How to make waterfall projects more agile
Published: 10 May 2015
Dr Thomas Grisham describes three different project management approaches – conventional, Pure Agile, and Scrum – and suggests how to decide which one(s) are the best fit.
Before we can discuss the process of Waterfall, we need to consider the definition of the process outlined in the Project Management Body of Knowledge (PMBOK). That is the sequential process groups of initiation, planning, executing, closing, and monitoring and controlling. Also for our purposes, assume that such a Waterfall process, or conventional project management, has strict gates at the beginning of each group. Of course in practice they are often blurred, but to make the distinctions more clear, assume they are crisp beginnings.
As for Agile, there are numerous models that have been developed for the IT industry, along with the terminology: XP, crystal, FDD, Scrum, Pure Agile, etc. For our purposes let us stick with Scrum as the representative of the Agile approach – it also is a better fit in our judgment. Sutherland’s recent book  provides a good description of the techniques and process, and the reason for bothering with it. With those basics in mind, let us have a look at the particulars.
Figure 1 shows a comparison of project management techniques with conventional Waterfall and Pure Agile at the extremes, and Scrum in the middle. You will note that the same five process groups are still present in Scrum, but they are mostly done in parallel rather than in sequence. The importance of a PMO is always a benefit in our experience, but for Scrum it is critically important. The reason is that projects which require more than 12 months to complete need someone to keep sight of the overall goal, and to enable effective and efficient hand-offs from one sprint to the next. This can be done by a Product Manager, but he or she may not understand the relationship with other projects ongoing. Regardless if it be a VP, Product Owner, Project Management Office (PMO), or some other title, a person tasked to see that all of the sprints are integrated.
Figure 1 - Comparison
The next difference is in the record keeping. I see companies that have adopted Scrum that actually create more paper than conventional projects, and others who keep no records. In theory, Scrum is intended to minimize documents so that the time is spent on the product, not on schedules, budgets, earned value, reports, and such. The key is to achieve the balance your organization requires.
The next consideration is an especially important one, resources. I have engaged with well over 400 companies in 65 countries, and can tell you that resource over allocation is endemic globally. One of the biggest benefits of Scrum in our view is that an organization can focus resources on a specific set of tasks for a limited amount of time. For example, imagine you have a project that requires 12 months to perform. The probability of having dedicated resources for this long in today’s economy is near zero, but for a Scrum sprint of two to four weeks better than 50%. There is a great deal of research into multitasking, but little that is definitive relating to productivity. A quick review suggests that productivity is on an inverted U curve, where productivity increases rapidly and then falls off rapidly. What has not been determined is what the maximum level is, and that is in our view because it is determined by the individual, task, and environment – too many variables. I can tell you from anecdotal discussions with thousands of corporate and academic students in 74 countries, that there seems to be leveling off around three concurrent projects that are not short duration.
It seems that people can juggle things that can be accomplished in a few hours at a time, so with text messages, email and breaks, a couple of hours seems to enable a person to cognitively focus effectively on a task. When the number exceeds five, people seem to be distracted, harried, impatient, and not well. For example, training for a company whose business volume increased by a factor of three, yet employees declined by 10%, I could see the stress. What also has not been studied is the effect on health and long term productivity in such circumstances.
The benefits of Scrum sprints of say two weeks, is that people can focus on perhaps one or two Scrums, and increase their effectiveness. And, on the sprints, if tasks are reduced to a single day or perhaps two, there is a much clearer quicker connection between task and the reward of completing it. On conventional projects a task with a duration of four weeks is simply too ambiguous, and hard to see the progress. Instead of doing ten things partially for a month, and having no sense of accomplishment daily, a person can start and finish a task in a few days, and look forward to something new. There is a good deal of neurological and biochemical research on the effects of quick rewards.
According to the work of Spira and Feintuch  knowledge workers lose 28% of their time due to interruptions. If one works 50 hours per week, normally this number is higher, that is a loss of one day per week. Bannister and Remenyi  cite other work which found that it takes a knowledge worker 25 minutes to refocus on a task. Yet others have found that multitasking increases productivity, but within the context that if a person is working on a mono-task there may be down time waiting on someone else. From our experience and anecdotal interviews with thousands of knowledge workers in global companies, most people work on average 50–60 hours per week. In general we find impatience, distracted, weary, and unfocused people on the average. Assume that the average worker begins at 80% productivity, we see far less, and then loses say 25% more due to interruptions and jumping from project to project. At 55% productivity that means that a person will have to work far more hours to keep up, the quality of the work will suffer, creativity will decline, and likely poor health will follow. So Agile seems to be an alternative, perhaps.
Figure 1 also addresses the nature of the teams employed to perform the work. In conventional projects the Project Manager has responsibility for the project, and the authority to direct the members of the team. She is in charge. For a Scrum Master he is a facilitator, and a guardian of the process. He does not have authority, and is often not part of the team for the sprint. The team is self-governing, self-directed, and somewhat immune from critique. The teams are mostly subject matter experts (SMEs), who determine the tasks, choose which ones they will do, and are responsible for how much or little time they spend on a project. In societal cultures where individualism is prevalent, Scrum is inherently more challenging as a result. In cultures that are collectivistic, it is an easier fit. Thus, a blend of conventional and Scrum can have obvious benefits on global virtual projects where both cultural tendencies are present.
Then there is the presence of the sponsor, customer, Product Owner, executive, business analyst or other title that represents the person paying for the project. In Pure Agile that person is present 100% of the time. This is unlikely in many instances so the Scrum alternative is more realistic. In Scrum the Product Owner is the person who knows the product, the reason for it, and what features it is to have. It is the person who can define the requirements for the project, and answer questions regularly. The job of the Scum Master is to get the attention of the Product Owner as needed, and as possible.
These then are the major dimensions to consider when deciding which project management approach(es) to use. The next consideration is how to filter projects into priorities to enable precious resources to be allocated across the business units.
Picking an approach
There are innumerable ways for global business to decide upon the projects that they do. What we suggest is a template to think about which approaches will best serve the organization. To use an example, assume an organization has 1,000 staff available for projects, and that each person works 160 hours per month or a total of 16,000 hours available. Let’s assume then that they are all multitasking on perhaps three projects, and the overall productivity rate is 70%. The company then has 112,000 person-hours available for projects, and the question is where and how to utilize them most effectively.
Figure 2 provides a framework for allocating resources, and determining which project management technique to employ. For critical projects Pure Agile would generally be a better option because of the focus and improved productivity for a short term. For projects longer than a month the pace is too intense, and it may prove that because of waiting times on development projects, having resources on a few projects concurrently may offer the possibility of utilizing down time more effectively. For the longer critical projects, having a PMO, Program Manager, Portfolio Manager, Product Manager or some other title is important for the long term continuity. Thus, our suggestion for a conventional framework split into Scrum sprints.
Figure 2 – Options
For the important items, we suggest dropping the Pure Agile and the Scrum fusion approach. And for the Nice to Have, rely on the conventional approach. The concept is that high intensity efforts like Pure Agile should be reserved for only the most critical projects as they require high intensity, and cannot be maintained effectively over a wide portfolio of projects. Similarly, Scrum Fusion projects require a great deal of effective communications, more so than normal. These two options place a heavier burden on the organization to do things in a special manner. Just as sprinters in track and field are not able to keep up the same pace for long distances, resources need to have a sustainable pace. In fact, having an occasional sprint is invigorating, as long as there is a respite at the end.
People need time to think, especially knowledge workers. Forcing Agile on a company that is already resource constrained will only hasten the problems and likely the failure rate. Using Agile as a way to selectively rush critical projects through the pipeline can be beneficial by providing a break in thinking patterns, and an increase in productivity. Provided it comes with the commensurate reduction in other tasks. I was with a company recently that had increased its projects by 50% per year, and decreased its staff by 10% per year – because of people quitting. The company wanted to learn Agile so that they could do projects more quickly with their current staffing because they saw the increase of projects continuing at the same pace. The staff saw this moving from crises once per week to crises once per day.
Agile will not solve problems caused by understaffing, it will exacerbate them. The time saved if Agile is properly implemented should revert to the staff, not to adding more projects.
1. PMBOK, A Guide to the Project Management Body of Knowledge, 5th Edition. 2013, Newtown Square, PA: Project Management Institute.
2. Sutherland, J., Scrum: The Art of Doing Twice the Work in Half the Time. 2014, New York: Crown Business.
3. Spira, J., Feintuch, J., The Cost of Not Paying Attention: How Interruptions Impact Knowledge Worker Productivity. 2005, New York: Basex.
4. Bannister, F., D. Remenyi, Multitasking: the Uncertain Impact of Technology on Knowledge Workers and Managers. Electronic Journal of Information Systems Evaluation, 2009. 12(1): p. 1–11.