I am often asked, “What is the difference between Waterfall and Agile?” and “When does it make sense to apply Agile vs. Waterfall? A google search will produce many results in answer to these questions and here are my thoughts on the subject.
What is Waterfall?
Waterfall is the termed used for a sequential, linear, one-directional process for working through the software development stages of Requirements, Analysis, Design, Development, Test, Deployment. All the work for one stage of the process is expected to be completed before moving to the next stage. Work flows down from one stage to the next, resembling a waterfall and hence the term “waterfall”.
This sequential process became mainstream in 1985 when the US Department of Defense captured and formalized this approach. For the history buffs, this process was first formally described in the 1970 article by Winston Royce – Managing the Development of Large Software Projects where he in fact, raised its risk as a flawed model particularly for large, complex projects and recommended a more iterative approach.
Originating from the manufacturing and construction industries, where the cost of change has traditionally increased as we get further into the project, the “waterfall” approach offers the opportunity to learn as much as possible as early as possible in an attempt to avoid change in later stages of the process. When using this approach, all the requirements are gathered up-front and a plan is put together with a well defined scope, time and budget before any work is started. The value of the work is realized only after all the stages are complete and the entire project is delivered in a big bang approach.
In situations where everything is known or knowable up-front, and where everything falls into place through the various stages according to the plan, this type of approach might work. However, with the increasing complexity of software systems we work with and the problems we are trying to solve, it is not possible to know everything up-front. With the accelerated pace of change, the bigger the project, the higher the chance of a late discovery or a new emerging reality resulting in changing priorities – putting the original plan in jeopardy and creating wasted effort.
Lack of feedback loops and late integration and testing raise the additional risks of this approach. Pressure to meet the constraints of staying within scope, budget and time, often results in a product of questionable quality being pushed out the door or a product that falls short of meeting the needs of the customer.
Agile processes are an attempt to address some of these risks and short comings and use change as an opportunity to gain a competitive advantage.
How is Agile different?
I like to define Agile processes are an “iterative and incremental approach to delivering something of value that enhances our ability to respond to and create change”. Instead of a big bang approach, work is planned and delivered as smaller increments of functionality that allow for early and continuous realization of value, and opportunities for feedback to better ensure the product meets the customer’s need.
Several lightweight frameworks and methods including Scrum, Kanban, XP, Scrumban, DSDM, Crystal etc. fall under the category of Agile processes. These processes have several things in common including a prioritized backlog, short feedback loops, an iterative and incremental approach to planning and delivery, a focus on value and working software, and a small, cross-functional, self-organizing team that makes all of it possible. The word Agile was coined to describe the underlying philosophy behind these frameworks, methods and practices and is captured as 4 values and 12 principles under the Agile manifesto.
In recent years, advances in software and technology have opened the door to lowering the cost of change. Agile processes allow us to take advantage of that! The incremental realization of value offers organizations greater freedom and flexibility to strategically change priorities in response to the accelerated pace of change and increase their ability to navigate and when done well, thrive in today’s VUCA (volatile, uncertain, complex and ambiguous) world.
“The quest for agility has resulted in a fewer giant projects and choosing to do more small things integrated in simpler ways” – RT Grady Booch
When does it make sense to use Agile processes?
The empirical approach of Agile processes make it a natural fit for product development and complex situations where everything cannot be known upfront. The needs of the user and solutions to problems are better discovered through feedback loops and sense and respond approach. However, I believe any type of effort, large or small, will benefit from the many advantages offered by Agile processes including visibility of the work, transparency around progress and impediments, focus on value, efficiency through smaller batch sizes and work in progress limits, clarity on prioritization, greater collaboration and engagement, better decisions through short feedback loops, and early and continuous realization of value.
The image below offers a side-by-side comparison of some key differences between waterfall and Agile from a process perspective.
Want to dig deeper?
Some great books on the subject include Succeeding with Agile, Mike Cohn and The Agile Mindset, Gil Broza