Methodology, how to Manage Modern Software Projects: Waterfall vs Agile

In 2000, the FBI started developing a criminal case management software system called the Virtual Case File. It was a complete failure. The cost of this project was estimated to be over $170 million dollars and after 5 years of developing, prototyping and designing they had to abandon the project with nothing to present.

Why did a project so large, with so many stakeholders and so much planning, fail so badly?

For many reasons; the absence of enterprise architecture, a poorly designed system, lack of project management skills and a strong dependence on external contractors.

But most of all, due to the methodology. They used the classic Waterfall methodology, where a 300 person team spent 6 months creating the requirements, with a grand design upfront, that resulted in 600 pages requirements, 400 documented ‘change requests’, 700,000 lines of code that were written and re-written time and time again, and no useful results.

The Waterfall Methodology

This doesn’t mean that every project that uses the Waterfall Methodology will fail. This methodology focuses on a more traditional and linear approach, where projects start at the first phase and only progress to the next when everything in the previous phase has been completed. All requirements must be gathered before any development occurs, (relying on heavy up-front analysis and documentation about the needs and problems of the client), followed by the process of design. Only once this is finished, will the implementation and development take place.

In the final phases, there is the verification and maintenance stage, where the customer already has the end product and can test it according to the specifications only when the product is totally finished. The developers will then maintain the code, fix any bug that appears and keep it updated.

For many years, this was the most widely used methodology and a large number of projects succeeded with it. When you see a large bridge, an airplane, a building of +50 floors, it most likely was build using Waterfall Methodology, taking into account that the client just won’t change the requirements middle way, since you would have to demolish the building or dismantle the airplane and that would cost money. Those projects must have a very long period of planning and up-front analysis to succeed.

But, there is another kind of project where iteration, flexibility, constant feedback, and adaptation are essential. This is where the Agile Methodology comes in.

The Agile Methodology

The Agile Methodology is based on iterative and incremental development instead of a linear approach. It does not build an entire system at once, but rather develops it incrementally. Less time is invested upfront for documentation and analysis, as clients are constantly seeing and testing the product and providing feedback. The development and feedback process adds accountability (tangible milestones of completed work, not just documentation), and tends to improve client satisfaction by allowing ongoing input.

Agile relies on a very high level of customer involvement throughout every phase of the project. The planning, design, develop, testing, release and feedback are in a constant cycle in a defined period of time. Instead of segmenting projects in stages, Agile development tends to address the projects as a whole.

This model emphasizes the rapid delivery of a completely functional application components. Rather than creating tasks and schedules, all time is separated into phases called “sprints.” Each sprint has a specific duration (usually in weeks) with a list of deliverables, planned at the beginning of the sprint. Deliverables are prioritized by business value as determined by the customer. If all planned work for the sprint cannot be completed, work is reprioritized and the information is used for future sprint planning.

This approach is more flexible. The requirements may change along the way and the team must be able to adapt easily, (the teams are usually smaller). There is greater transparency between the customer and developers, and the schedule and cost are predictable.

After the failure of the FBI system, they had to implement a different approach in order to get different results. In 2010 the project was internalized, the FBI CIO took ownership and Agile was adopted as the project framework. The design was broken into 670 user stories, there were self-organizing teams, 45 staff (not 300 as previous) the product owner prioritized the work, and they had two weeks sprints (instead of 6 months sprint) with a demo in every sprint.

The experience was successful, they reduced the cost in half, the agents started using the system on real cases, exceeding all expected targets.

PROS AND CONS

Waterfall Pros

• It is easy to understand and manage as stages are clearly defined.
• Meticulous record keeping and documentation.
• The client knows what to expect. The client will have an idea of the size, cost, and timeline for the project. The client will have a definite idea of what their product will do in the end.
• In the case of employee turnover, Waterfall’s strong documentation allows for minimal project impact.

Waterfall Cons

• It often becomes rigid and resistant to change.
• It relies heavily on initial requirements. However, if these requirements are faulty in any manner, the project is doomed.
• The whole product is only tested at the end. If errors are discovered late in the process, their existence may have affected the rest of the project.
• The plan does not take into account the client’s evolving needs throughout the project cycle.

Agile Pros

• It allows for changes to be made after the initial planning stage. It follows the client’s requirements changes.
• It is easier to add features that will keep the product up to date with the latest developments in the industry.
• At the end of each sprint, project priorities are evaluated. This allows clients to add their feedback so that they ultimately get the product they desire.
• The testing at the end of each sprint ensures that the errors are caught in each cycle.

Agile Cons

• This dynamic methodology is not suitable for processes that require a complex decision making of formal planning such as construction, manufacturing, military, health care system among others.
• As the initial project does not have a definitive plan, the final product can be grossly different from what was initially intended.

Moove It

At Moove It, our main focus is on producing high-quality value for our customers. Create a beautifully-designed and functional components application in a short period of time, with small teams that can adapt easily and be flexible with the client needs and problems. Synchronize with the customer, adapt, iterate, deliver, again and again until getting the best possible outcome for the customer.

Every client is different, the sprints, schedules, complexity, tools, teams, and technologies may change from customer to customer, but there is one common concept that applies throughout our projects; we use Agile methodology. And the reason is because it has been proved with multiple studies (and with our own experience), that for software development, the Agile methodology is more likely to help projects to succeed since it belongs to an industry that is constantly changing, and continuous improvement is highly valued, as is shown in the following image:

Ambysoft’s 2013 Project Success Rates Survey

The development team that uses Agile methodology has 64% success rate, compared to just 49% for the Waterfall model.

Also, Ambysoft’s survey analyzed the main factors that contribute to a project’s success or failure and found the following:

Agile methodologies are better suited for product quality, stakeholder value, ROI and Time/Schedule. Statistics clearly show that Agile methodologies consistently deliver more successful projects and fit better for software development teams than other methodologies. This is seconded by the 2017 study conducted by PWC, which also indicates that Agile projects are 28% more successful than traditional projects. This is why we apply this kind of methodology, not only statistics confirm Agile success, but also our team and happy customers.

Conclusion

Both of methodologies are widely used and many projects have succeeded with them. The Waterfall Methodology is more traditional, and for projects that need a big planning and a robust structure with no requirements being modified constantly, this is a great methodology to follow.

In the other side, the Agile Methodology is great for teams that need to adapt constantly according to the client’s needs, focused on continuous improvement, flexibility, getting results rapidly and business value.

However, Waterfall is not bad — the transition to Agile was largely dictated by the expectations of modern businesses, the nature of specific industries and the way projects are approached nowadays. Waterfall is still very useful (and the only methodology suitable) for many industries.

At the end what it matters is the result, and Agile is thriving in software development because it works.