Waterfall vs agile vs building homes

Many of us are grappling with how to embed agile ways of working into traditional organisations which are used to waterfall or lean mindsets and methods.

It’s a hard juggling act.

I’m temporarily overseeing the change programme in Homes England, a government arm funding house building. Currently, they invest over £4b in house building, rising to a total £27b over the next few years. It’s an ambitious programme in a growing organisation, set in a challenging political and economic environment. They’re transforming as an organisation, with FutureGov supporting through digital, service, and organisation change, so the agency can better meet these growing ambitions.

Having reset the programme from a very stiff waterfall six months ago, we have now aligned both agile and lean / waterfall*.

My heart, my dear heart says ‘agile’, yet my head, sometimes hitting a desk, says ‘waterfall’. Below, I briefly outline how. First a quick look at why:

Why waterfall can work

Most people are used to waterfall most of the time. Especially the decision-makers to whom we report; e.g.:

Boss: “Show me my thing is going to get delivered in six weeks when I told big chief who also loves big plans it will”.

Employee: “Yes boss, look at this big plan with lines stretching into the future”.

Boss: “Excellent, you’re not fired. See you next month”

Projects In A Controlled Environment (PrInCE, a well-used waterfall approach) exist. Some environmental factors are controllable, like budgets, structures (physical, organisational), or processes. Want to launch a learning and development programme? You can decide when and how. Controlling factors are possible and beneficial. It gives a sense of ‘grip’.

Detailed planning gives an indication of spending, timescales, scope and levels of risk. The Homes England change programme is currently managing 21 work packages (projects) within four workstreams (programmes). Projects which are big and diverse like the technology business case, office accommodation moves and staff onboarding experience have potentially endless variables. Keeping an eye on how they are all evolving ahead of time is important when asking for resources, measuring success, or deciding to change course.

Why agile works

The agile method allows us to easily change course when we realise we were wrong. Humans are so very often wrong and so very awful at admitting it. Just look at the news. Or most projects.

Much of the environment is, nonetheless, outside our control. Political upheaval, organisational leadership, your boss’ opinions, your opinions, what your user’s thought they needed vs what they actually need vs what they will expect very soon.

Agile is open, honest, and true to our instincts. For millennia humans have worked in open, multidisciplinary and collaborative teams, with quick feedback loops and minimal documentation, apart from the odd cave painting. This way of working brings energy, focus, and manages inherent risk. We love it.

Using agile & waterfall together

At Homes England, we’re measuring both outputs (a waterfall approach to deliver a thing for a point in time) and outcomes (deliver an objective in an agile stage).

These are different roadmaps for the same programme running right now.

The first allows us to plot artefacts into the future. It’s important because stakeholders need an understanding of what is happening, when and how it relates to the multiple other changes taking place.

The second allows us to get a sense of what came before and what is happening next within the lifecycle of the project.

Removing the advantages of the former overnight loses trust. Homes England needs to accelerate the building of homes and rapidly change its delivery capability. If I do not provide reassurance in a time of change, I get fired. The programme dies, probably along with the agile methodology. Leaders lose trust, I go home and we can’t make the necessary changes to get more homes built.

So, twin tracking makes sense. Waterfall provides a comfort blanket for many to explore agile whilst not losing momentum. Change is by its very nature is transitory so we need to build for both the old and new simultaneously.

For now.

Over time, the intention is to move away from Waterfall as much as possible. Not just as a project management tool, but an organisational mindset. The aim is to embed agile principles as much as agile rituals, positively affecting organisational culture, values and reward structures over time.

As Janet Hughes put it, leading change can feel like turning a”‘flotilla of tankers”. Multiple important changes, all at the same time.

So when it comes to the question between agile or waterfall, the answer might be “both”. If it means the wider mission wins.

*Other methods exist. Some of the better writing on this comes from Simon Wardley who draws a further distinction between Agile, Lean and Six Sigma. There are other blogs written by bigger geeks than me. I’ve emphasised on mindsets and principles over mechanics.