This post is a placeholder for some notes I jotted down about a recent .NET Rocks! podcast (episode 1262) where the hosts, Richard Campbell and Carl Franklin spoke with guest Matthew Renze about the economics, psychology, and science of Agile.
See also, my notes on a previous episode about Agile.
Notes
Matthew’s premise for talking about Agile is Eddie Obeng’s TED talk The World after Midnight. The old world (before midnight) is one where people learn faster than the world can change, and is seated in bureaucracy, process, and tradition. The new world (after midnight) is one where society is changing faster than people can learn, and is seated in communication, collaboration, and learning. Agile, with its continuous feedback loops makes it a good fit for the new world.
The Agile Manifesto outlines 4 value propositions and 12 principles that were common among a series of software development methodologies that were around at the time.
See this Dilbert cartoon about how Agile doesn’t mean going faster with no additional resources.
Why are these Agile practices producing better results? There are 4 levers — scope, resources, schedule, quality — that are constraints. Traditional project management involves a triangle: Fix the scope, then estimate resources and the schedule. Agile project management inverts this: Fix the resources and the schedule, then estimate scope. This solves several problems:
- Solves the mythical man month by fixing team sizes (i.e., resources) to limit the team so it can’t grow too big.
- Solves slipping release dates by fixing the schedule; we may have more or fewer features in any given release, but it will ship on this date.
- Solves scope creep by estimating scope.
- Solves technical debt by letting the team push back if it doesn’t think the work can be done with the right about of quality in the time given.
A typical use of the project management triangle: The scope and budget are fixed, so sacrifice quality.
Having empirical evidence about why Agile works makes it easier to convince management.
When traditional companies see other (Agile) companies being successful, that can influence them into embracing new world thinking.
What are some ways to ease people into the new world?
- Start with education at the top and have it work its way down.
- Explain Agile concepts using economics, psychology, and science (i.e., the language of business).
- Get the right people in the right seats to move the bus forward.
In companies dealing with a high degree of uncertainty, Agile works extremely well. If we were building an ATM or a tractor, Agile may not work as well.
Many companies have this mindset: “We want this much software by this date and here’s how much money we have for you. You can use whatever methodology you want, just get it done.”
Little-a agile means using what works for your team based on your specific constraints, goals, and capabilities as a team. Big-a Agile means doing it by a specific process, which consultants are happy to sell you.
Agile is good at providing transparency to the team and to management, and does so via information radiators. These radiators are anything that’s visible to others (e.g., Scrum board, dashboard) to indicate the level of success for the team.
The Pareto principle states that 80% of the effects are derived from 20% of the causes in any system that’s modeled by power law functions. For example, 20% of the software we produce brings 80% of the value. This is why we need a prioritized backlog (by value or ROI) so that we don’t write as much software that brings little value.
Calculating ROI is very time consuming. One way to estimate this is to rank complexity and value on a scale of 1 to 3 so that you can tease out the values without all the math and without using dollar amounts. This scale also allows you to prioritize features relative to one another. Note: To keep everything from being a “3”, have an imaginary bank account with only certain amounts of 1s, 2s, 3s so you have to choose more diligently.
Wisdom of the crowds involves the collective guesses of a group of people sometimes being better than experts (e.g., how valuable is this feature, how much time will it take to develop). This technique works well for things that use central tendency — for example, Planning Poker.
Complex adaptive systems — a collection of interacting things that changes in response to its environment to increase its survivability (e.g., a school of fish). This system is not a top-down, command-and-control structure; it’s an “ad-hocracy.”
Agile teams are good at utilizing feedback to adapt to changing markets / technology / requirements / preferences because things aren’t stable while the project finishes.
Paper: Bayesian Analysis of Empirical Software Engineering Cost Models by Chulani, Boehm, and Steece
The Cone of Uncertainty says that the variability of our estimates decreases over time as we learn. We’re often off by a factor of 4 at the beginning.
Feedback is the strongest factor in terms of success in learning. Agile has many feedback mechanisms built in at different scales. For example, test-driven development gives you feedback on the order of seconds, and release planning gives you feedback on the order of days/months.
The lean startup movement involves smart failures so we can pivot to another approach.
Feedback reduces risk.
Technical debt as a term resonates well with management because it’s an economic metaphor.
Build telemetry into the software to validate your hypotheses while you’re developing the software. For example, should we add a feature to this page when we’ve measured that only 1% of our users go to that page?
“Running on analytics” — making decisions based on a data science framework. The idea is to test and not make decisions unless you have data to support those decisions.
Developers get flak for providing bad estimates, but you can push back. “How much value will this feature you want really add (because we can measure that); i.e., how good are your estimates?”
If you’re building something very complex/expensive/risky, you need to do due diligence to get good estimates.
It’s easier to stare at your navel for weeks rather than build something that might fail. There’s a culture of fearing failure, and it comes from the top down.
Information and knowledge gleaned from failure is an asset. The insight gained from something that didn’t work may provide a perspective your competitor doesn’t have (because they haven’t failed in that exact way).