Real Software Engineering & The Agile Value Graph

Glen Vanderburg (@glv) recently reprised his talk "Real Software Engineering" as a keynote at Ruby Hoedown 2010. In the words of Jim Weirich "every software developer should hear this"... Motivated to see what all the buzz was about, I watched the recording of the same talk from earlier this year (http://j.mp/9dqw9Q). The thesis is simple--Agile IS the state of the art in Software Engineering, not some sort of anti-engineering, or accidental engineering.

This, of course, is not news to most practioners, but his approach to defending this thesis is pretty novel, I think. He begins by quoting a paper from the proceedings of a NATO computing conference in 1969:

"A software system can best be designed if the testing is interlaced with the designing instead of being used after the design."

Pretty compelling huh?

He then frames the inception and growth of the Waterfall methodology as stemming from two errors: 1) the misinterpretation of a 1970 paper by Dr. Winston Royce by Pointy-Haired Bosses (a la http://www.paulgraham.com/icad.html), and 2) the Barry Boehm "cost of errors" graph had a hidden bias in that only Waterfall projects were measured.

I hope you are interested enough to check out the talk. I couldn't do it justice here. And you will find he draws several other valuable conclusions. Personally, I was compelled to read another of Vanderburg's papers referenced at the end of this talk, "Extreme Programming Annealed" (http://vanderburg.org/Writing/xpannealed.pdf), published in ACM SIGPLAN proceedings 2005.

In this paper he attempts an exposition of the coupling between the 12 (+1) XP practices to understand how to cope with a situation where one of the practices is disallowed. I won't recapitulate that work here, but he did present an interesting arrangement of the practices in relation to time. To me there was an immediate correlation between the cost of errors graph and this arrangement.

As providence would have it, I had recently been thinking about visual depictions of the "value" of Agile/XP. The pointy-haired bosses need something simple(-ish) to understand why these Agile practices are important. Ideally, they'd like to see how they affect the bottom-line; everything in business is a trade-off, after all...

After a few iterations, I came up with something that I hope Edward Tufte wouldn't snarl at and Kent Beck might nod at approvingly. I present you, the "Value of Agile Methods"

Some notes:

• P(Error) is the probability of introducing an error. I make the facile assumption that there is a linear relationship between the frequency with which an activity is performed and the frequency of errors introduced by that activity.


• P(Error_Agile) is the result of flattening of P(Error) by four particular Agile methods (see downward arrows); in Vanderburg's paper these are called noise-filters


• the "x-axis" is a logarithmic arrangement of time; in the case of the probability of error curves, this is how frequently an activity is performed; in the case of the cost of errors curve, it is the length of an interval between when an error is introduced and when it is discovered


• the groupings of the time axis represent the "epochs" of different practices: engineering (code, test, vet req's etc.); process (define requirements, team meetings, project management, etc.); and strategy (direction set from executives, market research, etc.). There is obviously some overlap and flexibility in what activities correspond to which "epochs"

Inferences

I hope the graph would compel the following inferences:

• Agile practices reduce the frequency of errors in engineering activities while reducing the time between when an error is introduced and when it is identified in both engineering and process epochs.


• The cumulative effect of these practices is to reduce the probability of errors and limit the overall cost of errors.


• Engineering errors are invariably frequent and cheap to fix when applying Agile practices, in stark contrast to errors in strategic decisions that are both infrequent and very, very expensive.

Call to Action

If you think this graph is interesting, flawed, has potentional, etc. I encourage you to iterate it and share it.

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.