Real Software Engineering & The Agile Value Graph

Glen Vanderburg (@glv) recently reprised his talk "Real Software Engineering" as a keynote at Ruby Hoedown 2010. In the words of Jim Weirich "every software developer should hear this"... Motivated to see what all the buzz was about, I watched the recording of the same talk from earlier this year (http://j.mp/9dqw9Q). The thesis is simple--Agile IS the state of the art in Software Engineering, not some sort of anti-engineering, or accidental engineering.

This, of course, is not news to most practioners, but his approach to defending this thesis is pretty novel, I think. He begins by quoting a paper from the proceedings of a NATO computing conference in 1969:

"A software system can best be designed if the testing is interlaced with the designing instead of being used after the design."

Pretty compelling huh?

He then frames the inception and growth of the Waterfall methodology as stemming from two errors: 1) the misinterpretation of a 1970 paper by Dr. Winston Royce by Pointy-Haired Bosses (a la http://www.paulgraham.com/icad.html), and 2) the Barry Boehm "cost of errors" graph had a hidden bias in that only Waterfall projects were measured.

I hope you are interested enough to check out the talk. I couldn't do it justice here. And you will find he draws several other valuable conclusions. Personally, I was compelled to read another of Vanderburg's papers referenced at the end of this talk, "Extreme Programming Annealed" (http://vanderburg.org/Writing/xpannealed.pdf), published in ACM SIGPLAN proceedings 2005.

In this paper he attempts an exposition of the coupling between the 12 (+1) XP practices to understand how to cope with a situation where one of the practices is disallowed. I won't recapitulate that work here, but he did present an interesting arrangement of the practices in relation to time. To me there was an immediate correlation between the cost of errors graph and this arrangement.

As providence would have it, I had recently been thinking about visual depictions of the "value" of Agile/XP. The pointy-haired bosses need something simple(-ish) to understand why these Agile practices are important. Ideally, they'd like to see how they affect the bottom-line; everything in business is a trade-off, after all...

After a few iterations, I came up with something that I hope Edward Tufte wouldn't snarl at and Kent Beck might nod at approvingly. I present you, the "Value of Agile Methods"

Some notes:

• P(Error) is the probability of introducing an error. I make the facile assumption that there is a linear relationship between the frequency with which an activity is performed and the frequency of errors introduced by that activity.


• P(Error_Agile) is the result of flattening of P(Error) by four particular Agile methods (see downward arrows); in Vanderburg's paper these are called noise-filters


• the "x-axis" is a logarithmic arrangement of time; in the case of the probability of error curves, this is how frequently an activity is performed; in the case of the cost of errors curve, it is the length of an interval between when an error is introduced and when it is discovered


• the groupings of the time axis represent the "epochs" of different practices: engineering (code, test, vet req's etc.); process (define requirements, team meetings, project management, etc.); and strategy (direction set from executives, market research, etc.). There is obviously some overlap and flexibility in what activities correspond to which "epochs"

Inferences

I hope the graph would compel the following inferences:

• Agile practices reduce the frequency of errors in engineering activities while reducing the time between when an error is introduced and when it is identified in both engineering and process epochs.


• The cumulative effect of these practices is to reduce the probability of errors and limit the overall cost of errors.


• Engineering errors are invariably frequent and cheap to fix when applying Agile practices, in stark contrast to errors in strategic decisions that are both infrequent and very, very expensive.

Call to Action

If you think this graph is interesting, flawed, has potentional, etc. I encourage you to iterate it and share it.

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

Hipster Programming: Coding in F# on Mac OS X

The vitriol towards Microsoft has a long and graceless tradition with a lexicon sporting such inventive neologisms as the venerable "M$FT" and the puerile "Total Failure System" in reference to Team Foundation Server (TFS). Oh, how these countless little slights must wound the souls of those folks toiling away under the yoke of stodgy corporate masters! Their inner light having been dimmed and filtered by a soulless, crushing capitalism so much so that they unconsciously embrace their sameness, adopting a uniform of pique polo shirts and khaki pants.

The iconoclasts, the individualists, the thinkers--these bear the nome d'guerre "Rubyist", "Rails aficionado", "Python hacker", "open source developer". You will know them not by their similarity but their diversity, their vestiture definitively anti-establishment and their mocking of any tool that you don't have to download from github and compile from source.

As in all things, there is a middle road whose pedestrians are accused of compromise with the Devil by one side and "just trying to be different" on the other. Yet in this camp we find many sagacious, experienced craftspeople who have spent time in both camps. With the reader's indulgence, I shall call them hipster programmers.

For what its worth, I consider myself a hipster programmer, as evinced by how I spent my morning. Having been developing in .NET for nearly a decade and of recent years drawn strongly to functional programming, I am of course learning F#. But, I'm also experimenting with Erlang, Ruby, and more recently Clojure. As far as platforms go, you've got a lot of choice these days: Windows, Linux, Mac OS X. With Windows you certainly need Cygwin and a git port. Linux is probably the easiest to get going on but lacks some of the day-to-day niceties (e.g. iTunes). Mac OS X can't be (legally) virtualized, so it makes an obvious choice as the base system.

My work machine is a MacBook Pro running VMWare Fusion. My recent explorations of Clojure have kept my exclusive in the Java tools domain--Maven, Ant, Netbeans, etc., but the Chicago Clojure user group recently invited Dr. David Miller to come talk about clojure-clr, a C# implementation of Clojure for the CLR, very hipster! What could be cooler than that?

Well, porting clojure-clr to F# of course!

Sure, I could fire up Fusion and VS 2010 and start rocking, but I like to begin with the end in mind, namely moving seamlessly between writing code in Clojure targetting the CLR and the JVM. I'd like to use similar tool chains. And, I'd like to present to a group of the cool kids in their native tongue.

The first step is getting an environment setup. Robert Pickering has a great article to get Mono setup on Mac OS X. Don't skip the 'sudo install-mono.sh' step. At the time of this writing their is a strange error in his instructions regarding the mono.snk file. The install-mono.sh file makes this pretty clear, so just ignore that part. Basically, you have to give the F# assemblies a proper strong name to add it to the GAC.

Now, I love vim--it was my first code editor--but the Ruby kids have been using TextMate, so let's use that since someone kindly shared an F# bundle. That bundle uses iTerm to evaluate function in F# interactive, so we'll need that too. Finally, we need to alias "fsi" to run F# interactive using Mono, so the TextMate bundle will run it properly. Edit ~/.profile adding this command.

alias fsi='mono ~/path/to/fsi.exe'

So we've got the beginnings of a build environment going; what's next?

• Get a branch going on github, referencing the clojure-clr project


• Get a build system stack setup (Maven? Rake?)


• Find an unit testing suite for F#/Mono/Mac OS X


• Start hacking