IFormattable

Fostering a Culture of Effective Feedback

2011-08-02T22:47:00.009-04:00

As of the last few months, I've had numerous occasions to ruminate on the purpose, efficacy, and goals of feedback. It's a risky business to give someone unsolicited feedback, for example, and it is difficult to do so tactfully. So, most folks simply don't bother to the detriment of themselves and the person being deprived of needful insight into their blind spots.

In a retrospective today we were asked some poignant questions about how to foster a feedback culture. Initially as a joke, I challenged our project sponsor to use the word as an acronym to define itself, a la "GNU". As I began thinking about my own description of effective feedback, some ideas came to mind.

FEEDBACK

Fast—feedback should be given as close in time to the event that triggered the feedback as possible. Feedback given too long after an event risks being revisionist, or perceived as such, due to our fallible memories, making it more difficult to agree on what happened. Feedback given soon after an precipitating event will be received in context and have a proportionately stronger impact. It's much easier to take remedial, corrective action given incremental, regular feedback; likewise, regular positive feedback can raise morale and engender positive espirit de corps.

Expected—make no mistake, the surest way to have feedback blow up in your face is for it to find your interlocutor surprised to be receiving it. Different social situations have different norms of feedback expectations. In most situations that come to mind, thoughtful feedback from a "superior" to a "subordinate" is almost assuredly going to be received gladly, whereas the reverse is nearly equally likely to not go so well. A similar gradient exists between a senior and a junior employee, and peers are not likely to expect any feedback from each other at all. The challenge is a cultural one, and cultures are shaped by their rituals. Effort must be expended to normalize feedback, making it an expected part of day-to-day life in the culture.

Empathetic—how would you feel if you were receiving the feedback you intend to give? This is the most important question you can ask as it informs all other aspects of feedback. Some feedback can be delivered within earshot of others; often a quick private chat is more apropos. Before ever giving feedback, try to imagine being the receiver.

Definite—feedback should not be opinion and conjecture. Details should be concrete and accurate. The feedback must be clear. Saying, "you are acting like a jerk," is spurious; "I am put on edge when say X, and I fear others may be as well," does not invite interpretation nor ascribe subjective qualities. If the receiver cannot understand the feedback, it is likely to have a deleterious effect on the relationship.

Beneficial—it should be obvious that if there is not benefit to the team or the receiver the feedback will likely have a negative effect. What's important is that you can articulate the benefit of giving the feedback. If the receiver acts in a responsible way vis-à-vis the feedback, how does our situation improve?

Actionable—articulating the benefit necessarily highlights the question of appropriate actions that address the feedback. If feedback is not actionable, it cannot be beneficial. The giver and receiver should work together to identify action that can be taken to incorporate the feedback. This discussion can lead to a fuller understanding of the feedback that was given and how it is manifested in behavior.

Considered—much worse than delaying feedback too long is a rash, immediate response. The qualities that make for effective feedback are not to be evaluated retroactively; they should be considered before the feedback is given in order to ensure good outcomes.

Kind—"If you can't say something [nicely], say nothing." It is entirely possible to provide, accurate, beneficial, efficacious feedback and to do so with a negative spirit. Feedback is not revenge; it is a dish best served warm. Kindness is king, and if you want to get the most of a feedback culture, it must be compulsory. The goal of feedback is to give rise to high functioning teams, and a key indicator of such teams is social sensitivity.

How Agile Makes You Dumb (or Smart)

2011-07-27T00:34:00.004-04:00

While many of the thought leaders in the Agile community decry the inexorable march toward banal, brand-oriented adoption, entreating us to internalize the principles of Agile and not blindly follow prescriptive practices, proscribing such phrases as “being agile” or “doing agile”, there remains an elusiveness to what makes Agile work. We’ve heard the stories: high-functioning teams churning through a product backlog, steadily delivering week after week. Nothing impedes these paragons of Agile values, while the rest of us struggle.

The picture looks quite different for most Agile teams: stories are estimated inaccurately; pair performance varies widely; velocity suffers at the mercy of technical debt; knowledge silos emerge; defect cycles lengthen; customers start getting frustrated. Very soon managers start trying to identify problems with the process, invariably looking for their keys under the lamplight instead of where they dropped them—looking for problems they can solve instead of solving the problems.

One cold Chicago winter night, Robert C. Martin, “Uncle Bob”, related a tale wherein a company engaged in the adoption of Scrum saw a very promising early spike in velocity followed by decline. They brought in Ken Schwaber, looking for the problems in their Agile practices and methodologies. With some tweaking they saw another bump in velocity but were too soon again swimming upstream. The Scrum Master himself next recommended they bring in Uncle Bob. They had done all the Agile stuff right—everything but the engineering.

As Uncle Bob relates the tale, Extreme Programming (XP) neatly codified the marriage of what we now call Agile with fundamental, essential software engineering principles and techniques. Unfortunately, most managers have very little aptitude for or mastery of the engineering, and naturally focus on those things they perceive they can directly impact. Consequently, the Agile aspects got top billing, and the engineering relegated to understudy. The unequivocal result is underperforming teams and projects. Agile made them dumb.

“Whoa! What about those teams who are delivering,” I hear you ask! Those fabled unicorns of the software industry where every iteration delivers substantial business value and change is accommodated like an accidental in a Kind of Blue reprise… surely they’ve got the system right. Or, maybe it isn’t the system at all. That’s the pretext of the Agile guru’s foreboding about “doing Agile”—there is no system, no prescription… There’s just really smart, wise, motivated people working together effectively.

Organizations are driven to Agile practices for a number of reasons, but few managers would suggest that Agile makes their teams smarter. But that is truly the secret sauce of what makes so-called Agile teams successful. They have a high collective intelligence. As a group they can overcome a great many obstacles and continue performing at a high level. They create fewer problems for themselves and deal swiftly and precisely with the problems they don’t create. They work.

Recent research into high-performing teams—those with a high collective intelligence—shows two distinct and surprising facts. First, individual intelligence is only weakly correlated with team performance. Second, there is a strong positive correlation with the number of women on the team and team performance. Women make teams smarter than men. Full Stop.

Why? Those of us with a Y-chromosome will be relieved to know that there is a similar correlation with social sensitivity and team performance. Women generally rate significantly better at social sensitivity than men. The researcher states, "what it suggests is that if you don't know the social sensitivity of a group, it is a better bet to include females than not." "The team also found that groups in which members took turns speaking were more collectively intelligent." (New Scientist)

In most enterprises, groups develop software. The engineering aspects of software development are extremely difficult and require a great deal of intelligence. It stands to reason the groups with high emotional intelligence are better at developing software than not. A bunch of hotshots don’t make for a smart group. So how would we go about maximizing the collective intelligence of development team?

“…took turns speaking…” Sound familiar? Stand-ups and pair programming, when effectively fostered, should result in an effective increase in the social sensitivity of the individuals on the team. The corollary is that teams that are good at stand-ups and pairing are better at delivering software. Agile makes them smarter.

Managers should be cogitating on ways to increase the social sensitivity of their developers. To those that can’t break their addiction to quick fixes: fire all the asshole, prima dons and use the windfall to hire some competent women.

Code qua Art

2011-02-21T14:47:00.003-05:00

The positronic sqrt program in rod logic is art. Damian Conway, Thoughtstream: "Temporally Quaquaversal Virtual Nanomachine Programming In Multiple Topologically Connected Quantum-Relativistic Parallel Timespaces...Made Easy!"

The Qlobe is art.

Real Software Engineering & The Agile Value Graph

2010-09-06T15:33:00.007-04:00

Glen Vanderburg (@glv) recently reprised his talk "Real Software Engineering" as a keynote at Ruby Hoedown 2010. In the words of Jim Weirich "every software developer should hear this"... Motivated to see what all the buzz was about, I watched the recording of the same talk from earlier this year (http://j.mp/9dqw9Q). The thesis is simple--Agile IS the state of the art in Software Engineering, not some sort of anti-engineering, or accidental engineering.

This, of course, is not news to most practioners, but his approach to defending this thesis is pretty novel, I think. He begins by quoting a paper from the proceedings of a NATO computing conference in 1969:

"A software system can best be designed if the testing is interlaced with the designing instead of being used after the design."

Pretty compelling huh?

He then frames the inception and growth of the Waterfall methodology as stemming from two errors: 1) the misinterpretation of a 1970 paper by Dr. Winston Royce by Pointy-Haired Bosses (a la http://www.paulgraham.com/icad.html), and 2) the Barry Boehm "cost of errors" graph had a hidden bias in that only Waterfall projects were measured.

I hope you are interested enough to check out the talk. I couldn't do it justice here. And you will find he draws several other valuable conclusions. Personally, I was compelled to read another of Vanderburg's papers referenced at the end of this talk, "Extreme Programming Annealed" (http://vanderburg.org/Writing/xpannealed.pdf), published in ACM SIGPLAN proceedings 2005.

In this paper he attempts an exposition of the coupling between the 12 (+1) XP practices to understand how to cope with a situation where one of the practices is disallowed. I won't recapitulate that work here, but he did present an interesting arrangement of the practices in relation to time. To me there was an immediate correlation between the cost of errors graph and this arrangement.

As providence would have it, I had recently been thinking about visual depictions of the "value" of Agile/XP. The pointy-haired bosses need something simple(-ish) to understand why these Agile practices are important. Ideally, they'd like to see how they affect the bottom-line; everything in business is a trade-off, after all...

After a few iterations, I came up with something that I hope Edward Tufte wouldn't snarl at and Kent Beck might nod at approvingly. I present you, the "Value of Agile Methods"

Some notes:

P(Error) is the probability of introducing an error. I make the facile assumption that there is a linear relationship between the frequency with which an activity is performed and the frequency of errors introduced by that activity.

P(Error_Agile) is the result of flattening of P(Error) by four particular Agile methods (see downward arrows); in Vanderburg's paper these are called noise-filters

the "x-axis" is a logarithmic arrangement of time; in the case of the probability of error curves, this is how frequently an activity is performed; in the case of the cost of errors curve, it is the length of an interval between when an error is introduced and when it is discovered

the groupings of the time axis represent the "epochs" of different practices: engineering (code, test, vet req's etc.); process (define requirements, team meetings, project management, etc.); and strategy (direction set from executives, market research, etc.). There is obviously some overlap and flexibility in what activities correspond to which "epochs"

Inferences

I hope the graph would compel the following inferences:

Agile practices reduce the frequency of errors in engineering activities while reducing the time between when an error is introduced and when it is identified in both engineering and process epochs.

The cumulative effect of these practices is to reduce the probability of errors and limit the overall cost of errors.

Engineering errors are invariably frequent and cheap to fix when applying Agile practices, in stark contrast to errors in strategic decisions that are both infrequent and very, very expensive.

Call to Action

If you think this graph is interesting, flawed, has potentional, etc. I encourage you to iterate it and share it.

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

Hipster Programming: Coding in F# on Mac OS X

2010-09-04T12:00:00.005-04:00

The vitriol towards Microsoft has a long and graceless tradition with a lexicon sporting such inventive neologisms as the venerable "M$FT" and the puerile "Total Failure System" in reference to Team Foundation Server (TFS). Oh, how these countless little slights must wound the souls of those folks toiling away under the yoke of stodgy corporate masters! Their inner light having been dimmed and filtered by a soulless, crushing capitalism so much so that they unconsciously embrace their sameness, adopting a uniform of pique polo shirts and khaki pants.

The iconoclasts, the individualists, the thinkers--these bear the nome d'guerre "Rubyist", "Rails aficionado", "Python hacker", "open source developer". You will know them not by their similarity but their diversity, their vestiture definitively anti-establishment and their mocking of any tool that you don't have to download from github and compile from source.

As in all things, there is a middle road whose pedestrians are accused of compromise with the Devil by one side and "just trying to be different" on the other. Yet in this camp we find many sagacious, experienced craftspeople who have spent time in both camps. With the reader's indulgence, I shall call them hipster programmers.

For what its worth, I consider myself a hipster programmer, as evinced by how I spent my morning. Having been developing in .NET for nearly a decade and of recent years drawn strongly to functional programming, I am of course learning F#. But, I'm also experimenting with Erlang, Ruby, and more recently Clojure. As far as platforms go, you've got a lot of choice these days: Windows, Linux, Mac OS X. With Windows you certainly need Cygwin and a git port. Linux is probably the easiest to get going on but lacks some of the day-to-day niceties (e.g. iTunes). Mac OS X can't be (legally) virtualized, so it makes an obvious choice as the base system.

My work machine is a MacBook Pro running VMWare Fusion. My recent explorations of Clojure have kept my exclusive in the Java tools domain--Maven, Ant, Netbeans, etc., but the Chicago Clojure user group recently invited Dr. David Miller to come talk about clojure-clr, a C# implementation of Clojure for the CLR, very hipster! What could be cooler than that?

Well, porting clojure-clr to F# of course!

Sure, I could fire up Fusion and VS 2010 and start rocking, but I like to begin with the end in mind, namely moving seamlessly between writing code in Clojure targetting the CLR and the JVM. I'd like to use similar tool chains. And, I'd like to present to a group of the cool kids in their native tongue.

The first step is getting an environment setup. Robert Pickering has a great article to get Mono setup on Mac OS X. Don't skip the 'sudo install-mono.sh' step. At the time of this writing their is a strange error in his instructions regarding the mono.snk file. The install-mono.sh file makes this pretty clear, so just ignore that part. Basically, you have to give the F# assemblies a proper strong name to add it to the GAC.

Now, I love vim--it was my first code editor--but the Ruby kids have been using TextMate, so let's use that since someone kindly shared an F# bundle. That bundle uses iTerm to evaluate function in F# interactive, so we'll need that too. Finally, we need to alias "fsi" to run F# interactive using Mono, so the TextMate bundle will run it properly. Edit ~/.profile adding this command.


alias fsi='mono ~/path/to/fsi.exe'

So we've got the beginnings of a build environment going; what's next?

Get a branch going on github, referencing the clojure-clr project

Get a build system stack setup (Maven? Rake?)

Find an unit testing suite for F#/Mono/Mac OS X

Start hacking

REST: The Uniform Interface

2010-08-09T17:16:00.001-04:00

The central feature that distinguishes the REST architectural style from other network based styles is its emphasis on a uniform interface between components. (Fielding’s Dissertation, Sec. 5.1.5)

Street RESTers will immediately submit that the uniform interface is simply having a URL that reads: “/controller/method/id”. While this maps nicely to their server-side MVC application, it does not represent the necessary constraints to satisfy the uniform interface of REST.

In the dissertation Fielding defines four interface constraints for REST:

Identification of resources
Manipulation of resources through representations
Self-descriptive messages
Hypermedia as the engine of application state

The benefits realized by the uniform interface are best understood in terms of the effects upon the architecture when applying these constraints.

Identification of Resources

To understand this constraint we must first have a solid definition of a resource. A resource is some named entity that is provided by our application. On the web we generally use URLs to name these entities. The normal notion of a URL is “a link to a web page”. A URL is a special kind of Uniform Resource Identifier. REST constrains the URIs we use in a few ways:

The semantics of the mapping of a URI to a resource must not change. So, while the contents of example.com/Top10 can change over time, the thing that it names—e.g. the top 10 examples of the day—cannot.
A resource’s identity is independent of its value. So, two resources could point to the same value at some point in time, but they are not the same resource.
The provider of a resource is solely responsible for maintaining the semantic validity of the URI. This just means that we should choose good URLs that are easy to maintain.
A URI should not contain any reference to the media type used to represent the resource; example.com/Top10/json is verboten.

This seems like a simple constraint, and it is, but its significant benefits include:

There is just one way to get at a particular resource.
The value of a resource at a point in time (representation) can be served up in any appropriate media type at the time it is requested, based on the characteristics of the request (Accept header).
Since the semantics of resource identifiers are static, and the media type of the representation is determined at the time of the request, clients dependent on a resource do not have to change any identifiers in order for the content type to change.

Manipulation of Resources through Representations

The abstract notion of a resource named by a URI is reified by a representation in a media type selected based upon the nature of the request for that resource. A representation contains the data and metadata describing the data, such as the media type of the data. All of the perceived work done by a server in REST architecture is initiated by a client either: a) requesting a resource whereupon the server returns a representation of that resource; b) sending a representation of a resource whereupon the server nominally mutates/creates the resource.

There are additional details to the semantics of representation-based interactions, but the salient point is that URLs are not akin to call sites in a program. These interactions are more akin to message passing.

An important and often overlooked consequence of this constraint is that there is no distributed consistency, and so the notion of a “transactional REST” is anathema:

REST just says that there is no consistency -- only representations that indicate state at some point in the past and an implicit grant of use for some time into the future. –Fielding on rest-discuss

Besides decoupling the resource from a particular representation, the benefit to this approach are seen in the application of the next two constraints.

Self-Descriptive Messages

All of the details required to route, interpret, and process a message must be in the message itself. This enables the communication between components to be stateless and allows messages to be cached appropriately.

REST concentrates all of the control state into the representations received in response to interactions. The goal is to improve server scalability by eliminating any need for the server to maintain an awareness of the client state beyond the current request. (dissertation)

Hypermedia as the Engine of Application State

The model application is therefore an engine that moves from one state to the next by examining and choosing from among the alternative state transitions in the current set of representations. (dissertation)

This is probably the least applied of the REST constraints, especially with the proliferation of AJAX clients. The core idea is that the representation given to the client will have embedded hyperlinks that completely disambiguate what actions are available to interact with the server.

Dr. Fielding wrote an extended exposition of the hypermedia constraint wherein he makes explicit some rules that apply to a RESTful API. He also says this,

A truly RESTful API looks like hypertext. Every addressable unit of information carries an address, either explicitly (e.g., link and id attributes) or implicitly (e.g., derived from the media type definition and representation structure). Query results are represented by a list of links with summary information, not by arrays of object representations (query is not a substitute for identification of resources).

Lest you think you can apply REST to your architecture sans this “hypermedia constraint”, Dr. Fielding further clarifies.

ROA is supposed to be a kind of design method for RESTful services, apparently, but most folks who use the term are talking about REST without the hypertext constraint. In other words, not RESTful at all. REST without the hypertext constraint is like pipe-and-filter without the pipes: completely useless because it no longer induces any interesting properties. (blog)

Towards a RESTful Client

Taken together, the constraints that engender the uniform interface of REST-style architecture, in turn force certain characteristics in clients of a RESTful API. Among these client characteristics are:

Ability to process one or more media types available for a resource’s representation.
Ability to maintain and manage state by selecting among hyperlinks.
Awareness of potential inconsistency of representations.
Only enters the interaction from a known entry point.

If we are building RESTful web applications, we have to supply such a client that can run in a browser. For example, the user agent can navigate to the URL that identifies the client. This client is assembled in the browser via normal mechanisms for displaying a web page. The various JavaScript libraries loaded then augment the capabilities of the browser to exhibit the characteristics mentioned above while interacting with the RESTful API on the server.

Dynamic Dispatch (Multimethods) in C#?

2010-07-25T04:34:00.001-04:00

I’ve recently become enamored with the multimethods system of Clojure, as well as its approach to polymorphism and “type” hierarchies in general. Having never heard the term, I consulted Wikipedia about multimethods, hoping to have its origins elucidated, though it appears it is simply a synonym of multiple dispatch.

Polymorphism, “the ability of one type to appear as and be used like another type”, is limited to inheritance with simple overloading semantics (and generics) in C#. The ability to be “used as” another type is implemented by allowing subclasses to implement any virtual methods defined on their respective superclasses. Then, at compile time, the method to be called is determined based on the actual types of the objects upon which the function/method is invoked; we call this compile-time binding or early-binding.

Let’s look at it from a more mechanical perspective. In most OO languages, when we write obj.Foo() we are implicitly writing (Foo obj). In other words, obj is the first argument in the invocation of Foo; an argument called this in many languages. (See JavaScript’s call/apply.) So, in our inheritance example, the compiler looks at the actual type of this first argument obj (the target of the invocation) when determining the Foo to call. This is called single dispatch.

What about the other arguments to the function? Can we vary which function is called based on the other arguments to a method besides the target (i.e. the first implicit argument)? Well, of course, we can have method overloads that take a different number and/or type of arguments. However, unlike the first argument, there’s no mechanism in C# to evaluate the the derived types of these other arguments to make a dispatch decision; there is no multiple dispatch in C#.

This blog article, “The Visitor Pattern and Multiple Dispatch”, usefully explains the problem in terms of the Visitor pattern. As a means of implementing double dispatch the Visitor pattern has some shortcomings. First, the targets must be aware of and receive the visitor, violating the single responsibility principle. Second, the visitor itself must evaluate the type of the target and invoke the correct method. Though some suggest using reflection to invoke the correct method, that implementation, reflecting on the runtime type of the object and using dynamic invocation through the type to get to the right method can be expensive.

We can simplify that approach using the dynamic keyword to effect dynamic dispatch. Using the dynamic keyword to “box” the arguments to the target method (Foo above), we can let the runtime system do the reflection for us. In the code below, I build on the Visitor example in “The Visitor Pattern and Multiple Dispatch” article, see line 28.

Code Snippet
 class Program
{
    abstract class Expression { }
 
    class ConstantExpression : Expression
    {
        public int constant;
    }
 
    class SumExpression : Expression
    {
        public Expression left, right;
    }
 
    class EvaluateVisitor
    {
        public int Visit(Expression e)
        {
            throw new Exception("Unsupported type of expression"); // or whatever
        }
 
        public int Visit(ConstantExpression e)
        {
            return e.constant;
        }
        public int Visit(SumExpression e)
        {
            return Visit(e.left as dynamic) + Visit(e.right as dynamic);
        }
    }
    
    static void Main(string[] args)
    {
        var one = new ConstantExpression { constant = 1 };
        var two = new ConstantExpression { constant = 2 };
        var sum = new SumExpression { left = one, right = two };
        var vistor = new EvaluateVisitor();
        Console.WriteLine("Visit result {0}", vistor.Visit(sum));
        Console.ReadKey();
    }
 

This technique has some utility but should be used wisely. Obviously there will be some cost for “dynamic” dispatch. It’s important to note that this isn’t a generalized system for multiple dispatch, just a great spot welding technique to make the Visitor pattern more palatable.

In contrast, Clojures multimethods allow you to define a function on the arguments that is evaluated and cached, while the “overloads” define the results of that evaluation that they correspond to. In this way dispatch on multimethods in Clojure can consider not only the “types” and values of all the arguments, but really any sort of inspection or evaluation you choose.

Using the Web to Create the Web

2010-07-20T22:24:00.001-04:00

Wikis do this, as do blogs.

Fast JavaScript in browsers is enabling a new generation of programmers to develop applications completely in their browser. While the most obvious commercial example is Force.com, there are many other ideas out there. In no particular order:

The common connection between these frameworks is the notion of bootstrapping the web; that is, using the web to create the web.

If you’ll forgive the inchoate thoughts, let me attempt to connect some mental dots.

Dr. Alan Kay has of late been discussing the SmallTalk architecture of real objects (computers) all the way down and how this might improve the nature of software on the Internet.

In September 2009 in an interview, Dr. Kay said,

The ARPA/PARC research community tried to do as many things ‘no center’ as possible and this included Internet […] and the Smalltalk system which was ‘objects all the way down’ and used no OS at all. This could be done much better these days, but very few people are interested in it (we are). We’ve got some nice things to show not quite half way through our project. Lots more can be said on this subject.

This month in an interview with ComputerWorld Australia, Dr. Kay expounded,

To me, one of the nice things about the semantics of real objects is that they are “real computers all the way down (RCATWD)” – this always retains the full ability to represent anything. The old way quickly gets to two things that aren’t computers – data and procedures – and all of a sudden the ability to defer optimizations and particular decisions in favour of behaviours has been lost.

In other words, always having real objects always retains the ability to simulate anything you want, and to send it around the planet. If you send data 1000 miles you have to send a manual and/or a programmer to make use of it. If you send the needed programs that can deal with the data, then you are sending an object (even if the design is poor).

And RCATWD also provides perfect protection in both directions. We can see this in the hardware model of the Internet (possibly the only real object-oriented system in working order).

You get language extensibility almost for free by simply agreeing on conventions for the message forms.

My thought in the 70s was that the Internet we were all working on alongside personal computing was a really good scalable design, and that we should make a virtual internet of virtual machines that could be cached by the hardware machines. It’s really too bad that this didn’t happen.

Is OOP the wrong path? What is this RCATWD concept really about? Doesn’t the stateless communication constraint of REST force us to think of web applications in the browser as true peers of server applications? Should we store our stateful browser-based JavaScript applications in a cloud object-database, in keeping with the Code-On-Demand constraint of REST? Can we make a them “real objects” per Dr. Kay? Are RESTful server applications just functional programs? If so, shouldn’t we be writing them in functional languages?

I definitely believe we can gain many benefits from adopting a more message-passing oriented programming style. I would go so far as to say that OO classes should only export functions, never methods. (They can use methods privately of course, to keep things DRY.)

I’ve written extensively in a never published paper about related topics: single-page applications, not writing new applications to build and deliver applications for every web site, intent-driven design, event sourcing, and others. Hopefully I’ll find the time to return to that effort and incorporate some of this thinking.

RavenDB: In the Code, Part 1—MEF

2010-07-20T21:14:00.001-04:00

If you’ve not heard of RavenDB, it’s essentially a .NET-from-the-ground-up document database taking its design cues from CouchDB (and MongoDB to a lesser degree). Rather than go into the details about its design and motivations, I’ll let Ayende speak for himself.

Instead, I would like to document some of the great things I’ve found in the codebase of RavenDB, as I read to be a better developer. This series of articles discusses RavenDBs use of the following .NET 4 features.

Managed Extensibility Framework (MEF)
New Concurrency Primitives in .NET 4.0
The new dynamic keyword in C# 4

While discussing RavenDB’s use of these features, I hope to provide a gentle introduction to these technologies. In this, the first post of the series, we discuss MEF. For a very brief introduction to MEF and its core concepts, see the Overview in the wiki.

Managed Extensibility Framework

MEF was originally in the Patterns & Practices team and has since moved into the BCL as the System.ComponentModel.Composition namespace. Glenn Block has nominated it as a plug-in framework, an application partitioning framework, and has given many reasons why you may not want to attempt to use it as your inversion-of-control container (especially if you listen to Uncle Bob’s advice). RavenDB uses MEF to handle extensibility for it’s RequestResponder classes.

RavenDB’s communication architecture is essentially an HTTP server that has a number of registered handlers of requests, not unlike the front-controller model of ASP.NET MVC. Akin to MVC’s Routes, each RequestResponder provides a UrlPattern and SupportedVerbs to identify those requests it will handle. A given RequestResponder will vary it’s work depending on the HTTP verbs, headers, and body of the request. It is in this sense that RavenDB can be considered RESTful (even if it isn’t, see street REST).

Code Snippet
 public class HttpServer : IDisposable
    {
        [ImportMany]
        public IEnumerable<RequestResponder> RequestResponders { get; set; }
 

This HttpServer class dispatches requests to one of the items in the RequestResponders. This is populated by MEF because of the ImportManyAttribute. MEF looks in its catalogs and finds the RequestResponder class is exported, as is all of it’s subclasses; see below.

Code Snippet
 [InheritedExport]
public abstract class RequestResponder
 

The InheritedExportAttribute ensures that MEF considers all subclasses of the attributed class are themselves as exports. So, if your class inherits from RequestResponder and MEF can see your class, it will automatically be considered for each incoming request.

How does MEF “see your class”? Out-of-the-box MEF provides for the definition of what is discoverable in a number of useful ways. RavenDB makes use of these by providing it’s own MEF CompositionContainer.

Code Snippet
 public HttpServer(RavenConfiguration configuration, DocumentDatabase database)
{
    Configuration = configuration;
 
    configuration.Container.SatisfyImportsOnce(this);
 

Above, in the constructor of the HttpServer class, we see the characteristic call to SatisfyImportsOnce on the CompositionContainer. This instructs the container to satisfy all the imports for the HttpServer, namely the RequestResponders. The configuration.Container property is below:

Code Snippet
 public CompositionContainer Container
{
    get { return container ?? (container = new CompositionContainer(Catalog)); }
 

And the Catalog property is initialized in the configuration class’ constructor like this:

Code Snippet
 Catalog = new AggregateCatalog(
    new AssemblyCatalog(typeof (DocumentDatabase).Assembly)
    );
 

So the container is created with a single AggregateCatalog that can contain multiple catalogs. That AggregateCatalog is initialized with an AssemblyCatalog which pulls in all the MEF parts (classes with Import and Export attributes) in the assembly containing the DocumentDatabase class (more on that later).

That takes care of the built-in RequestResponders, because those are in the same assembly as the DocumentDatabase class. If that smells like it violates orthogonality, you are not alone. But, I digress; what about extensibility? How does Raven get MEF to see RequestResponder plugins?

The configuration class also has a PluginsDirectory property; in the setter, is the following code.

Code Snippet
 if(Directory.Exists(pluginsDirectory))
{
    Catalog.Catalogs.Add(new DirectoryCatalog(pluginsDirectory));
}
 

So, in Raven’s configuration you can specify a directory where MEF will look for parts. That’s the raison d'être of MEF’s DirectoryCatalog, since a plugins folder is such a common deployment/extensibility pattern. You can learn more about the various MEF catalogs in the CodePlex wiki.

Now, the real extensibility story for RavenDB is its triggers.

RavenDB Triggers

The previously mentioned DocumentDatabase class is responsible for the high-level orchestration of the actual database work. It maintains four groups of triggers.

Code Snippet
 [ImportMany]
public IEnumerable<AbstractPutTrigger> PutTriggers { get; set; }
 
[ImportMany]
public IEnumerable<AbstractDeleteTrigger> DeleteTriggers { get; set; }
 
[ImportMany]
public IEnumerable<AbstractIndexUpdateTrigger> IndexUpdateTriggers { get; set; }
 
[ImportMany]
public IEnumerable<AbstractReadTrigger> ReadTriggers { get; set; }
 

Following the same pattern as RequestResponders, the DocumentDatabase calls configuration.Container.SatisfyImportsOnce(this). So, the imports are satisfied in the same way, i.e. from DocumentDatabase’s assembly and from a configured plug-ins directory.

In RavenDB triggers are the way to perform some custom action when documents are “put” (i.e. upsert) or read or deleted. RavenDB triggers also provide a way to block any of these actions from happening.

Raven also allows for custom actions to be performed when the database spins up using the IStartupTask interface.

Startup Tasks

When the DocumentDatabase class is constructed, it executes the following method after initializing itself.

Code Snippet
 private void ExecuteStartupTasks()
{
    foreach (var task in Configuration.Container.GetExportedValues<IStartupTask>())
    {
        task.Execute(this);
    }
}
 

This method highlights the use of the CompositionContainer’s GetExportedValues<T> function, which returns all of the IStartupTasks in the catalogs created in the configuration object.

Conclusion

We’ve seen three important extensibility points in RavenDB supported by MEF: RequestResponders, triggers, and startup tasks. Next time, we’ll look at two more—view generators and dynamic compilation extensions—while learning more about RavenDB indices.

iPad: The InterPersonal Computer

2010-06-21T15:40:00.001-04:00

There’s no shortage of information on the “how” of the iPad. Apple’s reification of Alan Kay’s Dynabook makes no sacrifices in terms of processing, communications, display—even the audio is surprisingly good. But what does the A4 system-on-a-chip, IPS display, Wifi/Bluetooth/3G add up to in terms of experience?

Having spent a week with the iPad, I feel compelled to write down my answers to that question. The iPad is nothing short of a joy in my home. It’s the device we didn’t know we needed: the fourth screen. It’s the morning paper, the evening magazine, and the after dinner board game. It’s the vacation photo album, the argument settler, and the cookbook. The iPad is the first interpersonal computer (iPC); the PC has artfully been disguised as an intelligent, portable screen that facilitates rather than stymies interpersonal interaction.

Why did we need this device? Surely I could use Wikipanion on my phone to settle the debate on the national language of Côte d’Ivoire during the game. But, I couldn’t show you the map of the region from across the room. I definitely could have turned on my PC and connected my TV via DLNA to show our vacation photos. But I’d rather just hand you the album to scan at your leisure. We could get all the tiles out, flip them over, mix them up, and arrange them, but board games are much more fun (and more apt to be played) when you don’t have to set them up or put them away. I could have done an internet search for recipes that included lemon balm and printed one out, but it’s nice to just go straight from searching to cooking. If we had kids the raison d'être of this latter-day Dynabook would be handsomely fulfilled by an interactive periodic table, a sketchbook, musical toys, and a huge library of books. For now we’ll just have to settle for loving these apps as grown-ups.

This is a device for kids of all ages, to be sure. Each app acts as a mask, transforming the iPad to a device well-suited to the task at hand. Who wouldn’t find something to enjoy?

I’d be remiss if I didn’t offer some lament or prognostication on future enhancements. This device would be truly magical if I didn’t have to plug it in and synch with iTunes. If my photos, videos, and music were just wirelessly transported from the cloud on demand and cached locally, I wouldn’t have to wait forever to synch or chew up a bunch of space with things I rarely want.

I would love it if the apps knew me. Maybe a fingerprint scanner could be added to help applications identify me; when I launch a game or Twitter client, if I swiped my finger it could load my saved game or timeline. An iPC should be like a family friend, a unique relationship with each of us, but impartial and accessible to all.

AT&T+Apple could score quite the coup if the 3G was free up to a certain level of usage. How many snowbirds would buy an iPad to stay in touch with the family back home? How many more business travelers would be able to keep up with their inbox without having to be nickel-and-dimed all the time? More importantly, they would have a device that would be complete out-of-the-box; thank you for purchasing this magical screen that is connected to everyone, everywhere, anywhere you are—right now.

The Law of Demeter and Command-Query Separation

2010-05-17T11:36:00.001-04:00

My first encounter of the Law of Demeter (LoD) was in the Meilir Page-Jones book, Fundamentals of Object-Oriented Design in UML. It is also referenced in Clean Code by Robert C. Martin. Basically, the law states that methods on an object should only invoke methods on objects in their immediate context: locally created objects, containing instance members, and arguments to the method itself. This limits what Page-Jones calls the direct encumbrance of a class; the total set of types a class directly depends upon to do its work. Martin points out that if an object effectively encapsulates it’s internal state, we should not be able “navigate through it.” Not to put too fine of a point on it, but the kind of code we are talking about here is:

Code Snippet
 static void Main(string[] args)
{
    var a = new A();
    a.Foo().Bar();
}
 
class A
{
    public B Foo()
    {
        // do some work and yield B
    }
}
 
class B
{
    public void Bar()
    {
 
    }
}
 

Martin calls line 4 above a “train wreck” due to its resemblance to a series of train cars. Our Main program has a direct encumbrance of types A & B. We “navigate through” A and invoke a method of B. Whatever Foo() does, it is not effectively encapsulating it; we cannot change it to an implementation that uses C transparently.

LoD is a heuristic that leverages the encumbrance of a type to determine code quality. We observe that effective encapsulation directly constrains encumbrance, so we can say that the Law of Demeter is a partial corollary to an already well known OOP principle: encapsulation. Another such principle is Command-Query Separation (CQS) as identified by Bertrand Meijer in his work on the Eiffel programming language.

CQS simply states that methods should either be commands or query; they should either mutate state or return state without side-effects. Queries must be referentially transparent; that is you can replace all query call sites with the value returned by the query without changing the meaning of the program. Commands must perform some action but not yield a value. Martin illustrates this principle quite succinctly in Clean Code.

Referring to our snippet above again, we can see that if CQS were to have been observed, Foo() would return void and our main program would not have been returned an instance of B. CQS thus reinforces LoD, both of which manifest as specializations of OO encapsulation. Following these principles force us to change the semantics of our interfaces, creating contracts that are much more declarative.

CQS has many implications. Fowler observes that CQS allows consumers of classes to utilize query methods with a sense of “confidence, introducing them anywhere, changing their order.” In other words, CQS allows for arbitrary composition of queries; this is a very important concept in functional programming. Queries in CQS are also necessarily idempotent, by definition; this is extremely important in caching.

In Martin’s discussion of LoD, he notes that if B above were simply a data structure—if all of its operations were Queries—we would not have a violation of LoD, in principle. This is because LoD is really only concerned with the proper abstraction of Commands. From the C2 wiki,

In the discussion on the LawOfDemeter, MichaelFeathers offers the analogy, "If you want your dog to run, do you talk [to] your dog or to each leg? Further, should you be able to manipulate the dog's leg without it knowing about it? What if your dog wants to move its leg and it doesn't know how you left it? You can really confuse your dog."

To extend the analogy, when you are walking your dog, you don’t command it’s legs, you command the dog. But, if you want to have a smooth walk, you’ll stop and wait if you one of the dog’s legs is raised. It is in this sense that we can restate the Law of Demeter in terms of Command-Query Separation.

Whereas a Query of an object must:

never alter the observable state of the object, i.e. the results of any queries;
and return only objects entirely composed of queries, no commands.

A Command of an object must:

constrain its actions to be dependent upon the observable state, i.e. Queries, of only those objects in its immediate context (as defined above);
and hide the internal state changes that result of its actions from observers.

This restating is helpful in that it implies that we refactor to CQS before applying LoD. In the first phase we clearly separate commands from queries. In the second phase we alter the semantics of our commands and queries to comply with LoD. While the first phase is rather mechanical, it give us a good starting point to reconsider the semantics of our objects as we bring them into compliance with LoD.

Usability Apothegms

2010-04-22T16:04:00.001-04:00

A common saying in computing is that “Security is inversely proportional to usability”… or something like that. As we critical examine the security of our systems, we realize we need to put measures in place that make the system harder to access and thus harder to use. A good interaction design can help mitigate the usability issues, but at the end of the day a system that doesn’t require me to memorize a password or login is easier to use than a system that does.

We can say definitively that security and usability exist in tension.

As software architects we seek simplicity in our designs in the name of maintainability, if not intelligibility. We also seek modularity in the name of reusability. I submit that simplicity and modularity exist in tension.

Accepting a priori that simplicity is the absence of complexity, we can obtain the simplicity of a program by measuring its complexity. A field of computer science called algorithmic information theory defines the complexity of something to be the length of the simplest program for calculating it. We might infer from this that a monolithic program (no components, no objects, no abstractions, etc.) is a simpler program than our common object-oriented code. In general we can say that modularity implies no small increase in the use of abstractions to enable that modularity.

In object-oriented* systems, an increase in modularity results in a proportional increase in complexity.

I limit this to object-oriented systems purposefully. In my experience functional programming languages modularity is de rigueur.

Converting Oranges to Apple’s: Meta-competition in the Platform Wars

2010-04-12T19:53:00.001-04:00

Updates to the developer agreement in the new iPhone SDK restrict developers to using C, C++, or Objective-C to create their native iPhone applications. That is, they must have been “originally written” in one of those languages.

This move has made its rounds among the pundits and bloggers. Rather than rehashing any of those points, I’ll give my own opinions on why Apple made this move.

Make no mistake, this is about stewardship of the iPhone experience. Steve Jobs has responded to the criticism (emphasis my own),

[…] intermediate layers between the platform and the developer ultimately produces sub-standard apps and hinders the progress of the platform.

Before you go say they’re really just trying to kill Flash, let’s address that up front. Flash does not run in Safari on the iPhone for the same reason that you are prohibited in the developer agreement from hosting a virtual machine in your iPhone application. As the steward of the platform, Apple has to attempt to ensure that users have consistently good experience. Trying to squeeze a bloated swf player into a memory and processor constrained device is just not workable currently. As any iPhone user can attest, websites that are not optimized for mobile browsers are horribly slow to load. Flash content would just make that worse. And, guess what, my mom doesn’t know that your website or Flash animation is a pig, she just thinks her phone is slow.

Let’s look at it from another perspective. The latest incarnation of Microsoft’s flagship development suite, Visual Studio 2010, has been re-written on their WPF platform. Well, everything except the splash screen. You see, it takes too long to load WPF and .NET up to affect the desired result of a splash screen—giving you something to look at while the main application loads. The iPhone developer guide makes it abundantly clear that you should load something as quickly as possible to give the user the sense that your app is responsive. Try explaining to your mom that, yes, her phone has registered her tap on the application icon, but that it has to spin up a virtual machine or load some large libraries that abstract away Cocoa.

Next, try to explain to her why the extra cycles required by the VM and/or interoperability libraries drain her battery. She’ll just be annoyed that she’s constantly plugging the darn thing in.

Apologies to all the technical moms out there.

so, Flash [1] is not the target here, at least in the sense that Apple cares if Flash is around or perceives it as a threat. And, assuredly, we cannot say that stewardship of the performance of the phones is the only reason. That’s part of Steve’s “sub-standard apps” argument, and it is not entirely convincing given that it becomes an optimization problem, and developers are notoriously good at those when they set their mind to it. As a steward though, you really cannot afford to wait from them to figure it out. Nevertheless, I think the original ban on virtual machines and this subsequent tightening to a whitelist of languages comes down to a stewardship of a different kind: keeping out the riff-raff.

Keeping up with AppStore submissions is already pretty hard and very expensive. Developers are often frustrated by the time it takes to get their updates into the store, especially developers used to patching websites with no downtime. Don’t imagine for a second that your $99 covers the cost of the program. This is Apple’s investment in the ecosystem. Imagine the deluge of useless, crappy submissions they will get if every Flash developer and every VB developer can just tick a box to target the iPhone.

But, it’s not just about raising the bar for entry to keep out sub-standard developers and their apps. They have to look after all those developers who have committed to their platform. They aren’t going to invest significant resources to enabling C#, VB, Python, Ruby, or Flash developers to compete against their own.

Apple is not interested in seeing your app run on a Droid, Windows Mobile 7, and iPhone. They are actively controverting homogenization of the mobile application marketplace. They have an insurmountable lead[2] in the mobile applications marketplace, and the iPad, another device on the iPhone OS platform, marks the next step in their overall competitive strategy for leveraging that lead. At the center of all of this is the AppStore, perhaps the single most valuable asset in computing today. Whereas no one controls the Internet, Apple owns the AppStore outright with all the rights and responsibilities that entails; think of Salesforce.com and their AppExchange.

Microsoft has long understood that developers are what make their ecosystem work. Steve Ballmer’s famous rallying cry, “Developers! Developers! Developers!”, is poignant illustration of this reality. Apple understands that to usurp Microsoft’s position in the PC and enterprise markets, to continue their domination of consumer segments, they need a large population of developers that understand their platform. What they don’t need are more ways for developers to “write once, run everywhere.” In the platform wars, he who has the developers wins.

[1] Anyway, it isn’t just about Flash. Look no further than Microsoft’s Windows Mobile 7 announcements to see that. Silverlight, a Flash competitor to say that least, will be the native platform of those devices. Where Flash is Silverlight will surely follow.

[2] I believe that Google will continue to have limited success as a web-native device. There is a significant segment of the marketplace that just want a phone, email, web-browsing device. RIM has an Apple-style loyalty thing going for it. Microsoft? Well, they still have such a big footprint; they could fail and still make a huge impact. I think the move to Silverlight helps shield their effort from their internal Windows-Office power structure enough that it actually has a chance of competing, but they are way too far behind to win. There’s not a huge population of Silverlight developers in the world, after all.

Inchoate Thoughts on New Programming Abstractions for the Cloud

2009-12-29T19:23:00.001-05:00

I watched an InfoQ video of Barbara Liskov at OOPSLA this evening. At the end of her talk in a look at the challenges looking forward she said, “I’ve been waiting for thirty years for a new abstraction mechanism,” and, “There’s a funny disconnect in how we write distributed programs. You write your individual modules, but then when you want to connect them together you’re out of the programming language, and sort of into this other world. Maybe we need languages that are a little more complete now so that we can write the whole thing in the language.” Her talk was an interesting journey through the history of Computer Science through the lens of her work, but this notion connected with something I’ve been thinking about lately.

Earlier this month I was catching up on the happenings on Channel9 and stumbled on this conversation between Erik Meijer and Dave Thomas. I would have preferred a different interview-style, but Dave Thomas’ opinions are always interesting. “I find it hard to believe that people would build a business application in Java or C#. […] The real opportunity is recognizing that there is really no need for people to do all of the stuff they have to do today. And, I really believe that is fundamentally a language problem, whether that’s through an extended SQL, or its a new functional language, whether its a vector functional language; I can see lots of possibilities there. […] I see a future where there are objects in the modeling layer, but not in the execution infrastructure.”

I see a connection between Liskov’s desire “new abstraction mechanism” and Thomas’ “language problem”. If you look at the history of “execution infrastructure”, there has been an unremitting trend toward greater and greater abstraction of the hardware that makes it all happen. From twiddling bits, to manipulating registers, to compiled languages, to compiled to virtual machine bytecode and attendant technologies like garbage collection, interpreters, to very declarative constructs, we continually move away from the hardware to focus on the actual problems that we began writing the program to solve in the first place(1). Both Liskov and Thomas are bothered by the expressivity of languages; that is, programming languages are still burdened with the legacy of the material world.

I think this may very well be the point of “Oslo” and the “M” meta-language. One might view that effort as a concession that even DSLs are too far burdened by their host language’s legacy to effectively move past the machine programming era into the solution modeling era. So, rather than create a language with which to write solution code, create a language/runtime to write such languages. This is really just the logical next step, isn’t it?

I didn’t quite understand this viewpoint at the time, but I had this discussion to some extent with Shy Cohen at TechEd09. I just didn’t grasp the gestalt of “Oslo”. This certainly wasn’t a failing of his explanatory powers nor—hopefully—my ability to understand. Rather I think it’s because they keep demoing it as one more data access technology. The main demo being to develop an M grammar of some simple type and have a database created. Again, the legacy of computing strikes again. To make the demo relevant, they have to show you how it works with all your existing infrastructure.

So, maybe the solution to developing new abstractions is to once and for all abstract away the rest of the infrastructure. Both Liskov and Thomas were making this point. Why should we care how and where the data is stored? Why should most developers care about concurrency? Why should we have to understand the behemoth that is WCF in order to get to modules/systems/components talking to each other?

Let’s start over with an abstraction of the entirety of computing infrastructure(2): servers, networking, disks, backups, databases, etc.. We’ll call it, oh, I don’t know… a cloud. Now, what properties must a cloud exhibit in order to ensure this abstraction doesn’t leak? I suggest we can examine why existing abstractions in computing have been successful, understand the problems they solve at a fundamental level, and then ensure that our cloud can do the same. Some candidates for this examination are: relational databases, virtual machines/garbage collectors, REST, Ruby on Rails, and LINQ. I’m sure there are countless other effective abstractions that could serve as guideposts.

Should we not be able to express things naturally in a solution modeling language? Why can’t I write, “When Company X sends us a shipment notification for the next Widget shipment, make sure that all Whatzit production is expedited. And, let me know immediately if Whatzit production capacity is insufficient to meet our production goals for the month.” Sure, this is a bit imperative, but it’s imperatively modeling a solution, not instructions on how to push bits around to satisfy the solution model; in that sense it is very, very declarative. I believe the cloud abstraction enables this kind of solution modeling language.

How about a cloud as the ambient monad for all programs? What about a Google Wave bot as a sort of REPL loop for this solution modeling language? What about extensive use of the Maybe monad or amb style evaluation in a constraint driven lexicon so that imperative rules like those in the sample above are evaluated in a reactive rather than deterministic fashion?

Quantum computing productively remains undecided as to “how” it actually works. That is, “how” is somewhat dependent on your interpretation of quantum mechanics. Maybe an infinite number of calculation occur simultaneously in infinite universes with the results summed over via quantum mechanical effects.
I should say here that I think the Salesforce.com model is pretty far down this path, as is Windows Azure to a lesser extent. Crucially, neither of these platforms mitigate the overhead of persistence and communications constructs.

System.Tuple: More .NET 4.0 Functional Goodness

2009-12-21T08:52:00.001-05:00

In functional languages, tuples are pretty much a requirement. Of the functional languages I’ve used, including Erlang and T-SQL, tuples are bread’n’butter. Every release of .NET has included more and more support for functional programming paradigms, and we’re getting tuple support in the BCL in .NET 4.0, though first-class language support is still not there. Here’s an example of using System.Tuple with the new Zip LINQ operator.

Code Snippet
 var xs = new List<int> { 1, 3, 5 };
var ys = new List<string> { "hello", "world", "channel" };
var zs = xs.Zip(ys, Tuple.Create);
 
foreach (var z in zs)
    Console.WriteLine("{1}:{0}",z.Item1, z.Item2);
 

Quicksort Random Integers with PLINQ

2009-12-17T23:31:00.001-05:00

The first “homework assignment” of Dr. Erik Meijer’s functional programming course on Channel 9 was to write quicksort using C# (or VB) functional comprehensions. I did it in a very naïve way, then I wondered if I could get a “free” speed up simply by leveraging some of the Parallel Extensions to .NET 4.0.

My results were mixed. As the number of integers in the unsorted array increases, the parallel implementation begins to pull away, but with smaller numbers the sequential implementation is faster. I think I’ll try to do something very similar in Erlang this weekend. It should be an interesting comparison exercise since Erlang terms are immutable. A naïve implementation should look pretty similar I think. Neither of them would be true quicksort implementations, since the algorithm was specified by Hoare to work in a mutable fashion with the original array being changed in situ.

My dual core machine yielded times of about 80s and 100s when sorting an array of a thousand for the parallel (QsortP) and sequential (Qsort) methods respectively. So that’s definitely not impressive, but I did it in a very abstract way. The same function could sort strings or floats or anything else that implements IComparable.

Code Snippet
 using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Diagnostics;
 
namespace ConsoleApplication3
{
    class Program
    {
        static void Main(string[] args)
        {
            int[] ints = new int[100];
            var rand = new Random();
            for (int i = 0; i < ints.Length; i++)
                ints[i] = rand.Next(0, Int32.MaxValue);
            int[] sortedP;
            int[] sorted;
            Time(() => sortedP = QsortP(ints).ToArray());
            Time(() => sorted = Qsort(ints).ToArray());
            Debugger.Break();
        }
 
        public static IEnumerable<T> QsortP<T>(IEnumerable<T> arr) where T : IComparable
        {
            if (arr.Count() == 0)
                return arr;
            T pivot = arr.First();
            var sortLess = new Task<IEnumerable<T>>(() => QsortP(arr.Where(v => v.CompareTo(pivot) < 0)));
            var sortMore = new Task<IEnumerable<T>>(() => QsortP(arr.Where(v => v.CompareTo(pivot) > 0)));
            sortLess.Start(); sortMore.Start();
            var equal = arr.AsParallel().Where(v => v.CompareTo(pivot) == 0);
            return sortLess.Result.Concat(equal).Concat(sortMore.Result);
        }
 
        public static IEnumerable<T> Qsort<T>(IEnumerable<T> arr) where T : IComparable
        {
            if (arr.Count() == 0)
                return arr;
            T pivot = arr.First();
            return Qsort(arr.Where(v => v.CompareTo(pivot) < 0))
                .Concat(arr.Where(v => v.CompareTo(pivot) == 0))
                .Concat(Qsort(arr.Where(v => v.CompareTo(pivot) > 0)));
        }
 
        public static void Time(Action a)
        {
            var t = new Stopwatch();
            t.Start();
            a();
            t.Stop();
            Debug.Write(t.Elapsed.TotalSeconds);
        }
    }
}
 

Premature Optimization and LINQ to SQL

2009-12-15T14:57:00.001-05:00

Don’t count your chickens before they hatch. Let’s call that adage the “count-no” principle. I promise that will be funny later.

The common interpretation of this adage is that you should not rely on future results when making decisions in the present. It is generally an admonishment to not be too cocky about the future. What might the corollary be? Perhaps to concern ourselves with the details of the present moment rather than thinking about the future. We’ll call this corollary “nocount”. One specific example of this corollary is the programmer’s maxim about premature optimization. In fact, it was Donald Knuth who said, “"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil,” in a response to Dijkstra’s “Go to statement considered harmful”.

So, I’m debugging a particularly pernicious problem today. In production (and only production) we were getting a ChangeConflictException when updating a row in the database. Unfortunately, the logs indicated that there were no MemberChangeConflict objects. In other words we weren’t violating optimistic concurrency. So, I profiled the SQL statements and it appeared to be just what I expected, a simple update supporting optimistic concurrency.

Did you know you can eek out an exceedingly miniscule amount of performance by using the “SET NOCOUNT ON” option with SQL Server? Did you know that you can actually set this option at the server level? Do you know what happens when your LINQ to SQL DataContext uses the row count returned from your optimistically concurrent update statement to determine if one and only one row was updated?

Yes dear reader, premature optimization can cause the most insidious and difficult to discover problems. Please, don’t ever violate the “nocount princple”!

Create an XML Schema Enumeration of Supported .NET CultureInfo with Powershell

2009-11-10T13:44:00.001-05:00

Here’s a Powershell script to generate the enumeration elements of an XML Schema simpleType restriction. The first command loads the .NET Assembly you’ll need, sysglobl.dll.

PS>  [System.Reflection.Assembly]::Load("sysglobl, Version=2.0.0.0, Culture=neutral, PublicKeyToken= b03f5f7f11d50a3a, processorArchitecture=MSIL")

PS> [System.Globalization.CultureInfo]::GetCultures([System.Globalization.CultureTypes]::FrameworkCultures)    
| ? {$_.Name.Trim().Length -eq 5} | % { "<xs:enumeration value=`"$_`"/>"} | Out-File cultures.txt

See my earlier post on locale identifiers for more background.

Whoa!!! HP Software Sucks!

2009-10-30T20:17:00.001-04:00

This is the error that was reported after the HP installer forced me to restart my computer to continue installation…

TFS: Locked for Check-out

2009-10-15T10:12:00.001-04:00

Today a co-worker was attempting to edit a file, and TFS reported that it was “locked for check-out by [me] in workspace …”.

I didn’t have it checked out. I tried to use “tf lock /lock:none” on the file in question, but it would simply report:

TF10152: The item … must remain locked because its file type prevents multiple check-outs.

We don’t allow multiple checkouts for this Team Project. Unfortunately, this meant that not only didn’t I intend to lock the file, I couldn’t unlock it.

My solution was to just check-in some dummy changes, comments full of expletives if you must know, since TFS will not perform a check-in unless there are actually changes to the files.

So yeah, not TFS’ most shining moment.

Cloud Morphology

2009-08-06T00:01:00.001-04:00

Spent some time this evening tracking down the various component technologies of the myriad cloud offerings out there. Here’s the result. This will be updated over time.

NASA’s Cloud Platform: NEBULA, and Amorphous Morphology

2009-08-04T17:45:00.001-04:00

NEBULA is a Cloud Computing environment developed at NASA Ames Research Center, integrating a set of open-source components into a seamless, self-service platform. It provides high-capacity computing, storage and network connectivity, and uses a virtualized, scalable approach to achieve cost and energy efficiencies.

They always write it in all caps, though it doesn’t appear to be an acronym. My guess is that it’s easier to search for “NEBULA” and find what you want at NASA. And, what better name for a cloud computing platform?

To be frank, I didn’t recognize any of the technologies other than RabbitMQ and Apache. So I set out to find out what each piece does, and it’s these times that really make me envious of the LAMP folks. There is so much going on in the OSS world!

In fact, it’s that sheer volume of innovation that makes it such a nightmare. Here’s where having NASA or Amazon or Elastra or EngineYard or Rackspace or Google AppEngine discuss their respective cloud infrastructures is so beneficial. If you look at their technology choices as a Venn diagram, you might find an optimal set of technologies in the overlap. Beyond that, you can begin to form a generalized blueprint of a cloud platform, identifying the core components.

In this way the cloud begins to take shape. Were we to sit all the big industry players down and say, define the cloud al a the OSI model, no one would have gotten around to building one. If we find the clouds in the wild, we can dissect them and find out what makes them tick. We can reverse engineer our blueprint.

There’s a few major players missing here, of course, most notably Salesforce.com (SFDC) and Microsoft’s Azure platform. SFDC doesn’t have an infrastructure that you can replicate, per se, because it is inseparable from their proprietary technologies. In fact, Stu Charlton and others have noted that this is their biggest failing. Similarly, Microsoft’s Azure is also a monolithic platform play, with the major difference that the platform can be deployed on premises, a so-called private cloud.

Still though, we can include SFDC and Azure in our analysis. We can develop our morphology of the cloud platforms that have published their constituent technologies. With the language of this morphology in hand, we can classify the facilities of other cloud platforms. Perhaps a taxonomy will evolve that allows us to identify Platform-as-a-Service, Infrastructure-as-a-Service, Software-as-a-Service, hybrids, and perhaps some novel species.

Despite the amorphous character implied by the term cloud computing, these platforms have well-defined structure. Moreover, unlike the traditional use of clouds in architecture diagrams, the details of this structure are important.

Powershell: Get Size of Child Directories

2009-08-02T23:33:00.001-04:00

gci -recurse -force |
? { $_.GetType() -like 'System.IO.DirectoryInfo'} |
     select-object Name,@{Name="size"; Expression = {
    ($_.GetFiles() |
    Measure-Object -Property Length -Sum |
     Measure-Object -Property Sum -Sum).Sum}} | sort-object -Property size

Three Common Fallacies Concerning REST

2009-07-08T14:09:00.001-04:00

Better to light a candle than to curse the darkness. -Chinese Proverb

I purposely did not title this post “The 3 Fallacies of RESTful Computing” as I am certainly not an expert in either REST or fallacy. :) I am, however, quite well-versed in auto-didacticism, and over the past week I’ve been boning up on REST. Along the way I’ve had one of my early notions of REST disabused (it is neither an architecture nor a protocol) and noticed a few other common misconceptions in the blogosphere and tweet stream. If you are new to REST, or even if you aren’t, you might very well find a few edifying points in this post; I hope to light a candle or two out there.

Without further ado, here are three common fallacies concerning REST.

Fallacy #1 REST is CRUD

Perhaps the most common fallacy of RESTful computing is that REST is simply CRUD (Create, Read, Update, and Delete) over HTTP. Microsoft’s ADO.NET Data Services endeavors to provide developers a “data service being surfaced to the web as a REST-style resource collection[…]”; this would seem to further the notion that REST is another way of doing CRUD.

However, REST is not CRUD as Stu Charlton states in a blog post about REST design guidelines; Arnon Rotem-Gal-Oz says CRUD is bad for REST, too. If we are going to attempt to abridge RESTful architecture with an innocuous statement of the form “REST is X over HTTP” let us say that REST is using URLs to facilitate application state changes over HTTP.

Fallacy #2 POST is not RESTful

First, it is very important to note that REST is not tied to a specific protocol. As an architectural style, it is protocol agnostic; though to be sure HTTP is a natural fit for many reasons. As Fielding said in It is okay to use POST:

Search my dissertation and you won’t find any mention of CRUD or POST. The only mention of PUT is in regard to HTTP’s lack of write-back caching. The main reason for my lack of specificity is because the methods defined by HTTP are part of the Web’s architecture definition, not the REST architectural style.

Since the nominal use of POST is orthogonal to RESTfulness, by definition, it cannot be the case that POST is antithetical to REST. Nevertheless, it is important to understand the reasoning that generally goes into this fallacy, because it speaks directly to a core principle of REST. Most architectures expose an API that allows consumers to affect the state of the application only indirectly. To understand the implications of your actions as a consumer, you then have—at best—to be very familiar with the application architecture and be aware that you are making a lot of assumptions. The state mechanics are hidden from you. You cannot explicitly move the application from one state to another, nor can you directly observe the transition(s) that have taken place.

A primary goal of Representational State Transfer is to make the application’s state machine unambiguous by exposing representations of application resources that have embedded within them URIs pertinent to the resource. In this way a consumer of a RESTful service can discover the current state of the resource, the mechanisms to affect change in the resource, and other resources related to the current resource in the application’s state. This is what is known as Hypertext as the Engine of Application State (HatEoAS).

HatEoAS implies a model where all of your important resources are represented by unique URIs and all important state changes are done by interacting with representations sent to or retrieved from those URIs. Most people first approaching REST view it in terms of opposing other architectural “styles” such as SOA or get mired in implementation immediately and begin contrasting their understanding of REST over HTTP against WS-*. Another common problem is reducing the ethos of REST to “RPC is bad” and “we don’t need all that complexity, we have HTTP” (see REST is better than WS-*). These views are commonplace because REST is being promulgated as a better solution for many types of applications on the Web.

The specifics of how REST works over HTTP are beyond the scope of this article, and the subject of a lot of debate, but, since a lot of uses of POST seem very much like RPC-style invocations, people have a knee-jerk reaction that POST is not RESTful. By now you know this is not the case, but let’s hear from the experts.

From Fielding:

POST only becomes an issue when it is used in a situation for which some other method is ideally suited: e.g., retrieval of information that should be a representation of some resource (GET), complete replacement of a representation (PUT) […]

Stu Charlton’s design guidelines say nearly the same thing:

The problem with POST is when we abuse it by having it perform things that are more expressive in one of the other methods. GET being the obvious one that needs no hypermedia description. For the other methods, a good design guideline is that you MUST not break the general contract of the HTTP method you choose -- but you SHOULD describe the specific intent of that method in hypermedia.

Fallacy #3 REST is better than WS-*

In fact, Fielding’s thesis does address many of the problems and advantages of various network-based architectural styles, but no where does he claim REST is the one ring to rule them all. In his aforementioned blog post Fielding says (emphasis my own),

[…]there are plenty of information systems that can be designed using the REST architectural style and gain the associated benefits. Managing cloud instances is certainly one of those applications for which REST is a good fit[…]

From his thesis, here is the definition of the REST architectural style.

REST consists of a set of architectural constraints chosen for the properties they induce on candidate architectures. […] [It] is an abstraction of the architectural elements within a distributed hypermedia system. […] It encompasses the fundamental constraints upon components, connectors, and data that define the basis of the Web architecture, and thus the essence of its behavior as a network-based application.

So, the associated benefits are induced by constraints that collectively are referred to as the REST architectural style. The benefits are myriad, and many of the constraints are are recognizable in how the web works. Section #5 of the thesis is concise and readable, and I encourage the reader to internalize it.

The salient point here is that REST is not a protocol; it’s not even an architecture. REST is an architectural style! By understanding its constraints and benefits, we can make informed decisions about its applicability to our problem domain and appropriation of technologies. Comparing REST to the WS-* suite of protocols is comparing apples to oranges, though there are those who strongly argue the benefits of REST and HTTP over SOAP.

What, Not How & Why, Not When

2009-06-27T11:28:00.001-04:00

It occurred to me this morning that many software development principles seem to emerge from the rigorous application of the following principle:

Your architecture and code should make What & Why explicit without specifying How & When.

What, Not How
It is well known that we should prefer declarative over imperative code. By observing the Law of Demeter, we are in fact forced to create declarative, What, semantics in our interfaces since we can't get at the How, the imperative constructs. The Single-Responsibility Principle tends to force us to factor out the How into other classes, leaving us to consume other classes with What semantics. Further, the Interface Segregation Principle requires us to explicity segregate the interfaces our clients consume based on What they are trying to accomplish; if our focus was How, such segregation would be less of a concern. Event-Driven Architectures (EDA) are another example of What, Not How. Whereas the SOLID principles operate at the class design level of abstraction, EDA is concerned with system-level architecture. In an Event-Drive Architecture, we explicitly model happenings in the domain. Rather than coupling the site of the happening to a specific party designating for dealing with that happening, we create an Event and use reliabile messaging and subscription schemes to allow one or many services to handle it. In other words, instead of specifying How to deal with a happening at the site that generates it, we explicitly model What happened and let other parties worry about How to deal with it.
Why, Not When

Pareto, Zipf, Heap: The 80-20 Rule, Language, and Diminishing Returns

2009-06-26T14:33:00.001-04:00

Please consider this a sort of layman’s disambiguation page.

The classic “80-20 rule” refers to a Pareto distribution. The thin distribution of the 20% is the subject of Chris Anderson’s “The Long Tail”. Originally, the Pareto distribution referred to the fact that 20% of the people control 80% of the wealth, but it has turned up in many other contexts.

A Zipf’s Law is about rankings and frequency. The second item’s frequency will be half of the first; the third’s frequency will be half the second, and so on. The “half” may be some other factor, but it remains constant in the distribution; the rank is inversely proportional to the frequency. The item in Zipf’s Law is a word and its frequency is its appearance in a corpus of English text. However, Zipf’s Law holds for texts generated from a fixed alphabet by picking letters randomly with a uniform distribution.

Heaps’ Law is about diminishing returns. It has an exact formula, but generally it says that the more you look into a text, the fewer new discoveries of words you’ll find. So, as you read through the text it takes longer and longer to find new words in the text. Heaps’ Law applied to the general case where the “words” are just classifiers of some collection of things. So, it could be applied to the nationality of a collection of people; you’d have to gather more and more people from a random sampling to get a representative from all countries.

The implications of these laws in various contexts are the subject of much interesting study and postulation.

Value Objects and Value Domains

2009-06-16T21:29:00.001-04:00

I’ve written relatively extensively on the topic of replacing enum constructs with classes, so I won’t rehash the topic. Instead, I’d like to introduce you to some code I’ve written that enables you to create finite domains of Value Objects in C#. Please see my two previous posts on the subject to learn more about the benefits of this approach. To see how the Value Object semantics and making a finite domain a first class concept in the pattern improves the approach, read on.

First, we need some definitions. We are taking the concept of a Value Object from Eric Evans’ book “Domain Driven Design” p.99:

When you care only about the attributes of an element of the model, classify it as a Value Object. Make it express the meaning of the attributes it conveys and give it related functionality. Treat the Value Object as immutable. Don’t give it any identity […] The attributes that make up a Value Object should form a conceptual whole.

An example here is useful. Consider modeling the domain of a book publisher. At some point you need to capture all the different formats of books: A4, B, Pinched Crown, etc. The attributes of width, height, unit of measure, and name would go into your model. But, any instance of your A4 Value Object should be completely interchangeable from any other A4. And, it goes without saying that you can’t change the width or height or any other attribute of A4.

All of the different formats of books belong to a Value Domain. According to WordNet, a domain (we are using the mathematical notion) is:

the set of values of the independent variable for which a function is defined

The functional complement of a domain is the range of the function; i.e. domain is what you can put in, the range is what you can expect to get out. I like calling this concept in my approach a domain instead of a set, because it neatly captures a key benefit. When we are writing code. We want to declare our parameters as being a proper member of domain of values, instead of just primitive types or strings.

Now we’re ready to dive into the implementation. Let’s begin with the end in mind. I want to write a function, say, that let’s me search for a document with a particular format about an arbitrary topic.

void SearchDocs(DocFormat docFormat, string topic)

Ok, so we could create a base class or an interface called DocFormat and create Word doc, PDF, etc. that inherit from or implement DocFormat. Easy. But, SearchDocs has to be able to handle all current and future implementations of DocFormat; it must not violate the Liskov substitution principle. What if the repository or search algorithm depends on the what the doc format actually is? Also, we’d have a subclass of DocFormat to write for every document type, and we’d have to do a lot of work to remove object identity, since your instance of PDF is not the same as my instance of PDF. And, don’t forget to make the whole thing immutable. [Note: I know this is a contrived example with well-established OOP solutions that don’t require LSP violation. Work with me here. :)]

Clearly we have a lot of work to do to make our Value Objects a reality. It’s not impossible, though, and a quick ~~Bing~~ Google turns up a couple of investigations and approaches. Jimmy Bogard’s approach gave me a clue that I needed to get it working. What I wanted was a base type, ValueObject, that I could inherit from and get the Value Object semantics described in Evans.

Jimmy’s approach used a self-referencing open constructed type to allow the ValueObject base class to do all the work of providing Value Object semantics. This base class uses dynamic reflection to determine what properties to use in the derived class to do equality comparisons (an approach nominally improved upon here). He saw his approach as having a single fundamental flaw; it only worked for the first level of inheritance, i.e. the first descendent from ValueObject.

For my purposes—creating a bounded, finite domain of Value Objects to replace enum-type classes--this is not a flaw. Substantively, all that remains to do is introduce the concept of the the Value Domain into Jimmy’s approach and put the Value Objects in it. Because I wanted to use these Value Domains throughout the enterprise, I baked WCF support into my approach. Further, because the Value Domain is defined a priori, I didn’t have to play with object identity; I could simply look the value up in the domain. (It took a little out-of-the-box thinking to get that to work transparently.) Finally, I wanted it to be trivially easy to create these Value Domains, so I created a snippet for use in Visual Studio.

Here’s an example of Value Domain and Value Objects implementation:

[DataContract]
public sealed class DocFormats : ValueObject<DocFormats.DocFormat, string, DocFormats>.Values<DocFormats>
{
    private DocFormats()
    {
        Word = Add("Word");
        PDF = Add("PDF");
    }

    [DataMember]
    public readonly BindingType Word, PDF;

    [Serializable]
    public sealed class DocFormat : ValueObject<DocFormat, string, DocFormats>
    {
        private DocFormat(string v) : base(v) { }
    }
}

DocFormats is the Value Domain. DocFormat is the type of the Value Object. String is the underlying type of the values, but the pattern supports anything that implements IEquatable and IComparable. Methods can accept DocFormats.DocFormat as a parameter and only Word and PDF will be valid. Code can specify those values through a Singleton accessor: DocFormats.Instance.PDF. You’ll notice that the only constructor is private; the Singleton implementation is in the base value domain base class (ValueObject<…>.Values<…>).

// our new method interface
void SearchDocs(DocFormats.DocFormat docFormat, string topic)
{
    // referencing an individual value
    if(DocFormats.Instance.Word == docFormat)
    {
        //...
    }
}

Above you’ll note that the Value Object type definition (subclass of ValueObject<…>) is nested inside of the Value Domain. Doing that groups the two together syntactically in a very natural way (FooTypes.FooType); the entire domain of values is contained in one place. That locality makes the snippet to produce them cohesive too. [Note: I’ve included the snippet XML at the end of this post.]

Interestingly, the underlying implementation necessarily inverts this nesting; the Value Domain base class is nested in the Value Object base class. That allows the Value Domain Singleton to create instances of Value Objects. I only had to resort to reflection in order to allow the Value Domain base class to provide the Singleton implementation. Inversion of Control containers like Unity and Castle Windsor do this kind of reflection all the time, and it’s cheap since .NET 2.0.

Without further ado, here’s the base classes implementation.

using System;
using System.Collections.Generic;
using System.Reflection;

[Serializable]
public abstract class ValueObject<T, TValue, TValues> : IEquatable<T>, IComparable, IComparable<T>
    where T : ValueObject<T, TValue, TValues>
    where TValue : IEquatable<TValue>, IComparable<TValue>
    where TValues : ValueObject<T, TValue, TValues>.Values<TValues>
{
    /// <summary>
    /// This is the encapsulated value.
    /// </summary>
    public readonly TValue Value;
    protected ValueObject(TValue value)
    {
        Value = value;
    }

    #region equality
    public override bool Equals(object other)
    {
        return other != null && other is T && Equals(other as T);
    }

    public override int GetHashCode()
    {
        // TODO provide an efficient implementation
        // http://www.dotnetjunkies.com/weblog/tim.weaver/archive/2005/04/04/62285.aspx
        return Value.GetHashCode();
    }

    public bool Equals(T other)
    {
        return other != null && Value.Equals(other.Value);
    }

    public static bool operator ==(ValueObject<T, TValue, TValues> x, ValueObject<T, TValue, TValues> y)
    {
        // pointing to same heap location
        if (ReferenceEquals(x, y)) return true;

        // both references are null
        if (null == (object)(x ?? y)) return true;

        // auto-boxed LHS is not null
        if ((object)x != null)
            return x.Equals(y);

        return false;
    }

    public static bool operator !=(ValueObject<T, TValue, TValues> x, ValueObject<T, TValue, TValues> y)
    {
        return !(x == y);
    }
    #endregion

    public static implicit operator TValue(ValueObject<T, TValue, TValues> obj)
    {
        return obj.Value;
    }

    public static implicit operator ValueObject<T, TValue, TValues>(TValue val)
    {
        T valueObject;
        if (Values<TValues>.Instance.TryGetValue(val, out valueObject))
            return valueObject;

        throw new InvalidCastException(String.Format("{0} cannot be converted", val));
    }

    public override string ToString()
    {
        return Value.ToString();
    }

    #region comparison

    public int CompareTo(T other)
    {
        return Value.CompareTo(other);
    }

    public int CompareTo(object obj)
    {
        if (null == obj)
            throw new ArgumentNullException();

        if (obj is T)
            return Value.CompareTo((obj as T).Value);

        throw new ArgumentException(String.Format("Must be of type {0}", typeof(T)));
    }

    #endregion

    [Serializable]
    public abstract class Values<TDomain> where TDomain : Values<TDomain>
    {
        [NonSerialized]
        protected readonly Dictionary<TValue, T> _values = new Dictionary<TValue, T>();

        private void Add(T valueObject)
        {
            _values.Add(valueObject.Value, valueObject);
        }

        protected T Add(TValue val)
        {
            var valueObject = typeof(T).InvokeMember(typeof(T).Name,
                           BindingFlags.CreateInstance | BindingFlags.Instance |
                           BindingFlags.NonPublic, null, null, new object[] { val }) as T;
            Add(valueObject);
            return valueObject;
        }

        public bool TryGetValue(TValue value, out T valueObject)
        {
            return _values.TryGetValue(value, out valueObject);
        }

        public bool Contains(T valueObject)
        {
            return _values.ContainsValue(valueObject);
        }

        static volatile TDomain _instance;
        static readonly object Lock = new object();
        public static TDomain Instance
        {
            get
            {
                if (_instance == null)
                    lock (Lock)
                    {
                        if (_instance == null)
                            _instance = typeof(TDomain)
                                .InvokeMember(typeof(TDomain).Name,
                                                  BindingFlags.CreateInstance |
                                                  BindingFlags.Instance |
                                                  BindingFlags.NonPublic, null, null, null) as TDomain;
                    }
                return _instance;
            }
        }
    }
}

I’d love to hear your feedback. This is approach is not intended to support unbounded Value Domains, such as the canonical example of Address. It is meant for the enums-as-classes problem, i.e. finite, bounded Value Domains.

ValueObject.snippet

<?xml version="1.0" encoding="utf-8" ?>
<CodeSnippets  xmlns="http://schemas.microsoft.com/VisualStudio/2005/CodeSnippet">
    <CodeSnippet Format="1.0.0">
        <Header>
            <Title>ValueOjbect</Title>
            <Shortcut>vo</Shortcut>
            <Description>Code snippet for ValueObject and domain</Description>
            <Author>Christopher Atkins</Author>
            <SnippetTypes>
                <SnippetType>Expansion</SnippetType>
            </SnippetTypes>
        </Header>
        <Snippet>
            <Declarations>
                <Literal>
                    <ID>domain</ID>
                    <ToolTip>Value domain name</ToolTip>
                    <Default>MyValues</Default>
                </Literal>
        <Literal>
          <ID>value</ID>
          <ToolTip>ValueObject Class name</ToolTip>
          <Default>MyValueObject</Default>
        </Literal>
        <Literal>
          <ID>type</ID>
          <ToolTip>Value Type</ToolTip>
          <Default>string</Default>
        </Literal>
            </Declarations>
            <Code Language="csharp">
        <![CDATA[    
    [DataContract]
    public sealed class $domain$ : ValueObject<$domain$.$value$, $type$, $domain$>.Values<$domain$>
    {
        private $domain$()
        {
            Value1 = Add(String.Empty);
        }

        [DataMember]
        public readonly $value$ Value1;

        [Serializable]
        public sealed class $value$ : ValueObject<$value$, $type$, $domain$>
        {
            private $value$($type$ v) : base(v) { }
        }
    }
]]>
            </Code>
        </Snippet>
    </CodeSnippet>
</CodeSnippets>

Google Wave

2009-05-28T16:57:00.001-04:00

Upon the day mankind's children ask
Who it was made their world intelligible,
Those who set about the grand task
of creating the Wave will be eligible.

For never will they have occasion to think
how to create discourse, share knowledge,
Wonder about the universe and link
Without the this wonderful prodigy.

Like hieroglyphs in the tombs of Kings,
eee-may-uhls voy-s'-may-uhls eye-ems and tweets
will serious study to the curious bring,
to be shared in Waves while ancestors sleep.

How is it the first step of a journey
Can put the past so far behind thee?

Connascence: The Underlying Principle of OO Design?

2009-05-25T13:34:00.002-04:00

This is a great talk at a Ruby conference by Jim Weirich about his attempt to frame all object-oriented design (OOD) principles as special cases of an underlying principle called “connascence”. Connascence is a term co-opted by Meilir Page-Jones in the ‘90s for use in OOD; below is the definition from his book with Larry Constantine entitled, “Fundamentals of Object-Oriented Design in UML” (page 214):

Connascence between two software elements A and B means either

that you can postulate some change to A that would require B to be changed (or at least carefully checked) in order to perserve overall correctness, or

that you can postulate some change that would require both A and B to be changed together in order to preserve overall correctness.

UPDATE: Jim Weirich's talk can be found here.

WolframAlpha: A New Kind of Site

2009-05-21T11:24:00.001-04:00

TechCrunch called Google Squared the imminent Cain to WolframAlpha’s Abel. Dare Obsanajo warns Wikipedia to beware the ides of March, casting WolframAlpha as Brutus. But, I would cast WolframAlpha in quite a different role: Hamlet.

If you’ve not used WolframAlpha, head over there come back once you’ve exhausted its novelty. Go ahead, I’ll wait.

So, now you know why they call it a “computational knowledge engine” whose mission it is to “make the world’s knowledge computable”. Ignoring, for the present, the interesting epistemological discussion viz. the validity of that mission, let’s instead talk about why you might use it.

If you are looking for something, go to Google. This is not a search engine.

If you are looking to mine the web for hard data, be patient, Google Squared will be your guide (when it is baked).

If you would like to learn about a particular topic, Wikipedia stands ready to help you begin.

If you want to apply your existing domain knowledge to posing interesting questions about a wide variety of topics, if you can phrase such questions as nominal computations, then go to WolframAlpha, though your attempts might be frustrated, e.g. it fails to compute the following “annual energy consumption of the average American * projected worldwide population in 2020.”

In other words for 98% of the world, WolframAlpha is simply a curiosity. For the other 2%, it’s only useful as a way to experiment with such questions, since the rigor required of formal academia is not satisfied by simply saying, “WolframAlpha said so.”

To understand WolframAlpha’s raison d'être, you have to understand NKS: Stephen Wolfram’s “New Kind of Science”. To understand NKS you can either go read the book or trust what I’m about to say. At its core, the claim it makes to novelty is based on the notion that all phenomena can be viewed as computations, and thus real science can be done by exploring the “universe” of possible computations. Of course the only apparatus sufficient for doing this “new” science is Wolfram’s own Mathematica program, a program originally created by Dr. Wolfram in a single summer. Twenty or so years later after leaving academia to build the company that develops and sells Mathematica, Wolfram brought forth the the 1,200+ page tome that is NKS.

Wolfram is no lightweight; we’re talking about one of those few people in the world with the intellectual capacity and training to do modern quantum mechanics at age 17—when it comes to mathematics he is the real McCoy. But, very few seriously look at NKS as being fundamentally new, and no few academics treat Wolfram with vitriol and ire due to his claim of novelty. We all—it is universally agreed—stand upon the shoulders of giants. Further, many argue that NKS contributes nothing new to the existing corpus of mathematics or science. Here are some notes for the dramaturg:

The NKS book was self-published by Wolfram-Media
Wolfram opens the book with a statement of his childhood dream to “know everything”
NKS is the culmination of twenty years of work outside of academia
The precipitating event that put him on that path was his not being able to understand the pattern generated by a cellular automata (simple computer program) he’d written

My personal belief is that the cognitive dissonance created by having not contributed to science in any recognized, significant way over twenty years, in the face of soaring hubris born out of his prodigious intellect and early summiting of some of the highest peaks of academia, forces Dr. Wolfram to eschew—even denigrate—the trappings of mainstream academia and embrace his self-created role of the father of a new kind of science. But, hey, I don’t know the guy; and it goes without saying that I’m criticizing Michael Jordan’s cross-over. Maybe it will take twenty years for everyone else to get it; if it is new, it's nascent, and you have to walk before you can crawl.

He may be an egoist, but he has excellent taste. I think he’d make Edward Tufte proud.

What WolframAlpha owes its life to is the attempt to make NKS immediately relevant in a roundabout way. Remember, everything—everything—is simply a computation. Another concept that is critical to understanding WolframAlpha from NKS is “computational equivalence”; essentially this is the reason why NKS doesn’t permit predicting the future: to do that you would need a computer the equivalent of our universe. Reality, you see, is simply the current result of the continuous computation being performed by the fabric of our universe. (As strange as that may sound, it is not dissimilar from the beliefs of proponents of the much vaunted string theories of the universe.)

Since we don’t have a computer the size of the world, we can’t calculate everything in reality. What we can do, though, is examine the current results of the computation, in the form of data that can be collected and mined and extruded by domain experts into an symbolic ontology that is computable by Mathematica. We can then run computations with these synthetic ontologies, because they can be related via formal ontology of units we have created throughout human history. To what end? Well, I suppose that depends on the answers you get.

I hope by now you understand why it is called WolframAlpha. And, for those lovers of the theater who haven’t yet guessed, here is the cast of characters.

Hamlet: WolframAlpha
Fortinbras: Google Squared
Gertrude: Capitalism
Ophelia: Wikipedia
Claudius: Academia

I think it is obvious who plays the role of the murdered king. Ok, I've surely stretched the metaphor a bit too thin, but I really like the notion that these kinds of current events are manifestations of the same archetypes described in our great literature. I think Wolfram would like it too, but I have no doubt his protagonist wouldn't be from one of Shakespeare's tragedies.

Microsoft TechEd: Day 0

2009-05-11T01:05:00.001-04:00

It’s the evening before the official start of the Microsoft’s annual IT conference, TechEd. In 2009 we find ourselves in sunny, yet cool, Los Angeles. I couldn’t be happier with the weather and the hotel, but so far the conference sucks.

Why—or, more appropriately, how can I say it sucks when the conference hasn’t begun? Easy, go to msteched.com and attempt to build a schedule. Well, you have to be signed in with Windows Live and a registered attendee I believe, but if you could and did you would quickly discover a rather pathetic fail on the part of the conference planners. The session builder is garbage. You’ll spend the majority time watching this animation, since they’ve not seemed to master the partial page update without updating the entire 670K+ page weight. Seriously, I’d rather look at the source and see and XML Data Island than wait for the entire page to reload every time I updated my schedule.

Aesthetically, it isn’t bad, the site I mean, but the session builder is garbage. The session builder for Web 2.0 Expo at least displayed all the tracks and timeslots in a calendar-like fashion, so you could make your decision by sight and rather quickly.

But, you know, if the content is there, if the sessions are just awesome, who cares about some clunky, hacked-together conference website? Don’t judge a book by its cover, right? So, what are my options for tomorrow’s 11am timeslot? Well, the first keynote is this one.

KEY01 Moving Forward Together: The Potential of IT Innovation

Presenter: Bill Veghte

Mon 5/11 | 10:00 AM-11:30 AM | West Hall A

In today's economic environment, your company is faced with unprecedented pressures to reduce costs and drive operating efficiencies. At the same time, the demands on IT to deliver more connected services and greater flexibility continue to grow. With the upcoming release of Windows 7, Windows Server 2008 R2, and Exchange "14," Microsoft helps IT organizations better meet these competing demands. Join Bill Veghte, Senior Vice President of Windows Business, as we explore how these technologies combine to give you the tools and resources to better manage your infrastructure--from back-end services through client-side experiences, and the network in between.

Doesn’t exactly inspire. But, never fear, there is an alternative!

PAN59 Agile: A Process or an Excuse?

Presenters: Richard Campbell, Stephen Forte, Chris Menegay, Joel Semeniuk

Mon 5/11 | 11:00 AM-12:00 PM | 501C

Over the last few years, the community at large has created a number of agile development styles; Scrum, XP, and more are there for you and your team to choose. However, is agile really a process? On the other hand, is it an excuse to avoid accountability and proper development techniques? Come to this interactive session to see what Chris, Steve, and Joel have to say, and, if you are up for it, share your opinion.

Seriously, I’m not making this up. I came to Los Angeles for this?

The Coming Apocalypse of your "Friends"

2009-04-09T13:52:00.001-04:00

What's a friend on MySpace, Facebook, and Orkut? If you have five hundred friends, are they really friends? These are the main reasons why I have not, to this point, joined any of those sites. I don't have many friends, because I just can't give a lot of people the energy needed to maintain a true, lasting friendship.

What does this have to do with technology? Well, Facebook Connect and Google FriendConnect are two apis, with many more to come, that allow you to take your social graph with you. So now, instead of just being a bunch of people that post something they copy and pasted onto your "wall" at major holidays, they are now people who are going to affect recommendations you get on other websites.

Tell me, do you want five hundred of your "closest" friends opinions, tastes, and web surfing habits to have a direct effect on your web experience?

Web 2.0 Expo Day 2

2009-04-03T15:10:00.001-04:00

This morning started with keynotes, though there didn't seem to be a theme beyond "here is who we could get to come and talk." The sole exception was the very first speaker, Douglas Rushkoff, was fantastic. I'm really excited about his new book, Life Inc. His main theses were that corporations were not created to allow competition and artificial currency was not

Nokia's CEO showed us his vision for ten years from now. Wearable computers? Wow, now that's innovative. He got off on some weird metaphysical rant about projecting our consciousness into the 'net. What he really means is a constant data stream of my location, mood, and random thoughts. The only interesting thing he mentioned was geolocation through photography. I really feel like that is an idea whose time has come.

Next I attended a session that discussed the relative merits of building directly for the iPhone or the web, and the conclusion was use PhoneGap to do both. it had some interesting points in making the case to stay web-based, but none of them were very convincing. He attacked the following five arguments for going native:

Performance--the recent "fast javascript" movement combined with paring down the page bloat that has occurred in recent years
Offline Usage--there is HTML 5 (and thus embedded mySQL support) in iPhone since v2.0
Findability--there are now 25K+ apps in the AppStore
Device Capabilities (GPS, camera, accelerometer, etc.)--the GPS capabilities are "opening up", and PhoneGap let's you go native
Monetization--4% of devices get turned down, people use iPhone apps like tissue paper, you have to sell your app for $0.99 to get any traction

Some valid points there, but it's ironic that the solution proffered is to start web-based and enhance to native.

Microsoft Bizspark were the underwriters for Launch Pad at Web 2.0 Expo. There were five companies who presented:

PhoneGap
80legs
zeaLOG
Dub (at DubMeNow)
Bantam (Bantam Networks)

Honestly, I wish I had gone to a different session. I already knew about PhoneGap from the ten other times it was mentioned this week. What they are doing at 80legs is really cool, but I'm not sure about its applicability. They've made something that is really hard to do really cheap, but they've not made it easy. That is, you still have to write Java code. I'm more excited about what people are doing with it, a sure sign of a platform play. I hope they get acquired and rewarded for their hard work. One last thing, there was maybe a hundred people in the room. Maybe.

I went to a talk from the guys at EngineYard. Pretty good talk, but really centered on the implications of systems management in the could-era. I really enjoyed the talk, pretty fun. The gave it in this paired presenter style where one would pick up where the other left off. Nothing terrible prescient or relevant to me, but they host a lot of really awesome new companies, like github.

Werner Vogels

2009-04-01T21:17:00.001-04:00

Super cool guy. He's one of those super geeks that's past the point where he feels it necessary to get dragged into details but is totally upbeat. It's good to be the king, right? Perhaps my biggest surprise was to learn just how much Amazon views itself as a technology company. Amazon's retail experience is just a customer of AWS. Riiiight. I find that hard to believe, but he claims that Amazon.com the retail site is not even the biggest customer of AWS. Whoa.

My big takeaway from his talk and comments afterward is that only companies that really, really must scale to incredible extremes should build their applications for AWS, i.e. bake in use of SimpleDB, SQS, etc. What he said to me was, "run your Linux infrastructure on EC2 so you can leave if you don't like us." So, really, build great SOA systems and use AWS as a virtualization provider. He had a lot of great principles in his talk for building scalable systems, including:

Autonomy: individual components make decisions only based on local information
Asynchrony: Make progress under all circumstances
Controlled Concurrency: operations are designed such that limmited or no concurrency control is required
Controlled Parallelism: use fully decentralized (p2p) techniques to remove bottlenecks
Decentralize: Remove dependencies
Decompose: Break up into more granular services
Failure Tolerant: Failure is a normal part of operation
Local Responsibility for Consistency: Similar to Autonomy, don't look elsewhere for consistent state
Simplicity: pare away functionality until you cannot
Symmetry: all nodes can do any function in the system

Some other interesting bits were that the Amazon Homepage is produced by 200-300 services. Every service at Amazon is developed and operated by the same team. There are about 500-600 services/teams. Elasticity is both shrinking AND growing on demand.

Geolocation: The new W3C API

2009-04-01T21:04:00.001-04:00

A thought leader in geolocation from the company that created Loki gave a talk about geolocation on the web. His first recommendation was to build your applications to adopt latitude and longitude as the basic unit of location. He then went on to talk about all the other ways location can be represented. For example, the names of business, location descriptions, planar areas (rather than points) on a map, etc. These are generally much more welcome in the UX than map coordinates.

Some ways of getting local businesses are:

Yahoo! Local Search API
Yelp
Localeze

The skinny on geolocation for the web right now is that it relies on browser plug-ins or extensions. The nice thing is that a W3C working group put together a standard for accessing the geolocation capabilities of the browser from Javascript. It's quite easy to use, if not widely available.

One interesting stat is the precision one can expect from the various means of geolocation.

GPS can be accurate to within 10m for commercial applications
Wifi (database lookup and triangulation from SSIDs and signal strength) can be as good as 20m up to 80m
Cell towers are up around 1000m
IP address based schemes are wildly inaccurate and limited to city/region precision

Loki.com/how and Google Gears utilize hybrid approaches to do geolocation, obviously, to balance speed of acquisition, device capability, desire precision, etc.

Web Developer Tools from Web 2.0

2009-04-01T20:49:00.001-04:00

This was by far the best presentation I saw today. A couple of Mozilla guys gave this great talk really about the past, present, and future of the web developer experience. I'm really buzzed about the near future of HTML 5, "fast javascript", and the "web worker" model, enabling the kind of interactivity available on desktop applications.

The introduced a lot of tools, mostly around Firebug and its extensions. I wasn't familiar with the unit testing stuff in that space: Code Coverage and FireUnit. Some of the more interesting tools were the CSS tools: YUI CSS Grid Builder, blueprint, and the recently announced Microsoft SuperPreview.

They also mentioned some "future" tools like Bespin and Thunderhead. Bespin is wonderful, a source code editor on the web, with concurrent-versioning system support (git now and later SVN?). I really feel like this is boostrapping the web, you can begin to think of creating the web ON the web. Avi Bryant has already plugged Seaside into Bespin. Thunderhead is a prototype for marrying grid-layouts with the HTML 5 canvas. They are talking about doing an "Open Tools Directory".

One thing they mentioned that was only tagentially related to their talk was PhoneGap. This let's HTML/Javascript developers create native mobile phone applications (iPhone, Android, and Blackberrry are supported to different extents at this time). Atlas, which let's you write native Cocoa (Mac OS) applications in Javascript (their Objective J) was given an excited nod, too.

Why Scala?

2009-04-01T20:32:00.001-04:00

I'm trying to catch up with blogging about the sessions I've attended this first day of the Web 2.0 Expo. The second session I attended today was entitled, "Why Scala?" Presented by a fellow from Twitter, it was essentially a discussion on their adoption of Scala from their initial appraisal of the technology landscape through the pilot project successes to their company-wide embrace of Scala.

Despite not having any major investment in Java code, the JVM platform gave them access to the myriad Java libraries already out there, as well as the ability to do things like hydrate JSON as a JVM object and reason about it naturally within Scala. Given that they were looking for a language with a great concurrency story, this appears to be the number one reason for choosing Scala, that it is a JVM language written by people who really understand the JVM internals. Some of the other reasons put forth were:

Actor model--like Erlang the primary programming paradigm of Scala is independent workers that share no state and pass messages
Immutability--with built-in immutable data structures and the Actor model, Scala heavily encourages immutability, but doesn't force it, meaning you can take advantage of mutability when it is more expressive or performant
Type inference--yay, strong typing AND pretty code
First-class functions
Traits--not real clear on what this is, I think the Erlang/OTP equivalent would be behaviors
Pattern matching--this is similar to what Erlang does, but isn't limited to tuples or other terms, i.e. you can match objects
XML Literals & Query methods baked into the language
Case Classes--these sound like structs, maybe message types?

The "bad" thing about Scala is that it forces you to learn a bit more about the JVM than perhaps you were inclined to learn, and it is very complex. There seems to be quite a community evolving around Scala, and the mainstreaam web framework is called Lift.

Identity Wars and Joseph Smarr

2009-04-01T19:49:00.001-04:00

I remember Mr. Smarr's talk about OpenID and OAuth from a couple of years ago. I got the impression that his biggest fear was that Facebook would always be a walled garden. For reasons I don't quite understand, perhaps a sort of social consciousness (no pun intended), Facebook has opened their platform up, albeit in a limited and proprietary way.

Fortunately, there are companies like Google and JanRain who are hiding the differences between the Open Stack and Facebook's APIs. First, let's have a look at the Open Stack proper.

OpenID for authentication
OAuth for authorization
XRD a service discovery protocol
Portable Contacts (self-explanatory)
Open Social

JanRain has RPX, and Google has FriendConnect, so using the OpenID/OAuth stuff is easy without really knowing a lot. What is needed is a way to marry the existing authentication stacks of all the common web platforms (ASP.NET, PHP, etc.) with the Open Stack, I think.

For more information, here are some blogs:

TheSocialWeb.tv
JosephSmarr.com
TheRealMcCrea.com

SEO: From Soup to Nuts--Web 2.0 Expo Workshop

2009-04-01T12:22:00.001-04:00

It's workshop day in the first and only pre-conference day of Web 2.0 Expo San Francisco. Among the first sessions of the day, I opted to attend the presentation given by Stephan Spencer of Netconcepts: "SEO: From Soup to Nuts."

His presentation was organized around his "three pillars of SEO": Content, Architecture, and Links. His SEO practice is very data driven as evidenced by his egregious use of statistics. Among the highlights of those statistics:

1.25 to 1.5 times more conversions to PPC versus "natural" (normal search result click-through)
86% of search engine clicks are natural
47% click-through on first Google result falls to 10% on second, and half of the first natural for the first paid
Double click through rate for short URLs according to Marketing Sherpa

It definitely isn't all about statistics in the SEO game, though. In fact, it seemed as if getting hard data about efficacy was the biggest and enduring challenge. There are some hard-and-fast techniques to employ, thankfully. Page titles, using 301s, nofollow links, canonical tags, there should be only one url for any given content, etc., as well as some great tools (both free and pay) including:

SEOmoz.org's LinkScape
SEObrowser.com
Google Web Optimizer
SIFR
Adobe Search Engine SDK
Yahoo! Site Explorer

There were also a lot of surprises, at least to me:

Search result rankings can vary from city to city (of searcher)
Underscores do not count as word separators to Google
YouTube doesn't give link juice (nofollow)
Digg users do not buy anything
User generated content can get you so-called "long tail terms"

One thing I wondered, looking at this image that shows the distribution of eye movement on the Google results page, is what are people actually looking for one Google. They aren't looking for knowledge, but information, a gateway to knowledge, perhaps.

Web 2.0 Expo travel

2009-03-29T12:48:00.001-04:00

We have just been incredibly fortunate this far. I last flew commercial soon after 9/11, and the experience was harsh to say the least. Anotinio Gramsci once wrote that the downfall of a secular democracy was the continuous surrendering of freedoms in exchange for protection from our fellow citizens. Our travel experience to SF this weekend has taught me how much the human spirit can shine despite and perhaps paradoxically because of difficult circumstance. We have been graced by the goodwill of so many of our fellow citizens that I am tacitly aware of Gramsci's error if unable to put words to it.

Today has been incredible, but I must save my emotive prose for another time.

Sex-Software Simile

2009-03-18T22:35:00.001-04:00

Using software vendors' technology demos as the metric for your software development is like measuring your sexual prowess by watching porn films: accurate for edge cases but ultimately not relevant.

Why ORM? Why Cloud Computing?

2009-03-13T13:51:00.001-04:00

I think it was Fred Brooks that once wrote that someone will inevitably look at any software system and ask, "how did we get here?" And, the answer is invariably, "one logical choice at a time." Rather than answer the questions, "Why ORM?" and "Why Cloud Computing?", I will endeavor to elucidate those logical choices.

The monolithic database model (MDM) has served the IT sector very well for about fifty years. I will explicitly define it as a model of software development where myriad applications are integrated through a shared database, or--more accurately--more than one user interface is provided to a database. In this model the database itself is where business processes and domain modeling exist. Enterprises often expanded these databases to include all aspects of the business, or perhaps a few databases integrated via ETL, until the number of objects exceeded any one person's capacity to understand the system. EDI, data warehousing, and data marts are all extensions of the (MDM) for dealing with M&A, e-biz partners, etc. Given a large number of well-trained database professionals with rigorous communication and documentation requirements, the monolithic database model is efficacious and expedient.

As object-oriented programming techniques became more well-known, developers began modeling the domain and business process using advanced general purpose programming languages like Lisp, Smalltalk, C++, etc. They used object databases to persist these models and all was good and productive. Unfortunately, few developers grok those technologies, and even fewer have a mastery of the techniques needed to use them well, so it didn't catch on. Perhaps the biggest impediment to adoption was the success of the MDM. The MDM was working and understood. The deficiencies of the model were insufficient to move IT development en masse to the object crowd.

Still, a lot of software was being written outside of IT. Embedded developers loved object programming and had no need to store loads of data. So, the hardcore programmers--those that relish writing life support systems, flight control systems, nuclear power plant control systems, etc.--were exclusively "object guys". Meanwhile, Sun Microsystems was building a runtime for embedded systems to simplify programming across hardware--Java. A 4GL, Java had all the object-oriented bells and whistles, and Sun had written runtimes for Unix, Solaris, and even Windows! These runtimes came bundled with AWT, allowing developers to nominally "write once, run anywhere."

Suddenly there was the WWW. I say suddenly because even Bill Gates once purportedly remarked that it was irrelevant. As IT understood the implications of Metcalfe's Law for the Internet, it became the elephant in every boardroom. No single company in the world wanted to be a loser in the new, connected economy. All IT decisions were tempered by the Internet revolution.

Now the cohesive, insular world of the MDM was really looking problematic. Still, that's where the data was, and ultimately the Internet was about access to information. So, every IT vendor that wanted to compete was pitching platforms that claimed to do one central task better than anyone else: turn relational data into web pages. These platforms were either backed by scripting or 4GL, and DBI, JDBC, ODBC, ADO, etc., in all their versions and iterations were simply libraries used by the object guys to get at the data.

Concurrently, universities had been turning out software developers who started writing C++ or Java in their freshman year, graduating to "real" object languages in subsequent years. The worldwide demand for IT developers skyrocketed, and technical colleges met the challenge with programs designed to get people into the market with the minimal set of skills to create web applications. At this point the technologies for building web applications were infantile relative to RDBMS technologies. The result was a lot of really bad developers developing really bad applications, slowly.

The good developers, the object guys with strong Lisp kung fu, really resented having to interface with databases--to go from their object model to a data model. IT programmers oft complained of the inordinate amount of time they spent move data in and out of databases. Architects started designing systems built around messaging again, calling them web services this time, and the canon said that these messages were serialization of object state. So smart people started writing libraries for interfacing with databases in a tacit fashion, attempting to abstract away the "implementation details" of data storage and retrieval: object-relational mappers (ORM). The "real" model of business entities and processes, after all, were their objects.

Now, the old IT systems in DB2 and Cobol were "legacy" systems to be tolerated until they could eventually be re-written. New IT initiatives would set out in their own silo (usually an application server), with their own persistence engines that utilized ORM, and integrate with the legacy systems via message-oriented middleware (MOM): Biztalk, SOA, ESB, etc. Vendors were happy because MOM was expensive; object-guys were happy because their jobs now matched up with their education; and business owners were hopefully optimistic because mainframe vendors all operate on an annuity model, and the owners' hope was to escape from a world of multi-million dollar support contracts.

Lo, the MDM guys sneered, since the whole enterprise--the business itself--was still built upon their systems, and every attempt to re-write their systems was an incredibly expensive failure.

Business owners were getting pretty angry, too. All these application servers needed lots of new hardware, and the new servers needed to be partitioned by firewalls, and they needed new database servers too, since RDBMS was still the persistence engine of choice, and of course all of this needed fail-over and clustering. Meanwhile, what the developers were producing wasn't sufficient to run the entire business on, so they now had to maintain two systems in parallel, putting increased strain on the middleware servers. Further, the developer's velocity was slow, since everything had to be integrated, and years and years of process refinements implicit in the MDM had to be re-imagined and re-discovered. This integration stuff was expensive.

Again, though, not all software was being written in IT shops. There were companies building applications hosted on web servers that were available on-demand, so called Application Service Providers (ASP). These companies didn't have any legacy systems and drew the smartest object guys. Many of them were wildly successful, but when the dot-com bubble burst, the idea that a company that you relied upon for mission critical systems could just disappear caused many IT shops to reject the idea out-of-hand. The ASPs that survived were those companies that offered important but not "mission-critical" (like email, production data management, etc.) services, such as sales force automation or marketing support. Another class of ASPs that won were those that offered hosted services to SME customers who could not or chose not to afford the sunk costs of providing their own infrastructure.

In the U.S., capital was king; the financial markets were providing great returns and due to foreign investment and interest rate cuts the cost of capital was low. Businesses were now faced with enormous capital expenditures from their IT operations during a time when capital investments were the most prudent choice, but no one was willing to give up the very real productivity gains that had been made steadily in the last decade. The solution to the dilemma was to defer expenditures, to lease.

At the same time, the business owner's frustration with the lack of agility of their IT operations peaked. The costs were high, the pace of development was slow, the requirements gathering and definition process was pain-staking, and the failure rate was abominable. The maintenance costs alone for all the manifold servers, languages, and platforms was choking the life out of new business opportunities before they could be explored.

So, the market itself demanded cloud-computing. Rather than buying the infrastructure and paying to provision it, they wanted to "pay as you go." Elastic demand on IT systems meant provisioning for the worst-possible scenario; cloud-computing meant scaling-up (and down) on demand. Rather than having to pay for and maintain application servers, database servers, load-balancers, etc., they could just write the app and see if it sticks.

Yea, though I walk through the valley of the shadow of integration, I shall fear no SQL.

The problem of integration remains. No cloud platform fits any enterprise's IT needs completely, so now the challenge is how to integrate a world where the web application server platform is considered a legacy. Microsoft's solution is to position all ASP.NET development as "cloud" development, making it easy to keep the application on-premise or move it to the cloud (Azure). Ironically, the major impediment to this approach is the persistence engine: the database.

Moving a database that fully exploits the RDBMS platform to the cloud is non-trivial. In other words, developers that wrote stored procedures, utilized event brokers or extended stored procedures, created ETL jobs, made it unlikely that they can move their applications to Azure. If they instead had only used pure ORM (LINQ2SQL), this could be done mechanically.

This property of not violating the ORM abstraction, namely the ease with which the backing data store can be mechanically replaced, is the key reason why ORM advocates promulgate the principle that the application should never accept a dependency on the data store. In other words, no stored procedures allowed. The Entity Data Model is an attempt to abstract the universality of the data model--arguably the best feature of MDM--away from the persistent storage engine(s), in order to ensure the portability of the data model in the enterprise and beyond, i.e. to the cloud.

How We Will Develop for the Web in Five Years

2009-03-04T06:24:00.001-05:00

Okay, I admit that I'm predisposed to prognostication, but I'm less inclined to blog if I'm not confident. So comeback in five years so I can say, "Told ya so!"

In five years, nearly everyone who writes code for the web will be using a web-based IDE to create those applications. All of the libraries that you use to create your UI will be hosted on a CDN, freely available to all, and your IDE will be aware of the CDN. You'll add new widgets to your toolbox by specifying CDN URLs.

The MVCC for your web project will be hosted by your cloud infrastructure provider. Unit tests and code coverage requirements will be imposed on check-in, and continuous integration will be de rigueur. The other distributed developers on your team will receive a notification of failed builds in the web-based IDE, and workgroup chat capabilities in the IDE will be used to coordinate a fix.

On the server-side, the distinction between persistent storage and in-flight objects will have all but disappeared. No longer will there be a separate notion of a data store and middle-tier objects. Ad-hoc object composition will be supported via interfaces in the object runtime, as opposed to a query language. Backups and disaster recovery will be outmoded ideas, as the cloud infrastructure provider will ensure redundancy and edge computing performance in myriad datacenters. Data replication between these points-of-presence will be completely automatic and unnoticeable.

The standard application architecture will be server-coordinated real-time continuations. The majority of applications will have just a handful of screens in the UI, with one screen generally accounting for 80% of interactions. These screens will be HTML pages generated primarily using JavaScript UI toolkits. That is, the server will not generate individual pages, but will deliver client-side applications. These applications/screens will interact with the server via RESTful AJAX calls returning JSON that will move forward a state-machine facilitated by continuations on the server. Browser extensions like AIR/Flex, Gears, and Silverlight will be used primarily for their ability to allow offline application interactions, not for flashy graphics or rich-media, but this usage will eventually be deprecated by browser implementations of offline storage in HTML 5.

State machines and workflows will be specified using a constraint language layered on top of the fluent interface/DSL available ipso facto on a platform that requires an object model to get any work done. So, business rules will be clearer and organized, allowing more transparency in the development process.

This state of affairs will require domain-driven design a priori. While user interfaces and the mechanics of interactions could be built ahead of other development, all meaningful interactions will require the construction of workflows out of domain models.

Much of these ideas already exist. Seaside is a great example of browser-based development and server-side continuations. Salesforce.com's Force.com platform imposes code coverage restrictions for production code. LINQ provides an object-runtime method of programming without an explicit query language. Google hosts jQuery and other tools on its CDN. ASP.NET 4.0 stands poised to take client-side templating to the masses. AWS, Azure, AppEngine, and RailYard are providing the infrastructure we need. The pieces exist, what I wanted to do was articulate a vision for the future where these technologies are mainstream, integrated, and proven.

Public Global Resources

2009-02-16T13:32:00.001-05:00

In VS 2008, the addition of a resources (.resx) file to the App_GlobalResources file defaults the custom tool to GlobalResourceProxyGenerator. This tool generate a class (resource_file.designer.cs) that is internal (friend in VB), therefore classes outside of the web project assembly cannot access these resources. The solution is to simply change the custom tool from GlobalResourceProxyGenerator to PublicResXFileCodeGenerator in the properties of the resx files. This tool will create a publicly accessible in the designer file; however, it will also default the namespace to WebProjectName.App_GlobalResources. If you put "Resources" into the Custom Tool Namespace property of the resx file, you will be able to recompile without any problems, and your resources will be accessible outside of the web project's assembly.

One last note, it is my experience that you must also set the Build Action to "Embedded Resource" or you will receive an MissingManifestResourceException in your consuming web project at runtime.

Guilt

2009-01-30T15:11:00.001-05:00

Guilt is the feeling of being separate from our own model of true self; but we should build upon guilt, for it is only possible when we both aspire and look honestly upon our actions.

A Corollary to Ockham's Razor

2008-12-17T18:19:00.001-05:00

The popular phrasing of the maxim known as Ockham's Razor is, "the simplest solution is usually the right one." This is not the original statement or William of Ockham. He might better be paraphrased: "a solution with fewer assumptions is to be preferred." That restatement, though still not precisely correct, captures the spirit but leaves out the concept of simplicity. Karl Popper, the imminent epistemologist, would have us define simplicity as the "degree of falsifiability [sic]" of our solution; i.e., how easy is it to prove the solution incorrect? Popper goes further:

Simple statements, if knowledge is our object, are to be prized more highly than less simple ones because they tell us more; because their empirical content is greater; and because they are better testable.

He could have well been talking about programming! If would follow that declarative programming, currying, constraint programming, and domain-specific languages are to be preferred over alternate paradigms, but so too would an entry to the obfuscated C contest and your average unreadable Perl program. To rectify this seeming contradiction, we need to understand that a "simple statement", according to Popper, is one with fewer parameters--precisely because it is more testable.

So more testable code is better code, and we might induce that a test driven approach is to be preferred, insofar as the practice encourages the writing of simpler, more testable statements. But, Ockham's razor is about solutions, not approaches. So, what can we say about deciding between solutions?

I offer the Programmer's Corollary to Ockham's Razor:

To pick the best solution architecture, choose the one that consistently requires fewest number of parameters.

HOWTO: SQL Server Return Value with Enterprise Library

2008-12-16T18:16:00.001-05:00

    int returncode = 0;
    Database db = DatabaseFactory.CreateDatabase();
    DbCommand dbCommand = db.GetStoredProcCommand("<your procedure>");
    db.AddParameter(dbCommand, "@RETURN_VALUE", DbType.Int32, ParameterDirection.ReturnValue, String.Empty, DataRowVersion.Default, null);
    db.ExecuteNonQuery(dbCommand);
    if (!Int32.TryParse(Convert.ToString(dbCommand.Parameters["@RETURN_VALUE"].Value), out returncode))
        returncode = -1;

The returncode local variable is useful, but generally not necessary. It's good to state your assumptions about the operation of external code explicitly.

Everything Crashes, But Programmers Are Still Jocular

2008-11-12T13:59:00.001-05:00

This is completely off-topic, but I couldn't resist sharing this with anyone and everyone. Whilst researching the frequent crashes on my iPhone, I ran across this post, and the first comment made me chuckle.

Posted by Kee Hinckley
24 July 2007 @ 4am

It appears that if the OS was unable to get a traceback for the crash (which seems to be the case in a lot of my crashes), the exception code given is: 0×8badf00d.

Could be worse. I remember one company getting a confused call from a customer about a 0xdeadbeef crash.

Dealing with Globalization Differences in Windows Versions

2008-11-04T12:36:00.001-05:00

Recently I was charged with leading an effort to localize an ASP.NET application to a few different locales, including The Netherlands and India. I was reminded again how difficult it is to get this right. There are those who argue that using the same view across all the cultures supported in your application is a bad idea. For example, an RTL culture, e.g. Hebrew, when done right, requires a mirroring of the entire view layout: not an easy thing to accomplish without changing the view.

Regardless, the .NET Framework and ASP.NET provide many facilities for localizing your application around a single view: App_LocalResources, App_GlobalResources, CultureInfo, etc. ASP.NET has a particularly helpful feature; you can set the CultureInfo used by the application in the web.config using the globalization element.

There are lots of different ways of representing a culture in Windows: culture name, culture identifier, locale identifier (LCID), and others. Generally, though, the culture name is used, e.g. en-US, en-GB, and es-MX for English (US), English (Great Britain), and Spanish (Mexico), respectively. As the world's second most populous country with many ancient and highly varied cultures, India have no less than nine culture names defined in the .NET Framework. This is, however, a narrow view of India's diversity with its 28 states, 7 union territories, and literally hundreds of spoken languages. Obviously, there isn't 100% coverage of this diversity in the cultures supported in .NET.

One particularly useful culture name in use in Windows is en-IN. In India, the de facto language of the law and by extension government and business is English, due to both the historical influence of British colonization and the need for a lingua franca in such an amazingly diverse country. The en-IN culture code codifies this, making it possible for an application developer to effectively gloss over this diversity in the Indian market. Unfortunately, en-IN is not available in the core .NET Framework release. As you can see from this list, Windows Vista supports this culture, as does Windows 2008.

Obviously, this is problematic for developers doing development on Vista but deploying to Windows 2003 R2. Fortunately, Microsoft developers faced with the incredible diversity of the world's cultures provided a way to customize the available cultures on any Windows installation: the CultureAndRegionInfoBuilder (CARIB) class. There is a standard how-to create custom cultures with this class, but we will take a slightly different approach in this entry. We will export the en-IN culture from our Windows Vista development machine, and import it on our Windows 2003 R2 server.

We'll use PowerShell here, but the examples should be clear enough that you can write your own console or Windows application to execute these steps. Let's begin with exporting the en-IN culture. The CARIB class is in sysglobl.dll, so we need a "reference" to that assembly. We can load the assembly from the GAC in PowerShell. To do so we use its strong name, obtained by using "gacutil /l sysglobl" from a Visual Studio command prompt. In PowerShell on the Vista machine,

PS> [System.Reflection.Assembly]::Load("sysglobl, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL")

Now that we can reference the class, let's export the culture into an industry standard format called Locale Data Markup Language (LDML) version 1.1 using the Save method. We can import this later with the CreateFromLdml method.

PS> $enIN = New-Object System.Globalization.CultureAndRegionInfoBuilder("en-IN", "Replacement")
PS> $enIN.LoadDataFromCultureInfo((New-Object System.Globalization.CultureInfo("en-IN")))
PS> $enIN.LoadDataFromRegionInfo((New-Object System.Globalization.RegionInfo("IN")))
PS> $enIN.Save("enIN.ldml")

Now we can copy enIN.ldml to our server and import it. There are three very important modification we must make to enIN.ldml, however. The en-IN culture is defined in terms of some other Windows locale information that won't be found on Windows 2003 R2, specifically "text info", "sort", and "fallback". If you open enIN.ldml, you'll find the following three elements.

<msLocale:textInfoName type="en-IN" />
<msLocale:sortName type="en-IN" />
...
<msLocale:consoleFallbackName type="en-IN" />

If we try to load a CARIB from this file as-is, we'll receive the following error. For more information, see the section "Exporting Operating System-Specific Cultures" in this CodeProject on-line book.

Culture name 'en-in' is not supported.

We've got to change these to a sensible alternative that is supported on Windows 2003 R2. I chose "en-US", though "en-GB" would've also been appropriate. For other cultures, this may prove more difficult. My changed LDML file contains these lines.

<msLocale:textInfoName type="en-US" />
<msLocale:sortName type="en-US" />
...
<msLocale:consoleFallbackName type="en-US" />

With that done, we can load and register the culture on the Windows 2003 server.

PS> [System.Reflection.Assembly]::Load("sysglobl, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL")
PS> $enIN = [System.Globalization.CultureAndRegionInfoBuilder]::CreateFromLdml("enIN.ldml")
PS> $enIN.Register()

The culture en-IN is now available to all applications running on your server. To see this in action, you could now augment the web.config of you web application with a globalization element:

...
<system.web>
<globalization culture="en-IN" uiCulture="en-IN" />
...

Create a file called i18n.aspx and add the following code:

<%@ Page Language="C#"%>
<% = (10.5).ToString("c") %>

Navigate to i18n.aspx and you should see the following:

Rs. 10.50

I hope this helps!

Using ASP.NET to Write Files to a Remote UNC Share

2008-10-19T19:00:00.001-04:00

Scenario: Let's say you have an ASP.NET site on a public web server. The server itself is in an un-trusted domain, e.g. a workgroup server, and the site needs to write files to a UNC share that is in the domain. There are many reasons you may need to do this, but the difficulty is that you cannot specify NetworkCredentials for the DirectoryInfo class. You cannot, therefore, authenticate against the UNC share programmatically, and since your application is running in an un-trusted domain, the ASP.NET process identity is of no help. If possible, the easiest thing to do is write the file locally and have the system that needs the file pull from there via a UNC. Of course, if you're attempting to share the file across web applications, you'd likely be in the same scenario.

Solution: The solution I have developed for a slightly more complex scenario utilizes a few relatively little known facilities:. Basically, we'll use an ASP.NET FileUpload to receive a file, write it to a temporary location, perform any needed processing, and then use the WebClient class to perform an HTTP PUT to a local virtual directory that points to an UNC share. Let's start with the page that receives the file.

Using the FileUpload control couldn't be easier. Add it to your page and call its SaveAs(string) method to save the file locally. Of course, that doesn't help us get that file to a remote UNC share, does it? What we'll do is save that file to a temporary location on the web server, do any necessary processing, and then upload that file. Here's the code to do that.

    WebClient wc = new WebClient();
    string tempFileName = Path.GetTempFileName();
    fileUpload1.SaveAs(tempFileName);
    wc.UploadFile(uploadUrl, "PUT", tempFileName);

Let's talk about that uploadUrl variable. You are probably already familiar with GET and POST, but you may not have had occasion to use PUT and DELETE. When specifying PUT in an HTTP request, the request URL is the location where the request's file payload should be stored by the web server. IIS 6 supports HTTP PUT requests via its WebDAV module, but the functionality is disabled by default. Turn it on the same way you turn on ASP.NET, by clicking Allow after selecting WebDAV in the Web Extensions in IIS manager. Once it's turned on, the IIS server hosting your application will now accept HTTP PUT requests. If you do not turn on WebDAV, PUT requests will result in 405 Not Implemented being returned by IIS.

Having enabled PUT, you now need a place to put the files. Let's create a virtual directory in our site that points at the UNC path that is our final destination. There are three key points here. First, we need to ensure that the new virtual directory has Write access. Second, we need to tell IIS to connect as an account with the same username and password as an account on the machine where the UNC share exists. Finally, we have to allow Anonymous access to the virtual directory. If the UploadFile call returns a 401 Unauthorized, one of these three things is wrong. The account that authenticates to the UNC share should have the necessary permissions to to the share of course, including write access.

There you have it a few lines of code and a virtual directory. You're not likely to find a simpler solution to this problem.

BizTalk Mapper: Mapping a the Value of a Node when Present, Otherwise Nil

2008-07-17T11:25:00.001-04:00

If you have an optional node in your source schema (e.g. minOccurs="0") who value must map to a nillable node in your destination schema, this tidy little mapper construct is for you.

It occurs to me that a cookbook containing many of these patterns would be useful to folks just getting started with the BizTalk mapper.

Calling a WCF Service from a SQL Server 2005 CLR User-Defined Function

2008-07-10T17:20:00.001-04:00

On my current project I developed a Membership Service that consolidates all user identities into a single authority, allowing for things like enterprise single sign-on, impersonation, and centralized auditing. Now that we're ready to convert our users over from the previous system, we need to register them all through the service interface. Since the existing user data will be staged to a SQL Server, it was appropriate to give that import access to the service.

Enter a user-defined function. Adding a SQL Server Project to my Membership Service solution, I quickly wrapped the call to the RegisterUser web method in a SqlFunction, i.e. a method tagged with the Microsoft.SqlServer.Server.SqlFunction attribute. David Hayden has a good post on the steps necessary to get this right. Here's an even more detailed discussion to get you going. In addition to these two posts, this article covers how to impersonate the principal in calling your function; it also discusses how you can modify the proxy class to not require the UNSAFE permission set. The SQLCLR team also has a great post on doing this.

All of these authors used wsdl.exe to generate the proxy class for the web service. This is a fine approach, but I prefer the ease of adding a Web Service to my project.

Both approaches also used the sgen utility to generate an XML Serialization Assembly for the classes in the web service proxy that was created with wsdl.exe or adding a Web Reference. This knowledge base article explains why this is absolutely necessary when hosting your proxy in SQLCLR.

There is an addition consideration with using wsdl.exe and/or Web Reference generated proxies against a WCF Service that uses the wsHttpBinding, highlighted here. These proxies only support the basicHttpBinding, that is they don't support WS-Addressing as required by the wsHttpBinding. So, if you got your sql clr function or stored procedure calling your WCF web service, but you are getting a web service timeout, check to make sure it is using a compatible binding. You may also notice the System.ServiceModel.ActionMismatchAddressingException in the service log.

XML Schema dateTime Primer for C# Developers

2008-06-10T09:23:00.002-04:00

The XML Schema dateTime primitive data type represents the current date and time (June 10th, 2008 at 9:30am Eastern Daylight Savings at the time of this writing) in the following format:

2008-06-10T13:30:00Z

The above format represents the UTC date/time. UTC, or coordinated universal time, is just Greenwich Mean Time (or Zulu time, hence the "Z"). UTC is the SI name for this zero time offset. To get the UTC date/time for your local time, simply add your time zone offset. In Indiana, where I'm from, our GMT offset is currently -04:00 (colloquially, "four hours behind").

Here's a C# one-liner to format the current time in the XML Schema dateTime format.

string xsDateTime = System.DateTime.Now.ToUniversalTime().ToString("o");

PowerShell: Convert Unix line-ending to Windows CrLf

2008-06-05T12:46:00.001-04:00

function GetFiles
{
foreach ($i in $input) {
? -inputObject $i -filterScript { $_.GetType().Name -eq "FileInfo" }
}
}

function ConvertToCrLf
{
param([string] $path)
gci $path |
GetFiles |
% {
    $c = gc $_.FullName
    $c | % {
             if (([regex] "`r`n$").IsMatch($_) -eq $false) {
               $_ -replace "`n", "`r`n"
             } else {
               $_
             }
           } |
        set-content $path
}
}

SafeExecute: A Functional Pattern for Cross-cutting Exception Handling

2008-05-29T11:14:00.001-04:00

In some cases, you may find yourself implementing the exact same exception handling logic in several methods within a class, and your brain screams DRY! Here's way to use C# 3.0 to put that exception handling logic in one place:

        public string Foo(string name, string greeting)
        {
            return SafeExecute<string>(() =>
                {
                    return greeting + " " + name;
                });
        }

        public int Bar(int i)
        {
            return SafeExecute<int>(() =>
                {
                    return i + 1;
                });
        }


        protected T SafeExecute<T>(Func<T> funcToExecute)
        {
            try
            {
                return funcToExecute.Invoke();
            }
            catch (Exception)
            {
                throw;
            }
            //if your error handling does not throw an exception
            //return default(T);
        }

The details of proper exception handling are elided above to allow the reader to focus on the functional aspects of the pattern. This pattern is really only useful when you are using exactly the same exception handling logic. This is often the case in classes due to common implications of SRP.

Actually, this pattern is more generalized by the common functional pattern of wrapping the execution of another function, but this provides a useful concrete example.

Learning New Tools

2008-05-28T20:19:00.001-04:00

I sometimes wonder which is more complex, in the information theory sense: the functioning of a given vital organ or the functioning of a modern IDE. In the case of Emacs, I would tend to favor the the IDE.

I learned vi in college; because, truthfully, it was the default editor for elm, and I wanted to send email. Most other folks used pine, but I didn't hear about that for a while. Anyway, that was how it began, and I came to love vi and later vim. This loved flowered even after I found out Bill Joy wrote it, probably because he hated Emacs.

Now, there are a few flawed attempts to bring syntax highlighting and indenting for Erlang to my favorite little editor, but they're far from perfect. There's even an aborted attempt to bring Erlang to Eclipse. Alas, it too is seriously flawed. It would seem that most of the Erlang programming universe writes code in Emacs.

I've never been a joiner, and I wore my iconoclastic editor as a badge of honor. Now, however, I've resolved to follow the pack and learn Emacs. As a side benefit, it would appear that I'll be learning a little Lisp along the way, well elisp at least, more out of necessity than desire.

What bugs me is that I don't really want to learn Emacs, I'd rather spend that time exploring OTP. But, c'est la vie! It will just take some time to become proficient.

At work these days I'm learning the ins-and-outs of BizTalk. This, too, is less than ideal. BizTalk is really powerful, very enterprise-y message-oriented middleware. Truthfully, I wouldn't mind it so much if its development tools had any notion of refactoring. The whole thing just seems to brittle, and could there be a more miserable way to spend ones time mapping messages types?

Getting Erlang Going on a Vanilla Ubuntu Install

2008-05-28T00:36:00.001-04:00

Before you are able to configure and make Erlang OTP from source, you're going to need to be sure your have the right dependencies installed. For a vanilla Ubuntu installation, this means:
sudo apt-get install libc6-dev
sudo apt-get install libncurses6-dev
sudo apt-get install m4
sudo apt-get install libssl-dev
sudo apt-get install sun-java6-jdk

The last two aren't strictly necessary, but libssl-dev is the source files the OTP system needs to utilize OpenSSL. Now, that last dependency on the JDK may seem a bit odd, but the standard makefile for the OTP system tries to build jinterface, an OTP application that enables running an Erlang node in Java. (As an aside, there is a similar package called twotp that does the same thing for Python.)

Assuming all goes well, you should be able to cd into the source directory (wherever it is you extracted it to when you downloaded it) and execute the following commands in order:
./configure
make
sudo make install

Depending on the permissions to /usr/local/* you may not need elevated privileges, but again this is for a vanilla install. Now, you should be able to run Erlang. Next time, I'll have some notes on setting up vim and Emacs for Erlang.

The ALT.NET Times for May 24th, 2008

2008-05-24T15:43:00.001-04:00

A bit of a quiet week on the list this week. In the U.S. this coming Monday is Memorial Day, and we get the day off--a fact that compels many to plan their vacations around it. Whatever the reason, there wasn't much traffic, and so there's not much to report.

Probably the most interesting messages of the week revolved around ALT programming languages. The Boo programming language--which is a full-on CLR language in the style of Python--seemed to be a favorite of many on the list. Oren Eini is a primary contributor and uses Boo for his Windsor configuration tool, Binsor. There's a healthy community around the language, one which created VS2008 integration for it.

There was also some discussion of F#, a language that is something like a CLR implementation of OCaml, with some changes required to behave well with the runtime and other CLR languages. Additionally, it is missing one quintessential OCaml feature: functors. Nevertheless, F# delivers many of the familiar mainstays of a dynamic functional language--like currying, first-class functions, pattern matching, and tuples/records--while integrating with all the .NET libraries and providing strong typing through type inference. It also works seamlessly when installed over Visual Studio 2008. To learn more, a good place to start would be to RTM, or Ted Neward's F# Primer. The place where the F# community gathers is called hubFS.

So, that's the short version (or at least the thematically relevant version) of what I learned from the alt.netters this week. Have a great one!

VMWare Blues

2008-05-21T09:07:00.001-04:00

I suspect it's my fault. I let Vista update to SP1 while I had the VM paused then tried resuming. I got this BSOD while VMWare was restoring the virtual machine's state. Since Vista now shows 4GB of RAM instead of 3.5GB after the update, I'm guessing the memory manager changed, and that caused VMWare to choke on its old memory map.

The ALT.NET Times for May 17th, 2008

2008-05-17T12:46:00.001-04:00

They say if you do something three times, it becomes a habit. This is the second post in a series highlighting what I gleaned from the sharp minds in the ALT.NET scene over last week.

Early in the week, Harry McIntyre from London shared his process and custom code for managing change to DBML files, the XML that defines your classes in Linq to SQL. Folks using Linq to SQL know the pain of managing change to these files. You cannot regenerate them from your database without wiping out any customizations that you made. The Linq to SQL designer is pretty good about capturing most of the idiosyncrasies of your tables, but you inevitably have to customize the generated DBML to cover the corner cases, such as having a default value on a non-nullable CreatedOn column.

Harry's project is called Linq to DBML Runner, and he has a post about the process he uses. It's dependent on a couple of other projects, including Linq to XSD and in yet another case of trickle-down technology from Ruby: Migrator.NET. If you're committed to effectively managing change to your DBML files, and you will be if you use it on greenfield projects involving more than a couple of developers, check out Linq to DBML Runner.

There was a lot of talk about MVC and MVP, along with specializations formalized by Fowler including Passive View. A few astute alt.netters shared some exposition on the venerable MVC pattern as it was first conceived and developed in Smalltalk. Among these posts, Derek Greer's extensive exposition of these patterns was linked up. This is a fantastic, thorough, and cogent examination of the patterns and the origin, highly recommended.

You can also add another implementation to your MVP list: MVC#. This looks very interesting, but I haven't had a chance to delve into it.

Also in the rich-client/WPF world, I heard the first rumblings about Prism this week. Glenn Block has been a frequent poster to the mailing list in the past. He's the Technical Product Planner for the Client User Experience (UX) program in Microsoft's Patterns & Practices (PnP) team. Glenn posted about a code drop of a Prism reference implementation. This is essentially an application that will bake-in all the concepts that will eventually be extracted, formalized, and generalized into the Prism framework for composite applications. This is cool stuff, and it will be good to see a lot of the disparate PnP frameworks rolled up.

One ActionScript developer (now that is ALT) asked about dealing with nested asynchronous web service calls. Two suggestions were given: Flow-based programming and continuation passing style. The latter is familiar, I would imagine, to most developers, at least those with some exposure to functional programming, and it is now very easy to accomplish with C# and lambda expressions. The former, Flow-based programming, is an unfamiliar programming style to me, but at first blush it appears to fit in very well with the current interest in SOA.

Our final topic is an appropriate one to this column, namely, how do we process all the technology out there? Asked by J.P. Hamilton on the list, the most interesting response was from Oren:

I don't. I read some interesting blogs, but I don't go into technology unless I have a need for it. I try to keep an eye for new ideas, but technology itself is not something that I even try to keep up. I find that knowing he fundamentals helps immensely when you learn new stuff, so focusing on that is never a bad idea.

Have a great week!

Hpricot Ruby Script to Digest ISO Currency Codes

2008-05-12T13:27:00.001-04:00

UPDATE: I fixed a couple of bugs in this and changed the XML Schema DataType namespace alias to “xs”. You will likely want to remove some enumeration items because they aren’t particular useful for e-commerce applications, e.g. palladium troy ounce.

require 'hpricot'
require 'open-uri'
doc = Hpricot(open("http://en.wikipedia.org/wiki/ISO_4217"))
codetable = doc.search("//table[@class='wikitable sortable']")[0]
rows = codetable.search("//tr")
for i in 1..rows.length
    tds = rows[i].search("//td")
    unless rows[i] == nil
        puts '<xs:enumeration id="' + tds[3].search("//a[@title]").inner_html.inner_html.gsub(/\s/, '_') + '" value="' + tds[0].inner_html + '" />'
    end
end

Also, here's a Powershell script to process the ISO 3166 country code list (semi-colon delimited):

gc countrycodes.txt | ? {$_ -match ';'} | % { $s0 = $_.split(';')[0]; $s1 = $_.split(';')[1]; "<xsd:enumeration id=`"$s0`" value=`"$s1`" />" } | out-file codes.txt

The ALT.NET Times for May 10th, 2008

2008-05-10T11:01:00.001-04:00

This is the inaugural post of a series I propose to do over Saturday morning coffee. I hope to cherry-pick the posts on the ALT.NET mailing list, and record here for posterity (and my feeble, fallible brain) the best information of the week. Think of it as an ALT.NET Lazy Web distilled.

So, what did we learn this week? Well, if you're writing, thinking about writing, or desperately need an HTML parser, you should check out Hpricot. Incidentally, if you're not a fan of Ruby, beware! The usage syntax of this library might make you a convert. If you're looking for a .NET library, SgmlReader is a great place to start.

Speaking of Ruby, Dave Newman let us know about his VS plug-in for syntax highlighting of Haml. Haml is a templating language for creating views in RoR.

A topic that frequently comes up made another appearance this week: "TDD: Where to Start?" Ah, a fine question and one the ALT.NET crowd answers rather well. A resounding chorus of alt.netters always seem to respond with one book title in particular: Working Effectively with Legacy Code by Michael Feathers. Starting out with TDD is hard work, says the crowd, and starting out with TDD on a brownfield project that doesn't utilize inversion of control is considerably harder. Another popular suggestion is to see TDD in action.

The Managed Extensibility Framework got some pixels towards the beginning of the week. To quote Krzysztof Cwalina, contributing author of ".NET Framework Design Guidelines" and PM of the Application Framework Core team at Microsoft:

MEF is a set of features referred in the academic community and in the industry as a Naming and Activation Service (returns an object given a “name”), Dependency Injection (DI) framework, and a Structural Type System (duck typing). These technologies (and other like System.AddIn) together are intended to enable the world of what we call Open and Dynamic Applications, i.e. make it easier and cheaper to build extensible applications and extensions.

This is exciting news for .NET developers. Incorporating these features into the BCL means we don't have to get permission from our clients and project managers to use them. We won't have to take dependencies on Unity, Windsor, Structure Map, AutoFac, or Spring.NET to get inversion of control goodness into our applications.

One classically "ALT" exchange about distributing configuration changes mentioned some technologies with which I was unfamiliar. ActiveMQ is a huge, mature Apache project for Message Brokering and Enterprise Integration Patterns. Under its umbrella are projects such as the .NET Messaging API which provides a common API for interacting with messaging providers (ActiveMQ, STOMP, MSMQ, and EMS). IKVM is, to put it succinctly, Java running on Mono. The OP was keen on using custom WCF extensions to make distributed configuration caching with Memcached transparent. Oren Eini posted some time ago about using Memcached with .NET, and it is definitely something you should check out, should the need for a distributed object cache arise.

That's all I have to report this week. Have a great weekend and to quote Joe Krutulis of Appirio, "Think & Enjoy!"

Using LINQ from .NET 2.0

2008-05-05T11:18:00.001-04:00

A few months back when I was migrating a project to VS 2008, the PM was very concerned that .NET 3.5 would cause problems on our servers, interfering with existing applications or creating new bugs in the existing codebase. I did my best to educate my PM on the nature of .NET 3.5, that it still ran on the .NET 2.0 CLR, that it was a bunch of new libraries, that it include SDK tools that compiled the new C# 3.0 language, that it wouldn't affect existing applications. Ever the pragmatic fellow, he insisted on late night installs and system testing, nevertheless.

Now, I don't mind pragmatism, in fact, I applaud it; however, a different situation can arise than the one I just described. A non-technical PM or a draconian operations manager might insist that you don't "use .NET 3.5" and stick with ".NET 2.0". If you take this at face value, it means you can't use C# 3.0, LINQ, etc. Well, first you must get them into a Slammer worm recovery group. Next, you can still use some .NET 3.5 goodness without having to install it on the production server.

Here's a minimalist approach: include System.Core.dll in your .NET 2.0 project. You may already know that you can build strictly for .NET 2.0 from VS 2008, so just set the reference, set it to copy local, and roll. You'll have to convince them to let you install .NET 2.0 SP1, though. Another thing you could try is LINQBridge, a .NET 2.0 re-implementation of LINQ to Objects along with Action<T> and Func<T>.

At the end of the day, you should probably just stay within the environment's strictures. If you can make the case that .NET 3.5 will save time and money, you'll be much more successful in changing minds than if you just think it is cool.

Take Ownership and Full Control in Vista

2008-04-28T17:13:00.001-04:00

Sometimes you move a bunch of files from an old machine, in a different domain, and you want to modify those files in the new environment. Generally, the process is to take ownership of the files, then grant yourself full control. This can be a tedious process in Windows Vista, especially in light of LUA. So, I present here some scripting you can use from PowerShell perform this burdensome task.

First, taking ownership is still best accomplished using the "takeown" command. You can look the command syntax up, but if you wanted to take ownership of all the files in the current directory, you can do this from your PowerShell prompt:

gci | % { takeown /f $_}

Next, you want to grant yourself full control. This is normally done with the cacls command, but in Windows Vista this has been deprecated and you should use the icacls command. Here's how to grant yourself full control on all the files in the current directory, replacing your current permissions:

gci | % { icacls $_.FullName /grant:r your_domain\your_username:F }

Replace your_domain and your_username above with the information appropriate to your environment.

.NET 3.5 TDD Frameworks from the ALT.NET Scene

2008-04-20T22:04:00.001-04:00

From the ALTNET mailing list this week, I've come across to very capable frameworks for enabling true test-driven development. Both seem to be born out of a dissatisfaction with current implementations.

First up, I'll mention MoQ; pronounced "Maw-kyoo" or just "mawk", it is written alternatively as both moq and MoQ. The tagline from the MoQ Google code site says it is "The simplest mocking library for .NET 3.5 with deep C# 3.0 integration." From here, we can already say that his library isn't for everyone. Folks working on greenfield projects or converting existing projects may be able to choose .NET 3.5, but many are working in situations where lambdas and LINQ are out-of-bounds.

Those of us lucky enough to be using Func and Action in our code will find moq to be an elegant approach to unit testing interfaces. But, other Mocking frameworks support some of the new C# syntax, so why moq? According to Daniel Cazzulino (kzu), a developer on moq and a Microsoft XML MVP,

The value we think Moq brings to the community is simplicity through a more natural programming model.

So, what does kzu mean by "natural"? Well, traditional mocking uses a record/playback model for setting expectations in TDD, and, due to a legacy that often extends back to .NET 1.0, they have "more than one way to do it", to use an oft invoked Perl-ism and a big reason why Perl is frequently referred to as "write once, read never". Certainly, simpler APIs are to be preferred as long as they can get the job done, and APIs tend to become simpler and more natural over time as long as they aren't required to prevent breaking changes. So, moq was meant to be a simpler mocking framework that leverages C# 3.0 language features. And, so it does.

If you are just getting into TDD and are a little overwhelmed by the myriad mocking frameworks out there and their unfamiliar semantics, you should definitely look into MoQ.

Now, the next item on our agenda is Total Recall. Okay, not Total Recall, but another story by Phillip K. Dick: Autofac; in any event, we're going to take a look at the eponymous Inversion of Control container. Autofac is an MIT licensed project that has gotten some pixel time on the ALT.NET mailing list as of late. The community around Autofac writes on Google code that:

Autofac was designed with modern .NET features and obsessive object-orientation in mind. It will change the way you approach dependency injection in .NET.

Well, if you aren't doing DI just yet, it will certainly change the way you do it. If you're using many of the other .NET IoC containers out there, you're probably not leveraging lambda expressions and LINQ either, so Autofac would be a change. Like MoQ, Autofac sheds some of the legacy cruft and fully embraces .NET 3.5 as a platform. Also note that Autofac leverages LinqBridge to remain compatible with .NET 2.0 applications.

So, will I be doing my next greenfield project using DDD with moq, Autofac, and db4o? Well, I don't think my clients are ready for that yet.

Advanced Web Programming Techniques: Dynamic Script Tags

2008-04-13T22:23:00.001-04:00

I was researching some techniques for doing Comet programming, and I ran across a PowerPoint presentation on Dynamic Script Tags. If you speak GWT, you'll find they outline this technique in their FAQ. Here's the basic idea.

First, a little background about the Javascript Same-Origin Policy, first introduced in Netscape 2. From The Art of Software Security Assessment blog, "same-origin prevents a document or script loaded from one site of origin from manipulating properties of or communicating with a document loaded from another site of origin. In this case the term origin refers to the domain name, port, and protocol of the site hosting the document." These restrictions extended to XmlHttpRequests (xhr), i.e. a script cannot make an xhr to a domain other than the one from whence the script originated.

The Same-Origin Policy poses a bit of a problem for two kinds of AJAX applications. First, the developer attempting to make xhr calls to sub-domains (see my previous post on web performance optimization) will have problems. Generally (IE & FF only?), Resources are not of the same origin if they have a different URL, essentially. Scripts, however, from the same top-level domain can set the document.domain property to the TLD to allow them to interact. This doesn't solve the xhr problem. The second type of AJAX application that can be fettered by the Same-Origin Policy is the mashup. Most mashups proxy the browser request to get the data from the others sites. In other word, xyzmaps.com would have all of its scripts/xhr point to xyzmaps.com which would in turn make an HTTP request to e.g. maps.google.com, essentially proxying the browser request. This presents a huge difficulty for implementing mashups, as all requests have to be proxied through the origin server. A more thorough understanding of the Same-Origin Policy leads us to a better solution.

If you've ever used the Google maps API or the Virtual Earth api, you'll note that you include the api in your page via script tags. Of course, this works perfectly for many sites, but doesn't it violate the Same-Origin Policy? It does not, in fact, because the browser rightly assumes that scripts included via tags in the document are "safe" insofar as they are not alien. So, we can get around the Same-Origin Policy by adding script tags from other domains in our document. This works statically in or pages, but we can also add script tags dynamically via script DOM manipulation! Thus, a better solution than proxying requests for our mashups is found.

So, SiteA dynamically adds a script tag whose src attribute is set to a url on SiteB that will generate JavaScript, embedding the name of SiteA's callback method in the request.

There are several other rather unsatisfactory approaches outlined at Abe Fettig's Weblog.

Website Performance Talk

2008-03-29T14:51:00.002-04:00

This is a really enlightening talk from Steve Souders, author of High Performance Websites, and Chief Performance Yahoo!. Below is a summary of his talk, but they also have a page detailing their rules.

Make fewer HTTP requests: combing CSS and Javascript files, CSS sprites*, use image maps instead of multiple images where possible, inline images*
Use a Content Distribution Network. Akamai, Panther Express, Limelight, Mirror Image, SAVVIS. Distribute your static content before creating a distributed architecture for your dynamic content.
Add an Expires header. If you aren't already, you should be changing the URL of your resources when you version them, to ensure that all users will get updates and fixes. Since you then won't be updating a given resource, you should give it a far future expires header.
Gzip all content: html, css, and javascript.
Put stylesheets at the top. IE will not begin rendering until all CSS files are downloaded. Use the link HTML tag to pull in stylesheets, not the @import CSS directive, as IE will defer the download of the stylesheet.
Put javascripts at the bottom. HTTP 1.1 allows for two parallel server connections per hostname, but all downloads are blocked until the script is downloaded. The defer attribute of the script block is not supported in Firefox and doesn't really defer in IE.
Avoid CSS expressions. The speaker doesn't cover this, but I gather from my own experience that this can seriously delay rendering as the expression will not be evaluated until the page is fully loaded.
Make CSS and JS external. This rule can be bent in situations where page views are low and users don't revisit often. In this situation in-lining is suggested.
Reduce DNS lookups. He didn't cover this either, but it follows from rule #1 that additional network requests negatively impact performance. He does mention later in the talk that splitting your requests across domains can drastically increase response time. This is a consequence of the two connections per domain limit in the HTTP 1.1 specification. It is important to remember that JavaScripts from one domain cannot affect JavaScripts or pages from another domain, due to browser security restrictions. Also, this is a case of diminishing returns, as beyond four domains causes negative returns from, presumably DNS lookups and resource contention in the browser itself.
"Minify" Javascript. JSMin written by Doug Crockford is the most popular tool. This is just removing whitespace, generally. YUI compressor may be preferred and also does CSS.
Avoid redirects.
Remove duplicate scripts.
Configure ETags. These are used to uniquely identify a resource in space and time, such that if the ETag header received is identical to the cached result from a previous request for that resource, the browser can choose to use the local cache without need to download the remainder of the response stream. The speaker doesn't go into this subject, so questions about how this improves caching performance over caching headers remain unanswered until you read his book.
Make AJAX cacheable. If you embed a last modified token into the URL of your AJAX request, you can cause the browser to pull from cache when possible.

A great tool to analyze your site's conformance to these rules is YSlow. It's an add-on for Firebug. During development your YSlow grade can give you a very good indication of what is happening to the response time of your application. This metric, the YSlow grade, seems to me to be a much better quality gate for iterative development than does something as variable and volatile measured response time.

Some additional rules from the next version of the book:

As mentioned in my commentary in rule #9, split dominant content domains.
Reduce cookie weight.
Make static content cookie free.
Minify CSS (see rule #10 comments above).
Use iframes wisely. The are a very expensive DOM operation. Think about it--it's an entirely new page, but linked to another.
Optimize images. Should your GIFs be PNGs? Should your PNGs be JPGs? Can you shrink your color palette?

Erlang News

2008-03-19T19:41:00.001-04:00

Robert Virding's First Rule: "Any sufficiently complicated concurrent program in another language contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Erlang."

"I now know based on hard-won experience that you could replace "concurrent program" with "distributed system" in Robert's rule and it would still be just as accurate." --Steve Vinosky

Yariv wrote a cogent response to "What Sucks About Erlang".

Erlang Flavored Lisp anyone?

Programming Erlang: Chapter 8 Problem 2

2008-03-10T21:13:00.001-04:00

'Tis an odd title, I know. My puppy is prancing around the kitchen trying to get me interested in his chew toy, dropping it near me so I can see that it bounces and wiggles.

Do you want to see Erlang bounce and wiggle? The code below creates a circular linked list of Erlang N processes, then sends a Message around it M times. Pretty cool, huh? Well, it was meant to be used by readers of "Programming Erlang" to compare their favorite language to Erlang. I didn't bother doing this with C# threads, though it would be interesting to try it out with the concurrency and co-ordination runtime.

Here's the out put of the code (time is in milliseconds):

Sent 30000000 messages in 27453 / 29186

-module(problem2).
-compile(export_all).

start(N, M, Message) when is_integer(N), is_integer(M) ->
  statistics(runtime),
  statistics(wall_clock),
  FirstPid = spawn(?MODULE,loop,[[]]),
  LastPid = create(1, N, FirstPid),
  FirstPid ! LastPid, %% the ring is closed
  send(0, M, FirstPid, Message),
  {_, Time1} = statistics(runtime),
  {_, Time2} = statistics(wall_clock),
  io:format("Sent ~p messages in ~p / ~p~n", [N * M, Time1, Time2]).
  
send(C, C, _Pid,  _Message) -> void;
send(I, C, Pid, Message) ->
  Pid ! {self(), Message},
  receive
    finished -> void
  end,
  send(I+1, C, Pid, Message).

create(C, C, Pid) -> Pid; 
create(I, C, Pid) -> 
  create(I+1, C, spawn(?MODULE, loop, [Pid])).

loop(Next) ->
  receive
    {Caller, FirstPid, Message} ->
      if
        FirstPid =:= self() -> Caller ! finished;
        true -> Next ! {Caller, FirstPid, Message}
      end,
      loop(Next);
    {Caller, Message} ->
      Next ! {Caller, self(), Message},
      loop(Next);
    NewNext when Next =:= [] ->
      loop(NewNext)
  end.

Presenter First with POA

2008-02-15T15:42:00.001-05:00

Plain-old ASP.NET (POA) has a lot of advantages: rapid UI prototyping, incomparable support from 3rd-party vendors, and a simplified programming model that abstracts away the problems of a stateless application to name a few. But there remains the problem of testing. ASP.NET applications--any web-based application--is difficult to test. The traditional architecture of having your code-behind orchestrate your business logic components in response to events generated through the postback mechanism requires that you instigate those postbacks in order to test the orchestration. In other words, you have to drive the UI to test your application.

UI testing is expensive. If you've done it, you already know that. If you've not done it, then trust us on this one; the cost of tools, the complexity, and brittleness of web tests drive the cost up. Furthermore, if an application isn't designed to be UI tested, it becomes a nearly intractable problem to do so. The folks at Atomic Object know this from experience. As proponents of test-driven development, they have looked down the barrel of that gun and found a better way. They call it Presenter First.

The basic idea behind Presenter First (PF) is to obviate the need for UI testing. It is a Model-View-Presenter (MVP) variant that is closely related to Martin Fowler's Passive View. The application architecture is such that UI is so humble that there is no need to test the UI. This architecture requires that the Presenter be responsible for orchestrating the Model (business components) in response to events generated by the View, and updating the View appropriately. A View in PF only implements events and property set methods. This View can be defined by contract, e.g. IView, and mocked to allow for test-driven development.

A hallmark of PF is that Presenters are constructed with an IView and an IModel and do not implement any other public interface members. Thus, the encapsulated worfklow is driven completely through IView and (possibly) IModel events. Concordantly, it becomes evident that PF architectures are implicitly stateful, i.e. the Presenter is extant between events. In POA, the HttpContext only exists for the duration of the processing of a request. While the HttpSession remains there is a non-trivial amount of complexity involved with it; ultimately, this too should be considered transient in the general case. From these facts one might conclude that PF is not applicable to POA.

The purpose of this writing is to propose a method of implementing Presenter First in ASP.NET. The general approach is outlined below:

All page's implementing a PF IView (e.g. ICustomerEditView) must inherit from PFPage.
PFPage implements a Dictionary<typeof(IView),PresenterInstance> lookup.
Before Page processing, the PresenterInstance is given a reference to the View, i.e. the current page.
The IView events are fired (from postback) and the Presenter handles them appropriately, updating the View via setters.
The Presenter handles the Page.Unload event and drops its View reference

This approach diverges from "classical" PF in a few ways. Most importantly, the actual concrete View acted upon by the Presenter instance is transient. This should not pose any particular difficulty as the ASP.NET postback model ameliorates this stateless condition of web applications. Further, while this requires the Presenter interface to implement a mutator for its View reference, the Presenter implementation need not diverge from the "classical" PF approach, save in one important way. IModel events would need to be queued.[1]

Furthermore, the Presenter instance and the dictionary must be serializable if that pattern is to be used in an ASP.NET deployment where the HttpSession was out-of-process. This might propose problems with Presenter references to cached objects that can't be extended to support serialization; though this is considered a minor flaw in the general case and an impediment that can be ameliorated by recreating objects from metadata.

To compose views it would only be necessary to create a PFControl base class that implemented nearly identical functionality to the PFPage. Also, though declarative databinding is no longer possible, DTOs could be used in View mutators for databinding.

Through the use of mock objects, this approach to ASP.NET development using Presenter First enables Test-Driven Development of the business objects and their orchestration in response UI events to without the need to involve ASP.NET pages.[2] Additionally, this allows for the tandem of presentation (.aspx) and interaction (.aspx.cs) to be isolated from business logic (Model) and orchestration (Presenter).

Please let me know what you think. Is your organization using a very similar approach? How have you fared?

[1] Because the Presenter MUST NOT call mutator methods on the View between postbacks, IModel events must be queued until a new View instance is received.

[2] While issues such as declarative security and AJAX can make UI testing a necessity, we are assuming that these are not direct considerations for vetting the business logic and orchestration. Further this kind of UI testing will generally be done as a part of UAT and thus does not "cost" the development team.

Upgrading to Windows Vista

2008-02-12T22:30:00.001-05:00

There's a certain voodoo to searching for problems. I don't expect that anyone is actually going to read this before upgrading to Vista... but here are the three problems I had.

First, make sure if you are running DaemonTools that you uninstall it first. The Vista upgrade will force you to do this before proceeding, but sptd.sys will not be removed. So... remove it. It's in your Windows directory.

Next, make sure you download Vista updates for all your drivers. I couldn't get my external monitor for my laptop to use it's native resolution. The generic monitor driver in Vista didn't allow it. I went to my manufacturer's website, NEC's, and download Vista drivers for the monitor.

Lastly (I hope), you might have trouble connecting to a cheaper broadband router. Well, you'll be able to connect, but you won't get a DHCP address. This knowledge base article should do the trick.

So, what do I think of Vista...?

I guess I don't know what all the hullabaloo has been about in the blogs and press. Surely no one was expecting Mac OSX? I run Ubuntu Linux too, and I don't expect it to be OSX. In fact, I think Ubuntu's desktop experience sucks compared to Vista, but I love it as a Linux distro. I ran Slackware in '98, and I can tell you things have REALLY changed in ten years. And, I can tell you Vista is better than XP. You have to take that with a grain of salt; I relish change.

Sure, you will have more trouble pirating music, but it's more secure, prettier, and cooler.

I guess that's why Vista is making Microsoft money. Go figure.

Weekend Update with Christopher Atkins

2008-02-03T17:10:00.001-05:00

Dennis Miller was my favorite Weekend Update anchorman. His wry, sardonic--if verbose--commentaries always made me laugh. As I think about it, that sketch seems a direct ancestor of the news shows on Comedy Central so popular with the college crowd.

Anyway, my weekend has been interesting. I found out that despite what the documentation says, ECDiffieHellmanCng is not supported on Windows XP. In fact, its constructors check for the presence of NCrypt.dll: a library only available for Windows Vista and Windows Server 2003+. So, despite that fact that it ships with .NET 3.5, you can't use it on XP. Joy. I hope this is not a trend. You'll also find that you cannot use a BigInt, because it is marked internal. If you're trying to do modern cryptography on Windows XP, in other words, you are out of luck.

So, I spent the remainder of my weekend working through Programming Erlang, and I have to say I am even more excited about it. I'm only on page 51, but I've already learned about list comprehensions and custom flow control abstractions using higher-order functions. The cool thing is I can apply some of this stuff to C#, now that we have lambda expressions.

I'm excited about the year ahead. There are a lot of changes on the horizon. Imagine, Windows Server 2008, SQL Server 2008, a Yahoo! and Microsoft merger, widespread adoption of OpenID, and a new presidential election are all on their way. To quote the folk poet Bob Dylan:

Come writers and critics
Who prophesize with your pen
And keep your eyes wide
The chance won't come again
And don't speak too soon
For the wheel's still in spin
And there's no tellin' who
That it's namin'.
For the loser now
Will be later to win
For the times they are a-changin'

Competence != Compensation

2008-01-30T20:41:00.001-05:00

In America we are conditioned to believe that working hard, being diligent, and producing great work is the road to fiscal success. Anyone who has really worked here will tell you that is simply not the case. Prima facie, you must recognize that you are not paid what you are "worth"; that is, your personal worth has nothing to do with your income. One must never forget this simple fact. Despite the fact that you may do three times the work of your peers, you will not be compensated even twice as much.

Witness the Home Depot CEO who purposefully runs his stock in the ground, knowing well that a golden parachute awaits for him once the board of directors requests that he "step down." Negative output results in a cash payout worth over $200 million dollars. In America, you are worth what someone will pay you. There is no such thing as intrinsic value, only assessed value. In software development there are two ways of having your value assessed, by managers and by investors.

Make sure you don't hang your hopes on managers.

Continuous Integration with TFS: Pre-compiling a Web Application Project

2008-01-30T07:12:00.001-05:00

If you are looking to pre-compile your web application projects with TFS 2008 Team Build, here are some tips.

First, you'll want to add the following to your TFSBuild.proj:

<Target Name="AfterDropBuild">
  <AspNetCompiler
     PhysicalPath="$(DropLocation)\$(BuildNumber)\Release\_PublishedWebsites\web_project_name"
     Debug="false"
     Force="true"
     TargetPath="C:\path_to_copy_precompiled_web_to"
     VirtualPath="/Path_to_your_web"
/>
</Target>

There a few things to note here. First, the Target Name is crucial; this Target will get executed after the drop is made, i.e. after the build successfully runs and your files are copied to the drop folder you specified when you created the build. Secondly, remember that this is running from the machine where your build agent runs. The AspNetCompiler MSBuild tag shown here will launch aspnet_compiler.exe on that build agent machine, and so "/Path_to_your_web" points to the virtual directory on the default web site of your build agent machine. Please note that "/Path_to_your_web" would be "/" in the case of your application being in the root directory. Finally, the Force parameter is required since we will be writing over the existing pre-compiled web on subsequent builds.

If you want setup a build agent on another machine, perhaps a development web server where you would prefer to deploy the pre-compiled web application, you'll need to configure that agent according to these instructions. This is a good idea when running verification tests that drive the web UI.

Happy Coding!

On a Positive Note

2008-01-26T13:20:00.001-05:00

Jean-Paul Boodhoo is a developer and consultant who used to work at ThoughtWorks and has been featured on the Polymorphic Podcast and DNRtv. He recently posted a very encouraging post about dreams, risk, and reward.

Here's an excerpt.

I was speaking with a friend yesterday who made an interesting comment:

“You seem to regularly take on more stress than most other people would ever think to take on”

I corrected him and made this statement. I don’t feel like I am stressed out that much. In all honesty, the times that I feel stressed out almost always constitute a failure to plan on my part. I did say that what I do take on regularly is: Challenge and Risk. I am not afraid of the opportunity to fall flat on my face taking a risk, because I now that it is in the times of struggle/pain that growth happens.

Bravo. That's well said. Paul Graham (all your Lisp are belong to us) wrote a year ago explaining: How to Do What You Love. Here's a quote, emphasis added:

Once, when I was about 9 or 10, my father told me I could be whatever I wanted when I grew up, so long as I enjoyed it. I remember that precisely because it seemed so anomalous. It was like being told to use dry water. Whatever I thought he meant, I didn't think he meant work could literally be fun—fun like playing. It took me years to grasp that.

Sure, sure, I heard you saying... do what you love, right? Well, I love vacations, the ocean, playing guitar, and chasing my puppy--how do I get paid to do that? The truth is you don't; you won't; and you shouldn't. Again, from Mr. Graham:

Unproductive pleasures pall eventually. After a while you get tired of lying on the beach. If you want to stay happy, you have to do something.

I can't tell you how invigorating working in the yard can be or how exciting solving some mildly difficult programming problem can be. It's enough to make people blog about their experiences. But, even these are just short-term examples, what is it that you would love to do over the long haul? Is there some sort of heuristic that we can apply to help use decide? Here's Graham's idea:

I think the best test is one Gino Lee taught me: to try to do things that would make your friends say wow.

If you hate what you do, I cannot convince you that trying to impress your friends with it is going to make it fun. So, think of the things you do that you do try to impress your friends with? Do you love to cook? Master the art, learn the language, hone your technique, and blog about the experience. Start a in-home cooking class consultancy. Do you love to take photographs? Practice, learn, take your camera everywhere, make it your first thought. Do you love it enough to do it for free?

The test of whether people love what they do is whether they'd do it even if they weren't paid for it—even if they had to work at another job to make a living. How many corporate lawyers would do their current work if they had to do it for free, in their spare time, and take day jobs as waiters to support themselves?

If you don't have enough passion to work hard at something and get better, then you either don't love it or you don't have an audience. You might just need new friends. Seek them out in classes or online discussion groups. But seek them out.

If you are worried about money, if you're worried about being responsible for the well-being of your children or spouse, remember that you have a responsibility to them. You are responsible for loving them, for inspiring them, for trying to be the best person you can for them. Those are your true responsibilities; you can find a way to pay the really important bills. Don't let fear admit apathy. Don't use your family as an excuse.

If you don't already know what you love to do--and so few of us really do--Graham has further advice:

It's hard to find work you love; it must be, if so few do. So don't underestimate this task. And don't feel bad if you haven't succeeded yet. In fact, if you admit to yourself that you're discontented, you're a step ahead of most people, who are still in denial. [..] Finding great work you love does usually require discipline. Is there some test you can use to keep yourself honest? One is to try to do a good job at whatever you're doing, even if you don't like it. [...] Another test you can use is: always produce.

If you are telling yourself that you hate your day job, and one day you will be set free to do what you love, then you better being doing what you love and consistently. Otherwise, you're lying to yourself and everyone around you. The hardest thing to admit is that at age 20, 30, 40, or 50 is that you don't know what you love; you don't know what you want to do when you grow up. My point here is that you must do something; you must produce. Even if you prove to be rather amateurish in all you attempt, but you really gave it your best, you will be happier--and much more interesting. The worst thing you can be in your work-life, truly, is be a dilettante.

I will close by leaving you with these further words from Graham's essay (please read it):

"Always produce" is also a heuristic for finding the work you love. If you subject yourself to that constraint, it will automatically push you away from things you think you're supposed to work on, toward things you actually like. "Always produce" will discover your life's work the way water, with the aid of gravity, finds the hole in your roof.

One last note before parting that I have is simply this: seek out good teachers. My wife recently told me what the tuition for a four-year degree at local technical college is: nearly $84,000. Most of their students do not finish, and most of them are there because they want to make more money. And, sadly, most of them don't learn a thing--because they're not there to learn, they're there to make more money (via a diploma). This is not the road to happiness. It would be much better for those people to find work they love and to see out the best teachers. I guarantee you that those technical colleges do not have the best teachers.

IndyTFS January Meeting: These Guys Rawk!

2008-01-24T13:52:00.001-05:00

Jamie Kurtz of Open Solutions presented on TFS Work Items. Now, I really hate going to user group meetings in the midwest, because most of the time the speakers are not really all that knowledgeable about their topic. This session, however, could have been entitled, "TFS: All Questions Answered." Between Paul Hacker and Jamie Kurtz, not one question was a stumper. I learned more about TFS in that two hours than I could have in two days on my own. It was truly fantastic; they had a great attitude!

Here are some of the things we learned.

Customizing Work Item types: workflow, states, custom form fields, custom field controls
Details and gotchas on how to get business users setup to edit work items from Excel
Details about securing fields and work items
Editing MSBuild files to work appropriately with WorkItems

Kudos to both gentlemen. I encourage anyone using or considering using TFS to member up and start attending meetings.

List of Links for Learning LINQ

2008-01-12T19:11:00.001-05:00

Since I often use this blog as a brain back-up device. I will add another list of links, this time for learning LINQ.

Building a LINQ Provider

Links to LINQ (another link-post)

How LINQ Works: Creating Queries (part of a series of great posts on LINQ internals)

The .NET Standard Query Operators (slightly out-of-date, but very relevant)

Standard Query Operator Translation (LINQ to SQL)

Happy 70th Birthday to Don Knuth

2008-01-11T12:57:00.001-05:00

I still haven't read The Art of Computer Programming, Volume 1, but I'm definitely an admirer of this man. Here's to many, many more years where we are blessed to have him among us.

A Twitter-style Blog Post from a User Group

2008-01-10T19:46:00.001-05:00

Larry Clarkin is a developer evangelist for Microsoft based out of Milwaukee. He gave a talk this evening to the IndyNDA group at the Junior Achievement center on Keystone. He spent the first twenty minutes of his talk showing us Microsoft's Virtual Earth maps site. To use the Virtual Earth client libraries you need to include their script library and create a DIV to contain the map renderings. A very informative site for learning Virtual Earth was demonstrated; it's called "The Virtual Earth Interactive SDK". Very cool.

He's showed us a pretty tame example of overlaying a Silverlight control on a Virtual Earth map. The Silverlight control references the map via Javascript. Pretty simple, but I believe the concept is solid. You could really have a lot of fun with this approach.

7:09pm and we've been enlightened that SOAP has lost the battle to REST on the public web. Now we're getting some of the Twitter hype... boring. More Twitter, more boring. I guess I just don't get Twitter. It seems like a way to fill the gaps when you are alone with yourself. What's next--a microphone for your thoughts? I will never waste my time reading that crap.

7:23pm and we're learning that RSS is gee-wowie! "We need more things like RSS." Good to know. Larry likes Flickr.

The guy sitting behind me has returned to his seat and reeks of cigarette smoke. Joy!

Oooh, now I'm learning how to get an ugly slideshow in Blend from a Flickr RSS feed. It's 7:37pm and he's done--felt longer but under an hour.

The highlight was hearing guys get excited about being able to update Twitter via SMS. They thought, "Wow, I can point my applications at that feed and then send control codes via SMS." It feels like you should probably skip the middle-man. Though, a free SMS to RSS gateway is a fun idea for homebrew mash-ups.

Well, the pizza was better this time.

TFS Build Failure: MSBuild Tool missing sgen.exe

2008-01-09T18:50:00.001-05:00

We're running TFS 2008 now at my client. I installed the trial edition just to expedite things, as I'm not the one who works with the licenses. (You can do an in-place upgrade of the trial to a live license.) As I was getting the continuous integration build setup for our pilot project, I kept getting a build error similar to the following.

error MSB3091: Task failed because "sgen.exe" was not found, or the correct Microsoft Windows SDK is not installed. The task is looking for "sgen.exe" in the "bin" subdirectory beneath the location specified in the InstallationFolder value of the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v6.0A. You may be able to solve the problem by doing one of the following:  1) Install the Microsoft Windows SDK for Windows Server 2008 and .NET Framework 3.5.  2) Install Visual Studio 2008.  3) Manually set the above registry key to the correct location.  4) Pass the correct location into the "ToolPath" parameter of the task.

Some extensive searching only yielded this post, but I ignored the suggestion that followed the original posting... at first. Turns out, it works quite fine. Basically, you should have two registry keys under HKLM\SOFTWARE\Microsoft\Microsoft SDKs

The InstallationFolder value in the .NETFramework\v2.0 subkey is the thing we're after in the case. Make an exact copy of that value under the Windows\V6.0A key. I did this, as the previously mentioned post suggested, and things worked fine!

Mas Importante! You may need to install the .NET Framework 2.0 SDK onto your TFS Server/Build Agent machine in order to do the above.

Lambdas & Closures in C# 3.0

2008-01-05T22:18:00.001-05:00

Here's some code the exhibits the creation of a closure in C#. This was possible with the advent of anonymous delegates in C# 2.0, but it looks a lot cleaner in C# 3.0.

using System;
using System.Linq;
using System.Collections.Generic;

public class Program
{
    public static void Main(string[] args)
    {
        ClosuresWithLambdas a = new ClosuresWithLambdas();

        List<Func<int>> fs = a.GetNumberFunctions();

        Func<int> increments_i_func = fs[0];
        Func<int> just_returns_i_func = fs[1];

        //prints 1
        a.PrintReturnValue(increments_i_func);
        //prints 2
        a.PrintReturnValue(just_returns_i_func);
    }

    public class ClosuresWithLambdas
    {
        public List<Func<int>> GetNumberFunctions()
        {
            //this variable gets included in the closure
            int j = 1;

            //we use this to return two functions
            List<Func<int>> fs = new List<Func<int>>(2);

            //don't get confused by the postfix operator
            //the f lambda returns j BEFORE incrementing it
            Func<int> f = () => j++;

            //g just returns the variable j
            Func<int> g = () => j;

            //add the two functions to the list and return it
            fs.Add(f);
            fs.Add(g);
            return fs;
        }

        public void PrintReturnValue(Func<int> f)
        {
            Console.WriteLine(f());
        }
    }
}

This closure of which I speak is not immediately evident to the untrained eye. The basic concept in this example is that the variable "j" has to survive past the end of its normal scope, i.e. the execution of GetNumberFunctions. It has to survive because the two functions passed out of GetNumberFunctions--the lambdas "f" and "g"--refer to "j" outside of the function in which they were defined. The compiler detects this (perfectly legal) reference as a lexical closure; it generates a class to contain these functions and the variable "j". Here is that generated helper class viewed from Lutz Roeder's Reflector.

[CompilerGenerated]
private sealed class <>c__DisplayClass2
{
    // Fields
    public int j;

    // Methods
    public int <GetNumberFunctions>b__0()
    {
        return this.j++;
    }

    public int <GetNumberFunctions>b__1()
    {
        return this.j;
    }
}

As you can probably gather b__0 is "f" and b__1 is "g" (or, "increments_i_func" and "just_returns_i_func" respectively). They share "j" in the scope of an instance of this compiler generated sealed class instance. There are other ways for language and compiler to implement closures, but this generated inner class works just fine. For a practical example of using closures, please see my previous article on doing asynchronous network I/O with closures.

"Fixing" the Enter Key in CreateUserWizard

2007-12-28T16:33:00.001-05:00

Problem: You find that when you press the "Enter" or "Return" key on a page that contains the CreateUserStep of your CreateUserWizard (or any other step of any other wizard), your form does not get submitted. Put another way, something else on the page gets triggered instead of your Create User button.

Solution: In ASP.NET 2.0 the Page.Form has a new property: DefaultButton. This effectively traps the Enter key and causes the specified button to execute its onclick event handler. We would like to set this Page.Form.DefaultButton property to the UniqueID property of the CreateUserStep's Create User button. Unfortunately, we do not have a reliable way of accessing this button in our code-behind. The solution is to replace it with our own button and wire up the DefaultButton property in the button's pre-render event handler, since it is in a template and we cannot access it directly.

In your .aspx add the following CustomNavigationTemplate to your CreateUserStep:

In your .aspx.cs add the following:

protected void CreateUserButtonRender(object sender, EventArgs e)
{
this.Page.Form.DefaultButton = (sender as Button).UniqueID;
}

Choosing a Web Application Server Stack

2007-12-26T19:00:00.001-05:00

There's this great book entitled, "The Paradox of Choice: Why More Is Less," that really opened my eyes to a rather non-intuitive way to improve my experience in life. I won't go into it more than to say the author posits that there is an inflection point where "happiness" does not increase with additional accretion of choices. This is non-intuitive, but he does a good job of explaining the many factors underlying this phenomenon. For one, consider the opportunity cost of making a decision; you are buying what you consider the best at the time, but you are also carrying the burden of not having chosen from many other options. Very often, buyer's remorse sets in after a purchase, and we have no one to blame but ourselves.

So, as I tore through blogs, email archives, tutorials, and documentation today, looking for the "best" platform for my personal pet project, I became acutely aware of just how much choice there is available to build web applications these days. The result is that it's very late in the day, my weekend spent, and I am writing this blog post to reason aloud, as it were, and to force myself to pick one.

By way of introduction, let me say that my skill set falls squarely in the tried-and-true .NET 2.0 ASP.NET developer model. I don't do MVC, MVP, Presenter First (PF), O/RM, TDD, DDD, BDD, or any other TLAs, except maybe DRY, SRP, and all those other solid OOP principles. Also, I'm building this application for myself, by myself, and I can't seem to get a free copy of Visual Studio 2008, so .NET 3.5 and LINQ are out of reach. I know I need forums, but that's about my only "requirement". Sure, I've got lots of ideas of what I'm going to build, but I am staring at the blank canvas right now.

Here's the thing: I'm productive. I build good applications. I suspect that probably has more to do with my empathy and diligence rather than some prodigious development or architecture skills. I'm certain I can get better at the latter, but my competitive advantage, if you will, comes from design good interactions. I'm good a UI design and ideation. And, frankly, all those TLAs are a bit intimidating, as is the prospect of writing so much more code.

It may sound like I'm leaning toward an ASP.NET 2.0 application, so let me reflect on that. I am definitely not going that direction. The major strengths of that platform are:

Tooling support---Visual Studio beats vim an SciTE hands down
3rd-Party Controls---Telerik, Infragistics, etc.
Familiarity---myself and thousands like me use it every day

Those advantages sound great if you are an IT manager building line-of-business applications. That's not me. Among the disadvantages for me are:

Mundane---I use it for a living and wouldn't be learning anything new
Visual Studio 2005 is not free and the Express editions are...Express editions
I won't be purchasing any 3rd-party controls
The Page-based application model seems outmoded

So, what are my options? Well, I would like to try building an application using the MVC/MVP/PF paradigm. I've invested many hours learning about it, and I want to take a stab at it. This means, almost certainly, using an IoC container--but which one? Also, MVC differs significantly from MVP as does PF; which shall I use? I have to select an environment to do this all in as well.

Supervising Presenter First?

I have settled on Presenter First (PF). This doesn't have the widespread community support that MVC/MVP have today, which means less tooling and "free" functionality, but that's okay. PF takes SRP and the Humble Dialog to the extreme, forcing you to develop a testable presenter that reads like user stories and easily mocked views and models that can also be tested. Because PF dictates an stateless and ignorant view, it should be easy to replace and change the UI. Now, I can definitely say I won't be doing "pure" PF; I plan to allow tuples/ActiveRecord objects into my UI, because I want to use databinding and all the built-in goodness of ASP.NET when it is efficacious to do so. In this sense, I want PF with Supervising Controller leanings.

Views: Plain-old Pages

I believe ASP.NET Pages are a very strong candidate for views, despite what I have heard from the ALT.NET crowd. As a template engine, they are very mature; you can even nest master pages in 2008! The ASP.NET Membership provider (Authentication, Authorization, Personalization, etc.), declarative security, and databinding are a few great things you get out-of-the-box. There are lots of controls out there to work in ASP.NET Pages, including all the ASP.NET AJAX stuff. This absolutely does force you to think about your views in terms of pages at some level, but I believe partial page updates allow user controls to be views as well.

Choosing a Framework

There are lots of choices out there for doing MVC/MVP style ASP.NET application, each with their own peculiar twists. I have mentioned a couple of these before, but here are the ones I've looked at:

The main problem with each of these is that they are not PF pattern friendly. That isn't to say that they are antagonistic, not at all. I'm guessing, relatively blindly, that each of these would be equally difficult to implement PF. So, what other criteria can I use to cull the herd? Well, MonoRail doesn't play nice with ASP.NET Pages, so it's out. Cuyahoga is a real pain in the butt to configure, quite possibly the longest time-to-hello-world [Note: ~400MB QuickTime movie about alternative web frameworks, including plone] of all these frameworks--gone! Rhino Igloo has some very, very interesting ideas reminiscent of Guice and Spring javaconfig in its used of an InjectAttribute. WCSF does this, too. Both of these require that the request is made to and handled by the view (the Page), and they use DI tricks to get the Controller involved. ASP.NET MVC is using an extensible URL rewrite feature to put the controller first. This lets us be RESTful in addition to being PF-friendly, e.g. we can handle requests with our controller.

Continuations

My major complaint with all of these frameworks is that don't let me writing applications in a natural way. I don't write my application in terms of logical processes, instead I implement page flows and deal with all the problems of stateless web applications. In the nine years that I have been building web applications, my view of what a web application is and should be have changed a lot. Now, I believe I have seen the promised land, as it were. Web application servers and frameworks should allow me to develop my applications with logical continuations and take care of the plumbing for me.

REST + Continuations + Presenter First = ?

So, REST is good because it provides clean, bookmark-able URLs, a sort of coherent web-based command-line. Continuations are good because they let me write applications that simply do not care that the next program event is dependent on transient sessions over a stateless protocol. The Presenter First pattern is good because its raison d'être is making MVP more amenable to test-driven development. Unfortunately, there are no ASP.NET web frameworks out there that take these three to be their cardinal virtues. So, we'll just have to go invent our own.

In a follow-up post to this one, I plan to introduce my prototype for just such a framework. Utilizing some of the functionality of ASP.NET MVC and Windows Workflow Foundation, along with a lot of new code and concepts, I am building a prototype that I hope proves to be a huge evolution in web programming on the ASP.NET platform.

Liking LINQ: A Question of Efficacy

2007-12-21T13:28:00.001-05:00

Consider these two equivalent blocks of code. First, the LINQ way:

foreach(var assignable in (from Assembly a in BuildManager.GetReferencedAssemblies()
                           select a.GetTypes() into types
                           from t1 in types
                           where typeof(I).IsAssignableFrom(t1)
                           select t1))
         Cache.AddType(assignable.Name, Cache.ContainsType(assignable.Name) ? DUPLICATE_TYPE : assignable);

And, the "normal way":

foreach (Assembly assembly in BuildManager.GetReferencedAssemblies())
     foreach (Type type in assembly.GetTypes())
          if (typeof(I).IsAssignableFrom(type))
          {
                if (!Cache.ContainsType(type.Name))
                      Cache.AddType(type.Name, type);
                else
                      Cache.AddType(type.Name, DUPLICATE_TYPE);
          }

I cannot rightly make a judgement call on which way is better. They have the same result. Though, it is safe to say that normal way should perform better. It is also clear, oddly, that the normal way has fewer lines of meaningful code; thus, it is easier to grok. So, what do you think? If you had to maintain the code base, which one would you prefer. FWIW, I'm going to go with the normal way.

This leaves me with the question of efficacy. Certainly there are niches where LINQ is superior, or the only option, but for general object collection work, should we just ignore this language feature? Perhaps when PLINQ comes available we'll have a good reason to use it. Time will tell.

SQL Optimization: Substring Joins

2007-12-21T13:11:00.001-05:00

The DBA at my current client has some mad T-SQL skills. He took a naïve join implementation I had written and improved its performance by three orders of magnitude.

My initial query looked something like this:

SELECT A.*, B.* FROM A JOIN B ON A.Message LIKE '%' + B.Identifier + '%'

Obviously this isn't the best situation in the first place. We don't have a clean relation between these two tables, instead we've got to pick out an identifier (GUID/uniqueidentifier) from a larger text field. Notwithstanding some obvious ETL or trigger possibilities, the statement above seems the simplest solution. The only problem is it is slooow.

The DBA had a great optimization which was essentially to tease out the identifier from the field as you would if you were writing a trigger to create a join column. Putting this in your query allows you to join on this derived column in your code. The performance implications are well above what I would have guessed them to be, operating–as I am–from naïveté. Here's the generalized solution based on our example above:

SELECT B.*, J.* FROM B JOIN ( SELECT A2.*, CAST(SUBSTRING(A2.Message, A2.GuidStart, A2.GuidEnd - A2.GuidStart) AS uniqueidentifier) AS JoinGuid FROM ( SELECT A1.*, CHARINDEX('TokenAfterGuid', A1.Message, A1.Start) as GuidEnd FROM ( SELECT A.*, CHARINDEX('TokenBeforeGuid:', A.Message) + LEN('TokenBeforeGuid:') + 1 AS GuidStart FROM A ) AS A1 ) AS A2 ) ON B.Guid = J.JoinGuid

Of course, you would expand the wildcard (*) references. This is really a great technique considering the performance ramifications. Certainly in our case, where the query was for a view, this was a wonderful improvement. Obviously, the best option from a performance stand point would be to tease out the uniqueidentifier in an INSERT/UPDATE trigger, create an index on the new column, and join the tables on that; however, in situations where you don't have the option of doing ETL or triggers this can be useful.