Monday blues

Monday morning, 8:00 am, and I’m by myself in the office, watching the coastal fog very slowly recede over the hills. At 8:00 am, I’m usually alone in the office. Geeks are not a matinal species. The office usually starts filling in after 10 am only. There are often checkins into our source control system at 1 am though.

The startup I work for is in the same building as a few financial outfits, a branch of Morgan Stanley, a Bank of America, and more on the second floor. The necktie wearing people I crossed on that floor this morning were bleary eyed and “wandering around in a daze”, as they reported in the New York Times. Lehman Brothers in bankruptcy, Merryll Lynch bought out by Bank of America. Don’t worry, the windows don’t open in this building, the branch managers on the second floor won’t be able to defenestrate themselves.

It’s weekly report time for me, and in this blog, I want to write about geeks and irrationality. I thought a bunch of seasoned software engineers would be a pretty rational crowd, but it doesn’t appear to be so. We are building a really complex system here at company X and in good old Cartesian fashion, I thought it would be efficient to decompose the system into modules and sub-modules, and test those separately. In 1637, Descartes was already engineering that way. His second precept in the Discourse on the Method is “to divide each of the difficulties under examination into as many parts as possible, and as might be necessary for its adequate solution.” Alas, my colleagues haven’t read the Discourse on the Method (probably too old), and they insist on putting the big, complex system together and then testing it with “real data”. They feel that testing is not worth doing if it doesn’t use “real data”. I agree with that, but I still want to decompose into modules, have clear specifications for the modules and write tests against those specifications. I’m concerned that if we throw everything and the kitchen sink into the System and then turn on the ignition key with “real data”, the System will simply not perform adequately, and we’ll have one giant Mess to dig into to figure out what is going on. I’m pretty sure we’ll be worth off if we can’t analyze our system in terms of simpler modules. Maybe there isn’t enough pressure on us software engineers to get it right the first time around. In other engineering disciplines, when building an Airbus for example, they can’t overlook their subsystems till it’s time to take off. Or maybe I should change my view of testing. For the components I write, I shall henceforth not test them but simply throw them into the System, and wait till the System flounders miserably. It’ll still be time to write specifications and to test then, right? In the meantime here is a law of software engineering:

Complex Software System + Real Data + No Analysis into Modules = One Giant Mess.