Blogs are a useful and important tool for professional developers and engineers, for two main reasons. Firstly, by and large, we are not free to divulge the details of the work we do while under contract — but at our next interview, those details are precisely what the interview itself is there to judge. So blogs (and I include this one without question) can take on a professional portfolio role, showcasing examples of professional work as applied to side projects, giving professional opinions on relevant products and work practices and so forth, all of which informs an interviewer who does their research as to the suitability of a candidate for a position. This is an old technique that artists and graphic designers are explicitly familiar with.
Relevant professional blogs can also act as a cheap way to monitor your continuing professional development. They’re not training courses, they’re not accredited and they never will be; but observing what your peers are doing is a very old way of watching for trends that you might consider directing your own CPD efforts towards. Through RSS feeds, RSS aggregator sites and a decent RSS reader, it is relatively easy to monitor large numbers of blogs with little effort beyond the actual reading of new articles as they appear.
The problem is that some bloggers, so focussed on the first goal of presenting a good online portfolio, and so obsessed with metrics like pages-served-per-week as a measure of progress towards that first goal, post dross which can then impede others in the pursuit of the second goal because of the lack (and impossibility) of an effective peer review process. In some cases, bloggers seem to abandon completely the goal in pursuit of the metric – chasing after eyeballs without having anything of substance to then display to this newly captured audience.
The specific example that pushed me to write this comment is this latest post from Joel Spolsky’s blog entitled “The Duct Tape Programmer”. However, this is just the most recent example – it’s a sadly growing trend, and it’s long since turned nasty. Zed Shaw‘s now-infamous “Rails is a Ghetto” post, a forceful criticism of the programming community surrounding the Ruby on Rails platform, has been copied so many times by people attempting to surpass its levels of outrage that he has now in fact pulled the original post from the internet and will not re-release it, believing that these levels of unprofessional comment do no good. It’s a good sentiment, but some degree of comment is still called for — finding the right balance is an ongoing challange.
Returning to the motives for this comment with that restraint in mind, on the first day of engineering as an undergrad we were shown video footage of the Tacoma Narrows bridge collapse. For the few who have not heard of, or seen this, it’s a classic engineering cautionary tale, showing how a groundbreaking theoretical advance in one area of structural engineering failed because of a then-little-understood phenomenon in another area (aeroelastic flutter in this case). It’s often the first example of this shown because it’s got an excellent visual record (the footage is now preserved by the Library of Congress as being of significant cultural significance) and because it’s spectacular to watch with an obvious failure mode; and because no human lives were lost as a result (the unfortunate dog is rarely mentioned).
Later examples of what happens when engineering projects go awry are less pleasant to consider. Whether they be cases where the engineers themselves did nothing wrong, as in the Tacoma Narrows collapse; or cases where the engineers were not listened to, as in the Space Shuttle Challanger disaster or the Hyatt Regency walkway collapse; or in cases where poor engineering was the root cause of the error, like the failed Patriot missile intercept at Dhahran or the Therac 25 radiation treatment system; the end lesson was the same – an unacceptable outcome. A failure in an engineering system can cost lives, even in systems that appear noncritical at first glance. More often, such failures cost enormous sums of money – €355 million euro in today’s money for the Ariane 5 Flight 501 disaster, for example, an error caused by not testing a subsystem within its expected operating parameters. Even the most jaded and unethical cynic who place prices on human lives lost via the cost incurred in compensation and damages by the company held responsible, find such penalties unacceptable (which is, obviously, the justification for financial penalties in such cases).
The standard approach taken to minimise or prevent these unacceptable outcomes is conservative design, using trusted components, with testing and quality assurance being critical parts of the design and implementation processes. In older branches of engineering – civil and mechanical engineering to give good examples – this isn’t a standard approach, it is the only approach. Society decided quite some time ago that bridges that didn’t stay up were simply not acceptable. Younger branches of engineering, however, are only still learning this very old lesson. Computer and software engineering, the youngest branches of all, are in some quarters still wondering if it’s really necessary.
A lot of this is because of the desktop PC market having such prevalence. It’s a large and easily entered market segment, so it dominates teaching and its commercial entities are highly visible publicly – and that (because of the less obvious image of other, larger market segments like embedded systems) may not be a good thing. Few desktop applications have direct critical status where a failure would cost lives. Most software companies specifically preclude their software being used in such a capacity as part of their end user licences. And market forces in the desktop market are such that time-to-market and initial development costs need to be minimised. With pressure like that, and consequences not seen as critical, corners are cut relentlessly. The new sub-market of web programming is even more sensitive to these pressures and has in fact evolved new work practices to cope with them, even to the point of glorifying speed of development past the point of professional efficiency. And critically, that attitude feeds back into the teaching environment, polluting the learned work practices not only of those engineers and programmers who will go on to work in the desktop and web markets, but also of those who will go on to work in markets where failure is far less acceptable and far more costly. This unnecessarily extends training times and raises risks associated with new engineers.
Hence the provocation caused by Spolsky’s post, which states that a “duct tape programmer” (meaning one who rushes code out the door and using the specific example of Jamie Zawinski from his interview in Coders at Work) is a superior programmer. In one way, he does have a valid point, though badly made, with regard to complexity – as pointed out in Thirty Years of C suspiciously recently, some features of some languages (in this case the higher abstractions in C++) are sufficiently ill-understood and poorly standardised in implementation that it is a good work practice to avoid using them. However, while Dyer was making a point based on insufficient reliability and robustness in the available tools, Spolsky is making the same assertion purely on the basis that it takes more time to produce a good design than to use a simpler tool and “duct tape” a solution.
He then goes past the point where there was a defendable justification, agreeing with the throwing out, not only of complex designs with their long development time, but also multiple inheritance, multithreading, and unit testing amongst others. I would certainly agree that certain business practices are inefficient (and Scott Adams makes a fun living documenting many examples of this), but this dumbing-down of the engineers toolbox is ridiculous.
Multiple inheritance in many cases can be an avoidable and poor design choice – but in some cases, such as PyQT GUI design using QT designer, it is the simplest solution to an inherently complex problem. It also speeds development as an additional benefit.
Multithreading is difficult to design for if you rush the design. However, to throw it out on the basis that it is overly complex is to abandon large amounts of difficult problems that require solutions – some situations simply require concurrent processing and cannot be reliably handled in any other way. Embedded programmers have being using concurrent techniques for decades now, and while there are well-understood dangers (and less-well-understood ones, and presumably completely unknown ones as well), it’s a necessary tool in the engineer’s toolbox.
And as to throwing out unit testing on the grounds that it does not add to the final product because the customer doesn’t see it and that it costs time to add — there are some well-known if not terribly polite responses to such a suggestion which to my mind answer the idea insufficiently forcefully. Unit testing – testing of all kinds, in fact – is not an optional extra. To see its abandonment recommended publicly is somewhat outrageous.
Certainly, not everyone writes embedded system code or medical or financial software – and those outside these perceived critical zones might think this testing to be overkill, might feel that if your iPhone mashup app or even your iPhone needs to be restarted because of a bug, that that’s just a small glitch, easily fixed and not worth developer time to prevent. The problem with this school of thought is that we’re coming into (if not already deeply embedded in) an age where people have lost their mistrust of computer systems to the point where they will drive cars onto train tracks into the path of an oncoming commuter train because their GPS told them to, or drive a twelve-foot-tall busload of children under a nine-foot-tall stone bridge because their GPS didn’t warn them (even when the warning signs outside the bus did).
The domain of software which can be considered critical – where a failure can cost enormous amounts of money, or worse, lives – is expanding without much notice being taken. It’s later than you think – and you may already be in that domain.
Testing your code is not an optional extra. And duct-taping a solution can be harmful.
2009-09-25 » Mark Dennehy