Cleaning out his basement, Brian comes across a document that shows how little programming has changed in 23 years.
A few weeks ago I was cleaning out my basement, and while going through old papers I came across the following:
It was a hand-written root cause analysis data sheet from a project I had worked on back in 1988! It was amusing and a bit satisfying to know that I had been thinking about root cause analysis over 23 years ago, long before things like TDD, BDD, Agile, Scrum, Java, or Linux even existed. On the other hand, it was a bit depressing to realize that in many ways the field has not made much progress in the intervening two decades.
This project was for one of my first consulting jobs and the assignment was to develop either a basic checkwriting program or a full Accounts Payable and Billable system. If you’re puzzled by that “or,” so was I. The manager told me that they were not sure which product they were going to build but that I should start coding right away because I was an expensive hourly contractor, and that they’d decide which product I was building in a month or two.
I wish I could say I was making that up, but it’s true. Faced with that problem I decided to spend the limbo months developing utility classes that would be needed for either project.
Given this background it’s not surprising that the highest percent cause of the bugs came from incomplete requirements. That certainly hasn’t changed. It’s also interesting that the most reported category ended up being “not a bug.” That speaks to a low level of communications between development and testing (there was no notion of QA being a key part of the development team back then).
If we convert the categories from 1988 style to current terms and combine them a bit we get:
And if we then graph those categories we get:
That pie chart looks like it could have come from one of my current projects! With all of our progress, with our Integrated Development Environments, our Test Driven Development, Agile Methodologies, and such, we haven’t made a lot of progress of reducing the bugs or even changing where they tend to come from.
I wonder how much of my cumulative salary over the past 23 years has been spent working on what turned out to be late, incomplete, or flawed requirements? What is the cumulative delay in all the projects I’ve worked on due to such things? I still hear people say “we don’t have time” and “quality is too expensive.” I guess I’m preaching to the choir because if you’re reading this magazine it’s safe to assume that you get it, but clearly we haven’t sold our message to the mainstream—and I wonder why.
I have to believe that a lot of the problem comes from how we train developers. I’ve said it before but most software students have never seen a requirements document. And if you count a class assignment as a requirement they’ve certainly never seen feature creep or shifting requirements. How can they deal with something they’ve never trained for?
What Actually Got Built?
By the way, the project ended up creating a form-filling program that supported a family of small business forms such as invoices, change orders, customer statements, and checks. In all, we supported over 30 distinct form types, each with their own business rules and layouts. We also supported a wide variety of printers back in the day before generic printer drivers (which is where the “strange hardware” bug category came from).
By the way, in yet another past-is-prologue moment we were hit with a patent infringement claim from a disgruntled ex-employee the week before we were going to ship. Back in the day of continuous-form printing it was crucial to get the top of the form aligned with the top of the printer. The forms had numbers along the side and the program would print an “X” in the margin. The number where the “X” was printed told you how many lines to manually move the paper in the printer. The folks at the USPTO gave this ex-employee a patent on the formula “Lines Per Form - X_Location = LinesToScroll”!
In the week before our major product rollout we changed the alignment method to “print the alignment character and then adjust the paper. Repeat until the “X” is printed at line 1.” That apparently passed muster as being different enough to be non-infringing. Given the current Apple/Google/everyone patent wars, it doesn’t seem like much has changed!
Brian Tarbox is a Distinguished Member of Technical Staff at Motorola where he designs server-side solutions in the Video On Demand space. He writes a blog on the intersection of software design, cognition, and Tai Chi at briantarbox.blogspot.com. His primary contact point is about.me/BrianTarbox.