I authored the following article for CMP Media (c) 2003 which appeared in the Developer Market News, January 2003 issue. Access to the newsletter is limited, but this appears with permission.
Software’s Quest for Qualityby Jim Sherburne
Are We There Yet?The software industry has been mired down by its seeming inability to deliver products that actually do what they were designed to do in a consistent and reliable fashion. Last time I suggested that other mature industries like manufacturing found their answer in the works of W. Edwards Deming, first published back in the 1930’s. Deming, a statistician by training, was responsible for a revolution of quality management in production processes – first most significantly in Japan, and later throughout the Western world. Programs like Six Sigma owe much to Deming’s seminal works in this field.
Art vs. Science
The admittedly high levels of creativity that go into the development of truly innovative software have all too often made software developers reluctant to accept the kinds of rigorous disciplines associated with their contemporaries in more mature traditional industries. They feel they are more artists than engineers. I’ve always found it interesting that in fact many of the truly gifted software professionals that I have had the pleasure of working with over the years are artists of one type or another – often musicians.
Creating music without the discipline of having learned an instrument is typically not a terribly productive exercise. Just like with music, creativity in programming without some discipline attached will only get you so far. I would suggest that the industry has lost years of productivity believing that there was just no way to produce software in the same way that other durable goods were produced. With the formalization of various software development processes, like the waterfall and iterative methods, the industry has seen deployment of quality control methods like ISO 9001 and SEI CMM certification. This has provided at least a ray of hope for some that you could indeed product software using a controlled process. The success of such efforts however has been limited. Why?
· They create large amounts of documentation to certify that a company follows a written procedure for designing software.
· They require a significant human labor to verify and maintain, often mandating the existence of a bureaucracy whose sole mission is to sustain the process.
In other words, they add cost and perhaps more importantly they erode the existing pools of resources necessary to complete a project. Has anything been gained as a result of these additional costs?
Getting Closer
Despite their cumbersome bureaucratic shell certification processes like ISO 9001 and SEI CMM do at least give us an indication of the right direction to take.
The International Standards Organization (ISO) is a global federation of standards bodies that address the quality management aspects of virtually every aspect of modern commerce. ISO 9001 is actually only a requirement standard you use to assess your ability to meet customer and applicable regulatory requirements and as a result address customer satisfaction – not just for software, but virtually any industry.
Instead of only relying on the more generalized ISO 9001 standards, a software specific standards body has emerged from Carnegie Mellon University – the Software Engineering Institute (SEI). They produced what they feel is an objective way to measure software development through the use of a Capability Maturity Model
(tm) (CMM).
CMM has become a de facto standard for assessing and improving software processes. It uses a five-step classification process. The initial level 1 phase is characterized as primarily chaotic, with success typically dependent on individual effort and heroics. The process stages up incrementally to level 5, which is characterized as optimizing – meaning continuous, process improvement. This is enabled by a reliable flow of quantitative feedback from the process itself. Predictability, effectiveness and control are the watchwords here.
Challenges Are Huge
It all sounds wonderful in the abstract, but how realistic is wide spread adoption of something like CMM? Think about how difficult a situation we’re talking about here. Every single piece of software out there is unique. In my experience even the most active proponents of CMM will not – privately at least - try to lay claim to compliance with anything higher than level 3 on the CMM scale – which merely defines a process. Quality measures do not even become a part of the equation until level 4.
My point here is not to laud or condemn CMM and other similar efforts. The intent of CMM is admirable. Its implementation however is at best resource intensive, and at worst clumsy and overly bureaucratic. When was the last time you heard an IT manager at any level say, “Resources are not an issue, we’ve got plenty of people we can throw at the problem”? This is just not reality. All to often the result is to defocus an IT organization from their primary responsibilities, which are to get a functioning piece of software deployed on schedule.
Even if nirvana is achieved and an IT organization can legitimately lay claim to being CMM level 5, is it the process itself that is assuring you a quality product? No! It’s the human interactions with that process that accomplish that. Quality is hard under the best of circumstances, and making quality software is even harder.
Because of all this the state of quality management in software development today is still stuck. I would submit that it remains roughly comparable to that of manufacturing when Deming’s findings were first published – in about 1930! A number of national studies have documented that the typical commercial software development group spends between 50-75% of their time just chasing bugs, with an attendant cost in the United States alone in excess of $59.5 billion annually . This represents an enormous drain on resources and productivity for businesses.
We are not talking about a quick fix here. This is not simply a matter of needing to make minor adjustments to a process. Information technology is central to modern business.
What’s Missing?
What we are talking about is a need for fundamental changes in how software is written that will allow organizations to survive as a business. The weak link in all the efforts to date has been the need for human intervention into an inherently complex and ever changing process.
An ideal solution would be to have an array of intelligent, automated tools or processes that remove, or at least minimize human intervention. Another possibility would be software that corrects itself, again no human intervention. Such things simply have not existed for software in the past. That’s why quality management has historically never been a fundamental part of the software creation process. It is only in the last 3-4 years that we have seen a glimmer of hope that this is starting to change.
RETURN TO RESUME RETURN TO PROFESSIONAL BIOGRAPHY
