Software Inspections (Formal Inspections or Peer Reviews) have been used for decades to remove defects during software development from requirements, design, code, test, or documentation artifacts. Several years ago, I was attempting to quantify effectiveness (Return on Investment) for Software Inspections, in other words, how many need to be conducted, and how well, to achieve quality and cost goals. I searched the internet but found nothing useful. After developing a solution, I was troubled by the lack of other solutions on the internet. Why hadn’t a solution been found?
If brilliant minds had failed at this then my solution was likely a joke! I was relieved to find that companies apparently weren’t trying to find a solution. They didn’t question Software Inspection effectiveness, so there was no pressing need to quantify it. But why would they assume that Software Inspections are effective?
I was pleased to find an answer in Nassim Taleb’s prescient book, The Black Swan — The Impact of the Highly Improbable. His concept, the Triplet of Opacity, asserted that we often make assumptions about the past and then falsely rely on those assumptions. As explained below, his concept applies to Software Inspections and even Coconut Cream Pie! Quotations are from his book.
Taleb: “The human mind suffers from three ailments as it comes into contact with history, what I call the triplet of opacity”:
1. “The Illusion of Understanding”
This is the tendency to view the world as more “understandable, more explainable, and therefore more predictable than it actually is”. We typically, for example, assign simple causes for business failings or successes that actually stemmed from complex situations. In the same way, some users have mistakenly felt that they understood Software Inspection effectiveness because of their core function: removing defects so they don’t escape into formal test. This is a component of inspection effectiveness, but is, unfortunately, just one of the factors that influence effectiveness.
2. “Retrospective Distortion”
Our perception of a historical event is naturally more simplified and organized than what actually happened. For Software Inspections, this distortion relates to the processes we use to conduct them. These processes are our retrospective understanding of how inspections were successfully conducted in the past. Unfortunately, they rarely convey the manner (discipline and attention to detail) with which those inspections were conducted. They typically suggest, for example, inspecting less than 400 lines of new code in a single inspection. Now, one commonly sees engineers relaxing this guidance by inspecting much larger code sizes. That’s the rub. Using a proven process doesn’t guarantee that it will be executed with the same effectiveness as the pioneers who developed it.
3. “…Handicap of authoritative… people”
The issue with authoritative people (experts) is that we often place too much faith in the validity or applicability of their opinions. The problem here for Software Inspections is over-reliance on expert approved processes. Experts may have approved a Software Inspection process, but that doesn’t mean that a company is effectively using it.
Summary
Some of us (in the Software Industry) haven’t quantified the effectiveness of Software Inspections because, through no fault of our own, we have taken them for granted. This may be the result of the natural human reactions identified here: our understanding of how they work, how to conduct them, or the supposed credibility of the processes we use.
As an analogy of how easily this occurs, consider the following true story. I love Coconut Cream Pie so decided to try the recipe used by my wife’s aunt. The Triplet of Opacity nailed me: 1) I thought it would be straightforward but it wasn’t; 2) she sent me her recipe but gave me more credit for cooking skill than I deserve; and 3) she had a reputation for her pies so I brashly make enough for 2 pies! The filling was a disgusting brown and tasted like burned caramel pudding!
In summary, due to human nature and the passage of time, some companies may experience less Software Inspection effectiveness than they’ve assumed. Taking action to measure and improve software inspection effectiveness may provide significant leverage in meeting the quality and cost demands of today’s marketplace.
Joe Lindley
Content Copyright 2010, Quality Forensic Solutions, L.L.C. All rights reserved.
