Quality Forensic Solutions LLC - Consulting Services for Improving Software Quality and, for Local Businesses, Dominating Google Search

An Engineering Measures Tip For Improving Attitudes About Measures

admin — Thu, 17 Jun 2010 21:21:24 +0000

If you’re in a Software or Hardware Development Engineering Process or Measures group, you’re familiar with the usual complaints from engineers about measures they have to collect and provide to the organization. Even though the measures add value for projects and the organization, they are sometimes viewed as a mundane chore that is forced upon the projects by the organization.

This is a tip on how to improve some attitudes on this issue. It’s simple. Use the word “Forensics” instead of measures for the measures your engineers use for their own project analysis, monitoring, and control. Code Size Measure, for example, would be changed to Code Size Forensic. What you’d be doing is using “Forensic” as a Generative Metaphor: an analogy or metaphor used to 1) strengthen perspectives or understanding of something, and 2) constructively generate new perceptions, explanations, and inventions.In their book, Made to Stick, Chip and Dan Heath use the example of how Disney uses a Generative Metaphor very successfully with its employees. This is a quote from the book:

“Disney calls its employees cast members. This metaphor of employees as cast members in a theatrical production is communicated consistently throughout the organization:

Cast members don’t interview for a job, they audition for a role.

When they are walking around the park, they are onstage.

People visiting Disney are guests, not customers.

Jobs are performances, uniforms are costumes.”

So, if you start using this word, your engineers would be using Forensics, just like the popular personalities on TV shows like CSI Miami to crack the case, discover the problem, find the solution, etc. The use of the word adds professionalism, importance, and a bit of excitement. It is generative, in that there is an expectation: you don’t use a forensic measure unless you expect that it may be helpful in discovering something or solving a problem. This will enforce the attitude that Forensics are used to accomplish something, not just to collect dust in company archives.

There are several technical arenas that use “Forensics”, so we should also be able to use it with Engineering Measures!

Criminal Forensic Science is, of course, the originating specialty – providing evidence suitable in a court of law for cause of death.
Forensic Materials Engineering or Engineering Forensics is the specialty of providing evidence suitable in a court of law for the causes of equipment, materials, or structural failure.
Chemical Forensics is the specialty of providing evidence suitable in a court of law for the causes of chemical damage, health issues, or injury. It also pertains to laboratory testing.
Security, Computer, or Network Forensics are for examining computers or networks for evidence of security violations, hacking, viruses, etc. They use Forensics Labs as well.

I have used this with my clients and after they got past the initial explanation, they were comfortable and enthusiastic about it.

If you need help with SW Inspections or want to understand how they actually work visit my Peer Review Process ROI website. Your comments on this article are most welcome.

Joe Lindley

Press Release on Software Inspections – Consultant Urges Focusing on their Effectiveness…

admin — Mon, 14 Jun 2010 11:31:58 +0000

Garland, TX (PRWEB) June 9, 2010 — Software bugs just continue to crop up, in almost all industries. Joe Lindley, Principal Consultant with Quality Forensic Solutions, LLC (QFS), urges that providing improvements in how software engineers remove defects with Software Inspections will be significantly effective in preventing software bugs. Software engineers use Software Inspections (also known as Peer Reviews or Formal Inspections) to inspect and remove defects from each other’s code.

According to Lindley, “It would be wonderful if a company could train its engineers on a Software Inspection Process and then let them independently use it over time. Unfortunately it doesn’t always work that way.” His experience has shown that for various reasons effectiveness in removing defects with Software Inspections tends to decline, often unnoticed, over time. QFS offers solutions that strengthen the Software Inspection culture for a company with the goal that the culture will self-improve over time. Quality Forensics (Software Inspection measures) are used to help software engineers appreciate, understand, and improve upon their own effectiveness in reducing defects.

Mr. Lindley was formerly an Engineering Measures Expert with Raytheon, developing innovative and Best Practice software development process improvements. His Process Performance Model for Software Inspections provides an enlightening view of the nonlinear behavior of Software Inspection Effectiveness and Return on Investment.

Information on how to improve Software Inspection Effectiveness can be found on the QFS website at Peer Review Process ROI.

VIDEO: How Peer Reviews (Software Inspections) Actually Work

admin — Thu, 03 Jun 2010 17:50:31 +0000

While at Raytheon I developed a Process Performance Model for Peer Reviews (Software Inspections) that explained a number of issues related to how they actually work in terms of effectiveness, cost, and the impact of improvements. I conducted a presentation on that at the Systems Software Technology Conference in 2009. The same presentation was used in 2009 at national Raytheon and Software Engineering Institute conferences. The following video is an abridged recap of that presentation, which is protected by a 2009 Raytheon Copyright.

Joe Lindley

For additional information see other posts in this Peer Review Process ROI website. Your comments on this video are most welcome.

Software Inspection Effectiveness – Why Do We Take it for Granted?

admin — Fri, 14 May 2010 14:21:53 +0000

Software Inspections (Formal Inspections or Peer Reviews) have been used for decades to remove defects during software development from requirements, design, code, test, or documentation artifacts. Several years ago, I was attempting to quantify effectiveness (Return on Investment) for Software Inspections, in other words, how many need to be conducted, and how well, to achieve quality and cost goals. I searched the internet but found nothing useful. After developing a solution, I was troubled by the lack of other solutions on the internet. Why hadn’t a solution been found?

If brilliant minds had failed at this then my solution was likely a joke! I was relieved to find that companies apparently weren’t trying to find a solution. They didn’t question Software Inspection effectiveness, so there was no pressing need to quantify it. But why would they assume that Software Inspections are effective?

I was pleased to find an answer in Nassim Taleb’s prescient book, The Black Swan — The Impact of the Highly Improbable. His concept, the Triplet of Opacity, asserted that we often make assumptions about the past and then falsely rely on those assumptions. As explained below, his concept applies to Software Inspections and even Coconut Cream Pie! Quotations are from his book.

Taleb: “The human mind suffers from three ailments as it comes into contact with history, what I call the triplet of opacity”:

1. “The Illusion of Understanding”

This is the tendency to view the world as more “understandable, more explainable, and therefore more predictable than it actually is”. We typically, for example, assign simple causes for business failings or successes that actually stemmed from complex situations. In the same way, some users have mistakenly felt that they understood Software Inspection effectiveness because of their core function: removing defects so they don’t escape into formal test. This is a component of inspection effectiveness, but is, unfortunately, just one of the factors that influence effectiveness.

2. “Retrospective Distortion”

Our perception of a historical event is naturally more simplified and organized than what actually happened. For Software Inspections, this distortion relates to the processes we use to conduct them. These processes are our retrospective understanding of how inspections were successfully conducted in the past. Unfortunately, they rarely convey the manner (discipline and attention to detail) with which those inspections were conducted. They typically suggest, for example, inspecting less than 400 lines of new code in a single inspection. Now, one commonly sees engineers relaxing this guidance by inspecting much larger code sizes. That’s the rub. Using a proven process doesn’t guarantee that it will be executed with the same effectiveness as the pioneers who developed it.

3. “…Handicap of authoritative… people”

The issue with authoritative people (experts) is that we often place too much faith in the validity or applicability of their opinions. The problem here for Software Inspections is over-reliance on expert approved processes. Experts may have approved a Software Inspection process, but that doesn’t mean that a company is effectively using it.

Summary

Some of us (in the Software Industry) haven’t quantified the effectiveness of Software Inspections because, through no fault of our own, we have taken them for granted. This may be the result of the natural human reactions identified here: our understanding of how they work, how to conduct them, or the supposed credibility of the processes we use.

As an analogy of how easily this occurs, consider the following true story. I love Coconut Cream Pie so decided to try the recipe used by my wife’s aunt. The Triplet of Opacity nailed me: 1) I thought it would be straightforward but it wasn’t; 2) she sent me her recipe but gave me more credit for cooking skill than I deserve; and 3) she had a reputation for her pies so I brashly make enough for 2 pies! The filling was a disgusting brown and tasted like burned caramel pudding!

In summary, due to human nature and the passage of time, some companies may experience less Software Inspection effectiveness than they’ve assumed. Taking action to measure and improve software inspection effectiveness may provide significant leverage in meeting the quality and cost demands of today’s marketplace.

Joe Lindley

Software Inspection Effectiveness – Why is it Still a Struggle?

admin — Tue, 11 May 2010 04:51:52 +0000

Software Inspections are used during software development to remove defects from requirements, design, code, test, or documentation artifacts. A formal multi-step procedure is used and inspection meetings are conducted to ensure adequate detection and removal of defects. Otherwise known as Formal Inspections or Peer Reviews, they have been used for 3 decades and are considered essential for timely delivery of quality software. Some companies have significant success with them. Many, however, suspect that they are not conducting them effectively. Why is this happening? How could such an accepted process fall by the wayside? As described below, there are fundamental issues that render Software Inspections quite a challenge to master.To use an aircraft analogy, Software Inspections are aerodynamically unstable — they don’t fly smoothly ahead if you take your hands off the controls. It would be wonderful if a company could train its engineers on a Software Inspection process and then let them independently use it over time. Unfortunately it rarely works that way. Left on their own with a Software Inspection process, projects are likely to subvert the process till it loses effectiveness. Some fundamental reasons for this are explained below: (1) the cost vs. benefit “pill” is hard to swallow; (2) measurement difficulties; (3) the nature of inspections; and (4) the Death Spiral (which occurs when inspections are poorly executed).

1. The Cost vs. Benefit “Pill” is Hard to Swallow

Imagine that you are a software project lead and a process engineer informs you that if you’ll commit more engineering labor to inspections during design you’ll achieve a 400% Return on Investment (ROI). The process engineer’s claim is likely valid. However, it would likely seem too good to be true. You would also find it difficult to commit to inspections during early development. You are comfortable committing labor to defect removal later in the project if too many defects are detected. Early in development, though, it’s a hard pill to swallow. You’d be paying for this up front, but would have to wait till project completion to see if you’re getting the promised ROI.

2. Measurement Difficulties

Engineering process organizations seek to ensure that Software Inspections are done correctly by measuring inspections – which often doesn’t work very well. There are two problems with measurement:

Some factors that determine inspection effectiveness are too arcane or subjective to measure. Artifact maturity, for instance, is rarely measured, but plays a role in how many defects will be found in an artifact (inspect too soon and many defects are found vs. inspect too late and few are found).

Most of the data for Software Inspection measures isn’t normally distributed, which increases the difficulty of establishing good control limits. There are statistical techniques for handling this, but it is nevertheless tempting to use rough, assumed control limits.

3. The Nature of Inspections

Software Inspections are an unnatural social interaction between software development team members. Most are uncomfortable with sitting down with a team mate to tear apart his/her work. That team mate is likely even more uncomfortable! The rationale for this is usually covered during Software Inspection training. The problem occurs years after training as engineers gravitate to a more comfortable style of inspections that is more superficial (gathers fewer significant defects) but less effective.

4. The Death Spiral

Projects that lack Software Inspection effectiveness usually have engineers that dislike Software Inspections. This is a symptom of the Death Spiral. If engineers aren’t effectively conducting inspections, they soon realize that they are wasting time conducting them. Engineers hate to waste time, so they then find ways to avoid inspections by gaming the process. This makes inspections more wasteful, inviting more gaming and a continual cycle of declining effectiveness.

Summary

Consider the combined impact of these issues with Software Inspections. It’s hard to convince leaders to use them, and it’s difficult to measure effectiveness, which has a tendency to decline slowly or rapidly. Returning to the aircraft analogy: they are hard get off the ground, difficult to fly, and tend to lose altitude or crash! All is not lost, however. Peer Review (Software Inspection) ROI is achievable. We just need to implement effective remedies and keep our hands on the controls.

Joe Lindley

How Peer Reviews (SW Inspections) Actually Work – Nonlinear Impact

admin — Wed, 05 May 2010 16:23:38 +0000

It’s odd but true that even though we’ve been using Software Inspections (Peer Reviews) for 30 years, most of us don’t really know how they work. By “work”, I mean, how well can you expect them to work and what issues impact that. In other words, if you conduct 10% more inspections will you get a software product that has 10% fewer defects. The answer to that question is unfortunately, “it depends”. The answer to the broader question of whether they provide the glowing Return on Investment (ROI) we’ve been promised is a resounding “Yes”.

What I’d like to discuss here is a bit of the detail space between the two questions posed above: what issues make them work better or worse (yield higher vs. lower ROI). In this first post on this topic, I’ll cover one of the most interesting: the nonlinear impact of Software Peer Review defect removal.

This chart portrays the results from 10 software development projects (green squares) illustrating this nonlinear behavior. The vertical axis is Software Product Density (how many defects were found during formal test (Systems Integration) normalized by the amount of software developed). The lower this is, the better Peer Reviews performed in removing defects prior to formal test. The horizontal axis is a score, comprised of the amount and type of defects removed with Peer Reviews coupled with an adjustment called PEP which accounts for innate differences in projects related to defects (some are “good and lucky” and don’t have much likelihood of defects while others can expect more difficulty with defects). Note the shape of the curve fitted to the data. On the left side, where the projects removed few defects with Peer Reviews the curve drops quickly – which means that a slight increase in defect removal causes a relatively large drop in defect density. These projects obviously get a ton of ROI for doing more defect removal. This makes sense, because they are probably only conducting Peer Reviews on their worst artifacts, so will get a lot of benefit out of fixing them. As the curve moves to the right, though, it starts to flatten out, yielding less and less of a drop in Defect Density as more defects are removed. This also makes sense, because these projects are inspecting much more of their code base, so they are, in some cases, inspecting artifacts that have few or no defects. This additional effort, in other words, is still helping to remove defects, but is not nearly as effective. This therefore is the nonlinear impact of Peer Reviews: high ROI initially, but as more and more defects are removed, less and less ROI can be expected for additional effort invested.

There are other interesting issues with Peer Reviews (Software Inspections) I’ll cover later in Peer Review Process ROI posts.

Joe Lindley

Why Use QFS Consulting Services

admin — Tue, 04 May 2010 14:53:44 +0000

If the following is true (or partially true) for your organization, Quality Forensic Solutions, LLC (QFS) should be the best choice for consulting services:

There is a Peer Review Process in place or one is needed.
There is suspicion or proof that Peer Reviews are not performing as you expect.
There is interest in cost effectively improving the current process rather than replacing it with a new process.
There is an upcoming CMMI assessment.

If the above is true for your organization, QFS is the best choice because:

We drive positive, bottoms-up behavioral change (i.e. Peer Review practitioners learn to appreciate and improve how they conduct Peer Reviews).
We drive positive, top-down managerial change (i.e. managers learn to appreciate and improve how they manage Peer Reviews).
We provide proven support for CMMI compliance relative to measures and Process Performance Models (PPMs).

We will endeavor to provide useful and timely Peer Review Process ROI information on this website.

Joe Lindley

The History and Future of Peer Reviews

admin — Thu, 22 Apr 2010 05:17:21 +0000

Peer Reviews took the software development industry by storm in the 1970s and 1980s. They remain to this day a defacto standard for software development. They have also been adopted across the engineering development spectrum from hardware development to testing and documentation of any type.

The reality now, however, is that companies achieve mixed success in reaping the benefits they should from Peer Reviews. We see a broad spectrum here, from companies who barely, if at all, perform them, to companies who have outstanding Peer Review cultures.

Why do we have such a broad range of effectiveness? It boils down to 2 fundamental issues. First, Peer Reviews aren’t a natural social activity for engineers (criticizing the work of their team mates), so engineers, if not continually nudged into conducting Peer Reviews correctly, will inevitably shift into superficial and wasteful forms of Peer Reviews. Secondly, we haven’t had a practical way to measure real performance (Return on Investment). Couple these two issues and the current reality makes sense. Without effective measures to guide them in the right direction, engineers in many companies have (without notice) gradually shifted into bad habits.

The good news is that with Peer Review Process ROI concepts we have solutions now. We have effective approaches to measuring both Return on Investment and behavior changes that drive real improvements in performance. We are equipped now to change the course of Peer Review history. The future for Peer Reviews will be very different than the stagnation we’ve witnessed over the last 30 years.

Joe Lindley