Without Data, Talk About Quality Is Just Talk

by Alan Willett

Immutable Law Of Speed #4:
The Higher the Cost of Rework, the Slower You are Going.— From Alan Willett’s Book, Lead With Speed

This old picture I found well represents what I am talking about today. That broken axel has disrupted everyone’s workday. This disruption removes everyone involved from any creative work they were doing to advance the goals of their organizations.

Now consider some of the headlines I have collected about defects that escaped into the world, disrupting people and organizations.  

Two years ago:  Fiat Chrysler recalls 4.8 million US vehicles for a cruise control software defect.

Last year: GM is updating a brake controls recall they originally issued in December of 2019 because the first recall caused additional problems.

This year: Southwest Airlines cancels 500+ flights after a computer glitch grounds their fleet.

I have collected hundreds of headlines like these over the years. Each of these issues represents problems with customer loyalty. It cost hundreds, even thousands, of hours of lost productivity in the organization. Sometimes it even cost lives.

There is a problem I see consistently across many organizations. Quality is typically assumed. Leaders ask a lot of questions about dates. They want to know when the project will be done. They also have many requests for more features.  

However, there are typically few if any questions about quality. But those questions must be asked. And the answers to those questions must be based on data.

Here are a few examples.

  • Our quality levels in system test show 0.05 defects per thousand new lines of code compared to the average of 2 defects per thousand lines of code we found in our previous releases.
  • Our inspections had a defect removal efficiency of 42%.  58% of the defects escaped into testing or out to customers.
  • Our economics of quality numbers show that the defects captured in the system test are costing the organization an average of 125 hours per defect.  The smallest cost was 30 hours, and the greatest cost was over 1,000 hours.
  • Only 27% of product features were delivered to our customers within our criteria for high quality with no critical or level 1 defects. This is unacceptable. We are taking action to address the root causes.

You may not know what these data descriptions mean, but you should.

Claiming quality without knowing data like this is just talking.

And if it is just talking, the results you get will be talked about just like in the headlines I mentioned.

Yours in the calm pursuit of excellence,


Alan Willett