In one line: Variation, not the average, is what customers experience as poor quality. Statistics is the engineer’s tool for understanding it, and it is most valuable when used to build scientific understanding, not just to analyze data afterward.
Core idea
Every process and every product varies. Parts differ, materials differ, conditions differ, and so does use. Tim Davis, drawing on decades in automotive quality engineering, argues that many engineering failures come from incomplete understanding of the underlying science, and that statistics earns its place when it is used to learn the science: to design experiments that reveal cause and effect, and to understand how noise drives failure [1]. Used only to summarize test results after the design is fixed, it comes too late.
How it works
Core ideas every engineer should carry:
- Distributions, not points. Describe performance by its mean and its spread. Two designs with the same average can fail at very different rates.
- Common and special causes (after Deming): routine variation built into the process, versus unusual events. Treat them differently (see Deming’s Quality Philosophy).
- Process capability: compare the spread of a process with the specification width. Indices such as and summarize how comfortably a process fits, and are the basis of Six Sigma.
- Measurement systems vary too. Before trusting data, check that the gauge’s own variation is small compared with what you are measuring.
- Correlation is not causation. Designed experiments establish cause; observational data suggests it (see Design of Experiments).
- Noise causes failure. Most field failures come from noise (variation in use, environment, manufacturing) pushing a non-robust design outside its working range (see Robust Design).
In practice
A practical habit is to ask, for any key result, what is the spread, and how was it measured? Another is to look at the raw data plotted over time before computing anything, since many “problems” turn out to be a single special cause or a measurement artifact. The Challenger Launch Decision is the sobering example of data that held the answer but was not plotted in a way that made the trend unmistakable.
Questions to ask yourself
- Do I know the variation of my key performance metric, or only its average?
- How capable is the process my design depends on?
- Am I using statistics to understand the physics, or just to report results?
Connections
- Applications: Robust Design, Design of Experiments, Six Sigma, Reliability.
- Philosophy: Deming’s Quality Philosophy. Cautionary case: Challenger Launch Decision.
- Capabilities exercised: Inquiry, Decision Making.