Design of Experiments

In one line: Instead of changing one factor at a time, change several factors together in a planned pattern. You learn each one’s effect, and their interactions, from far fewer tests.

Core idea

The intuitive way to experiment is one factor at a time: hold everything else constant, vary one thing, then move to the next. It is slow, and it misses interactions, where the effect of one factor depends on another. Design of experiments (DOE) varies factors together according to a structured plan, so that the effect of each factor can be separated statistically. It is the core improvement tool in Six Sigma and the engine of Robust Design.

How it works

1. Define the response to measure, such as flight distance, yield or noise figure.
2. Choose the factors and their levels, typically two or three settings each.
3. Choose a design:
   • Full factorial: every combination. It is complete but grows fast: 4 factors at 3 levels is $3^4=81$ runs.
   • Fractional factorial or orthogonal array: a balanced subset in which every level of every factor appears equally often with every level of the others. For example, an L9 array tests 4 three-level factors in just 9 runs.
4. Randomize and replicate where possible, to guard against drift and to estimate noise.
5. Run and analyze:
   • main effects: the average response at each level of each factor
   • interactions: where two factors’ effects combine
   • variability: for robust design, how much each factor affects the spread
6. Confirm the predicted best settings with a confirmation run.

In practice

The program’s lab uses paper airplanes. Four factors (weight position, stabilizer flap angle, nose length and wing angle), each at three levels, are tested in a nine-run orthogonal array. Teams fly each configuration, plot the main effects, pick the best combination, and fly it to confirm. The method transfers directly to engineering work such as tuning a process recipe, a circuit’s bias points or a test setup. In those settings the cost per run is high, so fewer, better-planned runs matter.

Questions to ask yourself

• Where am I running one-factor-at-a-time experiments that a designed experiment would do faster?
• Which two factors in my design might interact?
• How many runs would a full factorial need here, and could an orthogonal array give me the main effects with far fewer?

Connections

• Purpose: Robust Design. Foundations: Statistics in Engineering. Home in: Six Sigma.
• Related: Prototyping (what to build for each run), PDCA Cycle (the learning loop).
• Capabilities exercised: Inquiry, Resourcefulness, Decision Making.

Sources

• Ulrich and Eppinger - Product Design and Development [1]
• Davis - Science, Engineering, and Statistics [2]
• Course lab: an orthogonal-array design-of-experiments exercise with paper airplanes.

[1]

K. T. Ulrich and S. D. Eppinger, Product Design and Development, 6th ed. New York, NY, USA: McGraw-Hill Education, 2016.

[2]

T. P. Davis, “Science, Engineering, and Statistics,” Applied Stochastic Models in Business and Industry, vol. 22, no. 5–6, pp. 401–430, 2006, doi: 10.1002/asmb.643.