Automate, assist or retain manual visual inspection on a finishing line
A vehicle component manufacturer decides whether machine vision should replace, support or leave alone the inspectors who check every part before despatch. The log shows why a single accuracy figure i...
The options
The question: Should visual inspection on the finishing line be automated, used to assist the inspectors, or left manual?
Option 1: Retain manual inspection and add a fifth inspector
For
- Evidence. The inspectors detect the fine defect classes at a rate no automated option has demonstrated, and can recognise defect types absent from the taxonomy because they judge parts against an expectation rather than a label set.
Against
- Risk. Inspection performance falls measurably across a shift. The re-inspection log shows the false reject rate rising in the final two hours, and adding a fifth inspector increases capacity without addressing that pattern.
- Risk. It leaves the inspection station as the line constraint at the next volume increase, and the recruitment market for experienced inspectors is the reason the fourth post took nine months to fill.
Option 2: Assist: machine vision flags candidate defects, an inspector adjudicates every flag
For
- Evidence. In the vendor trial on 4,000 labelled production parts, the model and the incumbent inspectors missed largely different defects. Coarse classes were detected by the model at a higher rate than by inspectors; fine classes at a substantially lower rate. Fine-class defects were few in the sample.
Against
- Risk. Under this option a part the model does not flag is never examined. Its escape rate therefore depends on how sensitively the model flags the fine classes, where it is weakest, and the trial sample contained too few fine-class defects — low-contrast hairline cracks in particular — to set that sensitivity with confidence. Raising sensitivity to compensate increases the flag load and erodes the throughput gain.
- Risk. Adjudicating flags is a different skill from examining parts, and inspectors who see mostly true flags tend over time to confirm rather than examine. The benefit depends on the inspector continuing to disagree with the model, which is a behaviour and not a setting.
Option 3: Automate: machine vision decides, with a sampled audit by an inspector
For
- Benefit. The option removes the shift-length performance decline entirely, and inspection capacity ceases to constrain line output at any volume currently forecast.
Against
- Risk. The option performs worst on precisely the defect classes the published benchmarks identify as hardest, and removes the inspector who currently covers them. Its escape rate is unconfirmed for that reason, and the constraint that no option may raise the escape rate cannot be shown to be met.
- Risk. A sampled audit detects a drift in model performance only after a period of production has passed. The quantity of material at risk between audits is set by the sampling interval and has not been agreed.
Option 4: Split by defect class: automate the coarse classes, retain manual inspection for the fine ones
For
- Rationale. The split follows the boundary the published benchmarks identify rather than an arbitrary one: the model is applied where it has been shown to outperform manual inspection and withheld where it has been shown not to.
- Research. Reviews of visual defect detection, among them A Review of Benchmarks for Visual Defect Detection in the Manufacturing Industry, report that small and low-contrast defects remain materially harder to detect than larger surface defects such as scratches, dents and delamination. No published figure transfers directly to this organisation's parts, and performance on its fine defect classes cannot be inferred from a headline figure quoted for the coarse ones.
- Recommendation. Split inspection by defect class: automate the coarse classes and retain manual inspection for the fine ones. It is the only option that raises throughput above the forecast demand while keeping full examination by an inspector on the fine classes, where the published benchmarks say the model is weakest. The assist option is cheaper and has a lower false reject rate, but a part it does not flag is never examined, so the model screens the fine defects first; and its benefit depends on inspectors continuing to disagree with the model's flags. Retaining manual inspection leaves the station as the line constraint at the next volume increase. The full automation option cannot be recommended while its escape rate is unconfirmed, because the constraint that no option may raise the escape rate is the one constraint that cannot be traded.
Against
- Risk. It requires the defect taxonomy to map cleanly onto part numbers, so that routing can be decided before inspection rather than after. Three part families carry both coarse and fine defect classes and would have to pass both stations, reducing the throughput benefit.
- Risk. Two inspection points is a more complex line than one, and the boundary between them has to be maintained as the defect taxonomy changes. A defect class that migrates from coarse to fine without the routing being updated is passed to a station that is not looking for it.
Criteria and values
| Criterion | Option 1 | Option 2 | Option 3 | Option 4 |
|---|---|---|---|---|
| Defect escape rate | 1.4% | 1.1% | — | 1% |
| False reject rate | 2.1% | 1.6% | 3.4% | 2.8% |
| Capability on fine defects | strong | adequate | weak | strong |
| First-year cost | 58,000 EUR | 285,000 EUR | 310,000 EUR | 295,000 EUR |
| Inspector roles affected | 0 | 4 | 4 | 2 |
| Inspection throughput | 210 | 340 | 620 | 430 |
| Time to validated operation | 9 month | — | — | — |
What constrained the decision
- Constraint. A defect escaping to the customer is a warranty and reputation event, and two of the four largest customers require notification of any change to the inspection method before it takes effect. No option may raise the escape rate above the current level.
- Constraint. Any automated inspection must be validated against a labelled sample of production parts covering every defect class in the taxonomy, including the classes that occur only a few times a year. Assembling that sample is the long pole in every option that involves a model.
- Assumption. Escape rates for every option, the current manual one included, are taken from the vendor trial's per-class results on the same 4,000-part sample and reweighted to the production defect mix. The manual figure of 1.4 per cent is the incumbent inspectors' result in that trial. Customer returns over the last twelve months are consistent with it but understate the true rate, because a customer who does not return a part does not appear in them, so they serve only as a cross-check.
- Assumption. The defect mix remains as observed over the last two years, at approximately 70 per cent coarse classes and 30 per cent fine. A shift toward fine defects would weaken every option that relies on the model for those classes.
Evidence
- Research. Reviews of machine-vision defect detection, among them State of the Art in Defect Detection Based on Machine Vision, evaluate detectors on measures such as precision and recall rather than on overall accuracy alone. For an inspection step the reason is the class balance: almost every part is good, so a high accuracy figure says little. False positive rate and false negative rate are the measures that carry information, and they must be reported separately. The comparison below therefore uses escape rate and false reject rate as two criteria rather than a single accuracy figure.