Resolution

Resolution in metrology is the smallest change in a measured quantity that a measurement instrument can detect — the fundamental capability limit that determines whether a semiconductor metrology tool can distinguish between parts that are within specification and those that are out of specification.

What Is Resolution?

  • Definition: The smallest increment of change in the measured value that the instrument can meaningfully detect and display — also called discrimination or readability.
  • Rule of Thumb: Resolution should be at least 1/10 of the specification tolerance — a gauge measuring to 1nm resolution is needed for ±5nm tolerances (10:1 rule).
  • Distinction: Resolution is the instrument's detectability limit; precision is how consistently it reads; accuracy is how close to truth it reads.

Why Resolution Matters

  • Specification Discrimination: If the specification tolerance is ±2nm and the gauge resolution is 1nm, the gauge can only distinguish 4 discrete levels within the tolerance — inadequate for process control.
  • SPC Sensitivity: Insufficient resolution causes "digital" control charts with stacked identical readings — obscuring real process trends and shifts.
  • Gauge R&R: The AIAG MSA manual requires the number of distinct categories (ndc) ≥ 5, which requires adequate resolution relative to part-to-part variation.
  • Process Optimization: Fine-resolution measurements enable detection of small process improvements — critical for continuous improvement at advanced nodes.

Resolution in Semiconductor Metrology

InstrumentTypical ResolutionApplication
CD-SEM0.1-0.5nmCritical dimension measurement
Scatterometer (OCD)0.01nmFilm thickness, CD profiles
Ellipsometer0.01nmThin film thickness
AFM0.1nm (Z), 1nm (XY)Surface topography
Wafer prober0.1mV, 1fAElectrical parameters
Overlay tool0.05nmLayer alignment

Resolution vs. Other Metrology Properties

  • Resolution: Can the gauge detect a change? (smallest detectable increment)
  • Precision: Does the gauge give consistent readings? (repeatability)
  • Accuracy: Does the gauge give the right answer? (closeness to true value)
  • Range: What span of values can the gauge measure? (minimum to maximum)
  • All four properties must be adequate for a measurement system to be capable.

Resolution is the first capability checkpoint for any semiconductor metrology tool — if the instrument cannot detect changes smaller than the process tolerance, no amount of calibration or averaging can make it capable of supporting reliable process control decisions.

Take resolution further

Ask the copilot about this term, or have our engineers assess it against your process.