Resolution, LSB, code, reading scaling and DAC output voltage
Quantisation staircase zoomed around the input voltage (8 LSB). The dashed line is the perfect conversion.
Example loaded: 4–20 mA, 0–10 bar pressure transmitter read across a 150 Ω resistor (0.6 to 3.0 V).
An analog-to-digital converter compares the input voltage with a reference voltage and expresses it as an integer between 0 and 2n − 1, where n is the number of bits of resolution. The voltage interval corresponding to each code is the LSB (least significant bit):
A 10-bit ADC at 5 V resolves 4.9 mV steps; a 12-bit one at 3.3 V, 0.81 mV steps. All voltages within the same step give the same code: this is the quantisation error, of up to 1 LSB (or ±½ LSB if the ADC centres the steps, as many do). With random noise, that error limits the ideal signal-to-noise ratio to 6.02·n + 1.76 dB.
Some manufacturers divide by 2n − 1 instead of 2n. The difference is less than 1 LSB and only matters for the last digit; what really matters is an accurate, stable reference, because a 1 % error in Vref is a 1 % error in the whole measurement.
Nominal resolution is not accuracy. Noise, non-linearity and reference error mean that a microcontroller's 12-bit ADC usually gives 9 to 11 effective bits (ENOB). Some guidelines:
Almost all industrial sensors are linear: two points (minimum and maximum voltage with their physical values) are enough to get a straight line value = a·code + b. In the example, a 4–20 mA signal across 150 Ω gives 0.6 to 3.0 V, which at 12 bits is about 2980 usable codes: the pressure resolution is about 3.4 thousandths of a bar. Using only part of the ADC range wastes resolution, so choose the resistor or divider to get close to the reference voltage without exceeding it. For resistive temperature sensors, see the Pt100 calculator.
The DAC does the opposite: it converts a code into a voltage Vout = code · Vref / 2n. The most common structure is the R-2R ladder, which needs only two resistor values: each bit switches a branch between the reference and ground, and the network adds the contributions with weights 1/2, 1/4, 1/8… Many microcontrollers include one (STM32, 8-bit on the ESP32); otherwise you can use an external I²C or SPI DAC (MCP4725, MCP4921) or a PWM with an RC filter, cheaper but slower and with ripple.
| Resolution | Levels | LSB at 3.3 V | LSB at 5 V | Typical use |
|---|---|---|---|---|
| 8 bits | 256 | 12.9 mV | 19.5 mV | Simple audio, rough levels |
| 10 bits | 1024 | 3.22 mV | 4.88 mV | Potentiometers, basic sensors (Arduino Uno) |
| 12 bits | 4096 | 0.806 mV | 1.22 mV | 32-bit microcontrollers, 4–20 mA |
| 16 bits | 65,536 | 50.4 µV | 76.3 µV | Instrumentation, strain gauges with amplifier |
| 24 bits | 16,777,216 | 0.197 µV | 0.298 µV | Load cells (HX711), thermocouples, sigma-delta |
Related: number system converter, PWM calculator, 4–20 mA loop and multiplexers (the 74HC4051 adds ADC inputs).