Digital Signal Processing
Digital signal processing is the numerical manipulation of sampled signals to filter, transform, detect, compress, modulate, or reconstruct information. Every layer of an AI chip's surrounding system — audio front-ends, radar, image sensors, SerDes PHYs, wireless modems — relies on DSP algorithms implemented in hardware MAC arrays, dedicated DSP cores, or programmable vector engines.
| Operation | Complexity | Hardware | Application |
|---|---|---|---|
| FIR filter (N taps) | O(N) MAC/sample | MAC array, SIMD | Anti-aliasing, channel EQ |
| IIR filter (N poles) | O(N) MAC/sample | Feedback MAC | Audio, control loops |
| DFT (N-point) | O(N²) | Rare directly | Baseline reference |
| FFT (N-point) | O(N log N) | cuFFT, HW FFT | OFDM, radar, spectrum |
| Convolution | O(N·M) or O(N log N) via FFT | Tensor core, DSP | FIR filtering, image |
Sampling theorem (Nyquist-Shannon) — a continuous signal must be sampled at a rate fs > 2·fmax to be reconstructed without aliasing. An anti-aliasing low-pass filter must remove all energy above fs/2 before the ADC. Violating this causes high-frequency content to fold back into the baseband as aliasing distortion, which is indistinguishable from legitimate signal. For voice (4 kHz bandwidth), fs = 8 kHz is exactly Nyquist; CD audio uses 44.1 kHz for 20 kHz bandwidth plus guard band.
FIR filters implement the convolution y[n] = Σ h[k]·x[n−k] using a tapped delay line: the input signal passes through N delay elements (z⁻¹), each tap is multiplied by a coefficient h[k], and the products are summed. FIR filters are unconditionally stable, have linear phase (constant group delay), and are fully specified by their coefficient vector — which is the sampled impulse response. The price is computational load: an N-tap FIR requires N multiplications and N−1 additions per output sample, which is why SIMD MAC arrays are universal in DSP hardware.
FFT and frequency-domain processing — the Discrete Fourier Transform (DFT) converts N time-domain samples to N complex frequency-domain coefficients. Direct computation costs O(N²); the Cooley-Tukey FFT exploits the DFT's periodicity and symmetry to reduce this to O(N log₂N) using a butterfly network of complex additions and twiddle-factor multiplications. An 8-point FFT requires 3 stages of 4 butterflies; a 4096-point FFT requires 12 stages of 2048 butterflies. OFDM modems (5G, Wi-Fi 6E, DOCSIS) implement 4096-point FFTs as hard IP blocks, processing one symbol per FFT latency.
AI connections — long-range attention in transformer models can be viewed as a form of learned convolution in the sequence domain. Monarch Mixer and other sub-quadratic attention proposals leverage FFT-based convolution (O(N log N)) to replace O(N²) attention. In a more direct sense, every GPU runs cuFFT for spectral analysis workloads, and AI inference chips for edge devices typically include a DSP subsystem for pre-processing sensor data — beamforming, voice activity detection, image ISP — before the neural-network accelerator core.
Fixed-point and quantization — embedded DSP systems represent samples in Q-format fixed-point (e.g., Q1.15 for 16-bit signed) to save area and power versus floating-point. The SNR of a uniform quantizer is approximately 6N + 1.76 dB for N bits, setting ADC resolution requirements: 12-bit gives ~74 dB, 16-bit gives ~98 dB. The same principle applies to AI inference: INT8 quantization trades 24 dB of numeric headroom for 4× throughput and 4× memory bandwidth reduction.
Read DSP through a frequency-domain decomposition lens rather than a time-domain operations lens: almost every DSP algorithm is most clearly understood in terms of which frequency components it preserves, attenuates, or shifts — the filter, the FFT, and the sampling theorem are all fundamentally statements about the frequency axis.