Random Network Distillation

RND (Random Network Distillation) is an exploration bonus method that detects novelty by measuring how well a predictor network can match a fixed random network's output — novel states produce high prediction error (the predictor hasn't been trained on similar states), providing an exploration bonus.

How RND Works

• Fixed Target: A randomly initialized and frozen network $f_{target}(s)$ — maps states to random embeddings.
• Predictor: A trained network $f_{predict}(s)$ — tries to match the fixed target's output.
• Novelty: $r_i = |f_{predict}(s) - f_{target}(s)|^2$ — high error = novel state (predictor not trained on similar states).
• Training: The predictor is trained on visited states — its error naturally decreases for familiar states.

Why It Matters

• No Stochasticity Problem: Unlike curiosity (ICM), RND is not confused by stochastic environments — the target is deterministic.
• Simple: Just two networks and an MSE loss — extremely simple to implement.
• Montezuma's Revenge: RND achieved breakthrough performance on Montezuma's Revenge — a notoriously hard-exploration Atari game.

RND is novelty through random targets — detecting unfamiliar states by measuring prediction error against a fixed random network.