Iqn

IQN (Implicit Quantile Network) is a distributional RL algorithm that can sample any quantile of the return distribution — instead of learning a fixed set of quantiles (like QR-DQN), IQN takes a quantile level $\tau \in [0,1]$ as input and outputs the corresponding quantile value.

IQN Architecture

• Input: State $s$ + sampled quantile level $\tau \sim U(0,1)$.
• Quantile Embedding: Embed $\tau$ using cosine features: $\phi(\tau)_j = \text{ReLU}(\sum_i \cos(pi i \tau) w_{ij})$.
• Combination: Hadamard product of state features and quantile embedding.
• Output: The return value at quantile $\tau$ for each action — $Z_\tau(s,a)$.

Why It Matters

• Arbitrary Quantiles: Can evaluate any quantile at inference — not limited to pre-defined quantile levels.
• Risk Policies: Optimize for any risk level — CVaR, worst-case, or custom risk measures.
• State-of-Art: IQN outperforms both C51 and QR-DQN on Atari benchmarks.

IQN is the universal quantile machine — computing any quantile of the return distribution on-demand for flexible risk-sensitive RL.