Inverse Reinforcement Learning

IRL (Inverse Reinforcement Learning) is the problem of recovering the reward function from expert demonstrations — given an expert's behavior, IRL solves for the reward function that makes the expert's policy optimal, then uses this reward to train a new policy via standard RL.

IRL Methods

• MaxEnt IRL: Find the reward function under which the expert's behavior is maximum entropy optimal.
• Feature Matching: Find a reward such that the learned policy's expected features match the expert's.
• Bayesian IRL: Posterior over reward functions — captures reward uncertainty.
• Deep IRL: Parameterize the reward function with a neural network — scales to high-dimensional spaces.

Why It Matters

• Transferable: The recovered reward function transfers to new environments — more general than a copied policy.
• Understanding: The reward reveals WHAT the expert is optimizing — interpretable understanding of expert behavior.
• Ambiguity: Many reward functions can explain the same behavior — IRL is inherently ill-posed.

IRL is inferring WHY the expert acts — recovering the hidden reward function from observed expert behavior.