Multi-Goal Rl

Multi-Goal RL is a reinforcement learning paradigm where the agent must learn to achieve multiple different goals — training a single policy $pi(a|s,g)$ that can accomplish any goal from a goal space, rather than training separate policies for each goal.

Multi-Goal Approaches

• Goal-Conditioned Policy: Policy takes goal as input — $pi(a|s,g)$ outputs actions conditioned on the current goal.
• UVFA: Universal value function $Q(s,a,g)$ estimates value for any state-action-goal triple.
• HER: Hindsight Experience Replay — relabel failed trajectories with achieved goals for dense learning signal.
• Curriculum: Automatically generate goals of increasing difficulty — adaptive goal curriculum.

Why It Matters

• Generalization: One agent handles a distribution of tasks — far more practical than single-task agents.
• Sample Efficiency: Sharing experience across goals massively improves sample efficiency.
• Robotics: A robot that can reach any position, grasp any object — multi-goal is the natural formulation.

Multi-Goal RL is one agent, many objectives — training a versatile agent that accomplishes any goal from a continuous goal space.