Goal-Conditioned Rl

Goal-Conditioned RL is a reinforcement learning framework where the policy takes both a state and a goal as input — $pi(a|s,g)$ learns to reach any specified goal $g$, enabling a single policy to accomplish many different tasks by conditioning on different goals.

Goal-Conditioned Components

• Universal Policy: $pi(a|s,g)$ — one policy handles all goals by conditioning on the goal.
• Goal Space: Goals can be target states, images, language descriptions, or abstract representations.
• Reward: Typically sparse — $r = -\mathbf{1}[|s - g| > \epsilon]$ — reward only when the goal is reached.
• HER: Hindsight Experience Replay is essential — relabel failed trajectories with achieved goals.

Why It Matters

• Generalization: One policy covers an entire space of goals — no need to retrain for each task.
• Composability: Goals can be composed sequentially for complex, multi-step tasks.
• Robotics: Goal-conditioned policies enable flexible robotic manipulation — reach any target position.

Goal-Conditioned RL is one policy, any goal — training a single universal policy to reach any specified goal through conditioning.