Soft Actor-Critic

SAC (Soft Actor-Critic) is a state-of-the-art off-policy reinforcement learning algorithm for continuous action spaces — based on maximum entropy RL, SAC simultaneously maximizes expected reward and policy entropy, achieving sample-efficient, stable learning with automatic temperature tuning.

SAC Components

• Actor: Policy network $pi_\theta(a|s)$ outputs a Gaussian distribution over continuous actions.
• Twin Critics: Two Q-networks $Q_{\phi_1}, Q_{\phi_2}$ — use the minimum to reduce overestimation bias.
• Entropy Term: Loss includes $-alpha H(pi)$ — temperature $alpha$ is automatically tuned.
• Off-Policy: Stores transitions in a replay buffer — high sample efficiency.

Why It Matters

• Sample Efficient: Off-policy + replay buffer makes SAC one of the most sample-efficient model-free RL algorithms.
• Stable: Entropy regularization + twin critics prevent training instability common in actor-critic methods.
• Continuous Control: State-of-art for robotics, process control, and continuous optimization tasks.

SAC is the stable explorer — combining maximum entropy RL with twin critics for robust, sample-efficient continuous control.