Actor-Critic Methods
Policy Gradient Methods needed a full Monte Carlo return to compute a low-noise gradient — waiting for episodes to finish, and paying for it in variance. Actor-critic methods replace that Monte Carlo estimate with a second, learned network that predicts value directly, combining the direct optimisation of policy gradients with the low variance of TD learning.