Reinforcement Learning End Term: 10 May 2026, Set 1 (January 2026 term)

State space, action space, reward function

Initiation set, option policy, termination function

Value function, behaviour policy, discount factor

Transition model, reward model, termination condition

To prevent the actor from updating faster than the critic

To stabilise training by preventing the TD target from changing too rapidly

To ensure the replay buffer contains on-policy data

To match the learning rate of the actor and critic networks

The policy gradient estimate becomes unbiased

The algorithm becomes on-policy

Temporal correlations between consecutive samples are broken

The critic no longer requires a target network