humansand.ai

The 4-bitter Lesson

Introduction RL Training Simulator. The simulator models an asynchronous RL system where samplers continuously generate rollouts while the trainer updates the policy. Policy mismatch arises from both off-policyness...