1 min lesson
Self-summarization: training past the context limit
Connect "Training past the context limit" to the decision you would make.
Step 1 of 2
Self-summarization: training past the context limit
A rollout that reaches 200K tokens will eventually hit the model's context limit mid-task. Rather than truncate and lose the thread, the model is trained to summarize its own progress and continue from the summary, so a single episode can run longer than one context window holds.
Long-horizon coding tasks generate more history than any context window can hold - hundreds of tool calls, file reads, test logs and prior edits. Without compaction the episode dies at the context wall before the task is done, so the agent could never learn behaviors that take more than one window to complete. Training the model to write and trust its own summary lets the rollout shed stale detail, keep the load-bearing state and push through tasks that genuinely require more than a single context window - which is most real engineering work.