Research

Post-Training

2 reports

Surveys

On-Policy Self-Distillation for Large Language Models: A Survey

A survey of 478 papers on on-policy self-distillation, where a language model learns from itself given privileged context. It covers what the teacher sees, signal shaping, teacher dynamics, coupling with RL, the debates over when OPSD helps and what privileged information adds, and applications from agents to diffusion models.

Surveys

Test-Time Training: A Survey of Learning at Inference, from Self-Supervision to Memory Layers, LLMs and Agents

Test-time training lets a model keep learning after deployment by optimizing an explicit objective on the test input, context, problem or stream. This survey of 605 papers organizes the field by what is learned at test time and from which signal — self-supervised TTT, TTT layers and test-time memory, LLM test-time training, test-time RL, and agents and robots — writes them in one notation, and weighs five open debates.