Surveys
Looped and Recurrent-Depth Transformers: Architectures, Training, Adaptive Inference, Theory and Systems
A looped (or recurrent-depth) transformer applies the same weight-tied block repeatedly to a token's hidden state, turning "depth" from a fixed property of the parameter count into a variable that can be spent at inference time.