Abstract

A looped (or recurrent-depth) transformer applies the same weight-tied block repeatedly to a token's hidden state, turning "depth" from a fixed property of the parameter count into a variable that can be spent at inference time. Since 2024 this idea has moved from theory constructions and small sequence-model experiments to pretrained language models in the billions of parameters, and to vision, diffusion, flow-matching and reinforcement-learning backbones. This survey maps that literature along four questions that turn out to be largely independent: where the loop sits in a network, how successive iterations are distinguished from one another, what state is carried between iterations, and who decides how many iterations a token receives. We show that the field's central design choices are best described as a small number of discrete options on those four axes, that training stability — not expressive power — has been the binding constraint, and that the empirical case for looping is still made mostly against parameter-matched rather than compute-matched baselines. We review adaptive-depth inference, where loops are now increasingly used as a resource to be scheduled rather than a fixed budget; the theory that establishes what looping can simulate and how far it generalises in depth; mechanistic evidence about what individual iterations compute; and the systems, safety and evaluation questions that the architecture raises. We close with the open problems we consider most consequential, including the shortage of compute-matched controls and the absence of any agreed inference-time scaling law for looped models.

---

Scope and evidence boundary

This survey is a narrative review of one architectural family, not a systematic review of latent reasoning as a whole. Its evidence base is a candidate pool of 358 papers retrieved from the Parallax library by a union of six retrieval routes (semantic recall plus keyword variants across the vocabulary of looping, recursion, weight sharing, universal and recurrent-depth transformers), date- and category-filtered to the window in which this literature exists. From that pool, 102 papers were read in full (per-paper structured reading notes with section-level provenance); the remainder were screened at title/abstract level.

Three consequences should be kept in mind throughout:

• Coverage is bounded by the library. Three of the field's foundational works — Universal Transformers [127], Adaptive Computation Time [128] and Deep Equilibrium Models [129] — are absent from the library and are therefore cited from their public arXiv/publisher metadata. They are marked as external in the references.
• Reading depth is uneven. 28 of the 102 dispatched papers had no full-text reading surface and were processed from abstract plus reading notes; they never stand alone behind a load-bearing claim.
• The field moves fast and is preprint-dominated. A large share of what is cited here is unreviewed and dated 2026. Where a claim rests on a single small-scale preprint we say so. Where several independent works agree, we say that too, and distinguish it from consensus.

A companion reading-pool artifact (the screened corpus of 358 candidates, with the AI-screening decisions that produced it) was used to assemble the evidence base; it is a working set, not a claim that these are all relevant papers.

Takeaways. The survey covers one architectural family and says so: 358 screened candidates, 102 full-text reads, 28 of those abstract-only, and three foundational works cited from external metadata. Load-bearing claims rest on works whose sections were actually read; the abstract-only and metadata-only works appear only where a grouped sentence is supported by their stated aims.

---

Part I — Foundations

1. Why loop the depth axis?

1.1 The problem the family addresses

A conventional transformer buys capability along two axes at once: parameters and per-token compute both grow with the number of stacked blocks. Looping separates them. A looped model has a small number of physical layers $L$ and an iteration count $T$ that is applied to the same weights:

$$h^{(t+1)} = B_{\theta}\big(h^{(t)}, x, t\big), \qquad t = 0, \dots, T-1 ,$$

where $x$ is the token/context the loop is conditioned on and $B_\theta$ is one shared block. Increasing $T$ increases effective depth without increasing parameter count; the same weights can then be run for more or fewer iterations per input, per token, or per generation step.

The idea is not new. Universal Transformers [127] proposed a weight-tied, adaptive-depth transformer in 2018, and Adaptive Computation Time [128] supplied the halting mechanism; the multi-step recurrent reasoning view was already present in work on recurrent networks with pondering. What changed is scale and intent. Between 2024 and 2026 the family acquired: billion-parameter pretrained instances (Huginn, Ouro) [1, 2]; a mixture-of-experts form that makes looping cheaper than parameter growth [7, 8]; a set of training recipes that actually converge at scale [42, 50, 103]; and an inference-time story in which the loop count becomes a controllable compute knob [64, 74].

1.2 Two things looping is used for

It is worth separating two purposes that are often conflated.

Capability. Looping can increase expressive power per parameter. The theoretical results in §8.1 show that a looped transformer of constant depth can simulate a small programmable computer [76] or execute graph algorithms [87] — a fixed-depth feed-forward transformer of the same size cannot. Empirically, a looped 1-layer model can match a 12-layer one on in-context learning tasks at one twelfth of the parameter count [78], and looped models trained on short sequences can extrapolate to inputs far beyond their training length where a standard transformer trained under the same protocol fails [77].

Controllability. Looping also makes token-level compute an explicit variable. Because iterations are indexed, a model can decide to run more of them for a hard token [64], to run a shorter trajectory when a cheaper answer will do [63, 66], or to reuse early iterations as a draft for speculative decoding [74] — mechanisms that have no analogue in a fixed-depth stack.

The two purposes have different evidence and different failure modes, and §11 argues that much of the apparent disagreement in the literature comes from not keeping them apart.

1.3 What this survey argues

Four claims organise the material that follows.

1. The design space is small and discrete. Despite the volume of recent work, almost every looped architecture is a choice of four things: the scope of the shared block (which part of the network repeats), the step conditioning (how iteration $t$ is signalled), the inter-iteration interface (what state is passed, replaced or accumulated), and the depth policy (whether $T$ is fixed, learned, or decided at inference). §3–§4 organise the literature this way.
2. Stability, not expressivity, is the binding constraint. The theoretically attractive regime — a contractive fixed point — is not what large models converge to, and the papers reporting the largest gains are largely papers reporting a stabilising trick [42, 43, 44, 45, 46, 47]. §5.2 is the core of the survey's discussion of training.
3. The evidence is mostly parameter-matched, not compute-matched. Looping's headline comparisons are against dense models with a similar number of parameters, or against the same model at $T=1$, rather than against a dense model of the same inference FLOPs. The minority of works that do match inference FLOPs tend to report a different and more mixed conclusion [50, 63, 64, 101, 102]. §11 tabulates what we have.
4. Adaptive depth at inference is where the field is most active and least validated. Almost all of §6's evidence comes from a small number of models, short single-seed runs, and synthetic or narrow benchmarks; the adaptivity gains reported are real but fragile, and disappear under some matched-cost comparisons.

1.4 Reading map

Figure 1 shows how the components of the survey relate. Part I fixes vocabulary and taxonomy. Part II is the architecture and training design space, which is where most of the field's engineering sits. Part III covers inference-time computation and the systems/safety constraints it creates. Part IV covers theory and mechanistic interpretability. Part V collects the compute-matched evidence, the applications outside text, and the open problems.

Figure 1: Map of the survey. Solid arrows are the design axes treated in Part II; the branches under "Depth policy" and "Training stability" are the two places where the field's practical difficulties concentrate.

Takeaways. Looping trades parameter count for a repeatable computation. It is used both to buy capability per parameter and to expose per-token compute as a controllable knob, and the two uses have different evidence. This survey treats four design axes — scope, step conditioning, interface, depth policy — as the organising structure, and returns repeatedly to a single evidentiary weakness: looping is usually evaluated against parameter-matched rather than compute-matched baselines.

2. One formalism for the loop

2.1 Definitions

Throughout, a looped transformer is a sequence model in which a parameter-tied sub-network $B_\theta$ is applied more than once to the same token position's representation along an iteration index, and in which the number of applications is a first-class quantity (fixed, learned, or scheduled). We use the following vocabulary consistently.

Table 1: Notation used throughout the survey. Effective depth and per-token FLOPs are proportional inside one architecture but not across architectures.

Three further distinctions matter and are used consistently below.

• Effective depth vs compute. Because one iteration of a $b$-block loop costs one forward pass through $b$ blocks, effective depth and per-token FLOPs are proportional within a single architecture, but not across architectures. A 24-layer stack whose middle twelve layers repeat, and a 24-iteration single-block loop, can have the same FLOPs and very different effective depths. Statements about "depth" and about "compute" must therefore be read from the architecture, not from the label.
• Loop scope. The shared region may cover the whole network ([3, 12, 14]), a middle span between unshared prelude and coda ([1, 2]), or a single layer or narrow set of layers ([26, 28]). We call these full, two-stage and localised scoping respectively.
• Semantic depth vs iteration index. Some authors report that the last iterations do not correspond to additional reasoning steps; the mapping from $t$ to "amount of reasoning" is an empirical question examined in §9.2, not an assumption.

2.2 The boundaries of the family

Four neighbouring lines of work are close enough that they are frequently confused with looped transformers. This survey treats all four as related directions and does not make them parallel subjects; each is included only where it directly informs a claim about looping.

Table 2: Adjacent families and why they are not this survey's object. Each is included only where it directly informs a claim about looping.

The last row deserves emphasis because two of the corpus's most-cited "recurrent" results are not about depth at all. The distinction is easy to lose in the vocabulary, since "recurrent" and "recursive" appear on both sides.

2.3 Position against existing surveys

The library contains no dedicated survey of looped / recurrent-depth transformers. Two recent surveys of latent reasoning are the nearest neighbours, and the relationship is one of partition rather than overlap.

Table 3: The two nearest surveys in the library and how this survey differs. Both are organised by latent-reasoning method rather than by architecture.

The differentiation this survey claims is therefore on architecture, training stability, inference-time scheduling, efficiency/systems and cross-modal use of the loop axis, with latent-reasoning method taxonomies treated as background. We make no claim to be systematic or exhaustive in the sense of a review protocol; the coverage statement in "Scope and evidence boundary" is the operational description.

2.4 How the family got here

Figure 2 places the works this survey relies on in time. The pattern is worth noting before the details: theory and constructive results dominate the early period, the first billion-parameter language models appear in 2025, and the 2026 literature splits into stability recipes, inference-time scheduling, systematic scaling laws and cross-modal transfers. The density of the most recent period is a property of the retrieval window, not evidence of a research frontier forming in a particular month.

Figure 2: Selective timeline. Only works used in this survey are shown; the 2026 cluster reflects when the retrieval routes were run, not a measured growth rate.

Takeaways. The family is defined by weight-tied reuse along an iteration axis, not by the word "recurrent": sequence-axis recurrence, fixed-depth early exit and implicit-equilibrium solving are distinct objects. Effective depth and per-token FLOPs are proportional inside one architecture but not across architectures, which is the source of much of the literature's apparent disagreement. No dedicated survey of this family exists in the library; the two nearest surveys partition the space by latent-reasoning method rather than by architecture.

3. A taxonomy

3.1 The four axes

Everything in Part II is a choice on four axes. Figure 3 shows the taxonomy with representative works at the leaves.

Figure 3: Taxonomy of looped architectures along the four axes used throughout Part II. Leaves name the design option; representative works are given only where a paper is the clearest instance.

3.2 Why these four axes and not others

The axes are not arbitrary. They correspond to the four questions a practitioner must answer to instantiate a looped model, and crucially they are close to orthogonal in the literature: the same scope appears with fixed and adaptive depth; the same interface appears with and without step conditioning.

A crude count over the works read for this survey illustrates the point. Papers that introduce a scope innovation (where the loop sits) almost never also introduce a depth-policy innovation, and papers about stability almost always keep scope and depth policy fixed. The two exceptions are the mixed architectures that pair MoE layers with a loop count (looping beyond twice [33], MoRE [32], MoR [30]) — there, scope and routing genuinely interact, because the routing decision is the depth decision.

3.3 A fifth, cross-cutting axis: what the iteration is for

There is a further distinction that cuts across the four axes and is not architecture at all: whether the intended role of additional iterations is iterative refinement (approaching a fixed point), serial computation (each iteration executes a step of an algorithm), or sampling (each iteration improves a generative distribution). These correspond to three different theoretical stories and three different prediction sets for what more loops should buy. §8.3 and §11.2 return to this, because several of the survey's own disagreements dissolve once the intended role is made explicit.

Takeaways. The design space reduces to four largely independent choices — scope, step conditioning, interface, depth policy. Most papers vary exactly one. A fifth, non-architectural axis — whether iterations are intended as refinement, serial computation, or sampling — predicts different behaviour under extra loops and is the most useful single question to ask of a new looped architecture.

Part II — The design space

4. Architecture: where the loop sits and how iterations differ

4.1 Scope: two-stage, full and localised loops

The most consequential and least discussed design choice is which part of the network repeats.

Two-stage scoping. The dominant pattern in large language models separates the network into an unshared prelude, a repeated core, and an unshared coda. Huginn [1] is the clearest instance: one prelude block, four shared core blocks, one coda block, with the full state re-entering the core each iteration. The motivation is that the prelude amortises the embedding/early-mixing work that should not be repeated, while the coda converts the converged latent state into logits. Ouro [2] and the Latent Recurrent Transformer [51] follow the same partition. The choice is not innocent: because the state is replaced wholesale at the boundary, the prelude/coda split is also what makes the loop's fixed-point behaviour analysable, and its cost is that all iterations share a single embedding of the input.

Figure 4: The two-stage (prelude–core–coda) pattern, as introduced by Huginn [1]. The entire state re-enters the core on each iteration; no per-iteration noise or halting signal is used. Figure reproduced from [1].

Full-network scoping. At the other extreme, every layer participates in the loop, and there is no privileged unshared region. This is the setting of the theoretical works on looped depth [3, 77, 78] and of several recent systems papers that argue the prelude/coda split is itself a source of instability [14, 43, 12]. The Fully Looped Transformer [43] makes the argument explicitly: letting only a subset of layers repeat leaves their residual statistics growing without bound, and folding the whole stack into the loop is what stabilises it. §5.2 takes this up.

Localised scoping. A third option repeats only a few layers in the middle of a deep stack. Depth-Attention [26] is the sharpest instance: it keeps a conventional $L$-layer transformer but replaces each layer's value matrix with a depth-mixed value formed from a sparse set of shallower layers using the current layer's queries. The complexity argument is stated in the paper as $O(TL/s) \ll O(T^2)$ — the mixing is sparse over depth, cheap relative to token-axis attention.

Figure 5: Depth-Attention [26] as a localised, sparse form of cross-iteration mixing. The mixing is over depth rather than over sequence positions. Figure reproduced from [26].

Localised loops are attractive for a specific reason: they can be retrofitted to an existing pretrained stack, and they avoid the wholesale state-replacement question entirely. T²MLR [27] takes the same "middle-layer recurrence" route, and Penelope [28] localises recurrence in order to contain its cost. The category is also where the family blends into ordinary architectural work — at what point does "reuse one layer twice" stop being a looped transformer? We adopt the operational boundary of §2.1: reuse is a loop when the iteration count is a controllable quantity, not merely a repeated sub-expression.

4.2 Step conditioning: telling iteration $t$ from iteration $t-1$

A shared block applied repeatedly is an equivariant map: without extra signal, iteration $t$ and iteration $t'$ are indistinguishable to $B_\theta$. Every nontrivial looped architecture therefore adds conditioning. Four mechanisms recur.

Timestep or positional encoding over the depth axis. The theoretical work of [17] shows that looped transformers without timestep encoding have limited expressive power in a precise sense and that adding it strictly increases it; LayerRoPE [19] generalises depth-wise position encoding to a magnitude-and-angle superposition, and LayerNorm is argued in [18] to act as an implicit gain control that plays a similar role for the residual path. This is the cheapest and most common option, and it is also the one most often omitted by accident — Huginn [1] uses none, which is a design position, not an oversight.

Per-loop low-rank adapters. Relaxed Recursive Transformers [9] fold a deep network into $K=L/B$ shared blocks plus a per-layer, per-loop LoRA delta (initialised by truncated SVD of the original layer), and then uptrain. The result is a continuum: at one end, full weight sharing; at the other, distinct layers. Mixture-of-LoRAs [11] turns the per-loop adapters into a routed set. The advantage is parameter efficiency — a few million extra parameters restore most of the loss from tying — and the cost is that "how much sharing" becomes a new hyperparameter that must be tuned.

Depth-structured attention. Rather than conditioning the block's parameters, cross-iteration attention conditions its information flow: ReFlux [25] writes the update back to the same tokens across iterations; Loop-Native Attention Residuals [24] push this to tens of thousands of test-time iterations by making the residual readout itself loop-aware. Depth-Attention [26] (Figure 5) is the localised version.

Figure 6: Depth-attention source weights learned by the 3B model of [26], averaged over tokens, heads and evaluation examples. The diagonal is each layer reading its own value state; the strong horizontal line at source layer 24 and the faint row at source layer 0 are the only non-diagonal sources the trained model uses. Figure reproduced from [26].

Per-loop experts. MoRE [32] treats "which iteration" as a routing dimension and reuses expert pools across iterations; the looped-MoE works in §4.5 do the same at the block level. This is the option that most tightly couples conditioning to routing.

4.3 The interface: what crosses an iteration boundary

The second structural question is what the next iteration reads.

Whole-state replacement. The simplest interface passes the full updated state and nothing else — Huginn [1] and Ouro [2]. It admits a clean fixed-point interpretation (§5.1) and is compatible with the depth-generalisation theory of [77].

Residual or injected updates. Injection — re-adding the input embedding on each iteration — is standard in the length-generalisation literature [77] and is used in the "iterative-KV" formulation, where the loop is written as a fixed-point iteration on the KV cache rather than on the hidden state.

Memory as state. A distinct and increasingly popular option carries an explicit memory across iterations. MeSH [10] routes the shared block's state through memory highways; LSTM-UT [21] replaces the naive carry with a bounded gated memory and argues that storing state is not the same as remembering it; Associative-State Universal Transformers [22] pair recurrence with sparse retrieval; Maglev [118] uses a sliding recurrent memory. The strongest evidence in this group is [95], which shows that a shared KV memory read by every recursion improves the accuracy/memory trade-off (Figure 7) — the authors' own reading is that loops whose attention mass lands on the shared memory recover most of the gradient signal that plain loops lose at depth.

Figure 7: Accuracy versus memory for looped prediction transformers with (lpt) and without (h-lpt) shared memory [95]. The comparison is parameter-matched; the paper does not report a decode-FLOP-matched control. Figure reproduced from [95].

Figure 8: Attention mass falling on the shared memory in the shared-memory looped transformer of [95], shown by layer and recursion for two model sizes (S and M) and averaged over queries and heads; the bottom row of each panel is the all-layer mean. Figure reproduced from [95].

Loop-local versus persistent state. A recurring finding, and one with practical consequences, is that the scope of the state matters more than its size. LSTM-UT [21] and Source-Centered State Evolution [23] both report that unbounded state accumulation degrades; Universal Transformers Need Memory [20] frames the whole problem as a depth–state trade-off. We read this cluster as converging on a single empirical claim: beyond a point, extra iterations help only if there is somewhere for the result to go. That reading is our inference, not a stated result of any one paper.

4.4 Depth policy: fixed, learned, elastic, scheduled

Fixed. $T$ is a hyperparameter. This includes the subtly important case of $T$ chosen small: the Looping-Beyond-Twice recipe [33] argues that the field has been over-focused on $b=2$ (a one-block loop of depth two) and that larger $b$ with larger $T$ scales better.

Learned. A gate predicts halting, in the tradition of ACT [128]. Dynamic Halt [54] is the thorough early study of recurrent transformers with a halting head; Sparse UT [8] combines a sparse mixture-of-experts block with a halting module and trains it end-to-end. Learned halting is attractive but the corpus's diagnostics are unflattering: §6.2 reports that learned gates are systematically miscalibrated [55] and that adaptive depth in practice mostly acts as a finishing stage rather than as a runtime scheduler [53].

Elastic. Train once, run at any depth. LoopFormer [63] trains over a family of trajectories by conditioning each step on a normalised time $t$ and step size $\Delta t$ with a shortcut-consistency loss; One Model to Loop Them All [66] makes elastic loop count the primary goal. The appealing property is that a single checkpoint serves a compute/quality frontier.

Scheduled at inference. The loop count is decided per token or per request at run time by a router, a decider, or an external schedule — Think-at-Hard [64], Program-of-Layers [65], continuous-depth batching [57], and the self-speculation decoders of §7.3. This is where most 2026 activity sits and is treated separately in §6.

Figure 9: The mixture-of-recursions step of [30]: each recursion applies a fixed shared layer stack, and a router emits a scalar per token that decides whether that token continues to the next recursion or exits. Figure reproduced from [30].

4.5 Looped mixture-of-experts and segment loops

The single most economical idea in the design space is to make the repeated block a mixture of experts, so that looping buys capacity-of-weights and capacity-of-computation separately.

MoEUT [7] is the foundational instance and the cleanest experiment. It replaces the shared block's attention and MLP with sparse mixtures and, critically, introduces a peri-layernorm arrangement in which layer normalisation is applied both before and after the residual addition — the authors report that this is what makes deep weight-tied mixtures trainable. Their own ablation is the informative part: a non-shared σ-MoE is reported as "significantly worse", i.e. the gain comes from the interaction of sparsity with tying rather than from sparsity alone.

Figure 10: The peri-layernorm arrangement that the authors of MoEUT [7] identify as necessary for stable deep weight-tied mixtures. Figure reproduced from [7].

The 2026 literature then splits this into finer options. Mixture-of-Recursions [30] lets each token exit the recursion at a different iteration and pairs this with a shared KV cache; Looping Beyond Twice [33] studies the $(b,T)$ trade-off directly; MoRE [32] reuses expert pools across iterations; LoopMoE [31] unifies iterative computation with MoE in a single formulation. On the scaling side, SMELT [101] provides compute-matched scaling laws for MoE looped transformers — a rare case where the alternative (more parameters, fewer loops) is matched on FLOPs rather than on parameter count.

4.6 Looped generation: diffusion, flow and visual tokenisation

A distinct branch applies the loop to a generative denoising process, where the iteration index replaces or augments the diffusion timestep. Looped Diffusion Transformer [35] and LiFT [36] are the canonical looped-diffusion and flow-matching instances; Looped Diffusion Language Models [37] and Recursive Scaling in Masked Diffusion Models [38] apply the same idea to discrete diffusion over token positions; Fixed-Point Masked Generative Modeling (CoFRe) [40] formulates masked generation explicitly as a fixed-point iteration and adds consistency regularisation; ALoDLM [39] makes the iteration count adaptive; LoopREG [126] adds representation supervision so that recurrent depth transfers at test time.

This branch is architecturally interesting precisely because it inverts the usual story: in a denoising model, "more iterations" has an unambiguous meaning tied to a target distribution, so the loop count is interpretable in a way it is not in an autoregressive language model. It is also the branch where the cross-iteration interface literature (publication-free reuse, consistency losses, reuse ablations) is most developed.

4.7 Comparative summary of architectures

Table 4 compares the architectures whose components this survey read in full, on the four axes of §3.1. Rows are ordered by scope. The "Evidence scale" column records the largest setting the paper reports, which is often much smaller than the setting the claims are stated for.

Table 4: Architecture comparison on the four taxonomy axes. "Evidence scale" is the largest setting reported; it is not an endorsement that the claims hold at that scale.

Takeaways. Scope is the first decision and the one with the largest downstream consequences: two-stage scoping gives analysable fixed points but shares one input embedding across iterations; full-network scoping avoids residual-statistic growth but abandons the prelude/coda separation; localised scoping retrofits onto existing stacks and blends into ordinary architecture work. Step conditioning is often omitted by design (Huginn, Ouro) and added deliberately elsewhere; timestep encoding is theoretically justified [17] but empirically under-tested at scale. The interface question — replacement, injection, or memory — is where the most consistent empirical finding appears: extra iterations help mainly when there is somewhere for their result to go [20, 21, 95]. Looped MoE inverts the usual trade-off by letting weights and computation grow separately, and its central ablation is that the gain comes from the interaction of sparsity with tying, not from sparsity alone [7].

5. Training: stability is the binding constraint

5.1 The stability problem, stated plainly

Unrolling a weight-tied block $T$ times is a deep computation with a shared Jacobian. Three pathologies follow, and they are different from the ones a normal deep network faces.

1. Residual growth or collapse. If the loop has no mechanism to regulate the state's scale, its norm drifts monotonically with $t$; at large $T$ the state either explodes or saturates and stops carrying information.
2. Gradient starvation at depth. The gradient of the loss with respect to early iterations decays with the number of intervening applications; if the decay is fast, iterations beyond a certain point receive no learning signal and become inert.
3. Fixed points that are not the ones you want. If the loop does converge, it converges to some attractor; nothing in the objective says the attractor should be a useful answer, and the basin structure can be adversarial.

Figure 11: The three failure modes of an unrolled shared block and the intervention families that address them, as reported across the training literature reviewed in this chapter. Drawn by the author from the mechanisms described in [41, 42, 43, 44, 45, 47, 61]; not a figure from any single paper.

The theoretically attractive resolution is to make the loop contractive and train at the fixed point with implicit differentiation, following the deep-equilibrium tradition [129]. The corpus's evidence is that this is not what large looped language models do. [15] replaces layer stacking with a deliberately contractive recurrent depth and reports gains in the small regime; [47] frames language and reasoning loops as attractor models and reports that the fixpoint formulation is what repairs stability; [48] builds "fixed-point reasoners" with an explicit stability constraint; [49] re-examines what to do at the fixed point (notably, which endpoint of the trajectory should stand in for it). But of the works reporting results at hundreds of millions to billions of parameters, most keep the unrolled objective and stabilise the architecture instead.

5.2 What the stabilising interventions actually are

Grouping the interventions by where they act makes the literature much easier to read.

Residual-path scaling. [42] studies residual scaling exponents for looped transformers directly and reports both stability and transferability benefits; DeepLoop [41] sweeps the residual-scaling exponent on GPT-2 small/medium backbones and reports that matching a scaled baseline is what "depth scaling" for loops requires. This is the most common single intervention.

Normalisation placement. MoEUT's peri-layernorm [7] is the prototype: normalising on both sides of the residual addition. [18] gives the complementary analysis, arguing that LayerNorm functions as implicit gain control precisely in the looped setting. Rotation-based normalisation [43] and blockwise conformality [44] are the more aggressive variants: instead of controlling scale, they constrain the geometry of the transition, with [44] reporting that the free diagonal of the transition operator is what collapses and that a blockwise conformal constraint restores it.

Figure 12: Where the stabilising interventions of this section act on the unrolled loop, grouped by the part of the computation they modify. Drawn by the author from the interventions described in [7, 18, 41, 42, 43, 44, 45, 61].

Bounded, gated state updates. [61] runs a controlled ablation study over recursive-reasoning models — varying one factor at a time within a single unified pipeline — and concludes that explicit hierarchy is unnecessary, and that stability comes from bounded, gated, normalised recurrent state updates combined with an intermediate gradient horizon and a large physical batch. The result is worth stating in the authors' terms because it is a negative claim about a popular design: the hierarchical (two-level) structure of HRM-style models is, in their experiments, not the source of the gain.

Stochasticity. [45] learns a stochastic stopping distribution so that the extrapolation behaviour at test time is stable; [59] analyses the slow oscillatory modes of recurrent reasoners and argues that rate-limiting them is what makes looping finish earlier than expected.

Two caveats apply to this whole cluster. First, most of these papers report a diagnosis plus a fix on a small or medium backbone, generally with a single seed; replicated evidence that a specific intervention scales is sparse. Second, the interventions are not independent — a residual-scaling exponent is meaningless without knowing the normalisation placement — so the "which trick works" question may not have a well-posed answer across architectures.

5.3 Seeing the failure: a worked diagnostic

The clearest single piece of evidence in the training literature is [43]'s diagnosis, reproduced in Figure 13. Comparing a fully looped transformer with conventional looped transformers at equal and larger per-loop depth over the first 2,000 optimisation steps, the paper reports three simultaneous divergences: the conventional loop's residual norm grows monotonically to tens of thousands while the fully looped model's stays near a thousand; the conventional loop's training loss plateaus around 7.9 while the fully looped one descends to ≈3.6; and the conventional loop's gradient norm collapses to near zero while the fully looped one retains a non-trivial gradient.

Figure 13: Training diagnostics from [43], contrasting a fully looped transformer (FLT) with conventional looped transformers (LT) of equal and larger per-loop depth. Residual norm (left), training loss (centre) and gradient norm (right). Figure reproduced from [43].

We read this figure as showing the three pathologies of §5.1 occurring together in one run: uncontrolled residual growth, a loss floor, and gradient starvation. The paper's own conclusion is that folding the entire stack into the loop is the remedy. Our inference is narrower than the authors': any architecture in which the repeated region is a proper subset of the network must show some residual-statistic discontinuity at the prelude/coda boundary, so the "fully looped" claim may well be a statement about boundary effects rather than about looping as such. Huginn [1] reports two failed training attempts diagnosed from early loss/gradient behaviour, which is the same phenomenon seen from the other side.

5.4 The objective: what is supervised, and over how many steps

Three questions distinguish the training objectives in this literature.

Is every iteration supervised, or only one? Standard next-token prediction supervises the readout at every iteration. [77]'s step-dependent supervision does the opposite: sample $i$ is supervised only at iteration $t = T_i$, where $T_i$ is the number of steps the task actually needs. Combined with full-output prediction and input injection, this is what produces the paper's headline length-generalisation result. [82] and [80] extend the analysis of depth extrapolation and its limits.

Over how many iterations does the gradient flow? Full backpropagation through $T$ iterations is expensive and can be unstable; truncation is cheap and loses the signal that early iterations carry. [61] identifies an intermediate gradient horizon as one of the three ingredients of stability — neither full BPTT nor aggressive truncation. This is one of the few quantitative findings in the area, and it is consistent with the gradient-starvation picture of §5.1.

Is there an auxiliary consistency or reuse objective? CoFRe [40] adds a reuse-and-consistency regulariser so that a single denoising step approximates a multi-step trajectory; LoopFormer [63] adds a shortcut-consistency loss pulling short trajectories toward the longest; [53] argues that adaptive-depth behaviour decomposes into a trajectory and a readout and that the readout can be supervised post hoc (PLN). Consistency objectives are what make a single checkpoint serve multiple loop counts, so they are as much an inference-time design choice as a training one.

5.5 Loop count and per-loop depth as hyperparameters

The most practically useful results in this area are about how to choose $(b, T)$.

More loops than two. [33]'s recipe explicitly targets the regime the field drifted into — one-block loops trained at $T=2$ — and reports that larger effective depth bought by looping scales better than the alternative. Their iso-FLOP comparison locates an optimum near five loops, while under a fixed non-iso-FLOP budget performance continues to improve out to nine loops. The distinction between those two curves is exactly the distinction this survey keeps returning to: how many loops does the loss improve with is a different question from how many loops are worth their FLOPs.

Adaptive rather than uniform. Mixture-of-Recursions [30] routes tokens to different recursion depths and reports compute-optimal scaling curves for that setting, and DeepLoop [41] sweeps loop count against backbone size under a residual-scaling sweep.

Training recipes at scale. Three 2026 works are the most useful references for practitioners. Closing the Loop [50] reports practical recipes and, importantly, an inference-FLOP-matched comparison against a dense model (discussed in §11). Parcae [103] derives scaling laws in which recurrence is an orthogonal compute axis and reports power laws for the optimal recurrence count against FLOP budget. Iso-depth scaling laws [102] reframe the question in the form "how much is one recurrence worth?" by holding depth fixed and asking what a recurrence buys relative to added width or tokens.

Figure 14: Compute-optimal validation loss per architecture together with the fitted joint scaling law in the iso-depth analysis of [102], whose question is how much one recurrence is worth relative to added width or tokens. Figure reproduced from [102].

5.6 What is not established about training

Three gaps are worth naming explicitly, because each is routinely glossed.

• Seed variance. Very few of the training-stability papers report more than a single run per configuration. The interventions in §5.2 are described with the confidence of replicated results but the corpus does not support that confidence. This is our assessment of the corpus, not a claim made by any of the papers.
• Scale transfer. Almost every stability diagnosis is performed at small or medium scale and then extrapolated. Parcae [103] is the exception that makes the rule, and its whole contribution is that stability behaves law-like.
• Interaction with data. Whether a stabilising intervention changes what the model can learn, rather than only whether it trains, is rarely tested. [61]'s negative result about hierarchy is one of the few that isolates this.

Takeaways. Looping is a deep computation with a shared Jacobian, and its three failure modes — residual drift, gradient starvation and unwanted attractors — are the problems the training literature actually solves. Large looped language models overwhelmingly keep the unrolled objective and stabilise the architecture: residual scaling, normalisation placement, bounded gated updates, stochasticity. The single clearest diagnostic shows all three failure modes co-occurring in one run [43], and the corpus's most systematic ablation argues that bounded gated normalised updates plus an intermediate gradient horizon and a large batch matter more than hierarchical structure [61]. Choosing $(b,T)$ is governed by two different questions — where the loss improves, and where the FLOPs pay off — which have different answers.

Part III — Inference-time computation

6. Adaptive inference: spending loops where they matter

6.1 The premise, and why it is not automatically true

If a loop count $T$ can be varied per input, then a looped model can in principle match a dense model's quality at a fraction of its average compute — spend many iterations on hard tokens, few on easy ones. Whether this works depends entirely on whether the distribution of required depth is predictable from something the model can see at run time, and whether the mechanism that predicts it is calibrated. Both are testable, and the evidence is mixed.

6.2 When to stop: halting is harder than it looks

The simplest adaptive scheme is a learned halting gate over iterations, in the tradition of ACT [128]. The corpus contains a detailed evaluation that should temper optimism about it. [55] reframes adaptive depth as jointly selecting a trajectory and a readout and reports, on synthetic parity/MANO tasks with a 4-layer $d=512$ loop at $T=6$ over 3 seeds, that a geometric-prior trajectory paired with a logit-margin readout reaches 95–99% accuracy at 1.38–1.55 average loops, whereas a learned linear gate needs 2.80 ± 0.30 and an MLP gate 3.58 ± 0.36 loops to reach 95%. On frozen Ouro-1.4B/2.6B checkpoints, post-hoc readouts beat the model's own on 5 of 12 model–benchmark pairs and are cheaper on 7 of 12. The authors' claim is about design of the readout; our reading is the weaker and safer one — learned gates are systematically miscalibrated in this setting, and a non-learned schedule plus a good readout can dominate them.

A second diagnostic points the same way. [53] argues that adaptive depth as usually implemented behaves as a "finishing stage" — the extra loops sharpen an answer that the uniform-depth trajectory has already determined — rather than as a scheduler that discovers that a token needs more computation. If that reading is right, adaptive depth's benefit is bounded by the quality of the underlying fixed schedule.

Two further results constrain what halting can be expected to achieve. [60] decomposes the quality loss from naive depth truncation into interpretable components on a 542.8M model, reporting that of a 1.4251-nat gap, roughly 47.9% [46.6, 49.3] is attributable to block applications, ≈ −2.8% [−3.2, −2.4] to distinct depth, 54.8% [53.3, 56.3] to inter-block composition and 25.6% [24.5, 26.6] to calibration; on Huginn-0125 they report only 0.045 temperature drift over a 32× depth change alongside a clean 1.83-nat depth effect. The practical implication is that most of what depth buys is composition, not per-iteration refinement — which is exactly the quantity a halting gate is worst placed to estimate. And [56] and [59] report that per-token fixed-point convergence and the rate-limiting of slow oscillatory modes are the right abstractions for stopping, in contrast to a scalar confidence gate.

A third variant is trained end-to-end rather than diagnosed after the fact: Sparse UT [8] puts a halting module inside a sparse mixture block and trains it jointly with the rest of the model, and reports that the amount of depth consumed varies from token to token within a single sequence (Figure 15) — evidence that what a gate has to predict is itself position-dependent.

Figure 15: The token-by-layer activity pattern of the Sparse Universal Transformer halting module of [8] on a Logical Inference example. Positions of the same sequence remain active to visibly different depths. Figure reproduced from [8].

6.3 What the extra loops buy: a controlled picture

The clearest evidence that loop requirements scale with problem difficulty comes from the forced-exit experiments of [55], reproduced in Figure 16: on synthetic tasks whose difficulty is controlled by the number of required operations, the accuracy-versus-loop-count curves shift rightward and flatten as the operation count rises, and at ten operations even the strongest configuration has not saturated by six loops.

Figure 16: Forced-exit accuracy against loop count as a function of problem depth [55]. Required loops increase with the number of operations. Figure reproduced from [55].

This is a synthetic setting and should be read as a mechanism demonstration, not as an estimate of what language benchmarks need. What it establishes cleanly is that the ceiling on adaptive-depth gains is set by whether the hardest inputs need more iterations than the budget allows — and that in this setting they do.

6.4 Deciding per token, per program, or per batch

Four positions on the spectrum are represented, and they differ in how much information the decision-maker sees.

Per token, from the first state. T-LoopFormer [62] trains a two-layer MLP router that predicts each token's loop count from the initial hidden state $h^{(0)}$, with gather–scatter execution and per-recursion KV caches. On FineWeb-Edu 25B tokens at $d=2048$ the authors report 24×-FLOPs perplexity of 20.13/19.14 and average accuracy 44.20 against LoopFormer's 21.96/20.09/43.48 and a non-recurrent 24-layer model's 20.17/19.22/43.8, with 0.197 ms/token versus LoopFormer's 0.441. The honest reading: elastic-depth training buys a large speed advantage at essentially the same quality as a dense model of equal FLOPs — it does not buy quality. The design's own limitation, acknowledged in the paper, is that the decision is made once from $h^{(0)}$ and cannot be revised once context is read.

Per token, after error detection. Think-at-Hard [64] inverts the usual order: it runs a first pass, predicts which tokens are likely wrong, and revisits only those, with a duo-causal attention over (token, depth) and a depth-aware LoRA active only for $d>1$. On Qwen3-Base 0.6B/1.7B/4B with $d_{\max}=2$, the 1.7B configuration averages 51.3 against a 47.5 standard baseline, 39.9 for "always think", and 46.9 for Ouro; only ≈7% of tokens receive a second round (6% on AIME25) at ≈1.04–1.05× per-token compute, with measured 301.4 s against 747.2 s for always-think. The paper's own limits are important: the oracle upper bound on HumanEval++ is 72.9 against 40.9, so the decider — reported as 83% accurate — is the bottleneck; and the method is sensitive to over-thinking errors.

Per program, before execution. Program-of-Layers [65] predicts a whole schedule of skip/repeat segments in advance, freezing the backbone and training only a 2.1M-parameter scheduler. On DART-Math with LLaMA-3.2-3B the authors report pass@5 on DM-1 of 74.5 against a 65.2 baseline. The paper's MCTS diagnostic — that skip-and-loop schedules are optimal on all 4 models × 5 difficulty levels — is presented as evidence that the schedule space is real; we read it as evidence that a search over schedules finds something, which is weaker than evidence that the trained scheduler finds it.

Elastic, without a decision. LoopFormer [66] and One Model to Loop Them All [66] sidestep the router by training a single model over a continuum of loop counts. Continuous-depth batching [57] is the systems-side counterpart: batching tokens that have reached different depths. [58] proposes changing the loop count mid-generation ("change of thought"), and [72, 73, 74] turn the early iterations into a draft for speculative decoding — the next subsection.

6.5 Recovering the cost: loops as drafts and as ensembles

A looped model's intermediate states are free by-products, and a small literature has grown around using them.

Figure 17: Depth-asynchronous self-speculation, in which the read depth of the verified prefix and the draft depth are decoupled [74]. Drawn by the author from the mechanism described in that paper; the other loop-based decoders in this chapter differ mainly in how they pipeline these steps.

Self-speculative decoding. The early iterations of a loop generate a cheap prediction that the full-depth pass verifies. LoopSpec [72] is a pipelined formulation (≤6.83× best-case speedup, training-free). WaveFront Decoding [73] parallelises the same idea. The strongest evidence is [74], which decouples draft depth from read depth of the verified prefix and reports that its DAS-Wave variant is fastest in all 16 model–task settings, with mean speedups of 4.00×–6.96× over paired full-depth autoregressive decoding, acceptance lengths of 15.6–18.1 on GSM8K against 5.2–12.2 for tuned sequential self-speculation, and a same-stack internal control of 1.70–3.72×. The paper reports paired bootstrap confidence intervals over 10,000 resamples, which is unusually careful for this literature. Its stated limitation is that the method needs depth-indexed prefix K/V and that without state carry the "mature" read can fall below 1× — i.e. the speedup is a property of the loop's state structure, not of the idea alone.

Contrastive decoding over iterations. [75] contrasts a looped model's output distribution against its own earlier loop distributions, training-free, and reports Ouro-2.6B-Thinking AIME 2024 pass@1 improving from 61.88 to 73.33 and Huginn HumanEval from 22.56 to 31.71 while halving the loop count at 22.5–48.2% lower forward FLOPs. This is a large claim from a single paper and has not been independently replicated within this corpus.

Parallel samplers. [69, 70, 71] replace sequential iteration with parallel decoding of blocks of the trajectory; [70] explicitly connects parallel sampling of recurrent-depth models to diffusion language models, which is theoretically the most interesting of the three because it makes the loop count and the denoising step count the same object.

6.6 The state of the evidence on adaptive inference

Summarising §6 honestly requires saying that the positive results are mostly of the form "adaptive depth reaches comparable quality at lower average cost", not "adaptive depth reaches better quality at comparable cost". The second kind of result — a genuine quality gain from allocating loops non-uniformly — appears mainly in [64] (+3.8 points on average for the 1.7B model) and [68] (accuracy–compute slope improving by 53%, 2.74 vs 1.79, with the gain growing with maximum depth). Both are from the same research line, which is a reason for caution rather than for dismissal. And in the one clean iso-compute comparison the survey found between an adaptive elastic-depth model and a dense model of equal FLOPs, the adaptive model's advantage is speed, not quality [62].

Takeaways. Adaptive depth works when the required depth is predictable and the decision is made with enough information: deciding per token after observing a first pass [64] and deciding a whole program in advance [65] both report gains, while deciding once from the initial state [62] and relying on a learned gate [55] do not. Most of what depth buys is inter-block composition rather than per-iteration refinement [60], which is precisely what a scalar confidence gate estimates worst. The clearest evidence on the ceiling of adaptive depth is synthetic: loop requirements grow with the number of required operations [55]. And the loop's intermediate states are a genuinely useful by-product — self-speculative decoding is the most thoroughly evidenced inference-time win in the corpus [74].

7. Efficiency, systems and safety

7.1 Where the FLOPs and the bytes go

Three accounting facts change what a looped model costs, and they are easily conflated.

• Training. Every iteration backpropagated through costs a forward and a backward pass; total training FLOPs scale roughly linearly in $T$ for fixed $b$. LoopFormer [63] reports about 1.5× FLOPs and 1.3× wall-clock overhead relative to its baseline for the family-of-trajectories objective.
• Prefill. A $T$-iteration loop over a prompt is $T$ passes over the shared blocks; the prelude and coda are paid once. MoEUT [7] reports training about 1.7× slower than a comparable dense model.
• Decoding — the case that matters most. In a sequence-recurrent model, the state is a fixed-size object and decoding is $O(1)$ per token. In a depth-recurrent model, the shared blocks attend to the full KV cache on every one of the $T$ iterations, so per-token decode cost multiplies by $T$ and the KV working set is read $T$ times. This is why so much of the recent systems literature is about the iteration-axis KV: shared-memory architectures [95], cross-loop KV compression [116], depth-hidden state carry [24], hybrid latent attention [117], memory-efficient looped transformers [96] and lazy-update implementations [97].

Figure 18: Where the cost of a depth loop accumulates, and the three families of remedy. Drawn by the author from the accounting described in [95, 96, 97, 116, 117] and the decoding schemes of [72, 73, 74].

The distinction matters for how results should be quoted. A reported speedup from reducing $T$ applies to decode FLOPs and not to the memory traffic, unless the architecture also changes what is read.

7.2 Systems results worth carrying forward

Shared memory [95] is the clearest architectural efficiency result: reusing one KV memory across recursions improves the accuracy/memory frontier at matched parameters, with the sharpest gains at larger model sizes and deeper recursions.

Lazy and pipelined execution [97, 96] reduce memory traffic by not materialising every iteration's state; quantisation [98] extends post-training quantisation to the loop dimension, where errors compound across iterations and therefore cannot be treated as a single-pass problem.

Depth-adaptive batching [57] is the systems fix for elastic depth: group tokens by iteration count so that a batch of mixed-depth work still runs dense kernels.

Parallelism [69] and parallel samplers [70, 71] trade sequential iteration for parallel decoding, which changes the latency profile but not the total FLOPs.

7.3 Safety: loops change the model's own controllability surface

Figure 19: How a serving-time loop count reaches safety-relevant behaviour. The three edges are the three claims named in this section; all three are abstract-level within this survey's corpus and none is treated as established. Drawn by the author.

Three findings, all thin but all pointing the same way.

• Refusal depends on the loop count. A recurrent-depth model's refusal behaviour has been reported to change with the number of loops executed at inference — that is, a safety-relevant behaviour is coupled to a compute knob that a serving system may vary dynamically. The corpus contains this claim at abstract level only, and we flag it as unverified within this survey's reading.
• A single looped model exposes many readouts. Work on alignment across depths argues that one looped model presents a broader readout space than a fixed-depth model, because every iteration is a candidate readout. Like the previous item, this is abstract-level within this corpus.
• Monitorability. [109] analyses the chain-of-thought monitorability of looped language models at abstract level. This connects to the broader latent-reasoning concern that a model whose intermediate computation is not in tokens is harder to inspect; within this corpus the looped-architecture-specific evidence is thin, and the honest statement is that safety and interpretability of looped models is an identified problem with very little direct evidence.

The mechanistic work in §9 partially mitigates this: depth-axis readouts (logit lens and coda lens over iterations [93]) do give a partial view into what the loop is doing. But "partially inspectable" is not "monitorable", and we do not treat the mechanistic results as answering the safety question.

Takeaways. The cost of a depth loop is dominated by decode: every iteration re-reads the context, so reducing the loop count multiplies decode FLOPs by $T$ while memory traffic is only relieved by changing what is read — which is why the systems literature concentrates on the iteration-axis KV. The best-evidenced efficiency result is shared cross-iteration memory [95]; the best-evidenced latency result is depth-asynchronous self-speculation [74]. Safety is the survey's thinnest chapter: the loop count plausibly changes refusal behaviour and the readout space is broadened by construction, but within this corpus the evidence is abstract-level and should not be treated as established.

Part IV — Understanding the loop

8. Theory: what looping can compute, and what it generalises to

Figure 20: The four theoretical threads reviewed in this chapter and what each establishes. Drawn by the author from [3, 17, 76, 77, 78, 81, 86, 87, 88, 105, 106, 107, 115].

8.1 Simulation: looping buys computational universality

The strongest theoretical case for looping is constructive. [76] shows that a constant-depth looped transformer can simulate a small programmable computer: the input sequence is read as a punched card holding instructions, memory and scratchpad; hard-coded weights implement a one-instruction-set computer (SUBLEQ, generalised to FLEQ, where each instruction may itself invoke a pre-built linear-algebra or nonlinearity block); the iteration index serves as the program counter; and the parameters are independent of the number of steps $T$, with a halting condition implemented through a self-referential end-of-input marker. The paper's Table 1 gives concrete configurations: a 9-layer, 2-head looped transformer runs SUBLEQ; a 13-layer, 1-head looped transformer runs Newton matrix inversion, power iteration and two-layer-network SGD to within $\epsilon$. The error analysis is explicit that this requires the readout scale to satisfy $\lambda > \log(Gdn^3/\epsilon)$ and per-step error at most $\epsilon/T$ for the T-step accumulation to stay within $\epsilon$.

It is essential to state what this does not establish, and the authors say so themselves: there are no training experiments, and the construction "has no resemblance" to how real language models are trained. The result is a statement about representational capacity under exact weights, not about learnability. [87] extends the same style of construction to graph algorithms — BFS, DFS and shortest paths simulated by looped transformers with graph-specific attention heads within bounded memory.

8.2 Depth generalisation: the most practically relevant theory

The theoretical result with the clearest empirical consequences concerns generalising to inputs longer than those seen in training. [77] defines a recurrence of the form

$$P = P_{\text{pre}} \circ (P')^{T(n)} \circ P_{\text{post}},$$

replaces next-token prediction with full-output prediction, uses a decoder-only architecture with no positional encoding and input injection, and adds step-dependent supervision (token $i$ is supervised only at iteration $t = T_i$). On six synthetic tasks with training lengths at most 20 (12 for multiplication) and five seeds, they report parity learned up to 20 bits generalising beyond 40, and addition and copy remaining near-perfect where standard next-token prediction fails at "maximum training length + 10". Inference uses either an oracle $T(n)$ or a maximum-confidence criterion. The limits are stated in the paper and are severe: the construction supports a single loop segment only, it needs ground-truth step counts at training time, and its training compute grows with the number of steps. Code is released.

This result should be read together with three follow-ups. [79] analyses the mechanisms and limits of length generalisation in looped transformers beyond the training horizon; [80] studies depth extrapolation directly, asking which training depth is best given a compute budget and reporting a population optimum for the test-depth window; [82] connects implicit reasoning in recurrent-depth models to depth generalisation. Together they form the corpus's most coherent theoretical thread — and the one with the clearest caveat, which is that all of it is measured on synthetic tasks with controllable ground-truth step counts.

A related negative-flavoured result is worth recording: [88] shows that transformers can shortcut the computation of automata by composing sub-computations in fewer layers than the sequence length requires, which shows that "effective depth" is not a monotone measure of computational work. Any claim that additional iterations buy additional serial computation has to contend with the fact that models are under no obligation to use them serially.

8.3 Loops, depth and chain-of-thought as alternative compute allocations

A second theoretical thread asks how looped computation compares with the alternatives it competes against: more layers, or more tokens of chain-of-thought.

The paper that most directly states the "loops as latent CoT" thesis is [3], which studies the power of looped transformers through the lens of latent thoughts — how many implicit reasoning steps are needed for a given problem — and quantifies a scaling relation between effective depth and problem class. It is the paper that most directly states the "loops as latent CoT" thesis. [86] compares chain-of-thought and looping formally, asking which can express which; [83, 84, 85] analyse what looped transformers can learn in context, showing that looped transformers can implement multi-step gradient descent over in-context examples and that looping removes an exponential sample-complexity dependence that a fixed-depth model has, with [85] relating this to task diversity.

The empirical counterpart is [78]: a looped model with one block, 30 batch steps and 15 iterations ties a 12-layer standard transformer on in-context linear regression with one twelfth of the parameters, matches it on higher-complexity function classes and exceeds it on sparse linear regression — beating even Lasso on 9 of 10 OpenML datasets. The paper's limits are candid: there is an error floor relative to exact fixed-point iteration; $T > 20$ becomes unstable; there is no equal-inference-compute comparison, and about 20 loops were needed to match 8 layers. Our inference is that this paper is best read as evidence that iterative refinement is an efficient way to implement an iterative algorithm, not as evidence that looping dominates depth at matched compute.

Figure 21: The four looping configurations compared in [3]: an iso-parameter baseline of k layers, a looped model in which k layers are reused for L iterations, an iso-FLOP baseline of kL distinct layers, and middle looping. Figure reproduced from [3].

8.4 Dynamical-systems analyses

A final cluster treats the loop as a dynamical system and asks about its qualitative behaviour. [105] asks when recurrence "becomes an algorithm", framing it as convergence selection in weight-tied looped transformers; [106] applies a Jacobian analysis to ask whether an interpretable global-workspace structure survives recurrence; [107] identifies dynamical phase selection as controlling compute scaling in looped transformers; [115] gives a geometric analysis of why a finite number of steps can move in the right direction and still land in the wrong place. These are recent and largely single-paper results, but they are the theoretical work most directly aimed at the question practitioners care about — why does the last iteration stop helping — and they are the right frame for §9's mechanistic findings.

8.5 What the theory does not establish

Three gaps should be stated plainly, because the constructive results are frequently cited as if they closed them.

• Learnability. The simulation results [76, 87] are existence proofs with hand-set weights. No result in this corpus shows that gradient descent on a real objective finds the simulating configuration.
• Tightness. The chain-of-thought-versus-looping comparisons [3, 86] establish expressivity relations, not efficiency equivalences at realistic scale; the empirical bridge [78] is small-scale and not compute-matched.
• Composition and readout. [81] shows that universal transformers achieve perfect length generalisation on circuit computations in tiny settings, and [60] shows that most of the depth effect in a trained model is inter-block composition. There is no theory in this corpus that predicts how much of a trained large model's depth effect is composition, refinement or noise.

Takeaways. Looping's strongest theoretical claim is constructive universality at constant depth [76, 87] — but with hand-set weights and no learnability result behind it. The most practically consequential theory is depth/length generalisation under step-dependent supervision [77], which is well evidenced on synthetic tasks and restricted to a single loop segment requiring ground-truth step counts. Comparing loops against chain-of-thought and against extra layers yields expressivity relations [3, 86] and a small-scale empirical bridge [78] that is not compute-matched. Transformers can shortcut serial computation [88], so effective depth is not a monotone measure of work — a fact that any "more loops means more reasoning" claim has to absorb.

9. Mechanistic interpretability of the loop

Figure 22: The three families of mechanistic evidence on the loop, and the caveat that bounds all of them. Drawn by the author from [89, 90, 92, 93, 94, 108, 110, 112, 114].

9.1 Do different iterations do different things?

The most basic mechanistic question is whether the iterations of a trained loop are functionally differentiated or whether they are near-identical refinements. The evidence says the former, but with an important qualification.

One study, [89], examines which computation each loop performs, asking whether the shared weights select different sub-computations at different iterations. Their method is unusually direct: they show that a learned linear layer can be used to rewrite the state entering the next iteration, and that patching the attention pattern of the controlled run into the uncontrolled run switches the model's prediction target — evidence that specific iterations are responsible for specific computations rather than for generic refinement.

A complementary question — what makes recurrence effective — is asked by [91], which finds that conditioning matters at two levels: a base loop that is unconditioned, plus a core loop whose conditioning (combined step/parameter signals) supplies the iteration-specific information. This is the mechanistic counterpart of §4.2's step-conditioning axis, and it is one of the few places where the mechanistic finding and the architectural design advice agree.

Two further results refine the picture. [34] separates "thinking harder" from "knowing more" by pairing adaptive loops with memory and reports that the two routes are not substitutes, which supports the §4.3 reading that extra iterations need somewhere to put their result. [104] dissects hierarchical reasoning models mech-anistically and [111] argues that latent reasoning in TRMs is best understood as a policy-improvement operator — a claim that recasts the loop as an optimisation procedure rather than as a sequence of reasoning steps.

Figure 23: Patching the second layer's attention pattern switches the prediction target in both directions: from two-hop behaviour toward a one-hop answer (a) and back (b), measured as answer rate over one-hop and two-hop responses. Figure reproduced from [89].

9.2 Reading the trajectory: lens methods

A second line treats the iteration index as a readable axis and asks when, and in what order, information becomes available.

The decoding study [93] applies logit lens and coda lens to each iteration's block outputs, and reports that the top-1 prediction's rank oscillates across iterations rather than converging monotonically — direct evidence against the "each loop is a reasoning step" reading. [90] introduces step-resolved data attribution, asking which training examples influence which iteration of the computation, and reports on parity that the answer-token hidden state traces a four-state limit cycle in PCA space; on Sudoku, test-time accuracy versus loop count stratified by difficulty; on GSM8K, the cumulative share of step-wise influence as a function of iteration, recomputed with full backpropagation at several analysis horizons $\tau$. The geometric-influence growth curve is the sharpest instrument in this literature for the question "which iterations matter".

Figure 24: The average rank of the final predicted token as a function of block number, read out with logit lens (left) and coda lens (right) in [93]. The rank oscillates rather than descending monotonically, which the authors read as evidence against treating each block as one reasoning step. Figure reproduced from [93].

A monitoring toolkit rather than a mechanism claim is offered by [94], which proposes five lenses for monitoring latent reasoning in looped LMs — hidden-state trajectories and attention focus, when the answer token is committed, and how the answer changes when noise is injected at the entry state — which is closer to an operational monitoring toolkit than to a mechanism claim. [92] measures attention-matrix norms across depth and extracts "stages of inference" within each recurrence, reporting that column-sum concentration is stable up to 128 recursions. [110] argues that latent reasoning discovers a recurrent search algorithm that underlies depth generalisation, tying the mechanistic story back to §8.2.

9.3 Dynamical readings: cycles, basins and scales

The most recent mechanistic work reads the loop as a dynamical system. [112] reports that latent reasoning can be trapped in fractal basins — that the basin structure of the iteration map contains fine-grained boundaries on which the outcome is extremely sensitive. That is a substantive claim with a practical corollary: if basin boundaries are fractal, then "how many loops" is not a smooth knob near the boundary. [113] proposes invariant-measure representations as a stabilising alternative; [114] and [52] study two-timescale and fast–slow latent dynamics, which is the mechanistic version of the architectural observation that some information should persist across iterations while other information should be refreshed.

9.4 What mechanistic evidence cannot settle

The disciplinary point is made directly in [108]: observable patterns in latent-reasoning models are not explanations, and a causal-geometric analysis is needed before the patterns can be interpreted. This is the right note on which to end this chapter, because the looped-transformer mechanism literature has a structural temptation — the iteration axis looks like a step counter, so a curve that rises with iteration invites a "the model is doing one more step" reading. The results above are genuinely informative about what varies with iteration; they are much weaker evidence about what the model is computing, and the strongest single counterexample is [93]'s finding that prediction rank oscillates rather than converging.

Within this corpus we found no mechanistic study of a looped language model at the multi-billion-parameter scale with replication of its central result. The mechanism literature is currently a small-scale, single-lab field, and its conclusions should be weighted accordingly.

Takeaways. Iterations of a trained loop are functionally differentiated, and [89] provides causal rather than correlational evidence for this by switching behaviour through state and attention patching. The mapping from iteration to "reasoning step" is nonetheless broken: prediction rank oscillates across iterations rather than converging [93], and the most useful instruments for the question "which iterations matter" are step-resolved attribution and geometric-influence growth [90] rather than a monotone convergence curve. Dynamical readings — limit cycles, fractal basins, two-timescale structure [52, 112, 114] — are the most promising current frame, and [108]'s warning that observable patterns are not explanations applies with full force to this subfield.

Part V — Beyond text, evidence, and outlook

10. The loop outside language modelling

The depth axis is not language-specific, and some of the cleanest evidence for looping comes from domains where "more iterations" has an unambiguous meaning. This chapter is deliberately short: the applications are mostly early and small, and its purpose is to show which of Part II's design choices survive the transfer.

10.1 Vision

Vision is the largest non-text cluster, and it splits cleanly along the scope axis.

Full and localised loops. LoopViT [119] applies looped transformers to visual ARC-style reasoning; LoopVL [120] frames recurrent visual intelligence at scale; reViT [99] uses one recurrent block with depth-programmed experts — the natural vision analogue of the per-loop adapters and per-loop experts of §4.2 — and reports results across multiple depths from one block. RD-ViT [121] extends the two-stage recurrent-depth pattern to semantic segmentation and reports reduced data dependence, which is the architectural claim of §1.2 translated to a dense-prediction setting.

Efficiency-oriented loops. [122] investigates single-block recurrence in vision transformers for recognition; MergeOver [123] merges tokens post-training so that later recursions process fewer of them — an inference-time saving that has no text analogue because images have a spatial redundancy structure tokens do not; GramLoop [125] uses a training-free gram-gated replay for robust dense prediction; LoopVAE [124] puts recurrent depth across scales in a visual tokeniser.

The vision results matter for one methodological reason: because the input is fixed and the loop is over a spatial-then-depth axis, the number of iterations can be varied at test time without changing the token count, which makes an iso-compute comparison against a deeper ViT unusually easy. Papers in this cluster report such comparisons more often than the language papers do, and their conclusions are correspondingly less dependent on parameter matching.

10.2 Generative modelling: diffusion, flow and discrete denoising

Diffusion and flow models are the domain in which the loop count is intrinsically meaningful, since iterating the denoiser is what the model is for. [35] applies a looped transformer as a diffusion backbone; LiFT [36] does so for flow matching; [37] and [38] take the same idea to discrete/masked diffusion over token positions, and [39] makes the iteration count adaptive. CoFRe [40] is the most theoretically explicit: masked generation is formulated as a fixed-point iteration, with a reuse-and-consistency regulariser tying a single step to the full trajectory.

The value of this cluster to the rest of the survey is that it supplies the closest thing to a ground truth for the depth axis. In a denoising model, the relationship between iteration count and output distribution is defined by the generative process; in a language loop it is not. Statements like "the last iterations stop helping" are therefore testable here in a way they are not in §6.

10.3 Reinforcement learning and control

FlexLoop [100] introduces depth-elastic looped policies for adaptive test-time computation in deep RL, which is the cleanest instance of the inference-time depth policy of §6.4 being applied to an agent rather than to a token stream. The interesting structural point is that in RL the cost of an extra iteration is part of the objective rather than a serving-budget compromise, so the depth policy is trained rather than bolted on.

10.4 Why the transfers work — and where they don't

Two design choices transfer cleanly across all four domains: shared weights with per-iteration conditioning (§4.2) and an elastic loop count (§4.4). One transfers badly: learned halting (§6.2), which appears in the vision and generative clusters mainly as a training-time regulariser rather than as a run-time controller.

Our reading — stated as inference — is that the successful transfers are exactly those where the loop's iterations have an externally defined meaning (denoising steps, refinement scales, spatial resolution) and the unsuccessful transfers are those where the model must invent the meaning of an iteration. That is consistent with §8.3 and §9.2, both of which find that trained loops do not naturally align iterations with reasoning steps.

Takeaways. The loop transfers to vision, diffusion/flow, discrete denoising, and RL, and the transfers that work are those where an iteration has an externally defined meaning. Vision is methodologically ahead of language modelling on the survey's central concern, because fixed input size makes iso-compute comparisons against deeper ViTs easy and they are reported more often. Learned halting is the one design choice that transfers badly.

11. What the evidence actually supports

Figure 25: A reading procedure for "looped versus dense" claims: identify the matched quantity first, because the answer changes with it. Drawn by the author from the comparisons tabulated in Table 5 and discussed in this chapter.

11.1 The comparison problem

The single largest obstacle to reading this literature is that "looped versus dense" comparisons are run at five different kinds of match, and they give different answers.

Table 5: The five kinds of match used in "looped versus dense" comparisons, and the finding each tends to produce. Inclusion rule: this table sorts the comparisons discussed in this chapter by the quantity held equal; it reports no measurements of its own.

The fifth row is worth naming because it explains a large amount of apparent disagreement. A comparison in which a looped model with effective depth 24 is compared against a dense 12-layer model is not a comparison at matched depth, and its result says nothing about whether looping is a good use of depth. Works that report "looped beats dense" frequently mean this, and works that report "looped is behind" frequently mean iso-FLOP.

11.2 The specific numbers, read carefully

Four cross-paper comparisons recur in discussions of this literature. Each is stated below in the form the paper reports, followed by what we think it does and does not establish.

Huginn's twin control [1]. The paper trains recurrent and non-recurrent models under otherwise identical conditions, and reports that at 0.18T tokens the recurrent-depth model reaches 9.02 on GSM8K with chain-of-thought against 1.82 for the non-recurrent control. This is a pareto-optimality argument: if a non-recurrent model with the same data and the same forward FLOPs per token reaches the same quality with more training, the recurrence is overhead. The paper's evidence is that it does not within its budget. What it does not establish is that the recurrent model is efficient at larger budgets, and the control is a single pair of runs.

Closing the Loop's iso-FLOP comparison [50]. A 1.4B looped model is compared against a 3.9B dense model at matched inference FLOPs and reported to be at parity on mathematics while behind by 2–9 points on knowledge and commonsense tasks. This is the most informative single comparison in the corpus precisely because it is unfavourable on a class of tasks, and because it suggests the mechanism: looping helps where the task is procedural and hurts where the answer must be retrieved from parameters. That reading is ours; the paper reports the task-wise gap.

Looping Beyond Twice's two curves [33]. Under an iso-FLOP budget the optimum is near five loops; under a fixed (non-iso) budget performance continues to improve out to nine. This is the cleanest available demonstration that "how many loops does loss improve with" and "how many loops are worth their FLOPs" are different questions.

Shared Memory's match [95]. The comparison is parameter-matched against other looped designs, not against a dense model at matched decode FLOPs, and the paper's own efficiency claims are about the accuracy/memory frontier within looped architectures. It is routinely cited as evidence about looping's efficiency in general; that is a stronger claim than the design supports.

Elastic depth versus dense at equal FLOPs [62]. At 24× FLOPs, T-LoopFormer's perplexity and accuracy are statistically indistinguishable from a non-recurrent 24-layer model, with a 2.24× decode speed advantage. Read together with [63]'s finding that under FLOP-matched training the advantage of elastic training disappears, this is a narrow but important result: elastic depth's current value is throughput, not quality.

11.3 Why the numbers are hard to compare across papers

Beyond the choice of match, four further problems make cross-paper arithmetic unsafe.

• Effective depth is not compute. [88]'s shortcut result means two architectures with the same nominal depth may do different amounts of serial work.
• The decode cost depends on the KV scheme, not the loop count. A 16-iteration loop with a shared KV memory [95] and a 16-iteration loop that re-reads a full cache per iteration have the same nominal FLOPs in a naive count and very different wall-clock, memory and acceptance behaviour.
• Architectural variants are not measured under a common protocol. [13] changes what a layer reads across positions, [16] compares hierarchical against flat iteration at fixed parameters, and per-token depth policies such as [67] change what a token's cost even means. There is no benchmark that makes these commensurable.
• Scale is inconsistent. The stability findings come from small models (§5.6); the quality findings from medium ones; the serving findings from whichever checkpoint the authors had. The three are rarely the same model.

11.4 What we take to be established, and what we do not

We separate three grades of conclusion.

Established within this corpus (multiple independent works, consistent results). Weight-tied recurrence is trainable at scale, but only with an explicit stabilising intervention [7, 41, 42, 43, 44, 61]. Extra iterations beyond a handful help mainly when there is an explicit cross-iteration state or memory to receive their result [20, 21, 95]. Loops can be turned into a per-token compute knob with a real but modest quality benefit and a larger throughput benefit [62, 63, 64, 66]. The intermediate states of a loop are usable as speculative drafts, and depth-asynchronous self-speculation is the best-evidenced inference-time win in the corpus [74].

Supported but thinly (one or two papers, or one research line). That adaptive depth produces a genuine quality gain at matched cost rather than only a cost saving [64, 68]. That contrastive decoding across iterations is a large, free improvement [75]. That most of the depth effect is inter-block composition rather than refinement [60]. That hierarchical structure is unnecessary for recursive reasoners [61]. All four are interesting and all four need independent replication.

Not established. Any inference-time scaling law for looped models comparable to what [102, 103] provide for training; whether the parameter-matched advantages of §11.2 survive at matched decode cost; whether looping helps tasks that require retrieval rather than procedure [50]; and whether the adaptivity behaviour observed in small synthetic studies persists in the models practitioners actually serve.

Takeaways. "Looped beats dense" is under-specified: the result depends on whether parameters, training FLOPs, inference FLOPs, decode cost or effective depth is held equal, and papers rarely say which. The most informative comparisons are the ones that are unfavourable in a specific way — iso-FLOP parity on mathematics with a knowledge deficit [50], and elastic-depth parity in quality with a throughput advantage [62]. Cross-paper arithmetic is unsafe because effective depth is not compute, decode cost depends on the KV scheme, and each family is measured on its own protocol.

12. Open problems

The works reviewed above suggest a list of problems that are well posed, consequential, and currently unanswered. They are ordered by what we judge to be their leverage.

12.1 A compute-matched benchmark for looping

The survey's most-repeated complaint is that the field compares at incompatible matches (§11.1). What is missing is not more comparisons but a protocol: a fixed benchmark, a fixed data budget, and a rule for what "equal" means, reported for every looped paper. The vision cluster shows this is feasible when input size is fixed (§10.1). The language case is harder because decode cost depends on the KV scheme, but that is an argument for reporting the KV scheme, not for abandoning the comparison.

12.2 An inference-time scaling law

Training-side scaling laws for looping now exist [102, 103] and are the most directly useful scaling results in the field: they say how much one recurrence is worth and how the optimal recurrence count moves with FLOP budget. There is no counterpart for inference: no result in this corpus predicts, for a trained looped model, how quality or cost varies with the scheduling of iterations — per token, per batch, per generation step. Given that §6 is the most active area of the field, this is the largest theory gap.

12.3 Does looping help retrieval, or only procedure?

The most specific empirical finding in the corpus is Closing the Loop's task-wise gap [50]: iso-FLOP parity on mathematics, a 2–9 point deficit on knowledge and commonsense. If that pattern replicates, the correct use of a looped model is not "instead of a dense model" but "for the subset of inference that is procedural", which would change how the architecture is deployed. We regard this as the single highest-value replication target in the survey.

Our reading of the mechanism — an inference, not a result — is that looping adds serial computation, which is what procedural tasks need, and does not add retrievable parameters, which is what knowledge tasks need. If that reading is right, then looped mixture-of-experts (§4.5) should close the gap, and iso-FLOP comparisons of looped MoE against dense should show a smaller knowledge deficit than [50] does. [101] is the closest existing test.

12.4 What makes an iteration meaningful?

The mechanistic evidence says iterations of a trained loop are functionally differentiated [89] but that their correspondence to reasoning steps is unreliable [93]. The theory says extra iterations are not necessarily used serially [88] and that most of the depth effect is composition [60]. Nobody has shown how to train a loop so that iteration $t$ means what a designer wants it to mean. This is the natural formalisation of the intuition behind step-dependent supervision [77] and depth-aware schedules [65], and it is currently the most promising bridge between the theory of §8 and the systems of §7.

12.5 Safety and monitorability of a compute knob

If inference-time loop count is a serving-time knob, and safety-relevant behaviour depends on it, then some safety properties are not properties of a model but of a deployment configuration. Within this corpus the evidence is thin (§7.3): two abstract-level reports that refusal behaviour varies with loop count and that a looped model exposes a broader readout space. Establishing whether either holds, and what a serving system should therefore pin, is a prerequisite for deploying looped models where refusal behaviour matters.

12.6 Stability at scale, with seeds

Every stability finding in §5.2 is a small-scale, usually single-seed diagnosis, extrapolated to a regime nobody has measured. Parcae [103] shows the question is answerable at scale. Until a few of the interventions are tested at 1B+ parameters with multiple seeds under a common protocol, "looped models need residual scaling and peri-layernorm" remains a plausible engineering heuristic rather than a result.

Takeaways. The problems above are ordered by leverage, and the first three share a single cause: the field compares looped models against dense models at five different kinds of match without saying which. A compute-matched protocol, an inference-time scaling law, and a replication of the iso-FLOP knowledge deficit [50] would between them resolve more disagreement than any new architecture. The remaining three — making an iteration mean something, treating loop count as a safety-relevant deployment parameter, and testing stability interventions at scale with seeds — are prerequisites for using looping deliberately rather than opportunistically.

---

13. Conclusion

Looping the depth axis is, in retrospect, an obvious idea whose difficulty turned out to lie entirely in the engineering. The expressive-power case was made in 2023 with constructions showing that constant-depth weight-tied transformers can simulate small computers and graph algorithms [76, 87]. The capability case was made on synthetic length generalisation [77] and in-context learning [78]. What the 2024–2026 literature added was the part that the theory does not cover: how to make an unrolled shared block train at billion-parameter scale without its residual statistics diverging or its early iterations losing their gradient; how to choose between whole-network, two-stage and localised loops; and what to do at inference time with a model whose compute per token is now a variable.

Three conclusions seem to us well supported. First, the design space is small and its axes are close to independent: scope, step conditioning, inter-iteration interface and depth policy (§3–§4), and almost every paper varies one of them. Second, training stability, not expressivity, has been the real constraint, and the interventions that work — residual scaling, normalisation placement, bounded gated updates, an intermediate gradient horizon [41, 42, 43, 44, 61] — are all ways of keeping an unrolled shared Jacobian from degenerating. Third, the inference-time story is real but is currently about throughput more than about quality: the best-evidenced wins come from reusing the loop's intermediate states as speculative drafts [74], and the cleanest iso-FLOP comparisons show parity in quality at a speed advantage [62, 63].

The conclusion we would most want a reader to carry away is a caution about the word "depth". Looping increases effective depth, and effective depth is not the same as computation, not the same as FLOPs, and not the same as reasoning steps. The theory says a looped transformer is free to shortcut its iterations [88]; the mechanism literature says prediction rank oscillates across iterations rather than converging [93]; the depth-decomposition result says the bulk of the effect is inter-block composition rather than per-iteration refinement [60]; and the mechanistic work that most directly addresses the question concludes that observable patterns along the iteration axis are not explanations [108]. Loop count is a real and useful control on a transformer's behaviour. What it controls is still, for the most part, an open question.

---

Appendix — Figures, tables and reference count

This review carries 25 numbered figures and 5 numbered tables. The bibliography contains 129 numbered references: 126 works from the screened candidate pool and 3 foundational works verified from external public metadata.

Figures, in reading order. Figure 1, survey map (§1.4); Figure 2, timeline (§2.4); Figure 3, taxonomy (§3.1); Figure 4, the two-stage recurrent-depth architecture, from [1] (§4.1); Figure 5, depth-attention, from [26] (§4.1); Figure 6, depth-attention source weights learned by a 3B model, from [26] (§4.2); Figure 7, accuracy versus memory for shared-memory loops, from [95] (§4.3); Figure 8, attention mass on the shared memory, from [95] (§4.3); Figure 9, the mixture-of-recursions step, from [30] (§4.4); Figure 10, the MoEUT peri-layernorm block, from [7] (§4.5); Figure 11, failure modes of an unrolled loop and the interventions that address them (§5.1); Figure 12, where the stabilising interventions act (§5.2); Figure 13, training diagnostics for a fully looped transformer, from [43] (§5.3); Figure 14, the iso-depth joint scaling law, from [102] (§5.5); Figure 15, the token-by-layer halting pattern of Sparse UT, from [8] (§6.2); Figure 16, forced-exit accuracy against loop count, from [55] (§6.3); Figure 17, depth-asynchronous self-speculation (§6.5); Figure 18, where the cost of a depth loop accumulates (§7.1); Figure 19, how a serving-time loop count reaches safety-relevant behaviour (§7.3); Figure 20, the four theoretical threads (§8); Figure 21, four looping configurations, from [3] (§8.3); Figure 22, the three families of mechanistic evidence (§9.1); Figure 23, attention-pattern patching and target switching, from [89] (§9.1); Figure 24, rank oscillation across blocks, from [93] (§9.2); Figure 25, a reading procedure for matched-cost claims (§11.1).

Tables, in reading order. Table 1, notation (§2.1); Table 2, adjacent families and why they are not this survey's object (§2.2); Table 3, the nearest surveys (§2.3); Table 4, architecture comparison on the four taxonomy axes (§4.7); Table 5, the five kinds of match in "looped versus dense" comparisons (§11.1).

Figures reproduced from the papers themselves: Figures 4, 5, 6, 7, 8, 9, 10, 13, 14, 15, 16, 21, 23 and 24 — fourteen in total. The remaining eleven figures are diagrams drawn by the author.

References

[1] Jonas Geiping, Sean McLeish, Neel Jain, et al.. Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach. 2025. arXiv preprint. arXiv:2502.05171

[2] Rui-Jie Zhu, Zixuan Wang, Kai Hua, et al.. Scaling Latent Reasoning via Looped Language Models. 2025. arXiv preprint. arXiv:2510.25741

[3] Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, Sashank J. Reddi. Reasoning with Latent Thoughts: On the Power of Looped Transformers. 2025. arXiv preprint. arXiv:2502.17416

[4] Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng, et al.. A Survey on Latent Reasoning. 2025. arXiv preprint. arXiv:2507.06203

[5] Xinghao Chen, Anhao Zhao, Heming Xia, et al.. Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning. 2025. arXiv preprint. arXiv:2505.16782

[6] Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, et al.. Training Large Language Models to Reason in a Continuous Latent Space. 2024. arXiv preprint. arXiv:2412.06769

[7] Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber, Christopher Potts, Christopher D. Manning. MoEUT: Mixture-of-Experts Universal Transformers. 2024. arXiv preprint. arXiv:2405.16039

[8] Shawn Tan, Yikang Shen, Zhenfang Chen, Aaron Courville, Chuang Gan. Sparse Universal Transformer. 2023. arXiv preprint. arXiv:2310.07096

[9] Sangmin Bae, Adam Fisch, Hrayr Harutyunyan, Ziwei Ji, Seungyeon Kim, Tal Schuster. Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA. 2024. arXiv preprint. arXiv:2410.20672

[10] Chengting Yu, Xiaobo Shu, Yadao Wang, et al.. MeSH: Memory-as-State-Highways for Recursive Transformers. 2025. arXiv preprint. arXiv:2510.07739

[11] Mohammadmahdi Nouriborji, Morteza Rohanian, Omid Rohanian. Improving Recursive Transformers with Mixture of LoRAs. 2025. arXiv preprint. arXiv:2512.12880

[12] Abbas Zeitoun, Lucas Torroba-Hennigen, Yoon Kim. Hyperloop Transformers. 2026. arXiv preprint. arXiv:2604.21254

[13] Costin-Andrei Oncescu, Depen Morwani, Samy Jelassi, Alexandru Meterez, Mujin Kwun, Sham Kakade. The Recurrent Transformer: Greater Effective Depth and Efficient Decoding. 2026. arXiv preprint. arXiv:2604.21215

[14] Zanmin Wang. A Single-Layer Model Can Do Language Modeling. 2026. arXiv preprint. arXiv:2605.10643

[15] Guillaume Godin. SCORE: Replacing Layer Stacking with Contractive Recurrent Depth. 2026. arXiv preprint. arXiv:2603.10544

[16] Sang-Il Han. Hierarchical vs. Flat Iteration in Shared-Weight Transformers. 2026. arXiv preprint. arXiv:2604.14442

[17] Kevin Xu and Issei Sato. On Expressive Power of Looped Transformers: Theoretical Analysis and Enhancement via Timestep Encoding. 2024. arXiv preprint. arXiv:2410.01405

[18] Matthias M. M. Buehlmaier. LayerNorm as Implicit Gain Control in Looped Transformers. 2026. arXiv preprint. arXiv:2607.10681

[19] Shikhar Srivastava, Christopher Kanan. LayerRoPE: Dynamic Depth-wise Magnitude & Angular Superposition. 2026. arXiv preprint. arXiv:2610.09179

[20] Grigory Sapunov. Universal Transformers Need Memory: Depth-State Trade-offs in Adaptive Recursive Reasoning. 2026. arXiv preprint. arXiv:2604.21999

[21] Aras Kavuncu, Muhammad Burhan Hafez. Storing Is Not Remembering: LSTM-UT and Bounded Gated Memory for Looped Transformers. 2026. arXiv preprint. arXiv:2609.19521

[22] Liu Xiao. Associative-State Universal Transformers: Sparse Retrieval Meets Structured Recurrence. 2026. arXiv preprint. arXiv:2604.25930

[23] Bum Jun Kim, Kohei Hayashi, Shunsuke Kamiya, Masanori Koyama, Yusuke Iwasawa, Yutaka Matsuo. Looped Transformers with Source-Centered State Evolution. 2026. arXiv preprint. arXiv:2607.27656

[24] Pengxiang Li, Dilxat Muhtar, Di He, Guinan Su, Lu Yin, Shiwei Liu. Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals. 2026. arXiv preprint. arXiv:2610.11570

[25] Wencheng Ye, Anning Hu, Xiangdong Zhang, et al.. Write Back the Δ: Revisiting the Same Tokens with Fresh Representations. 2026. arXiv preprint. arXiv:2609.32457

[26] Boyi Zeng, Yiqin Hao, Zitong Wang, et al.. Depth-Attention: Cross-Layer Value Mixing for Language Models. 2026. arXiv preprint. arXiv:2606.05014

[27] Ziyang Cai, Xingyu Zhu, Yihe Dong, Yinghui He, Sanjeev Arora. T^2MLR: Transformer with Temporal Middle-Layer Recurrence. 2026. arXiv preprint. arXiv:2607.15178

[28] Yutong Chen, Shouqian Shi, Xinran Liu, et al.. Penelope: Localized Latent Recurrence for Efficient Structured Reasoning. 2026. arXiv preprint. arXiv:2607.25915

[29] Chengting Yu, Xiaobo Shu, Yadao Wang, et al.. SpiralFormer: Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursion. 2026. arXiv preprint. arXiv:2602.11698

[30] Sangmin Bae, Yujin Kim, Reza Bayat, et al.. Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation. 2025. arXiv preprint. arXiv:2507.10524

[31] Wenkai Chen, Tianshu Li, Wenyong Huang, Yichun Yin, Lifeng Shang, Chengwei Qin. LoopMoE: Unifying Iterative Computation with Mixture-of-Experts for Language Modeling. 2026. EMNLP 2026. arXiv:2606.04438

[32] Eric S. Qiu, Utku Umur Acikalin, Justin Lovelace, et al.. MoRE: Mixture of Reused Experts. 2026. arXiv preprint. arXiv:2609.18176

[33] Di He, Pengxiang Li, Da Chang, Qingyan Meng, Lu Yin, and Shiwei Liu. Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts. 2026. arXiv preprint. arXiv:2610.01153

[34] Markus Frey, Behzad Shomali, Ali Hamza Bashir, David Berghaus, Joachim Koehler, Mehdi Ali. Adaptive Loops and Memory in Transformers: Think Harder or Know More?. 2026. arXiv preprint. arXiv:2603.08391

[35] Yong Xien Chng, Tianyi Chen, Wenwen Tong, et al.. Looped Diffusion Transformer. 2026. arXiv preprint. arXiv:2609.40305

[36] Mohammad Mahdi Derakhshani, Pedro M. P. Curvo, Gertjan J. Burghouts, Jan-Willem van de Meent, Cees G. M. Snoek. LiFT: Loop Flow Transformers. 2026. arXiv preprint. arXiv:2610.05538

[37] Sanghyun Lee, Chunsan Hong, Seungryong Kim, Jonghyun Lee, Jongho Park, Dongmin Park. Looped Diffusion Language Models. 2026. arXiv preprint. arXiv:2605.26106

[38] Alba Carballo-Castro and Julianna Piskorz and Paulius Rauba and Mihaela van der Schaar and Pascal Frossard. Recursive Scaling in Masked Diffusion Models. 2026. arXiv preprint. arXiv:2606.18022

[39] Liancheng Fang, Zhuowei Li, Youngeun Kim, et al.. ALoDLM: Adaptively Looped Diffusion Language Models. 2026. arXiv preprint. arXiv:2610.04198

[40] Andrea Miele, Yiming Qin, Alba Carballo-Castro, Justin Deschenaux, Pascal Frossard. Fixed-Point Masked Generative Modeling. 2026. arXiv preprint. arXiv:2605.31215

[41] Shuzhen Li, Yifan Zhang, Jiacheng Guo, Quanquan Gu, Mengdi Wang. DeepLoop: Depth Scaling for Looped Transformers. 2026. arXiv preprint. arXiv:2607.13491

[42] Shaowen Wang, Bingrui Li, Ge Zhang, Wenhao Huang, Shen Yan, Jian Li. On the Residual Scaling of Looped Transformers: Stability and Transferability. 2026. arXiv preprint. arXiv:2606.18524

[43] Rao Fu, Zixuan Yang, Jiankun Zhang, Jing Ma, Hechang Chen, Yu Li, Yi Chang. Simply Stabilizing the Loop via Fully Looped Transformer. 2026. arXiv preprint. arXiv:2605.18797

[44] Alexey Zemtsov, Albina Klepach, Vadim Lomshakov, et al.. Rotate the Loop: Blockwise Conformality for Looped Transformers. 2026. arXiv preprint. arXiv:2610.14030

[45] Hsun-Yu Kuo, El Mahdi Chayti, Patrik Reizinger, Wieland Brendel, Martin Jaggi. Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping. 2026. arXiv preprint. arXiv:2606.29983

[46] Xiao-Wen Yang, Ziyu Han, Xi-Hua Zhang, et al.. Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Models. 2026. arXiv preprint. arXiv:2605.26733

[47] Jacob Fein-Ashley, Paria Rashidinejad. Solve the Loop: Attractor Models for Language and Reasoning. 2026. arXiv preprint. arXiv:2605.12466

[48] Sajad Movahedi, Vera Milovanović, Shlomo Libo Feigin, et al.. Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers. 2026. arXiv preprint. arXiv:2606.18206

[49] Benhao Huang, Chufan Shi, Junlin Chen, et al.. Towards Looped Models Done Right, Part II: Rethinking at Fixed Points. 2026. arXiv preprint. arXiv:2610.06833

[50] Andrei Marchenko, Viacheslav Bezrukov, Oleg Kashurin, et al.. Closing the Loop: Practical Training Recipes for Looped Language Models. 2026. arXiv preprint. arXiv:2610.00673

[51] Zeyi Huang, Xuehai He, LiLiang Ren, et al.. Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior. 2026. arXiv preprint. arXiv:2605.26797

[52] Shota Takashiro, Masanori Koyama, Takeru Miyato, Yusuke Iwasawa, Yutaka Matsuo, Kohei Hayashi. Exploration of Fast-Slow Latent Recurrence for Train-Short, Test-Long Generalization. 2026. arXiv preprint. arXiv:2604.01577

[53] Andrei Piterbarg, Sean McLeish, Dhruv Pai, Benjamin Keigwin. Adaptive-Depth Is a Finishing Stage. 2026. arXiv preprint. arXiv:2610.14252

[54] Jishnu Ray Chowdhury, Cornelia Caragea. Investigating Recurrent Transformers with Dynamic Halt. 2024. arXiv preprint. arXiv:2402.00976

[55] Andrei Cristian Popescu, Haitz Sáez de Ocáriz Borde, Pietro Liò. Adaptive Depth in Looped Transformers: Diagnosing Learned Halting Gates and Trajectory Readouts. 2026. arXiv preprint. arXiv:2607.20519

[56] Joe Logan. Per-Token Fixed-Point Convergence in Depth-Recurrent Transformers. 2026. arXiv preprint. arXiv:2607.14427

[57] Kristian Schwethelm, Daniel Rueckert, Georgios Kaissis. Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching. 2026. arXiv preprint. arXiv:2608.09444

[58] Mrinal Mathur, Mike Doan, Barak Pearlmutter, Sergey Plis. Change of Thought: Adaptive Test-Time Computation. 2025. arXiv preprint. arXiv:2507.13569

[59] Sumin Park, T. Konstantin Rusch. Looped Reasoning Finishes Earlier Than You Think. 2026. arXiv preprint. arXiv:2610.18174

[60] Ha Van Dau, Thanh Tung Khuat, Nguyen Thanh Dung. Beyond Depth Truncation: Controlled Evaluation of Depth Utilization in Recursive Language Models. 2026. arXiv preprint. arXiv:2609.19934

[61] Yuliana Shakhvalieva, Dmitrii Kharchev, Viacheslav Bezrukov, et al.. What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling. 2026. arXiv preprint. arXiv:2609.39967

[62] Mingqian Yu, Wenpeng Zhang, Shaobo Cui, Peilin Zhao. T-LoopFormer: Token-Level Elastic-Depth Looped Transformers for Latent Reasoning with Dynamic Routing. 2026. arXiv preprint. arXiv:2609.15160

[63] Ahmadreza Jeddi, Marco Ciccone, Babak Taati. LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation. 2026. arXiv preprint. arXiv:2602.11451

[64] Tianyu Fu, Yichen You, Zekai Chen, Guohao Dai, Huazhong Yang, Yu Wang. Think-at-Hard: Dynamic Looped Transformers for Improved Reasoning. 2025. arXiv preprint. arXiv:2511.08577

[65] Ziyue Li, Yang Li, Tianyi Zhou. Skip a Layer or Loop It? Learning Program-of-Layers in LLMs. 2026. arXiv preprint. arXiv:2606.06574

[66] Faisal Ladhak, Shantanu Acharya, Santiago Akle Serano, Simeng Sun, Boris Ginsburg, Krishna C Puvvada. One Model to Loop Them All: Elastic Latent Recurrence for Language Models. 2026. arXiv preprint. arXiv:2610.12582

[67] Shixiang Song, He Li, Zitong Wang, et al.. AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depth. 2026. arXiv preprint. arXiv:2603.01914

[68] Yichen You, Tianyu Fu, Aosong Feng, et al.. Improving Test-Time Scaling with Adaptive Looped Transformers. 2026. arXiv preprint. arXiv:2609.35748

[69] Bohong Wu, Mengzhao Chen, Xiang Luo, et al.. Parallel Loop Transformer for Efficient Test-Time Computation Scaling. 2025. arXiv preprint. arXiv:2510.24824

[70] Jonas Geiping, Xinyu Yang, Guinan Su. Efficient Parallel Samplers for Recurrent-Depth Models and Their Connection to Diffusion Language Models. 2025. arXiv preprint. arXiv:2510.14961

[71] Chris Cameron, Wangzheng Wang, Nikita Ivanov, et al.. One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models. 2026. arXiv preprint. arXiv:2604.18839

[72] SangLyul Cho, Langqing Cui, Sehoon Kim, Dongsu Han, Insu Han. LoopSpec: Pipelined Self-Speculative Decoding for Looped Transformers. 2026. arXiv preprint. arXiv:2609.17184

[73] Hyeongju Ha, Jae-Joon Kim. WaveFront Decoding: Parallelized Self-Speculative Decoding for Looped Language Models. 2026. arXiv preprint. arXiv:2609.23033

[74] Guanghao Li, Zihan Su, Hao Yu, et al.. Shallow Queries, Mature Values: Depth-Asynchronous Self-Speculation for Looped Transformers. 2026. arXiv preprint. arXiv:2609.34538

[75] Weihao Liu, Huangjie Zheng, Tianrong Chen, et al.. Decoding Looped Transformers Better for (Almost) Free. 2026. arXiv preprint. arXiv:2610.02185

[76] Angeliki Giannou, Shashank Rajput, Jy-yong Sohn, Kangwook Lee, Jason D. Lee, Dimitris Papailiopoulos. Looped Transformers as Programmable Computers. 2023. arXiv preprint. arXiv:2301.13196

[77] Ying Fan, Yilun Du, Kannan Ramchandran, Kangwook Lee. Looped Transformers for Length Generalization. 2024. arXiv preprint. arXiv:2409.15647

[78] Liu Yang, Kangwook Lee, Robert Nowak, Dimitris Papailiopoulos. Looped Transformers are Better at Learning Learning Algorithms. 2023. arXiv preprint. arXiv:2311.12424

[79] Jia Liang, Xi Jin, Liangming Pan. Beyond the Training Horizon: Mechanisms and Limits of Length Generalization in Looped Transformers. 2026. arXiv preprint. arXiv:2609.33144

[80] Huitao Yang, Jianhao Huang, Zixuan Wang, Baharan Mirzasoleiman. Learning to Extrapolate in Depth with Looped Transformers. 2026. arXiv preprint. arXiv:2610.14456

[81] Takuya Ito, Ruchir Puri, Murray Campbell, Parikshit Ram. Universal Transformers for Circuit Computations: Perfect Length Generalization in Tiny Transformers. 2026. arXiv preprint. arXiv:2608.31067

[82] Harsh Kohli, Srinivasan Parthasarathy, Huan Sun, Yuekun Yao. Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers. 2026. arXiv preprint. arXiv:2604.07822

[83] Khashayar Gatmiry, Nikunj Saunshi, Sashank J. Reddi, Stefanie Jegelka, Sanjiv Kumar. Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?. 2024. arXiv preprint. arXiv:2410.08292

[84] Bo Chen, Xiaoyu Li, Yingyu Liang, Zhenmei Shi, Zhao Song. Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent. 2024. arXiv preprint. arXiv:2410.11268

[85] Khashayar Gatmiry, Nikunj Saunshi, Sashank J. Reddi, Stefanie Jegelka, Sanjiv Kumar. On the Role of Depth and Looping for In-Context Learning with Task Diversity. 2024. arXiv preprint. arXiv:2410.21698

[86] Kevin Xu, Issei Sato. To CoT or To Loop? A Formal Comparison Between Chain-of-Thought and Looped Transformers. 2025. arXiv preprint. arXiv:2505.19245

[87] Artur Back de Luca and Kimon Fountoulakis. Simulation of Graph Algorithms with Looped Transformers. 2024. arXiv preprint. arXiv:2402.01107

[88] Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, Cyril Zhang. Transformers Learn Shortcuts to Automata. 2022. arXiv preprint. arXiv:2210.10749

[89] Jiaju Wu, Yi Hu, Muhan Zhang. Shared Weights, Selected Computations: How Looped Transformers Route What Each Loop Does. 2026. arXiv preprint. arXiv:2609.39892

[90] Georgios Kaissis, David Mildenberger, Juan Felipe Gomez, Martin J. Menten, Eleni Triantafillou. Step-resolved data attribution for looped transformers. 2026. arXiv preprint. arXiv:2602.10097

[91] Xinlin Zhuang, Siyuan Wang, Imran Razzak, Weiyang Liu. What Makes Recurrence Effective in Looped Language Models?. 2026. arXiv preprint. arXiv:2609.36636

[92] Hugh Blayney, Álvaro Arroyo, Johan Obando-Ceron, et al.. A Mechanistic Analysis of Looped Reasoning Language Models. 2026. arXiv preprint. arXiv:2604.11791

[93] Wenquan Lu, Yuechuan Yang, Kyle Lee, Yanshu Li, Enqi Liu. Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer. 2025. arXiv preprint. arXiv:2507.02199

[94] Pawel Olszowiec, Paulina Tomaszewska, Maciej Żelaszczyk, Michal Byra, Neo Christopher Chung. Monitoring latent reasoning in looped language models. 2026. arXiv preprint. arXiv:2610.18127

[95] Giovanni Monea, Keshav Ramji, Yousef El-Kurdi, et al.. The Surprising Effectiveness of Shared Memory in Looped Transformers. 2026. arXiv preprint. arXiv:2610.02383

[96] Victor Conchello Vendrell, Arnau Padres Masdemont, Niccolò Grillo, et al.. Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Models. 2026. arXiv preprint. arXiv:2605.07721

[97] Wanqi Yang, Shiwei Liu. FlashLoop: Fast and Memory-Efficient Looped Transformers via Lazy Updates. 2026. arXiv preprint. arXiv:2609.29812

[98] Zhixiong Zhao, Zukang Xu, Xing Hu, Lifeng Liu, Guangyu Sun, Dawei Yang. LoopQuant: Accurate and Efficient Post-Training Quantization for Looped Transformers. 2026. arXiv preprint. arXiv:2610.16368

[99] Adrian Bulat, Yassine Ouali, Georgios Tzimiropoulos. One Block, Multiple Depths: Recurrent Vision Transformers with Depth-Programmed Experts. 2026. arXiv preprint. arXiv:2610.12448

[100] Xun Wang, Ruishuo Chen, Yu Chen, Zhuoran Li, Longbo Huang. FlexLoop: Depth-Elastic Looped Policies for Adaptive Test-Time Computation in Deep RL. 2026. arXiv preprint. arXiv:2609.34488

[101] Shaowen Wang, Ge Zhang, Kairong Luo, et al.. SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers. 2026. arXiv preprint. arXiv:2609.01343

[102] Kristian Schwethelm, Daniel Rueckert, Georgios Kaissis. How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models. 2026. arXiv preprint. arXiv:2604.21106

[103] Hayden Prairie, Zachary Novack, Taylor Berg-Kirkpatrick, and Daniel Y. Fu. Parcae: Scaling Laws For Stable Looped Language Models. 2026. arXiv preprint. arXiv:2604.12946

[104] Leo Raphael Rodrigues, Jian Kang. Dissecting Hierarchical Reasoning Models: A Mechanistic Study. 2026. arXiv preprint. arXiv:2609.22197

[105] Tong Zhang, Junhao Hu, Yun Peng, Tao Xie. When Does Recurrence Become an Algorithm? Convergence Selection in Weight-Tied Looped Transformers. 2026. arXiv preprint. arXiv:2607.20594

[106] Wenlong Wang, Fergal Reid. Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?. 2026. arXiv preprint. arXiv:2609.01924

[107] Gunn Kim. Dynamical phase selection controls compute scaling in looped transformers. 2026. arXiv preprint. arXiv:2608.26556

[108] Darpan Aswal, Thomas Palmeira Ferraz, Yongxin Zhou, Maxime Peyrard. Observable Patterns Are Not Explanations: A Causal-Geometric Analysis of Latent Reasoning Models. 2026. arXiv preprint. arXiv:2606.12689

[109] Han Wang, Ishwar B Balappanawar, Huan Zhang. On the Chain-of-Thought Monitorability of Looped Language Models. 2026. arXiv preprint. arXiv:2610.02741

[110] Huzi Cheng, Zhewei Zhang. Not All Thinking is Created Equal: Latent Reasoning Discovers a Recurrent Search Algorithm for Depth Generalization. 2026. arXiv preprint. arXiv:2609.35643

[111] Arip Asadulaev, Rayan Banerjee, Fakhri Karray, Martin Takac. Latent Reasoning in TRMs is Secretly a Policy Improvement Operator. 2025. arXiv preprint. arXiv:2511.16886

[112] Jeffrey Lai, Anthony Bao, John Quinn, William Gilpin. Fractal basins trap latent reasoning. 2026. arXiv preprint. arXiv:2609.04963

[113] Yuto Inui, Takuya Konishi, Yoshinobu Kawahara. Invariant-Measure Reasoners: Stable Representations for Latent Reasoning. 2026. arXiv preprint. arXiv:2610.10996

[114] Francesco Pappone, Donato Crisostomi, Emanuele Rodolà. Two-Scale Latent Dynamics for Recurrent-Depth Transformers. 2025. arXiv preprint. arXiv:2509.23314

[115] Zhihao Guo, Zonghan Wu, Haizhou Du, et al.. Right Direction, Wrong Step: Geometric Analysis of Finite-Step Failure in Looped Transformers. 2026. arXiv preprint. arXiv:2609.16665

[116] James O' Neill, Fergal Reid. Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers. 2026. arXiv preprint. arXiv:2607.15456

[117] Yuhan Chen, Siyuan Zhang, Nan Wang, Feiyang Kang, Ruoxi Jia. Hybrid Latent Attention for Looped Language Models. 2026. arXiv preprint. arXiv:2610.07940

[118] Bo Liu, Qiang Liu. Maglev: Sliding Recurrent Memory. 2026. arXiv preprint. arXiv:2608.02870

[119] Wen-Jie Shu, Xuerui Qiu, Rui-Jie Zhu, Harold Haodong Chen, Yexin Liu and Harry Yang. LoopViT: Scaling Visual ARC with Looped Transformers. 2026. arXiv preprint. arXiv:2602.02156

[120] Zhe Qian, Ziyang Gong, Zhongxing Xu, et al.. LoopVL: Recurrent Visual Intelligence. 2026. arXiv preprint. arXiv:2609.38426

[121] Renjie He. RD-ViT: Recurrent-Depth Vision Transformer for Semantic Segmentation with Reduced Data Dependence. 2026. arXiv preprint. arXiv:2605.03999

[122] Michal Byra, Pawel Olszowiec, Grzegorz Stefanski, Grzegorz Gruszczynski, Alberto Presta. Investigating Single-Block Recurrence in Vision Transformers for Image Recognition. 2026. arXiv preprint. arXiv:2605.10661

[123] Junseo Kim, Uraz Odyurt, Amirreza Yousefzadeh. MergeOver: Post-Training Token Merging for Recursive Vision Transformers. 2026. arXiv preprint. arXiv:2608.13141

[124] Zhiying Lu. LoopVAE: Recurrent Depth Across Scales for Visual Tokenization. 2026. arXiv preprint. arXiv:2609.11516

[125] Yang Chen, Canyu Shen, Xinzhe Rao, et al.. GramLoop: Training-Free Gram-Gated Replay for Robust Dense Prediction. 2026. arXiv preprint. arXiv:2608.29113

[126] Zipeng Wang, Anping Xiong, Jing Zhang, et al.. LoopREG: Representation Supervision Enables Test-Time Recurrent Depth in Diffusion Transformers. 2026. arXiv preprint. arXiv:2610.17877

[127] Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, Łukasz Kaiser. Universal Transformers. 2019. International Conference on Learning Representations (ICLR). arXiv:1807.03819 (external — not present in the library collection used for this survey)

[128] Alex Graves. Adaptive Computation Time for Recurrent Neural Networks. 2016. arXiv preprint. arXiv:1603.08983 (external — not present in the library collection used for this survey)

[129] Shaojie Bai, J. Zico Kolter, Vladlen Koltun. Deep Equilibrium Models. 2019. Advances in Neural Information Processing Systems (NeurIPS). arXiv:1909.01377 (external — not present in the library collection used for this survey)