Subliminal Clocks: How Diffusion Language Models Track Hidden Time

Diffusion Language Models generate text by starting with noise and iteratively refining it toward coherent language—but unlike traditional diffusion models, they don't receive explicit timestep information. The central finding here is surprising: these models secretly encode what stage of denoising they're in within their internal neural activations, even without being told. This hidden "clock" guides their generation process and can be externally manipulated to control model behavior.

The Hidden Signal

The authors trained probes—lightweight classifiers—to predict the diffusion timestep from the model's internal activations across different layers. The fact that these probes worked reliably means the information about denoising progress is genuinely present and accessible in the model's residual streams (the pathways where information flows between layers). This wasn't obvious: the model never sees explicit timestep conditioning during training or inference, yet learns to track progress anyway.

The geometry of this representation is interpretable. Rather than being scattered randomly across activations, the timestep information occupies a structured, low-dimensional subspace—a thin slice of the high-dimensional activation space. This suggests the model organizes denoising progress in a compact, systematic way.

Steering the Model's Clock

Understanding where and how this signal lives enabled a powerful manipulation: the authors identified the subspace representing timestep information and systematically shifted model activations along it. By pushing the model's activations to "think" it was at an earlier or later denoising stage than it actually was, they could predictably change the model's behavior. Early stages generate high-entropy (uncertain, exploratory) tokens; later stages produce high-confidence, refined completions.

This steering directly affected two measurable quantities: the model's confidence in its predictions and the entropy of its output distribution. Tweaking the latent clock made the model more or less certain about its choices in predictable ways, demonstrating causal control over a fundamental aspect of generation.

Why This Matters

The discovery reveals how DLMs solve a critical self-awareness problem: without explicit timestep conditioning, they must infer where they are in the generation process. The existence of a robust, decodable internal clock suggests this is solved through implicit learning. For practitioners, it opens a path to finer-grained control over generation behavior—adjusting confidence or diversity without retraining.

More broadly, the work contributes to mechanistic interpretability of diffusion models: understanding what models represent internally and how those representations drive behavior. As DLMs become competitive with autoregressive models, knowing how they internally scaffold the generation process becomes crucial for safety, debugging, and alignment. The fact that this information is recoverable through simple probes and geometrically structured suggests that diffusion language models may be more interpretable than previously assumed.