1. Introduction: Breaking the Efficiency Bottleneck of dLLMs

Diffusion-based Large Language Models (dLLMs) offer a compelling non-autoregressive alternative to standard Transformers, but they suffer from a critical efficiency flaw. Unlike Autoregressive models that rely on causal masking to easily cache past computations, dLLMs utilize bidirectional attention, requiring the model to recompute Key-Value (KV) states for the entire sequence at every denoising step. This computational redundancy creates a severe bottleneck, making inference prohibitively slow as sequence length increases.

To bridge this gap, we introduce d²Cache, a training-free framework designed to accelerate dLLMs without retraining. By employing a dual adaptive selection mechanism, d²Cache dynamically identifies and updates only the most critical tokens at each step while caching the rest, effectively simulating a quasi left-to-right generation process. Our experiments demonstrate that d²Cache not only achieves substantial inference speedups (up to 4.8×) but also consistently improves generation quality by stabilizing the diffusion process against premature errors.

2. Preliminary

2.1 Generation process of dLLMs

Unlike Autoregressive models that generate text token-by-token, Diffusion-based LLMs (dLLMs) model the generation as a reverse denoising process. Starting from pure noised sequence \(x_T\), the model iteratively removes noise to recover the target text \(x_0\). At each step \(t\), the model predicts the clean data estimate \(\hat{x}_0\) based on the current noisy state \(x_t\): ​

However, as shown in Figure 1, this iterative generation process cannot use a standard KV cache, which severely reduces inference throughput.

Figure 1. (a) In ARMs, causal attention requires each token to interact only with its preceding tokens. (b) In dLLMs, bidirectional attention requires each token to attend to both its preceding and subsequent tokens, such that any modification in the subsequent tokens necessitates recomputation of the entire sequence.

2.2 Key Observations

Previous methods, such as Fast-dLLM, found significant cosine similarity between KV states in adjacent steps. Therefore, the computational cost of a single forward step could be reduced by periodically refreshing certain sequence segments. To find a more fine-grained refresh strategy, we have the following three observations:

Figure 2. (a) PCA of the trajectory of the 77th masked token on LLaMA-8B-Instruct with GSM8K (sequence length 328, generation length 256, generate in 256 steps). (b) Sequential distances between token pairs decoded in adjacent steps, accumulated from 64 GSM8K samples.

Figure 3. Attention rollout analysis over sequence, where the example and setting are the same as in Figure 2. (a) Attention rollout visualization at step 126, showing the sum of rollout values over all key positions (top) and the pairwise rollout values across different positions (bottom). (b) The total absolute differences in rollout values between each two adjacent decoding steps.

3. d²Cache: Dual Adaptive Cache

Motivated by the previous observations, we present Dual aDaptive Cache (d²Cache), a training-free approximate KV cache framework for accelerating dLLM inference. To exploit approximate KV cache in dLLMs, d²Cache seeks to adaptively identify tokens whose KV states should be actively updated at each decoding step, while caching the remaining tokens for reuse in a subsequent decoding step.

We first group tokens in dLLMs into three categories: prompt tokens, masked tokens, and decoded tokens. Based on this categorization, we introduce a two-stage fine-grained token selection strategy. ❶ Certainty prior-guided selection from masked tokens. After each forward pass, d²Cache assigns each masked token a score combining prediction confidence and a certainty prior, which represents the density of known tokens (i.e., prompt or decoded ones) in the local context. d²Cache then adaptively selects a subset of masked tokens with higher scores. In light of this, d²Cache naturally enables an alternative decoding scheme guided by the certainty prior rather than prediction confidence alone, yielding higher reliability. ❷ Attention-aware selection from remaining tokens. Furthermore, for the remaining tokens (especially prompt and decoded tokens), d²Cache adaptively selects a subset of tokens with higher attention activations. Finally, for the tokens selected in these two stages, d²Cache updates their KV states at each decoding step, while caching the KV states of the remaining tokens for reuse in a subsequent decoding step.

For formal descriptions, please refer to Section 4 of the original paper.

4. Experimental Results

4.1 Experimental Configuration

We evaluate d²Cache on two representative dLMMs: LLaDA-8B and Dream-v0-7B, referred to as LLaDA-Base/Inst and Dream-Base/Inst. Experiments are conducted on six benchmark datasets: GSM8k, MBPP, Humaneval, Math-500, GPQA and MMLU-Pro. For fair comparisons, we report both inference throughput and latency, where throughput denotes the average number of tokens generated per second and latency denotes the average inference time per sample. All experiments are performed on NVIDIA 3090 24GB GPUs.

Compared Methods

We compare d²Cache with the following baseline methods:

• Vanilla: At each decoding step, the masked position with the highest confidence is replaced with its predicted token.
• dLLM-Cache: The prompt segment and response segment are refreshed at different time intervals, and the cache in the response segment is dynamically refreshed according to changes in V states.
• Fast-dLLM: During semi-autoregressive generation, only the current block and subsequent blocks are recalculated. After each block is generated, the entire sequence is recalculated, and the prompt and completed blocks are cached.
• d²Cache: Our method, which dynamically selects the tokens that need to be updated at the token level at each step.

4.2 Main Results

Performance Comparison

Comprehensive evaluation results on LLaDA-Inst and Dream-Inst.

Bold numbers denote best results and blue texts denote the speedup ratios.

Recommended citation: Jiang Y, Cai Y, Luo X, et al. d $^ 2$ Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching[J]. arXiv preprint arXiv:2509.23094, 2025.
Download Paper