Open Models Almanac
Methodsinference & servingKV cache management

single mention · inference & serving

You Only Cache Once (YOCO)

You Only Cache Once (YOCO) is a KV-cache sharing scheme in which some attention layers reuse the keys and values produced by an earlier layer instead of maintaining their own caches.1,2 In the described architecture, the KV producer sits at the model's halfway point, with consumer layers in the second half drawing on that cached representation.3,4

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. layers that previously relied on keys and values produced by earlier layers then need to compute and maintain their own KV caches — technical report
  2. their second-half YOCO consumers continued to reuse the KV representation from the first half — technical report
  3. the preceding KV-producing long layer at the model’s halfway point — technical report
  4. preserving YOCO sharing among the original global-attention layers in the second half of the network — technical report

Also called You Only Cache Once (YOCO) KV-cache reuse, YOCO.

Filed under inference & serving › KV cache management