single mention · inference & serving
You Only Cache Once (YOCO)
You Only Cache Once (YOCO) is a KV-cache sharing scheme in which some attention layers reuse the keys and values produced by an earlier layer instead of maintaining their own caches.1,2 In the described architecture, the KV producer sits at the model's halfway point, with consumer layers in the second half drawing on that cached representation.3,4
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
layers that previously relied on keys and values produced by earlier layers then need to compute and maintain their own KV caches
— technical reporttheir second-half YOCO consumers continued to reuse the KV representation from the first half
— technical reportthe preceding KV-producing long layer at the model’s halfway point
— technical reportpreserving YOCO sharing among the original global-attention layers in the second half of the network
— technical report
Also called You Only Cache Once (YOCO) KV-cache reuse, YOCO.
Filed under inference & serving › KV cache management