single mention · model architecture
Cross-layer attention sharing
Cross-layer attention sharing is a KV-cache sharing method in which some attention layers reuse the keys and values produced by earlier layers instead of maintaining their own.1 It acts as a KV-cache compression mechanism that reduces what must be stored, with affected layers taking a similar residual stream input to the producing layer.2,3
Trade-off Removing CLA sacrifices inference optimizations in exchange for model performance closer to a full-attention model.4
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
it is more difficult to remove CLA and YOCO KV-cache sharing: layers that previously relied on keys and values produced by earlier layers then need to compute and maintain their own KV caches
— technical reportThe goal was to remove KV-cache compression mechanisms
— technical reportwhich is close to the affected layer and takes a very similar residual stream as input
— technical reportThe goal was to remove KV-cache compression mechanisms to achieve an architecture as close as possible to a full-attention model. Although this approach temporarily sacrificed inference optimizations, we prioritized model performance
— technical report
Also called Cross-layer attention sharing (CLA), attention sharing method.
Filed under model architecture › token mixer