specific method · filed under model architecture
Mamba
Mamba is a token-mixing architecture that, during autoregressive decoding, stores its SSM state in a constant-sized cache per batch element.1 Unlike attention's KV cache, this state does not grow with sequence length, though it can still exceed the KV cache at short sequence lengths.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
In Mamba autoregressive decoding, the SSM state is stored in a constant-sized cache for each batch element
— Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic ReasoningThe KV cache size, on the other hand, grows with sequence length. However, there is a crossover point below which the Mamba cache size is larger than the KV cache size
— Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning