specific method · filed under software implementation
Mooncake
Disaggregated inference infrastructure that separates prefill and decode across different node pools.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
Moonshot's own serving uses the Mooncake disaggregated inference infrastructure, which separates prefill and decode across different node pools
usedinference servingin Kimi K3 serving systemMoonshot AI
Filed alongside
Other methods under software implementation :: infrastructure service.
Automatic provider failoverCustom placement engineDeepSeek Elastic Compute (DSec)Dependency unificationDurable peer mailboxEnd-to-end lineageFresh-clone repository sanitizationHTTP/HTTPS sidecar proxyIn-house batch schedulerIncremental sandbox checkpointing and resumptionLocal container-image cachingModel FactoryNeMo GuardrailsNeMo SwitchyardPer-node pooling of initialization actorsRayReplacing short-lived Ray actors with tasksSandbox pause and resumeVendoring external dependenciesWrite-ahead logZ3 SMT solver integration