specific method · filed under inference & serving
Request-class resource-budget admission control
An admission-control method that assigns separate resource budgets to request classes to limit each class's capacity use.
Also called budget-based admission control.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
core 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
We therefore adopt budget-based admission control, allocating separate resource budgets to different request classes so that bursty long-context traffic consumes at most its own share of the capacity
coreinference servingin Kimi K3Moonshot AI
Filed alongside
Other methods under inference & serving :: inference scheduling.
Asynchronous schedulingNUMA binding of workers to GPU-local CPU socketsComposite-key sorting for topology-aware GPU rank assignmentContinuous batchingCross-group pinning of cache-hit blocksDual-cluster consistent-hash failover for cache affinityEnvoy-based proxy with custom orchestratorExacto routingGreedy feasible-rank placement by remaining capacityHost-side scheduling optimizationIncreasing batch size for inferenceLatency-sensitive execution class with priority isolationMax sequences tuningPer-node hard admission constraintPrefix-cache-aware session affinity schedulingRegistering NVLink domains as Ray custom resourcesRuntime-signal-based rollout concurrency auto-throttlingSub-NUMA partitioning with per-VM NUMA bindingWorkload-aware routed-expert GEMM scheduling