specific method · filed under inference & serving
Latency-sensitive execution class with priority isolation
A resource-management method that isolates latency-sensitive tasks through scheduling priority and core scheduling.
Also called a latency-sensitive (LS) execution class.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
DSec therefore introduces a latency-sensitive (LS) execution class. We apply SCHED_IDLE to non-LS tasks to minimize their scheduling priority, and use core scheduling to ensure only tasks of the same priority class execute simultaneously on sibling hyperthreads to eliminate interference.
usedsoftware implementationin DeepSeek Elastic Compute (DSec)DeepSeek
Filed alongside
Other methods under inference & serving :: inference scheduling.
Asynchronous schedulingNUMA binding of workers to GPU-local CPU socketsComposite-key sorting for topology-aware GPU rank assignmentContinuous batchingCross-group pinning of cache-hit blocksDual-cluster consistent-hash failover for cache affinityEnvoy-based proxy with custom orchestratorExacto routingGreedy feasible-rank placement by remaining capacityHost-side scheduling optimizationIncreasing batch size for inferenceMax sequences tuningPer-node hard admission constraintPrefix-cache-aware session affinity schedulingRegistering NVLink domains as Ray custom resourcesRequest-class resource-budget admission controlRuntime-signal-based rollout concurrency auto-throttlingSub-NUMA partitioning with per-VM NUMA bindingWorkload-aware routed-expert GEMM scheduling