TensorRT™-LLM is an open source library built to deliver high-performance, real-time inference optimization for large language models
specific method · filed under software implementation
TensorRT-LLM
TensorRT-LLM is an open source library.1 It is built to deliver high-performance, real-time inference optimization for large language models on NVIDIA GPUs.1
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
TensorRT™-LLM is an open source library built to deliver high-performance, real-time inference optimization for large language models like Nemotron on NVIDIA GPUs
— NVIDIA Nemotron AI Models
Also called high-throughput serving, TensorRT-LLM inference optimization, TRTLLM.
Documented in
Evidence
4 quotes from 3 sources, one card per model, each opening with its strongest.
How sources treat it
optional 3used 1
usedinference servingin NVIDIA TensorRT-LLMNVIDIA
Mistral-Large-4
Mistral AIthe open source TensorRT LLM library for high-throughput serving.
optionalunclearin Mistral Large 4Mistral AI
VLLM_FLASHINFER_MOE_BACKEND=latency(TRTLLM-Gen)
optionalsoftware implementationin Nemotron 3 UltraNVIDIA
1 more from this model
TRT-LLM deployment snippet provided
optionalsoftware implementationin Nemotron 3 UltraNVIDIA