Open Models Almanac
Methodssoftware implementationinference engine

specific method · filed under software implementation

TensorRT-LLM

TensorRT-LLM is an open source library.1 It is built to deliver high-performance, real-time inference optimization for large language models on NVIDIA GPUs.1

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. TensorRT™-LLM is an open source library built to deliver high-performance, real-time inference optimization for large language models like Nemotron on NVIDIA GPUs — NVIDIA Nemotron AI Models

Also called high-throughput serving, TensorRT-LLM inference optimization, TRTLLM.

sources
3
models
3
labs adopt it
2
strongest
used

Documented in

Evidence

4 quotes from 3 sources, one card per model, each opening with its strongest.

How sources treat it
optional 3used 1
↑