SOURCE REVIEWED / 2026-11-20
NVIDIA TensorRT LLM
Inspect NVIDIA GPU inference scheduling, cache management, precision formats and serving.
Where it fits
Read exact versioned runtime and platform requirements
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
License and practical limits
Apache-2.0 default; specified NVIDIA kernels and third-party exceptions. Read exact platform support and migration guide. Rubin support is preview; specified kernel directories use NVIDIA software terms. No local performance or device-fit test.
Source reviewed; not independently installed or tested. No cloud service or hardware purchase is required to read the source.
Repository snapshot
15,108 stars and 2,947 forks when checked 2026-11-20. These numbers change and are not a security or quality endorsement.
Read the official repository ↗
Before you run it
Read installation requirements and release notes, verify the download source, and keep your own projects backed up. Review third-party plugins and model licenses separately.