Skip to content
AIBRAINWORX / OpenSourceCoder.comLOCAL-FIRST. BUILT IN THE OPEN.

SOURCE REVIEWED / 2026-11-20

NVIDIA TensorRT LLM

Inspect NVIDIA GPU inference scheduling, cache management, precision formats and serving.

Where it fits

Read exact versioned runtime and platform requirements

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

License and practical limits

Apache-2.0 default; specified NVIDIA kernels and third-party exceptions. Read exact platform support and migration guide. Rubin support is preview; specified kernel directories use NVIDIA software terms. No local performance or device-fit test.

Source reviewed; not independently installed or tested. No cloud service or hardware purchase is required to read the source.

Repository snapshot

15,108 stars and 2,947 forks when checked 2026-11-20. These numbers change and are not a security or quality endorsement.

Read the official repository ↗

Before you run it

Read installation requirements and release notes, verify the download source, and keep your own projects backed up. Review third-party plugins and model licenses separately.

Pass the signal.