NVIDIA-NeMo/Guardrails

What it is

NVIDIA NeMo Guardrails (nemoguardrails, Apache 2.0) is a Python library for putting programmable rails between application code and an LLM. The latest release on the page is 0.24.1. It supports Python 3.10 to 3.13. The develop branch tracks top-of-tree development.

How it works

Integration replaces direct LLM calls with a guardrails layer. You load a config folder and call generate or generate_async. The message format follows the OpenAI Chat Completions API.

from nemoguardrails import LLMRails, RailsConfig
config = RailsConfig.from_path("PATH/TO/CONFIG")
rails = LLMRails(config)
completion = rails.generate(
    messages=[{"role": "user", "content": "Hello world!"}]
)

A config folder holds config.yml (models and active rails), config.py (init code), actions.py (custom Python actions) and .co files (Colang flows). Active rails are listed by flow name under rails.input and rails.output in config.yml, for example check jailbreak, mask sensitive data on input, self check facts and self check hallucination. Masked entity types are set under sensitive_data_detection, for example PERSON and EMAIL_ADDRESS.

A config with no rails forwards requests to the LLM unchanged.

Colang 1.0 is the default. Colang 2.0 is supported but must be chosen.

What to do

Install, then run as a server if you want a network boundary:

pip install nemoguardrails
nemoguardrails server [--config PATH/TO/CONFIGS] [--port PORT]

The server exposes POST /v1/chat/completions and selects a config with config_id. A Dockerfile is provided. Other CLI commands are chat, evaluate and actions-server.

nemoguardrails evaluate covers topical rails, fact-checking, moderation (jailbreak and output) and hallucination. The repository includes a sample vulnerability scan of the ABC Bot to compare configurations.

LangChain integration is opt-in. Set the variable below or call set_default_framework("langchain"), then install the LangChain packages your config needs:

export NEMOGUARDRAILS_LLM_FRAMEWORK=langchain

To disable telemetry, use any one of these before the library starts:

export NEMO_GUARDRAILS_NO_USAGE_STATS=1
# or
export DO_NOT_TRACK=1
# or
mkdir -p ~/.config/nemoguardrails && touch ~/.config/nemoguardrails/do_not_track

Caveats

Telemetry is on by default. One event is sent when LLMRails, IORails or Guardrails is instantiated, followed by periodic heartbeats from a daemon thread. The payload includes:

• library, Python and OS versions;
• the Colang version;
• LLM provider names, but not model names;
• rail counts and active built-in feature names;
• the deployment mode;
• a per-process random UUID.

The page says prompts, completions, API keys, endpoints, file paths and IP addresses are not in the payload. Setting the opt-out after startup does not stop a heartbeat thread that is already running. The library also tries to write each event to ~/.config/nemoguardrails/usage_stats.json. The opt-out covers only the library, not any third-party inference endpoint you use.

The page says the built-in rails "may or may not be suitable" for a given production use case and must be validated per application. It also says NVIDIA Build endpoints are for evaluation only and not for production. It describes safety evaluation as an open research question and gives no detection-rate figures in the fetched text.