Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/about/architecture/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Architecture Guide

This document provides more details on the architecture and the approach that the NeMo Guardrails toolkit takes for implementing guardrails.
This document provides more details on the architecture and the approach that the NeMo Guardrails Library takes for implementing guardrails.

![Overall Architecture](overall-architecture.png)

## The Guardrails Process

This section explains in detail the process under the hood, from the utterance sent by the user to the bot utterance that is returned.

The guardrails runtime uses an event-driven design (i.e., an event loop that processes events and generates back other events). Whenever the user says something to the bot, a `UtteranceUserActionFinished` event is created and sent to the runtime.
The guardrails runtime uses an event-driven design (i.e., an event loop that processes events and generates back other events). Whenever the user says something to the bot, an `UtteranceUserActionFinished` event is created and sent to the runtime.

The process has three main stages:

Expand Down Expand Up @@ -69,7 +69,7 @@ Regardless of the path taken, there are two categories of next steps:

When an action needs to be executed, the runtime will invoke the action and wait for the result. When the action finishes, an `InternalSystemActionFinished` event is created with the result of the action.

**Note**: the default implementation of the runtime is async, so the action execution is only blocking for a specific user.
**Note**: The default implementation of the runtime is async, so the action execution is only blocking for a specific user.

When the bot should say something, the process will move to the next stage, i.e., generating the bot utterance.

Expand Down Expand Up @@ -246,13 +246,13 @@ Notice the various sections included in the prompt: the general instruction, the

## Interaction with LLMs

This toolkit relies on LangChain for the interaction with LLMs. Below is a high-level sequence diagram showing the interaction between the user's code (the one using the guardrails), the `LLMRails`, LangChain and the LLM API.
This library relies on LangChain for the interaction with LLMs. Below is a high-level sequence diagram showing the interaction between the user's code (the one using the guardrails), the `LLMRails`, LangChain and the LLM API.

![Sequence Diagram LLMRails](sequence-diagram-llmrails.png)

## Server Architecture

This toolkit provides a guardrails server with an interface similar to publicly available LLM APIs. Using the server, integrating a guardrails configuration in your application can be as easy as replacing the initial LLM API URL with the Guardrails Server API URL.
This library provides a guardrails server with an interface similar to publicly available LLM APIs. Using the server, integrating a guardrails configuration in your application can be as easy as replacing the initial LLM API URL with the Guardrails Server API URL.

![Guardrails Server](guardrails-server.png)

Expand Down
4 changes: 2 additions & 2 deletions docs/about/how-it-works/guardrails-process.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ The following diagram depicts the guardrails process in detail:

The guardrails process has multiple stages that a user message goes through:

1. **Input Validation stage**: The user input is first processed by the input rails. The input rails decide if the input is allowed, whether it should be altered or rejected.
1. **Input Validation stage**: The user input is first processed by the input rails. The input rails decide if the input is allowed, or whether it should be altered or rejected.
2. **Dialog stage**: If the input is allowed and the configuration contains dialog rails (i.e., at least one user message is defined), then the user message is processed by the dialog flows. This will ultimately result in a bot message.
3. **Output Validation stage**: After a bot message is generated by the dialog rails, it is processed by the output rails. The Output rails decide if the output is allowed, whether it should be altered, or rejected.
3. **Output Validation stage**: After a bot message is generated by the dialog rails, it is processed by the output rails. The Output rails decide if the output is allowed, or whether it should be altered or rejected.

## The Dialog Rails Flow

Expand Down
4 changes: 2 additions & 2 deletions docs/about/how-it-works/how-rails-work.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
---
title: Rails Overview
description: Learn how the NeMo Guardrails toolkit applies guardrails at multiple stages of the LLM interaction.
description: Learn how the NeMo Guardrails Library applies guardrails at multiple stages of the LLM interaction.
---

# Rails Overview

The NeMo Guardrails toolkit applies guardrails at multiple stages of the LLM interaction.
The NeMo Guardrails Library applies guardrails at multiple stages of the LLM interaction.

| Stage | Rail Type | Common Use Cases |
|-------|-----------|------------------|
Expand Down
12 changes: 6 additions & 6 deletions docs/about/how-it-works/index.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
---
title: How It Works
description: Learn how the NeMo Guardrails toolkit works.
description: Learn how the NeMo Guardrails Library works.
---

# How It Works

The NeMo Guardrails toolkit is for building guardrails for your LLM applications. It provides a set of tools and libraries for building guardrails for your LLM applications.
The NeMo Guardrails Library is for building guardrails for your LLM applications. It provides a set of tools and libraries for building guardrails for your LLM applications.

Read the following pages to learn more about how the toolkit works and how you can use it to build a guardrails system for your LLM applications.
Read the following pages to learn more about how the library works and how you can use it to build a guardrails system for your LLM applications.

::::{grid} 1 1 2 2
:gutter: 3
Expand All @@ -16,21 +16,21 @@ Read the following pages to learn more about how the toolkit works and how you c
:link: how-rails-work
:link-type: doc

Learn how the NeMo Guardrails toolkit applies guardrails at multiple stages of the LLM interaction.
Learn how the NeMo Guardrails Library applies guardrails at multiple stages of the LLM interaction.
:::

:::{grid-item-card} Guardrails Sequence Diagrams
:link: guardrails-process
:link-type: doc

This guide provides an overview of the process of invoking guardrails.
Get an overview of the process of invoking guardrails.
:::

:::{grid-item-card} Architecture Guide
:link: ../architecture/README
:link-type: doc

This document provides more details on the architecture and the approach that the NeMo Guardrails toolkit takes for implementing guardrails.
Learn about the architecture and the approach that the NeMo Guardrails Library takes for implementing guardrails.
:::

::::
Expand Down
38 changes: 19 additions & 19 deletions docs/about/overview.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
---
title: Overview
description: Learn about the NeMo Guardrails toolkit and its capabilities.
description: Learn about the NeMo Guardrails Library and its capabilities.
---

<!--
SPDX-FileCopyrightText: Copyright (c) 2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0
-->

# Overview of the NeMo Guardrails Toolkit
# Overview of the NeMo Guardrails Library

The NVIDIA NeMo Guardrails toolkit is an open-source Python package for adding programmable guardrails to LLM-based applications. It intercepts inputs and outputs, applies configurable safety checks, and blocks or modifies content based on defined policies.
The NVIDIA NeMo Guardrails library is an open-source Python package for adding programmable guardrails to LLM-based applications. It intercepts inputs and outputs, applies configurable safety checks, and blocks or modifies content based on defined policies.

```{mermaid}
%%{init: {'theme': 'neutral', 'themeVariables': { 'background': 'transparent' }}}%%

flowchart TB
A("Application Code")
B("NeMo Guardrails Toolkit")
B("NeMo Guardrails Library")
C("Large Language Model (LLM)")

A <--> B
Expand All @@ -40,26 +40,26 @@ end
style F fill:#f0f7e6,stroke:#76b900
```

*Application code interacting with LLMs through the NeMo Guardrails toolkit.*
*Application code interacting with LLMs through the NeMo Guardrails Library.*

---

## What You Can Do with the NeMo Guardrails Toolkit
## What You Can Do with the NeMo Guardrails Library

The following are the top use cases of the NeMo Guardrails toolkit that you can apply to protect your LLM applications.
The following are the top use cases of the NeMo Guardrails Library that you can apply to protect your LLM applications.

::::{grid} 1 1 2 2
:gutter: 3
:class-container: sd-equal-height

:::{grid-item-card} Text Content Safety
:::{grid-item-card} Content Safety - Text
:link: ../getting-started/tutorials/nemotron-safety-guard-deployment
:link-type: doc

Deploy Nemotron Safety Guard to detect harmful content in text inputs and outputs.
Deploy Nemotron Content Safety to detect harmful content in text inputs and outputs.
:::

:::{grid-item-card} Multimodal Content Safety
:::{grid-item-card} Content Safety - Multimodal
:link: ../getting-started/tutorials/multimodal
:link-type: doc

Expand All @@ -70,26 +70,26 @@ Add safety checks to images and text using vision models as LLM-as-a-judge.
:link: ../getting-started/tutorials/nemoguard-jailbreakdetect-deployment
:link-type: doc

Deploy NemoGuard Jailbreak Detection NIM to block adversarial prompts.
Deploy NemoGuard Jailbreak Detect to block adversarial prompts.
:::

:::{grid-item-card} Topic Control
:link: ../getting-started/tutorials/nemoguard-topiccontrol-deployment
:link-type: doc

Deploy NemoGuard Topic Control NIM to restrict conversations to allowed topics.
Deploy NemoGuard Topic Control to restrict conversations to allowed topics.
:::

:::{grid-item-card} PII Handling
Identify and mask Personally Identifiable Information in inputs and outputs using regex patterns, Presidio integration, or custom detection logic.
Identify and mask personally identifiable information (PII) in inputs and outputs using regex patterns, Presidio integration, or custom detection logic.
:::

:::{grid-item-card} Knowledge Base / RAG
In RAG scenarios, verify LLM responses against retrieved source documents to detect unsupported claims or hallucinations.
:::

:::{grid-item-card} Agentic Workflows
Apply execution rails to secure LLM agents that perform multi-step reasoning or interact with external systems. Validate agent decisions, restrict allowed actions, and enforce policies before execution proceeds.
Apply execution rails to LLM agents that perform multi-step reasoning or interact with external systems. Validate agent decisions, restrict allowed actions, and enforce policies before execution proceeds.
:::

:::{grid-item-card} Tool Integration
Expand All @@ -102,7 +102,7 @@ Validate inputs and outputs when the LLM calls external tools or APIs. Execution

## Tools

The following are the tools you can use to interact with the NeMo Guardrails toolkit.
The following are the tools you can use to interact with the NeMo Guardrails Library.

### Python SDK

Expand All @@ -129,15 +129,15 @@ The server exposes an HTTP API compatible with OpenAI's `/v1/chat/completions` e

---

## Toolkit vs Microservice
## Library vs Microservice

This documentation covers the open-source NeMo Guardrails toolkit. The NeMo Guardrails Microservice is a separate product that packages the same core functionality for Kubernetes deployment.
This documentation covers the open-source NeMo Guardrails Library. The NeMo Guardrails Microservice is a separate product that packages the same core functionality for Kubernetes deployment.

| | Toolkit | Microservice |
| | Library | Microservice |
|------------------|----------------------------------|----------------------------------|
| Distribution | PyPI (`pip install`) | Container image |
| Deployment | Self-managed | Kubernetes with Helm |
| Scaling | Application-level | Managed by orchestrator |
| Configuration | Same YAML/Colang format | Same YAML/Colang format |

Configurations are portable between the toolkit and microservice.
Configurations are portable between the library and microservice.
18 changes: 9 additions & 9 deletions docs/about/supported-llms.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
---
title: Supported LLMs and Providers
description: Browse the LLMs and their providers supported by the toolkit.
description: Browse the LLMs and their providers supported by the NeMo Guardrails Library.
---

# Supported LLMs

The NeMo Guardrails toolkit supports a wide range of LLM providers and their models.
The NeMo Guardrails Library supports a wide range of LLM providers and their models.

## LLM Providers

The toolkit supports the following LLM providers.
The NeMo Guardrails Library supports the following LLM providers:

### NVIDIA NIM

The toolkit supports NVIDIA NIM microservices for local deployment and NVIDIA API Catalog for hosted models.
The NeMo Guardrails Library supports NVIDIA NIM microservices for local deployment and NVIDIA API Catalog for hosted models.

- **Locally-deployed LLM NIM Microservices**: LLMs deployed on your own infrastructure.
- **NVIDIA API Catalog**: Hosted LLMs on [build.nvidia.com](https://build.nvidia.com/models).
- **Specialized NIM Microservices**: NemoGuard Content Safety, Topic Control, and Jailbreak Detection.
- **Specialized NIM Microservices**: Nemo Content Safety, NeMo Topic Control, and NeMo Jailbreak Detect.

### External LLM Providers

The toolkit supports the following external LLM providers.
The NeMo Guardrails Library supports the following external LLM providers:

- OpenAI
- Azure OpenAI
Expand All @@ -31,7 +31,7 @@ The toolkit supports the following external LLM providers.

### Self-Hosted

The toolkit supports the following self-hosted LLM providers.
The NeMo Guardrails Library supports the following self-hosted LLM providers:

- HuggingFace Hub
- HuggingFace Endpoints
Expand All @@ -40,11 +40,11 @@ The toolkit supports the following self-hosted LLM providers.

### Providers from LangChain Community

The toolkit supports any LLM providers from the LangChain Community. Refer to [All integration providers](https://docs.langchain.com/oss/python/integrations/providers/all_providers) in the LangChain documentation.
The NeMo Guardrails Library supports any LLM provider from the LangChain Community. Refer to [All integration providers](https://docs.langchain.com/oss/python/integrations/providers/all_providers) in the LangChain documentation.

## Embedding Providers

The toolkit supports the following embedding providers.
The NeMo Guardrails Library supports the following embedding providers:

- NVIDIA NIM
- FastEmbed
Expand Down
Loading