diff --git a/docs/about/architecture/README.md b/docs/about/architecture/README.md index 050b8d4fd..40b164229 100644 --- a/docs/about/architecture/README.md +++ b/docs/about/architecture/README.md @@ -1,6 +1,6 @@ # Architecture Guide -This document provides more details on the architecture and the approach that the NeMo Guardrails toolkit takes for implementing guardrails. +This document provides more details on the architecture and the approach that the NeMo Guardrails Library takes for implementing guardrails. ![Overall Architecture](overall-architecture.png) @@ -8,7 +8,7 @@ This document provides more details on the architecture and the approach that th This section explains in detail the process under the hood, from the utterance sent by the user to the bot utterance that is returned. -The guardrails runtime uses an event-driven design (i.e., an event loop that processes events and generates back other events). Whenever the user says something to the bot, a `UtteranceUserActionFinished` event is created and sent to the runtime. +The guardrails runtime uses an event-driven design (i.e., an event loop that processes events and generates back other events). Whenever the user says something to the bot, an `UtteranceUserActionFinished` event is created and sent to the runtime. The process has three main stages: @@ -69,7 +69,7 @@ Regardless of the path taken, there are two categories of next steps: When an action needs to be executed, the runtime will invoke the action and wait for the result. When the action finishes, an `InternalSystemActionFinished` event is created with the result of the action. -**Note**: the default implementation of the runtime is async, so the action execution is only blocking for a specific user. +**Note**: The default implementation of the runtime is async, so the action execution is only blocking for a specific user. When the bot should say something, the process will move to the next stage, i.e., generating the bot utterance. @@ -246,13 +246,13 @@ Notice the various sections included in the prompt: the general instruction, the ## Interaction with LLMs -This toolkit relies on LangChain for the interaction with LLMs. Below is a high-level sequence diagram showing the interaction between the user's code (the one using the guardrails), the `LLMRails`, LangChain and the LLM API. +This library relies on LangChain for the interaction with LLMs. Below is a high-level sequence diagram showing the interaction between the user's code (the one using the guardrails), the `LLMRails`, LangChain and the LLM API. ![Sequence Diagram LLMRails](sequence-diagram-llmrails.png) ## Server Architecture -This toolkit provides a guardrails server with an interface similar to publicly available LLM APIs. Using the server, integrating a guardrails configuration in your application can be as easy as replacing the initial LLM API URL with the Guardrails Server API URL. +This library provides a guardrails server with an interface similar to publicly available LLM APIs. Using the server, integrating a guardrails configuration in your application can be as easy as replacing the initial LLM API URL with the Guardrails Server API URL. ![Guardrails Server](guardrails-server.png) diff --git a/docs/about/how-it-works/guardrails-process.md b/docs/about/how-it-works/guardrails-process.md index c8c6d6c52..a61ce186a 100644 --- a/docs/about/how-it-works/guardrails-process.md +++ b/docs/about/how-it-works/guardrails-process.md @@ -12,9 +12,9 @@ The following diagram depicts the guardrails process in detail: The guardrails process has multiple stages that a user message goes through: -1. **Input Validation stage**: The user input is first processed by the input rails. The input rails decide if the input is allowed, whether it should be altered or rejected. +1. **Input Validation stage**: The user input is first processed by the input rails. The input rails decide if the input is allowed, or whether it should be altered or rejected. 2. **Dialog stage**: If the input is allowed and the configuration contains dialog rails (i.e., at least one user message is defined), then the user message is processed by the dialog flows. This will ultimately result in a bot message. -3. **Output Validation stage**: After a bot message is generated by the dialog rails, it is processed by the output rails. The Output rails decide if the output is allowed, whether it should be altered, or rejected. +3. **Output Validation stage**: After a bot message is generated by the dialog rails, it is processed by the output rails. The Output rails decide if the output is allowed, or whether it should be altered or rejected. ## The Dialog Rails Flow diff --git a/docs/about/how-it-works/how-rails-work.md b/docs/about/how-it-works/how-rails-work.md index 56d77ee0c..8cf77091f 100644 --- a/docs/about/how-it-works/how-rails-work.md +++ b/docs/about/how-it-works/how-rails-work.md @@ -1,11 +1,11 @@ --- title: Rails Overview -description: Learn how the NeMo Guardrails toolkit applies guardrails at multiple stages of the LLM interaction. +description: Learn how the NeMo Guardrails Library applies guardrails at multiple stages of the LLM interaction. --- # Rails Overview -The NeMo Guardrails toolkit applies guardrails at multiple stages of the LLM interaction. +The NeMo Guardrails Library applies guardrails at multiple stages of the LLM interaction. | Stage | Rail Type | Common Use Cases | |-------|-----------|------------------| diff --git a/docs/about/how-it-works/index.md b/docs/about/how-it-works/index.md index ed66ec6b8..eee6608f6 100644 --- a/docs/about/how-it-works/index.md +++ b/docs/about/how-it-works/index.md @@ -1,13 +1,13 @@ --- title: How It Works -description: Learn how the NeMo Guardrails toolkit works. +description: Learn how the NeMo Guardrails Library works. --- # How It Works -The NeMo Guardrails toolkit is for building guardrails for your LLM applications. It provides a set of tools and libraries for building guardrails for your LLM applications. +The NeMo Guardrails Library is for building guardrails for your LLM applications. It provides a set of tools and libraries for building guardrails for your LLM applications. -Read the following pages to learn more about how the toolkit works and how you can use it to build a guardrails system for your LLM applications. +Read the following pages to learn more about how the library works and how you can use it to build a guardrails system for your LLM applications. ::::{grid} 1 1 2 2 :gutter: 3 @@ -16,21 +16,21 @@ Read the following pages to learn more about how the toolkit works and how you c :link: how-rails-work :link-type: doc -Learn how the NeMo Guardrails toolkit applies guardrails at multiple stages of the LLM interaction. +Learn how the NeMo Guardrails Library applies guardrails at multiple stages of the LLM interaction. ::: :::{grid-item-card} Guardrails Sequence Diagrams :link: guardrails-process :link-type: doc -This guide provides an overview of the process of invoking guardrails. +Get an overview of the process of invoking guardrails. ::: :::{grid-item-card} Architecture Guide :link: ../architecture/README :link-type: doc -This document provides more details on the architecture and the approach that the NeMo Guardrails toolkit takes for implementing guardrails. +Learn about the architecture and the approach that the NeMo Guardrails Library takes for implementing guardrails. ::: :::: diff --git a/docs/about/overview.md b/docs/about/overview.md index 7e664cc80..0511b6ea8 100644 --- a/docs/about/overview.md +++ b/docs/about/overview.md @@ -1,6 +1,6 @@ --- title: Overview -description: Learn about the NeMo Guardrails toolkit and its capabilities. +description: Learn about the NeMo Guardrails Library and its capabilities. --- -# Overview of the NeMo Guardrails Toolkit +# Overview of the NeMo Guardrails Library -The NVIDIA NeMo Guardrails toolkit is an open-source Python package for adding programmable guardrails to LLM-based applications. It intercepts inputs and outputs, applies configurable safety checks, and blocks or modifies content based on defined policies. +The NVIDIA NeMo Guardrails library is an open-source Python package for adding programmable guardrails to LLM-based applications. It intercepts inputs and outputs, applies configurable safety checks, and blocks or modifies content based on defined policies. ```{mermaid} %%{init: {'theme': 'neutral', 'themeVariables': { 'background': 'transparent' }}}%% flowchart TB A("Application Code") - B("NeMo Guardrails Toolkit") + B("NeMo Guardrails Library") C("Large Language Model (LLM)") A <--> B @@ -40,26 +40,26 @@ end style F fill:#f0f7e6,stroke:#76b900 ``` -*Application code interacting with LLMs through the NeMo Guardrails toolkit.* +*Application code interacting with LLMs through the NeMo Guardrails Library.* --- -## What You Can Do with the NeMo Guardrails Toolkit +## What You Can Do with the NeMo Guardrails Library -The following are the top use cases of the NeMo Guardrails toolkit that you can apply to protect your LLM applications. +The following are the top use cases of the NeMo Guardrails Library that you can apply to protect your LLM applications. ::::{grid} 1 1 2 2 :gutter: 3 :class-container: sd-equal-height -:::{grid-item-card} Text Content Safety +:::{grid-item-card} Content Safety - Text :link: ../getting-started/tutorials/nemotron-safety-guard-deployment :link-type: doc -Deploy Nemotron Safety Guard to detect harmful content in text inputs and outputs. +Deploy Nemotron Content Safety to detect harmful content in text inputs and outputs. ::: -:::{grid-item-card} Multimodal Content Safety +:::{grid-item-card} Content Safety - Multimodal :link: ../getting-started/tutorials/multimodal :link-type: doc @@ -70,18 +70,18 @@ Add safety checks to images and text using vision models as LLM-as-a-judge. :link: ../getting-started/tutorials/nemoguard-jailbreakdetect-deployment :link-type: doc -Deploy NemoGuard Jailbreak Detection NIM to block adversarial prompts. +Deploy NemoGuard Jailbreak Detect to block adversarial prompts. ::: :::{grid-item-card} Topic Control :link: ../getting-started/tutorials/nemoguard-topiccontrol-deployment :link-type: doc -Deploy NemoGuard Topic Control NIM to restrict conversations to allowed topics. +Deploy NemoGuard Topic Control to restrict conversations to allowed topics. ::: :::{grid-item-card} PII Handling -Identify and mask Personally Identifiable Information in inputs and outputs using regex patterns, Presidio integration, or custom detection logic. +Identify and mask personally identifiable information (PII) in inputs and outputs using regex patterns, Presidio integration, or custom detection logic. ::: :::{grid-item-card} Knowledge Base / RAG @@ -89,7 +89,7 @@ In RAG scenarios, verify LLM responses against retrieved source documents to det ::: :::{grid-item-card} Agentic Workflows -Apply execution rails to secure LLM agents that perform multi-step reasoning or interact with external systems. Validate agent decisions, restrict allowed actions, and enforce policies before execution proceeds. +Apply execution rails to LLM agents that perform multi-step reasoning or interact with external systems. Validate agent decisions, restrict allowed actions, and enforce policies before execution proceeds. ::: :::{grid-item-card} Tool Integration @@ -102,7 +102,7 @@ Validate inputs and outputs when the LLM calls external tools or APIs. Execution ## Tools -The following are the tools you can use to interact with the NeMo Guardrails toolkit. +The following are the tools you can use to interact with the NeMo Guardrails Library. ### Python SDK @@ -129,15 +129,15 @@ The server exposes an HTTP API compatible with OpenAI's `/v1/chat/completions` e --- -## Toolkit vs Microservice +## Library vs Microservice -This documentation covers the open-source NeMo Guardrails toolkit. The NeMo Guardrails Microservice is a separate product that packages the same core functionality for Kubernetes deployment. +This documentation covers the open-source NeMo Guardrails Library. The NeMo Guardrails Microservice is a separate product that packages the same core functionality for Kubernetes deployment. -| | Toolkit | Microservice | +| | Library | Microservice | |------------------|----------------------------------|----------------------------------| | Distribution | PyPI (`pip install`) | Container image | | Deployment | Self-managed | Kubernetes with Helm | | Scaling | Application-level | Managed by orchestrator | | Configuration | Same YAML/Colang format | Same YAML/Colang format | -Configurations are portable between the toolkit and microservice. +Configurations are portable between the library and microservice. diff --git a/docs/about/supported-llms.md b/docs/about/supported-llms.md index e2fe2e32c..4f24d13fa 100644 --- a/docs/about/supported-llms.md +++ b/docs/about/supported-llms.md @@ -1,27 +1,27 @@ --- title: Supported LLMs and Providers -description: Browse the LLMs and their providers supported by the toolkit. +description: Browse the LLMs and their providers supported by the NeMo Guardrails Library. --- # Supported LLMs -The NeMo Guardrails toolkit supports a wide range of LLM providers and their models. +The NeMo Guardrails Library supports a wide range of LLM providers and their models. ## LLM Providers -The toolkit supports the following LLM providers. +The NeMo Guardrails Library supports the following LLM providers: ### NVIDIA NIM -The toolkit supports NVIDIA NIM microservices for local deployment and NVIDIA API Catalog for hosted models. +The NeMo Guardrails Library supports NVIDIA NIM microservices for local deployment and NVIDIA API Catalog for hosted models. - **Locally-deployed LLM NIM Microservices**: LLMs deployed on your own infrastructure. - **NVIDIA API Catalog**: Hosted LLMs on [build.nvidia.com](https://build.nvidia.com/models). -- **Specialized NIM Microservices**: NemoGuard Content Safety, Topic Control, and Jailbreak Detection. +- **Specialized NIM Microservices**: Nemo Content Safety, NeMo Topic Control, and NeMo Jailbreak Detect. ### External LLM Providers -The toolkit supports the following external LLM providers. +The NeMo Guardrails Library supports the following external LLM providers: - OpenAI - Azure OpenAI @@ -31,7 +31,7 @@ The toolkit supports the following external LLM providers. ### Self-Hosted -The toolkit supports the following self-hosted LLM providers. +The NeMo Guardrails Library supports the following self-hosted LLM providers: - HuggingFace Hub - HuggingFace Endpoints @@ -40,11 +40,11 @@ The toolkit supports the following self-hosted LLM providers. ### Providers from LangChain Community -The toolkit supports any LLM providers from the LangChain Community. Refer to [All integration providers](https://docs.langchain.com/oss/python/integrations/providers/all_providers) in the LangChain documentation. +The NeMo Guardrails Library supports any LLM provider from the LangChain Community. Refer to [All integration providers](https://docs.langchain.com/oss/python/integrations/providers/all_providers) in the LangChain documentation. ## Embedding Providers -The toolkit supports the following embedding providers. +The NeMo Guardrails Library supports the following embedding providers: - NVIDIA NIM - FastEmbed diff --git a/docs/about/use-cases.md b/docs/about/use-cases.md index cefc18b13..20a3f3f06 100644 --- a/docs/about/use-cases.md +++ b/docs/about/use-cases.md @@ -1,11 +1,11 @@ --- title: Use Cases -description: Browse the different use cases of the NeMo Guardrails toolkit. +description: Browse the different use cases of the NeMo Guardrails Library. --- # Use Cases -The NeMo Guardrails toolkit supports a wide range of use cases for protecting LLM-based applications. +The NeMo Guardrails Library supports a wide range of use cases for protecting LLM-based applications. The following sections describe the primary use cases. --- @@ -13,24 +13,24 @@ The following sections describe the primary use cases. ## Content Safety Content safety guardrails help ensure that both user inputs and LLM outputs are safe and appropriate. -The NeMo Guardrails toolkit provides multiple approaches to content safety: +The NeMo Guardrails Library provides multiple approaches to content safety: - **LLM self-checking**: Use the LLM itself to check inputs and outputs for harmful content. -- **NVIDIA safety models**: Integration with [Llama 3.1 NemoGuard 8B Content Safety](https://build.nvidia.com/nvidia/llama-3_1-nemoguard-8b-content-safety) for robust content moderation. -- **Community models**: Support for [LlamaGuard](../user-guides/community/llama-guard.md), [Fiddler Guardrails](../user-guides/community/fiddler.md), and other content safety solutions. -- **Third-party APIs**: Integration with [ActiveFence](../user-guides/guardrails-library.md#activefence), [Cisco AI Defense](../user-guides/community/ai-defense.md), and other moderation services. +- **NVIDIA safety models**: Integrate with [Llama 3.1 NemoGuard 8B Content Safety](https://build.nvidia.com/nvidia/llama-3_1-nemoguard-8b-content-safety) for robust content moderation. +- **Community models**: Use [LlamaGuard](../user-guides/community/llama-guard.md), [Fiddler Guardrails](../user-guides/community/fiddler.md), and other community content safety solutions. +- **Third-party APIs**: Integrate with [ActiveFence](../user-guides/guardrails-library.md#activefence), [Cisco AI Defense](../user-guides/community/ai-defense.md), and other moderation services. For more information, refer to the [Content Safety section](../user-guides/guardrails-library.md#content-safety) in the Guardrails Library and the [Getting Started guide](../getting-started/index.md). ## Jailbreak Protection -Jailbreak detection helps prevent adversarial attempts to bypass safety measures and manipulate the LLM into generating harmful or unwanted content. -The NeMo Guardrails toolkit provides multiple layers of jailbreak protection: +Jailbreak protection helps prevent adversarial attempts from bypassing safety measures and manipulating the LLM into generating harmful or unwanted content. +The NeMo Guardrails Library provides multiple layers of jailbreak protection: - **Self-check jailbreak detection**: Use the LLM to identify jailbreak attempts. -- **Heuristic detection**: Pattern-based detection of common jailbreak techniques. -- **NVIDIA NemoGuard**: Integration with [NemoGuard Jailbreak Detection NIM](../getting-started/tutorials/nemoguard-jailbreakdetect-deployment.md) for advanced threat detection. -- **Third-party integrations**: Support for [Prompt Security](../user-guides/community/prompt-security.md), [Pangea AI Guard](../user-guides/community/pangea.md), and other services. +- **Heuristic detection**: Use pattern-based detection for common jailbreak techniques. +- **NVIDIA NemoGuard**: Integrate with [NemoGuard Jailbreak Detection NIM](../getting-started/tutorials/nemoguard-jailbreakdetect-deployment.md) for advanced threat detection. +- **Third-party integrations**: Use [Prompt Security](../user-guides/community/prompt-security.md), [Pangea AI Guard](../user-guides/community/pangea.md), and other services. For more information, refer to the [Jailbreak Detection section](../user-guides/guardrails-library.md#jailbreak-detection) in the Guardrails Library and [LLM Vulnerability Scanning](../evaluation/llm-vulnerability-scanning.md). @@ -39,21 +39,21 @@ For more information, refer to the [Jailbreak Detection section](../user-guides/ Topic control guardrails ensure that conversations stay within predefined subject boundaries and prevent the LLM from engaging in off-topic discussions. This is implemented through: -- **Dialog rails**: Pre-defined conversational flows using the Colang language. +- **Dialog rails**: Pre-define conversational flows using the Colang language. - **Topical rails**: Control what topics the bot can and cannot discuss. -- **NVIDIA NemoGuard**: Integration with [NemoGuard Topic Control NIM](../getting-started/tutorials/nemoguard-topiccontrol-deployment.md) for semantic topic detection. +- **NVIDIA NemoGuard**: Integrate with [NemoGuard Topic Control NIM](../getting-started/tutorials/nemoguard-topiccontrol-deployment.md) for semantic topic detection. For more information, refer to the [Topical Rails tutorial](../getting-started/6-topical-rails/README.md) and [Colang Language Syntax Guide](../user-guides/colang-language-syntax-guide.md). ## PII Detection Personally Identifiable Information (PII) detection helps protect user privacy by detecting and masking sensitive data in user inputs, LLM outputs, and retrieved content. -The NeMo Guardrails toolkit supports PII detection through multiple integrations: +The NeMo Guardrails Library supports PII detection through multiple integrations: -- **Presidio-based detection**: Built-in support using [Microsoft Presidio](../user-guides/community/presidio.md) for detecting entities such as names, email addresses, phone numbers, social security numbers, and more. -- **Private AI**: Integration with [Private AI](../user-guides/community/privateai.md) for advanced PII detection and masking. -- **AutoAlign**: Support for [AutoAlign PII detection](../user-guides/community/auto-align.md) with customizable entity types. -- **GuardrailsAI**: Access to [GuardrailsAI PII validators](../user-guides/community/guardrails-ai.md) from the Guardrails Hub. +- **Presidio-based detection**: Use [Microsoft Presidio](../user-guides/community/presidio.md) for detecting entities such as names, email addresses, phone numbers, social security numbers, and more. +- **Private AI**: Integrate with [Private AI](../user-guides/community/privateai.md) for advanced PII detection and masking. +- **AutoAlign**: Use [AutoAlign PII detection](../user-guides/community/auto-align.md) with customizable entity types. +- **GuardrailsAI**: Access [GuardrailsAI PII validators](../user-guides/community/guardrails-ai.md) from the Guardrails Hub. PII detection can be configured to either detect and block content containing PII or to mask PII entities before processing. @@ -64,10 +64,10 @@ For more information, refer to the [Presidio Integration](../user-guides/communi Agentic security provides specialized guardrails for LLM-based agents that use tools and interact with external systems. This includes: -- **Tool call validation**: Execution rails that validate tool inputs and outputs before and after invocation. -- **Agent workflow protection**: Integration with [LangGraph](../integration/langchain/langgraph-integration.md) for multi-agent safety. -- **Secure tool integration**: Guidelines for safely connecting LLMs to external resources (refer to [Security Guidelines](../security/guidelines.md)). -- **Action monitoring**: Detailed logging and tracing of agent actions. +- **Tool call validation**: Execute rails that validate tool inputs and outputs before and after invocation. +- **Agent workflow protection**: Integrate with [LangGraph](../integration/langchain/langgraph-integration.md) for multi-agent safety. +- **Secure tool integration**: Review guidlines for safety connecting LLMs to external resources (refer to [Security Guidelines](../security/guidelines.md)). +- **Action monitoring**: Monitor detailed logging and tracing of agent actions. Key security considerations for agent systems: @@ -80,9 +80,9 @@ For more information, refer to the [Tools Integration Guide](../integration/tool ## Custom Rails -The NeMo Guardrails toolkit provides extensive flexibility for creating custom guardrails tailored to your specific requirements: +The NeMo Guardrails Library provides extensive flexibility for creating custom guardrails tailored to your specific requirements. -### Custom Rails into Guardrails +### Add Custom Rails into Guardrails You can create custom rails using one or more of the following approaches: @@ -134,7 +134,7 @@ You can create custom rails using one or more of the following approaches: ### Integrate Guardrails into LLM-based Applications -The NeMo Guardrails toolkit can be integrated into applications in multiple ways: +The NeMo Guardrails Library can be integrated into applications in multiple ways: 1. **Python SDK integration**: Add guardrails directly into your Python application. diff --git a/docs/getting-started/installation-guide.md b/docs/getting-started/installation-guide.md index 704b00432..0231a0ee0 100644 --- a/docs/getting-started/installation-guide.md +++ b/docs/getting-started/installation-guide.md @@ -1,11 +1,11 @@ --- title: Install -description: Install the toolkit with pip and set up your environment. +description: Install the NeMo Guardrails Library with pip and set up your environment. --- # Installation Guide -This guide walks you through the following steps to install the NeMo Guardrails toolkit. +This guide walks you through the following steps to install the NeMo Guardrails Library. 1. Check the requirements. 2. Set up a fresh virtual environment. @@ -16,16 +16,16 @@ This guide walks you through the following steps to install the NeMo Guardrails ## Requirements -Review the following requirements to install the NeMo Guardrails toolkit. +Review the following requirements to install the NeMo Guardrails Library. | Requirement Type | Details | |----------------------|----------------------------------------------------------------------------------------------------------------------------------------------| -| **Hardware** | The toolkit runs on CPUs (no GPUs required).
It acts as a process manager between your app front-end and the backend LLM. | +| **Hardware** | The library runs on CPUs (no GPUs required).
It acts as a process manager between your app front-end and the backend LLM. | | **Software** | Python 3.10, 3.11, 3.12, or 3.13 | ### Additional Dependencies -The NeMo Guardrails toolkit uses [annoy](https://github.com/spotify/annoy), which is a C++ library with Python bindings. To install it, you need to have a valid C++ runtime on your computer. +The NeMo Guardrails Library uses [annoy](https://github.com/spotify/annoy), which is a C++ library with Python bindings. To install it, you need to have a valid C++ runtime on your computer. Most systems already have installed a C++ runtime. If the **annoy** installation fails due to a missing C++ runtime, you can install a C++ runtime as follows: #### Installing a C++ runtime on Linux, Mac, or Unix-based OS @@ -40,7 +40,7 @@ Install the [Microsoft C++ Build Tools](https://visualstudio.microsoft.com/visua ## Setting up a virtual environment -To experiment with the NeMo Guardrails toolkit from scratch, use a fresh virtual environment. Otherwise, you can skip to the following section. +To experiment with the NeMo Guardrails Library from scratch, use a fresh virtual environment. Otherwise, you can skip to the following section. ### Setting up a virtual environment on Linux, Mac, or Unix-based OS @@ -74,7 +74,7 @@ Use the `mkvirtualenv` *name* command to activate a new virtual environment call ## Install the NeMo Guardrails Toolkit -Install the NeMo Guardrails toolkit using **pip**: +Install the NeMo Guardrails Library using **pip**: ```sh pip install nemoguardrails @@ -82,7 +82,7 @@ Install the NeMo Guardrails toolkit using **pip**: ## Installing from source code -The NeMo Guardrails toolkit is under active development and the main branch always contains the latest development version. To install from source: +The NeMo Guardrails Library is under active development and the main branch always contains the latest development version. To install from source: 1. Clone the repository: @@ -103,11 +103,11 @@ The `nemoguardrails` package also defines the following extra dependencies: - `dev`: packages required by some extra Guardrails features for developers, such as the **autoreload** feature. - `eval`: packages used for the Guardrails [evaluation tools](../../nemoguardrails/evaluate/README.md). -- `openai`: installs the latest `openai` package supported by the NeMo Guardrails toolkit. -- `sdd`: packages used by the [sensitive data detector](../user-guides/guardrails-library.md#sensitive-data-detection) integrated in the NeMo Guardrails toolkit. +- `openai`: installs the latest `openai` package supported by the NeMo Guardrails Library. +- `sdd`: packages used by the [sensitive data detector](../user-guides/guardrails-library.md#sensitive-data-detection) integrated in the NeMo Guardrails Library. - `all`: installs all extra packages. -To keep the footprint of `nemoguardrails` as small as possible, these are not installed by default. To install any of the extra dependency you can use **pip** as well. For example, to install the `dev` extra dependencies, run the following command: +To keep the footprint of `nemoguardrails` as small as possible, these are not installed by default. To install any of the extra dependencies you can use **pip** as well. For example, to install the `dev` extra dependencies, run the following command: ```sh > pip install nemoguardrails[dev] @@ -133,12 +133,12 @@ as shown in the following example, where *YOUR_KEY* is your OpenAI key. export OPENAI_API_KEY=YOUR_KEY ``` -Some NeMo Guardrails toolkit LLMs and features have specific installation requirements, including a more complex set of steps. For example, [AlignScore](../user-guides/advanced/align_score_deployment.md) fact-checking, using [Llama-2](../../examples/configs/llm/hf_pipeline_llama2/README.md) requires two additional packages. +Some NeMo Guardrails Library LLMs and features have specific installation requirements, including a more complex set of steps. For example, [AlignScore](../user-guides/advanced/align_score_deployment.md) fact-checking using [Llama-2](../../examples/configs/llm/hf_pipeline_llama2/README.md) requires two additional packages. For each feature or LLM example, check the readme file associated with it. ## Using Docker -The NeMo Guardrails toolkit can also be used through Docker. For details on how to build and use the Docker image see [NeMo Guardrails with Docker](../user-guides/advanced/using-docker.md). +The NeMo Guardrails Library can also be used through Docker. For details on how to build and use the Docker image see [NeMo Guardrails with Docker](../user-guides/advanced/using-docker.md). ## What's next? diff --git a/docs/getting-started/tutorials/index.md b/docs/getting-started/tutorials/index.md index 6b279c281..8cd090ae8 100644 --- a/docs/getting-started/tutorials/index.md +++ b/docs/getting-started/tutorials/index.md @@ -5,33 +5,33 @@ description: Follow hands-on tutorials to build your first guardrails configurat # Tutorials -This section contains tutorials that help you get started with NeMo Guardrails Toolkit. +This section contains tutorials that help you get started with the NeMo Guardrails Library. ::::{grid} 1 1 2 2 :gutter: 3 -:::{grid-item-card} Text Content Safety +:::{grid-item-card} Content Safety - Text :link: nemotron-safety-guard-deployment :link-type: doc -Deploy Nemotron Safety Guard to detect harmful content in text inputs and outputs. +Deploy Nemo Safety Guard to detect harmful content in text inputs and outputs. ::: :::{grid-item-card} Topic Control :link: nemoguard-topiccontrol-deployment :link-type: doc -Deploy NemoGuard Topic Control NIM to restrict conversations to allowed topics. +Deploy Nemo Topic Control to restrict conversations to allowed topics. ::: :::{grid-item-card} Jailbreak Detection :link: nemoguard-jailbreakdetect-deployment :link-type: doc -Deploy NemoGuard Jailbreak Detection NIM to block adversarial prompts. +Deploy Nemo Jailbreak Detect to block adversarial prompts. ::: -:::{grid-item-card} Multimodal Content Safety +:::{grid-item-card} Content Safety - Multimodal :link: multimodal :link-type: doc diff --git a/docs/getting-started/tutorials/multimodal.md b/docs/getting-started/tutorials/multimodal.md index 82401f820..b889a7e4e 100644 --- a/docs/getting-started/tutorials/multimodal.md +++ b/docs/getting-started/tutorials/multimodal.md @@ -12,7 +12,7 @@ description: Add safety checks to images and text using vision models as LLM-as- ## About Working with Multimodal Data -NeMo Guardrails toolkit supports adding safety checks to multimodal content---images and text. +NeMo Guardrails Library supports adding safety checks to multimodal content---images and text. The support is for input and output guardrails only. Depending on the image reasoning model, you can specify the image to check as a base64 encoded data or as a URL. diff --git a/docs/getting-started/tutorials/nemoguard-jailbreakdetect-deployment.md b/docs/getting-started/tutorials/nemoguard-jailbreakdetect-deployment.md index 0f8fd9f60..f7da5e6c8 100644 --- a/docs/getting-started/tutorials/nemoguard-jailbreakdetect-deployment.md +++ b/docs/getting-started/tutorials/nemoguard-jailbreakdetect-deployment.md @@ -1,11 +1,11 @@ --- title: Jailbreak Detection -description: Deploy NemoGuard Jailbreak Detection NIM to block adversarial prompts. +description: Deploy NeMo Jailbreak Detect to block adversarial prompts. --- -# Jailbreak Detection with NemoGuard NIM +# Jailbreak Detection with NeMo Jailbreak Detect -The NemoGuard Jailbreak Detect model is available via the [Jailbreak Detection Container](jailbreak-detection-deployment.md) or as an [NVIDIA NIM](https://docs.nvidia.com/nim/#nemoguard). +NeMo Jailbreak Detect is available via the [Jailbreak Detection Container](jailbreak-detection-deployment.md) or as an [NVIDIA NIM](https://docs.nvidia.com/nim/#nemoguard). ## NIM Deployment diff --git a/docs/getting-started/tutorials/nemoguard-topiccontrol-deployment.md b/docs/getting-started/tutorials/nemoguard-topiccontrol-deployment.md index eff458d26..215814206 100644 --- a/docs/getting-started/tutorials/nemoguard-topiccontrol-deployment.md +++ b/docs/getting-started/tutorials/nemoguard-topiccontrol-deployment.md @@ -1,17 +1,17 @@ --- title: Topic Control -description: Deploy NemoGuard Topic Control NIM to restrict conversations to allowed topics. +description: Deploy NeMo Topic Control to restrict conversations to allowed topics. --- -# Topic Control with NemoGuard NIM +# Topic Control with NeMo Topic Control -The TopicControl model is available to download as a LoRA adapter module through Hugging Face or as an [NVIDIA TopicControl NIM microservice](https://docs.nvidia.com/nim/llama-3-1-nemoguard-8b-topiccontrol/latest/index.html) for low-latency optimized inference with [NVIDIA TensorRT-LLM](https://docs.nvidia.com/tensorrt-llm/index.html). +The Topic Control model is available to download as a LoRA adapter module through Hugging Face or as an [NVIDIA TopicControl NIM microservice](https://docs.nvidia.com/nim/llama-3-1-nemoguard-8b-topiccontrol/latest/index.html) for low-latency optimized inference with [NVIDIA TensorRT-LLM](https://docs.nvidia.com/tensorrt-llm/index.html). -This guide covers how to deploy the TopicControl model as a NIM microservice and use it in a NeMo Guardrails configuration. +This guide covers how to deploy NeMo Topic Control as a NIM microservice and use it in a NeMo Guardrails configuration. ## NIM Deployment -Follow the instructions below to deploy the TopicControl NIM microservice and configure it in a NeMo Guardrails application. +Follow the instructions below to deploy NeMo Topic Control and configure it in a NeMo Guardrails application. ### Access @@ -44,9 +44,9 @@ docker run -it --name=$MODEL_NAME \ $NIM_IMAGE ``` -### Use TopicControl NIM Microservice in NeMo Guardrails App +### Use NeMo Topic Control in the NeMo Guardrails App -A locally running TopicControl NIM microservice exposes the standard OpenAI interface on the `v1/chat/completions` endpoint. NeMo Guardrails provides out-of-the-box support for engines that support the standard LLM interfaces. In Guardrails configuration, use the engine `nim` for the TopicControl NIM microservice as follows. +A locally running Topic Control NIM exposes the standard OpenAI interface on the `v1/chat/completions` endpoint. NeMo Guardrails provides out-of-the-box support for engines that support the standard LLM interfaces. In the Guardrails configuration, use the engine `nim` for the Topic Control NIM as follows. ```yaml models: @@ -68,7 +68,7 @@ rails: A few things to note: -- `parameters.base_url` should contain the IP address of the machine the NIM was hosted on, the port should match the tunnel forwarding port specified in the docker run command. +- `parameters.base_url` should contain the IP address of the machine the NIM was hosted on, and the port should match the tunnel forwarding port specified in the docker run command. - `parameters.model_name` in the Guardrails configuration needs to match the `$MODEL_NAME` used when running the NIM container. - The `rails` definitions should list `topic_control` as the model. @@ -76,7 +76,9 @@ A few things to note: If you'd like to not build TRTLLM engines from scratch every time you run the NIM container, you can cache it in the first run by just adding a flag to mount a local directory inside the docker to store the model cache. -To achieve this, you simply need to mount the folder containing the cached TRTLLM assets onto the docker container while running it using `-v $LOCAL_NIM_CACHE:/opt/nim/.cache`. See below instructions for the full command. Important: make sure that docker has permissions to write to the cache folder (`sudo chmod 666 $LOCAL_NIM_CACHE`). +To achieve this, you simply need to mount the folder containing the cached TRTLLM assets onto the docker container while running it using `-v $LOCAL_NIM_CACHE:/opt/nim/.cache`. See below instructions for the full command. + +Important: Make sure that docker has permissions to write to the cache folder (`sudo chmod 666 $LOCAL_NIM_CACHE`). ```bash ### To bind a $LOCAL_NIM_CACHE folder to "/opt/nim/.cache" @@ -99,10 +101,10 @@ docker run -it --name=$MODEL_NAME \ $NIM_IMAGE ``` -## More details on TopicControl model +## More details on NeMo Topic Control -For more details on the TopicControl model, check out the other resources: +For more details on NeMo Topic Control, check out these resources: -- NeMo Guardrails library for [NVIDIA NemoGuard models](../guardrails-library.md#nvidia-models) -- TopicControl topic safety example [configuration and prompts](https://github.com/NVIDIA/NeMo-Guardrails/tree/develop/examples/configs/topic_safety) +- NeMo Guardrails Library for [NVIDIA NemoGuard models](../guardrails-library.md#nvidia-models) +- NeMo Topic Control topic safety example [configuration and prompts](https://github.com/NVIDIA/NeMo-Guardrails/tree/develop/examples/configs/topic_safety) - [Paper at EMNLP 2024](https://arxiv.org/abs/2404.03820) diff --git a/docs/getting-started/tutorials/nemotron-safety-guard-deployment.md b/docs/getting-started/tutorials/nemotron-safety-guard-deployment.md index 4bb2f9e4c..62dc9051d 100644 --- a/docs/getting-started/tutorials/nemotron-safety-guard-deployment.md +++ b/docs/getting-started/tutorials/nemotron-safety-guard-deployment.md @@ -1,6 +1,6 @@ --- -title: Text Content Safety -description: Deploy Nemotron Safety Guard to detect harmful content in text inputs and outputs. +title: Content Safety - Text +description: Deploy NeMo Content Safety to detect harmful content in text inputs and outputs. --- -# Text Content Safety with Nemotron Safety Guard NIM +# Content Safety (Text) with NeMo Content Safety ## Adding Multilingual Content Safety Guardrails diff --git a/docs/index.md b/docs/index.md index a0779c640..ab2b4c47b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -3,13 +3,13 @@ SPDX-License-Identifier: Apache-2.0 --> -# NVIDIA NeMo Guardrails Toolkit Developer Guide +# NVIDIA NeMo Guardrails Library Developer Guide -The NeMo Guardrails toolkit is an open-source Python package for adding programmable guardrails to LLM-based applications. It intercepts inputs and outputs, applies configurable safety checks, and blocks or modifies content based on defined policies. +The NeMo Guardrails library is an open-source Python package for adding programmable guardrails to LLM-based applications. It intercepts inputs and outputs, applies configurable safety checks, and blocks or modifies content based on defined policies. -## About the NeMo Guardrails Toolkit +## About the NeMo Guardrails Library -Learn about the toolkit and its capabilities in the following sections. +Learn about the library and its capabilities in the following sections. ::::{grid} 1 1 2 2 :gutter: 3 @@ -18,35 +18,35 @@ Learn about the toolkit and its capabilities in the following sections. :link: about/overview :link-type: doc -Learn about the NeMo Guardrails toolkit and its capabilities. +Learn about the NeMo Guardrails library and its capabilities. ::: :::{grid-item-card} Use Cases :link: about/use-cases :link-type: doc -Browse the different use cases of the NeMo Guardrails toolkit. +Browse the different use cases of the NeMo Guardrails library. ::: :::{grid-item-card} How It Works :link: about/how-it-works/index :link-type: doc -Learn how the NeMo Guardrails toolkit works. +Learn how the NeMo Guardrails library works. ::: :::{grid-item-card} Supported LLMs and Providers :link: about/supported-llms :link-type: doc -Browse the LLMs and their providers supported by the toolkit. +Browse the LLMs and their providers supported by the library. ::: :::: ## Get Started -Follow these steps to start using the NeMo Guardrails toolkit. +Follow these steps to start using the NeMo Guardrails library. ::::{grid} 1 1 2 2 :gutter: 3 @@ -55,14 +55,14 @@ Follow these steps to start using the NeMo Guardrails toolkit. :link: getting-started/installation-guide :link-type: doc -Install the toolkit with pip and set up your environment. +Install the library with pip and set up your environment. ::: :::{grid-item-card} Tutorials :link: getting-started/tutorials/index :link-type: doc -Follow hands-on tutorials to build your first guardrails configuration. +Follow hands-on tutorials to build your first Guardrails configuration. ::: :::: @@ -107,7 +107,7 @@ Monitor and troubleshoot your guardrails applications. :link: deployment/index :link-type: doc -Deploy your guardrails using the toolkit's local server, Docker, or as a production microservice. +Deploy your guardrails using the library's local server, Docker, or as a production microservice. ::: :::{grid-item-card} Integrate @@ -120,8 +120,8 @@ Connect with LangChain, LangGraph, and other frameworks. :::: ```{toctree} -:caption: About NeMo Guardrails Toolkit -:name: About NeMo Guardrails Toolkit +:caption: About NeMo Guardrails Library +:name: About NeMo Guardrails Library :hidden: Overview