NVIDIA/NeMo-Guardrails

Features

• LLM Guardrail Frameworks - Provides a comprehensive framework for enforcing safety and structural constraints on LLM inputs and outputs.
• Dialogue Flow Management - Provides the ability to define structured interaction flows and constraints using configuration files.
• Declarative Dialogue Modeling - Provides a specialized modeling language for defining conversational flows and constraints through declarative rules.
• Hallucination Detection - Includes mechanisms for identifying and scoring AI-generated content against retrieved context to prevent hallucinations.
• Safety Logic Integration - Enables the injection of safety logic into agentic workflows to maintain consistent standards across execution pipelines.
• Conversational Dialogue Systems - Controls the flow of AI conversations to ensure the assistant follows specific interaction scripts.
• Programmable Dialogue Constraints - Allows the definition of interaction flows and safety checks using a programmable modeling language.
• Dialogue Flow Orchestration - Uses a modeling language to map conversational paths and keep assistants on topic.
• Prompt Injection Detectors - Protects models from jailbreak attempts and malicious instructions using input inspection.
• Input Quality Filters - Inspects and modifies user prompts to mask sensitive data and block malicious content before they reach the model.
• LLM Safety Enforcers - Filters user inputs and model outputs to block jailbreaks, PII, and prohibited content.
• Output Guardrails - Validates and filters responses generated by language models against safety and accuracy criteria.
• LLM Security - Implements a security layer to identify and mitigate vulnerabilities specific to LLMs, such as prompt injection.
• Input and Output Guardrails - Intercepts and filters data flowing into and out of the model to ensure safety and integrity.
• Guardrail Execution Traces - Records each guardrail's action and content modifications as observability spans for monitoring and debugging.
• Guardrail Reliability Evaluation - Ships a dedicated tool for quantifying the accuracy and reliability of moderation and hallucination filters.
• Model-Driven Action Triggering - Implements mechanisms that allow the dialogue manager to execute custom Python functions and external tools.
• Context Retrieval Filtering - Filters retrieved data chunks against safety policies to remove sensitive information before prompting the model.
• Middleware-Based Request Pipelines - Implements a pipeline of middleware layers that intercept and modify messages before they reach the model.
• AI Agent Security - Applies validation and safety guardrails to the tools and multi-step pipelines used by AI agents.
• Command Execution Guardrails - Validates model outputs to ensure only safe and authorized system commands are executed via custom actions.
• LLM Security Scanning - Evaluates the system against common attack vectors to determine the effectiveness of existing security constraints.
• LLM Judge Accuracy Validators - Measures response reliability by validating outputs against expected results and safety guidelines.
• LLM Evaluation - Evaluates the effectiveness of security policies by scanning for weaknesses and running safety evaluations.
• AI Security and Red Teaming - Toolkit for adding programmable guardrails to LLM applications.
• Guardrails and AI Safety - Listed in the “Guardrails and AI Safety” section of the The Incredible Pytorch awesome list.
• Privacy and Safety - Toolkit for adding programmable safety guardrails to LLMs.
• Red Teaming and Security - Programmable guardrails for controlling conversational system behavior.
• Safety and Security - Toolkit for programmable conversational guardrails.
• Security & Privacy - Programmable toolkit for adding safety guardrails to conversational applications.

Star history