BoundaryML/baml

Features

• Prompt Engineering - Designing and iterating on complex prompt templates with conditional logic and reusable snippets in a dedicated environment.
• LLM Response Parsers - Processes raw LLM output and converts it into strongly typed objects defined by the prompt function.
• API Client Generators - Compiles prompt definitions into language-specific, type-safe client libraries for API communication.
• LLM Client Generators - A compiler that produces language-specific boilerplate code for API communication, output parsing, and error recovery across different model providers.
• AI Workflow Orchestrators - A platform for managing model fallbacks, retry policies, and multimodal inputs to ensure production reliability for AI applications.
• AI Workflow Reliability Strategies - Implements automatic retries, model fallbacks, and output validation to ensure stable production AI behavior.
• Automated Prompt Optimization - Uses genetic algorithms to iteratively refine prompt text based on performance benchmarks.
• Genetic Prompt Evolution - Improves prompt quality and accuracy using a genetic algorithm that iterates on candidate prompts based on tests.
• LLM Execution Tracing - Monitors token usage, inspects raw API requests, and traces AI function executions for debugging and optimization.
• LLM Provider Integrations - Provides configurations and authentication adapters to connect to multiple diverse LLM providers.
• LLM Response Streaming - Sends prompt function output incrementally to the client to display results before computation completes.
• Multimodal Input Processors - Processes images, audio, PDFs, and videos by converting URLs into formats consumable by AI models.
• Structured Output Generators - Uses classes, enums, unions, and maps to ensure LLM responses conform to specific machine-readable schemas.
• Prompt Engineering Frameworks - A system for defining type-safe AI prompts as functions and compiling them into client libraries for multiple languages.
• Return Type Injection - Inserts a formatted description of the expected return type into a prompt to guide parsable data generation.
• Prompt Iteration Workflows - Provides a development extension for iteratively validating prompt logic and output formatting with instant feedback.
• Prompt Templates - Provides a dedicated language for designing complex prompt templates with conditional logic and reusable snippets.
• Structured Data Extraction - Transforms unstructured text into predefined schemas by identifying entities and relationships via LLMs.
• Schema-Enforced Output Parsers - Transforms unstructured LLM responses into strongly typed objects using alignment algorithms and schemas.
• Prompt Grounding Attachments - Integrates external knowledge sources into prompts to improve accuracy and prevent hallucinations.
• Language Type Mappers - Generates native type definitions across different programming languages from a centralized prompt schema.
• Prompt Template Injection - Inserts dynamic values into prompt templates using string interpolation to customize the input.
• Structured Object Streaming - Transforms raw LLM token streams into semantically valid partial objects for real-time structured rendering.
• Interactive Prompt Testing - Runs individual or batch test cases against defined functions to verify output quality and performance.
• LLM Function Wrappers - Enables the execution of predefined AI prompts as type-safe functions to integrate model logic directly into application code.
• Model Provider Integrations - Integrates with Anthropic's messages API to send prompts to Claude models with configurable settings.
• Fallback Sequences - Implements high reliability by chaining model providers in a sequence for automated failover.
• OpenAI-Compatible APIs - Routes requests to HuggingFace models using an OpenAI-compatible client configuration.
• Model Request Routing - Implements logic to dynamically switch between different LLM providers and model clients during execution for traffic distribution and testing.
• AI Observability Tracing - BAML connects local execution logs to a remote dashboard for centralized observability.
• AI Provider Integrations - Connects to models hosted on Azure AI Foundry using OpenAI-compatible API configurations.
• Chain of Thought Implementations - Guides the model to reason through a problem step-by-step before producing a final structured output.
• Optimization Benchmarks - Creates specialized performance benchmarks using named checks to guide the automated optimization process.
• Conversation History Management - Manages the sequence of previous messages to maintain conversational context for LLM prompts.
• Field Metadata Descriptions - Adds descriptive metadata to fields or values to help the model understand the intended meaning of data.
• Function Parameter Extraction - Translates unstructured user input into typed function arguments for tool execution.
• Gemini Integrations - Sends prompts to Google Gemini models via the Google AI API for text generation and streaming.
• LLM Observability - A set of tools for tracking token usage, inspecting raw HTTP traffic, and monitoring function execution through distributed tracing.
• Prompt-Level - Tunes prompts against competing metrics like accuracy and latency by assigning relative weights to each goal.
• Chat Role Assignment - Specifies the persona for different prompt sections to ensure correctly structured chat messages.
• Model-Side Tool Integrations - Integrates external capabilities like function calling and web search into the model's reasoning process.
• Fallback Configurations - Defines backup model chains to automatically handle requests when primary AI services fail.
• Model Generation Tuning - Adjusts model behavior by setting temperature, token limits, safety thresholds, and reasoning budgets.
• Instructional Input Optimizers - Reduces token count by pruning data or splitting prompts to minimize model confusion and hallucinations.
• Runtime Provider Switching - Allows dynamic swapping of the active model or provider for a specific function during runtime execution.
• Model Response Racing - Sends prompts to several models simultaneously and returns the first successful response while canceling others.
• Multimodal Processing - Handling images, audio, and PDF inputs within prompting workflows across various model providers.
• OpenAI API Clients - Interacts with OpenAI APIs hosted on Azure using specific resource names and deployment IDs.
• Schema-Aware Prompting - Adds aliases and descriptions to data fields to improve model understanding and parsing.
• Template Syntax Formatting - Uses template syntax to dynamically inject variables and logic into prompts before delivery.
• Conditional Logic - Uses conditional statements within templates to vary prompt text based on input variables.
• Nested Prompt Fragments - Defines string templates that can be nested and called within prompts to organize complex logic.
• Multimodal Prompt Validation - Executes test cases against functions using various input types including text, images, audio, and video.
• Prompt Visualizers - Visualizes the final prompt sent to the provider, including resolved macros and raw requests.
• Reasoning Field Embedding - Includes dedicated fields within the output object to force the model to provide justifications.
• Provider Failover Handlers - Automatically redirects requests to alternative AI providers when primary services fail to ensure reliability.
• Source Citations - Captures exact source quotes from knowledge bases as structured arrays to provide verifiable references.
• Text Classification - Categorizes text into single or multiple labels by combining defined schemas with targeted prompts.
• Token Optimizers - Minimizes token consumption by using an optimized schema format during structured data extraction.
• Dynamic Schema Extensions - Generates tool schemas at runtime from source code to avoid duplicating definitions in application logic.
• Vertex AI Integrations - Connects to Google Vertex AI services to execute prompts using Gemini and Anthropic Claude models.
• LLM Output Constraints - Enforces constraints on extracted fields using custom logic to ensure AI responses meet specific requirements.
• Dynamic Schema Adaptation - Updates classes and enums during execution to allow the LLM output schema to adapt to dynamic data sources.
• Model Behavioral Annotations - Attaches metadata or behavioral modifiers to fields to control how the AI model processes specific values.
• Tool Selection Logic - Determines which tool or set of tools to invoke by returning a union or list of typed objects.
• Editor-Integrated Test Execution - Executes tests and visualizes results directly within the editor through real-time syntax validation and a dedicated playground.
• Prompt Playgrounds - Simulates model responses in a playground environment to preview prompts and validate tests without consuming tokens.
• Language Server Integrations - Provides syntax highlighting, jump-to-definition, and error diagnostics via Language Server Protocol integration.
• OpenAPI Specification Generators - Generates machine-readable JSON specifications and interactive UIs to document available API endpoints.
• AWS Bedrock Integrations - Connects to AWS Bedrock text-output models using the Converse API within a type-safe workflow.
• Containerized AI Function Deployment - Hosts defined prompt functions in containers to be called as services over HTTP.
• AI Model Load Balancers - Distributes calls across multiple model providers using a round-robin strategy to manage capacity.
• Provider Rotation - Rotates outgoing calls across multiple providers in sequence, including automatic switching during retries.
• Token Streaming - Controls whether specific fields stream token-by-token or appear only when fully complete via semantic attributes.
• Structured Object Streaming - Converts raw LLM token streams into partially valid structured objects for real-time typed data rendering.
• Token-Efficient Field Aliasing - Replaces internal property names with abstract symbols to reduce model bias and optimize token usage.
• Runtime Type Registrars - Creates or modifies data structures using the prompting language directly within the application code at runtime.
• API Key Management - Passes authentication credentials dynamically during specific calls to support per-user API key management.
• Parallel LLM Execution - Runs multiple prompt functions in parallel across different threads or asynchronous tasks to improve throughput.
• Error Handling - BAML captures parsing failures and client timeouts to provide raw output and fallback history for debugging.
• LLM Response Repair - BAML uses a secondary LLM prompt to automatically repair malformed responses by passing parsing errors back to a function.
• Prompt-Optimized Field Renaming - Maps internal code identifiers to more intuitive names to improve prompt performance.
• Value Constraint Enforcers - Restricts model outputs to specific literal values or named constants to ensure consistent classification.
• Prompt and Code Decoupling - Isolates prompt definitions into dedicated directories to manage AI instructions independently from application code.
• Retry Policies - Implements retry policies with exponential backoff and fallback chains to maintain AI service stability.
• Structured Trace Inspection - Captures calls as structured function executions with typed parameters instead of raw JSON.
• Prompt Request Inspection - Generates the raw HTTP request for a prompt, including headers and body, for inspection without sending it.
• Metrics Collection - BAML tracks function calls and usage metrics through a collector to access performance data.
• Call Inspection - BAML captures raw requests, responses, and timing information to debug workflows without abstraction layers.
• Observability Tracing - BAML records the inputs and outputs of application functions in an observability dashboard using a decorator.
• Token Usage Analytics - Calculates cumulative token consumption and identifies cached input tokens for providers.
• Usage Dashboard Builders - Provides a dashboard to track and visualize model calls and resource consumption across functions.
• Automated Assertion Validators - Runs test cases against AI functions in parallel using programmatic assertions to ensure consistent output quality.
• Prompt List Iteration - Loops through collections of items to inject multiple values into prompts using specific attributes.
• AI Integration Hooks - Provides type-safe React hooks for managing AI request states, streaming responses, and loading conditions within frontend components.
• Type-Safe Hook Generation - BAML creates server actions and hooks from prompt definitions to handle LLM requests in streaming or non-streaming modes.
• Provider-Agnostic LLM Routing - Routes prompts to external AI providers by overriding base URLs while maintaining standard client protocols.
• RESTful APIs - Serves defined prompting functions over a RESTful interface for external consumption.
• Artificial Intelligence - Prompting language for building reliable AI workflows.

Star history