googleapis/python-genai
Python Genai
This project is a Python software development kit and framework for building applications that integrate with large language models. It serves as a multimodal content generator and vector embedding library, enabling the production and editing of text, images, audio, and video.
The toolkit provides specialized capabilities for adapting base models through supervised and reinforcement training. It further distinguishes itself by offering tools for orchestrating complex workflows, including stateful chat sessions, the enforcement of structured output via schemas, and the integration of external tools and function calling.
The library covers a broad range of functional areas, including large-scale AI inference via asynchronous batch predictions, semantic search through text embeddings, and the management of server-side context caching to reduce latency. It also provides utilities for calculating token counts, managing uploaded media files, and configuring model hyperparameters.
Features
- Generative - Acts as a framework for orchestrating LLM workflows, stateful chats, and structured output enforcement.
- Generative AI Models - Produces text, code, or media by interfacing with large-scale generative AI models.
- Generative Content APIs - Provides programmatic interfaces for sending prompts to AI models and retrieving generated content.
- Multimodal Input Processing - Processes multimodal inputs including text, images, and audio within a single request.
- Multimodal Media Generation - Produces a variety of media types including text, images, and audio based on multimodal prompts.
- Chat Session Management - Provides stateful conversation management to track dialogue history across multiple turns.
- Conversation State Management - Tracks dialogue history across multiple turns to maintain context and session state during interactive chats.
- External Tool Integrations - Integrates external utilities like search engines and code execution to extend model functionality.
- Function Calling Interfaces - Provides function calling interfaces that enable models to execute external tools and APIs.
- Automatic Tool Executions - Executes external tools automatically when the language model triggers a function call.
- Generative AI Development - Provides a comprehensive SDK for building applications that integrate large language models for text, code, and media generation.
- Image Editing - Provides capabilities to generate new images from text prompts and edit existing visuals using AI.
- Tool Call Executions - Performs the actual runtime invocation of external functions based on structured model outputs.
- Multimodal Content Generation - Creates high-resolution images, videos, and text from mixed-modality prompts.
- Multimodal Content Generators - Serves as a toolkit for producing and editing text, images, audio, and video using multimodal inputs.
- Tool Calling - Connects a model to custom functions or external tools to request and receive real-world data.
- Structured Output Enforcements - Constrains model responses to specific formats using schemas or predefined values for reliable processing.
- Text Embeddings - Converts text into numerical vector representations to enable semantic searches and similarity tasks.
- Video Generation - Produces video content from text or image prompts and allows extending existing video clips.
- Model Fine-Tuning - Enables fine-tuning of base models using custom datasets to optimize performance for specific tasks.
- Supervised Fine-Tuning - Supports adapting base models through supervised training on custom datasets to create specialized model versions.
- AI Image Processing - Provides AI-powered image generation, upscaling, and segmentation with aspect ratio control.
- LLM Schema Outputs - Enforces structured output by constraining model responses to a provided JSON schema.
- Schema-Constrained Outputs - Forces model responses to conform to specific machine-readable structures using JSON schemas.
- Multimodal Search - Performs multimodal search across uploaded files and documents to provide grounded responses.
- JSON Output Constraints - Enforces a specific schema for generated responses to ensure the output is machine-readable.
- LLM Client Libraries - Ships as a comprehensive Python SDK for programmatic integration with large language models.
- Generative Video Editing - Implements generative video editing, including frame interpolation and clip extension.
- Multimodal Content Generation - Produces and edits a wide range of media types including text, images, audio, and video.
- LLM Orchestration - Orchestrates complex model interactions including stateful chat sessions, tool calling, and structured output enforcement.
- Tool-Call Dispatchers - Maps model-generated function requests to executable Python functions or external servers.
- Batch Prediction Processing - Implements batch prediction processing for high-throughput inference on large datasets.
- Code Execution Tools - Integrates built-in code execution tools for performing mathematical and algorithmic tasks.
- High Volume AI Processing - Supports high-throughput inference through asynchronous batch predictions and server-side context caching.
- Hyperparameter Configurations - Provides tools to manage and tune model hyperparameters like temperature, presence penalty, and frequency penalty.
- Context Caching - Implements context caching for prompt and conversation history to improve response speed.
- Large Language Model Fine-Tuning - Enables adaptation of base models through supervised and reinforcement training on custom datasets.
- Asynchronous Batch Processing - Provides high-throughput inference by processing large volumes of requests as asynchronous background jobs.
- LLM Fine-Tuning Toolsets - Provides a toolkit for supervised fine-tuning and parameter-efficient updates of language models.
- Model Response Streaming - Streams model-generated content incrementally to the client to improve responsiveness.
- Real-time Multimodal Sessions - Establishes low-latency multimodal connections supporting voice activity detection and audio transcription.
- Stateful Agent Orchestration - Orchestrates complex, multi-step agent workflows using state-based transitions and tool execution.
- Vector Embeddings - Provides a library for generating vector representations of text to enable semantic search.
- Semantic Vector Search - Converts text into vector embeddings to enable semantic search and grounded data retrieval.
- Content Caching Controls - Controls the storage and expiration of cached content to optimize performance and operational cost.
- Server-Side Context Caches - Stores large inputs and system instructions on the server to reduce latency and token costs.
- Input Caches - Caches large inputs and system instructions to reduce latency and token costs.
- Batch Processing Jobs - Implements batch processing jobs for handling large volumes of requests with configurable dataset inputs and outputs.
- Web Search Grounding - Grounds model responses using real-time web search results to ensure factual accuracy.
- Uploaded File Management - Manages the lifecycle of media files and documents uploaded for use in model prompts.
- AI Content Safety Filtering - Filters and sanitizes AI inputs and outputs to ensure safety and policy compliance.
- Token Count Calculation - Calculates the number of tokens in a prompt using remote APIs or local tokenizers.
- Incremental Response Streaming - Delivers model-generated content and function arguments to clients in real-time via incremental streaming.
- Agent Frameworks - Official SDK for integrating Google generative models into Python applications.
