blakeblackshear/frigate
Frigate
Frigate is a self-hosted network video recorder that functions as a private, local AI-powered vision engine. It manages video streams by performing real-time object detection, tracking, and classification directly on local hardware, ensuring that security monitoring and activity recording remain independent of cloud services.
The system distinguishes itself through a modular, hardware-accelerated video pipeline that offloads intensive decoding and machine learning inference to dedicated GPUs, NPUs, or specialized accelerators like Coral TPUs and Hailo modules. It utilizes state-based object tracking to maintain persistent identity and spatial coordinates for detected objects, enabling advanced behavioral analysis such as loitering detection and speed estimation. Users can further refine these capabilities through semantic search, which allows for text-to-image and image-to-image similarity queries across recorded footage.
Beyond core detection, the platform provides comprehensive tools for spatial configuration, including declarative geometric masks and zone-based filtering to minimize false positives. It supports low-latency, peer-to-peer streaming for live viewing and integrates with smart home ecosystems to bridge camera feeds and event notifications. The system also includes specialized features for face recognition, license plate detection, and audio event analysis, all managed through a secure, token-authenticated API.
The software is designed for containerized deployment, utilizing environment variables for configuration and standard protocols for certificate management and performance metric exposure.
Features
- Video Surveillance Systems - Provides a local, AI-powered security system for detecting and recording activity.
- Network Video Recorders - Manages video streams, performs real-time object detection, and maintains searchable archives of security footage.
- Video Analytics Pipelines - Decodes video frames and performs machine learning inference to detect objects and manage searchable activity history.
- Vision Engines - Uses machine learning models to identify, track, and classify objects within live video feeds.
- Transcoding Engines - Frigate processes raw camera streams through dedicated GPU or NPU hardware to perform efficient decoding and real-time object inference.
- Inference Engines - Frigate decouples detection logic from hardware backends to support diverse accelerators like Coral TPUs, Hailo modules, and OpenVINO.
- Object Tracking - Maintains persistent identity and spatial coordinates for detected objects across consecutive frames to enable behavioral analysis and loitering detection.
- Semantic Search - Enables similarity-based searching of historical events using AI models.
- WebGL & GPU Rendering - Processes raw camera streams by offloading video decoding and machine learning inference to dedicated GPU or NPU hardware.
- Computer Vision Analytics - Frigate provides distance measurements between zone coordinates to calculate the speed of objects moving through a monitored area.
- Face Recognition - Processes camera streams to identify individuals based on detected facial features.
- Hardware Acceleration - Enables accelerated object detection by connecting to specialized hardware units.
- Inference Abstractions - Decouples detection logic from specific hardware backends to support diverse accelerators like TPUs and integrated graphics processors.
- Local AI Integrations - Connects self-hosted vision-language models to the system with hardware-specific performance tuning.
- Spatial Detection Zones - Limits alerts and detections to specific areas of interest within a camera view.
- Spatial Filtering - Uses user-defined geometric masks and zone rules to filter detections and trigger alerts based on specific movement patterns.
- Media Downloaders - Frigate manages circular buffers and long-term archival of video segments triggered by motion, audio, or object detection events.
- GPU & Performance - Offloads video decoding and inference to GPUs and NPUs for improved performance.
- Stream Management - Provides network endpoints for simultaneous viewing of camera feeds.
- Motion Detection Configurations - Frigate sets minimum time requirements for objects in specific zones to prevent false triggers from objects simply passing through the area.
- Classification Models - Defines object labels and detection thresholds to categorize tracked items for event reporting.
- Motion Detection - Adjusts sensitivity thresholds to balance motion detection against environmental noise.
- Video Streaming - Security camera monitoring with local AI-based processing.
- Home Automation Integrations - Connects camera feeds and events into broader home automation platforms.
- Media Archival - Manages circular buffers and long-term storage by triggering recording segments based on motion, audio, or object detection events.
- Hardware Media Processors - Offloads decoding and inference tasks to specialized local hardware like GPUs and NPUs.
- Compositing Pipelines - Provides a visual editor to define detection zones and exclusion masks.
- Low-Latency Streaming - Transmits live feeds using peer-to-peer protocols to minimize delay.
- Peer-to-Peer Streaming - Transmits live video feeds directly to browser clients using low-latency protocols to minimize server overhead and viewing delay.
- Authentication Strategies - Exchanges user credentials for secure bearer tokens to authorize subsequent requests and ensure communication remains protected.
- Live Stream Dashboards - Renders camera feeds on the dashboard using adaptive streaming technologies for smooth viewing.
- Detection Filtering - Sets minimum confidence thresholds to distinguish true positives from false positives.
- Event Analysis - Uses generative AI to summarize and retrieve specific moments from recorded footage.
- Face Recognition Training - Assigns detected faces to individuals to build a personalized recognition database.
- Model Training Interfaces - Allows users to label example images to improve object detection accuracy.
- Semantic Metadata Indexing - Generates structured summaries and embeddings for recorded footage to enable text-based search and intelligent event retrieval.
- Intel Hardware Acceleration - Configures hardware acceleration presets for Intel GPUs to improve video decoding performance.
- NVIDIA Hardware Acceleration - Configures device reservations to enable NVIDIA GPU hardware acceleration for efficient video processing.
- Camera Autotracking - Enables motorized cameras to automatically follow detected objects by centering them in the frame and returning to a preset position.
- HomeKit Bridges - Connects local security cameras to smart home ecosystems by managing pairing credentials and accessory links.
- Access Control - Secures network restream endpoints with authentication to prevent unauthorized access.
- Stream Management - Assigns specific roles like detection or recording to different resolution streams to balance performance.
- Generative Summarization - Produces structured summaries and narratives from recorded footage for improved activity monitoring.
- License Plate Recognition - Processes vehicle streams to detect and track license plates.
- Data Retention Policies - Manages storage usage by automatically deleting old video segments after a set period.
- Audio Processing - Frigate monitors audio levels on camera streams to trigger recordings and snapshots whenever sound exceeds defined thresholds for security and monitoring purposes.
- Spatial Audio - Frigate uses user-defined geometric masks and spatial rules to filter detections and trigger alerts based on object movement patterns.
- Asset Pipelines - Automatically selects the highest quality frame for event snapshots.
- Camera Bridges - Exposes local camera streams and event notifications to external home automation and monitoring ecosystems.
- Remote Hardware Controls - Connects cameras via standard protocols to enable remote movement and zoom functionality.
- Monitoring Dashboards - Defines display modes and camera overrides for unified monitoring of multiple feeds.
- API Orchestration - Exposes system controls and media access through a secure, token-authenticated interface for external integrations.
- WebRTC Configurations - Sets up network candidates and port forwarding to enable low-latency live video streaming.
