screenpipe/screenpipe
Screenpipe
Screenpipe is a local-first platform designed to record, index, and analyze desktop activity. By capturing screen, audio, and keyboard input, it creates a comprehensive and searchable history of computer usage. The system functions as an activity recorder and automation framework, providing a persistent, context-aware memory that allows artificial intelligence agents to observe and interact with local desktop environments.
The platform distinguishes itself through a privacy-focused architecture that processes all data locally. It utilizes on-device computer vision and speech recognition to transcribe audio and extract metadata from screen content, while simultaneously applying automated redaction to filter sensitive information before storage. All captured logs are maintained in a local relational database, ensuring that user data remains under local control and is protected by authenticated encryption at rest.
Beyond simple recording, the platform supports complex workflow automation and business process analysis. It exposes captured activity data to external tools through standardized protocols, enabling the execution of custom scripts and agents that can trigger actions across third-party software. These capabilities allow for the mapping of work patterns, the identification of process bottlenecks, and the generation of structured datasets for training computer-use models.
The software provides a command-line interface for managing capture policies and supports fleet-wide deployment within organizational environments. All processing, including data sanitization and indexing, occurs on the host machine to maintain security and compliance.
Features
- Desktop Automation Frameworks - Provides a framework for intelligent agents to observe, analyze, and execute actions across local desktop environments using standardized protocols.
- Activity Indexers - Records and indexes screen, audio, and keyboard input to create a searchable history of computer usage.
- Screen Recording Tools - Records screen, audio, and keyboard input to create a searchable, context-aware history of computer usage for AI agents.
- Local-First Privacy Tools - Ensures all captured desktop activity and sensitive logs remain stored and processed locally on the host machine to maintain user privacy.
- Assistant Context Integrations - Provides context-aware memory to external AI assistants by exposing local activity data.
- Model Context Protocol Integrations - Exposes local history to external AI assistants through standardized protocols for context-aware memory.
- Automatic Redaction - Automatically detects and obscures personally identifiable information and secrets within captured images and text using local processing.
- Activity Monitors - Indexes and records user interactions across applications to provide a comprehensive, queryable history of computer usage.
- Speech Transcription - Converts system and microphone audio into text in real-time using local speech recognition models.
- Business Process Automation - Deploys intelligent agents that trigger actions in external software based on observed user activity.
- Business Workflow Automation - Automates business processes by executing custom tasks and agents in response to system events.
- Workflow Analytics - Mapping actual work patterns from captured data to identify bottlenecks, redundant steps, and opportunities for process improvement in professional environments.
- Local-First Databases - Stores captured activity data in a local relational database to ensure user ownership and offline availability.
- Natural Language Search Interfaces - Enables natural language queries across screen text, OCR data, and audio transcripts.
- Data Sanitization - Filters and redacts personally identifiable information using local models before data is indexed.
- Workflow Analytics - Maps actual work patterns from captured activity intelligence to identify bottlenecks and redundant steps in professional environments.
- Computer Vision - Performs on-device image analysis and text recognition to extract metadata from screen content.
- Developer Tools and Utilities - Capture and analyze screen/audio history for context-aware agents.
- Audio and Video - Local AI-powered screen and microphone recording.
- Screen Recording - Listed in the “Screen Recording” section of the Awesome Mac awesome list.
- AI Assistant Tools - Local AI screen and microphone recording.
- AI Assistants and Tools - Local AI tool for 24/7 screen and mic recording.
- Productivity Assistants - Open-source tool for recording and searching screen and audio activity.
- Training Data Pipelines - Processes and sanitizes desktop activity data into structured datasets suitable for training computer-use models and automating professional workflows.
- Model Context Protocols - Provides a consistent interface for external AI assistants to query local history through standardized protocols.
- Authenticated Encryption - Protects recorded activity logs using authenticated encryption to ensure data security at rest.
- Data Encryption - Secures captured activity data through authenticated encryption and automated local privacy detection.
- Desktop Deployment Management - Enforces data capture policies across organizational devices using standard management tools.
- Event-Driven Pipelines - Triggers recording pipelines based on specific system events to optimize resource consumption.
- Granular Access Controls - Enforces granular, deterministic access controls for AI agents and external integrations interacting with local activity data.
- Automated Task Execution Engines - Executes scheduled tasks and automated agents that query screen data and perform system actions.
- Plugin Architectures - Provides a plugin-based architecture for executing user-defined scripts and agents that interact with local data.
- Training Data Generation - Exports structured logs of human interactions to create datasets for training computer-use models.
- Third-Party Service Integrations - Synchronizes recorded activity and insights with third-party business platforms to automate reporting tasks.
