mediar-ai/screenpipe
Screenpipe
Screenpipe is a local screen and audio recorder that captures and indexes digital activity to create a searchable archive of computer usage. It functions as an AI context engine, providing a local database of visual and auditory history to ground large language models.
The system serves as a Model Context Protocol server, delivering screen history and meeting transcriptions to external AI assistants. It utilizes an OCR screen search tool to extract text from visual data and a speech-to-text transcription tool for identifying speakers in system and microphone audio.
The software includes capabilities for natural language activity search, chronological activity indexing, and local vector storage for semantic retrieval. It also provides OS-level permission filtering to restrict AI agent access to sensitive content and a local REST API for programmatic activity analysis.
Features
- Agent Context Providers - Serves as a context provider that injects real-time and historical screen data into AI assistants.
- Digital Activity Recorders - Provides a private system for capturing and indexing screen and audio activity for later retrieval and analysis.
- Model Context Protocol - Implements the Model Context Protocol to serve historical screen and audio data to external AI assistants.
- Meeting Transcriptions - Converts system audio and microphone input into text with speaker identification to document digital conversations.
- Model Context Protocol Servers - Implements a standardized interface to serve screen history and meeting transcriptions to external AI assistants.
- Screen Text Extractors - Extracts searchable text from screen frames using optical character recognition engines.
- Speech-to-Text Engines - Converts system and microphone audio into timestamped text with speaker identification.
- Speech-to-Text Integrations - Processes system and microphone audio into timestamped text with integrated speaker identification.
- Speech Transcription Engines - Converts system and microphone audio into text transcripts using optimized ML models.
- Digital Activity Tracking - Records and stores a private history of screen content and audio to create a searchable archive of computer activity.
- Local Interaction Stores - Stores real-time screen captures and audio within a private local interaction database.
- Screen Capture Tools - Captures screen content and accessibility trees periodically to build a chronological activity record.
- Visual Context Awareness Engines - Functions as a visual context awareness engine that grounds LLMs with a local database of visual and auditory history.
- Recurring Agent Scheduling - Provides scheduled AI agents that trigger actions based on observed user work activity.
- Natural Language Querying - Enables keyword and natural language querying across captured screen text and audio transcriptions.
- Personal Knowledge Management - Creates a searchable database of all digital interactions to act as a personal knowledge base.
- Vector Storage - Uses a local vector database to store screen text and audio embeddings for semantic retrieval.
- Activity Timelines - Organizes screenshots and audio transcripts into a time-series timeline for rapid playback and contextual querying.
- Application Permissions - Provides OS-level permission filtering via configuration files to restrict AI agent access to sensitive application content.
- Activity Visualizers - Provides a scrollable visual record of screen history for audio playback and screenshot review.
- End-User Applications - Local AI that records, searches, and automates tasks based on your screen and audio.
- Screen Recording - AI-powered 24/7 screen and microphone recording.
