# basedhardware/omi

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/basedhardware-omi).**

12,869 stars · 2,064 forks · Dart · MIT

## Links

- GitHub: https://github.com/BasedHardware/omi
- Homepage: https://omi.me
- awesome-repositories: https://awesome-repositories.com/repository/basedhardware-omi.md

## Topics

`ai` `app` `bci` `c` `flutter` `friend` `mobile` `necklace` `nextjs` `omi` `personas` `python` `smartglasses` `summary` `transcription` `wearable`

## Description

Omi is an open-source wearable AI platform that captures audio and screen data to provide real-time conversational assistance and memory. It integrates a wearable hardware development kit with a vector memory database and large language model capabilities to create a persistent digital record of user interactions.

The platform is distinguished by its BLE audio streaming pipeline, which transmits raw audio from wearable hardware for real-time transcription and speaker identification. It utilizes a plugin-based agent tool framework that allows AI assistants to autonomously invoke custom functions and interact with external services.

The system covers broad capability areas including semantic memory retrieval, voice-driven workflow automation, and multimodal activity capture. It manages the full lifecycle of AI interactions through automated conversation summarization, persona emulation, and the programmatic management of memories and action items.

The project provides a choice between self-hosting the backend or using a managed cloud service, with available SDKs for building third-party applications.

## Tags

### Artificial Intelligence & ML

- [Cross-Session Conversation Memories](https://awesome-repositories.com/f/artificial-intelligence-ml/conversational-agent-sessions/cross-session-conversation-memories.md) — Persists information and interaction history across multiple user sessions to provide long-term AI memory. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))
- [Fact Extraction Pipelines](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-architectures/memory-management-systems/long-term-memory-stores/fact-extraction-pipelines.md) — Processes unstructured audio transcripts to extract and store confirmed facts for long-term AI memory. ([source](https://docs.omi.me/))
- [Knowledge Base Retrieval](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-rag-development/knowledge-base-retrieval.md) — Provides a system for retrieving information from the total historical record of encountered data for context-aware responses. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))
- [Agent Tool Execution](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/integration-deployment/agent-frameworks/tool-use-and-execution/agent-tool-execution.md) — Enables AI agents to autonomously select and invoke external functions to provide comprehensive answers. ([source](https://docs.omi.me/doc/developer/backend/chat_system))
- [Agentic Tool-Use Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/integration-deployment/agent-frameworks/tool-use-and-execution/agentic-tool-use-frameworks.md) — Provides a plugin-based framework that allows LLMs to interact with external tools and knowledge bases.
- [AI Agent Tool Integrations](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-agent-integrations/ai-agent-tool-integrations.md) — Provides a plugin framework that allows LLMs to autonomously invoke custom functions and interact with external services.
- [Plugin-Based](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-assistants/plugin-based.md) — Provides a plugin-based framework allowing AI assistants to autonomously invoke custom functions and interact with external services.
- [Audio Speaker Embeddings](https://awesome-repositories.com/f/artificial-intelligence-ml/audio-speaker-embeddings.md) — Computes speaker embeddings from audio to enable precise identification and similarity matching of different speakers. ([source](https://docs.omi.me/doc/developer/apps/AudioStreaming))
- [Audio Transcription](https://awesome-repositories.com/f/artificial-intelligence-ml/audio-transcription.md) — Converts live audio into text using a provider-agnostic system with interchangeable transcription services. ([source](https://docs.omi.me/doc/developer/backend/backend_deepdive))
- [Real-Time Transcription](https://awesome-repositories.com/f/artificial-intelligence-ml/audio-transcription/real-time-transcription.md) — Converts streaming audio to text via WebSockets, delivering multi-language transcripts as JSON. ([source](https://docs.omi.me/doc/developer/backend/backend_deepdive))
- [Embedding Retrieval](https://awesome-repositories.com/f/artificial-intelligence-ml/embedding-retrieval.md) — Retrieves semantically similar past discussions by converting conversation text into high-dimensional vector embeddings.
- [Tool Invocation Handlers](https://awesome-repositories.com/f/artificial-intelligence-ml/mcp-tool-connectors/tool-invocation-handlers.md) — Handles the invocation and execution of registered custom functions called by an AI assistant during conversations.
- [Memory Structuring](https://awesome-repositories.com/f/artificial-intelligence-ml/natural-language-processing-analysis/memory-structuring.md) — Uses large language models to analyze raw captured activity and structure it into meaningful, persistent memory entries. ([source](https://docs.omi.me/doc/developer/backend/Backend_Setup))
- [Personal Data Retrieval](https://awesome-repositories.com/f/artificial-intelligence-ml/personal-data-retrieval.md) — Utilizes vector search and metadata filters to retrieve relevant past conversations and people from a user's private history. ([source](https://docs.omi.me/doc/developer/backend/chat_system))
- [Real-Time Speech Transcription](https://awesome-repositories.com/f/artificial-intelligence-ml/real-time-speech-processing/real-time-speech-transcription.md) — Converts live audio streams from wearable hardware into text with speaker identification and chronological ordering.
- [Real-Time Speaker Identifiers](https://awesome-repositories.com/f/artificial-intelligence-ml/speaker-diarization/real-time-speaker-identifiers.md) — Distinguishes between different speakers by assigning labels to transcribed words during live audio streaming. ([source](https://docs.omi.me/))
- [Speaker Verification](https://awesome-repositories.com/f/artificial-intelligence-ml/speaker-embeddings/speaker-verification.md) — Matches live voice embeddings against stored profiles to authenticate and distinguish between different speakers. ([source](https://docs.omi.me/doc/developer/backend/listen_pusher_pipeline))
- [Transcript Summarizers](https://awesome-repositories.com/f/artificial-intelligence-ml/speech-to-text-transcription/summarizations/transcript-summarizers.md) — Summarizes audio transcripts using AI models to extract overviews and action items. ([source](https://docs.omi.me/doc/developer/backend/backend_deepdive))
- [Structured Data Extraction](https://awesome-repositories.com/f/artificial-intelligence-ml/structured-data-extraction.md) — Uses AI to identify and extract structured factual data about user preferences and habits from conversations. ([source](https://docs.omi.me/doc/developer/backend/chat_system))
- [Semantic Vector Search](https://awesome-repositories.com/f/artificial-intelligence-ml/vector-embeddings/semantic-vector-search.md) — Retrieves relevant past interactions and facts using vector embeddings and semantic similarity search. ([source](https://docs.omi.me/doc/developer/backend/chat_system))
- [Wearable AI Platforms](https://awesome-repositories.com/f/artificial-intelligence-ml/wearable-ai-platforms.md) — Integrates wearable hardware, real-time sensing, and LLMs to provide a comprehensive conversational AI platform.
- [RESTful Memory Administration APIs](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-architectures/memory-management-systems/long-term-memory-stores/hybrid-short-and-long-term-memory/graph-structured-memory-apis/restful-memory-administration-apis.md) — Provides REST endpoints for the programmatic management of AI memories, conversations, and action items. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))
- [Voice Activity Detection](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/conversational-voice-interaction/voice-agents/voice-activity-detection.md) — Identifies speech segments within audio streams to trigger downstream processing and manage conversational turns. ([source](https://docs.omi.me/doc/developer/apps/AudioStreaming))
- [Custom Extension Integrations](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/integration-deployment/agent-ecosystems/extensible/custom-extension-integrations.md) — Offers interfaces for developers to create custom tools and applications that extend AI agent capabilities. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))
- [Query Path Routing](https://awesome-repositories.com/f/artificial-intelligence-ml/agentic-systems-frameworks/memory-context-systems/agent-memory-architectures/composable-memory-architectures/three-tier-memory-architectures/query-path-routing.md) — Classifies user questions into distinct paths to determine if personal memory retrieval or specific persona logic is required.
- [External API Tool Exposures](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-agent-integrations/ai-agent-tool-integrations/external-api-tool-exposures.md) — Exposes external service functions as callable tools for AI assistants via a manifest file. ([source](https://docs.omi.me/doc/developer/apps/ChatTools))
- [AI Chat Actions](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-chat-actions.md) — Defines custom functions that can be invoked directly within a chat conversation to perform external actions. ([source](https://docs.omi.me/doc/developer/apps/Introduction))
- [Streaming Chat Responses](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-chat-clients/streaming-chat-responses.md) — Sends AI-generated response text and tool-execution status to the client incrementally using Server-Sent Events. ([source](https://docs.omi.me/doc/developer/backend/chat_system))
- [Persona Configurations](https://awesome-repositories.com/f/artificial-intelligence-ml/custom-ai-assistant-development/persona-configurations.md) — Allows configuration of specific AI personalities and behavioral constraints via a web-based interface. ([source](https://docs.omi.me/))
- [Spoken Language Detection](https://awesome-repositories.com/f/artificial-intelligence-ml/language-detection-tools/spoken-language-detection.md) — Automatically identifies the spoken language from an audio stream without requiring a preset language code. ([source](https://docs.omi.me/doc/developer/backend/transcription))
- [Conversation Summaries](https://awesome-repositories.com/f/artificial-intelligence-ml/long-context-training-optimizations/context-summarizations/conversation-summaries.md) — Produces concise AI-generated condensations and actionable items from recorded conversation threads. ([source](https://docs.omi.me/))
- [Persona Interaction Interfaces](https://awesome-repositories.com/f/artificial-intelligence-ml/persona-management-systems/persona-imitation/persona-interaction-interfaces.md) — Provides interfaces for users to engage in themed conversations with specific AI character personas. ([source](https://docs.omi.me/))
- [Persona Emulation](https://awesome-repositories.com/f/artificial-intelligence-ml/prompt-engineering-tools/persona-instruction-sets/persona-emulation.md) — Generates AI responses using specific character prompts and curated tool sets to maintain behavioral consistency. ([source](https://docs.omi.me/doc/developer/backend/chat_system))
- [Streaming Transcription Events](https://awesome-repositories.com/f/artificial-intelligence-ml/real-time-speech-processing/real-time-speech-transcription/streaming-transcription-events.md) — Streams real-time transcription arrays and system status updates to the client via WebSockets. ([source](https://docs.omi.me/doc/developer/backend/listen_pusher_pipeline))
- [Silence-Based Conversation Triggers](https://awesome-repositories.com/f/artificial-intelligence-ml/silence-based-conversation-triggers.md) — Uses audio silence detection to determine the end of a conversation and trigger post-processing. ([source](https://docs.omi.me/doc/developer/backend/transcription))
- [Speaker Identification Frameworks](https://awesome-repositories.com/f/artificial-intelligence-ml/speaker-identification-frameworks.md) — Matches voice input against stored speech profiles to identify specific users during transcription. ([source](https://docs.omi.me/doc/developer/backend/transcription))
- [Voice-Driven Interfaces](https://awesome-repositories.com/f/artificial-intelligence-ml/speech-recognition/voice-driven-interfaces.md) — Enables the extraction of action items and management of calendars and issue trackers through natural language voice input.
- [Source Citations](https://awesome-repositories.com/f/artificial-intelligence-ml/structured-output-enforcements/source-citations.md) — Generates AI responses with numbered citations that provide verifiable references to the original data sources. ([source](https://docs.omi.me/doc/developer/backend/chat_system))

### Data & Databases

- [Conversation Memory Retrieval](https://awesome-repositories.com/f/data-databases/indexing-and-search/recall-optimization/conversation-memory-retrieval.md) — Uses vector embeddings to store and retrieve a lifelong history of user interactions for context-aware AI responses.
- [Transcript-Vector Hybrid Storage](https://awesome-repositories.com/f/data-databases/hybrid-vector-and-full-text-search/transcript-vector-hybrid-storage.md) — Implements a specialized storage architecture combining full-text conversation transcripts with semantic vector summaries for dual-mode retrieval.
- [Conversation Insights](https://awesome-repositories.com/f/data-databases/relational-data-modeling/complex-data-modeling/data-insight-generators/generative-ai-insights/conversation-insights.md) — Extracts titles, summaries, and personal facts from conversation transcripts using generative AI. ([source](https://docs.omi.me/doc/developer/backend/backend_deepdive))
- [Agent-Specific Query Routing](https://awesome-repositories.com/f/data-databases/semantic-query-routing/agent-specific-query-routing.md) — Directs user queries to specific processing paths or AI agent personas based on the required context. ([source](https://docs.omi.me/doc/developer/backend/chat_system))
- [Vector Databases](https://awesome-repositories.com/f/data-databases/vector-databases.md) — Employs a vector database to query high-dimensional embeddings and metadata filters for memory retrieval. ([source](https://docs.omi.me/doc/developer/backend/chat_system))
- [Vector Memory Stores](https://awesome-repositories.com/f/data-databases/vector-memory-stores.md) — Utilizes a hybrid vector memory store to maintain long-term context and retrieve semantically similar past interactions.
- [Vector Storage](https://awesome-repositories.com/f/data-databases/vector-storage.md) — Utilizes a vector database to store high-dimensional embeddings of conversations for efficient semantic retrieval. ([source](https://docs.omi.me/doc/developer/backend/Backend_Setup))
- [Conversation Processing Pipelines](https://awesome-repositories.com/f/data-databases/conversation-processing-pipelines.md) — Routes transcribed text through an event-driven pipeline to generate memories and summaries while buffering during outages. ([source](https://docs.omi.me/doc/developer/backend/listen_pusher_pipeline))
- [Document Storage Managers](https://awesome-repositories.com/f/data-databases/data-collections-datasets/collection-lifecycle-management/document-storage-managers.md) — Manages the storage and retrieval of full conversation transcripts and associated metadata using a document database. ([source](https://docs.omi.me/doc/developer/backend/backend_deepdive))
- [Full Text Search](https://awesome-repositories.com/f/data-databases/full-text-search.md) — Implements keyword-based full-text indexing and retrieval across captured conversational data. ([source](https://docs.omi.me/doc/developer/backend/Backend_Setup))
- [Event-Driven Memory Triggers](https://awesome-repositories.com/f/data-databases/indexing-and-search/recall-optimization/contextual-memory-recall/event-driven-memory-triggers.md) — Implements automated triggers to capture or process memories based on runtime system events. ([source](https://docs.omi.me/doc/developer/apps/Introduction))
- [Interaction History](https://awesome-repositories.com/f/data-databases/interaction-history.md) — Persists a chronological record of user interactions and associated media files in a cloud database. ([source](https://docs.omi.me/doc/developer/backend/backend_deepdive))

### Business & Productivity Software

- [Digital Activity Recorders](https://awesome-repositories.com/f/business-productivity-software/digital-activity-recorders.md) — Continuously captures and indexes screen and audio activity to create a searchable digital archive of interactions. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))
- [Action Item Tracking](https://awesome-repositories.com/f/business-productivity-software/action-item-tracking.md) — Extracts tasks from conversation transcripts and allows manual creation to manage user obligations. ([source](https://docs.omi.me/api-reference/introduction))
- [Voice-to-Issue Workflows](https://awesome-repositories.com/f/business-productivity-software/issue-tracking/issue-creation-tools/voice-to-issue-workflows.md) — Transcribes spoken feedback into professional GitHub issues with AI-generated titles and descriptions. ([source](https://docs.omi.me/doc/developer/apps/examples/Github))
- [Natural Language Task Creation](https://awesome-repositories.com/f/business-productivity-software/project-task-management/natural-language-task-creation.md) — Enables the creation and updating of tasks from natural language requests within a conversation. ([source](https://docs.omi.me/doc/developer/backend/chat_system))
- [User Preference Management](https://awesome-repositories.com/f/business-productivity-software/user-preference-management.md) — Captures and persists timeless information regarding user preferences and personal relationships for AI context. ([source](https://docs.omi.me/api-reference/introduction))

### Graphics & Multimedia

- [Screen and Audio Capture](https://awesome-repositories.com/f/graphics-multimedia/audio-music/audio-recording/screen-and-audio-capture.md) — Records synchronized audio and screen content across devices to create a persistent record of user activity. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))
- [Real-Time Transcription Pipelines](https://awesome-repositories.com/f/graphics-multimedia/audio-music/audio-streaming-engines/audio-playback-engines/chunked-audio-streaming/generative-audio-chunking/audio-streaming-pipelines/real-time-transcription-pipelines.md) — Implements a real-time data pipeline that streams raw audio over BLE for immediate transcription and event emission.

### Hardware & IoT

- [Wearable Hardware Development Kits](https://awesome-repositories.com/f/hardware-iot/wearable-hardware-development-kits.md) — Supplies a comprehensive hardware development kit including schematics and firmware tools for building AI wearables.
- [BLE Device Management](https://awesome-repositories.com/f/hardware-iot/ble-device-management.md) — Implements a standard interface for BLE communication and data streaming to integrate wearable hardware.
- [Over-the-Air Device Updates](https://awesome-repositories.com/f/hardware-iot/over-the-air-device-updates.md) — Supports delivering and installing system software updates to wearable hardware remotely via a companion app. ([source](https://docs.omi.me/doc/hardware/DevKit2))

### Networking & Communication

- [PCM Audio Streaming](https://awesome-repositories.com/f/networking-communication/socket-networking/audio-streaming-servers/pcm-audio-streaming.md) — Transmits raw PCM audio from a wearable device over BLE using configurable codecs and packet fragmentation. ([source](https://docs.omi.me/doc/developer/Protocol))
- [Bidirectional Audio Transports](https://awesome-repositories.com/f/networking-communication/websocket-to-stream-bridges/audio-transcription-websockets/bidirectional-audio-transports.md) — Streams live audio from hardware to a backend using a combination of Bluetooth Low Energy and WebSockets. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))
- [Websocket Connection Managers](https://awesome-repositories.com/f/networking-communication/connection-management/websocket-connection-managers.md) — Maintains persistent WebSocket connections using keepalive pings and inactivity timeouts for reliable real-time communication. ([source](https://docs.omi.me/doc/developer/backend/listen_pusher_pipeline))
- [WebSocket Clients and Servers](https://awesome-repositories.com/f/networking-communication/distributed-systems-p2p/distributed-computing/communication-protocols/websocket-implementations/websocket-clients-and-servers.md) — Maintains persistent bidirectional WebSocket connections for streaming live transcripts and system events between client and server.
- [Cross-Device Data Availability](https://awesome-repositories.com/f/networking-communication/distributed-systems-p2p/distributed-computing/data-synchronization-consistency/distributed-file-synchronization/cross-device-data-availability.md) — Ensures seamless access and synchronization of captured data across wearable, mobile, and desktop devices. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))

### Development Tools & Productivity

- [Automated Task Generation](https://awesome-repositories.com/f/development-tools-productivity/automation-action-collections/automated-task-generation.md) — Analyzes captured conversation data to automatically produce concise summaries and actionable task lists. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))
- [Conversation Memory SDKs](https://awesome-repositories.com/f/development-tools-productivity/conversation-memory-sdks.md) — Provides software development kits and interfaces for building custom applications that process memories and conversations. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))
- [Community Extension Integrators](https://awesome-repositories.com/f/development-tools-productivity/platforms-runtimes-language-services/developer-ecosystems/community-extension-integrators.md) — Provides a framework for installing and running third-party community applications to extend system features. ([source](https://docs.omi.me/))

### DevOps & Infrastructure

- [Cloud Storage](https://awesome-repositories.com/f/devops-infrastructure/cloud-infrastructure/cloud-computing-serverless/cloud-storage.md) — Persists binary data including audio recordings and photos in cloud-based storage. ([source](https://docs.omi.me/doc/developer/backend/backend_deepdive))
- [Serverless Function Executions](https://awesome-repositories.com/f/devops-infrastructure/event-driven-triggers/serverless-function-executions.md) — Utilizes serverless function execution via Modal for scalable, on-demand processing of audio and conversation data.
- [Cloud or Self-Hosted Deployments](https://awesome-repositories.com/f/devops-infrastructure/feature-flags/cloud-or-self-hosted-deployments.md) — Provides flexible deployment options allowing users to choose between a managed cloud service or a self-hosted backend. ([source](https://docs.omi.me/))

### Mobile Development

- [Open-Source Hardware Sync](https://awesome-repositories.com/f/mobile-development/mobile-infrastructure-security/mobile-synchronization/wearable-to-mobile-sync/open-source-hardware-sync.md) — Connects open-source AI wearables to a mobile application for continuous audio and screen data capture. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))

### Software Engineering & Architecture

- [Event Bus Architectures](https://awesome-repositories.com/f/software-engineering-architecture/event-bus-architectures.md) — Uses an asynchronous event bus to route completed conversation segments for automated summarization and memory creation.

### User Interface & Experience

- [Citation-to-Source Navigators](https://awesome-repositories.com/f/user-interface-experience/citation-to-source-navigators.md) — Provides UI components that allow users to navigate from numbered citations directly to the source conversation context. ([source](https://docs.omi.me/doc/developer/backend/chat_system))

### Web Development

- [Integration SDKs](https://awesome-repositories.com/f/web-development/third-party-api-integrations/integration-sdks.md) — Offers SDKs and REST endpoints for building third-party applications using captured conversation data. ([source](https://cdn.jsdelivr.net/gh/basedhardware/omi@main/README.md))