30 open-source projects similar to thunlp/wantwords, ranked by shared indexed features. Tags may describe platforms or build tools rather than the same primary purpose. Check each project’s use case, license, and deployment requirements before treating it as a replacement.
This project is a transformer-based framework for generating dense and sparse vector embeddings of text and multimodal data. It serves as a library for fine-tuning models to perform semantic similarity tasks, retrieval, and reranking. The system is distinguished by its support for diverse architectural patterns, including bi-encoders for fast similarity search and cross-encoders for high-precision reranking. It provides dedicated pipelines for multimodal embeddings, mapping text and images into a shared vector space, and implements knowledge distillation to compress large models into smaller,
qmd is a local semantic search engine and RAG knowledge base indexer that functions as a Model Context Protocol server. It converts local documents, markdown files, and codebases into a searchable database to provide retrieval augmented generation capabilities for AI agents. The system exposes its search and retrieval tools via stdio or HTTP. It utilizes local model files for embeddings and reranking, supporting query expansion across multiple languages. The project employs abstract syntax tree based chunking to split source code at function and class boundaries. It implements hybrid vector-
ECDICT is a collection of structured linguistic datasets and an English-Chinese dictionary database. It provides bilingual word definitions, phonetic symbols, and parts of speech, alongside a bilingual geographic gazetteer that maps English place names to Chinese equivalents. These resources are available as a multi-format lexicon export in CSV, SQL, StarDict, and MDX formats. The project distinguishes itself by integrating a linguistic corpus dataset that includes word frequency rankings and academic syllabus markers derived from national corpora. It functions as an educational vocabulary re
mgrep is an LLM-powered semantic search engine and local file indexer designed to retrieve information from local directories and web content using natural language queries. It functions as a semantic document retriever that uses meaning and context rather than exact keyword matches to locate relevant data. The tool distinguishes itself by combining local file indexing with real-time web content retrieval to synthesize comprehensive answers. It employs retrieval-augmented generation to transform retrieved snippets from both local and remote sources into direct, concise responses. The system
Open Semantic Search is an open-source enterprise discovery platform designed to index, analyze, and explore large, diverse document collections. It functions as a comprehensive search engine and analytics suite that transforms unstructured data into structured information through automated processing pipelines. The platform distinguishes itself by integrating semantic exploration with traditional retrieval methods. It utilizes knowledge graph entity linking and thesaurus-driven query expansion to connect related concepts, allowing users to navigate datasets beyond simple keyword matching. Th
Khoj is a self-hosted artificial intelligence platform designed for personal knowledge management and semantic information retrieval. It functions as a private assistant that indexes your local documents, notes, and external workspaces, allowing you to interact with your data through natural language queries and conversational chat. By maintaining a local-first architecture, the system ensures that your information remains under your control while providing context-aware responses grounded in your personal knowledge base. The platform distinguishes itself through a modular, cross-platform int
Vespa is a distributed search engine, vector database, and machine learning ranking engine. It serves as an AI search platform designed to handle large-scale document indexing and complex query processing across a cluster of nodes, combining keyword retrieval with high-dimensional embedding storage for semantic similarity search. The platform distinguishes itself by integrating machine learning models directly into the search pipeline to perform real-time inference and ranking. It converts these models into ranking expressions to score and order results based on relevance, while providing a s
This project is a knowledge base plugin and RAG context manager that uses a local vector database interface to enable semantic search and relationship mapping. It transforms text into numerical vectors to find semantically related notes and excerpts based on conceptual meaning rather than keyword matches. The system differentiates itself through a semantic graph visualizer that maps notes into clusters to reveal conceptual connections. It also features a context manager capable of bundling local notes and excerpts into reusable packs to provide grounded factual bases for large language model
fuzzysort is a JavaScript library for performing approximate string matching and ranking results. It functions as a string matching engine and weighted search utility designed to identify approximate matches within text and object lists. The library features a pre-indexed search implementation that processes target strings into an optimized format to accelerate repeated lookups. It supports weighted object retrieval, allowing users to search through lists of objects by matching multiple keys and applying custom weights to prioritize specific fields. The engine provides capabilities for searc
This project is a framework for training and deploying transformer-based models that map text, images, audio, and video into dense or sparse vector representations. It functions as a multimodal embedding library and semantic search engine used to retrieve relevant documents by calculating vector similarity between meanings. The framework provides specialized tools for both cross-encoder reranking, which calculates precise similarity scores to refine search results, and vector quantization to compress embedding vectors for reduced memory usage and increased retrieval speed. The project covers
pg_textsearch is a full-text search integration for PostgreSQL that provides large-scale text indexing and BM25 relevance ranking. It implements a scalable indexing architecture that uses a memtable system to spill data to disk segments, allowing for the processing of massive datasets. The project distinguishes itself through support for multilingual search via language-specific partial indexes and the ability to index complex expressions, such as JSONB fields or concatenated columns. It ensures high availability by utilizing PostgreSQL-native streaming replication and write-ahead logs to syn
Fuse is a JavaScript fuzzy search library and client-side search engine designed to index and query JSON data. It provides utilities for approximate string matching and ranking results by relevance, allowing applications to perform fast filtering and searching of datasets without a dedicated backend. The library distinguishes itself through a token-based search implementation that supports word-order independence and relevance weighting. It utilizes edit-distance scoring to handle typos and insertions, and employs a system of field weighting to prioritize matches in high-value data keys. The
Quivr is a framework for building retrieval-augmented generation pipelines that connect large language models to custom knowledge bases. It serves as a generative AI integration layer that abstracts the process of transforming diverse document sources into searchable context for AI responses. The project orchestrates the end-to-end flow between document ingestion, vector storage management, and model provider interfaces. It features a vector-store-agnostic retrieval system and a modular API layer that allows for flexible switching between different generative model providers. The system cove
Sonic is a high-performance, lightweight search backend designed to provide real-time full-text search and autocomplete capabilities for applications. It functions as a persistent indexing server that maps text terms to object identifiers, allowing developers to integrate rapid search functionality without storing raw document content directly within the search engine. The system distinguishes itself through a specialized graph-based index that enables real-time word prediction and typo correction. Communication is handled via a custom, low-latency binary protocol over raw TCP sockets, which
This project is an autonomous agent framework designed to integrate large language models with popular messaging platforms. It functions as a middleware platform that enables automated, multimodal interactions by decomposing complex user goals into sequential plans, executing them through external tools, and maintaining persistent context across sessions. The framework distinguishes itself through a modular skill architecture and a hybrid memory system. Users can extend system capabilities by installing custom logic modules from community hubs or generating them through natural language. The
Mem0 is an agent-agnostic memory layer designed to provide intelligent agents with long-term persistence and cross-session state management. By acting as a centralized service, it allows diverse AI agents to recall user preferences, past interactions, and historical context, ensuring continuity across multiple workflows and independent agent systems. The platform distinguishes itself through a multi-signal retrieval engine that combines semantic vectors, keyword matching, and entity-linked metadata to surface the most relevant information. It employs an adaptive memory engine that automatical
Weaviate is an AI-native vector database designed to store and index high-dimensional vector embeddings alongside traditional data objects. It serves as a backend infrastructure for retrieval-augmented generation, enabling applications to ground language model responses in private, context-aware data. The platform distinguishes itself by combining vector similarity search with traditional keyword filtering through a hybrid storage architecture. It integrates directly with external machine learning models to automate the generation of embeddings and perform complex inference tasks during inges
Mempalace is a local-first long-term memory store for large language models and AI agents. It provides a persistent storage system for verbatim conversation history and agent data, utilizing a local-first knowledge graph to track evolving entity relationships and timelines. The project implements a standardized memory protocol that allows external AI clients to read and write persistent memory via standard input and output. It features a hybrid semantic search engine that combines keyword boosting and reranking to find precise historical information across scoped categories. The system inclu
Realtime is a real-time data distribution and synchronization engine that enables applications to stream database changes and coordinate state between clients. It functions as a synchronization layer that monitors database write-ahead logs to provide change data capture and pushes updates to authorized clients via WebSockets. The project features a real-time presence server for tracking the online status of active users and a broadcast service for sending ephemeral messages without database persistence. It organizes communication through channel-based message routing and uses a structured JSO
Corpora is a library of curated, domain-specific text corpora and structured datasets. It provides collections of categorized nouns, adjectives, and verbs in JSON format, intended for use as static training or testing data for conversational agents and automated systems. The project organizes linguistic data across diverse fields, including science, art, and geography. These datasets are distributed as standalone files using a language-neutral schema and a static JSON data model, allowing for direct import into applications without an API layer. The library supports chatbot content generatio
Searx is a privacy-respecting metasearch engine and search result aggregator. It functions as a self-hosted search proxy that queries diverse web services, databases, and local indices to present a single unified list of results. The project prevents user tracking and profiling by acting as an intermediary between the client and search services. It strips identifying information from queries, removes tracker URLs and HTTP referrers from outgoing links, and can route traffic through proxies or the Tor network to mask user identity. The system supports multilingual search and result filtering
This project is a containerized search infrastructure designed to deploy a privacy-focused metasearch engine. It acts as a self-hosted search proxy that aggregates results from multiple external web, image, and academic search providers while anonymizing requests and stripping trackers to protect user identity. The system utilizes Docker to orchestrate the search instance, integrating caching mechanisms and reverse proxy support to ensure a private and efficient search environment. It employs a modular adapter-based integration to standardize diverse external API responses and a processing pi
Wukong is a distributed full-text search engine designed for indexing and retrieving text documents. It functions as a customizable search backend that employs a BM25 relevance ranker to order search results based on term frequency and inverse document frequency. The system includes a specialized Chinese text segmenter to break continuous character strings into meaningful words for accurate indexing and retrieval. To handle large datasets and high request volumes, it utilizes a distributed search index that employs hash-based sharding to split documents across multiple nodes. The engine prov
This project is a full text search engine and enterprise search infrastructure designed for indexing and retrieving large sets of documents. It provides a comprehensive framework for information discovery using ranked results and linguistic analysis. The system integrates high-dimensional vector similarity search for semantic retrieval alongside traditional full-text capabilities. It distinguishes itself through support for geospatial data retrieval, multilingual text processing, and a search suggestion workflow that includes typo-tolerant query completion and spellchecking. The platform cov
Game-Cheats-Manager is a centralized dashboard for downloading, organizing, and updating software modifications and trainers for single-player games. It functions as a community trainer aggregator that collects and distributes game modifications from multiple repositories and sources into a single searchable index. The project includes a file organization system that handles archive extraction and local folder imports to prevent duplicate downloads. To ensure tools run without being blocked, it features an antivirus exclusion tool that adds specific directories to security whitelists and prov
lunr.js is a JavaScript full-text search library and client-side search engine. It creates in-memory search indexes for fast keyword retrieval and ranked document matching within browser or Node.js environments. The library utilizes a JSON serializable search index, allowing the search structure to be converted to and from JSON for storage and distribution of pre-built search data. This enables search functionality for static websites by indexing content into portable files. The system supports advanced querying capabilities, including fuzzy text matching to account for typos, field-scoped i
Pagefind is a static site search engine that indexes HTML files to provide a browser-based search experience without the need for a backend server or API. It consists of a multilingual search indexer and a set of prebuilt, customizable user interface components for rendering search inputs and result lists. The system is designed for global content, utilizing a multilingual search indexer that detects page languages and creates independent index bundles to provide language-specific stemming and results. It further optimizes performance by using a compressed index and offloading query execution
Zombodb is a database extension and relational data indexer that integrates PostgreSQL with Elasticsearch. It provides a SQL search interface, allowing users to execute complex search queries and aggregations using standard SQL functions and syntax instead of native JSON APIs. The project synchronizes relational data from PostgreSQL to a remote search engine to enable high-performance full-text search and analytics. The system distinguishes itself by bridging relational structures with search engine capabilities, specifically through geospatial search integration for geometry and geography ty
Letta is a framework for building, deploying, and managing autonomous AI agents that maintain persistent state across long-term interactions. It provides a comprehensive suite of primitives for defining agents with configurable personas, modular memory blocks, and tool-use capabilities, enabling them to retain user preferences and conversation history over extended sessions. The platform distinguishes itself through its advanced memory management and orchestration capabilities. It allows agents to autonomously update their own memory, perform retrieval-augmented generation, and coordinate com