Projects sharing features with Dedupe

Fuzzywuzzy is a Python library and text processing utility designed to calculate similarity scores between strings. It functions as a text similarity scoring engine and an approximate string matching tool used to identify the closest textual matches within a list of candidate strings. The library provides a suite of tools for measuring the degree of similarity between pieces of text, accounting for typos and formatting differences. These capabilities include extracting the best match from a candidate list and performing fuzzy string matching through various scoring methods. The toolset cover

FalkorDB is a high-performance graph database management system and vector graph database. It serves as a knowledge graph construction tool and a GraphRAG knowledge store, integrating structured property graphs with vector search to provide grounded context for large language models. The engine is designed as a multi-tenant graph engine, capable of hosting thousands of isolated datasets within a single instance. The system distinguishes itself by using linear algebra for query execution, treating relationship tensors as matrix multiplications to achieve low-latency multi-hop traversals. It ut

This project is a transformer-based framework for generating dense and sparse vector embeddings of text and multimodal data. It serves as a library for fine-tuning models to perform semantic similarity tasks, retrieval, and reranking. The system is distinguished by its support for diverse architectural patterns, including bi-encoders for fast similarity search and cross-encoders for high-precision reranking. It provides dedicated pipelines for multimodal embeddings, mapping text and images into a shared vector space, and implements knowledge distillation to compress large models into smaller,

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

Skim is an interactive text filter and terminal selection tool written in Rust. It functions as a command line interface utility that processes input streams to isolate specific entries through real-time user queries and sorting. The tool differentiates itself through ANSI compatibility, allowing it to parse color codes and maintain text formatting during the search process. It supports multiple matching strategies, including configurable fuzzy matching algorithms and regular expression integration. The application covers a broad range of capabilities including field-specific filtering, resu

This is a Python fuzzy string matching library used for calculating string similarity and edit distances. It serves as a collection of string distance algorithms, a sequence alignment tool, and an approximate string search engine to measure text similarity. The library provides a wide array of metrics to quantify string closeness, including Levenshtein, Jaro-Winkler, Hamming, and Damerau-Levenshtein distances. It supports similarity analysis through longest common subsequence calculations, token-based comparisons, and weighted scoring to account for differences in content and word order. Bey

Ulauncher is a keyboard-driven application launcher and extensible command palette for Linux desktop environments. It provides a searchable interface for launching installed software and navigating local files. The system features a Python-based plugin architecture that allows the integration of third-party extensions and custom functionality. It includes a themable interface that supports custom color schemes and visual styles to match the system environment. Core capabilities include fuzzy-string matching for software and file retrieval, an integrated mathematical calculator for instant ev

toolong is a terminal log viewer and TUI log manager designed for monitoring live log streams and navigating large log files. It functions as a log aggregator and JSONL formatter, capable of merging multiple log files into a single chronological view by automatically detecting timestamps. The application supports the visualization of structured data by pretty printing JSONL files and applying syntax highlighting to common web server log patterns. It handles large-scale data efficiently through virtual-sized scrollable views, allowing users to open compressed logs or files of any size without

match-sorter is a JavaScript string matching and array filtering utility designed to rank and sort lists based on search string relevance. It functions as a deterministic best-match sorting library and fuzzy search engine for filtering object arrays. The tool prioritizes results using weighted match heuristics that favor exact matches, acronyms, and string containment. It employs a deterministic ranking system to ensure consistent ordering and supports diacritic-insensitive normalization to match characters regardless of accents. The library covers match criteria specification via key-path p

Lnx is a high-performance search engine and indexing database designed for managing large datasets. It provides a scalable infrastructure for full-text retrieval, allowing applications to process and query structured data with consistent responsiveness. The platform distinguishes itself through a schema-driven approach to data ingestion, which enforces structured field definitions to optimize storage and query precision. It incorporates advanced search capabilities, including fuzzy matching, spell correction, and similarity-based content recommendations, to ensure accurate results even with i

Skim is a cross-platform interactive fuzzy finder that runs as a terminal application, a Rust library, a Vim and Neovim plugin, and a shell integration tool. It provides real-time filtering and selection from lists of items, supporting keyboard and mouse navigation, live preview panes, and multi-select functionality across Linux, macOS, and Windows. The tool distinguishes itself through a composable query expression tree that supports fuzzy, exact, inverse, prefix, suffix, and logical AND/OR operators, combined with a Smith-Waterman scoring engine that penalizes typos and gaps for natural rel

Sensitive-lexicon is a sensitive word detection service and content moderation tool designed to identify prohibited text. It utilizes a curated lexicon of thousands of categorized terms and a fuzzy matching text scanner to detect restricted words and phrases. The project features specialized filters for Chinese language content across political, social, and adult domains. It supports approximate string matching to identify terms that use noise characters or whitespace to evade standard keyword filters. The system includes a network interface for hosting the detection service, allowing for re

Data-Juicer is an open-source framework for cleaning, filtering, deduplicating, and transforming multimodal datasets to prepare them for training large language and vision models. It functions as a distributed data pipeline engine that runs processing jobs across Ray clusters, handling billions of samples with automatic operator fusion and adaptive parallelism. The framework provides a library of operators that leverage large language models for semantic extraction, filtering, and data synthesis within processing pipelines. The project distinguishes itself through a YAML-based data recipe sys

Peco is an interactive text filter and fuzzy finder for the terminal. It serves as a terminal user interface selection tool that filters standard input in real-time using fuzzy matching and regular expressions. The tool preserves and renders ANSI color escape sequences from piped input streams while performing matching logic on plain-text versions. It supports multi-stage filtering, allowing users to freeze result sets to create a new base for subsequent refinements. Capability areas include advanced search filtering with negative matching, multi-item selection, and the ability to pipe selec

mini.nvim is a comprehensive library of independent modules designed to extend Neovim with a wide array of navigation, user interface, and text manipulation tools. It serves as a modular plugin collection, a UI toolkit for creating custom statuslines and notifications, and a package manager for installing and pinning external plugins from Git. The project provides a specialized fuzzy picker framework for filtering files and symbols, an LSP completion engine with interactive snippet expansion, and a dedicated plugin test framework that uses headless editor instances and remote procedure calls

Saws is an interactive shell wrapper and resource manager for the AWS CLI. It provides a command-line environment designed to enhance the execution of AWS commands through predictive text, resource suggestions, and improved navigation. The tool implements fuzzy searching and case-insensitive autocomplete to accelerate the discovery and selection of cloud resources. It reduces manual entry via a system of command shortcuts and aliases that map short strings to complex commands. The interface includes local caching of resource data to minimize API requests, persistent command history, and the

lunr.js is a JavaScript full-text search library and client-side search engine. It creates in-memory search indexes for fast keyword retrieval and ranked document matching within browser or Node.js environments. The library utilizes a JSON serializable search index, allowing the search structure to be converted to and from JSON for storage and distribution of pre-built search data. This enables search functionality for static websites by indexing content into portable files. The system supports advanced querying capabilities, including fuzzy text matching to account for typos, field-scoped i

Zed is a terminal-based code editor built in Rust that provides a full-featured editing experience with familiar keybindings, mouse support, and multiple cursors. It runs entirely in the terminal while offering capabilities typically found in graphical editors, including split panes, a command palette, and integrated language server protocol support for real-time diagnostics, completions, go-to-definition, and code actions across multiple languages. The editor distinguishes itself through a plugin system that runs sandboxed TypeScript plugins in a QuickJS runtime, with an asynchronous bridge

list.js is a JavaScript search and sort library used to add real-time filtering, sorting, and pagination to HTML lists and tables without backend dependencies. It functions as a DOM data indexer and template-driven HTML renderer, allowing developers to manage how data is displayed and discovered on the client side. The library distinguishes itself through a fuzzy string matching engine that handles approximate matches and typos, and a DOM data indexer that extracts values directly from HTML data attributes to build a searchable internal index. It uses a template-driven rendering system to gen

ctrlp.vim is a fuzzy file navigation tool for the Vim editor. It enables the location and opening of files, buffers, tags, and recently used items through approximate string matching and regular expressions. The project identifies project roots by scanning for version control markers and configuration files. It allows for the creation of new files and their required parent directories directly from the search interface, and can open multiple files simultaneously. Broad capabilities include text editor resource management and workflow automation, such as executing specific commands immediatel

apollo-ios is a GraphQL client library for iOS and Apple platforms that enables type-safe network communication. It transforms GraphQL operations into generated Swift models, ensuring that network responses are validated at compile time to eliminate manual mapping. The library features a normalized cache manager that stores entities in a flat structure to maintain data consistency across different application views. It also optimizes network performance using hash-based persisted queries to reduce payload sizes and supports real-time data streaming via WebSockets or HTTP subscriptions. The p

Tiny Universe is an educational monorepo that delivers multiple independent implementations of core AI subsystems as self-contained Jupyter notebooks. It provides from-scratch constructions of foundational architectures including a complete Transformer model built from the original paper specification, a denoising diffusion probabilistic model for image generation, and a ReAct-style autonomous agent framework that equips an LLM with tools for planning and multi-step task execution. The project distinguishes itself by covering the full lifecycle of modern AI systems through hands-on implementa

DataFlow is an agent-based workflow orchestrator and data pipeline designed to synthesize, clean, and augment large-scale datasets for training large language models. It functions as a synthetic data generator and text curation tool, utilizing an intelligent assistant to assemble modular processing operators into functional pipelines based on user requirements. The project distinguishes itself through a low-code approach, providing a web-based visual interface for designing and monitoring multi-stage execution flows. It features an operator-based registry system that allows for the integratio

LLM4Decompile is a toolset and framework for binary-to-source code translation. It uses large language models to transform machine code into readable source code and recover the original logic of compiled executables. The project includes a specialized pipeline for generating synthetic training datasets by converting source code into assembly pairs. It provides a fine-tuning framework to optimize deep learning models on these binary-to-source datasets, increasing the accuracy of code recovery. The system also features capabilities for refining decompiled pseudo-code. This process focuses on