Awesome GitHub RepositoriesMemory-Mapped Storage

This project is a high-performance library designed for the similarity search and clustering of dense vectors across massive datasets. It functions as a vector similarity search engine, providing the necessary tools to organize complex numerical data into specialized structures that facilitate rapid retrieval and efficient querying of millions of records. The library distinguishes itself through a variety of advanced indexing and compression techniques, including hierarchical navigable small worlds for logarithmic time complexity and inverted file indexing to partition vector spaces into mana

Datasets is a library designed for the management, processing, and sharing of large-scale data collections for machine learning workflows. It functions as both a data processing framework and a versioning platform, providing tools to organize, filter, and transform massive datasets while ensuring reproducibility across research and development teams. The library distinguishes itself by enabling the handling of datasets that exceed available system memory. It utilizes memory-mapped file access, disk-based caching, and lazy iterative streaming to maintain performance when working with large-sca

This project is a terminal-based log analyzer and real-time traffic monitoring system designed for web server environments. It processes raw server access logs to generate structured statistical insights, providing visibility into visitor behavior, bandwidth usage, and request latency. The tool functions as a high-performance utility capable of handling large-scale datasets through multi-threaded parsing and memory-efficient data structures. The software distinguishes itself by offering both an interactive terminal interface and a web-based dashboard that updates in real-time via WebSocket st

Sandboxie is an operating system-level virtualization tool designed to run Windows applications in isolated, secure environments. By intercepting system calls and redirecting file system and registry modifications to a separate, discardable storage area, it prevents untrusted software from making permanent changes to the host system. This containment ensures that browser history, temporary files, and potential malware remain trapped within the sandbox, protecting the integrity and privacy of the underlying host. The software distinguishes itself through granular control over the isolation env

Bolt is a single-file embedded key-value store for Go applications. It is an ACID transactional database that organizes data in B+trees on disk to provide efficient sorted key retrieval and range scans. The system uses a memory-mapped model to map the database file directly into the process address space for fast random-access reads. The project distinguishes itself through a multi-version concurrency control architecture that allows multiple simultaneous readers to access a consistent snapshot of data without blocking a writer. It employs a single-writer multi-reader locking model and uses a

FastImageCache is an iOS image caching library that provides a persistent disk-based image store. It utilizes a persistent bitmap cache to store images in uncompressed formats and incorporates an image pre-processing pipeline to optimize assets before they are committed to storage. The library optimizes rendering performance by using memory-mapped image tables for constant-time retrieval and byte-aligned data layouts to prevent memory copies. It organizes images of identical dimensions into shared tables and manages disk space through a least-recently-used cache eviction system. The project

pcileech is a toolkit for executing DMA attacks, analyzing PCIe bus traffic, performing kernel patching, and conducting remote volatile memory forensics. It functions as a hardware memory acquisition tool and a PCIe DMA attack framework designed to read and write remote system memory via direct hardware interfaces. The project provides capabilities for capturing and displaying raw transaction layer packets from the PCIe bus and mounting live RAM as local drives for analysis. It enables the modification of system memory signatures and the execution of shellcode or implants within the kernel wi

LMCache is a distributed key-value cache manager and tiering system designed to accelerate large language model inference. It functions as a tiered storage layer that offloads tensors from GPU memory to CPU RAM, local disks, or remote object stores, enabling the reuse of cached prefixes across different inference sessions and serving engines. The system differentiates itself through a disaggregated prefill-decode model, which separates prompt processing from token generation by transferring caches between distributed compute nodes. It utilizes peer-to-peer orchestration to share and retrieve

zvec is an embedded vector database engine and indexing library designed for high-dimensional similarity search. It functions as a hybrid search engine and a retrieval-augmented generation knowledge base, allowing for the storage and retrieval of dense and sparse vectors. The system is distinguished by its hybrid retrieval pipeline, which fuses vector similarity, full-text keyword matching, and scalar metadata filtering into single query operations. It supports a plugin-based model integration system for registering custom embedding models and rerankers, as well as language bindings for nativ

هذا المشروع عبارة عن مكتبة تقنية وقاعدة معرفية لإطار عمل Qt، تعمل كدليل لبرمجة واجهات المستخدم الرسومية (GUI) بلغة C++ ومورد للتطوير عبر المنصات. يوفر مجموعة من الأدلة التقنية، وأفضل الممارسات، والمواد التعليمية التي تركز على بناء التطبيقات باستخدام نظام Qt البيئي. يركز المستودع على تحسين أداء التطبيقات وسير عمل التطوير. ويغطي الأنماط المعمارية لتحسين سرعات البناء وكفاءة التنفيذ، بما في ذلك استخدام قواعد البيانات داخل الذاكرة واستدعاءات الدوال غير المتزامنة. كما يقدم توجيهاً متخصصاً حول دمج كود Java عبر مغلفات الواجهة الأصلية للتطبيقات الهجينة. يمتد نطاق المشروع إلى تصميم الواجهات المخصصة والتوافق عبر المنصات. ويشمل ذلك تقنيات لتقديم الرسومات عالية الأداء، والتحجيم المتجاوب عبر كثافات البكسل المختلفة، وإدارة مسارات الملفات وترميز النصوص عبر أنظمة تشغيل مختلفة. كما يتضمن مجموعة مختارة من الكتب الإلكترونية، ومعايير البرمجة، ودروساً تعليمية لاستخدام أدوات التطوير.

PowerDNS is DNS server software used for serving authoritative and recursive DNS records. It functions as both an authoritative DNS server, providing definitive records for specific domains from trusted databases, and a recursive DNS resolver that finds answers for any domain by iteratively querying external servers. The project distinguishes itself through a backend-abstraction layer that decouples server logic from data storage, allowing records to be served from SQL databases, zone files, and external APIs. It includes a geo-aware DNS router to direct users to the nearest server based on t

This project is a comprehensive command-line reference and toolkit designed for Linux system administration and network security assessment. It provides a collection of technical snippets and operational guides focused on managing remote environments, orchestrating shell sessions, and executing administrative tasks through native terminal utilities. The repository distinguishes itself by offering specialized techniques for stealthy operations and infrastructure manipulation. It covers methods for establishing encrypted tunnels to bypass firewalls, obfuscating process identities and command hi

Chronicle Queue is a high-performance data handling system featuring off-heap message queues, memory-mapped file stores, and replicated message stores. It provides a binary compatible memory layout that enables different programming languages to share data without serialization overhead. The system utilizes a replicated message store to synchronize data across multiple nodes, ensuring high availability and instant failover. Its memory-mapped architecture supports deterministic replay from disk and low-latency data recording. The project implements off-heap memory management and zero-allocati

NutsDB is an ACID-compliant, embedded transactional storage engine that functions as both a disk-backed key-value store and an in-memory data structure store. It provides atomic and serializable transactions with commit and rollback capabilities to ensure strict data consistency for applications requiring a lightweight persistence layer. The engine distinguishes itself by supporting a variety of complex data types, including lists, sets, and sorted sets, alongside standard byte-slice storage. It implements a transactional storage model featuring hot backups and a compaction algorithm to maint

LMDB is an embedded key-value storage engine that provides ACID-compliant data persistence. It is a memory-mapped database that utilizes B+ trees to store key-value pairs, ensuring atomicity, consistency, isolation, and durability. The engine maps files directly into the virtual address space to minimize data copying and system calls. This approach enables high-performance local caching and low-latency data access, specifically optimizing for read-heavy database workflows. The system implements a transactional model with copy-on-write versioning and single-writer multi-reader locking. These