Awesome GitHub RepositoriesContent Type Detection

CrewAI is a multi-agent orchestration framework designed for building autonomous systems that execute complex, multi-step workflows. It provides a development platform where specialized agents are defined with specific roles, goals, and tool sets to perform tasks collaboratively. By leveraging a declarative workflow engine, the system manages task dependencies, state transitions, and execution logic, allowing for the creation of structured, stateful sequences of operations. The framework distinguishes itself through its hierarchical management capabilities, which utilize manager agents to coo

Magika is an AI content type classifier and MIME type prediction engine that uses deep learning to identify file formats based on binary data. It analyzes byte sequences through a neural network to predict the content type of a file and provide associated confidence scores. The system features a foreign function interface that allows the core detection logic to be integrated across different programming languages. It includes a mechanism for configuring detection sensitivity and per-type thresholds to balance precision and recall. The project provides capabilities for bulk file analysis via

Sherpa-ONNX is an ONNX-based speech processing toolkit that provides a local speech recognition engine, an on-device voice synthesis tool, and a speaker identification framework. It is designed as a cross-platform speech API that enables speech-to-text, text-to-speech, and speaker verification tasks to be executed locally on a device without requiring network access. The project is distinguished by its ability to perform zero-shot voice cloning and speaker diarization on-device. It supports a wide range of hardware accelerations, including GPU and various NPU architectures, and provides a Web

This repository contains the HTML specification, which defines the core standards for web page structuring, content organization, and document rendering. It establishes the fundamental algorithms for state-machine-based tokenization, tree construction for the document object model, and origin-based security isolation. The specification provides a framework for defining custom elements with independent lifecycles and registries. It also details the requirements for cross-document communication, session history management, and the synchronization of interface properties with content attributes.

PyMuPDF is a comprehensive PDF manipulation library and document analysis tool. It serves as a text extraction tool, OCR engine, and image converter, providing a programmatic interface to edit, merge, split, and optimize PDF and Office documents. The project distinguishes itself through high-performance capabilities, including the use of C-bindings for low-level manipulation and parallelized page processing to accelerate workloads. It provides specialized conversion paths, such as transforming PDF content into Markdown for retrieval-augmented generation and large language model pipelines. It

Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo

Espectre is an edge machine learning framework and motion detection platform that uses Wi-Fi Channel State Information to identify human presence and movement. It functions as a sensing toolkit for ESP32 microcontrollers, enabling the detection of motion through walls without the use of cameras or wearables. The project distinguishes itself by executing compact neural network classifiers and mathematical detection algorithms directly on the microcontroller. It utilizes a MicroPython runtime to allow for the prototyping and deployment of sensing logic and wireless signal processing algorithms

SQLiteStudio is an open-source graphical tool for browsing, editing, and managing SQLite database files. It combines a full-featured SQL editor with syntax highlighting, a visual database schema designer for creating entity-relationship diagrams, and a plugin-based extensibility platform that allows adding custom functionality through C/C++, JavaScript, Tcl, or Python. The application distinguishes itself through its multi-language scripting engine, which embeds JavaScript, Tcl, and Python interpreters to enable user-defined functions and scripts within SQL queries. It supports encrypted data

Markup is a tool for converting various documentation formats and manual pages into structured HTML. It functions as a rendering engine selector and converter that transforms raw markup files into web-ready output using a pluggable pipeline. The system identifies the appropriate rendering engine by utilizing file extensions or analyzing document content through pattern-based language detection. This allows the pipeline to handle documents where file extensions are missing or ambiguous. The project covers a broad range of content transformation, including the conversion of traditional man pag

Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interfac

jscpd is a code duplication detector that scans source code across 223 languages to find identical or near-identical copy-pasted blocks using a rolling hash algorithm. It is built with a Rust core for high performance, exposed through a Node.js API and CLI, and can be run as a standalone binary with no runtime dependencies. The tool detects duplicates in embedded languages within Vue, Svelte, Astro, and Markdown files by tokenizing each language block separately, and it supports extensionless scripts via shebang detection. The project distinguishes itself through its integration capabilities

OpenWeChat este un SDK și API wrapper pentru limbajul Go, conceput pentru integrarea mesageriei și gestionării conturilor WeChat în aplicațiile Go. Servește ca un framework de boți și bibliotecă de mesagerie pentru gestionarea evenimentelor de chat în timp real și a interacțiunilor programatice cu platforma. Proiectul oferă un sistem cuprinzător pentru gestionarea sesiunilor, inclusiv autentificarea prin cod QR și persistența cookie-urilor de sesiune în format JSON pentru a menține accesul după restartări. Se distinge prin oferirea de capabilități pentru a intercepta și păstra mesajele pe care expeditorii încearcă să le revoce, precum și suport pentru gestionarea sesiunilor multi-cont. Biblioteca acoperă o gamă largă de arii funcționale, inclusiv livrarea automatizată a mesajelor și logica de răspuns, gestionarea contactelor și a grupurilor și preluarea datelor de profil ale utilizatorilor. Include, de asemenea, instrumente pentru gestionarea media, cum ar fi descărcarea fișierelor partajate și identificarea tipurilor de conținut prin analiza header-ului fișierului. SDK-ul implementează un sistem de înlănțuire a mesajelor bazat pe middleware și utilizează detectarea evenimentelor bazată pe polling pentru a sincroniza datele primite.

file-type is a binary file type detector that identifies file extensions and MIME types by analyzing magic numbers and signature bytes in binary data. It functions as a magic number parser and MIME type resolver, mapping binary signatures to standardized media type strings. The project is an extensible file format identifier that allows for the addition of custom detector plugins to recognize uncommon or non-binary file formats. The engine supports binary format identification across various data sources, including buffers and data streams. It utilizes a supported format registry and provide

PasteMD is a clipboard-based document processor and productivity tool designed to convert Markdown or HTML content into formatted office documents. It transforms markup and mathematical formulas from the clipboard into rich text for direct insertion into word processors and spreadsheets. The system functions as a style orchestrator, using reference documents and templates to apply specific fonts, colors, layouts, and margins to the converted text. This allows for the customization of output appearances to match specific document requirements. The tool handles technical document composition b

Tika is a content analysis toolkit and Java library designed for detecting and extracting metadata and text from thousands of different file types. It functions as a universal document text extractor and metadata extraction engine, converting complex files into plain text or XHTML. The system employs a specialized MIME type detector that identifies document formats using magic bytes and metadata to determine the correct parser. It serves as an OCR integration gateway, connecting to external text recognition tools to extract content from image files. The project covers a broad range of extrac

Defuddle is a command line web parser and content extractor designed to isolate the primary article body from web pages and convert the result into standardized markdown. It functions as a content cleaner that removes layout clutter, such as sidebars and headers, to retrieve the main text and associated metadata. The tool provides a terminal interface that processes content from remote URLs, local files, or piped HTML streams. It supports custom content targeting, allowing users to specify CSS selectors to manually define the main content area when automatic detection is insufficient. The sy

Active Storage Validations este o bibliotecă pentru Ruby on Rails care oferă ajutoare de validare la nivel de model pentru atașamentele de fișiere. Se integrează direct în ciclul de viață standard de validare pentru a se asigura că conținutul încărcat respectă cerințele de business definite înainte de a fi persistat în stocare. Framework-ul impune constrângeri privind dimensiunea fișierului, tipul de conținut, dimensiunile și numărul de atașamente. Include mecanisme de securitate pentru a detecta și bloca spoofing-ul tipului de conținut prin inspectarea semnăturii binare a fișierelor pentru a verifica dacă formatul lor real corespunde extensiei declarate. Dincolo de validarea de bază, biblioteca suportă controlul calității activelor media prin impunerea unor rapoarte de aspect și dimensiuni specifice. De asemenea, oferă matchere de testare pentru a verifica dacă aceste constrângeri sunt configurate corect în fluxurile de lucru de dezvoltare.