Awesome GitHub RepositoriesContent Type Detection

CrewAI is a multi-agent orchestration framework designed for building autonomous systems that execute complex, multi-step workflows. It provides a development platform where specialized agents are defined with specific roles, goals, and tool sets to perform tasks collaboratively. By leveraging a declarative workflow engine, the system manages task dependencies, state transitions, and execution logic, allowing for the creation of structured, stateful sequences of operations. The framework distinguishes itself through its hierarchical management capabilities, which utilize manager agents to coo

Magika is an AI content type classifier and MIME type prediction engine that uses deep learning to identify file formats based on binary data. It analyzes byte sequences through a neural network to predict the content type of a file and provide associated confidence scores. The system features a foreign function interface that allows the core detection logic to be integrated across different programming languages. It includes a mechanism for configuring detection sensitivity and per-type thresholds to balance precision and recall. The project provides capabilities for bulk file analysis via

Sherpa-ONNX is an ONNX-based speech processing toolkit that provides a local speech recognition engine, an on-device voice synthesis tool, and a speaker identification framework. It is designed as a cross-platform speech API that enables speech-to-text, text-to-speech, and speaker verification tasks to be executed locally on a device without requiring network access. The project is distinguished by its ability to perform zero-shot voice cloning and speaker diarization on-device. It supports a wide range of hardware accelerations, including GPU and various NPU architectures, and provides a Web

This repository contains the HTML specification, which defines the core standards for web page structuring, content organization, and document rendering. It establishes the fundamental algorithms for state-machine-based tokenization, tree construction for the document object model, and origin-based security isolation. The specification provides a framework for defining custom elements with independent lifecycles and registries. It also details the requirements for cross-document communication, session history management, and the synchronization of interface properties with content attributes.

PyMuPDF is a comprehensive PDF manipulation library and document analysis tool. It serves as a text extraction tool, OCR engine, and image converter, providing a programmatic interface to edit, merge, split, and optimize PDF and Office documents. The project distinguishes itself through high-performance capabilities, including the use of C-bindings for low-level manipulation and parallelized page processing to accelerate workloads. It provides specialized conversion paths, such as transforming PDF content into Markdown for retrieval-augmented generation and large language model pipelines. It

Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo

Espectre is an edge machine learning framework and motion detection platform that uses Wi-Fi Channel State Information to identify human presence and movement. It functions as a sensing toolkit for ESP32 microcontrollers, enabling the detection of motion through walls without the use of cameras or wearables. The project distinguishes itself by executing compact neural network classifiers and mathematical detection algorithms directly on the microcontroller. It utilizes a MicroPython runtime to allow for the prototyping and deployment of sensing logic and wireless signal processing algorithms

SQLiteStudio is an open-source graphical tool for browsing, editing, and managing SQLite database files. It combines a full-featured SQL editor with syntax highlighting, a visual database schema designer for creating entity-relationship diagrams, and a plugin-based extensibility platform that allows adding custom functionality through C/C++, JavaScript, Tcl, or Python. The application distinguishes itself through its multi-language scripting engine, which embeds JavaScript, Tcl, and Python interpreters to enable user-defined functions and scripts within SQL queries. It supports encrypted data

Markup is a tool for converting various documentation formats and manual pages into structured HTML. It functions as a rendering engine selector and converter that transforms raw markup files into web-ready output using a pluggable pipeline. The system identifies the appropriate rendering engine by utilizing file extensions or analyzing document content through pattern-based language detection. This allows the pipeline to handle documents where file extensions are missing or ambiguous. The project covers a broad range of content transformation, including the conversion of traditional man pag

Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interfac

jscpd is a code duplication detector that scans source code across 223 languages to find identical or near-identical copy-pasted blocks using a rolling hash algorithm. It is built with a Rust core for high performance, exposed through a Node.js API and CLI, and can be run as a standalone binary with no runtime dependencies. The tool detects duplicates in embedded languages within Vue, Svelte, Astro, and Markdown files by tokenizing each language block separately, and it supports extensionless scripts via shebang detection. The project distinguishes itself through its integration capabilities

OpenWeChat एक Go भाषा सॉफ़्टवेयर डेवलपमेंट किट और API रैपर है जिसे Go एप्लिकेशन में WeChat मैसेजिंग और खाता प्रबंधन को एकीकृत करने के लिए डिज़ाइन किया गया है। यह वास्तविक समय चैट इवेंट्स और प्लेटफ़ॉर्म के साथ प्रोग्रामेटिक इंटरैक्शन को संभालने के लिए एक बॉट फ्रेमवर्क और मैसेजिंग लाइब्रेरी के रूप में कार्य करता है। यह प्रोजेक्ट सत्र प्रबंधन के लिए एक व्यापक प्रणाली प्रदान करता है, जिसमें QR कोड प्रमाणीकरण और पुनरारंभ (restarts) के बाद पहुंच बनाए रखने के लिए JSON प्रारूप में सत्र कुकीज़ का स्थायित्व शामिल है। यह उन संदेशों को इंटरसेप्ट और संरक्षित करने की क्षमताओं की पेशकश करके खुद को अलग करता है जिन्हें भेजने वाले रद्द (revoke) करने का प्रयास करते हैं, साथ ही मल्टी-अकाउंट सत्र प्रबंधन के लिए समर्थन भी प्रदान करता है। यह लाइब्रेरी स्वचालित संदेश वितरण और उत्तर तर्क, संपर्क और समूह प्रबंधन, और उपयोगकर्ता प्रोफ़ाइल डेटा पुनर्प्राप्ति सहित कार्यात्मक क्षेत्रों की एक विस्तृत श्रृंखला को कवर करती है। इसमें मीडिया हैंडलिंग के लिए उपकरण भी शामिल हैं, जैसे कि साझा फ़ाइलें डाउनलोड करना और फ़ाइल हेडर विश्लेषण के माध्यम से सामग्री के प्रकारों की पहचान करना। SDK एक मिडलवेयर-आधारित संदेश चेनिंग प्रणाली को लागू करता है और आने वाले डेटा को सिंक्रोनाइज़ करने के लिए पोलिंग-आधारित इवेंट डिटेक्शन का उपयोग करता है।

file-type is a binary file type detector that identifies file extensions and MIME types by analyzing magic numbers and signature bytes in binary data. It functions as a magic number parser and MIME type resolver, mapping binary signatures to standardized media type strings. The project is an extensible file format identifier that allows for the addition of custom detector plugins to recognize uncommon or non-binary file formats. The engine supports binary format identification across various data sources, including buffers and data streams. It utilizes a supported format registry and provide

PasteMD is a clipboard-based document processor and productivity tool designed to convert Markdown or HTML content into formatted office documents. It transforms markup and mathematical formulas from the clipboard into rich text for direct insertion into word processors and spreadsheets. The system functions as a style orchestrator, using reference documents and templates to apply specific fonts, colors, layouts, and margins to the converted text. This allows for the customization of output appearances to match specific document requirements. The tool handles technical document composition b

Tika is a content analysis toolkit and Java library designed for detecting and extracting metadata and text from thousands of different file types. It functions as a universal document text extractor and metadata extraction engine, converting complex files into plain text or XHTML. The system employs a specialized MIME type detector that identifies document formats using magic bytes and metadata to determine the correct parser. It serves as an OCR integration gateway, connecting to external text recognition tools to extract content from image files. The project covers a broad range of extrac

Defuddle is a command line web parser and content extractor designed to isolate the primary article body from web pages and convert the result into standardized markdown. It functions as a content cleaner that removes layout clutter, such as sidebars and headers, to retrieve the main text and associated metadata. The tool provides a terminal interface that processes content from remote URLs, local files, or piped HTML streams. It supports custom content targeting, allowing users to specify CSS selectors to manually define the main content area when automatic detection is insufficient. The sy

Active Storage Validations Ruby on Rails के लिए एक लाइब्रेरी है जो फाइल अटैचमेंट के लिए मॉडल-स्तरीय सत्यापन सहायक प्रदान करती है। यह सुनिश्चित करने के लिए कि अपलोड की गई सामग्री स्टोरेज में पर्सिस्ट होने से पहले परिभाषित व्यावसायिक आवश्यकताओं के अनुरूप है, यह सीधे मानक सत्यापन लाइफसाइकिल में इंटीग्रेट होती है। फ्रेमवर्क फाइल आकार, कंटेंट प्रकार, आयाम और अटैचमेंट की संख्या पर बाधाएं लागू करती है। इसमें फाइलों के बाइनरी हस्ताक्षर का निरीक्षण करके कंटेंट-प्रकार स्पूफिंग का पता लगाने और ब्लॉक करने के लिए सुरक्षा तंत्र शामिल हैं ताकि यह सत्यापित किया जा सके कि उनका वास्तविक प्रारूप घोषित एक्सटेंशन से मेल खाता है। बुनियादी सत्यापन से परे, लाइब्रेरी विशिष्ट पहलू अनुपात और आयामों को लागू करके मीडिया एसेट गुणवत्ता नियंत्रण का समर्थन करती है। यह डेवलपमेंट वर्कफ़्लो के भीतर इन बाधाओं को सही ढंग से कॉन्फ़िगर किया गया है या नहीं, यह सत्यापित करने के लिए परीक्षण मैचर्स भी प्रदान करती है।