Projects sharing features with Camelot

qsv is a high-performance command line toolkit for querying, transforming, and analyzing comma-separated value files. It functions as a data wrangling interface and a tabular data profiler, featuring a query engine capable of executing SQL statements and joins directly on flat files without requiring a database. The project is distinguished by its ability to process massive datasets that exceed available system memory. This is achieved through disk-based external memory processing, including multithreaded merge sorting, on-disk hash tables for deduplication, and lightweight file indexing for

This repository is a comprehensive collection of instructional guides and practical examples for Python development, focusing on machine learning, data science, and web scraping. It provides implementations for neural networks, reinforcement learning algorithms, and deep learning architectures using PyTorch, alongside detailed manuals for scientific computing and data visualization. The project distinguishes itself by offering specialized tutorials on concurrent programming to optimize CPU performance and guides for setting up Linux development environments. It covers the implementation of ad

This project is a pandas data analysis cookbook and Python data science guide. It provides a collection of programmatic recipes and examples for cleaning, manipulating, and analyzing structured data. The project focuses on providing a containerized analysis environment to ensure a consistent workspace and reproducible dependencies when executing data processing scripts. It covers a broad range of data science capabilities, including data ingestion from external sources, raw data cleaning, and exploratory data analysis. These recipes demonstrate how to perform structured data analysis through

Cleanlab is a data-centric AI library and toolkit designed to improve machine learning model performance by detecting label errors and increasing overall dataset quality. It implements a confident learning framework that iteratively refines label noise estimates by comparing model predictions with estimated label probabilities to identify mislabeled examples. The project provides specialized utilities for active learning optimization, allowing for the selection of the most impactful examples for labeling or re-labeling. It also includes an outlier detection tool to identify atypical data poin

This project is a comprehensive pandas data analysis tutorial and instructional guide designed for learning data manipulation and analysis. It serves as a tabular data processing guide and a manual for time series analysis, providing a structured approach to cleaning, merging, and transforming datasets. The repository functions as a data feature engineering course, providing tutorials on constructing and selecting dataset features to improve machine learning model performance. It also includes a vectorized data operations guide for performing element-wise mathematical computations and matrix

This project is a collection of reference materials and guidelines for implementing data audit frameworks. It serves as a data quality reference guide and a dataset validation manual for identifying common structural and statistical errors in datasets. The project provides a structured knowledge base for data cleaning, featuring a catalog of real-world data errors and practical strategies for their detection and resolution. It includes specific frameworks for evaluating data provenance and the reliability of aggregated information. The material covers a broad range of data analysis capabilit

diff-pdf is a visual comparison tool for PDF files that identifies differences between two documents. It functions as both a command line utility for verifying if documents are visually identical and a visualizer for pinpointing specific changes. The project provides a graphical interface for manual inspection, featuring zoom and offset controls to analyze visual shifts. It also generates difference maps that visually highlight the specific areas where two PDF documents differ. The tool supports automated regression testing by returning status codes to indicate whether pixel-level discrepanc

Danfo.js is a data analysis and preprocessing library for JavaScript that provides high-performance labeled data structures. It implements data frames and series to enable complex data analysis, statistical computing, and the manipulation of structured tabular data. The project serves as a machine learning preprocessing library, offering utilities for categorical label encoding, one-hot encoding, and numeric feature scaling and standardization. It specifically facilitates the conversion of labeled data structures into tensors for model training and evaluation. The library covers a broad set

AutoGluon is an automated machine learning framework and multimodal library designed to automate the end-to-end pipeline from data preprocessing to high-accuracy model training and validation. It functions as an automated model trainer for tabular, image, text, and time series data, as well as a tool for time series forecasting and foundation model finetuning. The project is distinguished by its ability to jointly process and fuse different data types, allowing for the construction of multimodal neural networks that integrate images, text, and structured tables. It supports zero-shot inferenc

csvkit is a composable Unix-style command-line toolkit for converting, filtering, and analyzing CSV files directly from the terminal. It provides a suite of focused single-purpose commands that can be combined via pipes to build complex data processing workflows, with a modular architecture that includes a column-type inference engine for automatically detecting data types and a streaming-pipeline design for efficient handling of tabular data. The toolkit distinguishes itself through its SQL-engine abstraction layer, which allows users to run SQL queries directly against CSV files without req

This project provides a collection of processed Chinese conversational datasets and preprocessing workflows designed for training and instruction tuning of large language models. It functions as a training corpus of cleaned, standardized Chinese text formatted as query-answer pairs. The repository includes a preprocessing pipeline and dataset aggregator that combine multiple public chat sources into unified files. These tools normalize text by converting traditional Chinese characters to simplified characters and transforming complex dialogue threads into a standardized sequence of single tur

This library provides a comprehensive system for defining data schemas and enforcing structural integrity within JavaScript applications. It enables developers to construct declarative rules for complex, nested objects, ensuring that data conforms to expected types and constraints before it is processed or stored. By integrating directly with the Meteor framework, it serves as a central tool for maintaining consistent data structures across collections and remote procedure calls. The project distinguishes itself through its deep integration with reactive data systems, allowing validation stat

This project is an AI-powered IDE extension and LLM coding assistant that provides a conversational interface for generating, refactoring, and debugging code. It functions as an AI agent framework and a Model Context Protocol client, connecting AI models to external data sources and tools to automate complex development tasks. The system is distinguished by its use of autonomous AI agents capable of multi-step task execution, including the ability to read files, modify code, and run terminal commands iteratively. It supports recursive agent orchestration through subagent delegation and employ

unioffice is a comprehensive document processing suite that provides a PDF document processor, an Open XML document library, a document security toolkit, and a document content extractor. It is designed to programmatically create, read, and modify Word, Excel, and PowerPoint files, as well as generate and edit PDF documents. The project is distinguished by its native language implementation of the Open XML standard, which removes native binary dependencies to simplify container deployments. It features advanced capabilities for digital document security, including hardware-based PDF signing,

Pdfarranger is a PDF page organizer, document editor, image converter, and booklet generator. It provides a visual drag-and-drop interface to reorder, merge, split, and delete pages within PDF documents. The application includes specialized tools for creating booklet printing layouts and converting image files into PDF pages or exporting PDF pages as PNG and JPEG images. It allows for the modification of document metadata while preserving internal outlines and hyperlinks. The software covers a range of structural manipulations, including page rotation, resizing, cropping, and overlaying. It

Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo

pdf2htmlEX is a PDF to HTML converter that transforms documents into web pages while preserving the original layout, fonts, and formatting. It functions as a layout engine and text extractor, mapping PDF coordinate data to HTML and CSS to maintain visual fidelity. The tool converts PDF content into searchable and selectable native HTML text by embedding original document fonts. It maintains document interactivity by preserving internal links, bookmarks, and outlines, converting them into functional web navigation. The conversion process supports flexible output structures, allowing documents

Libpostal is a C library designed for international address parsing and normalization. It utilizes statistical NLP and a language classifier to decompose unstructured global address strings into structured components and standardize street addresses by expanding abbreviations and resolving regional naming variations across multiple languages. The project provides tools for text transliteration, converting various scripts into standardized Latin-ASCII or NFD forms. It also includes capabilities for address deduplication, using symmetric fuzzy matching to identify whether different address reco

Zerox is a multimodal document parser and OCR tool that uses vision models to convert PDF files and images into structured Markdown text. It functions as a visual layout extraction engine, leveraging large multimodal models to digitize documents while maintaining their original structural formatting. The system differentiates itself through the use of coordinate-based element mapping and multimodal layout analysis to identify structural elements like tables, charts, and headers. It utilizes rasterization to convert vector PDF pages into high-resolution bitmaps, ensuring consistent input for t

PyMuPDF is a comprehensive PDF manipulation library and document analysis tool. It serves as a text extraction tool, OCR engine, and image converter, providing a programmatic interface to edit, merge, split, and optimize PDF and Office documents. The project distinguishes itself through high-performance capabilities, including the use of C-bindings for low-level manipulation and parallelized page processing to accelerate workloads. It provides specialized conversion paths, such as transforming PDF content into Markdown for retrieval-augmented generation and large language model pipelines. It

This project is a collection of educational resources and study materials focused on scientific computing and data analysis using Python. It consists of translated notes and Jupyter notebooks designed to guide learners through the Python data ecosystem. The content covers specialized workflows including numerical computation, data cleaning, and time series analysis. These materials provide a reference for performing complex data manipulations and processing sequential data to identify patterns. The resource is organized as a series of static files and markdown documents using a flat-file dir

This project serves as an educational and practical resource for mastering machine learning workflows using Python. It provides a comprehensive collection of code examples and exercises designed to guide users through the implementation of predictive systems, ranging from fundamental algorithms to deep learning architectures. The repository distinguishes itself by offering a structured approach to both classical machine learning and neural network training. It covers the full lifecycle of model development, including the orchestration of reusable data transformation pipelines, advanced ensemb

This project is a comprehensive machine learning educational resource and tutorial series delivered as a collection of interactive Jupyter Notebooks. It provides practical Python implementations for the end-to-end machine learning lifecycle, covering supervised and unsupervised learning, deep learning, and reinforcement learning. The resource distinguishes itself by providing detailed implementation guides for complex architectures, including transformers, generative adversarial networks, and convolutional neural networks. It also features specialized courseware for developing reinforcement l

OpenPDF is a Java library and document processor used for creating, editing, rendering, and encrypting PDF documents. It functions as a toolkit for generating new files from scratch, modifying existing document structures, and extracting text content. The project includes a dedicated engine for transforming HTML and CSS content into PDF documents by parsing markup and applying styles. It also provides a rendering engine to convert PDF pages into image formats for thumbnails and previews, alongside a security utility for protecting content via document encryption. The library supports the add

msgspec is a high-performance data modeling, serialization, and schema validation toolkit for Python. It serves as a type-safe serialization framework that integrates schema enforcement and data parsing into a single pass, functioning as both a data serialization library and a schema validation system based on standard Python type annotations. The project distinguishes itself through high-performance structural primitives, including compilation-based routine generation and zero-copy buffer parsing. It optimizes memory usage via garbage collection-aware layouts and reduces processing overhead

pdfplumber is a PDF data extraction library and layout analysis tool used to retrieve text, tables, and geometric objects from PDF files using precise coordinate-based analysis. It functions as a layout analyzer and table parser that identifies the bounding boxes and visual coordinates for every character and image on a page. The library distinguishes itself through visual debugging capabilities, allowing users to render PDF pages as images and draw annotations to verify the position of extracted data. It employs line and intersection analysis to identify cell structures and convert unstructu

PubTabNet is a large dataset for image-based table recognition, containing 568k+ images of tabular data annotated with the corresponding HTML representation of the tables. The table images are extracted from the scientific publications included in the PubMed Central Open Access Subset…