Awesome GitHub RepositoriesColumn Transformation

Polars is a high-performance columnar data processing library designed for efficient analytical workflows. It functions as a structured data library that organizes information into typed columns, utilizing the Apache Arrow memory format to enable zero-copy data sharing and cache-friendly, vectorized operations. The engine is built to handle large-scale tabular datasets, providing both local and distributed analytical runtimes that scale from single-machine environments to multi-node clusters. The project distinguishes itself through a sophisticated lazy query engine that constructs abstract e

This project is a machine learning array framework and tensor computation library designed for high-performance numerical computing. It provides a comprehensive suite of tools for constructing and training neural networks, featuring an automatic differentiation engine that facilitates gradient-based optimization and complex mathematical modeling. The library distinguishes itself through a unified memory architecture that allows data to be shared across CPU and GPU devices without explicit copies, significantly reducing data movement overhead. Its execution model relies on a lazy evaluation en

This is a machine learning educational repository consisting of a collection of notebooks and code examples. It provides practical implementations of diverse machine learning algorithms and workflows, ranging from traditional scientific computing to deep learning. The project features specific implementations of Scikit-Learn models, such as decision trees, random forests, and support vector machines, as well as TensorFlow examples for building neural networks, convolutional layers, and recurrent architectures. It also includes tutorials on reinforcement learning development and the creation o

Calibre is a comprehensive suite for digital library management, serving as a centralized hub for organizing, converting, and editing e-book collections. It functions as a multi-purpose platform that combines a relational database for metadata tracking with a powerful processing engine capable of transforming document formats and restructuring internal markup. Beyond local management, the software acts as a content server, enabling users to host their libraries over a network for remote access and reading via standard web browsers. The project distinguishes itself through its deep extensibili

This project is a Python-based framework that functions as a generative AI agent for programmatic data analysis. It enables users to interact with structured data sources through natural language prompts, translating these requests into executable code to perform analysis, data cleaning, and visualization. By maintaining conversational context across multi-turn interactions, the system allows for iterative exploration and the building of complex data narratives. The framework distinguishes itself through a robust semantic layer and secure execution model. It maps raw datasets to descriptive m

dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d

PRQL is a functional, modular data transformation language that serves as a compiler for relational data pipelines. It allows developers to write expressive, pipelined queries that are translated into standard SQL dialects. By abstracting complex data manipulation into a readable, sequential syntax, the project enables the construction of maintainable workflows that remain independent of specific database engines. The language distinguishes itself through a robust compilation infrastructure that performs type validation and relational algebra analysis before generating target-specific code. I

LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters

This project is an educational resource and a collection of instructional materials for performing data manipulation and statistical analysis using Python. It provides a comprehensive set of guides and code examples for using the Pandas, NumPy, and Matplotlib libraries to analyze structured data. The resource includes a dedicated guide for reshaping, cleaning, and aggregating tabular data and time series via Pandas, alongside a reference for high-performance vectorized operations and linear algebra using NumPy. It also features tutorials for creating publication-quality charts, distribution p

VisiData is a terminal-based interactive data analysis tool and browser designed for exploring, filtering, and sorting large tabular datasets. It functions as a structured data inspector that loads and flattens complex formats like JSON, XML, and PCAP into interactive sheets, as well as a terminal file manager for navigating directories and performing staged filesystem operations. The project distinguishes itself by rendering data visualizations, such as scatter plots and histograms, directly in the terminal using Unicode Braille characters. It provides a Python-based data wrangling environme

Open edX is a web-based learning management system and online course delivery platform that provides a complete environment for hosting, delivering, and managing online courses. It includes a dedicated course authoring studio for creating, organizing, and publishing course materials and assessments, along with a learning analytics dashboard for viewing visualizations and metrics of course data to analyze learner performance and engagement. The platform supports educational content structuring through tools for assembling course materials, and offers course data analytics capabilities for anal

InfoSpider is a personal data aggregator and digital footprint analyzer. It extracts user activity and history from social platforms and local browser database files to consolidate information into a unified format. The system functions as a social media archiving tool that converts feed data and albums from external links into downloadable PDF documents for offline preservation. It also serves as a browser history extractor that reads local SQLite database files to retrieve and analyze web navigation history. The project covers capabilities for data aggregation, digital footprint analysis,

Snapdom is a DOM capture engine that serializes live web page elements into images, videos, documents, and other formats. It converts any DOM subtree into PNG, JPG, WebP, SVG, PDF, or self-contained HTML, and can record animating elements as video files or animated GIFs using the browser's MediaRecorder API. The library distinguishes itself through a plugin-based architecture that allows custom output format handlers and pipeline hooks to extend the capture process without re-cloning the source element. It manages separate caches for images, styles, and fonts with preloading and clearing meth

notion-zh_CN is a localization proxy and translation layer designed to adapt the Notion interface for Chinese users. It functions as a serverless tool that intercepts network traffic to deliver translated interface text and localized content in real time. The project provides a specialized proxy worker that translates user interface elements and slash commands into Chinese. It further enables localized command discovery by mapping pinyin keystrokes to application functions, allowing users to trigger internal commands without using English. The system also manages network traffic routing to o

Lance is a versioned columnar data format and storage engine designed as a multimodal AI lakehouse. It serves as a vector database storage engine and a cloud object store dataset manager, organizing images, video, audio, and embeddings into a unified format optimized for machine learning workflows. The project distinguishes itself by combining a columnar layout for structured data with a specialized blob store for large multimodal tensors. It implements a hybrid search engine that integrates vector similarity search, full-text search, and SQL analytics on a single dataset, supported by a stor

SQLiteStudio is an open-source graphical tool for browsing, editing, and managing SQLite database files. It combines a full-featured SQL editor with syntax highlighting, a visual database schema designer for creating entity-relationship diagrams, and a plugin-based extensibility platform that allows adding custom functionality through C/C++, JavaScript, Tcl, or Python. The application distinguishes itself through its multi-language scripting engine, which embeds JavaScript, Tcl, and Python interpreters to enable user-defined functions and scripts within SQL queries. It supports encrypted data

csvkit is a composable Unix-style command-line toolkit for converting, filtering, and analyzing CSV files directly from the terminal. It provides a suite of focused single-purpose commands that can be combined via pipes to build complex data processing workflows, with a modular architecture that includes a column-type inference engine for automatically detecting data types and a streaming-pipeline design for efficient handling of tabular data. The toolkit distinguishes itself through its SQL-engine abstraction layer, which allows users to run SQL queries directly against CSV files without req

Tridactyl is a Vim-like Firefox extension that provides a comprehensive keyboard-driven interface for browsing, tab management, and page interaction. It replaces traditional mouse-based navigation with Vim-style keybindings, an ex-mode command line, and a hint overlay system for selecting and interacting with page elements. The extension is built around a core infrastructure that includes a modal command parser, a keybinding configuration system, and a content-script command bridge for executing commands in page context. The extension distinguishes itself through its deep integration with Fir

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a config-driven system for instantiating components, orchestrating distributed training, and managing parameter-efficient fine-tuning with quantization support, all through YAML-based configurations and command-line overrides. The library distinguishes itself through its comprehensive post-training workflow orchestration, combining supervised fine-tuning, preference optimization (DPO, PPO, GRPO), knowledge distillation, and quantization-aware training in a single configurable pip

Cuberite is a high-performance multiplayer game server for Java Edition clients, designed to provide a low-memory and low-CPU environment for hosting shared virtual spaces. The server is built for cross-platform deployment across various operating systems and hardware types. It allows for the extension of game mechanics and server logic through a Lua scripting interface, enabling functionality changes without recompiling the core engine. The project includes tools for server administration via a remote console and world data management for analyzing statistics and optimizing save file storag