Analytics, dataframes and notebooks

This project is an interactive data science environment that combines code execution, rich media visualization, and narrative documentation into a persistent, browser-based platform. It serves as a comprehensive educational resource for scientific computing, providing a framework for iterative data analysis and machine learning prototyping. The environment is distinguished by its focus on high-performance numerical computing, utilizing vectorized array operations and memory-mapped data structures to handle large-scale computations efficiently. It features a unified estimator interface that standardizes machine learning workflows, allowing users to build, train, and evaluate predictive models through consistent pipelines. Additionally, the project includes a configuration-driven visualization engine that separates aesthetic style definitions from data rendering, enabling the creation of publication-quality graphical outputs. Beyond its core modeling capabilities, the project provides an extensive exploratory programming toolkit. This includes dynamic namespace introspection, performance profiling, and interactive debugging tools that allow users to inspect object metadata and navigate code in real-time. The repository is structured as a collection of executable notebooks and technical documentation, designed to facilitate hands-on learning of data science techniques and programming workflows.

This project is an interactive educational textbook and comprehensive machine learning resource designed for deep learning education. It provides a structured curriculum that combines narrative prose with executable code, utilizing literate programming to create reproducible learning experiences within a collection of Jupyter Notebooks. The repository distinguishes itself by teaching machine learning through applied research and modular design. It demonstrates a callback-driven training loop, a declarative data-block pipeline, and a layered abstraction API that allows users to transition between high-level convenience functions and low-level control. By employing dynamic dispatching, the system automatically resolves processing logic based on input data structures, enabling users to experiment with advanced architectures and transition models into production environments. The curriculum covers a broad range of technical topics, including foundational neural network theory, computer vision, natural language processing, and tabular modeling. These concepts are explored through guided exercises that address both the implementation of modern algorithms and the practical considerations of deploying models for real-world use. The entire resource is authored as a series of interactive documents, allowing for hands-on experimentation directly within a browser-based notebook environment.

This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, while also examining the architectural patterns for both batch and stream processing pipelines. Beyond foundational theory, the project covers the implementation of event-driven systems, including event sourcing, log-structured storage, and message brokering. It addresses the complexities of maintaining system consistency, enforcing transactional integrity, and managing derived data views in environments prone to network failures and concurrency challenges. The documentation is available in multiple formats, including an exportable digital book version, to support study and reference across various devices.

Metabase is a business intelligence platform designed to connect to various storage systems and relational databases for data exploration, visualization, and reporting. It provides a centralized environment where users can build queries through a graphical interface or raw code, transforming raw information into interactive dashboards and charts. The platform is built to support self-service analytics, allowing non-technical team members to extract insights without requiring deep knowledge of database syntax. The platform distinguishes itself through a metadata-driven modeling layer that abstracts complex database schemas into user-friendly business entities. It includes an automated workflow engine that enables users to trigger external processes and update records directly from the interface, bridging the gap between data analysis and operational action. For organizations requiring external distribution, the software provides an embedded analytics solution that allows secure integration of dashboards into third-party websites and applications, supported by sandboxing to isolate visual components. Beyond core visualization, the system incorporates artificial intelligence to assist with query generation and data summarization through natural language interactions. It maintains strict data governance through granular role-based access control, ensuring that permissions are managed consistently across all connected information assets. The platform handles the full lifecycle of data retrieval, including orchestration, caching, and translation of high-level inputs into database-specific syntax.

LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters into a single ranked result set. The project covers a broad range of capabilities, including automated vector embedding generation, multimodal data ingestion, and large-scale feature engineering. Its search surface includes approximate nearest neighbor indexing, precision reranking, and late-interaction multivector retrieval. Additionally, it provides tools for dataset curation, model evaluation, and zero-copy data streaming for training loops. The database is accessible via multi-language SDKs and a standardized REST API, supporting deployments across local filesystems and cloud object storage providers.

Polars is a high-performance columnar data processing library designed for efficient analytical workflows. It functions as a structured data library that organizes information into typed columns, utilizing the Apache Arrow memory format to enable zero-copy data sharing and cache-friendly, vectorized operations. The engine is built to handle large-scale tabular datasets, providing both local and distributed analytical runtimes that scale from single-machine environments to multi-node clusters. The project distinguishes itself through a sophisticated lazy query engine that constructs abstract execution plans. By deferring data operations until collection, the engine performs predicate and projection pushdown to minimize memory overhead and data passes. It further optimizes performance through a multi-threaded parallel execution model and a streaming batch processor, which allows for the analysis of datasets that exceed available system memory by processing them in manageable chunks. The library provides a comprehensive expression framework for complex data engineering, supporting aggregation, arithmetic, and logical transformations across various data types, including nested structures and categorical data. It integrates with external systems through native connectivity for cloud storage, relational databases, and remote repositories, while offering diagnostic tools to visualize query plans and monitor performance. Polars is available as a native library with language bindings for Python and R, allowing users to integrate high-performance data manipulation into existing analytical pipelines without complex build steps.

Modin is a distributed dataframe library and parallel data processing engine designed to handle large datasets that exceed system memory. It functions as a distributed computing framework that parallelizes data manipulation tasks across multiple CPU cores or clusters to increase throughput and avoid memory errors. The project mirrors the Pandas API, allowing for the distribution of data workflows without changing core code logic. It utilizes a pluggable backend interface, which enables users to switch between different distributed execution engines to optimize performance based on available hardware. The library provides capabilities for out-of-core memory management and partition-based data distribution. These features allow it to process datasets larger than available RAM by loading and computing on data partitions from disk on demand.

ClickHouse is a high-performance, columnar analytical database designed for real-time query execution and large-scale data aggregation. It functions as a distributed data warehouse capable of processing petabytes of information, while also providing an embedded engine that integrates directly into applications for native query capabilities without external dependencies. The system is built to handle high-throughput ingestion and complex analytical workloads, delivering millisecond-level latency for interactive dashboards and operational monitoring. The platform distinguishes itself through advanced storage and execution techniques, including vectorized query processing and a merge tree storage engine that maintains performance during massive insertions. It features adaptive subcolumn mapping for semi-structured data and supports native vector search for machine learning and generative AI applications. To facilitate efficient data movement, the engine utilizes zero-copy shared memory buffers, minimizing overhead when interacting with external analytical tools or processing diverse file formats like Parquet, JSON, and Arrow. Beyond its core storage and processing capabilities, the project provides a comprehensive suite of tools for observability, security, and data integration. It includes built-in support for natural language querying, automated workflow orchestration for AI agents, and extensive diagnostic features for query plan inspection. The platform also offers robust cloud infrastructure management, including support for private networking, compliant deployment strategies, and integrated billing consolidation.

This project serves as a comprehensive textbook and educational resource for data analysis using the Python ecosystem. It provides a structured guide to manipulating, cleaning, and processing datasets, focusing on the core tools required for numerical computing and statistical analysis. The repository distinguishes itself by offering a collection of practical code examples and workflows that demonstrate how to perform complex data tasks. It covers the application of vectorized numerical computations, the management of time-indexed data, and the creation of statistical visualizations to communicate analytical findings. The content spans the full lifecycle of data science projects, including loading external data formats, aggregating and grouping information, and integrating statistical modeling libraries. These materials are presented through interactive notebooks that interleave narrative documentation with executable code to support reproducible analysis and skill building.

OpenBB is a financial data platform and investment research terminal designed to aggregate, normalize, and distribute market data across analytical workflows. It functions as a comprehensive ecosystem that bridges disparate financial data providers with custom applications, spreadsheets, and internal modeling infrastructure. The platform distinguishes itself through a provider-based data abstraction layer that normalizes heterogeneous financial APIs into a consistent, schema-driven format. This architecture supports quantitative research automation and the construction of interactive, widget-based dashboards, allowing users to maintain control over data within secure, self-hosted, or private infrastructure environments. Beyond its core terminal interface, the project provides a modular, plugin-driven architecture for integrating proprietary data feeds and external services. These capabilities enable the embedding of live market and historical datasets directly into custom software products and business intelligence platforms, ensuring consistent data availability for cross-platform analysis.

Nushell is a cross-platform shell and programming language designed to treat all input and output as structured data rather than raw text streams. By enforcing data types and command signatures, it provides a consistent environment for building robust, pipeline-oriented workflows. The shell allows users to chain commands that pass structured objects between stages, enabling complex data processing and automation tasks that remain predictable across different operating systems. What distinguishes the project is its focus on interactive data exploration and modular extensibility. Users can query, sort, and visualize local files, databases, and remote API responses directly within the terminal using native structured data primitives. The shell supports a plugin-based architecture that allows external binaries to register as native commands, alongside a module system that enables the creation of reusable, scoped command-line tools. These features are complemented by a flexible configuration system that allows for deep customization of the shell environment, including prompts, keybindings, and persistent settings. The platform provides a comprehensive suite of tools for managing data and execution flow. It includes built-in support for structured data manipulation, such as record and table operations, as well as advanced features like concurrent pipeline processing, background job management, and runtime error handling. The shell also offers a sophisticated line editor with support for modal editing and interactive menus to streamline command entry. Documentation and configuration are managed through standard files, allowing users to define custom commands, aliases, and environment variables that persist across sessions. The system is designed to integrate seamlessly with existing external commands, automatically converting between structured data and text or binary formats to maintain compatibility with standard system utilities.

This project is an open-source, privacy-focused web analytics platform designed for high-throughput data ingestion and multi-tenant data management. It provides a cookie-less tracking engine that captures visitor interactions using ephemeral request metadata, ensuring comprehensive traffic visibility while maintaining strict privacy standards. The architecture utilizes an event-driven ingestion pipeline and aggregated metric storage to decouple data collection from processing, enabling efficient long-term retrieval and responsive dashboard performance. What distinguishes this platform is its emphasis on first-party data collection and proxy-based routing. By allowing tracking requests to be routed through a custom domain, the system effectively masks analytics traffic as internal requests, bypassing ad-blocking software and privacy filters that typically interfere with client-side scripts. This approach, combined with server-side event processing, ensures that site owners maintain accurate traffic data even when browser-based limitations are present. The platform offers a broad capability surface for managing complex organizational needs, including granular role-based access control, SAML-based single sign-on, and automated reporting workflows. Users can programmatically manage site configurations, integrate external data sources, and export raw event logs for deep analysis in third-party business intelligence tools. The system also supports advanced conversion funnel tracking, allowing teams to define and measure specific user journeys and revenue-generating actions across multiple websites from a centralized dashboard.

This is a pandas-based technical analysis library and financial feature engineering tool. It serves as a vectorized indicator calculator that transforms raw price and volume data into derived metrics for time series analysis. The library uses a NumPy-based engine to perform mathematical operations across entire arrays, avoiding iterative loops to maintain high performance. It organizes technical indicators into a modular class hierarchy with a consistent interface, allowing for bulk feature generation and the direct appending of results as new columns to a pandas DataFrame. The system covers a wide range of financial metrics, including momentum oscillators, asset return metrics, and trend direction indicators. It also provides tools for measuring volatility through statistical bands and analyzing market pressure via volume-weighted metrics. To ensure dataset completeness for machine learning, the processor includes configurable strategies for filling missing values and allows for the tuning of indicator parameters such as moving average periods.

Umami is a self-hosted, privacy-focused web analytics platform designed to provide full control over infrastructure and user data. It captures website traffic and visitor behavior through anonymous tracking methods that avoid cookies, browser fingerprinting, and the storage of personally identifiable information. The platform distinguishes itself through a comprehensive suite of behavioral analysis tools, including session replays, heatmaps, and cohort-based retention reporting. It features a multi-tenant architecture that allows teams to manage multiple websites within a single, collaborative dashboard, supported by granular role-based access controls and the ability to share specific insights via public links. Beyond core traffic monitoring, the system includes a robust event tracking framework for capturing custom user interactions, conversion funnels, and marketing campaign attribution. It also provides diagnostic capabilities for web performance, allowing users to track core web vitals and troubleshoot data collection through detailed session logs and visitor activity searches. The software supports flexible deployment strategies, including containerized installations and source-code-based setups, and can be integrated into various environments via a standard API or pre-built plugins.

VectorBT is a vectorized trading strategy backtesting framework that simulates thousands of strategy configurations in a single pass over historical price data. It operates as a parameter optimization engine, a portfolio performance analyzer, a technical indicator calculator, and a financial data fetcher, all built around a DataFrame-centric data model that uses NumPy broadcasting for signal alignment and compiled code acceleration for performance. The framework distinguishes itself through its ability to run large-scale parameter sweeps by constructing every combination of strategy parameters as a single array dimension, enabling one-pass evaluation of the full grid. It includes a walk-forward validation framework for testing strategy robustness across changing market conditions, and generates interactive visualizations using Plotly for exploring backtest results and indicators. The project also provides external data source abstraction for fetching market data from providers like Yahoo Finance. Beyond its core backtesting and optimization capabilities, VectorBT supports computing custom technical indicators, generating crossover trading signals, and analyzing portfolio performance with trade-level metrics and drawdown statistics. It can schedule recurring analyses and send notifications through Telegram, and offers a one-line backtesting interface for quick strategy evaluation.

DuckDB is an in-process analytical database engine designed to run directly within an application process. As a zero-dependency, embedded system, it provides enterprise-grade SQL data processing capabilities without the overhead of managing a dedicated database server. It is built to handle complex analytical and aggregation tasks by storing and retrieving information in columns, allowing for high-performance relational data manipulation. The engine distinguishes itself through a columnar vectorized execution model that maximizes CPU cache efficiency during query operations. It employs adaptive query optimization to dynamically select execution plans at runtime and utilizes zero-copy ingestion to map external data formats directly into memory. To facilitate integration with analytical programming environments, the system supports high-performance data exchange through standardized memory formats and provides specialized connectors for Python, R, and Java. The project covers a broad capability surface, including advanced relational join operations, incremental result streaming for large datasets, and flexible data ingestion from various file formats. It supports complex data types and provides a comprehensive command-line interface for interactive session management and batch processing. The codebase is designed for portability, offering single-file amalgamation to simplify integration into external projects and build systems.

VisiData is a terminal-based interactive data analysis tool and browser designed for exploring, filtering, and sorting large tabular datasets. It functions as a structured data inspector that loads and flattens complex formats like JSON, XML, and PCAP into interactive sheets, as well as a terminal file manager for navigating directories and performing staged filesystem operations. The project distinguishes itself by rendering data visualizations, such as scatter plots and histograms, directly in the terminal using Unicode Braille characters. It provides a Python-based data wrangling environment where users can clean and transform datasets using Python expressions and regular expressions to calculate new values or split columns. Broad capabilities include exploratory data analysis through pivot tables and summary statistics, as well as data management via SQL database connections and Pandas integration. The system also supports command-based macro recording, a plugin architecture for extending application logic, and the ability to process tabular data within shell pipelines.

Pandas is a high-performance data analysis library that provides a comprehensive framework for manipulating, cleaning, and transforming structured datasets. It centers on labeled one-dimensional and two-dimensional data structures, allowing users to construct, filter, and reshape tabular information while performing complex arithmetic and logical operations. The library distinguishes itself through a sophisticated indexing engine that enables automatic data alignment during calculations and relational merges. By utilizing a block-based memory layout, it optimizes cache locality for vectorized operations across columns. Its capabilities extend to a robust split-apply-combine pattern for grouping, as well as specialized tools for time series analysis that handle calendar-aware offsets, frequency resampling, and time zone management. Beyond core manipulation, the project offers extensive support for data lifecycle management, including ingestion and serialization across diverse file formats and database systems. It provides advanced features for hierarchical multi-index mapping, relational joins, and flexible missing data handling, ensuring that datasets are normalized and ready for statistical or analytical workflows.

Pygwalker is a library that transforms tabular data into interactive, drag-and-drop interfaces for exploratory analysis and visualization. It functions as a grammar-based framework that translates user interactions into declarative chart definitions, allowing for the creation of dynamic data exploration environments directly within notebooks or embedded web applications. The system distinguishes itself by offloading heavy analytical computations to backend kernels, which maintains responsiveness when visualizing large datasets. It supports the serialization of visual states into portable configurations, enabling developers to save, share, and restore specific chart layouts and data views across different sessions. Beyond core exploration, the project provides capabilities for embedding self-service analytical tools into web applications, allowing end-users to manipulate data tables through graphical interfaces. It includes options for read-only modes and automated workflow management to support diverse data analysis requirements.

This project is a research-oriented repository that serves as a centralized database for system-level prompts and internal behavioral instructions extracted from various large language models. Its primary purpose is to provide a transparent, accessible reference for researchers and developers to study how artificial intelligence models are configured, constrained, and governed. The repository distinguishes itself by cataloging the hidden directives and operational guidelines that define model personas and safety boundaries. By archiving these instruction sets, it enables comparative analysis of how different models maintain their internal logic and respond to user interactions. The project functions as a resource for investigating the transparency of AI systems, offering a structured collection of data that helps clarify the underlying mechanisms of model behavior. It supports the broader goal of understanding the configuration and constraints inherent in modern language models.