awesome-repositories.com
Blog
awesome-repositories.com

Découvrez les meilleurs dépôts open-source grâce à notre recherche par IA.

ExplorerRecherches sélectionnéesAlternatives open sourceLogiciels auto-hébergésBlogPlan du site
ProjetÀ proposNotre méthodologiePresseServeur MCP
Mentions légalesConfidentialitéConditions d'utilisation
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·
Rdatatable avatar

Rdatatable/data.table

0
View on GitHub↗
3,894 stars·1,045 forks·R·MPL-2.0·10 vuesr-datatable.com↗

Data.table

Ce projet est un framework de traitement de données tabulaires haute performance pour R, conçu pour gérer des jeux de données massifs avec efficacité mémoire et vitesse. Il fournit une structure de données améliorée qui utilise la sémantique de référence et la modification sur place pour effectuer des transformations complexes sans la surcharge de copies d'objets inutiles.

La bibliothèque se distingue par ses optimisations architecturales de bas niveau, incluant le traitement parallèle multi-threadé, le tri basé sur radix et l'analyse de fichiers mappés en mémoire. En déchargeant les routines critiques de manipulation et d'agrégation de données vers du code C compilé, elle permet une exécution rapide des tâches qui seraient autrement coûteuses en calcul. Son moteur principal prend en charge des opérations relationnelles avancées, telles que les jointures non-équi, glissantes et à intervalles chevauchants, parallèlement à l'indexation secondaire automatique pour accélérer l'accès répété aux données.

Au-delà de ses capacités de traitement principales, le projet offre une suite complète d'outils pour la gestion du cycle de vie des données. Cela inclut des utilitaires d'ingestion et de sérialisation à haute vitesse avec détection automatique de type, ainsi qu'un support spécialisé pour l'analyse de séries temporelles et l'agrégation multidimensionnelle. Le framework est conçu pour évoluer, permettant aux utilisateurs d'effectuer des opérations complexes de regroupement, de filtrage et de remodelage sur des jeux de données contenant des milliards de lignes tout en maintenant la stabilité et les performances du système.

Features

  • External File Imports - Reads text files into memory by automatically detecting separators, column types, and quote-escaping rules.
  • R Data Manipulation Libraries - Provides a high-performance framework for tabular data manipulation, aggregation, and relational joining within the R language.
  • Tabular Data Manipulations - Performs high-performance data wrangling, including filtering, aggregation, and reshaping, using efficient memory management and reference semantics.
  • Tabular Data Processors - Provides a high-performance tabular data processing framework for filtering, aggregating, and joining large datasets.

Recherche par IA

Explorez plus de dépôts awesome

Décrivez vos besoins en langage naturel — l'IA classe des milliers de projets open source sélectionnés par pertinence.

Start searching with AI
  • Filtrage par recherche binaire - Retrieves specific rows using indices or computations with options to return all, first, or last matches.
  • Data Import and Export - Provides efficient import and export of delimited files using high-performance parsing and serialization.
  • Data Pipeline Acceleration - Accelerates complex data reshaping and aggregation tasks using optimized C-based internal routines.
  • Delimited Text Parsing - Parses delimited text files into memory with automatic detection of separators and column types.
  • Data Reshaping Operations - Converts data between wide and long formats using melting and casting with pattern-based column selection.
  • Dataset Joins - Combines multiple datasets using various join modes while minimizing memory overhead and supporting complex merging.
  • E/S rapides de fichiers délimités - Provides high-speed ingestion of large delimited text files with automatic type detection and decompression.
  • File-Based Data Ingestion - Reads large files into memory at high speeds with automatic type detection and flexible parsing options.
  • Group-By Aggregations - Computes expressions across subsets of data defined by grouping variables using a concise, flexible syntax.
  • High-Performance Data Analysis - Orders rows using high-performance sorting algorithms to accelerate data processing tasks.
  • In-Memory Data Processors - Provides a high-speed in-memory engine for filtering, grouping, and reshaping large-scale datasets.
  • In-Place Data Modifiers - Modifies data structures in place without creating memory-intensive copies to improve performance during large-scale data processing.
  • Variable In-Place Mutations - Supports in-place mutation of data structures to avoid expensive object copying during transformations.
  • In-Place Data Mutators - Modify specific values or add new columns directly within the existing memory structure to avoid the performance cost of duplicating the entire dataset.
  • Sous-ensemble par recherche binaire basée sur des clés - Enables high-performance binary search subsetting by setting columns as keys to physically reorder data.
  • Large-Scale Data Computation - Enables high-speed processing and aggregation of massive datasets with billions of rows.
  • Sous-ensemble de table logique - Retrieves specific rows and columns based on logical conditions, keys, or variable names.
  • Agrégations de données multi-threadées - Distributes grouped computation tasks across multiple CPU cores to handle billions of rows efficiently.
  • In-Place - Updates data by reference to avoid expensive object copying and reduce memory overhead.
  • Grouped Query Executions - Filters rows and computes expressions across specific groups within a single, efficient operation.
  • Agrégations tabulaires évolutives - Computes summary statistics and groupings across billions of rows using multi-threaded execution and memory-efficient processing.
  • Secondary Indexes - Computes and stores secondary indices to accelerate data access without requiring full table reordering.
  • Table Joining Operations - Combines multiple datasets using equi, non-equi, rolling, range, or interval join methods.
  • Advanced Joins - Merges datasets using rolling, overlapping range, non-equi, or aggregate join logic.
  • Tabular Data Frames - Implements an enhanced, memory-efficient tabular data structure that supports in-place modification and accelerated binary search subsetting.
  • Tabular Data Wrangling - Provides high-performance tools for cleaning, transforming, and reshaping large tabular datasets.
  • Sous-ensemble de lignes et colonnes tabulaires - Filters rows and selects specific columns using a concise syntax for fast data retrieval.
  • In-Place Data Structures - Uses reference semantics and in-place modification to handle massive datasets with minimal memory overhead.
  • Memory Efficiency Strategies - Provides high-performance, memory-efficient utilities for importing and exporting tabular data.
  • Implémentations natives en C - Offloads critical data manipulation and aggregation routines to compiled C code for maximum execution speed.
  • Radix Sorts - Orders rows based on one or more columns using a fast internal radix sort algorithm.
  • Performance and Optimization - Accelerates grouping, rolling calculations, and transformations using high-performance internal execution paths.
  • Combinaison de tables basée sur les références - Concatenates tables side-by-side using reference semantics to avoid memory-intensive object copying.
  • 64-bit Integer Handling - Detects and preserves the precision of integers larger than 2^31 using a specialized 64-bit data type.
  • Batch Value Updates - Updates specific values without duplicating the entire object during the modification process.
  • Fusion multi-colonnes - Unpivots a dataset by converting multiple columns into row pairs or melting into multiple columns simultaneously.
  • Column Management - Provides comprehensive tools for reordering columns and handling missing data during table manipulation.
  • Remplacements de valeurs conditionnels - Evaluates logical conditions to replace values within columns based on specified criteria.
  • CSV Exporters - Export datasets to CSV files using multi-threaded processing to reduce write time.
  • Memory-Mapped File Access - Uses memory-mapped file access to sample and infer data structures before loading.
  • Inférence de type - Samples file contents using memory-mapped access to determine the most efficient data types before loading.
  • Data Filtering - Provides mechanisms for filtering data based on field conditions.
  • Data Format Transformations - Converts tabular data between wide and long formats using optimized casting and melting operations.
  • Table Format Translators - Transforms diverse data structures into a high-performance table format while maintaining internal element structures.
  • Automatic Type Detection - Automatically infers column data types during the ingestion process to optimize memory usage.
  • Conversions de types par lots - Performs batch type conversions across multiple columns simultaneously using pattern-based selection.
  • Grouped Analysis - Calculates expressions and filters data across specific groups using a concise and efficient syntax.
  • Pipeline Configuration Export and Import - Allows configuration of separators, missing value handling, and type coercion during data import and export.
  • Data Indexing Structures - Organizes data structures using keys to enable fast retrieval and efficient filtering.
  • Jointures par chevauchement d'intervalles - Merges two tables based on overlapping intervals or ranges.
  • Non-Equi Joins - Matches rows using comparison operators like inequalities to compare numeric ranges or dates.
  • Parallel Dataframe Operations - Utilizes multi-threading to speed up computationally intensive data processing tasks across large datasets.
  • Range Data Extraction - Retrieves raw data objects associated with a specified range of values.
  • Data Serialization and Parsing - Reads and writes structured files with automatic format detection, encoding support, and progress reporting.
  • High-Performance CSV Exporters - Writes data to CSV files using a high-performance writer.
  • Fusion récursive de tables - Joins a sequence of tables from left to right using specified join types and keys.
  • Import Filtering - Filters data by selecting or dropping columns during the reading process to optimize memory.
  • Projections d'importation au niveau des colonnes - Selects specific columns by name or index during the initial read process to reduce memory consumption.
  • Conditional Row Filters - Identifies and filters out records based on the absence of values within a specified set.
  • Date and Time Libraries - Handles dates and times using integer-based classes for improved performance and easier manipulation.
  • Extraction de composantes temporelles - Extracts and formats date and time components such as years, quarters, and weeks with optimized efficiency.
  • Duplicate Row Filtering - Removes duplicate records from result sets or counts unique values across columns.
  • Identification des lignes en double - Locates repeated entries by returning logical vectors or the index of the first duplicate.
  • Construction améliorée de Data Frame - Creates high-performance tabular structures from lists or direct function calls.
  • Paramétrage d'expression - Substitutes variables, function names, and character values within filtering and computation arguments using a provided environment.
  • Extensions de requêtes fonctionnelles - Filters and transforms datasets using any expression or external package function within a query.
  • Mise en cache globale de chaînes - Uses global string caching to minimize memory usage for repetitive text data during import.
  • Grouped Expression Executions - Runs arbitrary expressions and functions on subsets of data grouped by specific columns.
  • Grouped Aggregations - Calculates summary statistics across groups using optimized functions like sum and mean to increase execution speed.
  • Multi-Dimensional Aggregations - Summarizes data using grouping sets, cubes, and roll-ups with support for custom labels.
  • Grouped Function Application - Executes custom calculations on subsets of data within each group for complex analytical workflows.
  • Coercition tabulaire haute performance - Converts various data structures like matrices and lists into a high-performance tabular format with minimal overhead.
  • Cartesian Join Prevention - Blocks joins that would result in an explosive number of rows to protect system memory.
  • Value Filtering - Filters entries based on a predicate applied to their values within specified bounds.
  • Long-to-Wide Reshaping - Converts data from a long format back to a wide format and applies aggregate functions to handle multiple observations.
  • Casting haute performance - Transforms data from long to wide formats using a fast, optimized implementation.
  • Remodelage optimisé pour la mémoire - Transforms data by aggregating values and spreading them across new columns for memory efficiency.
  • Missing Data Removal - Drops rows containing missing values from a dataset using high-performance internal routines.
  • Missing Value Imputation - Replaces missing entries in vectors using strategies like last observation carried forward.
  • Coalescing Utilities - Fills missing data points by replacing them with the first available non-missing value from a set.
  • Ensembles de regroupement multidimensionnels - Calculates summaries using rollup, cube, and grouping set operations for multi-dimensional data analysis.
  • Importateurs de données multi-sources - Supports loading datasets from diverse sources including files, URLs, raw strings, and shell pipes.
  • Correspondance optimisée de vecteurs de chaînes - Finds the first occurrences of character strings using high-performance sorting algorithms.
  • Multi-threaded Matrix Operations - Distributes grouped computation and sorting tasks across multiple CPU cores for parallel processing.
  • Parallel Sorting - Orders datasets using multi-threaded radix sort algorithms for high-performance data alignment.
  • Automatic Indexing - Accelerates subsequent queries by generating and saving indices during the first execution of a filter operation.
  • Query Execution Optimizations - Applies automatic indexing and internal performance enhancements to accelerate filtering, grouping, and sorting.
  • Query Performance Tuning - Optimizes data retrieval through automatic secondary indexing and pre-allocation of column slots.
  • Extraction de données par expression régulière - Extracts specific patterns from text using named regular expressions to reshape data into structured formats.
  • Advanced Relational Joins - Supports advanced relational operations including non-equi, rolling, and overlapping interval joins for complex dataset merging.
  • Row Deletions - Removes specific rows from a table without creating a full copy to optimize memory usage.
  • Récupération d'optima de groupe - Retrieves the specific row containing the maximum or minimum value for every distinct group.
  • Group-Specific Row Extractions - Retrieves specific rows, such as the first or last entry, independently for every group in a dataset.
  • Indexation temporelle basée sur des entiers - Uses integer-based storage for temporal data to accelerate sorting operations and minimize memory footprint.
  • Integer-Based Date Classes - Provides specialized date and time classes that use integer storage for memory efficiency and faster arithmetic operations.
  • String Pattern Filters - Filters tabular data based on string patterns and regular expressions.
  • Conversions d'objets de séries temporelles - Converts tabular data into time-series objects by utilizing a temporal column as the primary index.
  • Structured Data File Extractors - Analyzes file layouts to automatically detect field separators, headers, and row counts.
  • Optimisations de jointure par chevauchement - Executes overlap joins and creates join tables to combine datasets efficiently.
  • Filtrage de lignes basé sur les jointures - Identifies rows that only exist in one table or overlap between two tables without combining columns.
  • Logique de jointure conditionnelle - Supports merging tables using relationships that change based on the specific characteristics of the data rows.
  • Table Stacking - Merges multiple tables vertically into a single large dataset for high-speed processing.
  • Column Structural Modifications - Modifies table columns directly within the existing memory structure to avoid expensive object copying.
  • Détection de doublons tabulaires - Identifies and counts repeated records or unique entries within a data table.
  • Sous-ensemble dynamique de colonnes - Enables dynamic column subsetting using regular expressions or logical functions for flexible data retrieval.
  • Time Series Analysis - Implements specialized tools for calculating rolling window aggregates and adaptive statistics on sequential data.
  • Arrondi temporel - Adjusts temporal data to the nearest interval, such as hours or months, to facilitate grouped analysis.
  • K-Nearest Value Search - Finds values with the minimum absolute difference to a target in sorted datasets.
  • Unique Value Counting - Identifies distinct entries and calculates their occurrence frequencies in a dataset.
  • Wide-to-Long Reshaping - Transforms wide-format data into long-format by collapsing multiple columns into key-value pairs.
  • Implémentations de Fast Casting - Reshapes data from a long format to a wide format using fast casting operations.
  • Pivotement multi-colonnes - Expands multiple value columns from a long format into a wide format in a single operation.
  • Rolling Statistics - Applies functions across a moving window of data to calculate trends and summaries.
  • 64-bit Integer Types - Detects high-precision numbers exceeding standard limits and automatically assigns them to specialized sixty-four bit data types.
  • Gzip File Read-Write Operations - Supports reading and writing of compressed tabular data formats like gzip and zip.
  • File Read and Write Operations - Imports and exports delimited text files at high speeds to facilitate efficient data ingestion and persistence.
  • Data Ranking Utilities - Assigns numerical ranks to data elements using various tie-breaking strategies for statistical analysis.
  • Rolling Window Functions - Computes moving averages, sums, and other windowed metrics across sequential data.
  • Fenêtres adaptatives - Calculates rolling window aggregates where the window size varies based on observation intervals.
  • Dimensionnement dynamique de fenêtre - Determines the width of a rolling window based on time-series indices.
  • Conditional Element Assignment - Performs fast element-wise conditional checks and value assignments across large datasets.
  • Remplacement de valeur multi-conditions - Evaluates a series of conditions and returns corresponding values based on the first true match.
  • Large Dataset Explorers - Manipulates and aggregates massive data structures with high memory efficiency and speed to support large-scale data analysis workflows.
  • Deep Copy Utilities - Enables the creation of fully independent, deep copies of datasets to prevent unintended side effects during in-place modifications.
  • Parallèle - Orders large datasets using a multi-threaded radix sort implementation for high-performance data alignment.
  • Data Manipulation - High-performance data manipulation syntax.
  • Bibliothèques numériques - Fast aggregation and manipulation of large datasets in R.
  • Package and Dependency Management - High-performance data manipulation package for R.

    • Historique des stars

    Graphique de l'historique des stars pour rdatatable/data.tableGraphique de l'historique des stars pour rdatatable/data.table

    Questions fréquentes

    Que fait rdatatable/data.table ?

    Ce projet est un framework de traitement de données tabulaires haute performance pour R, conçu pour gérer des jeux de données massifs avec efficacité mémoire et vitesse. Il fournit une structure de données améliorée qui utilise la sémantique de référence et la modification sur place pour effectuer des transformations complexes sans la surcharge de copies d'objets inutiles.

    Quelles sont les fonctionnalités principales de rdatatable/data.table ?

    Les fonctionnalités principales de rdatatable/data.table sont : External File Imports, R Data Manipulation Libraries, Tabular Data Manipulations, Tabular Data Processors, Filtrage par recherche binaire, Data Import and Export, Data Pipeline Acceleration, Delimited Text Parsing.

    Quelles sont les alternatives open-source à rdatatable/data.table ?

    Les alternatives open-source à rdatatable/data.table incluent : tidyverse/dplyr — dplyr is an R data manipulation library that provides a grammar for transforming tabular data frames. It functions as… apache/pinot — Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It… jtablesaw/tablesaw — Tablesaw is a Java dataframe library designed for manipulating, filtering, and aggregating structured data. It serves… datawhalechina/joyful-pandas — This project is a comprehensive pandas data analysis tutorial and instructional guide designed for learning data… kotlin/dataframe — This library is a data processing framework for the JVM that provides a type-safe environment for manipulating… hosseinmoein/dataframe — DataFrame is a C++ tabular data library and manipulation engine designed for managing heterogeneous data in contiguous…

    Alternatives open source à Data.table

    Projets open source similaires, classés selon le nombre de fonctionnalités partagées avec Data.table.
    • tidyverse/dplyrAvatar de tidyverse

      tidyverse/dplyr

      5,034Voir sur GitHub↗

      dplyr is an R data manipulation library that provides a grammar for transforming tabular data frames. It functions as an in-memory data frame processor and a relational data algebra tool, using a consistent set of verbs to filter, select, and summarize data. The project includes a SQL translation engine that converts high-level data manipulation expressions into optimized queries. This allows users to perform transformations directly on remote relational databases and cloud storage without pulling data locally. The library covers a broad range of tabular operations, including column mutation

      R
      Voir sur GitHub↗5,034
    • apache/pinotAvatar de apache

      apache/pinot

      6,098Voir sur GitHub↗

      Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer

      Java
      Voir sur GitHub↗6,098
    • jtablesaw/tablesawAvatar de jtablesaw

      jtablesaw/tablesaw

      3,753Voir sur GitHub↗

      Tablesaw is a Java dataframe library designed for manipulating, filtering, and aggregating structured data. It serves as a toolkit for statistical analysis, data visualization, and machine learning execution within the Java Virtual Machine. The project provides specialized tools for computing descriptive statistics and generating cross-tabulations. It includes a visualization library for creating histograms and scatter plots, as well as a framework for executing linear regression, clustering, and classification tasks through integration with statistical libraries. The library covers a broad

      Java
      Voir sur GitHub↗3,753
    • datawhalechina/joyful-pandasAvatar de datawhalechina

      datawhalechina/joyful-pandas

      5,164Voir sur GitHub↗

      This project is a comprehensive pandas data analysis tutorial and instructional guide designed for learning data manipulation and analysis. It serves as a tabular data processing guide and a manual for time series analysis, providing a structured approach to cleaning, merging, and transforming datasets. The repository functions as a data feature engineering course, providing tutorials on constructing and selecting dataset features to improve machine learning model performance. It also includes a vectorized data operations guide for performing element-wise mathematical computations and matrix

      Jupyter Notebookpandas
      Voir sur GitHub↗5,164
    Voir les 30 alternatives à Data.table→