31 repository-uri
Capabilities for querying and loading data directly from flat files like CSV and Parquet.
Distinct from File-to-Table Importers: Focuses on the ability to treat files as database tables via SQL, which is distinct from schema validation or generic file importers.
Explore 31 awesome GitHub repositories matching data & databases · File-Based Data Import. Refine with filters or upvote what's useful.
DuckDB este o bază de date SQL analitică încorporată, în proces, și un sistem de gestionare a bazelor de date OLAP. Funcționează ca un motor de date pentru fișiere Parquet și CSV, permițând utilizatorilor să execute interogări SQL complexe pe seturi de date mari fără a necesita un proces de server separat. Sistemul este conceput pentru procesare analitică locală și fluxuri de lucru de știință a datelor încorporate. Acesta permite interogarea și analiza directă a fișierelor Parquet și CSV de pe disc, evitând necesitatea de a încărca datele într-o bază de date permanentă. Motorul oferă execuție SQL analitică de înaltă performanță, inclusiv suport pentru funcții de fereastră și subinterogări imbricate. Acesta încorporează un layout de stocare pe coloane și execuție vectorială a interogărilor pentru a gestiona manipularea și explorarea datelor la scară largă. Baza de date este accesibilă printr-o interfață de linie de comandă autonomă și binding-uri specifice limbajelor pentru Python, R, Java și Wasm.
Enables direct loading and querying of CSV and Parquet files by referencing them within SQL queries.
This project is a collection of educational resources and reference implementations for the Apache Flink stream processing framework. It provides a learning resource focused on mastering distributed stream processing through implementation guides, performance tuning tutorials, and practical examples. The repository features detailed walkthroughs for building real-time data pipelines using the DataStream and Table APIs. It includes specific integration examples for connecting Apache Flink with Kafka brokers and Elasticsearch indices, as well as reference implementations for real-time deduplica
Reads text or formatted files from a path to process data once or continuously as a stream.
This project is an automated machine learning framework and toolkit designed for training and tuning custom models for classification, regression, and recommendations. It functions as a multimodal machine learning toolkit capable of processing and training models using a combination of text, image, audio, and sensor data. The framework distinguishes itself as a multimodal data processor that can handle and visualize large datasets on a single machine using column-oriented disk storage. It includes a core machine learning model generator that converts trained models into formats compatible wit
Loads information from local or remote files into a scalable tabular structure.
Akaunting is a modular business enterprise resource planning system and self-hosted accounting software. It provides a comprehensive platform for small business financial management, centering on a double-entry bookkeeping system with a general ledger and chart of accounts. The platform is designed for extensibility through a module-based architecture and a dedicated marketplace for procuring third-party applications. It supports multi-tenant data isolation and utilizes role-based access control to manage granular user permissions. Its capability surface covers a wide range of business opera
Provides a way to save custom field mappings for specific file structures to reuse for future data uploads.
This project is an AI-powered IDE extension and LLM coding assistant that provides a conversational interface for generating, refactoring, and debugging code. It functions as an AI agent framework and a Model Context Protocol client, connecting AI models to external data sources and tools to automate complex development tasks. The system is distinguished by its use of autonomous AI agents capable of multi-step task execution, including the ability to read files, modify code, and run terminal commands iteratively. It supports recursive agent orchestration through subagent delegation and employ
Allows opening datasets from local CSV and Excel files or notebook variables for analysis.
AI Town is a TypeScript-based simulation engine used to create virtual environments where autonomous characters interact and socialize. It functions as a framework for orchestrating multiple AI agents within a persistent digital world, utilizing language models and a game engine to drive character behavior and social interactions. The project differentiates itself through a dedicated agent sandbox and a vector database agent store, which allow for the management of agent memories and world state. It integrates generative AI for background music and provides tools for simulation world design,
Provides capabilities to load data from files into specified database tables.
Apache JMeter is a Java-based performance testing tool and multi-protocol traffic simulator used to analyze the stability and scalability of servers and networks. It functions as a distributed load testing framework that coordinates remote worker nodes from a single controller to generate high volumes of concurrent traffic. The project is distinguished by its ability to simulate traffic across diverse backend systems, including HTTP, JDBC, LDAP, JMS, FTP, and TCP. It provides a headless command-line interface for automated execution and a reporting system that transforms raw sample logs into
Loads comma-separated values from external files to provide unique data for test iterations.
RedisInsight is a graphical user interface and management tool for browsing, analyzing, and administering Redis databases. It provides a visual environment for exploring key-value data structures, managing database instances, and performing data analysis across different operating systems and deployments. The tool distinguishes itself by providing dedicated visual managers for complex operations, including a vector database manager for configuring embeddings and similarity searches, a query workbench for executing raw commands and Lua scripts, and a performance monitoring dashboard for tracki
Reads content from standard input or text files to set key values and execute commands.
Dejavu is a containerized administration panel and web interface for managing data within Elasticsearch and OpenSearch clusters. It serves as a search index management tool for browsing, editing, and deleting records through a visual explorer rather than raw API queries. The project distinguishes itself by providing a search interface prototyping tool. This allows users to visually design search screens to test result relevancy and export the final layout configuration as usable code. The tool covers broad data management capabilities, including structured data import from CSV or JSON files
Supports loading structured data from CSV or JSON files into search clusters via guided mapping.
JimuReport is an open-source reporting and dashboard engine designed to be embedded directly into Spring Boot applications. Its core identity centers on generating data reports and full-screen dashboards from natural language descriptions, eliminating the need for manual design. The platform also provides a conversational query interface that translates plain-language questions into database queries, returning results as tables and charts without requiring SQL knowledge. What distinguishes JimuReport is its integration of AI skills that can be installed with a single command, enabling report
Loads data from Excel, CSV, and JSON files as a data source for report generation.
Dawarich is a self-hosted location history manager and travel journaling platform. It functions as a personal travel archive that collects GPS coordinates and movement data, providing a private alternative to proprietary tracking services. The system utilizes a PostgreSQL geospatial database to store coordinates, visits, and custom geofence boundaries. The project distinguishes itself as a geospatial data converter and visualization tool, capable of transforming location history between formats such as GPX, KML, and GeoJSON. It allows users to organize GPS tracks and geotagged photos into nam
Combines multiple GPS exchange files into a single unified track within the browser.
Azure Data Studio is a cross-platform SQL database management IDE used for writing queries, managing schemas, and administering relational databases. It functions as a comprehensive environment for relational database management, providing a structured interface for executing SQL queries and browsing database objects. The platform is distinguished by its interactive data notebooks, which combine executable code cells, narrative text, and visualizations for data analysis. It also includes specialized tools for database migration, allowing users to assess and transfer schemas and data from on-p
Includes a guided wizard to load data from CSV, text, and JSON files into database tables.
AlaSQL is a JavaScript SQL database engine that allows for the filtering, grouping, and joining of in-memory object arrays and JSON data. It functions as an in-memory SQL database and client-side data processor, enabling the execution of SQL statements against JavaScript arrays and external data sources in both browser and server environments. The project serves as a universal data query tool capable of performing relational joins across diverse sources, such as merging Google Spreadsheets, SQLite files, and remote APIs into a single result set. It also acts as an IndexedDB SQL wrapper, allow
Imports data from legacy .xls files directly into the database for SQL querying.
IPFS Desktop este un client grafic pentru stocarea, preluarea și gestionarea conținutului pe o rețea peer-to-peer. Funcționează ca un sistem de fișiere distribuit și instrument de administrare a nodurilor, permițând utilizatorilor să ruleze un nod local și să gestioneze date adresate prin conținut fără a utiliza o linie de comandă. Aplicația include un vizualizator de rețea pentru monitorizarea peer-ilor conectați și analizarea topologiei rețelei globale. Servește, de asemenea, ca handler de protocol de sistem capabil să intercepteze și să proceseze URI-uri ipfs și ipns. Software-ul acoperă stocarea descentralizată a fișierelor și găzduirea distribuită a conținutului, oferind instrumente pentru importul datelor prin drag-and-drop sau arhive binare. Oferă utilitare pentru organizarea fișierelor, pinning-ul conținutului la distanță și colectarea gunoiului (garbage collection) pentru a gestiona spațiul pe disc. Capabilitățile de gestionare a rețelei includ configurarea peering-ului permanent, PubSub și setările IPNS.
Enables loading specific data sets into a node using Content Addressable Archive files.
Speedscope is a web-based performance profiler that visualizes profiling data through interactive flamegraphs and timeline views. It ingests performance profiles from a wide range of sources, including Chrome, Firefox, Safari, Node.js, .NET Core, Instruments, Hermes, GHC, and Ruby, normalizing them into a common schema for unified analysis. The tool distinguishes itself with a canvas-based rendering engine that draws flamegraphs without DOM nodes for each frame, and a WebAssembly-based rendering pipeline for high-performance drawing. It offers left-heavy stack sorting to surface the most time
Captures JavaScript performance data from Hermes engine in development mode via the React Native Dev Menu and saves it as a .cpuprofile file for analysis.
SQLiteStudio is an open-source graphical tool for browsing, editing, and managing SQLite database files. It combines a full-featured SQL editor with syntax highlighting, a visual database schema designer for creating entity-relationship diagrams, and a plugin-based extensibility platform that allows adding custom functionality through C/C++, JavaScript, Tcl, or Python. The application distinguishes itself through its multi-language scripting engine, which embeds JavaScript, Tcl, and Python interpreters to enable user-defined functions and scripts within SQL queries. It supports encrypted data
Imports data from files into database tables, creating the table if it does not exist.
InvenTree is an open-source inventory management platform built on Django, designed for tracking parts, stock levels, and supply chain operations through a web interface and REST API. The system uses barcodes—including QR codes, 1D barcodes, and Data Matrix codes—as primary identifiers for scanning, linking, and triggering inventory actions, and extends core functionality through a Python plugin framework supporting custom actions, UI panels, barcode handlers, and scheduled tasks. The platform distinguishes itself through a comprehensive plugin-based extensibility system that allows custom in
InvenTree uses a data import wizard to step through selecting a file and mapping its data to create parts.
Apache Hive is a SQL-on-Hadoop data warehouse that enables querying and managing petabytes of data stored in distributed storage such as HDFS and cloud storage services. It provides a familiar SQL interface for batch analytics and reporting, supported by a core set of components including the HiveServer2 Thrift service for remote query execution, the Hive Metastore Service for central metadata management, the Hive ACID Transaction Engine for concurrent read-write operations, and the Hive LLAP Interactive Engine for low-latency analytical processing. The WebHCat REST API offers an HTTP interfac
Imports data from HDFS files into Hive tables using external table definitions or load commands.
Acest proiect este un curriculum educațional de machine learning și o platformă de învățare livrată prin Jupyter Notebooks interactive. Servește drept ghid cuprinzător pentru stăpânirea toolkit-ului de data science Python, oferind tutoriale structurate pentru calcul numeric, manipularea datelor tabelare și vizualizarea statistică. Curriculum-ul include ghiduri specifice de implementare pentru Scikit-Learn și un curs practic despre TensorFlow pentru construirea, antrenarea și deployment-ul rețelelor neuronale și a modelelor de computer vision. Acoperă procesul end-to-end de construire a modelelor predictive, de la formularea inițială a problemei și categorizarea sarcinilor până la deployment-ul modelelor prin interfețe web interactive. Proiectul acoperă o suprafață largă de capabilități, inclusiv calcul numeric cu array-uri multidimensionale, analiză exploratorie a datelor și rutine de preprocesare a datelor. Oferă fluxuri de lucru detaliate pentru învățarea supervizată și nesupervizată, pipeline-uri de machine learning automatizat, optimizarea hiperparametrilor și evaluarea modelelor folosind metrici de clasificare și cross-validation. Conținutul educațional este organizat ca o serie de notebook-uri care intercalează codul Python cu explicații narative pentru a documenta fluxurile de lucru în data science.
Provides capabilities for querying and loading data directly from flat files like CSV and Excel.
Reth is a modular, high-performance Ethereum execution layer client written in Rust. It serves as a full Ethereum node that syncs, validates, and serves blockchain data, functioning as an archive node implementation, a high-throughput RPC node server, and a snapshot sync tool. The project is built around a modular component architecture that allows assembling custom node behavior by swapping independent Rust crates for consensus, execution, mempool, and networking. The client distinguishes itself through a staged sync pipeline that downloads headers and bodies online before processing the res
Reads previously exported chain data files and writes their blocks and state into the local database.