1 个仓库
Awesome GitHub RepositoriesAnalysis Dataset Optimization
Processes for merging data files and filtering fields to reduce memory footprint before database ingestion.
Distinct from Dataset Preparation Scripts: Focuses on memory optimization and field filtering for analysis, whereas Dataset Preparation Scripts focus on downloading and organizing public datasets.
Explore 1 awesome GitHub repository matching data & databases · Analysis Dataset Optimization. Refine with filters or upvote what's useful.
Awesome Analysis Dataset Optimization GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- turboway/bigdata_analyse
TurboWay/bigdata_analyse
5,238在 GitHub 上查看This project is a collection of big data frameworks and pipelines, including an Apache Hive analysis framework, a behavioral data analytics platform, a predictive analytics engine, and real-time data pipelines. It provides the infrastructure for building Extract, Transform, Load (ETL) workflows to process large datasets for distributed storage and SQL-based analysis. The system supports diverse analytical implementations, such as a predictive engine using linear regression for value forecasting and a real-time architecture that moves data through message brokers for immediate reporting. It in
Includes processes to merge data files and filter fields to optimize memory usage before loading into databases.
Pythonhqlpythonsql
