30 open-source projects similar to twitter/scalding, ranked by how many features they have in common. Compare stars, activity and what each one does to find the best Scalding alternative.
Streaming MapReduce with Scalding and Storm
ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.
Ray is a distributed computing framework designed to scale Python and Java applications across clusters by abstracting task scheduling and resource management. It functions as a resource-aware execution engine that manages task dependencies, placement, and fault tolerance across networked compute nodes. At its core, the system provides a stateful actor model, allowing developers to define classes that run in dedicated processes to maintain and mutate internal state across remote method calls. The framework distinguishes itself through a robust cross-language interoperability layer, enabling f
Breeze is/was a numerical processing library for Scala.
This repository is a comprehensive collection of instructional guides and practical examples for Python development, focusing on machine learning, data science, and web scraping. It provides implementations for neural networks, reinforcement learning algorithms, and deep learning architectures using PyTorch, alongside detailed manuals for scientific computing and data visualization. The project distinguishes itself by offering specialized tutorials on concurrent programming to optimize CPU performance and guides for setting up Linux development environments. It covers the implementation of ad
This repository serves as an educational collection of interactive notebooks and code examples designed to demonstrate fundamental machine learning and deep learning concepts. It provides a structured environment for exploring data science workflows, ranging from basic numerical computing and statistical analysis to the construction of complex neural network architectures. The project distinguishes itself through a focus on hands-on experimentation, offering practical implementations for tasks such as computer vision, natural language processing, and statistical simulation. Users can engage w
This project provides a suite of interfaces and tools for accessing electricity carbon intensity and production metrics. It includes an API for real-time and historical data, a geographic power data map for visualizing regional carbon intensity and renewable energy percentages, and a system for extracting datasets required for standardized greenhouse gas emissions reporting. The project features an interactive API sandbox that allows users to test requests and inspect data responses without writing code. It also includes mechanisms for institutional email verification to manage access to hist
This project is a collection of educational notes and tutorials focused on Python programming, scientific computing, and data analysis. It serves as a reference for learning language basics, advanced techniques, and object-oriented design. The materials include implementation guides for building linear, logistic, and convolutional neural networks using symbolic graph frameworks. It also provides instruction on manipulating and visualizing structured data frames and performing complex mathematical operations through numerical libraries. The repository includes a system for converting interact
This project is a collection of educational resources and reference implementations for neural network development using TensorFlow. It serves as a comprehensive learning course, machine learning curriculum, and practical implementation guide for building deep learning architectures. The codebase provides instructional materials and examples covering a wide range of model types, including convolutional neural networks for image classification, recurrent networks and long short-term memory cells for sequential data, and autoencoders for generative modeling. It also includes implementations for
This project is a comprehensive technical reference and programming cheatsheet for the Python language. It serves as a curated catalog of language features, syntax patterns, and standard library functions designed to help developers identify and apply correct coding patterns. The documentation covers a broad range of functional areas, including language fundamentals such as object-oriented structuring, functional logic, and list comprehensions. It also provides guidance on utilizing the standard library for data analysis, file management, networking, and concurrent execution. The reference e
Apache MXNet is a deep learning framework and distributed machine learning library designed for training and deploying neural networks across distributed systems, mobile devices, and hardware accelerators. It functions as a cross-platform runtime and a dynamic dataflow scheduler that optimizes neural network execution. The framework provides a multi-language API, enabling the development of machine learning models using Python, R, Julia, Scala, Go, and JavaScript. It supports high-performance model training and the scaling of workloads across multiple GPUs and machines. The system covers cap
Apache Flink is a distributed processing engine designed for both high-throughput, low-latency data streams and finite batch workloads. It functions as a stateful stream processor and a SQL stream processing engine, providing a unified runtime to execute relational queries and event-based transformations. The system is distinguished by its ability to manage persistent operator state to ensure exactly-once processing guarantees and consistency during failures. It features specialized capabilities for complex event processing to detect temporal patterns and handles out-of-order events using eve
Build LLM-powered applications in Ruby
Aerosolve is a machine learning framework designed for training and deploying interpretable models. It functions as a feature engineering tool and a model trainer that utilizes sparse feature modeling to simplify weight debugging and accelerate data iteration. The system includes a specialized domain-specific transformation language for converting raw data families into model-ready representations. It also provides capabilities for visual content analysis by mapping images into dense high-dimensional vector spaces to rank and organize data by style or content. The framework allows for human-
Hyperparameter Experiments with TensorFlow and Keras
A collection of machine-learned models for use in autonomous driving applications.
Einops is a tensor manipulation library that provides a framework-agnostic interface for reshaping, Einstein summation, and multi-dimensional array operations. It serves as an abstraction layer that works across NumPy, PyTorch, TensorFlow, and JAX, allowing for tensor transformations without changing the API. The library distinguishes itself through a declarative notation system that uses readable string patterns to describe tensor rearrangements and reductions. This approach includes an extended Einstein summation interface that supports multi-letter axis names and a named dimension mapping