alibaba/graph-learn
Graph Learn
Graph-learn is a distributed graph processing engine and graph neural network framework designed for large-scale graph data. It provides specialized query interfaces to extract training subgraphs and node neighborhoods, enabling the construction and training of complex graph neural network models on massive datasets. The system integrates a real-time inference server to serve live predictions by sampling dynamic graphs with low latency while processing streaming graph updates.
The project features a C++ core engine integration that executes graph sampling and tensor operations natively, coupled with a distributed graph partitioning engine that fragments graph topologies across multiple worker nodes for parallel traversal. It includes a neighbor sampling pipeline that feeds balanced mini-batches into neural network training loops, alongside a partition-based distributed storage layer with local neighborhood indexing. A high-level Python API bridges user-defined workflows with the underlying distributed execution engine.
Features
- Graph Neural Networks - Enables the construction and training of complex graph neural network models on massive datasets.
- Neighborhood Sampling - Extracts localized subgraphs and node neighborhoods on demand to feed balanced mini-batches into training loops.
- Distributed Graph Engines - Fragments massive graph topologies across multiple cluster nodes for parallel traversal and model training.
- Graph Neural Network Training Routines - Supports building and training complex graph neural network models on massive datasets using specialized sampling.
- Graph Neighborhood Sampling - Extracts training subgraphs and node neighborhoods using specialized query interfaces for graph models.
- Distributed Graph Storage - Stores fragmented graph topologies across worker nodes with local neighborhood indexing for fast parallel access.
- Graph-Augmented Inference - Serves live predictions by sampling dynamic graphs with low latency and processing streaming updates.
- Real-Time Prediction Serving - Serves live graph predictions with low latency by querying dynamic graph structures and streaming updates.
- C++ Core Implementations - Executes heavy graph sampling and tensor operations natively in C++ for maximum execution throughput.
