6 repository-uri
Tools for segmenting large graph structures into smaller subgraphs for memory-efficient processing.
Distinct from Graph Processing: Distinct from general graph processing: focuses specifically on memory-efficient partitioning for mini-batch training.
Explore 6 awesome GitHub repositories matching data & databases · Graph Partitioning Utilities. Refine with filters or upvote what's useful.
This project is a deep learning library designed for training neural networks on irregular data structures, including graphs, 3D meshes, and point clouds. It functions as an extension to the PyTorch framework, providing specialized layers and kernels that enable the processing of complex, non-Euclidean information. The library distinguishes itself through a geometric deep learning toolkit that manages the unique requirements of graph-based data. It utilizes sparse matrix-based message passing to aggregate information across nodes and employs dynamic computational graph construction to accommo
Segments large graph structures into smaller subgraphs to allow memory-efficient processing of datasets that exceed single-device capacity.
DGL is a Python library for building and training graph neural networks. It functions as a graph message passing framework and a geometric deep learning tool, enabling the development of models that analyze graph-structured data. The library is designed for large-scale graph processing, utilizing distributed training and neighbor sampling to handle datasets with billions of edges. It provides specialized support for heterogeneous graph modeling, allowing for the representation of complex real-world entities with multiple node and edge types. Its capabilities cover a wide range of graph tasks
Implements utilities for splitting large-scale graphs across multiple machines to enable training on massive datasets.
Titan este o bază de date graf distribuită și un motor de calcul conceput pentru stocarea și interogarea seturilor masive de date de noduri și muchii interconectate în clustere multi-mașină. Funcționează ca un strat de stocare graf scalabil și un magazin tranzacțional, oferind un framework pentru executarea sarcinilor de procesare graf la scară largă și a traversărilor profunde. Sistemul se distinge prin backend-ul său de stocare pluggable, care decuplează motorul graf de stratul de persistență fizică. Utilizează partiționarea datelor de tip vertex-cut pentru a echilibra sarcinile de procesare și un model de proprietăți set-cardinality care permite proprietăților unice să stocheze mai multe valori. Platforma acoperă o gamă largă de capabilități, inclusiv indexarea graf multi-model pentru căutări geografice și full-text, gestionarea globală a schemei pentru re-indexarea seturilor de date și operațiuni tranzacționale asigurate prin write-ahead logging. De asemenea, încorporează expirarea elementelor prin setări de tip time-to-live și monitorizarea performanței sistemului pentru urmărirea activității de interogare și a latenței tranzacțiilor.
Balances cluster load by segmenting large graph structures into smaller subgraphs using vertex or edge cuts.
AITemplate is an ahead-of-time deep learning compiler that translates PyTorch neural networks into standalone C++ source code. It functions as a PyTorch to C++ compiler and a GPU kernel fusion engine, producing self-contained executable binaries that run inference without requiring a Python interpreter or deep learning framework runtime. The project generates optimized CUDA and HIP C++ code specifically for NVIDIA TensorCores and AMD MatrixCores. It focuses on maximizing throughput for half-precision floating-point operations through a system that combines multiple neural network operators in
Handles unsupported operators by delegating specific graph segments to an external engine while accelerating the rest.
OpenSfM is a computer vision library and structure-from-motion pipeline designed to reconstruct three-dimensional scenes and camera trajectories from overlapping images. It functions as a 3D reconstruction engine and photogrammetry toolkit, utilizing automated feature-based image matching and incremental bundle adjustment to derive spatial geometry. The system distinguishes itself as a geospatial alignment tool, integrating GPS and inertial sensor data to align reconstructed 3D models with real-world geographic coordinates. It employs a hybrid Python and C++ execution model to manage large-sc
Organizes large image collections into manageable sub-graphs to reduce the computational complexity of reconstruction.
POT is an optimal transport library providing a collection of solvers for computing Wasserstein, Gromov-Wasserstein, and Fused Gromov-Wasserstein distances between probability distributions. It functions as a differentiable tensor framework that integrates with various tensor libraries to enable automatic differentiation and GPU acceleration. The project is distinguished by its ability to align data distributions across different metric spaces by comparing internal relational structures rather than coordinates. It implements mathematical optimization algorithms as differentiable layers, allow
Divides graphs into clusters and selects representative nodes to optimize the performance of quantized transport solvers.