Awesome GitHub RepositoriesCompute Graph Slicing

PowerInfer is a high-performance local large language model inference engine and sparse inference framework. It provides a runtime for executing models on consumer-grade hardware, utilizing a GPU acceleration backend to optimize tensor operations for graphics processors. The system distinguishes itself through a sparse inference framework that increases generation speed by skipping computations based on activation sparsity in model weights. It includes a GGUF model converter for transforming weights and metadata into a unified binary format, as well as an OpenAI API compatible server for inte

TensorFlowOnSpark is a distributed framework for running TensorFlow machine learning workloads and model training across Apache Spark clusters. It functions as a cluster computing orchestrator that manages worker processes and resource allocation to scale deep learning tasks across multiple computing nodes. The platform enables distributed deep learning training and large-scale model inference, allowing users to execute tasks across a cluster of servers to handle datasets that exceed the memory of a single machine. It integrates deep learning workloads with Spark data processing to create end