Projects sharing features with Isaac GR00T

ManiSkill is a GPU-accelerated robot simulation framework designed for training robotic manipulation skills, benchmarking learning algorithms, and generating synthetic datasets. It serves as a reinforcement learning environment where robot control policies can be developed and evaluated using parallelized physics and rendering on the GPU. The platform is distinguished by its ability to perform sim-to-real transfer, allowing policies trained in virtual environments to be deployed onto physical robotic hardware. It features ray-traced parallel rendering for producing high-frame-rate RGBD and se

DeepEP is a distributed model accelerator and expert-parallel communication library designed to optimize the training and inference of large-scale neural networks. It provides specialized GPU communication kernels and a remote GPU memory interface to facilitate high-throughput data exchange between hardware nodes. The system utilizes dynamic kernel generation to compile optimized GPU kernels during execution, removing the need for separate installation compilation steps. It implements virtual-lane traffic isolation to prevent interference between different data streams and employs routing met

WhisperLive is a real-time speech-to-text server that converts live audio streams into text using Whisper models. It functions as a backend service that receives microphone input via WebSockets and provides incremental transcriptions with word-level timestamps. The system utilizes a GPU-accelerated inference engine and a keyword-boosted transcription API to improve the recognition accuracy of domain-specific jargon, acronyms, and product names. It also includes a speaker diarization tool that clusters audio embeddings to identify and label different participants within a recording. Additiona

Vocode-core is a framework for building real-time conversational AI voice agents. It serves as a conversational orchestrator and pipeline that integrates speech-to-text, large language models, and text-to-speech services to enable low-latency voice interactions. The project features a provider-agnostic interface that allows for swappable speech and language model providers, including support for both cloud APIs and local binaries. It distinguishes itself through a specialized telephony integration layer that enables agents to be deployed across phone lines, WebRTC, and virtual meeting platfor

NVIDIA DALI is a GPU-accelerated data loading and preprocessing library designed for deep learning workflows. It constructs high-performance data pipelines that offload decoding, augmentation, and normalization to the GPU, eliminating CPU bottlenecks in training and inference. The library reads data from multiple storage formats and streams it directly into GPU memory, with support for multi-GPU execution to scale throughput across large-scale workloads. DALI distinguishes itself by enabling data pipelines to be built once and executed across multiple deep learning frameworks without code cha

tensorrtx is a computer vision inference engine and model implementation library designed for graphics processor acceleration. It provides a framework for optimizing deep learning models through a GPU inference optimizer, a deep learning model converter for transforming weights from frameworks like TensorFlow and PyTorch, and a custom plugin library to implement operations not natively supported by the TensorRT API. The project distinguishes itself through a comprehensive collection of pre-defined network implementations, ranging from various YOLO versions and DETR transformers for object det

Whisper is a high-performance speech-to-text inference engine that uses graphics hardware shaders to accelerate the transcription of spoken audio into written text. It implements a GPU-accelerated automatic speech recognition framework specifically designed to run Whisper models. The system focuses on high-speed processing for both recorded audio files and live microphone streams. It utilizes voice activity detection to analyze raw audio in real time, triggering the inference engine only when human speech is detected. The engine covers a broad range of capabilities including real-time audio

lite.ai.toolkit is a C++ computer vision toolkit designed for edge AI deployment. It enables the execution of pre-trained models for object detection, image classification, and segmentation on resource-constrained devices. The project features a multi-backend inference engine that supports the ONNX model runtime, allowing AI models to run across different hardware targets. It includes a GPU-accelerated pipeline specifically for NVIDIA hardware to reduce latency and increase processing speed. The toolkit covers a broad range of facial analysis capabilities, including emotion detection, gender

Swift is a toolkit for the full-parameter and parameter-efficient fine-tuning of large language and multimodal models. It functions as a multimodal model trainer for text, image, video, and audio data, and includes specialized tools for model compression and reinforcement learning from human feedback. The framework provides an alignment toolkit for optimizing model behavior using preference learning algorithms and reinforcement learning. It integrates parameter-efficient fine-tuning methods to adapt models with minimal memory and compute requirements, alongside utilities for reducing hardware

NCCL is a high-performance communication library and distributed GPU computing framework designed for executing collective and point-to-point data exchanges across multiple GPUs in single or multi-node systems. It serves as an RDMA GPU transport layer and memory orchestrator, facilitating high-bandwidth synchronization of data and model gradients for distributed GPU training and inference. The library is distinguished by its ability to execute communication primitives directly from GPU kernels, removing the host CPU from the critical path. It utilizes topology-aware path selection to optimize

Magic is an all-in-one productivity environment and agent platform designed for deploying, orchestrating, and managing multi-agent workflows. It functions as a coordination system that dispatches complex tasks to specialized agents, serving as both a workflow engine and a knowledge management system that synthesizes information from PDFs, websites, and databases into structured digital assets. The platform distinguishes itself through a multimodal content suite capable of generating professional business deliverables, including high-fidelity graphic assets, technical diagrams, and presentatio

This project is a comprehensive computer vision library for the PyTorch ecosystem, providing a standardized collection of neural network architectures, datasets, and high-performance transformation utilities. It serves as a foundational framework for building, training, and deploying deep learning models, offering a centralized model registry that allows developers to instantiate architectures with pre-trained weights for tasks such as image classification, object detection, and semantic segmentation. The library distinguishes itself through its modular approach to data and compute management

AutoGluon is an automated machine learning framework and multimodal library designed to automate the end-to-end pipeline from data preprocessing to high-accuracy model training and validation. It functions as an automated model trainer for tabular, image, text, and time series data, as well as a tool for time series forecasting and foundation model finetuning. The project is distinguished by its ability to jointly process and fuse different data types, allowing for the construction of multimodal neural networks that integrate images, text, and structured tables. It supports zero-shot inferenc

This project is a cross-platform machine learning inference engine designed to execute pre-trained models across diverse operating systems and hardware environments. It functions as a standardized execution framework that manages the entire lifecycle of model inference, from loading and graph optimization to hardware-accelerated execution and generative sequence management. The runtime distinguishes itself through a highly modular architecture that decouples model logic from hardware-specific kernels. By utilizing an execution provider abstraction, it enables developers to offload computation

FlashMLA is an LLM attention kernel library and inference acceleration library providing a collection of high-performance CUDA kernels. It implements multi-head latent attention mechanisms designed to reduce memory overhead and increase throughput during the forward and backward passes of large language model inference. The library utilizes quantized cache attention kernels to improve computation efficiency across both sparse and dense token processing. It specifically optimizes the prefill and decoding phases of model inference through these latent attention implementations. The project cov

Drake is a robotics simulation framework and control system modeling tool used for designing, simulating, and verifying the dynamics of complex robotic systems. It functions as a multibody dynamics simulator and a mathematical optimization library, providing a suite of algorithms for trajectory optimization and the simulation of articulated robots. The framework is distinguished by its block-diagram system for composing dynamical subsystems and its ability to formulate and solve diverse mathematical programs, including linear, quadratic, and nonconvex nonlinear problems. It supports specializ

Ecoute is a live transcription tool that provides real-time transcripts for both the user's microphone input (You) and the user's speakers output (Speaker) in a textbox.

Video-Pre-Training is a machine learning framework designed for training autonomous agents to perform complex tasks by observing and mimicking human behavior from video recordings. It provides a comprehensive toolkit for imitation learning and reinforcement learning research, enabling the development of agents that can replicate human actions within simulated digital environments. The framework distinguishes itself through its ability to process large-scale, unlabeled video datasets to bootstrap agent capabilities. It utilizes inverse dynamics modeling to infer control inputs from frame trans

Webots is a physics-based robot simulator and development environment used for modeling, programming, and testing the behavior of robots in a simulated 3D physical world. It serves as a virtual prototyping tool to verify mechanical and electronic systems through the creation of virtual robot models and control logic. The platform enables a full robotics simulation workflow, including the development of robot controllers and the programming of autonomous agent behaviors. It focuses on physical system modeling to represent the mechanical properties of hardware and simulate real-world interactio

cuda-python provides low-level Python bindings for the CUDA Driver and Runtime APIs. It serves as a programmatic wrapper for controlling device memory, managing hardware toolchains, and orchestrating execution graphs on NVIDIA GPUs, allowing for the compilation and launching of parallel kernels directly from Python. The project enables the development of SIMT kernels and the execution of mathematical algorithms on device memory. It integrates pre-compiled bytecode as custom operators and interfaces with accelerated device libraries to access low-level hardware functions without leaving the la

Lorax is a GPU-accelerated inference server and multi-adapter engine designed for serving large language models. It functions as a high-throughput system capable of deploying models via Kubernetes and managing the dynamic swapping of Low-Rank Adaptation adapters per request. The server distinguishes itself through multi-adapter dynamic batching, which allows requests using different adapter weights to be processed in a single GPU forward pass. It employs just-in-time adapter loading and weighted adapter merging to maximize throughput and enable multi-tasking without sacrificing performance.

Thrust is a heterogeneous computing library and C++ template library that provides a collection of high-level templates for executing data-parallel operations. It functions as a parallel algorithms library designed to work across different hardware backends, including multicore CPUs and NVIDIA GPU hardware. The framework utilizes a header-only implementation and a generic-programming policy interface to abstract the differences between CPU and GPU memory and execution models. It employs an iterator-based data abstraction to provide a uniform interface for accessing elements across host RAM an

IsaacGymEnvs is a GPU-accelerated physics sandbox and robotics policy training suite designed for reinforcement learning. It serves as a vectorized robotic simulator that runs thousands of parallel environments on GPUs to accelerate the training of neural networks. The project provides a sim-to-real transfer framework that utilizes domain randomization and physics variations to ensure policies trained in simulation are robust enough for deployment on real hardware. It distinguishes itself through a high-performance architecture that uses tensor-based state management to handle observations an

Falco is an eBPF runtime security monitor and cloud native detection engine that identifies abnormal behavior and security threats across hosts and containers. It functions as a Linux kernel event auditor, capturing system calls and kernel events in real-time to detect malicious activity. The system distinguishes itself through a rule-based threat detection model that evaluates system activity against a library of community-maintained rules and custom security definitions. It enriches raw kernel events with container and Kubernetes metadata to provide observability into isolated environments

Neural Compressor is a deep learning model compression toolkit and AI inference acceleration engine. It functions as an automated model quantization tool and hardware-aware model compiler designed to reduce the memory footprint of neural networks and decrease execution latency. The project provides specialized frameworks for optimizing large language models, utilizing weight-only quantization and hardware-specific kernels to improve the operational efficiency of generative AI workloads. It maps neural network operators to specialized CPU and GPU vector instructions to accelerate model executi

BentoML is a machine learning model serving framework and GPU-accelerated inference server designed to package, deploy, and scale AI models as production-ready REST APIs. It functions as an AI model lifecycle manager and an inference graph orchestrator, enabling the chaining of multiple models and custom logic into complex pipelines for advanced task sequences. The framework distinguishes itself through a dynamic batching engine that optimizes GPU throughput and an artifact-based packaging system that bundles model weights and dependencies into immutable archives for consistent deployment. It

This project is a comprehensive framework for constructing, managing, and evaluating knowledge graphs through multi-agent reasoning and deep search capabilities. It provides an end-to-end pipeline that ingests multi-format documents, extracts entities and relationships based on configurable schemas, and maintains structured knowledge bases to support evidence-based retrieval. The system distinguishes itself through its multi-agent orchestration, which decomposes complex queries into parallel research steps and synthesizes long-form reports. It leverages advanced graph-based techniques, includ