Project-HAMi/HAMi

Features

• GPU Resource Virtualization - Partitions physical accelerators into smaller virtual slices to enable efficient sharing across multiple containers.
• Dynamic Hardware Partitioning - Partitions physical GPU hardware into isolated virtual slices to allow multiple containers to share the same device.
• Hardware Acceleration Abstractions - Provides a unified abstraction for scheduling workloads across diverse accelerator pools from different vendors.
• Topology-Aware Schedulers - Assigns tasks to hardware nodes using topology-aware policies to maximize throughput and proximity.
• GPU Resource Orchestrators - Dynamically manages, partitions, and allocates GPU hardware across shared AI workloads.
• Device Plugin Integration - Acts as a Kubernetes extension to manage the allocation and partitioning of hardware accelerators.
• Heterogeneous Device Schedulers - Manages workload placement across diverse hardware accelerators using topology and packing policies.
• Hardware Device Sharing - Allows multiple concurrent containers to access a single physical device using resource offsets and shared access handles.
• Hardware Memory Limiters - Enforces strict hardware memory limits and allocates capacity via absolute values or percentages to ensure isolation.
• Multi-Tenant Hardware Sharing - Allows multiple containers simultaneous access to the same hardware devices to increase overall resource utilization.
• Device Plugin Virtualization - Implements a Kubernetes device plugin to partition physical accelerators into isolated virtual slices for containers.
• Multi-Tenant Isolation - Enforces strict memory limits and compute quotas to prevent interference between workloads on shared hardware.
• Hardware-Level Isolation - Enforces strict hardware memory limits at the driver level to provide secure hardware-backed isolation.
• Compute Core Isolation - Separates memory and compute cores on hardware to allow multiple workloads to share a single physical device securely.
• Resource Quotas - Applies hard memory and compute limits at runtime to prevent interfering workloads in shared environments.
• Workload Placement Policies - Uses packing and topology policies to optimize where containerized workloads are placed on hardware nodes.
• Resource Management Interfaces - Provides a web-based graphical interface to administer virtualization environments and hardware allocations.
• Mixed-Accelerator Orchestration - Enables running tasks across different types of hardware accelerators within a single unified workflow.
• Virtualization Management - Provides a graphical interface and backend for partitioning physical hardware into isolated virtual slices.
• Hardware Performance Monitoring - Collects real-time memory and compute utilization data from various hardware vendors for system dashboards.
• Hardware Resource Tracking - Provides a dedicated endpoint for monitoring the consumption of hardware accelerator resources.
• Accelerator Device Targeting - Provides the ability to target specific accelerators or exact hardware devices using unique identifiers to meet strict requirements.
• Hardware Telemetry Collection - Collects real-time memory and core utilization data across hardware vendors and exports it via open standards.
• Accelerator Monitoring - Tracks performance metrics and resource allocation for specialized hardware accelerators across vendors.
• Prometheus System Metrics Exporters - Exposes real-time hardware utilization data via a Prometheus-compatible REST endpoint for monitoring.

Star history