HAMi

Features

GPU Resource Virtualization - Partitions physical accelerators into smaller virtual slices to enable efficient sharing across multiple containers.

Dynamic Hardware Partitioning - Partitions physical GPU hardware into isolated virtual slices to allow multiple containers to share the same device.

Hardware Acceleration Abstractions - Provides a unified abstraction for scheduling workloads across diverse accelerator pools from different vendors.

Topology-Aware Schedulers - Assigns tasks to hardware nodes using topology-aware policies to maximize throughput and proximity.

GPU Resource Orchestrators - Dynamically manages, partitions, and allocates GPU hardware across shared AI workloads.

Device Plugin Integration - Acts as a Kubernetes extension to manage the allocation and partitioning of hardware accelerators.

Heterogeneous Device Schedulers - Manages workload placement across diverse hardware accelerators using topology and packing policies.

Hardware Device Sharing - Allows multiple concurrent containers to access a single physical device using resource offsets and shared access handles.

Hardware Memory Limiters - Enforces strict hardware memory limits and allocates capacity via absolute values or percentages to ensure isolation.

Multi-Tenant Hardware Sharing - Allows multiple containers simultaneous access to the same hardware devices to increase overall resource utilization.

Device Plugin Virtualization - Implements a Kubernetes device plugin to partition physical accelerators into isolated virtual slices for containers.

Multi-Tenant Isolation - Enforces strict memory limits and compute quotas to prevent interference between workloads on shared hardware.

Hardware-Level Isolation - Enforces strict hardware memory limits at the driver level to provide secure hardware-backed isolation.

Compute Core Isolation - Separates memory and compute cores on hardware to allow multiple workloads to share a single physical device securely.

Resource Quotas - Applies hard memory and compute limits at runtime to prevent interfering workloads in shared environments.

Workload Placement Policies - Uses packing and topology policies to optimize where containerized workloads are placed on hardware nodes.

Resource Management Interfaces - Provides a web-based graphical interface to administer virtualization environments and hardware allocations.

Mixed-Accelerator Orchestration - Enables running tasks across different types of hardware accelerators within a single unified workflow.

Virtualization Management - Provides a graphical interface and backend for partitioning physical hardware into isolated virtual slices.

Hardware Performance Monitoring - Collects real-time memory and compute utilization data from various hardware vendors for system dashboards.

Hardware Resource Tracking - Provides a dedicated endpoint for monitoring the consumption of hardware accelerator resources.

Accelerator Device Targeting - Provides the ability to target specific accelerators or exact hardware devices using unique identifiers to meet strict requirements.

Hardware Telemetry Collection - Collects real-time memory and core utilization data across hardware vendors and exports it via open standards.

Accelerator Monitoring - Tracks performance metrics and resource allocation for specialized hardware accelerators across vendors.

Prometheus System Metrics Exporters - Exposes real-time hardware utilization data via a Prometheus-compatible REST endpoint for monitoring.

HAMi is a hardware orchestration and virtualization system designed to manage accelerators within Kubernetes. It functions as a device plugin that partitions physical hardware into isolated virtual slices, enabling multiple containers to share a single device through enforced memory limits and compute quotas.

The project provides a virtualization manager and a heterogeneous compute scheduler that distributes tasks across diverse accelerator types. It uses packing and topology policies to optimize workload placement and allows for specific hardware targeting using unique device identifiers.

The system includes a monitoring tool that collects real-time memory and core utilization metrics across different hardware vendors, exporting this data via Prometheus-compatible endpoints for cluster health dashboards. Administrative tasks and hardware allocations are managed through a web-based graphical interface.

Features