huggingface/huggingface_hub
Huggingface Hub
The Hugging Face Hub Python client is a library that provides programmatic access to the Hugging Face Hub, a centralized platform for hosting and collaborating on machine learning models, datasets, and demo applications. It serves as the primary SDK for interacting with the Hub's API, enabling users to download and upload models and datasets, manage repositories, authenticate via tokens or OAuth, and run inference on hosted models through a unified interface.
The client distinguishes itself through a comprehensive set of capabilities that go beyond basic file transfer. It includes a CLI extension system for adding custom commands, chunk-based deduplication for accelerated downloads, server-side file operations that avoid local transfers, and a virtual filesystem mounting feature that exposes remote Hub files as a local directory. The library also supports OIDC token exchange for CI/CD workflows, webhook management for event-driven automation, and the ability to execute cloud compute jobs on remote GPUs or TPUs.
Beyond its core storage and authentication features, the client offers tools for managing the full lifecycle of Hub resources. This includes creating and configuring Spaces for interactive demos, deploying and scaling inference endpoints, managing S3-like object storage buckets, and participating in community discussions and pull requests. The library also provides search and discovery capabilities for browsing models, datasets, and Spaces by task, framework, or author, along with cache management utilities for inspecting, pruning, and verifying the integrity of locally stored files.
The client is installed via pip or conda and supports configuration for HTTP proxies and custom Hub endpoint overrides, making it adaptable to network-restricted or self-hosted environments.
Features
- Model and Dataset Hubs - Provides the primary SDK for uploading, downloading, and versioning models and datasets on the Hub.
- Hub Dataset Uploads - Uploads datasets to a remote hub with files and metadata for sharing and versioning.
- Hub File Uploads - Uploads local files to a remote hub repository supporting models, datasets, and demos.
- Hub Dataset Clients - Provides a Python client for downloading, uploading, and managing datasets on the Hub.
- Model Hub Integrations - Hugging Face Hub's API integration for sending input data to a model hosted on the Hub and retrieving its predictions.
- Inference API Clients - Ships a client for running inference on hosted models through the Hub's inference endpoints.
- Inference Endpoint Integrations - Creates, configures, and manages remote inference endpoints for scalable model serving.
- Repository Listings - Lists models, datasets, and spaces stored on the remote hub via an HTTP client.
- Repository Creations - Hugging Face Hub's data storage feature for creating an empty model, dataset, or Space repository on the remote hub with configurable visibility and cloud region.
- Model Registries - Provides a client for discovering, downloading, and uploading models from a centralized model registry.
- Model Downloaders - Hugging Face Hub's data storage feature for downloading a machine learning model from the remote hub to a local directory using its repository identifier.
- Library-Aware Model Downloads - Downloads models from the hub that were created with dozens of integrated ML libraries.
- Hub Dataset Downloads - Downloads datasets from a remote hub including files and metadata for local use.
- Repository-to-Repository Transfers - Enables server-side file transfers between repositories and buckets without local downloads.
- Repository Renamings - Hugging Face Hub's data storage feature for changing the name or owner of a repository, including moving it from a user to an organization.
- Hugging Face Cache Managers - Manages cached files from the Hub by listing, removing, pruning unreferenced items, and verifying checksums.
- CLI Workflow Integrations - Provides a command-line interface for managing repositories, files, and authentication on the Hub.
- Repository Lifecycle Operations - Hugging Face Hub's data storage feature for creating, deleting, moving, renaming, and updating settings for remote repositories, including branches and tags.
- Repository File Management Tools - Hugging Face Hub's data storage feature for listing, deleting, and moving files within a remote repository using the client library.
- Model Inference APIs - Ships a unified API client for sending inputs to hosted model endpoints and retrieving predictions.
- Python API Clients - Ships a native Python SDK that wraps the Hub API for repository management, authentication, and file operations.
- Server-Side File Copying - Copies files between repositories on the server without downloading to the client.
- Hub Python Clients - Provides the primary Python library for interacting with the Hugging Face Hub.
- Model Uploaders - Hugging Face Hub's data storage feature for uploading a machine learning model to the remote hub, including its files and metadata, for sharing and versioning.
- Library-Aware Model Uploads - Uploads models trained with dozens of integrated ML libraries to the hub for sharing.
- Spaces Deployments - Creates, configures, and manages hosted machine learning demos and applications on the Hub.
- Interactive Demo Deployments - Creates and configures hosted Spaces for running interactive ML applications or demos.
- Space Hardware Configurations - Sets hardware resources and secret environment variables for remote Space deployments.
- Space Secrets Management - Sets environment variables and secret keys for Spaces without exposing them in source code.
- Repository Metadata Management - Hugging Face Hub's data storage feature for creating, deleting, and updating repositories, including their branches, tags, and settings, through HTTP requests.
- Repository Management APIs - Communicates with a remote hub using standard HTTP requests to manage repositories, files, and metadata.
- Model Endpoint Deployments - Deploys models on optimized inference endpoints or GPU-backed Spaces with per-hour pricing.
- Endpoint Lifecycle Controls - Starts, stops, scales, and deletes inference endpoints to control cost and availability.
- Remote File Downloads - Hugging Face Hub's data storage feature for downloading a single file from a remote repository by its path, supporting version pinning and local caching.
- Cached - Downloads files from remote repositories with local caching to avoid redundant transfers.
- Concurrent Repository Downloads - Downloads all files in a repository concurrently with local caching for efficient reuse.
- Hub File Downloads - Downloads models, datasets, and files from a remote hub with caching and resumable transfers.
- Hub Repository Downloads - Downloads files or entire repositories from the terminal using a single command with URI-based references.
- Version-Aware File Downloads - Downloads remote files with version-aware caching and checksum verification.
- Remote File Operations - Hugging Face Hub's data storage feature for creating, updating, and deleting files in remote repositories programmatically through a Python API.
- Access Token Management - Creates, deletes, and refreshes user access tokens with selectable roles and names.
- Hub Logins - Logs in to a remote account via browser OAuth or a token, saving credentials for subsequent commands.
- OAuth Authentication - Logs users in through a third-party OAuth flow so they can access protected resources without sharing credentials.
- Repository Visibility Controls - Toggles repository visibility between public and private to control access.
- Token Authentication - Passes a remote API token to authenticate requests for accessing private or gated resources.
- Hugging Face Authenticators - Sets an authentication token as an environment variable to authorize API calls without hardcoding credentials.
- Scoped Token Authentications - Uses scoped tokens to authenticate applications and notebooks for read, write, or fine-grained access.
- Token-Based Authentication - Authenticates users via scoped tokens and OAuth flows for secure access to private resources.
- Cached File Version Trackers - Stores downloaded files locally with version tracking and checksum verification to avoid redundant transfers.
- Repository - Filters repository search results by task, framework, author, and other attributes.
- Document Summarization - Hugging Face Hub's API integration for condensing a longer text into a shorter summary while preserving key information.
- Browser-Based Inference - Hugging Face Hub's API integration for testing a model directly in the browser using a built-in inference widget, without writing any code.
- Language Translation Services - Hugging Face Hub's API integration for converting text from one language into another using a trained translation model.
- CLI Search Interfaces - Provides a CLI for searching and listing models, datasets, and spaces with filtering and sorting.
- Hub Resource Listers - Lists and retrieves details about models, datasets, Spaces, and papers on the Hub.
- Hub Search Queries - Queries the Hub for models, datasets, or spaces using text filters and returns matching results.
- Repository File Listings - Hugging Face Hub's data storage feature for listing all files in a remote repository directory, optionally filtered by pattern or revision.
- Task-Based Dataset Browsers - Discovers and explores over five hundred thousand datasets organized by task and popularity.
- Model API Providers - Hugging Face Hub's API integration for accessing over forty-five thousand models from leading AI providers through a single unified API.
- Semantic Space Searches - Discovers hosted demo applications using natural language semantic search queries.
- Sentiment Analysis Tools - Hugging Face Hub's API integration for analyzing the emotional tone or polarity of text to determine if it is positive, negative, or neutral.
- Task-Based Model Browsers - Discovers and explores over two million models organized by task and popularity.
- Text Classification - Hugging Face Hub's API integration for assigning a label from a predefined set to a given text or image input based on its content.
- Text Completion Engines - Hugging Face Hub's API integration for producing a continuation of a provided text prompt using a language model to predict the next words.
- Question Answering - Hugging Face Hub's API integration for extracting a span of text from a given context that directly answers a natural language question.
- Pull Request Retrieval - Provides API capabilities for retrieving discussions and pull requests with pagination and filtering.
- Chunk-Based Deduplicators - Accelerates downloads by fetching only needed file ranges using chunk-based deduplication.
- Collaboration Discussion Tools - Provides API capabilities for creating, reading, and commenting on discussions and pull requests.
- Card Publishing - Pushes model cards to remote repositories with optional pull request creation.
- Evaluation Results - Embeds evaluation metrics and dataset details into model card metadata.
- Template-Based Generation - Generates model cards from Jinja2 templates with customizable sections.
- Bucket Creation - Creates S3-like object storage buckets on the Hub for large files.
- Bucket Deletion - Irreversibly removes buckets and all their contents from the Hub.
- Hub Bucket Managers - Manages S3-like buckets on the Hub for storing large mutable files without version control.
- Model Card Retrievers - Retrieves model documentation and YAML metadata from model cards by repository ID.
- Metadata Updates - Modifies metadata key-value pairs on repository cards with pull request support.
- Search Result Sorting - Sorts repository search results by criteria such as downloads and returns only the top results.
- Branch and Tag Management - Manages Git branches and tags on any repository type for organizing development streams.
- Community Extension Installers - Adds new top-level CLI commands by installing community extensions from public GitHub repositories.
- Hub Repository Searches - Queries the Hub for repositories and metadata to find models, datasets, or Spaces.
- Account Repository Listings - Retrieves all repositories for an account or organization sorted by storage usage.
- Remote Repository Deletions - Hugging Face Hub's data storage feature for permanently removing a repository from the remote hub after confirming the irreversible action.
- Pull Request Creation Interfaces - Provides API capabilities for opening new discussion threads and pull requests on repositories.
- Pull Request Merging Tools - Provides API capabilities for merging, closing, or reopening pull requests on repositories.
- Resource Group Access Controls - Restricts access to models and datasets using resource groups, audit logs, and single sign-on.
- Pattern-Based File Deletion - Deletes files from buckets by pattern with recursive and dry-run options.
- GPU/TPU Job Executions - Executes code on remote GPUs or TPUs using a Docker-like command-line interface.
- Community Collaboration - Creates, reviews, and manages discussions and pull requests on repositories for collaboration.
- Bucket File Downloads - Downloads specific files from Hub buckets to local paths with metadata pre-fetching.
- Remote Repository Mounts - Mounts remote Hub files as a local filesystem for seamless browsing and editing via Python.
- Organization Token Policy Enforcements - Applies organization-level policies requiring administrator approval and fine-grained token restrictions.
- Organization Token Issuances - Generates tokens for organization members via OAuth Token Exchange for internal platforms.
- Event Webhook Management - Hugging Face Hub's API integration for listing, creating, and deleting webhooks that trigger actions on Hub events like pushes or discussions.
- Hub Virtual Filesystem Browsers - Hugging Face Hub's data storage feature for opening a virtual filesystem over the remote hub so users can list, read, write, and delete files using standard Python file operations.
- Metadata Card Creation - Creates structured metadata cards for repositories with model and dataset details.
- Metadata Card Reading - Loads and parses repository metadata cards for programmatic inspection.
- Metadata Card Updating - Updates repository metadata cards by merging new content with existing data.
- Collaborative Discussion Threads - Provides API capabilities for opening new discussion threads with titles and descriptions.
- Collaborative Commenting Systems - Provides API capabilities for adding comments to discussion and pull request threads.
- Model and Dataset Collections - Ships a feature for creating named collections of models, datasets, and Spaces.
