OpenCSGs/csghub
Csghub
CSGHub is a model management platform and dataset registry designed for storing, distributing, and managing large language models. It provides a centralized hub for AI assets accessible via a web interface, software development kit, and command line.
The platform functions as a self-hosted AI infrastructure, allowing for on-premise installation within private networks for secure and offline operations. It includes a model storage system that maintains Python SDK compatibility with the Hugging Face ecosystem to facilitate the migration of existing scripts.
The system covers the full model lifecycle, including versioning, distribution, and the preparation of training data through intelligent annotation and multi-source synchronization. Organizational security is managed through enterprise access control and role-based permissions to restrict access to sensitive machine learning assets.
Features
- Model Asset Hubs - Acts as a centralized repository specifically designed for versioning and distributing ML model weights and datasets.
- LLM Asset Management - Provides a centralized hub for storing and distributing large language models and datasets via web and SDK.
- Dataset Registries - Provides a repository for organizing and versioning training data with synchronization and annotation tools.
- Model Lifecycle Management - Manages the end-to-end process of versioning and distributing models from development to deployment.
- Model and Dataset Hubs - Functions as a centralized platform for storing, versioning, and sharing machine learning models and datasets.
- Private AI Infrastructure - Provides a platform for hosting and managing machine learning models and workflows on private servers.
- On-Premise Deployment - Supports full installation of the platform on private servers for secure and offline operations.
- Self-Hosted AI Infrastructure - Offers a private deployment solution for managing AI services and model assets on private hardware.
- Machine Learning Datasets - Facilitates the creation of structured collections of labeled data for training machine learning models.
- Registry Compatibility Layers - Maintains Python SDK compatibility with the Hugging Face ecosystem to facilitate script migration.
- Training Data Annotations - Provides systems for creating annotated datasets through intelligent labeling of raw information.
- Multi-Source Data Integration - Coordinates and organizes information from multiple external data sources to streamline asset collection.
- Distributed Storage - Provides high-availability storage for large binary model files and datasets across scalable backends.
- ML SDK - Provides an API layer that mimics popular library standards to allow seamless migration of ML scripts.
- Annotation Pipelines - Implements structured processing steps to transform raw information into labeled training sets.
- Enterprise Security Controls - Implements mechanisms for enforcing security and restricting access to sensitive assets in restricted environments.
- Organization-Based Access Controls - Separates teams into organizations with independent role-based permissions to protect sensitive ML assets.
- Role-Based Access Control - Manages data security by assigning specific permissions to users and groups across organizational levels.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
