Awesome GitHub RepositoriesIn-Memory Model Caching

AutoGluon is an automated machine learning framework and multimodal library designed to automate the end-to-end pipeline from data preprocessing to high-accuracy model training and validation. It functions as an automated model trainer for tabular, image, text, and time series data, as well as a tool for time series forecasting and foundation model finetuning. The project is distinguished by its ability to jointly process and fuse different data types, allowing for the construction of multimodal neural networks that integrate images, text, and structured tables. It supports zero-shot inferenc

sd-scripts is a suite of utilities designed for fine-tuning generative models, preprocessing datasets, and converting model weights. It provides a collection of scripts for executing Stable Diffusion training through methods such as DreamBooth, textual inversion, and full fine-tuning, alongside a framework for creating and managing Low-Rank Adaptation weights. The project features specialized capabilities for model weight conversion between different architectures and precision formats. It includes tools for merging adaptation weights into base models, extracting weights from trained models,

KServe is an open platform for deploying and serving generative and predictive AI models on Kubernetes. It defines inference services as custom resources with declarative YAML specifications, enabling a Kubernetes-native approach to model deployment and lifecycle management. The platform leverages Knative-based serverless scaling for automatic scale-to-zero and revision management, and supports a pluggable serving runtime architecture that maps model formats to containerized execution environments. KServe distinguishes itself through model-aware autoscaling that scales replicas based on token

KServe is a Kubernetes-native platform for deploying and serving machine learning models as scalable inference services. It supports both generative AI models, including large language models, and traditional predictive models from frameworks such as TensorFlow, PyTorch, Scikit-Learn, XGBoost, and ONNX. The platform manages the full lifecycle of model deployments, including revision tracking, canary rollouts, A/B testing, and automatic rollbacks, and provides serverless scale-to-zero capabilities for cost-efficient resource management. KServe distinguishes itself through a standardized infere

This project provides a self-hosted server for automatic speech recognition, functioning as a containerized inference engine for the Whisper model. It exposes core transcription and translation capabilities through a standardized web interface, allowing for the integration of speech-to-text services into external applications. The service distinguishes itself by incorporating advanced audio analysis tools, including speaker diarization to attribute text to specific individuals and voice activity detection to filter non-speech segments. It supports automated language detection and provides out

Pocket-tts is a text-to-speech server and neural speech synthesizer that converts written text into audible speech. It includes a CPU-optimized inference engine and a voice cloning tool capable of analyzing audio samples to reproduce specific speaker characteristics. The system differentiates itself through the use of dynamic int8 quantization to reduce memory usage and increase generation speed on processors. It supports real-time speech synthesis by streaming audio chunks incrementally and utilizes voice state caching to store processed embeddings as portable files, bypassing redundant proc

ComfyUI-SeedVR2_VideoUpscaler is an AI video upscaling tool that uses diffusion models to increase the resolution of videos and images while maintaining visual consistency across frames. The project implements distributed video rendering by splitting datasets into chunks for parallel processing across multiple GPUs. It utilizes model compilation and specialized attention backends to reduce inference latency and increase throughput. Additional capabilities include video color correction using wavelet and LAB matching methods to preserve color fidelity. Hardware memory is managed through block

BERT Score is a text evaluation tool that assesses the quality of generated text by computing precision, recall, and F1 metrics between candidate and reference texts. It transforms text tokens into dense contextual vectors using pretrained transformer models, calculating token-level similarity matrices through pairwise cosine distances. The system computes scores by greedily matching tokens between sequences and supports multilingual assessment across dozens of different languages by utilizing language-specific or cross-lingual transformer backends. The library includes features for baseline