Awesome GitHub RepositoriesZero-Shot Classification Models

CLIP is a neural network architecture designed to map visual and textual data into a shared latent vector space. By utilizing transformer-based feature extraction and multi-modal tokenization, the system aligns images and natural language strings, enabling cross-modal similarity analysis and semantic classification. The project functions as a zero-shot classification engine, identifying image content by calculating the cosine similarity between visual features and arbitrary text labels without requiring task-specific retraining. Beyond inference, it serves as a research toolkit for evaluating

Open CLIP is an open source framework for training and deploying Contrastive Language-Image Pre-training models. It serves as a vision-language training framework and multimodal embedding engine that maps images and text into a shared vector space for similarity searches and zero-shot classification. The project provides a toolkit for distributed training of contrastive models and includes an image-to-text generative model for producing natural language descriptions. It supports custom text encoder integration and utilizes teacher-student model distillation to transfer knowledge from large pr

This project is a research library and toolkit for deep learning computer vision, focused on implementing transformer and mixer-based architectures for image classification. It processes visual data by converting images into sequences of patches, allowing standard attention mechanisms to capture global dependencies without relying on traditional convolutional operations. The framework distinguishes itself through its support for multimodal embedding analysis, which maps images and text into a shared latent vector space. This capability enables zero-shot classification and cross-modal retrieva

This PyTorch-based deep learning library provides a framework for analyzing and forecasting temporal data. It implements specialized architectures for time series forecasting, anomaly detection, data imputation, and classification. The project distinguishes itself through the inclusion of zero-shot inference capabilities, allowing large-scale temporal models to be evaluated on unseen datasets without requiring task-specific fine-tuning. The framework covers a broad range of analytical capabilities, including the recovery of missing values in incomplete datasets, the identification of irregul

This project is a multimodal translation framework and large language model capable of speech-to-speech, speech-to-text, and text-to-text translation across nearly 100 languages. It provides a real-time speech translation engine and a comprehensive toolkit for converting spoken audio between languages. The system is distinguished by its ability to preserve the original speaker's tone, pace, and prosody during translation. It utilizes a specialized on-device inference toolkit that converts model checkpoints into C-based libraries, enabling low-latency execution on mobile and edge hardware with

InternVL is a vision-language model framework that fuses a visual encoder with a large language model to translate image features into textual tokens for reasoning. It provides a system for multimodal inference and dialogue, enabling the processing of images and text to answer questions or generate descriptions. The project is distinguished by its high-resolution image processing, which uses dynamic tiling to maintain detail for images up to 4K resolution, and its chain-of-thought visual reasoning for solving complex mathematical and spatial problems. It also supports temporal frame sampling

AutoGluon is an automated machine learning framework and multimodal library designed to automate the end-to-end pipeline from data preprocessing to high-accuracy model training and validation. It functions as an automated model trainer for tabular, image, text, and time series data, as well as a tool for time series forecasting and foundation model finetuning. The project is distinguished by its ability to jointly process and fuse different data types, allowing for the construction of multimodal neural networks that integrate images, text, and structured tables. It supports zero-shot inferenc

ImageBind is a multi-modal embedding model and joint representation learner that maps images, text, audio, and other modalities into a single shared vector space. It functions as a cross-modal retrieval framework designed to bind multiple sensory inputs into one cohesive mathematical embedding. The system uses a contrastive learning architecture to align disparate data types by maximizing the similarity between related samples. This allows the model to perform zero-shot multimodal classification and execute cross-modal data retrieval, such as locating visual content via natural language descr

Skorch 是一个将 PyTorch 神经网络包装在 scikit-learn 兼容接口中的库，允许在标准机器学习流水线和超参数优化工具中使用深度学习模型。它充当数据适配器、训练管理器和优化工具，弥合了深度学习模块与传统机器学习工作流之间的差距。 该项目通过提供用于自动化 PyTorch 训练生命周期的工具包而脱颖而出，包括集成的检查点保存、提前停止和学习率调度。它还通过用于冻结特定模型层和针对特定任务微调预训练权重的实用程序，实现了迁移学习。 能力面扩展到数据转换，包括将表格数据和数值数组转换为张量格式以及注册文本分词器。它还提供了用于硬件加速选择、即时模块编译以及用于不确定性量化的概率数据建模的工具。 该系统包括用于将超参数映射到命令行参数的实用程序，以确保实验的可重复性。

Muzic 是一个用于 AI 驱动的音乐分析、创作和合成的深度学习平台和框架。它作为一个音乐生成框架和分析工具，利用大型语言模型和自主智能体来编排符号音乐和音频音乐的创作与解读。 该项目以其跨模态能力而著称，将自然语言和符号音乐映射到共享的联合嵌入空间中，用于零样本分类和信息检索。它采用了多种专门的架构，包括用于音频合成的扩散框架、用于长序列结构一致性的双粒度注意力机制，以及结合音乐理论规则与神经网络的混合系统。 该平台涵盖了广泛的功能，包括从文本和歌词生成 MIDI 序列、神经歌声合成以及自动歌词转录。它还提供用于音乐结构建模、基于属性的符号生成以及通过自主智能体编排外部音乐工具的工具。 支持性实用程序包括用于大规模 MIDI 二进制化、数据集编码的数据工程流水线，以及用于旋律音符提取和语音到音素对齐的音频信号处理。

Spark NLP 是一个构建在 Apache Spark 分布式计算框架之上的可扩展文本分析和机器学习工具包。它提供了一个多模态机器学习框架和一个用于对标注器进行排序以处理大规模语言数据的分布式流水线系统。该库包含一个用于生成上下文向量嵌入的 Transformer 文本处理器，以及一个用于管理大型语言模型的专用推理引擎。 该项目通过其在统一视觉-语言架构内处理异构数据类型（包括文本、音频和图像）的能力而脱颖而出。它支持高级生成式 AI 功能，如提示工程、具有约束 JSON 输出的结构化实体提取，以及消除网络延迟的本地推理。此外，它还提供跨文本和图像模态的跨语言翻译和零样本分类工具。 该框架涵盖了广泛的功能，包括用于实体识别和情感分析的监督模型训练，以及抽取式问答和文档摘要。它集成了向量数据库支持以进行相似性搜索，并为 GPU 加速和通过集中式注册表进行模型生命周期管理提供了基础设施。 该工具包允许通过公共仓库分发自定义模型和流水线，并支持通过 REST API 部署模型。

Nixtla 是一个以基于 Transformer 的基础模型为中心的时序分析平台。它为预测和异常检测提供零样本推理，允许系统在无需模型重新训练的情况下预测新时序的未来值。 该项目专为大规模分析而设计，使用分布式推理扩展和预测并行化来处理数百万个数据序列。它支持微调适配以针对特定领域数据集调整预训练权重，并提供从本地执行和私有容器到作为 Snowflake 内存储过程集成等多种部署选项。 能力包括长周期和间歇性需求预测、假设场景分析以及预测不确定性量化。该系统还提供了一个完整的数据工程流水线，用于审计、清理和使用外生变量及基于日期的指标来丰富时序数据。 模型可靠性通过交叉验证回测、预测准确性验证以及用于超参数记录的实验跟踪来管理。

This project is a research framework and toolkit designed for training large-scale vision transformers and multimodal language models. It provides a comprehensive suite for vision-language pretraining, enabling the development of models that map images and text into shared latent spaces. The framework is distinguished by its capabilities in high-fidelity image generation and multimodal research, utilizing normalizing flows and variational autoencoders to produce images from text prompts or class labels. It supports the development of both generative and contrastive models, allowing for a wide

LLM Guard is a security firewall and guardrail framework designed to scan and sanitize inputs and outputs for large language models. It functions as a proxy gateway and security layer to block prompt injections, toxicity, and sensitive data leakage while ensuring that model interactions remain compliant with organizational policies. The system distinguishes itself through a modular scanner pipeline that utilizes local model orchestration to eliminate external network dependencies. It supports real-time security filtering via streaming chunk analysis and implements a fail-fast execution model