Awesome GitHub RepositoriesContrastive Learning Models

CLIP is a neural network architecture designed to map visual and textual data into a shared latent vector space. By utilizing transformer-based feature extraction and multi-modal tokenization, the system aligns images and natural language strings, enabling cross-modal similarity analysis and semantic classification. The project functions as a zero-shot classification engine, identifying image content by calculating the cosine similarity between visual features and arbitrary text labels without requiring task-specific retraining. Beyond inference, it serves as a research toolkit for evaluating

This project is a framework for training and deploying transformer-based models that map text, images, audio, and video into dense or sparse vector representations. It functions as a multimodal embedding library and semantic search engine used to retrieve relevant documents by calculating vector similarity between meanings. The framework provides specialized tools for both cross-encoder reranking, which calculates precise similarity scores to refine search results, and vector quantization to compress embedding vectors for reduced memory usage and increased retrieval speed. The project covers

This project is a transformer-based framework for generating dense and sparse vector embeddings of text and multimodal data. It serves as a library for fine-tuning models to perform semantic similarity tasks, retrieval, and reranking. The system is distinguished by its support for diverse architectural patterns, including bi-encoders for fast similarity search and cross-encoders for high-precision reranking. It provides dedicated pipelines for multimodal embeddings, mapping text and images into a shared vector space, and implements knowledge distillation to compress large models into smaller,

This is a PyTorch implementation of a text-to-image model designed for synthesizing high-fidelity images from natural language descriptions. It utilizes a diffusion image generator to transform latent embeddings into visual data through an iterative denoising process. The system employs a two-stage latent mapping process, using a CLIP-based latent prior to map text embeddings to image embeddings before decoding them into pixels. It features a cascading diffusion decoder that produces high-resolution imagery by passing low-resolution outputs through a sequence of models at increasing scales.

GroundingDINO is a deep learning vision model and open-vocabulary object detector designed to map natural language prompts to spatial coordinates. It functions as a text-to-bounding-box framework that enables zero-shot image localization, allowing the system to identify and locate arbitrary objects without requiring predefined classes or specific training for those categories. The project distinguishes itself by matching visual features to natural language descriptions to achieve open-set visual recognition. It supports text-guided image localization and the isolation of specific objects base

ImageBind is a multi-modal embedding model and joint representation learner that maps images, text, audio, and other modalities into a single shared vector space. It functions as a cross-modal retrieval framework designed to bind multiple sensory inputs into one cohesive mathematical embedding. The system uses a contrastive learning architecture to align disparate data types by maximizing the similarity between related samples. This allows the model to perform zero-shot multimodal classification and execute cross-modal data retrieval, such as locating visual content via natural language descr

This project is a comprehensive machine learning interview guide and technical study resource designed for individuals preparing for machine learning and AI engineering roles. It provides a collection of materials and practice problems covering core algorithms, theoretical fundamentals, and the implementation of neural network architectures. The resource serves as a technical reference for generative AI development, focusing on the design and optimization of large language models and diffusion systems. It includes frameworks for system design, covering the architecture of production machine l

Chinese-CLIP es un framework multimodal y modelo de visión-lenguaje diseñado para la recuperación intermodal y la generación de representaciones utilizando texto e imágenes en chino. Emplea una arquitectura de aprendizaje contrastivo para mapear datos visuales y textuales en un espacio vectorial compartido para cálculos de similitud. El sistema permite la búsqueda bidireccional, facilitando la recuperación de texto a imagen e imagen a texto. También proporciona clasificación de imágenes zero-shot, que identifica objetos dentro de imágenes sin requerir entrenamiento específico para la tarea. El proyecto incluye herramientas para el ajuste fino (fine-tuning) de modelos preentrenados en conjuntos de datos especializados mediante entrenamiento distribuido y aprendizaje contrastivo. También proporciona utilidades para exportar pesos de modelos a formatos optimizados para aumentar la velocidad de inferencia en entornos de producción.

BLIP is a vision-language model framework that combines contrastive, matching, and language modeling objectives to align images with text. Built on a multimodal encoder-decoder architecture, it supports distributed data-parallel training with cosine learning rate scheduling and sliding-window metric tracking for training stability. The framework provides capabilities for image captioning, visual question answering, and cross-modal retrieval, scoring semantic alignment between images and text through learned embeddings. It includes toolkits for fine-tuning pre-trained models on custom datasets

moco es una implementación en PyTorch de momentum contrast diseñada para el aprendizaje de representación visual auto-supervisado. Sirve como un framework basado en investigación para extraer características de imagen de alto nivel de datasets sin etiquetar, maximizando la similitud entre diferentes vistas de la misma imagen. El sistema utiliza una arquitectura de codificador asimétrico que consiste en un codificador online de aprendizaje rápido y un codificador de momentum de evolución lenta para estabilizar el entrenamiento. Emplea un enfoque basado en diccionario que compara imágenes de consulta contra una cola dinámica de muestras negativas para aprender características visuales distintivas sin etiquetado humano manual. El framework cubre el flujo de trabajo de aprendizaje contrastivo de extremo a extremo, incluyendo el aprendizaje de representación visual no supervisada y el análisis de imágenes sin etiquetar. Aprovecha operaciones de tensores aceleradas por GPU para cálculos de similitud de vectores de alta dimensión y entrenamiento de modelos.

PlugNPlay-Modules is a collection of reusable PyTorch computer vision modules and deep learning architectural components. It provides a library of standardized building blocks for constructing neural networks, focusing on attention mechanisms, signal processing layers, and feature fusion modules. The project is distinguished by its extensive variety of attention primitives, covering spatial, channel, and temporal weighting, as well as specialized variants like deformable, frequency-enhanced, and linear-complexity attention. It also implements advanced signal processing tools within the neural

Caffe es un framework de deep learning de alto rendimiento y una librería de redes neuronales convolucionales diseñada para entrenar y desplegar redes neuronales. Funciona como un motor de machine learning acelerado por GPU con un núcleo implementado en C++ para permitir operaciones de tensores de alto rendimiento. El proyecto utiliza un sistema de configuración declarativa donde las arquitecturas de modelos y los hiperparámetros se definen en archivos de texto externos, separando el diseño de la red del código de ejecución. Incluye un sistema de serialización de modelos para exportar pesos y topologías entrenadas a archivos binarios para un despliegue eficiente en diferentes entornos de hardware. El framework cubre una amplia gama de capacidades, incluyendo el diseño de arquitectura de redes neuronales, entrenamiento de modelos supervisados con optimización basada en gradientes y flujos de trabajo de clasificación de imágenes. Proporciona herramientas para el preprocesamiento de datasets, extracción de características neuronales y ajuste fino de modelos preentrenados. El núcleo en C++ es accesible a través de una interfaz multilenguaje con bindings oficiales para Python y MATLAB.

Recognize-anything is a multimodal foundation model designed for image recognition, visual tagging, and the generation of descriptive text captions from visual input. It functions as a multimodal embedding model that maps images and text into a shared vector space to enable cross-modal retrieval and recognition. The system implements zero-shot image classification and open-vocabulary object detection, allowing it to recognize object categories not present in the original training data through custom label embeddings. It also features a visual tagging engine and a captioning system that produc

Lightly is a self-supervised learning framework and computer vision data curation tool designed to manage large image datasets and train models on unlabeled data. It functions as a PyTorch vision library and dataset management SDK, providing tools to convert raw images into high-dimensional vectors for similarity search, visualization, and feature extraction. The project implements a variety of self-supervised architectures, including MoCo, SimCLR, VICReg, Barlow Twins, and masked image modeling. It distinguishes itself by combining these learning frameworks with active learning capabilities,

GraphSAGE is a graph neural network framework designed for inductive representation learning on large-scale graphs. It functions as an inductive graph embedding tool and neighborhood aggregation engine, enabling the generation of numerical node representations that generalize to previously unseen data. The system distinguishes itself by computing node embeddings through the aggregation of features from local neighborhoods rather than relying on a global lookup table. This approach allows the framework to operate as both a supervised graph classifier for predicting categorical node classes and

Instructor-embedding es un framework de procesamiento de lenguaje natural diseñado para transformar texto no estructurado en vectores numéricos de alta dimensión. Al utilizar una arquitectura de codificador basada en transformers, el sistema facilita la recuperación semántica, la clasificación de datos y el análisis de similitud en grandes datasets. El framework se distingue por la proyección vectorial condicionada por instrucciones, que incorpora instrucciones en lenguaje natural directamente en el proceso de embedding para mejorar el rendimiento en tareas específicas sin requerir entrenamiento adicional. Funciona como una librería de aprendizaje contrastivo, permitiendo a los usuarios ajustar modelos de lenguaje preentrenados en datasets personalizados para crear embeddings especializados para dominios de nicho. El proyecto proporciona un conjunto integral de herramientas para gestionar representaciones vectoriales, incluyendo capacidades para realizar benchmarks de precisión de modelos contra métricas estandarizadas e indexar embeddings para búsquedas de similitud rápidas. Para soportar el despliegue en entornos con recursos limitados, el framework incluye características de optimización como la cuantización de modelos de precisión mixta para reducir el uso de memoria y acelerar la velocidad de inferencia.