Awesome GitHub RepositoriesDense Embeddings

This project is a framework for training and deploying transformer-based models that map text, images, audio, and video into dense or sparse vector representations. It functions as a multimodal embedding library and semantic search engine used to retrieve relevant documents by calculating vector similarity between meanings. The framework provides specialized tools for both cross-encoder reranking, which calculates precise similarity scores to refine search results, and vector quantization to compress embedding vectors for reduced memory usage and increased retrieval speed. The project covers

This project is a transformer-based framework for generating dense and sparse vector embeddings of text and multimodal data. It serves as a library for fine-tuning models to perform semantic similarity tasks, retrieval, and reranking. The system is distinguished by its support for diverse architectural patterns, including bi-encoders for fast similarity search and cross-encoders for high-precision reranking. It provides dedicated pipelines for multimodal embeddings, mapping text and images into a shared vector space, and implements knowledge distillation to compress large models into smaller,

This project is a self-supervised vision foundation model based on a vision transformer architecture. It is designed to learn dense visual representations from unlabeled images, serving as a general-purpose backbone for a wide variety of downstream vision tasks. The system is distinguished by its use of self-distillation and masked image modeling to extract semantic and geometric features. It also incorporates an image-text alignment model that maps visual embeddings to textual descriptions, enabling zero-shot image recognition, zero-shot segmentation, and cross-modal retrieval. The project

This project is an educational implementation guide and framework for building Retrieval Augmented Generation systems. It provides a workflow for constructing a knowledge base pipeline that partitions documents, indexes them as vectors, and provides external context for language model prompts. The system features a document chunking framework that uses recursive character splitting to fit text into model context windows. It includes an in-memory vector store and a similarity search system that retrieves relevant text segments by calculating the mathematical distance between dense embedding ve

该项目是一个用于文本分类的卷积神经网络的 TensorFlow 实现。它作为一个深度学习文本分类器，通过识别和分析训练集中的学习模式，将预定义标签分配给文本文档。 该模型利用嵌入层向量化、用于特征提取的卷积层和用于处理文本数据的最大池化下采样序列。最终类别概率通过密集层分类系统确定。 该工作流涵盖了端到端的机器学习生命周期，包括监督文本学习、具有可配置嵌入维度和过滤器大小的模型训练，以及使用验证数据集和基于检查点的模型持久化进行的性能评估。

Flashlight is a standalone C++ machine learning library and tensor library used for building and training neural networks. It functions as a comprehensive neural network framework and automatic differentiation engine, providing the tools to construct computation graphs and calculate gradients via backpropagation. The project serves as a distributed training framework, utilizing all-reduce operations to synchronize gradients and parameters across multiple compute nodes and devices. It distinguishes itself through deep integration of high-performance tensor manipulation, native device memory in

该项目是一个专为命名实体识别（NER）和文本分类设计的自然语言处理系统。它使用机器学习方法从原始文本中识别特定名称和关键信息，从而将非结构化内容组织为结构化格式。 该系统实现了多层架构，结合了用于嵌入的预训练 Transformer、用于序列建模的双向长短期记忆网络（Bi-LSTM）以及用于标签转换的条件随机场（CRF）。它通过在特定任务数据集上微调这些模型来支持迁移学习。 该项目包含使用指定配置和词汇表文件在自定义数据集上训练模型的能力。它还提供了一种将训练好的模型部署为网络服务的机制，允许通过 HTTP 服务器进行文本分类和实体识别。

该项目是一个命名实体识别框架和基于 TensorFlow 的自然语言处理模型。它提供了一个流水线，用于将预训练语言模型适配到特定的实体识别和文本分类任务。 该系统实现了序列标注架构，结合了基于 Transformer 的嵌入、双向序列建模和条件随机场（CRF）解码。它包括用于微调模型权重和训练网络以识别并分类非结构化文本中实体的工具。 该框架还包含一个客户端-服务器架构，通过 HTTP API 暴露训练好的模型。这允许通过网络接口进行远程推理、命名实体预测和文本分类。

This project is an educational implementation of a small-scale generative pre-trained transformer designed to teach the fundamentals of neural network architecture and training. It serves as a reference implementation and tutorial for constructing a text-generating neural network from scratch. The codebase demonstrates the mechanics of tokenization, self-attention, and the construction of a lightweight language model. It focuses on the step-by-step process of building a generative model to illustrate how large language models are constructed. The implementation covers transformer-based archi

Engram 是一个用于大语言模型的动态知识检索系统和记忆增强框架。它作为一个可扩展的记忆查找层和稀疏架构组件，旨在将静态模型知识与动态外部状态融合，以提高事实准确性并减少幻觉。 该系统利用条件记忆检索和可微记忆寻址，将输入 Token 映射到大规模关联记忆存储中的特定索引。这允许模型通过将权重存储在外部查找表中，并仅为给定输入激活相关的知识片段，从而增加其总可用参数量。 该框架涵盖了模型稀疏性优化和可扩展增强，使用键值检索和动态参数融合来提升特定任务的性能，而无需对网络进行全面重训练。

Tiny Universe is an educational monorepo that delivers multiple independent implementations of core AI subsystems as self-contained Jupyter notebooks. It provides from-scratch constructions of foundational architectures including a complete Transformer model built from the original paper specification, a denoising diffusion probabilistic model for image generation, and a ReAct-style autonomous agent framework that equips an LLM with tools for planning and multi-step task execution. The project distinguishes itself by covering the full lifecycle of modern AI systems through hands-on implementa

这是一个 TensorFlow 机器学习示例集合，为各种神经网络范式提供了参考实现。它涵盖了监督学习、无监督学习、强化学习和序列学习模型。 该仓库包含了专注于图像分类和排序的卷积神经网络实现，以及用于时间序列预测和序列到序列翻译的循环神经网络。此外，它还提供了通过奖励优化训练的强化学习智能体，以及用于数据聚类的自编码器和自组织映射等无监督学习技术。 其他功能涵盖了监督回归和分类、语义嵌入生成，以及用于序列数据建模的隐马尔可夫模型。该项目还包括用于张量操作管理和通过仪表板进行模型性能可视化的实用工具。 内容以一系列 Jupyter Notebook 的形式提供。

tiny-llm is a large language model inference engine and transformer model implementation. It serves as a quantized model runtime and paged key-value cache manager, providing a specialized inference stack optimized for Apple Silicon. The system distinguishes itself through high-throughput execution techniques, including continuous batching and paged attention. It utilizes a paged memory system to eliminate fragmentation during token generation and employs on-the-fly dequantization of compressed weights to reduce the memory footprint during matrix multiplication. The project covers a broad ran

本项目是一门教育课程及教学材料集，旨在通过 Python 从零开始构建大语言模型。它提供了一份分步指南和实用教程，重点关注 Transformer 架构的内部机制和预训练工作流。 该仓库提供了一个用于实现和比较不同模型家族（包括 Llama、GLM 和 RWKV）的框架。它使用配置驱动的组装方法来分析这些不同架构的结构差异和内部机制。 代码库涵盖了完整的开发流水线，包括文本预处理、字节对编码（BPE），以及带有因果掩码的多头注意力机制实现。它还包括训练工具，如梯度裁剪、学习率调度和用于在未标记语料库上进行预训练的超参数优化。 该项目使用 Jupyter Notebooks 实现。

这是一个基于 TensorFlow 的语音转换框架和深度学习音频工具包，专为神经语音风格迁移而设计。它作为一个语音合成引擎，将源说话人的语音频谱特征转换为目标说话人的声音特征。 该系统采用基于音素的语音转换方法，将音频话语分类为与说话人无关的音素，并使用目标声音重新合成它们。此流水线允许通过映射不同说话人之间的音频特征来转换语音特征。 该工具包包括跨多个 GPU 进行音频模型训练、张量数据归一化以及管理模型超参数的功能。它还提供了用于监控性能的工具，例如通过混淆矩阵可视化分类准确率。

This project is a generative AI educational resource and natural language processing course. It serves as a technical implementation guide for building, pre-training, and fine-tuning a large language model from scratch using PyTorch. The curriculum provides a step-by-step tutorial on large language model development, focusing specifically on the design of transformer-based text generation models. It includes dedicated instruction on parameter-efficient fine-tuning to optimize training by updating only a small subset of model weights. The material covers the end-to-end generative AI training

TransformerLens is a library for mechanistic interpretability research designed to reverse engineer the learned algorithms within large language models. It provides a standardized framework for wrapping diverse transformer architectures, allowing researchers to extract, manipulate, and analyze internal activations and weights through a consistent interface. The project distinguishes itself through a comprehensive system of activation hooks that can capture, patch, and ablate internal tensors during the forward pass. It includes specialized utilities for decomposing fused projections, material