Awesome GitHub RepositoriesLong-Context Models

Qwen2.5 is a suite of large language model foundation models designed for natural language generation, code production, and complex mathematical reasoning. The project encompasses a multilingual language model capable of processing dozens of languages and a specialized code generation model for technical problem solving and debugging. The framework is distinguished by its long context capabilities, enabling the analysis of massive inputs ranging from 256K up to 1 million tokens. It further functions as an agentic framework, utilizing standardized templates and parsers to execute autonomous wo

Qwen2.5-VL est un transformeur multimodal autorégressif conçu pour traiter des séquences entrelacées de jetons de texte et visuels. Il intègre des intégrations de caractéristiques visuelles dans un espace de modèle de langage partagé pour effectuer un raisonnement transmodal et générer des réponses cohérentes ou du code de mise en page structuré. Le projet se distingue par la cartographie vision-langage-action, lui permettant de percevoir des interfaces visuelles et de traduire cette perception en commandes exploitables pour faire fonctionner des écrans numériques et du matériel robotique. Il utilise un encodage d'image à résolution dynamique et une indexation vidéo à trame temporelle pour gérer diverses tailles d'image et des séquences visuelles de longue durée. Le modèle couvre une large surface de capacités, notamment la reconnaissance optique de caractères multilingue pour la numérisation de documents, la mise à la terre spatiale pour localiser des objets via des boîtes englobantes, et l'analyse de contenu vidéo long format. Il prend également en charge le raisonnement mathématique multimodal pour résoudre des problèmes en utilisant des graphiques et des diagrammes, et étend sa compréhension à une longueur de contexte d'un million de jetons.

The Gemini Cookbook is a comprehensive collection of implementation patterns, code samples, and development guides designed for building applications with Google Gemini models. It serves as a central resource for developers to integrate multimodal generative artificial intelligence into their software, providing the necessary frameworks to manage model interactions, stateful workflows, and structured data extraction. The repository distinguishes itself by offering specialized toolkits for autonomous agent orchestration, enabling the construction of agents that can execute code, browse the web

CodeQwen1.5 is a large language model designed for generating, completing, and analyzing code. It functions as an AI code generator capable of writing programming logic across hundreds of different languages. The model is distinguished by its long-context capabilities, allowing it to process up to one million tokens to reason across entire software repositories. It also operates as a function calling model, utilizing specialized formats to execute complex coding tasks and browser-based automation. The system supports intelligent code completion through fill-in-the-middle capabilities, which

Qwen2.5-Coder is a code-centric large language model designed to generate, complete, and analyze source code. It serves as a polyglot programming model capable of producing functional code across hundreds of different programming languages. The model is optimized for reasoning over extensive software repositories, utilizing a context window that supports up to one million tokens. It also functions as an agentic coding framework, executing multi-step workflows and browser tasks through specialized function call formats. Its capabilities include large-scale codebase analysis, intelligent parti

ChatGLM2-6B is an open-weight large language model designed for natural language conversations and text generation in both English and Chinese. It functions as a bilingual chat model capable of processing and maintaining coherence across text sequences up to 32K tokens. The model is optimized for local deployment through precision quantization, which reduces memory requirements to allow execution on consumer-grade hardware. It supports distributing model weights across multiple graphics cards to handle parameters that exceed the memory of a single device. The project covers capabilities for

ChatGLM2-6B is a bilingual chat large language model designed for natural conversation and text generation in both English and Chinese. It functions as a fine-tunable language model that supports updating weights via specialized scripts to adapt to specific datasets and tasks. The project serves as a quantized inference engine and multi-GPU model orchestrator, enabling the execution of large models on consumer-grade hardware. It is capable of processing long context sequences up to 32K tokens to maintain understanding across extended documents. The system covers capabilities for multilingual

Qwen3-Coder is a specialized large language model designed for software development, technical reasoning, and automated code synthesis. Built on transformer-based sequence modeling, it functions as a multilingual programming assistant capable of generating, completing, and debugging source code across more than one hundred programming languages. The model distinguishes itself through its capacity to process and maintain logical coherence across massive datasets, supporting context windows of up to one million tokens. This allows for repository-scale reasoning, enabling the model to analyze co

cc-connect is an AI agent messaging bridge and session manager that connects local AI coding agents to third-party messaging platforms. It acts as a multimodal AI chat relay and a OneBot protocol gateway, allowing users to control local AI agents remotely via a variety of chat interfaces. The project distinguishes itself by providing a remote AI agent controller that enables the management of agents through slash commands and a web management dashboard. It supports multi-tenant project orchestration and session-based context isolation, ensuring that independent conversation threads are mainta

Pipecat is a framework and software development kit for building real-time multimodal AI agents and speech-to-speech systems. It utilizes a frame-based data pipeline to route audio, video, and text through a modular sequence of processors, enabling the orchestration of low-latency conversational AI. The project is distinguished by its ability to coordinate complex multimodal services, including speech-to-text, language models, and text-to-speech, within a single pipeline. It features semantic voice activity detection for natural turn-taking, state-machine conversation flows for dialogue manag

EdgeGPT is a reverse engineered API wrapper and programmatic client for interacting with Bing Chat and associated large language model services. It enables the retrieval of text responses, code snippets, and suggested questions through a structured interface. The project uses exported browser cookies for authentication and implements an automated session rotation system to bypass daily request limits and regional restrictions. It manages multiple cookie sets to maintain continuous service uptime. The system also includes capabilities for AI image generation, automating requests to create vis

Yi is a bilingual language model and foundation model designed for natural language processing, reasoning, and reading comprehension in both English and Chinese. It is built as a transformer-based architecture capable of general purpose text generation and conversational tasks. The model is distinguished by its ability to function as a long context system, processing and analyzing extended input sequences up to 200k tokens. It also supports quantized versions that use low-bit precision to reduce memory footprints, enabling execution on consumer-grade hardware. The project covers a broad rang

InternLM is a large language model and a comprehensive suite of weights designed for text generation and complex reasoning. It functions as an inference engine for serving responses, a fine-tuning framework for adjusting model weights, and a platform for building autonomous AI agents. The system is capable of processing long-context input sequences up to one million tokens for document analysis. It employs chain-of-thought reasoning to solve knowledge-intensive tasks by generating intermediate logic steps before producing a final answer. The project covers model weight optimization through s

This project provides a Chinese large language model based on the LLaMA architecture. It is an instruction-tuned model optimized for natural language processing and multi-turn conversations in Chinese. The system includes a framework for parameter-efficient fine-tuning using low-rank adaptation and quantization to reduce memory requirements. It also implements retrieval augmented generation for local document question answering and supports long-context processing for sequences up to 64K tokens. The project covers a broad set of capabilities including supervised instruction tuning, reinforce

GLM-4 is a large language model and fine-tuning framework designed for human-like text production, complex reasoning, and multilingual conversation. It functions as a multimodal system capable of processing high-resolution visual content and as a long-context model designed to analyze documents with a context window of up to one million tokens. The project differentiates itself through a function calling interface that enables AI agent development by connecting the model to external APIs and real-time web browsing. It includes specialized capabilities for generating functional programming cod

This project is a multimodal AI proxy and content generation hub that provides a unified web interface for interacting with multiple large language models and generative AI services. It functions as a secure API access gateway, routing requests from a single dashboard to various external AI backends using configurable base URLs and API keys. The platform is delivered as a cross-platform progressive web application, allowing for installation on Linux, Windows, and MacOS. It distinguishes itself by consolidating text, image, audio, and video generative controls into a standardized interface, su

Ce projet est un framework de traitement du langage naturel axé sur un pré-entraîneur autorégressif généralisé conçu pour la représentation de langage non supervisée. Il implémente un modèle de langage qui combine un entraînement basé sur la permutation avec une architecture Transformer-XL pour fonctionner comme un processeur de texte à long contexte. Le système se distingue par sa capacité à gérer des séquences de texte dépassant les limites de longueur standard grâce à l'utilisation de la récurrence au niveau du segment et du codage positionnel relatif. Il met à l'échelle le pré-entraînement haute performance sur plusieurs clusters de GPU et TPU en utilisant des implémentations d'entraînement distribué. La base de code couvre l'ensemble du workflow de machine learning, incluant le nettoyage de texte et la tokenisation par sous-mots pour le prétraitement des données, ainsi que le fine-tuning spécifique à une tâche pour la réponse aux questions, la compréhension de lecture et la classification de texte. Il inclut des utilitaires pour l'optimisation des paramètres, la planification du taux d'apprentissage et l'évaluation des probabilités de réponse via des métriques de précision-rappel. Le projet fournit des configurations pour gérer les hyperparamètres du modèle et l'entraînement accéléré par le matériel sur plusieurs hôtes.

x-transformers est une bibliothèque PyTorch et un toolkit de recherche pour construire des architectures transformer. Il fournit un framework modulaire pour implémenter la recherche transformer expérimentale, incluant une suite de mécanismes d'attention avancés, des outils de modélisation de séquences longues et un framework pour les vision transformers. Le projet se distingue par son accent sur les composants haute performance et économes en mémoire, tels que Flash Attention avec des noyaux tuilés et l'attention multi-requêtes. Il implémente également des méthodes spécialisées pour étendre les fenêtres de contexte, incluant la récurrence de séquence et les plongements positionnels rotatifs. La bibliothèque couvre un large éventail de capacités architecturales, incluant divers schémas de normalisation pour stabiliser l'entraînement, des réseaux feedforward à portes et des topologies de couches personnalisées comme les réseaux Macaron. Elle prend en charge les constructions d'encodeur et de décodeur, fournissant des outils pour la génération de séquence autorégressive et les tâches vision-langage comme la légende d'image.

The official PyTorch implementation of Google's Gemma models