Awesome GitHub RepositoriesLong-Context Models

Qwen2.5 is a suite of large language model foundation models designed for natural language generation, code production, and complex mathematical reasoning. The project encompasses a multilingual language model capable of processing dozens of languages and a specialized code generation model for technical problem solving and debugging. The framework is distinguished by its long context capabilities, enabling the analysis of massive inputs ranging from 256K up to 1 million tokens. It further functions as an agentic framework, utilizing standardized templates and parsers to execute autonomous wo

Qwen2.5-VL este un transformer multimodal autoregresiv conceput pentru a procesa secvențe intercalate de token-uri de text și vizuale. Acesta integrează embedding-urile caracteristicilor vizuale într-un spațiu comun de model de limbaj pentru a efectua raționamente cross-modale și a genera răspunsuri coerente sau cod de layout structurat. Proiectul se distinge prin maparea viziune-limbaj-acțiune, permițându-i să perceapă interfețele vizuale și să traducă acea percepție în comenzi acționabile pentru operarea ecranelor digitale și a hardware-ului robotic. Utilizează codificarea imaginilor cu rezoluție dinamică și indexarea video pe cadre temporale pentru a gestiona dimensiuni diverse ale imaginilor și secvențe vizuale de lungă durată. Modelul acoperă o suprafață largă de capabilități, inclusiv recunoașterea optică a caracterelor multilingve pentru digitizarea documentelor, ancorarea spațială pentru localizarea obiectelor prin bounding boxes și analiza conținutului video de lungă durată. De asemenea, suportă raționamentul matematic multimodal pentru a rezolva probleme folosind grafice și diagrame și își extinde înțelegerea la o lungime de context de un milion de token-uri.

The Gemini Cookbook is a comprehensive collection of implementation patterns, code samples, and development guides designed for building applications with Google Gemini models. It serves as a central resource for developers to integrate multimodal generative artificial intelligence into their software, providing the necessary frameworks to manage model interactions, stateful workflows, and structured data extraction. The repository distinguishes itself by offering specialized toolkits for autonomous agent orchestration, enabling the construction of agents that can execute code, browse the web

CodeQwen1.5 is a large language model designed for generating, completing, and analyzing code. It functions as an AI code generator capable of writing programming logic across hundreds of different languages. The model is distinguished by its long-context capabilities, allowing it to process up to one million tokens to reason across entire software repositories. It also operates as a function calling model, utilizing specialized formats to execute complex coding tasks and browser-based automation. The system supports intelligent code completion through fill-in-the-middle capabilities, which

Qwen2.5-Coder is a code-centric large language model designed to generate, complete, and analyze source code. It serves as a polyglot programming model capable of producing functional code across hundreds of different programming languages. The model is optimized for reasoning over extensive software repositories, utilizing a context window that supports up to one million tokens. It also functions as an agentic coding framework, executing multi-step workflows and browser tasks through specialized function call formats. Its capabilities include large-scale codebase analysis, intelligent parti

ChatGLM2-6B is an open-weight large language model designed for natural language conversations and text generation in both English and Chinese. It functions as a bilingual chat model capable of processing and maintaining coherence across text sequences up to 32K tokens. The model is optimized for local deployment through precision quantization, which reduces memory requirements to allow execution on consumer-grade hardware. It supports distributing model weights across multiple graphics cards to handle parameters that exceed the memory of a single device. The project covers capabilities for

ChatGLM2-6B is a bilingual chat large language model designed for natural conversation and text generation in both English and Chinese. It functions as a fine-tunable language model that supports updating weights via specialized scripts to adapt to specific datasets and tasks. The project serves as a quantized inference engine and multi-GPU model orchestrator, enabling the execution of large models on consumer-grade hardware. It is capable of processing long context sequences up to 32K tokens to maintain understanding across extended documents. The system covers capabilities for multilingual

Qwen3-Coder is a specialized large language model designed for software development, technical reasoning, and automated code synthesis. Built on transformer-based sequence modeling, it functions as a multilingual programming assistant capable of generating, completing, and debugging source code across more than one hundred programming languages. The model distinguishes itself through its capacity to process and maintain logical coherence across massive datasets, supporting context windows of up to one million tokens. This allows for repository-scale reasoning, enabling the model to analyze co

cc-connect is an AI agent messaging bridge and session manager that connects local AI coding agents to third-party messaging platforms. It acts as a multimodal AI chat relay and a OneBot protocol gateway, allowing users to control local AI agents remotely via a variety of chat interfaces. The project distinguishes itself by providing a remote AI agent controller that enables the management of agents through slash commands and a web management dashboard. It supports multi-tenant project orchestration and session-based context isolation, ensuring that independent conversation threads are mainta

Pipecat is a framework and software development kit for building real-time multimodal AI agents and speech-to-speech systems. It utilizes a frame-based data pipeline to route audio, video, and text through a modular sequence of processors, enabling the orchestration of low-latency conversational AI. The project is distinguished by its ability to coordinate complex multimodal services, including speech-to-text, language models, and text-to-speech, within a single pipeline. It features semantic voice activity detection for natural turn-taking, state-machine conversation flows for dialogue manag

EdgeGPT is a reverse engineered API wrapper and programmatic client for interacting with Bing Chat and associated large language model services. It enables the retrieval of text responses, code snippets, and suggested questions through a structured interface. The project uses exported browser cookies for authentication and implements an automated session rotation system to bypass daily request limits and regional restrictions. It manages multiple cookie sets to maintain continuous service uptime. The system also includes capabilities for AI image generation, automating requests to create vis

Yi is a bilingual language model and foundation model designed for natural language processing, reasoning, and reading comprehension in both English and Chinese. It is built as a transformer-based architecture capable of general purpose text generation and conversational tasks. The model is distinguished by its ability to function as a long context system, processing and analyzing extended input sequences up to 200k tokens. It also supports quantized versions that use low-bit precision to reduce memory footprints, enabling execution on consumer-grade hardware. The project covers a broad rang

InternLM is a large language model and a comprehensive suite of weights designed for text generation and complex reasoning. It functions as an inference engine for serving responses, a fine-tuning framework for adjusting model weights, and a platform for building autonomous AI agents. The system is capable of processing long-context input sequences up to one million tokens for document analysis. It employs chain-of-thought reasoning to solve knowledge-intensive tasks by generating intermediate logic steps before producing a final answer. The project covers model weight optimization through s

This project provides a Chinese large language model based on the LLaMA architecture. It is an instruction-tuned model optimized for natural language processing and multi-turn conversations in Chinese. The system includes a framework for parameter-efficient fine-tuning using low-rank adaptation and quantization to reduce memory requirements. It also implements retrieval augmented generation for local document question answering and supports long-context processing for sequences up to 64K tokens. The project covers a broad set of capabilities including supervised instruction tuning, reinforce

GLM-4 is a large language model and fine-tuning framework designed for human-like text production, complex reasoning, and multilingual conversation. It functions as a multimodal system capable of processing high-resolution visual content and as a long-context model designed to analyze documents with a context window of up to one million tokens. The project differentiates itself through a function calling interface that enables AI agent development by connecting the model to external APIs and real-time web browsing. It includes specialized capabilities for generating functional programming cod

This project is a multimodal AI proxy and content generation hub that provides a unified web interface for interacting with multiple large language models and generative AI services. It functions as a secure API access gateway, routing requests from a single dashboard to various external AI backends using configurable base URLs and API keys. The platform is delivered as a cross-platform progressive web application, allowing for installation on Linux, Windows, and MacOS. It distinguishes itself by consolidating text, image, audio, and video generative controls into a standardized interface, su

Acest proiect este un framework de procesare a limbajului natural axat pe un pre-antrenor autoregresiv generalizat conceput pentru reprezentarea limbajului nesupervizat. Implementează un model de limbaj care combină antrenamentul bazat pe permutare cu un backbone Transformer-XL pentru a funcționa ca un procesor de text cu context lung. Sistemul se distinge prin capacitatea de a gestiona secvențe de text care depășesc limitele standard de lungime prin utilizarea recurenței la nivel de segment și a codificării poziționale relative. Acesta scalează pre-antrenamentul de înaltă performanță pe mai multe GPU-uri și clustere TPU folosind implementări de antrenament distribuit. Codul sursă acoperă întregul flux de lucru de machine learning, inclusiv curățarea textului și tokenizarea subcuvintelor pentru preprocesarea datelor, precum și fine-tuning-ul specific sarcinii pentru răspunsul la întrebări, înțelegerea lecturii și clasificarea textului. Include utilitare pentru optimizarea parametrilor, programarea ratei de învățare și evaluarea probabilităților de răspuns prin metrici de precizie-rechemare. Proiectul oferă configurații pentru gestionarea hiperparametrilor modelului și antrenamentul accelerat hardware pe mai multe gazde.

x-transformers este o bibliotecă PyTorch și un toolkit de cercetare pentru construirea arhitecturilor de tip transformer. Oferă un framework modular pentru implementarea cercetării experimentale în domeniul transformer, incluzând o suită de mecanisme avansate de atenție, instrumente de modelare a secvențelor lungi și un framework pentru vision transformers. Proiectul se distinge prin accentul pus pe componente eficiente din punct de vedere al memoriei și de înaltă performanță, cum ar fi Flash Attention cu nuclee (kernels) tiled și atenție multi-query. De asemenea, implementează metode specializate pentru extinderea ferestrelor de context, inclusiv recurența secvențelor și embedding-uri poziționale rotative. Biblioteca acoperă o gamă largă de capabilități arhitecturale, inclusiv diverse scheme de normalizare pentru stabilizarea antrenamentului, rețele feedforward cu porți (gated) și topologii de straturi personalizate precum rețelele Macaron. Suportă construcții atât de tip encoder, cât și decoder, oferind instrumente pentru generarea autoregresivă de secvențe și sarcini vision-language, cum ar fi generarea de subtitrări pentru imagini.

The official PyTorch implementation of Google's Gemma models