awesome-repositories.com
Blog
awesome-repositories.com

Descubre los mejores repositorios open-source con nuestra búsqueda potenciada por IA.

ExplorarBúsquedas curadasAlternativas open-sourceSoftware autohospedableBlogMapa del sitio
ProyectoAcerca deCómo clasificamosPrensaServidor MCP
Aviso legalPrivacidadTérminos
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·
microsoft avatar

microsoft/unilm

0
View on GitHub↗
22,030 estrellas·2,694 forks·Python·mit·9 vistasaka.ms/GeneralAI↗

Unilm

This project is a comprehensive framework and toolkit for developing, optimizing, and deploying transformer-based models across multimodal, document intelligence, and natural language processing tasks. It provides a unified neural architecture that processes text, vision, audio, and document layout data through a shared set of weights, enabling researchers and developers to build foundational models that align cross-modal representations.

The platform distinguishes itself through advanced training and inference strategies designed for large-scale deep learning. It incorporates specialized mechanisms such as retentive state processing for efficient sequence generation, differential attention for improved focus, and distributed weight partitioning to handle memory-intensive computations. These capabilities are complemented by techniques for sparse decoding and model compression, which maintain performance while reducing the computational footprint of large-scale architectures.

The project covers a broad capability surface, including end-to-end pipelines for data curation, synthetic data generation, and tokenization across diverse modalities. It supports extensive workflows for pre-training, instruction tuning, and fine-tuning, with specific focus areas in document understanding, speech synthesis, and cross-lingual transfer. Diagnostic tools for attention analysis and benchmarking further assist in evaluating model performance on complex reasoning and retrieval tasks.

Features

  • Intelligent Document Processing - Provides a comprehensive framework for extracting information from visually-rich documents by integrating text, layout, and image analysis.
  • Language Model Fine-Tuning - Supports large language model fine-tuning to adapt pre-trained models to specific domains and downstream tasks.
  • Large Language Models - Offers a complete toolkit for pretraining, instruction tuning, and optimizing transformer-based models for diverse natural language tasks.
  • Language Model Training - Provides distributed pre-training pipelines for building large-scale language models from scratch.

Búsqueda con IA

Explora más repositorios increíbles

Describe lo que necesitas en lenguaje sencillo: la IA clasifica miles de proyectos open-source curados por relevancia.

Start searching with AI
Multimodal AI Systems - Provides a comprehensive framework for multimodal AI development, integrating text, vision, audio, and document layout data.
  • Structured Document Extraction - Extracts information from structured documents like forms and receipts by analyzing both textual content and visual layout features.
  • Document Question Answering Pipelines - Analyzes the structure and content of web pages to provide accurate answers to natural language queries about the document.
  • Unified Frameworks - Provides a research platform for training and fine-tuning unified transformer models across text, vision, audio, and document modalities.
  • Training Efficiency - Provides efficient model training workflows through distributed training, memory optimization, and hardware-aware kernels.
  • Modular Backbone Architectures - Provides a unified transformer backbone that processes text, vision, and audio inputs through a shared set of weights.
  • Multimodal Models - Enables the development of foundational models that align and process cross-modal data including speech, images, and text.
  • Document Layout Analysis - Identifies and segments structural elements within document images such as text blocks, figures, and tables.
  • Speech Synthesis - Generates natural-sounding human speech from short text prompts using neural codec language models.
  • Information Extraction - Processes text and markup from visually-rich documents to identify and pull specific data points.
  • Long Context Training Optimizations - Implements dilated attention mechanisms to handle context windows of up to one billion tokens efficiently.
  • Model-Driven Text Extraction - Identifies text within images and assigns precise spatial coordinates to enable document-level text recognition.
  • Multimodal Layout Analysis - Integrates text, spatial layout, and visual image data into a unified model to extract information and understand visually-rich documents.
  • Vision Model Fine-Tuning - Adapts pre-trained transformer models to specific downstream vision tasks like image classification and semantic segmentation.
  • Inference Optimization - Reduces memory usage and improves computational efficiency during sequence generation using gated retention mechanisms.
  • Language Model Fine-Tuning - Adapts pre-trained document understanding models to specific downstream tasks like question answering or form extraction using task-specific datasets.
  • Mixture of Experts - Scales deep learning models using specialized architectures like mixture-of-experts and retentive networks.
  • Unified Understanding and Generation Training - Trains neural networks on combined datasets to perform both natural language understanding and text generation within a single architecture.
  • Speech Processing - Supports speech and audio processing for automatic speech recognition, voice synthesis, and acoustic representation learning.
  • Memory Optimization Techniques - Reduces GPU memory consumption during training using distributed strategies and activation checkpointing.
  • Model Optimization Suites - Implements a suite of techniques for accelerating inference and reducing memory usage in large-scale deep learning architectures.
  • Model Compression Suites - Reduces model size and computational requirements through self-attention distillation.
  • Attention Kernel Fusion - Executes differential attention operations efficiently using hardware-aware kernels to accelerate training and inference.
  • Weight Distribution - Provides distributed weight partitioning strategies to handle memory-intensive computations across multiple processors during large-scale model development.
  • Multimodal Large Language Models - Integrates visual and textual data into a unified model to enable multimodal understanding and generation tasks across different input modalities.
  • Multimodal Training - Trains unified models capable of processing and generating across text, vision, speech, and document modalities.
  • Multilingual Text Recognition - Processes visual input using transformer architectures to generate text output for handwritten and printed documents.
  • Synthetic Data Generation - Provides synthetic data generation pipelines to create pseudo-test inputs for evaluating and filtering training data.
  • Training Data Generation - Supports synthetic training data generation to create large-scale instruction-tuning datasets for model improvement.
  • Retentive State Mechanisms - Implements retentive state processing to enable efficient sequence generation and handle long-context data during inference.
  • Image Segmentation - Adapts pre-trained vision models to perform pixel-level semantic segmentation for identifying and labeling distinct objects within an image.
  • Dataset Preparation Tools - Provides comprehensive training dataset curation tools for organizing general knowledge, code, and mathematical datasets.
  • Distributed Training - Distributes large model training across multiple processors by partitioning model weights to handle memory-intensive computations efficiently.
  • Image Classification - Fine-tunes pre-trained vision models to categorize images into predefined classes with high accuracy.
  • Document Image Model Pre-training - Learns visual representations from large-scale unlabeled document images using self-supervised techniques.
  • Vision Transformer Pre-training - Trains image transformer models using masked image modeling to learn visual representations from large-scale datasets.
  • Speech Model Fine-Tuning - Adapts pre-trained audio representations to specific downstream tasks like speaker verification and speech separation.
  • Model Performance Benchmarking - Runs standardized benchmarks on mathematical reasoning datasets to measure model accuracy and output quality.
  • Model Training Pipelines - Executes instruction-tuning pipelines on large-scale grounded image-text datasets for unified vision-language systems.
  • Model Fine-Tuning and Adaptation - Offers comprehensive tools for refining pre-trained models across various domains and task requirements.
  • Synthetic Speech Generation - Converts text input into natural-sounding audio using pre-trained models and vocoders.
  • Self-Supervised Speech Representations - Trains large-scale self-supervised models on extensive audio datasets to generate robust representations for speech processing.
  • Model Deployment Toolkits - Provides toolkits for efficient sequence-to-sequence decoding and model compression for production environments.
  • Multimodal Token Interleaving - Implements multimodal data tokenization to align audio, text, and visual inputs into unified sequences for model training.
  • Self-Supervised Embedding Trainers - Trains vision models using self-supervised learning to create reusable feature representations.
  • Multimodal Document Pre-training - Learns joint representations of text, spatial layout, and visual image features for document understanding tasks.
  • Preference Optimization - Refines model outputs using direct preference optimization by comparing responses against feedback.
  • Sequence Decoders - Predicts multiple tokens simultaneously during sequence generation to reduce decoding steps.
  • Subword Tokenization - Implements subword text tokenization to convert raw text into numerical sequences for transformer architectures.
  • Vision-Language Grounding Models - Links text spans such as noun phrases and referring expressions to specific image regions to enable phrase grounding and comprehension.
  • Advanced Learning - BERT-style pre-training for image transformers.
  • Attention Optimization - Implements decoder-decoder architectures to optimize cache usage.
  • Foundation Models - Grounding multimodal models to real-world entities.
  • Generalization And Learning - Neural codec models for zero-shot in-context learning.
  • Large Language Models - Self-supervised pre-training across tasks and modalities.
  • Model Architectures - Decoder-decoder architecture for efficient caching.
  • Model Quantization Tools - Native 4-bit activations for 1-bit LLM architectures.
  • Multimodal Foundation Models - General-purpose foundation model for vision and language tasks.
  • Multimodal Models - Framework for transformer-based optical character recognition.
  • Self-Supervised Pretraining - Applies BERT-style pretraining to image transformers.
  • Sequence To Sequence Models - Unified framework for various sequence-to-sequence generation tasks.
  • Vision Language Models - Transformer-based models for aligning perception with linguistic capabilities.
  • Vision Models - Document image transformer for self-supervised pre-training.
  • Document Processing - LayoutLM-v3 model for document understanding.
  • Reading Order Predictors - Analyzes text and spatial layout information within document images to determine the logical sequence in which text lines should be read.
  • AI-Generated Captions - Produces descriptive text summaries for images by interpreting visual content and generating corresponding natural language captions.
  • Visual Tokenizers - Implements visual data tokenization to convert raw images into discrete tokens using encoder-decoder architectures.
  • Sparse Caching Strategies - Implements sparse key-value caching to maintain accuracy while accelerating the processing of long text sequences.
  • Attention Mechanisms - Calculates attention scores using a differential mechanism that subtracts two separate attention maps to improve model focus and performance.
  • Audio Tokenization - Learns acoustic representations from raw audio data using iterative tokenization for downstream classification tasks.
  • Data Preparation - Provides pipelines for multilingual training data preparation, converting raw text and parallel pairs into memory-mapped binary formats.
  • Multilingual Extractors - Extends document understanding capabilities to multiple languages by training on cross-lingual datasets to extract key-value pairs from international document formats.
  • Reading Order Benchmarks - Provides large-scale datasets of document images paired with ground-truth reading order information to evaluate and train document analysis models.
  • Text-to-Image Generators - Creates images containing coherent text by using text prompts and layout guidance.
  • Visual Text Renderers - Generates visual output from text inputs using pre-trained models or fine-tuned adapters to render specific text styles and layouts.
  • Knowledge Distillation - Transfers knowledge from teacher models to student retrievers to improve performance and efficiency.
  • Mathematical Reasoning Training - Trains models on large-scale synthetic instruction datasets to enhance mathematical problem-solving capabilities.
  • Long Context Retrieval Testing - Assesses model recall capabilities within long sequences using needle-in-a-haystack and multi-needle retrieval experiments.
  • Custom Vision Training - Supports modifying pre-trained vision and language weights to master downstream tasks like visual question answering.
  • Biencoder Pipelines - Executes a multi-stage supervised fine-tuning pipeline to develop high-performance biencoder models for information retrieval tasks.
  • Model Fine-Tuning - Adapts pre-trained models to specific document understanding objectives like semantic entity recognition and relation extraction using labeled datasets.
  • Cross-Lingual Objectives - Trains language models using masked language modeling, translation language modeling, and contrastive learning objectives to improve cross-lingual representation.
  • Retrieval Model Pre-training - Compresses input information into a representation bottleneck to create specialized models for dense passage retrieval.
  • Ternary Weight Optimizations - Optimizes large language model architectures by using ternary weights to reduce memory footprint and computational requirements.
  • Sequence-to-Sequence Tasks - Trains parameter-efficient transformer models to perform tasks like grammatical error correction and abstractive summarization on resource-constrained devices.
  • Speech Translation Systems - Translates spoken language by processing audio input and generating text output through sequence-to-sequence models.
  • Multilingual Text Processing - Facilitates multilingual text processing by tokenizing and converting raw text into binary formats for large-scale training.
  • Multimodal Prompt Adapters - Customizes pre-trained multimodal models for text-intensive image understanding tasks by applying supervised training with task-specific prompts.
  • Speech-to-Text Engines - Converts model outputs into text using language models and lexicons to improve transcription accuracy.
  • Cross-Lingual Translation Training - Provides cross-lingual transformer encoders to improve the accuracy and scalability of automated translation workflows.
  • Referring Expression Generators - Produces descriptive text for specific image regions based on provided visual context using zero-shot or few-shot learning techniques.
  • Markdown Converters - Transforms visual document layouts into structured markdown format by capturing both the text content and its original styling.
  • Document Classification - Categorizes documents based on their visual structure and content to automate sorting and organization workflows.
  • Audio Processing - Applies fine-tuned acoustic models to categorize audio inputs into specific classes based on learned patterns.
  • Cross-Modal Representations - Applies iterative word alignment and contrastive loss functions during training to synchronize semantic representations across different languages.
  • Custom Diffusion Model Training - Trains two-stage diffusion models on large-scale image-text datasets annotated with character-level segmentation masks and optical character recognition data.
  • Vocabulary Builders - Supports incremental vocabulary generation to expand token sets for domain-specific terminology.
  • Embedding Generators - Transforms text inputs into high-dimensional vector representations using pre-trained language models to support semantic search and information retrieval tasks.
  • Layout Planners - Optimizes models to predict spatial arrangements for text elements within generated images to ensure coherent composition.
  • Pre-made Models - Leverages pre-trained model weights to accelerate development of systems for complex document layout analysis.
  • Backbone Model Integration - Utilizes established transformer architectures as backbones to initialize and accelerate training of document understanding systems.
  • Result Reranking - Optimizes re-ranking models to refine retrieval results by evaluating the relevance between queries and passages more precisely.
  • Voice Cloning - Modifies speaker identity or characteristics of audio input while preserving linguistic content.
  • Data Input Interfaces - Provides automated input data processing to detect and handle raw text or pre-tokenized dataset structures.
  • Audio Feature Extraction - Processes audio input through pre-trained models to generate numerical representations for downstream speech analysis.
  • Training Cycles - Executes training cycles and performance testing on document datasets to optimize model accuracy for specialized document understanding tasks.

    • Historial de estrellas

    Gráfico del historial de estrellas de microsoft/unilmGráfico del historial de estrellas de microsoft/unilm

    Preguntas frecuentes

    ¿Qué hace microsoft/unilm?

    This project is a comprehensive framework and toolkit for developing, optimizing, and deploying transformer-based models across multimodal, document intelligence, and natural language processing tasks. It provides a unified neural architecture that processes text, vision, audio, and document layout data through a shared set of weights, enabling researchers and developers to build foundational models that align cross-modal representations.

    ¿Cuáles son las características principales de microsoft/unilm?

    Las características principales de microsoft/unilm son: Intelligent Document Processing, Language Model Fine-Tuning, Large Language Models, Language Model Training, Multimodal AI Systems, Structured Document Extraction, Document Question Answering Pipelines, Unified Frameworks.

    ¿Qué alternativas de código abierto existen para microsoft/unilm?

    Las alternativas de código abierto para microsoft/unilm incluyen: zhaochenyang20/awesome-ml-sys-tutorial — This project provides a comprehensive technical guide and framework for engineering large-scale machine learning… sgl-project/sglang — Sglang is a high-performance inference engine and serving system designed for large language and multimodal models. It… paddlepaddle/lark — LARK is a development toolkit for training, fine-tuning, and deploying large language models and multimodal models… liguodongiot/llm-action — This project is a comprehensive framework for the training, fine-tuning, and deployment of large language models. It… axolotl-ai-cloud/axolotl — Axolotl is a configuration-driven framework designed for the fine-tuning, evaluation, and quantization of large… facebookresearch/fairseq — Fairseq is a PyTorch toolkit for sequence-to-sequence modeling, specializing in neural machine translation, automatic…

    Alternativas open-source a Unilm

    Proyectos open-source similares, clasificados según cuántas características comparten con Unilm.
    • zhaochenyang20/awesome-ml-sys-tutorialAvatar de zhaochenyang20

      zhaochenyang20/Awesome-ML-SYS-Tutorial

      5,371Ver en GitHub↗

      This project provides a comprehensive technical guide and framework for engineering large-scale machine learning systems. It covers the full lifecycle of model development, focusing on the infrastructure and computational principles required to build, train, and serve generative AI models across distributed GPU clusters. The repository distinguishes itself by offering deep-dive tutorials and implementation strategies for complex system challenges. It emphasizes high-performance architectural primitives, such as collective communication orchestration, distributed tensor sharding, and static gr

      Python
      Ver en GitHub↗5,371
    • sgl-project/sglangAvatar de sgl-project

      sgl-project/sglang

      29,079Ver en GitHub↗

      Sglang is a high-performance inference engine and serving system designed for large language and multimodal models. It provides a programmable interface for orchestrating complex generation workflows, enabling developers to coordinate multi-turn dialogues, tool invocations, and reasoning chains through a domain-specific language. The platform is built to support production-scale deployments, offering an OpenAI-compatible API that allows for integration with existing application ecosystems. The system distinguishes itself through a disaggregated architecture that separates compute-intensive pr

      Pythonattentionblackwellcuda
      Ver en GitHub↗29,079
    • paddlepaddle/larkAvatar de PaddlePaddle

      PaddlePaddle/LARK

      7,717Ver en GitHub↗

      LARK is a development toolkit for training, fine-tuning, and deploying large language models and multimodal models based on PaddlePaddle. It functions as a comprehensive framework that includes an LLM training orchestrator, an inference server, and a multimodal model framework for processing text, image, and video inputs. The project features a retrieval-augmented generation system for building conversational applications that integrate web search and private knowledge bases. It provides specific capabilities for multimodal reasoning and complex logic, enabling the extraction of structured da

      Python
      Ver en GitHub↗7,717
  • liguodongiot/llm-actionAvatar de liguodongiot

    liguodongiot/llm-action

    23,169Ver en GitHub↗

    This project is a comprehensive framework for the training, fine-tuning, and deployment of large language models. It functions as a distributed deep learning platform that enables users to scale model workflows across multiple hardware nodes while providing tools for model evaluation and performance benchmarking. The platform distinguishes itself by offering specialized utilities for model compression and weight transformation, allowing users to reduce memory footprints and latency through quantization and pruning. It supports the adaptation of large models for consumer-grade hardware, facili

    HTMLllmllm-inferencellm-serving
    Ver en GitHub↗23,169
    • Ver las 30 alternativas a Unilm→