ml-explore/mlx-examples
Mlx Examples
This repository provides a collection of reference implementations and code examples for training and deploying machine learning models using the MLX framework. It serves as a practical guide for executing distributed training, fine-tuning large language models, converting model weights, and implementing multimodal generative workflows.
The project distinguishes itself through specialized examples for local hardware execution, featuring weight quantization to reduce memory usage and low-rank adaptation for parameter-efficient fine-tuning. It also includes scripts for transforming external model formats into MLX-compatible versions and merging adapter weights for standalone deployment.
The examples cover a broad range of capabilities, including natural language processing with decoder-only and mixture-of-experts architectures, computer vision for image classification and segmentation, and audio processing for speech-to-text and music generation. Additionally, it demonstrates generative AI workflows for text-to-image and text-to-video synthesis, alongside graph-based neural networks and multimodal systems that utilize shared embedding spaces.
Features
- Reference Model Implementations - Provides a comprehensive collection of reference implementations for training and deploying models using the MLX framework.
- Generative AI Workflows - Provides sequences of automated steps for creating images, video, music, and text using diffusion models and transformers.
- Large Language Model Fine-Tuning - Offers code samples for adapting large language models using LoRA and quantization for local hardware.
- Model Fine-Tuning - Implements procedures for adapting pre-trained models to specific datasets using parameter-efficient training.
- Low-Rank Adaptation - Provides reference implementations for parameter-efficient fine-tuning using low-rank adaptation to reduce hardware requirements.
- Language Model Training - Implements transformer-based language models trained on text corpora to predict subsequent tokens.
- Weight Quantization - Includes techniques for compressing model weights into 4-bit or 8-bit precision to reduce memory footprint on local hardware.
- Text Generation - Enables the generation of human-like text by running large language models on local hardware.
- MLX Format Conversions - Provides scripts to transform model weights into formats optimized for the MLX framework on Apple Silicon.
- Local Model Deployment - Provides examples for running and managing large language models on local hardware using quantization.
- Model Weight Conversions - Ships scripts for transforming model weights and formats into MLX-compatible versions with precision quantization.
- Mistral Model Integrations - Provides an optimized implementation for integrating and executing Mistral large language models.
- Multimodal Generation - Implements generative workflows for producing text, images, audio, and video from mixed-modal inputs.
- Distributed Training - Provides reference code for parallelizing model training and computation across multiple devices.
- Data-Parallel Training - Includes examples for distributing training workloads across multiple hardware units by synchronizing gradients.
- Text-to-Video Generators - Synthesizes high-resolution video sequences from textual descriptions using a diffusion transformer.
- Latent Diffusion Models - Implements generative architectures that perform iterative denoising within compressed latent spaces to create visual content.
- Inference Acceleration - Accelerates inference by skipping redundant transformer computations when embeddings remain similar.
- Mixture of Experts - Supports the execution of large-scale mixture-of-experts models for efficient text generation.
- Adapter Fusion - Provides utilities to merge trained low-rank adaptation weights into the base model for standalone deployment.
- Multi-Modal Embedding Models - Implements neural networks that map images and text into a shared vector space for joint retrieval.
- Multimodal AI Systems - Implements systems that combine text, image, and audio data for joint embeddings and generative workflows.
- Multimodal Embeddings - Implements shared vector spaces that map images and text together to enable joint retrieval and multimodal generation.
- Image-to-Video Generation - Transforms static images into animated video sequences using model parameters.
- Multimodal Generation - Enables the production of text descriptions and answers based on combined image and text inputs.
- Audio and Speech Models - Implements speech recognition, neural audio compression, and text-to-music generation using specialized acoustic models.
- Unified Memory Systems - Leverages a shared memory pool between CPU and GPU to eliminate redundant data copying during tensor operations.
