Mozilla-Ocho/llamafile

Features

• Model-Runtime Bundlers - Combines language models and their execution logic into single, portable executable files.
• AI Model Distribution - Enables the distribution of language models as single executable files for immediate local execution.
• Local Model Execution - Enables local execution of large language models via a single-file, zero-installation distribution.
• LLM Runtimes - Provides a self-contained environment for running large language models across different operating systems.
• Local LLM Execution - Allows running large language models on local hardware without managing complex environment dependencies.
• Standalone LLM Executables - Provides a single-file distribution format for zero-installation local execution of large language models.
• Self-Extracting Binaries - Bundles model weights and runtimes into a single binary that unpacks itself upon execution.
• Standalone Model Binaries - Deploys machine learning models into a single file that bundles both the runtime and weights.
• Model Packaging - Packages model weights and execution logic into a single portable file for simplified distribution.
• Local Speech-to-Text - Converts audio to text using a standalone local machine learning executable.
• C-Based Engines - Integrates a lightweight C-based inference engine for minimal runtime overhead.
• Memory-Mapped Weight Loaders - Maps model weight files directly from the executable into memory to optimize RAM usage.
• Speech to Text Transcription - Provides standalone capabilities to transcribe and translate audio files into text.
• Content-Addressed File Systems - Employs a content-addressed storage layer to manage large model files and prevent data redundancy.
• Static Binary Compilers - Statically links all required libraries into the executable to ensure cross-platform portability.
• Model Package Distributions - Packages model weights and runtimes into a single file for simplified cross-platform distribution.
• Local HTTP Servers - Embeds a lightweight local HTTP server to provide a standardized API for model interaction.
• Local LLM Tools - Packages LLMs into single executable files.

Star history