# cocktailpeanut/dalai

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/cocktailpeanut-dalai).**

12,920 stars · 1,334 forks · CSS

## Links

- GitHub: https://github.com/cocktailpeanut/dalai
- Homepage: https://cocktailpeanut.github.io/dalai
- awesome-repositories: https://awesome-repositories.com/repository/cocktailpeanut-dalai.md

## Topics

`ai` `llama` `llm`

## Description

The simplest way to run LLaMA on your local machine

## Tags

### Artificial Intelligence & ML

- [Local LLM Execution](https://awesome-repositories.com/f/artificial-intelligence-ml/on-device-models/local-llm-execution.md) — Executes large language models directly on a personal computer without cloud connectivity.
- [Local](https://awesome-repositories.com/f/artificial-intelligence-ml/agent-deployment-servers/llm-inference-servers/local.md) — Starts a web server that exposes a locally installed language model for inference requests. ([source](https://github.com/cocktailpeanut/dalai/blob/main/Dockerfile))
- [Model Installers](https://awesome-repositories.com/f/artificial-intelligence-ml/ai-application-frameworks/local-on-device-ai/ollama-engine-integrations/ollama-model-runners/model-installers.md) — Downloads specific model variants by name from a CDN for local use. ([source](https://github.com/cocktailpeanut/dalai#readme))
- [Model](https://awesome-repositories.com/f/artificial-intelligence-ml/audio-transcription/cli-transcription-tools/http-api-servers/model.md) — Exposes the model runtime as a RESTful HTTP server for remote inference requests and parameter tuning.
- [Local Model Execution](https://awesome-repositories.com/f/artificial-intelligence-ml/local-model-execution.md) — Executes language models directly on a personal computer without requiring cloud connectivity. ([source](https://github.com/cocktailpeanut/dalai/tree/main/cmds))
- [Local Model Integrations](https://awesome-repositories.com/f/artificial-intelligence-ml/local-model-integrations.md) — Enables sending a text prompt to a locally running model and receiving the generated completion in return. ([source](https://github.com/cocktailpeanut/dalai/tree/main/demo))
- [Local Model Runners](https://awesome-repositories.com/f/artificial-intelligence-ml/local-model-runners.md) — Runs large language models on a personal computer without requiring cloud connectivity.
- [LLM API Servers](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-deployment-and-serving/local-and-on-device-inference/local-api-servers/llm-api-servers.md) — Starts an HTTP server exposing a locally running language model for inference requests.
- [Local LLM API Servers](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-deployment-and-serving/local-and-on-device-inference/local-api-servers/local-llm-api-servers.md) — Starts an HTTP server that exposes the locally running language model for interaction through a web interface or API. ([source](https://github.com/cocktailpeanut/dalai/blob/main/package.json))
- [Local Model Query APIs](https://awesome-repositories.com/f/artificial-intelligence-ml/model-api-integrations/local-model-query-apis.md) — Provides an API to send prompts to a locally running model and receive streamed text responses. ([source](https://github.com/cocktailpeanut/dalai#readme))
- [Local LLM Installers](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-deployment-and-serving/local-and-on-device-inference/local-llm-configurations/local-llm-installers.md) — Downloads and configures a large language model on the local machine for offline use. ([source](https://github.com/cocktailpeanut/dalai/blob/main/Dockerfile))

### Part of an Awesome List

- [Local LLM Execution](https://awesome-repositories.com/f/awesome-lists/ai/local-llm-execution.md) — Downloads and runs large language models on a local machine using a single command-line instruction. ([source](https://github.com/cocktailpeanut/dalai#readme))
- [Large Language Model Deployments](https://awesome-repositories.com/f/awesome-lists/ai/local-model-deployment/large-language-model-deployments.md) — Loads and executes large language models on a personal computer using a command-line tool. ([source](https://github.com/cocktailpeanut/dalai/blob/main/.prettierignore))

### Development Tools & Productivity

- [Model Managers](https://awesome-repositories.com/f/development-tools-productivity/command-line-model-inferences/model-managers.md) — Manages language models through terminal commands for local use.
- [Model Installers](https://awesome-repositories.com/f/development-tools-productivity/command-line-model-inferences/model-installers.md) — Downloads and configures model files from a content delivery network using a single terminal command.

### DevOps & Infrastructure

- [LLaMA Runners](https://awesome-repositories.com/f/devops-infrastructure/model-serving/llama-cpp-backend-runners/llama-runners.md) — Executes LLaMA models locally using a simple command-line interface.
- [Docker Container Deployments](https://awesome-repositories.com/f/devops-infrastructure/container-orchestration/container-runtimes/runtime-configuration-interfaces/docker-socket-orchestrators/docker-target-configurators/docker-container-deployments.md) — Packages the entire application and model into a Docker container for isolated, reproducible execution.
- [LLM Docker Images](https://awesome-repositories.com/f/devops-infrastructure/container-orchestration/container-runtimes/runtime-configuration-interfaces/docker-socket-orchestrators/docker-target-configurators/docker-container-deployments/docker-container-execution/repository-built-container-hook-runs/ready-to-run-docker-images/llm-docker-images.md) — Builds and runs a language model inside a Docker container with persistent storage and network access. ([source](https://github.com/cocktailpeanut/dalai/blob/main/docker-compose.yml))
- [LLM Deployments](https://awesome-repositories.com/f/devops-infrastructure/container-orchestration/container-runtimes/runtime-configuration-interfaces/docker-socket-orchestrators/docker-target-configurators/docker-container-deployments/llm-deployments.md) — Deploys and runs language models inside a Docker container for isolated execution.
- [Model Deployments](https://awesome-repositories.com/f/devops-infrastructure/container-orchestration/container-runtimes/runtime-configuration-interfaces/docker-socket-orchestrators/docker-target-configurators/docker-container-deployments/model-deployments.md) — Runs language models inside Docker containers for isolated execution.

### Networking & Communication

- [LLM Servers](https://awesome-repositories.com/f/networking-communication/local-http-servers/llm-servers.md) — Serves a locally running language model through an HTTP API for remote inference.
- [Token Streaming](https://awesome-repositories.com/f/networking-communication/real-time-event-streams/token-streaming.md) — Streams generated text token-by-token over a socket.io connection for real-time browser interaction.

### Programming Languages & Runtimes

- [C++ Inference Runtimes](https://awesome-repositories.com/f/programming-languages-runtimes/c-inference-runtimes.md) — Loads and executes large language models directly in C++ for maximum performance on local hardware.
- [LLM Integrations](https://awesome-repositories.com/f/programming-languages-runtimes/node-js-runtime-integration/llm-integrations.md) — Embeds a language model runtime into a Node.js application for programmatic use.
- [Model Embedding Runtimes](https://awesome-repositories.com/f/programming-languages-runtimes/node-js-runtime-integration/model-embedding-runtimes.md) — Allows integrating the model runtime into an existing Node.js project for programmatic use without a separate server. ([source](https://github.com/cocktailpeanut/dalai/tree/main/docs))

### Web Development

- [Model Streaming Over WebSocket](https://awesome-repositories.com/f/web-development/websockets/rpc-over-websocket/model-streaming-over-websocket.md) — Starts a socket.io server that exposes the model so browsers or other applications can query it remotely. ([source](https://github.com/cocktailpeanut/dalai#readme))