josStorer/RWKV-Runner

Features

• Local Model Runners - Launches RWKV large language models locally with one-click setup and automatic dependency management.
• Local LLM Tools - Runs large language models locally on consumer hardware with one-click setup and automatic dependency management.
• OpenAI-Compatible - Translates OpenAI chat and embedding request schemas into internal model calls for local inference.
• Chat Interfaces - Provides a built-in chat interface with presets, file attachments, and parameter tuning for RWKV models.
• State-Based Inference Engines - Runs the RWKV model as a recurrent neural network using a fixed-size state vector for linear-time inference.
• Gated Linear Recurrent Layers - Runs the RWKV model as a recurrent neural network using a fixed-size state vector for linear-time inference.
• Model Launchers - Launches large language models with a single click, automating dependency installation and configuration.
• OpenAI-Compatible Servers - Ships a local server that exposes an OpenAI-compatible API for chat and embeddings.
• LoRA Fine-Tuning Tools - Fine-tunes RWKV models on custom data using low-rank adaptation with a one-click training interface.
• Multi-Backend GPU Inference Engines - Switches between NVIDIA, AMD, Intel, and other GPU backends to run language model inference.
• LoRA Fine-Tuning Pipelines - Implements a one-click LoRA fine-tuning pipeline for RWKV models on custom data.
• LoRA Training - Adapts RWKV models to custom data using LoRA training with a single click on Windows.
• Model Downloaders - Downloads RWKV model weights and configuration files from remote sources for local use.
• Inference Compute Backends - Switches between CUDA, ROCm, and Vulkan backends to run RWKV models on various graphics hardware.
• Model Format Converters - Provides a built-in command-line tool to convert models between RWKV, GGML, and PyTorch formats.
• RWKV Format Converters - Converts model files between RWKV, GGML, and PyTorch formats using a built-in command-line tool.
• Text Embedding Generators - Generates vector embeddings from text inputs for semantic search and similarity tasks.
• Model Downloaders - Downloads, inspects, and updates RWKV models through a built-in manager with automatic update support.
• GPU Backend Switchers - Switches between GPU backends to run computations on NVIDIA, AMD, Intel, and other graphics cards.
• Unified GPU Backend Abstractions - Wraps CUDA, ROCm, and Vulkan behind a single inference interface with automatic backend switching.
• Preset-Based - Provides a chat UI with configurable system prompts and parameter sets stored as reusable presets.

Istoric stele