MOSS

MOSS is a conversational AI API server and framework designed to manage stateful multi-turn dialogues via session identifiers for remote interaction. It functions as a tool-augmented language model framework and a quantized inference engine.

The project integrates external plugins, such as search engines and calculators, to provide factual and computed data within model responses. It also includes a supervised fine-tuning toolkit for adapting base language models to specific conversational datasets and behavioral instructions.

The system supports inference optimization through 4-bit and 8-bit weight quantization to reduce GPU memory and computation costs. It further provides capabilities for model API hosting and the deployment of interactive demos via web or command-line interfaces.

Features

Conversational AI APIs - Functions as a web service that manages stateful multi-turn dialogues via session identifiers.

Conversational AI Deployments - Hosts a language model as a network service to manage stateful multi-turn dialogues.

Conversational Session Management - Implements context tracking across multi-turn dialogues using unique session identifiers.

External Tool Integration - Integrates external plugins like search engines and calculators to augment model responses with factual data.

Tool Augmentations - Combines model outputs with external tool data from search engines and calculators for improved accuracy.

LLM Conversational AI Frameworks - Provides a framework coordinating a large language model with external tool plugins for factual augmentation.

Model Serving APIs - Exposes language models as network-accessible services via standard API endpoints for remote interaction.

Conversational Dialogue Systems - Provides a system for generating human-like text across multi-turn dialogues and following behavioral instructions.

Quantized Inference Runtimes - Provides a quantized inference engine that supports 4-bit and 8-bit weight precision to reduce GPU memory and computation costs.

ML Model Hosting - Provides infrastructure for deploying and serving a language model as a dedicated network service.

Inference Optimization - Optimizes inference by using 4-bit and 8-bit quantization to reduce GPU memory and computation.

Weight Quantization - Utilizes 4-bit and 8-bit precision to compress model weights and reduce GPU memory costs.

Supervised Fine-Tuning - Supports training base models on custom conversational datasets to adapt specific behaviors.

Supervised Fine-Tuning Frameworks - Ships a toolkit for aligning pretrained models to specific conversational roles using labeled datasets.

Plugin Execution Engines - Executes external software modules during the inference loop for real-time data retrieval and computation.

Natural Language Processing - Listed in the “Natural Language Processing” section of the FunNLP awesome list.

OpenMOSSMOSS

Features

Star history