3 repositorios
Capabilities for running multiple language model requests simultaneously to increase total throughput.
Distinct from Concurrent Task Execution: Specifically targets the parallelization of LLM prompt functions rather than general asynchronous task coroutines.
Explore 3 awesome GitHub repositories matching software engineering & architecture · Parallel LLM Execution. Refine with filters or upvote what's useful.
G0DM0D3 is a static web client and multi-model chat gateway designed for AI research, prompt optimization, and red teaming. It provides a unified interface to query numerous AI models in parallel, allowing for the simultaneous evaluation of different prompt variations and sampling parameters to identify the most successful outputs. The project features specialized tooling for probing safety filters and bypassing model constraints through an input perturbation engine that applies text obfuscation and character substitution. It includes a composite scoring system to rank model performance and a
Executes multiple prompt and model combinations simultaneously to identify the most effective response patterns.
BAML is a prompt engineering framework and LLM client generator that defines AI prompts as type-safe functions. It serves as a structured data extraction tool and workflow orchestrator, transforming unstructured model responses into strongly typed objects using a custom schema language and alignment algorithms. The project distinguishes itself by using a compiler to generate language-specific boilerplate code for API communication and output parsing. It features a dedicated environment for designing complex prompt templates with conditional logic and reusable snippets, and employs genetic alg
Runs multiple prompt functions in parallel across different threads or asynchronous tasks to improve throughput.
This project is an LLM research orchestrator and autonomous AI agent framework designed to automate the scientific lifecycle. It functions as an end-to-end research pipeline and model training toolkit, managing everything from initial literature reviews and hypothesis testing to the final drafting of academic papers. The system is distinguished by its ability to convert unstructured academic PDFs into machine-executable knowledge layers, allowing agents to reproduce and extend research findings. It employs a two-loop orchestration architecture and a specialized research engineering skill libr
Runs multiple training configurations simultaneously and logs real-time metrics through a dedicated SDK.