Awesome GitHub RepositoriesManaged Infrastructure Deployment

Lightning is a PyTorch training framework and distributed AI training orchestrator designed to decouple core research logic from the engineering boilerplate required for model training. It functions as a deep learning workflow manager that automates the process of pretraining and finetuning models across diverse compute environments. The project distinguishes itself by providing a hardware-agnostic training wrapper, allowing the same model code to execute on CPUs, GPUs, or TPUs without modification. It further manages the scaling of workloads from single devices to multi-node clusters and ser

Hyperframes is an HTML-to-video rendering engine and composition tool that transforms web layouts and CSS into encoded video files. It functions as a headless browser video pipeline and a distributed video rendering framework, allowing users to create seekable animations and programmatic motion designs using HTML, CSS, and JavaScript. The project differentiates itself as an AI agent video orchestrator, enabling the automation of video scripts and compositions through natural language prompts. It supports distributed video encoding by splitting rendering tasks across multiple serverless functi

Prefect is a workflow orchestration platform designed to define, schedule, and monitor complex data pipelines as Python code. It functions as a container-native engine that wraps individual tasks in isolated environments, ensuring consistent dependencies and resource allocation across diverse infrastructure. By utilizing a state-machine-based orchestration model, the system tracks execution progress through discrete transitions and persistent event logs to maintain reliable and observable task processing. The platform distinguishes itself through a decoupled worker-API architecture, which sep

kops is a Kubernetes cluster provisioner and lifecycle manager designed to automate the creation, maintenance, and destruction of production-grade clusters on cloud infrastructure. It functions as a declarative infrastructure manager, synchronizing the live state of a cluster with versioned manifests stored in remote object storage to ensure idempotent operations. The project distinguishes itself by offering comprehensive automation for the entire cluster lifecycle, including high-availability control plane deployment, incremental rolling updates, and automated version upgrades. It also serve

Terraformer is a reverse engineering tool and infrastructure-to-code generator that transforms existing live cloud resources into declarative configuration files and state manifests. It functions as a cloud infrastructure exporter, allowing users to extract resource metadata from cloud providers to create reproducible infrastructure deployments. The tool specializes in reverse engineering by querying cloud provider APIs to map active resource configurations and translate them into Terraform resource blocks. It supports infrastructure state recovery by reconstructing state files from live envi

Unstructured is an enterprise-grade data orchestration engine designed to transform raw, unstructured files into structured, machine-readable formats. It functions as a comprehensive platform for document ingestion, partitioning, and enrichment, specifically engineered to prepare complex data for retrieval-augmented generation and agentic AI workflows. The platform distinguishes itself through its sophisticated document processing strategies, which combine rule-based extraction with vision-language models to handle diverse file layouts, tables, and images. It provides a modular architecture t

Redpanda is a distributed event streaming engine designed to serve as a high-performance, drop-in replacement for existing event-driven architectures. It provides a foundation for building and scaling applications that require reliable data movement, analytical querying, and strict operational compliance across both cloud and self-managed environments. The platform distinguishes itself through a shared-nothing architecture that utilizes thread-per-core execution and a non-blocking asynchronous input/output engine to maximize throughput. It maintains data consistency through a consensus-based

Zappa is a deployment tool and orchestrator designed to package and deploy Python web applications to AWS Lambda. It functions as a wrapper and framework that adapts standard Python applications to work with serverless infrastructure and API Gateway events. The project enables the hosting of Python web frameworks on serverless environments without requiring the application to be rewritten. It automates the process of pushing code from local environments to the cloud and manages the lifecycle of these deployments to remove manual server management. Its capabilities include automated infrastru

Promptflow is a development framework and orchestrator for building applications powered by large language models. It functions as a suite of tools for designing, orchestrating, and deploying AI workflows by linking prompts, custom Python code, and language models into executable sequences. The project is distinguished by a visual AI workflow designer that allows for the creation of directed acyclic graphs of logic nodes. It provides a dedicated prompt engineering environment for versioning and comparing templates, alongside stateful execution tracing to record function calls and variable val

Azure Docs is the official technical documentation repository for Microsoft Azure, the cloud computing platform. It provides comprehensive guidance on the full spectrum of Azure services, covering everything from core infrastructure components like virtual machines, Kubernetes clusters, and serverless computing to platform services for AI, machine learning, data analytics, and storage. The documentation details how to provision, manage, and govern cloud resources at scale, including policy enforcement, identity management, and cost optimization. The documentation distinguishes Azure through i

AutoMQ is a cloud-native streaming platform and Kafka-compatible message broker. It implements the Kafka protocol to provide integration with existing clients and ecosystems while functioning as a message queue that persists data directly to cloud object storage. The system decouples compute from storage, allowing processing power and storage capacity to scale independently. It utilizes a shared-log architecture and object-storage-based persistence to remove dependencies on local disks, which reduces operational costs and eliminates manual disk management. The platform includes mechanisms fo

This is an infrastructure as code tool and serverless deployment orchestrator that provides a shorthand syntax for defining serverless infrastructure. It functions as a framework for transforming concise resource declarations into full AWS CloudFormation templates to automate the provisioning of cloud functions, APIs, and databases. The project distinguishes itself by using a macro-based transformation system to expand simplified resource types into detailed infrastructure components. It includes an automated permission mapping system that translates high-level resource interaction intents in

Leantime est une plateforme de gestion de projet open source conçue pour suivre les tâches, les jalons et les flux de travail à l'aide de tableaux Kanban et de diagrammes de Gantt. Il fonctionne comme un outil de planification stratégique qui lie les tâches opérationnelles quotidiennes aux objectifs commerciaux de haut niveau et aux objectifs à long terme, tout en servant également de système de suivi du temps et de facturation pour gérer les heures de travail et les estimations des clients. La plateforme se différencie par un assistant de projet intégré alimenté par l'IA qui décompose les tâches complexes, génère des rapports d'état et fournit des informations sur le projet. Elle distingue également son offre avec une base de connaissances auto-hébergée pour centraliser les wikis de projet et les notes internes, ainsi que des outils d'alignement stratégique pour mapper les résultats aux activités via une théorie du changement. Le système couvre un large éventail de capacités, y compris la collaboration d'équipe agile, la gestion d'identité d'entreprise via LDAP et OIDC, et la coordination de portefeuille de projets entre différentes unités commerciales. Il fournit également des outils de brainstorming visuel comme des cartes mentales et des wireframes, un contrôle d'accès basé sur les rôles pour la collaboration avec des clients externes, et un modèle d'extensibilité basé sur une place de marché de plugins. Le logiciel prend en charge le stockage d'objets compatible S3 pour les fichiers de projet et fournit des options d'infrastructure de projet gérée.

RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen

LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters

Apache Beam is a distributed data pipeline framework and unified data processing model designed to handle both bounded batch data and unbounded real-time streams. It provides a system for building scalable, data-parallel workflows that operate across compute clusters using a single programming model. The framework utilizes a cross-runner pipeline abstraction that decouples the data processing logic from the underlying execution backend, allowing the same pipeline to run on different distributed compute engines. It supports multi-language pipeline development by translating high-level code fro

Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and monitor large language model applications. It serves as a prompt management system for versioning and testing templates, and as a self-hosted AI operations infrastructure for managing telemetry and experiments. The platform differentiates itself through a specialized embedding visualization tool used to detect data drift and optimize vector search. It provides a comprehensive evaluation suite that utilizes judge-based evaluators and ground-truth datasets to score model outputs, and

Travis CI is a continuous integration platform and CI/CD pipeline orchestrator that automates the testing and building of code changes from version control systems. It functions as a multi-language test runner and build infrastructure manager, ensuring software quality through automated testing across various programming languages and runtimes. The platform is distinguished by its use of virtual-machine-based isolation for reproducible environments and a configuration-driven approach to pipeline generation. It supports complex testing strategies through parallel matrix execution, allowing job

Moto is a cloud service mockery framework and API mock server that simulates AWS infrastructure locally. It allows developers to test cloud-dependent code and verify infrastructure-as-code templates without deploying real resources or incurring costs. The project functions as an SDK interceptor that can patch existing service clients to redirect requests to a local mock environment. It can also be run as a standalone HTTP server, enabling any programming language to interact with the simulated endpoints. The framework covers a vast array of simulated capabilities, including data storage, com

LiveTalking is an interactive talking head engine and AI avatar management platform designed to synchronize synthetic speech with facial movements. It functions as a real-time orchestrator that connects large language models and text-to-speech services to neural-rendered digital humans. The project distinguishes itself through low-latency streaming capabilities and the ability to handle real-time conversational interruptions. It supports advanced audio-visual customization, including human voice cloning and the ability to drive avatar expressions using real-time webcam data. The platform cov