Open-source alternatives to Serving

Kubeless is a Kubernetes-native serverless framework that deploys and runs stateless functions as custom resources managed by an in-cluster controller. It functions as a Function-as-a-Service platform, launching function runtime pods on demand and scaling them to zero when idle to optimise resource usage. Functions are invoked automatically through HTTP requests or a publish-subscribe messaging bus, enabling event-driven execution for workloads and microservices. The platform supports running functions written in Golang, Python, Node.js, Ruby, PHP, .NET, and Ballerina, with the ability to add

KServe is an open platform for deploying and serving generative and predictive AI models on Kubernetes. It defines inference services as custom resources with declarative YAML specifications, enabling a Kubernetes-native approach to model deployment and lifecycle management. The platform leverages Knative-based serverless scaling for automatic scale-to-zero and revision management, and supports a pluggable serving runtime architecture that maps model formats to containerized execution environments. KServe distinguishes itself through model-aware autoscaling that scales replicas based on token

The Event Gateway is a unified routing layer designed to connect HTTP requests and cloud events to serverless functions across multiple providers and infrastructure environments. It functions as a centralized platform for managing event-driven architectures, enabling developers to register, map, and trigger remote functions in response to incoming web traffic or asynchronous system events. The platform distinguishes itself through a plugin-based middleware pipeline that allows for custom logic execution during the request lifecycle, enabling developers to intercept, validate, or transform eve

This project is a Kubernetes serverless framework and OCI container function platform. It provides a system for deploying event-driven functions and microservices as compatible container images onto a Kubernetes cluster. The platform includes an event-driven function orchestrator that triggers executions via HTTP requests or message streams. It features an auto-scaling function manager that adjusts the number of active instances based on real-time demand and scales down to zero during inactivity. A background queuing system is included to process asynchronous tasks and maintain application re

Inngest is a durable execution framework and event-driven automation engine designed to orchestrate background workflows. It enables developers to build resilient, stateful processes by memoizing function steps, ensuring that long-running tasks can automatically resume from the last successful operation after failures, timeouts, or infrastructure restarts. The platform distinguishes itself through its event-driven architecture, which uses a schema-validated bus to trigger functions and coordinate complex, multi-step logic. It employs an onion-model middleware approach for cross-cutting concer

The container native, cloud agnostic serverless platform.

Fission is a function-as-a-service platform and serverless framework for Kubernetes. It manages the lifecycle and execution of code snippets as serverless functions, providing an orchestrator that triggers these functions based on HTTP requests, message queues, or scheduled events. The platform features a cold-start optimized runtime that utilizes warm container pools and dynamic loaders to achieve millisecond execution. It includes a native autoscaler to adjust the number of function instances based on real-time traffic demand and supports canary release testing to split incoming traffic bet

The Functions Framework for Python is a library designed for building and deploying portable serverless functions. It enables developers to write modular code that executes in response to incoming HTTP requests or cloud events, abstracting away the need to manage underlying server infrastructure or complex request handling logic. The framework distinguishes itself by providing a consistent runtime environment that packages code into portable container images, ensuring identical behavior across diverse cloud-native platforms. It utilizes decorator-based registration to define function entry po

KEDA is a Kubernetes event-driven autoscaler and cloud event scaling engine. It functions as a custom metrics provider that monitors external event sources—including message brokers, databases, and cloud metrics—to dynamically adjust the replica counts of containerized workloads. The project is distinguished by its scale-to-zero workflow, which reduces workloads to zero replicas during inactivity and automatically restarts them when new events are detected. It operates as a multi-cloud event trigger system, using a pluggable scaler interface to integrate with a wide array of third-party servi

CloudEvents is an open specification for describing event data in a common format across cloud platforms and services. It defines a standard structure and set of metadata attributes for events, enabling interoperability across different systems so producers and consumers can exchange events without custom translation. The specification provides a protocol-agnostic serialization framework that maps CloudEvents attributes and payloads to multiple serialization formats including JSON, Avro, and Protobuf, and defines transport bindings for mapping events onto protocols like HTTP, AMQP, Kafka, MQTT

The Serverless Framework is a declarative infrastructure-as-code tool designed to automate the deployment, scaling, and lifecycle management of cloud-native applications. It provides a unified command-line interface that translates high-level configuration files into provider-specific resource templates, enabling developers to orchestrate complex architectures, event-driven functions, and cloud resources within a single project structure. What distinguishes this framework is its focus on developer experience and multi-environment parity. It supports local function invocation and event proxyin

Easegress is a cloud-native traffic orchestration platform that functions as a multi-protocol API gateway and service mesh proxy. It routes, transforms, and orchestrates HTTP and MQTT traffic through configurable filter chains, providing high availability, security controls, and resilience for backend services. The platform integrates with service discovery systems like Kubernetes, Consul, and Eureka to dynamically route traffic to discovered services with automatic failover and health monitoring. The system distinguishes itself through a pipeline-based filter chain architecture where request

This project provides a comprehensive architectural blueprint and implementation set for building a platform-as-a-service on Kubernetes. It serves as a technical resource for deploying container orchestration environments, managing the full software development lifecycle, and integrating a complete DevOps toolchain. The implementation emphasizes automated software delivery through the integration of build and delivery pipelines, private container registries, and distributed configuration systems. It enables the decoupling of application settings from images via a centralized configuration man

Flagger is a Kubernetes operator designed to automate the lifecycle of application deployments through progressive delivery. It functions as a controller that monitors custom resource definitions to orchestrate complex release strategies, including canary, blue/green, and A/B testing. By continuously reconciling the desired cluster state with the actual environment, it ensures that deployments adhere to defined specifications while managing the underlying infrastructure required for traffic routing. The project distinguishes itself through a sophisticated metric-driven analysis loop that eval

RocketMQ is a distributed messaging and streaming platform designed for building event-driven applications. It serves as middleware to decouple services using publish-subscribe and request-reply patterns, and functions as a transactional messaging system that ensures atomicity by linking message delivery to local transaction outcomes. The platform includes specialized capabilities as a Kubernetes-native message broker for container orchestration environments and an MQTT broker for ingesting event data from mobile applications and hardware terminals. The system covers high-throughput data str

Dapr is a distributed application runtime that provides a sidecar-based infrastructure layer for building resilient microservices and event-driven applications. By utilizing a sidecar proxy pattern, it abstracts complex infrastructure tasks into standardized, network-accessible APIs, allowing developers to focus on application logic while the runtime handles service discovery, state management, and secure communication. The platform distinguishes itself through a pluggable component architecture and language-agnostic design, enabling services written in any programming language to interact wi

Text Embeddings Inference is a high-performance inference server designed to host text embedding and sequence classification models as scalable API endpoints. It provides a vector embedding API to convert text into dense representations and a cross-encoder reranking server for scoring the relevance of document sequences against a query. The project features a GPU-accelerated inference engine that utilizes dynamic batching and specialized kernels to maximize throughput. It offers a high-performance binary interface via gRPC as an alternative to standard HTTP to reduce network latency and seria

Boto3 is the AWS SDK for Python, providing a programmatic interface for managing and automating AWS cloud infrastructure and services. It serves as a cloud management API client and resource manager for provisioning, configuring, and scaling virtual servers, databases, and storage. The library enables the implementation of infrastructure-as-code through declarative templates and scripts, allowing for the deployment of identical resource stacks across multiple accounts and geographic regions. It also provides a framework for coordinating distributed workflows, serverless functions, and contain

jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU hardware. Its primary purpose is to enable real-time computer vision and AI inference at the edge with low latency and high throughput. The project distinguishes itself through high-performance streaming analytics and the ability to execute concurrent AI pipelines on auto-grade silicon. It provides specialized support for multi-sensor stream processing, utilizing zero-copy data transport to load camera frames directly into GPU memory. The codebase covers a broad surface of capabiliti

Mercure is an HTTP-based message broker and real-time pub/sub server that routes asynchronous data updates to authorized subscribers. It functions as a centralized hub that receives data via standard HTTP requests and broadcasts it to web clients using the Server-Sent Events protocol. The project distinguishes itself by replacing WebSockets with a unidirectional HTTP connection, leveraging HTTP/2 and HTTP/3 for multiplexed, low-latency delivery. It implements a secure access control layer using JSON Web Tokens and JSON Web Signatures to validate the identity of publishers and subscribers, and

Kill Bill is a subscription billing platform and usage-based billing engine designed to manage recurring invoicing and automated payment collection. It functions as a multi-tenant billing infrastructure, providing isolated environments for different organizational entities through a dedicated API. The system is distinguished by a plugin-based extension framework that allows for the integration of third-party payment gateways and custom business logic. It includes a payment gateway orchestrator to handle transactions and refunds, as well as a revenue recognition system to allocate contract rev

This project is a memory-safe implementation of the QUIC transport protocol and HTTP/3, designed for high-throughput and efficient network communication. It provides a comprehensive toolkit for building secure, low-latency network applications by managing the full lifecycle of transport connections, including protocol negotiation, stream data exchange, and connection state management. The library distinguishes itself through a focus on performance and protocol integrity. It utilizes a formal state machine to enforce strict adherence to transport rules and employs zero-copy buffer management t

Pangolin is a zero-trust remote access platform designed to provide secure, identity-aware connectivity to private network resources. It functions as a cloud-native network controller that orchestrates encrypted tunnels, traffic routing, and access policies across distributed environments. By leveraging WireGuard for secure data transport, the platform enables authenticated access to internal web applications, terminal sessions, and remote desktops without exposing services to the public internet. The platform distinguishes itself through a declarative infrastructure model that synchronizes n

The AWS Cloud Development Kit is an infrastructure-as-code framework that enables developers to define and provision cloud resources using familiar programming languages. By utilizing construct-based synthesis, it translates high-level, object-oriented code into declarative templates, allowing for the automated management of complex cloud environments through a centralized, code-driven control plane. The framework distinguishes itself through its ability to model infrastructure as a dependency-aware resource graph, ensuring that components are provisioned and updated in the correct order. It

Calico is a cloud-native networking and security solution designed to connect containerized workloads across virtual machines, bare metal, and multi-cloud environments. It provides a routing solution based on the Border Gateway Protocol to manage cluster traffic and implement the Container Network Interface for pod connectivity and IP address management. The project distinguishes itself through a security layer that enforces network policies based on identities and labels rather than static addresses. It includes a policy engine for controlling traffic flow, a cluster network encryptor for se

This project serves as a comprehensive knowledge base and directory for the container orchestration ecosystem. It functions as a curated index of tools, tutorials, and technical documentation designed to assist developers and operators in managing, scaling, and maintaining containerized applications. The repository operates through community-driven content curation, maintaining a structured collection of resources within a version-controlled environment. It distinguishes itself by providing a vetted index that helps users locate specific utilities for infrastructure management and DevOps task

CAP is a .NET distributed transaction framework and event bus designed to manage asynchronous communication in microservices. It implements the outbox pattern to ensure eventual consistency and reliable message delivery by persisting messages in local database tables until transactions commit. The framework includes a distributed message monitor and web dashboard for tracking the status of sent and received messages. It provides tools for event traffic visualization, distributed request tracing, and the ability to manually trigger retries for failed delivery attempts. The system supports var