# cortexlabs/cortex **Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/cortexlabs-cortex).** 8,013 stars · 595 forks · Go · Apache-2.0 ## Links - GitHub: https://github.com/cortexlabs/cortex - Homepage: https://cortexlabs.com/ - awesome-repositories: https://awesome-repositories.com/repository/cortexlabs-cortex.md ## Topics `infrastructure` `machine-learning` ## Description Cortex is a Kubernetes-based machine learning infrastructure platform designed for deploying, scaling, and managing models and workloads. It functions as a serverless inference engine and GPU cluster orchestrator, providing the tools necessary to execute real-time, asynchronous, and batch model predictions. The platform utilizes declarative infrastructure-as-code for provisioning model clusters and environments. It optimizes operational costs by elastically scaling CPU and GPU resources through the use of spot instances. The system covers a broad set of operational capabilities, including workload orchestration, private cloud network isolation with integrated identity management, and observability pipelines that stream logs and performance metrics to external monitoring tools. ## Tags ### Artificial Intelligence & ML - [Production Serving Infrastructure](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-deployment-and-serving/deployment-pipelines-and-endpoints/model-deployment-pipelines/production-serving-infrastructure.md) — Deploys and serves machine learning models in production environments with scalable infrastructure and automated settings. - [Serverless Inference Engines](https://awesome-repositories.com/f/artificial-intelligence-ml/serverless-inference-engines.md) — Provides a serverless inference engine that automatically scales real-time, asynchronous, and batch model predictions. - [GPU Resource Scaling](https://awesome-repositories.com/f/artificial-intelligence-ml/gpu-resource-scaling.md) — Dynamically adjusts GPU compute capacity using spot instances to balance performance and operational costs. - [Private AI Deployments](https://awesome-repositories.com/f/artificial-intelligence-ml/private-ai-deployments.md) — Deploys machine learning workloads on private infrastructure to ensure data security and access control. ([source](https://github.com/cortexlabs/cortex#readme)) ### DevOps & Infrastructure - [GPU Resource Orchestrators](https://awesome-repositories.com/f/devops-infrastructure/gpu-resource-orchestrators.md) — Elastically provisions and optimizes CPU and GPU resources using spot instances for AI workloads. - [Kubernetes ML Platforms](https://awesome-repositories.com/f/devops-infrastructure/kubernetes-ml-platforms.md) — Provides a production platform for deploying, scaling, and managing machine learning models and workloads on Kubernetes. - [ML Infrastructure Managers](https://awesome-repositories.com/f/devops-infrastructure/ml-infrastructure-managers.md) — Automates the provisioning and scaling of CPU and GPU compute clusters for large-scale ML workloads. - [ML Orchestration Deployments](https://awesome-repositories.com/f/devops-infrastructure/ml-orchestration-deployments.md) — Orchestrates the deployment and scaling of machine learning models across production infrastructure to handle traffic loads. ([source](https://github.com/cortexlabs/cortex/blob/master/.gitbook.yaml)) - [Model Inference Clusters](https://awesome-repositories.com/f/devops-infrastructure/model-inference-clusters.md) — Provisions specialized infrastructure and environment settings specifically for serving machine learning models. ([source](https://github.com/cortexlabs/cortex/blob/master/Makefile)) - [Serverless Inference Engines](https://awesome-repositories.com/f/devops-infrastructure/serverless-function-management/model-inference-functions/serverless-inference-engines.md) — Executes real-time or batch model predictions that scale automatically based on request volume or queue length. - [Workload Orchestration](https://awesome-repositories.com/f/devops-infrastructure/workload-orchestration.md) — Orchestrates real-time and batch processes that scale automatically based on request volume or queue length. - [Asynchronous Task Processing](https://awesome-repositories.com/f/devops-infrastructure/automation-orchestration/task-execution-frameworks/task-job-management/task-queues/asynchronous-task-processing.md) — Provides a queued system for executing non-real-time machine learning workloads through background workers. - [Cloud Infrastructure Cost Optimization](https://awesome-repositories.com/f/devops-infrastructure/cloud-infrastructure-cost-optimization.md) — Reduces operational expenses through the use of spot instances and elastic compute scaling. - [Infrastructure Provisioning Tools](https://awesome-repositories.com/f/devops-infrastructure/cloud-infrastructure/infrastructure-provisioning-management/infrastructure-provisioning-tools.md) — Automates the creation of model clusters using declarative infrastructure-as-code configurations. ([source](https://github.com/cortexlabs/cortex#readme)) - [Virtual Private Clouds](https://awesome-repositories.com/f/devops-infrastructure/cloud-infrastructure/networking-connectivity/virtual-private-clouds.md) — Runs ML workloads within isolated virtual private clouds with integrated identity management for secure access. - [Compute Instance Scaling](https://awesome-repositories.com/f/devops-infrastructure/cluster-node-management/capacity-scaling/compute-instance-scaling.md) — Elastically scales CPU and GPU compute instances using spot instances to reduce operational expenses. ([source](https://github.com/cortexlabs/cortex#readme)) - [Declarative Infrastructure Tools](https://awesome-repositories.com/f/devops-infrastructure/declarative-infrastructure-tools.md) — Uses infrastructure-as-code and configuration templates to provision machine learning environments and clusters. ### Security & Cryptography - [Private Network Security](https://awesome-repositories.com/f/security-cryptography/private-network-security.md) — Runs workloads within isolated virtual private clouds with integrated identity management for secure access control. ### System Administration & Monitoring - [Observability Pipelines](https://awesome-repositories.com/f/system-administration-monitoring/observability-pipelines.md) — The project tracks system behavior and errors by streaming metrics and logs to external monitoring tools or dashboards. ([source](https://github.com/cortexlabs/cortex/blob/master/README.md)) ### Part of an Awesome List - [General Machine Learning](https://awesome-repositories.com/f/awesome-lists/ai/general-machine-learning.md) — Platform for deploying ML models in production.