feast-dev/feast

Features

• Feature Stores - Configures how the feature store runs on infrastructure through a declarative configuration file at the repository root.
• Batch and Streaming Transformations - Defines, computes, and transforms features from both batch and streaming data sources for ML pipelines.
• Feature Execution DAGs - Feast automatically resolves and executes graphs of dependent feature views during materialization or historical retrieval.
• Batch Feature Ingestion - Loads batch feature data into the online store through materialization for low-latency serving.
• Composable Batch Pipelines - Feast defines features from batch data sources or other feature views, supporting transformations, aggregations, and recursive chaining.
• CI Feature Definition Application - Applies version-controlled feature definitions from a CLI to update infrastructure and persist them in a registry.
• Component Orchestration - Coordinates offline store, online store, and compute infrastructure within a target cloud environment to work together seamlessly.
• Store Backend Configurations - Wires Postgres, Redis, DuckDB, or file-based stores for offline, online, and registry storage.
• Entity-Key Collocated Storage - Feast stores all features for the same entity in a single document to reduce the number of queries needed during retrieval.
• Feature Registries - Defines features, entities, and data sources in a version-controlled repository and stores them in a shared registry.
• Declarative Definitions - Stores all feature definitions as protobuf objects in a central registry, synchronized from Python files via a CLI apply command.
• Feature Repository Definitions - Defines core ML feature objects like entities, feature views, and data sources in Python files.
• Version-Controlled Feature Definitions - Feast tracks and manages feature schemas and transformations over time, allowing rollback and audit of feature changes.
• Feature View Catalogs - Feast stores all feature definitions and their metadata in a central catalog for discovery and collaboration.
• Historical Feature Retrieval - Builds point-in-time correct training datasets from historical feature data for model development.
• Offline Store Integrations - Connects to various offline storage backends for historical feature retrieval and batch scoring.
• Online Feature Retrieval - Connects directly to the online store from a Python service to fetch feature values for real-time model inference.
• On-Demand Retrieval Transformations - Feast applies user-defined transformations to stored features and request-time input data as a post-processing step during retrieval.
• Online Feature Writes - Feast writes computed feature values to the online store for later retrieval.
• Online Store Integrations - Connects to various online storage backends for low-latency feature serving during real-time inference.
• User-Defined Transformations - Feast supports user-defined SQL queries or UDFs to perform custom feature transformations during materialization or retrieval.
• Training Data Pipelines - Ingests and serves feature data for model training pipelines, enabling reproducible and consistent feature sets.
• Feature Store - Feast constructs training datasets from historical time-series feature values stored in data sources.
• Model Inference and Serving - Serves pre-computed feature values at low latency from online stores to power real-time model predictions.
• On-Demand Feature Transformations - Applies Python-based feature transformations at retrieval time, combining precomputed features with request-time data for flexible serving.
• Historical Feature Joins - Combines entity data with feature values from an offline store to produce training datasets for model development.
• Access Control and RBAC - Enforces access control on feature serving endpoints using Kubernetes service account tokens and TokenAccessReview.
• Feature Store Permissions - Detects authentication tokens from client requests and applies permission policies to restrict access to feature services.
• Kubernetes RBAC Validations - Validates every API call against ClusterRole-based permissions using ServiceAccount tokens to control who can read or write features.
• Feature Groupings - Defines named collections of features from one or more feature views to simplify retrieval for a specific model version.
• ML Feature Services - Defines logical groups of features from multiple feature views for use by a specific model version.
• Temporal Join Alignment - Implements point-in-time correct joins that align historical feature values with entity timestamps to prevent data leakage.
• Point-in-Time Training Joins - Joins entity labels with historical feature values at specified timestamps to produce point-in-time correct training datasets.
• Temporal Leakage Prevention - Generates point-in-time correct feature sets so future values do not leak into training data.
• Feature Materialization Pipelines - Reads batch features from offline stores and writes them to online stores using pluggable distributed compute engines like Spark or Ray.
• Batch Feature Materialization - Computes and writes the latest batch feature values into an online store for low-latency serving.
• Feature View Execution - Executes feature view logic including transformations, aggregations, and UDFs to generate features.
• Incremental Materializations - Provides incremental materialization to keep online features fresh without full recomputation.
• Data Ingestion Sources - Ingests feature data from a wide range of batch and streaming sources including cloud warehouses and databases.
• Distributed Computing - Feast executes feature computation DAGs across clusters, automatically distributing work across available workers for scalable processing.
• Request-Time Data Sources - Feast supplies input columns alongside entity rows or entity DataFrames so transformations can incorporate values provided at query time.
• Entity-Key Collocated Stores - Stores all features for a single entity in one document or hash to reduce online reads to a single lookup per request.
• Feature Schemas - Declares typed fields for features using a unified type system that mediates between different storage backends.
• Feature Views - Groups related time-series features from a data source into a logical collection for training and serving.
• Execution Plans - Resolves feature view definitions into execution plans and runs transformations, aggregations, joins, and filters.
• Large-Scale Feature Transformations - Feast executes feature computation DAGs across a cluster, automatically scaling workers and managing resources for large-scale processing.
• Pluggable Storage Backends - Provides a pluggable abstraction layer that allows swapping offline and online storage backends without changing retrieval code.
• Real-Time Feature Computation - Pushes data from source systems into the online store for low-latency feature serving.
• Real-Time Prediction Serving - Serves real-time predictions by retrieving features from online stores and scoring them with pre-trained models.
• Storage Abstractions - Feast provides a single data access layer that decouples feature storage from retrieval, keeping models portable across environments.
• Stream Feature Views - Defines feature views that process streaming data from sources like Kinesis for real-time feature computation.
• Stream Feature Ingestion - Pushes feature data from streaming services directly into the feature store via a push API.
• Streaming Feature Definitions - Feast defines features that are continuously updated from a streaming data source for real-time machine learning.
• Stream Ingestion - Consumes data from stream sources and loads it into online and offline stores for feature serving.
• Stream-to-Online Store Ingestion - Writes feature values from a Kinesis stream to the online store for low-latency serving.
• Feature Server Deployments - Runs feature servers as Kubernetes deployments for online feature retrieval in production.
• Declarative Infrastructure Synchronizers - Synchronizes feature repository definitions to the registry and cloud infrastructure via a declarative apply command.
• Feature Version Associations - Feast associates a named set of features with a model version so training and inference always use the same feature schema.
• Feature Serving Protocols - Exposes the same feature values through REST, gRPC, and MCP protocols simultaneously for diverse client ecosystems.
• OpenID Connect Token Validations - Feast validates bearer tokens from an external OpenID Connect provider on every API call.
• OIDC Identity Integrations - Validates JWT tokens against a configurable OIDC provider to authenticate feature store API requests.
• Role-Based Access Control - Assigns permissions for operations and resources to users or groups, enforcing authorization decisions based on roles.
• Access Control - Applies Kubernetes RBAC roles or OIDC authentication and TLS to secure all feature store service endpoints.
• Feature Access Abstractions - Abstracts feature storage across offline and online backends, enabling portable and environment-independent ML pipelines.
• Background Cache Refresh - Periodically re-downloads registry metadata in a background thread so serving requests never block on cache updates.
• Document Embedding Pipelines - Provides a single class for chunking, embedding, and writing documents to the online vector store.
• Custom Compute Engine Interfaces - Adds a custom compute engine to run batch materialization jobs on a non-default processing platform.
• Retrieval-Augmented Generation - Stores and queries embeddings for similarity search and LLM context augmentation in RAG workflows.
• Custom Provider Interfaces - Adds a custom provider to override default deployment and orchestration logic for the feature store.
• Spark Feature Loaders - Reads feature data from Spark tables, queries, or Parquet files for offline training or online inference.
• RAG Document Retrieval - Feast queries stored embeddings to find relevant document passages for retrieval-augmented generation.
• CLI Feature Management - Views and updates feature definitions and metadata from the command line.
• Custom Data Sources - Ships a plugin interface for creating custom data source subclasses that serialize connection info for feature views.
• Custom Store Backends - Provides a plugin architecture and guides for adding custom offline and online store backends.
• Embedding Stores - Stores vector embeddings alongside traditional features and retrieves them via a single API call for GenAI applications.
• Join Key Aliases - Feast maps columns in request data to different column names in the source table, enabling reuse of a feature view with multiple entity aliases.
• Environment Isolation - Namespaces feature store resources per project to keep development, staging, and production environments separate.
• Environment Separation - Separates staging and production environments for feature definitions, allowing safe testing before promoting changes.
• Environment-Specific Registries - Configures separate registries per environment to track infrastructure changes independently for staging and production.
• Remote Registry Connections - Pushes or pulls registry state to a remote store so multiple deployments share the same feature definitions.
• Safe Object Applications - Feast adds or updates only the specified objects in the registry, leaving all existing objects untouched for safe team contributions.
• SQL-Backed Registries - Stores feature registry metadata in a SQL database for concurrent transactional updates without data loss.
• Startup Registrations - Runs feast apply on startup to sync all feature views with the registry automatically.
• Topology Management - Uses local or remote registries, cross-namespace references, and TLS to share a single registry across services.
• Entityless Retrievals - Fetches historical features by specifying a time window instead of an entity dataframe for time-series models.
• Lazy Retrieval Jobs - Provides deferred execution handles for historical feature queries, returning results as DataFrames or Arrow tables.
• MsSQL Feature Queries - Retrieves point-in-time correct feature values from a Microsoft SQL Server or Azure Synapse SQL source for training datasets.
• Ray Dataset Exports - Returns historical feature data directly as a Ray Dataset for distributed processing and ML pipelines.
• Redshift Feature Queries - Retrieves feature data from Redshift for batch training and historical backfills.
• Remote Feature Retrieval - Fetches online features by sending requests to a remote feature server over the network.
• Validated Historical Retrievals - Feast validates a newly retrieved historical feature dataset against a reference profile to detect data drift or quality regressions.
• HTTP Feature Serving - Exposes an HTTP endpoint that accepts JSON requests and returns feature values from the online store for any language.
• TLS-Encrypted Feature Serving - Accepts encrypted HTTPS connections to secure feature data transmitted between clients and the server.
• Kubernetes Provisioners - Creates a feature repository inside pods by cloning a Git repo or scaffolding a new project on Kubernetes.
• Latest Feature Value Retrieval - Returns the most recent feature observation per entity from the offline store using server-side aggregation.
• Logged Feature Persistence - Feast persists logged feature data to a remote offline store for later retrieval and analysis.
• Multi-Backend Feature Routings - Routes feature operations to different store backends based on tags or source types for mixed latency and compliance requirements.
• DataFrame Persistence to Offline Stores - Persists dataframes to the offline store, primarily for push-based feature ingestion.
• Multi-Offline Store Routings - Routes feature retrieval and materialization to different offline store backends based on each FeatureView's batch source.
• Concurrent Retrievals - Handles thousands of concurrent feature retrieval requests using native async I/O without blocking.
• Entity-Key Collocated Retrievals - Retrieves all features for an entity in a single read by storing them as a pre-computed blob.
• Online Store Infrastructure Management - Automatically creates and tears down database tables and indices when applying or removing feature views.
• Push-Based Online Store Ingestion - Pushes feature values from data producers to the online store for low-latency retrieval during real-time inference.
• Parallel Feature Materialization - Materializes each feature view in a separate job or distributes the work across a cluster.
• Precomputed Feature Transformations - Computes feature values outside of the feature store or inside the feature server before writing them to the online store.
• Flink UDF Transformations - Feast applies user-defined functions that receive and return PyFlink table objects when a BatchFeatureView transformation uses Flink mode.
• Push-Based Feature Ingestion - Accepts feature data via a REST endpoint and writes it to the online or offline store for immediate or batch use.
• TLS-Enabled Feature Servers - Starts the online feature server with HTTPS by providing private and public key file paths.
• Value Validations - Checks that feature data conforms to declared types, including timestamp columns used for point-in-time joins.
• Vector Database Backends - Persists feature values to vector databases for low-latency retrieval during online inference.
• Dedicated Transformation Servers - Feast deploys a dedicated HTTP server that executes on-demand transformations independently for horizontal scaling during real-time serving.
• Hybrid Precomputed and On-Demand Features - Feast uses a hybrid approach that mixes precomputed and on-demand transformations to balance latency and freshness.
• Write-Time Precomputations - Moves on-demand feature computation to materialization time to reduce serving latency.
• Retrieval Augmented Generation Pipelines - Combines embedding storage, vector search, and structured feature retrieval to supply context for LLM responses.
• Streaming Aggregation Tilings - Optimizes streaming feature aggregation using intermediate tiled representations for accuracy and efficiency.
• Vector Similarity Search - Feast queries a vector database for the nearest neighbors to a given query vector and returns the matching document vectors.
• Tag-Based Store Routings - Routes online feature reads and writes to different backend stores based on configurable FeatureView tags.
• Feature Drift Detectors - Feast profiles and validates feature data against expectations to catch drift or errors before they affect models.
• Custom Compute Engine Interfaces - Implements a defined interface and feature builder to run feature pipelines on a user-provided backend or execution framework.
• Application-Level RBAC Definitions - Defines permissions as Python objects in the feature repository and enforces them independently of Kubernetes RBAC.
• Team Object Ownership Restrictions - Limits teams to creating only feature services and on-demand views, preventing modification of core objects.
• Full Registry Synchronizations - Feast adds, updates, and deletes objects in the registry to maintain a complete and authoritative state for central platform teams.
• Entity-Based Feature Groupings - Groups semantically related features under a named entity for discovery, reuse, and joining on a common key.
• Custom Materialization Engines - Replaces the default materialization engine with a user-defined class for batch ingestion, historical retrieval, and infrastructure lifecycle.
• Batch Processing Schedulers - Runs a batch engine on a recurring schedule to materialize features.
• Online Store Connection Pool Tuning - Tunes connection pool sizes, timeouts, and retry settings per store type to avoid exhaustion and keep tail latency low.
• Feature Server - Tunes replica count, CPU, memory, connection pooling, and registry cache settings for feature server performance.
• Chunked Materialization Writes - Feast splits large offline-to-online writes into chunks to keep peak memory proportional to the batch size rather than the full dataset.
• Flink Execution Engines - Feast runs batch materialization and historical retrieval through PyFlink Table API, using Flink SQL for joins, filters, aggregations, and deduplication.
• Spark-Based Materialization - Runs batch materialization operations on a Spark cluster to transform and write large volumes of feature data.
• Streaming Processors - Applies transformations to streaming data via a pluggable interface with reference implementations.
• Pluggable Stream Processors - Applies user-defined transformations to streaming data through a pluggable Stream Processor interface.
• Row-Level Feature Transformations - Feast wraps a Python function with a decorator that accepts stored feature views and request sources, returning a new feature view computed per row.
• Data Type Mappings - Translates native types from offline and online stores into a common type system for consistent feature representation.
• Join Strategy Selectors - Feast automatically chooses between broadcast and distributed windowed joins based on dataset size to balance memory use and performance.
• Document Ingestion Pipelines - Chunks, embeds, and writes documents into the feature store in a single configurable pipeline.
• Feature Store Document Ingestion - Feast chunks, embeds, and writes documents to the online store in a single step using a configurable pipeline.
• Time-Window Aggregations - Feast computes batch or streamable aggregations such as sums or averages on data within a specified time period.
• Schema Validations - Checks that declared feature columns exist and match expected types during materialization and historical retrieval.
• Composite - Feast joins features from different sources or views into a single composite feature view using an inner join on entity ID.
• Kubernetes Horizontal Pod Autoscalers - Increases FeatureStore replicas via the Feast Operator, kubectl, or KEDA autoscaling to handle higher request loads.
• Vector Stores - Enables PGVector support to store vector embeddings and retrieve the top-k closest vectors for similarity search.
• Document Embedding Stores - Stores vector embeddings of document chunks in a feature store for later retrieval by similarity search.
• Document Embedding Generations - Generates millions of document embeddings in parallel using distributed compute engines like Spark or Ray.
• Worker Resource Allocation - Feast assigns CPU, memory, GPU, and custom resource requirements to individual tasks for precise scheduling across workers.
• Team Schema Isolation - Feast separates each team's feature data into its own database schema to prevent accidental cross-team interference.
• SQL Query Sources - Points feature stores to Trino tables or views so their columns become available as feature values.
• Stream Transformations - Feast applies transformation logic, such as user-defined functions, to raw streaming data to generate features.
• Streaming Data Sources - Registers Kafka streams as data sources for real-time feature computation and ingestion.
• Schema Inference - Automatically detects feature names and types from data source columns when no explicit schema is provided.
• Kafka Stream Ingestion - Feast ingests feature values from a Kafka stream into the online store for low-latency serving.
• Lazy Feature Retrieval - Returns a retrieval job that executes queries lazily and produces results as a DataFrame or Arrow table.
• Multi-Format Data Loading - Reads feature data from Parquet, CSV, JSON, HuggingFace, MongoDB, SQL, and more using Ray's native readers.
• Document Embedding Pipelines - Processes raw documents through chunking and embedding generation, then stores the results as features.
• Ingestion-Time Embedding Generation - Automates chunking, embedding generation, and writing to the online store for vector search.
• Vector Similarity Search - Queries the online store using vector similarity search to find feature values for nearest neighbors.
• Faiss Index Stores - Feast queries a Faiss index to retrieve feature vectors closest to a given embedding, enabling nearest-neighbor lookups.
• Kubernetes CronJob Schedulers - Deploys a Kubernetes CronJob that runs materialization on a recurring schedule to keep the online store fresh.
• REST APIs - Sends JSON POST requests to feature endpoints from any language, avoiding SDK overhead and local registry loading.
• Feature Retrieval APIs - Retrieves feature values for entities and feature references through a RESTful endpoint for low-latency online inference.
• Multi-Tenant Topologies - Creates a multi-tenant deployment with namespace isolation, managed stores, and fine-grained RBAC.
• GPU-Accelerated Transformations - Feast assigns GPU nodes to transformation workers through configuration, enabling GPU-native libraries for batch processing.
• Standard Production Topologies - Deploys a high-availability setup with HPA autoscaling, TLS ingress, and Kubernetes RBAC.
• Kubernetes Job Execution - Schedules batch feature materialization as Kubernetes Jobs that run on a recurring or ad-hoc basis.
• Kubernetes Operators - Uses a Kubernetes Operator to deploy, configure, and manage Feast resources via custom resource definitions.
• Minimal Production Topologies - Sets up a single-namespace Kubernetes deployment with no high availability for small teams.
• Client Certificate Authentication - Requires clients to present a certificate signed by a trusted CA for mutual authentication on server connections.
• Offline Server TLS Securings - Starts the offline store server with gRPC TLS by providing private and public key file paths.
• Registry Server TLS Securings - Starts the registry server with gRPC TLS by providing private and public key file paths.
• API Request Authentication - Feast extracts and parses authorization tokens from request headers to validate user identity for secured endpoints.
• OIDC Authentication Plugins - Feast validates Bearer tokens against an external OpenID Connect provider such as Keycloak or Dex for API access control.
• SSL/TLS Connection Security - Supports TLS and mutual TLS authentication by providing certificate and key paths to encrypt client-server communication.
• Kubernetes Token Authentications - Feast provides user tokens from CLI, API, or SDK to authenticate client requests to the Feast feature store.
• Feature Baseline Computations - Feast automatically computes reference statistics for new features when they are registered, establishing a baseline for drift detection.
• Infrastructure Component Overrides - Swaps individual store or infrastructure components while keeping the rest of the environment's provider configuration intact.
• Model and Lookup Table Pre-Loading - Pre-loads heavy resources like models and lookup tables at server startup to reduce per-request overhead.
• Machine Learning Operations - A feature store for machine learning that manages and serves ML features to models.
• Databases and Analytics - Feature store for managing ML data.
• Feature Engineering - Provides an open-source feature store for ML.
• Feature Stores - Open-source feature store for managing ML infrastructure.
• MLOps and Deployment - Feature store for managing ML model features.
• MLOps and Pipelines - Feature store for machine learning models.
• Data Science Tools - Feature store for managing machine learning features.

Star history