Gorse

Gorse is a personalized recommendation engine server and machine learning pipeline designed to suggest items to users based on their behavior and preferences. It operates as a distributed system that separates training, candidate generation, and serving nodes to support high-throughput workloads.

The system utilizes a multi-stage recommendation pipeline to refine results through retrieval, scoring, and reranking. It generates personalized suggestions using collaborative filtering, matrix factorization, and item-to-item similarity models, while also providing non-personalized and fallback recommendations when individual profiles are unavailable.

The infrastructure supports scaling via Kubernetes and provides a management dashboard secured with OpenID Connect. Broad capabilities include model training and evaluation, a pluggable database backend, and observability via OpenTelemetry request tracing.

Users can integrate the engine into applications through REST endpoints or dedicated SDKs for Go, Python, Rust, TypeScript, Java, and .NET.

Features

Recommendation Engines - Provides a complete system for predicting and ranking items to deliver personalized content suggestions.
Personalized Recommendation Retrieval - Generates item suggestions for specific users by leveraging collaborative filtering and item-to-item similarity logic.
Recommendation List Generators - Generates personalized item suggestions for specific users using collaborative filtering and item-to-item logic.
Collaborative Filtering Models - Uses matrix factorization to train collaborative filtering models from user-item interaction data.
Candidate Generation - Spreads the computation of item and user similarities across multiple worker nodes to handle large-scale datasets.
Interaction Matrix Factorizers - Learns user and item latent features from historical interaction data to predict engagement probabilities.
Model Training Pipelines - Implements end-to-end workflows for sourcing interaction data, training, and validating recommendation models.
Candidate Sourcing Pipelines - Retrieves a broad set of potential items using similarity strategies and external data sources.
Recommendation Engine Pipelines - Orchestrates the flow from candidate generation and scoring to final reranking.
Model Training - Calculates similarities and trains prediction models using historical interaction data.
Recommendation Scoring and Ranking - Calculates engagement probability for candidate items and sorts them to present the most relevant content first.
Recommendation Pipelines - Processes candidates through a sequence of retrieval, scoring, and reranking stages to refine the final item list.
User-to-User Similarity - Identifies users with shared labels to suggest items based on the feedback of similar peers.
User Feedback Collection - Provides mechanisms for inserting and updating user interactions with items to inform underlying recommendation models.
Recommendation Systems - A distributed framework for building and serving personalized recommendation engines at scale.
Vector Similarity Search - Calculates item similarities using embedding vectors to suggest related content.
Workflow Execution Scaling - Distributes workloads across multiple server and worker nodes to increase the volume of recommendations generated.
Distributed Recommendation Infrastructure - Deploys a distributed system across Kubernetes clusters to handle high request volumes and large datasets.
Role-Based Scaling - Distributes tasks across master, worker, and server nodes to separate heavy model training from real-time serving.
Item-to-Item Similarity - Suggests items similar to a target by calculating neighbor similarity based on a chosen field.
Recommendation APIs - Delivers real-time recommendation results to external services through a set of standardized web endpoints.
Model Evaluation Metrics - Calculates accuracy metrics using training and testing datasets to estimate algorithm quality.
Reranking Models - Implements a reranking stage that applies ML models to candidate pools to improve result relevance.
External Source Integration - Executes external scripts to retrieve suggested items from third-party sources.
Fallback Strategies - Supplies general non-personalized content when personalized recommendation sources are unavailable.
Non-personalized Recommendations - Suggests popular or discovery items to all users regardless of individual interaction history.
Hyperparameter Tuning - Tunes model hyperparameters through periodic trials and early stopping to maximize prediction accuracy.
Pipeline Visual Configurators - Offers a visual interface to modify the sequence of recommendation stages to refine item suggestion logic.
Recommendation Filtering - Removes previously viewed items and swaps insufficient results with alternative suggestions.
Recommendation Success Metrics - Tracks positive feedback rates and visualizes results to determine user engagement levels.
Hybrid Recommendation Integration - Combines results from external APIs to supplement or replace internally generated model outputs.
Non-personalized Recommendations - Generates global recommendation lists using custom expressions and scoring independent of user profiles.
Recommendation Merging - Combines item suggestions from remote endpoints into a final recommendation list using automated scripts.
Result Caching - Stores intermediate computation results and final lists to reduce CPU load and improve response times.
Pluggable Storage Backends - Implements a pluggable storage architecture allowing the use of various database engines for persistence and caching.
Item Management - Provides APIs to create, modify, delete, and retrieve the individual items suggested to users.
Recommendation Entity Management - Provides an interface to inspect relationships and delete specific user or item records.
Single-Binary Distributions - Provides a standalone executable containing all dependencies for easy installation on a single node.
Kubernetes Application Deployments - Provides automated Helm charts to bootstrap the engine and database on Kubernetes clusters for rapid installation.
Service Deployment - Supports installing the system via binaries, containers, or clusters to establish a running recommendation service.
Single-Node Deployment - Packages all necessary components into a single container for simplified installation on a single machine.
Prediction Workload Distribution - Distributes the computation of item and user similarities across multiple worker nodes to handle large-scale datasets.
API Key Authentication - Restricts access to recommendation endpoints by validating a unique identifier passed in the request header.
Database Abstraction Layers - Abstracts the storage layer to allow switching between different database engines for persisting users, items, and feedback.
Distributed Role Separation - Implements a distributed architecture that separates master, worker, and server nodes to decouple model training from real-time serving.
Multi-Language SDKs - Provides pre-built client libraries in multiple programming languages to ensure consistent access to the recommendation engine.
Cluster Health Monitoring - Provides a terminal interface to track the health and progress of background tasks and distributed cluster nodes.
Metric and Performance Monitors - Tracks accuracy metrics alongside user and item counts to evaluate model effectiveness.
Recommendation Management Dashboards - Provides a web-based dashboard for configuring recommendation flows and monitoring cluster health.
System Monitoring Dashboards - Provides a centralized administrative interface to monitor system health and handle data movement.
Recommendation SDKs - Offers dedicated SDKs for Go, Python, Rust, TypeScript, Java, and .NET to consume recommendation services.
External Recommendation Integrations - Retrieves item suggestions from third-party services to expand the local recommendation set.
AI and Machine Learning - AI-powered recommender system engine.
Recommender Systems - Universal open-source recommender system for online services.

d2l-ai/d2l-en

29,001View on GitHub

This project is an educational platform and research toolkit designed to teach deep learning through a combination of mathematical theory, visual diagrams, and executable code. It provides a comprehensive environment for building, training, and evaluating neural networks, grounding complex concepts in interactive computational notebooks that allow for hands-on experimentation. The framework distinguishes itself by interleaving theoretical foundations—including linear algebra, calculus, and probability—with practical implementations across multiple industry-standard libraries. It supports flex

datawhalechina/fun-rec

7,177View on GitHub

fun-rec is a learning guide and framework for building personalized recommendation systems, covering everything from deep learning ranking to generative recommendation paradigms. It provides instructional content on constructing industrial-grade architectures that span offline data processing and real-time online serving. The project distinguishes itself by focusing on generative recommendation, treating the suggestion process as a sequence-to-sequence task using large language models and transformer models to generate item identifiers rather than traditional ranking lists. It also emphasizes

recommenders-team/recommenders

21,769View on GitHub

This project is a recommendation system framework designed for building, evaluating, and operationalizing personalized item suggestion engines. It provides a comprehensive toolkit for implementing collaborative filtering and content-based algorithms, supported by an end-to-end machine learning pipeline for preparing datasets and deploying predictive models. The framework distinguishes itself through the integration of knowledge graphs to provide richer context for recommendations and the use of industry-specific patterns to accelerate system deployment. It also includes a specialized model ev

lyst/lightfm

5,095View on GitHub

LightFM is a Python recommendation library and machine learning framework designed to predict user preferences. It implements a hybrid recommendation engine that combines collaborative filtering with content filtering by integrating user-item interaction data with descriptive metadata. The system utilizes hybrid matrix factorization to learn latent representations of users and items. It is specifically designed to handle implicit feedback, utilizing specialized loss functions such as Weighted Approximate Rank Pairwise and Bayesian Personalized Ranking to optimize item preferences for datasets

gorse-iogorse

Features