# twitter/the-algorithm-ml

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/twitter-the-algorithm-ml).**

10,545 stars · 2,245 forks · Python · agpl-3.0

## Links

- GitHub: https://github.com/twitter/the-algorithm-ml
- Homepage: https://blog.twitter.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm
- awesome-repositories: https://awesome-repositories.com/repository/twitter-the-algorithm-ml.md

## Description

The algorithm-ml is a machine learning ranking engine designed to personalize content feeds by calculating relevance scores for items based on user interests and historical interaction data. It functions as a recommendation system that processes user behavior and item metadata to determine the optimal order of content for individual users.

The system utilizes a multi-stage ranking architecture that filters large pools of candidate items into smaller sets before applying computationally expensive scoring models. It employs gradient-boosted decision tree ensembles to capture non-linear relationships within engagement data and uses feature-cross techniques to analyze specific interactions between user preferences and content attributes.

The platform supports large-scale operations through distributed model serving and a centralized feature store that provides low-latency access to precomputed attributes for real-time inference. Model refinement is managed through offline batch training pipelines that consume historical interaction logs to iteratively update predictive weights.

## Tags

### Artificial Intelligence & ML

- [Content Ranking Models](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/recommendation-engines/content-ranking-models.md) — Calculates relevance scores to determine the optimal order of content within personalized user feeds. ([source](https://blog.twitter.com/engineering/en_us/topics/open-source/2023/twitter-recommendation-algorithm))
- [Recommender Systems](https://awesome-repositories.com/f/artificial-intelligence-ml/recommender-systems.md) — Implements mathematical models to rank candidate items and determine the optimal order of content for users.
- [Feature Stores](https://awesome-repositories.com/f/artificial-intelligence-ml/feature-stores.md) — Utilizes a centralized feature store to provide low-latency access to precomputed attributes for real-time inference.
- [Recommendation Engines](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/recommendation-engines.md) — Builds recommendation systems that predict user preferences to surface relevant content from vast item pools.
- [Gradient Boosting Libraries](https://awesome-repositories.com/f/artificial-intelligence-ml/ensemble-learning-libraries/gradient-boosting-libraries.md) — Employs gradient-boosted decision tree ensembles to capture non-linear relationships in engagement data.
- [High-Throughput Model Serving](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-deployment-and-serving/inference-servers-and-runtimes/high-throughput-model-serving.md) — Provides high-throughput model serving infrastructure to maintain low latency for real-time scoring requests.
- [Model Inference and Serving](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-inference-serving.md) — Manages and serves predictive models to deliver tailored experiences within large-scale applications.
- [Feature Cross Scoring](https://awesome-repositories.com/f/artificial-intelligence-ml/evaluation-metrics/scoring-pipelines/feature-cross-scoring.md) — Uses feature-cross techniques to analyze complex interactions between user preferences and content attributes during ranking.
- [Training Log Analysis](https://awesome-repositories.com/f/artificial-intelligence-ml/training-log-analysis.md) — Consumes historical interaction logs in batch pipelines to iteratively refine predictive model weights.

### Content Management & Publishing

- [Personalized Feed Orchestrators](https://awesome-repositories.com/f/content-management-publishing/content-aggregation-curation/personalized-feed-orchestrators.md) — Delivers tailored content streams by processing user behavior and item metadata.

### Data & Databases

- [Ranking Engines](https://awesome-repositories.com/f/data-databases/ranking-engines.md) — Provides a machine learning ranking engine to personalize content feeds based on user interests and interaction data.
- [Ranking Pipelines](https://awesome-repositories.com/f/data-databases/data-processing-pipelines/document-llm-preparation/multi-stage-pipeline-processing/ranking-pipelines.md) — Implements multi-stage ranking pipelines to filter and score candidate items for personalized content delivery.
- [Offline Training Pipelines](https://awesome-repositories.com/f/data-databases/data-processing-pipelines/batch-processing-systems/batch-processing-utilities/offline-training-pipelines.md) — Manages model refinement through offline batch training pipelines that process historical interaction logs.

### Web Development

- [Interest Modeling](https://awesome-repositories.com/f/web-development/user-profiles/interest-modeling.md) — Analyzes historical interaction data to build profiles that inform real-time content prioritization.