# twintproject/twint

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/twintproject-twint).**

16,319 stars · 2,786 forks · Python · mit · archived

## Links

- GitHub: https://github.com/twintproject/twint
- awesome-repositories: https://awesome-repositories.com/repository/twintproject-twint.md

## Topics

`elasticsearch` `kibana` `osint` `python` `scrape` `scrape-followers` `scrape-following` `scrape-likes` `tweep` `tweets` `twint` `twitter`

## Description

Twint is an open-source intelligence and data extraction framework designed to gather public social media information. It functions as a command-line utility that retrieves posts, user profiles, and follower lists directly from web interfaces, bypassing the need for official platform developer credentials or authentication keys.

The tool distinguishes itself by enabling automated, large-scale data collection through terminal-based orchestration. It supports granular filtering by keywords, geographic locations, time ranges, and account status, allowing researchers to build targeted datasets for sentiment analysis or network relationship mapping. The architecture includes state tracking to resume interrupted sessions and an integrated pipeline for real-time text translation during the collection process.

Beyond raw extraction, the project provides a modular output system that streams data into local files, databases, or external search engines. This design facilitates integration with visualization tools for generating network graphs and interactive dashboards, supporting long-term research workflows and investigative analysis.

## Tags

### Development Tools & Productivity

- [Command Line Interfaces](https://awesome-repositories.com/f/development-tools-productivity/command-line-interfaces.md) — Provides a command-line interface for orchestrating large-scale social media data collection tasks.
- [OSINT Automation Frameworks](https://awesome-repositories.com/f/development-tools-productivity/open-source-tools/osint-automation-frameworks.md) — Automates the retrieval of public web content and mapping of account connections for investigative OSINT analysis.
- [Social Media Research Tools](https://awesome-repositories.com/f/development-tools-productivity/social-media-research-tools.md) — Enables researchers to filter and retrieve social media content for investigative and academic analysis.
- [CLI Task Managers](https://awesome-repositories.com/f/development-tools-productivity/terminal-shell-cli/terminal-cli-enhancements/cli-task-managers.md) — Executes data extraction tasks directly from the terminal using specific search parameters. ([source](https://github.com/twintproject/twint/blob/master/setup.py))

### Web Development

- [Social Media Extraction Tools](https://awesome-repositories.com/f/web-development/social-media-extraction-tools.md) — Collects public social media posts, profiles, and follower lists without requiring official developer credentials. ([source](https://github.com/twintproject/twint/blob/master/README.md))
- [Social Media Scrapers](https://awesome-repositories.com/f/web-development/social-media-scrapers.md) — Extracts public social media posts and user profiles via command-line tools without needing official platform API access.

### Business & Productivity Software

- [Social Media Intelligence Gatherers](https://awesome-repositories.com/f/business-productivity-software/social-media-management-platforms/social-media-intelligence-gatherers.md) — Collects and filters public social media activity to support intelligence gathering and long-term research.
- [Automated Extraction Schedulers](https://awesome-repositories.com/f/business-productivity-software/scheduling-automation/automated-extraction-schedulers.md) — Automates recurring data collection tasks to maintain up-to-date datasets without manual intervention. ([source](https://github.com/twintproject/twint/blob/master/requirements.txt))

### Networking & Communication

- [HTTP Request Builders](https://awesome-repositories.com/f/networking-communication/http-request-builders.md) — Fetches public web content by mimicking browser requests to bypass platform authentication and rate limits.

### Software Engineering & Architecture

- [Public Data Gathering Frameworks](https://awesome-repositories.com/f/software-engineering-architecture/public-data-gathering-frameworks.md) — Gathers public social data from platforms without needing official access keys. ([source](https://github.com/twintproject/twint#readme))
- [Social Network Analysis Tools](https://awesome-repositories.com/f/software-engineering-architecture/social-network-analysis-tools.md) — Maps and visualizes social network connections and influence patterns using collected follower and interaction data.
- [Scraping Resumption Handlers](https://awesome-repositories.com/f/software-engineering-architecture/execution-pausing/inference-task-interruption/scraping-resumption-handlers.md) — Tracks progress to resume interrupted data collection sessions from the last processed item. ([source](https://github.com/twintproject/twint/blob/master/README.md))
- [Modular Architectures](https://awesome-repositories.com/f/software-engineering-architecture/modular-architectures.md) — Features a modular output architecture that streams collected data into various local files and external databases.

### Data & Databases

- [Web Data Pipelines](https://awesome-repositories.com/f/data-databases/data-integration-synchronization/event-driven-data-pipelines/web-data-pipelines.md) — Provides automated workflows to extract and load public social media data into local storage for research.
- [Data Query Filters](https://awesome-repositories.com/f/data-databases/data-query-filters.md) — Enables granular filtering of social media activity by keywords, location, and time ranges. ([source](https://github.com/twintproject/twint/blob/master/test.py))
- [Search Result Filtering](https://awesome-repositories.com/f/data-databases/search-result-filtering.md) — Filters and searches social media content using criteria like keywords, location, and verification status. ([source](https://github.com/twintproject/twint#readme))
- [Web Data Extraction](https://awesome-repositories.com/f/data-databases/web-data-extraction.md) — Extracts public profile data directly from web interfaces for research and analysis. ([source](https://github.com/twintproject/twint/blob/master/setup.py))
- [Structured Data Extraction](https://awesome-repositories.com/f/data-databases/structured-data-extraction.md) — Extracts structured data from raw web responses by parsing HTML document elements.
- [Cursor-Based Pagination](https://awesome-repositories.com/f/data-databases/data-access-querying/data-access-abstraction/cursor-based-pagination.md) — Implements cursor-based pagination to maintain state and resume interrupted data collection sessions.
- [Data Exporters](https://awesome-repositories.com/f/data-databases/data-exporters.md) — Exports scraped data into multiple formats and streams it directly into external databases. ([source](https://github.com/twintproject/twint/blob/master/README.md))
- [Local Data Stores](https://awesome-repositories.com/f/data-databases/local-data-stores.md) — Saves gathered information into local files and databases for long-term research. ([source](https://github.com/twintproject/twint#readme))

### Programming Languages & Runtimes

- [Relationship Graph Visualizers](https://awesome-repositories.com/f/programming-languages-runtimes/programming-utilities/data-structure-type-helpers/data-structures/hierarchical-tree-structures/relationship-graph-visualizers.md) — Generates network graphs from stored user and follower data to visualize social relationships. ([source](https://github.com/twintproject/twint/wiki/Graph))