twintproject/twintArchived
Twint
Twint is an open-source intelligence and data extraction framework designed to gather public social media information. It functions as a command-line utility that retrieves posts, user profiles, and follower lists directly from web interfaces, bypassing the need for official platform developer credentials or authentication keys.
The tool distinguishes itself by enabling automated, large-scale data collection through terminal-based orchestration. It supports granular filtering by keywords, geographic locations, time ranges, and account status, allowing researchers to build targeted datasets for sentiment analysis or network relationship mapping. The architecture includes state tracking to resume interrupted sessions and an integrated pipeline for real-time text translation during the collection process.
Beyond raw extraction, the project provides a modular output system that streams data into local files, databases, or external search engines. This design facilitates integration with visualization tools for generating network graphs and interactive dashboards, supporting long-term research workflows and investigative analysis.
Features
- Command Line Interfaces - Provides a command-line interface for orchestrating large-scale social media data collection tasks.
- OSINT Automation Frameworks - Automates the retrieval of public web content and mapping of account connections for investigative OSINT analysis.
- Social Media Extraction Tools - Collects public social media posts, profiles, and follower lists without requiring official developer credentials.
- Social Media Scrapers - Extracts public social media posts and user profiles via command-line tools without needing official platform API access.
- Social Media Intelligence Gatherers - Collects and filters public social media activity to support intelligence gathering and long-term research.
- Social Media Research Tools - Enables researchers to filter and retrieve social media content for investigative and academic analysis.
- HTTP Request Builders - Fetches public web content by mimicking browser requests to bypass platform authentication and rate limits.
- Public Data Gathering Frameworks - Gathers public social data from platforms without needing official access keys.
- Web Data Pipelines - Provides automated workflows to extract and load public social media data into local storage for research.
- Data Query Filters - Enables granular filtering of social media activity by keywords, location, and time ranges.
- Search Result Filtering - Filters and searches social media content using criteria like keywords, location, and verification status.
- Web Data Extraction - Extracts public profile data directly from web interfaces for research and analysis.
- OSINT Intelligence - Scrape and analyze Twitter data without using the official API.
- Structured Data Extraction - Extracts structured data from raw web responses by parsing HTML document elements.
- CLI Task Managers - Executes data extraction tasks directly from the terminal using specific search parameters.
- Social Network Analysis Tools - Maps and visualizes social network connections and influence patterns using collected follower and interaction data.
- Cursor-Based Pagination - Implements cursor-based pagination to maintain state and resume interrupted data collection sessions.
- Data Exporters - Exports scraped data into multiple formats and streams it directly into external databases.
- Scraping Resumption Handlers - Tracks progress to resume interrupted data collection sessions from the last processed item.
- Automated Extraction Schedulers - Automates recurring data collection tasks to maintain up-to-date datasets without manual intervention.
- Local Data Stores - Saves gathered information into local files and databases for long-term research.
- Relationship Graph Visualizers - Generates network graphs from stored user and follower data to visualize social relationships.
- Modular Architectures - Features a modular output architecture that streams collected data into various local files and external databases.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
