Open-source alternatives to Linkedin Scraper

Katana is a web crawler and spider designed for security reconnaissance and web application mapping. It functions as a utility for identifying endpoints, forms, and API structures across web targets by combining standard HTTP request traversal with headless browser automation to render dynamic, JavaScript-heavy content. The tool distinguishes itself through its ability to maintain authenticated sessions and handle complex web interactions, such as automated form submission and captcha resolution. It provides granular control over the discovery process, allowing users to define specific crawl

This project is a web scraping and automation tool designed to collect public data from Instagram and perform automated social media interactions. It functions by gathering profile details, captions, media files, and engagement metrics directly from web pages, bypassing the need for official developer interfaces or platform-specific credentials. The tool distinguishes itself by combining data extraction with automated engagement capabilities. It allows users to programmatically interact with content by liking posts that match specific search criteria or hashtags, aiming to increase account vi

TikTokDownloader is a containerized automation tool designed for the systematic collection and archiving of social media content. It functions as an HTTP API-driven service that enables users to trigger automated media retrieval and metadata extraction tasks through standard web requests, facilitating the storage of videos, images, and platform data for offline analysis. The utility distinguishes itself through session-based web scraping, which utilizes browser cookies to authenticate requests and access restricted platform content that is otherwise unavailable. By maintaining active session

OpenWeChat is a Go language software development kit and API wrapper designed for integrating WeChat messaging and account management into Go applications. It serves as a bot framework and messaging library for handling real-time chat events and programmatic interactions with the platform. The project provides a comprehensive system for session management, including QR code authentication and the persistence of session cookies in JSON format to maintain access across restarts. It distinguishes itself by offering capabilities to intercept and preserve messages that senders attempt to revoke, a

igbot is an Instagram automation bot that handles routine social media interactions on Instagram, including following, unfollowing, liking, and commenting on posts and profiles. It also functions as a video metadata extractor, retrieving download URLs, view counts, and audio information from Instagram Reels and public short videos. To carry out these actions, the bot combines GraphQL API scraping with headless browser automation, mimicking human behavior through randomized action timing, session cookie persistence, and user agent rotation. A task queue with exponential backoff retry ensures r

Spider_XHS is a data extraction and automation tool built specifically for the Xiaohongshu social platform. It orchestrates multi-step workflows that combine comment tree traversal, cookie-based session reuse, high-resolution media retrieval, keyword search, proxy-backed retries, QR-code login, structured file export, and aggregated user profile collection into a single pipeline. The tool distinguishes itself through its integrated authentication and publishing capabilities, supporting login via QR code scanning or phone verification codes to establish and maintain authenticated sessions. It

Browserless is a service-oriented platform designed for remote browser automation and headless execution. It provides a distributed infrastructure that manages browser sessions through containerized isolation, allowing users to execute scripts and interact with web content without maintaining local browser state or infrastructure. The platform functions as a remote API and WebSocket-based control layer, enabling stateless HTTP requests for tasks like document generation and real-time browser interaction. It incorporates proxy-based routing to manage traffic signatures and supports the integra

Crawlee is a web scraping framework designed for building scalable, reliable, and distributed data extraction pipelines. It provides a unified interface for managing headless browser automation and lightweight HTTP requests, allowing developers to handle complex web navigation, dynamic content rendering, and large-scale data collection within a single, modular architecture. The project distinguishes itself through its resource-aware concurrency controller, which dynamically scales task execution based on real-time CPU and memory usage to prevent host machine exhaustion. It also features a rob

This project is a social media automation tool designed to publish videos and images across multiple social networks programmatically. It functions as a headless browser content publisher and a multi-platform posting API, allowing for automated social media posting and content distribution. The system utilizes browser automation to execute uploads and logins on platforms without official public APIs. It features a social media command-line manager for executing batch media uploads and managing account sessions, as well as a programmatic interface for triggering uploads and scheduling content

Scraperjs is a JavaScript web scraping library and headless browser automation tool designed to extract structured data from both static HTML and JavaScript-rendered websites. It functions as an asynchronous scraping workflow orchestrator and a selector-based engine for transforming unstructured web content into usable data. The library distinguishes itself by managing a shared browser instance across multiple tasks to optimize resource usage and reduce startup latency. It supports custom script injection into sandboxed browser environments to manipulate the DOM or execute browser-side logic

Puppeteer is a JavaScript library for programmatically controlling Chrome and Firefox through the Chrome DevTools Protocol or the WebDriver BiDi protocol. It launches and manages browser instances—typically without a visible user interface—to automate interactions with web pages, enabling navigation, clicking, typing, and data extraction entirely through code. The library distinguishes itself through deep integration with the Chromium embedding layer, allowing fine-grained process configuration with custom flags, permissions, and sandbox policies. It maintains multiple concurrent command stre

chromedp is a browser automation framework and driver that controls web browsers via the Chrome DevTools Protocol. It functions as a headless browser automation tool and web browser controller, enabling the programmatic management of browser sessions, targets, and network responses through a remote debugging interface. The project provides specialized capabilities for Chrome DevTools Protocol automation, including headless browser testing, web scraping and data extraction, and mobile device emulation. It also supports browser-based visual regression by capturing precise screenshots of web pag

Firecrawl is a headless browser automation tool and web crawling engine designed to extract structured data from the web. It functions as an API that transforms raw website content and documents into clean markdown and JSON formats to serve as context for large language models. The project distinguishes itself by using natural language prompts to translate human instructions into targeted data extraction tasks and browser actions. It can execute interactive page navigation, such as clicking and scrolling, and perform automated web research to retrieve structured data without manual interventi

Crawlee-python is a web crawling framework for building scalable scrapers using Python. It serves as a comprehensive tool for web scraping automation, providing a system to extract structured data from websites using both lightweight HTTP requests and headless browser automation. The framework is distinguished by its anti-bot evasion capabilities, which include browser fingerprint impersonation and tiered proxy rotation to bypass detection systems and solve challenges such as Cloudflare. It also incorporates artificial intelligence for autonomous website navigation and schema-based data extra

OpenCLI is an AI browser automation framework designed to automate web navigation, data extraction, and repetitive browser tasks. It functions as a browser-based CLI generator that converts website interfaces into command-line interactions by controlling authenticated web browser sessions. The project features a web-to-CLI adapter platform for mapping web elements to programmatic command-line inputs and outputs. It includes a browser profile manager to organize and switch between isolated session profiles to maintain different user identities. The toolkit provides capabilities for web conten

This project is a collection of Python scripts and tools designed for web scraping, browser automation, and large-scale data extraction. It provides a set of implementations for retrieving information from websites and private APIs, including tools for multimedia downloading and social media data archiving. The toolset includes specialized mechanisms for bypassing anti-scraping measures through IP proxy pool rotation and multi-threaded crawlers. It also features capabilities for simulating browser sessions to handle authentication, intercepting session cookies, and decrypting network payloads

Telegram-Media-Downloader is a web media scraper and restricted content downloader designed to extract images and videos from the Telegram web interface for local storage. It functions by programmatically automating a web browser to capture media from chats and stories. The utility specifically enables the recovery of restricted content from groups and channels where administrators have disabled the native download function. It bypasses these restrictions by identifying media elements within the browser to locate source URLs. The system includes capabilities for session-based authentication,

Spider is a web-based platform designed for automated data extraction, providing a centralized framework to collect, process, and route structured information from websites. It functions as a comprehensive pipeline that manages the entire lifecycle of data gathering, from initial configuration to final storage in external databases or message queues. The platform distinguishes itself through a visual configuration interface that allows users to define extraction rules and manage scraping templates without writing custom code. It supports both static and dynamic content retrieval by integratin

FlareSolverr is a proxy server designed to provide programmatic access to websites protected by automated security challenges and firewall restrictions. It functions by orchestrating headless browser instances to render web pages, execute JavaScript, and retrieve the necessary cookies and content required to bypass common security hurdles. The service distinguishes itself by maintaining persistent browser sessions in memory, which allows for the reuse of authenticated states across multiple requests. It integrates with external captcha resolution services to handle interactive security challe

This project is a specialized TikTok API scraper and data extractor. It functions as a proxy-based web scraper designed to collect user metadata, video posts, and trend feeds, while providing a webhook data pipeline to route scraped information to external URLs via HTTP requests. The tool includes a watermark-free video downloader that saves high-definition content to local storage. It employs cryptographic request signing for server authentication and utilizes session cookie authentication combined with proxy rotation to manage network traffic and avoid rate limits. Capabilities cover bulk

LastPass CLI is a command-line tool for managing secure credentials, authenticating user identities, and synchronizing encrypted password vaults directly from the terminal. It provides a terminal interface for storing, retrieving, and organizing sensitive account data with local encryption and remote synchronization, functioning as an encrypted vault management tool for handling credential storage across devices using remote authorization endpoints. The application communicates with remote servers over HTTPS while securing user credentials locally and in transit using client-side symmetric en

ytdlnis is a mobile application that serves as a graphical client for the yt-dlp engine on Android. It functions as a media downloader and manager, providing a user interface to retrieve video and audio from websites. The project distinguishes itself by integrating directly with the Android system share menu and intents to trigger background downloads from external apps. It includes a dedicated authentication cookie manager to import and sync browser session data, enabling the retrieval of private, age-restricted, or premium content. The application covers broad capability areas including au

Maxun is an open-source web scraping and automation platform designed to transform dynamic website content into structured data. By leveraging artificial intelligence to interpret natural language prompts, the system identifies page elements and extracts information without requiring manual selector configuration. It serves as a bridge between raw web content and intelligent workflows, providing structured outputs in formats optimized for large language model ingestion and agent-based applications. The platform distinguishes itself through its ability to handle complex, authenticated, and dyn

This project is a ticket purchase automation tool and browser automation bot designed to secure high-demand event tickets. It functions as a web scraping purchase script that monitors availability and executes checkout transactions programmatically. The tool utilizes a hybrid execution model that combines headless browser automation for authentication and session management with direct HTTP requests to ticketing server APIs. This approach is used to bypass user interface latency and handle high-speed request processing during flash sales. The system includes capabilities for automated availa

biliTickerBuy is a Bilibili ticket purchase automator and web-based ticketing bot. It functions as an e-commerce checkout script designed to automate the acquisition of limited inventory tickets on the Bilibili platform to increase purchase speed. The tool handles Bilibili event ticketing and the acquisition of limited inventory items. It automates the checkout process to bypass manual entry and increase the success rate of securing high-demand event entries. The project utilizes headless browser automation and request-based sequence scripting. It includes capabilities for polling-based inve

Playwright-cli is a command line interface for executing web tasks and automating browser interactions using the Playwright framework. It serves as a browser binary manager for downloading and installing specific browser engines and their required system dependencies, as well as a tool for running automated test suites across multiple engines to verify application behavior. The utility functions as a browser session controller, managing browser profiles and persistent storage states via the command line. It enables the execution of automation suites across different browser engines and config