joeyism/linkedin_scraper

Features

• Headless Browser Automation - Uses a headless browser to simulate human interaction and render dynamic content for data extraction.
• Headless Browser Orchestrators - Implements a headless browser system to simulate human interaction and extract dynamic professional profile data.
• Job Market Data - Extracts job listings and requirements to facilitate analysis of hiring trends and industry demands.
• Job Market Scraping - Collects job requirements and application links using keyword and location filters.
• Professional Profile Scraping - Gathers career data and company history from professional networking sites using automated browser tools.
• DOM-Based Extractions - Extracts professional data by querying the HTML structure of rendered web pages using JavaScript.
• Profile Extractors - Gathers work experience, education, and contact details from individual professional user pages.
• Session Management - Manages authentication cookies and browser settings to access protected professional profiles.
• Session and Credential Management - Captures and stores user session tokens and credentials in files for persistence across multiple requests.
• Browser Session Authentication - Utilizes browser cookies to authenticate automated requests to protected professional profiles.
• Session-Based Scrapers - Utilizes browser session cookies to scrape restricted profiles, company details, and job listings from LinkedIn.
• Session-Cookie Persistences - Saves and reuses authentication cookies in local files to maintain persistent access across script runs.
• Web Session Management - Coordinates the login process and maintains state to access restricted areas of the professional network.
• Company Intelligence Utilities - Gathers business overviews and industry classifications to build professional company profiles.
• Organizational Data Extraction - Collects industry details, organizational size, and headquarters locations from company information pages.
• Competitive Market Research - Provides capabilities to extract company overviews and industry data for competitive analysis.
• Professional Networking Automation - Automates the collection of posts and profile updates to monitor industry activity.
• Contact Discovery - Identifies and extracts contact information associated with professional user profiles.
• Post & Comment Scraping - Fetches post content, timestamps, and engagement metrics from organization social feeds.
• Lead Enrichment - Extracts and augments lead profiles with contact information and career history from professional networking sites.
• Profile Information Retrieval - Fetches specific personal and professional profile details using unique identifiers.
• User Profile Retrieval - Retrieves detailed work experience, education, and skill sets from professional user profiles.
• User Agent Rotation - Cycles through user agent strings and viewport settings to mimic human behavior and avoid bot detection.
• Search Filtering Logic - Narrows down job listings and professional profiles based on user-provided keywords and location criteria.
• Browser Argument Configuration - Configures headless browser arguments, viewport size, and user agents to simulate human activity.

Star history