30 open-source projects similar to naibowang/easyspider, ranked by shared indexed features. Tags may describe platforms or build tools rather than the same primary purpose. Check each project’s use case, license, and deployment requirements before treating it as a replacement.
Automa is a browser-based automation platform that enables users to build, schedule, and execute repetitive web tasks through a visual, no-code interface. By operating as a browser extension, it provides a canvas-based environment where users construct workflows by connecting functional blocks to interact with web elements, manage browser state, and process data. The platform distinguishes itself through its deep integration with the browser environment, allowing for complex orchestration such as event-driven triggers, cross-origin request handling, and the ability to package workflows as sta
Spider-flow is a Java-based web crawling and data extraction platform that provides a centralized environment for managing automated information gathering. It functions as a no-code tool, allowing users to define complex data collection pipelines through a visual, drag-and-drop interface rather than manual programming. The platform distinguishes itself through a graph-based workflow orchestration system where users link discrete nodes to define navigation and parsing logic. It supports dynamic content crawling by integrating headless browsers to execute JavaScript and render page content that
SuperAGI is a comprehensive marketing automation platform and customer data system designed to orchestrate multi-channel engagement workflows. It functions as a no-code workflow orchestrator, allowing users to build complex, automated task sequences triggered by real-time user behavior, transactional data, or scheduled events. By centralizing customer profiles and interaction history, the platform enables businesses to manage end-to-end marketing operations from a single interface. The platform distinguishes itself through its deep integration with e-commerce storefronts and its ability to ex
CodeceptJS is a Node.js end-to-end testing framework used for writing acceptance and functional tests for web and mobile applications. It functions as an automated browser testing suite and cross-browser automation engine that allows users to simulate end-user interactions and verify page content using a JavaScript-based domain-specific language. The framework implements a behavior-driven development approach, enabling the creation of scenario-driven tests that describe application behavior in a human-readable format. It utilizes the page object model to encapsulate UI elements and logic into
OpenFaaS is a serverless function platform that provides a container-native framework for deploying and managing event-driven code. It functions as an abstraction layer over container orchestrators, allowing developers to package code into scalable functions that run across Kubernetes clusters or edge computing environments. The platform distinguishes itself through a developer-centric runtime that utilizes standardized language templates and automated build pipelines to simplify the creation of container images. It features a central API gateway that manages request routing, authentication,
Portia is a containerized scraping platform and visual web scraper that enables no-code data extraction. It serves as a Scrapy visual scraping tool and spider generator, allowing users to design and deploy web scrapers through a graphical interface instead of writing manual selector code. The system distinguishes itself by converting visual web page annotations into executable Scrapy spider code and structured JSON specifications. This visual-to-code mapping allows users to define scraping logic and extraction rules through a point-and-click interface, which can then be exported for use in ex
Automatisch is an open-source, self-hosted automation platform designed to orchestrate multi-stage workflows across various third-party web services. It functions as a no-code integration engine that allows users to connect disparate applications, enabling the automated movement of data and the execution of tasks without manual intervention. By running the platform on private infrastructure, users maintain full control over their data and ensure compliance with internal security policies. The platform distinguishes itself through a focus on secure, local credential management and flexible int
Firetable is a backend administration platform and low-code orchestrator for Firebase databases. It provides a spreadsheet-like interface for managing cloud database records, defining rich data fields, and controlling user access permissions. The project functions as a no-code automation platform and cloud function orchestrator, allowing for the deployment and management of serverless logic through a visual browser interface. It enables the creation of backend workflows and external API integrations that are triggered by specific data changes at the field level. The platform covers broad cap
This platform is a modular, metadata-driven framework designed for building custom business applications and data management systems without traditional coding. It functions as a low-code environment where data models, user interfaces, and business logic are defined through visual configurations rather than hardcoded views. The architecture supports multi-tenant isolation, allowing multiple independent applications to run within a single shared memory space while maintaining strict logical separation of data and configurations. What distinguishes this system is its deep integration of artific
This platform is a low-code database system that combines the flexibility of a spreadsheet interface with the structured power of a relational database. It serves as a collaborative workspace for managing complex datasets, building custom business applications, and automating operational workflows without requiring traditional software development. The platform distinguishes itself through deep integration of artificial intelligence, which enables users to query databases using natural language, generate content, and deploy custom conversational agents trained on internal data. It supports re
Baserow is a self-hosted, no-code relational database platform built on PostgreSQL. It provides a spreadsheet-like interface for structuring and managing data without writing code, while exposing all database resources via a REST API to support headless architectures. The platform distinguishes itself by integrating large language models and embedding servers to power AI assistants and automated data generation. It further extends its utility as a no-code application builder, allowing users to create custom internal portals, dashboards, and business tools using visual logic and managed data.
Heyform is an open-source form builder and self-hosted data collection platform. It provides a no-code designer for creating dynamic web-based surveys and input forms, supported by an extensible backend for managing submissions and storing results in a private database. The system distinguishes itself through advanced form logic and branding controls. It includes a conditional logic engine to show or hide sections based on user responses and allows for precise visual identity customization through configurable themes and custom CSS injection. The platform covers a broad range of operational
Flowy is a JavaScript library designed for building interactive, drag-and-drop flowchart interfaces. It enables the creation of canvas-based environments where users can arrange blocks to design custom logic sequences, decision trees, and visual process maps. The library provides a structured approach to diagramming by organizing elements into a hierarchical parent-child tree. It features coordinate-based snapping logic to ensure blocks automatically align into organized layouts during user interaction. To support complex system configurations, the tool includes event-driven callback hooks th
This project is an agentic framework designed to enable autonomous web navigation and browser automation. It functions as a controller that translates natural language instructions into deterministic browser actions, allowing agents to interact with websites, perform data extraction, and manage complex authentication flows. By leveraging accessibility trees and semantic element resolution, the framework mimics human-like navigation, moving beyond brittle DOM selectors to interact reliably with modern web interfaces. The framework distinguishes itself through its focus on secure, scalable exec
This project is a high-performance headless browser engine designed for scalable web automation, data extraction, and AI agent integration. It provides a specialized environment that allows autonomous agents and testing frameworks to interact with web content through standardized remote control protocols. By executing pages in a lightweight, headless state, the engine minimizes resource consumption while maintaining the ability to perform complex navigation and dynamic content rendering. The platform distinguishes itself through deep integration with AI-centric communication layers and advanc
This repository is a collection of node-based pipeline configurations, examples, and templates for generating AI media. It provides a workflow library and a curated gallery of blueprints designed for creating images, videos, and 3D assets using diffusion models. The project specifically offers a set of pre-configured node graphs for implementing advanced image generation and refinement techniques, with a focus on Stable Diffusion workflows. These examples demonstrate how to interconnect processing nodes to define complex generative logic without writing code. The available templates cover a
Eko is a framework for designing and deploying agentic workflows, featuring an LLM agent workflow orchestrator and a browser automation engine. It provides a server-side process manager for executing system-level operations and managing local files, alongside a human-in-the-loop agent controller for manual oversight and direction during automated decision processes. The system coordinates multi-agent collaboration through role-based partitioning and workflow orchestration, dividing complex tasks into distinct roles and managing execution handoffs. It integrates the Model Context Protocol to s
This project is a Model Context Protocol server that connects large language models to web scraping and crawling tools. It functions as a bridge, allowing LLM clients to utilize a web crawling engine and scraping utilities to extract and process web data. The server integrates a markdown web converter that transforms dynamic web pages and PDF documents into clean markdown to optimize consumption by AI models. It also provides a browser automation interface for controlling headless sessions and bypassing access restrictions. The system covers broad capabilities including large-scale website d
AgentGPT is a browser-based platform for deploying autonomous AI agents. It serves as a web-based orchestrator and self-hosted framework that allows users to configure agents that decompose high-level goals into smaller, actionable tasks for iterative execution. The system manages the full lifecycle of autonomous agents, from defining behaviors and parameters to overseeing goal-oriented task automation. It enables the deployment of agents that use a recursive loop of planning and analysis to reach a desired outcome. The platform includes a command line interface for bootstrapping the project
Azure Docs is the official technical documentation repository for Microsoft Azure, the cloud computing platform. It provides comprehensive guidance on the full spectrum of Azure services, covering everything from core infrastructure components like virtual machines, Kubernetes clusters, and serverless computing to platform services for AI, machine learning, data analytics, and storage. The documentation details how to provision, manage, and govern cloud resources at scale, including policy enforcement, identity management, and cost optimization. The documentation distinguishes Azure through i
Spider is a web-based platform designed for automated data extraction, providing a centralized framework to collect, process, and route structured information from websites. It functions as a comprehensive pipeline that manages the entire lifecycle of data gathering, from initial configuration to final storage in external databases or message queues. The platform distinguishes itself through a visual configuration interface that allows users to define extraction rules and manage scraping templates without writing custom code. It supports both static and dynamic content retrieval by integratin
Maxun is an open-source web scraping and automation platform designed to transform dynamic website content into structured data. By leveraging artificial intelligence to interpret natural language prompts, the system identifies page elements and extracts information without requiring manual selector configuration. It serves as a bridge between raw web content and intelligent workflows, providing structured outputs in formats optimized for large language model ingestion and agent-based applications. The platform distinguishes itself through its ability to handle complex, authenticated, and dyn
TestCafe is a Node.js end-to-end web testing framework used to automate browser tests with JavaScript or TypeScript. It serves as a cross-browser testing tool and a command-line execution engine designed for integration into continuous integration pipelines. The framework supports behavior-driven development by mapping human-readable Gherkin syntax to automation logic. It also includes an integrated web accessibility auditor to identify violations within web applications. The toolset covers a broad range of automation capabilities, including parallel test execution across multiple browser in
Selenium is a comprehensive browser automation framework that provides a standardized interface for controlling web browsers to perform automated tasks, user interactions, and data extraction. It functions as a cross-browser testing tool, enabling developers to execute identical automation scripts across various browser engines and operating systems to ensure consistent application behavior. By implementing the WebDriver protocol, it maps high-level automation commands to browser-specific drivers using a standardized HTTP-based wire protocol. The project distinguishes itself through its distr
Protractor is a WebDriver-based end-to-end testing framework and browser automation tool. It serves as a frontend integration test suite used to verify web application flows by simulating user behavior and executing JavaScript within a browser. The framework is specifically designed for testing Angular applications, providing specialized locators and synchronization tools that align with the framework lifecycle. It distinguishes itself through automatic test step synchronization, which pauses execution until pending page tasks are completed to ensure stable browser execution. The tool covers
Taiko is a browser automation framework and web end-to-end testing library used to perform programmatic user actions and verify application behavior. It functions as a headless browser testing tool capable of simulating real interactions and asserting page states in Chromium and Firefox. The project includes a browser interaction recorder that captures live actions and exports them as executable JavaScript automation scripts. It also serves as a web accessibility auditor, analyzing pages to detect accessibility violations and ensure compliance with inclusive design standards. The framework c
Scratch-blocks is a block-based visual programming library and framework used to create graphical programming interfaces. It provides a visual environment that translates interlocking blocks into executable code, serving as a foundation for educational coding interfaces and visual programming languages. The project is implemented as a customized extension of the Google Blockly ecosystem. It enables the development of domain-specific languages through a drag-and-drop interface, supporting the creation of custom block libraries and no-code tool development. The framework handles the structural
Playwright for Python is a browser automation framework designed for end-to-end testing, web scraping, and user interaction simulation. It functions as a headless browser controller that enables programmatic navigation, data extraction, and the execution of complex workflows across multiple rendering engines. The framework distinguishes itself through an actionability-aware interaction engine that automatically verifies element readiness before performing actions, significantly reducing test flakiness. It utilizes isolated browser contexts to maintain separate storage and cookies for parallel
Kestra is a declarative workflow orchestrator designed to manage complex task dependencies and automated processes through versioned configuration files. It functions as a distributed platform that decouples task scheduling from execution by offloading computational workloads to a fleet of worker nodes. The system uses a reactive, event-driven engine to initiate workflows automatically in response to external signals, webhooks, schedules, or file system changes. The platform distinguishes itself through a modular plugin architecture that allows for the integration of custom tasks and external
Crawlee-python is a web crawling framework for building scalable scrapers using Python. It serves as a comprehensive tool for web scraping automation, providing a system to extract structured data from websites using both lightweight HTTP requests and headless browser automation. The framework is distinguished by its anti-bot evasion capabilities, which include browser fingerprint impersonation and tiered proxy rotation to bypass detection systems and solve challenges such as Cloudflare. It also incorporates artificial intelligence for autonomous website navigation and schema-based data extra