xuxueli/xxl-job
Xxl Job
xxl-job is a distributed task scheduling platform and job orchestrator designed to manage and trigger timed jobs across a cluster of remote executor nodes. It provides a centralized system for scheduling tasks, linking dependent jobs, and managing complex execution lifecycles through a relational database that persists configurations and logs.
The platform distinguishes itself through a web-based interface for cron job management, allowing users to create and update scheduled tasks without modifying source code. It supports cross-language task execution by triggering logic on third-party executors via a standardized web interface and enables distributed data sharding to broadcast tasks across all cluster executors.
The system covers broad capability areas including task routing and load balancing, real-time remote monitoring of execution logs, and automated failure handling with retry limits. It also incorporates access control for user permissions and auditing of system activity to track operational changes.
Features
- Job Schedulers - Provides a centralized job scheduler to orchestrate background tasks and automate execution based on time-based schedules.
- Distributed Task Schedulers - Serves as a centralized system for managing and triggering timed jobs across a distributed cluster of worker nodes.
- Scheduler-Worker Topologies - Employs a central scheduling server to manage task distribution while remote executor nodes handle the actual job execution.
- Job State Persistence - Persists job metadata and execution logs in a relational database to ensure reliability across system failures.
- Relational Database Persistence - Utilizes relational database persistence to maintain consistent state for job configurations and execution history.
- Cron Scheduling - Provides a web-based interface for defining and managing task execution using cron expressions.
- Cron Trigger Management - Provides a system for creating and managing cron-based triggers to automate task execution timing.
- Management Interfaces - Offers a dedicated web interface for creating and updating scheduled tasks without modifying source code.
- Multi-Strategy Task Triggers - Executes jobs using multiple timing patterns such as cron expressions, fixed intervals, or external API events.
- Distributed Job Orchestration - Provides a framework for linking dependent tasks and managing complex job lifecycles across a distributed environment.
- Distributed Task Orchestrators - Coordinates and scales parallel task execution and dependencies across distributed computing resources.
- HTTP Task Dispatching - Communicates with distributed executors via a web interface to trigger tasks and receive real-time status updates.
- Job Dependency Coordination - Manages complex asynchronous workflows through parent-child job dependencies and execution lifecycles.
- Job Sharding - Implements techniques for splitting a single logical task into parallel shards for distributed execution across a cluster.
- Data Partitioning - Splits workloads across multiple executors by assigning unique indices to nodes for parallel data processing.
- Task-Based Shard Broadcasting - Broadcasts tasks across multiple cluster executors so each instance processes a unique subset of a larger dataset.
- Online Scripting Environments - Provides a web-based editor to write and deploy task scripts without needing to restart active services.
- Task Dependency Management - Defines execution order and prerequisites for tasks, ensuring child processes start only after parents complete successfully.
- Distributed Job Executors - Integrates with external distributed executors to enable cross-language task execution via a standardized interface.
- Capacity Scaling - Supports increasing or decreasing the number of executor nodes to meet task resource demands dynamically.
- Distributed Execution Gateways - Enables triggering operations on third-party executors written in different programming languages via a standardized web interface.
- Cluster Load Balancing - Distributes computational workloads across cluster nodes using specific routing algorithms to maximize hardware utilization.
- Asynchronous Task Execution - Implements mechanisms for executing long-running operations with real-time progress polling and result retrieval from remote executors.
- Failure Handling Policies - Implements failure handling policies including automated retries and error alerts when tasks fail.
- Clustered Task Distribution - Distributes scheduled jobs across a cluster of executors using routing strategies to ensure high availability.
- Remote Executor Frameworks - Provides a system for triggering logic on external servers through a web interface with real-time execution monitoring.
- Task Status Monitors - Provides real-time tracking of execution status and remote log viewing for troubleshooting failures.
- Job Status Monitoring - Tracks real-time execution status and provides rolling logs from remote executors to verify task completion.
