Awesome GitHub RepositoriesParallel Task Execution

Colly is a web scraping framework and concurrent crawler written in Go. It provides a system for traversing web pages, following links, and extracting structured data from HTML and XML documents. The framework includes a distributed scraping engine designed to spread data collection tasks across multiple instances to increase throughput. It ensures compliance with website owner policies by automatically reading and respecting robots.txt files. The system manages request lifecycles through domain-based rate limiting, concurrency controls, and session management via a stateful cookie jar. It s

Codon is an LLVM-based Python compiler and statically typed implementation that translates source code into optimized machine instructions. It functions as a high-performance numerical backend and a GPU computing framework designed to remove runtime overhead. The project implements a compiled alternative to NumPy, translating array logic directly into machine code. It differentiates itself by generating specialized hardware kernels for graphics processors and utilizing static type inference to enable aggressive machine-code optimization. The system provides capabilities for parallel workload

Edict is a multi-agent orchestration system and framework designed to coordinate specialized large language model agents. It functions as a workflow designer and orchestrator that decomposes complex objectives into structured plans, using directed acyclic graphs and role-based hierarchies to execute sub-tasks. The system is distinguished by its event-driven architecture, utilizing a publish-subscribe event bus and transactional outbox to manage agent communications and task transitions. It features a dedicated skill management system that allows for the importation, updating, and sandboxed ex

fp-ts is a TypeScript library that brings pure functional programming patterns to the language through algebraic data types, type class abstractions, and composable combinators. It provides foundational data types like Option for optional values, Either for typed error handling, and Task for lazy asynchronous computations, all designed to make invalid states unrepresentable and side effects explicit. The library is built on category theory concepts, offering type classes such as Functor, Applicative, Monad, Semigroup, and Monoid with lawful instances for common data structures. The library di

oneAPI Threading Building Blocks (oneTBB)

oneTBB 是一个 C++ 并行库和框架，旨在为应用程序添加多核并行性。它提供了一种基于任务的并行模型，将逻辑计算任务映射到可用的硬件核心，从而消除了手动线程管理的需要。 该库作为多核扩展工具，利用通用模板跨处理器扩展数据并行操作，以实现可移植的性能。它采用基于任务的框架，确保计算工作负载分布在硬件资源上。 该项目涵盖共享内存并行、多核任务调度和数据并行扩展。它利用工作窃取（work-stealing）任务调度器、递归范围拆分和动态负载均衡，在运行时管理核心间的工作分配。

Arrow is a functional programming library for Kotlin that provides tools for implementing data-oriented programming patterns. It serves as a framework for typed error handling, a concurrency toolkit, and a library for the manipulation of immutable data. The project distinguishes itself through specialized capabilities for managing application failures using explicit types instead of exceptions and implementing resilience patterns such as circuit breakers and retry policies for distributed services. It also provides optics to update and query deeply nested immutable data structures without man

OCaml is a strongly typed functional language featuring a sophisticated type system and a focus on safety and expressiveness. It provides a comprehensive compiling toolchain that transforms source code into either portable bytecode or high-performance native binaries. The project is distinguished by a shared memory parallel runtime that executes computations across multiple processor cores using domains, and an algebraic effect system for managing side effects and control flow through execution context handlers. It also includes a dedicated parser generator to automatically create lexers and

Open Multi-Agent is a TypeScript framework for multi-agent orchestration that decomposes natural language goals into a runtime-generated directed acyclic graph of tasks. It functions as a task orchestrator and workflow state manager, coordinating multiple AI models to execute parallel and sequential operations. The framework is distinguished by a proposer-judge consensus protocol used to validate agent outputs through a quorum of agreement. It employs provider-agnostic model routing to assign specific models to tasks based on roles or execution phases and utilizes state-based workflow checkpo

Moleculer is a Node.js microservices framework designed for building distributed systems. It functions as a distributed service broker, task orchestrator, and service mesh framework, enabling a decentralized architecture with built-in service discovery and load balancing. The project differentiates itself through a pluggable transport layer supporting protocols such as NATS, Redis, TCP, and Kafka, as well as a dedicated microservices API gateway that maps external HTTP and WebSocket requests to internal service actions. It includes built-in fault tolerance mechanisms, including circuit breake

Elvish is a shell that combines interactive command-line use with a structured scripting language, designed to make both everyday terminal work and automation tasks more predictable and readable. It parses, compiles, and executes code in three phases, catching syntax and variable errors before any code runs, and it aborts execution on command failure by default to prevent silent errors. The shell introduces value-oriented pipelines that pass structured data like lists, maps, and closures between commands, preserving types without serialization. It also mixes traditional byte streams with thes

vcmi 是《魔法门之英雄无敌 III》的开源游戏引擎，重现了原版策略游戏的玩法、逻辑和渲染。它作为一个跨平台游戏运行时，支持 Windows、Linux、macOS、Android 和 iOS，提供用于战略世界地图导航和战术战斗的回合制策略模拟器。 该项目构建为可模组化的游戏框架，具有数据驱动的模组系统和沙盒 Lua 脚本集成。这些允许通过外部配置文件和模式验证来定义自定义游戏逻辑、法术效果，并添加新的阵营、角色和资产。 该引擎包含一个多人游戏状态同步器，通过二进制序列化在客户端之间协调同步回合和游戏状态更新。它还集成了用于策略模拟的 AI、专用的地图创作与生成系统，以及用于资产覆盖和本地化的工具。 该软件支持多平台构建配置，并提供用于提取游戏数据和调试游戏状态的工具。

Leaf 是一个用 Go 编写的游戏服务器框架，专为构建多人游戏后端而设计。它提供模块化的后端架构，将服务器逻辑组织为独立的模块，并包含一个用于管理有序、延迟或循环任务的并发任务调度器。 该框架具有 TCP 和 WebSocket 服务器，可通过单一接口管理并发连接。它集成了一个消息路由器，能够解码 Protobuf 和 JSON 数据，将传入的网络包映射到特定的内部服务器模块。 该系统包含多协议网络路由、多核工作负载分配和系统事件日志记录功能。它还提供将 CSV 配置文件加载到内存索引结构中以进行数据查询的实用工具。

Inngest is a durable execution framework and event-driven automation engine designed to orchestrate background workflows. It enables developers to build resilient, stateful processes by memoizing function steps, ensuring that long-running tasks can automatically resume from the last successful operation after failures, timeouts, or infrastructure restarts. The platform distinguishes itself through its event-driven architecture, which uses a schema-validated bus to trigger functions and coordinate complex, multi-step logic. It employs an onion-model middleware approach for cross-cutting concer

GraphQL-Ruby is a Ruby library for building GraphQL APIs with a strongly typed schema and a dedicated query execution engine. It provides a comprehensive framework for mapping application objects to a formal type system, enabling structured data fetching through defined resolvers. The project distinguishes itself with advanced performance and delivery mechanisms, including a data loader for batching and caching to prevent N+1 query patterns. It supports high-performance data delivery through incremental response streaming, deferred query responses, and parallel data fetching using fibers. Add

SparkInternals 是一份技术参考和架构指南，详细介绍了 Apache Spark 分布式计算引擎的内部设计和实现。它作为大数据引擎分析的研究资料，重点关注系统如何管理集群执行以及驱动节点（Driver）、执行器（Executor）和工作节点（Worker）之间的交互。 该项目详细分解了逻辑计划如何转换为物理执行阶段。它专门分析了数据 Shuffle 操作、内存管理以及分布式作业调度协调的机制。 该文档涵盖了广泛的分布式计算功能，包括查询执行规划、数据依赖管理和内存缓存策略。它还研究了任务分配、并行执行以及用于故障恢复和数据持久化的过程。

FluidX3D 是一个 GPU 加速的计算流体动力学（CFD）求解器和基于体素的流体模拟器。它利用格子玻尔兹曼方法（LBM）通过支持图形处理器和 CPU 的 OpenCL 硬件抽象来模拟气体和液体行为、压力和速度。 该系统专注于使用流体体积法（VOF）进行自由表面建模和液滴的多相流模拟。它包括用于建模热传递和热对流的热模拟器，以及用于计算物理对象上的升力、阻力和扭矩的空气动力学力分析工具。 该软件提供了一套全面的模拟功能，包括用于双向流体相互作用的拉格朗日粒子追踪、通过周期性重体素化的旋转物体动画，以及用于高雷诺数的湍流稳定。它通过三角网格几何导入处理数据，并支持多种可视化方法，包括行进立方体（Marching Cubes）表面提取、内存光栅化和视频渲染。 性能通过多 GPU 域分解和半精度内存压缩进行管理。

Radash 是一个 TypeScript 函数式实用工具库，专为数据转换、验证和工作流管理而设计。它提供了一系列类型化的辅助函数来操作数组、对象和字符串。 该项目具有用于管理 Promise、重试和并发限制的异步工作流工具包，以及一套用于验证原始类型、对象和 Promise 的布尔验证器。它还实现了函数式编程模式，如函数组合、偏函数应用和错误优先响应模式。 该库涵盖了广泛的功能，包括具有深克隆和合并的对象状态管理、复杂的数组操作和字符串格式化。它还提供了用于执行优化的工具，如速率限制和结果缓存。

statsforecast 是一个高性能统计时间序列预测库，旨在生成点预测和预测区间。它作为一个分布式时间序列框架，利用基于 C 的预测引擎和自动模型选择器来识别并拟合数据集中每个唯一序列的最佳统计模型。该系统还包括一个时间序列异常检测器，通过将观测值与概率预测区间进行比较来识别异常数据点。 该项目的特色在于其处理数百万个独立序列的大规模并行预测的能力。它通过分布式计算框架、多核并行执行和加速核心 ARIMA 及指数平滑逻辑的编译 C 内核来实现这一点。该系统进一步利用长格式数据布局和惰性求值数据流水线来优化大规模处理，以减少内存开销。 该库提供了一套全面的模型，包括 AutoARIMA、用于间歇性或季节性需求的各种指数平滑方法、Theta 分解以及用于金融风险的 GARCH 波动率建模。它涵盖了更广泛的功能领域，例如带有外生变量的多元预测、时间序列分解以及通过历史交叉验证和滑动窗口分析进行模型评估。 该库与 Polars 等高性能数据结构集成，并提供将保存的模型作为 REST 端点提供服务以进行网络可访问预测的实用程序。

This repository is a collection of JavaScript coding challenges and a comprehensive interview guide. It provides reference implementations and educational examples designed to help developers master language fundamentals and prepare for technical interviews. The project covers a wide array of specialized implementations, including functional programming patterns like currying and partial application, as well as asynchronous patterns for concurrency control. It also includes practical examples of Document Object Model manipulation and the implementation of common data structures such as stacks