1 repo
Tools that split large documents into smaller segments for use in language models.
Explore 1 awesome GitHub repository matching data & databases · Document Chunking Utilities. Refine with filters or upvote what's useful.
Pathway is a high-performance data processing framework designed for building unified batch and streaming pipelines. It functions as an orchestrator for complex data transformations, utilizing a differential dataflow engine to process updates incrementally. By treating static datasets and continuous event streams with