2 个仓库
Awesome GitHub RepositoriesDocument Batching for Embedding
Splitting oversized document collections into smaller groups that fit within an embedding model's token limit.
Distinct from Document Embedding Generations: Distinct from Document Embedding Generations: focuses on batching documents to respect token limits, not parallel generation at scale.
Explore 2 awesome GitHub repositories matching data & databases · Document Batching for Embedding. Refine with filters or upvote what's useful.
Awesome Document Batching for Embedding GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- spring-projects/spring-ai
spring-projects/spring-ai
9,001在 GitHub 上查看Spring AI is an application framework for Java that provides a portable, fluent API for integrating AI models, tools, and vector stores into applications. It wraps multiple AI providers behind a common interface, allowing developers to switch between chat, embedding, image, and speech models without changing application code. The framework includes a chainable chat client API similar to WebClient or RestClient, supports both synchronous and streaming interactions, and offers structured output conversion that transforms unstructured AI responses into strongly-typed Java objects. The framework
Splits large document collections into smaller batches to fit within embedding model token limits.
Javaartificial-intelligencejavaspring-ai - timescale/pgai
timescale/pgai
5,802在 GitHub 上查看pgai 是一个 PostgreSQL AI 工具包和框架，旨在将大语言模型和向量嵌入直接集成到数据库中。它充当了在标准数据库查询中执行机器学习模型请求和进行文本转 SQL 翻译的桥梁。 该项目提供了一个自动化的向量嵌入流水线，负责处理来自表和非结构化文档的文本加载、解析和分块。该系统利用后台工作进程在源数据发生变化时自动同步嵌入，并包含用于构建检索增强生成（RAG）应用和语义搜索引擎的专用工具。 该工具包涵盖了广泛的功能领域，包括利用 OCR 处理非结构化数据、创建将数据库模式映射到自然语言的语义目录，以及通过向量索引和结果重排序实现高性能相似度搜索。它还支持通过 SQL 调用外部模型，从而实现数据增强、分类和内容审核。
Manages large-scale batch processing of embeddings with built-in resilience against failures and API rate limits.
PLpgSQL
