Awesome GitHub RepositoriesText Vectorizers

This project is a comprehensive framework for building and managing autonomous agent systems. It provides a unified architecture for orchestrating multi-agent societies, where specialized agents collaborate through roleplay to decompose and solve complex tasks. The system integrates language models with external environments, enabling agents to perform real-world actions through a standardized tool-calling abstraction layer. The framework distinguishes itself through its focus on iterative reasoning and data reliability. It employs automated feedback loops to refine agent outputs and self-eva

Weaviate is a cloud-native vector database and distributed vector store designed to save high-dimensional vectors alongside structured data. It functions as a hybrid search engine that combines vector similarity, keyword matching, and structured metadata filtering within a single query. The system is optimized for retrieval-augmented generation, integrating vector search with generative AI and reranking to power question-and-answer workflows. It distinguishes itself through the ability to merge semantic search with traditional keyword queries and structured metadata filters to improve result

Gensim is a natural language processing toolkit designed for large-scale text analysis and the training of semantic vector embeddings. It provides a framework for identifying latent thematic structures within document collections and calculating semantic similarity between text segments using unsupervised statistical algorithms. The project is distinguished by its ability to handle datasets that exceed available system memory through incremental corpus streaming, which processes documents one at a time from disk. It utilizes sparse vector representations and dictionary-based token mapping to

This repository is a comprehensive collection of instructional guides and practical examples for Python development, focusing on machine learning, data science, and web scraping. It provides implementations for neural networks, reinforcement learning algorithms, and deep learning architectures using PyTorch, alongside detailed manuals for scientific computing and data visualization. The project distinguishes itself by offering specialized tutorials on concurrent programming to optimize CPU performance and guides for setting up Linux development environments. It covers the implementation of ad

This project is a comprehensive knowledge base and study resource designed for mastering technical interviews. It provides structured guides, roadmaps, and curricula focused on data structures, algorithms, system design, and frontend engineering to help candidates prepare for software engineering screenings. The repository distinguishes itself by offering a holistic approach to professional advancement. Beyond technical drills, it includes a career development handbook covering resume optimization, salary benchmarking, and strategic negotiation coaching. It also provides detailed methodologie

Pattern is a Python web mining library that functions as an HTML web scraper, a natural language processing toolkit, and a network analysis tool. It provides a mathematical framework for categorizing datasets through a vector space model library. The project enables the extraction of structured data from web services and the creation of searchable web content indexes. It processes unstructured text using sentiment analysis, part-of-speech tagging, and n-gram searching. The library covers machine learning classification through the training of models using perceptron algorithms and support ve

CatBoost is a gradient boosting machine learning library used to train decision tree ensembles for regression, classification, and ranking tasks. It functions as a high-performance framework that provides a categorical data processor for transforming non-numeric features, a distributed trainer for large-scale datasets, and GPU acceleration to speed up model construction. The library distinguishes itself through native handling of categorical data and text features, removing the need for manual encoding. It includes a specialized model interpretability tool that leverages SHAP values and featu

This project is a machine learning implementation library featuring a collection of code examples that implement supervised, unsupervised, and reinforcement learning algorithms from scratch. It provides a comprehensive set of toolkits for core machine learning components, including a natural language processing toolkit, a reinforcement learning framework, and suites for data dimensionality reduction and pattern mining. The library includes specialized implementations for reinforcement learning, such as Q-Learning, Deep Q-Networks, and Actor-Critic agents. The natural language processing capab

Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo

GloVe is a distributed word representation system and a C implementation for training and using Global Vectors for word embeddings. It provides a word embedding training tool to learn numerical representations of words based on global co-occurrence statistics from a text corpus. The project includes a pre-trained word vector library learned from large web datasets, allowing for the import of these representations to perform semantic analysis without local training. It enables word vector generation to identify semantic relationships, analogies, and nearest neighbors. The system covers the fu

WantWords is an open-source multilingual reverse dictionary and semantic search engine that retrieves words from written descriptions rather than exact spellings or prefixes. It solves the tip-of-the-tongue problem by letting users describe a concept and returning matching words across Chinese and English language pairs, with support for part-of-speech filtering to narrow results to specific grammatical categories like nouns, verbs, or adjectives. The tool distinguishes itself through embedding-based semantic matching that converts user descriptions and dictionary entries into vector represen

Este proyecto es un curso de desarrollo y plan de estudios de aprendizaje centrado en la construcción de chatbots de modelos de lenguaje grande (LLM). Proporciona una serie estructurada de tutoriales para crear agentes conversacionales mediante la aplicación de procesamiento de lenguaje natural y modelos de deep learning. Los materiales incluyen un recorrido técnico para implementar redes neuronales y embeddings de palabras para manejar tareas automatizadas de preguntas y respuestas. También proporciona una guía para construir corpora de conversación a gran escala a partir de fuentes de texto externas para entrenar y evaluar sistemas de diálogo. El plan de estudios cubre técnicas centrales de análisis de texto, incluyendo tokenización y análisis sintáctico, para ayudar a los usuarios a comprender los patrones del lenguaje humano.

ZenML is an extensible machine learning orchestration framework designed to manage the end-to-end lifecycle of data pipelines and AI agent workflows. It functions as a durable orchestrator that executes machine learning tasks as directed acyclic graphs, ensuring that every step is containerized for consistent performance across local, cloud, and hybrid infrastructure. By decoupling pipeline code from underlying compute and storage backends, the platform allows developers to define infrastructure-agnostic stacks that remain portable across diverse environments. The project distinguishes itself

ZenML is an orchestration platform designed for building, deploying, and monitoring reproducible machine learning pipelines and agentic workflows. It provides a unified framework that manages the entire lifecycle of machine learning assets, from data processing and model training to the deployment of persistent inference services. By decoupling pipeline logic from underlying compute and storage, the platform enables teams to transition workflows seamlessly from local development environments to production-grade cloud infrastructure. The platform distinguishes itself through a service-oriented

This repository provides a collection of reference implementations, toolkits, and orchestration tools for training and deploying large-scale AI models on Cloud TPU hardware. It serves as a framework for managing the lifecycle of accelerator clusters, including hardware orchestration and the provisioning of high-performance compute infrastructure for machine learning workloads. The project specifically enables the pre-training of foundation models, large language models, and complex reasoning architectures through distributed training toolkits and multi-host scaling recipes. It further provide

zvec is an embedded vector database engine and indexing library designed for high-dimensional similarity search. It functions as a hybrid search engine and a retrieval-augmented generation knowledge base, allowing for the storage and retrieval of dense and sparse vectors. The system is distinguished by its hybrid retrieval pipeline, which fuses vector similarity, full-text keyword matching, and scalar metadata filtering into single query operations. It supports a plugin-based model integration system for registering custom embedding models and rerankers, as well as language bindings for nativ

This is an interactive notebook-based course that teaches machine learning from Python fundamentals through deep learning and natural language processing. It uses real datasets and multiple frameworks within a structured, hands-on curriculum that combines concise explanations with executable code cells, built-in datasets, and embedded exercise checkpoints. Learning progresses through data preparation and exploration, classical machine learning workflows, computer vision with convolutional neural networks, and natural language processing with deep learning, all delivered as a cohesive progressi

Este proyecto es un pipeline de clasificación multietiqueta diseñado para la predicción de géneros. Implementa un flujo de trabajo de aprendizaje automático que asigna múltiples etiquetas de categoría a un solo elemento procesando datos de entrada tanto textuales como visuales. El sistema utiliza extracción de características multimodal para transformar imágenes y descripciones de texto en vectores semánticos. Este proceso incluye el uso de redes preentrenadas para la extracción de características visuales y el promedio de palabras semánticas para el análisis de texto, permitiendo que el modelo integre diferentes tipos de datos en una entrada unificada. El pipeline cubre todo el ciclo de vida del aprendizaje automático, incluyendo la integración de metadatos de conjuntos de datos desde bases de datos externas y la organización de los datos en un pipeline lineal de múltiples etapas. El rendimiento se mide mediante la evaluación de métricas de verdad fundamental utilizando cálculos de precisión y exhaustividad (recall), mientras que las relaciones entre categorías se analizan mediante matrices de coocurrencia por pares.

nlpaug es una librería de aumento de datos diseñada para generar texto sintético, audio y datos de espectrogramas con el fin de mejorar la robustez de los modelos de machine learning. Funciona como un sintetizador de datos textuales y un aumentador de señales de audio, proporcionando herramientas especializadas para expandir datasets mediante diversos métodos de transformación. El proyecto se distingue por su capacidad para orquestar flujos de trabajo complejos mediante un orquestador de pipelines, que permite encadenar múltiples funciones de aumento de forma secuencial o aleatoria. Soporta síntesis de texto sofisticada mediante back-translation, embeddings de palabras contextuales e integración de modelos de lenguaje pre-entrenados, además de ofrecer aumento de imágenes de espectrogramas mediante enmascaramiento de tiempo y frecuencia. La librería cubre un amplio rango de capacidades, incluyendo modificación de señales de audio con inyección de ruido y pitch shifting, alteraciones de texto basadas en reglas para simular errores tipográficos y ortográficos, y expansión de datasets mediante generación de oraciones y sustitución semántica. También proporciona controles para el volumen de aumento y filtrado de objetivos mediante expresiones regulares para proteger tokens específicos de la modificación.

DeepKE es un kit de herramientas y framework de extracción de conocimiento diseñado para transformar texto no estructurado en grafos de conocimiento estructurados. Proporciona una tubería para identificar y clasificar entidades nombradas, relaciones semánticas y eventos, convirtiendo conjuntos de datos crudos en triples estructurados. El proyecto utiliza modelos de lenguaje grandes como llamadores de herramientas a través de un protocolo de contexto estandarizado para impulsar procesos automatizados de extracción de datos. Admite la extracción basada en esquemas en múltiples dominios y texto bilingüe, empleando la extracción conjunta de entidades y relaciones para identificar componentes en una única salida estructurada. El kit de herramientas incluye capacidades para el entrenamiento y ajuste fino de modelos, optimización de hiperparámetros y preparación de datos mediante supervisión distante y etiquetado automático de relaciones. También cuenta con entrenamiento distribuido en GPU, optimización de memoria de modelos mediante cuantización y la capacidad de desplegar modelos entrenados como servicios de inferencia a través de endpoints de API.