Awesome GitHub RepositoriesText Vectorizers

This project is a comprehensive framework for building and managing autonomous agent systems. It provides a unified architecture for orchestrating multi-agent societies, where specialized agents collaborate through roleplay to decompose and solve complex tasks. The system integrates language models with external environments, enabling agents to perform real-world actions through a standardized tool-calling abstraction layer. The framework distinguishes itself through its focus on iterative reasoning and data reliability. It employs automated feedback loops to refine agent outputs and self-eva

Weaviate is a cloud-native vector database and distributed vector store designed to save high-dimensional vectors alongside structured data. It functions as a hybrid search engine that combines vector similarity, keyword matching, and structured metadata filtering within a single query. The system is optimized for retrieval-augmented generation, integrating vector search with generative AI and reranking to power question-and-answer workflows. It distinguishes itself through the ability to merge semantic search with traditional keyword queries and structured metadata filters to improve result

Gensim is a natural language processing toolkit designed for large-scale text analysis and the training of semantic vector embeddings. It provides a framework for identifying latent thematic structures within document collections and calculating semantic similarity between text segments using unsupervised statistical algorithms. The project is distinguished by its ability to handle datasets that exceed available system memory through incremental corpus streaming, which processes documents one at a time from disk. It utilizes sparse vector representations and dictionary-based token mapping to

This repository is a comprehensive collection of instructional guides and practical examples for Python development, focusing on machine learning, data science, and web scraping. It provides implementations for neural networks, reinforcement learning algorithms, and deep learning architectures using PyTorch, alongside detailed manuals for scientific computing and data visualization. The project distinguishes itself by offering specialized tutorials on concurrent programming to optimize CPU performance and guides for setting up Linux development environments. It covers the implementation of ad

This project is a comprehensive knowledge base and study resource designed for mastering technical interviews. It provides structured guides, roadmaps, and curricula focused on data structures, algorithms, system design, and frontend engineering to help candidates prepare for software engineering screenings. The repository distinguishes itself by offering a holistic approach to professional advancement. Beyond technical drills, it includes a career development handbook covering resume optimization, salary benchmarking, and strategic negotiation coaching. It also provides detailed methodologie

Pattern is a Python web mining library that functions as an HTML web scraper, a natural language processing toolkit, and a network analysis tool. It provides a mathematical framework for categorizing datasets through a vector space model library. The project enables the extraction of structured data from web services and the creation of searchable web content indexes. It processes unstructured text using sentiment analysis, part-of-speech tagging, and n-gram searching. The library covers machine learning classification through the training of models using perceptron algorithms and support ve

CatBoost is a gradient boosting machine learning library used to train decision tree ensembles for regression, classification, and ranking tasks. It functions as a high-performance framework that provides a categorical data processor for transforming non-numeric features, a distributed trainer for large-scale datasets, and GPU acceleration to speed up model construction. The library distinguishes itself through native handling of categorical data and text features, removing the need for manual encoding. It includes a specialized model interpretability tool that leverages SHAP values and featu

This project is a machine learning implementation library featuring a collection of code examples that implement supervised, unsupervised, and reinforcement learning algorithms from scratch. It provides a comprehensive set of toolkits for core machine learning components, including a natural language processing toolkit, a reinforcement learning framework, and suites for data dimensionality reduction and pattern mining. The library includes specialized implementations for reinforcement learning, such as Q-Learning, Deep Q-Networks, and Actor-Critic agents. The natural language processing capab

Kreuzberg is a document extraction engine that converts PDFs, Office files, images, and over 90 other formats into clean, structured text and metadata. It is built around a compiled Rust core that can be used as a native library, a command-line tool, a REST API server, or a WebAssembly module for browser-based processing. The system is designed to run entirely on self-hosted infrastructure, with no data leaving the user's environment. What distinguishes Kreuzberg is its breadth of integration surfaces and its pipeline architecture. It exposes extraction capabilities through native bindings fo

GloVe is a distributed word representation system and a C implementation for training and using Global Vectors for word embeddings. It provides a word embedding training tool to learn numerical representations of words based on global co-occurrence statistics from a text corpus. The project includes a pre-trained word vector library learned from large web datasets, allowing for the import of these representations to perform semantic analysis without local training. It enables word vector generation to identify semantic relationships, analogies, and nearest neighbors. The system covers the fu

WantWords is an open-source multilingual reverse dictionary and semantic search engine that retrieves words from written descriptions rather than exact spellings or prefixes. It solves the tip-of-the-tongue problem by letting users describe a concept and returning matching words across Chinese and English language pairs, with support for part-of-speech filtering to narrow results to specific grammatical categories like nouns, verbs, or adjectives. The tool distinguishes itself through embedding-based semantic matching that converts user descriptions and dictionary entries into vector represen

Ce projet est un cours de développement et un programme d'apprentissage axé sur la construction de chatbots basés sur de grands modèles de langage (LLM). Il fournit une série structurée de tutoriels pour créer des agents conversationnels via l'application du traitement du langage naturel et des modèles de deep learning. Les matériaux incluent une procédure technique pour implémenter des réseaux de neurones et des embeddings de mots pour gérer des tâches automatisées de questions-réponses. Il fournit également un guide pour construire des corpus de conversation à grande échelle à partir de sources textuelles externes afin d'entraîner et d'évaluer des systèmes de dialogue. Le programme couvre les techniques fondamentales d'analyse de texte, incluant la tokenisation et l'analyse syntaxique, pour aider les utilisateurs à comprendre les modèles de langage humain.

ZenML is an extensible machine learning orchestration framework designed to manage the end-to-end lifecycle of data pipelines and AI agent workflows. It functions as a durable orchestrator that executes machine learning tasks as directed acyclic graphs, ensuring that every step is containerized for consistent performance across local, cloud, and hybrid infrastructure. By decoupling pipeline code from underlying compute and storage backends, the platform allows developers to define infrastructure-agnostic stacks that remain portable across diverse environments. The project distinguishes itself

ZenML is an orchestration platform designed for building, deploying, and monitoring reproducible machine learning pipelines and agentic workflows. It provides a unified framework that manages the entire lifecycle of machine learning assets, from data processing and model training to the deployment of persistent inference services. By decoupling pipeline logic from underlying compute and storage, the platform enables teams to transition workflows seamlessly from local development environments to production-grade cloud infrastructure. The platform distinguishes itself through a service-oriented

This repository provides a collection of reference implementations, toolkits, and orchestration tools for training and deploying large-scale AI models on Cloud TPU hardware. It serves as a framework for managing the lifecycle of accelerator clusters, including hardware orchestration and the provisioning of high-performance compute infrastructure for machine learning workloads. The project specifically enables the pre-training of foundation models, large language models, and complex reasoning architectures through distributed training toolkits and multi-host scaling recipes. It further provide

zvec is an embedded vector database engine and indexing library designed for high-dimensional similarity search. It functions as a hybrid search engine and a retrieval-augmented generation knowledge base, allowing for the storage and retrieval of dense and sparse vectors. The system is distinguished by its hybrid retrieval pipeline, which fuses vector similarity, full-text keyword matching, and scalar metadata filtering into single query operations. It supports a plugin-based model integration system for registering custom embedding models and rerankers, as well as language bindings for nativ

This is an interactive notebook-based course that teaches machine learning from Python fundamentals through deep learning and natural language processing. It uses real datasets and multiple frameworks within a structured, hands-on curriculum that combines concise explanations with executable code cells, built-in datasets, and embedded exercise checkpoints. Learning progresses through data preparation and exploration, classical machine learning workflows, computer vision with convolutional neural networks, and natural language processing with deep learning, all delivered as a cohesive progressi

Ce projet est un pipeline de classification multi-étiquettes conçu pour la prédiction de genre. Il implémente un workflow de machine learning qui assigne plusieurs étiquettes de catégorie à un seul élément en traitant à la fois des données textuelles et visuelles. Le système utilise l'extraction de caractéristiques multimodales pour transformer les images et les descriptions textuelles en vecteurs sémantiques. Ce processus inclut l'utilisation de réseaux pré-entraînés pour l'extraction de caractéristiques visuelles et la moyenne sémantique des mots pour l'analyse textuelle, permettant au modèle d'intégrer différents types de données dans une entrée unifiée. Le pipeline couvre l'intégralité du cycle de vie du machine learning, incluant l'intégration de métadonnées de jeux de données provenant de bases externes et l'organisation des données dans un pipeline linéaire multi-étapes. La performance est mesurée par une évaluation basée sur la vérité terrain avec des calculs de précision et de rappel, tandis que les relations entre catégories sont analysées via des matrices de co-occurrence par paires.

nlpaug est une bibliothèque d'augmentation de données conçue pour générer du texte, de l'audio et des données de spectrogramme synthétiques afin d'améliorer la robustesse des modèles de machine learning. Elle fonctionne comme un synthétiseur de données textuelles et un augmentateur de signaux audio, fournissant des outils spécialisés pour étendre les jeux de données via diverses méthodes de transformation. Le projet se distingue par sa capacité à orchestrer des flux de travail complexes à l'aide d'un orchestrateur de pipeline, permettant d'enchaîner plusieurs fonctions d'augmentation de manière séquentielle ou aléatoire. Il prend en charge la synthèse de texte sophistiquée via la rétro-traduction, les plongements lexicaux contextuels et l'intégration de modèles de langage pré-entraînés, tout en offrant une augmentation d'images de spectrogrammes par masquage temporel et fréquentiel. La bibliothèque couvre un large éventail de capacités, incluant la modification de signaux audio avec injection de bruit et changement de hauteur (pitch shifting), des altérations de texte basées sur des règles pour simuler des fautes de frappe et d'orthographe, et l'expansion de jeux de données par génération de phrases et substitution sémantique. Elle fournit également des contrôles pour le volume d'augmentation et le filtrage des cibles via des expressions régulières pour protéger certains jetons de toute modification.

DeepKE est une boîte à outils et un framework d'extraction de connaissances conçu pour transformer du texte non structuré en graphes de connaissances structurés. Il fournit un pipeline pour identifier et classer les entités nommées, les relations sémantiques et les événements, convertissant des jeux de données bruts en triplets structurés. Le projet utilise de grands modèles de langage comme appelants d'outils via un protocole de contexte standardisé pour piloter des processus d'extraction de données automatisés. Il prend en charge l'extraction pilotée par schéma à travers de multiples domaines et le texte bilingue, employant une extraction conjointe d'entités et de relations pour identifier les composants dans une sortie structurée unique. La boîte à outils inclut des capacités pour l'entraînement et le réglage fin de modèles, l'optimisation des hyperparamètres et la préparation des données via une supervision distante et l'étiquetage automatique des relations. Elle dispose également d'un entraînement GPU distribué, d'une optimisation de la mémoire des modèles via la quantification, et de la capacité de déployer des modèles entraînés en tant que services d'inférence via des endpoints API.