How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
Rust native ready-to-use NLP pipelines and transformer-based models (BERT, DistilBERT, GPT2,...)
The main features of guillaume-be/rust-bert are: Artificial Intelligence, Machine Learning Frameworks, Natural Language Processing, Rust NLP Libraries.
Projects with overlapping indexed features include: huggingface/tokenizers — This project is a high-performance library for converting raw text into tokens and IDs for machine learning models. It… ericlbuehler/mistral.rs — mistral.rs is an inference engine for large language models that runs locally and exposes models behind OpenAI and… apache/incubator-mxnet — Apache MXNet is a deep learning framework and distributed machine learning library designed for training and deploying… arc53/docsgpt — DocsGPT is a retrieval-augmented generation platform and private knowledge base used to build AI agents that perform… edobashira/speech-language-processing — A curated list of speech and natural language processing resources. chelsea0x3b/dfdx — Deep learning in Rust, with shape checked tensors and neural networks.
This project is a high-performance library for converting raw text into tokens and IDs for machine learning models. It functions as a fast text encoder and a text preprocessing pipeline designed to transform strings into numerical representations with high throughput for research and production. The library includes a subword tokenizer trainer used to analyze text datasets and create custom vocabularies using algorithms such as byte-pair encoding and wordpiece. It provides capabilities for subword vocabulary training and text alignment, allowing character offsets to be tracked during normaliz
Deep learning in Rust, with shape checked tensors and neural networks
Apache MXNet is a deep learning framework and distributed machine learning library designed for training and deploying neural networks across distributed systems, mobile devices, and hardware accelerators. It functions as a cross-platform runtime and a dynamic dataflow scheduler that optimizes neural network execution. The framework provides a multi-language API, enabling the development of machine learning models using Python, R, Julia, Scala, Go, and JavaScript. It supports high-performance model training and the scaling of workloads across multiple GPUs and machines. The system covers cap
DocsGPT is a retrieval-augmented generation platform and private knowledge base used to build AI agents that perform grounded search and analysis. It functions as a multi-model AI orchestrator and enterprise agent builder, allowing for the integration of various local and cloud language models to customize reasoning and text generation. The project provides a visual environment for developing automated assistants using conditional logic and third-party API connectivity. It enables the creation of private AI agents capable of performing enterprise search and detailed document analysis using pr