Open-source alternatives to Laserembeddings

This repository is a comprehensive collection of instructional guides and practical examples for Python development, focusing on machine learning, data science, and web scraping. It provides implementations for neural networks, reinforcement learning algorithms, and deep learning architectures using PyTorch, alongside detailed manuals for scientific computing and data visualization. The project distinguishes itself by offering specialized tutorials on concurrent programming to optimize CPU performance and guides for setting up Linux development environments. It covers the implementation of ad

This project is a transformer-based language model and autoregressive text generator designed to predict the next token in a sequence to produce human-like prose and synthetic text. It functions as a large language model that utilizes a transformer architecture to learn linguistic patterns from large datasets for unsupervised multitask learning. The repository provides a distribution of pre-trained weights, enabling natural language processing tasks without requiring additional training. This allows the model to perform zero-shot task generalization by applying learned patterns to new tasks.

Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo

gpt-neox is a distributed training system and framework for building large-scale autoregressive language models. It implements the transformer architecture and provides a toolkit for training models with billions of parameters by distributing weights across compute clusters. The framework distinguishes itself through extensive support for distributed model parallelism, including pipeline and sequence parallelism, to overcome single-device memory limits. It further supports sparse model architectures using a mixture of experts system with Sinkhorn-based routing. The project covers a broad ran

LAnguage Model Analysis

PyTorch original implementation of Cross-lingual Language Model Pretraining.

ColossalAI is a distributed deep learning framework designed for training and deploying massive artificial intelligence models across clusters of hardware accelerators. It functions as a parallel computing engine that partitions model workloads and data across multiple processors to maximize memory efficiency and throughput. The platform distinguishes itself through a comprehensive suite of parallelization strategies, including multi-dimensional tensor parallelism and pipeline-based model parallelism, which segment neural network layers and stages across devices. To support large-scale genera

This project is a PyTorch transformer model library and pre-trained model framework. It serves as a deep learning model hub and multimodal inference engine, providing a centralized system for loading, executing, and fine-tuning state-of-the-art model checkpoints. The library focuses on multimodal machine learning, enabling predictions across text, vision, and audio data. It provides specialized capabilities for model framework interoperability, allowing the conversion of weights and definitions between different deep learning libraries. The platform covers the full model lifecycle, including

Transformers is a comprehensive library for machine learning that provides a unified interface for training, fine-tuning, and deploying transformer-based models. It supports a wide range of tasks, including text classification, language modeling, question answering, and sequence-to-sequence translation, while offering specialized architectures for both text and vision processing. The framework includes tools for managing the entire model lifecycle, from data preprocessing and tokenization to distributed training and inference. The library features extensive support for model optimization and

A Model for Natural Language Attack on Text Classification and Inference

Generating multiple choice questions from text using Machine Learning.

Generate Hacker News Comments from Titles

GPT2-Chinese is a Chinese language model implementation based on the GPT-2 architecture. It provides a causal language model trainer and a natural language generation tool designed for training and generating human-like Chinese text sequences. The system integrates a BERT tokenizer to process Chinese corpora into manageable units for machine learning. It enables the development of predictive text models that can generate specific patterns, such as news or poetry, through prompt-based text completion. The project covers a full workflow including text tokenization, model training using a trans

bert中文分类实践

A List of Big Models

LARK is a development toolkit for training, fine-tuning, and deploying large language models and multimodal models based on PaddlePaddle. It functions as a comprehensive framework that includes an LLM training orchestrator, an inference server, and a multimodal model framework for processing text, image, and video inputs. The project features a retrieval-augmented generation system for building conversational applications that integrate web search and private knowledge bases. It provides specific capabilities for multimodal reasoning and complex logic, enabling the extraction of structured da

inferring sql queries from plain-text questions about tables

Fine-tuned pre-trained GPT2 for custom topic specific text generation. Such system can be used for Text Augmentation.

使用预训练语言模型ALBERT做中文NER

Benchmarking synthetic data generation methods.

标注数据，可以说是AI模型训练里最艰巨的一项工作了。自然语言处理的数据标注更是需要投入大量人力。相对计算机视觉的图像标注，文本的标注通常没有准确的标准答案，对句子理解也是因人而异，让这项工作更是难上加难。 但是！谷歌最近发布的BERT大大的解决了这个问题！根据我们的实验，BERT在文本多分类的任务中，能在极小的数据下，带来显著的分类准确率提升。并且，实验主要对比的是仅仅5个月前发布的State of the art 语言模型迁移学习模型 - ULMFiT (https://arxiv.org/abs/1801.06146)， 结果有着明显的提升。

一行代码使用BERT生成句向量，BERT做文本分类、文本相似度计算

This Word Does Not Exist

Awesome papers on Language-Model-as-a-Service (LMaaS)

Dataset for couplets. 70万条对联数据库。