GLM-130B is a pre-trained foundation model and bilingual large language model designed for natural language processing tasks in both English and Chinese. It functions as an autoregressive language model and text generator capable of producing long-form content and predicting missing phrases.
The model utilizes an autoregressive blank-filling architecture and a bidirectional dense transformer to process text. This approach allows the system to transition between understanding context through masked language modeling and generating sequential text using specific mask tokens.
The project covers capabilities for bilingual text generation, high-performance model inference, and large language model evaluation. It supports hardware-specific quantization to reduce memory usage and increase inference speed, alongside a configuration-driven system for measuring performance across various datasets.
