awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
facebookresearch avatar

facebookresearch/LASERArchived

0
View on GitHub↗

LASER

LASER is a cross-lingual sentence embedding library and multilingual text encoder. It functions as a parallel text mining tool that maps sentences from multiple languages into a shared vector space for similarity and classification tasks.

The system converts raw text into fixed-length embeddings, enabling the discovery of translation pairs by calculating the vector distance between sentences. This shared representation allows for cross-lingual document classification, where a model trained on one language can be used to categorize documents in another.

The library includes a sentence-piece tokenizer to split multilingual strings into subword units. It also provides mechanisms for pre-trained model management, allowing users to download and store language models locally for offline embedding generation.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Features

  • Cross-Lingual Alignment - Provides a shared vector space to compare meanings across different languages regardless of the original script.
  • Shared Latent Spaces - Maps sentences from multiple languages into a single coordinate system for semantic comparison.
  • Multilingual NLP - Prepares multilingual text for machine learning tasks through tokenization and model management.
  • Multilingual Encodings - Converts raw text into fixed-length embeddings using pre-trained multilingual models.
  • Sentence Embeddings - Maps sentences from multiple languages into a shared vector space for similarity and classification.
  • Cross-Lingual Transfer - Enables classification tasks in one language using a model trained in another via shared embeddings.
  • Fixed-Length Vector Encodings - Transforms variable-length text sequences into single constant-size vectors for efficient similarity comparisons.
  • Text Tokenization - Splits raw text into smaller units using sentence-piece models for encoding.
  • Subword Tokenization - Uses subword tokenization to handle unseen words and diverse language scripts.
  • Vector Similarity Search - Calculates angular distance between high-dimensional vectors to identify translation pairs.
  • Parallel Corpus Mining - Discovers translation pairs across different languages by calculating vector distance between embeddings.
  • Cross-Lingual - Assigns categories to documents in one language using a model trained on another via shared embeddings.
  • Natural Language Processing - Library for computing and using multilingual sentence embeddings.
  • Natural Language Corpora - Massive multilingual parallel text corpus for cross-lingual tasks.
3,659 stars·461 forks·Jupyter Notebook·other·19 views

Star history

Star history chart for facebookresearch/laserStar history chart for facebookresearch/laser

How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.

Projects sharing features with LASER

These projects share indexed features with LASER. Shared tags can include platform or build tooling; verify the primary use case before treating a result as a replacement.
  • chatopera/synonymschatopera avatar

    chatopera/Synonyms

    5,107View on GitHub↗

    Synonyms is a natural language processing library and semantic similarity engine specifically designed for Chinese text. It functions as a word embedding toolkit and tokenizer that extracts semantic meaning and identifies synonyms by calculating the conceptual closeness between words and sentences. The system provides a toolkit for Chinese word embedding and synonym discovery, allowing for the retrieval of semantically similar words to expand vocabulary. It distinguishes itself through a configuration-driven approach to model loading, which supports the integration of custom word embeddings t

    Pythonaichatbotnlp
    View on GitHub↗5,107
  • facebookresearch/fairseqfacebookresearch avatar

    facebookresearch/fairseq

    32,228View on GitHub↗

    Fairseq is a PyTorch toolkit for sequence-to-sequence modeling, specializing in neural machine translation, automatic speech recognition, and large-scale language model training. It provides a framework for processing and aligning diverse data sources, including text, audio, and video, to support tasks such as speech-to-text conversion and multimodal sequence learning. The project is distinguished by its distributed training capabilities, which utilize parameter sharding, mixed-precision training, and CPU offloading to handle models that exceed single-device memory. It also includes specializ

    Python
    View on GitHub↗32,228
  • johnsnowlabs/spark-nlpJohnSnowLabs avatar

    JohnSnowLabs/spark-nlp

    4,135View on GitHub↗

    Spark NLP is a toolkit for scalable text analysis and machine learning built on the Apache Spark distributed computing framework. It provides a multimodal machine learning framework and a distributed pipeline system for sequencing annotators to process large-scale linguistic data. The library includes a transformer text processor for generating contextual vector embeddings and a dedicated inference engine for managing large language models. The project distinguishes itself through its ability to process heterogeneous data types, including text, audio, and images, within a unified vision-langu

    Scala
    View on GitHub↗4,135
  • google-research/bertgoogle-research avatar

    google-research/bert

    39,869View on GitHub↗

    This project is a transformer-based language model and natural language processing toolkit designed to generate deep contextual representations of text. By utilizing a transformer-based encoder architecture, the system processes input sequences through stacked self-attention layers to capture the semantic meaning of tokens based on their surrounding sentence structure. The model distinguishes itself through bidirectional contextual processing, which analyzes text in both directions simultaneously, and masked language modeling, which trains the system by predicting hidden tokens within a seque

    Pythongooglenatural-language-processingnatural-language-understanding
    View on GitHub↗39,869
Compare all 30 related projects→

Frequently asked questions

What does facebookresearch/laser do?

LASER is a cross-lingual sentence embedding library and multilingual text encoder. It functions as a parallel text mining tool that maps sentences from multiple languages into a shared vector space for similarity and classification tasks.

What are the main features of facebookresearch/laser?

The main features of facebookresearch/laser are: Cross-Lingual Alignment, Shared Latent Spaces, Multilingual NLP, Multilingual Encodings, Sentence Embeddings, Cross-Lingual Transfer, Fixed-Length Vector Encodings, Text Tokenization.

Which projects share features with facebookresearch/laser?

Projects with overlapping indexed features include: chatopera/synonyms — Synonyms is a natural language processing library and semantic similarity engine specifically designed for Chinese… facebookresearch/fairseq — Fairseq is a PyTorch toolkit for sequence-to-sequence modeling, specializing in neural machine translation, automatic… johnsnowlabs/spark-nlp — Spark NLP is a toolkit for scalable text analysis and machine learning built on the Apache Spark distributed computing… google-research/bert — This project is a transformer-based language model and natural language processing toolkit designed to generate deep… autogluon/autogluon — AutoGluon is an automated machine learning framework and multimodal library designed to automate the end-to-end… huggingface/sentence-transformers — This project is a transformer-based framework for generating dense and sparse vector embeddings of text and multimodal…