microsoft/SynapseML

Features

• Distributed ML Pipeline Managers - Provides a composable framework for sequencing data featurization and model training across distributed compute clusters.
• Distributed Machine Learning Integrators - Provides an interface for training and evaluating machine learning models on large-scale datasets using parallelized Spark structures.
• Distributed Data Processing - Distributes machine learning workloads across a cluster of nodes using the Spark distributed data processing engine.
• AI Service Integrations - Provides a layer to integrate and apply cloud-based AI services for sentiment and language analysis within distributed pipelines.
• Anomaly Detection - Identifies multivariate outliers and unusual patterns in high-dimensional data and time series.
• Cloud AI Integrations - Wraps external cloud AI services as pipeline steps by communicating over HTTP to process distributed data.
• Computer Vision - Executes image recognition, face analysis, and object detection tasks across multi-node clusters.
• Deep Learning Inference Engines - Executes pre-trained deep learning models on CPU or GPU hardware to generate predictions for large datasets.
• Spark Cluster Connectivity - Establishes network connections to distributed Spark clusters to execute machine learning workflows across multiple nodes.
• Distributed Inference Engines - Implements a distributed inference engine that splits and executes machine learning workloads across multiple cluster nodes.
• Distributed Text Analytics - Performs sentiment analysis, entity extraction, and language translation on massive textual datasets using distributed processing.
• Machine Learning Workflow Libraries - Provides a standardized framework for building and scaling machine learning pipelines across distributed Spark clusters.
• Distributed Training - Scales the training and evaluation of machine learning models across multiple compute nodes.
• Model Inference - Runs pretrained deep learning models across clusters to generate large-scale predictions using hardware acceleration.
• Natural Language Processing Analysis - Executes linguistic analysis tasks including sentiment analysis, language detection, and entity extraction.
• Scalable Anomaly Detection - Provides distributed implementations of anomaly detection to identify outliers in large-scale, high-dimensional datasets.
• Text Analytics - Processes textual data at scale using distributed interfaces to extract insights and patterns.
• Machine Learning Pipelines - Orchestrates composable workflows that integrate data featurization, model training, and external AI services.
• Composable Architectures - Sequences data featurization and model training into unified workflows via a modular, composable interface.
• Face Analysis - Detects human faces in images to perform verification, identification, grouping, and similarity matching.
• Cybersecurity Machine Learning - Applies specialized machine learning models to detect and analyze cybersecurity threats.
• Gradient Boosting - Implements distributed training for gradient boosted decision trees to process large-scale datasets.
• Nonlinear Detection - Identifies anomalies in high-dimensional data using a distributed forest of isolation trees.
• Image Content Analyzers - Provides tools for detecting objects and text within images to automate metadata extraction.
• Machine Learning Pipelines - Embeds distributed machine learning models into pipelines to support batch, streaming, and serving workloads.
• Distributed Execution - Runs vision-based machine learning models across distributed clusters to process large-scale image data.
• ONNX Runtime Inference - Executes pre-trained models on CPU or GPU hardware using the cross-platform ONNX runtime for distributed inference.
• Medical Relationship Extraction - Identifies medical entities and maps relationships within unstructured clinical documents and health records.
• Multilingual Document Translation - Translates text and full documents across multiple languages while preserving original structural formatting.
• Speech and Text Conversion - Provides integrated pipelines for transcribing audio to text and synthesizing text into neural audio.
• Time Series Anomaly Detection - Generates models to identify irregularities and anomalous data points within temporal time series data.
• Elastic Scaling - Executes model training and evaluation across compute environments that dynamically resize based on the workload.
• Anomaly Detection Algorithms - Implements algorithms to identify outliers across multiple data streams by analyzing inter-correlations and dependencies.
• Computation Serving - Exposes cluster-based computations as web services to deliver results with sub-millisecond response times.
• Direct Memory Data Transfer - Optimizes data movement and memory usage between distributed partitions and native datasets using direct memory transfer.
• Similarity Search - Identifies the nearest neighbors for a query across large datasets based on feature similarity.
• Structured Data Extraction - Extracts key-value pairs and tables from business forms and IDs into structured formats.

Star history