glouppe/info8010-deep-learning

Features

• Deep Learning Exercises - Provides structured lecture materials and programming exercises for learning deep learning theory and implementation.
• Automatic Differentiation Engines - The framework calculates exact derivatives of functions defined by programs by decomposing them into primitive operations and applying chain rule accumulation.
• Automatic Differentiation Engines - Provides an engine for calculating exact partial derivatives via computational graph traversal and chain rule application.
• Neural Network Construction - Provides the process for designing and building deep learning architectures using layers and model abstractions.
• Neural Network Regularization - Implements dropout and other regularization techniques to prevent overfitting and improve model generalization during training.
• Parameter Optimizers - Updates neural network weights using gradient-based methods to minimize loss functions during training.
• Deep Learning Courses - Provides comprehensive training materials on deep learning architectures, optimization, and generative modeling.
• Deep Learning Curriculum - Offers a structured academic curriculum covering the theoretical and technical foundations of neural networks.
• Neural Network Tutorials - Includes interactive notebooks and exercises for building and training deep learning models from scratch.
• Computational Graphs - Constructs execution structures for mathematical operations by recording primitive sequences to enable automatic differentiation.
• Neural Network Training from Scratch - Builds and trains complex models from scratch, including transformers and convolutional networks.
• Activation Functions - Applies non-linear activation functions to layer outputs to model complex relationships.
• Causal - Implements causal masking to prevent access to future tokens during autoregressive generation.
• Attention Algorithm Optimizations - Accelerates autoregressive sequence generation by caching key and value vectors to eliminate redundant attention computations.
• Input Sequence Attentions - Computes weighted sequence representations by measuring similarity between query and key vectors.
• Attention Scoring Functions - Computes attention scores by measuring similarity between query and key vectors.
• Image Sequence Architectures - The framework transforms image data into sequences of flattened patches to enable the application of transformer architectures to computer vision tasks.
• Instance Segmentation Engines - Generates pixel-level masks to identify and isolate individual object instances within a visual scene.
• Object Detection - Identifies and localizes objects within images by predicting bounding boxes and class scores.
• Semantic Segmentation Training - Implements semantic segmentation training using configurable backbones and decoders to generate precise pixel-wise masks.
• Convolutional Feature Extraction - Extracts spatial patterns from multi-dimensional inputs using convolutional kernels.
• Synthetic Data Generators - Provides tools for learning probabilistic models to simulate realistic synthetic data samples.
• Empirical Risk Minimization Procedures - Estimates model parameters by minimizing loss functions over training datasets to approximate optimal functions.
• Graph Aggregators - The framework computes node representations by aggregating features from adjacent nodes using fixed coefficients, learned weights, or attention mechanisms.
• Weather Forecast Generation - Computes atmospheric predictions using graph-based neural network models for meteorological forecasting.
• Image Synthesis Models - Generates high-resolution visual content using adversarial learning architectures.
• Generative AI Models - Implements adversarial, diffusion, and score-based models to synthesize high-quality data.
• Latent Space Disentanglement - Enforces semantic structure in latent spaces using variational inference to improve feature disentanglement.
• Latent Diffusion Models - Performs generative learning by operating on compressed latent representations using diffusion priors.
• Conditional Generative Modeling - Guides generative models by injecting conditioning signals into pre-trained architectures without additional training.
• Generative Adversarial Network Training - Optimizes generator and discriminator networks through minimax games to produce synthetic data.
• Gradient Computation - Calculates partial derivatives of loss functions with respect to model parameters using computational graphs.
• Gradient Clipping Utilities - Rescales gradient norms during backpropagation to prevent exploding gradients in deep architectures.
• Graph Neural Networks - Applies deep learning techniques to non-Euclidean data structures for scientific and relational analysis.
• Learning Rate Schedulers - Adjusts step sizes dynamically during training using decay strategies to improve optimization stability.
• Neural Network Classification - Predicts class probabilities for spatially structured inputs using convolutional neural networks.
• Normalization Layers - Includes layers that normalize input activations to stabilize training and improve convergence.
• Machine Learning Training - Guides students through hands-on implementation of tensor operations and optimization algorithms using interactive notebooks.
• Mixture of Experts - Supports routing and recording expert paths in mixture-of-experts models.
• Bidirectional Recurrent Neural Networks - Implements recurrent neural networks that process sequences in both forward and backward directions.
• Model Capacity Management Techniques - Adjusts the complexity of the hypothesis space to balance the trade-off between underfitting and overfitting.
• Neural Network Stacks - Builds deep neural architectures by stacking parameterized functional blocks that transform input data.
• Hyperparameter Tuning - Uses dedicated validation datasets to tune model settings and select optimal configurations without biasing evaluation.
• Model Performance Evaluators - Assesses model generalization using independent test datasets to measure performance on unseen data.
• Adversarial Training Procedures - Implements simultaneous optimization of competing generator and discriminator networks through minimax games.
• Momentum Optimizers - Accelerates convergence by incorporating historical gradient information into the parameter update process.
• Multi-Head Attention Mechanisms - Captures diverse sequence dependencies using parallel attention mechanisms on distinct projections.
• Neural Network Composition - Provides methods for stacking and organizing neural network layers into hierarchical structures.
• Neural Network Layers - The framework adapts the final network layer to specific tasks by selecting appropriate activation functions for classification or regression.
• Residual Networks - Utilizes skip-connections to facilitate gradient flow and mitigate vanishing gradients.
• Autoencoder Compression - Reduces high-dimensional input data into compact latent representations using autoencoder architectures.
• Gradient-Based Weight Optimization - Updates model parameters iteratively using gradient-based methods to minimize loss functions.
• Positional Encodings - Integrates spatial or temporal context into token embeddings using fixed, learned, or rotational encodings.
• Protein Structure Prediction - Predicts three-dimensional protein structures from amino acid sequences using transformer-based neural networks.
• Recurrent Neural Networks - Maintains a recurrent state updated at each time step to model inputs of variable length.
• Gated Recurrent Units - Uses gated recurrent units to manage information flow and long-term dependencies.
• Signal Restoration Models - Trains models to capture dependencies between signal components for mapping noisy inputs back to clean states.
• Spatial Pooling Layers - Reduces spatial dimensions using pooling layers to capture global dependencies.
• Transfer Learning - Adapts pre-trained model backbones to new tasks by fine-tuning or using them as feature extractors.
• Weight Initialization - Sets initial parameter values using statistical distributions tailored to specific activation functions.
• Orthogonal Initializers - Sets recurrent weight matrices to be orthogonal to preserve gradient stability.
• Weight Regularization - Penalizes large parameter values during training to prevent overfitting and improve model generalization.
• Zero-Shot Vision Foundation Models - Leverages pre-trained vision foundation models to perform zero-shot classification and segmentation across diverse domains.
• Graph Pooling - Aggregates node-level representations into single vectors for graph-level predictions.
• Score-based Models - Implements score-based generative training using Langevin dynamics and stochastic differential equations.
• Generative Data Synthesis - Synthesizes audio, text-to-image mappings, and scientific simulations using adversarial generative models.
• Training Memory Optimizers - Reduces training memory consumption by checkpointing intermediate variables and recomputing them during the backward pass.
• Training Image Augmentations - Multiplies effective dataset size by applying stochastic transformations to improve model robustness and performance.
• Molecular Property Prediction - Evaluates chemical compounds using graph neural networks to identify promising drug candidates via virtual screening.
• Continuous-Time Density Transformations - Models complex data distributions by applying invertible transformations and continuous-time differential equations.

Star history