cs231n/cs231n.github.io

Features

• Neural Network Implementations - Provides low-level implementations of neural network architectures and training pipelines built from first principles.
• Neural Network Training - Provides a comprehensive curriculum for building and training multi-layer neural networks to solve classification tasks.
• Deep Learning Education - Serves as a comprehensive educational resource for learning deep learning theory and practice through a structured curriculum.
• Educational Courses - Provides a comprehensive structured curriculum and educational materials for computer vision and deep learning.
• Programming Course Materials - Hosts structured programming course materials including exercises, notebooks, and projects for computer vision.
• Activation Functions - Explains the application of mathematical functions like ReLU and Sigmoid to introduce non-linearity in neural networks.
• Computer Vision Features - Provides instructional material on using learnable filters to extract visual patterns from image data.
• Convolutional Neural Networks - Provides a comprehensive curriculum and implementation guides for training convolutional neural networks for image classification.
• Decoder Architectures - Provides technical guides on building transformer decoders for autoregressive sequence generation.
• Fully Connected Layers - Teaches the implementation of fully connected layers to transform input activations into final class scores.
• Gradient Computation - Provides detailed guidance on computing analytic gradients of loss functions through backpropagation.
• K-Nearest Neighbor Classifiers - Teaches the implementation of supervised learning models that classify images using majority votes of the closest training samples.
• Linear Classifiers - Teaches the process of mapping flattened pixel vectors to confidence scores via linear mappings.
• Multiclass SVM Loss Calculators - Provides the mathematical foundation and implementation for calculating multiclass SVM loss.
• Softmax Loss Calculators - Provides the mathematical foundation and implementation for calculating softmax cross-entropy loss.
• Convolution Layers - Explains how to stack multiple convolutional layers to build complex visual feature representations.
• Layered Architectures - Explains how to stack modular layers, such as convolutional and fully connected layers, to build complex feature representations.
• Network Architectures and Layers - Teaches how to define the number of layers and neurons to balance model capacity and overfitting.
• Multi-Head Attention Mechanisms - Teaches the implementation of multi-headed attention mechanisms to capture diverse sequence dependencies.
• Parameter Optimization Strategies - Finds the weights that minimize a loss function using random search or gradient-based descent.
• Perceptron Classifiers - Covers the implementation of simple linear binary classification models that learn weights via error correction.
• Cross-Entropy Loss Functions - Instructs on implementing cross-entropy loss functions and regularization to guide the optimization of classifiers.
• Stochastic Gradient Descent - Teaches the implementation of stochastic gradient descent using mini-batches to minimize loss functions.
• Transformer Encoders - Includes detailed instructions on implementing transformer encoders using self-attention and positional encoding.
• Weight Regularization - Instructs on strategies for penalizing model weights to improve generalization to unseen data.
• Computer Vision Education - Provides a public educational website with lecture notes and technical guides focused on computer vision and deep learning.
• Image Classifiers - Provides instructional materials on classifying images by analyzing pixel arrays as input data.
• Backpropagation Theory - Explains the mathematical foundations of backpropagation and gradient descent for updating model weights.
• Computer Vision Tutorials - Delivers a structured academic curriculum covering image classification, visual recognition, and generative modeling.
• Academic Course Materials - Provides structured academic course materials and lecture notes focusing on convolutional neural networks.
• Linear Algebra - Provides foundational education on linear algebra and matrix operations essential for processing neural network activations.
• Softmax Normalization - Explains how to use softmax normalization to convert raw class scores into probability distributions.
• Activation Visualizations - Displays internal layer activations to identify dead filters or observe feature sparsity during training.
• Adversarial Machine Learning - Covers the generation of adversarial examples and the implementation of denoising blocks to increase model robustness.
• Attention Masking - Masks portions of the alignment matrix in self-attention layers to prevent access to future vectors.
• Attention Mechanisms - Provides instructional content on implementing attention layers using linear maps and alignment operations.
• Autoregressive Image Generation - Describes the process of producing image pixels sequentially using recurrent networks.
• Image Processing - Includes practical resources for reading, writing, and resizing images as a foundation for computer vision.
• Convolutional Layer Conversion - Explains how to convert fully connected layers into convolutional layers to enable spatial evaluation of images.
• Adversarial Denoising - Covers methods to remove noise from adversarial inputs using denoising blocks to protect high-level features.
• Deep Learning Dataset Expanders - Explains how to use random transformations like flipping and cropping to expand training datasets.
• Density Estimation - Teaches methods for modeling image probability by decomposing pixels into conditional density estimations.
• Dimensionality Reduction - Encodes unlabeled training data into low-dimensional feature vectors to capture factors of variation.
• Discrete Pixel Modeling - Instructs on representing pixel intensities as discrete values using softmax layers.
• Symmetry Breaking Initializers - Provides instruction on using random distributions for weight initialization to break symmetry and improve model convergence.
• Feature Scale Normalization - Explains how to normalize dataset features to zero mean and unit variance for stable model convergence.
• Gated Memory Mechanisms - Teaches the use of cell states and gating mechanisms to preserve information over long sequences.
• Latent Space Generative Models - Covers the use of latent space generative models to transform simple distributions into realistic synthetic data samples.
• Sampling - Teaches procedures for generating noise and latent vectors to serve as inputs for conditional synthesis.
• Synthetic Media Generators - Teaches how to use generator and discriminator networks to transform noise into realistic high-dimensional data.
• Quality Evaluators - Provides guidance on measuring the realism and diversity of generated images using metrics like FID and Inception Score.
• Autoregressive Image Generation - Teaches how to produce image pixels sequentially by feeding previous predictions back into the neural network.
• Parallel Pixel Generation - Explains the use of masked convolutions to predict multiple image pixels simultaneously.
• Generative Model Development - Includes instructional content on creating realistic high-dimensional data using GANs, variational autoencoders, and neural radiance fields.
• Layer Health Diagnostics - Analyzes activation and gradient distributions via histograms to detect vanishing or exploding gradients.
• Gradient Verification Tools - Instructs on using centered difference formulas to verify the accuracy of analytic gradient implementations.
• Numerical Gradient Approximations - Estimates the gradient of a function at a specific point using finite difference approximations.
• Hyperparameter Optimization - Provides technical guides on tuning learning rates and regularization settings via hyperparameter optimization.
• Image Density Estimation - Teaches how to predict the probability of an image by decomposing joint pixel likelihood.
• Composition-Controlled Generators - Explains the use of mapping networks and noise to control the style and variety of generated images.
• Generative Adversarial Architectures - Provides instructional material on training generative adversarial networks to synthesize high-quality visual content.
• Novel View Synthesis Engines - Includes instructional content on synthesizing novel 3D viewpoints using neural radiance fields.
• Knowledge Distillation - Explains how to transfer soft labels from teacher to student networks via knowledge distillation to improve resilience.
• Learning Rate Schedulers - Reduces the learning rate over time using decay schedules to help the model settle into minima.
• Loss Functions - Measures the discrepancy between predictions and ground truth using metrics like Hinge or Softmax loss.
• Stacked - Provides instructional material on building deep models by stacking multiple recurrent layers.
• Gradient Flow Analysis - Provides theoretical guidance on how operations like addition and multiplication influence gradient flow.
• Training Progress Monitoring - Tracks loss, accuracy, and weight-to-update ratios over epochs to diagnose learning rate issues.
• Hyperparameter Tuning - Determines optimal algorithm settings by splitting training data into training and validation sets.
• Data Preprocessing - Teaches methods for cleaning and scaling input features via normalization and whitening.
• Adversarial Training Procedures - Details training procedures that incorporate adversarial samples to increase model immunity against attacks.
• Implementation Sanity Checkers - Provides validation scripts to verify the functional integrity of model initializations and overfitting capabilities.
• Model Weight Inspections - Inspects convolutional layer weights to determine if the network has converged or is overfitting.
• Model Weight Visualizations - Plots the weights of the first layer to identify unconverged networks or improper regularization.
• Neural Network Debugging - Teaches how to analyze gradient flow and visualize internal activations to diagnose issues like vanishing gradients.
• Neural Network Interpretability - Retrieves images that trigger the highest response from specific neurons to understand detected visual features.
• Neural Network Regularization - Teaches regularization techniques like dropout to prevent neuron co-adaptation and improve generalization.
• Adaptive Learning Rate Optimizers - Adjusts learning rates per parameter using algorithms like Adam to normalize updates based on gradient magnitudes.
• Gradient Descent Algorithms - Updates model parameters by moving in the opposite direction of the gradient to minimize loss.
• Positional Encodings - Explains how to use sine and cosine functions to inject sequence order into transformer models.
• Approximate Inference Methods - Teaches how to optimize a lower bound on likelihood using latent variables to model data distributions.
• Saliency Mapping - Generates a heatmap by sliding an occluder over an image to identify influential pixels.
• Search Space Definition - Teaches best practices for defining and sampling hyperparameter search spaces to optimize learning rates and regularization.
• Sequential Data Models - Covers frameworks and architectures used for predicting and processing data sequences.
• Spatial Pooling Layers - Teaches the use of spatial pooling layers to downsample feature maps and reduce total parameter counts.
• Stochastic Gradient Descent Optimizations - Provides educational material and implementation guides for updating model parameters using stochastic gradient descent and momentum.
• Transfer Learning - Explains how to apply transfer learning by adapting pretrained networks through fine-tuning or feature extraction.
• Transformer Architecture Implementation - Provides technical guides on building transformer encoders and decoders using multi-headed self-attention and positional encoding.
• Unpaired Image Translation - Includes educational content on models trained to map images between domains without requiring corresponding training pairs.
• Variational Inference - Teaches the decomposition of data likelihood using latent variables and optimization of a lower bound.
• Weight Initialization - Sets starting parameter values using random distributions to break symmetry and improve convergence.
• Self-Attention Implementations - Guides the implementation of self-attention modules that derive queries, keys, and values from a single input set.
• Differentiable Programming - Provides instructional material on the mathematical foundations of differentiable programming and backpropagation for parameter optimization.
• Dimensionality Reduction - Provides guidance on reducing data dimensionality using PCA and random projections.
• Sequence To Sequence Models - Includes technical guides on building sequence-to-sequence models using recurrent layers and attention mechanisms.
• Adversarial Attacks - Teaches how to generate adversarial examples using gradient-based methods to test model robustness.
• Data Visualization - Provides tutorials on rendering 2D data and images using scientific computing visualization libraries.
• Array Manipulation Guides - Offers educational guides on performing element-wise mathematical operations and slicing on multidimensional arrays.
• Programming Assignments - Manages a centralized directory of coding exercises and project requirements for students.
• Arithmetic Broadcasting - Provides instruction on executing arithmetic operations between arrays of different shapes using broadcasting semantics.
• Python Educational Fundamentals - Provides instructional materials covering Python syntax and fundamental programming concepts specifically tailored for AI implementation.
• Computational Graph Decompositions - Teaches how to decompose complex functions into modular gates to simplify the computation of partial derivatives.
• Sequence Likelihood Estimators - Teaches methods for evaluating image probability by decomposing joint likelihoods into conditional sequence estimators.
• Neural Network Architectures - Course materials for convolutional neural networks in visual recognition.

Star history