Course: STAT 946 -- Adv Topics: Mathematical Foundations of Deep Learning
Blurb:
The goal of this course is to introduce and explore some of the recent theoretical advances that
aim to understand modern deep learning methods and training regimes.
Topics may include: Universal approximations, Uniform convergence, Benign overparamterization, functional limit theory/scaling limits,
NTK, comparison to kernel methods, Training dynamics and related phenomenology (Implicit regularization, Training regimes, sample complexity,
Mean-Field/hydrodynamic limits, DMFT/Effective Dynamics), Loss landscapes and geometry, transformers, diffusion models.
There will be a heavy focus on the analysis of training dynamics (sample complexity and scaling limits).
The course will be interspersed with primers on important tools and techniques from probability theory such as
concentration of measure, random matrix theory, stochastic analysis, and related ideas inspired by statistical physics.
This will be a fast-paced, research-level, seminar-style course. We will be learning as a group.
Students will play an active role in the course: as the course progresses students will pick up and explore these
or other topics to catch us all up!
Lectures
Week 1: Universal Approximations
Week 2: Uniform Convergence
- Lec 2: TBA
- Lec 3: TBA
- Reading:
Week 3: TBA
Week 4: TBA
Week 5: TBA
Week 6: TBA
Week 7: Reading Week
Week 8: TBA
Week 9: TBA
Week 10: CANCELLED
Week 11: TBA
Week 12: TBA
Week 13: TBA
Week 14: TBA