Advanced Deep Learning

• Most lectures: Guillaume Charpiat
• Practical sessions: Styliani Douka, Rémy Hosseinkhan, Cyriaque Rousselot and Théo Rudkiewicz (incl. materials by Victor Berger, Alessandro Bucci, Loris Felardos, Wenzhuo Liu, Matthieu Nastorg, Francesco Pezzicoli and Antoine Szatkownik)

• 4 practicals (notebooks), presented in class, to do at home in groups of 1-3 people, and to hand in within 2 weeks
• 1 final exam (on paper, in class, alone)

• Session 1 : Thursday Janury 23rd (Auditorium 2 Michelin, Eiffel building)
• Session 2 : Thursday Janury 30th (Auditorium 3 Michelin, Eiffel building)
• 3 week break
• Session 3 : Thursday February 27th (Théâtre e.070, Bouygues building)
• Session 4 : Monday March 3rd (Théâtre e.070, Bouygues building) -- unusual day!
• Session 5 : Thursday March 6th (Auditorium 3 Michelin, Eiffel building) -- NB: 2 sessions the same week!
• Session 6 : Thursday March 13th (Auditorium 3 Michelin, Eiffel building)
• Session 7 : Thursday March 20th (Théâtre e.070, Bouygues building)
• Session 8 : Thursday March 27th (Amphitheatre Alain Aspect, ENS Pari-Saclay) -- Final exam

• Session 1 : Deep learning vs. classical machine learning and optimization
  → Practical session: hyperparameters and training basics: instructions and the 2 problems of TP1: Pb 1, and Pb 2 with its dataset. NB: TP1 will not be graded.
  → Practical session: github repo with general MLOps advice
  → Environment used in practical sessions
  → Practical tips and tricks to train neural networks
  → [2022] Course notes (pdf) (handwritten with drawings) and lesson summary (html)
  → [2022] Video recording: lecture [270MB]
  
  
• Session 2 : Interpretability (part 1: visualization and analysis)
  → Practical session: visualization with grad-CAM: notebook and test images
  → Link to hand in TP2 (password given in the registration email)
  → [2023] Course notes (pdf) (handwritten with drawings) and lesson summary (html)
  → [2023] Video recording: part 1 [500MB], part 2 [700MB]
  
  
• Session 3 : Architectures
  → Practical session on graph-NN: notebook with auxiliary files class_model_gnn.py and conv_as_message_passing.py
  → Link to hand in TP3 (same password as usual)
  → [2023] Course notes (pdf) part 1 + part 2 (handwritten with drawings) and [2025] lesson summary
  → [2023] Video recording: part 1 [500MB] (theory: prior, initialization, ...) and part 2 [500MB] (architecture zoo, attention, graph-NN)
  
  
• Session 4 : Interpretability (part 2: biases and issues with datasets)
  → [2023] Course notes (pdf) (handwritten with drawings) and lesson summary (html)
  → [2023] Video recording: part 3 [700MB], part 4 [400MB]
  
  
• Session 5 : Small data and frugal AI: weak supervision, transfer, compression and incorporation of priors
  → Practical session on transfer learning: notebook and python script
  → Link to hand in TP4
  → [2023] Course notes (pdf) (handwritten with drawings) and lesson summary
  → [2023] Video recording: part 1 [500MB] and part 2 [500MB]
  
  
• Session 6 : Modeling: deep learning and physics (exploiting known invariances, priors or physical properties)
  → Practical session on learning dynamical systems: notebook + image (included by the notebook) (Credits: Alessandro Bucci, now Safran)
  → Link to hand in TP5
  → [2023] Course notes (pdf) part 1 + part 2 (handwritten with drawings) and lesson summary
  → [2023] Video recording: part 1 [850MB] and part 2 [500MB]
  → [2023] Course notes (pdf) about metrics (handwritten with drawings)
  → Lesson summary about modeling tasks and losses and about learning from noisy data
  
  
• Session 7 : Generative models (GAN, VAE, Normalizing Flows, Diffusion Models and Flow Matching) and Auto-DeepLearning (incl. Neural Architecture Search) by guest speaker Romain Égelé
  → Practical session on generative models: notebook + images included by the notebook: VAE, diffusion model, and auxiliary detail.
  → Link to hand in TP6
  → [2023] Course notes (pdf) on generative models (handwritten with drawings)
  → [2023] Video recording of the course on generative models: part 1 [400MB] and part 2 [550MB]
  → [2025] AutoML slides (Optimization of Learning Workflows Hyperparameter Optimization, Neural Architecture Search, Ensemble) by Romain Égelé and its DeepHyper repository
  → [2023] Video recordings: course on Modeling tasks and losses + presentation "AutoML via Bayesian Optimization and Meta-Learning" by Lisheng Sun-Hosoya, [400MB] and presentation "Neural Architecture Search" by Romain Égelé [260MB]
  → [2023] AutoML slides + Neural Architecture Search slides
  
  
• Session 8 : Final exam
  
  

• ours:
  • Links between explainability, frugal AI, robustness and formal proofs of neural networks: looking for statistically-meaningful concepts and enhancing them. In collaboration with PEPR IA SAIF. Contact me for more details!
  • Deep learning for partially-observed dynamical systems
  • Deep learning inference of cardiac arrhythmias
  • Neural Architecture Growth for Frugal Learning
  • Learning dynamical systems with the right statistics: distribution of extreme events in complex physical systems
  • Learning causality
  • and lots of other very interesting topics, on demand
  
  
• more from the TAU team (deep for population genetics, AutoML, etc.), in particular:
  • Equivariant Graph Neural Networks for amorphous materials
  • Observational Causal Discovery: Stability as a necessary condition for Identifiability
  • A Divide and Conquer strategy for Observational Causal Discovery