Deep Learning in Practice

• Most lectures: Guillaume Charpiat
• Practical sessions: Wenzhuo Liu, Lucas Meyer and Matthieu Nastorg (incl. materials by Victor Berger, Alessandro Bucci and Loris Felardos)

• January 13th : Deep learning vs. classical machine learning and optimization
  → Practical session: hyperparameters and training basics: instructions and the 2 problems of TP1: Pb 1, and Pb 2 with its dataset
  → Link to hand in TP1 (same password as usual)
  → Environment used in practical sessions
  → Practical tips and tricks to train neural networks
  → [2022] Course notes (pdf) (handwritten with drawings) and lesson summary (html)
  → Video recording: lecture [270MB], TP presentation [130MB]
  
  
• January 20th : Interpretability
  → Practical session: visualization with grad-CAM: notebook and test images
  → Link to hand in TP2 (same password as usual)
  → [2022] Course notes (pdf) (handwritten with drawings) and lesson summary (html)
  → Video recording: lectures on visualization (part 1) [380MB], visualization (part 2) [260MB], interpretability [100MB], biases (part 1) [100MB], biases (part 2) + privacy [116MB]; TP presentation [50MB]
  
  
• January 27th : Interpretability (bis)
  → materials: cf previous session above
  
  
• February 3rd : Architectures
  → Practical session on graph-NN: instructions (with installation instructions), code and baseline
  → Link to hand in TP3 (same password as usual)
  → [2022] Course notes (pdf) (handwritten with drawings) and lesson summary
  → Video recording: part 1 [250MB] (theory: prior, initialization, ...) and part 2 [380MB] (architecture zoo, attention, graph-NN, TP presentation, more graph-NN)
  
  
• February 10th and 17th : no course!
  
  
• February 24th : Small data: weak supervision, transfer, and incorporation of priors
  → Practical session on transfer learning: jupyter notebook
  → Link to hand in TP4 (same password as usual)
  → Video recording: course, part 1 [350MB] and course, part 2 + TP4 presentation [290MB]
  → [2022] Course notes (pdf) (handwritten with drawings) and lesson summary
  
  
• March 3rd : Modeling: deep learning and physics (exploiting known invariances, priors or physical properties), with Michele Alessandro Bucci (LISN, TAU team / Safran)
  → Practical session on learning dynamical systems (notebook + image included by the notebook), with its dedicated submission site
  → Video recording: part 1 (by G.C.) [220MB] and part 2 (by M.A.B.) + TP5 presentation (by L.M.) [580MB]
  → [2022] References (articles presented by M.A.B.)
  → [2022] Course notes (pdf) (handwritten with drawings) and lesson summary
  → [2021] Last year's Lionel Mathelin's slides
  
  
• March 10th : Generative models (GAN, VAE and Normalizing Flows) + talk on uncertainty quantification by Maxence Ernoult (IBM Research) : Simple techniques for Out-of-distribution Detection & Error Detection in Statistical Models
  → Practical session on generative models, and its dedicated submission site
  → [2022] Course notes (pdf) on generative models, (handwritten with drawings)
  → Video recording: part 1 (by M.E.) [280MB] and part 2 (by G.C.) + TP6 presentation (by L.M.) [240MB]
  
  
• March 17th : Modeling tasks and losses + Generalization and guarantees + Auto-ML / Auto-DeepLearning (guest talk by Adrian El Baz, from MILA)
  → Video recordings: Auto-DL presentation by Adrian El Baz [125MB] and lesson about Modeling and generalization [360MB] (skip the first minutes)
  → [2022] Course notes (pdf) (handwritten with drawings)
  → Lesson summary about modeling tasks and losses and about learning from noisy data
  → [2021] Lisheng Sun's talk: AutoML slides + more details about Bayesian optimization for hyper-parameters
  

• ours:
  • Deep learning for population genetics (interpretability and DNA generators)
  • Sampling 3D Molecular Distributions with Deep Neural Networks
  • Deep Active Modeling for Physics
  • Deep Learning for Fluid Mechanics
• more from the TAU team (deep for population genetics, AutoML, etc.)