- 8/26: Intro
- 8/28: Intro
(Audio
of lecture 2)
- 9/4: MDP(Audio
of lecture 3 (9/4))
- 9/9: MDP (Audio of lecture 4 (9/9))
- 9/11: MDP (Slide video
of lecture 5 (9/11))
- 9/16 Infinitie Horizon MDP, Value Iteration, its convergence (Audio of Lecture 6) ((Partial Slide video of lecture 6 (9/16))
- 9/18: Policy Iteration; Stochastic Shortest Path MDP
(Slide video
of lecture 7 (9/18))
- 9/23: More on SSP; compiling FH and IHDR to SSP; RL start (Slide video of lecture 8 (9/23))
- 9/25: RL Dimensions: Model-based vs. Model-free; active vs. passive;--
--simulator vs real world. Model-based and Model-free--
learning for passive RL. Model-based for active RL. The
importance of exploration
(Slide video
of lecture 9 (9/25))
- 9/30 (MDP Homework Due) Exploration policies; passive model-free learning and temporal-difference learning. Relating temporal difference learning and montecarlo learning. The concept of k-step returns and TD(\lamda) learning. Active model-free learning, and the need to keep track of Q-values. (Slide video of lecture 10 (9/30))
- 10/2
RL end; start of approximation approaches for MDPs (Slide video
of lecture 11 (10/2))
- 10/7: LRTDP and UCP discussion. (Slide Video)
- 10/9 (no-class; see videos I pointed you to).
- 10/16 POMDP Model (Slide Video)
- 10/21 POMDP Model contd. (Slide Video)
- 10/23 End of POMDP Model (Slide Video)
- 10/28 Midterm (Tentative)(includes MDP,RL,POMDP)(Rao on travel)
- 10/30 Overview of Factored Approaches for MDPs and Reinforcement
Learning (Slide Video)
- 11/4 Factored Reinforcement Learning--and discussion on gradient-descent approaches for regression; Policy Gradient (Slide Video)
- 11/6 (Discussed Bayes Network inference--under the assumption--
--that you watched L19 and L20)
- 11/13 Discussion of MLE approaches for learning on Bayes Networks
Learning (Slide Video)
- 11/18 Bayesian Learning with continuous hypothesis spaces (Slide Video)
- 11/20: Plate Models & Markov Networks--a quick introduction (Slide Video)
- 11/22 (Extra Class): Markov Networks continued; conditional
independences, inference, learning (and some discussion on connections
to deep learning)
(Slide Video)
- 11/25 Start of statistical models of language. (Slide Video)
- 11/27: Delving deep into topic models, understanding the nature of
the topics, seeing the connection with dimensionality reduction,
supervised vs. unsupervised learning etc. Going from unigram to single
topic to multi-topic model(Slide Video)
- 12/2: Discussion on LDA model and its applications (Slide Video)
- 12/4: Inference on LDA model (Slide Video)