CSE 574: Planning & Learning.

Fall 2022 (Fridays 1:30--4:15pm; CAVC351)

This course will focus on sequential decision-making problems under a variety of conditions--with known (given) model--normally called planning, and unknown (partially known) model--normally called -- Reinforcement Learning. We will study both deterministic and stochastic settings, atomic as well as lifted (symbolic) models, and function approximation models (e.g. deep reinforcement learning).

The last time I (Rao) taught CSE574 was back in 2008 (!!). The lectures from that offering are below. The Fall 2022 version of the Planning/Learning class will change in significant ways to keep up with the new developments. One important goal is to draw clearer connections between planning and reinforcement learning.

Please note that this course will assume familiarity with Intro AI (see here for Rao's version)

This course will be a cross between a normal lecture course and a seminar course. In addition to instructor lectures, the course will involve paper readings and discussions. It is thus most useful for people with research interests in this area.

(Student comments on Rao's courses can be found here)

Lectures from the 2008 edition of CSE574 are linked below (Note that this edition didn't have any videos. The closest set of videos of my lectures for this material can be found HERE)

1. 1/15: Introduction. (By Menkes/J)(audio is here)
2. 1/17: Progression, regression and Means-Ends analysis planning. (audio is here)

3. 1/22: Plan-space planning (aka partial order planning or least commitment planning); handling conditional effects in progression/regression/plan-space planning; Lifted planning (audio is here)
4. 1/24: Refinement planning as a unifying view (audio is here)

5. 1/29: Reachability heuristics for classical planning (audio is here) (You can also get a "lecture" style discussion of the material from here)

6. 1/31:Bounded-length Planning: Graphplan; SAT/CSP/IP Compilation (audio is here)

7. 2/5: Bounded-length planning: Planning as Model-finding (audio is here)
8. 2/7:Knowledge-based planning (audio is here)

9. 2/12: Knowledge-based planning (contd) (audio is here)
10. 2/14: Learning search control for planning (audio is here)

11. 2/19: Over-Subscription/Partial Satisfaction Planning (audio is here)
12. 2/21: Replanning & Execution MOnitoring (audio is here)

13. 2/26: Temporal Planning: Modeling issues (audio is here)
14. 2/28: Temporal Planning: Search (audio is here)

15. 3/4: Temporal CSPs (audio is here)
16. 3/6: Scheduling (audio is here)

    Week of 3/9 is spring break.

17. 3/18: Integrating Planning & Scheduling (audio is here)
18. 3/20: Belief Space Search (audio is here)

19. 3/25: Belief Space Search Continued (Conformant vs. Conditional Planning) (audio is here)
20. 3/27: Belief Space Search: Heuristics (audio is here) (Also check out a BDD mini-tutorial here)

21. 4/1: Decision Theoretic Planning/Markov Decision Processes (audio is here)
    • Here is another set of slides on VI and PI
22. 4/3: FOMDPS: Special Cases/Efficient Algorithms (audio is here)

23. 4/8: Reinforcement Learning (lecture by Sungwook Yoon) (audio is here)
24. 4/10: Efficient Search (LAO*; RTDP) and Factored Reps for FOMDPS (audio is here)

25. 4/15: Probabilistic Planning Competition; Sampling methods for solving MDPs. (audio is here)
26. 4/17: Plan Recognition: Overview and Consistency-based methods. (audio is here)

27. 4/22: Plan Recognition: Probabilistic Methods; Decision-Theoretic Assistance (audio is here)
    • 4/23: The Myth and Reality of Web Services Composition (audio is here)
28. 4/24:Online Continual Planning to Control Modular Production Printer

29. 4/29: Final Class.

30. 5/6: Final Exam (2:40--4:30pm)

Planned Topics

1. Introduction, representation & search (1 week)
2. State Space and Plan Space Planning, Lifting (1 week)
3. Reachability heuristics (1- week)
4. SAT/CSP/IP based planning graph search; Planning as model-finding (1- week)
5. Refinement Planning as a unifying framework (1 week?)
   • RECITATION: Case studies of heuristic planners. Graphplan search

   ---

6. Partial satisfaction planning (1 class)
7. Knowledge-based planning with some emphasis on HTN planning (1 week)
8. Model-lite planning (1 class?)

   ---

9. Metric/Temporal Planning (1 week)
10. Scheduling (1 week)

    ---

11. Non-deterministic Planning: Conformant and Conditional planning (1 week)
12. Probabilistic planning: MDPs and POMDPS (2+ weeks)
13. Plan & Activity recognition (1 week)

    ---

14. Monitoring and Diagnosis (1/2 class)
15. Online planning (1/2 class)
16. Multi-agent planning (1 class)
17. Planning & Learning (1+ class)