In the past several years, significant strides have been made in designing algorithms to support decision making in realistic worlds that have actions with stochastic effects, whose states are only partially observable (due to sensor limitations), and which are inhabited by multiple adversarial or co-operative agents. This seminar course will cover these advances. Specific topics to be discussed include:

• Handling metric resources and temporal constraints

• Temporal representations
• TCSP paper by Schwalb and Dechter --AIJ-1997 (Processing disjunctions in TCSP problems).
• TCSP Tutorial (AAAI-96) by Vilain, Boddy and Schwalb.
• Integrating symbolic and numeric reasoning methods

• e.g. Satisfiability and Integer Linear Programming

• Integrating planning and scheduling

• Dynamic constraint satisfaction problems

• 1st Paper: Bridging the gap between planning and scheduling by David Smith, Jeremy Frank and Ari Jonsson.
• Handling sensor limitations

• Contingency planning

• Quantified boolean formulae

• Interleaving planning and execution
• Chapter 13 from Russell & Norvig.
• Original CNLP algorithm (basis for CPOP)
• Puccini Algorithm and Example Trace
• LCW paper from AAAI-94
• Planning in stochastic domains

• Markov Decision Processes

• Value and Policy iteration
• Factored representations

• Value and Policy function approximations

• Stochastic Satisfiability
Littman's paper on Stochastic Satisfiability
• Handling uncerainity and partial-observability together

• Partially observable markov decision processes

• Handling unknown environment dynamics

• Reinforcement learning

Section 4 of the paper "Four Directions in Machine Learning"

• Reactive planning
• Risk-averse planning
• Multi-agent planning

• Issues of coordination/collaboration

There will be no prescribed textbook. Instead, we will read a set of state-of-the-art papers on the various topics. Grading for the course will be based on class participation, a semester project and examinations.