1: List five ideas/techniques that you learned in this course that you thought were most interesting?

Student	Answer
Dan Bryce	1) Refinement Planning Generalization 2) Being able to understand PO Planning, 3) The discussion on local Search Techniques, but would have liked a more general discussion, not just LPG 4) The discussion of TCN’s. 5) Planning under uncertainty discussions.
J. Benton	1) EBL and memoization 2) Using the planning graph to derive heuristics. 3) Metric Temporal Planning 4) Conformant Planning 5) Progression, Regression and causal proofs.
Mike Rush	1) Explanation Based Learning 2) Graphplan 3) Knowledge Based Planning 4) Heuristics 5) Markov Decision Problems
Xin Z.	1) Planning Graphs 2) Planning Heuristics 3) Conditional Planning 4) Planning with incompleteness 5) Scheduling
Preetha A.	1) Partial Order planning: not committing to position. 2) Posing planning as CSP and seeing that they actually speed up search 3) The idea of planning graphs for reachability as well as heuristics 4) The local search idea and seeing how it worked to give faster search times but not optimal plans 5) Seeing that instead of committing to one approach if we combine advantages of different approaches in planning, it turns out that better ways to solve problems are obtained.
Corey M.	1) Conditional planning 2) Conformant planning 3) MDPs 4) Metric Temporal Planning 5) Temporal Constrain Networks
Monika S	1) Least-commitment- It provides a lot o freedom when it comes to deciding a position of an action, especially in early stages of creating a plan. 2) Memos in Graphplan search. It was great way of pruning search. 3) Planning with Knowledge-based planners. It can do wonders in improving plan quality. 4) Capturing n-sized mutexes and how that captures n-ary relations between state variables. 5) I liked creating my own domain in PDDL 2.1, even though the lack of support for adl was very limiting.
Menkes v.	1) The graphplan algorithm 2) Heuristic estimation in Graphplan 3) POP Algorithm 4) DDB and EBL 5) Metric temporal planning approaches by SAPA and LPG
Josh R.	1) Conformant planning 2) The value of graph plan and similar ideas even in non classical domains 3) How much easier (well for the user anyway) metric temporal planning is compared to how hard it sounds… this is a big hurray to PDDL 2.1 4) The value of using sets of states compared to individual states. 5) The cost of using sets of states compared to individual states.

2: List five ideas/technqiues that you thought got over-play and were not worth the time we spent on them.

Student	Answer
Dan Bryce	1) Too much testing of planners (BB, GPCSP, Sapa, LPG) in projects. 2) Proofs. Writing causal proof for homework and midterm was too much. 3) LPG. Nice to learn about, but we didn’t go into such detail with other leading planning approaches. 4) Difference between conformant and conditional planning. You spent too much time outlining the differences in the problem definitions and not enough on the solutions techniques. 5) Not enough on learning, just the EBL/DDB stuff.
J. Benton	1) Knowledge Based Planning 2) CSP encoding of Graphplan (good to discuss but I think overplayed) 3) Partial order planning (spent too much time on) 4) Compilation to IP 5) Would have liked to have tarted temporal planning sooner so we could cover it for a longer period of time
Mike Rush	I didn’t think any topic was over covered. I would have liked to spend more time on temporal problems rather than classical ones. For me, they seemed to be more “real life” even though the methods are not as developed.
Xin Z.	1) SAT encoding 2) ILP encoding 3) CSP encoding 4) SAPA 5) ZENO
Preetha A.	1) Initial parts of progression and regression search, too much time was spent on it. 2) More on metric temporal planning could have been done. 3) Knowledge based planners: What sorts of problems do they actually solve?
Corey M.	1) Basic Progression 2) Basic Regression 3) Partial Order Planning 4) Graphplan 5) CSP/SAT
Monika S	1) Converting planning graphs to CSP and especially SAT encoding. I thought that there was too much busy-work involved with it. 2) I think that too much time was spent on discussion how to fix open conditions in different planning schemas. 3) Empirical problems. Running sets of problems with different planners. I think it would have been more productive if there was less busy-work and more per-example explanations, i.e. this planner is generally better in these types of problems, but it failed in problem 1, because … 4) There was too much busy work with making mutexes and generally too much discussion about mutexes relations. I do realize though, that it was important in capturing negative relations between actions and in pruning search. 5) There was to omuch time spen on serial plans versus parallel plans, i.e. we spent a lot of time talking about the blocks world. It’s great and easy to use in different examples, but I would like to see more examples of domains with parallel plans.
Menkes v.	1) Hmm, difficult. Actually, I felt like we went through a lot of things very quickly, especially in the first couple of weeks. I strongly feel that he time spent on classical planning is time well spent. It gives you a solid basis to tackle more advanced planning problems like MTP and planning under uncertainty.
Josh R.	Most things seemed to lead into something else eventually, even if they felt overplayed at the time, looking back I don’t think it would be very fair to call any of them over played.

3: List any ideas that you thought would be covered in the course, and would have liked them to see covered, but finally were not.

Student	Answer
Dan Bryce	1) It would have been nice to talk more about applications (domains) that are used in industry. Plus, where the demand is. 2) I would have liked to learn more about monitoring plan execution. (RAX) 3) It would have been nice to have a general overview of algorithms that are used, and in what scenarios they are most applicable. 4) I wanted to hear more (or something) about multi-agent systems. 5) I was disappointed that we were not assigned individual semester projects, in place of homogenous, smaller projects.
J. Benton	1) I was expecting a little more information about learning techniques, especially the monte carlo methods of reinforcement learning
Mike Rush	1) I would have liked some discussion of how all these methods relate to how humans solve these planning problems (which we seem to do all the time). Even just a survey of hypotheses with brief discussion would have been nice.
Xin Z.	1) The class is called “Planning and Learning”, but planning takes 90% of the class time, and learning only covered a little bit. 2) I would like to see more scheduling part. 3) Also, we may need to spend one class on CSP, since not everyone learn it before 4) So, my suggestion is (might be wrong, but just my opinion) that planning 50% and rest other topics such as scheduling, learning.
Preetha A.	1) Fully concentrated on planning, except EBL methods. There wasn’t anything on learning. 2) However the time was too short and you packed as much as possible. I worked hard and learned so much about planning from a level that I did not commit to a single method. Encourages to think on different approaches.
Corey M.	I would have liked to see coverage of more advanced topics in Planning, such as disjunctive goals. Also, it would have been interesting to have leed of more applications of planning in the real world.
Monika S	I would like to see more planning based on agents acquiring knowledge from its experiences. I was also hoping for planning in dynamic domains. It would have been nice to be able to use planners that are able of handling conditional effects, negative preconditions, disjunctive preconditions, etc. I also thought I would see more elaborate planning used in strategy games and simulators, like the “Massive” (crowd simulator)
Menkes v.	When the notion of lifted actions was introduced I was looking forward to see a full class (maybe two) on this particular topic. It is one of a few topics I am still not very comfortable with.
Josh R.	The probabilistic stuff could have been fun, but I feel we covered a lot of stuff for one semester as is so that’s ok.

4: List five "non-obvious" (to you) details or cross-connections that _you_ were able to appreciate over the semester

Student	Answer
Dan Bryce	1) The connections between Graphplan, GPCSP, and Blackbox were good to learn. 2) I liked the three categories for planning under uncertainty: problems, solutions, and success measures. 3) It was nice to see the 101 ways to that the planning graph can be used in different planning scenarios. 4) I liked the explanation of the continuum between planning and scheduling. 5) The connections between SAT, CSP, and ILP were interesting, but I would have liked a deeper examination.
J. Benton	1) Calculating the slack between two events can be used as a heuristic for finding an ordering of events, using de Floyd-Warshall algorithm to tighten constraints on simple temporal constraint network. 2) Being unable to observe your state and having non-deterministic actions are the “same thing”. 3) Serial planning graph finds the shortest plan in terms of numbers of action but normal parallel graph finds the shortest plan in terms of number of levels. 4) Solving a compiled to CSP grpahplan allows the ability to work on different levels of the graphplan at the same time. 5) It is possible that graphplan will not find the optimal parallel plan to a problem.
Mike Rush	1) Planning is search, search and search. 2) Search spaces come in different varieties (state, belief, plans, etc). 3) MDP applicability to finances 4) The limits on classical planning and how those limits affect the range of problems which can be solved.
Xin Z.	1) I learned planning heuristics. 2) I also learned different kinds of techniques about planning. At beginning of the class, I have little idea about planning, but now I think I know it, even though not completely, but at least got some ideas. 3) Temporal planning is very interesting. 4) Planning with incompleteness is hard, it seemed that there is not much work have been done compared with other topics we covered. But the introduction of this idea is good. I want to hear more about it, very good presentation of this part. 5) Conditional planning is also god knowledge that I learned from this class. It’s interesting and Dr. Rao gave good class in this part.
Preetha A.	1) Mutexes doing binary mutexes till level off does not guarantee a plan of that length (because we do not calculate 3,4 mutexes, etc) 2) Regression search looks faster and more efficient up front, but actually we search in sets of states many of which may not even be legal, so it is costly. 3) What encoding to follow CSP/SAT/IP depends of the problem too, there is not clear winner. 4) Notion of preconditions and effects of actions that wasn’t quite true while moving into conformant/conditional planning in belief states. It becomes more like do the action and if the precondition is true the effect will happen, if not nothing happens.
Corey M.	1) Reachability heuristics 2) EBL 3) Bellman equations and their convergence 4) Search in AND/OR Graphs 5) Non-classical planning in general.
Monika S	1) Earlier planners could have solved the Sussman-anomaly problem if the goal state description would have been complete. Intuitively would have said that the more relaxed the goal state, the easier would be to satisfy it. 2) Memos are n-ary mutexes. For some reason it was not obvious to me, but it was really helpful in realizing a link between different searches. 3) If a set of subgoals have mutual exclusion relations among them and appear together for the first time at step n without any pair being mutex, does not guarantee that thre is a plan to achieve these subgoals in an n-step plan, because only mutexes between pairs are being considered. There might be 3,4,5, ..-ary mutexes. The cross connection is that this can be compared to arc consistency does not guarantee there is solution, n-consistency (for n step plan) does. 4) A partial order planner is a least commitment regression planner. 5) Given that planners had very hard time finding solutions when plan required temporarily getting away from the goal, one of the hardest problems to solve for the planners would be the rubik’s cube. The evil part is that (quoting from http://www.rubiks.com/): “Theoretically the shortest possible path to solving Rubik’s cube from any scrambled position is as few as 22 twists . So far no one has succeded in demonstrating this method.
Menkes v.	1) The planning circle of Graphplan-CSP-SAT-IP 2) EBL and memorization 3) Solving classical planning as MDP
Josh R.	1) It’s not really that non-obvious , but I had not realized the connection between no-goods and mutex, until you said it and then I said “duh” 2) The actual speed and quality of a greedy planner (LPG) really caught m off guard.