CSE 571: On the implementation part of the MDP homework

I got some questions on what the implementation should be--for "general" state space, transition and reward functions, or for specific grid-world like domains. While *my* preference is to leave my code some space for extension (as I suggested), for this project you will get the full credits if (1) your implementations run correctly with required input (exercise 17.8, Figure 17.1) and (2) your experiment analysis makes sense (i.e., consistent with what you learned from class). This also means that you should pay attention to your report (it will be extremely hard for me if you just give me your code and some numbers without any comments/observations).