an MDP value iteration applet you can play with

To: Rao Kambhampati <rao@asu.edu>
Subject: an MDP value iteration applet you can play with
From: Subbarao Kambhampati <rao@asu.edu>
Date: Wed, 29 Sep 2010 14:16:07 -0700
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=aprR35GL4HCSidGRE7Uzc/7PkpoM5BtklBNNb5qSpLQ=; b=FoKeE/buI+yH5kMo0E9ZekZgL6TX6Zss+2r54qVwLrKeu8JWoZ9gP8JHJRMdGH3WS7 N2kWOivEdpnkosPD+ODsLAL2r4f2QAabx09M8+tUwNK2qb/8u1mfmrTkUYOG91UPhEyE w5e+cd/dlxg0YWsuVEfNZSqRQD6BXNV1XTHYQ=
Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; b=ZZprYyDYG7ToqMBzctUWo3acnyimzu0+XbqR4BMFn4/FExKc765caD5tFufzHFOVBj aXsf2qqUQj8N+wvYsj8DumFig7Cx51zv8pHDc5uzSOwdpbS54RN8dBPx/IxcpC1IYrWH CQ+dh7wuQjMkRbSBTr8VTS4+7BzfVLzyN7M64=
Sender: subbarao2z2@gmail.com

check this one out:

you can change discount factor, change whether or not there are sink states, and see how soon/slowly the value iteration converges.

Rao