Markoviana: A reading group

On Probabilistic Reasoning in AI


Spring 2005


Friday 2:30-4:30pm. BY 576


Coordinated by:  Subbarao Kambhampati


Mailing List Archive


Weekly meetings. Each meeting approx ~2 hours.


Students encouraged to register for at least 1 credit of independent study (mostly as a  sign of serious commitment).

(you can take it also for 2 credits—which will involve making one presentation at least

and 3 credits which will involve making 2 presentations. The grade will be Pass/Fail.)


Everyone is expected to read the week’s material before showing up.


This is not a formal lecture course. We will read papers/chapters together and participants

 will give presentations on various weeks.


Each participant must lead at least one reading session. 



Current schedule:


     1/28:  R&N Chapter 15, Sections 15.1-15.3. Speaker: Will Cushing. Here are the notes.

      2/4:   Rabiner’s tutorial on HMMs (also the chapter on HMMs from Durbin et al). Speaker: Fatih Gelgi.  Here are the notes

     2/11:  Kalman Filters (R&N Chapter 15, Sections 15.4 + Kalman filter tutorial). Speaker: Srini Vadrevu. Here are the notes.

     2/18:  Particle filters (R&N 15.5 and Particle Filter Tutorial). Speaker: Will Cushing. Here are Slides used

      2/25: Monte Carlo Localization for Robots; Inferring transportation routines: Speaker: Dan Bryce . Here are slides used.

      3/4: Statistical Learning – R&N 20 (1,2)  (Heckerman Tutorial is another reference) [ Jicheng's Slides] (A more complete but uncommented set is here)

     3/11 :  Statistical learning (EM algorithm) R&N 20.3 + Dellaert's paper). [R&N slides on EM]. [Fatih's Frank Dellart Slides]

      3/18:  Spring break!

      3/25:  Cancelled due to AAAI exhaustion

      4/1:  Structural Probabilistic Relational Models: Inference/Learning (R&N 14.6 + Getoor et. al. chapter) [NamTran]

     4/8:  Markov Networks (undirected graphical models) (Koller & Friedman, Chapter 5)[Srini Vadrevu]

     4/15: Conditional Random Fields


     4/22:  POMDP  (paper?) [Dan Bryce]

     4/29:   Markov Games




  Kernel methods—Support vector machines  (paper TBDr) [Mariano]

  Utility and MDPs (R&N 16 and 17) [Menkes]

   Reinforcement learning  (R&N 21) [J Benton]


      (semester ends 5/3)



Related Sites:



------------Expected Evolution







Filtering and estimation

  --Hidden Markov Models

  --Kalman Filters

  --Particle Filters

  --Plan recognition

R&N chapter on Temporal reasoning

Chapter 15 (15.1-15.5) [2 weeks]




Paper on particle filters and kalman filters


AAAI 2004 best paper


VLDB 2004 best paper


Srini Vadrevu


Statistical learning

  EM etc


Kernel machines and support vector machines




R&N chapter 20


Source for Kernel m/c?





   --MDP methods

   --POMDP basic solution methods


  LRTDP or ICAPS 2005 paper from CMU







Reinforcement learning 




R&N chapter on reinforcement learning



Markov Games

  (Multi-agent MDPs)




Bayes Nets; MCMC etc


First-order probabilistic inference


Markov Random Fields


From R&N and Koller book