Bayesian Analysis of Markov Chains

Suppose we observe data and want to test if it comes from a Markov chain. If it does, we may want to estimate the transition operator. Working in a Bayesian way, we have to specify priors and compute posteriors. Interesting things happen if we want to put priors on reversible Markov chains. There are useful connections with reinforced random walk (work with Silke Rolles). On large-scale application to protein folding will be described. More generally, these problems arise in approximating a dynamical system by a Markov chain.

For continuous state spaces, the usual conjugate prior analysis breaks down. Thesis work of Wai Liu (Stanford) gives useful families of priors where computations are "easy." These seem to work well in test problems and can be proved consistent.