Markov chains
Supplemental study note
Older Chapter 4 supplement + original explanation
Supplemental status
Markov chains appear in the older Chapter 4 handout but are not explicit in the current course outline.
A transition matrix is row-stochastic: entries are nonnegative and each row sums to 1. A state distribution moves one step by multiplying the row distribution by the transition matrix. Repeating this process gives d P^k.