Markov chain — random walk on a 4-state graph

Each step moves from the current state to a neighbour chosen with probability given by the transition matrix P (edge thickness ∝ probability). The left panel is the transition graph with the current state highlighted; the right panel tracks the empirical visit frequency of each state, which converges to the stationary distribution π (red markers).

steps 0 current state A π = (...)

Transition graph

Empirical frequency vs stationary π

empirical frequency stationary π current state