Preview
IB DP Mathematics: Applications and Interpretation | HL | Statistics and Probability | Topic 4.19
📋 Learning Intentions
- Define a Markov chain and state the conditions required for a sequence of events to be modelled as a Markov chain.
- Construct a transition matrix from a transition diagram or a table of conditional probabilities.
- Calculate state probability vectors after \(n\) steps using repeated matrix multiplication: \(\mathbf{s}_n = T^n \mathbf{s}_0\).
- Find the steady-state (equilibrium) distribution by solving \(T\mathbf{s} = \mathbf{s}\) or by raising \(T\) to a large power on the GDC.
- Interpret the long-run behaviour of a Markov chain in context, including what the steady-state distribution means for a real-world system.
- Justify whether a unique steady-state distribution exists for a given transition matrix.
Real-World Introduction
A supermarket tracks customer loyalty: each week, a shopper either stays with the same brand or switches to a competitor. The probability of switching depends only on where they shopped this week — not on any previous history. This memoryless property is the defining feature of a Markov chain, and it makes long-run prediction tractable.
The same mathematical structure models weather patterns (sunny today → probability 0.7 of sunny tomorrow), population migration between cities, PageRank (Google's original algorithm for ranking web pages), and the spread of disease between compartments. In every case, the system moves between a finite set of states, and the transition probabilities are fixed.
🌦️ Analogy — Weather Forecasting Think of a simplified weather model with two states: Sunny (S) and Rainy (R). If it is sunny today, there is a 70% chance it will be sunny tomorrow and a 30% chance of rain. If it is rainy today, there is a 40% chance of sun tomorrow and a 60% chance of rain. A Markov chain captures exactly this: the next state depends only on the current state, and the transition matrix stores all four probabilities in one compact object.
Core Concepts & Definitions
Before writing any matrix, it is essential to understand what a Markov chain actually is. The key insight is that the system has no memory: the probability of moving to any future state depends only on the current state, not on how the system arrived there. This is called the Markov property .
🔑 Key Definition — Markov Chain A Markov chain is a sequence of random variables \(X_0, X_1, X_2, \ldots\) taking values in a finite set of states \(\{1, 2, \ldots, k\}\) such that: \[ P(X_{n+1} = j \mid X_n = i, X_{n-1}, \ldots, X_0) = P(X_{n+1} = j \mid X_n = i) \] The probability of transitioning from state \(i\) to state \(j\) depends only on the current state \(i\), not on any earlier history. These probabilities are called transition probabilities and are assumed to be constant (time-homogeneous).
The transition probabilities are collected into a square matrix called the transition matrix \(T\). Each entry \(T_{ij}\) gives the probability of moving from state \(i\) to state \(j\) in one step. Because these are probabilities, each row must sum to 1 — this is the defining property of a stochastic matrix .
🔑 Key Definition — Transition Matrix For a Markov chain with \(k\) states, the transition matrix \(T\) is a \(k \times k\) matrix where:
- Entry \(T_{ij}\) = probability of moving from state \(i\) to state \(j\)
- Every entry satisfies