What is a Markov Chain?

Suppose you have a system that changes state over time, with some random variability. The time sequence of states of the system is called a stochastic process. A stochastic process is a Markov chain if at any point in time, the probability of future states is only dependent on the current state, not on anything that has gone before. Another term used is memoryless. The system does not remember what happened before in the sense that future states are independent of past states.

Consider a model of Brownian motion:

• There are #n# particles of pollen which we keep track of.
• The particles of pollen tend to keep moving in the same direction with the same velocity.
• The movement is changed by the random impact of unseen smaller particles which we are not tracking.

If our model only maintained a state which only described the instantaneous position of each particle of pollen at each time #t# then this would not be a Markov chain: The future position of each particle is influenced by the current velocity of the particle too - and probably its angular momentum.

If however the model maintained a state which described not only the position of each pollen particle, but also its current velocity and angular momentum then it probably is a Markov chain.