Real World Applications of Markov Decision Process (MDP)

mdpreinforcement-learningdecision-makingtutorial

Abstraction: Real-world finite MDP formulations with worked transition graph examples

Key points:

MDP requires defining states, actions, rewards, and state transition probabilities; Markov property means actions depend only on the current state
Four worked examples: salmon fishing yield optimization, quiz game show play-or-quit decisions, traffic light timing, hospital bed admission management
Transition graphs use large circles for state nodes and small solid black circles for action nodes; arrows show (transition probability, reward)
Salmon example illustrates shaped rewards: fishing at "high" state yields $100K but risks transitioning to "empty" which incurs $200K re-breeding cost
Traffic light example uses exponentially growing penalty for blocking opposing traffic direction, satisfying Markov property by encoding duration in the state
Reference: Sutton and Barto "Reinforcement Learning: An Introduction" (2nd ed.) Chapter 3

Connections: Sutton Barto · Markov Decision Process · Reinforcement Learning · Dynamic Programming