Return to Index
Operations Research Models and Methods
 
Computation Section
Subunit Discrete-Time Markov Chain
 - Steady State Analysis

The Steady State worksheet computes the Markov Chain steady state probabilities. The average cost per period is computed at the right. The expected net present worth depends on the initial state and is computed in the vector below the steady-state vector.

For the example, we have noted that the transient solution approaches a steady state, and indeed this vector is computed independently on this worksheet. Steady state results are independent of the initial state, and they are available only for Markov Chains that have a single recurrent class.

 

The entry in B4 indicates that the steady-state is solved with a formula involving the inverse of the transition matrix located elsewhere on the worksheet (it is hidden). The yellow fields hold formulas that should not be changed. The steady-state probabilities are linked the data in the transition matrix through these formulas.

For problems with more than 20 states, the inverse matrix may be difficult or impossible to compute. The steady-state probability vector is a solution to a set of linear equations that can be obtained with the Excel Solver add-in. For problems with more than 20 states, the program asks the user whether to use the Solver or the inverse. With the Solver option, the program constructs a Solver model. Clicking the Solve SS button finds the solution and places it in the green cells holding the steady-state probabilities. The figure below shows part of the solution to a randomly generated DTMC with 30 states.

   
 
  
Return to Top

tree roots

Operations Research Models and Methods
Internet
by Paul A. Jensen
Copyright 2004 - All rights reserved