A natural extension of a DTMC occurs when time is treated as a continuous parameter. In this section, we consider continuous-time, discrete-state stochastic processes but limit our attention to the case where the Markovian property holds; that is, the future realization of a system depends only on the current state and the random events that proceed from it.This is call a Continuous Time Markov Chain (CTMC). Some times we use the term Markov Process for this kind of system.

It happens that the Markov property is only satisfied in a continuous-time stochastic process if all activity durations are exponentially distributed. Although this may sound somewhat restrictive, many practical situations can be modeled as CTMC and many powerful analytical results can be obtained. A primary example is an M/M/s queueing system in which customer arrivals and service times follow an exponential distribution. Because it is possible to compute the steady-state probabilities for such systems, it is also possible to compute many performance-related statistics such as the average wait and the average number of customers in the queue. In addition, many critical design and operational questions can be answered with little computational effort.