Brief review of conditional probability and expectation followed by a study of Markov chains, both discrete and continuous time. Queuing theory, terminology, and single queue systems are studied with ...

A Markov-modulated Poisson Process (MMPP) is a Poisson process that has its parameter controlled by a Markov process. These arrival processes are typical in communications modeling where time-varying ...

The Annals of Statistics, Vol. 4, No. 6 (Nov., 1976), pp. 1219-1235 (17 pages) The paper deals with continuous time Markov decision processes on a fairly general state space. The rewards are ...

It is shown that sequences of generalized semi-Markov processes converge in the sense of weak convergence of random functions if associated sequences of defining elements (initial distributions, ...

This paper proposes a hidden state Markov model (HMM) that incorporates workers’ unobserved labor market attachment into the analysis of labor market dynamics. Unlike previous literature, which ...