We will first consider the computation of the expected number R(i, j) of visits to j and the probability F(i, j) of ever reaching j, both starting at i. Considering the recurrent states we will show how to compute the probability Pn(i, j) of being in state j at time n, starting at i, for large n. Studying the periodic states is easily reduced to the aperiodic case, and we will show how to do that. Then we will take up the computation of the probability of remaining in a set of transient states forever. Finally, we will give a brief treatment of two important models in queueing theory and an
...more
