Mason Archival Repository Service

Recursive Parameter Estimation for Continuous-time Bivariate Markov Chains

Show simple item record

dc.contributor.advisor Ephraim, Yariv Li, Zhuxuan
dc.creator Li, Zhuxuan 2014-05-12 2014-10-21T20:16:12Z 2014-10-21T20:16:12Z 2014-10-21
dc.description.abstract A continuous-time bivariate Markov chain comprises a pair of continuous-time random processes which are jointly Markov. One of the two processes is an underlying process while the other is assumed observable. An important special bivariate Markov chain is given by the continuous-time Markov modulated Poisson process (MMPP). The underlying process of an MMPP is a Markov chain, and the observable process is conditionally Poisson. Discretetime bivariate Markov chains may also be de ned, but they shall not be studied in this thesis. Bivariate Markov chains are useful in modeling ion-channel currents in living cells, Internet tra c, and in other problems in queuing theory. In this thesis we focus on recursive estimation of the parameter of a bivariate Markov chain which comprises its in nitesimal generator. We study a stochastic approximation approach using a newly developed recursion for the gradient of the log-likelihood function of the observed signal. The recursive algorithm is compared with a batch expectation-maximization (EM) algorithm developed earlier. The bias and mean squared error obtained in estimating each component of the parameter using each algorithm are evaluated and compared. The recursive algorithm requires far more data to provide an estimate comparable to that obtained by the EM algorithm, but the EM algorithm iterates over the entire data multiple times. The main advantage of the recursive estimator is its ability to adapt to slow changes in the underlying statistics of the model. The EM algorithm is a batch approach which must be re-applied whenever new data becomes available or there is a change in the underlying statistics of the model.
dc.language.iso en_US en_US
dc.rights Copyright 2014 Zhuxuan Li en_US
dc.subject Bivariate Markov chain en_US
dc.subject Markov modulated Poisson process en_US
dc.subject recursive parameter estimation en_US
dc.subject EM algorithm en_US
dc.title Recursive Parameter Estimation for Continuous-time Bivariate Markov Chains en_US
dc.type Thesis en Master of Science in Electrical Engineering en_US Master's en Electrical Engineering en George Mason University en

Files in this item

This item appears in the following Collection(s)

Show simple item record

Search MARS


My Account