Summary:
Hidden Markov models are used in a wide range of applications due to their construction that
renders them mathematically tractable and allows for the use of efficient computational techniques.
There are methods for the estimation of the model’s parameters, such as the EM algorithm, but also
for the estimation of the hidden states of the underlying Markov chain, such as the Viterbi algorithm.
In applications where the dimension of the data is comparable to the sample size, the sample
covariance matrix is known to be ill-conditioned, which directly affects the maximisation step (M-
step) of the EM algorithm, where its inverse is involved in the computations. This problem might be
amplified if there are rarely visited states resulting in a small sample size for the estimation of the
corresponding parameters. Therefore, the direct implementation of these methods can be proved to
be troublesome, as many computational problems might occur in the estimation of the covariance
matrix and its inverse, further affecting the estimation of the one-step transition probability matrix
and the reconstruction of the hidden Markov chain.
In this paper, a modified version of the EM algorithm is studied, both theoretically and computa-
tionally, in order to obtain the shrinkage estimator of the covariance matrix during the maximisation
step. This is achieved by maximising a penalised log-likelihood function, which is also used in the
estimation step (E-step). A variant of this modified version, where the penalised log-likelihood func-
tion is only used in the maximisation step (M-step), is also studied computationally.
Keywords:
Hidden Markov Models, EM Algorithm, Shrinkage, NYSE, AMEX, NASDAQ
