a hidden markov model (HMM) is a probabilistic sequence model: given a sequence of units (words, letters, morphemes, sentences, whatever), it computes a probability distribution over possible sequences of labels and chooses the best label sequence.
a hidden markov model allows us to talk about both observed events (like words that we see in the input) and hidden events (like part-of-speech tags) that we think of as causal factors in our probabilistic model. an HMM is specified by the following components:
- hmm-states
: a set of
states,
- hmm-trans-prob
: a transition probability matrix
. each
representing the probability of moving from state
to state
,
,
- hmm-obs
a sequence of
observations, each one drawn from a vocabulary
,
- hmm-obs-like
: a sequence of observation likelihoods, also called emission probabilities, each expressing the probability of an observation
being generated from a state
,
- hmm-init-prob
: an initial probability distribution over states.
is the probability that the markov chain will start in state
. some states
may have
, meaning that they cannot be initial states. also,
.
a first-order hidden markov model instantiates two simplifying assumptions. first, as with a first-order markov chain, the probability of a particular state depends only on the previous state:
second, the probability of an output observation
depends only on the state that produced the observation
and not on any other states or any other observations: