let denote the joint pdf or pmf of the sample . then, given that is observed, the function of defined by
is called the likelihood function.
if is a discrete random vector, then . if we compare the likelihood function at two parameter points and find that
then the sample we actually osberved is more likely to have occurred if than if which can be interpreted as saying that is a more plausible value for the true value of than is . many different ways have been proposed to use this information, but certainly it seems reasonable to examine the probability of the sample we actually observed under various possible values of . this is the information provided by the likelihood function.
if is a continuous, real-valued random variable and if the pdf of is continuous in , then, for small , is approximately (this follows from the definition of a derivative). thus,
and comparison of the likelihood function at two parameter values again gives an approximate comparison of the probability of the observed sample value, .
def-like almost seems to be defining the likelihood function to be the same as the pdf or pmf. the only distinction between these two functions is which variable is considered fixed and which is varying. when we consider the pdf or pmf , we are considering as fixed and as the variable; when we consider the likelihood function , we are considering to be the observed sample point and to be varying over all possible parameter values.
let be sample observations taken on corresponding random variables whose distribution depends on a parameter . then, if are discrete random variables, the likelihood of the sample, , is defined to be the joint probability of .
if are continuous random variables, the likelihood is defined to be the joint density evaluated at .