MLE's bias pathology, model updated MLE, and Wallace's minimum message length method
Journal
IEEE Transactions on Information Theory
Date Issued
March 1, 2015
Author(s)
DOI
10.1109/TIT.2014.2386329
10.1109/TIT.2014.2386329
Abstract
The inherent bias pathology of the maximum likelihood estimation method is confirmed for models with unknown parameters θ and ψ when maximum likelihood estimate (MLE) ψ is function of MLE θ. To reduce ψ's bias the likelihood equation to be solved for ψ is updated using the model for the data Y in it. For various models with ψ a shape parameter model updated (MU) MLE, ψMU, reduces ψ's bias. For the Pareto model ψMU reduces in addition ψ's variance. The results explain the difference that puzzled Fisher, between biased ψ and the unbiased estimate he obtained for two models with the abandoned two-stage procedure used in MUMLE's implementation. ψMU is also obtained with the minimum message length method thus motivating the use of priors in frequentist inference.