MLE's bias pathology, model updated MLE, and Wallace's minimum message length method
Journal
IEEE Transactions on Information Theory
Date Issued
March 1, 2015
Author(s)
Abstract
The inherent bias pathology of the maximum likelihood estimation method is confirmed for models with unknown parameters θ and ψ when maximum likelihood estimate (MLE) ψ is function of MLE θ. To reduce ψ's bias the likelihood equation to be solved for ψ is updated using the model for the data Y in it. For various models with ψ a shape parameter model updated (MU) MLE, ψMU, reduces ψ's bias. For the Pareto model ψMU reduces in addition ψ's variance. The results explain the difference that puzzled Fisher, between biased ψ and the unbiased estimate he obtained for two models with the abandoned two-stage procedure used in MUMLE's implementation. ψMU is also obtained with the minimum message length method thus motivating the use of priors in frequentist inference.
Affiliation