MLE's bias pathology, model updated MLE, and Wallace's minimum message length method

Abstract

The inherent bias pathology of the maximum likelihood estimation method is confirmed for models with unknown parameters θ and ψ when maximum likelihood estimate (MLE) ψ is function of MLE θ. To reduce ψ's bias the likelihood equation to be solved for ψ is updated using the model for the data Y in it. For various models with ψ a shape parameter model updated (MU) MLE, ψMU, reduces ψ's bias. For the Pareto model ψMU reduces in addition ψ's variance. The results explain the difference that puzzled Fisher, between biased ψ and the unbiased estimate he obtained for two models with the abandoned two-stage procedure used in MUMLE's implementation. ψMU is also obtained with the minimum message length method thus motivating the use of priors in frequentist inference.