Application of credibility ceilings probes the robustness of meta-Analyses of biomarkers and cancer risk

Abstract

Objectives Meta-Analyses of biomarkers often present spurious significant results and large effects. We applied sensitivity analyses with the use of credibility ceilings to assess whether and how the results of meta-Analyses of biomarkers and cancer risk would change. Study Design and Setting We evaluated 98 meta-Analyses, 43 (44%) of which had nominally statistically significant results. We assumed that any single study cannot give more than a maximum certainty 100 - c% (c, credibility ceiling) that the effect estimate [odds ratio (OR)] exceeds 1 (null) or 1.2. Results Nominal statistical significance was maintained for 21 (21%) meta-Analyses, for c = 10% and OR >1, and these proportions changed to 7%, 3%, and 6% with ceilings of 20%, 30%, and 40%, respectively. For ceilings for OR >1.2, the respective proportions were 37%, 21%, 7%, and 3%. Seven meta-Analyses on infectious agents retained statistical significance even with a high ceiling of c = 20% for OR >1.00. Meta-Analyses without other hints of bias (large between-study heterogeneity, small-study effects, excess significance) were more likely to retain statistical significance than those that had such hints of bias. Conclusion Credibility ceilings may be helpful in meta-Analyses of biomarkers to understand the robustness of the results to different levels of uncertainty.