Algorithmically outsourcing the detection of statistical errors and other problems

Scientists are well aware that publications are not necessarily error‐free, even if they have undergone peer review. To some extent, the publication record is self‐correcting by virtue of new published findings, but we still rely upon the scientific literature as our most authoritative source of established knowledge. Thus, minimizing errors in publications is important, but how important it is depends on how frequent the errors are, their magnitude, and the probability they will lead to erroneous conclusions or wasted effort in reproducing results. Recent studies have begun to address the first two issues by quantifying the frequency and magnitude of mathematical and statistical discrepancies in the literature algorithmically. This creates an interesting situation: if discrepancies/errors can be found algorithmically after publication, they can also, in theory, be found and corrected before publication. The purpose of this commentary is to discuss errors from the standpoint of studying them systematically, summarize what some of …