Systematic image screening at EMBO Press uncovers many problems, most of which can be resolved. Standardized pre‐publication image screening would make for a more reliable literature. Flagging putative image manipulation post‐publication remains an important control mechanism, but must not fall victim to an overzealous response. The STAP stem cell papers are a case in point.
The EMBO Journal undertakes systematic manual screens for image aberrations in manuscripts that passed peer review. Using image processing routines similar to those suggested by the US Office of Research Integrity and applied by the Journal of Cell Biology, our data integrity analyst uncovers problems in around one‐fifth of the manuscripts assessed—excluding statistics, missing information such as MW markers and excessive contrast adjustments or cropping. The issues range from non‐disclosed vertical splice marks in blots and reassembled micrographs to the selective deletion of data, image duplication, and the cloning of bands. The high incidence of these problems—all of which would have minimally led to a corrigendum in the journal after publication—is cause for concern and appears to betray a lack of training and supervision at a time when high publishing pressures can conspire with digital tools that make illicit or overzealous image processing all too easy.
Publishing real data
Part of the problem is the widespread perception that journals want to publish ‘clean’ and ‘striking’ data. Actually, reputable journals want to publish well controlled, reproducible, compelling, and informative data framed by careful interpretation. Biological data are complex, and findings cannot always be shoehorned into simple conclusions or made to fit ‘dogma’ (a word that we think should be banned from biology).
Research papers present data contextualized by the author's description and interpretation. The data should be presented in a form that allows re‐analysis and integration with other research. Scientific advances usually progress in increments, building on the discoveries of others. It is therefore essential that researchers can access and extract ‘real data’—what we call Source Data—from papers. We have successfully encouraged publication of such minimally processed data—including the display of replicates—alongside figures for several years, adding transparency and depth to papers (Pulverer, 2014).
When to blow the whistle
Any image aberrations uncovered by our referees, editors, or data analyst are carefully evaluated in the context of the data affected and the type of manipulation to decide what action to take: was it a careless mistake or down to a lack of awareness of acceptable image processing, or could there have been intent to deceive? Can we find out by requesting source data? Will the author's explanations resolve the issues beyond doubt or is an institutional investigation necessary? Consider Fig 1, an example where we can assume a lack of intent to deceive and publish the unprocessed figure. Remarkably, we regularly find image duplications, offset or inverted, that turn out to be mistakes.
We are not able to catch carefully executed fraud, but rather we uncover cases of incompetent image processing, beautification, and mistakes—very occasionally serious image manipulation. We do not claim to be—nor do we wish to be—data police; we aim to prevent mistakes and unacceptable image processing from entering the literature. We know that mistakes happen all too easily as we try to navigate today's digital data flood. Also, there remain divergent sensitivities to allowable image processing and a lack of training and education. We see no reason to fundamentally question the veracity of the literature—we retain full trust in molecular biology research.
We hope that a similar ethos is adopted by whistle blowers—occasional or full time, anonymous or open. Importantly, we must avoid judging the whole literature by today's standards. Good scientific practice has not fundamentally changed, but the nature of the data and its acquisition have changed dramatically. It is unreasonable to hold researchers accountable for not retaining lab books and original data after the data archiving mandates of their institutions and funders have expired.
Nobody will have missed the furore around the two STAP stem cell papers published earlier this year, after image manipulations surfaced, followed by concerns about reproducibility (Obokata et al, 2014a,b). The claims made were both groundbreaking and extremely surprising. While research institutions and journals cannot be expected to independently validate the reproducibility of findings before publication, they do have an obligation to take reasonable measures for quality assurance. The more important and surprising the finding, the higher the stakes; in our view, it follows that a higher level of scrutiny be applied pre‐publication. Once a questionable finding enters the canon of the scientific literature, it starts to mislead other researchers, at minimum squandering resources. Importantly, it also has a toxic effect in generally undermining trust in scientific research—not least by the public.
To assess our own image screens, when the first comments circulated about the STAP papers, we asked our image analyst to undertake her routine screen on the papers. She readily identified the image aberrations and duplications (as well as another splice mark, but not the duplications from the thesis). In this case, we feel confident that the issues could have been caught before publication through routine screens.
In the wake of the STAP case, research in Japan has come under much greater scrutiny and a number of researchers approached us about problems with their own published papers. This is good, as is circumspect whistle blowing—anonymous or not. What we must avoid is the evolution of a vigilante culture that risks dampening scientific progress and innovation, and sacrifice the careers of innocent researchers. When researchers like Shinya Yamanaka feel under pressure to publicly announce that anonymous allegations about a putative band duplication in an EMBO Journal paper from 2000 was found to be baseless, the pressures on scientists and an open research environment have gone too far. Our own analysis concludes that there is no definitive evidence at the resolution published that the bands are identical (see Fig 2).
The systematic archiving of source data, where possible alongside the figures in papers, and the publication of anonymous referee reports with papers will do much to alleviate potential concerns.
- © 2014 The Author