Statuses

Richard Morey

Statistical Forensics

Wednesday, July 31, 7-8:30
Reader, School of Psychology
Cardiff University

Presentation Abstract: Most of scientific work happens behind the scenes: every published scientific result is an intermediate step in a process that may have taken years to realize. Opportunistic analyses, publication bias, and outright fraud may be important to consider when trying to assess the trustworthiness of a result, but they are, of course, not mentioned in the report. However, they might leave statistical traces: for instance, Fisher (1936) pointed out that Mendel’s (1866) results with pea plants appeared to be too close to their theoretical values to be accounted for by chance variation, possibly intentionally falsified either for didactic reasons or by an assistant trying to please Mendel. Similar methods have been at the center of assessing the credibility of more recent research, but the essential character of modern methods is a straighforward extension of Fisher’s logic (which itself is significance testing). We can call these “statistical forensics” methods whose goal is to shed light on whether a body of research is trustworthy and perhaps to try to correct for issues that might cause doubt. I will outline some of these methods, describe where they have been used in practice, and discuss potential objections.