The Legal Sequelae of the 2016 American Statistical Association P-Value Statement

Nathan A. Schachtman

Monday August 5 7-8:30

In 2016, the American Statistical Association issued an unusual guidance document in which it attempted to redress its perception that p-values and statistical significance were widely misunderstood and misinterpreted. In addition to providing guidance on the meaning and use of attained significance probabilities, the ASA also encouraged the use of “other methods that emphasize estimation over testing,” including Bayesian methods. Although the ASA guidance document warned against misuses of p-values, it did not warn of the potential for misapplication of these “other methods.” The reaction of some segments of the legal community was prompt, both in interpreting the 2016 guidance as a rejection of p-values and significance testing, as well as an encouragement to use “other methods,” for which the judiciary would have far less experience and acumen to detect invalid inferences. In this presentation, I will discuss how the ASA Statement was used rhetorically to justify causal claims that had been rejected by the FDA and scientific organizations, and to advance a “Bayesian hypothesis” test to support a claim that a meta-analysis showed that there was an 85 percent probability that testosterone replacement therapy caused either heart attack or stroke. (Fuller Discussion (pdf))