The use and misuse of p-values

The American Statistical Association task-force shares a statement.

Back in 2019, the President of the American Statistical Association (ASA) launched a task-force into p-values. This task-force started after an editorial in The American Statistician, an ASA journal. People may have mistaken that editorial for official ASA policy.

An informal definition of a p-value is:

the probability under a specified statistical model that a statistical summary of the data would be equal to or more extreme than its observed value.

The task-force’s statement says:

Much of the controversy surrounding statistical significance can be dispelled through a better appreciation of uncertainty, variability, multiplicity, and replicability.

Often, journal articles will use p-values as a statistical summary.

Principles of use

The statement offers four principles of proper usage of p-values.

• Capturing uncertainty is critical: Different measures of uncertainty complement one another. Reports should describe sources of variation.
• Replicability and uncertainty are at the heart of statistics: A study replicates if similar results follow from similar studies. There is inherent uncertainty, even in…