What is a p-value?

This value does not mean the probability the null hypothesis is true.

The p-value is a misunderstood statistic. Sometimes, textbooks carry incorrect definitions — perpetuating misinterpretations.

For example, a Twitter user shared a 2003 biology textbook definition:

The P-value is the bottom line of most statistical tests. It is simply the probability that the hypothesis being tested is true. So if a P-value is given as 0.06, that indicates that the hypothesis has a 6% chance of being true.

This is wrong.

A better definition of the p-value is, from the American Statistical Association:

the probability under a specified statistical model that a statistical summary of the data would be equal to or more extreme than its observed value.

Often, you see this definition where the specified model is a null (or nil) hypothesis. One such hypothesis is of no difference between studied groups.

Despite its brevity, there are many aspects of this definition to examine.

The p-value indicates compatibility between the model and data. The calculation assumes a particular model holds. The p-value is then a measure of how extreme the observed data is. With low compatibility, this value provides evidence against the hypothesis or underlying assumptions.