How do you calculate the p value?

by Posted on

How do you calculate the p value?

P values are usually automatically calculated by your statistical program (R, SPSS, etc.).

You can also find tables for estimating the p value of your test statistic online. These tables show, based on the test statistic and degrees of freedom (number of observations minus number of independent variables) of your test, how frequently you would expect to see that test statistic under the null hypothesis.

The calculation of the p value depends on the statistical test you are using to test your hypothesis:

  • Different statistical tests have different assumptions and generate different test statistics. You should choose the statistical test that best fits your data and matches the effect or relationship you want to test.
  • The number of independent variables you include in your test changes how large or small the test statistic needs to be to generate the same p value.
Example: Choosing a statistical test
If you are comparing only two different diets, then a two-sample t test is a good way to compare the groups. To compare three different diets, use an ANOVA instead – doing multiple pairwise comparisons will result in artificially low p values and lets you overestimate the significance of the difference between groups.

No matter what test you use, the p value always describes the same thing: how often you can expect to see a test statistic as extreme or more extreme than the one calculated from your test.

P values and statistical significance

P values are most often used by researchers to say whether a certain pattern they have measured is statistically significant.

Statistical significance is another way of saying that the p value of a statistical test is small enough to reject the null hypothesis of the test.

How small is small enough? The most common threshold is p < 0.05; that is, when you would expect to find a test statistic as extreme as the one calculated by your test only 5% of the time. But the threshold depends on your field of study – some fields prefer thresholds of 0.01, or even 0.001.

The threshold value for determining statistical significance is also known as the alpha value.

Example: Statistical significance
Your comparison of the two mouse diets results in a p value of less than 0.01, below your alpha value of 0.05; therefore you determine that there is a statistically significant difference between the two diets.

Published by Henry