We already say that the Hypothesis testing rejects or does not the H0 basis on some acceptance criteria. In do that, the test can make an error and the error ar of two types:
- Type 1 error: The null hypothesis is correct, but we reject it for error;
- Type 2 error: The null hypothesis is incorrect, but it’s not rejected.
We already know that the acceptable risk of the Type 1 error is the probability alpha, and It also called alpha level. The acceptable risk of the Type 2 error is the probability beta, also called beta level.
We want to discuss how to reject the hypothesis H0 because this depends on our alpha level. There are two methods:
- Using the critical value;
- Using the p-value.
The critical value method is the easiest to explain. You already know that the critical value divides the critical region, that is, the rejection area, from the area of non-rejection.
So with this method, you calculate the statistics defined in the chosen hypothesis test and then compare the statistic with the critical value. This comparison depends on the kind of test:
- Two-tail test: you have two critical values, a left, and a right. If your statistic isn’t among these two values, you reject H0;
- Left-tail test: you have only one left critical value. If your statistic is smaller (at left) than this critical value, you reject H0;
- Right-tail test: you have only one right critical value. If your statistics are bigger (is at right) than this critical value, you reject H0.
You get the critical value from the table for the chosen statistics using the correct alpha level. For a two-tail test, remember that you have two critical values and use alpha/2.
The P-value is the probability, assuming H0 true, of seeing data at least as extreme as the experimental data. In other words, the p-value helps to understand if the variability of the data depends on the sampling techniques (so it’s there by chance) or there is a statistical significance difference (there is an assignable cause of variability).
So high P-value means that you have high probability of look an extreme value only by chance. So not worry, you H0 is still true.
On the other hands, low p-value means that there is low probability that the extreme is looked only by chance so, with high probability the H0 hypothesis is false and you need to reject it.
But how the p-value method works?
The P-value method, like the other one, required that you calculate the specific statistics of your test. Next, you need to pass from the value of the statistics to the probability of having that value (of the statistics), assuming H0 true that is the p-value (or probability value). You can calculate the p-value in a different way depending on the type of test:
- Two-tail test: You have one probability for each tails, Pr(T >= t| H0) and Pr(T<=t|H0). To calculate the p-value, you need to get the smaller one and multiply for 2;
- Left-tail test: You have only the probability Pr ( T <= t | H0) that is you P-Value;
- Right-tail test: You have only the probability Pr( T >= t | H0) that is you P-Value.
How to calculate this probability? Depends on the distribution of your sample. You can find a table for this value in most cases, but you need not get confused by the critical value table. In fact, with the P-Value, you know t, and you want to get the probability of t. With the critical value methods, you know the probability of your critical value (that is alpha or alpha/2), and you want to get t.
After you have calculated your P-value (in all the three cases above) if P-value <= alpha we reject H0.
In conclusion, don’t worry if sometimes you will look at the test run with the critical value and, in another case, with the p-value. Are just two alternative ways to run a hypothesis test; it’s only essential to doesn’t make confusion. In addition, remember that you can use software like excel and Minitab to compute this test in real life, but only by knowing how the test works can you use this software in the right way.
For a step-by-step example of the critical value, I suggest looking at the one-sample t-test chapter.
For a step-by-step example of the p-value, I suggest looking at the one-sample sign chapter.