Author: Tara Fowler, RRT CHT MHS

Hypothesis testing uses a probability value (p-value) to determine the statistical significance of experimental results (Lee, 2019). The significance of the study’s conclusion is typically assessed with an index referred to as p-value. The determination of the statistical significance of an experimental hypothesis is based upon three key elements: hypothesis testing, normal distribution and p-values.

Hypothesis Testing

Hypothesis testing in statistics is a way to test the results of an experiment to determine if the results are meaningful. It tests whether the results are valid by figuring out the odds that the results have happened by chance (Glen, 2020). If the results have happened by chance, the experiment won’t be repeatable and has little use. Science uses a battery of processes to prove or disprove theories, making sure than any new hypothesis has no flaws. Inclusion of both a null and an alternate hypothesis is one safeguard to ensure research isn’t flawed.

The null hypothesis (H0) is the commonly accepted fact; it is the opposite of the alternate hypothesis. A researcher works to reject or disprove the null hypothesis. Researchers come up with an alternate hypothesis; one that they think explains a phenomenon, and then work to reject the null hypothesis. The researcher makes a claim (H0) and then uses sample data to check if the claim is valid. When the claim isn’t valid, the researcher then is able to choose the alternative hypothesis instead.

To know if a claim is valid or not, the researcher calculates a p-value to weigh the strength of the evidence to see if it’s statistically significant. If the evidence supports the alternative hypothesis, then the null hypothesis is rejected, and the alternative hypothesis is accepted.

Normal Distribution

The normal distribution is the most important probability distribution in statistics because many continuous data in nature display this bell-shaped curve when compiled and graphed. The most powerful (parametric) statistical tests used by statisticians require data to be normally distributed. If the data does not resemble a bell curve, researchers may have to use a less powerful type of statistical test, called non-parametric statistics. Data values in a normal distribution are converted to z-scores in a standard normal distribution (McLeod, 2019).

The empirical rule describes the percentage of the data that fall within specific numbers of standard deviations from the mean for bell-shaped curves. The empirical rule allows researchers to calculate the probability of randomly obtaining a score from a normal distribution (McLeod, 2019).

P-Values

A P-value is a number calculated by running a hypothesis test on the collected research data. The smaller the P-value, the stronger the evidence is to reject the null hypothesis. A P-value of 0.05 (5%) or less is usually enough to claim that the results are repeatable. A P-value of less than 0.001 is considered highly statistically significant (less than one in a thousand chance of being wrong).

The P-values only mean the probability of accepting the null hypothesis and do not mean the probability of accepting the ‘study hypothesis (Vidgen & Yasseri, 2016). The P-values do not tell how 2 groups are different. The degree of difference is referred as ‘effect size’.

Statistical significance is not equal to scientific significance. P-values alone cannot confirm whether the researcher’s argument is correct or not. It is recommended that the proper inference should not be based solely on the P-values (Vidgen & Yasseri, 2016). Contextual factors should also be considered to derive scientific inferences. Factors such as study design, the quality of the measurements, and the logical basis of the assumptions for the study are also important.

References

Glen, S. (2020). Hypothesis testing from statistics. HowTo.com: Elementary Statistics for the rest of us!
Cited from : https://www.statisticshowto.com/probability-and-statistics/hypothesis-testing/

Lee, A. (2019). P-values Explained By Data Scientist. Medium.
https://towardsdatascience.com/p-values-explained-by-data-scientist-f40a746cfc8

McLeod, S. A. (2019, May 28). Introduction to the normal distribution (bell curve).
Simply psychology: https://www.simplypsychology.org/normal-distribution.html

Vidgen B and Yasseri T (2016) P-Values: Misunderstood and misused. Front. Phys. 4:6. doi: 10.3389/fphy.2016.00006