Statistical Power in Hypothesis Testing

Discussing power and what its influences are in hypothesis testing

Image Source

Effect Size and Cohen’s D

The extent of the difference between the two samples that you are studying is called the effect size. You can use a t-test to decide if the coin is a fair coin after flipping the coin n number of times, with a null hypothesis (H0) of H0(tails)=0.5. To do this, if you compared two different coins or you compared the coin you’re testing against a known distribution, you would compare the mean of the sample to that of the other sample. In these instances, the metric often used for the effect size is Cohen’s D. By mathematical definition, Cohen’s D is d=m1-m2/s, where m1 and m2 are the respective means of the samples and s is the samples’ total standard deviation. In (hopefully) simpler terms, Cohen’s D is equivalent to the difference of the sample means divided by the combined standard deviation of the samples when looking at the difference of means between two populations. The combined standard deviation of the samples is the average allocation of all the pieces of data across the collective mean for the two samples.

Analyzing Power

When calculating power for a statistical test, there are three aspects to be considered: alpha, effect size and sample size. You can take a look at the plots of the power of your t-tests (provided there are differing sample sizes), which will allow you to gain a better understanding of the relationship between these quantities and what comprises a compelling statistical test.

Fitness, Sports, Data — And not necessarily in that order

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store