OK, so John Lennon didn’t really write this , but statistical power is a very abstract concept and the ability to “imagine” really helps.
Power is the probability that you will find a statistically significant difference in your study SAMPLE if it truly exists in the larger POPULATION.
Beta is the probability that you will not be able to detect a difference if it is truly there in the population.
The study you can’t afford: There are 72,000,000 people with hypertension in the U.S. If you could study them all you would find that the new drug Lowpressure® lowers blood pressure by 7mm more than Whocaresapine.
To test the difference on an affordable scale, you need:
Power Calculations Before the Study !!
We will describe the process in four simple steps.
Decide how big a difference you consider clinically important.
For our Hypothetical Dilemma Example: You think a 7mm difference or more is clinically important.
How variable is the outcome we are testing? (this is a guess, based on available facts)
Fact: The measure of variability used in power calculations is variance or (standard deviation)2
Hypothetical Dilemma: From previous studies, we know that standard deviation of the mean blood pressure has been 5mmg Hg ( so variance for the calculation would be 25 (5)2)
Fact: The more variable the outcome, the more difficult it is to be statistically confident that the difference you observe is real and not due to random chance (or variation)
Fact: the more variable the data , the more people you have to study to get statistical significance.
How sure do we need to be? The usual beta is 0.20 (giving a power of 80%)
If you haven’t picked this up yet, One minus beta = Power.
The Dreaded Calculation Because the concept is the important thing, we will spare you the headache of the power equation and just tell you that these assumptions yield a calculated N required of approximately 100 in each group. Hypothetical Dilemma Example: You will need to study 200 people, randomized to the drug “Lowpressure” or the drug “Whocaresapine” to have an 80 percent power to detect a 7mm difference or more.
However, studies often don’t enroll exactly the number of people they need so you may have to do ….
Power Calculations After the Study !!
Don’t despair, there are only three steps for this part.
If you found a statistically significant difference (p less than 0.05)…You had enough POWER. You don’t need power calculations. Really.
If you found a statistically non-significant difference (p greater than 0.05). There are two main possibilities.
A. There really is no difference in the population
B. You didn’t have enough power. (Congratulations! You have succeeded in making a Type II error
Since you are the insatiably curious type, we can now calculate The Power we had to detect the difference we said was significant.
We will use:
- N: the number of people you actually enrolled
- Sigma: the measured variance of the blood pressures in your study population (not estimated as before)
- The difference you decided before the study was clinically important (7 mm Hg in this case)
The power you calculate from this is the probability you had of detecting a clinically important difference if it is present in the larger population.