Statistical Power

• From the statistical power wiki "The power of a statistical test is the probability that the test will reject a false null hypothesis".
• There are four things that influence the conclusions you can draw from a statistical test
  1. sample size
  2. effect size (this is merely how strong the relation between the variables compared to the noise)
  3. significance
  4. power, the odds you will observe an effect
• Given values of any three, you can calculate the fourth.
• Power analysis is often conducted before a study to determine sample size.
• Calculate the effect size. You can do this by calculating Cohen's d where d = (mean1-mean2)/(sqrt(SD1^2 + SD2^2)/2)
• I calculated this on some of the ttest data and got an effect of .07558. Apparently, 0.2 is idnicative of a small effect 0.5 a medium effect, and 0.8 a large effect.
• I plugged test value 7, sample average 6.7, sample size, standard dev (3.6) and alpha error (20% it didn't take 60%) into the DSS research caculator and got a power of 21.9%.
• Apparently 80% is acceptable.
• So as expected, this reinforces the null hypothesis, i.e. that the two groups are not different. (We know they're not.)