I am currently preparing a proposal for submission and one piece of information that the agency suggests is the power required to distinguish effects. This is obviously a perfectly reasonable piece of information to request; however, power calculations fall into that class of things that I know that I should know but I don't. It is one of those topics that every statistics book will tell you is important, but either a) glosses over the topic, or b) provides such a deep background that it is impossible to follow what the authors are talking about. Additionally, power calculations are complicated enormously by the fact that sample designs can become very complicated.

In contrast to this traditional treatment, Andrew Gelman and Jennifer Hill's book, Data Analysis Using Regression and Multilevel/Hierarchical Models, provides a very clear description of simple power analyses, which -- thankfully -- is all that I really need for this project. To make sure that I don't forget, I record below how to find the required sample size, n, for varying levels of between-group effect differences, Δ, at 80% power. The formula is relatively easy (see pp. 437-447 for more info): (5.6σ/Δ)². Therefore, if I measure change in units of standard deviations, sd, then I can estimate the sample size n for each unit of change.

drop _all
range sd 0 1 41
gen n = (5.6/sd)^2

I can then make a graph of the expected sample size required for a standard unit change using the command twoway line n sd; or, alternatively, just print a table of numbers using list.