Paul Pharoah raises two important points. First, it is well-known that null hypothesis significance testing (NHST) doesn’t really tell us what we want to know. As Jacob Cohen put it in his delightfully titled article, “The earth is round (p < .05)" (Am Psychol, 1994, 49(12), 997-1003), what we want to know is the probability of the data given that the null hypothesis is true. What we actually get, though, is the probability of the null given the data. Problems with NHST have been highlighted since 1938 (Berkson J. Some difficulties of interpretation encountered in the application of the chi-square test. J Am Stat Assoc. 1938;33:526-542), but the reality is that, unless you are a Bayesian, we've been using it for going on a century, and it has served our purposes.
His other point is that a microscope doesn't help if the phenomenon isn't there. There's no way to argue with this. What this means is that we have to determine the magnification (or sample size) a priori, based on our best guess of the phenomenon's size. Then, if we don't see it, it likely ain't there.

]]>