Archive for the ‘Bayes’ Category.

## How often can Thomas Bayes check the results of his A/B test?

Stopping your A/B test once you reach significance is a great way to find bogus results…if you’re a frequentist.  Checking before you have the statistical power to detect the phenomenon will often lead to false positives if you rely on classical/frequentist methods.  A Bayesian with an informative null-result prior can avoid these problems.  Let’s think about why. Continue reading ‘How often can Thomas Bayes check the results of his A/B test?’ »

What is a methods-careful practitioner to do when the number of observations ($n$) is small?  I don’t know how many times I’ve been told by a well-meaning Bayesian some variation of
Bayesian estimation addresses the “small $n$ problem”