How often can Thomas Bayes check the results of his A/B test?
Stopping your A/B test once you reach significance is a great way to find bogus results…if you’re a frequentist. Checking before you have the statistical power to detect the phenomenon will often lead to false positives if you rely on classical/frequentist methods. A Bayesian with an informative null-result prior can avoid these problems. Let’s think about why. Continue reading ‘How often can Thomas Bayes check the results of his A/B test?’ »
) is small? I don’t know how many times I’ve been told by a well-meaning Bayesian some variation of