Archive for the ‘Bayes’ Category.

How often can Thomas Bayes check the results of his A/B test?

Stopping your A/B test once you reach significance is a great way to find bogus results…if you’re a frequentist.  Checking before you have the statistical power to detect the phenomenon will often lead to false positives if you rely on classical/frequentist methods.  A Bayesian with an informative null-result prior can avoid these problems.  Let’s think about why. Continue reading ‘How often can Thomas Bayes check the results of his A/B test?’ »

Regain your confidence (intervals)

Next time you see someone “misinterpret” a confidence interval, wait a second.  They’re actually probably okay. Continue reading ‘Regain your confidence (intervals)’ »

Bayes fixes small n, doesn’t it?

What is a methods-careful practitioner to do when the number of observations (n) is small?  I don’t know how many times I’ve been told by a well-meaning Bayesian some variation of

Bayesian estimation addresses the “small n problem”

This is right and wrong. Continue reading ‘Bayes fixes small n, doesn’t it?’ »