If you are a Bayesian and not a Frequentist, the main way you can apply Bayesian Statistics to an AB test result is through using an online tool or a programming language such as Python or R... well until this blog post. If you do not have any programming skills, here I share a Google Sheets that leverage Bayesian Statistics to calculate the probability of your variant being better than the control as well as the relative uplift associated. All you need to do is plug the result from your AB test into the tool, easy peasy. Let’s have a look.
I am not going to take you through a step by step guide on how to build the calculator, but instead introduce the one I built and use. Feel free to make a copy and retro-engineer. The Bayesian AB-test calculator in Google-Sheets is made of three parts:
I wanted to close this blog post by sharing a video from Cassie Kozyrkov where she explains very skillfully the differences between Frequentist and Bayesian statistics: very useful if you are still trying to wrap your head around these concepts.