Shiny Contest Submission: A/B Testing Sample Size Calculator

In the world of website optimization, A/B testing is used to determine which user experiences are most effective. This requires calculating how long a test will need to run based on the traffic volume to the page/site being tested, the desired/expected observed difference, and the tolerance for Type I and Type II errors.

This calculator enables the test planner to input different values as part of the planning process and then visualize the impact on the test duration and the likelihood of a Type I or Type II error as a means of providing business users who do not have a strong statistical background a degree of intuition regarding the different levers they can pull in their test design.

Key technical details:

  • Ultimately, this is just a bunch of interactivity and visualizations built around a simple use of power.prop.test()
  • It uses flexdashboard, so, technically, is combining both RMarkdown and Shiny
  • Obviously, there is heavy use of ggplot2


A screen cap of the app:


This topic was automatically closed 54 days after the last reply. New replies are no longer allowed.

If you have a query related to it or one of the replies, start a new topic and refer back with a link.