Online FDR Explore
Authors: Lathan Liou, David S Robertson
Abstract: This is a Shiny App interface to the onlineFDR package hosted on Bioconductor. It is designed to assist researchers in controlling the False Discovery Rate (FDR) in a novel "online" framework as well as introduce to paradigm in an educational way.
Full Description: Overview
The app provides a family of algorithms that users can use, a tool to help decide which one to use, and several features including plotting the adjusted significance thresholds against a Bonferroni correction as well as the compare tool to compare two algorithms against each other. Users are also able to download the results. An interactive guide is included to walk users through how to navigate the app. A default dataset is provided with the app, but users can feel free to upload their correctly formatted data.
Background
Multiple hypothesis testing is a fundamental problem in statistical inference, and the failure to manage multiple testing problems has been highlighted as one of the elements contributing to the replicability crisis in science (Ioannidis 2015). Methodologies have been developed to manage the multiple testing situation by adjusting the significance levels for a family of hypotheses, in order to control error metrics such as the familywise error rate (FWER) or the false discovery rate (FDR).
Frequently, modern data analysis problems have a further complexity in that the hypotheses arrive sequentially in a stream.
This introduces the challenge that at each step, the investigator must decide whether to reject the current null hypothesis without having access to the future p-values or the total number of hypotheses to be tested, but with the knowledge of the historic decisions to date.
The onlineFDR package provides a family of algorithms you can apply to a historic or growing dataset to control the FDR or FWER in an online manner. At a high-level, these algorithms rely on a concept called “alpha wealth” in which experiments cost some amount of error from your “budget” but a discovery earns some of the budget back.
For more information, please see our vignette!
Keywords: education, statistics, error rate, research
Shiny app: http://shiny.mrc-bsu.cam.ac.uk/apps/onlineFDRexplore
Repo: GitHub - latlio/onlineFDRexplore
RStudio Cloud:
Thumbnail:
Full image: