How to calculate power (or sample size) for a multiple comparison when Bonferroni-adjusted p-value?

[Dennis007]

Dear all,

We conduct a study for comparing the difference of a key indicator (continuous variable) between six groups.

When continuous variables follow a normal distribution, one should use ANOVA, and when they do not follow a normal distribution, the Kruskal-Wallis non-parametric test is employed. We have completed the analysis.

Nonetheless, how can we calculate power (or sample size) for the multiple comparison experiment (ANOVA or Kruskal-Wallis non-parametric test, respectively) when using the Bonferroni-adjusted p-value method?

Thanks!

[technocrat]

Without a reprex (see the FAQ), the best I can do is to relay a chat, which I haven't tested.

To calculate power or sample size for multiple comparison experiments (ANOVA or Kruskal-Wallis non-parametric test) when using the Bonferroni-adjusted p-value method, you can use the MultNonParam package in R for the Kruskal-Wallis test[1]. For ANOVA, you can use the pwr package in R.

For the Kruskal-Wallis test, you can use the kwpower function from the MultNonParam package[3]:

# Install the package if not already installed
if (!requireNamespace("MultNonParam", quietly = TRUE)) {
  install.packages("MultNonParam")
}

# Load the package
library(MultNonParam)

# Calculate power for Kruskal-Wallis test
kwpower(nreps, shifts, distname = c("normal", "logistic"), level = 0.05, ...)

For ANOVA, you can use the pwr.anova.test function from the pwr package:

# Install the package if not already installed
if (!requireNamespace("pwr", quietly = TRUE)) {
  install.packages("pwr")
}

# Load the package
library(pwr)

# Calculate power for ANOVA
pwr.anova.test(k, n, f, sig.level = 0.05, power = NULL)

For a more user-friendly tool, you can use GPower to calculate the required sample size for the Kruskal-Wallis test[2]. GPower is a free software that can be downloaded from the following website: http://www.gpower.hhu.de/

Please note that the Bonferroni-adjusted p-value method is used for multiple comparisons after the initial ANOVA or Kruskal-Wallis test. The power or sample size calculations mentioned above are for the initial tests, and the Bonferroni adjustment is applied afterward to control the family-wise error rate.

Citations:
[1] nonparametric - Power analysis for Kruskal-Wallis or Mann-Whitney U test using R? - Cross Validated
[2] Kruskal-Wallis-Test - calculate required sample size with G*Power - YouTube
[3] kwpower: Power for the Kruskal-Wallis test. in MultNonParam: Multivariate Nonparametric Methods
[4] https://www.datanovia.com/en/lessons/kruskal-wallis-test-in-r/
[5] https://med.und.edu/research/daccota/_files/pdfs/berdc_resource_pdfs/sample_size_r_module.pdf
[6] https://med.und.edu/research/daccota/_files/pdfs/berdc_resource_pdfs/sample_size_

[Dennis007]

Thank you for your consistent professionalism and insight, which have provided me with a lot of inspiration. Your provided code will also be of great assistance to us.

I have a question: we have completed multiple sets of comparative analyses (we conducted the multiple comparisons under Bonferroni-adjusted p-values). As we have divided the data into 6 groups , the reviewers are requesting us to calculate the power to ensure an adequate sample size.

In this situation, is it necessary to calculate the power under the Bonferroni-adjusted p-values for the post hoc multiple comparisons? If so, how can this be implemented?

We often use the PASS software to calculate sample sizes and effect sizes. However, this time, we only found a mention of the Bonferroni method in the "Multiple contrasts (simulation)" module of the software（see supplement PDF）.

It mentions the Dunn-Bonferroni test method, but we haven't seen any reference to Bonferroni-adjusted p-values, and it doesn't seem to fit our needs. We're wondering if this is indeed the case.

Alternatively, are there other ways to achieve this, perhaps using R packages or other software?

Thanks !
Multiple_Contrasts-Simulation.pdf (314.9 KB)

[technocrat]

This isn't something that I can help with in the abstract without the concreteness provided by a reprex.

[FrederickJax]

I'm working on a research project involving six groups, and we're comparing a continuous key indicator. While we've successfully conducted the analysis using either ANOVA or the Kruskal-Wallis test based on the normality of the distribution, we're now looking to calculate the power or determine the sample size for our multiple comparison experiment. We want to use the Bonferroni-adjusted p-value method to control for familywise error rate.

[Kingston3454]

I'm attempting to perform a complex hierarchical clustering analysis on a large dataset using RStudio. However, I keep encountering an error that I can't seem to resolve. The error message I'm getting reads: "Error in hclust(d, method = 'complete') : 'd' must be a dissimilarity structure." I've checked my data and tried various approaches to resolve this issue, such as ensuring my data is in the correct format and checking the clustering method.
serpentine font

[nirgrahamuk]

I'm not confident this thread is the place to continue a discussion of your hclust issues, as before your post, hclust wasnt mentioned...
regardless, as the error said, d must be a dissimilarity structure, and those are generated from calls to the stats::dist() function. Does your code call dist() at any point ?