I have a question about the Tukey correction in emmeans. See the example below. It says "P value adjustment: tukey method for comparing a family of 3 estimates." Does this mean that the error rate for all of these comparisons (which were run together) is 5%? What does "a family of 3 estimates" refer to?
The two factors here were "group," which had 3 levels, and "condition," which had 4 levels, so there are a total of 12 comparisons. If each comparison had been run separately, the error rate would have been 5% comparison. I'm hoping that because these were all run together with the Tukey correction, the error rate is down to 5% for all of them. What I'm confused about is the reference to "3 estimates," and I'm wondering if this means the error rate is 5% for each set of three group comparisons (one set per condition), meaning the error rate is actually 1 - 0.95^4 (about 18%)?
Thank you!
> emmeans(model_1, pairwise ~ group | condition)
$emmeans
condition = EN-TVJT-R-BRF:
group emmean SE df lower.CL upper.CL
EN 0.0456 0.0174 219 0.01129 0.0799
HS 0.1515 0.0211 368 0.11004 0.1930
SB 0.0208 0.0246 500 -0.02745 0.0691
condition = EN-TVJT-R-BRT:
group emmean SE df lower.CL upper.CL
EN 0.9848 0.0174 219 0.95043 1.0191
HS 0.8712 0.0211 368 0.82974 0.9127
SB 0.9479 0.0246 500 0.89964 0.9962
condition = EN-TVJT-R-THF:
group emmean SE df lower.CL upper.CL
EN 0.0254 0.0174 219 -0.00891 0.0597
HS 0.1742 0.0211 368 0.13277 0.2157
SB 0.0417 0.0246 500 -0.00661 0.0899
condition = EN-TVJT-R-THT:
group emmean SE df lower.CL upper.CL
EN 0.9898 0.0174 219 0.95548 1.0241
HS 0.8939 0.0211 368 0.85247 0.9354
SB 0.9688 0.0246 500 0.92047 1.0170
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95
$contrasts
condition = EN-TVJT-R-BRF:
contrast estimate SE df t.ratio p.value
EN - HS -0.1059 0.0266 783 -3.983 0.0002
EN - SB 0.0248 0.0294 783 0.842 0.6768
HS - SB 0.1307 0.0317 783 4.117 0.0001
condition = EN-TVJT-R-BRT:
contrast estimate SE df t.ratio p.value
EN - HS 0.1136 0.0266 783 4.271 0.0001
EN - SB 0.0368 0.0294 783 1.252 0.4230
HS - SB -0.0767 0.0317 783 -2.417 0.0420
condition = EN-TVJT-R-THF:
contrast estimate SE df t.ratio p.value
EN - HS -0.1488 0.0266 783 -5.597 <.0001
EN - SB -0.0162 0.0294 783 -0.552 0.8454
HS - SB 0.1326 0.0317 783 4.177 0.0001
condition = EN-TVJT-R-THT:
contrast estimate SE df t.ratio p.value
EN - HS 0.0959 0.0266 783 3.606 0.0010
EN - SB 0.0211 0.0294 783 0.716 0.7543
HS - SB -0.0748 0.0317 783 -2.357 0.0489
Degrees-of-freedom method: kenward-roger
P value adjustment: tukey method for comparing a family of 3 estimates