How do you interpret Tukey's significance when there is no group clustering alone?

Question

[DNAngel]

I am confused by my results. I tested significance of means for 5 groups of my data and ANOVA confirmed there was a significant result - so there is a mean that is statistically significantly different from other means. Then I use Tukey HSD to determine which pair of groups is causing the significant difference. I've only ever worked with simple data where it was very obvious which means clustered, but the following results are confusing me:

From a tutorial online the results from a Tukey HSD test showed:

## Site   alpha groups
## Sil 72.99945      a
## Mah 62.03473      a
## Jam 32.62941      b

Clearly this means that group Jam clusters alone (it is labeled as b) so its mean is significantly different from the means of the other two groups.

Now here are my results:

Group  alpha groups
1 1.5712     a
2 0.9868     ab
3 0.7421     ab
4 0.6506     b
5 0.4565     b

My questions/confusions are:

1. So this means that group 1 clusters alone in a, but then it is not significantly different from any of the other groups because 'a' also appears for groups 2 and 3? Or perhaps it is significantly different because it is the only one clustering to 'a' by itself?

2. Or perhaps the interpretation is that when comparing group 1 and 4 or group 1 and 5, it is significantly different because their lettering is different. But when comparing groups 2 and 3 to any other group, those pair are not sig diff because letters are shared?

3. Since my ANOVA on this dataset detected significance, does this automatically mean there is at least one group's mean that has to be significantly different from all other groups?

4. Finally, I've run into a scenario where my ANOVA shows significance but after running Tukey HSD, all groups have the same significance letter - how is this possible?

Answer 1

[DNAngel]

I figured it out. In case any other noobs wonder about this, by looking at the Tukey HSD results using Tukey.HSD in R, it is clear which pairs are significant. Those that have different letters are considered significantly different against each other. In my example above, group 1 and 4 is significantly different, as well as group 1 and 5 because neither pair share a letter.

If ANOVA does show a significant difference amongst means in a dataset, this does not mean that another post hoc test like Tukey's will show the same result. In fact, many times multiple tests are needed. I am still not sure why this may happen or what it means but from this interpretation alone, I would still say yes ANOVA does say that one mean is different from all other means but Tukey HSD may not detect the same thing at all. Could be that the sensitivity of Tukey is not as strong.