How To Interpret A T-Test Output Produced By R

Question

[Panos]

Hi!

I have this R output from a Welch t-test (to compare means of two samples)

data:  sampleA and sampleB
t = 11.8184, df = 49705.89, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 3.162714 4.420318
sample estimates:
mean of x mean of y
 6.128647  2.337131

Looking at the output, can someone tell me if the means of my samples are significantly different? I'm confused with the "alternative hypothesis" line...

[Michael]

Ok, in consequence, sometimes it's good to battle it out. Closing a question as of off-topic is always very subjective, and please don't take it personally. I am not always sure if it is correct, and I'd prefer if closing a question would be a voting process as well. Therefore, it is good to have strong advocacy from the community as feedback.

[Michael]

sorry, off topic, this is very basic statistics not bioinformatics

[Istvan Albert]

I think "off topic" should mean topics that cannot possibly have relevance to the field. While it is true that strictly speaking this is a question about statistics and a the way R displays a quantity we should allow more leeway in applying the rules - especially if the question garners a number of good answers - if anything that is proof that the question is of interest to many here.

[Michael]

Sure, I wouldn't want to be overly strict either, and one or more borderline questions don't hurt us, BUT: There are places where the op might get a more correct answer. In fact none of the answers touched upon the reason for the confusion about the alternative hypotheses yet. I believe that a question that can be formulated as a pure statistics question (even when it has -possibly- a biological background behind it) is better asked on a statistics site: e.g. http://stats.stackexchange.com/ , we have seen the same with pure programming questions and they have been pretty much handled that way.

[Istvan Albert]

I think that if the first or second answer to this were what you state above then I think it would be fine to also close the question and move on. But once there are a good number of answers and comments I think we should let the question take its course and allow people to edit/change their answers and overall sort it out. Closing the question at that point freezes everything in place and does not allow any kind of improvements to take place. In the end I think there are lot of interesting discussions below. To be honest with you I think p-values are a very subtle concept and perhaps statisticians are not always the right people to explain it.

[Arun]

I have to disagree with you on the last point on p-values. Most biologists, if not all, tend to allow for instincts in deciding which candidate gene is better than the other according to their p-value, especially in multiple testing cases (GWAS, candidate gene approaches etc..). At the end of the day, its a number. It doesn't have a biological meaning. But people tend to give it too much importance, 10^-5 or 10^-25? 10^-25 is a "strong" candidate. Numerically, yes. Biologically, both are. And I think understanding this point and accepting that intuition can lead to a conclusion otherwise rarely happens if you are NOT a statistician.

[Istvan Albert]

What I meant is that the proper statistical terminology is not easy to understand. Biologists misusing p-value is also a major pet-peeve of mine and don't really have a disagreement there. I like to make the joke that the chance that all authors on a paper being habitual liars is bigger than E^-25 therefore no matter what context you see that reported is beyond the expected accuracy anyhow ;-). The problem is further compounded by not accounting for multiple comparisons when trying to come up with something "interesting".

[Panos]

Thanks for the link for the stats site Michael. Didn't even know it existed!

Since I'm working in bioinformatics, biostars.org was the only place it came to mind for asking such a question.

I'll have in mind the other site, however, for my next question on stats!

[Arun]

Not a rebuttal, but I wonder how one would construe a question on "error using samtools", for example. It is, strictly speaking, a software bug or misuse, even though it is a "bioinformatics" software. I am just saying its very hard to draw the line. I agree with Istvan's definition of "off-topic". Maybe this must be cited somewhere in the website where people can take easy notice?

[seidel]

Yes I never use basic statistics in any of my bioinformatics work, no one I've met has ever been confused by it, and I agree that people doing bioinformatics shouldn't discuss such topics. ;)

[Michael]

You can spare your irony. But in fact I want to give an example, you and most people in bioinformatics use computers, though it's most likely not the right place to discuss the choice of your computer (or telephone or car or whatever) brand here, unless you can make a connection to bioinformatics very clear. IMO it is the responsibility of the poster to prove the connection to bioinformatics if it is not obvious initially.

Also, questions that ask only for explanations or definitions of basic concepts and terms that could be easily read up elsewhere or textbook, googled should be discouraged (imho), especially when they belong to a different discipline.

[seidel]

When questions like this get closed, I fear sparing the irony would jeopardize my sanity. I tend to agree with Istvan that question closing is for varieties that "...cannot possibly have relevance to the field." Historically, off-topic is touchy because it is SO subjective. In the context of Istvan's comment, I simply prefer a hierarchy as follows. I agree with you that it is the op's job to connect their question as clearly as possible to bioinformatics. Failing that, people who answer can choose to provide some connection in their answers. Most of the controversial closed questions have a connection to bioinformatics that simply hasn't been explicitly stated, yet for many of us is as plain as day. Faced with a choice to close a question or provide a connection, I wish those who assumed moderation power would take the benevolent shepherd approach and choose to reconstruct the limb rather than amputate and cauterize the wound, as it were. As for asking only in the best place? Agreement on "best" will be tough.

Answer 1

[bdemarest]

Assuming sampleA and sampleB are normally distributed independent variables with equal variances: You should reject the null hypothesis (p < 0.00000000000000022).

Another way of explaining: The difference between mean(sampleA) and mean(sampleB) is 6.128647 - 2.337131 = 3.791516. A difference this large (or larger) would only happen by chance 1 time in 2.2 * 10^16 experiments. Thus, you can conclude that the difference in means is not chance, but is due to a real biological difference.

Note that the true p-value could be smaller than 2.2e-16. Numbers smaller than that cannot be reliably distinguished from one another (limit of floating-point precision).

[Joachim]

Nice explanation! Much more applicable than my answer. However, I like to add that the emphasis on rejecting the null hypothesis by p-value can be misleading. There are some nice points about common p-value interpretation issues under "Misunderstandings" in the p-value page of Wikipedia. The p-value in the t test is only to support the argument of rejecting the null hypothesis, where the latter is purely determined by the difference of the means.

[bdemarest]

Thanks for the Wikipedia link. Though I'm not sure that I completely follow your point, I must admit that I very much simplified the issue. My answer is not mathematically rigorous at all, but I hope it is not misleading.

[Arun]

(+1) To be precise, I'd delete the line "but is due to real biological difference"; you just don't know that, even though it is very likely. A statistical test allows acceptance or rejection of NULL hypothesis. For one to understand what it means, he/she has to have designed the test by hypothesising NULL and alternatives first. As such, the test just tells the means are significantly different. The statistical test stops there. The inference of the test begins by interpreting what your NULL was.

[Joachim]

I think your answer is not misleading as such. It was just my intention to raise awareness that the p-value is just one of the many statistical outputs you get. Sorry for the confusion.

[David W]

Minor point: This is a Welch t-test so the variances don't have to be equal.

also, (+1) for Joachin - obsessive noting of p-values is usually bad thing. Was there really any chance that groups and A and B would be exactly equal? If null was biologically unlikely then we shouldn't be surprised that a tonne of data has given is a low p-value. In this case I'd instead look at the confidence interval. The true difference between the mean of the A and B groups is likely to fall between 3.16 and 4.42.

[Rm]

nice explanation!!!

[Panos]

I couldn't get a clearer answer bdemarest! Many thanks!

Answer 2

[Joachim]

The "alternative hypothesis" is the counterpart of the "null hypothesis". http://en.wikipedia.org/wiki/Null_hypothesis

In a nutshell:

Your "null hypothesis" is saying "these two samples have the same means." Your "alternative hypothesis" is the opposite, which says "these two samples actually have different means."

You can check Welch's t test on Wikipedia for more details, but it essentially states that the "null hypothesis" is testing whether the two means are equal. Now, Welch's t test is a specialized form of the Student's t test, which has some good examples and goes into a bit more detail about the means.

With these t tests, if your means would be equal, then your "null hypothesis" is true and your data appears not to be interesting. Well, you have to take that with a pinch of salt, because you are only focusing on the means there and you are assuming some other constraints that your data adheres to. However, if the means are not equal, then your "alternative hypothesis" is true and your data is not all of the same kind. That can be taken as an indicator that you are measuring something of statistical significance, which is expressed by the p-value above. The smaller the p-value is, the more interesting your result is.

Hope that helps.

[Arun]

@Joachim: "However, if the means are not equal, then your "alternative hypothesis" is true and your data is not all of the same kind."

Just a small point on this. Accepting alternative hypothesis and rejecting Null hypothesis aren't exactly the same, even though its tempting to intuitively derive there. The statistical test just rejects the null hypothesis.

Answer 3

[Woa]

You need to specify another parameter, the significance level or alpha which is typically set to 0.01 or 0.05. If the p-value is less than alpha then you can reject the null hypothesis. Depending upon the case there can be issues about Multiple hypothesis testing corrected or adjusted p-values. Check for yourself.

[Chris Evelo]

Although very commonly done, you don't "need to" specify alpha. That is for a large part a left over from the time we used paper tables to derive significance levels. You can just report the actual p-value found and leave it to the reader to decide whether p<0.049 is really that much more significant than p<0.051. Unfortunately many reviewers will agree with you though.

[Arun]

(+1) very true.

Answer 4

[Steve Lianoglou]

The "alternative hypothesis line" is just an another way state the null, no? Where the null hypothesis is something like: "the difference between the means of your two samples is 0".

I'll leave the "drawing of conclusions" on what this means for your data up to you.