Microbiome data: Should unmatched reads be included in Shannon Diversity calculations?

Question

[Lina F]

Hi all,

I was wondering how everyone deals with unmatched reads when calculating Shannon Diversity.

Since they may be valid sequences and match an unknown 16S sequence, should they be included?

Or is it skewing the analysis because they most likely don't all come from the same organism?
In that case, is it safe to just discard them?

Thanks for any thoughts!
~Lina

[Brian Bushnell]

Normally, I would expect the researcher to cluster reads, and determine whether a sequence is valid based on cluster size and purity regardless of whether the sequence matches anything known.