samtools write-index

Question

[R.L.]

Hi Community! I am writing a pipeline for automations, and I am suffering an issue like this:

[W::hts_idx_load3] The index file is older than the data file: alignment.sorted.bam.bai

And I employed minimap2 like this:

minimap2 -ax map-ont --secondary=no --MD -t 18 /data/renqgli/project/refseq/hg38.p14.fa $fastqgz_all\
|samtools sort -O BAM --write-index -o alignment.sorted.bam##idx##alignment.sorted.bam.bai -

I actually have no idea on what's happening, the reason I use this is because I do not need intermediate files, and save space for my disk.

Thank you!

[Pierre Lindenbaum]

can you please retry, but add

rm -vf alignment.sorted.bam.bai

before invoking minimap2. And -o alignment.sorted.bam should be enough.

[R.L.]

Really appreciate your help Pierre! I am a little bit confused btw. do you mean this:

rm -vf alignment.sorted.bam.bai
minimap2 -ax map-ont --secondary=no --MD -t 18 /data/renqgli/project/refseq/hg38.p14.fa $fastqgz_all\
|samtools sort -O BAM --write-index -o alignment.sorted.bam##idx##alignment.sorted.bam.bai -

Thanks! Lucis

[R.L.]

-o alignment.sorted.bam##idx##alignment.sorted.bam.bai

This portion is to get .bai index, instead of .csi indexing. Although Idk the difference between these two lol, but I prefer .bai the familiar one lol

[GenoMax]

The index file is older than the data file: alignment.sorted.bam.bai

I don't think that warning is going to cause any issues. Not sure why that happens but I have seen it as well. Likely due to using multiple threads for commands.

[R.L.]

Yeah, I did multiple split tests of my pipeline (just subset 400k rows of my fastq and run through my pipeline for quick tests) and none of them raised this issue. However, when I input the human WGS into it for read world challenge, the warning raised...

Answer 1

[aw7]

Someone, presumably you, opened an issue for the same problem. The answer was:

This problem should be fixed in htslib 1.20 (included with the samtools 1.20 release). Before that, the index and output files could be closed in the wrong order leading to the misleading warning that you got.

To fix, you can either:

Upgrade to version 1.20 (best, you fix other bugs too)

Index the output file separately using samtools index

Use touch on the index file so the timestamp is later than the one on the sort output

Or just ignore the warning as it should be harmless

[R.L.]

Oh God, thank you so much for helping me in both platforms! I am gonna upgrade my samtools to stay tracked with you and your team! Thank you again!