bcftools merging 3540 individual files

Question

[realtreeecat]

Hi all,

I have 3540 sequences individual vcf files. I first created a list for all my vcf files using this command:

ls  input_file*.vcf.gz > list_of_file.txt

Then I created index for them and merged with this:

bcftools merge -l merge.txt -Oz -o merge.vcf.gz

Indexing command created .tbi fles for each vcf files however it shows this error :

[E::hts_idx_load3] Could not load local index file 'EPI_ISL_13330943.vcf.gz.tbi'
Failed to open EPI_ISL_13330943.vcf.gz: could not load index

If I index this file individually, then it shows the same error, however it says failed to load .csi file this time. I check my ulimit and it's 7823 while the number of items in my folder is just 7079.

How do I fix this?

Thank you

[GenoMax]

Are your VCF files sorted and bgzipped before indexing? .csi indexes can be made by using bcftools index -c vcf.gz.

Looks like you could also use --write-index with your bcftools merge command to create the index.

[realtreeecat]

GenoMax Hello, Yes! My files are .gz zipped and I also sorted them and these are my steps again 1) Created a list of vcf text file

ls EPI_ISL_*.vcf.gz > list_of_files.txt

2) Sort it

cat list_of_files.txt | while read file do bcftools sort $file -Oz -o "${file/input_file/sorted_input_file}" bcftools index -t "${file/input_file/sorted_input_file}" done

(The sort command automatically creates .tbi index for all files)

3) Next i created .csi index but with this below script .the bcftool -c indexing was only indexing a single file only and not the rest of my vcf files so i used this script to loop and index all of them

cd "$bcfcopy" || exit

for vcf_file in *.vcf.gz; do

if [ -f "$vcf_file" ]; then

    bcftools index "$vcf_file"

    echo "Indexed $vcf_file"
else
    echo "File not found: $vcf_file"
fi

done

4) After this i tried merging

cftools merge -l list_of_files.txt -Oz -o merge.vcf.gz

5) And i got the same error :

[E::hts_idx_load3] Could not load local index file 'EPI_ISL_13330943.vcf.gz.csi' Failed to open EPI_ISL_13330943.vcf.gz: could not load index

PLSSS what am i doing wrong? Thank you