error with CombineGVCFs to join large number of files

Question

[rj.rezwan]

Hi, I have 64 different accessions files having *g.vcf output files after haplotyepcalling in GATK. Now I want to combine them in one file, and I am using CombineGVCFs but unable to get the output and file showing error. Can someone suggest how to combine them in the best smart way or is there any issue with combineGCVF for combining many samples files? the code is here

#!/bin/bash
#
#SBATCH --job-name=combine_files
#SBATCH --output=combine_files.%j.out
#SBATCH --partition=batch
#SBATCH --cpus-per-task=20
#SBATCH --time=100:00:00
#SBATCH --mem=600G

module load gatk/4.1.2.0

ref_dir=(~/path/PitayaGenomic.fa)

gvcf_dir=(~/path/*.g.vcf)

gatk CombineGVCFs -R ${ref_dir} $(printf -- '--variant %s ' "${gvcf_dir[@]}") -O joint_files.g.vcf

[raphael.B]

Without error message, it is hard to say where the problem is. You could try increasing memory (--java-options "-Xmx50g" for example). Just as a remark, GATK recommends using GenomicDBImport instead of CombineGVCFs especially when dealing with large numbers of files (more than 1000). But I don't think it is your case here.

[Pierre Lindenbaum]

You have to comment and /or to validate the answers for all your other questions: is it mandatory to use fixmate? ; does warning has an impact on the markduplication output file? ; why sorted.bam file size is increased after using fixmate in picard ; Apple gene ID conversion tool ; Submission of RNA-seq data in NCBI Sequence Read Archive (SRA) ; Gene ontology and KEGG pathway analysis of rice RNA-seq data ;

[rj.rezwan]

sure, I will make a comment ASAP within a few days with some suitable answers which would be useful for new users

[Pierre Lindenbaum]

Not just a comment. Validate (green mark on the left) the good answers for your previous questions.

Answer 1

[Pierre Lindenbaum]

instead of

gvcf_dir=(~/path/*.g.vcf)
gatk CombineGVCFs -R ${ref_dir} $(printf -- '--variant %s ' "${gvcf_dir[@]}") -O joint_files.g.vcf

do

find ~/path/ -type f -name "*.g.vcf" > gvcfs.list
gatk CombineGVCFs -R "${ref_dir}"  --variant gvcfs.list -O joint_files.g.vcf.gz