AWS STAR Genome Index Error

Question

[SHXVRR]

Hello,

I have been trying to run this line of code for the longest time:

STAR --runThreadN 20 --runMode genomeGenerate --genomeDir genomeDir/ --genomeFastaFiles Homo_sapiens.GRCh38.dna.toplevel.fa --sjdbGTFfile Homo_sapiens.GRCh38.110.chr.gtf

I first tried running it on my home terminal but then realized that that it would take several days so I switched to AWS. I have created an instance for EC2 and then a S2 bucket and then connected it all through IAM. But whenever I try and run it I keep on getting:

Oct 05 03:02:26 ..... started STAR run   
Oct 05 03:02:26 ... starting to generate Genome files        
terminate called after throwing an instance of 'std::bad_alloc'      
  what():  std::bad_alloc
Aborted (core dumped)

I then switched the code to

STAR --runThreadN 20 --runMode genomeGenerate --genomeDir genomeDir/ --genomeFastaFiles Homo_sapiens.GRCh38.dna.toplevel.fa --sjdbGTFfile Homo_sapiens.GRCh38.110.chr.gtf --limitGenomeGenerateRAM 30G --genomeSAsparseD 2

But it still doesn't work. My storage is a max of 30G. Are the files too big and should I be working with smaller ones to create the STAR genome index or is there a better way of dealing with this issue of running a genome index.

Thanks

[ATpoint]

My storage is a max of 30G

Do you know the difference between memory and disk space? Please find out what your maximum allowed memory is.

[SHXVRR]

I didn't mean storage but the instance that I am running the Linux on has a maximum of 30G.

Answer 1

[GenoMax]

See STAR genome generate memory requirements

You are using the wrong genome file.

[SHXVRR]

when I replaced it, I still got the same error:

(base) [ec2-user@ip-172-31-28-142 ~]$ STAR --runThreadN 20 --runMode genomeGenerate --genomeDir genomeDir/ --genomeFastaFiles Homo_sapiens.GRCh38.dna.primary_assembly.fa --sjdbGTFfile Homo_sapiens.GRCh38.110.chr.gtf --limitGenomeGenerateRAM 30G --genomeSAsparseD 2 Oct 05 18:16:00 ..... started STAR run Oct 05 18:16:00 ... starting to generate Genome files terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc Aborted (core dumped) (base) [ec2-user@ip-172-31-28-142 ~]$

[GenoMax]

30 GB of RAM is not enough. Consider allocating at least 40G. If not you are not going to be able to do this with VM you have.

[SHXVRR]

I created a brand new instance and made it have 50G of Storage but still is giving me the same error