Ubuntu exits (Cellranger mkref, custom reference)

Question

[kyle.mcandrews]

Hello,

I am trying to generate a custom mouse reference genome that includes 3 transgenes (eGFP, tdTomato, and Cre), I have added these to the reference genome and the gtf as well as filtered the gtf with no issues.

When I run mkref (usung Ubuntu within windows, version 2204.2.33.0), it ends up exiting Ubuntu abruptly with no warning, with the following as the contents of the log.out file:

STAR version=                 2.7.2a
STAR compilation time,server,dir=__REDACTED__
##### Command Line:
/home/kmcandre/cellranger-7.1.0/lib/bin/STAR --runMode genomeGenerate --genomeDir /home/kmcandre/genomes/Mus/star --runThreadN 16 --genomeFastaFiles /home/kmcandre/genomes/Mus/fasta/genome.fa --sjdbGTFfile /home/kmcandre/genomes/Mus/genes/genes.gtf --limitGenomeGenerateRAM 178241142784 --genomeSAsparseD 1 --genomeSAindexNbases 14 --genomeChrBinNbits 18
##### Initial USER parameters from Command Line:
###### All USER parameters from Command Line:
runMode                       genomeGenerate     ~RE-DEFINED
genomeDir                     /home/kmcandre/genomes/Mus/star     ~RE-DEFINED
runThreadN                    16     ~RE-DEFINED
genomeFastaFiles              /home/kmcandre/genomes/Mus/fasta/genome.fa        ~RE-DEFINED
sjdbGTFfile                   /home/kmcandre/genomes/Mus/genes/genes.gtf     ~RE-DEFINED
limitGenomeGenerateRAM        178241142784     ~RE-DEFINED
genomeSAsparseD               1     ~RE-DEFINED
genomeSAindexNbases           14     ~RE-DEFINED
genomeChrBinNbits             18     ~RE-DEFINED
##### Finished reading parameters from all sources

##### Final user re-defined parameters-----------------:
runMode                           genomeGenerate
runThreadN                        16
genomeDir                         /home/kmcandre/genomes/Mus/star
genomeFastaFiles                  /home/kmcandre/genomes/Mus/fasta/genome.fa   
genomeSAindexNbases               14
genomeChrBinNbits                 18
genomeSAsparseD                   1
limitGenomeGenerateRAM            178241142784
sjdbGTFfile                       /home/kmcandre/genomes/Mus/genes/genes.gtf

-------------------------------
##### Final effective command line:
/home/kmcandre/cellranger-7.1.0/lib/bin/STAR   --runMode genomeGenerate   --runThreadN 16   --genomeDir /home/kmcandre/genomes/Mus/star   --genomeFastaFiles /home/kmcandre/genomes/Mus/fasta/genome.fa      --genomeSAindexNbases 14   --genomeChrBinNbits 18   --genomeSAsparseD 1   --limitGenomeGenerateRAM 178241142784   --sjdbGTFfile /home/kmcandre/genomes/Mus/genes/genes.gtf
----------------------------------------

Finished loading and checking parameters
Jun 28 12:18:34 ... starting to generate Genome files
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 0  "1a" chrStart: 0
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 1  "1b" chrStart: 262144
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 2  "1c" chrStart: 524288
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 3  "1" chrStart: 786432
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 4  "10" chrStart: 196345856
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 5  "11" chrStart: 327155712
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 6  "12" chrStart: 449314816
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 7  "13" chrStart: 569638912
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 8  "14" chrStart: 690225152
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 9  "15" chrStart: 815267840
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 10  "16" chrStart: 919339008
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 11  "17" chrStart: 1017643008
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 12  "18" chrStart: 1112801280
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 13  "19" chrStart: 1203765248
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 14  "2" chrStart: 1265369088
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 15  "3" chrStart: 1447559168
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 16  "4" chrStart: 1607729152
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 17  "5" chrStart: 1764491264
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 18  "6" chrStart: 1916534784
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 19  "7" chrStart: 2066481152
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 20  "8" chrStart: 2211971072
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 21  "9" chrStart: 2341470208
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 22  "MT" chrStart: 2466250752
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 23  "X" chrStart: 2466512896
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 24  "Y" chrStart: 2637692928
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 25  "JH584299.1" chrStart: 2729443328
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 26  "GL456233.1" chrStart: 2730491904
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 27  "JH584301.1" chrStart: 2731016192
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 28  "GL456211.1" chrStart: 2731278336
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 29  "GL456350.1" chrStart: 2731540480
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 30  "JH584293.1" chrStart: 2731802624
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 31  "GL456221.1" chrStart: 2732064768
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 32  "JH584297.1" chrStart: 2732326912
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 33  "JH584296.1" chrStart: 2732589056
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 34  "GL456354.1" chrStart: 2732851200
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 35  "JH584294.1" chrStart: 2733113344
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 36  "JH584298.1" chrStart: 2733375488
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 37  "JH584300.1" chrStart: 2733637632
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 38  "GL456219.1" chrStart: 2733899776
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 39  "GL456210.1" chrStart: 2734161920
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 40  "JH584303.1" chrStart: 2734424064
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 41  "JH584302.1" chrStart: 2734686208
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 42  "GL456212.1" chrStart: 2734948352
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 43  "JH584304.1" chrStart: 2735210496
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 44  "GL456379.1" chrStart: 2735472640
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 45  "GL456216.1" chrStart: 2735734784
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 46  "GL456393.1" chrStart: 2735996928
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 47  "GL456366.1" chrStart: 2736259072
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 48  "GL456367.1" chrStart: 2736521216
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 49  "GL456239.1" chrStart: 2736783360
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 50  "GL456213.1" chrStart: 2737045504
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 51  "GL456383.1" chrStart: 2737307648
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 52  "GL456385.1" chrStart: 2737569792
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 53  "GL456360.1" chrStart: 2737831936
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 54  "GL456378.1" chrStart: 2738094080
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 55  "GL456389.1" chrStart: 2738356224
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 56  "GL456372.1" chrStart: 2738618368
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 57  "GL456370.1" chrStart: 2738880512
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 58  "GL456381.1" chrStart: 2739142656
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 59  "GL456387.1" chrStart: 2739404800
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 60  "GL456390.1" chrStart: 2739666944
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 61  "GL456394.1" chrStart: 2739929088
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 62  "GL456392.1" chrStart: 2740191232
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 63  "GL456382.1" chrStart: 2740453376
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 64  "GL456359.1" chrStart: 2740715520
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 65  "GL456396.1" chrStart: 2740977664
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 66  "GL456368.1" chrStart: 2741239808
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 67  "JH584292.1" chrStart: 2741501952
/home/kmcandre/genomes/Mus/fasta/genome.fa : chr # 68  "JH584295.1" chrStart: 2741764096
Estimated genome size=2943026240   2742026240   201000000
GstrandBit=32
Number of SA indices: 5305573360
Jun 28 12:19:11 ... starting to sort Suffix Array. This may take a long time...
Number of chunks: 16;   chunks size limit: 2829639120 bytes
Jun 28 12:19:17 ... sorting Suffix Array chunks and saving them to disk...
Writing 1108621928 bytes into /home/kmcandre/genomes/Mus/star/SA_15 ; empty space on disk = 1012894220288 bytes ... done

My system has 128GB of RAM and I set the --memgb attribute above that (I have also set it at 120, 128, etc) and have the same issue every time.

The output folder "fasta" contains "genome.fa" and "genome.fa.fai"; folder "genes" contains "genes.gtf", and folder "star" contains "chrLength.txt", "chrName.txt", "chrNameLength.txt", and "chrStart.txt", as well as "SA_15" which is >1GB.

Any advice would be greatly appreciated, I tried to install Ubuntu desktop on a separate partition of one of my drives but was having issues booting from USB.

Thank you for your time!!!!

Kyle

[Ram]

Just to clarify, you're running Ubuntu ON Windows and giving the process more memory than you physically have?

[kyle.mcandrews]

For my most recent attempt, yes I did set the RAM over what I physically have, previously I had tried default settings (16gb) as well as 120, 128. (I have 128GB), and yes it is Ubuntu for windows.

[Ram]

Set it to something like 48 or 64 GB - you're already consuming a bunch of RAM running Ubuntu on Windows.

Answer 1

[kyle.mcandrews]

Ok, so I believe I have solved the problem, I am new to Linux/Ubuntu and did not realize the drive where all of this was being written to was low on storage, I mounted a partition with 500GB free space and so far it seems to be running fine (well past where it had become hung up before)

Thank you for your reply I appreciate your willingness to help this newB.

Kyle

[Ram]

Glad you figured it out! And thank you for posting the solution and bringing closure to your post. I'll move your post to an answer. Please accept it to let the system know your issue was resolved.