Segmentation Fault error when running salmon quant

Question

[Daniel]

When running salmon quant on multiple reads, I get the error:

"/var/lib/slurm-llnl/slurmd/job00431/slurm_script: line 12: 213183 Segmentation fault (core dumped) salmon quant -i /home/user/fol2/reference_transcriptome/transcripts_index -l A -r /home/user/fol2/raw_fastq/D_{1..3}/.gz --validateMappings -o transcripts_quant"

Everything seemed to run normally before. I read online that a segmentation fault means that the program is trying to access memory it doesn't have access to. Currently, my server only has 1 node available (that I am sharing with someone else who has an active program running).

I am curious if the issue then is that the server doesn't have enough memory available. I am a bit confused because if that was the error, wouldn't slurm tell me in the beginning before it started?

Additionally, if this is a salmon issue (though I couldn't find anything online), it would be very helpful to know that too.

Thanks so much!

[h.mon]

I don't know the cause of the segfault, but you probably want to think twice about your Salmon command.

Salmon will output only one quantification, as it considers multiple files as technical replicates which one wants to quantify together. With D*_{1..3}/*.gz, you are globing over all files from all folders matching the pattern. Judging by the names of your input files, this isn't what you want - you probably want to quantify all files from each sample folder together (and don't forget to exclude the test_clean.fq.gz file.

[Daniel]

Thanks so much! It looks like that was the cause of the segfault (maybe because it was trying to quantify all 16 files together, it was too memory extensive?)

I fixed it by adding the directories to a .txt and looping over the txt, calling salmon quant one-by-one. I'm not sure I did this in the best way (for example in STAR, there is a way to have multiple files aligned into one output file, but I did not seem to see this with salmon.)

Once it finishes running, do you know if I can just combine the different output files on their rows?

[ATpoint]

Please post the salmon log. Hoe much memory did you request from that job? Please shoe the slurm submission headers/command.

[Daniel]

Thanks for your response! Here's the submission with the command:

#!/bin/bash
#SBATCH --job-name=mapReads
#SBATCH --output="slurm-logSalmonMapping.txt"
#SBATCH --ntasks=1
#SBATCH --nodes=1
#SBATCH --mem=0

salmon quant -i /home/user/fol2/reference_transcriptome/transcripts_index -l A \
-r /home/user/fol2/raw_fastq/D*_{1..3}/*.gz \
--validateMappings -o transcripts_quant

Here's the entire log:

Version Info: This is the most recent version of salmon.
### salmon (selective-alignment-based) v1.9.0
### [ program ] => salmon
### [ command ] => quant
### [ index ] => { /home/user/fol2/reference_transcriptome/transcripts_index }
### [ libType ] => { A }
### [ unmatedReads ] => { /home/user/fol2/raw_fastq/DMSO_1/01-dmsoR01_S1_L001_R1_001.fastq.gz /home/user/fol2/raw_fastq/DMSO_1/01-dmsoR01_S1_L002_R1_001.fastq.gz /home/user/fol2/raw_fastq/DMSO_1/01-dmsoR01_S1_L003_R1_001.fastq.gz /home/user/fol2/raw_fastq/DMSO_1/01-dmsoR01_S1_L004_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_1/04-DOXR01_S4_L001_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_1/04-DOXR01_S4_L002_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_1/04-DOXR01_S4_L003_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_1/04-DOXR01_S4_L004_R1_001.fastq.gz /home/user/fol2/raw_fastq/DMSO_2/02-dmsoR02_S2_L001_R1_001.fastq.gz /home/user/fol2/raw_fastq/DMSO_2/02-dmsoR02_S2_L002_R1_001.fastq.gz /home/user/fol2/raw_fastq/DMSO_2/02-dmsoR02_S2_L003_R1_001.fastq.gz /home/user/fol2/raw_fastq/DMSO_2/02-dmsoR02_S2_L004_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_2/05-DOXR02_S5_L001_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_2/05-DOXR02_S5_L002_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_2/05-DOXR02_S5_L003_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_2/05-DOXR02_S5_L004_R1_001.fastq.gz /home/user/fol2/raw_fastq/DMSO_3/03-dmsoR03_S3_L001_R1_001.fastq.gz /home/user/fol2/raw_fastq/DMSO_3/03-dmsoR03_S3_L002_R1_001.fastq.gz /home/user/fol2/raw_fastq/DMSO_3/03-dmsoR03_S3_L003_R1_001.fastq.gz /home/user/fol2/raw_fastq/DMSO_3/03-dmsoR03_S3_L004_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_3/06-DOXR03_S6_L001_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_3/06-DOXR03_S6_L002_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_3/06-DOXR03_S6_L003_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_3/06-DOXR03_S6_L004_R1_001.fastq.gz /home/user/fol2/raw_fastq/DOX_3/test_clean.fq.gz }
### [ validateMappings ] => { }
### [ output ] => { transcripts_quant }
Logs will be written to transcripts_quant/logs
[2023-01-22 12:58:46.641] [jointLog] [info] setting maxHashResizeThreads to 64
[2023-01-22 12:58:46.641] [jointLog] [info] Fragment incompatibility prior below threshold.  Incompatible fragments will be ignored.
[2023-01-22 12:58:46.642] [jointLog] [info] Usage of --validateMappings implies use of minScoreFraction. Since not explicitly specified, it is being set to 0.65
[2023-01-22 12:58:46.642] [jointLog] [info] Setting consensusSlack to selective-alignment default of 0.35.
[2023-01-22 12:58:46.642] [jointLog] [info] parsing read library format
[2023-01-22 12:58:46.644] [jointLog] [info] There is 1 library.
[2023-01-22 12:58:46.650] [jointLog] [info] Loading pufferfish index
[2023-01-22 12:58:46.651] [jointLog] [info] Loading dense pufferfish index.
-----------------------------------------
| Loading contig table | Time = 20.006 s
-----------------------------------------
size = 36145765
-----------------------------------------
| Loading contig offsets | Time = 152.15 ms
-----------------------------------------
-----------------------------------------
| Loading reference lengths | Time = 120.27 us
-----------------------------------------
-----------------------------------------
| Loading mphf table | Time = 1.8247 s
-----------------------------------------
size = 3751182992
Number of ones: 36145764
Number of ones per inventory item: 512
Inventory entries filled: 70598
-----------------------------------------
| Loading contig boundaries | Time = 4.984 s
-----------------------------------------
size = 3751182992
-----------------------------------------
| Loading sequence | Time = 1.0573 s
-----------------------------------------
size = 2666810072
-----------------------------------------
| Loading positions | Time = 11.933 s
-----------------------------------------
size = 3267117988
-----------------------------------------
| Loading reference sequence | Time = 923.04 ms
-----------------------------------------
-----------------------------------------
| Loading reference accumulative lengths | Time = 139.18 us
-----------------------------------------




/var/lib/slurm-llnl/slurmd/job00432/slurm_script: line 13: 282083 Segmentation fault      (core dumped) salmon quant -i /home/user/fol2/reference_transcriptome/transcripts_index -l A -r /home/user/fol2/raw_fastq/D*_{1..3}/*.gz --validateMappings -o transcripts_quant