Cellranger count fails at mapping

Question

[gnomee]

Hi all, I am fairly new to scRNA-Seq. I am trying to align 10X datasets using cellranger. However, cellranger fails at the mapping step during chemistry detection, and I do not understand why.

This is my code:

cellranger count --id=4CHH4R_tumor1 \
        --fastqs=data/K46R-4CHH4R/tumor1-h01 \
        --transcriptome=ReferenceGenome/refdata-cellranger-GRCh38-3.0.0/ \
        --localcores=12

Running it results in the following error:

/software/cellranger/3.0.1/cellranger-cs/3.0.1/bin
cellranger count (3.0.1)
Copyright (c) 2018 10x Genomics, Inc.  All rights reserved.
-------------------------------------------------------------------------------

Martian Runtime - '3.0.1-v3.1.0'
2019-11-19 14:12:15 [jobmngr] WARNING: configured to use 540GB of local memory, but only 180.2GB is currently available.
Serving UI at http://odcf-worker01:40484?auth=MIDdzQZmtbTMaiF1LkOT2qnuVXfoTepZAIa60S11Us0

Running preflight checks (please wait)...
2019-11-19 14:12:16 [runtime] (ready)           ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.EXPAND_SAMPLE_DEF
2019-11-19 14:12:16 [runtime] (run:local)       ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.EXPAND_SAMPLE_DEF.fork0.chnk0.main
2019-11-19 14:12:28 [runtime] (chunks_complete) ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.EXPAND_SAMPLE_DEF
Checking sample info...
Checking FASTQ folder...
Checking reference...
Checking reference_path (ReferenceGenome/refdata-cellranger-GRCh38-3.0.0/) on odcf-worker01...
Checking chemistry...
Checking optional arguments...
mrc: '3.0.1-v3.1.0'

mrp: '3.0.1-v3.1.0'

Anaconda: Python 2.7.14 :: Anaconda, Inc.

numpy: 1.14.2

scipy: 1.0.1

pysam: 0.14.1

h5py: 2.8.0

pandas: 0.22.0

STAR: STAR_2.5.1b

samtools: samtools 1.7
Using htslib 1.7
Copyright (C) 2018 Genome Research Ltd.

2019-11-19 14:12:50 [runtime] (ready)           ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY
2019-11-19 14:12:50 [runtime] (run:local)       ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY.fork0.chnk0.main
2019-11-19 14:12:50 [runtime] (ready)           ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.SC_RNA_ANALYZER.CHOOSE_DIMENSION_REDUCTION
2019-11-19 14:12:50 [runtime] (run:local)       ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.SC_RNA_ANALYZER.CHOOSE_DIMENSION_REDUCTION.fork0.chnk0.main
2019-11-19 14:12:50 [runtime] (ready)           ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.DISABLE_FEATURE_STAGES
2019-11-19 14:12:50 [runtime] (run:local)       ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.DISABLE_FEATURE_STAGES.fork0.chnk0.main
2019-11-19 14:12:53 [runtime] (chunks_complete) ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.DISABLE_FEATURE_STAGES
2019-11-19 14:12:53 [runtime] (chunks_complete) ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.SC_RNA_ANALYZER.CHOOSE_DIMENSION_REDUCTION
2019-11-19 14:13:02 [runtime] (progress)        ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY.fork0.chnk0: Building transcriptome...
2019-11-19 14:18:16 [runtime] (progress)        ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY.fork0.chnk0: Building kmer index...
2019-11-19 14:19:47 [runtime] (progress)        ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY.fork0.chnk0: Mapping reads...
2019-11-19 14:19:51 [runtime] (failed)          ID.4CHH4R_tumor1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY

[error] Pipestance failed. Error log at:
4CHH4R_tumor1/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/CHEMISTRY_DETECTOR/DETECT_CHEMISTRY/fork0/chnk0-u55c3d3ea52/_errors

Log message:
Traceback (most recent call last):
  File "/software/cellranger/3.0.1/martian-cs/v3.1.0/adapters/python/martian_shell.py", line 554, in _main
    stage.main()
  File "/software/cellranger/3.0.1/martian-cs/v3.1.0/adapters/python/martian_shell.py", line 519, in main
    self._run(lambda: self._module.main(args, outs))
  File "/software/cellranger/3.0.1/martian-cs/v3.1.0/adapters/python/martian_shell.py", line 488, in _run
    cmd()
  File "/software/cellranger/3.0.1/martian-cs/v3.1.0/adapters/python/martian_shell.py", line 519, in <lambda>
    self._run(lambda: self._module.main(args, outs))
  File "/software/cellranger/3.0.1/cellranger-cs/3.0.1/mro/stages/chemistry_detector/detect_chemistry/__init__.py", line 348, in main
    chemistry_name, _report, metrics = infer_sc3p_or_sc5p(chunks, txome_idx, vdj_idx)
  File "/software/cellranger/3.0.1/cellranger-cs/3.0.1/mro/stages/chemistry_detector/detect_chemistry/__init__.py", line 218, in infer_sc3p_or_sc5p
    metrics[read_type] = map_reads(fq_specs[read_type], kmer_idx_path, map_out_path)
  File "/software/cellranger/3.0.1/cellranger-cs/3.0.1/mro/stages/chemistry_detector/detect_chemistry/__init__.py", line 91, in map_reads
    '--fastqs', fq_spec_path])
  File "/software/cellranger/3.0.1/cellranger-cs/3.0.1/mro/stages/chemistry_detector/detect_chemistry/__init__.py", line 85, in run
    tk_subproc.check_call(args)
  File "/software/cellranger/3.0.1/cellranger-cs/3.0.1/tenkit/lib/python/tenkit/log_subprocess.py", line 37, in check_call
    return subprocess.check_call(*args, **kwargs)
  File "/software/cellranger/3.0.1/miniconda-cr-cs/4.3.21-miniconda-cr-cs-c10/lib/python2.7/subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['detect_chemistry', 'map-reads', '4CHH4R_tumor1/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/CHEMISTRY_DETECTOR/DETECT_CHEMISTRY/fork0/chnk0-u55c3d3ea52/files/kmers.idx', '/dev/null', '4CHH4R_tumor1/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/CHEMISTRY_DETECTOR/DETECT_CHEMISTRY/fork0/chnk0-u55c3d3ea52/files/R1_out.json', '--initial-reads', '10000', '--fastqs', '4CHH4R_tumor1/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/CHEMISTRY_DETECTOR/DETECT_CHEMISTRY/fork0/chnk0-u55c3d3ea52/files/R1_in.json']' returned non-zero exit status 101

Can anybody give me an idea what is going wrong here?

I downloaded the reference genome from the 10X genomics website.

[GenoMax]

Are the fastq's in cellranger format i.e. the index sequence is in a separate file (I1)?

[swbarnes2]

Newer versions of cellranger don't need that.

[GenoMax]

OP appears to be using v.3.0.1 which is not the latest (v.3.1.0).

[swbarnes2]

I'm pretty sure all the 3+ versions do not need the indices as a separate file. I just did a cellranger testrun with 3.0.2 after messing up the name of the index read, and it worked fine.

[gnomee]

I rerun it with the latest cellranger version (3.1.0) now, however this results in the same error.

Answer 1

[gnomee]

I think I solved the issue now. I realized that the directory specified in --fastqs only contained symlinks to the acutal fastq files, which cellranger apparently could not handle. Why it failed only at that specific step I do not know, but it runs smoothly now that I specified the correct directory with the fastq files.

[bioinformatics2020]

The one thing that bothers me about CellRanger's pipelines are the vague error messages. Both times I have run into issues have resulted in error messages than made the issue become more trivial than it should have been.

[Ömer An]

I don't think this could be a problem. I am also running cellranger using symlinks for --fastqs parameter and it works fine on CSI NGS Portal

Answer 2

[swbarnes2]

Since the command that's failing is "detect chemistry", that might be the problem.

How were the fastqs generated? Can you confirm that every read in both files has the right length?

One thing to try is to make mini fastqs, say, 100 lines or 100 sequences, see if you can get those to work.