nfcore/fetchngs : Error: fasterq-dump cannot create this file

Question

[Pierre Lindenbaum]

I'm trying to run nfcore/fetchngs https://nf-co.re/fetchngs/1.11.0 (singularity+slurm ) with an input only containing SRX10474968 and with --force_sratools_download

all my processes FASTERQDUMP fail:

    8       a0/971d51       3250085 NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOLS_FASTERQDUMP (SRX10474968_SRR14103032) FAILED  3       2024-01-24 22:05:49.415 29.9s   25s     -       -       -    --
    9       20/a9ce28       3250087 NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOLS_FASTERQDUMP (SRX10474968_SRR14103035) FAILED  3       2024-01-24 22:05:54.398 1m 20s  50s     -       -       -    --
    10      67/4c125b       3250089 NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOLS_FASTERQDUMP (SRX10474968_SRR14103033) FAILED  3       2024-01-24 22:05:54.507 1m 20s  50s     -       -       -    --
    11      2e/785287       3250091 NFCORE_FETCHNGS:SRA:FASTQ_DOWNLOAD_PREFETCH_FASTERQDUMP_SRATOOLS:SRATOOLS_FASTERQDUMP (SRX10474968_SRR14103034) FAILED  3       2024-01-24 22:05:59.398 1m 15s  50s     -       -       -    --

with a .command.err starts with:

            Error: fasterq-dump cannot create this file: '(...)/2e/785287648c6a7c99e4acada7006045/SRX10474968_SRR14103034'

            Error: fasterq-dump cannot create this file: '(...)/2e/785287648c6a7c99e4acada7006045/SRX10474968_SRR14103034'

    spots read      : 46,524,055
    reads read      : 93,048,110

the command in .command.sh is:

export NCBI_SETTINGS="$PWD/user-settings.mkfg"


fasterq-dump \
    --split-files --include-technical \
    --threads 6 \
    --outfile SRX10474968_SRR14103033 \
     \
    SRR14103033

(...)

how can I fix this please ?

cross-posted: https://nfcore.slack.com/archives/C024915L4KB/p1706130702887529

[ATpoint]

Not helping, but why all the overhead with such a pipeline when you can enter the accession at sra-explorer.info and get a wget link? It could not be any easier!

[Pierre Lindenbaum]

ATpoint yes, but in real life I've got ~200 accessions numbers :-P . Furthermore, If I can setup such 'simple' pipeline on our new cluster, then I can run more complicated workflows like SAREK... :-)

Answer 1

[Pierre Lindenbaum]

looks like a bug in SRA: https://github.com/ncbi/sra-tools/issues/865

I am currently trying to download a couple of raw sequencing data files using sra-tools prefetch and fasterq-dump. Prefetch works fine but I get a weird error when trying to convert the generated *.sra file to fastq with fasterq-dump. The data is paired-end and the actual path should be /scratch/daniel.malzl/work/aa/7ab6e5d29db7a0352a1f1cd4af2af3/SRX10737613_SRR14385311 but judging by the error message there seems to be some bug in the renaming code because it says the following:

[pbioinf]

Reading that github thread, is there a dot (.) in your path somewhere other than in the file extension.

[Pierre Lindenbaum]

yes, the scratch partition where I can work is my email.... :-P

/scratch/nautilus/users/[e-mail]/work/....

[pbioinf]

I feel your pain, I've had a similar issue with a linux username that contained a dot because it was derived from an active directory username and it broke docker.

Answer 2

[Philipp Bayer]

On our cluster, you need to use cache = 'lenient' in the nextflow config to get around timing problems with the shared network storage which lead to similar errors. Alternatively, do you have file number limits on your working directory?

[Pierre Lindenbaum]

sorry that didn't work.