Nextflow Rmd render script cannot find input files?

Question

[Eliveri]

My nextflow workflow calls Rmd render (NXF version 22.11.0-edge run with sge, apptainer in scratch directory). But the Rmd file is unable to find the files, directory path is passed in as a param.

nextflow.config has param outdir = "../results"

params.rscript = "$projectDir/run_quality_report.Rmd"

process run_report {
    publishDir params.outdir, mode:'copy'
    input:
    tuple val(pair_id), path('file.tsv'),
    path rscript

    output:
    file('report.html')

    script:
    """
    Rscript -e 'rmarkdown::render(input = "$rscript", output_dir = getwd(), params = list(directory = "${params.outdir}"))'
    """
}

Error message. The file is not being found.

Command executed:

  Rscript -e 'rmarkdown::render(input = "run_quality_report.Rmd", output_dir = getwd(), params = list(directory = "../results"))'

Command error:  
  processing file: run_quality_report.Rmd
  Error: '../results/Bam_stats_pf_Final.tsv' does not exist in current working directory ('/scratch/769451.1.long.q/nxf.jI7GjjSY7c').
  Execution halted

Work dir:
  /wynton/scratch/finterly/new_workflow/work/91/b1888cccc40f39d394ed315b4ba432

The Rmd file looks like this:

title: ...
params:
  directory:
    value: x
---
```
file_df <-read_tsv(file.path(params$directory, "file.tsv"), show_col_types = FALSE)
```

[Pierre Lindenbaum]

tuple val(pair_id), path('file.tsv'),

I'm surprised that the input contains a path with a quoted string. It looks like an error.

[Pierre Lindenbaum]

and how about all your previous questions left without comment or validation ? Nextflow Gatk DepthOfCoverage: Failure working with the tmp directory ; Nextflow workflow errors when run from Scratch dir? ; Nextflow: [E::bwa_idx_load_from_disk] fail to locate the index files ; Nextflow Singularity/Apptainer error: command not found ; Fetching UniProt ID Mapping results as TSV using Python? ; etc...

[Eliveri]

Hi. Thank you for bringing this to my attention. I will try my best to leave comments and validation (as in select an answer?).

Answer 1

[Pierre Lindenbaum]

use a full path for params.outdir

not:

  --outdir ../results/Bam_stats_pf_Final.ts

but

  --outdir ${PWD}/../results/Bam_stats_pf_Final.ts

[Pierre Lindenbaum]

and it looks like it was the same problem as previously: Nextflow: [E::bwa_idx_load_from_disk] fail to locate the index files ...

[Eliveri]

Thank you. Making sure the file path is a full path and passing the outdir param in as an input resolved the issue.

input:
    tuple val(pair_id), path(file_tsv),
    path rscript
    path outdir