Error running cutadapt in multithread mode [SOLVED]

Question

[Francesco]

I'm running cutadapt on MacOS Catalina 15.10.6:

This is cutadapt 2.10 with Python 3.8.3

I have pigz installed and the script is working perfetly in single core. I also tried specifying to use just 2 cores but I get the same error. I also tried to execute the trimming of the adapter in single ended mode but still nothing.

My code:

cutadapt -a GATCGGAAGAGCACACGTCTGAACTCCAGTCACACAGTGATCTCGTATGCCGTCTTCTGCTTG \
  -A AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT \
  --output $WD/out_R1.trimmed.fastq.gz \
  --paired-output $WD/out_R2.trimmed.fastq.gz \
  --cores=0 \
 $FILE_DIR/in_R1.fastq.gz \
 $FILE_DIR/in_R2.fastq.gz

My error :

Traceback (most recent call last):
  File "/Users/tucos/.local/bin/cutadapt", line 8, in <module>
    sys.exit(main())
  File "/Users/tucos/.local/lib/python3.8/site-packages/cutadapt/__main__.py", line 855, in main
    stats = r.run()
  File "/Users/tucos/.local/lib/python3.8/site-packages/cutadapt/pipeline.py", line 735, in run
    workers, connections = self._start_workers()
  File "/Users/tucos/.local/lib/python3.8/site-packages/cutadapt/pipeline.py", line 730, in _start_workers
    worker.start()
  File "/Users/tucos/opt/anaconda3/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/tucos/opt/anaconda3/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/tucos/opt/anaconda3/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Users/tucos/opt/anaconda3/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/tucos/opt/anaconda3/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/tucos/opt/anaconda3/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/tucos/opt/anaconda3/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_io.BufferedWriter' object

[ATpoint]

https://github.com/marcelm/cutadapt/issues/469

and https://github.com/marcelm/cutadapt/blob/master/CHANGES.rst on top.

issue 469: Cutadapt did not run under Python 3.8 on recent macOS versions.

Guess you need the devel version or just run single-threaded. I personally always pipe the putput directly into the aligner and then the bottleneck is always the alignment so single-threaded is sufficient.

Answer 1

[Francesco]

Thank you, I created a conda enviroment for cutadapt with python 3.7 and now it's working in multicore. Thank you so much!