FALCON running ERROR

Question

[1348988193]

Hi,everyone.I'am running a test on falcon,but there are some problems.Can anyone help me ?Thanks a lot.

Heading

I run the test by virtualenv. (fc_env) root@iZwz9gfo7g3450iygsp5zpZ:~/FALCON-integrate/FALCON/examples/test# fc_run.py fc_run.cfg

---

[output] [1333]$(u'lfs setstripe -c 12 /root/FALCON-integrate/FALCON/examples/test') sh: 1: lfs: not found [1333]WARNING: Call u'lfs setstripe -c 12 /root/FALCON-integrate/FALCON/examples/test' returned 32512. [INFO]Setup logging from file "None". [INFO]fc_run started with configuration fc_run.cfg [INFO] No target specified, assuming "assembly" as target [WARNING]In simple_pwatcher_bridge, pwatcher_impl=<module 'pwatcher.blocking'="" from="" '="" root="" falcon-integrate="" pypeflow="" pwatcher="" blocking.pyc'=""> [INFO]In simple_pwatcher_bridge, pwatcher_impl=<module 'pwatcher.blocking'="" from="" '="" root="" falcon-integrate="" pypeflow="" pwatcher="" blocking.pyc'=""> [INFO]job_type='string', job_queue='bash -C ${CMD}', sge_option='-pe smp 8 -q bash -C ${CMD}', use_tmpdir=False, squash=True, job_name_style='1' [INFO]u'Serializing 52 records' [INFO]u" Opened for write: u'./config.json'" [INFO]u' Serialized to 0.0MB as json' [INFO]u"Deserializing from u'/root/FALCON-integrate/FALCON/examples/test/config.json'" [INFO]u" Opened for read: u'/root/FALCON-integrate/FALCON/examples/test/config.json'" [INFO]u' Read 0.0MB as json' [INFO]u' Deserialized 52 records' [INFO]gen_task(python -m falcon_kit.mains.build_rdb --input-fofn-fn={input.raw_reads_fofn} --config-fn={input.config} --run-jobs-fn={output.run_jobs} --length-cutoff-fn={output.length_cutoff} --job-done-fn={output.db_build_done} touch {output.db_build_done}

inputs={u'raw_reads_fofn': 't1.fofn', u'config': u'config.json'},
outputs={u'raw_reads_db': u'0-rawreads/raw_reads.db', u'run_jobs': u'0-rawreads/run_jobs.sh', u'length_cutoff': u'0-rawreads/length_cutoff', u'db_build_done': u'0-rawreads/rdb_build_done'})

[INFO]gen_task(python -m falcon_kit.mains.daligner_split --wildcards={params.wildcards} --db-prefix={params.db_prefix} --skip-checks={params.skip_checks} --pread-aln={params.pread_aln} --run-jobs-fn={input.run_jobs} --db-fn={input.db} --split-fn={output.split} --bash-template-fn={output.bash_template}

inputs={u'run_jobs': u'0-rawreads/run_jobs.sh', u'db': u'0-rawreads/raw_reads.db'},
outputs={u'split': u'0-rawreads/daligner-split/all-units-of-work.json', u'bash_template': u'0-rawreads/daligner-split/daligner_bash_template.sh'})

[INFO]Num unsatisfied: 2, graph: 2 [INFO]About to submit: Node(0-rawreads) [INFO]!chmod +x /root/FALCON-integrate/FALCON/examples/test/0-rawreads/run-P0_0_rawreads_e15eb137ba238898f3c20c1f66b04914.bash [INFO]sleep 0.1s [INFO]Popen: 'bash -C /root/FALCON-integrate/pypeFLOW/pwatcher/mains/job_start.sh' executable=${PYPEFLOW_JOB_START_SCRIPT} + executable=/root/FALCON-integrate/FALCON/examples/test/0-rawreads/run-P0_0_rawreads_e15eb137ba238898f3c20c1f66b04914.bash timeout=${PYPEFLOW_JOB_START_TIMEOUT:-60} # wait 60s by default + timeout=60

Wait up to timeout seconds for the executable to become "executable",

then exec.

timeleft = int(timeout)

while [[ ! -x "${executable}" ]]; do if [[ "${timeout}" == "0" ]]; then echo "timed out waiting for (${executable})" exit 77 fi echo "not executable: '${executable}', waiting ${timeout}s" sleep 1 timeout=$((timeout-1)) done + [[ ! -x /root/FALCON-integrate/FALCON/examples/test/0-rawreads/run-P0_0_rawreads_e15eb137ba238898f3c20c1f66b04914.bash ]]

bash ${executable} + bash /root/FALCON-integrate/FALCON/examples/test/0-rawreads/run-P0_0_rawreads_e15eb137ba238898f3c20c1f66b04914.bash

• '[' '!' -d /root/FALCON-integrate/FALCON/examples/test/0-rawreads ']'
• cd /root/FALCON-integrate/FALCON/examples/test/0-rawreads
• eval '/bin/bash run.sh' ++ /bin/bash run.sh cd /root/FALCON-integrate/FALCON/examples/test/0-rawreads
• cd /root/FALCON-integrate/FALCON/examples/test/0-rawreads bash task.sh
• bash task.sh iZwz9gfo7g3450iygsp5zpZ HOME=/root LANG=en_US.UTF-8 LESSCLOSE=/usr/bin/lesspipe %s %s LESSOPEN=| /usr/bin/lesspipe %s LOGNAME=root LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:.tar=01;31:.tgz=01;31:.arc=01;31:.arj=01;31:.taz=01;31:.lha=01;31:.lz4=01;31:.lzh=01;31:.lzma=01;31:.tlz=01;31:.txz=01;31:.tzo=01;31:.t7z=01;31:.zip=01;31:.z=01;31:.Z=01;31:.dz=01;31:.gz=01;31:.lrz=01;31:.lz=01;31:.lzo=01;31:.xz=01;31:.bz2=01;31:.bz=01;31:.tbz=01;31:.tbz2=01;31:.tz=01;31:.deb=01;31:.rpm=01;31:.jar=01;31:.war=01;31:.ear=01;31:.sar=01;31:.rar=01;31:.alz=01;31:.ace=01;31:.zoo=01;31:.cpio=01;31:.7z=01;31:.rz=01;31:.cab=01;31:.jpg=01;35:.jpeg=01;35:.gif=01;35:.bmp=01;35:.pbm=01;35:.pgm=01;35:.ppm=01;35:.tga=01;35:.xbm=01;35:.xpm=01;35:.tif=01;35:.tiff=01;35:.png=01;35:.svg=01;35:.svgz=01;35:.mng=01;35:.pcx=01;35:.mov=01;35:.mpg=01;35:.mpeg=01;35:.m2v=01;35:.mkv=01;35:.webm=01;35:.ogm=01;35:.mp4=01;35:.m4v=01;35:.mp4v=01;35:.vob=01;35:.qt=01;35:.nuv=01;35:.wmv=01;35:.asf=01;35:.rm=01;35:.rmvb=01;35:.flc=01;35:.avi=01;35:.fli=01;35:.flv=01;35:.gl=01;35:.dl=01;35:.xcf=01;35:.xwd=01;35:.yuv=01;35:.cgm=01;35:.emf=01;35:.ogv=01;35:.ogx=01;35:.aac=00;36:.au=00;36:.flac=00;36:.m4a=00;36:.mid=00;36:.midi=00;36:.mka=00;36:.mp3=00;36:.mpc=00;36:.ogg=00;36:.ra=00;36:.wav=00;36:.oga=00;36:.opus=00;36:.spx=00;36:.xspf=00;36: MAIL=/var/mail/root PATH=/root/FALCON-integrate/fc_env/bin:/root/FALCON-integrate/fc_env/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/root/FALCON-integrate/fc_env/bin:/root/FALCON-integrate/fc_env/bin PWD=/root/FALCON-integrate/FALCON/examples/test/0-rawreads PYPEFLOW_JOB_START_SCRIPT=/root/FALCON-integrate/FALCON/examples/test/0-rawreads/run-P0_0_rawreads_e15eb137ba238898f3c20c1f66b04914.bash PYPEFLOW_JOB_START_TIMEOUT=60 SHELL=/bin/bash SHLVL=5 SSH_CLIENT=36.149.129.242 26124 22 SSH_CONNECTION=36.149.129.242 26124 172.16.252.178 22 SSH_TTY=/dev/pts/0 TERM=xterm USER=root _=/usr/bin/env VIRTUAL_ENV=/root/FALCON-integrate/fc_env XDG_RUNTIME_DIR=/run/user/0 XDG_SESSION_ID=1 /root/FALCON-integrate/FALCON/examples/test/0-rawreads 2018-03-13 20:36:09,873 - root - DEBUG - Running "/root/FALCON-integrate/pypeFLOW/pypeflow/do_task.py /root/FALCON-integrate/FALCON/examples/test/0-rawreads/task.json" 2018-03-13 20:36:09,874 - root - DEBUG - Checking existence of '/root/FALCON-integrate/FALCON/examples/test/0-rawreads/task.json' with timeout=30 2018-03-13 20:36:09,874 - root - DEBUG - Loading JSON from '/root/FALCON-integrate/FALCON/examples/test/0-rawreads/task.json' 2018-03-13 20:36:09,875 - root - DEBUG - {u'bash_template_fn': u'template.sh', u'inputs': {u'config': u'/root/FALCON-integrate/FALCON/examples/test/config.json', u'raw_reads_fofn': u'/root/FALCON-integrate/FALCON/examples/test/t1.fofn'}, u'outputs': {u'db_build_done': u'rdb_build_done', u'length_cutoff': u'length_cutoff', u'raw_reads_db': u'raw_reads.db', u'run_jobs': u'run_jobs.sh'}, u'parameters': {u'topdir': u'..'}} 2018-03-13 20:36:09,875 - root - DEBUG - CD: '/root/FALCON-integrate/FALCON/examples/test/0-rawreads' <- '/root/FALCON-integrate/FALCON/examples/test/0-rawreads' 2018-03-13 20:36:09,875 - root - DEBUG - Checking existence of u'/root/FALCON-integrate/FALCON/examples/test/t1.fofn' with timeout=30 2018-03-13 20:36:09,875 - root - DEBUG - Checking existence of u'/root/FALCON-integrate/FALCON/examples/test/config.json' with timeout=30 2018-03-13 20:36:09,875 - root - DEBUG - Checking existence of u'template.sh' with timeout=30 2018-03-13 20:36:09,875 - root - DEBUG - CD: '/root/FALCON-integrate/FALCON/examples/test/0-rawreads' <- '/root/FALCON-integrate/FALCON/examples/test/0-rawreads' 2018-03-13 20:36:09,876 - root - INFO - /bin/bash user_script.sh hostname
• hostname iZwz9gfo7g3450iygsp5zpZ pwd
• pwd /root/FALCON-integrate/FALCON/examples/test/0-rawreads date
• date Tue Mar 13 20:36:09 CST 2018

  Substitution will be similar to snakemake "shell".

python -m falcon_kit.mains.build_rdb --input-fofn-fn=/root/FALCON-integrate/FALCON/examples/test/t1.fofn --config-fn=/root/FALCON-integrate/FALCON/examples/test/config.json --run-jobs-fn=run_jobs.sh --length-cutoff-fn=length_cutoff --job-done-fn=rdb_build_done + python -m falcon_kit.mains.build_rdb --input-fofn-fn=/root/FALCON-integrate/FALCON/examples/test/t1.fofn --config-fn=/root/FALCON-integrate/FALCON/examples/test/config.json --run-jobs-fn=run_jobs.sh --length-cutoff-fn=length_cutoff --job-done-fn=rdb_build_done [INFO]sleep 0.2s INFO:root:Building rdb from '/root/FALCON-integrate/FALCON/examples/test/t1.fofn' ('/root/FALCON-integrate/FALCON/examples/test/config.json'), to write 'run_jobs.sh' INFO:root:u"Deserializing from '/root/FALCON-integrate/FALCON/examples/test/t1.fofn'" INFO:root:u" Opened for read: '/root/FALCON-integrate/FALCON/examples/test/t1.fofn'" INFO:root:u"Deserializing from '/root/FALCON-integrate/FALCON/examples/test/config.json'" INFO:root:u" Opened for read: '/root/FALCON-integrate/FALCON/examples/test/config.json'" INFO:root:u' Read 0.0MB as json' INFO:root:u' Deserialized 52 records' INFO:root:$(u'bash -vex build_rdb.sh')

!/bin/bash

set -vex + set -vex echo "PBFALCON_ERRFILE=$PBFALCON_ERRFILE" + echo PBFALCON_ERRFILE= PBFALCON_ERRFILE= set -o pipefail + set -o pipefail rm -f raw_reads.db .raw_reads.* # in case of re-run + rm -f raw_reads.db '.raw_reads.*'

fc_fasta2fasta < my.t1.fofn >| fc.fofn

while read fn; do cat $fn | fasta2DB -v raw_reads -i${fn##*/}; done < my.t1.fofn + read fn + fasta2DB -v raw_reads -it1.fasta + cat /root/FALCON-integrate/FALCON/examples/test/t1.fasta Adding 't1.fasta.fasta' ... File t1.fasta.fasta, Line 1: Pacbio header line format error WARNING:root:Call u'bash -vex build_rdb.sh' returned 256. Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/root/FALCON-integrate/fc_env/lib/python2.7/site-packages/falcon_kit-1.0+git.0b5ac4e9e358eaee1c7bb7d9d8934ad93d3fa736-py2.7-linux-x86_64.egg/falcon_kit/mains/build_rdb.py", line 74, in <module> main() File "/root/FALCON-integrate/fc_env/lib/python2.7/site-packages/falcon_kit-1.0+git.0b5ac4e9e358eaee1c7bb7d9d8934ad93d3fa736-py2.7-linux-x86_64.egg/falcon_kit/mains/build_rdb.py", line 70, in main run(vars(args)) File "/root/FALCON-integrate/fc_env/lib/python2.7/site-packages/falcon_kit-1.0+git.0b5ac4e9e358eaee1c7bb7d9d8934ad93d3fa736-py2.7-linux-x86_64.egg/falcon_kit/mains/build_rdb.py", line 28, in run io.syscall('bash -vex {}'.format(script_fn)) File "/root/FALCON-integrate/fc_env/lib/python2.7/site-packages/falcon_kit-1.0+git.0b5ac4e9e358eaee1c7bb7d9d8934ad93d3fa736-py2.7-linux-x86_64.egg/falcon_kit/io.py", line 158, in syscall raise Exception(msg) Exception: Call u'bash -vex build_rdb.sh' returned 256. 2018-03-13 20:36:09,950 - root - DEBUG - CD: '/root/FALCON-integrate/FALCON/examples/test/0-rawreads' -> '/root/FALCON-integrate/FALCON/examples/test/0-rawreads' 2018-03-13 20:36:09,950 - root - DEBUG - CD: '/root/FALCON-integrate/FALCON/examples/test/0-rawreads' -> '/root/FALCON-integrate/FALCON/examples/test/0-rawreads' 2018-03-13 20:36:09,950 - root - CRITICAL - Error in /root/FALCON-integrate/pypeFLOW/pypeflow/do_task.py with args="{'json_fn': '/root/FALCON-integrate/FALCON/examples/test/0-rawreads/task.json',\n 'timeout': 30,\n 'tmpdir': None}" Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/root/FALCON-integrate/pypeFLOW/pypeflow/do_task.py", line 250, in <module> main() File "/root/FALCON-integrate/pypeFLOW/pypeflow/do_task.py", line 242, in main run(vars(parsed_args)) File "/root/FALCON-integrate/pypeFLOW/pypeflow/do_task.py", line 236, in run run_cfg_in_tmpdir(cfg, tmpdir) File "/root/FALCON-integrate/pypeFLOW/pypeflow/do_task.py", line 212, in run_cfg_in_tmpdir run_bash(bash_template, myinputs, myoutputs, parameters) File "/root/FALCON-integrate/pypeFLOW/pypeflow/do_task.py", line 173, in run_bash util.system(cmd) File "/root/FALCON-integrate/pypeFLOW/pypeflow/util.py", line 35, in system raise Exception('cwd: {}\nrc: {} <- {!r}'.format(os.getcwd(), rc, cmd)) Exception: cwd: /root/FALCON-integrate/FALCON/examples/test/0-rawreads rc: 256 <- '/bin/bash user_script.sh'

real 0m0.127s user 0m0.096s sys 0m0.012s + finish + echo 'finish code: 1' finish code: 1 [ERROR]Task Node(0-rawreads) failed with exit-code=1 [INFO]recently_satisfied: set([]) [INFO]Num satisfied in this iteration: 0 [INFO]Num still unsatisfied: 2 [ERROR]Some tasks are recently_done but not satisfied: set([Node(0-rawreads)]) [ERROR]ready: set([]) submitted: set([]) [ERROR]Noop. We cannot kill blocked threads. Hopefully, everything will die on SIGTERM. Traceback (most recent call last): File "/root/FALCON-integrate/fc_env/bin/fc_run.py", line 11, in <module> load_entry_point('falcon-kit==1.0+git.0b5ac4e9e358eaee1c7bb7d9d8934ad93d3fa736', 'console_scripts', 'fc_run.py')() File "/root/FALCON-integrate/fc_env/lib/python2.7/site-packages/falcon_kit-1.0+git.0b5ac4e9e358eaee1c7bb7d9d8934ad93d3fa736-py2.7-linux-x86_64.egg/falcon_kit/mains/run1.py", line 625, in main main1(argv[0], args.config, args.logger) File "/root/FALCON-integrate/fc_env/lib/python2.7/site-packages/falcon_kit-1.0+git.0b5ac4e9e358eaee1c7bb7d9d8934ad93d3fa736-py2.7-linux-x86_64.egg/falcon_kit/mains/run1.py", line 159, in main1 input_fofn_plf=input_fofn_plf, File "/root/FALCON-integrate/fc_env/lib/python2.7/site-packages/falcon_kit-1.0+git.0b5ac4e9e358eaee1c7bb7d9d8934ad93d3fa736-py2.7-linux-x86_64.egg/falcon_kit/mains/run1.py", line 262, in run parameters={}, File "/root/FALCON-integrate/fc_env/lib/python2.7/site-packages/falcon_kit-1.0+git.0b5ac4e9e358eaee1c7bb7d9d8934ad93d3fa736-py2.7-linux-x86_64.egg/falcon_kit/pype.py", line 87, in gen_parallel_tasks wf.refreshTargets() File "/root/FALCON-integrate/pypeFLOW/pypeflow/simple_pwatcher_bridge.py", line 273, in refreshTargets self._refreshTargets(updateFreq, exitOnFailure) File "/root/FALCON-integrate/pypeFLOW/pypeflow/simple_pwatcher_bridge.py", line 339, in _refreshTargets raise Exception(msg) Exception: Some tasks are recently_done but not satisfied: set([Node(0-rawreads)])

[2668428466]

Have you solved your problem? I met the same question and want to ask for help.