How to install `BETA` on linux server?

Question

[Dan]

Hello:

I am trying to install BETA on HPC:

http://cistrome.org/BETA/index.html#inst

I create virtual environment using conda:

conda create -n BETA python=2.7

conda activate BETA

unzip BETA_1.0.7.zip
cd BETA_1.0.7
python setup.py install


cc -Wall -g motif.c misp.c -o misp -O3 -lz -lm
In file included from motif.c:9:
motif.h:18:10: fatal error: zlib.h: No such file or directory
   18 | #include <zlib.h>
      |          ^~~~~~~~
compilation terminated.
misp.c:1:10: fatal error: zlib.h: No such file or directory
    1 | #include <zlib.h>
      |          ^~~~~~~~
compilation terminated.
make: *** [all] Error 1
running install
running bdist_egg
running egg_info
writing requirements to BETA_Package.egg-info/requires.txt
writing BETA_Package.egg-info/PKG-INFO
writing top-level names to BETA_Package.egg-info/top_level.txt
writing dependency_links to BETA_Package.egg-info/dependency_links.txt
reading manifest file 'BETA_Package.egg-info/SOURCES.txt'
writing manifest file 'BETA_Package.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/bayesian_motif_comp.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/fastafrombed.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/Up_Down_score.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/motif_scan.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/runbeta.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/OptValidator.py -> build/bdist.linux-x86_64/egg/BETA
creating build/bdist.linux-x86_64/egg/BETA/templates
copying build/lib/BETA/templates/styles.css -> build/bdist.linux-x86_64/egg/BETA/templates
copying build/lib/BETA/templates/motiflogos.zip -> build/bdist.linux-x86_64/egg/BETA/templates
copying build/lib/BETA/templates/script.js -> build/bdist.linux-x86_64/egg/BETA/templates
copying build/lib/BETA/templates/template.html -> build/bdist.linux-x86_64/egg/BETA/templates
copying build/lib/BETA/templates/subtemplate.html -> build/bdist.linux-x86_64/egg/BETA/templates
copying build/lib/BETA/PScore.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/__init__.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/Up_Down_distance.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/permp.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/expr_combine.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/motif_clustering.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/corelib.py -> build/bdist.linux-x86_64/egg/BETA
creating build/bdist.linux-x86_64/egg/BETA/references
copying build/lib/BETA/references/cistrome.db -> build/bdist.linux-x86_64/egg/BETA/references
copying build/lib/BETA/references/hg19.refseq -> build/bdist.linux-x86_64/egg/BETA/references
copying build/lib/BETA/references/cistrome.xml -> build/bdist.linux-x86_64/egg/BETA/references
copying build/lib/BETA/references/hg18.refseq -> build/bdist.linux-x86_64/egg/BETA/references
copying build/lib/BETA/references/hg19_CTCF_bound.bed -> build/bdist.linux-x86_64/egg/BETA/references
copying build/lib/BETA/references/mm9.refseq -> build/bdist.linux-x86_64/egg/BETA/references
copying build/lib/BETA/references/Entrez_Symbol_DBD.txt -> build/bdist.linux-x86_64/egg/BETA/references
copying build/lib/BETA/references/hg38.refseq -> build/bdist.linux-x86_64/egg/BETA/references
copying build/lib/BETA/references/mm10.refseq -> build/bdist.linux-x86_64/egg/BETA/references
copying build/lib/BETA/references/mm9_CTCF_bound.bed -> build/bdist.linux-x86_64/egg/BETA/references
copying build/lib/BETA/fileformat_check.py -> build/bdist.linux-x86_64/egg/BETA
copying build/lib/BETA/MotifParser.py -> build/bdist.linux-x86_64/egg/BETA
byte-compiling build/bdist.linux-x86_64/egg/BETA/bayesian_motif_comp.py to bayesian_motif_comp.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/fastafrombed.py to fastafrombed.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/Up_Down_score.py to Up_Down_score.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/motif_scan.py to motif_scan.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/runbeta.py to runbeta.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/OptValidator.py to OptValidator.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/PScore.py to PScore.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/Up_Down_distance.py to Up_Down_distance.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/permp.py to permp.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/expr_combine.py to expr_combine.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/motif_clustering.py to motif_clustering.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/corelib.py to corelib.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/fileformat_check.py to fileformat_check.pyc
byte-compiling build/bdist.linux-x86_64/egg/BETA/MotifParser.py to MotifParser.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
installing scripts to build/bdist.linux-x86_64/egg/EGG-INFO/scripts
running install_scripts
running build_scripts
creating build/bdist.linux-x86_64/egg/EGG-INFO/scripts
copying build/scripts-2.7/misp -> build/bdist.linux-x86_64/egg/EGG-INFO/scripts
copying build/scripts-2.7/BETA -> build/bdist.linux-x86_64/egg/EGG-INFO/scripts
changing mode of build/bdist.linux-x86_64/egg/EGG-INFO/scripts/misp to 755
changing mode of build/bdist.linux-x86_64/egg/EGG-INFO/scripts/BETA to 755
copying BETA_Package.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying BETA_Package.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying BETA_Package.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying BETA_Package.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying BETA_Package.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/BETA_Package-1.0.7-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing BETA_Package-1.0.7-py2.7.egg
Removing /lila/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/BETA_Package-1.0.7-py2.7.egg
Copying BETA_Package-1.0.7-py2.7.egg to /lila/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages
BETA-Package 1.0.7 is already the active version in easy-install.pth
Installing misp script to /home/lid/miniconda3/envs/BETA/bin
Installing BETA script to /home/lid/miniconda3/envs/BETA/bin

Installed /lila/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/BETA_Package-1.0.7-py2.7.egg
Processing dependencies for BETA-Package==1.0.7
Searching for numpy
Reading https://pypi.org/simple/numpy/
Downloading https://files.pythonhosted.org/packages/e4/a9/6704bb5e1d1d778d3a6ee1278a8d8134f0db160e09d52863a24edb58eab5/numpy-1.24.2.tar.gz#sha256=003a9f530e880cb2cd177cba1af7220b9aa42def9c4afc2a2fc3ee6be7eb2b22
Best match: numpy 1.24.2
Processing numpy-1.24.2.tar.gz
Writing /tmp/easy_install-7cwqYP/numpy-1.24.2/setup.cfg
Running numpy-1.24.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-7cwqYP/numpy-1.24.2/egg-dist-tmp-JA2SyI
Traceback (most recent call last):
  File "setup.py", line 59, in <module>
    main()
  File "setup.py", line 52, in main
    'Topic :: Database',
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/command/install.py", line 117, in do_egg_install
    cmd.run(show_deprecation=False)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 424, in run
    self.easy_install(spec, not self.no_deps)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 666, in easy_install
    return self.install_item(None, spec, tmpdir, deps, True)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 713, in install_item
    self.process_distribution(spec, dist, deps)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 758, in process_distribution
    [requirement], self.local_index, self.easy_install
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/pkg_resources/__init__.py", line 782, in resolve
    replace_conflicting=replace_conflicting
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1065, in best_match
    return self.obtain(req, installer)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1077, in obtain
    return installer(requirement)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 685, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 711, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 896, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1164, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1150, in run_setup
    run_setup(setup_script, args)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/sandbox.py", line 253, in run_setup
    raise
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/sandbox.py", line 166, in save_modules
   saved_exc.resume()
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/sandbox.py", line 141, in resume
    six.reraise(type, exc, self._tb)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/sandbox.py", line 154, in save_modules
    yield saved
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/sandbox.py", line 195, in setup_context
    yield
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/sandbox.py", line 250, in run_setup
    _execfile(setup_script, ns)
  File "/home/lid/miniconda3/envs/BETA/lib/python2.7/site-packages/setuptools/sandbox.py", line 44, in _execfile
    code = compile(script, filename, 'exec')
  File "/tmp/easy_install-7cwqYP/numpy-1.24.2/setup.py", line 45
    raise RuntimeError(f'Cannot parse version {FULLVERSION}')
                                                           ^
SyntaxError: invalid syntax

How should I install BETA on HPC? Thanks

Answer 1

[ATpoint]

Given you use conda already, why not just using it for the actual installation?

Else, use it to get the missing zlib dependency.

https://anaconda.org/bioconda/cistrome_beta

Answer 2

[colindaven]

If you can't install this yourself, then ask your admin why it is not available

misp.c:1:10: fatal error: zlib.h: No such file or directory

[lennykovac]

The compilation aborts, cause the script cant find zlib. A default header under any unix distro.

Try this: whereis zlib

zlib: /usr/include/zlib.h /mnt/c/Program Files/Intel/WiFi/bin/zlib1.dll /usr/share/man/man3/zlib.3.gz

If the output looks not like this, you can just download zlib and add it to your path. Repo for zlib.h

Answer 3

[Michael]

Each HPC is different (and all are very different from the average Ubuntu box), you should first consult the documentation at your institution: https://hpcdocs.mskcc.org/display/CLUS/Installing+Software

Many HPC use the module command to load software, check if the compile suite module is loaded with module list. It should show gcc or another optimized compiler suite. If not, check what is available with module avail. The support staff at you institute offers to install software globally or help you set it up for you. You should take their offer because only they know the specifics of their environment. They also mention singularity and docker. It is often easier to run software that way if you can find a ready container on Dockerhub (and that is compatible with the Hardware of your HPC).