CWL: Linker error doing 'pip install cwlref-runner' on CentOS 6 system

Question

[Kevin Ernst]

I get the following linker error trying to pip install cwel-runner on a CentOS 6 system, within a virtualenv using python27-python from the "centos-sclo-rh" repository:

$ virtualenv venv
$ source venv/bin/activate
(venv)$ pip install cwlref-runner
# >>> the usual 'pip' chatter, followed by a lot of compiler output <<<
gcc -pthread -shared -L/data/lab/opt/lib -L/data/lab/opt/lib64 -L/data/lab/local/lib -I/data/lab/opt/include -I/data/lab/local/include build/temp.linux-x86_64-2.7/ext/_ruamel_yaml.o build/temp.linux-x86_64-2.7/ext/api.o build/temp.linux-x86_64-2.7/ext/writer.o build/temp.linux-x86_64-2.7/ext/dumper.o build/temp.linux-x86_64-2.7/ext/loader.o build/temp.linux-x86_64-2.7/ext/reader.o build/temp.linux-x86_64-2.7/ext/scanner.o build/temp.linux-x86_64-2.7/ext/parser.o build/temp.linux-x86_64-2.7/ext/emitter.o -L/opt/rh/python27/root/usr/lib64 -lpython2.7 -o build/lib.linux-x86_64-2.7/_ruamel_yaml.so
    /usr/bin/ld: /data/lab/opt/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
    /data/lab/opt/lib/libpython2.7.a: could not read symbols: Bad value
    collect2: ld gab 1 als Ende-Status zurück
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/users/myuserid/devel/cwl-workshop-20170614/venv/bin/python2 -c "import setuptools, tokenize;__file__='/tmp/pip-build-4J6vNA/ruamel.yaml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-_0i2fv-record/install-record.txt --single-version-externally-managed --compile --install-headers /users/ern6xv/devel/cwl-workshop-20170614/venv/include/site/python2.7/ruamel.yaml" failed with error code 1 in /tmp/pip-build-4J6vNA/ruamel.yaml

The problem seems to be with some component of the ruamel.yaml package failing to compile ("a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and map key order" according to its PyPI description). The German error message from collect2 means "ld returned 1 as exit code" or something along those lines.

Is there perhaps a quick resolution to this problem, without needing to hunt down where the linker error is, or worse, having to patch or recompile the offending module?

Answer 1

[Kevin Ernst]

It's entirely possible this situation arose in my case because I export CFLAGS and LDFLAGS, which contain another (older) Python's include and library files.

But if anyone finds themselves in this situation on CentOS 6, there is a relatively painless solution which just involves updating pip (within the virtualenv) and trying again:

(venv)$ pip install --upgrade pip
(venv)$ pip install ruamel.yaml  # succeeds this time, no errors
(venv)$ pip install cwlref-runner

Upgrade the virtualenv to use the latest version of pip (9.0.1 as of this writing), which can then fetch a precompiled "wheel" for ruamel.yaml appropriate for the current host's architecture, precluding the need to compile from source.

At that point, the pip install cwlref-runner runs through without further incident.