Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pysam-0.8.4+gx1.tar.gz breaks on TravisCI #2575

Closed
peterjc opened this issue Jun 29, 2016 · 4 comments
Closed

pysam-0.8.4+gx1.tar.gz breaks on TravisCI #2575

peterjc opened this issue Jun 29, 2016 · 4 comments
Assignees
@dannon

Comments

@peterjc
Copy link
Contributor

peterjc commented Jun 29, 2016

Filing here as per @natefoo on Twitter: https://twitter.com/natefoo/status/748182338006319105

See https://travis-ci.org/peterjc/galaxy_blast/jobs/141112142 which downloads https://wheels.galaxyproject.org/packages/pysam-0.8.4+gx1.tar.gz and tries to install it, but fails with what looks like pysam-developers/pysam#79 suggesting the Galaxy wheel fo pysam is missing some headers?

  Running setup.py install for pysam
    package init file 'htslib/__init__.py' not found (or not a regular file)
    package init file 'htslib/htslib/__init__.py' not found (or not a regular file)
    package init file 'samtools/__init__.py' not found (or not a regular file)
    package init file 'samtools/win32/__init__.py' not found (or not a regular file)
    building 'pysam.csamtools' extension
    gcc -pthread -fno-strict-aliasing -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FILE_OFFSET_BITS=64 -D_USE_KNETFILE= -Isamtools -Ipysam -Ihtslib -I/opt/python/2.7.9/include/python2.7 -c pysam/csamtools.c -o build/temp.linux-x86_64-2.7/pysam/csamtools.o -Wno-error=declaration-after-statement -DSAMTOOLS=1
    gcc: error: pysam/csamtools.c: No such file or directory
    gcc: fatal error: no input files
    compilation terminated.
    error: command 'gcc' failed with exit status 4
    Complete output from command /home/travis/virtualenv/python2.7.9/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-ehLQC8/pysam/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-2wp2Q7-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/travis/virtualenv/python2.7.9/include/site/python2.7:
    running install

    running build

    running build_py

    creating build

    creating build/lib.linux-x86_64-2.7

    creating build/lib.linux-x86_64-2.7/pysam

    copying pysam/Pileup.py -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/__init__.py -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/namedtuple.py -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/version.py -> build/lib.linux-x86_64-2.7/pysam

    creating build/lib.linux-x86_64-2.7/pysam/include

    copying pysam/include/__init__.py -> build/lib.linux-x86_64-2.7/pysam/include

    package init file 'htslib/__init__.py' not found (or not a regular file)

    package init file 'htslib/htslib/__init__.py' not found (or not a regular file)

    package init file 'samtools/__init__.py' not found (or not a regular file)

    package init file 'samtools/win32/__init__.py' not found (or not a regular file)

    copying pysam/calignedsegment.pxd -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/calignmentfile.pxd -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/cbcf.pxd -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/cfaidx.pxd -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/chtslib.pxd -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/csamfile.pxd -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/csamtools.pxd -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/ctabix.pxd -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/ctabixproxies.pxd -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/cutils.pxd -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/cvcf.pxd -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/htslib_util.h -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/pysam_stream.h -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/pysam_util.h -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/samfile_util.h -> build/lib.linux-x86_64-2.7/pysam

    copying pysam/tabix_util.h -> build/lib.linux-x86_64-2.7/pysam

    creating build/lib.linux-x86_64-2.7/pysam/include/htslib

    copying htslib/config.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib

    copying htslib/hfile_internal.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib

    copying htslib/version.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib

    creating build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/bgzf.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/faidx.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/hfile.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/hts.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/hts_defs.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/kfunc.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/khash.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/khash_str2int.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/klist.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/knetfile.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/kseq.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/ksort.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/kstring.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/regidx.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/sam.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/synced_bcf_reader.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/tbx.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/vcf.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/vcf_sweep.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    copying htslib/htslib/vcfutils.h -> build/lib.linux-x86_64-2.7/pysam/include/htslib/htslib

    creating build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/bam.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/bam2bcf.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/bam_endian.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/bam_lpileup.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/bam_plbuf.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/bam_tview.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/errmod.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/kprobaln.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/pysam.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/sam.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/sam_header.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/sample.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/samtools.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/stats_isize.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    copying samtools/version.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools

    creating build/lib.linux-x86_64-2.7/pysam/include/samtools/win32

    copying samtools/win32/xcurses.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools/win32

    copying samtools/win32/zconf.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools/win32

    copying samtools/win32/zlib.h -> build/lib.linux-x86_64-2.7/pysam/include/samtools/win32

    running build_ext

    building 'pysam.csamtools' extension

    creating build/temp.linux-x86_64-2.7

    creating build/temp.linux-x86_64-2.7/pysam

    creating build/temp.linux-x86_64-2.7/samtools

    creating build/temp.linux-x86_64-2.7/samtools/misc

    creating build/temp.linux-x86_64-2.7/samtools/test

    creating build/temp.linux-x86_64-2.7/htslib

    creating build/temp.linux-x86_64-2.7/htslib/cram

    gcc -pthread -fno-strict-aliasing -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D_FILE_OFFSET_BITS=64 -D_USE_KNETFILE= -Isamtools -Ipysam -Ihtslib -I/opt/python/2.7.9/include/python2.7 -c pysam/csamtools.c -o build/temp.linux-x86_64-2.7/pysam/csamtools.o -Wno-error=declaration-after-statement -DSAMTOOLS=1

    gcc: error: pysam/csamtools.c: No such file or directory

    gcc: fatal error: no input files

    compilation terminated.

    error: command 'gcc' failed with exit status 4

    ----------------------------------------
@natefoo
Copy link
Member

natefoo commented Jun 29, 2016

Thanks for submitting this. I was aware of it but haven't had a chance to fix it yet - typically it does not come up since pip should prefer our wheels over our sdists. I'll fix the sdist ASAP.

Your Travis builds aren't using Galaxy pip, which is why it isn't downloading the wheels. I am not sure why - I know this was already fixed once. If you remove --skip-venv from bash scripts/common_startup.sh --skip-venv --dev-wheels, does this fix it?

@peterjc
Copy link
Contributor Author

peterjc commented Jun 29, 2016 •
edited
Loading

I'm intending to update my TravisCI setup to more closely follow the IUC setup, so problem may go away for me...

@peterjc
Copy link
Contributor Author

peterjc commented Jun 29, 2016

Failing TravisCI was using this:

#Older Galaxy versions using eggs (like 15.10) will ignore the extra arguments:
 - bash scripts/common_startup.sh --skip-venv --dev-wheels

Just switched to this which I was already using with TravisCI on another repository:

#Older Galaxy versions using eggs (like 15.10) will ignore the extra arguments:
 - bash scripts/common_startup.sh --no-create-venv --dev-wheels

Thanks! I'll leave this issue open for the underlying problem with the pysam not building.

@dannon dannon self-assigned this Nov 3, 2017
@natefoo
Copy link
Member

natefoo commented Nov 13, 2017

Pysam source builds (even for the older versions) should be fixed now.

It should be less of an issue going forward because we'll be using an unmodified upstream pysam. That said, unless I keep making modified source tarballs as described in galaxyproject/starforge#180 and uploading them to wheels.galaxyproject.org for whatever releases we pin, local builds will fail if you do not have Cython installed.

At least, that will be the case until pip with PEP 518 pyproject.toml support: pypa/pip#4144 is released, and pysam is updated to use pyproject.toml.

But, I'm closing this, as the issue is fixed for now.

@natefoo natefoo closed this as completed Nov 13, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dannon dannon

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@peterjc @dannon @natefoo