Drop conda dependency: replace hmmer/prodigal/seqkit with pyhmmer/pyrodigal/Python#98
Merged
Merged
Conversation
…odigal/Python Pynteny shelled out to three conda-only binaries (hmmsearch/hmmscan, prodigal and seqkit). This replaces all of them with pure-Python, pip-installable equivalents so Pynteny can be installed via pip without conda. - wrappers.run_HMM_search: run searches with pyhmmer and write a standard HMMER3 tabular (--tblout) file, so the existing Bio.SearchIO parser and result reuse are unchanged. Adds a translator for common hmmsearch_args (--cut_ga/_nc/_tc, -E/-T/--domE/-Z, ...). - wrappers.run_seqkit_nodup: reimplement seqkit rmdup -s with pyfastx. - GeneAnnotator.annotate: predict genes in-process with pyrodigal (threaded; GIL released during find_genes), writing prodigal-format FASTA so from_prodigal_output is unchanged. Removes run_prodigal and the split-by-contigs + multiprocessing dance. - FASTA.filter_by_IDs: replace seqkit grep with a case-insensitive pyfastx ID filter. - FASTAmerger.merge: replace the `printf | xargs cat` shell-out with Python file concatenation. Packaging: declare runtime deps in pyproject.toml (previously only in the conda recipe); drop hmmer/prodigal/seqkit from both conda recipes and the dev env (pyhmmer/pyrodigal via pip); convert the Dockerfile to a pure-pip build; update README install and dependency sections. Also fixes a pre-existing bug in FASTA.add_prefix_to_records: it opened output_file with "w+" while pyfastx read the same path lazily, truncating the source whenever output_file == self.file_path (so `pynteny build --prefix` produced an empty database). It now writes to a temp file and moves it into place. test_build_fasta is strengthened to assert non-empty, prefixed output. Closes #97 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- tests.yml: replace the conda env (conda-incubator/setup-miniconda + bioconda) with a pure-pip matrix (actions/setup-python, `pip install .`) over Python 3.9/3.12 on ubuntu + macos. Conda is no longer required after the pyhmmer/pyrodigal migration, and this also fixes the macOS job, which failed creating a py3.8/bioconda env on the ARM64 runner. Add `synchronize` to the pull_request triggers so pushes to a PR re-run the tests. - filter.py: reformat with black 24.8.0 to satisfy the quality check (pre-existing drift; file otherwise unchanged). - preprocessing.py: make Database.is_fasta/is_gbk tolerant of parse errors. Biopython >=1.85 raises (instead of returning no records) when a file is not valid FASTA (e.g. a GenBank file with leading comments), which broke build() on GenBank input once biopython is pulled from PyPI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DataFrame.applymap was deprecated in pandas 2.1 and removed in pandas 3.0,
breaking get_meta_info_for_HMM on Python 3.12 (which pulls pandas 3.x). The
applymap call only replaced NaN with "", which fillna("") does directly and
works across all supported pandas versions.
Verified: full suite passes in a clean Python 3.12 venv (pandas 3.0.3,
biopython 1.87, pyfastx 2.3.1, pyhmmer 0.12.1, pyrodigal 3.7.1).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- docs/index.md: lead with `pip install pynteny`; drop the conda-required framing and the obsolete macOS ARM64 conda workaround (pip wheels now work on Apple Silicon). - CONTRIBUTING.md: remove the "HMMER and Prodigal are not available in pip" claim; switch the developer setup to a venv + `pip install -e .` (conda env file kept as optional). - docs/examples/example_api_colab.ipynb: replace the miniconda bootstrap cell with a plain `pip install pynteny`. Also drop paths-ignore from the tests workflow's pull_request trigger: the matrix jobs are required status checks, so a docs-only PR would otherwise skip them and deadlock waiting for a status that never reports. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #97.
Replaces the three conda-only external binaries Pynteny shelled out to with pure-Python, pip-installable equivalents, so Pynteny can be installed via
pipwithout conda.Changes
Engine swap (no more
subprocess):wrappers.run_HMM_search→ runs searches with pyhmmer and writes a standard HMMER3--tbloutfile, so the existingBio.SearchIO("hmmer3-tab")parser andreuse_hmmer_resultscaching are unchanged. Includes a translator for commonhmmsearch_args(--cut_ga/_nc/_tc,-E/-T/--domE/-Z, …).wrappers.run_seqkit_nodup→ reimplementsseqkit rmdup -swith pyfastx.GeneAnnotator.annotate→ predicts genes in-process with pyrodigal (threaded; GIL released duringfind_genes), writing prodigal-format FASTA sofrom_prodigal_outputis unchanged. Removes the split-by-contigs + multiprocessing dance.FASTA.filter_by_IDs→ replacesseqkit grepwith a case-insensitive pyfastx ID filter.FASTAmerger.merge→ replaces theprintf | xargs catshell-out with Python file concatenation.Packaging:
pyproject.toml(previously they lived only in the conda recipe).hmmer/prodigal/seqkitfrom both conda recipes and the dev env (pyhmmer/pyrodigal via pip).Bonus bug fix:
FASTA.add_prefix_to_recordsopenedoutput_filewith"w+"while pyfastx read the same path lazily, truncating the source wheneveroutput_file == self.file_path— sopynteny build --prefixproduced an empty database. It now writes to a temp file and moves it into place. This was pre-existing onmainand masked becausetest_build_fastaonly asserted the return type; that test is now strengthened to assert non-empty, prefixed output.Verification (local)
unittest/pytestsuite: 24/24 pass against the edited code (editable install).test_integration_search.hmmsearch/prodigal/seqkitremoved fromPATH— both succeeded, confirming nothing shells out to the binaries.🤖 Generated with Claude Code