Rfam ncrna

HISTORY.md

@@ -1,14 +1,42 @@
 History
 =======
 
-1.10.2 (2026-07-20)
+1.11.0 (2026-08-10)
 -------------
 
+**Requires a database update - re-run `pharokka install` before using this version**
+
+* Adds ncRNA annotation with [Infernal](http://eddylab.org/infernal/) `cmscan` against 
+  [Rfam](https://rfam.org) 15.1 (4,227 covariance models). This picks up structured RNAs 
+  that `pharokka` previously missed entirely - riboswitches, ribozymes, regulatory sRNAs, 
+  group I/II introns and leader elements
+    * Runs by default in single-genome mode, since it is cheap for a phage genome 
+      (~5 seconds for a 40 kb phage, ~14 seconds for a 140 kb phage on 8 threads, and 
+      scales well with `--threads`). Pass `--skip_rfam` to disable it
+    * Skipped by default in meta mode (`-m`) since runtime scales with assembly size 
+      (~2 min/Mbp on 8 threads). Pass `--meta_rfam` to run it anyway in meta mode
+    * By default, Rfam tRNA (RF00005) and tmRNA (RF00023) hits are discarded, since 
+      tRNAscan-SE and ARAGORN already annotate these and are more sensitive for phage 
+      sequence. Pass `--rfam_keep_trna` to keep them
+    * Rfam is purely additive - it does **not** replace tRNAscan-SE, ARAGORN or MinCED
+    * Adds new outputs: `{prefix}_ncrna.tsv` (one row per ncRNA hit, with Rfam accession, 
+      family, type, coordinates, bit score and E-value), `{prefix}_cmscan.tblout` (raw 
+      Infernal output), and `ncRNA` features in the `.gff`/`.gbk` (plus an `ncRNAs` row 
+      per contig in `{prefix}_cds_functions.tsv`)
+    * Requires [Infernal](http://eddylab.org/infernal/) `>= 1.1.4` (`conda install -c 
+      bioconda infernal`), only checked when Rfam annotation will actually run
+    * Requires the v1.11.0 database (adds the pressed Rfam covariance models) - running 
+      against an older database will fail with an explanatory error unless `--skip_rfam` 
+      is passed
 * `pharokka run --citation` (and the end-of-run summary) now also print our 
   [protocols paper](https://doi.org/10.1002/cpz1.70405) citation alongside the 
   existing Pharokka citation
 * Fixes a duplicated author name (`Vreugde S.`) in the protocols paper citation 
   in the README and docs
+* Fixes the database tarball filename being hardcoded to `v1.8.0` rather than derived 
+  from the database version
+* The PHROGs database version marker file is now derived from the database version 
+  instead of being hardcoded
 
 1.10.1 (2026-07-09)
 -------------

README.md

@@ -22,6 +22,8 @@
 pharokka
 =================
 
+### 🌐 Try `pharokka`, `phold` and `phynteny` in your browser - no installation required: **[https://phage-annotation.org](https://phage-annotation.org)**
+
 <p align="center">
   <img src="img/pharokka_logo.png" alt="pharokka Logo" height=400>
 </p>

docs/changelog.md

@@ -1,5 +1,62 @@
 # Changelog
 
+## v1.11.0 — ncRNA annotation with Infernal and Rfam
+
+### ncRNA annotation, on by default
+
+`pharokka run` now annotates non-coding RNAs by scanning the genome against
+[Rfam](https://rfam.org) 15.1 (4,227 covariance models) with
+[Infernal](http://eddylab.org/infernal/) `cmscan`.  This picks up structured
+RNAs that pharokka previously could not see at all — riboswitches, ribozymes,
+regulatory sRNAs, group I/II introns and leader elements.
+
+It runs **by default**, because it is inexpensive for a phage genome: roughly
+5 seconds for a typical 40 kb phage and 14 seconds for a 140 kb phage on 8
+threads. `--threads` scales it well (4–5x on 8 cores) even for a single genome.
+
+To turn it off:
+
+```bash
+pharokka run -i phage.fasta -o output -d database --skip_rfam
+```
+
+**In meta mode (`-m`) it is skipped by default**, since runtime scales with
+assembly size at roughly 2 min/Mbp on 8 threads. Use `--meta_rfam` to run it
+in meta mode anyway.
+
+New outputs:
+
+* `{prefix}_ncrna.tsv` — one row per ncRNA, with Rfam accession, family, type,
+  coordinates, bit score and E-value.
+* `{prefix}_cmscan.tblout` — the raw Infernal output.
+* `ncRNA` features in the `.gff` and `.gbk`, and an `ncRNAs` row per contig in
+  `{prefix}_cds_functions.tsv`.
+
+By default, Rfam tRNA (RF00005) and tmRNA (RF00023) hits are discarded, since
+tRNAscan-SE and ARAGORN already annotate these and are more sensitive on phage
+sequence.  Pass `--rfam_keep_trna` to keep them.
+
+**Rfam does not replace tRNAscan-SE, ARAGORN or MinCED** — it is purely
+additive.
+
+### Requirements — action needed when upgrading
+
+* **Infernal >= 1.1.4 must be installed** (`conda install -c bioconda infernal`).
+  It is only checked when Rfam annotation will actually run.
+* **The v1.11.0 database is required**, which adds the pressed Rfam covariance
+  models. Re-run `pharokka install` to update.
+
+Because ncRNA annotation is on by default, running v1.11.0 against a v1.10.x or
+older database will fail with an explanatory error. Either update the database
+or pass `--skip_rfam`, which restores the previous behaviour exactly.
+
+### Other changes
+
+* Fixed the database tarball filename being hardcoded to `v1.8.0` rather than
+  derived from the database version.
+* The PHROGs database version marker file is now derived from the database
+  version instead of being hardcoded.
+
 ## v1.10.0 — CLI Redesign & Polars Refactor
 
 ### New subcommand-based CLI

docs/citation.md

@@ -24,6 +24,11 @@ With the following full citations for the constituent tools below where relevant
 * Larralde M. and Camargo A., (2023) Pyrodigal-gv: A Pyrodigal extension to predict genes in giant viruses and viruses with alternative genetic code. https://github.com/althonos/pyrodigal-gv.
 * Shimoyama, Y. (2022). pyCirclize: Circular visualization in Python [Computer software]. https://github.com/moshi4/pyCirclize
 
+If you use `--rfam` to annotate non-coding RNAs, please additionally cite:
+
+* Nawrocki E.P., Eddy S.R., "Infernal 1.1: 100-fold faster RNA homology searches", Bioinformatics (2013) https://doi.org/10.1093/bioinformatics/btt509.
+* Ontiveros-Palacios N., Cooke E., Nawrocki E.P., Triebel S., Marz M., Rivas E., Griffiths-Jones S., Petrov A.I., Bateman A., Sweeney B., "Rfam 15: RNA families database in 2025", Nucleic Acids Research (2025) https://doi.org/10.1093/nar/gkae1023.
+
 If you also use `phold` (recommended) to improve annotations after `pharokka`, please additionally cite:
 
 * Bouras G., Grigson S.R., Mirdita M., Heinzinger M., Papudeshi B., Mallawaarachchi V., Green R., Kim S.R., Mihalia V., Psaltis A.J., Wormald P-J., Vreugde S., Steinegger M., Edwards R.A. Protein Structure Informed Bacteriophage Genome Annotation with Phold. *Nucleic Acids Research*, Volume 54, Issue 1, 13 January 2026, gkaf1448, https://doi.org/10.1093/nar/gkaf1448

docs/install.md

@@ -71,6 +71,10 @@ pharokka --help
 
 * **Note: v1.8.0 uses a new MMseqs2 PHROG profile database format that is incompatible with the v1.4.0 database. If upgrading from v1.7.x or earlier, you must re-run `pharokka install` to fetch the updated database.**
 
+* **Note: v1.11.0 adds the Rfam 15.1 covariance models used for ncRNA annotation, which make the database download considerably larger. Because ncRNA annotation runs by default, v1.11.0 requires the v1.11.0 database — re-run `pharokka install` when upgrading. If you would rather not download it, `--skip_rfam` restores the previous behaviour and works with an older database.**
+
+* **Note: v1.11.0 also requires [Infernal](http://eddylab.org/infernal/) >= 1.1.4 (`conda install -c bioconda infernal`) unless `--skip_rfam` is used.**
+
 To install the pharokka database to the default directory:
 
 ```bash

docs/output.md

@@ -18,7 +18,30 @@ The main output is a `.gff` GFF3 file suitable for use in downstream pangenomic
 
 * A `.tbl` file, which is a flat-file table suitable for upload to NCBI's BankIt.
 
-* A `_cds_functions.tsv` file, which includes counts of CDSs, tRNAs, CRISPRs and tmRNAs, and functions assigned to CDSs according to the PHROGs database.
+* A `_cds_functions.tsv` file, which includes counts of CDSs, tRNAs, CRISPRs and tmRNAs, and functions assigned to CDSs according to the PHROGs database. If `--rfam` is used, an `ncRNAs` count is also included.
+
+* If `--rfam` is used, a `_ncrna.tsv` file with one row per non-coding RNA detected by Infernal against Rfam. The columns are:
+
+  | Column | Description |
+  |--------|-------------|
+  | `contig` | Contig name |
+  | `locus_tag` | pharokka locus tag, e.g. `PHAGE_ncRNA_1` |
+  | `start`, `stop` | 1-based inclusive coordinates (always `start` < `stop`) |
+  | `strand` | `+` or `-` |
+  | `rfam_acc` | Rfam accession, e.g. `RF00050` |
+  | `rfam_id` | Rfam family name, e.g. `FMN` |
+  | `type` | Rfam family type, e.g. `Cis-reg; riboswitch;` |
+  | `description` | Rfam family description |
+  | `clan` | Rfam clan accession, if the family belongs to one |
+  | `bitscore` | Infernal bit score |
+  | `evalue` | Infernal E-value |
+  | `gc` | GC fraction of the hit |
+  | `trunc` | Whether the hit is truncated at a sequence end |
+  | `mdl_from`, `mdl_to` | Coordinates within the covariance model, which show whether the match is partial |
+
+  ncRNAs also appear as `ncRNA` features in the `.gff` and `.gbk` files.
+
+* If `--rfam` is used, a `_cmscan.tblout` file holding the raw Infernal `cmscan` output.
 
 * A `_length_gc_cds_density.tsv` file, which outputs the phage's length, GC percentage, translation table and CDS coding density.
 

docs/run.md

@@ -145,6 +145,43 @@ As of v1.5.0, you can skip running tRNAscan-SE 2, MinCED and Aragorn using `--sk
 pharokka run -i <fasta file> -o <output folder> -d <path/to/database_dir> -t <threads> --skip_extra_annotations
 ```
 
+## ncRNA annotation with Rfam
+
+As of v1.11.0, `pharokka` annotates non-coding RNAs (riboswitches, ribozymes, regulatory sRNAs, introns, leader elements) by scanning against [Rfam](https://rfam.org) with [Infernal](http://eddylab.org/infernal/). **This runs by default** and requires Infernal >= 1.1.4 and the v1.11.0 database or newer.
+
+To turn it off, use `--skip_rfam`:
+
+```bash
+pharokka run -i <fasta file> -o <output folder> -d <path/to/database_dir> -t <threads> --skip_rfam
+```
+
+It is on by default because it is cheap for a phage genome. Measured on 8 cores against the full Rfam 15.1 database:
+
+| genome | size | 1 thread | 8 threads |
+|---|---|---|---|
+| NC_043029 | 7.6 kb | 3.0 s | 0.7 s |
+| NC_004617 | 42.7 kb | 26.4 s | 4.9 s |
+| SAOMS1 | 140 kb | 75.3 s | 13.5 s |
+| 100 microviruses | 587 kb | 253 s | 63 s |
+
+`--threads` helps here, including on a single genome — `cmscan` divides the covariance model database across threads, so there is always work to parallelise regardless of how many contigs you have.
+
+**In meta mode (`-m`), Rfam is skipped by default.** Runtime scales with assembly size at roughly 2 minutes per Mbp on 8 threads, so a 100 Mbp metagenome would take around 3 hours — fine for a phage isolate, painful for a metagenome. To run it anyway, use `--meta_rfam`:
+
+```bash
+pharokka run -i <fasta file> -o <output folder> -d <path/to/database_dir> -t <threads> -m --meta_rfam
+```
+
+One thing worth knowing:
+
+* **Rfam does not replace tRNAscan-SE, ARAGORN or MinCED.** It is purely additive. Rfam's tRNA (RF00005) and tmRNA (RF00023) models are less sensitive on phage sequence than the specialised tools, so hits to them are discarded by default to avoid duplicate and conflicting annotations. If you want them anyway:
+
+```bash
+pharokka run -i <fasta file> -o <output folder> -d <path/to/database_dir> -t <threads> --rfam_keep_trna
+```
+
+Results are written to `{prefix}_ncrna.tsv`, and as `ncRNA` features in the `.gff` and `.gbk`. See [Output](output.md) for the column descriptions.
+
 There is also support for alternative genetic codes if `pharokka` is run with prodigal as a gene predictor using the `-c` flag. See Prodigal's [documentation](https://github.com/hyattpd/prodigal/wiki/Advice-by-Input-Type#alternate-genetic-codes), along with [Yutin et al. 2021](https://doi.org/10.1038/s41467-022-32979-6) and [Peters et al. 2022](https://doi.org/10.1038/s41467-022-32979-6) for more information:
 
 ```bash

scripts/build_rfam_db.sh

@@ -0,0 +1,59 @@
+#!/bin/bash
+# Builds the Rfam component of the pharokka database.
+#
+# The Rfam release is PINNED - pharokka ships a fixed Rfam version inside the
+# database tarball so that annotations are reproducible across runs and
+# machines.  To move to a new Rfam release you must also bump RFAM_VERSION and
+# the database VERSION in src/pharokka/databases.py, and upload a new tarball.
+#
+# Requires: Infernal (cmpress) on $PATH.
+#
+# Output files, which belong at the top level of the pharokka database dir:
+#   Rfam.cm.i1f  Rfam.cm.i1i  Rfam.cm.i1m  Rfam.cm.i1p
+#   Rfam.clanin
+#   Rfam_metadata.tsv
+#
+# The Rfam.cm flatfile is deleted after pressing: cmscan reads the .i1* files
+# and only uses the flatfile path as a base name, so shipping it would add
+# 329 MB to every user's database for nothing.
+
+set -euo pipefail
+
+RFAM_VERSION="15.1"
+RFAM_FTP="https://ftp.ebi.ac.uk/pub/databases/Rfam/${RFAM_VERSION}"
+OUTDIR="${1:-rfam_db}"
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+mkdir -p "$OUTDIR"
+cd "$OUTDIR"
+
+echo "==> Downloading Rfam ${RFAM_VERSION}"
+curl -sSL -O "${RFAM_FTP}/Rfam.cm.gz"
+curl -sSL -O "${RFAM_FTP}/Rfam.clanin"
+curl -sSL -O "${RFAM_FTP}/database_files/family.txt.gz"
+curl -sSL -O "${RFAM_FTP}/database_files/clan_membership.txt.gz"
+
+echo "==> Decompressing covariance models"
+gunzip -f Rfam.cm.gz
+
+echo "==> Pressing covariance models (this takes a few minutes)"
+cmpress -F Rfam.cm
+
+echo "==> Building metadata table"
+python "${SCRIPT_DIR}/build_rfam_metadata.py" \
+    --family family.txt.gz \
+    --clan-membership clan_membership.txt.gz \
+    --cm Rfam.cm \
+    --out Rfam_metadata.tsv
+
+echo "==> Cleaning up intermediates"
+# Rfam.cm is only needed to build the pressed files and the metadata table.
+# cmscan reads the .i1* files, so the 329 MB flatfile is not shipped.
+rm -f family.txt.gz clan_membership.txt.gz Rfam.cm
+
+echo "==> Done. Files in ${OUTDIR}:"
+ls -la Rfam.cm.i1? Rfam.clanin Rfam_metadata.tsv
+echo
+echo "Copy these into the pharokka database directory before creating the"
+echo "database tarball, then update db_url and md5 in src/pharokka/databases.py."