Cannot use custom GFF3 file for annotations

[rickymagner]

Hi, I wanted to experiment using SpliceAI a bit and was wondering how I can use an updated version of GENCODE (e.g. v49). I tried passing this directly to the -A flag, but get the error:

ERROR:root:Gene annotation file ../Exome/gencode.v49.annotation.gff3.gz not formatted properly: Error tokenizing data. C error: Expected 1 fields in line 8, saw 9

It looks like the tool hardcodes some assumptions about the header, which break in GENCODE v49 (and maybe earlier versions). Is there any way I can reformat the file to use in this case, or if the tool can extend to handle this case? If there's a dependency on a GFF3 parser maybe updating that version would help?

P.S. I tried using the tool with updated package versions using -A grch38 and hit an numpy error: np.fromstring is being used but was depricated many versions ago, which should be replaced with np.frombuffer. This finally triggered an error in >= 2.3.0 so might be worth replacing this method.

[bw2]

https://github.com/broadinstitute/SpliceAI-lookup/tree/master/annotations
has scripts for this.
See https://github.com/broadinstitute/SpliceAI-lookup/blob/master/annotations/update_SpliceAI_annotation_txt_files.sh

[rickymagner]

Thanks for sharing. It looks like that script works for GTF but not GFF3, as I get an error when trying to run it. (Also, I needed to replace the imports that say bw2_annotation_utils with annotation_utils).

[bw2]

Fixed. Why not download the GTF from gencode?

[rickymagner]

I had GFF3 already downloaded, and felt it was a bit more "future-proof" since it's the more modern format, but I think I'll have to download the GTF to get it to work with this tool.

[rickymagner]

Thanks for fixing the imports. I tried using the hg38 GTF (and removed the hg19 version from the loop) and end up hitting an error in the second part:

++ echo gencode.v49.basic.annotation.gtf.gz
++ sed s/.gtf.gz/.transcript_annotations.json/
+ json_path=gencode.v49.basic.annotation.transcript_annotations.json
+ python3 convert_gtf_to_SpliceAI_annotation_input_format.py -a gencode.v49.basic.annotation.transcript_annotations.json gencode.v49.basic.annotation.gtf.gz
+ tee process_gencode.v49.basic.annotation.log
usage: convert_gtf_to_SpliceAI_annotation_input_format.py [-h]
                                                          -a ANNOTATION_JSON_PATH
                                                          [--gtf-id-field {transcript_id,gene_id}]
                                                          gtf_gz_path
convert_gtf_to_SpliceAI_annotation_input_format.py: error: File not found: gencode.v49.basic.annotation.transcript_annotations.json

real    0m0.146s
user    0m0.121s
sys     0m0.023s

I see the file exists gzipped, but when I tried to rerun removing the .gz (using gunzip) I hit a different error. Any ideas?

[bw2]

You fixed the 1st issue (by unzipping the file).
What other error are you seeing?

[rickymagner]

It's telling me it expects to see a gzipped file:

Traceback (most recent call last):
  File "/Users/ricky/Repositories/SpliceAI-lookup/annotations/convert_gtf_to_SpliceAI_annotation_input_format.py", line 105, in <module>
    main()
    ~~~~^^
  File "/Users/ricky/Repositories/SpliceAI-lookup/annotations/convert_gtf_to_SpliceAI_annotation_input_format.py", line 31, in main
    transcript_annotations = json.load(f)
  File "/Users/ricky/.local/share/uv/python/cpython-3.14.0-macos-aarch64-none/lib/python3.14/json/__init__.py", line 293, in load
    return loads(fp.read(),
                 ~~~~~~~^^
  File "/Users/ricky/.local/share/uv/python/cpython-3.14.0-macos-aarch64-none/lib/python3.14/gzip.py", line 349, in read
    return self._buffer.read(size)
           ~~~~~~~~~~~~~~~~~^^^^^^
  File "/Users/ricky/.local/share/uv/python/cpython-3.14.0-macos-aarch64-none/lib/python3.14/compression/_common/_streams.py", line 118, in readall
    while data := self.read(sys.maxsize):
                  ~~~~~~~~~^^^^^^^^^^^^^
  File "/Users/ricky/.local/share/uv/python/cpython-3.14.0-macos-aarch64-none/lib/python3.14/gzip.py", line 570, in read
    if not self._read_gzip_header():
           ~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Users/ricky/.local/share/uv/python/cpython-3.14.0-macos-aarch64-none/lib/python3.14/gzip.py", line 539, in _read_gzip_header
    last_mtime = _read_gzip_header(self._fp)
  File "/Users/ricky/.local/share/uv/python/cpython-3.14.0-macos-aarch64-none/lib/python3.14/gzip.py", line 499, in _read_gzip_header
    raise BadGzipFile('Not a gzipped file (%r)' % magic)
gzip.BadGzipFile: Not a gzipped file (b'{\n')

Maybe the code is trying to handle both cases but mixing them up?