Exception: Error: error in the implementation.

[Grant-Fox]

Hello!

I am trying to run the squigualiser reform to get a .paf file to visualize my data. I did get the error right before this to add the @sq to the header which I did and then I ran this CLI squigualiser reform --sig_move_offset 0 --kmer_length 1 -c --bam sq_test_squig.sam -o test.paf.

I received this output/error:
kmer_length: 1 sig_move_offset: 0 input bam: sq_test_squig.sam output format: paf output file: test.paf Info: Default stride: 5 Traceback (most recent call last): File "/PATH/", line 8, in <module> sys.exit(main()) ~~~~^^ File "/PATH/squigualiser/venv3/lib/python3.13/site-packages/src/__init__.py", line 56, in main args.func(args) ~~~~~~~~~^^^^^^ File "/PATH/squigualiser/venv3/lib/python3.13/site-packages/src/reform.py", line 228, in run raise Exception("Error: error in the implementation. Please report the command with minimal reproducible data. Read_id: {}".format(sam_record.query_name)); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Exception: Error: error in the implementation. Please report the command with minimal reproducible data. Read_id: 49174ed0-756e-4be7-a5e4-e00e62ceb5f7

Please let me know if there is anything more I can provide to help.

Best,
Grant

[hiruna72]

Hey @Grant-Fox,

Thanks for reporting this.
Could you please also provide the following?

1. a slow5 file with the particular read - 49174ed0-756e-4be7-a5e4-e00e62ceb5f7
2. the list of commands you ran to generate sq_test_squig.sam file. Please provide the basecaller version as well.

Thank you again!

[Grant-Fox]

Hello @hiruna72,

Slow5, fast5, and blow5 file formats are not supported on github. I am not able to attach them. I did attach a text file that is a copy version of the slow5 file that you requested.

PAU73218_f2d0d057_ad2e3979_0.47_0.txt.tar.gz

Basecaller version is Dorado Basecall Service Software, (C)Oxford Nanopore Technologies plc. Version 7.4.12+0e5e75c49, client-server API version 20.0.0

Below are the commands I used to generate sq_test_squig.sam.

buttery-eel -g ~/Downloads/ont-dorado-server/bin/ --config rna_rp4_130bps_sup.cfg -i ~/Downloads/test/blow5_dir/PAU73218_f2d0d057_ad2e3979_0.47_0.blow5 -o test_squig.sam --port 5558 --use_tcp -x cpu --moves_out

cd ..

cd squigualiser

source venv3/bin/activate

squigualiser reform --sig_move_offset 0 --kmer_length 1 -c --bam ../test/test_squig.sam -o test.paf

echo -e fake_reference'\t'0 > fake_reference.fa.fai

samtools view ../test/test_squig.sam -h -t fake_reference.fa.fai -o sq_test_squig.sam

squigualiser reform --sig_move_offset 0 --kmer_length 1 -c --bam sq_test_squig.sam -o test.paf

[hiruna72]

Hey @Grant-Fox,

Sorry for the delayed response.

Unfortunately, the .gz file you shared didn't work.

Can you create a slow5 file like follows,

slow5tools get PAU73218_f2d0d057_ad2e3979_0.47_0.blow5 49174ed0-756e-4be7-a5e4-e00e62ceb5f7 -o read.slow5

mv read.slow5 read.slow5.txt

and upload the read.slow5.txt

Thank you!

[Grant-Fox]

Hi @hiruna72,

No problem, attached below is the file you requested.

One thing to note is that when I ran slow5tools get PAU73218_f2d0d057_ad2e3979_0.47_0.blow5 49174ed0-756e-4be7-a5e4-e00e62ceb5f7 -o read.slow5.

I got this error:
[slow5_idx_get::ERROR] Read ID '49174ed0-756e-4be7-a5e4-e00e62ceb5f7' was not found. At src/slow5_idx.c:539 [slow5_get::ERROR] Exiting on error. At src/slow5.c:2529

To get the read.slow5 for read 49174ed0-756e-4be7-a5e4-e00e62ceb5f7, I ran slow5tools get *.blow5 49174ed0-756e-4be7-a5e4-e00e62ceb5f7 -o read.slow5

read.slow5.txt

Thank you!

Grant

[hiruna72]

Hey @Grant-Fox,

The file you sent does not have a signal.
I assume that this particular read is a split read. Such split reads do not have a signal record in the slow5 file.
The latest (0.6.3) squigualiser skip such split reads.
Can you verify that you are using the latest squigualiser version?

python3.8 -m venv venv3
source venv3/bin/activate
pip install --upgrade pip
pip install squigualiser
squigualiser --version

[Grant-Fox]

Hello @hiruna72

Thank you for your assistance!

I checked the ran the commands you requested to confirm the version of squigualiser. I am using the lastest version of 0.6.3.

Pip is also up to date with 24.3.1
Along with all requirements for pip install squigualiser are satisfied

Grant

[hiruna72]

Hello @Grant-Fox,

In that case, since the record is not to be found in the slow5 file, can you send me the record in the move table?

cat test_squig.sam | grep 49174ed0-756e-4be7-a5e4-e00e62ceb5f7 > move_record.txt

[Grant-Fox]

@hiruna72 Attached is move_record.txt you requested.
move_record.txt

Grant

[hiruna72]

Thank you very much.

I found the bug. In reform I filter the split reads using sp aux tag in the sam records.
However, this record does not have sp. It has pi (strange!!!).
https://github.com/nanoporetech/dorado/blob/release-v0.9/documentation/SAM.md#split-read-tags

I have updated the dev branch to include this additional filtering condition.

For now either you can use dev branch or filter the split reads as follows,

cat sq_test_squig.sam | grep -v pi:Z: > filtered_input_to_reform.sam

PS:
I will tag @Psy-Fer so he can check when he is back whether this is a bug in buttery-eel or dorado-sever 7.4.12+0e5e75c49.
I just tested with dorado standalone 0.8.3 and can confirm that all the split reads have both sp:i and pi:Z tags.

[Grant-Fox]

Thank you for your help @hiruna72! I am using the dev branch and it alleviated my error. The test.paf file that it generates is empty because it is skipping the split reads. I will need to try out with another read to see if I get the same results.

Grant

[hiruna72]

Hi @Grant-Fox,

Usually, only the the split reads have sp and pi tags. And the number of split reads in a dataset is very small.
Can you try with slow5-dorado basecaller?
https://github.com/hiruna72/slow5-dorado/releases/tag/v0.8.3

[Grant-Fox]

Hi @hiruna72 & @Psy-Fer,

This data was sequenced using the PRO-FLO004 (new direct RNA kit). Could this possible be the issue where when using buttereel, I would be using the incorrect config file?

Grant

[Psy-Fer]

Hey

Nah the issue is I don't add the sp tag to the Sam files because when I wrote that part of the software, that flag didn't exist. Only the pi tag was used for the parent ID.

Anyways, if someone could make an issue on buttery-eel and link this issue I will get to it when I get back from my break.

Cheers,
James

[Psy-Fer]

hey @hiruna72 maybe you can help me out

looking to calculate this sp tag

here is some raw data I get out of the basecaller for a split read

split read 1

channel: 8
daq_offset: -244.0
daq_scaling: 0.1870698481798172
duration: 10050
is_duplex_parent: False
mean_qscore: 12.38630485534668
med_abs_dev: 0.0079876109957695
median: -743.1718139648438
modbase_model_version_id: 
model_stride: 6
model_version_id: dna_r10.4.1_e8.2_400bps_fast@v4.3.0
mux: 2
num_events: 1675
num_minknow_events: 0
read_id: 33de40c6-0c51-4b48-b25f-30e1613dfbc3
run_id: 95efb5e9a97737fb4740d9f8e6163cac19756b7b
sampling_rate: 5000.0
scaling_med_abs_dev: 0.0079876109957695
scaling_median: -743.1718139648438
scaling_version: pa
sequence_length: 613
start_time: 95941863
strand_id: a6888069-8429-49bf-949c-7aba9536d50f
trim_front: 0
trim_rear: 0
trimmed_duration: 10050
trimmed_events: 1675
trimmed_samples: 0

split read 2

channel: 8
daq_offset: -244.0
daq_scaling: 0.1870698481798172
duration: 19589
is_duplex_parent: False
mean_qscore: 10.307790756225586
med_abs_dev: 0.0079876109957695
median: -743.1718139648438
modbase_model_version_id: 
model_stride: 6
model_version_id: dna_r10.4.1_e8.2_400bps_fast@v4.3.0
mux: 2
num_events: 3264
num_minknow_events: 0
read_id: 33de40c6-0c51-4b48-b25f-30e1613dfbc3
run_id: 95efb5e9a97737fb4740d9f8e6163cac19756b7b
sampling_rate: 5000.0
scaling_med_abs_dev: 0.0079876109957695
scaling_median: -743.1718139648438
scaling_version: pa
sequence_length: 631
start_time: 95951919
strand_id: 3b26a7ba-5549-4214-b6fa-0bf2b0ac62df
trim_front: 0
trim_rear: 0
trimmed_duration: 19589
trimmed_events: 3264
trimmed_samples: 0

Can you see what I would need to do to build the sp value?

Cheers,

James