test data with Forward and reverse reads not paired. Check that BAM files have the same read names and are sorted.

[h20gg702]

`HiC-Pro -i /project/HIC_CRC/test_data/test_data
-o /project/HIC_CRC/test_data/hicpro_latest_test
-c /project/HIC_CRC/test_data/config_test_latest.txt

/project/HIC_CRC/test_data/hicpro_latest_test folder already exists. Do you want to overwrite it ? (y/n) [n] :
y

Run HiC-Pro 3.1.0

Sun Apr 13 21:53:05 PDT 2025
Bowtie2 alignment step1 ...
Logs: logs/dixon_2M_2/mapping_step1.log
Logs: logs/dixon_2M/mapping_step1.log

---

Sun Apr 13 21:53:34 PDT 2025
Bowtie2 alignment step2 ...
Logs: logs/dixon_2M_2/mapping_step2.log
Logs: logs/dixon_2M/mapping_step2.log

---

Sun Apr 13 21:53:45 PDT 2025
Combine R1/R2 alignment files ...
Logs: logs/dixon_2M_2/mapping_combine.log
Logs: logs/dixon_2M/mapping_combine.log

---

Sun Apr 13 21:53:49 PDT 2025
Mapping statistics for R1 and R2 tags ...
Logs: logs/dixon_2M_2/mapping_stats.log
Logs: logs/dixon_2M/mapping_stats.log

---

Sun Apr 13 21:53:50 PDT 2025
Pairing of R1 and R2 tags ...
Logs: logs/dixon_2M_2/mergeSAM.log
make: *** [scripts/Makefile:144: bowtie_pairing] Error 1
`
output of mergeSAM.log is below

`python scripts/mergeSAM.py -q 0 -t -v
-f bowtie_results/bwt2/dixon_2M_2/SRR400264_01_R1_GRCh38.primary_assembly.genome_1_22_XYM.bwt2merged.bam
-r bowtie_results/bwt2/dixon_2M_2/SRR400264_01_R2_GRCh38.primary_assembly.genome_1_22_XYM.bwt2merged.bam
-o bowtie_results/bwt2/dixon_2M_2/SRR400264_01_GRCh38.primary_assembly.genome_1_22_XYM.bwt2pairs.bam

[E::idx_find_and_load] Could not retrieve index file for '...R1_....bwt2merged.bam'
[E::idx_find_and_load] Could not retrieve index file for '...R2_....bwt2merged.bam'

mergeBAM.py

forward= ...

reverse= ...

output= ...

min mapq= 0

report_single= False

report_multi= False

verbose= True

Merging forward and reverse tags ...

Forward and reverse reads not paired. Check that BAM files have the same read names and are sorted.
`
Thanks in advance.

[h20gg702]

I verified that three types of BAM files are generated in the pipeline:
Here is how I checked the issue:

samtools view bowtie_results/bwt2/dixon_2M/SRR400264_00_R1_*.bwt2merged.bam | cut -f1 > R1.txt
samtools view bowtie_results/bwt2/dixon_2M/SRR400264_00_R2_*.bwt2merged.bam | cut -f1 > R2.txt
diff R1.txt R2.txt | head

Output:

50066d50065
< SRR400264.50066
64386d64384
< SRR400264.64386
92478a92477
> SRR400264.92479
184109a184109
> SRR400264.184111
192700a192701
> SRR400264.192703

Then I compared the read names at the global and local alignment levels:

samtools view bowtie_results/bwt2_global/dixon_2M/SRR400264_00_R1_*.bam | cut -f1 > global_R1.txt
samtools view bowtie_results/bwt2_global/dixon_2M/SRR400264_00_R2_*.bam | cut -f1 > global_R2.txt
samtools view bowtie_results/bwt2_local/dixon_2M/SRR400264_00_R1_*.bam | cut -f1 > local_R1.txt
samtools view bowtie_results/bwt2_local/dixon_2M/SRR400264_00_R2_*.bam | cut -f1 > local_R2.txt

Comparison results:

Global alignment:

0a1
> SRR400264.11
2a4,6
> SRR400264.29
> SRR400264.30
> SRR400264.31
3a8
> SRR400264.34
4a10,11
> SRR400264.46

Local alignment:

11d10
< SRR400264.11
27,29d25
< SRR400264.29
< SRR400264.30
< SRR400264.31
31d26
< SRR400264.34
42,43d36
< SRR400264.46

This suggests that reads like SRR400264.11, .29, .30, etc. are present in R2 only in the global BAM and in R1 only in the local BAM. After merging (samtools merge -n), these reads exist in both R1 and R2 BAMs.

The merged BAM files were sorted by read name using samtools sort -n, but mergeSAM.py still fails to pair the reads. Do you have any idea why this might be happening?

Thanks in adavance.

[Qimengling]

I also encountered the same problem, have you solved it ？

[XIXIKA-123]

Actually, you should sort the sam file by name. eg: samtools sort -n file.bam. And there is no need to index the sorted file, then , you can run the bowtie_pairing.sh to get the result file.

[l1y1y]

Actually, you should sort the sam file by name. eg: samtools sort -n file.bam. And there is no need to index the sorted file, then , you can run the bowtie_pairing.sh to get the result file.

HiC-Pro has built‑in code for sorting, right? Why does the error still occur? I also checked the source code and found this part:

Sort merge file. In theory, should be perform by "merge -n", but do not work in some cases ... depending on read name ?

cmd="${SAMTOOLS_PATH}/samtools sort -@ ${N_CPU} -m ${SORT_RAM}M -n -T ${TMP_DIR}/$tmp_prefix -o ${BOWTIE2_FINAL_OUTPUT_DIR}/${prefix}.bwt2merged.sorted.bam ${BOWTIE2_FINAL_OUTPUT_DIR}/${prefix}.bwt2merged.bam"
exec_cmd $cmd 2>&1

Sort merge file. In theory, should be perform by "merge -n", but do not work in some cases ... depending on read name ?

cmd="${SAMTOOLS_PATH}/samtools sort -@ ${N_CPU} -m ${SORT_RAM}M -n -T ${TMP_DIR}/$tmp_prefix -o ${BOWTIE2_FINAL_OUTPUT_DIR}/${prefix}.bwt2merged.sorted.bam ${BOWTIE2_FINAL_OUTPUT_DIR}/${prefix}.bwt2merged.bam"
exec_cmd $cmd 2>&1
Why does the final error still appear even though this sorting step is included?