Annotate pattern regions with custom match-modes

[jakob-schuster]

The user can specify error rate globally with --error-rate, but they can also override it per-region with ~.

# match with a 0.2 error rate
if read matches [_ AGCTAGCTAG~0.2 _] => count!('contains primer')

Currently, the user can also specify match mode globally with --match-mode, but they cannot override it per-region; this would be a useful feature for fine-tuning the matching behaviour where necessary. Even though this would make these patterns more verbose, it would only be for niche scenarios where the user needs to specify fine-grained match semantics.

if read matches [_ GGG::{mode='all'} _ AAA::{mode='first'} _] => count!('match')

TTTTTTTGGGTTTTTTTTGGGTTTTTTAAATTTTTAAATTTTT
a      ^^^.................^^^
b                 ^^^......^^^
c      ^^^.........................^^^
d                 ^^^..............^^^

For this read, only the matches {a, b} are identified; all occurrences of GGG are matched, but only the first occurrence of AAA.

Maybe a more practical example, this could be used to specify matching to the end of a poly(A) tail:

# extract everything from the first occurrence of a primer to the end of a >=16bp poly(A) tail.
if read matches [_ trimmed:(AGCTAGCTAG::{error=0.2, mode='first'} _ AAAAAAAAAAAAAAAA::{error=0, mode='last'}) _ ] =>
  trimmed.out!('out.fq')

By using a struct for these annotations, we would allow for both mode and distance to be specified with GGG::{error=0.2, mode='all'}, and allow for future extensions to what can be annotated in this way. Existing syntax GGG~0.2 could be retained but treated as sugar for GGG::{error=0.2}. The double-colon :: is just a first thought of course; alternative syntax ideas welcome.

[jakob-schuster]

This raises the question of how to resolve scenarios where the order in which each region is matched affects which final matches are chosen.

if read matches [_ GGG::{mode='all'} _ AAA::{mode='first'} _] => count!('match')

TTTTTTTTTTGGGTTAAATTTTTTTGGGTTTTTTTTTTTTTTTTTTAAA
a         ^^^..^^^
b         ^^^.................................^^^
c                        ^^^..................^^^

For a given pattern and read, there is an initial set of all possible matches, here {a,b,c}. Then, the match-mode of each region constrains the initial set, but the order in which these constraints are applied can affect the final subset, resulting in either {a} or {a,c}.

• If the AAA::{mode='first'} region is resolved first, then only match a will be identified. First the first AAA is selected, and then although the mode of the GGG region is all, there is only the first GGG to select.
• If the GGG::{mode='all'} region is resolved first, then the matches identified will be {a, c}. (Note that b is still not identified, because after the first GGG is matched, the first AAA is chosen instead of the second AAA)

This suggests a need for a rank/priority/order field, which is of Num type. Regions are matched in ascending order of rank. When breaking ties, the leftmost region wins (this means, when no ranks are specified by the user, they are all 0 and the current behaviour of matchbox remains unchanged).

With rank, the following two scripts could be used to specify either the first or second option from the example above:

# searches for `AAA` before `GGG`, identifying only match a in the above read
if read matches [_ GGG::{mode='all',rank=1} _ AAA::{mode='first',rank=0}] => count!('match')

# searches for `GGG` before `AAA`, identifying matches {a, c} in the above read
if read matches [_ GGG::{mode='all',rank=0} _ AAA::{mode='first',rank=1}] => count!('match')