Test out some more recent positional embedding techniques, like RoPE.

[eihli]

I'm just curious to see the difference and to learn more about different positional embedding techniques. But this is low priority.