ARE chunk the sentence, causing the the same token's hidden layer activation value at the same position to differ

[jasonrichdarmawan]

Problem: forwarding chunk of a sentence will result in different activation value

FP32 difference: 0.0003
FP16 difference: 0.4060
BF16 difference: 3.3125

AnyEdit/memit_ARE/compute_z.py

Line 72 in ac8e652

current_target_ids = target_ids[start:end]

Reproducible source code: https://gist.github.com/jasonrichdarmawan/16d718cd75311d478e0cbe056f9fa0b6

Solution: padding

FP32 difference: 0
FP16 difference: 0
BF16 difference: 0

Reproducible source code: https://gist.github.com/jasonrichdarmawan/42212c1c1a3453d1a5c62a9219c73196