Question about reproduce ScenePair text editing results reported in the paper

[ycy918]

Hi, thanks for releasing the code.

I am trying to reproduce the ScenePair text editing results reported in the paper, but I am not sure what the exact intended pipeline is for ScenePair. From the current repository, it seems that the released scripts mainly focus on OmniText-Bench, and I could not find an official reproduction script for the ScenePair benchmark, including preprocessing, inference, conversion to the crop-level format, and evaluation.

Could you please clarify whether there is an official script or set of scripts for reproducing the ScenePair text editing results reported in the paper? If such scripts are not available, I would really appreciate it if you could share the exact preprocessing, inference, and evaluation steps used for ScenePair.

In particular, I would like to ask which ScenePair files should be used for reproduction, such as i_full, i_s_full.txt, i_t.txt, i_t_full.txt, etc. I am also not sure how the full OmniText outputs should be converted into the crop-level format used for evaluation.

Any guidance on the intended ScenePair pipeline would be very helpful. Thank you so much!

[agusgun]

Hi, the main script is similar with editing in OmniText-Bench.

There is additional preprocessing e.g., contraction of the target mask during editing due to the sensitivity of the backbone.
In addition, we perform crop for the big image to focus on the editing part. We have written the details in the paper.
You can see the legacy notebook for rescaling.
char_masks = generate_char_masks(first_attn_mask, len(source_text_list[idx_ann]), len(target_text_list[idx_ann]))

For the file, we use similar setting like TextCtrl.

In summary, we use:

1. Input image: Cropped (512x512) original image
2. Target text: i_t_full
3. Target mask: i_t_full and i_s_full (simple algorithm to contract the mask if the target text has fewer number of characters)

Hope this can help you.