Reload malloc:// bin info from the live buffer with obR

[atirna]

Your checklist for this pull request

• I've read the guidelines for contributing to this repository.
• I made sure to follow the project's coding style.
• I've documented every RZ_API function and struct this PR changes.
• I've added tests that prove my changes are effective (required for changes to RZ_API).
• I've updated the Rizin book with the relevant information (if needed).
• I've used AI tools to generate fully or partially these code changes and I'm sure the changes are not copyrighted by somebody else.

Detailed description

empty malloc:// lands on the dummy any plugin at open. writing elf/mach0 magic into it used to leave format stuck on any unless you reopened the file, which throws away the buffer.

obR already reloads bin info from the current IO buffer (the help text even mentions malloc://). this PR uses that instead of probing on every wx:

• wx does not change format. scripts that set asm.arch then write code are not surprised by a reload.
• obR re-runs plugin autodetection on the live buffer and does not discard it.
• hex:// with magic already in the uri still detects at open (rz_core_bin_load).
• rizin - loads bin info after stdin is written into malloc://.
• om / rz_core_file_malloc_copy_chunk writes the copied bytes before bin load, so the probe sees the data.
• core_bin_reload keeps the replacement binfile on the core file after the old one is deleted.

reverted an unrelated avr check_buffer tweak that was in an earlier revision of this PR.

AI: Cursor Agent / cursor-grok-4.6-high for this rework. earlier revisions also used Codex / gpt-5.6-terra and Hermes Agent / glm-5.3. i reviewed the diff and checks.

Test plan

• ./build/test/unit/test_core_bin 13/13 (write stays any; obR reloads elf and keeps the IO bytes; mach0 via obR; hex:// elf at open)
• rz-test db/cmd/file_malloc 6/6
• manual: rizin -q malloc://128 then wx 7f45...; i~format stays any; obR; i~format is elf; e io.va=0; p8 4 @ 0 still 7f454c46

Closing issues

closes #6607

[atirna]

done. the elf special case is gone: after a write the re-probe now accepts any plugin that claims the buffer, so mach0/elf64/pe/... all reload bin info, not just elf32.

that broad gate was the reason the elf check existed though, some check_buffer implementations are loose enough to claim shellcode. so instead of narrowing the gate i tightened the claimers that actually misfire:

• avr no longer claims from the reset vector alone (arm ldr bytes decode as avr rjmp too). it now requires a full interrupt vector table, smallest known board has 19 vectors, so foreign shellcode stays on any
• when a plugin claims by magic but fails to load, rz_bin_reload now keeps the current bin file instead of leaving the bin empty

the implicit re-probe also stopped clobbering asm.arch/asm.cpu and seeking to the entrypoint mid-script, only the detected plugin becomes current.

checked: wx cefaedfe070000010300000002000000; i~format gives mach0 now, elf64 works the same way, arm shellcode stays any, and db/cmd/file_malloc + db/formats/avr pass 17/17 with the stricter avr check.

[wargio]

I feel like this should be done via a command to reload the bin & autodetect without discarding the current buffer as autodetecting this may do weird stuff.

[wargio]

@notxvilka @thestr4ng3r @Rot127 please have a look

[atirna]

probing on wx was not needed. dropped that.

obR already reloads bin info from the current buffer without discarding it, so that can be used. wx leaves format as any, obR picks up elf/mach0 and the malloc bytes can stay. hex:// still detects at open because the buffer is already filled.