Chain multiple hooks to the same function in a single transaction

[m417z]

Currently, it's assumed that there can only be a single operation per target function in a single transaction. Attaching multiple hooks to the same function doesn't work properly, and removing one hook and setting another doesn't work either.

This change addresses this limitation as following:

• Hook addition operations are done before hook removal operations during the commit. Hook addition relies on the disassembly of the original function as it was seen before the commit started. Hook removal operations may modify the function and invalidate this assumption.
• Consecutive hook addition operations for the same target are now chained.

Note that hook removal operations handle the case of the function changing during the commit. In this case, the hook will be put in bypass mode, resulting in a correct removal of the hook but in a memory leak for the allocated trampoline. This case can be optimized in the future.

[m417z]

This feature was used in production for a large amount of users since December 2025, and had no reported issues or crashes.

I rebased it over the recent ARM64EC changes, made some minor improvements, and also added a demo/test that demonstrates it and makes sure it works correctly.

[RatinCN]

This feature was used in production for a large amount of users since December 2025, and had no reported issues or crashes.

I rebased it over the recent ARM64EC changes, made some minor improvements, and also added a demo/test that demonstrates it and makes sure it works correctly.

Sorry for the late reply. I understand the need for multiple handlers for the same function. However, I think this is better handled at the application or host level by installing a single hook and dispatching to the individual handlers.

This would also give the upper layer explicit control over handler ordering, lifetime, and chaining semantics—for example, allowing a handler to modify arguments, skip the next handler, modify the return value, or invoke the next handler more than once—without requiring multiple inline hooks on the target function.

I’d prefer to keep SlimDetours focused on the low-level hooking mechanism rather than extend its transaction and trampoline management to support hook chains. Is there a particular constraint in your use case that makes a single-hook dispatcher impractical?

[m417z]

Perhaps my use case is too specific. The idea is that there are different independent DLLs that are provided with an API to hook functions:

BOOL Wh_SetFunctionHook(void* targetFunction, void* hookFunction, void** originalFunction);

Two DLLs can set a hook for the same function. For performance and stability, the engine sets all hooks in a single transaction, so I need this case to be supported.

Is there a particular constraint in your use case that makes a single-hook dispatcher impractical?

I don't think it's possible to support arbitrary function hooks this way. I'll need to know the hooked function calling convention, and to even then, I can't call it without some inline assembly or code generation. Other things will break as well, such as the ability to use _ReturnAddress intrinsic in the hook to check where the function is called from.

In any case, if you think it's out of scope, we can close this PR and I can keep maintaining it in a fork.