Non-async tools are evaluated in reverse order

[LeonAmp9]

Please update gptel first -- errors are often fixed by the time they're reported.

• I have updated gptel to the latest commit and tested that the issue still exists

Bug Description

First of all, great tool, thank you. Lots of potential. Now the issue.

One would think that non-async tools will be evaluated in the order the LLM requests them. But they are not.

This leads to catastrophic results if the tools mutate something, be it Emacs state or filesystem. For me, it was when I had a tool for "dump particular text into a new emacs buffer with particular name" and a tool for "show a particular buffer to the user" (for presentation of time-consuming tasks). The LLM would correctly sequence the 2 tool calls in one "turn" but the second call would consistently fail with buffer allegedly not existing. I was quite confused because the buffer was indeed created and the arguments used by the LLM were indeed correct. And the second tool would consistently succeed if called in a second conversation "turn".

After making a minimum viable test case (see the test case section), the underlying cause (see the title of this report) was clear.

Is this behavior intended? I thought, maybe it's an intended limitation because the LLM can mix async and sync tools and it would be difficult to guarantee an ordering, so it's just evaluated in a YOLO manner. But I find it weird that the calls are not merely out of order, they are so reliably and in a particular way - always exactly reverse. I couldn't find any mention of this surprising behavior in the manual.

I don't have the elisp skills necessary to understand this codebase to see why this is happening.

Backend

Other (please specify in Additional Context)

Steps to Reproduce

Tools:

(defvar counter 0)

(gptel-make-tool
 :name "tool-x"
 :description "Use this when the user requests tool-x"
 :category "core"
 :async nil
 :function
 (lambda ()
   (setq counter (+ counter 1))
   (message "X %0.6f %d" (float-time) counter)))

(gptel-make-tool
 :name "tool-y"
 :description "Use this when the user requests tool-y."
 :category "core"
 :async nil
 :function
 (lambda ()
   (setq counter (+ counter 1))
   (message "Y %0.6f %d" (float-time) counter)))

(gptel-make-tool
 :name "tool-z"
 :description "Use this when the user requests tool-z."
 :category "core"
 :async nil
 :function
 (lambda ()
   (setq counter (+ counter 1))
   (message "Z %0.6f %d" (float-time) counter)))

LLM chat:

### use tool-x, tool-y and tool-z


``` tool (tool-x nil)
(:name "tool-x" :args nil)

X 1784533301.935472 3
```
``` tool (tool-y nil)
(:name "tool-y" :args nil)

Y 1784533301.935415 2
```
``` tool (tool-z nil)
(:name "tool-z" :args nil)

Z 1784533301.935281 1
```

As can be seen from both the timestamps and the counter variable printout, the tools are requested by the LLM in order X-Y-Z but are evaluated as Z-Y-X.

Additional Context

Emacs: 30.2
OS: NixOS unstable
gptel: commit 8701e2b, per gptel-pkg.el
backend: llama.cpp with qwen3

Backtrace

Log Information

[LeonAmp9]

There is already an issue open for similar problem: #970

But if I understood that one correctly, it's about async tools (because MCP): #970 (comment) This report is about non-async tools.

[karthink]

In general tool calls in a single LLM turn should not be ordered, because the LLM cannot know if they will be evaluated asynchronously. But we can certainly fix the ordering of synchronous tool calls in gptel.

EDIT: Sorry, I see you're using llama.cpp. I'll take a look at how the tool calls are ordered.

[karthink]

Does this happen irrespective of whether the tool calls require confirmation?

This is going to be difficult to fix in the event that some calls require confirmation and some don't.

For example, if tool-y requires user confirmation above but tool-x and tool-z don't, then the following happens (even after fixing the reverse ordering)

1. tool-x doesn't require confirmation, so it runs immediately.
2. tool-y requires confirmation, it doesn't run.
3. tool-z doesn't require confirmation, so it runs immediately.
4. The user sees the prompt for tool-y and approves it.

So the tools run in the order tool-x, tool-z, tool-y.

[LeonAmp9]

Actually, it's even weirder with :confirm t set to all 3.

Before confirmation, its presented as X-Y-Z:

After confirmation (only 1 confirmation was requested), its presented as Z-Y-X (reverse):

Yet it was evaluated as X-Y-Z. It's as if the confirmation reverses it again, thus twice reversed, thus correct order with :confirm t.

[LeonAmp9]

This is going to be difficult to fix in the event that some calls require confirmation and some don't.

For example, if tool-y requires user confirmation above but tool-x and tool-z don't, then the following happens (even after fixing the reverse ordering)

1. tool-x doesn't require confirmation, so it runs immediately.

2. tool-y requires confirmation, it doesn't run.

3. tool-z doesn't require confirmation, so it runs immediately.

4. The user sees the prompt for tool-y and approves it.

So the tools run in the order tool-x, tool-z, tool-y.

I think I understand, just looked at gptel--handle-tool-use. I suppose the user can wrap mutating tools in some sort of async-pretending wrapper that schedules the actual sync lambdas in a custom queue, taking into account the confirms too.

I tried to put into system prompt that the LLM should only use 1 tool at a time but it gets context-trashed over time.

[LeonAmp9]

Made an extremely crude fix. It fixes both the reversal in the original issue text and also the weird double reversal in the case with all 3 having :confirm t. It doesnt fix the case you described (only some having :confirm t), where indeed the non-confirms will run ahead of the confirm.

That's good enough for me. Haven't noticed any regressions (fingers crossed).

0001-fix.patch

BTW I noticed the file contains some stray carriage returns and form-feeds (?).