Add Gemma 4 (26B A4B) prompt-mode handler

[tumbak]

Summary

Adds a prompt-mode handler for Google Gemma 4 26B A4B Instruct
(google/gemma-4-26B-A4B-it), which BFCL does not currently support. The
existing Gemma 3 handler hardcodes the Gemma 3 chat template and is
incompatible with Gemma 4's new turn-token format, so a dedicated handler is
needed.

To my knowledge these are the first BFCL numbers published for Gemma 4. Full
methodology, per-category scores, and reproduction commands:
https://algollabs.com/blog/gemma4-bfcl

What this changes

• New bfcl_eval/model_handler/local_inference/gemma4.py — Gemma4Handler
  (subclasses OSSHandler).
• Modified bfcl_eval/constants/model_config.py — register the model + import.
• Modified bfcl_eval/constants/supported_models.py — add the model id.
• Modified SUPPORTED_MODELS.md — add the table row.

Implementation notes

1. Chat template. _format_prompt delegates to
   tokenizer.apply_chat_template(...) so the correct Gemma 4 template is
   picked up from the HF tokenizer (same approach as QuickTestingOSSHandler),
   rather than hardcoding tokens like the Gemma 3 handler.

2. Native tool-call syntax (decode_ast / decode_execute). With
   --jinja, Gemma 4 emits <|tool_call>call:fn(arg="x")<tool_call|> rather
   than the legacy [fn(arg="x")]. Single-turn cases are coerced by the
   system-prompt preprocessing; in multi-turn the model reverts to native
   syntax and the default parser fails ("Failed to decode the model response").
   A small converter rewrites native → BFCL form before delegating to the
   default decoder, handling five observed emission variants (standard,
   empty-args, bracket-prefix, fullwidth colon, JSON-dict args). Responses
   already in BFCL format and plain-prose refusals pass through untouched.

3. Tool results in multi-turn (_add_execution_results_prompting). The
   Gemma 4 chat template silently drops role="tool" messages (BFCL's
   default), so the model never sees tool feedback and loops on failed
   actions. Switching to role="tool_response", which the template renders,
   is required — multi-turn accuracy floors at ~3% without it.

4. Context-length detection. Gemma 4 is multimodal; its top-level HF config
   has no max_position_embeddings (it lives under text_config), so
   spin_up_local_server falls back to the tokenizer sentinel (~1e18). The
   handler corrects max_context_length to the real value, 262144. See the
   open question below.

Evaluation details

• Prompt mode (is_fc_model=False).
• Sampling used for the published run: temperature=1.0, top_p=0.95, top_k=64
  per Google's Gemma guidance (BFCL default is 0.001).
• Open-weight model, publicly accessible on Hugging Face.

Open question for maintainers

The context-length fix is currently handler-local (corrected in inference()
because spin_up_local_server is decorated @final). The root cause is
generic to multimodal HF configs, so it may belong in base_oss_handler.py
instead — reading text_config.max_position_embeddings when the top-level
attribute is absent. Happy to move it there if you'd prefer the general fix.

🤖 Generated with Claude Code