Rebase b9840#176

Merged

gianni-cor merged 330 commits into

tetherto:temp-9840from

makaveli10:qvac-b9840

Jul 17, 2026

This pull request is big! We're only showing the most recent 250 commits

Commits on Jul 1, 2026

Vulkan: add support for fp32 OUT_PROD op

Italo Nicola
authored and
Guilherme Gallo
committed
metal: port OUT_PROD, SILU_BACK, SOFT_MAX_BACK, RMS_NORM_BACK ops to split architecture

authored and
Guilherme Gallo
committed
Initialize sin_sign in rope kargs to fix broken positional encoding.

zoq
authored and
Guilherme Gallo
committed
Fix Windows build by using path::string() for wchar_t conversion

zoq
authored and
Guilherme Gallo
committed
Fix format specifiers for int64_t portability.

zoq
authored and
Guilherme Gallo
committed
Add missing resume_from_batch arg to llama_opt_epoch call.

zoq
authored and
Guilherme Gallo
committed
Fix TQ2_0 dequantization.

authored and
Guilherme Gallo
committed
Fix slow ReBAR reads on discrete GPUs and relax contiguity checks for backward pass.

zoq
authored and
Guilherme Gallo
committed
Use VMA random-access host alloc, skip n_ctx padding and host-buft override during training.

authored and
Guilherme Gallo
committed
Fix loss calculation and TQ2_0 dequantization.

authored and
Guilherme Gallo
committed
ggml-vulkan: workaround for Adreno MUL_MAT Q6_K

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-vulkan: workaround for Adreno MUL_MAT TQ1

Italo Nicola
authored and
Guilherme Gallo
committed
vulkan: ensure host coherent memory on UMA devices

makaveli10
authored and
Guilherme Gallo
committed
ggml-vulkan: fix GGML_VULKAN_CHECK_RESULTS

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-vulkan: skip CROSS_ENTROPY_LOSS_MASKED for check_results

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-vulkan: skip COUNT_EQUAL_MASKED for check_results

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-vulkan: improve OUT_PROD Q4 performance

Italo Nicola
authored and
Guilherme Gallo
committed
Fix LLAMA_LORA_TARGET_ALL bitmask

makaveli10
authored and
Guilherme Gallo
committed
Preserve C API compatibility for llama_opt_epoch

makaveli10
authored and
Guilherme Gallo
committed
lora: enhance LoRA init safety and simplify caller

makaveli10
authored and
Guilherme Gallo
committed
training: add llama_opt_default_params and use it in examples

makaveli10
authored and
Guilherme Gallo
committed
training: add reproducible seed, improve safety and style in LoRA training

makaveli10
authored and
Guilherme Gallo
committed
training: refactor LoRA tensor init to use exceptions

makaveli10
authored and
Guilherme Gallo
committed
ggml-opt: refactor batch memory copying to use lambda

makaveli10
authored and
Guilherme Gallo
committed
fix: typo in ggml.c & README

makaveli10
authored and
Guilherme Gallo
committed
training: document masking constraints and fix metadata extension

makaveli10
authored and
Guilherme Gallo
committed
tests: add ops tests for cross_entropy_loss_masked

makaveli10
authored and
Guilherme Gallo
committed
Add bounds check for --chat-template argument parsing & remove stray backslash

makaveli10
authored and
Guilherme Gallo
committed
training: add TODO for refactoring CLI argument parsing

makaveli10
authored and
Guilherme Gallo
committed
training: add a comment about dropout not being used yet

makaveli10
authored and
Guilherme Gallo
committed
training: add static_assert to catch llama_layer padding issues

makaveli10
authored and
Guilherme Gallo
committed
graph: restore ggml_view_4d for non-contiguous Q tensor support

makaveli10
authored and
Guilherme Gallo
committed
ggml-vulkan: Add buffer sync to cross_entropy_loss_masked_back op

afrantzis
authored and
Guilherme Gallo
committed
ggml-vulkan: add support for tiling as a workaround for memory issues

authored and
Guilherme Gallo
committed
ggml: Wrap tensor and make it contiguous.

zoq
authored and
Guilherme Gallo
committed
Add comment on graph_max_nodes bump for LoRA finetuning.

zoq
authored and
Guilherme Gallo
committed
fix: resume_from_batch=0 incorrectly treated as no-resume in opt_epoch

gianni-cor
authored and
Guilherme Gallo
committed
Fix memory leak in optimizer state loading.

zoq
authored and
Guilherme Gallo
committed
Disable command-buffer concurrency by default on iOS.

zoq
authored and
Guilherme Gallo
committed
Override and default to n_cb=2 on iOS.

zoq
authored and
Guilherme Gallo
committed
fix: restore context state for inference after training cleanup

gianni-cor
authored and
Guilherme Gallo
committed
Add @autoreleasepool to encode_async block to prevent ObjC object accumulation on GCD worker threads.

zoq
authored and
Guilherme Gallo
committed
fix: keep output tensor on CPU for iOS to avoid Metal buffer limits

gianni-cor
authored and
Guilherme Gallo
committed
Remove unused variable 'tensor_name'.

zoq
authored and
Guilherme Gallo
committed
training: fix LLAMA_LORA_TARGET_ALL for ISO C compliance

makaveli10
authored and
Guilherme Gallo
committed
ci: disable native CPU optimizations for x64-cpu-low-perf builds

authored and
Guilherme Gallo
committed
Add resume-from-checkpoint support to Metal LoRA fine-tuning

makaveli10
authored and
Guilherme Gallo
committed
Fix missing parameters for llama_swift_finetune_options

Italo Nicola
authored and
Guilherme Gallo
committed
tests: disable TQ2_0 tests in test-backend-ops due to llvmpipe bug

authored and
Guilherme Gallo
committed
resume patch removed (#103)

dev-nid
authored and
Guilherme Gallo
committed
Update Readme (#104)

gianni-cor
authored and
Guilherme Gallo
committed
Check individual leafs to fix parallel GPU fine-tuning crash.

zoq
authored and
Guilherme Gallo
committed
Fix type mismatch errors in Metal backend.

zoq
authored and
Guilherme Gallo
committed
Fix Metal shader runtime errors by removing superseded individual unary kernels.

zoq
authored and
Guilherme Gallo
committed
Use CMAKE_CURRENT_SOURCE_DIR to fix include/ resolution when built as an Android subdirectory.

zoq
authored and
Guilherme Gallo
committed
Fix duplicate symbol link error on RISC-V by removing tq1_0/tq2_0.

zoq
authored and
Guilherme Gallo
committed
Fix common include paths to use CMAKE_CURRENT_SOURCE_DIR so common/ and vendor/ resolve correctly.

zoq
authored and
Guilherme Gallo
committed
Add OUT_PROD support to CPU repack backend and fix LoRA buffer allocation to use plain CPU buffers for F32/F16 training tensors.

zoq
authored and
Guilherme Gallo
committed
Added a build option that will only build and install the dependency ggml.

authored and
Guilherme Gallo
committed
Reduce VMA block size on integrated GPUs and release empty memory blocks on cleanup to prevent OOM.

zoq
authored and
Guilherme Gallo
committed
ggml-opencl: Add on-disk kernel binary cache for faster startup.

authored and
Guilherme Gallo
committed
lora: Fix SFT loss mask to exclude role tokens from supervised sequence

makaveli10
authored and
Guilherme Gallo
committed
Use tiled OUT_PROD shaders on AMD RADV embedded GPUs.

authored and
Guilherme Gallo
committed
ggml-vulkan: Fix NaN outputs on Mali

makaveli10
authored and
Guilherme Gallo
committed
ggml-vulkan: apply F16 coopmat workaround to Qualcomm Adreno GPUs

makaveli10
authored and
Guilherme Gallo
committed
Remove broken VMA defragmentation from ggml_vk_cleanup.

zoq
authored and
Guilherme Gallo
committed
Add load/unload test case.

zoq
authored and
Guilherme Gallo
committed
log: add opencl memory debug messages

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-opencl: remove Q4_K and Q4_1 from GPU supports_op

authored and
Guilherme Gallo
committed
ggml-opencl: remove Q4_0, Q4_1, Q6_K from L4_LM dispatch

authored and
Guilherme Gallo
committed
Convert bf16 projector weights to f16 on load when the GPU backend lacks bfloat support.

authored and
Guilherme Gallo
committed
Probe simdgroup intrinsic support at init and skip Metal devices that lack it.

zoq
authored and
Guilherme Gallo
committed
metal: Fall back to a temporary shared buffer.

zoq
authored and
Guilherme Gallo
committed
vulkan: fix F16 coopmat1 corruption on ARM Mali/Adreno by using non-coopmat fp32 shaders

makaveli10
authored and
Guilherme Gallo
committed
Mark read_alignment/has_direct_io as override to silence clang -Werror,-Winconsistent-missing-override on macOS.

zoq
authored and
Guilherme Gallo
committed
Route llama_file_disk::read_aligned_chunk through pimpl->read_raw on Windows; the Win32 impl has no aligned-chunk reader.

zoq
authored and
Guilherme Gallo
committed
Fix Metal soft_max_back/rms_norm_back precision by dropping the nth=std::min(nth, ne00[/4]) clamps.

zoq
authored and
Guilherme Gallo
committed
Fix mtmd PRIVATE_HEADER path so cmake --install can find debug/mtmd-debug.h from any source dir.

zoq
authored and
Guilherme Gallo
committed
ci fix: Include scripts pkgs into requirements-all
Guilherme Gallo
committed
common: Split jinja headers, add aux functions

zoq
authored and
Guilherme Gallo
committed
vulkan: Force f32 src1 through the strided cpy path which uses correct strides

makaveli10
authored and
Guilherme Gallo
committed
Keep TQ1_0/TQ2_0 on F32-src1 matmul path on Mali coopmat1.

zoq
authored and
Guilherme Gallo
committed
ggml-metal: Guard against NULL return from ggml_metal_buffer_init / ggml_metal_buffer_map in the backend buffer-type allocators, so iOS allocation failures surface as clean error returns instead of…

zoq
authored and
Guilherme Gallo
committed
ggml-backend: assert hash_set.used/keys are non-NULL on entry to sched_reserve, sched_split_graph and hash_set_reset, so the next iOS Gemma 4 mtmd crash points to the exact lifecycle violation inst…

zoq
authored and
Guilherme Gallo
committed
ggml-metal: cache ggml_metal_device_get by index so each device's embedded Metal library is compiled exactly once.

zoq
authored and
Guilherme Gallo
committed
ggml-metal: gate the simdgroup_mm probe behind MTLGPUFamilyApple1 so it only fires on Apple GPUs, Intel/AMD Mac GPUs cannot compile simdgroup_half8x8 and the pipeline compiler's GGML_LOG_ERROR() on…

zoq
authored and
Guilherme Gallo
committed
ggml-backend: log the offending tensor's name, op and buffer/data/view_src state before the GGML_ASSERTs in ggml_backend_tensor_alloc, so an iOS Heavy9-Gemma4 abort in the CLIP image-encode allocat…

zoq
authored and
Guilherme Gallo
committed
mtmd-clip: drop the Gemma 4 set_warmup_n_tokens(256) override

zoq
authored and
Guilherme Gallo
committed
Revert "fix: keep output tensor on CPU for iOS to avoid Metal buffer limits"

zoq
authored and
Guilherme Gallo
committed
ggml-metal: drop the residency-set keep-alive default from 3 minutes to 1 second so Metal buffers can be reclaimed under iOS memory pressure.

zoq
authored and
Guilherme Gallo
committed
ggml-vulkan, llama, common, mtmd: SSM_CONV dispatch fix

zoq
authored and
Guilherme Gallo
committed
ggml-vulkan: align host buf alloc to minMemoryMapAlignment
Guilherme Gallo
committed
ggml-vulkan: disable ROPE+VIEW+SET_ROWS fusion on Mali/Adreno coopmat1

zoq
authored and
Guilherme Gallo
committed
ggml-vulkan: route TQ1_0/TQ2_0 MUL_MAT to CPU on Mali/Adreno coopmat1

zoq
authored and
Guilherme Gallo
committed
common: declare common_chat_msg_diff_to_json_oaicompat in chat.h

zoq
authored and
Guilherme Gallo
committed
llama : skip gemma4 sliding_window_pattern read on vocab-only ggufs

zoq
authored and
Guilherme Gallo
committed
ggml-vulkan: Add debug log for coopmat-v1 flags

korcan-h
authored and
Guilherme Gallo
committed
ggml-vulkan: fix VkCooperativeMatrixPropertiesKHR list selection

korcan-h
authored and
Guilherme Gallo
committed
ggml-vulkan: workaround-fix for coopmat properties with large shape sizes

korcan-h
authored and
Guilherme Gallo
committed
ggml-vulkan: add new cm1 multi-mat path for f32 coopmat inputs

korcan-h
authored and
Guilherme Gallo
committed
ggml-vulkan: fix tile sizes for coopmat shape sizes greater than 16 (per axis)

korcan-h
authored and
Guilherme Gallo
committed
ggml-vulkan: remove Mali/Andreno specific workaround fixes for coopmats

korcan-h
authored and
Guilherme Gallo
committed
ggml-opencl: restore Adreno kernels lost in b9341 rebase

zoq
authored and
Guilherme Gallo
committed
ggml-metal: drain residency set at teardown instead of asserting.

zoq
authored and
Guilherme Gallo
committed
ci: disable native CPU optimizations for low-perf builds

jesusmb1995
authored and
Guilherme Gallo
committed
ggml-opencl: support quantized KV cache rows

authored and
Guilherme Gallo
committed
llama : exclude QWEN3NEXT/QWEN35/QWEN35MOE from SPLIT_MODE_TENSOR

zoq
authored and
Guilherme Gallo
committed
hip: allow-list gated_linear_attn_f32<64> VGPRs
Guilherme Gallo
committed
ui: Uprev playwright to 1.60 and add timeouts
Guilherme Gallo
committed
tests: download test model with retry + backoff

Guilherme Gallo
and
claude
committed
ggml-cpu: Add portable CPU prefetch hint macro

jesusmb1995
authored and
Guilherme Gallo
committed
ggml,ggml-cpu: TurboQuant core/cpu api changes

jesusmb1995
authored and
Guilherme Gallo
committed
ggml-vulkan: TurboQuant vulkan backend and shader integration

jesusmb1995
authored and
Guilherme Gallo
committed
cmake: install convert_hf_to_gguf.py alongside vulkan_profiling_analyzer.py

zoq
authored and
Guilherme Gallo
committed
feat: support M-RoPE K-shift (#146)

gianni-cor
authored and
Guilherme Gallo
committed
ggml-vulkan,common,cmake: fix TurboQuant copy_to_quant codebook swap, restore reasoning_budget, fix llama install include path

zoq
authored and
Guilherme Gallo
committed
ggml-opencl: serialize lazy device-context and kernel-program init to fix a multi-instance clBuildProgram race that corrupts the Adreno driver heap.

zoq
authored and
Guilherme Gallo
committed
docs: update lora finetuning documentation.

zoq
authored and
Guilherme Gallo
committed
Eval scripts for longbench, ZeroSCROLLS, L-Eval, NIAH added

dev-nid
authored and
Guilherme Gallo
committed
NIAH eval removed

dev-nid
authored and
Guilherme Gallo
committed
Code refactored, tests: extract shared KV-cache eval helpers into kv_cache_{eval,bench}_common

dev-nid
authored and
Guilherme Gallo
committed
tests: add gkamradt-style NIAH KV-cache eval; fix ARG_MAX in _kv_infer_completion for >16k ctx

dev-nid
authored and
Guilherme Gallo
committed
common: fix use-after-free when adopting an externally-loaded model

jpgaribotti
authored and
Guilherme Gallo
committed
common: fix common_init_from_model_and_params returns nullptr on load failure

jpgaribotti
authored and
Guilherme Gallo
committed
model-load: register split tensors in tensors_by_name during incremental load

jpgaribotti
authored and
Guilherme Gallo
committed
opencl : add CONV_2D_DW, POOL_2D, HARDSWISH, HARDSIGMOID kernels

olyasir
authored and
Guilherme Gallo
committed
kv-cache : cover GGSQ M-RoPE ext restore

authored and
Guilherme Gallo
committed
ggml-vulkan: remove TQ MUL_MAT -> CPU fallback on Mali/Adreno coopmat1

makaveli10
authored and
Guilherme Gallo
committed
metal : add CONV_2D_DW (depthwise conv2d) kernel

olyasir
authored and
Guilherme Gallo
committed
metal : trim redundant kargs comments in conv_2d_dw dispatch

olyasir
authored and
Guilherme Gallo
committed
metal : conv_2d_dw f32-only weights + enable CWHN

olyasir
authored and
Guilherme Gallo
committed
test : extend test_conv_2d_dw coverage (5x5, dilation, batch)

olyasir
authored and
Guilherme Gallo
committed
ggml: add GGML_OP_SIGMOID_BACK op

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-cpu: implement sigmoid_back

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-vulkan: implement sigmoid_back

Italo Nicola
authored and
Guilherme Gallo
committed
test-backend-ops: add sigmoid_back test

Italo Nicola
authored and
Guilherme Gallo
committed
ggml: Add stop-gradient backward for SSM, L2_NORM, GATED_DELTA_NET ops

makaveli10
authored and
Guilherme Gallo
committed
ggml: Add concat cpu backward pass

makaveli10
authored and
Guilherme Gallo
committed
ggml: Add TANH backward pass for cpu

makaveli10
authored and
Guilherme Gallo
committed
ggml: add GELU backward

makaveli10
authored and
Guilherme Gallo
committed
ggml: add MUL_MAT_ID backward (back_a, back_b)

makaveli10
authored and
Guilherme Gallo
committed
ggml: generalize get_rows_back to N-D tensors

makaveli10
authored and
Guilherme Gallo
committed
ggml-vulkan: implement gelu_back

Italo Nicola
authored and
Guilherme Gallo
committed
test-backend-ops: add gelu_back test

Italo Nicola
authored and
Guilherme Gallo
committed
ggml: add L2_NORM_BACK op with CPU implementation

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-vulkan: implement L2_NORM_BACK

Italo Nicola
authored and
Guilherme Gallo
committed
ggml: stop using stop-grad for L2_NORM_BACK

Italo Nicola
authored and
Guilherme Gallo
committed
test-backend-ops: add L2_NORM_BACK test

Italo Nicola
authored and
Guilherme Gallo
committed
tests: fix crash for GET_ROWS_BACK

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-vulkan: implement MUL_MAT_ID_BACK_A

Italo Nicola
authored and
Guilherme Gallo
committed
test-backend-ops: add MUL_MAT_ID_BACK_A test

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-vulkan: implement MUL_MAT_ID_BACK_B

Italo Nicola
authored and
Guilherme Gallo
committed
test-backend-ops: add MUL_MAT_ID_BACK_B test

Italo Nicola
authored and
Guilherme Gallo
committed
test-backend-ops: add MUL_MAT_ID_BACK perf cases

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-rpc: account for new backward ops (GGML_OP_COUNT 100->105), bump proto patch version

zoq
authored and
Guilherme Gallo
committed
ggml-metal: implement gelu_back, l2_norm_back, and mul_mat_id_back_a/b backward ops, cont the rms_norm_back gradient for QK-norm graphs, and log the outdated-gemma4-template warning once.

zoq
authored and
Guilherme Gallo
committed
gml-vulkan: implement GATED_DELTA_NET_B.

zoq
authored and
Guilherme Gallo
committed
ggml: add SSM_CONV_BACK_SX and SSM_CONV_BACK_C ops

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-cpu: implement SSM_CONV_BACK_SX and SSM_CONV_BACK_C

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-vulkan: implement SSM_CONV_BACK_SX and SSM_CONV_BACK_C

Italo Nicola
authored and
Guilherme Gallo
committed
test-backend-ops: add SSM_CONV_BACK tests and enable ssm_conv grad check

Italo Nicola
authored and
Guilherme Gallo
committed
ggml-metal: implement ssm_conv_back_sx/_c and gated_delta_net_back backward ops.

authored and
Guilherme Gallo
committed
feat(mtmd/qwen3vl): multi-tile batching with --image-tile-mode

authored and
Guilherme Gallo
committed
feat(mtmd/qwen3vl): global overview thumbnail for multi-tile encoding

authored and
Guilherme Gallo
committed
llama: add sequence token count memory API

authored and
Guilherme Gallo
committed
llama: add seq_token_count override for the DSA KV cache

zoq
authored and
Guilherme Gallo
committed
Document TurboQuant benchmarks

authored and
makaveli10
committed
Link TurboQuant test coverage

authored and
makaveli10
committed
Mention TurboQuant validation scripts

authored and
makaveli10
committed
Update TurboQuant eval script links

authored and
makaveli10
committed
Add PQ rows to Qwen benchmark highlights

authored and
makaveli10
committed
Use LongBench in TurboQuant quality summary

authored and
makaveli10
committed
Add Qwen3.5 f16 benchmark baseline

authored and
makaveli10
committed
Add Qwen3.5 8k f16 benchmark baseline

authored and
makaveli10
committed
Avoid URL substring assertion in MCP test

authored and
makaveli10
committed
flake8 lint fix

dev-nid
authored and
makaveli10
committed
flak8 noprint lint fix, ty check fix

dev-nid
authored and
makaveli10
committed
ggml: gemma 4 chat template parsing for LoRA finetune
makaveli10
committed
llama-lora-training: MoE expert LoRA & tighten module matcher
makaveli10
committed
llama-graph: training-mode workarounds for MoE backward in build_moe_ffn
makaveli10
committed

Commits on Jul 7, 2026

ggml-metal: implement the sigmoid_back backward op

zoq
authored and
Guilherme Gallo
committed
mtmd: use the M-RoPE position type in mtmd_test_create_input_chunks so test-mtmd-c-api's decoder-pos assertions match the fixture layout

zoq
authored and
Guilherme Gallo
committed

Commits on Jul 8, 2026

Commits on Jul 13, 2026

Commits on Jul 14, 2026

ggml-opencl: build flash-attn kernels without finite-math

olyasir
authored and
makaveli10
committed
ggml-opencl: treat null attention mask as bidirectional, not causal

olyasir
authored and
makaveli10
committed
ggml-opencl: add trailing barrier in f32/f16 flash-attn tile loop + guard upscale zero dims

iancris
authored and
makaveli10
committed
ci : disable OpenMP in wasm webgpu build
Guilherme Gallo
committed

Uh oh!

Commits

Commits on Jul 1, 2026