Repository navigation

Added AoT for Qwen, Phi, Gemma & Whisper - #4

Closed

neuropilot-captain wants to merge 726 commits into

mainfrom

support_qwen_phi_gemma_whisper

Closed

Added AoT for Qwen, Phi, Gemma & Whisper#4
neuropilot-captain wants to merge 726 commits into
mainfrom
support_qwen_phi_gemma_whisper

This pull request is big! We're only showing the most recent 250 commits

Commits on Sep 3, 2025
- Arm backend: Refactor check_platform_support (#13904)
  perheld
  authored
  cac1a71
- NXP backend: Add preprocessing pass to split multilayer `GRU`. (#13757)
  MartinPavella
  authored
  5abad6c
- Add extra logging in CoreML (#13890)
  metascroy
  authored
  0b4de82
- Revert "Use extension module for pybind (#13858)" (#13911)
  lucylq
  authored
  c393d17
- Bump tokenizers to 4ed91cc (#13888)
  GregoryComer
  authored
  537d30b
- Clean up optimized op_le
  manuelcandales
  authored
  8f724e0
- Enforce range setting only used with per-channel (#13889)
  rohansjoshi
  authored
  f0368db
- Add qnn deps to LlamaDemo gradle (#13912)
  kirklandsign
  authored
  33c8f76
- Qualcomm AI Engine Direct - Support Flip & Index_Select (#13906)
  winskuo-quic
  authored
  b2a8550
- [Windows] Run Python unit test CI on Windows (#13716)
  GregoryComer
  authored
  d7fd78b
- Arm backend: Enable capture of debug information during TOSA serialization (#13908)
  tom-arm
  authored
  ad19cb8
- XNNPACK: Kleidi QP8 and SME2 (#13887)
  digantdesai
  authored
  932818c
- Optimize index_out via fast path
  manuelcandales
  authored
  874d3c1
- [Windows] Add model run CI (#13836)
  GregoryComer
  authored
  66799e3
- Fix bug in elu
  manuelcandales
  authored
  18263c2
- Remove EDSR from android benchmarking (#13877)
  kirklandsign
  authored
  17cad55
- Enable per-row/per-col grouping in CoreML LUT ops (#13674)
  metascroy
  authored
  4fc602b
- Add audio to multimodal runner (#13851)
  pytorchbot
  authored
  3f2f7d6
- Add constraints for cadence idma ops.
  hsharma35
  authored
  ae862f7
- Post memory planning passes.
  hsharma35
  authored
  a99b64f
- Add Voxtral runner (#13853)
  pytorchbot
  authored
  f7de2da
- Make token and stat callback optional in multimodal runner (#13854)
  pytorchbot
  authored
  ab44d06
- Support Half/BFloat16 in native_group_norm
  manuelcandales
  authored
  ae07cb6
- Include audio preprocessing for raw audio tensor (#13855)
  pytorchbot
  authored
  624463e
Commits on Sep 4, 2025
- Add 16A8W linear ops support and test (#13922)
  
  pytorchbot
  and
  Ninja91
  authored
  b759ae8
- Static attention batch size > 1
  sxu
  authored
  9cc092c
- Back out "Add XNNPACK backend option for workspace sharing"
  GregoryComer
  authored
  e9e868c
- Force -O3 for executorch op_div.cpp in "clang 17" also
  akrieger
  authored
  aa08df5
- Adjust atol/rtol for ring attention's quantized kv cache test (#13909)
  kimishpatel
  authored
  76a8906
- Revert #13851 #13853 #13854 #13855
  kirklandsign
  authored
  9fa7edf
- let backends set info on the lowered_backend.meta from preprocess
  JacobSzwejbka
  authored
  32e82bc
- Add backend-agnostic implementation for quantized_linear
  DrJessop
  authored
  ea5cf49
- First working llama shared weights flow
  neuropilot-captain
  committed
  486dd4e
- Arm backend: Fix broken logic when checking os in utils.sh (#13939)
  perheld
  authored
  6aac82a
- Update Neutron Converter
  JakeStevens
  authored
  8c79a53
- Arm backend: Move tosa files to tosa/ directory (#13938)
  Sebastian-Larsson
  authored
  f6d90ef
- Summary: Add detailed error message to exception flow (#13929)
  
  psiddh
  and
  Github Executorch
  authored
  f9ce98f
- [EXIR] Register _clone_dim_order op and map aten.clone (#13735)
  
  authored
  c5ff74c
- Add audio to multimodal runner (#13948)
  
  pytorchbot
  and
  jackzhxng
  authored
  39594fa
- Add Voxtral runner (#13949)
  
  pytorchbot
  and
  jackzhxng
  authored
  c1d973e
- Make token and stat callback optional in multimodal runner (#13950)
  
  pytorchbot
  and
  jackzhxng
  authored
  e289f6c
- Fix reduction over dim list for empty input
  manuelcandales
  authored
  497f59c
- Include audio preprocessing for raw audio tensor (#13951)
  
  pytorchbot
  and
  jackzhxng
  authored
  544cece
- Update backend-agnostic quantize_per_tensor to use scale_inv to match the quantize_per_tensor_out implementation
  DrJessop
  authored
  30e7497
- Backend-agnostic implementation of quantized_layer_norm_per_tensor
  DrJessop
  authored
  14d0745
- Scriptify mel spectrogram processor (#13961)
  jackzhxng
  authored
  f4ec01a
- Revert exposed_as_util functionality and refactor stack utils
  manuelcandales
  authored
  19d22ab
- Add Voxtral build and run instructions (#13962)
  jackzhxng
  authored
  646e0c4
- Deprecate method retrieval in module (#13913)
  shoumikhin
  authored
  686bb71
Commits on Sep 5, 2025
- Support qwen 0.5 b (#13972)
  cccclai
  authored
  d5a5164
- Add support for checking more than one output from delegate in numerical comparator
  kimishpatel
  authored
  09a2e88
- Disable xnnpack jobs in macos (#13959)
  metascroy
  authored
  fcdcd8e
- [Executorch][target recipes] Add target based recipes for lowering models to a target device (#13983)
  
  pytorchbot
  and
  abhinaykukkadapu
  authored
  b02db12
- Merge remote-tracking branch 'upstream/main' into extract_share
  neuropilot-captain
  committed
  7cc7321
- Merge branch 'main' into extract_share
  neuropilot-captain
  authored
  b2303b2
- Fix conflict
  neuropilot-captain
  committed
  dbe864d
- Update for delegate interface changes
  neuropilot-captain
  committed
  6491161
- Arm backend: Replace asserts with report_reject in operator_support (#13985)
  Sebastian-Larsson
  authored
  7060fc5
- Summary: Add Initial support to pico2 (Arm Cortex M) (#12518)
  
  psiddh
  and
  sidart
  authored
  cfd9b68
- Arm backend: Auto generate cycle table if ETDump is used (#13943)
  zingo
  authored
  c6a6caa
- Arm backend: Add INT64 support to to_copy (#13993)
  oscarandersson8218
  authored
  cec1400
- Arm backend: Add VGF minimal example to out-of-box testing (#13861)
  SaoirseARM
  authored
  01258d2
- Fix qwen2_5_xnnpack_q8da4w.yaml (#13965)
  jackzhxng
  authored
  0eed262
- model : support LiquidAI LFM2 hybrid family (#13805)
  
  Tarek Dakhran
  and
  jackzhxng
  authored
  8607c89
- Arm backend: Refactor compiler choices in setup.sh (#13995)
  perheld
  authored
  8456567
- Update README.md (#14008)
  jackzhxng
  authored
  f3b902a
- Fix internal tests for LiquidAI LFM2
  jackzhxng
  authored
  4628f89
- Fix crash in op_upsample_bilinear2d_aa
  mergennachin
  authored
  9dfc0d6
- Arm backend: Propagate debug info to TOSA flatbuffer (#13998)
  
  tom-arm
  and
  mansnils
  authored
  3899beb
- Fix qwen2.5 1.5b test (#14013)
  cccclai
  authored
  364f493
- fix facto contraints to avoid char,byte in full
  zonglinpeng
  authored
  2261848
- Arm backend: Add decomposition of div.Tensor_mode (#13940)
  wwwind
  authored
  7a7e939
- Add boolean support for op_unbind_copy
  mergennachin
  authored
  303e2a0
- Fix crash in optimized log softmax
  mergennachin
  authored
  accc65e
- Fix torchao quantization for deployment targets under iOS18 (#13896)
  metascroy
  authored
  7e76ba2
- Update OSS flow to call the right APIs
  mcremon-meta
  authored
  2517376
- [Backend Tester] Don't run fp64 model tests (#13982)
  GregoryComer
  authored
  41c299f
Commits on Sep 6, 2025
- Backend-agnostic implementation of quantized_conv_nchw
  DrJessop
  authored
  d3e19eb
- [retake] Use extension module for pybind (#14029)
  
  pytorchbot
  and
  lucylq
  authored
  fa4d935
- Use OSS kernels everywhere
  mcremon-meta
  authored
  705150c
- Arm backend: fwd fix for tosa dir refactor for buck files (#14031)
  digantdesai
  authored
  f01198f
- Fix voxtral instructions (#14026)
  mergennachin
  authored
  51b3329
- Arm backend: Refactor setup.sh to break out vulkan (#14009)
  perheld
  authored
  d0e46a8
- Fixing the broken unit tests caused by missing portable lib import. (#14014)
  jirioc
  authored
  79744f0
- Delete copy of quantized SDPA in torchao/experimental
  metascroy
  authored
  1a7441f
Commits on Sep 7, 2025
- Fix gemma3 AoT SWA Mask
  neuropilot-captain
  committed
  40d6d43
- Added platform-config argument
  neuropilot-captain
  committed
  62352f4
- Update runner support for varying number of inputs
  neuropilot-captain
  committed
  34e286b
- Merge remote-tracking branch 'origin/extract_share' into support_qwen_phi_gemma_whisper
  neuropilot-captain
  committed
  d417b06
- Support share weights for phi, gemma, qwen
  neuropilot-captain
  committed
  781e953
Commits on Sep 8, 2025
- Qualcomm AI Engine Direct - Strengthen Unit Test Robustness (#13892)
  shewu-quic
  authored
  bf22ab7
- [ET-VK] Quantized Int8 Linear (#14041)
  pytorchbot
  authored
  c1ea7e9
- Arm backend: Add 16A8W support and test for add operation (#14039)
  
  pytorchbot
  and
  Ninja91
  authored
  e19d555
- [ET-VK] Quantized Int8 Convolution (#14042)
  pytorchbot
  authored
  2c93fd2
- [ET-VK][AOT] Enable exporting Q8 Quantized Linear + Convolution (#14043)
  pytorchbot
  authored
  d25f6c4
- [ET-VK][ez] Enable max_pool2d.default (#14044)
  pytorchbot
  authored
  8da72c3
- [ET-VK] Fast path for choose_qparams (#14045)
  pytorchbot
  authored
  038ff0b
- [ET-VK] Implement linear_q4gsw (#14046)
  pytorchbot
  authored
  fbda3a9
- Fix lint errors and update llama sample run script
  neuropilot-captain
  committed
  6b30094
- Add program-data separation to pybindings
  lucylq
  authored
  6c12956
- Arm backend: Move vgf files to own folder (#13862)
  Erik-Lundell
  authored
  90153fa
- Arm Backend: Split ops_unary into op_ceil, op_floor and op_logical_not (#14059)
  agrima1304
  authored
  2845fd3
- Support pytorch checkpoints for Qwen3/Phi4 mini (#13984)
  metascroy
  authored
  61b0070
- Support weight sharing in PTE (#13941)
  neuropilot-captain
  authored
  a90e907
- Add create_mutable_buffer util (#13813)
  digantdesai
  authored
  9ce07da
- text_llm_runner manage start_pos internally (#14028)
  kirklandsign
  authored
  34b6a2d
- Arm backend: tweak setup.sh --enable-vulkan-sdk (#14058)
  mansnils
  authored
  edc0847
- Try testing more pytest (#14037)
  mergennachin
  authored
  0eb4361
- Executorch Wasm Internal Build
  Conarnar
  authored
  151f3be
- Del redundant-static-def in executorch/schema/extended_header.cpp +2
  r-barnes
  authored
  9ffcf4b
- CI: Enable trunk jobs on PRs on main/release (#14060)
  digantdesai
  authored
  339e9fc
- Backend-agnostic quantized_conv_nhwc (channels last)
  DrJessop
  authored
  5948238
- CI: Enable trunk jobs on PRs (#14087)
  digantdesai
  authored
  ea37202
- Backend-agnostic quantized_relu
  DrJessop
  authored
  1d4c6ba
- Allow custom ops to log outputs and profiling events for etdump
  kimishpatel
  authored
  855c083
- Fix missing things from OSS kernels updates
  mcremon-meta
  authored
  00b50b2
Commits on Sep 9, 2025
- Objective-C wrapper for multimodal LLM runner. (#14075)
  shoumikhin
  authored
  c22a2b0
- Voxtral documentation fix (#14097)
  rohansjoshi
  authored
  d79ef5b
- [ET-VK] Implemement linear_dq8ta_q4gsw (#14098)
  pytorchbot
  authored
  1882ea1
- Qualcomm AI Engine Direct - Test QNN2.37 uplevel Fix (#13927)
  winskuo-quic
  authored
  879e7ba
- Exynos Backend for Executorch to bring up on Exynos SoC (#13677)
  
  authored
  823dea1
- Jni layer use multimodal runner (#13825)
  kirklandsign
  authored
  de0ff26
- Fix batch norm partitioning with Conv3d (#13696)
  GregoryComer
  authored
  d05a793
- Remove quantized_conv and leave just per tensor variants
  DrJessop
  authored
  f478f2f
- Arm backend: Preserve output order (#13997)
  wwwind
  authored
  d324799
- Arm backend: Make ops supported check TOSA profile-aware (#14063)
  
  YufengShi-dudu
  and
  Erik-Lundell
  authored
  589d3cd
- Arm backend: Refactor setup.sh $PATH creation (#14083)
  perheld
  authored
  3db7900
- remove Numpy-2.0 incompatible use of numpy.product
  nbadami123
  authored
  b90c743
- Update module call graph for export program builder.
  hsharma35
  authored
  bbe8943
- Replace aten linalg svd with cadence version.
  hsharma35
  authored
  53b4cb9
- Merge remote-tracking branch 'origin/extract_share' into support_qwen_phi_gemma_whisper
  neuropilot-captain
  committed
  ea5cd4d
- Update runner for weights sharing
  neuropilot-captain
  committed
  fd52664
- Arm backend: Fix run_vkml_emulation_layer tests (#14107)
  Erik-Lundell
  authored
  3c533aa
- NXP backend: Add pre-processing pass to replace nodes with known outputs with their data. (#13937)
  MartinPavella
  authored
  396d722
- Arm backend: Align dim_order ops handling (#14064)
  AdrianLundell
  authored
  59b9707
- forward fix for D81697327 (#14113)
  
  pytorchbot
  and
  Gasoonjia
  authored
  245630a
- Fix missing kernel header in stack util (#14069)
  
  jathu
  and
  Jathu Satkunarajah
  authored
  397b8df
- Arm backend: Fix tensor_div_mode decomp for scalar input (#14114)
  AdrianLundell
  authored
  c6e52f3
- Revert for fix conflict
  neuropilot-captain
  committed
  14ce449
- Merge branch 'main' into support_qwen_phi_gemma_whisper
  neuropilot-captain
  authored
  0a28404
- Arm backend: Refactor compile spec handling (#14111)
  Erik-Lundell
  authored
  29cec35
- Merge branch 'main' into support_qwen_phi_gemma_whisper
  neuropilot-captain
  authored
  a427f62
- Fix conflicts
  neuropilot-captain
  committed
  018e574
- Arm backend: Remove _vgf_ op pytest xfails (#14116)
  mansnils
  authored
  465e65f
- Update default executor runner with new optional options (#14017)
  mansnils
  authored
  16aac24
- Qualcomm AI Engine Direct - Improve GA Qwen 2.5 (#14047)
  shewu-quic
  authored
  c3e4ed9
- [EXIR] Update RemoveCloneOpsTransform to be dim order aware (#12976)
  
  keyprocedure
  and
  Gasoonjia
  authored
  dbac09c
- NXP backend: Update the pass `move_auxiliary_operator_into_separate_qdq_cluster`. (#13841)
  MartinPavella
  authored
  2a06efb
- Use native APIs in LLaMA demo app.
  shoumikhin
  authored
  8973eeb
- Add torchao checkpoint tests (#14074)
  metascroy
  authored
  dc944fe
- Update torch pin to Sep 06, 2025 (#14129)
  mergennachin
  authored
  8496f27
Commits on Sep 10, 2025
Commits on Sep 11, 2025
- Revert "Arm backend: Refactor compile spec handling (#14111)" (#14175)
  swolchok
  authored
  f294074
- Support for all quantized linear ops
  DrJessop
  authored
  de30390
- Android Refactor generate() (#14183)
  kirklandsign
  authored
  53b7ec5
- Revert "Update default executor runner with new optional options" (#14193)
  mergennachin
  authored
  dc190f9
- [Windows] Add CI wheel build jobs for Windows (#13837)
  GregoryComer
  authored
  56d5186
- Fix lint-url errors
  neuropilot-captain
  committed
  f09ffd3
- Add qnn backend to pip packages (#13794)
  cccclai
  authored
  068d341
- Arm backend: Add 16A8W support and test for mul operation (#14196)
  
  pytorchbot
  and
  Ninja91
  authored
  fe9447e
- [ET-VK] Misc code cleanup to recent Quantized Linear + SDPA implementations (#14197)
  pytorchbot
  authored
  1d37845
- Support more ops and verify more models (#14106)
  
  authored
  63481e3
- [ET-VK][DCE] Remove redundant quantized linear implementations (#14198)
  pytorchbot
  authored
  9c85266
- Allow zero-element tensors for shared inputs.
  hsharma35
  authored
  b2ae2b4
- Deprecate debug buffer in Module (#14199)
  
  pytorchbot
  and
  lucylq
  authored
  80fbede
- Arm backend: Document Ethos-U memory modes and add Ethos-U porting guide (#14144)
  gggekov
  authored
  6ed10e5
- Fix lint-url error
  neuropilot-captain
  committed
  a8ccdb6
- forward fix for copy_remove pass dependency (#14204)
  
  pytorchbot
  and
  Gasoonjia
  authored
  9e37d87
- support etdump generation in executorch.runtime (#14205)
  
  pytorchbot
  and
  Gasoonjia
  authored
  2805a81
- Support BFloat16 in exir
  manuelcandales
  authored
  a7eefd0
- Remove torchcodec from install_requirements.py (#14211)
  mergennachin
  authored
  9d2d188
- Revert "Add qnn backend to pip packages (#13794)" (#14212)
  swolchok
  authored
  1ea9feb
- Arm backend: Add 16A8W support and test for sigmoid operation (#14213)
  
  pytorchbot
  and
  Ninja91
  authored
  6b9c0a6
- Arm backend: Add 16A8W support and test for tanh operation (#14214)
  
  pytorchbot
  and
  Ninja91
  authored
  add30f6
- Arm backend: Add 16A8W support and test for slice operation (#14215)
  
  pytorchbot
  and
  Ninja91
  authored
  7f95941
- Qualcomm AI Engine Direct - PTQ for llama3.2 1b/3b (#12700)
  haowhsu-quic
  authored
  2e44184
- Allow PUBLIC visibility and make it the default
  swolchok
  authored
  06362f3
- Mass migrate to runtime.python_library
  swolchok
  authored
  39bdb41
- Arm backend: Add 16A8W support for view and transpose operations (#14216)
  
  pytorchbot
  and
  Ninja91
  authored
  c5d9c6f
- Arm backend: Add 16A8W support and test for cat operation (#14217)
  
  pytorchbot
  and
  Ninja91
  authored
  fe26bfd
- Arm backend: Add 16A8W FCNode support with BMM dependency fix (#14218)
  
  pytorchbot
  and
  Ninja91
  authored
  e4b1d51
- Add quantized fully connected ops
  DrJessop
  authored
  d28f924
- Use optimized kernels when available in top-level selective build (#14203)
  GregoryComer
  authored
  7fbb88c
- Fix bug where source transform passes untransformed model to next stage (#14186)
  abhinaykukkadapu
  authored
  8a0a25b
- Fix XNNPACK handling of negative permute dims (#14169)
  GregoryComer
  authored
  3124a6b
- Fix BFloat16 support for op_index
  manuelcandales
  authored
  c638851
- Bump torchao pin and use v2 torchao tensors (#14171)
  metascroy
  authored
  a4b7de0
- Qualcomm AI Engine Direct - GA Static Smollm3 3B (#14149)
  DannyYuyang-quic
  authored
  9af908d
- All variants of quantized relu
  DrJessop
  authored
  c97c2c1
- Update selective build example + CI for top-level targets (#13741)
  GregoryComer
  authored
  6779d15
- Fix get_max_seq_len metadata method not found (#14192)
  jackzhxng
  authored
  d4d1df7
- Add Bfloat16 support to op_convolution
  manuelcandales
  authored
  c969a4f
- Get tensor data
  lucylq
  authored
  ab8c244
- Test any_dims_out with empty dim list
  manuelcandales
  authored
  892e70f
- Reapply "Update default executor runner with new optional options"
  mansnils
  authored
  7f49d4f
- Arm backend: Fix bug in memory allocation
  gggekov
  authored
  8358516
- Add segment_data_size to extended header (#14226)
  
  pytorchbot
  and
  lucylq
  authored
  0e3d6d3
- Migrate from cpp_python_extension to runtime.cxx_python_extension (#14154)
  swolchok
  authored
  6d8583d
- Add GitHub Actions workflow for building CPU Aarch64 Linux wheels (#13946)
  chizkiyahu
  authored
  0cac45b
- Unify and reorder replace passes.
  eigen-k
  authored
  587113a
- [Windows] Run native unit tests in CI (#13923)
  GregoryComer
  authored
  f1ca55a
- Remove LLaMA demo app. (#14195)
  shoumikhin
  authored
  9d292a3
- Arm backend: Update backends/arm/README.md (#14206)
  tom-arm
  authored
  9e28fd7
- Revert "Migrate from cpp_python_extension to runtime.cxx_python_extension (#14154)" (#14233)
  swolchok
  authored
  b23f883
- Manual merge of #14154 (#14234)
  swolchok
  authored
  0935300
- Manual merge of #14155 (#14235)
  swolchok
  authored
  9decdf1
- Manual merge of #14156 (#14236)
  swolchok
  authored
  6bb6356
- Manual merge of #14157 (#14237)
  swolchok
  authored
  e207142
- Revert "Revert "Add qnn backend to pip packages (#13794)"" (#14220)
  cccclai
  authored
  f7df6b1
Commits on Sep 12, 2025
- Merge branch 'main' into support_qwen_phi_gemma_whisper
  neuropilot-captain
  authored
  4f65d18
- Compare full key value when retrieving named data
  LeeOHzzZ
  authored
  62a60d6
- Support bert distilbert (#14138)
  
  neuropilot-captain
  and
  neuropilot-captain
  authored
  36c2dd1
- [ET-VK] Enable automatic dtype conversion when copying to/from staging buffer (#14244)
  pytorchbot
  authored
  b63b358
- flat_tensor check size
  lucylq
  authored
  9a9db14
- [ET-VK] Add 'half' variants to some Llama operators + enable llama vulkan export with force_fp16 flag (#14245)
  pytorchbot
  authored
  35c1f52
- try fix (#14246)
  cccclai
  authored
  f2eb38e
- Add EXECUTORCH_THREADPOOL_SIZE options, default to using only performance cores (#14090)
  GregoryComer
  authored
  72d50b2
- fix ci lint error with workaround patch
  neuropilot-captain
  committed
  8e237e2
- Merge branch 'main' into support_qwen_phi_gemma_whisper
  neuropilot-captain
  authored
  ddc8ce8
- remove large files for lint
  neuropilot-captain
  committed
  97e73d2
- Remove large files
  neuropilot-captain
  committed
  66cad81
- Add gemma3 and phi4 run.sh
  neuropilot-captain
  committed
  ed29c7d
Commits on Sep 13, 2025
- Update README
  neuropilot-captain
  committed
  4a281fe
- Remove NeuronAdapter.h
  neuropilot-captain
  committed
  ac2a41c

Commits

Commits on Sep 3, 2025

Commits on Sep 4, 2025

Commits on Sep 5, 2025

Commits on Sep 6, 2025

Commits on Sep 7, 2025

Commits on Sep 8, 2025

Commits on Sep 9, 2025