Ascend AutoKernel Model-Level Pipeline

This repository is currently focused on model_level_pipeline/: an Agent-assisted Huawei Ascend NPU model-level optimization framework. The active path is not the old root single-operator AutoKernel harness.

Active Workflow

user-provided model + configured strategy list
  -> optional profile / run artifacts / trace evidence
  -> AnalysisReport
  -> match evidence to configured Candidate entries
  -> choose a candidate or beam/variant plan within the configured strategy set
  -> dispatch Candidate.action_id
  -> apply one focused patch
  -> full-model verify and measure
  -> after-analysis
  -> KEEP / MOVE_ON / REVERT
  -> record evidence, patch summary, and verdict
  -> optional strategy-owned operator drilldown

Single-operator optimization remains available only as an optional sub-flow under model_level_pipeline/strategies/operator/. A faster local operator is supporting evidence only; acceptance requires full-model correctness and end-to-end improvement.

Main Entrypoints

# Build a model-level plan from an existing run artifact directory
python model_level_pipeline/pipeline.py --run-dir /path/to/model-run-dir

# Print planned profile / verify / operator-drilldown commands without executing
python model_level_pipeline/pipeline.py \
  --run-dir /path/to/model-run-dir \
  --commands --workflow \
  --module transformers --class-name AutoModelForCausalLM \
  --pretrained /path/to/local-or-modelscope-model \
  --input-shape 1,2048 --dtype float16

# Capture profile evidence with the model-level profiler module
python -m model_level_pipeline.tools.profile \
  --module transformers --class-name AutoModelForCausalLM \
  --pretrained /path/to/local-or-modelscope-model \
  --dataset-path data/prompts_real.jsonl \
  --input-shape 1,2048 --dtype float16

# Optional operator drilldown command targets
python -m model_level_pipeline.strategies.operator.extract --backend npu --top 5
python -m model_level_pipeline.strategies.operator.orchestrate next
python -m model_level_pipeline.strategies.operator.bench > run.log 2>&1

Repository Layout

model_level_pipeline/
  analyzers/        run/profile/trace evidence readers and hotspot attribution
  core/             plan, state, decision, automation flow, pipeline orchestration
  executors/        experiment wiring, configured model-session runners, patch dispatch
  reports/          experiment recorder, timeline, patch_points.md
  strategies/       Candidate/action registry and strategy families
    operator/       strategy-owned operator extract/orchestrate/bench drilldown
  tools/            model-level profile, verify, dataset, run-artifact helpers
  tuners/           local cost model / search helpers
  tests/            unit tests

verify.py           current full-model verification harness
reference.py        shared reference helpers
sitecustomize.py    environment bootstrap for torch/torch_npu imports
data/               prompt JSONL inputs
workspace/          generated runtime artifacts

Model Sources

Configured model sessions should load a user-provided local model directory. For domestic model download, use ModelScope into a local cache/workspace path first, then pass that local directory as --pretrained or through RealSessionConfig. Do not write into read-only model source directories.

Development Notes

• Start from AGENTS.md for repository routing.
• Current work should prefer model_level_pipeline/ and model_level_pipeline/tests/.
• Do not accept an optimization without full-model correctness plus end-to-end metric improvement.
• Keep generated artifacts under workspace/ or explicit model-level session directories.