Skip to content

suspend-to-off: STM32 Stop 1 sleep window (A/B candidate 1)#93

Open
evgeny-boger wants to merge 19 commits into
feature/ec-suspend-modefrom
feature/ec-stop-mode
Open

suspend-to-off: STM32 Stop 1 sleep window (A/B candidate 1)#93
evgeny-boger wants to merge 19 commits into
feature/ec-suspend-modefrom
feature/ec-stop-mode

Conversation

@evgeny-boger

@evgeny-boger evgeny-boger commented Jul 8, 2026

Copy link
Copy Markdown
Member

Что происходит; кому и зачем нужно:

Suspend-to-off для WB 8.5: на время окна «сна» EC засыпает в STM32 Stop 1 вместо busy-poll и просыпается по RTC-будильнику, кнопке питания (EXTI) или страховочному WUT-дедлайну. Нужно для снижения потребления контроллера в suspend-to-off (вклад EC в floor). Ветка полностью проверена на железе (стенд 2026-07-08…09); диагностическая инструментализация снята отдельным коммитом 959db0d, эксплуатационные принты оставлены.

Что поменялось для пользователей:

Итоговый UX-контракт пробуждения кнопкой: нажал → пищит под пальцем через ~0.8 с → отпустил. Случайное касание (<0.5 с) отфильтровывается — контроллер не просыпается; удержание 8 с — принудительное выключение, как и раньше; пробуждения по будильнику и дедлайну — молча (необслуживаемые). Время от пробуждения до SPL — ~0.3 с (PWRON нажимается сразу, без ожидания самозапуска PMIC).

Пять багов, найденных и исправленных на железе (у каждого — регрессионный тест):

  1. Grace-окно бросало удерживаемые нажатия — окно ожидания мерилось от фронта EXTI, и удержание >~1.2 с роняло EC обратно в Stop посреди нажатия; теперь grace продлевается, пока антидребезженный уровень «нажато». Тест: нет повторного входа в Stop при удержании (интеграционный, проверен fault-injection).
  2. Бип на обесточенной шине → перенесён на возврат 3.3В — пищалка питается от 3.3В, выключенного всё окно: PWM-вызов был идеален и беззвучен; бип перенесён в последовательность пробуждения — звучит в первый момент возврата 3.3В, до PWROK, пока SoC в сбросе. Тест: порядок «бип → PWROK» в wake-последовательности (PS_WAKE_BEEP_WAIT).
  3. Отпускание wake-нажатия доезжало до Linux после resume — пережившее выход из окна нажатие фиксировало новый short-press на отпускании → hard off; теперь события кнопки «глотаются» до наблюдаемого отпускания (с учётом антидребезга). Тест: оба порядка отпускания (до/после входа в WORKING).
  4. linux_booted терялся через resume → hard-off — каждый resume проходил power-on-sequence, сбрасывавший linux_booted, и нажатие в первые 20 с после resume жёстко рубило питание; resume теперь сохраняет linux_booted (Linux возобновляется, а не загружается). Тест: resumed-press → graceful IRQ + сохранён легитимный hard-off при холодной загрузке (fault-injection).
  5. USART не дожидался конца передачи и вешал общую консоль — инвертированное условие ожидания TC: вход в Stop замораживал USART посреди кадра, TX залипал на всё окно и клал общий debug-мост; теперь принт полностью выталкивается до входа в Stop. Тест: кросс-оконное кнопочное покрытие (регистровая suite mcu-pwr + интеграционные alarm→button, button→button, button→alarm).

Как проверял/а:

На стенде WB 8.5 (захват общей debug-консоли), 2026-07-08…09: пробуждение кнопкой с бипом под пальцем, пробуждение по будильнику (rtcwake), страховочный дедлайн, повторные окна; wake-to-SPL ~0.3 с. Осталось до перевода из draft: замер floor потребления (MDP) + мини-прогон на 10 циклов — в очереди на стенд; после них — строка в таблицу «Проверка устройств».

Покрыл/а изменения юниттестом и если нет, то почему:

Да: каждый из пяти фиксов несёт регрессионный тест, для grace/классификации/linux_booted дополнительно проверено fault-injection (тест падает при откате фикса). Полный прогон входит в make MODEL_WB85: 458 тестов, 0 падений; WB74 также собирается.

🤖 Generated with Claude Code


Известная проблема (2026-07-10): «проглоченный» PWRON при мгновенном пробуждении.
Если выход из Stop происходит сразу после входа (будильник истёк ещё до засыпания EC — на стенде это случается при отладочном CRC-подсчёте ~80 с), импульс PWRON гонится с переходом PMIC в сон: спадающий фронт POK восстанавливает шины, но дебаунс ONLEVEL (~128 мс) может отбросить само нажатие — PMIC остаётся «полупроснувшимся» (3.3В есть, ядра не стартуют). Восстанавливается вторым (силовым) таймаутом EC (~4.5 мин). Частота — только на zero-dwell циклах (~1/15); на обычных пробуждениях не воспроизводится (0 из 60+ циклов). Две попытки аппаратного обхода (задержка 500 мс + retry по отсутствию 3.3В; полноширинное нажатие 200 мс) протестированы на стенде и откачены (30b658c): вторая вносит вероятностный двойной старт с потерей резюма. Дальше — только после осциллографирования POK/PWROK/шин вокруг входа в сон; дешёвая митигация-кандидат: укороченный первый таймаут вотчдога после resume-пробуждения. Подробности: dossier в bench-заметках.

evgeny-boger and others added 7 commits July 8, 2026 01:45
The suspend-to-off Stop-1 sleep window reuses the RTC wakeup timer (WUT)
as its deadline backstop and IWDG-feed tick (systick is dead in Stop, so
the ms deadline can no longer advance). Salvage the deadline-machinery
test hook from the parked Standby experiment: teach the rtc mock to
record the last programmed WUT period so the Stop tests can assert the
feed/deadline timer is armed with the expected period.

Previously rtc_set_periodic_wakeup() was a no-op in the mock and only
its disable was observable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
During suspend-to-off the EC stayed fully awake at 64 MHz, busy-polling
for the wake condition for the whole sleep window and burning run-current
on top of the ~0.212 W suspend floor. Once the sleep has really begun
(3.3V dropped), drop the EC into STM32 Stop 1 (WFI + SLEEPDEEP, LPMS=001)
between wakes instead.

Unlike the parked Standby experiment, Stop retains SRAM and the GPIO
output latches and resumes in place (no reset), so PD1/PD3/PB3 hold their
driven levels through the whole window - the wake-reset / DRAM-loss
failure family is eliminated by construction, and the wake sequence
(5V-en -> VMON V33 poll -> PWRON -> PWROK pulse) runs unchanged.

Design (see ec-stop-mode-brief.md):
- mcu-pwr: mcu_stop_window_prepare wires the wake sources missing today -
  RTC Alarm A + WUT (direct EXTI line 19) and the PWRON button (PA0/EXTI0
  falling edge) - and masks the SoC-CS EXTI9. The cause-aware ISRs dismiss
  the line without clobbering ALRAF (mask ALRAIE for an alarm, clear only
  CWUTF for a tick) so the polled alarm consumer still sees the wake.
  mcu_stop_enter feeds the IWDG, clears only stale WUT/button/CWUF (never
  CALRAF), enters Stop, and on wake restores the 64 MHz PLL, systick, ADC
  and the vmon settle gate.
- wbec: the off-mode poll block arms the window once, classifies the wake
  (Alarm A / PWRON / deadline) at the top of the pass consuming the
  latches the super-loop drivers set earlier, and enters Stop at the
  bottom. The deadline moves off the (now-frozen) systick onto a WUT-tick
  accumulator. A button EXTI edge keeps the EC awake so the normal 500 ms
  debounce can confirm the press, preserving the accidental-touch filter.
- IWDG: fed from the < 10 s WUT wake (no option-byte programming), keeping
  the exact watchdog envelope the product ships today.
- Forced-safe outputs held for the window: heater (PD2) and V_OUT (PD0)
  are driven off - their NTC / V_IN monitoring is frozen in Stop and their
  GPIO latches would otherwise persist unmonitored.

Unit-test stubs (mcu-pwr Stop primitives, temperature/gpio suspend) added
so the existing suites keep building and passing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add integration coverage for the Stop-1 sleep window decisions (the Stop
entry/exit itself is hardware, stubbed; the wake-classification logic
around it is what these exercise):

- window arm forces the heater (PD2) and V_OUT (PD0) safe, arms the WUT
  with the feed period, never requests Standby, and a fresh Alarm A wakes
  via REASON_RTC_ALARM, finishing the window and disabling the WUT;
- a PWRON EXTI edge keeps the EC awake so the normal 500 ms debounce
  confirms the press (REASON_POWER_KEY), and a brush that never debounces
  re-enters Stop instead of waking;
- the deadline backstop fires off the WUT-tick accumulator (REASON_
  WATCHDOG) and, the mirror regression, a freely advancing systick with no
  WUT ticks never fires it - proving the deadline moved off the (in Stop
  frozen) systick;
- the IWDG is fed on every Stop feed-tick (the direct regression for the
  Standby ~10 s reset).

The existing 2ac7e28 exit-arm regression tests are kept unchanged and stay
green - Stop is always a same-core resume, so they matter even more.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Restore the interrupt state to exactly what it was before the sleep
window so normal operation runs byte-for-byte unchanged: on the real-wake
exit, disable the Stop-only wake sources (PWRON EXTI0 and RTC EXTI19 /
NVIC) and clear their latched causes, in addition to restoring the
SoC-CS EXTI9 mask. Keeps the suspend window fully self-contained instead
of leaving new interrupts enabled after the first resume.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Stop-mode RTC ISR masked the fired AlarmA with a bare
`RTC->CR &= ~RTC_CR_ALRAIE`, but this firmware keeps RTC write-protection
LOCKED in steady state, and the Stop-arm path (rtc_set_periodic_wakeup ->
end_init_enable_wpr) leaves WPR locked at the moment we enter Stop. A
write-protected RTC_CR write is silently dropped by hardware, so ALRAIE
stayed set. EXTI line 19 is a DIRECT line = (ALRAF & ALRAIE); with ALRAF
latched by a fired alarm and ALRAIE never actually cleared, the line stays
asserted and the NVIC tail-chains RTC_TAMP_IRQ forever. On a Linux-requested
alarm wake the core never returns to thread mode: the wbec classify never
runs (linux_cpu_pwr_seq_wakeup never fires, the SoC is never restarted) and
the IWDG is never reloaded -> ~10 s reset -> DRAM self-refresh lost -> cold
boot. That is the exact Standby failure family Stop was meant to eliminate,
hitting the most important wake path.

Add rtc_mask_alarm_irq(), which clears ALRAIE under a WPR unlock/relock
(no INIT mode needed for the interrupt-enable bit) and, crucially, does NOT
touch ALRAF: the poller rtc_alarm_do_periodic_work still reads the ALRAF
latch to recognise the alarm wake. The ISR now calls this helper. The WUT
path (RTC->SCR, not write-protected) and the button path (EXTI->FPR1) were
already correct.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The load-bearing register + ISR logic in mcu-pwr.c had ZERO automated
coverage: the only harness exercising the suspend flow (wbec-integration)
fully MOCKS mcu-pwr, so the real ISRs, the LPMS=001/SLEEPDEEP writes, the
CWUTF/FPR0/CWUF-only clearing and the ALRAF-preservation never executed
under any test. That is why the alarm-line blocker shipped all-green.

Add unittests/mcu-pwr: a register-stub suite that compiles the REAL
src/mcu-pwr.c against test-owned RTC/EXTI/PWR/SCB instances (bit masks copied
verbatim from stm32g030xx.h / core_cm0plus.h) with NVIC and __WFI intercepted.
It asserts, at the register level:
 - Stop entry writes LPMS=001 (Stop 1, never 011/Standby), SLEEPDEEP set at
   the WFI, clears only CWUTF/FPR0/CWUF and NEVER CALRAF, clears the EXTI0
   NVIC pending, and feeds the IWDG before and after sleep;
 - the alarm ISR masks ALRAIE via the WPR-unlocked helper while PRESERVING
   ALRAF (never CALRAF) - this fails on a bare CR write, so the blocker can
   no longer regress-pass;
 - the WUT ISR clears only WUTF and leaves the armed alarm untouched;
 - the button ISR reports the wake and W1C-clears FPR0;
 - prepare arms EXTI19/EXTI0 + NVIC and masks EXTI9; finish restores them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…filter

test_stop_wake_by_button_edge_then_debounced_press confirmed the press with
utest_pwrkey_set_short_press(true) - injecting a ready-made short press at the
mock boundary - so the stay-awake-and-debounce interaction with the 500 ms
filter was never exercised end-to-end.

Give the pwrkey mock an OPT-IN debounce model (default off, so every existing
consumer is byte-for-byte unchanged) that mirrors src/pwrkey.c: a held raw
press becomes a confirmed short press only after > PWRKEY_DEBOUNCE_MS held
and > PWRKEY_DEBOUNCE_MS released, driven by system time through
pwrkey_do_periodic_work. Call pwrkey_do_periodic_work in the integration
sim loop (as main.c does). Rewrite the button test to hold PWRON, assert the
held-not-released press does NOT wake, then release and assert the debounced
short press wakes with REASON_POWER_KEY - within the debounce+grace window.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@evgeny-boger

Copy link
Copy Markdown
Member Author

Decision: Stop 1 selected as the primary window implementation — proceeding to bench validation (G2). #94 stays as the documented alternative.

…h-triage tests

Bench triage 2026-07-08 ("the window instant-wakes, every cycle, reason=RTC
alarm") root cause: NOT an EC bug. The diagnostic BL31 build spends ~80-90 s
computing DRAM checksums between the EC announce (0xA4) and the actual 3.3V
drop, so 30-60 s wake requests expire while the EC is still awake waiting for
the sleep to start; the awake poller latches the fired alarm and the pre-sleep
guard correctly wakes the board the moment the window opens (wake time already
passed; identical behavior at the a655128 busy-poll baseline). The 300 s rungs
in the same capture show real multi-minute Stop windows ending in clean in-Stop
alarm wakes (DRAM CLEAN, all CPUs up, PM: suspend exit), including a clean
window immediately after an alarm-ended one - no stale-ALRAF carryover.

- wbec: print "EC sleeping in Stop 1" once, at the real window arm (3.3V
  drop), so the bench log can distinguish "EC never slept yet" from "EC woke
  instantly". This ambiguity is what produced the false diagnosis.
- integration tests: (a) two consecutive windows where the first ends via a
  fired alarm - the second must not wake on a stale latch and must report the
  fresh alarm as REASON_RTC_ALARM; (b) an alarm firing between the announce
  and the 3.3V drop must wake the board immediately at window entry with
  REASON_RTC_ALARM - codifies the correct behavior seen on the bench so it is
  not "fixed" away later.

No functional firmware change. Bench guidance: rungs must request sleeps
longer than the BL31 suspend-entry latency (>= 120 s with the DRAM-sums
diagnostic in place), and the power floor must be measured inside a long
rung's quiet window.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@evgeny-boger

Copy link
Copy Markdown
Member Author

Bench status: Stop 1 confirmed working on hardware — genuine ~219 s Stop window observed with in-Stop RTC-alarm wake, clean DRAM retention verdict, full resume. An earlier suspected 'instant-wake' bug was a false alarm: the BL31 CRC-verify diagnostic adds ~80-90 s of awake time at suspend entry, so short (30-60 s) alarms legitimately expire pre-window and the EC correctly wakes immediately (baseline-identical, now locked by two new regression tests in df0f545, which also adds a window-entry console marker). Validation note: with the CRC diagnostic armed, use windows ≥120 s. Remaining for undraft: power-floor measurement (MDP), wake-source matrix (button, deadline), 10-cycle mini-soak on rungs that actually sleep.

…r resume

Bench 2026-07-08 (button-wake test): one press produced wake + poweroff +
cold boot. The wake itself worked, but while the button was held during the
sleep window, the normal WORKING-state handler (linux_booted ->
irq_set_flag(IRQ_PWR_OFF_REQ)) also latched a "power off request" for Linux -
from the EC's point of view Linux is "on" for the whole suspend window, and
the sleeping Linux never acks IRQ flags. After resume the wbec-pwrkey driver
read the stale flag and reported the same press again; logind's
HandlePowerKey=poweroff then shut the freshly woken board down, and with no
alarm set the EC turned that into a reboot.

Fix: on the suspend-to-off wake exit, clear the pending press indications
(wbec_ctx.pwrkey_pressed and the IRQ_PWR_OFF_REQ flag) - a press consumed as
the wake cause, or latched while Linux was powered down, must not stay
pending. A press during normal running keeps today's behavior (the clear
happens only on the window exit), and the powered-off Standby path is
untouched (different state machine, WKUP-based).

NOTE: the a655128 baseline has the same latent double-delivery in its
busy-poll window (same :519 level-handler + same classify consumer); it was
simply never exercised - the D4 button matrix never ran there. If the
fallback ladder ever ships a655128, this fix must be cherry-picked onto it.

Tests (integration, fail-without-fix verified by fault injection):
- test_stop_button_wake_press_not_redelivered_to_linux: hold the button in
  the window until the running-state handler latches IRQ_PWR_OFF_REQ
  (asserted as the bug precondition), release -> wake by button, assert the
  flag is no longer pending for Linux.
- test_running_press_still_delivers_pwr_off_req: a press with Linux booted
  and no suspend must still deliver IRQ_PWR_OFF_REQ (guard against
  overcorrection).
Also deduplicate the poweron-reason regmap helper the previous commit added
(the file already had get_poweron_reason_from_regmap and the UTEST_REASON_*
enum).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@evgeny-boger

Copy link
Copy Markdown
Member Author

Bench button-wake matrix result + fix (4421ec8): the wake path works end-to-end, but the waking press was double-delivered — the running-state level-check (wbec.c:519) latched IRQ_PWR_OFF_REQ during the in-window debounce, the kernel pwrkey driver re-reported it post-resume, logind powered the box off (one press = wake + shutdown + reboot, reproduced on bench with full console chain). Fix: clear the pwrkey latch + IRQ_PWR_OFF_REQ at suspend-window wake-exit only; normal-running press semantics unchanged (regression-tested both ways, fail-without-fix verified by fault injection).

…-starts

Bench 2026-07-08 (Evgeny): every suspend-to-off wake paid a fixed ~1.0 s
between "wake up, restart PMIC via PWRON" and the actual PWRON press,
dominating the 1.3-1.5 s wake-to-SPL time. Cause: PS_ON_STEP1_WAIT_3V3 waits
1000 ms for 3.3V to appear by itself - correct for a cold power-on (5V is
applied and the PMIC auto-starts) but guaranteed dead time on a suspend wake:
5V never dropped, the AXP is in sleep and only a PWRON (POK) press wakes it,
so the timeout always expired before the first press.

Fix: in PS_ON_STEP1_WAIT_3V3, when wake_pending is set (the suspend-resume
marker the wake path already carries) and 3.3V is absent, press PWRON
immediately instead of waiting out the 1 s self-start timeout. The V33
reading is already trustworthy at that point: do_periodic_work is gated on
vmon_ready(), i.e. 100 ms of settled post-Stop samples. The 3.3V-present
branch (late SoC suspend abort -> PWROK pulse, no press) stays first; the
STEP2/STEP3 retry ladder is untouched and remains the fallback; the cold
power-on path keeps its 1000 ms self-start wait. The press width is
unchanged: PWRON is held until 3.3V appears (>= 100-200 ms), well above the
AXP POK onlevel filter - the same hold that already woke the PMIC reliably
after the old delay.

Expected wake latency: press at ~105 ms after the wake decision (vmon settle
dominates), 3.3V up at ~250-300 ms, PWROK pulse 100 ms => wake-to-SPL around
0.4-0.5 s, from 1.3-1.5 s. Wake-to-PMIC-restart meets the < 200 ms target.

Bench verification items: scope the POK low width on the immediate press vs
the AXP POK sleep-wake filter (BL31 arms POK-negedge wake, REG41 bit3), and
confirm no V33 false-present reads at the press decision point right after
Stop (the 100 ms vmon settle should guarantee this).

NOTE: this wake path (wake_pending + WAIT_3V3) is byte-identical at the
a655128 baseline - the same fixed 1 s loss exists there and this fix
cherry-picks cleanly if the fallback ladder ships a655128.

Test (integration, fail-without-fix verified by fault injection):
test_stop_wake_presses_pwron_immediately_when_pmic_sleeps - PMIC modeled as
on real hardware (stays asleep until PWRON is held), the board must be booted
within 600 ms of the alarm wake (~350 ms with the fix, ~1.3 s without).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@evgeny-boger

Copy link
Copy Markdown
Member Author

Wake-latency fix (be7bb1c): every suspend wake paid a fixed 1 s in PS_ON_STEP1_WAIT_3V3 — the cold-boot AXP self-start timeout, always futile on a wake (PMIC sleeps until POK). wake_pending now short-circuits to an immediate PWRON press (~105 ms, vmon-settle-gated); retry ladder unchanged as fallback. Expected wake-to-SPL 0.4-0.5 s (from 1.3-1.5 s). Regression test with hw-faithful PMIC model, fail-without-fix verified. Same latent 1 s exists at the a655128 baseline (cherry-picks cleanly).

…rame

Bench 2026-07-09 00:37 ("button wake is dead"): the capture shows every
window's arm marker followed by a flood of garbage spaces, 1 Hz whitespace
lines, and finally the debug-console USB bridge hanging up 39 s into the
window - the capture was BLIND from ~3 s after the window armed, so nothing
the EC or SoC printed after that (including any button wake) could be seen.

Root cause: usart_wait_tranmission_complete() had an inverted condition
(`while (ISR & TC)`) - during an active transmission TC=0, so the "blocking"
print returned with 1-2 frames still in the shift register. Latent since
forever, harmless while the EC stayed awake after printing. df0f545 added the
first print that is followed microseconds later by Stop entry (the window arm
marker): the USART clock froze MID-FRAME, the TX line stuck at a data-bit
level for the entire window, and the shared debug bridge streamed garbage
until it wedged. That is the actual regression behind the "dead button"
report; the 881f7e6..be7bb1c diff contains no change to the EXTI0/button
path, and the arm marker in the log proves mcu_stop_window_prepare() ran for
the affected window.

Fix: wait for TC=1 (last frame fully shifted out) - the honest cost is <= 2
frames (~170 us) per print. Every print now drains before Stop entry (and
before usart_tx_deinit's USART reset, a second latent victim).

Cross-window button coverage added while at it (the sequence the bench ran),
all passing - the arm/teardown lifecycle is correct in firmware:
- mcu-pwr register suite: test_stop_two_windows_button_works_after_alarm_wake
  - real prepare/ISR/finish across two windows; window 2 must fully re-arm
  EXTI0/FTSR/NVIC/handler and deliver the button edge to the classifier.
- integration suite: alarm->button (the bench sequence), button->button,
  button->alarm - full announce/sleep/wake cycles, window 2 must wake with
  the right reason.

Why the earlier two-window tests did not catch the bench symptom: there is no
firmware EXTI bug to catch - the failure was on the console path, which no
harness models (and the integration harness stubs mcu-pwr/EXTI entirely; the
register-level suite is the right home for wake-source lifecycle tests, now
including a cross-window one).

Bench re-run notes: press must be held >= PWRKEY_DEBOUNCE_MS (500 ms) - a
shorter tap wakes the EC, silently fails the debounce and the EC re-enters
Stop by design (no console marker for that; same on 881f7e6). Expect clean
marker lines with no trailing garbage and a live console throughout the
window; the 00:37:48 cold boot happened while the capture was blind and
should be re-tested on this build.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@evgeny-boger

Copy link
Copy Markdown
Member Author

False-regression resolved (40adfd3): the 'button wake dead' report was the DEBUG CONSOLE dying, not the button — usart_wait_tranmission_complete() polarity was inverted (latent forever), and the df0f545 arm-marker print entered Stop microseconds later with the UART frozen mid-frame; the shared USB console bridge wedged and the capture went blind exactly during the button test. Fixed with a proper TC drain (~170 µs/print) + register-level cross-window wake-source regression tests (alarm→button, button→button, button→alarm). Bench note: presses must be held ≥500 ms (debounce, by design).

evgeny-boger and others added 8 commits July 9, 2026 13:11
…tay silent

UX requirement (Evgeny, 2026-07-09): "If I press the button when the board is
suspended it should beep and resume. Do not send any poweroff keypresses to
the running system."

- Beep: emitted at the wake CLASSIFICATION, in the button branch only - the
  earliest point where the wake decision exists. Placement is deliberate with
  respect to the f3face8 lesson (a beep during resume corrupts the DRAM
  re-init over self-refresh and cold-boots the board): at classification the
  SoC is still in reset, the PMIC is asleep and DRAM sits in autonomous
  self-refresh; PWRON is pressed ~105 ms later (vmon settle) and the fragile
  SPL DRAM re-init starts no earlier than ~0.6 s - the 100 ms beep
  (EC_BUZZER_BEEP_SUSPEND_WAKE_MS) ends before both. The WORKING-entry beep
  stays suppressed for every suspend resume (suspend_resume_no_beep,
  unchanged) - that one *is* the DRAM killer.
- Silent unattended wakes: alarm and deadline branches emit nothing; the
  blanket WORKING-entry suppression already covered them - now regression-
  tested per cause.
- No poweroff keypress to Linux: already fixed in 4421ec8
  (wbec_ctx.pwrkey_pressed + IRQ_PWR_OFF_REQ cleared at the window exit);
  the beep assert is added to the same test so the requirement pair is
  checked together.
- Buzzer readiness at that moment: TIM3 registers and clock enable are
  retained through Stop, and classification runs after the W3 64 MHz restore,
  so buzzer_enable()'s SystemCoreClock-based PSC/ARR math is correct; the
  timed beep stop (buzzer_subsystem_do_periodic_work) keeps running through
  the wake sequence in the super-loop.

Wake timeline for a button press (hold >= 500 ms debounce): press t0 ->
press-begin at ~t0+0.5 s (pwrkey layer also beeps 300 ms here, its normal
press feedback - bench to confirm the double feedback is acceptable) ->
release + 0.5 s release debounce -> classification: wake-confirm beep (new,
100 ms) -> PWRON ~+105 ms -> SPL ~+0.6 s.

Tests (integration, fail-without-fix verified for the beep asserts): button
wake beeps exactly once at classification with the suspend-wake duration and
leaves no IRQ_PWR_OFF_REQ pending; alarm wake silent; deadline wake silent;
brush (no confirmed press) silent; cold-boot WORKING beep unchanged (the
baseline "1" in every count). Beep asserts gated on EC_GPIO_BUZZER (WB74 has
no buzzer).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n console

Bench 2026-07-09 13:26 on 4060b9b: a button press woke the EC (wake print
seen) but NO beep sounded (neither the pwrkey press-begin beep nor the new
classification beep), and ~11 s after resume Linux received a power-key event
anyway ("Power off request from Linux" -> alarm armed by rtcwake -s 600 ->
"Powering off" -> standby -> cold boot). The combination (no beeps at all,
yet a delivered press) is not explained by any single code path; the mock
suites cannot adjudicate. This build prints the ground truth (all prints are
safe now that TX drains fully before Stop entry, 40adfd3):

- "DIAG exit cause=..." - the classified wake cause + wut_ticks, debounced
  pwrkey level, raw irq flags, grace state, at every window exit;
- "DIAG exit-clear ..." - what the window exit actually clears;
- "DIAG btn-edge wake, grace begins" / "DIAG grace expired..." - the EXTI0
  edge path and the debounce verdict;
- "DIAG wut-tick N" - every feed-tick (window heartbeat visible);
- "DIAG PWR_OFF_REQ latched (running-state handler) susp=XY" - first latch of
  the power-off request toward Linux, with suspend_mode/suspend_started;
- "DIAG pk press-begin" / "DIAG pk short-press latched" - the pwrkey debounce
  state machine verdicts;
- "DIAG beep f=.. d=.. sysclk=.." - every buzzer_beep invocation with the
  SystemCoreClock the PWM math will use (adjudicates "beep called but
  inaudible" vs "beep never called");
- "DIAG WORKING-entry beep suppressed (resume)" - the no_beep gate decision.

Classify-priority review for the alarm-armed-but-NOT-fired case (rtcwake -s
600): rtc_alarm_take_fired() consumes a latch that rtc_alarm_do_periodic_work
sets only on a FIRED ALRAF (RTC->SR bit); an armed alarm cannot latch it, so
"armed misread as fired" is unlikely - the DIAG exit-cause print will confirm
on hardware.

pwrkey unittest Makefile: add the console stub (real pwrkey.c now prints).

TO BE REVERTED after the bench run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…exposed

Bench 2026-07-09 14:46 (diag build 93d307e), decoded print chain: btn-edge ->
pk press-begin (+500 ms) -> DIAG beep (silent!) -> PWR_OFF_REQ latched ->
"grace expired, re-enter Stop" MID-HOLD -> x4 abandoned presses -> exit
cause=ALARM (570 s early, irqf=3) -> post-resume release latched a new short
press -> hard off -> cold boot. Four distinct bugs:

1. GRACE WINDOW DROPPED HELD PRESSES (the wake-killer). The wait was measured
   from the EXTI edge, but a short press is only latched on RELEASE: any hold
   longer than ~1.2 s expired the grace and the EC went back to Stop in the
   middle of the press - the release (rising edge, EXTI armed falling-only)
   could never wake it. Fix: while the debounced level reads pressed, keep
   refreshing the grace timestamp - the EC stays awake for the whole hold and
   classifies on release. Mock gap that kept 444 tests green: previous tests
   held the button exactly DEBOUNCE+50 ms; and the sim kept ticking pwrkey
   "inside Stop", so a lost press still latched. The new test asserts the
   real property: no Stop re-entry while held.

2. BEEP PHYSICALLY IMPOSSIBLE DURING THE WINDOW. Schematic WB 8.5.3: the
   buzzer (SG1 MLT-7525, Q11 gate from PC7) is POWERED FROM THE 3.3V RAIL -
   which is off for the entire suspend window by definition. The PWM call was
   perfect (f=1000 d=300 sysclk=64) and silent. Fix: the classification only
   REQUESTS the beep (linux_cpu_pwr_seq_wake_beep_request); the wake sequence
   emits it at the first instant 3.3V is back while the SoC is still held in
   reset (DRAM in autonomous self-refresh, nothing executing - safe per the
   f3face8 lesson), then pulses PWROK after the beep (new PS_WAKE_BEEP_WAIT
   state, +~120 ms on button wakes only). Audible beep lands ~0.4 s after
   the release, ~1.5 s before Linux is back.

3. RELEASE RE-DELIVERED AFTER RESUME. A press that survives the window exit
   (alarm/deadline wake mid-hold - pk_lvl=1 in the log) releases after
   resume; the release-latch then hit the !linux_booted branch = immediate
   hard off + standby (the "poweroff 11 s after resume" - it never involved
   Linux at all). Fix: if the button is still held at the window exit, swallow
   pwrkey short-press events until the debounced level reads released. Long
   press (force-off) is consumed earlier in the loop and stays functional.
   Tested for both orderings (release before/after WORKING entry). Mock fix
   required: utest pwrkey_pressed() now returns the DEBOUNCED level when the
   debounce model is on (hardware semantics: level flip and short-press latch
   land in the same pass) - it returned the raw level before (mock gap).

4. PHANTOM ALARM (cause=ALARM 570 s early, irqf=3 = IRQ_ALARM|IRQ_PWR_OFF_REQ,
   i.e. the poller genuinely read ALRAF=1). Root cause not yet proven; prime
   suspect: the dead SoC's floating SPI lines clocking garbage regmap WRITES
   into the EC during the ~4 s of accumulated awake grace windows -> the
   RTC_ALARM region "changed" -> alarm re-programmed to a garbage time.
   Containment: disable the SPI2 NVIC line for the whole window
   (mcu_stop_window_prepare/finish); the first post-resume CS edge runs
   reset_and_init_spi(), so regmap comms self-heal. Plus a DIAG print in the
   poller fire branch (now= vs alrm= time) to adjudicate on the next run.
   With bug 1 fixed the exposure window shrinks to ~1.2 s per press anyway.

DIAG prints kept in for the hardware verification run; unittest Makefiles for
rtc-alarm-subsystem and linux-power-control gained console/buzzer stubs.
Fault-injection verified: bug 1 and bug 3 tests fail with their fixes removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… held

Final UX contract (Evgeny, 2026-07-09): "press - hear the beep - release".
On 65f3d72 the wake (and thus the beep) waited for the RELEASE; now the
classify treats a debounce-CONFIRMED press (+500 ms of hold after the EXTI0
edge that woke the EC) as the button wake cause. Timeline: press t0 ->
confirm +0.5 s -> wake sequence -> PWRON ~+0.6 s -> 3.3V back ~+0.8 s ->
BEEP ~+0.8 s with the finger still down -> PWROK -> SPL ~+1.1 s; the release
happens whenever and is swallowed.

- Classify: button = (stop_button_wake && pwrkey_pressed()) ||
  pwrkey_handle_short_press() - the level term is the confirmation, the latch
  term is a safety net for a release that completed inside the window.
- Brush filter unchanged: below 500 ms the level never confirms; the grace
  expires silently and the EC re-enters Stop.
- The release now ALWAYS lands post-wake: the exit arms the swallow whenever
  a press is in flight (level pressed OR the edge seen but not yet confirmed
  - the second case closes a real hole: an alarm/deadline exit during a
  just-started press would let its post-exit confirm/release hit the
  !linux_booted hard-off). Swallow clearance is debounce-aware: it lifts only
  after the level reads released for > PWRKEY_DEBOUNCE_MS + 200 ms
  continuously, so an in-flight-but-unconfirmed press cannot slip through the
  gap between the exit and its debounced confirmation.
- Very long hold semantics (stated per request): after the confirm-wake the
  EC stays awake; once the hold reaches the pwrkey long-press threshold (8 s)
  the normal running-system force-off takes over (5V cut + poweroff standby).
  Anything shorter than long-press delivers nothing to Linux.
- Alarm/deadline wakes stay silent; DIAG prints kept for one more hardware
  pass.

Tests (integration; fault-injection: reverting the classify to
release-latch-only fails all four button tests with "must wake BEFORE
release"): edge test asserts no wake below 500 ms, wake + beep while still
held, release delivers nothing; long-hold (2.5 s) wakes at confirm with no
Stop re-entry during the hold; very-long-hold reaches force-off; the
mid-hold alarm tests now model the only remaining form (alarm during an
unconfirmed press) for both release orderings. Test-model lessons fixed
along the way: the sleeping PMIC must stay crashed until PWRON revives it
(clearing pmic_crashed early lets the sim's PMIC self-recover and boot the
SoC before the confirm), and post-standby asserts must run immediately (the
mcu_goto_standby stub returns; the stub-world afterlife diverges from
hardware). The very-long-hold test must let the release debounce out in sim
time before injecting the long-press latch - the handler busy-waits on the
debounced level and sim time does not advance inside it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ross wakes

Bench 2026-07-09 17:33: a short press on a fully RESUMED system hard-cut the
power instead of requesting a graceful shutdown. Every suspend-to-off wake
routes through POWER_ON_SEQUENCE_WAIT, whose completion resets
linux_booted (correct for cold power-ons - Linux must boot for 20 s before
presses are forwarded to it). But a suspend wake RESUMES Linux, it does not
boot it: nothing re-announces, so for the first 20 s after every resume a
press landed in the "linux not booted" branch = immediate hard off + standby
(the DIAG chain shows short-press latched -> "Power off and go to standby
now" with no PWR_OFF_REQ line).

Fix: the suspend-window exit sets suspend_wake_resume; the power-on-sequence
completion computes linux_booted = linux_initial_powered_on ||
suspend_wake_resume and consumes the flag. Cold power-ons (poweroff-standby
wake, power application, watchdog hard resets, reboot requests) never set it,
so the legit "press before Linux is up = hard off" behavior is intact.
Corner case accepted: if a resume degrades into the 5V-reset escalation
(DRAM lost, Linux actually cold-boots), presses during that boot take the
graceful path instead of hard-off for up to 20 s.

Also dedupe the power-key IRQ (the doubled "PWR_OFF_REQ latched" at 17:34:
Linux acked the IRQ mid-hold and the level-based handler re-latched it - one
press delivered twice): IRQ_PWR_OFF_REQ now latches on the press EDGE
(debounced level transition), tracked across the swallow/booted/not-booted
branches so a swallowed tail cannot fabricate an edge.

Tests (integration; fault-injection: reverting the preservation fails the
resumed-press test on the graceful-IRQ assert):
- test_resumed_system_press_is_graceful_not_hard_off: suspend -> button wake
  -> resume -> new press => IRQ latched, 5V stays on, no restart.
- test_fresh_boot_press_before_linux_up_hard_offs: cold boot, press inside
  the 20 s window => immediate hard off (legit path preserved). Test-model
  note: the debounce baseline must settle out of UNINIT first - a press from
  UNINIT generates no events by design ("button held at power-on").
- test_running_press_single_irq_despite_ack_mid_hold: ack mid-hold does not
  re-latch for the same press; a new press latches a new IRQ.
- Existing cold-boot graceful test (press after 20 s) unchanged.

DIAG prints kept for one more hardware pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remove all "DIAG "-prefixed console instrumentation added for the
button-wake bench triage (93d307e and the follow-up traces interleaved
into 65f3d72, 06cf384, 21c9663):

- buzzer.c: beep-invocation trace (f/d/sysclk);
- pwrkey.c: press-begin / short-press-latched debounce verdicts;
- rtc-alarm-subsystem.c: ALRAF fire trace (now= vs alrm=);
- wbec.c: window-exit cause/clear dumps, btn-edge/grace/wut-tick
  heartbeat, PWR_OFF_REQ latch trace, WORKING-entry linux_booted and
  suppressed-beep traces;
- unittest Makefiles (pwrkey, rtc-alarm-subsystem): drop the console
  stubs that existed only for those prints.

All fixes and the operational prints (window-entry marker, wake/restart
PMIC, PWRON prompt, power-off lines) stay untouched. The comment citing
the silent-beep bench evidence now references the diagnostic build by
hash instead of the removed print text.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…w it

Bench (2026-07-09, 3 real failures): when the Stop-mode exit is instant
(the RTC alarm was already pending at window entry - exit cause=ALARM,
WUT ticks=0), the wake PWRON pulse fired ~0.1-0.2 s after BL31's SRAM blob
armed the PMIC sleep / dropped the rails. The AXP853T, still completing its
sleep-entry transition, swallowed the pulse: the SoC never powered up and
nothing was left to wake the PMIC. The board stayed dark until the EC
deadline ladder (warm reset, then a full power reset ~2 min later) - a
~4.5 min outage.

Two complementary hardening fixes on the suspend-wake PWRON path:

1. Minimum arming-to-pulse delay. Anchor the wake at stop_armed_ts - the
   systick captured when the Stop window is armed, which coincides with the
   first 3.3V drop, i.e. ~when BL31 put the PMIC to sleep. The power
   sequence holds the first PWRON until stop_armed_ts +
   WBEC_SUSPEND_WAKE_PWRON_MIN_DELAY_MS (500 ms) so the PMIC finishes
   entering sleep before it is poked. The delay is requested only for an
   instant exit (suspend_wut_ticks == 0): on a real sleep the Stop window
   already burned wall-clock and the transition is long done, and systick -
   frozen across Stop - would otherwise mis-measure the anchor. The wait
   happens in the awake POWER_ON_SEQUENCE_WAIT super-cycle, where the IWDG
   is fed by the main-loop watchdog_reload() just like the existing STEP1-3
   waits.

2. Verify-and-retry. After pulsing PWRON, poll the 3.3V rail; if it does not
   return within WBEC_SUSPEND_WAKE_PWRON_RETRY_MS (~1 s) the PMIC swallowed
   the pulse - press again, up to WBEC_SUSPEND_WAKE_PWRON_ATTEMPTS (3) total,
   logging "PWRON swallowed, retry N". Only after all attempts fail does the
   wake path fall through to the existing 5V-reset backstop (which fully
   power-cycles the PMIC) and, beyond that, the deadline escalation. The
   cold-boot PWRON fallback (STEP2/STEP3) is left at its 1.5 s cadence.

No extra beeps or button-UX changes: the resume-beep still keys off the
V33-return path (PS_WAKE_BEEP_WAIT) and the retries do not touch it. Normal
cold boot is untouched (the new logic is gated on wake_pending).

Tests: linux-power-control gains 4 unit tests (min-delay boundary,
immediate press without the request, retry-count/timing + 5V-reset
fall-through, retry that succeeds when 3.3V returns); wbec-integration
gains 2 end-to-end tests (instant-exit holds PWRON by the min delay and
still boots; every-PWRON-swallowed recovers via the 5V backstop). Full
suite 466 tests, all green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t swallow it"

This reverts commit 70eb9c9c together with its bench-tested successor
(042021a, never pushed): both made the wake path worse on silicon.

Bench findings (2026-07-10, zero-dwell wake hammering):
- 70eb9c9's 500 ms arming-to-pulse delay engaged (492 ms visible on the
  console) but did not reduce the swallow rate: 2 swallows / 21 cycles
  vs ~3/25 baseline. The V33-absent retry never fired because a
  swallowed press leaves a HALF-WOKEN PMIC: rails restored by the POK
  falling edge, press discarded by the ONLEVEL debounce, V33 present,
  SoC cores never started (warm reset then fails, only the second,
  power-cycle timeout recovers).
- 042021a's full-width 200 ms press fixed the swallow but introduced a
  DOUBLE-START: the edge-started boot races the press-commit; when the
  debounce completes with POK still held, the PMIC re-sequences the
  already-running SoC ~300 ms after the resume jump. The cleared resume
  vector correctly fail-safes into a cold boot, so the resume (and the
  user's session) is lost — strictly worse than the ~1/15 swallow it
  was fixing.

The swallow is a zero-dwell-only phenomenon (alarm expiring during
suspend entry -> instant Stop-exit -> PWRON racing the PMIC sleep
transition); settled wakes never swallowed in 60+ bench cycles. Proper
fix needs scope characterization of POK/PWROK/rail timing around the
sleep transition, not another blind pulse-shape iteration. Mitigation
candidates recorded in the plan: shorter first watchdog timeout after a
suspend wake (cuts worst-case dark time 4.5 min -> ~30 s), and the
existing EC deadline ladder already self-recovers every case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@evgeny-boger evgeny-boger marked this pull request as ready for review July 10, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant