Skip to content

Replace demo benchmarks with captured synchronization and performance measurements - #1363

Merged
sargunv merged 13 commits into
mainfrom
sargunv/compose-map-desync
Sep 9, 2026
Merged

Replace demo benchmarks with captured synchronization and performance measurements#1363
sargunv merged 13 commits into
mainfrom
sargunv/compose-map-desync

Conversation

@sargunv

@sargunv sargunv commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Description

Replace the demo's projection-callback latency and Compose-frame drop estimates with repeatable camera animation, camera setter, and input-response scenarios. The same scene runs from the demo panel or capture runner, using local map content and a production placedAt overlay, with configurable Android presentation, FPS cap, and geometry load. Capture launches bypass normal demo state creation.

The runner retains raw artifacts and measures visible map/overlay separation. Android adds scheduled app CPU time, window GPU duration and missed deadlines, available driver GPU work periods, FrameTimeline events, and input-to-captured-display latency bounds. Desktop and iOS add process CPU counters, including runs without video. Chromium currently supports visual measurements only. Unavailable GPU/presentation data is not reported as zero; captured-display latency excludes the physical touchscreen and panel.

Performance-only results require a matching artifact/configuration/device visual capture whose raw pixels pass validation. Capture and lifecycle timeouts cover the full allowed startup, workload, and shutdown. Delete the previous benchmark workloads, tile-prefetch machinery, and timing estimators. Keep synthetic measurement tests in hygiene CI; the demo does not gain a unit-test suite.

Renderer mitigations stay outside this PR. This establishes their measurement harness for #407; selected designs are recorded in #1366, #1367, and #1368.

Validation

  • Eighteen Python analyzer tests cover known pixel offsets, latency bounds, sparse input recordings, strict Android capture clocks, incomplete metric logs, rejection of mismatched or stationary visual references, frame-interval filtering, desktop artifact identity, low-FPS captures, external URLs, and Android animation-scale validation.
  • Static checks and Android release build. A release capture verified 727 window frames within the trace interval from 795 collected reports, with none dropped. Android API 36 emulator capture with a deliberate 14-second launch delay retained the complete measurement interval; matched performance-only capture and reanalysis verified reference handling.
  • Earlier harness validation included Android Lint, iOS 26.5 simulator, macOS Metal desktop, and Chromium/WebGL builds and captures, plus CPU-only runs on Android, desktop, and iOS. Forty matched mitigation/control captures used one verified artifact per platform with runtime experiment flags; those prototype changes are not included here, and comparisons need replication after the capture-launch isolation correction.
  • No physical-phone GPU or touch-to-photon validation. GPU/deadline metrics and calibrated input capture remain Android-specific.

AI assistance

Implemented and investigated with OpenAI Codex (GPT-6).

@sargunv
sargunv marked this pull request as ready for review September 9, 2026 00:57
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-09T03:21:28.351590Z c54d098 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@sargunv
sargunv temporarily deployed to cloudflare-pages September 9, 2026 01:01 — with GitHub Actions Inactive
@sargunv
sargunv temporarily deployed to cloudflare-pages September 9, 2026 01:11 — with GitHub Actions Inactive

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1fd056f4a0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread benchmarks/android.pbtxt Outdated
Comment thread demo-app/common/src/commonMain/kotlin/org/maplibre/compose/demoapp/DemoApp.kt Outdated
Comment thread benchmarks/run.py Outdated
@sargunv
sargunv temporarily deployed to cloudflare-pages September 9, 2026 01:20 — with GitHub Actions Inactive
@sargunv
sargunv temporarily deployed to cloudflare-pages September 9, 2026 01:30 — with GitHub Actions Inactive

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d4804c66a0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread benchmarks/run.py
@sargunv
sargunv enabled auto-merge (squash) September 9, 2026 02:27

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 131b67315a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread benchmarks/run.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2eaf82badc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread benchmarks/run.py
Comment thread benchmarks/analyze.py Outdated
Comment thread benchmarks/performance.py Outdated
@sargunv
sargunv merged commit 3272bd9 into main Sep 9, 2026
18 checks passed
@sargunv
sargunv deleted the sargunv/compose-map-desync branch September 9, 2026 04:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant