feat(experimentation): add conversion-over-time rows to experiment results - #8451
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
github-actions
Bot
added
api
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Team Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 8 included reviews per hour; 4 remain after this review. 📝 WalkthroughWalkthroughThe experimentation results API adds conversion bucket and conversion time-series dataclasses. It builds ClickHouse queries for first post-exposure conversions on occurrence metrics. The service retrieves exposure and conversion buckets, groups them by time bucket and variant, and attaches chart data to result summaries. Tests cover query generation, row decoding, warehouse access, serialisation, and summary computation. Observability catalogue references are updated for moved service lines. Estimated code review effort: 3 (Moderate) | ~30 minutes Merge Risk: ⚪ Minimal · up to This change documents exposure time-series behavior for conversion analytics. No current merge-blocking risk remains. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
feature
Docker builds report
|
github-actions
Bot
added
feature
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8451 +/- ##
========================================
Coverage 98.81% 98.81%
========================================
Files 1621 1621
Lines 66138 66277 +139
========================================
+ Hits 65354 65493 +139
Misses 784 784 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
✅ private-cloud · depot-ubuntu-latest-16 — run #20111 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
🗂️ Previous results✅ private-cloud · depot-ubuntu-latest-arm-16 — run #20111 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
✅ oss · depot-ubuntu-latest-arm-16 — run #20111 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
✅ oss · depot-ubuntu-latest-16 — run #20111 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
✅ private-cloud · depot-ubuntu-latest-arm-16 — run #20109 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
✅ private-cloud · depot-ubuntu-latest-16 — run #20109 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
✅ private-cloud · depot-ubuntu-latest-16 — run #20107 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
✅ private-cloud · depot-ubuntu-latest-arm-16 — run #20107 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
✅ oss · depot-ubuntu-latest-arm-16 — run #20109 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
✅ oss · depot-ubuntu-latest-arm-16 — run #20107 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
✅ oss · depot-ubuntu-latest-16 — run #20107 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
✅ oss · depot-ubuntu-latest-16 — run #20109 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
|
Visual Regression19 screenshots compared. See report for details. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Team
Run ID: 1bbea4f2-7787-4678-9819-534f721ae992
📒 Files selected for processing (6)
api/experimentation/dataclasses.pyapi/experimentation/results_query.pyapi/experimentation/services.pyapi/tests/unit/experimentation/test_models.pyapi/tests/unit/experimentation/test_services.pydocs/docs/deployment-self-hosting/observability/_events-catalogue.md
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.
… conversions by metric
github-actions
Bot
added
docs
github-actions
Bot
added
docs
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Team
Run ID: c12e7b7d-0959-420e-906f-bec1f5b92c2f
📒 Files selected for processing (7)
api/experimentation/dataclasses.pyapi/experimentation/results_query.pyapi/experimentation/services.pyapi/tests/unit/experimentation/test_models.pyapi/tests/unit/experimentation/test_services.pyapi/tests/unit/experimentation/test_tasks.pydocs/docs/deployment-self-hosting/observability/_events-catalogue.md
Included review availability: Your plan provides up to 8 included reviews per hour; 6 remain after this review.
github-actions
Bot
added
docs
Zaimwa9
left a comment
Zaimwa9
left a comment
There was a problem hiding this comment.
Approving as I have non-blocking comments.
I wonder if we really need the hourly granularity though or if we are ok to tease with "incoming results" if the experiment is younger than 72h
2 participants
Thanks for submitting a PR! Please check the boxes below:
docs/if required so people know about the feature.Changes
Contributes to the conversion-rate-over-time chart on the experiment results page (backend half; frontend follows).
The results payload only carried headline statistics, so the dashboard had nothing to draw a per-variant conversion rate over time from.
ResultsSummary.exposures_timeseries: per-bucket newly exposed identities per variant, computed in the same run as the metrics so both sides of the rate share oneas_of.MetricResult.conversions_timeseries: per-bucket identities whose first post-exposure conversion landed in that bucket, per variant. Occurrence metrics only;nullfor value metrics.get_results_aggregates, runs the results, conversions and exposures reads with a singleResultsQueryBuilder. The conversions query reuses the exposures CTE and each slot's attribution condition, so bucket totals equal the metric'ssum, and joins only the charted metrics' events.How did you test this code?
Unit tests for the query builder, bucket decoding and summary assembly. Ran the gatherer against a local ClickHouse with seeded events: pre-exposure conversions ignored, duplicate events counted once, quarantined identities excluded, and day/hour bucket totals match each metric's
sum.