feat(Segment Membership): Skip traitless identities when seeding - #8455
feat(Segment Membership): Skip traitless identities when seeding#8455matthewelwell wants to merge 3 commits into
Conversation
An identity carrying no traits at all can only match a segment by percentage split or `is not set`, and some environments hold millions of them. Leave them out of the ClickHouse mirror, matching the Edge CDC filter that now drops them at the stream. System traits count as traits here, so cohort members are kept. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Team Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review. 📝 WalkthroughWalkthroughThe identity seeding task excludes identities with neither identity traits nor system traits. It skips empty batches before inserting data into ClickHouse. Unit tests cover traitless identities and identities with cohort membership in Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to Identity seeding now omits traitless identities while retaining identities needed for cohort membership, with regression coverage for empty batches. No current merge-blocking risk is identified. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Docker builds report
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Team
Run ID: 85a2210c-9809-44c8-8865-117e9e467499
📒 Files selected for processing (2)
api/segment_membership/tasks.pyapi/tests/unit/segment_membership/test_unit_segment_membership_tasks.py
Included review availability: Your plan provides up to 8 included reviews per hour; 7 remain after this review.
❌ oss · depot-ubuntu-latest-16 — run #20135 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Failed testsfirefox › tests/environment-test.pw.ts › Environment Tests › Environments can be created, renamed, and deleted @oss 🗂️ Previous results❌ private-cloud · depot-ubuntu-latest-16 — run #20135 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-16)Details
✅ private-cloud · depot-ubuntu-latest-arm-16 — run #20126 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)
Details
✅ private-cloud · depot-ubuntu-latest-16 — run #20126 (attempt 1)Playwright Test Results (private-cloud - depot-ubuntu-latest-16)
Details
✅ oss · depot-ubuntu-latest-arm-16 — run #20126 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
✅ oss · depot-ubuntu-latest-arm-16 — run #20128 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-arm-16)
Details
✅ oss · depot-ubuntu-latest-16 — run #20126 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
✅ oss · depot-ubuntu-latest-16 — run #20128 (attempt 1)Playwright Test Results (oss - depot-ubuntu-latest-16)
Details
Skipped testsfirefox › tests/onboarding-tests.pw.ts › Onboarding › New user connects via the single-page onboarding flow @oss |
Visual Regression19 screenshots compared. See report for details. |
One identity per batch, so the traitless one forms a batch with nothing left to write, and assert no INSERT is issued with an empty row list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
github-actions
Bot
added
feature
1 participant
docs/if required so people know about the feature.Changes
Contributes to https://github.com/Flagsmith/edge-api/pull/714
seed_organisation_identitiesmirrors every identity in an environment, including ones carrying no traits at all. Those can only match a segment by percentage split oris not set, and some environments accumulate them in the millions — one org's were expensive enough that we excluded their environment from the Edge CDC path outright. The edge-api PR above narrows that exclusion to just the traitless records, and this does the same for the backfill, so a seed doesn't load into ClickHouse what CDC is now filtering out.It also decouples the two: that org's empty identities are being deleted by a long-running script, and without this, seeding them has to wait for it to finish.
The
system_traitshalf of the check keeps cohort members, whose membership is the only thing on their document. Note the seed mapper doesn't readsystem_traitsyet (edge-api's CDC mapper does, since Flagsmith/edge-api#707), so those identities are still mirrored withtraits = NULLuntil that's fixed separately — this change just makes sure they aren't dropped entirely in the meantime.How did you test this code?
Added
test_seed_organisation_identities__traitless_identities__are_not_mirrored, covering an identity with no traits (dropped) and one carrying only a system trait (kept).Ran
tests/unit/segment_membership— 48 passed. The ClickHouse-marked tests were deselected; I don't have a local ClickHouse to run them against.🤖 Generated with Claude Code