Skip to content

Systematic apparent-age screening suggests the under-18 filtering (paper §3.1) has measurable false-negative leakage #37

Description

@kaanyor

Hi maintainers,

We're building a camera app with a strict internal policy against processing images of minors, and EasyPortrait is one of our aggregate-statistics sources (photometric/framing bands only — no image redistribution, no identity embeddings computed or stored). Before relying on the dataset we wanted to independently verify the paper's stated filtering (§3.1: "checking for children under 18, naked people, religious signs, and watermarks").

What we did: apparent-age visual screening (Claude-based multi-perspective review — an adversarial "try to refute under-18" pass plus an independent anatomical/facial-maturity pass) over the full locally-available train+val pool (34,000 images / 7,866 unique user_ids from meta.tsv). This is a full census of that pool, not a sample.

Headline finding: roughly 9% of individuals were independently assessed (2+ reviewers, high confidence) as more likely under 18 than over, plus a further ~18% we couldn't confidently place at 18+ either way. A handful of representative examples (age range = independent reviewer estimates, not a claim of certainty):

image_name reviewer-estimated age range confidence
ce08b4ce-88b0-4e7d-9af9-5dc1ad803706 early-mid adolescent high (this is the case that originally triggered our review)
91dbdf31-32cd-4c8e-b212-b29b0f3411f6 ~13-17 high
ebdb8c2b-6b99-4184-9e31-ba4c34b9b16f ~6-9 high
58490a0b-3dc1-447a-88f6-aa41bcee1cdf ~10-14 high

Separate data-integrity finding: in about 20 accounts, the frames attributed to one user_id clearly depicted different individuals — in a few cases a young child alongside an adult in the same account's photo set (e.g. f976f194-24a7-4886-b4b4-e2ab4662d7ae, one of several frames under a single user_id where the other frames show a clearly older individual). This lines up with what @hukenovs noted in #32 — that some workers may have uploaded photos that weren't of themselves — and it means per-account rather than per-image filtering can miss a minor mixed into an otherwise-adult account.

We're not sharing the full confirmed list here (id's tied to real minors, felt wrong to post in a public thread), but we're happy to send it privately — the full census, user_id-grouped, with our confidence tier per person — if that's useful for re-running your filter or auditing the crowdsourcing pipeline. Just let us know a preferred channel.

Really appreciate the dataset and the consent-oriented design intent — filing this in that spirit, not as a complaint.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions