METADATA: CUSTOM_PROPERTY scores HIGH 100 on the substring "confidential", so a Purview sensitivity label outranks a real secret

[sectoramen]

A custom document property is typed and scored by two mechanisms that have nothing to do with custom-property risk, and the purpose-built analyzer for it never runs.

What happens

inferMetadataType assigns CUSTOM_PROPERTY on the Custom_ name prefix alone (internal/validators/metadata/metadata_validator.go:1366). The confidence then comes from the generic metadata scorer CalculateConfidence, whose containsEnhancedCopyright check matches a bare substring anywhere in the line (metadata_validator.go:1596-1612):

copyrightPatterns := []string{
    "copyright", "©", "(c)", "all rights reserved", "proprietary",
    "confidential", "trade secret", "trademark", "patent",
}

A hit adds +0.4 (metadata_validator.go:396-399), which saturates the score at 100.

So the value Confidential — the standard Microsoft Purview / MSIP sensitivity label carried by a large fraction of enterprise documents — promotes the property to HIGH 100. That is the document stating its own classification marking, not a disclosure of sensitive data.

Measured on synthetic fixtures (one custom property each, identical benign body text):

custom property	confidence	validation_checks
MSIP_Label_<guid>_Name: Confidential	100 (HIGH)	contains_enhanced_copyright
Notice: Proprietary	100 (HIGH)	contains_enhanced_copyright
DocumentPurpose: Quarterly summary	60 (MEDIUM floor)	none

Note (c) is also in that list as a three-character substring.

Why this matters

The confidence contract puts HIGH at >=90. A classification label reaching 100 sits above genuinely disclosed values: on a local corpus of 304 .docx files, CUSTOM_PROPERTY produced 292 HIGH findings out of 2,236 METADATA findings total — the single largest HIGH population in the run, ahead of AUTHOR_INFO (13 HIGH). HIGH is the band operators triage first, so the labels crowd out the findings that matter.

The analyzer that was written for this is dead on the live path

analyzeCustomPropertyRisk (metadata_validator.go:3065) already tiers custom properties properly — CRITICAL for classification, HIGH for PII/financial, MEDIUM for organizational — and stamps custom_prop_risk_level, custom_prop_name and custom_prop_risk_factors into the match metadata.

None of those three keys appear on any of the 506 CUSTOM_PROPERTY findings in the corpus run. The emitter that fires is a different code path, so the tiering never applies. This is the same dual-path split recorded in #251, which fixed the value-shape half and left the custom-property half on the generic scorer.

Suggested direction

1. Route CUSTOM_PROPERTY scoring through analyzeCustomPropertyRisk so the tiering is load-bearing, and assert the custom_prop_* metadata keys are present — their absence is what makes this silently regress.
2. Do not treat a classification label as a disclosure. A property whose value is only a label belongs in a distinct low-severity type, or should be demoted rather than vetoed (cf. the reserved-example-value reasoning).
3. Word-bound the copyright patterns, and drop or gate (c).

Also, while reading: metadata_validator.go:1366-1377 contains the same if strings.HasPrefix(lineLower, "custom_") { return "CUSTOM_PROPERTY" } block twice, with an identical comment. The second copy is unreachable.

No customer content is quoted here; corpus figures are aggregate counts only and every value shown above is from a synthetic fixture.

[sectoramen]

Fixed in #319, which also carries #308 because the dedup changes the numbers this was measured against.

Two corrections to this issue, since I wrote it and it was wrong in one place:

analyzeCustomPropertyRisk is NOT dead on the live path. I inferred that from the custom_prop_* metadata keys being absent from all 506 findings. They are absent because the formatter has a metadata allowlist (internal/formatters/shared/structures.go) and custom_property_* is not on it, so the keys are stripped from output. The analyzer runs and its name-based tiering works — verified by holding the value constant and varying only the name:

Custom_DocumentPurpose: Quarterly summary  ->  60
Custom_ProjectCode:     Quarterly summary  ->  90
Custom_Classification:  Quarterly summary  -> 100

The real defect is narrower and sharper than "scores HIGH on a substring": the same classification word was charged by TWO scorers — the generic containsEnhancedCopyright (+0.4 on a bare substring) and the analyzer's own classification branch (+0.5). The consequence is an ordering failure, not just an inflated number:

Custom_Notice: Confidential                                 -> 100
Custom_Notice: Confidential - Project Nightjar acquisition  -> 100

A document that merely carries a Purview label ranked identically to one carrying the label and naming a live acquisition project.

Also worth recording: the word-bounding of the copyright patterns that this issue asked for was not done, and deliberately. containsEnhancedCopyright is shared by every metadata type, so changing it has blast radius beyond custom properties — for an image's Copyright field the IP heuristic is the correct signal. Withdrawing its contribution for CUSTOM_PROPERTY only achieves the same result with no risk to other types, and a test asserts the heuristic still fires elsewhere. If the (c) three-character pattern is worth tightening on its own merits, that deserves its own change with its own precision measurement.

The duplicated unreachable custom_ block at metadata_validator.go:1366-1377 is still there — it is genuinely dead (the first copy returns), so it is cosmetic and I left it out of a behaviour change.

Remaining limitation, tracked as #320: a marking dressed with harmless decoration (Confidential - Draft, Confidential FY25) still reaches 100.