fix: EPSS source precision, reconciling stats totals, known-issues fetch for migrated KB pages

CLAUDE.md

@@ -29,10 +29,10 @@ On Windows in this repo use `.venv/Scripts/python -m pytest` etc. — the venv w
 - `tools/formatters.py` — optional `format="markdown"|"csv"` triage renderings; JSON is always included and unchanged.
 - `tools/profiles.py` — named product watchlists for `msrc_search` (`product_profile=`, plus ad-hoc `products=[...]`/`product_families=[...]`). Built-ins (`identity-core`, `endpoint`, `server-infrastructure`) merged under a `MSRC_PROFILES_PATH` JSON override with strict validation; unknown/invalid → `invalid_input`, never an unscoped fallback. Matching is local substring (union: any product OR family); profile contents are never sent upstream or to telemetry.
 - `tools/prompts.py` — the `monthly_triage` MCP prompt (registered in `server.py` via `mcp.prompt`), a guided analyst workflow built entirely on `msrc_search`; optional `product_profile`/`month` args. Portable copies live in `prompts/` (plain-text prompt) and `skills/patch-tuesday-triage/` (agent skill) — keep them in sync with `tools/prompts.py` when the workflow changes.
-- `feeds/http_client.py` — shared httpx client: `follow_redirects=False` (hardcoded hosts; redirects are never followed automatically), `get_bounded()` which streams responses with a byte cap instead of buffering unbounded bodies, and `get_location()` which returns a redirect's status+Location without reading the body so a caller can follow one hop manually after validating the target.
+- `feeds/http_client.py` — shared httpx client: `follow_redirects=False` (hardcoded hosts; redirects are never followed automatically), `get_bounded()` which streams responses with a byte cap instead of buffering unbounded bodies, and `get_location()` which returns a redirect's status+Location without reading the body so a caller can follow hops manually after validating each target.
 - `feeds/msrc_api.py` — MSRC index + monthly CVRF fetch with in-process TTL caches (`MAX_FULL_MONTHS_CACHED = 12` — matches `MAX_TREND_MONTHS` so a max trend doesn't evict its own months; 40 slim), LRU eviction (hits refresh recency), per-month asyncio locks, `FETCH_CONCURRENCY = 3` semaphore, `force_refresh` bypass, freshness metadata. Response bodies capped via `MCP_MSRC_MAX_RESPONSE_BYTES` (64 MiB default).
 - `feeds/enrichment.py` — KEV catalog + EPSS fetches (batches of 100, fetched concurrently with `EPSS_FETCH_CONCURRENCY = 3`), cached; EPSS cache capped at `MAX_EPSS_CACHE_ENTRIES = 50_000`; failures return empty ({}) — enrichment must never break a search. Bodies capped via `MCP_ENRICHMENT_MAX_RESPONSE_BYTES` (32 MiB default).
-- `feeds/known_issues.py` — Microsoft-confirmed known issues per KB (`include_known_issues=True`), scraped best-effort from the public support.microsoft.com KB page (no keyless API exists; the Graph windowsupdates API needs AAD). Honest three-way status: `published` / `none_published` / `unavailable` — a fetch/parse failure is never reported as "none". Coverage boundary: known-issues sections exist mainly for Windows OS cumulative/preview updates; Office/SharePoint/SQL/.NET pages usually have none → `none_published`. Source quirks handled: `/help/{kb}` answers one same-host redirect (followed manually via `get_location()` after validating the target host), and bogus KB numbers redirect to *unrelated* articles that still echo the requested id in an analytics meta tag, so a landing page is only trusted if its URL slug or title names the requested KB (pages that never self-reference, like some .NET update pages, conservatively report no per-KB page). TTL cache 6 h (`MAX_CACHE_ENTRIES = 500`, LRU; failures aren't cached), `FETCH_CONCURRENCY = 3` semaphore shared by batch prefetch, bodies capped via `MCP_KNOWN_ISSUES_MAX_RESPONSE_BYTES` (4 MiB default). Parser is stdlib-only (regex slicing + `html.parser`); on layout drift it degrades to `unavailable` with the source URL — the `--run-live` smoke test is the drift canary.
+- `feeds/known_issues.py` — Microsoft-confirmed known issues per KB (`include_known_issues=True`), scraped best-effort from the public support.microsoft.com KB page (no keyless API exists; the Graph windowsupdates API needs AAD). Honest three-way status: `published` / `none_published` / `unavailable` — a fetch/parse failure is never reported as "none". Coverage boundary: known-issues sections exist mainly for Windows OS cumulative/preview updates; Office/SharePoint/SQL/.NET pages usually have none → `none_published`. Source quirks handled: `/help/{kb}` answers a short same-host redirect chain (bounded at `MAX_REDIRECT_HOPS = 3`; each hop followed manually via `get_location()` after validating the target host — the 2026 site migration added a second hop to `/servicing/os/...`), two markup generations are parsed (legacy `ocpSection` divs with `<b class="ocpLegacyBold">` labels, and the newer `<details>/<summary>` blocks with `<strong>` labels under an `<h3>` heading), and bogus KB numbers redirect to *unrelated* articles that still echo the requested id in an analytics meta tag, so a landing page is only trusted if its URL slug or title names the requested KB (pages that never self-reference, like some .NET update pages, conservatively report no per-KB page). TTL cache 6 h (`MAX_CACHE_ENTRIES = 500`, LRU; failures aren't cached), `FETCH_CONCURRENCY = 3` semaphore shared by batch prefetch, bodies capped via `MCP_KNOWN_ISSUES_MAX_RESPONSE_BYTES` (4 MiB default). Parser is stdlib-only (regex slicing + `html.parser`); on layout drift it degrades to `unavailable` with the source URL — the `--run-live` smoke test is the drift canary.
 - `models/vulnerability.py` — CVRF parsing into `Vulnerability`; numeric CVRF enums are documented constants (remediation types: 0=workaround, 1=mitigation, 2=vendor fix/KB, 4=will-not-fix). `to_summary_dict()` vs `to_detail_dict()` control output size; new fields are opt-in flags (`include_references`, `include_kb_details`, `include_kev_details`, `include_temporal`, filter-triggered `cwe`/`exploitation_assessment`).
 - `models/cvss.py` — lenient CVSS v3.x vector parser; fails open to `None`, never raises.
 - `middleware/` — per-IP token-bucket rate limit and request body cap, both with telemetry callbacks (`on_request`/`on_throttled`, `on_rejected`). X-Forwarded-For is honored only from private/loopback peers or `MCP_TRUSTED_PROXIES` members — a public direct peer can never forge it.

src/patch_tuesday_mcp/feeds/enrichment.py

@@ -187,8 +187,11 @@ async def fetch_batch(batch: list[str]) -> tuple[list[str], dict | None]:
             for entry in data.get("data", []):
                 cve = entry.get("cve")
                 try:
-                    score = float(entry["epss"])
-                    percentile = float(entry["percentile"])
+                    # EPSS publishes decimal strings at 5-significant-decimal
+                    # precision; round once here so no downstream view can
+                    # surface float-repr digits beyond what the source states.
+                    score = round(float(entry["epss"]), 5)
+                    percentile = round(float(entry["percentile"]), 5)
                 except (KeyError, TypeError, ValueError):
                     continue
                 if cve:

src/patch_tuesday_mcp/feeds/known_issues.py

@@ -16,12 +16,18 @@
   masquerade as "no known issues".
 
 Fetch failures never raise into a search; ``fetch_known_issues`` always
-returns a dict. Two quirks of the source shape this module: the /help/<kb>
-URL answers with a single same-host redirect to the canonical article (we
-follow exactly one validated hop; the shared client never follows redirects
-on its own), and unknown KB numbers redirect to *unrelated* articles — which
+returns a dict. Quirks of the source that shape this module: the /help/<kb>
+URL answers with a short chain of same-host redirects to the canonical
+article (historically one hop to /topic/...; the 2026 site migration added a
+second hop to /servicing/os/...), so we follow a bounded number of hops,
+validating each target ourselves — the shared client never follows redirects
+on its own. Unknown KB numbers redirect to *unrelated* articles — which
 still echo the requested id in an analytics meta tag — so a landing page is
-only trusted when its URL slug or its title names the requested KB.
+only trusted when its URL slug or its title names the requested KB. Two
+markup generations coexist: legacy pages use ``ocpSection``/``ocpExpando``
+divs with ``<b class="ocpLegacyBold">`` segment labels, while the newer
+/servicing/ pages use ``<details>``/``<summary>`` blocks with ``<strong>``
+labels under an ``<h3>`` heading; both are parsed.
 """
 
 import asyncio
@@ -46,6 +52,7 @@
 KNOWN_ISSUES_TTL_SECONDS = 6 * 3600
 MAX_CACHE_ENTRIES = 500  # parsed results are ~1-4 KB, so worst case ~2 MB
 FETCH_CONCURRENCY = 3
+MAX_REDIRECT_HOPS = 3  # /help -> /topic -> /servicing today; one spare
 
 # A known-issues section with no parseable issue entries is an explicit
 # "none" statement when its prose is short; anything longer suggests issue
@@ -93,23 +100,36 @@ def _elapsed_ms(start: float) -> float:
 
 # --- HTML parsing (stdlib only; the source publishes no structured format) ---
 
-# The section heading as observed on Windows/.NET update pages; tolerate the
-# small wording variants Microsoft has used over the years.
+# The section heading across both markup generations: legacy pages carry the
+# text directly in an <h2>, the newer /servicing/ pages wrap it in inline
+# tags (e.g. <strong>) under an <h3>. Tolerate the small wording variants
+# Microsoft has used over the years.
 _HEADING_RE = re.compile(
-    r"<h2[^>]*>[^<]*known issues (?:in|with) this (?:security )?update", re.IGNORECASE
+    r"<h(?P<level>[23])[^>]*>\s*(?:<[a-z][^>]*>\s*)*[^<]*"
+    r"known issues (?:in|with) this (?:security )?update",
+    re.IGNORECASE,
 )
-# The known-issues section ends where the next top-level section or heading
-# starts; issue entries inside it are div-based, never <section>.
+# Legacy layout: the section ends at the next top-level section or <h2>;
+# issue entries inside it are div-based, never <section>, and legacy issue
+# bodies may legitimately contain <h3> tags.
 _REGION_END_RE = re.compile(r"<h2[\s>]|<section\s", re.IGNORECASE)
+# /servicing/ layout: the <h3> section ends at the next heading of either
+# level (the page repeats the section under a second <h3> for the combined
+# SSU+LCU package — only the first is parsed).
+_REGION_END_NEW_RE = re.compile(r"<h[23][\s>]", re.IGNORECASE)
 _ISSUE_BLOCK_RE = re.compile(r'<div class="ocpSection">', re.IGNORECASE)
 _ISSUE_TITLE_RE = re.compile(
     r'class="ocpExpandoHeadTitleContainer"[^>]*>(?P<title>.*?)</div>', re.DOTALL
 )
 _ISSUE_BODY_RE = re.compile(r'class="ocpExpandoBody"[^>]*>', re.IGNORECASE)
-# Bold paragraph labels segmenting an issue body.
+# /servicing/ layout: one collapsible <details> element per issue.
+_DETAILS_RE = re.compile(r"<details[\s>].*?</details>", re.IGNORECASE | re.DOTALL)
+_SUMMARY_RE = re.compile(r"<summary[^>]*>(?P<title>.*?)</summary>", re.IGNORECASE | re.DOTALL)
+# Bold paragraph labels segmenting an issue body (legacy <b>, servicing <strong>).
 _MARKER_RE = re.compile(
-    r'<b class="ocpLegacyBold">[^<]*?(?P<label>symptoms?|workaround|next steps?|resolution)'
-    r"[^<]*?</b>",
+    r'<(?:b class="ocpLegacyBold"|strong)[^>]*>[^<]*?'
+    r"(?P<label>symptoms?|workaround|next steps?|resolution)"
+    r"[^<]*?</(?:b|strong)>",
     re.IGNORECASE,
 )
 _RESOLVED_BY_RE = re.compile(
@@ -168,7 +188,17 @@ def _segment_issue_body(body_html: str) -> dict[str, str]:
     return {field: " ".join(parts) for field, parts in segments.items()}
 
 
+def _attach_resolved_by(issue: dict) -> dict:
+    resolved = _RESOLVED_BY_RE.search(
+        f"{issue.get('workaround', '')} {issue.get('resolution', '')}"
+    )
+    if resolved:
+        issue["resolved_by"] = f"KB{resolved.group(1)}"
+    return issue
+
+
 def _parse_issue_block(block_html: str) -> dict | None:
+    """Parse a legacy ocpSection issue block."""
     title_match = _ISSUE_TITLE_RE.search(block_html)
     if not title_match:
         return None
@@ -180,13 +210,21 @@ def _parse_issue_block(block_html: str) -> dict | None:
     body_match = _ISSUE_BODY_RE.search(block_html)
     if body_match:
         issue.update(_segment_issue_body(block_html[body_match.end() :]))
+    return _attach_resolved_by(issue)
 
-    resolved = _RESOLVED_BY_RE.search(
-        f"{issue.get('workaround', '')} {issue.get('resolution', '')}"
-    )
-    if resolved:
-        issue["resolved_by"] = f"KB{resolved.group(1)}"
-    return issue
+
+def _parse_details_block(block_html: str) -> dict | None:
+    """Parse a /servicing/-layout <details> issue block."""
+    title_match = _SUMMARY_RE.search(block_html)
+    if not title_match:
+        return None
+    title = _text(title_match.group("title"))
+    if not title:
+        return None
+
+    issue: dict = {"title": title}
+    issue.update(_segment_issue_body(block_html[title_match.end() :]))
+    return _attach_resolved_by(issue)
 
 
 def _parse_known_issues(html: str, kb_number: str, source_url: str) -> dict:
@@ -207,14 +245,20 @@ def _parse_known_issues(html: str, kb_number: str, source_url: str) -> dict:
         }
 
     tail = html[heading.end() :]
-    region_end = _REGION_END_RE.search(tail)
+    end_re = _REGION_END_RE if heading.group("level") == "2" else _REGION_END_NEW_RE
+    region_end = end_re.search(tail)
     region = tail[: region_end.start()] if region_end else tail
 
     issues = []
     for block in _ISSUE_BLOCK_RE.split(region)[1:]:
         issue = _parse_issue_block(block)
         if issue:
             issues.append(issue)
+    if not issues:
+        for match in _DETAILS_RE.finditer(region):
+            issue = _parse_details_block(match.group(0))
+            if issue:
+                issues.append(issue)
 
     if issues:
         return {"status": "published", "issues": issues, "source_url": source_url}
@@ -256,23 +300,32 @@ def _resolve_redirect(location: str | None) -> str:
 
 
 async def _fetch_kb_page(kb_number: str) -> tuple[str | None, str | None]:
-    """Fetch the support page for a KB, following one validated redirect hop.
-
-    Returns (final_url, html), or (None, None) when Microsoft has no page for
-    this KB. Raises KnownIssuesError for anything that is a retrieval failure
-    rather than a definitive absence.
+    """Fetch the support page for a KB, following bounded validated redirects.
+
+    The /help/{kb} resolver answers with a short same-host redirect chain to
+    the canonical article (currently two hops: /topic/... then
+    /servicing/os/...); each hop's target is validated before following and
+    the hop count is capped at MAX_REDIRECT_HOPS. Returns (final_url, html),
+    or (None, None) when Microsoft has no page for this KB. Raises
+    KnownIssuesError for anything that is a retrieval failure rather than a
+    definitive absence.
     """
-    url = SUPPORT_KB_URL.format(kb=kb_number)
+    target = SUPPORT_KB_URL.format(kb=kb_number)
     try:
-        status, location = await http_client.get_location(url, timeout=30.0)
-        if status == 404:
-            return None, None
-        if status in _REDIRECT_STATUSES:
-            target = _resolve_redirect(location)
-        elif status == 200:
-            target = url
-        else:
+        for _ in range(MAX_REDIRECT_HOPS + 1):
+            status, location = await http_client.get_location(target, timeout=30.0)
+            if status == 404:
+                return None, None
+            if status == 200:
+                break
+            if status in _REDIRECT_STATUSES:
+                target = _resolve_redirect(location)
+                continue
             raise KnownIssuesError(f"KB page lookup returned HTTP {status}")
+        else:
+            raise KnownIssuesError(
+                f"KB page redirected more than {MAX_REDIRECT_HOPS} times"
+            )
 
         body_status, body = await http_client.get_bounded(
             target, timeout=30.0, max_bytes=MAX_RESPONSE_BYTES

src/patch_tuesday_mcp/models/vulnerability.py

@@ -289,11 +289,17 @@ def compute_stats(vulnerabilities: list[Vulnerability]) -> dict:
     publicly_disclosed = 0
     kev = 0
 
+    unrated = 0
+    unspecified_impact = 0
     for v in vulnerabilities:
         if v.severity:
             by_severity[v.severity] = by_severity.get(v.severity, 0) + 1
+        else:
+            unrated += 1
         if v.impact:
             by_impact[v.impact] = by_impact.get(v.impact, 0) + 1
+        else:
+            unspecified_impact += 1
         for family in v.product_families:
             by_family[family] = by_family.get(family, 0) + 1
         if v.exploited:
@@ -303,6 +309,14 @@ def compute_stats(vulnerabilities: list[Vulnerability]) -> dict:
         if v.kev is not None:
             kev += 1
 
+    # Entries MSRC ships without a severity rating or impact category
+    # (e.g. Chromium/third-party CVEs) get an explicit bucket so the
+    # breakdowns always sum to the reported total.
+    if unrated:
+        by_severity["Unrated"] = unrated
+    if unspecified_impact:
+        by_impact["Unspecified"] = unspecified_impact
+
     def ranked(d: dict[str, int]) -> list[dict]:
         return [
             {"name": name, "count": count}

tests/fixtures/kb_servicing_layout.html

@@ -0,0 +1,41 @@
+<!DOCTYPE html>
+<html lang="en-us">
+<head>
+<meta charset="utf-8" />
+<meta name="awa-kb_id" content="5094126" />
+<title>June 9, 2026&#x2014;KB5094126 (OS Build 17763.8880) | Microsoft Support</title>
+</head>
+<body>
+<main>
+<h2 id="summary">Summary</h2>
+<p>This security update includes quality improvements. For an overview of Windows 10, version 1809, see its <a href="../../2020/11/windows-10-and-windows-server-2019-update-history" data-linktype="relative-path">update history page</a>.</p>
+<h3 id="known-issues-in-this-update"><strong>Known issues in this update</strong></h3>
+<details>
+<summary>Microsoft Office applications might fail to open from certain third-party apps</summary>
+<p><strong>Symptoms</strong></p>
+<p>Microsoft has received reports of an issue in which certain third-party applications might be unable to launch Microsoft Office applications or open documents after installing this update. This issue affects certain third-party applications that use <a href="https://learn.microsoft.com/windows/win32/api/oaidl/nf-oaidl-idispatch-invoke" data-linktype="external">OLE automation</a>&#160;to interact with Microsoft Office applications.</p>
+<p><strong>Third-party information disclaimer</strong></p>
+<p>The third-party products listed are manufactured by companies that are independent of Microsoft.</p>
+<p><strong>Resolution</strong></p>
+<p>A resolution is in progress and will be included in a future Windows update. To work around this issue, open the application or document directly instead of launching it from the affected third-party application.</p>
+</details>
+<details>
+<summary>Deleting from Recycle Bin displays an internal file name</summary>
+<p><strong>Symptoms</strong></p>
+<p>After you install this update, the confirmation dialog might display the internal Recycle Bin file name (for example, $Rxxxxx.ext) instead of the original file name when you permanently delete a single item.</p>
+<p><strong>Workaround</strong></p>
+<p>This issue is addressed in KB5095093. Until you install that update, dismiss the dialog to continue.</p>
+</details>
+<h3 id="servicing-stack-update">Servicing stack update information</h3>
+<p>This update also includes the servicing stack update.</p>
+<h3 id="known-issues-in-this-update-1"><strong>Known issues in this update</strong></h3>
+<details>
+<summary>Duplicate section issue that must not be parsed twice</summary>
+<p><strong>Symptoms</strong></p>
+<p>The combined SSU and LCU package repeats the known-issues section.</p>
+</details>
+<h2 id="how-to-get-this-update">How to get this update</h2>
+<p>Highlights of the release channels follow.</p>
+</main>
+</body>
+</html>