Add compliance overrides, scoped-key CRUD, register, and search#4
Merged
Conversation
Continuity for the API's agent-friendly phases: - Compliance policy overrides (allow_domains / deny_patterns / respect_robots) on extract/bulk/crawl, via a shared helper that omits unset fields so an override never clobbers the key's own policy. - Scoped API keys: create_key / list_keys / revoke_key, plus get_logs, with CreatedKey / ApiKeyInfo / RevokedKey / LogEntry / LogsPage. - register(): self-provision a free, extract-only key with no existing credential (sync + async). - search(): web search plus extraction in one synchronous call, with SearchResult / SearchResults. Results carry a per-page status, so a page that failed extraction is marked rather than dropped. Error mapping is by HTTP status, so the new API error codes needed no new exception classes. Sync and async clients kept in lockstep. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jul 20, 2026
- Hoist the mid-file test imports to the top (ruff E402). They had crept in with the phase-5/6 test sections. - policy_overrides: annotate the bare `dict` return/local so mypy has type arguments. - backoff_seconds: annotate `base` as float — `int ** int` widens to Any (a negative exponent would yield float), which leaked Any out of a function declared to return float. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
extract/bulk/crawl gain `format` (markdown | json | chunks | html | links), mirrored across the sync and async clients. Results now inherit a shared `_Content` base carrying the five format-dependent fields plus `metrics`. Exactly one content field is populated per result; `markdown` still holds the default, so existing code is unaffected. New models: ContentBlock, Chunk, ContentMetrics — all exported. `BulkItem.ok` / `CrawlItem.ok` now key on `error` plus the new `content` property rather than on `markdown`. The old form reported every successful links/html/chunks item as failed, since those leave markdown None. Pinned by test_bulk_item_ok_is_true_for_non_markdown_formats. ruff, mypy and 66 pytest all clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…d.io Product decision: users get no capability to point WellMarked anywhere but the hosted API. base_url is gone from both clients and the register() helpers, and the bring-your-own-httpx.Client seam goes with it (a custom transport is the same capability by another door). The client now always owns its httpx client, so the ownership bookkeeping collapses too. Tests keep their respx strategy (it intercepts httpx globally, no seam needed) and gain a negative control pinning that base_url/http_client raise TypeError and the client targets the real API. BREAKING vs the published package: both parameters were public. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
format + the policy overrides now thread through search exactly as they do on bulk/crawl, and SearchResult inherits the shared _Content fields so a non-markdown format lands in its own field with metrics. Docstrings note the Pro+ gate on json/chunks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
retry: server-side re-attempts on target_timeout, fresh connection per attempt, default 0, no upper bound. Not offered on search() — the API's 15s per-hit deadline can't absorb one. Distinct from the client's own max_retries (transport retries between the SDK and the API). crawl(max_pages=N) narrows the plan's page cap; omitted = plan cap alone. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
retry's value stays uncapped, but the server stops re-attempting once a job's 6-hour lifetime is spent — state it where the parameter is documented so the docstring's promise matches the API's behavior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the SDK in line with the API's agent-friendly work.
What's here
allow_domains/deny_patterns/respect_robots) onextract/bulk/crawl. A shared helper omits unset fields, so an override can only narrow the key's own policy — never clobber it.create_key/list_keys/revoke_key, plusget_logs, with the matching models.register(): self-provision a free, extract-only key with no existing credential (sync + async).search(): web search + extraction in one synchronous call.SearchResultcarries a per-pagestatus, so a page that failed extraction is marked rather than silently dropped.Notes
Verification
ruff+mypy+pytestall green locally — 60 tests pass (57 + 3 new forsearch). CI runs the full 6-version matrix (3.9–3.14).Merge/release order matters: this SDK must be published to PyPI before
langchain-wellmarkedandllama-index-readers-wellmarked, whose CI installs the publishedwellmarkedand needsSearchResult/SearchResults.🤖 Generated with Claude Code