From 7c655e8d87bea67dcb4cb8cdd416467238ed22bb Mon Sep 17 00:00:00 2001 From: Grady Dillon Date: Thu, 23 Jul 2026 20:23:06 -0400 Subject: [PATCH 1/2] Document WellMarkedRetriever in the README MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The package ships a WellMarkedRetriever (search the web → extract → return Documents) alongside the loader, but the README only covered the loader. Add a Retriever section with its options, and mention it in the intro. Docs only. Co-Authored-By: Claude Opus 4.8 --- README.md | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index eb11565..fbd9194 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # langchain-wellmarked -The official [LangChain](https://www.langchain.com/) integration for **[WellMarked](https://wellmarked.io)** — load any URL as clean Markdown `Document`s. +The official [LangChain](https://www.langchain.com/) integration for **[WellMarked](https://wellmarked.io)** — load any URL as clean Markdown `Document`s, or retrieve them straight from a web search. ```bash pip install langchain-wellmarked @@ -50,6 +50,28 @@ docs[0].metadata["depth"] # how far from the root URL this page sits In `crawl` mode, pages that fail to extract (timeouts, robots-disallowed, no content) are skipped; only successful pages become `Document`s. +## Retriever + +`WellMarkedRetriever` is the natural surface for RAG chains where the context should come from the live web: each retrieval searches the web, extracts the top results to clean Markdown, and returns them as `Document`s — search and extraction in one round trip, no URLs to wrangle. Search requires a Pro plan or above. + +```python +from langchain_wellmarked import WellMarkedRetriever + +retriever = WellMarkedRetriever(num_results=5) +docs = retriever.invoke("best open-source vector databases") + +docs[0].page_content # clean Markdown of a result page +docs[0].metadata # {"source": ..., "title": ..., "snippet": ...} +``` + +| Parameter | Default | Description | +|---------------|-----------|----------------------------------------------------------------| +| `api_key` | env var | WellMarked API key; falls back to `WELLMARKED_API_KEY` | +| `num_results` | `5` | How many results to fetch + extract (clamped to 1..10) | +| `render_js` | `False` | Render JS-heavy result pages with a headless browser (Pro and above) | + +Only successfully extracted results become `Document`s; a result whose page timed out or was blocked is skipped. + ## Related - [`wellmarked`](https://pypi.org/project/wellmarked/) — the underlying Python SDK (this package wraps it) From 077b873ea19661634e2a24e519e26aea4aa27976 Mon Sep 17 00:00:00 2001 From: Grady Dillon Date: Thu, 23 Jul 2026 20:34:16 -0400 Subject: [PATCH 2/2] Pin ruff's default rule set so unpinned CI stays deterministic MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CI installs ruff unpinned (`ruff>=0.4`/`>=0.6`) and runs `ruff check .`. A newer ruff release broadened its *default* rule selection to include pyupgrade / flake8-simplify / isort / ruff-native rules, which turned CI red on code that was clean at merge time (UP006/UP007/UP035/UP037/UP045, SIM117/SIM118, I001, RUF012) — none of them E/F errors. Pin `[tool.ruff.lint] select` to ruff's classic default (E4/E7/E9/F) so `ruff check` verifies what these repos were written against regardless of which ruff version CI installs. Adopting the newer rule families is a deliberate migration for a separate PR. Co-Authored-By: Claude Opus 4.8 --- pyproject.toml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/pyproject.toml b/pyproject.toml index c3903cc..a14e433 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -40,6 +40,15 @@ dev = [ line-length = 100 target-version = "py39" +[tool.ruff.lint] +# Pin ruff's classic default rule set (pyflakes + the pycodestyle subset). +# CI installs ruff unpinned, so without an explicit selection `ruff check` +# follows whatever the *current* ruff release defaults to — and newer releases +# broadened those defaults (pyupgrade/simplify/isort/ruff-native), which turns +# CI red on unchanged code. Adopting those rule families is a deliberate, +# separate migration, not an accident of an upstream release. +select = ["E4", "E7", "E9", "F"] + [project.urls] Homepage = "https://wellmarked.io" Documentation = "https://wellmarked.io/docs"