Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 23 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# langchain-wellmarked

The official [LangChain](https://www.langchain.com/) integration for **[WellMarked](https://wellmarked.io)** — load any URL as clean Markdown `Document`s.
The official [LangChain](https://www.langchain.com/) integration for **[WellMarked](https://wellmarked.io)** — load any URL as clean Markdown `Document`s, or retrieve them straight from a web search.

```bash
pip install langchain-wellmarked
Expand Down Expand Up @@ -50,6 +50,28 @@ docs[0].metadata["depth"] # how far from the root URL this page sits

In `crawl` mode, pages that fail to extract (timeouts, robots-disallowed, no content) are skipped; only successful pages become `Document`s.

## Retriever

`WellMarkedRetriever` is the natural surface for RAG chains where the context should come from the live web: each retrieval searches the web, extracts the top results to clean Markdown, and returns them as `Document`s — search and extraction in one round trip, no URLs to wrangle. Search requires a Pro plan or above.

```python
from langchain_wellmarked import WellMarkedRetriever

retriever = WellMarkedRetriever(num_results=5)
docs = retriever.invoke("best open-source vector databases")

docs[0].page_content # clean Markdown of a result page
docs[0].metadata # {"source": ..., "title": ..., "snippet": ...}
```

| Parameter | Default | Description |
|---------------|-----------|----------------------------------------------------------------|
| `api_key` | env var | WellMarked API key; falls back to `WELLMARKED_API_KEY` |
| `num_results` | `5` | How many results to fetch + extract (clamped to 1..10) |
| `render_js` | `False` | Render JS-heavy result pages with a headless browser (Pro and above) |

Only successfully extracted results become `Document`s; a result whose page timed out or was blocked is skipped.

## Related

- [`wellmarked`](https://pypi.org/project/wellmarked/) — the underlying Python SDK (this package wraps it)
Expand Down
9 changes: 9 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,15 @@ dev = [
line-length = 100
target-version = "py39"

[tool.ruff.lint]
# Pin ruff's classic default rule set (pyflakes + the pycodestyle subset).
# CI installs ruff unpinned, so without an explicit selection `ruff check`
# follows whatever the *current* ruff release defaults to — and newer releases
# broadened those defaults (pyupgrade/simplify/isort/ruff-native), which turns
# CI red on unchanged code. Adopting those rule families is a deliberate,
# separate migration, not an accident of an upstream release.
select = ["E4", "E7", "E9", "F"]

[project.urls]
Homepage = "https://wellmarked.io"
Documentation = "https://wellmarked.io/docs"
Expand Down