diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 2f22bc0..fba8021 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -15,20 +15,69 @@ There are many ways to contribute:
 A special thanks to the [contributors](https://github.com/adbar/courlan/graphs/contributors) who have played a part in Courlan.
 
 
-## Testing and evaluating the code
+## Testing, development setup, and CI expectations
 
-Courlan requires Python 3.10 or higher. Here is how you can run the tests and code quality checks. Pull requests will only be accepted if the changes are tested and if there are no errors.
+Courlan requires Python 3.10 or higher. Follow these steps to set up a
+local development environment, run tests and linters, and prepare a
+pull request.
 
-1. Install the package along with its development dependencies from a checkout: `pip install -e ".[dev]"`
-2. Run the tests and code quality tools:
-   - Tests with `pytest`
-   - Linting and import sorting with `ruff check courlan tests`
-   - Code formatting with `ruff format courlan tests`
-   - Type checking with `mypy -p courlan`
+1. Clone and create a virtual environment
 
+```bash
+git clone https://github.com/adbar/courlan.git
+cd courlan
+python -m venv .venv
+# macOS / Linux
+source .venv/bin/activate
+# Windows (PowerShell)
+.\.venv\Scripts\Activate.ps1
+```
 
-For further questions you can use [GitHub issues](https://github.com/adbar/courlan/issues) or [E-Mail](https://adrien.barbaresi.eu/).
+2. Install dependencies
+
+```bash
+pip install --upgrade pip
+pip install -e '.[dev]'
+```
+
+3. Run tests and quality checks (recommended sequence)
+
+```bash
+# run unit tests
+pytest -q
+
+# linting
+ruff check courlan tests
+
+# apply formatting if needed
+ruff format courlan tests
+
+# static typing
+mypy -p courlan
+```
+
+4. Pre-commit and CI
+
+- Run `pre-commit run --all-files` if pre-commit is configured locally.
+- Ensure all CI checks pass (tests, ruff, mypy) before opening a PR. CI
+  expectation: tests green, linting passes, and type checks report no
+  new errors.
+
+5. Pull request guidance
+
+- Branch from `master` and use a descriptive branch name: `fix/url-cleaning`
+  or `feat/urlstore-persistence`.
+- Update or add tests for bug fixes and new features.
+- Keep commits small and focused. Use conventional commit messages
+  where helpful. Include the Co-authored-by trailer when relevant.
+- In the PR description, explain the problem, your approach, and any
+  user-facing changes (CLI flags, default behavior).
+
+6. Contact and support
+
+If you have questions, open an issue on GitHub or reach out via the
+contact details in the README.
 
 Thanks,
 
-Adrien
+Adrien
\ No newline at end of file
diff --git a/README.md b/README.md
index c0dfdfe..4c0c8c8 100644
--- a/README.md
+++ b/README.md
@@ -4,7 +4,37 @@
 [![Python package](https://img.shields.io/pypi/v/courlan.svg)](https://pypi.python.org/pypi/courlan)
 [![Python versions](https://img.shields.io/pypi/pyversions/courlan.svg)](https://pypi.python.org/pypi/courlan)
 [![Code Coverage](https://img.shields.io/codecov/c/github/adbar/courlan.svg)](https://codecov.io/gh/adbar/courlan)
-[![Documentation](https://readthedocs.org/projects/courlan/badge/?version=latest)](http://courlan.readthedocs.org/en/latest/)
+[![Documentation](https://readthedocs.org/projects/courlan/badge/?version=latest)](https://courlan.readthedocs.io/en/latest/)
+
+
+## Quickstart (1–2 minutes)
+
+Install and try courlan from PyPI:
+
+```bash
+pip install courlan
+```
+
+Python quickstart — validate and clean a URL:
+
+```python
+from courlan import check_url
+result = check_url('https://example.org/page?utm_source=twitter')
+if result:
+    cleaned, domain = result
+    print(cleaned)  # 'https://example.org/page'
+    print(domain)   # 'example.org'
+```
+
+Command-line quickstart — filter a file of URLs:
+
+```bash
+# one URL per line in urls.txt
+courlan -i urls.txt -o cleaned.txt -d discarded.txt
+# cleaned.txt contains accepted URLs, discarded.txt contains rejected ones
+```
+
+These examples are minimal — see the docs for advanced usage: language filtering, strict mode, sampling, and UrlStore persistence.
 
 
 ## Why coURLan?
@@ -50,7 +80,7 @@ retrieval:
 
 **Let the coURLan fish up juicy bits for you!**
 
-<img src="https://raw.githubusercontent.com/adbar/courlan/master/courlan_harns-march.jpg" width="65%" alt="Courlan bird"/>
+<img src="https://raw.githubusercontent.com/adbar/courlan/master/courlan_harns-march.jpg" alt="" role="presentation" style="max-width: 65%;"/>
 
 Here is a [courlan](https://en.wiktionary.org/wiki/courlan) (source:
 [Limpkin at Harn's Marsh by
@@ -87,7 +117,7 @@ All useful operations chained in `check_url(url)`:
 ``` python
 >>> from courlan import check_url
 
-# return url and domain name
+# return url and domain name (None if rejected)
 >>> check_url('https://github.com/adbar/courlan')
 ('https://github.com/adbar/courlan', 'github.com')
 
@@ -95,57 +125,16 @@ All useful operations chained in `check_url(url)`:
 >>> check_url('http://666.0.0.1/')
 >>>
 
-# tracker removal
->>> check_url('http://test.net/foo.html?utm_source=twitter#gclid=123')
-('http://test.net/foo.html', 'test.net')
-
-# use strict for further trimming
->>> my_url = 'https://httpbin.org/redirect-to?url=http%3A%2F%2Fexample.org'
->>> check_url(my_url, strict=True)
-('https://httpbin.org/redirect-to', 'httpbin.org')
-
-# check for redirects (HEAD request)
->>> url, domain_name = check_url(my_url, with_redirects=True)
-
-# include navigation pages instead of discarding them
->>> check_url('http://www.example.org/page/10/', with_nav=True)
-
-# remove trailing slash
->>> check_url('https://github.com/adbar/courlan/', trailing_slash=False)
-```
-
-Language-aware heuristics, notably internationalization in URLs, are
-available in `lang_filter(url, language)`:
-
-``` python
-# optional language argument
->>> url = 'https://www.un.org/en/about-us'
-
-# success: returns clean URL and domain name
->>> check_url(url, language='en')
+# language-aware filtering
+>>> check_url('https://www.un.org/en/about-us', language='en')
 ('https://www.un.org/en/about-us', 'un.org')
-
-# failure: doesn't return anything
->>> check_url(url, language='de')
->>>
-
-# optional argument: strict
->>> url = 'https://en.wikipedia.org/'
->>> check_url(url, language='de', strict=False)
-('https://en.wikipedia.org', 'wikipedia.org')
->>> check_url(url, language='de', strict=True)
+>>> check_url('https://www.un.org/en/about-us', language='de')
 >>>
 ```
 
-Define stricter restrictions on the expected content type with
-`strict=True`. This also blocks certain platforms and page types
-where machines get lost.
-
-``` python
-# strict filtering: blocked as it is a major platform
->>> check_url('https://www.twitch.com/', strict=True)
->>>
-```
+For the full set of options (`strict`, `with_redirects`, `with_nav`,
+`trailing_slash`, …) see the
+[documentation](https://courlan.readthedocs.io/en/latest/api/core.html).
 
 ### Sampling by domain name
 
@@ -156,252 +145,72 @@ where machines get lost.
 # optional: exclude_min=None, exclude_max=None, strict=False, verbose=False
 ```
 
+See the [API reference](https://courlan.readthedocs.io/en/latest/api/index.html) for details.
+
 ### Web crawling and URL handling
 
-Link extraction and preprocessing:
+Use `extract_links()` for general-purpose link extraction. For
+crawl-aware extraction with robots.txt rules and link prioritization,
+use `filter_links()` instead — it returns two lists: regular links and
+priority (navigation) links.
 
 ``` python
 >>> from courlan import extract_links
 >>> doc = '<html><body><a href="test/link.html">Link</a></body></html>'
->>> url = "https://example.org"
->>> extract_links(doc, url)
+>>> extract_links(doc, "https://example.org")
 {'https://example.org/test/link.html'}
-# other options: external_bool, no_filter, language, strict, redirects, ...
-```
-
-The `filter_links()` function provides additional filters for crawling
-purposes: use of robots.txt rules and link prioritization. It returns two
-lists: regular links and priority (navigation) links.
-
-``` python
->>> from courlan import filter_links
->>> doc = '<html><body><a href="page1.html">1</a><a href="/tag/listing">Tag</a></body></html>'
->>> links, links_priority = filter_links(doc, "https://example.org")
->>> links
-['https://example.org/page1.html']
->>> links_priority
-['https://example.org/tag/listing']
-```
-
-Determine if a link leads to another host:
-
-``` python
->>> from courlan import is_external
->>> is_external('https://github.com/', 'https://www.microsoft.com/')
-True
-# default
->>> is_external('https://google.com/', 'https://www.google.co.uk/', ignore_suffix=True)
-False
-# taking suffixes into account
->>> is_external('https://google.com/', 'https://www.google.co.uk/', ignore_suffix=False)
-True
-```
-
-Other useful functions dedicated to URL handling:
-
--   `extract_domain(url, fast=True)`: find domain and subdomain or just
-    domain with `fast=False`
--   `get_base_url(url)`: strip the URL of some of its parts
--   `get_host_and_path(url)`: decompose URLs in two parts: protocol +
-    host/domain and path
--   `get_hostinfo(url)`: extract domain and host info (protocol +
-    host/domain)
--   `fix_relative_urls(baseurl, url)`: prepend necessary information to
-    relative links
-
-``` python
->>> from courlan import *
->>> url = 'https://www.un.org/en/about-us'
-
->>> get_base_url(url)
-'https://www.un.org'
-
->>> get_host_and_path(url)
-('https://www.un.org', '/en/about-us')
-
->>> get_hostinfo(url)
-('un.org', 'https://www.un.org')
-
->>> fix_relative_urls('https://www.un.org', 'en/about-us')
-'https://www.un.org/en/about-us'
 ```
 
-Other filters dedicated to crawl frontier management:
-
--   `is_not_crawlable(url)`: check for deep web or pages generally not
-    usable in a crawling context
--   `is_navigation_page(url)`: check for navigation and overview pages
-
-``` python
->>> from courlan import is_navigation_page, is_not_crawlable
->>> is_navigation_page('https://www.randomblog.net/category/myposts')
-True
->>> is_not_crawlable('https://www.randomblog.net/login')
-True
-```
-
-See also [URL management page](https://trafilatura.readthedocs.io/en/latest/url-management.html)
-of the Trafilatura documentation.
-
+For frontier management utilities (`is_external`, `is_navigation_page`,
+`is_not_crawlable`, …) see the
+[crawling guide](https://courlan.readthedocs.io/en/latest/usage/crawling.html).
 
 ### Python helpers
 
-Helper function, scrub and normalize:
-
 ``` python
 >>> from courlan import clean_url
 >>> clean_url('HTTPS://WWW.DWDS.DE:443/')
 'https://www.dwds.de'
 ```
 
-Basic scrubbing only:
+For `normalize_url`, `validate_url`, `get_base_url`, `get_hostinfo`,
+and other utilities see the
+[API reference](https://courlan.readthedocs.io/en/latest/api/index.html).
 
-``` python
->>> from courlan import scrub_url
-```
-
-Basic canonicalization/normalization only, i.e. modifying and
-standardizing URLs in a consistent manner:
+Courlan uses an internal cache to speed up URL parsing. It can be
+reset with `courlan.meta.clear_caches()`.
 
-``` python
->>> from urllib.parse import urlparse
->>> from courlan import normalize_url
->>> my_url = normalize_url(urlparse(my_url))
-# passing URL strings directly also works
->>> my_url = normalize_url(my_url)
-# remove unnecessary components and re-order query elements
->>> normalize_url('http://test.net/foo.html?utm_source=twitter&post=abc&page=2#fragment', strict=True)
-'http://test.net/foo.html?page=2&post=abc'
-```
-
-Basic URL validation only:
-
-``` python
->>> from courlan import validate_url
->>> validate_url('http://1234')
-(False, None)
->>> validate_url('http://www.example.org/')
-(True, ParseResult(scheme='http', netloc='www.example.org', path='/', params='', query='', fragment=''))
-```
 
-### Troubleshooting
+## UrlStore class
 
-Courlan uses an internal cache to speed up URL parsing. It can be reset
-as follows:
+The `UrlStore` class allows for storing and retrieving domain-classified
+URLs, where a URL like `https://example.org/path/testpage` is stored as
+the path `/path/testpage` within the domain `https://example.org`:
 
 ``` python
->>> from courlan.meta import clear_caches
->>> clear_caches()
+>>> from courlan import UrlStore
+>>> store = UrlStore()
+>>> store.add_urls(['https://example.org/page1', 'https://example.org/page2'])
+>>> store.get_url('https://example.org')
+'https://example.org/page1'
+>>> store.find_unvisited_urls('https://example.org')
+['https://example.org/page2']
 ```
 
-## UrlStore class
-
-The `UrlStore` class allow for storing and retrieving domain-classified
-URLs, where a URL like `https://example.org/path/testpage` is stored as
-the path `/path/testpage` within the domain `https://example.org`. It
-features the following methods:
-
-- URL management
-   - `add_urls(urls=None, appendleft=None, visited=False)`: Add a
-     list of URLs to the (possibly) existing one. Optional:
-     append certain URLs to the left, specify if the URLs have
-     already been visited.
-   - `add_from_html(htmlstring, url, external=False, lang=None, with_nav=True)`:
-     Extract and filter links in a HTML string.
-   - `discard(domains)`: Declare domains void and prune the store.
-   - `dump_urls()`: Return a list of all known URLs.
-   - `print_urls()`: Print all URLs in store (URL + TAB + visited or not).
-   - `print_unvisited_urls()`: Print all unvisited URLs in store.
-   - `get_all_counts()`: Return all download counts for the hosts in store.
-   - `get_known_domains()`: Return all known domains as a list.
-   - `get_unvisited_domains()`: Find all domains for which there are unvisited URLs.
-   - `total_url_number()`: Find number of all URLs in store.
-   - `is_known(url)`: Check if the given URL has already been stored.
-   - `has_been_visited(url)`: Check if the given URL has already been visited.
-   - `filter_unknown_urls(urls)`: Take a list of URLs and return the currently unknown ones.
-   - `filter_unvisited_urls(urls)`: Take a list of URLs and return the currently unvisited ones.
-   - `find_known_urls(domain)`: Get all already known URLs for the
-     given domain (ex. `https://example.org`).
-   - `find_unvisited_urls(domain)`: Get all unvisited URLs for the given domain.
-   - `reset()`: Re-initialize the URL store.
-
-- Crawling and downloads
-   - `get_url(domain)`: Retrieve a single URL and consider it to
-     be visited (with corresponding timestamp).
-   - `get_rules(domain)`: Return the stored crawling rules for the given website.
-   - `store_rules(website, rules)`: Store crawling rules for a given website.
-   - `get_crawl_delay()`: Return the delay as extracted from robots.txt, or a given default.
-   - `get_download_urls(time_limit=10, max_urls=10000)`: Get a list of immediately
-     downloadable URLs according to the given time limit per domain.
-   - `establish_download_schedule(max_urls=100, time_limit=10)`:
-     Get up to the specified number of URLs along with a suitable
-     backoff schedule (in seconds).
-   - `download_threshold_reached(threshold)`: Find out if the
-     download limit (in seconds) has been reached for one of the
-     websites in store.
-   - `unvisited_websites_number()`: Return the number of websites
-     for which there are still URLs to visit.
-   - `is_exhausted_domain(domain)`: Tell if all known URLs for
-     the website have been visited.
-
-- Persistance
-   - `write(filename)`: Save the store to disk.
-   - `load_store(filename)`: Read a UrlStore from disk (separate function, not class method).
-
-- Optional settings:
-   - `compressed=True`: activate compression of URLs and rules
-   - `language=XX`: focus on a particular target language (two-letter code)
-   - `strict=True`: stricter URL filtering
-   - `verbose=True`: dump URLs if interrupted (requires use of `signal`)
+For the full method reference, optional settings (`compressed`, `language`,
+`strict`, `trailing_slash`, `verbose`), and crawl scheduling see the
+[UrlStore documentation](https://courlan.readthedocs.io/en/latest/api/urlstore.html).
 
 
 ## Command-line
 
-The main fonctions are also available through a command-line utility:
-
 ``` bash
 $ courlan --inputfile url-list.txt --outputfile cleaned-urls.txt
 $ courlan --help
-usage: courlan [-h] -i INPUTFILE -o OUTPUTFILE [-d DISCARDEDFILE] [-v]
-               [-p PARALLEL] [--strict] [-l LANGUAGE] [-r] [--sample SAMPLE]
-               [--exclude-max EXCLUDE_MAX] [--exclude-min EXCLUDE_MIN]
-
-Command-line interface for Courlan
-
-options:
-  -h, --help            show this help message and exit
-
-I/O:
-  Manage input and output
-
-  -i INPUTFILE, --inputfile INPUTFILE
-                        name of input file (required)
-  -o OUTPUTFILE, --outputfile OUTPUTFILE
-                        name of output file (required)
-  -d DISCARDEDFILE, --discardedfile DISCARDEDFILE
-                        name of file to store discarded URLs (optional)
-  -v, --verbose         increase output verbosity
-  -p PARALLEL, --parallel PARALLEL
-                        number of parallel processes (not used for sampling)
-
-Filtering:
-  Configure URL filters
-
-  --strict              perform more restrictive tests
-  -l LANGUAGE, --language LANGUAGE
-                        use language filter (ISO 639-1 code)
-  -r, --redirects       check redirects
-
-Sampling:
-  Use sampling by host, configure sample size
-
-  --sample SAMPLE       size of sample per domain
-  --exclude-max EXCLUDE_MAX
-                        exclude domains with more than n URLs
-  --exclude-min EXCLUDE_MIN
-                        exclude domains with less than n URLs
 ```
 
+See the [CLI documentation](https://courlan.readthedocs.io/en/latest/usage/cli.html) for all options.
+
 
 ## License
 
@@ -414,12 +223,9 @@ Versions prior to v1 were under GPLv3+ license.
 ## Settings
 
 `courlan` is optimized for English and German but its generic approach
-is also usable in other contexts.
-
-Details of strict URL filtering can be reviewed and changed in the file
-`settings.py`. To override the default settings, clone the repository and
-[re-install the package
-locally](https://packaging.python.org/tutorials/installing-packages/#installing-from-a-local-src-tree).
+is also usable in other contexts. See the
+[settings reference](https://courlan.readthedocs.io/en/latest/api/settings.html)
+for how to review and override filtering rules.
 
 
 ## Author
@@ -440,15 +246,15 @@ Reach out via the software repository or the [contact
 page](https://adrien.barbaresi.eu/) for inquiries, collaborations, or
 feedback.
 
-For more on Courlan's' software ecosystem see [this
+For more on Courlan's software ecosystem see [this
 graphic](https://github.com/adbar/trafilatura/blob/master/docs/software-ecosystem.png).
 
 
 ## Similar work
 
-These Python libraries perform similar handling and normalization tasks
-but do not entail language or content filters. They also do not
-primarily focus on crawl optimization:
+These Python libraries perform URL handling and normalization but do not
+provide language-aware filtering, content heuristics, crawl scheduling,
+or a domain-classified URL store:
 
 -   [furl](https://github.com/gruns/furl)
 -   [ural](https://github.com/medialab/ural)
diff --git a/courlan/core.py b/courlan/core.py
index 18bfcc6..3ebf5b7 100644
--- a/courlan/core.py
+++ b/courlan/core.py
@@ -57,8 +57,9 @@ def check_url(
         with_redirects: set to True for redirection test (per HTTP HEAD request)
         language: set target language (ISO 639-1 codes)
         with_nav: set to True to include navigation pages instead of discarding them
-        trailing_slash: keep trailing slashes on non-root paths (default True);
-                  the root slash is always stripped
+        trailing_slash: preserve trailing slashes (default True); when False,
+                  strip them from paths without a query string. A bare root
+                  slash is always stripped unless a query or fragment is present
 
     Returns:
         A tuple consisting of canonical URL and extracted domain
@@ -174,8 +175,9 @@ def extract_links(
         no_filter: override settings and bypass checks to return all possible URLs
         language: set target language (ISO 639-1 codes)
         strict: set to True for stricter filtering
-        trailing_slash: keep trailing slashes on non-root paths (default True);
-                  the root slash is always stripped
+        trailing_slash: preserve trailing slashes (default True); when False,
+                  strip them from paths without a query string. A bare root
+                  slash is always stripped unless a query or fragment is present
         with_nav: set to True to include navigation pages instead of discarding them
         redirects: set to True for redirection test (per HTTP HEAD request)
         reference: provide a host reference for external/internal evaluation
diff --git a/docs/source/api/clean.md b/docs/source/api/clean.md
index 14433d4..c496cf0 100644
--- a/docs/source/api/clean.md
+++ b/docs/source/api/clean.md
@@ -1,24 +1,11 @@
 # courlan.clean
 
-Core URL cleaning and normalization utilities.
+URL cleaning and normalization utilities.
+
+For usage examples, see the [Python Usage guide](../usage/python.md).
 
 ```{automodule} courlan.clean
 :members:
 :undoc-members:
 :show-inheritance:
 ```
-
-## Common usage
-
-```python
-from courlan import clean_url, scrub_url, normalize_url, validate_url
-
-# Clean and normalize a URL (returns str or None if invalid)
-url = clean_url('HTTPS://WWW.EXAMPLE.COM:443/path?utm_source=x')
-
-# Basic validation
-is_valid, parsed = validate_url('https://example.com')
-
-# Normalization only
-normalized = normalize_url('http://example.com/path?z=1&a=2#fragment')
-```
diff --git a/docs/source/api/cli.md b/docs/source/api/cli.md
deleted file mode 100644
index 3e323aa..0000000
--- a/docs/source/api/cli.md
+++ /dev/null
@@ -1,11 +0,0 @@
-# courlan.cli
-
-Command-line interface implementation and argument parsing.
-
-This module contains the CLI entry point and internal helpers. Most users interact with this through the `courlan` command rather than importing it directly — see the [CLI Reference](../usage/cli.md) for full flag documentation.
-
-```{automodule} courlan.cli
-:members:
-:undoc-members:
-:show-inheritance:
-```
diff --git a/docs/source/api/core.md b/docs/source/api/core.md
index 0afb2b2..1093b18 100644
--- a/docs/source/api/core.md
+++ b/docs/source/api/core.md
@@ -1,38 +1,11 @@
 # courlan.core
 
-Core URL checking utilities.
+Core URL checking, link extraction, and filtering.
+
+For usage examples, see the [Python Usage guide](../usage/python.md).
 
 ```{automodule} courlan.core
 :members:
 :undoc-members:
 :show-inheritance:
 ```
-
-## Common usage
-
-```python
-from courlan import check_url, extract_links, filter_links
-
-# check_url returns (url, domain) or None if the URL is rejected
-result = check_url('https://example.com/article')
-if result:
-    url, domain = result
-
-# Strict mode and language filtering
-result = check_url('https://example.com/article', strict=True, language='en')
-
-# Extract links from HTML (returns a set)
-links = extract_links(html, 'https://example.com', external_bool=False)
-
-# Extract and prioritize links for crawling (returns links, priority_links)
-links, priority_links = filter_links(html, 'https://example.com', lang='en')
-```
-
-## Filtering cost
-
-Options add overhead in this order, from cheapest to most expensive:
-
-1. **Basic** — `check_url(url)`
-2. **Language filtering** — `check_url(url, language='en')` — minimal overhead
-3. **Strict mode** — `check_url(url, strict=True)` — more conditions checked
-4. **Redirect checks** — `check_url(url, with_redirects=True)` — network I/O; avoid on large datasets
diff --git a/docs/source/api/filters.md b/docs/source/api/filters.md
index 3a4ce58..2c4a887 100644
--- a/docs/source/api/filters.md
+++ b/docs/source/api/filters.md
@@ -2,31 +2,10 @@
 
 URL filtering heuristics for content validation and crawler optimization.
 
+For usage examples, see the [Python Usage guide](../usage/python.md).
+
 ```{automodule} courlan.filters
 :members:
 :undoc-members:
 :show-inheritance:
 ```
-
-## Common usage
-
-```python
-from courlan import check_url, filter_links, lang_filter, is_valid_url
-
-# check_url returns (url, domain) or None if rejected
-result = check_url('https://example.com/article', language='en', strict=True)
-if result:
-    url, domain = result
-
-# Extract and filter links from HTML
-html = '<a href="/page1">Link</a><a href="/tag/spam">Tag</a>'
-links, priority_links = filter_links(html, 'https://example.com', lang='en')
-
-# Test if a URL matches a target language heuristically
-if lang_filter('https://example.com/en/article', language='en'):
-    print("Language matches")
-
-# Basic structural validity check (no network call)
-if is_valid_url('https://example.com/path'):
-    print("Valid URL structure")
-```
diff --git a/docs/source/api/index.md b/docs/source/api/index.md
index ebe3d53..f5985a8 100644
--- a/docs/source/api/index.md
+++ b/docs/source/api/index.md
@@ -6,15 +6,44 @@ This section is generated from the courlan package. Click a module to jump to it
 :maxdepth: 1
 :caption: Modules
 
-clean
-cli
 core
 filters
-meta
-network
+clean
+urlutils
+urlstore
 sampling
 settings
-urlstore
-urlutils
 ```
 
+
+## Internal modules
+
+### courlan.network
+
+HTTP redirect resolution used by `check_url(with_redirects=True)`.
+
+```{automodule} courlan.network
+:members:
+:undoc-members:
+:show-inheritance:
+```
+
+### courlan.meta
+
+LRU cache management. Use `clear_caches()` in long-running processes.
+
+```{automodule} courlan.meta
+:members:
+:undoc-members:
+:show-inheritance:
+```
+
+### courlan.cli
+
+Entry point for the `courlan` command. See the [CLI reference](../usage/cli.md) for usage.
+
+```{automodule} courlan.cli
+:members:
+:undoc-members:
+:show-inheritance:
+```
diff --git a/docs/source/api/meta.md b/docs/source/api/meta.md
deleted file mode 100644
index 6dd052d..0000000
--- a/docs/source/api/meta.md
+++ /dev/null
@@ -1,66 +0,0 @@
-# courlan.meta
-
-Cache management and meta-utilities.
-
-```{automodule} courlan.meta
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-## Cache Management
-
-Courlan uses LRU (Least Recently Used) caches to speed up URL parsing and language detection. For long-running processes, you can clear these caches to reclaim memory.
-
-### clear_caches()
-
-**Purpose**: Reset all internal LRU caches.
-
-Use in long-running processes handling many distinct URLs, in memory-constrained environments, or between crawl phases.
-
-**What gets cleared**: urllib.parse results, language detection scores.
-
-**Example**:
-```python
-from courlan import check_url
-from courlan.meta import clear_caches
-
-for i in range(10000):
-    result = check_url(f'https://example.com/page{i}')
-    if (i + 1) % 1000 == 0:
-        clear_caches()
-```
-
----
-
-## Usage in Batch Workflows
-
-```python
-from courlan import UrlStore, check_url
-from courlan.meta import clear_caches
-
-store = UrlStore(compressed=True)
-store.add_urls(many_urls)
-
-# Process in batches
-batch_size = 5000
-processed = 0
-
-while store.unvisited_websites_number() > 0:
-    for domain in store.get_unvisited_domains():
-        url = store.get_url(domain)
-        if url:
-            check_url(url, strict=True, language='en')
-            processed += 1
-            
-            # Clear caches periodically
-            if processed % batch_size == 0:
-                clear_caches()
-                print(f"Processed {processed} URLs, caches cleared")
-```
-
----
-
-## See Also
-
-- [Web Crawling guide](../usage/crawling.md) — cache clearing in crawler workflows
diff --git a/docs/source/api/network.md b/docs/source/api/network.md
deleted file mode 100644
index 592ee1e..0000000
--- a/docs/source/api/network.md
+++ /dev/null
@@ -1,19 +0,0 @@
-# courlan.network
-
-Network helpers for redirect checking and HTTP operations.
-
-```{automodule} courlan.network
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-## Common usage
-
-```python
-# Redirect checking is typically used through check_url() function
-from courlan import check_url
-
-# Check if URL redirects (makes HTTP HEAD request)
-url, domain = check_url('https://example.com/old-page', with_redirects=True)
-```
diff --git a/docs/source/api/sampling.md b/docs/source/api/sampling.md
index 03e58e0..4dcfa4e 100644
--- a/docs/source/api/sampling.md
+++ b/docs/source/api/sampling.md
@@ -1,74 +1,9 @@
 # courlan.sampling
 
-Sampling utilities to produce per-host URL samples.
+URL sampling by domain. See the [Python Usage guide](../usage/python.md) for examples.
 
 ```{automodule} courlan.sampling
 :members:
 :undoc-members:
 :show-inheritance:
 ```
-
-## Common usage
-
-```python
-from courlan import sample_urls
-
-# Generate sample: up to 10 URLs per domain
-urls = ['https://example.com/p1', 'https://example.com/p2', 'https://other.org/a']
-sample = sample_urls(urls, 10)
-
-# With exclusion filters
-sample = sample_urls(urls, samplesize=5, exclude_min=2, exclude_max=100)
-```
-
-## Example: Sampling output
-
-**Input URLs** (8 total, 3 domains):
-
-```
-https://github.com/adbar/courlan
-https://github.com/adbar/trafilatura
-https://github.com/adbar/htmldate
-https://example.com/some/page
-https://example.com/another/page
-https://example.com/third/page
-https://another.example/path
-https://another.example/blog/post
-```
-
-**Sample with `samplesize=2`** (2 URLs per domain):
-
-```python
-from courlan import sample_urls
-
-urls = [
-    'https://github.com/adbar/courlan',
-    'https://github.com/adbar/trafilatura',
-    'https://github.com/adbar/htmldate',
-    'https://example.com/some/page',
-    'https://example.com/another/page',
-    'https://example.com/third/page',
-    'https://another.example/path',
-    'https://another.example/blog/post',
-]
-
-sample = sample_urls(urls, samplesize=2)
-# Result: 6 URLs (2 per domain)
-for url in sample:
-    print(url)
-```
-
-**Output**:
-```
-https://github.com/adbar/courlan
-https://github.com/adbar/trafilatura
-https://example.com/some/page
-https://example.com/another/page
-https://another.example/path
-https://another.example/blog/post
-```
-
-**Reduction**: 8 input URLs → 6 sampled URLs (2 per domain)
-
-For CLI sampling, see the [CLI Reference](../usage/cli.md).
-
diff --git a/docs/source/api/settings.md b/docs/source/api/settings.md
index 8e463aa..515b753 100644
--- a/docs/source/api/settings.md
+++ b/docs/source/api/settings.md
@@ -17,7 +17,7 @@ Configuration constants for URL filtering and content detection.
 | `LANG_PARAMS` | `set[str]` | Query parameter names used for language detection (e.g. `lang`, `language`) |
 | `TARGET_LANGS` | `dict[str, set[str]]` | ISO 639-1 codes mapped to accepted variants (e.g. `"de"` → `{"de", "deutsch", "ger"}`) |
 
-## Customizing Settings
+## Customizing settings
 
 Settings are module-level objects loaded at import time. Patch them at runtime before any filtering calls:
 
@@ -26,7 +26,33 @@ import courlan.settings as settings
 
 settings.BLACKLIST.add("myservice.com")
 settings.ALLOWED_PARAMS.add("story_id")
-settings.TARGET_LANGS["fr"].add("français")
+settings.TARGET_LANGS.setdefault("fr", set()).add("français")
 ```
 
 For permanent changes, edit `courlan/settings.py` directly and reinstall in editable mode (`pip install -e .`).
+
+## Default values
+
+The most commonly-tuned defaults live in `courlan/settings.py`. Current
+values (refer to the file for the authoritative list) include:
+
+- BLACKLIST — a set of domain fragments excluded by default (social
+  media, CDNs, common platforms). Example entries: `"facebook"`,
+  `"amazonaws"`, `"youtube"`.
+- ALLOWED_PARAMS — query parameter names preserved during cleaning
+  (content IDs, pagination), e.g. `"page"`, `"id"`, `"post"`.
+- LANG_PARAMS — query parameter names used for language signals,
+  typically `{ "lang", "language" }`.
+- TARGET_LANGS — mapping of ISO 639-1 codes to accepted variants, e.g.
+  `{"en": {"en", "english"}, "de": {"de", "deutsch"}}`.
+
+To inspect defaults at runtime:
+
+```python
+import courlan.settings as settings
+print(settings.BLACKLIST)
+print(settings.ALLOWED_PARAMS)
+```
+
+Be cautious: overly aggressive changes (e.g., emptying BLACKLIST) can
+significantly alter filtering behavior.
diff --git a/docs/source/api/urlstore.md b/docs/source/api/urlstore.md
index ebe6976..04ef819 100644
--- a/docs/source/api/urlstore.md
+++ b/docs/source/api/urlstore.md
@@ -2,115 +2,10 @@
 
 Domain-classified URL storage for web crawling workflows.
 
+For usage examples, see the [URL Store guide](../usage/urlstore.md) and the [Web Crawling guide](../usage/crawling.md).
+
 ```{automodule} courlan.urlstore
 :members:
 :undoc-members:
 :show-inheritance:
 ```
-
-For crawler-oriented usage (crawl loops, scheduling, robots.txt, HTML link extraction), see the [Web Crawling guide](../usage/crawling.md).
-
-## Examples
-
-### Basic URL tracking
-
-```python
-from courlan import UrlStore
-
-store = UrlStore()
-store.add_urls([
-    'https://example.com/page1',
-    'https://example.com/page2',
-    'https://example.org/article',
-])
-
-while store.unvisited_websites_number() > 0:
-    for domain in store.get_unvisited_domains():
-        url = store.get_url(domain)  # marks URL as visited
-        if url:
-            print(f"Processing: {url}")
-            store.add_urls(['https://example.com/page3'])
-```
-
-### Persistent store (save/load)
-
-```python
-from courlan import UrlStore, load_store
-
-# Build store over time
-store = UrlStore()
-store.add_urls(['https://example.com/1', 'https://example.com/2'])
-store.get_url('https://example.com')  # mark as visited
-
-# Save to disk
-store.write('my_urls.db')
-
-# Later: load from disk (different session)
-store = load_store('my_urls.db')
-
-# Continue where you left off
-print(f"Total URLs: {store.total_url_number()}")
-print(f"Unvisited domains: {store.get_unvisited_domains()}")
-```
-
-```{warning}
-`write()`/`load_store()` use Python's pickle format, which can execute
-arbitrary code when loading. Only load files you have written yourself.
-```
-
-### Statistics and reporting
-
-```python
-from courlan import UrlStore
-
-store = UrlStore()
-store.add_urls([
-    'https://a.com/1', 'https://a.com/2', 'https://a.com/3',
-    'https://b.org/x', 'https://b.org/y',
-    'https://c.net/article',
-])
-
-# Mark some as visited
-store.get_url('https://a.com')
-store.get_url('https://a.com')
-
-# Generate statistics
-print(f"Total URLs: {store.total_url_number()}")
-print(f"Known domains: {store.get_known_domains()}")
-print(f"Unvisited domains: {store.get_unvisited_domains()}")
-
-# Per-domain stats
-for domain in store.get_known_domains():
-    all_urls = store.find_known_urls(domain)
-    unvisited = store.find_unvisited_urls(domain)
-    print(f"{domain}: {len(all_urls)} total, {len(unvisited)} unvisited")
-```
-
-### Filtering and deduplication
-
-```python
-from courlan import UrlStore
-
-store = UrlStore()
-store.add_urls(['https://example.com/page', 'https://example.org/post'])
-
-# Check if URL is already known
-if store.is_known('https://example.com/page'):
-    print("Already in store")
-
-# Filter unknown URLs
-new_urls = ['https://example.com/page', 'https://example.com/new']
-unknown = store.filter_unknown_urls(new_urls)
-print(f"Unknown URLs: {unknown}")
-
-# Filter unvisited URLs
-unvisited = store.filter_unvisited_urls(new_urls)
-```
-
-## Performance tips
-
-- **For large crawls**: Use `compressed=True` to reduce memory
-- **Storage**: Save the store periodically with `write(filename)`
-- **Scheduling**: Use `establish_download_schedule()` to respect crawl delays
-- **Languages**: Set language filter at init to filter links automatically: `UrlStore(language='en')`
-
diff --git a/docs/source/api/urlutils.md b/docs/source/api/urlutils.md
index 95c72ee..6c3f113 100644
--- a/docs/source/api/urlutils.md
+++ b/docs/source/api/urlutils.md
@@ -2,33 +2,10 @@
 
 URL parsing, decomposition, and relative URL resolution utilities.
 
+For usage examples, see the [Python Usage guide](../usage/python.md).
+
 ```{automodule} courlan.urlutils
 :members:
 :undoc-members:
 :show-inheritance:
 ```
-
-## Common usage
-
-```python
-from courlan import extract_domain, get_base_url, get_host_and_path, fix_relative_urls
-from courlan import get_hostinfo, filter_urls
-
-# Extract domain from URL
-domain = extract_domain('https://www.example.com/path', fast=True)
-
-# Get base URL (scheme + netloc)
-base = get_base_url('https://example.com/path/page?q=1')
-
-# Decompose URL into host and path
-host, path = get_host_and_path('https://example.com/articles/post')
-
-# Convenience: domain name + base URL in one call
-domainname, base_url = get_hostinfo('https://www.example.com/path')
-
-# Resolve relative URLs
-absolute = fix_relative_urls('https://example.com', 'articles/post.html')
-
-# Filter a list of URLs by substring pattern (None = deduplicate only)
-subset = filter_urls(link_list, urlfilter='example.com')
-```
diff --git a/docs/source/changelog.md b/docs/source/changelog.md
new file mode 100644
index 0000000..965e618
--- /dev/null
+++ b/docs/source/changelog.md
@@ -0,0 +1,8 @@
+---
+myst:
+  html_meta:
+    description: "Courlan release history, changelog, and migration notes."
+---
+
+```{include} ../../HISTORY.md
+```
diff --git a/docs/source/conf.py b/docs/source/conf.py
index b35434f..33dbef9 100644
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -16,6 +16,9 @@
     "sphinx.ext.intersphinx",
     "sphinx.ext.viewcode",
     "sphinx_copybutton",
+    "sphinx_design",
+    "sphinxext.opengraph",
+    "sphinx_sitemap",
 ]
 
 napoleon_google_docstring = True
@@ -36,9 +39,28 @@
     "colon_fence",
 ]
 
-html_theme = "sphinx_rtd_theme"
+# -- HTML output -----------------------------------------------------------
+
+html_theme = "furo"
 html_static_path = ["_static"]
-html_title = project
+html_title = "courlan — URL Filtering and Normalization for Python"
+
+html_theme_options = {
+    "source_repository": "https://github.com/adbar/courlan",
+    "source_branch": "master",
+    "source_directory": "docs/source/",
+}
+html_baseurl = "https://courlan.readthedocs.io/en/latest/"
+sitemap_url_scheme = "{link}"
+sitemap_excludes = ["search.html", "genindex.html"]
+
+# -- OpenGraph metadata ----------------------------------------------------
+
+ogp_site_url = "https://courlan.readthedocs.io/en/latest/"
+ogp_site_name = "courlan"
+ogp_description_length = 200
+
+# -- Intersphinx -----------------------------------------------------------
 
 intersphinx_mapping = {
     "python": ("https://docs.python.org/3", None),
diff --git a/docs/source/getting-started.md b/docs/source/getting-started.md
index fd8e5b1..519a55b 100644
--- a/docs/source/getting-started.md
+++ b/docs/source/getting-started.md
@@ -1,6 +1,4 @@
-# Getting Started
-
-This single guide covers both installation and a minimal, runnable Quickstart.
+# Getting Started with Courlan
 
 ## Prerequisites
 - Python 3.10+
@@ -8,90 +6,57 @@ This single guide covers both installation and a minimal, runnable Quickstart.
 
 ## Install
 
-Install the latest release from PyPI, e.g. with pip or uv:
+:::::{tab-set}
 
+::::{tab-item} pip
 ```bash
 pip install courlan
 ```
+::::
 
-Or install from source for development:
+::::{tab-item} uv
+```bash
+uv add courlan
+```
+::::
 
+::::{tab-item} From source
 ```bash
 git clone https://github.com/adbar/courlan.git
 cd courlan
 pip install -e .
 ```
+::::
 
-## Minimal CLI Quickstart (hands-on)
+:::::
 
-1) Create a simple input file (one URL per line):
 
-```bash
-cat > urls.txt <<'EOF'
-https://github.com/adbar/courlan
-https://example.com/some/page
-https://another.example/path
-EOF
-```
+## Quick check
 
-2) Run a full processing pass (filters + optional redirect checks). `-i/--inputfile` and `-o/--outputfile` are required:
+```python
+from courlan import check_url
 
-```bash
-courlan -i urls.txt -o cleaned.txt
-# or
-courlan --inputfile urls.txt --outputfile cleaned.txt
+# returns (cleaned_url, domain) or None if rejected
+check_url('https://example.org/page?utm_source=twitter')
+# ('https://example.org/page', 'example.org')
 ```
 
-Result: cleaned.txt contains accepted URLs (one per line). Exit code 0 indicates success.
-
-
-## Troubleshooting & tips
-- If the command fails, run `courlan --help` to inspect flags and check your input file encoding.
-- For development, prefer `pip install -e .` so local changes take effect immediately.
-
-## End-to-end example
-
-Create an input file, filter it, and inspect the results:
+From the command line:
 
 ```bash
-cat > urls.txt <<'EOF'
-https://www.example.com/page1
-https://www.example.com/page2?utm_source=twitter
-https://login.example.com/signin
-https://cdn.example.com/image.jpg
-https://example.org/valid-article
-EOF
-
-courlan -i urls.txt -o cleaned.txt -d discarded.txt --strict -v
+courlan -i urls.txt -o cleaned.txt
 ```
 
-`cleaned.txt` — accepted URLs; `discarded.txt` — rejected ones (trackers, login pages, media files, etc.).
-
-Inspect from Python:
-
-```python
-from courlan import check_url, UrlStore
-
-# check_url returns (url, domain) or None if rejected
-result = check_url('https://example.org/valid-article')
-if result:
-    url, domain = result
-    print(f"Accepted: {url} ({domain})")
 
-store = UrlStore()
-with open('cleaned.txt') as f:
-    store.add_urls([line.strip() for line in f])
-
-for domain in store.get_known_domains():
-    print(f"{domain}: {len(store.find_known_urls(domain))} URL(s)")
-```
-
-Sample by domain for large lists:
+## Troubleshooting
+- If the command fails, run `courlan --help` to inspect flags and check your input file encoding (UTF-8 expected).
+- For development, prefer `pip install -e .` so local changes take effect immediately.
 
-```bash
-courlan -i urls.txt -o sample.txt --sample 5 --exclude-min 2
-```
 
 ## Where to go next
-- **CLI Reference**: all flags and examples
-- **API Reference**: programmatic integration
+- **[Python Usage](usage/python.md)**: URL checking, cleaning, link extraction, and sampling
+- **[URL Store](usage/urlstore.md)**: domain-classified URL storage
+- **[Web Crawling](usage/crawling.md)**: building crawlers with courlan
+- **[CLI Reference](usage/cli.md)**: all flags and examples
+- **[Troubleshooting](troubleshooting.md)**: common issues and fixes
+- **[API Reference](api/index.md)**: full module reference
diff --git a/docs/source/index.md b/docs/source/index.md
index a80875d..7c93853 100644
--- a/docs/source/index.md
+++ b/docs/source/index.md
@@ -1,14 +1,87 @@
-# courlan
+---
+myst:
+  html_meta:
+    description: "courlan — Python library for URL filtering, normalization, cleaning, and web crawl scheduling."
+---
 
-courlan cleans, filters, normalizes, and samples URLs. It is designed as a building block for web crawlers and scrapers: steer clear of low-value pages, identify content by language, and deduplicate URL collections at scale.
+# courlan — URL Filtering and Normalization for Python
+
+courlan provides an additional "brain" for web crawling, scraping, and document management. It facilitates web navigation through a set of filters to enhance the quality of resulting document collections: save bandwidth by steering clear of low-value pages, identify content by language, and deduplicate URL collections at scale.
+
+## Common tasks
+
+::::{grid} 2
+:gutter: 3
+
+:::{grid-item-card} Validate & filter URLs
+:link: usage/python
+:link-type: doc
+
+`check_url` — validate, normalize, and filter in one call.
+:::
+
+:::{grid-item-card} Clean & normalize
+:link: usage/python
+:link-type: doc
+
+`clean_url`, `normalize_url` — fix up messy URLs without full filtering.
+:::
+
+:::{grid-item-card} Extract links from HTML
+:link: usage/python
+:link-type: doc
+
+`extract_links`, `filter_links` — general-purpose and crawl-aware extraction.
+:::
+
+:::{grid-item-card} Sample by domain
+:link: usage/python
+:link-type: doc
+
+`sample_urls` — pick N URLs per domain from a larger collection.
+:::
+
+:::{grid-item-card} URL Store
+:link: usage/urlstore
+:link-type: doc
+
+`UrlStore` — domain-classified storage with visit tracking and persistence.
+:::
+
+:::{grid-item-card} Web crawling
+:link: usage/crawling
+:link-type: doc
+
+Crawl delays, robots.txt, download scheduling, and frontier management.
+:::
+
+:::{grid-item-card} Command line
+:link: usage/cli
+:link-type: doc
+
+`courlan` CLI — filter and sample URL files from the terminal.
+:::
+
+:::{grid-item-card} Settings
+:link: api/settings
+:link-type: doc
+
+Customize blacklists, allowed parameters, and language rules.
+:::
+
+::::
 
 ```{toctree}
 :maxdepth: 2
 :caption: Contents
 
 getting-started
-usage/cli
+usage/python
+usage/urlstore
 usage/crawling
+usage/cli
+troubleshooting
+changelog
 api/index
 
 ```
diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md
new file mode 100644
index 0000000..a087b04
--- /dev/null
+++ b/docs/source/troubleshooting.md
@@ -0,0 +1,72 @@
+# Courlan Troubleshooting & FAQ
+
+This page lists common issues and quick fixes when using courlan.
+
+## CLI: nothing written to output
+
+- Ensure input file is UTF-8 encoded and contains one URL per line.
+- Confirm flags: `-i INPUTFILE -o OUTPUTFILE` are provided.
+- Run with `-v` for verbose logging to see why URLs were rejected.
+
+## Slow performance / high memory
+
+- Use `UrlStore(compressed=True)` to reduce memory use.
+- For bulk processing, split the input into chunks and run multiple
+  jobs (see CLI docs). Use `-p` for parallel workers in batch mode.
+- Clear internal caches in long-running processes: `courlan.meta.clear_caches()`.
+
+## Redirect checks are slow
+
+- Redirect checks (`--redirects` / `with_redirects=True`) perform HTTP
+  HEAD requests per URL and are network-bound. Only enable for small
+  datasets or when resolving chains is necessary.
+- Consider running redirect checks as a separate validation step on a
+  filtered subset.
+
+## UrlStore "missing" URLs
+
+- `add_urls()` silently drops URLs rejected by filters. If URLs
+  disappear, validate them with `check_url()` to see which rule
+  rejected them.
+- Ensure `language` and `strict` constructor options match your needs.
+
+## Pickle / load_store warnings
+
+- Saved state uses Python pickle. Never load pickle files from untrusted
+  sources. For sharing, export with `dump_urls()` instead.
+
+## Language detection surprises
+
+- Language signals use path segments, query parameters, and subdomains.
+  Some sites do not encode language explicitly; use `language=None` to
+  disable strict filtering or add custom `TARGET_LANGS` entries.
+
+## Tests failing locally
+
+- Ensure dev dependencies are installed: `pip install -e '.[dev]'`.
+- Run `pytest -q` to see failures. Formatting and linting issues can be
+  auto-fixed with `ruff format`.
+
+## Network timeouts and retries
+
+- Use a downloader with retry/backoff for transient network errors.
+- Keep `--redirects` off for large batches to avoid long-running HTTP
+  calls.
+
+## TypeError with language parameter
+
+`UrlStore.add_from_html` and `filter_links` use `lang=` while most other functions use `language=`. Passing the wrong name raises a `TypeError`:
+
+```python
+# wrong — raises TypeError
+store.add_from_html(html, url, language='en')
+
+# correct
+store.add_from_html(html, url, lang='en')
+```
+
+## Still stuck?
+
+Open an issue on GitHub with: minimal reproduction, input sample, exact
+command or code, and observed vs expected behavior. Include relevant
+log output when possible.
\ No newline at end of file
diff --git a/docs/source/usage/cli.md b/docs/source/usage/cli.md
index 8d8cef6..d82b397 100644
--- a/docs/source/usage/cli.md
+++ b/docs/source/usage/cli.md
@@ -1,6 +1,6 @@
-# CLI Reference
+# Courlan Command-Line Interface
 
-The courlan command-line utility is installed as the `courlan` entry point.
+The main functions are also available through the `courlan` command-line utility, which reads URLs from a text file and writes accepted ones to an output file.
 
 ```bash
 courlan -i INPUTFILE -o OUTPUTFILE [options]
@@ -14,7 +14,7 @@ courlan -i INPUTFILE -o OUTPUTFILE [options]
 | `-o, --outputfile` | Output file (required) |
 | `-d, --discardedfile` | Write rejected URLs to this file |
 | `-v, --verbose` | Enable debug logging |
-| `-p, --parallel` | Worker processes for batch mode (default: 1) |
+| `-p, --parallel` | Worker processes for batch mode (default: number of CPUs) |
 | `--strict` | Enable more restrictive filtering |
 | `-l, --language` | Keep only URLs matching this ISO 639-1 code (e.g. `en`, `de`) |
 | `-r, --redirects` | Check HTTP redirects (slow — see below) |
@@ -27,11 +27,18 @@ courlan -i INPUTFILE -o OUTPUTFILE [options]
 - **Batch mode** (default): processes all URLs, writes accepted URLs to `--outputfile` and rejected ones to `--discardedfile` if specified. Parallelism controlled by `-p`.
 - **Sampling mode** (`--sample`): samples N URLs per domain; `-p` is ignored.
 
-## Complete CLI examples
+## Examples
 
-### Example 1: Basic filtering with output capture
+### Basic filtering
 
-**Input file** (`urls.txt`):
+```bash
+courlan -i urls.txt -o cleaned.txt -d discarded.txt
+```
+
+:::{dropdown} Show input/output
+:icon: file-code
+
+**Input** (`urls.txt`):
 ```
 https://www.example.com/page1
 https://www.example.com/page2
@@ -40,47 +47,46 @@ https://cdn.example.com/image.jpg
 https://example.org/article
 ```
 
-**Command**:
-```bash
-courlan -i urls.txt -o cleaned.txt -d discarded.txt
-```
-
-**Output files**:
-
-`cleaned.txt` (accepted URLs):
+**`cleaned.txt`** (accepted):
 ```
 https://www.example.com/page1
 https://www.example.com/page2
+https://example.com/archive
 https://example.org/article
 ```
 
-`discarded.txt` (rejected URLs):
+**`discarded.txt`** (rejected):
 ```
-https://example.com/archive
 https://cdn.example.com/image.jpg
 ```
+:::
 
-### Example 2: Strict filtering with language detection
+### Strict filtering with language detection
 
-**Command**:
 ```bash
 courlan -i urls.txt -o cleaned.txt -d discarded.txt --strict -l en
 ```
 
 More restrictive filtering applied; only English URLs kept.
 
-### Example 3: Parallel processing with verbose output
+### Parallel processing with verbose output
 
-**Command** (4 worker processes, debug logging):
 ```bash
 courlan -i urls.txt -o cleaned.txt -p 4 -v
 ```
 
-Outputs debug information about each URL processing step.
+4 worker processes, debug logging for each URL processing step.
+
+### Sampling by domain
+
+```bash
+courlan -i large_urls.txt -o sample.txt --sample 2 --exclude-min 2
+```
 
-### Example 4: Sampling by domain
+:::{dropdown} Show input/output
+:icon: file-code
 
-**Input file** (`large_urls.txt`):
+**Input** (`large_urls.txt`):
 ```
 https://github.com/adbar/courlan
 https://github.com/adbar/trafilatura
@@ -91,20 +97,14 @@ https://example.com/page3
 https://another.org/article
 ```
 
-**Command** (2 URLs per domain, exclude domains with <2 URLs):
-```bash
-courlan -i large_urls.txt -o sample.txt --sample 2 --exclude-min 2
-```
-
-**Output** (`sample.txt`):
+**`sample.txt`** (2 per domain, domains with <2 URLs excluded):
 ```
 https://github.com/adbar/courlan
 https://github.com/adbar/trafilatura
 https://example.com/page1
 https://example.com/page2
 ```
-
-Result: 4 URLs selected (2 per domain that meets the exclusion criteria).
+:::
 
 ## Large inputs
 
diff --git a/docs/source/usage/crawling.md b/docs/source/usage/crawling.md
index 01477d9..bc04ca1 100644
--- a/docs/source/usage/crawling.md
+++ b/docs/source/usage/crawling.md
@@ -1,12 +1,17 @@
+---
+myst:
+  html_meta:
+    description: "Build web crawlers with courlan: crawl delays, robots.txt, download scheduling, frontier management."
+---
+
 # Web Crawling with Courlan
 
-Guide to building web crawlers with courlan: frontier management, crawl delays, link extraction, and persistence.
+Guide to building web crawlers with courlan: crawl delays, robots.txt, download scheduling, frontier management, and link extraction.
 
-## UrlStore for Crawler State
+This guide assumes familiarity with `UrlStore` basics — see the [URL Store guide](urlstore.md) first.
 
-The `UrlStore` class manages the crawl frontier: tracking visited/unvisited URLs per domain and handling robots.txt rules.
 
-### Basic Crawler Loop
+## Basic crawler loop
 
 ```python
 from courlan import UrlStore
@@ -25,28 +30,13 @@ while store.unvisited_websites_number() > 0:
             continue
         print(f"Visiting: {url}")
         # response = requests.get(url, timeout=10)
-        # store.add_urls(extract_links(response.text, url))
+        # store.add_from_html(response.text, url)
 ```
 
-### Key UrlStore Methods
 
-| Method | Purpose |
-|--------|---------|
-| `add_urls(urls)` | Add URLs to frontier |
-| `get_url(domain)` | Retrieve next URL and mark as visited |
-| `get_unvisited_domains()` | Domains with unvisited URLs |
-| `unvisited_websites_number()` | Count of domains with remaining URLs |
-| `establish_download_schedule(max_urls, time_limit)` | Batch URLs with per-domain delays |
-| `download_threshold_reached(threshold)` | Check if time limit exceeded |
-| `find_unvisited_urls(domain)` | List unvisited URLs for a domain |
-| `is_exhausted_domain(domain)` | Check if domain has no more URLs |
-| `write(filename)` | Save state to disk |
+## Crawl delays and robots.txt
 
----
-
-## Crawl Delays
-
-Use `get_crawl_delay()` to read the delay from stored robots.txt rules, and `store_rules()` / `get_rules()` to persist them.
+Use `store_rules()` / `get_rules()` to persist robots.txt rules, and `get_crawl_delay()` to read the delay.
 
 ```python
 from courlan import UrlStore
@@ -56,7 +46,7 @@ import time
 store = UrlStore()
 domain = 'https://example.com'
 
-# Store robots.txt rules after fetching
+# Fetch and store robots.txt rules (requires network access)
 rules = RobotFileParser(f'{domain}/robots.txt')
 rules.read()
 store.store_rules(domain, rules)
@@ -67,7 +57,8 @@ time.sleep(delay)
 url = store.get_url(domain)
 ```
 
-### Scheduled Download Strategy
+
+## Scheduled downloads
 
 For large crawls, `establish_download_schedule()` batches URLs with appropriate per-domain delays:
 
@@ -83,6 +74,9 @@ store.add_urls([
 
 schedule = store.establish_download_schedule(max_urls=100, time_limit=10)
 
+# or get a flat list of immediately-downloadable URLs (no delays)
+ready = store.get_download_urls(max_urls=50, time_limit=10)
+
 for delay, url in schedule:
     time.sleep(delay)
     print(f"Fetching: {url}")
@@ -91,11 +85,10 @@ for delay, url in schedule:
         break
 ```
 
----
 
-## Crawler Frontier Management
+## Frontier management
 
-### Scope Detection
+### Scope detection
 
 ```python
 from courlan import is_external
@@ -104,66 +97,45 @@ if not is_external(found_url, 'https://example.com', ignore_suffix=False):
     store.add_urls([found_url])
 ```
 
-### Navigation Page Detection
+### Crawlability detection
 
 ```python
-from courlan import is_navigation_page
+from courlan import is_not_crawlable, is_navigation_page
 
 for url in candidate_urls:
-    if not is_navigation_page(url):
-        store.add_urls([url])  # content page, high priority
+    if is_not_crawlable(url):
+        continue  # skip login pages, deep web, etc.
+    if is_navigation_page(url):
+        continue  # skip listing/index pages
+    store.add_urls([url])
 ```
 
-### Crawlability Detection
-
-```python
-from courlan import is_not_crawlable
-
-for url in candidate_urls:
-    if not is_not_crawlable(url):
-        store.add_urls([url])
+```{note}
+`filter_links` already separates navigation pages into a priority list
+(via `with_nav=True` by default). The manual check above is useful when
+you process URLs outside of `filter_links`.
 ```
 
----
 
-## Extracting Links from HTML
+## Extracting links from HTML
 
 ```python
 from courlan import extract_links
 
 links = extract_links(
     html,
-    base_url,
+    url,
     external_bool=False,
     language='en',
-    strict=True,
+    # strict=True is the default for extract_links
 )
 store.add_urls(links)
 ```
 
-`extract_links` also accepts `no_filter`, `redirects`, and `with_nav` — see the API reference for details.
+`extract_links` also accepts `no_filter`, `trailing_slash`, `with_nav`, `redirects`, `reference`, and `base_url` — see the [API reference](../api/core.md) for details.
 
----
-
-## Persistence and Resume
-
-```python
-from courlan import UrlStore, load_store
-
-store = UrlStore()
-store.add_urls(['https://example.com/page1', 'https://example.com/page2'])
-store.get_url('https://example.com')
-
-store.write('crawler_state.db')
-
-# Later session:
-store = load_store('crawler_state.db')
-print(f"Unvisited domains: {store.get_unvisited_domains()}")
-```
 
----
-
-## Best Practices
+## Best practices
 
 | Practice | Reason |
 |----------|--------|
@@ -171,14 +143,13 @@ print(f"Unvisited domains: {store.get_unvisited_domains()}")
 | Set crawl delays | Avoid overloading servers |
 | Identify User-Agent | Tell servers who you are |
 | Save crawler state | Resume after interruptions |
-| Skip navigation pages | Focus on content |
+| Separate navigation pages | Crawl them for link discovery, deprioritize for content extraction |
 | Validate URLs | Avoid malformed requests |
 | Handle errors gracefully | Don't crash on bad pages |
 | Limit crawl scope | Stay on target domain(s) |
 
----
 
-## Complete Example
+## Complete example
 
 ```python
 from courlan import UrlStore, extract_links, is_not_crawlable
@@ -205,9 +176,8 @@ while store.unvisited_websites_number() > 0 and pages_crawled < 100:
 store.write('crawler_state.db')
 ```
 
----
 
-## Troubleshooting
+## Troubleshooting crawls
 
 **URL not added to store** — `UrlStore.add_urls()` silently drops invalid URLs. Validate first:
 
diff --git a/docs/source/usage/python.md b/docs/source/usage/python.md
new file mode 100644
index 0000000..26fe3ef
--- /dev/null
+++ b/docs/source/usage/python.md
@@ -0,0 +1,275 @@
+---
+myst:
+  html_meta:
+    description: "How to validate, clean, normalize, and filter URLs in Python with courlan."
+---
+
+# URL Checking, Cleaning, and Filtering in Python
+
+Most filters revolve around the `strict` and `language` arguments. This page covers URL checking, cleaning, normalization, link extraction, and sampling.
+
+
+## Checking URLs with check_url
+
+`check_url` is the main entry point — it validates, normalizes, and filters a URL in one call. Returns `(url, domain)` on success or `None` if rejected.
+
+```{note}
+`check_url` returns `None` for rejected URLs — always check the return value before unpacking.
+```
+
+```python
+from courlan import check_url
+
+result = check_url('https://example.com/article?utm_source=twitter')
+if result:
+    url, domain = result
+    # url = 'https://example.com/article', domain = 'example.com'
+```
+
+### Language filtering
+
+Pass a two-letter ISO 639-1 code to keep only URLs that match the target language (detected from path segments, subdomains, and query parameters):
+
+```python
+# accepted: English path segment
+check_url('https://www.un.org/en/about-us', language='en')
+# ('https://www.un.org/en/about-us', 'un.org')
+
+# rejected: English URL but German requested
+check_url('https://www.un.org/en/about-us', language='de')
+# None
+```
+
+For standalone language detection without the full check_url pipeline, use `lang_filter`:
+
+```python
+from courlan.filters import lang_filter
+
+if lang_filter('https://example.com/en/article', language='en'):
+    print("Language matches")
+```
+
+### Strict mode
+
+`strict=True` enables more aggressive filtering — blacklisted domains, adult content, and suspicious paths are rejected, query parameters are trimmed more aggressively, and language detection uses subdomains too.
+
+:::{dropdown} Full comparison: default vs strict
+:icon: table
+
+| Area | Default | With `strict=True` |
+|------|---------|-------------------|
+| **Query parameters** | Only known trackers removed | All parameters removed except a small allowlist (`page`, `id`, `post`, etc.) |
+| **URL fragments** | Normalized | Stripped entirely |
+| **File extensions** | Non-web extensions rejected (`.jpg`, `.pdf`, `.zip`, etc.) | Same, plus pattern-based detection (e.g. `/img/`, `?format=pdf`) |
+| **Adult/video content** | Not checked | URLs with adult or video path patterns rejected |
+| **Domain blacklist** | Not applied | URLs from ~77 blacklisted platforms rejected (social media, CDNs, e-commerce, etc.) |
+| **Path filtering** | Not applied | URLs with suspicious path patterns rejected (e.g. long query-heavy paths) |
+| **Language detection** | Path-based only | Subdomain-based language signals also considered |
+:::
+
+```python
+# blocked in strict mode: major platform in the blacklist
+check_url('https://www.twitch.com/', strict=True)
+# None
+
+# query parameters trimmed more aggressively
+check_url('https://httpbin.org/redirect-to?url=http%3A%2F%2Fexample.org', strict=True)
+# ('https://httpbin.org/redirect-to', 'httpbin.org')
+
+# adult content pattern rejected in strict mode
+check_url('https://example.com/porn/page', strict=True)
+# None
+```
+
+The blacklist and allowlists can be customized at runtime — see the [settings reference](../api/settings.md).
+
+### Other options
+
+| Option | Effect |
+|--------|--------|
+| `with_redirects=True` | Follow HTTP redirects (HEAD request — slow) |
+| `with_nav=True` | Accept navigation/listing pages instead of discarding them |
+| `trailing_slash=False` | Strip trailing slashes |
+
+### Filtering cost
+
+Options add overhead in this order, from cheapest to most expensive:
+
+1. **Basic** — `check_url(url)`
+2. **Language filtering** — `check_url(url, language='en')` — minimal overhead
+3. **Strict mode** — `check_url(url, strict=True)` — more conditions checked
+4. **Redirect checks** — `check_url(url, with_redirects=True)` — network I/O; avoid on large datasets
+
+All options compose freely, e.g. `check_url(url, language='en', strict=True)`.
+
+
+## Cleaning and normalizing URLs
+
+```{warning}
+`clean_url` normalizes but does **not** validate. It can return `None`
+on malformed input but will happily return a munged string for
+structurally valid garbage. Use `check_url` or `validate_url` if you
+need to reject invalid URLs.
+```
+
+For cleaning without the full filtering pipeline:
+
+```python
+from courlan import clean_url
+
+# Lowercase scheme/host, strip default port, remove trackers
+clean_url('HTTPS://WWW.DWDS.DE:443/')
+# 'https://www.dwds.de'
+```
+
+For low-level scrubbing only (strip markup residues, control characters, and tracking artifacts without normalizing):
+
+```python
+from courlan import scrub_url
+
+scrub_url('<![CDATA[http://example.com]]>')
+# 'http://example.com'
+```
+
+For canonicalization only (reorder query params, strip fragments):
+
+```python
+from courlan import normalize_url
+
+normalize_url('http://test.net/foo.html?utm_source=twitter&post=abc&page=2#fragment', strict=True)
+# 'http://test.net/foo.html?page=2&post=abc'
+```
+
+For structural validation without normalization:
+
+```python
+from courlan import validate_url
+
+validate_url('http://1234')
+# (False, None)
+
+validate_url('http://www.example.org/')
+# (True, SplitResult(...))
+```
+
+For a simple boolean check (wrapper around `validate_url`):
+
+```python
+from courlan import is_valid_url
+
+is_valid_url('http://www.example.org/')
+# True
+```
+
+
+## URL parsing and decomposition utilities
+
+Decompose and manipulate URLs with `extract_domain`, `get_base_url`, `get_host_and_path`, `get_hostinfo`, and `fix_relative_urls`:
+
+```python
+from courlan import extract_domain, get_base_url, get_host_and_path, get_hostinfo, fix_relative_urls
+
+url = 'https://www.un.org/en/about-us'
+
+extract_domain(url)             # 'un.org'
+
+get_base_url(url)               # 'https://www.un.org'
+get_host_and_path(url)          # ('https://www.un.org', '/en/about-us')
+get_hostinfo(url)               # ('un.org', 'https://www.un.org')
+
+fix_relative_urls('https://www.un.org', 'en/about-us')
+# 'https://www.un.org/en/about-us'
+```
+
+Filter and deduplicate URL lists:
+
+```python
+from courlan import filter_urls
+
+subset = filter_urls(url_list, urlfilter='example.com')
+```
+
+
+## Extracting links from HTML with extract_links
+
+Use `extract_links` for general-purpose link extraction from HTML:
+
+```python
+from courlan import extract_links
+
+html = '<html><body><a href="test/link.html">Link</a></body></html>'
+links = extract_links(html, 'https://example.org')
+# {'https://example.org/test/link.html'}
+```
+
+For crawl-aware extraction with robots.txt rules and link prioritization, use `filter_links` — it returns two lists separating regular links from navigation/listing links:
+
+```python
+from courlan import filter_links
+
+html = '<a href="page1.html">Article</a><a href="/tag/listing">Tag</a>'
+links, priority_links = filter_links(html, 'https://example.org', lang='en')
+# links = ['https://example.org/page1.html']
+# priority_links = ['https://example.org/tag/listing']
+```
+
+`extract_links` accepts `external_bool`, `no_filter`, `language`, `strict`, `trailing_slash`, `with_nav`, `redirects`, `reference`, and `base_url`; `filter_links` accepts `external`, `lang`, `rules`, `strict`, and `with_nav`. See the [API reference](../api/core.md) for full signatures.
+
+
+## Sampling URLs by domain with sample_urls
+
+Sample a fixed number of URLs per domain from a larger collection:
+
+```python
+from courlan import sample_urls
+
+urls = ['https://example.org/' + str(x) for x in range(100)]
+sample = sample_urls(urls, samplesize=10)
+# 10 randomly selected URLs from example.org
+```
+
+Exclude domains that are too small or too large:
+
+```python
+sample = sample_urls(urls, samplesize=5, exclude_min=2, exclude_max=1000)
+```
+
+See also `courlan --sample` in the [CLI reference](cli.md).
+
+
+## Scope and crawlability checks
+
+Determine if a link leads to another host:
+
+```python
+from courlan import is_external
+
+is_external('https://github.com/', 'https://www.microsoft.com/')
+# True
+
+# Ignore domain suffixes — the default (treats .com and .co.uk as same)
+is_external('https://google.com/', 'https://www.google.co.uk/')
+# False
+```
+
+Check if a URL is usable in a crawling context:
+
+```python
+from courlan import is_not_crawlable, is_navigation_page
+
+is_not_crawlable('https://example.com/login')
+# True
+
+is_navigation_page('https://example.com/category/myposts')
+# True
+```
+
+
+## Cache management
+
+Courlan uses LRU caches internally. In long-running processes, clear them periodically to reclaim memory:
+
+```python
+from courlan.meta import clear_caches
+clear_caches()
+```
diff --git a/docs/source/usage/urlstore.md b/docs/source/usage/urlstore.md
new file mode 100644
index 0000000..2814e89
--- /dev/null
+++ b/docs/source/usage/urlstore.md
@@ -0,0 +1,197 @@
+# URL Store — Domain-Classified URL Storage
+
+The `UrlStore` class allows for storing and retrieving domain-classified URLs, where a URL like `https://example.org/path/page` is stored as the path `/path/page` within the domain `https://example.org`. It tracks visited and unvisited URLs per domain, supports crawl scheduling with per-domain delays, and can persist state to disk.
+
+
+## Basic UrlStore usage
+
+```{note}
+`add_urls` silently drops URLs that fail validation. If URLs seem to
+disappear, check them with `check_url` first to see why they are rejected.
+```
+
+```python
+from courlan import UrlStore
+
+store = UrlStore()
+store.add_urls([
+    'https://example.com/page1',
+    'https://example.com/page2',
+    'https://example.org/article',
+])
+
+# Retrieve a URL (marks it as visited with a timestamp)
+url = store.get_url('https://example.com')
+# 'https://example.com/page1'
+
+# Check what's left
+store.find_unvisited_urls('https://example.com')
+# ['https://example.com/page2']
+```
+
+
+## UrlStore constructor options
+
+| Option | Effect |
+|--------|--------|
+| `compressed=True` | Compress stored URLs and rules to reduce memory |
+| `language='en'` | Filter added URLs by target language (ISO 639-1 code) |
+| `strict=True` | Apply stricter URL filtering on add |
+| `trailing_slash=True` | Preserve trailing slashes (set to `False` to strip them) |
+| `verbose=True` | Dump URLs on interrupt (requires `signal`) |
+
+```python
+store = UrlStore(language='en', strict=True, compressed=True)
+```
+
+
+## Tracking visited and unvisited URLs
+
+```python
+# Check if a URL is already known
+store.is_known('https://example.com/page1')
+# True
+
+# Check if it has been visited
+store.has_been_visited('https://example.com/page1')
+# True (we called get_url above)
+
+# Filter a list to only unknown or unvisited URLs
+new_urls = ['https://example.com/page1', 'https://example.com/new']
+store.filter_unknown_urls(new_urls)
+# ['https://example.com/new']
+store.filter_unvisited_urls(new_urls)
+# ['https://example.com/new']
+```
+
+
+## Adding links from HTML
+
+Extract, filter, and add links from a page in one call:
+
+```python
+html = '<a href="/page3">Link</a><a href="https://other.com/x">External</a>'
+store.add_from_html(html, 'https://example.com')
+# internal links added; external links ignored by default
+
+# include external links
+store.add_from_html(html, 'https://example.com', external=True)
+
+# filter by language (note: the parameter is `lang`, not `language`)
+store.add_from_html(html, 'https://example.com', lang='en')
+```
+
+
+## UrlStore statistics and inspection
+
+```python
+store.total_url_number()         # total URLs across all domains
+store.get_known_domains()        # list of all domains
+store.get_unvisited_domains()    # domains with unvisited URLs
+store.unvisited_websites_number()  # count of such domains
+store.get_all_counts()           # download counts per host
+store.is_exhausted_domain('https://example.com')  # all URLs visited?
+
+# Per-domain inspection
+store.find_known_urls('https://example.com')
+store.find_unvisited_urls('https://example.com')
+store.dump_urls()                # all URLs as a flat list
+```
+
+
+## Printing and resetting
+
+```python
+store.print_unvisited_urls()     # print all unvisited URLs to stdout
+store.print_urls()               # print all URLs with visited/unvisited status
+store.reset()                    # clear the store and internal caches
+```
+
+
+## Discarding domains
+
+Remove entire domains from the store:
+
+```python
+store.discard(['https://spam.example.com'])
+```
+
+
+## Saving and loading UrlStore state
+
+Save and restore state across sessions. `UrlStore.write()` stores state
+using Python's pickle format (see security note below). `load_store()`
+restores a previously saved store.
+
+```python
+from courlan import UrlStore, load_store
+
+store = UrlStore()
+store.add_urls(['https://example.com/1', 'https://example.com/2'])
+store.get_url('https://example.com')  # visit one
+
+# Save state to disk
+store.write('crawler_state.db')
+
+# Later: resume from disk
+store = load_store('crawler_state.db')
+print(store.get_unvisited_domains())
+```
+
+### Persistence tips and compressed mode
+
+- Use `compressed=True` at construction to reduce memory and on-disk
+  footprint when saving large crawls: `UrlStore(compressed=True)`.
+- For very large crawls, combine periodic saves with incremental files
+  (e.g., `crawler_state-0001.db`, `crawler_state-0002.db`) to avoid
+  single large writes and to make resuming more robust.
+- If you need a human-inspectable export, use `store.dump_urls()` and
+  write the flat list to a newline-delimited file.
+
+### Common crawl workflow
+
+A typical crawl loop using UrlStore:
+
+```python
+store = UrlStore(language='en', strict=True, compressed=True)
+seed_urls = ['https://example.com']
+store.add_urls(seed_urls)
+
+while store.unvisited_websites_number() > 0:
+    domains = store.get_unvisited_domains()
+    if not domains:
+        break
+    url = store.get_url(domains[0])
+    html = fetch(url)  # your downloader
+    store.add_from_html(html, url, external=False)
+    # process page and enqueue new links
+    if should_checkpoint():
+        store.write('crawler_state_checkpoint.db')
+```
+
+This pattern: keep frequently-accessed state in memory, persist
+periodically, and prefer compressed mode for long runs.
+
+```{warning}
+`write()`/`load_store()` use Python's pickle format, which can execute
+arbitrary code when loading. Only load files you have written yourself
+or that you trust. Consider exporting via `dump_urls()` for sharing.
+```
+
+
+## Thread safety
+
+Readers are safe and writers are serialized, but a logical write is not globally atomic — drive mutations from a single writer thread.
+
+
+## Performance tips
+
+- Use `compressed=True` for large crawls to reduce memory usage
+- Save periodically with `write()` to allow resume after interruptions
+- Set `language` at construction time to filter URLs on add rather than later
+- Clear internal caches periodically in long-running processes: `courlan.meta.clear_caches()`
+
+
+## Next steps
+
+For crawl-specific workflows (robots.txt, crawl delays, download scheduling, frontier management), see the [Web Crawling guide](crawling.md).
diff --git a/docs/test_docs.py b/docs/test_docs.py
index e3eaae3..f86f84b 100644
--- a/docs/test_docs.py
+++ b/docs/test_docs.py
@@ -1,27 +1,25 @@
-import shutil
+"""Build the documentation with Sphinx to catch broken pages or references.
+Run in CI for the Python version used on Read the Docs."""
+
 import subprocess
 from pathlib import Path
 
 import pytest
 
 DOCS_SOURCE = Path(__file__).parent / "source"
-DOCS_BUILD = Path(__file__).parent / "_build"
-DOCS_HTML = DOCS_BUILD / "html"
-
-
-@pytest.fixture(autouse=True, scope="module")
-def clean_build():
-    if DOCS_BUILD.exists():
-        shutil.rmtree(DOCS_BUILD)
-    yield
 
 
-def test_sphinx_build_succeeds():
-    cmd = ["sphinx-build", "-W", "-b", "html", str(DOCS_SOURCE), str(DOCS_HTML)]
-    result = subprocess.run(cmd, capture_output=True, text=True, timeout=120)
+def test_sphinx_build_succeeds(tmp_path):
+    """The documentation compiles cleanly (warnings treated as errors)."""
+    result = subprocess.run(
+        ["sphinx-build", "-W", "-b", "html", str(DOCS_SOURCE), str(tmp_path)],
+        capture_output=True,
+        text=True,
+        timeout=120,
+    )
     if result.returncode != 0:
         pytest.fail(
             f"Sphinx build failed (exit {result.returncode})\n"
             f"STDOUT:\n{result.stdout}\nSTDERR:\n{result.stderr}"
         )
-    assert (DOCS_HTML / "index.html").exists()
+    assert (tmp_path / "index.html").exists()
diff --git a/pyproject.toml b/pyproject.toml
index 19d51ee..ac02ff0 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -65,6 +65,7 @@ courlan = "courlan.cli:main"
 
 [project.urls]
 "Homepage" = "https://github.com/adbar/courlan"
+"Documentation" = "https://courlan.readthedocs.io/en/latest/"
 "Blog" = "https://adrien.barbaresi.eu/blog/"  # /tag/courlan.html
 "Tracker" = "https://github.com/adbar/courlan/issues"
 
@@ -80,8 +81,11 @@ dev = [
 docs = [
     "sphinx>=6.2",
     "myst-parser>=0.19.0",
-    "sphinx-rtd-theme>=1.2.0",
+    "furo>=2024.1.29",
     "sphinx-copybutton>=0.5.0",
+    "sphinx-design>=0.6.0",
+    "sphinxext-opengraph>=0.9.0",
+    "sphinx-sitemap>=2.5.0",
 ]
 
 [tool.pytest.ini_options]