userscripting

CHANGELOG.md

@@ -2,6 +2,213 @@
 
 ## Unreleased
 
+- Userscript engine: Greasemonkey/Tampermonkey-style scripts injected into
+  matching pages, managed at runtime from the web UI.
+  - New **Settings → Userscripts** page: install a script by pasting it or by
+    URL (e.g. from Greasyfork), toggle scripts individually, edit a script's
+    source in place, and uninstall it. A master switch disables the whole
+    engine without clearing the per-script selection. Each entry shows its
+    version, `@run-at`, match patterns, grant count and `@noframes` state, and
+    a script whose stored body no longer parses is called out as not being
+    injected instead of failing silently.
+  - New `[userscripts]` configuration section holding the master `enabled` flag
+    and one `[[userscripts.scripts]]` entry per script. As with filter lists,
+    only metadata is stored in the configuration file; bodies live under
+    `userscripts/` in the configuration directory (override with
+    `PRIVAXY_USERSCRIPT_PATH`), keyed by a hash of the source URL so
+    re-installing the same script reuses its file. Configurations written
+    before this release parse unchanged and default to an enabled engine with
+    no scripts.
+  - Scripts are matched on the same canonical URL the adblock engine uses,
+    supporting `@match` (Chrome match-pattern syntax including `<all_urls>` and
+    `*.host` wildcards), `@include`/`@exclude` (globs, or regular expressions
+    written `/…/`), `@exclude-match`, `@run-at`
+    (`document-start`/`-body`/`-end`/`-idle`) and `@noframes`. Exclusions take
+    precedence over inclusions. A script declaring neither `@match` nor
+    `@include` — or with a malformed pattern or no metadata block — is rejected
+    with a `422` at install time rather than being stored and never firing.
+  - The in-page runtime provides `GM_info`, `unsafeWindow`, `GM_addStyle`
+    (nonce-stamped so it survives the page's CSP), `GM_log`, `GM_openInTab`,
+    `GM_setClipboard`, `GM_notification`, `GM_registerMenuCommand` and the
+    promise-based `GM.*` namespace.
+  - `@require` libraries and `@resource` payloads are fetched server-side and
+    cached on disk under `userscripts/assets/`, keyed by URL hash. Requires are
+    evaluated inside the script's own wrapper ahead of its body, so their
+    top-level declarations are visible to the script without leaking into the
+    page, and `GM_getResourceText`/`GM_getResourceURL` read the fetched
+    resources. Cached assets are never re-fetched (the convention is to pin a
+    versioned URL); delete the `assets` directory to refresh. An asset that
+    cannot be fetched degrades its script rather than dropping it, and the
+    failure is reported on the Userscripts page.
+  - `GM_setValue`/`GM_getValue`/`GM_deleteValue`/`GM_listValues` are persisted,
+    scoped per script, in `userscripts/gm_storage.json` (writes are coalesced
+    on a short debounce and written atomically; the configuration file is
+    deliberately not used, since these are written far too often to
+    re-serialize it each time). Because `GM_getValue` is synchronous in the GM
+    API, values are preloaded into each script's descriptor at injection time
+    and read from that snapshot in-page, so an ordinary `GM_getValue` costs no
+    request. Uninstalling a script drops its values.
+  - `GM_xmlhttpRequest` is relayed server-side, so it has no CORS restrictions
+    — the one capability a real content script cannot have. Three independent
+    controls gate it: the origin-bound token, the requesting script's own
+    `@connect` declarations (as Tampermonkey requires, so compatibility is
+    unaffected), and a filter rejecting loopback, RFC1918, carrier-grade NAT,
+    link-local (including the cloud metadata address) and IPv4-mapped
+    equivalents. Redirects are followed manually so `@connect` and address
+    filtering re-run on every hop rather than letting an allow-listed host
+    bounce the request to `127.0.0.1`. Methods are restricted, `Host` and
+    hop-by-hop request headers cannot be set, and responses are size-capped.
+  - `@resource` payloads may be binary. They are stored as bytes with the
+    content type they were served as; text small enough to matter is inlined
+    into the script's descriptor so `GM_getResourceText` stays synchronous,
+    while anything binary or oversized is reachable through
+    `GM_getResourceURL`, served from the reserved path with its original bytes,
+    content type and `X-Content-Type-Options: nosniff`. `GM_getResourceText` on
+    a binary resource returns `null` and logs a note pointing at
+    `GM_getResourceURL` rather than returning mojibake.
+  - `GM_addValueChangeListener`/`GM_removeValueChangeListener` are implemented.
+    Changes fire locally, reach other same-origin tabs over `BroadcastChannel`
+    (free, no request), and reach other origins and other devices behind the
+    same proxy by polling a read endpoint — which only runs while at least one
+    listener is registered, so a page with no listeners issues no extra
+    requests. The read endpoint requires the requesting URL to satisfy the
+    script's own `@match`/`@include`, so it can never reveal more than the
+    page's own descriptor already contained.
+  - `GM_registerMenuCommand` now has a real surface: a small floating menu,
+    injected only once a script actually registers a command, hosted in a closed
+    shadow root so neither the page's CSS nor Privaxy's can reach across.
+    Commands are still reachable from the console via
+    `__privaxyUserscriptMenu()`.
+  - Userscripts can be disabled for a single tab from that menu, backed by
+    `sessionStorage` — per-tab by construction, so the proxy needs no notion of
+    a tab. Other tabs are unaffected; `__privaxyUserscriptsEnableTab()` restores
+    them, since disabling removes the menu that turned them off.
+  - Userscripts installed from a URL are re-fetched on the same 24h timer as the
+    filter lists and recompiled in place, so upstream changes are picked up
+    without a restart. The refresh re-reads the configuration from disk rather
+    than reusing the updater's own copy: userscript changes deliberately bypass
+    that channel, so its copy is stale with respect to them and recompiling from
+    it would drop every script installed since startup.
+  - `@updateURL` and `@downloadURL` are honored, both defaulting to the URL the
+    script was installed from. When they differ, only the (small) `@updateURL`
+    document is fetched to compare `@version`, and the body is downloaded solely
+    when that version is newer — versions are ordered as dotted numbers, so
+    `1.2.10` correctly supersedes `1.2.9`, falling back to plain inequality for
+    schemes that cannot be ordered. Scripts that split metadata and body, a
+    common Greasyfork layout, previously re-downloaded the whole body on every
+    cycle.
+  - A **Check for updates** button on the Userscripts page refreshes on demand
+    instead of waiting out the timer, reporting per script whether it was
+    updated, already current, or failed and why. Unlike the periodic refresh it
+    holds the save lock, so a changed `@name` or `@version` is persisted.
+  - New `userscripts.allow_private_network_requests` setting (default off, with
+    a toggle on the Userscripts page) permits the relay to reach private
+    addresses. It is off by default because the relay runs server-side: the
+    proxy usually sits *inside* a LAN and can reach routers, admin panels and
+    metadata endpoints no page could contact. Changes apply immediately, with
+    no reload.
+  - Changes apply to the next page load with no reload or restart: compiled
+    scripts live in a shared store that each API mutation replaces in place,
+    and the store is also refreshed on `SIGHUP` so a hand-edited
+    `[userscripts]` section takes effect. Each script is emitted in its own
+    `nonce`d script element, so a syntax error in one script cannot abandon the
+    others or the ad-blocking payload, and the CSP nonce is kept in a closure
+    rather than published on `window`.
+  - Note that a userscript installed here runs on **every client behind the
+    proxy**, in the page's main world (a proxy has no isolated world to offer),
+    which is a wider blast radius than a browser extension installed in one
+    profile. The Userscripts page says so where scripts are added.
+  - **Not supported.** A userscript engine built into a proxy cannot reach full
+    Tampermonkey parity, and some of what is missing is structural rather than
+    unfinished. Known gaps, so a script that misbehaves can be diagnosed instead
+    of guessed at:
+    - *No isolated world, and there cannot be one.* Scripts run in the page's
+      main world, so `unsafeWindow === window`, page scripts can read and clobber
+      anything a userscript leaves reachable, and anti-adblock can detect the
+      injection. Each script is still wrapped in its own function, so its `var`,
+      `let`, `const`, `function` and `class` declarations do not leak to the page
+      — only an undeclared assignment or an explicit `window.x = …` does.
+    - *Pages the proxy never sees get nothing.* A site with an active service
+      worker serving navigations from cache, `file://`, `chrome://`, browser-cache
+      hits and any traffic not routed through Privaxy are all invisible to it, so
+      no script runs there. A browser extension sees all of them.
+    - *`@grant` is parsed and displayed but not enforced.* Every script receives
+      every implemented API regardless of what it declared, including
+      `@grant none`. This is deliberately forgiving — a script that forgot to
+      declare a grant still works — but it is a deviation.
+    - *Not implemented:* `GM_cookie` (the proxy has no cookie jar for the
+      browser's cookies; it only sees `Cookie` headers in flight),
+      `GM_getTab`/`GM_saveTab`/`GM_getTabs` (no tab identity exists on the proxy
+      side), `GM_download` (it would mean the proxy writing files to its own disk
+      on a page's behalf), `GM_addElement`, the batch `GM_setValues`/`GM_getValues`
+      forms, and `window.onurlchange`.
+    - *`GM_notification` writes to the console* rather than raising a real
+      notification, and `GM_setClipboard` needs a user gesture like any page-context
+      clipboard write.
+    - *Metadata ignored:* `@sandbox`, `@unwrap`, `@top-level-await`, `@icon`,
+      `@supportURL`, `@antifeature`. Unknown directives are skipped, not rejected.
+    - *`@resource` payloads are byte-exact but not inlined as text when binary or
+      over 256 KB* — `GM_getResourceText` returns `null` for those and the data is
+      reachable only through `GM_getResourceURL`.
+    - *`GM_xmlhttpRequest` is not a transparent `XMLHttpRequest`.* Responses are
+      decoded as lossy UTF-8, so binary bodies are unusable; `abort()` only
+      suppresses the callbacks, since the server-side request is already in
+      flight; there is no `onprogress`/`onreadystatechange`; redirects are capped
+      at 5 hops, timeouts at 60s (default 30s) and responses at 8 MB; and the
+      request only reaches hosts the script declared with `@connect`.
+    - *Value-change notification is not instant across origins.* Same-origin tabs
+      are updated immediately over `BroadcastChannel`; a change made on a
+      different origin or another device is picked up by a 15s poll, and only
+      while a listener is registered.
+    - *Per-tab disable is per-tab **per origin*** — it is `sessionStorage`, so
+      disabling on one site does not disable on another in the same tab.
+    - *Storage limits:* 1000 keys per script and 64 KB per value; a script body
+      or fetched `@require`/`@resource` is capped at 2 MB. Writes are flushed on a
+      500 ms debounce, so values set immediately before a crash can be lost.
+    - *No script ordering or import/export in the UI.* Injection order is
+      configuration order, changeable only by editing the file.
+- Proxy performance overhaul:
+  - *HTML responses now stream.* The proxy previously withheld an HTML
+    response — status line, headers and all — until the entire upstream
+    document had been downloaded and fed through the rewriter, so the browser
+    could not start parsing (or prefetching subresources) until the last
+    upstream byte arrived. The rewritten document now streams to the client as
+    it is produced, and the rewriter pipeline is bounded end-to-end, so a slow
+    client backpressures the upstream download instead of the whole document
+    buffering in memory.
+  - *WebSocket tunnels no longer squeeze through a 32-byte buffer.* The duplex
+    buffer bridging the client and upstream halves of an upgraded connection
+    was 32 bytes, forcing a task wakeup roughly every 32 bytes transferred;
+    it is now 64 KiB.
+  - *The adblock engine is shared, not funneled through one thread.* The
+    `single-thread` adblock feature is dropped; the engine (Send + Sync) is
+    now called directly from request tasks, removing a channel round-trip and
+    two cross-thread handoffs from every request. Matching itself still
+    serializes briefly on the engine's internal regex-manager lock — the same
+    one-core ceiling as the old blocker thread, far above proxy request rates
+    — but the per-request overhead around it is gone. Filter-list updates
+    build the replacement engine on the blocking pool and swap it in
+    atomically, so requests keep matching against the old engine during a
+    multi-second list rebuild instead of stalling behind it.
+  - *One cosmetic lookup per page instead of two.* The URL-scoped cosmetic
+    lookup (`url_cosmetic_resources`) ran once for the `<head>` injection and
+    again at end-of-body; the end-of-body pass now reuses the first lookup and
+    only resolves the generic class/id-indexed selectors on top.
+  - Dashboard events are only constructed when a client is actually watching
+    the live requests feed; statistics counters are atomics instead of
+    mutexes; the HTML rewriter no longer compiles a (redundant) regex per
+    response and scans each element once instead of twice.
+  - New 5-minute read timeout on proxied requests bounds a peer that stops
+    sending mid-response without closing (previously such a request hung
+    forever); generous enough not to disturb long-polls or quiet SSE streams.
+- A configuration file that fails to parse no longer takes the server down.
+  `read_configuration` unwrapped the parse error, so a hand-edited file with (for
+  example) a duplicate TOML key panicked a worker on `SIGHUP` and killed both the
+  proxy and web-UI loops while the process kept running — both ports stopped
+  listening with no way back except a restart. The last configuration that parsed
+  is now kept and reused, so a reload over a broken file logs the error and
+  carries on serving with the previous settings, then picks up the corrected file
+  on the next reload. The CA reload in the same path no longer unwraps either.
 - The PAC route now also answers at `/wpad.dat`, so DNS-based WPAD
   auto-discovery (`http://wpad.<search domain>/wpad.dat`) can point straight
   at Privaxy without needing a rewrite in a fronting reverse proxy.

privaxy/Cargo.toml

@@ -70,7 +70,14 @@ tokio-util = { version = "0.7.4", features = ["full"] }
 tokio-stream = "0.1"
 bytes = "1"
 socket2 = "0.5"
-adblock = { version = "0.12.5" }
+# default-features = false only to drop `single-thread`: the multi-thread
+# engine is Send + Sync, which lets requests share one engine directly instead
+# of funneling through a dedicated blocker thread. The other two features are
+# adblock defaults, restated because default-features is all-or-nothing.
+adblock = { version = "0.12.5", default-features = false, features = [
+  "embedded-domain-resolver",
+  "full-regex-handling",
+] }
 openssl = { version = "0.10.43", features = ["vendored"] }
 include_dir = "0.7.3"
 chrono = { version = "0.4.23", features = ["serde"] }
@@ -100,10 +107,10 @@ hyper-rustls = { version = "0.27", default-features = false, features = [
 log = "0.4.17"
 env_logger = "0.11.10"
 uluru = "3.1.0"
+portable-atomic = "1"
 regex = "1.7.0"
 lazy_static = "1.4.0"
 lol_html = "1.2.1"
-crossbeam-channel = "0.5.6"
 thiserror = "2"
 url = "2.3.1"
 futures = "0.3.25"