diff --git a/CHANGELOG.md b/CHANGELOG.md index 91b6906..2549366 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,213 @@ ## Unreleased +- Userscript engine: Greasemonkey/Tampermonkey-style scripts injected into + matching pages, managed at runtime from the web UI. + - New **Settings → Userscripts** page: install a script by pasting it or by + URL (e.g. from Greasyfork), toggle scripts individually, edit a script's + source in place, and uninstall it. A master switch disables the whole + engine without clearing the per-script selection. Each entry shows its + version, `@run-at`, match patterns, grant count and `@noframes` state, and + a script whose stored body no longer parses is called out as not being + injected instead of failing silently. + - New `[userscripts]` configuration section holding the master `enabled` flag + and one `[[userscripts.scripts]]` entry per script. As with filter lists, + only metadata is stored in the configuration file; bodies live under + `userscripts/` in the configuration directory (override with + `PRIVAXY_USERSCRIPT_PATH`), keyed by a hash of the source URL so + re-installing the same script reuses its file. Configurations written + before this release parse unchanged and default to an enabled engine with + no scripts. + - Scripts are matched on the same canonical URL the adblock engine uses, + supporting `@match` (Chrome match-pattern syntax including `` and + `*.host` wildcards), `@include`/`@exclude` (globs, or regular expressions + written `/…/`), `@exclude-match`, `@run-at` + (`document-start`/`-body`/`-end`/`-idle`) and `@noframes`. Exclusions take + precedence over inclusions. A script declaring neither `@match` nor + `@include` — or with a malformed pattern or no metadata block — is rejected + with a `422` at install time rather than being stored and never firing. + - The in-page runtime provides `GM_info`, `unsafeWindow`, `GM_addStyle` + (nonce-stamped so it survives the page's CSP), `GM_log`, `GM_openInTab`, + `GM_setClipboard`, `GM_notification`, `GM_registerMenuCommand` and the + promise-based `GM.*` namespace. + - `@require` libraries and `@resource` payloads are fetched server-side and + cached on disk under `userscripts/assets/`, keyed by URL hash. Requires are + evaluated inside the script's own wrapper ahead of its body, so their + top-level declarations are visible to the script without leaking into the + page, and `GM_getResourceText`/`GM_getResourceURL` read the fetched + resources. Cached assets are never re-fetched (the convention is to pin a + versioned URL); delete the `assets` directory to refresh. An asset that + cannot be fetched degrades its script rather than dropping it, and the + failure is reported on the Userscripts page. + - `GM_setValue`/`GM_getValue`/`GM_deleteValue`/`GM_listValues` are persisted, + scoped per script, in `userscripts/gm_storage.json` (writes are coalesced + on a short debounce and written atomically; the configuration file is + deliberately not used, since these are written far too often to + re-serialize it each time). Because `GM_getValue` is synchronous in the GM + API, values are preloaded into each script's descriptor at injection time + and read from that snapshot in-page, so an ordinary `GM_getValue` costs no + request. Uninstalling a script drops its values. + - `GM_xmlhttpRequest` is relayed server-side, so it has no CORS restrictions + — the one capability a real content script cannot have. Three independent + controls gate it: the origin-bound token, the requesting script's own + `@connect` declarations (as Tampermonkey requires, so compatibility is + unaffected), and a filter rejecting loopback, RFC1918, carrier-grade NAT, + link-local (including the cloud metadata address) and IPv4-mapped + equivalents. Redirects are followed manually so `@connect` and address + filtering re-run on every hop rather than letting an allow-listed host + bounce the request to `127.0.0.1`. Methods are restricted, `Host` and + hop-by-hop request headers cannot be set, and responses are size-capped. + - `@resource` payloads may be binary. They are stored as bytes with the + content type they were served as; text small enough to matter is inlined + into the script's descriptor so `GM_getResourceText` stays synchronous, + while anything binary or oversized is reachable through + `GM_getResourceURL`, served from the reserved path with its original bytes, + content type and `X-Content-Type-Options: nosniff`. `GM_getResourceText` on + a binary resource returns `null` and logs a note pointing at + `GM_getResourceURL` rather than returning mojibake. + - `GM_addValueChangeListener`/`GM_removeValueChangeListener` are implemented. + Changes fire locally, reach other same-origin tabs over `BroadcastChannel` + (free, no request), and reach other origins and other devices behind the + same proxy by polling a read endpoint — which only runs while at least one + listener is registered, so a page with no listeners issues no extra + requests. The read endpoint requires the requesting URL to satisfy the + script's own `@match`/`@include`, so it can never reveal more than the + page's own descriptor already contained. + - `GM_registerMenuCommand` now has a real surface: a small floating menu, + injected only once a script actually registers a command, hosted in a closed + shadow root so neither the page's CSS nor Privaxy's can reach across. + Commands are still reachable from the console via + `__privaxyUserscriptMenu()`. + - Userscripts can be disabled for a single tab from that menu, backed by + `sessionStorage` — per-tab by construction, so the proxy needs no notion of + a tab. Other tabs are unaffected; `__privaxyUserscriptsEnableTab()` restores + them, since disabling removes the menu that turned them off. + - Userscripts installed from a URL are re-fetched on the same 24h timer as the + filter lists and recompiled in place, so upstream changes are picked up + without a restart. The refresh re-reads the configuration from disk rather + than reusing the updater's own copy: userscript changes deliberately bypass + that channel, so its copy is stale with respect to them and recompiling from + it would drop every script installed since startup. + - `@updateURL` and `@downloadURL` are honored, both defaulting to the URL the + script was installed from. When they differ, only the (small) `@updateURL` + document is fetched to compare `@version`, and the body is downloaded solely + when that version is newer — versions are ordered as dotted numbers, so + `1.2.10` correctly supersedes `1.2.9`, falling back to plain inequality for + schemes that cannot be ordered. Scripts that split metadata and body, a + common Greasyfork layout, previously re-downloaded the whole body on every + cycle. + - A **Check for updates** button on the Userscripts page refreshes on demand + instead of waiting out the timer, reporting per script whether it was + updated, already current, or failed and why. Unlike the periodic refresh it + holds the save lock, so a changed `@name` or `@version` is persisted. + - New `userscripts.allow_private_network_requests` setting (default off, with + a toggle on the Userscripts page) permits the relay to reach private + addresses. It is off by default because the relay runs server-side: the + proxy usually sits *inside* a LAN and can reach routers, admin panels and + metadata endpoints no page could contact. Changes apply immediately, with + no reload. + - Changes apply to the next page load with no reload or restart: compiled + scripts live in a shared store that each API mutation replaces in place, + and the store is also refreshed on `SIGHUP` so a hand-edited + `[userscripts]` section takes effect. Each script is emitted in its own + `nonce`d script element, so a syntax error in one script cannot abandon the + others or the ad-blocking payload, and the CSP nonce is kept in a closure + rather than published on `window`. + - Note that a userscript installed here runs on **every client behind the + proxy**, in the page's main world (a proxy has no isolated world to offer), + which is a wider blast radius than a browser extension installed in one + profile. The Userscripts page says so where scripts are added. + - **Not supported.** A userscript engine built into a proxy cannot reach full + Tampermonkey parity, and some of what is missing is structural rather than + unfinished. Known gaps, so a script that misbehaves can be diagnosed instead + of guessed at: + - *No isolated world, and there cannot be one.* Scripts run in the page's + main world, so `unsafeWindow === window`, page scripts can read and clobber + anything a userscript leaves reachable, and anti-adblock can detect the + injection. Each script is still wrapped in its own function, so its `var`, + `let`, `const`, `function` and `class` declarations do not leak to the page + — only an undeclared assignment or an explicit `window.x = …` does. + - *Pages the proxy never sees get nothing.* A site with an active service + worker serving navigations from cache, `file://`, `chrome://`, browser-cache + hits and any traffic not routed through Privaxy are all invisible to it, so + no script runs there. A browser extension sees all of them. + - *`@grant` is parsed and displayed but not enforced.* Every script receives + every implemented API regardless of what it declared, including + `@grant none`. This is deliberately forgiving — a script that forgot to + declare a grant still works — but it is a deviation. + - *Not implemented:* `GM_cookie` (the proxy has no cookie jar for the + browser's cookies; it only sees `Cookie` headers in flight), + `GM_getTab`/`GM_saveTab`/`GM_getTabs` (no tab identity exists on the proxy + side), `GM_download` (it would mean the proxy writing files to its own disk + on a page's behalf), `GM_addElement`, the batch `GM_setValues`/`GM_getValues` + forms, and `window.onurlchange`. + - *`GM_notification` writes to the console* rather than raising a real + notification, and `GM_setClipboard` needs a user gesture like any page-context + clipboard write. + - *Metadata ignored:* `@sandbox`, `@unwrap`, `@top-level-await`, `@icon`, + `@supportURL`, `@antifeature`. Unknown directives are skipped, not rejected. + - *`@resource` payloads are byte-exact but not inlined as text when binary or + over 256 KB* — `GM_getResourceText` returns `null` for those and the data is + reachable only through `GM_getResourceURL`. + - *`GM_xmlhttpRequest` is not a transparent `XMLHttpRequest`.* Responses are + decoded as lossy UTF-8, so binary bodies are unusable; `abort()` only + suppresses the callbacks, since the server-side request is already in + flight; there is no `onprogress`/`onreadystatechange`; redirects are capped + at 5 hops, timeouts at 60s (default 30s) and responses at 8 MB; and the + request only reaches hosts the script declared with `@connect`. + - *Value-change notification is not instant across origins.* Same-origin tabs + are updated immediately over `BroadcastChannel`; a change made on a + different origin or another device is picked up by a 15s poll, and only + while a listener is registered. + - *Per-tab disable is per-tab **per origin*** — it is `sessionStorage`, so + disabling on one site does not disable on another in the same tab. + - *Storage limits:* 1000 keys per script and 64 KB per value; a script body + or fetched `@require`/`@resource` is capped at 2 MB. Writes are flushed on a + 500 ms debounce, so values set immediately before a crash can be lost. + - *No script ordering or import/export in the UI.* Injection order is + configuration order, changeable only by editing the file. +- Proxy performance overhaul: + - *HTML responses now stream.* The proxy previously withheld an HTML + response — status line, headers and all — until the entire upstream + document had been downloaded and fed through the rewriter, so the browser + could not start parsing (or prefetching subresources) until the last + upstream byte arrived. The rewritten document now streams to the client as + it is produced, and the rewriter pipeline is bounded end-to-end, so a slow + client backpressures the upstream download instead of the whole document + buffering in memory. + - *WebSocket tunnels no longer squeeze through a 32-byte buffer.* The duplex + buffer bridging the client and upstream halves of an upgraded connection + was 32 bytes, forcing a task wakeup roughly every 32 bytes transferred; + it is now 64 KiB. + - *The adblock engine is shared, not funneled through one thread.* The + `single-thread` adblock feature is dropped; the engine (Send + Sync) is + now called directly from request tasks, removing a channel round-trip and + two cross-thread handoffs from every request. Matching itself still + serializes briefly on the engine's internal regex-manager lock — the same + one-core ceiling as the old blocker thread, far above proxy request rates + — but the per-request overhead around it is gone. Filter-list updates + build the replacement engine on the blocking pool and swap it in + atomically, so requests keep matching against the old engine during a + multi-second list rebuild instead of stalling behind it. + - *One cosmetic lookup per page instead of two.* The URL-scoped cosmetic + lookup (`url_cosmetic_resources`) ran once for the `` injection and + again at end-of-body; the end-of-body pass now reuses the first lookup and + only resolves the generic class/id-indexed selectors on top. + - Dashboard events are only constructed when a client is actually watching + the live requests feed; statistics counters are atomics instead of + mutexes; the HTML rewriter no longer compiles a (redundant) regex per + response and scans each element once instead of twice. + - New 5-minute read timeout on proxied requests bounds a peer that stops + sending mid-response without closing (previously such a request hung + forever); generous enough not to disturb long-polls or quiet SSE streams. +- A configuration file that fails to parse no longer takes the server down. + `read_configuration` unwrapped the parse error, so a hand-edited file with (for + example) a duplicate TOML key panicked a worker on `SIGHUP` and killed both the + proxy and web-UI loops while the process kept running — both ports stopped + listening with no way back except a restart. The last configuration that parsed + is now kept and reused, so a reload over a broken file logs the error and + carries on serving with the previous settings, then picks up the corrected file + on the next reload. The CA reload in the same path no longer unwraps either. - The PAC route now also answers at `/wpad.dat`, so DNS-based WPAD auto-discovery (`http://wpad./wpad.dat`) can point straight at Privaxy without needing a rewrite in a fronting reverse proxy. diff --git a/Cargo.lock b/Cargo.lock index 0c26f8e..580489f 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -418,15 +418,6 @@ dependencies = [ "cfg-if", ] -[[package]] -name = "crossbeam-channel" -version = "0.5.15" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "82b8f8f868b36967f9606790d1903570de9ceaf870a7bf9fbbd3016d636a2cb2" -dependencies = [ - "crossbeam-utils", -] - [[package]] name = "crossbeam-deque" version = "0.8.6" @@ -2179,7 +2170,6 @@ dependencies = [ "base64", "bytes", "chrono", - "crossbeam-channel", "dirs", "env_logger", "filterlists-api", @@ -2201,6 +2191,7 @@ dependencies = [ "mime_guess", "once_cell", "openssl", + "portable-atomic", "regex", "reqwest", "rustls", diff --git a/privaxy/Cargo.toml b/privaxy/Cargo.toml index dc2a7a5..94c35d5 100644 --- a/privaxy/Cargo.toml +++ b/privaxy/Cargo.toml @@ -70,7 +70,14 @@ tokio-util = { version = "0.7.4", features = ["full"] } tokio-stream = "0.1" bytes = "1" socket2 = "0.5" -adblock = { version = "0.12.5" } +# default-features = false only to drop `single-thread`: the multi-thread +# engine is Send + Sync, which lets requests share one engine directly instead +# of funneling through a dedicated blocker thread. The other two features are +# adblock defaults, restated because default-features is all-or-nothing. +adblock = { version = "0.12.5", default-features = false, features = [ + "embedded-domain-resolver", + "full-regex-handling", +] } openssl = { version = "0.10.43", features = ["vendored"] } include_dir = "0.7.3" chrono = { version = "0.4.23", features = ["serde"] } @@ -100,10 +107,10 @@ hyper-rustls = { version = "0.27", default-features = false, features = [ log = "0.4.17" env_logger = "0.11.10" uluru = "3.1.0" +portable-atomic = "1" regex = "1.7.0" lazy_static = "1.4.0" lol_html = "1.2.1" -crossbeam-channel = "0.5.6" thiserror = "2" url = "2.3.1" futures = "0.3.25" diff --git a/privaxy/build.rs b/privaxy/build.rs index 99bfefa..3043b92 100644 --- a/privaxy/build.rs +++ b/privaxy/build.rs @@ -1,12 +1,136 @@ -use std::path::PathBuf; +use std::path::{Path, PathBuf}; use std::process::Command; fn main() { let manifest_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR")); + let out_dir = PathBuf::from(std::env::var("OUT_DIR").expect("cargo provides OUT_DIR")); + + build_procedural_shim(&manifest_dir, &out_dir); + build_scriptlet_resources(&manifest_dir, &out_dir); +} + +/// Strip the comments out of the procedural-cosmetics shim before it is embedded. +/// +/// The shim is injected inline into every HTML response that has cosmetic rules, +/// and the proxy re-sends responses uncompressed, so its comments would be real +/// bytes on the wire on every page load — roughly two thirds of the file. Keeping +/// them in the source and dropping them here means the reasoning lives next to +/// the code without being paid for at runtime. +/// +/// Deliberately not done with Node: the cross-compile path in +/// `build_scriptlet_resources` exists precisely because Node isn't always +/// available, and the shim has to be embedded in every build. +fn build_procedural_shim(manifest_dir: &Path, out_dir: &Path) { + let source_path = manifest_dir.join("src/resources/procedural_cosmetics.js"); + println!("cargo:rerun-if-changed={}", source_path.display()); + + let source = std::fs::read_to_string(&source_path).unwrap_or_else(|e| { + panic!( + "failed to read the procedural shim at {}: {e}", + source_path.display() + ) + }); + + let embedded = match strip_whole_line_comments(&source) { + Some(stripped) => stripped, + None => { + // Embedding the file verbatim is always correct, only larger, so an + // unrecognised construct degrades instead of risking a corrupted shim + // on every page. + println!( + "cargo:warning=procedural_cosmetics.js contains a construct the comment \ + stripper does not handle (a multi-line string, or a comment sharing a line \ + with code); embedding it verbatim. See strip_whole_line_comments in \ + privaxy/build.rs." + ); + source + } + }; + + std::fs::write(out_dir.join("procedural_cosmetics.js"), embedded) + .expect("failed to write the stripped procedural shim to OUT_DIR"); +} + +/// Remove comment-only lines, blank lines, and indentation from JavaScript. +/// +/// This is a line-oriented pass, not a JavaScript lexer: it decides what to drop +/// purely from how a line *starts*, and never inspects the interior of a line it +/// keeps. That is what makes it safe without parsing — a `//` inside a string or +/// a regex literal cannot be mistaken for a comment, because such a line is kept +/// whole. +/// +/// It returns `None` rather than guessing whenever it meets something that would +/// invalidate that reasoning: +/// +/// * a backtick, which may open a template literal spanning lines: a line +/// inside one could begin with `//` and would then be dropped wrongly; +/// * a trailing backslash, i.e. a string continued onto the next line, for the +/// same reason; +/// * a comment sharing a line with code, where telling comment from string +/// genuinely does need a lexer. +/// +/// Line numbers shift, which costs nothing here: the shim is concatenated after +/// the scriptlet payload before injection, so they never matched the file anyway. +fn strip_whole_line_comments(source: &str) -> Option { + let mut out = String::with_capacity(source.len()); + let mut in_block_comment = false; + + for line in source.lines() { + let trimmed = line.trim(); + + if in_block_comment { + match trimmed.find("*/") { + // Code trailing the end of a block comment needs a lexer to split. + Some(end) if end + 2 != trimmed.len() => return None, + Some(_) => in_block_comment = false, + None => {} + } + continue; + } + + if trimmed.starts_with("//") { + continue; + } + + if trimmed.starts_with("/*") { + match trimmed.find("*/") { + Some(end) if end + 2 != trimmed.len() => return None, + // `/*/` both starts and ends with a delimiter without being a + // complete comment, hence the length check. + Some(_) if trimmed.len() >= 4 => {} + _ => in_block_comment = true, + } + continue; + } + + if trimmed.is_empty() { + continue; + } + + if trimmed.contains('`') + || trimmed.ends_with('\\') + || trimmed.contains("//") + || trimmed.contains("/*") + { + return None; + } + + out.push_str(trimmed); + out.push('\n'); + } + + // An unterminated block comment means the file was not what we assumed. + if in_block_comment { + return None; + } + + Some(out) +} + +fn build_scriptlet_resources(manifest_dir: &Path, out_dir: &Path) { let scriptlets_src = manifest_dir.join("src/resources/vendor/ublock/scriptlets.js"); let builder = manifest_dir.join("build-scriptlets.mjs"); - let out_dir = std::env::var("OUT_DIR").expect("cargo provides OUT_DIR"); - let out_path = PathBuf::from(&out_dir).join("scriptlets-resources.json"); + let out_path = out_dir.join("scriptlets-resources.json"); println!("cargo:rerun-if-changed={}", scriptlets_src.display()); println!("cargo:rerun-if-changed={}", builder.display()); diff --git a/privaxy/src/resources/procedural_cosmetics.js b/privaxy/src/resources/procedural_cosmetics.js index 5be040f..4139546 100644 --- a/privaxy/src/resources/procedural_cosmetics.js +++ b/privaxy/src/resources/procedural_cosmetics.js @@ -7,6 +7,20 @@ * This shim receives those rules as JSON and applies them on load and on every * subsequent DOM mutation. * + * It also adopts the plain-CSS cosmetic rules into every shadow root it finds. + * That server-side "# ); diff --git a/privaxy/src/server/proxy/mitm.rs b/privaxy/src/server/proxy/mitm.rs index 8e532ed..0b1b2b8 100644 --- a/privaxy/src/server/proxy/mitm.rs +++ b/privaxy/src/server/proxy/mitm.rs @@ -1,5 +1,6 @@ use super::serve::UpgradeClient; use super::tls_failures::TlsFailureStore; +use super::userscripts::UserScriptContext; use super::{empty_body, exclusions::LocalExclusionStore, serve::serve, ProxyBody}; use crate::{ blocker::AdblockRequester, cert::CertCache, configuration::DohConfig, statistics::Statistics, @@ -35,6 +36,7 @@ pub(crate) async fn serve_mitm_session( scriptlet_debug_logging: bool, tls_failure_store: TlsFailureStore, gui_base_url: Option, + user_scripts: UserScriptContext, ) -> Result, hyper::Error> { let raw_authority = match req.uri().authority().cloned() { Some(authority) => authority, @@ -125,6 +127,7 @@ pub(crate) async fn serve_mitm_session( doh_config.clone(), scriptlet_debug_logging, gui_base_url.clone(), + user_scripts.clone(), ) }), ) @@ -208,6 +211,7 @@ pub(crate) async fn serve_mitm_session( doh_config, scriptlet_debug_logging, gui_base_url, + user_scripts, ) .await } diff --git a/privaxy/src/server/proxy/mod.rs b/privaxy/src/server/proxy/mod.rs index 09a6022..0c162b0 100644 --- a/privaxy/src/server/proxy/mod.rs +++ b/privaxy/src/server/proxy/mod.rs @@ -3,8 +3,10 @@ pub(crate) mod mitm; pub(crate) mod serve; pub(crate) use mitm::serve_mitm_session; pub(crate) mod exclusions; +pub(crate) mod gm; pub(crate) mod html_rewriter; pub(crate) mod tls_failures; +pub(crate) mod userscripts; use bytes::Bytes; use http_body_util::{combinators::BoxBody, BodyExt, Empty, Full, StreamBody}; diff --git a/privaxy/src/server/proxy/serve.rs b/privaxy/src/server/proxy/serve.rs index 42536c4..ded7508 100644 --- a/privaxy/src/server/proxy/serve.rs +++ b/privaxy/src/server/proxy/serve.rs @@ -1,5 +1,7 @@ use super::doh::{self, DohAction}; +use super::gm::endpoint as gm_endpoint; use super::html_rewriter::Rewriter; +use super::userscripts::UserScriptContext; use super::{body_channel, boxed_incoming, empty_body, full_body, BodySender, ProxyBody}; use crate::blocker::AdblockRequester; use crate::configuration::DohConfig; @@ -11,7 +13,7 @@ use bytes::Bytes; use futures::TryStreamExt; use http::uri::{Authority, Scheme}; use http::{HeaderMap, HeaderValue, Request, Response, StatusCode, Uri}; -use http_body_util::BodyStream; +use http_body_util::{BodyExt, BodyStream, Limited}; use hyper::body::{Frame, Incoming}; use hyper_rustls::HttpsConnector; use hyper_util::client::legacy::connect::HttpConnector; @@ -19,6 +21,7 @@ use hyper_util::client::legacy::Client as HyperClient; use hyper_util::rt::TokioIo; use std::net::IpAddr; use tokio::sync::broadcast; +use url::Url; /// Type of the hyper client used for upgrade tunneling (websockets, etc.). pub(crate) type UpgradeClient = HyperClient, ProxyBody>; @@ -45,6 +48,10 @@ const CSP_HEADERS: [&str; 3] = [ "x-webkit-csp", ]; +/// Cap on a buffered request body for the reserved endpoints. Generous for a +/// batch of `GM_setValue` writes and far below anything worth buffering. +const MAX_RESERVED_ENDPOINT_BODY_BYTES: usize = 256 * 1024; + /// 16 random bytes → 22 chars of url-safe base64. Plenty of entropy and /// avoids `=` padding that some CSP parsers historically choked on. fn generate_csp_nonce() -> String { @@ -277,6 +284,7 @@ pub(crate) async fn serve( doh_config: DohConfig, scriptlet_debug_logging: bool, gui_base_url: Option, + user_scripts: UserScriptContext, ) -> Result, hyper::Error> { let scheme_string = scheme.to_string(); @@ -299,6 +307,25 @@ pub(crate) async fn serve( return Ok(perform_two_ends_upgrade(request, uri, hyper_client).await); } + // Requests to the reserved path are answered by the proxy on the page's own + // origin and never forwarded upstream, which is what lets a userscript + // running in the page's main world reach Privaxy without CORS. Handled + // before any statistics counting: these are not proxied traffic. + if gm_endpoint::is_reserved(uri.path()) { + let (parts, body) = request.into_parts(); + // Bounded so a hostile page cannot make the proxy buffer an unbounded + // body; an oversized request simply fails to parse below. + let collected = Limited::new(body, MAX_RESERVED_ENDPOINT_BODY_BYTES) + .collect() + .await + .map(|collected| collected.to_bytes()) + .unwrap_or_default(); + + return Ok( + gm_endpoint::handle(&uri, &parts.method, &collected, &user_scripts, &client).await, + ); + } + let (mut parts, body) = request.into_parts(); parts.uri = uri.clone(); @@ -310,7 +337,7 @@ pub(crate) async fn serve( statistics.increment_top_clients(client_ip_address); - let request_type = request_type_from_headers(req.headers()).to_string(); + let request_type = request_type_from_headers(req.headers()); // Canonical URL (default port stripped) for all adblock-engine matching — // see `url_for_matching`. The raw `uri` (which may carry `:443`) is still @@ -322,36 +349,40 @@ pub(crate) async fn serve( let doh_action = doh::classify(&doh_config, req.headers(), &uri); if let DohAction::Block = &doh_action { log::debug!("Refusing DoH request: {}", uri); + // Events only feed the live requests view; when nobody is subscribed, + // building one (two Strings and a timestamp) is wasted work. + if broadcast_sender.receiver_count() > 0 { + let _result = broadcast_sender.send(Event { + now: chrono::Utc::now(), + method: req.method().to_string(), + url: req.uri().to_string(), + is_request_blocked: true, + }); + } + statistics.increment_blocked_requests(); + return Ok(get_empty_response(StatusCode::BAD_GATEWAY)); + } + + let (is_request_blocked, blocker_result) = adblock_requester.is_network_url_blocked( + &match_url, + match req.headers().get(http::header::REFERER) { + Some(referer) => referer.to_str().unwrap_or(&match_url), + // When no referer, we default to `uri` as we otherwise may get many false + // positives due to the blocker thinking it's third party requests. + None => &match_url, + }, + request_type, + ); + + if broadcast_sender.receiver_count() > 0 { let _result = broadcast_sender.send(Event { now: chrono::Utc::now(), method: req.method().to_string(), url: req.uri().to_string(), - is_request_blocked: true, + is_request_blocked, }); - statistics.increment_blocked_requests(); - return Ok(get_empty_response(StatusCode::BAD_GATEWAY)); } - let (is_request_blocked, blocker_result) = adblock_requester - .is_network_url_blocked( - match_url.clone(), - match req.headers().get(http::header::REFERER) { - Some(referer) => referer.to_str().unwrap().to_string(), - // When no referer, we default to `uri` as we otherwise may get many false - // positives due to the blocker thinking it's third party requests. - None => match_url.clone(), - }, - request_type.clone(), - ) - .await; - - let _result = broadcast_sender.send(Event { - now: chrono::Utc::now(), - method: req.method().to_string(), - url: req.uri().to_string(), - is_request_blocked, - }); - if is_request_blocked { statistics.increment_blocked_requests(); statistics.increment_top_blocked_paths(format!( @@ -477,36 +508,70 @@ pub(crate) async fn serve( let new_response = Response::from_parts(parts, new_new_body); if is_html { - let (sender_rewriter, receiver_rewriter) = crossbeam_channel::unbounded::(); + // Bounded so the whole backpressure chain holds: client ← response body + // ← rewriter ← this channel ← upstream. Before, every hop here was + // unbounded and the entire document buffered in memory. + let (sender_rewriter, receiver_rewriter) = tokio::sync::mpsc::channel::(32); // Resolve the URL-scoped payloads up-front so the rewriter can prepend // them inside before any page scripts execute: the uBO scriptlet // and the procedural cosmetic filters (both URL-specific, not dependent - // on collected IDs/classes). The end-of-body cosmetic lookup still runs - // for hide/style selectors, which do depend on collected IDs/classes. + // on collected IDs/classes). This is the single `url_cosmetic_resources` + // lookup for the page — the end-of-body pass only resolves the generic + // class/id-indexed selectors on top of it, using the exception set + // carried in this result. let head_cosmetics = adblock_requester - .get_cosmetic_response(match_url.clone(), Vec::new(), Vec::new()) + .get_cosmetic_response(match_url.clone()) .await; + // Userscripts are matched against the same canonical URL the adblock + // engine uses. The store is consulted per request rather than captured + // once per proxy start, so scripts added or toggled in the web UI apply + // to the very next page load without a reload. + let matched_user_scripts = if user_scripts.store.is_empty() { + Vec::new() + } else { + match Url::parse(&match_url) { + Ok(url) => user_scripts.store.matching(&url), + Err(err) => { + log::debug!("Not matching userscripts against {match_url}: {err}"); + Vec::new() + } + } + }; + + // Minted per page and handed to the in-page runtime so it can persist + // GM values. Bound to this origin; see `userscript_token`. + let endpoint_token = gm_endpoint::origin_of(&uri) + .map(|origin| super::gm::token::mint(&origin, &user_scripts.endpoint_signing_key)); + let rewriter = Rewriter::new( - match_url.clone(), adblock_requester, receiver_rewriter, sender, statistics, csp_nonce.expect("csp_nonce is Some whenever is_html"), - head_cosmetics.injected_script, - head_cosmetics.procedural_filters, + head_cosmetics, scriptlet_debug_logging, + matched_user_scripts, + user_scripts.gm_storage.clone(), + endpoint_token, ); tokio::task::spawn_blocking(|| rewriter.rewrite()); - while let Ok(Some(chunk)) = response.chunk().await { - if let Err(_err) = sender_rewriter.send(chunk) { - break; + // Drain the upstream body on its own task so the response (headers plus + // the streaming rewritten body) is returned to the client immediately. + // Holding the response until the whole document had been downloaded + // meant the browser could not start parsing — or prefetching + // subresources — until the very last upstream byte had arrived. + tokio::spawn(async move { + while let Ok(Some(chunk)) = response.chunk().await { + if sender_rewriter.send(chunk).await.is_err() { + break; + } } - } + }); return Ok(new_response); } @@ -645,7 +710,11 @@ async fn perform_two_ends_upgrade( uri: Uri, hyper_client: UpgradeClient, ) -> Response { - let (mut duplex_client, mut duplex_server) = tokio::io::duplex(32); + // The duplex buffer caps how many bytes can be in flight between the two + // `copy_bidirectional` tasks bridging client and upstream. A tiny buffer + // forces the tunnel to ping-pong wakeups every few bytes, throttling + // WebSocket throughput; 64 KiB matches a typical socket buffer. + let (mut duplex_client, mut duplex_server) = tokio::io::duplex(64 * 1024); // Captured for log context; `uri` is moved into `new_request` below. let request_uri = uri.to_string(); diff --git a/privaxy/src/server/proxy/userscripts.rs b/privaxy/src/server/proxy/userscripts.rs new file mode 100644 index 0000000..68f96d5 --- /dev/null +++ b/privaxy/src/server/proxy/userscripts.rs @@ -0,0 +1,247 @@ +//! Runtime store of compiled userscripts. +//! +//! Userscript changes made in the web UI must take effect on the next request, +//! not on the next reload. The proxy loop reads most configuration once per +//! (re)start — `scriptlet_debug_logging` is read that way in `lib.rs` — so +//! userscripts instead live behind this cheaply-cloneable handle, following the +//! same pattern as [`crate::proxy::exclusions::LocalExclusionStore`] and +//! [`crate::proxy::tls_failures::TlsFailureStore`]. Each API mutation replaces +//! the contents in place and every subsequent page load sees the new set. + +use super::gm::storage::GmStorageStore; +use crate::configuration::{CompiledUserScript, Configuration}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, RwLock}; +use url::Url; + +/// Everything the request path needs in order to serve userscripts. +/// +/// Bundled rather than passed as three more arguments through +/// `serve_mitm_session` -> `serve` -> `Rewriter`, which already carry more +/// parameters than is comfortable. +#[derive(Debug, Clone)] +pub struct UserScriptContext { + /// Compiled scripts, replaced in place whenever the configuration changes. + pub store: UserScriptStore, + /// Persistent `GM_setValue` data. + pub gm_storage: GmStorageStore, + /// Key the reserved-endpoint tokens are derived from. Read once per proxy + /// (re)start, so both minting and verification always use the same value. + pub endpoint_signing_key: String, + /// Mirrors `userscripts.allow_private_network_requests`, consulted by the + /// `GM_xmlhttpRequest` relay. + /// + /// Shared and atomic rather than a captured `bool`: the API updates it in + /// place so toggling it in the web UI takes effect immediately, like every + /// other userscript setting, instead of waiting for a reload. + pub allow_private_network_requests: PrivateNetworkAccess, +} + +/// Live switch for the relay's private-address filter. +#[derive(Debug, Clone, Default)] +pub struct PrivateNetworkAccess(Arc); + +impl PrivateNetworkAccess { + pub fn new(allowed: bool) -> Self { + Self(Arc::new(AtomicBool::new(allowed))) + } + + pub fn is_allowed(&self) -> bool { + self.0.load(Ordering::Relaxed) + } + + pub fn set(&self, allowed: bool) { + self.0.store(allowed, Ordering::Relaxed); + } +} + +/// Shared, cheaply-cloneable set of compiled userscripts. +/// +/// Scripts are held behind an `Arc` so matching a URL only clones pointers, +/// never script bodies, and the read lock is released before any of them is +/// used. Lock scopes are short and never held across an await point. +#[derive(Debug, Clone, Default)] +pub struct UserScriptStore(Arc>>>); + +impl UserScriptStore { + pub fn new(scripts: Vec) -> Self { + Self(Arc::new(RwLock::new( + scripts.into_iter().map(Arc::new).collect(), + ))) + } + + /// Swap in a freshly compiled set, discarding the previous one. + pub fn replace(&self, scripts: Vec) { + let mut guard = self.0.write().unwrap(); + *guard = scripts.into_iter().map(Arc::new).collect(); + } + + /// Scripts whose `@match`/`@include` declarations select `url`, in + /// configuration order. + pub fn matching(&self, url: &Url) -> Vec> { + let guard = self.0.read().unwrap(); + + guard + .iter() + .filter(|script| script.matches(url)) + .cloned() + .collect() + } + + /// Whether any script is loaded. Lets the request path skip URL parsing + /// entirely in the common case of no userscripts installed. + pub fn is_empty(&self) -> bool { + self.0.read().unwrap().is_empty() + } + + /// The compiled form of one script, by file name. Used by the API to report + /// compile warnings without redoing the work. + pub fn find(&self, file_name: &str) -> Option> { + self.0 + .read() + .unwrap() + .iter() + .find(|script| script.file_name == file_name) + .cloned() + } +} + +/// Compile every active script in `configuration`, reading bodies from disk and +/// resolving each script's `@require`/`@resource` assets (cached on disk after +/// the first fetch). +/// +/// A script that fails to load or no longer parses is logged and dropped rather +/// than aborting the whole rebuild — the same policy +/// [`crate::configuration::get_filters_content`] applies to filter lists. An +/// asset that fails to load only degrades its own script; see +/// [`CompiledUserScript::resolve_assets`]. +pub async fn compile_active_userscripts( + configuration: &Configuration, + http_client: &reqwest::Client, +) -> Vec { + let mut compiled = Vec::new(); + + for script in configuration.userscripts.active_scripts() { + let body = match script.read_body().await { + Ok(body) => body, + Err(err) => { + log::warn!( + "Dropping userscript '{}' whose body could not be read: {err}", + script.title + ); + continue; + } + }; + + match CompiledUserScript::new(script, body) { + Ok(mut compiled_script) => { + compiled_script.resolve_assets(http_client).await; + + for warning in &compiled_script.warnings { + log::warn!("Userscript '{}': {warning}", compiled_script.title); + } + + compiled.push(compiled_script); + } + Err(err) => log::warn!( + "Dropping userscript '{}' that no longer parses: {err}", + script.title + ), + } + } + + log::debug!("Compiled {} userscript(s)", compiled.len()); + + compiled +} + +/// Rebuild the store from `configuration`. Called at startup and after every +/// mutation from the API so changes apply without a proxy restart. +pub async fn reload_userscripts( + store: &UserScriptStore, + configuration: &Configuration, + http_client: &reqwest::Client, +) { + store.replace(compile_active_userscripts(configuration, http_client).await); +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::configuration::UserScript; + + fn compiled(name: &str, match_pattern: &str) -> CompiledUserScript { + let body = format!( + "// ==UserScript==\n// @name {name}\n// @match {match_pattern}\n// ==/UserScript==\nvoid 0;\n" + ); + let script = UserScript { + enabled: true, + title: name.to_string(), + file_name: format!("{name}.user.js"), + url: None, + }; + + CompiledUserScript::new(&script, body).expect("compiles") + } + + #[test] + fn matching_selects_only_scripts_whose_patterns_match() { + let store = UserScriptStore::new(vec![ + compiled("example-only", "https://example.com/*"), + compiled("everywhere", ""), + ]); + + let matched = store.matching(&Url::parse("https://example.com/page").unwrap()); + assert_eq!(matched.len(), 2); + + let matched = store.matching(&Url::parse("https://other.test/page").unwrap()); + assert_eq!(matched.len(), 1); + assert_eq!(matched[0].title, "everywhere"); + } + + /// A `@require` that cannot be fetched must degrade the script, not drop + /// it: the body still runs and the failure is reported as a warning. + #[tokio::test] + async fn unreachable_require_becomes_a_warning() { + let body = "// ==UserScript==\n// @name Needs a library\n// @match \n\ + // @require http://127.0.0.1:1/never-listening.js\n// ==/UserScript==\nvoid 0;\n"; + let script = UserScript { + enabled: true, + title: "Needs a library".to_string(), + file_name: "needs-a-library.user.js".to_string(), + url: None, + }; + + let mut compiled = CompiledUserScript::new(&script, body.to_string()).expect("compiles"); + compiled.resolve_assets(&reqwest::Client::new()).await; + + assert!(compiled.requires.is_empty()); + assert_eq!(compiled.warnings.len(), 1); + assert!( + compiled.warnings[0].contains("@require"), + "warning should name the failing directive: {}", + compiled.warnings[0] + ); + // Still injectable. + assert!(compiled.matches(&Url::parse("https://anything.test/").unwrap())); + } + + /// The store is what makes web-UI changes take effect without a reload, so + /// a replacement must be visible through an already-cloned handle. + #[test] + fn replace_is_visible_through_existing_clones() { + let store = UserScriptStore::new(Vec::new()); + let handle = store.clone(); + assert!(handle.is_empty()); + + store.replace(vec![compiled("added", "")]); + + assert!(!handle.is_empty()); + assert_eq!( + handle + .matching(&Url::parse("https://anything.test/").unwrap()) + .len(), + 1 + ); + } +} diff --git a/privaxy/src/server/statistics.rs b/privaxy/src/server/statistics.rs index d065425..133984b 100644 --- a/privaxy/src/server/statistics.rs +++ b/privaxy/src/server/statistics.rs @@ -1,3 +1,4 @@ +use portable_atomic::{AtomicU64, Ordering}; use serde::Serialize; use std::{ collections::HashMap, @@ -21,9 +22,9 @@ pub struct SerializableStatistics { #[derive(Debug, Clone)] pub struct Statistics { - pub proxied_requests: Arc>, - pub blocked_requests: Arc>, - pub modified_responses: Arc>, + pub proxied_requests: Arc, + pub blocked_requests: Arc, + pub modified_responses: Arc, pub top_blocked_paths: Arc>>, pub top_clients: Arc>>, } @@ -37,9 +38,9 @@ impl Default for Statistics { impl Statistics { pub fn new() -> Self { Self { - proxied_requests: Arc::new(Mutex::new(0)), - blocked_requests: Arc::new(Mutex::new(0)), - modified_responses: Arc::new(Mutex::new(0)), + proxied_requests: Arc::new(AtomicU64::new(0)), + blocked_requests: Arc::new(AtomicU64::new(0)), + modified_responses: Arc::new(AtomicU64::new(0)), top_blocked_paths: Arc::new(Mutex::new(LRUCache::default())), top_clients: Arc::new(Mutex::new(HashMap::new())), } @@ -63,31 +64,22 @@ impl Statistics { } pub fn increment_proxied_requests(&self) -> u64 { - let mut proxied_requests = self.proxied_requests.lock().unwrap(); - - *proxied_requests += 1; - *proxied_requests + self.proxied_requests.fetch_add(1, Ordering::Relaxed) + 1 } pub fn increment_blocked_requests(&self) -> u64 { - let mut blocked_requests = self.blocked_requests.lock().unwrap(); - - *blocked_requests += 1; - *blocked_requests + self.blocked_requests.fetch_add(1, Ordering::Relaxed) + 1 } pub fn increment_modified_responses(&self) -> u64 { - let mut modified_responses = self.modified_responses.lock().unwrap(); - - *modified_responses += 1; - *modified_responses + self.modified_responses.fetch_add(1, Ordering::Relaxed) + 1 } pub fn get_serialized(&self) -> SerializableStatistics { SerializableStatistics { - proxied_requests: *self.proxied_requests.lock().unwrap(), - blocked_requests: *self.blocked_requests.lock().unwrap(), - modified_responses: *self.modified_responses.lock().unwrap(), + proxied_requests: self.proxied_requests.load(Ordering::Relaxed), + blocked_requests: self.blocked_requests.load(Ordering::Relaxed), + modified_responses: self.modified_responses.load(Ordering::Relaxed), top_blocked_paths: { let top_blocked_paths = self.top_blocked_paths.lock().unwrap(); let mut top_blocked_paths_iterator = top_blocked_paths.iter(); diff --git a/privaxy/src/server/web_gui/mod.rs b/privaxy/src/server/web_gui/mod.rs index 65586d3..0c86525 100644 --- a/privaxy/src/server/web_gui/mod.rs +++ b/privaxy/src/server/web_gui/mod.rs @@ -1,7 +1,9 @@ use crate::configuration::FilterFailureStore; use crate::logging::LogHandle; use crate::proxy::exclusions::LocalExclusionStore; +use crate::proxy::gm::storage::GmStorageStore; use crate::proxy::tls_failures::TlsFailureStore; +use crate::proxy::userscripts::{PrivateNetworkAccess, UserScriptStore}; use crate::statistics::Statistics; use crate::WEBAPP_FRONTEND_DIR; use crate::{blocker::BlockingDisabledStore, configuration::Configuration}; @@ -26,6 +28,7 @@ mod pac; pub(crate) mod settings; pub(crate) mod statistics; pub(crate) mod tls_failures; +pub(crate) mod userscripts; #[derive(Debug, Serialize)] pub(crate) struct ApiError { @@ -41,6 +44,9 @@ pub(crate) fn get_frontend( local_exclusions_store: &LocalExclusionStore, tls_failure_store: &TlsFailureStore, filter_failure_store: &FilterFailureStore, + user_script_store: &UserScriptStore, + gm_storage: &GmStorageStore, + private_network_access: &PrivateNetworkAccess, notify_reload: Arc, log_handle: LogHandle, ) -> BoxedFilter<(impl warp::Reply,)> { @@ -66,6 +72,9 @@ pub(crate) fn get_frontend( local_exclusions_store, tls_failure_store, filter_failure_store, + user_script_store, + gm_storage, + private_network_access, http_client, notify_reload, log_handle, @@ -119,6 +128,9 @@ fn create_api_routes( local_exclusions_store: &LocalExclusionStore, tls_failure_store: &TlsFailureStore, filter_failure_store: &FilterFailureStore, + user_script_store: &UserScriptStore, + gm_storage: &GmStorageStore, + private_network_access: &PrivateNetworkAccess, http_client: reqwest::Client, notify_reload: Arc, log_handle: LogHandle, @@ -217,6 +229,17 @@ fn create_api_routes( .and(require_auth.clone()) .and(filterlists::create_routes()); + let userscripts_route = + warp::path("userscripts") + .and(require_auth.clone()) + .and(userscripts::create_routes( + configuration_save_lock.clone(), + http_client.clone(), + user_script_store.clone(), + gm_storage.clone(), + private_network_access.clone(), + )); + // Note: `.recover` is attached to the inner combinator, NOT to the // outer `api_path.and(...)`. If we attached it to the outer filter, // recover would also fire when `api_path` itself didn't match (e.g. @@ -234,6 +257,7 @@ fn create_api_routes( .or(settings_route) .or(options_route) .or(filterlists_route) + .or(userscripts_route) .recover(handle_rejection); api_path.and(api_inner).with(def_headers).boxed() diff --git a/privaxy/src/server/web_gui/userscripts.rs b/privaxy/src/server/web_gui/userscripts.rs new file mode 100644 index 0000000..6e3fad2 --- /dev/null +++ b/privaxy/src/server/web_gui/userscripts.rs @@ -0,0 +1,594 @@ +//! `/api/userscripts` — install, edit, toggle and remove userscripts at +//! runtime. +//! +//! Every mutation persists the configuration and then replaces the contents of +//! the [`UserScriptStore`], so a change made in the web UI applies to the next +//! page load without a proxy reload. +//! +//! Unlike the filter routes, these deliberately do *not* push through +//! `configuration_updater_sender`: that channel makes the updater re-read every +//! filter list from disk and rebuild the adblock engine, which a userscript +//! change has no reason to trigger. The updater's own configuration copy only +//! ever touches filters, so leaving it untouched is safe. + +use super::{get_error_response, get_unprocessable_response}; +use crate::configuration::{ + calc_local_userscript_filename, calc_userscript_filename, fetch_userscript, Configuration, + ConfigurationError, RefreshOutcome, UserScript, UserScriptMetadata, UserScriptRefresh, + UserScriptUpdate, +}; +use crate::proxy::gm::storage::GmStorageStore; +use crate::proxy::userscripts::{reload_userscripts, PrivateNetworkAccess, UserScriptStore}; +use serde::{Deserialize, Serialize}; +use std::{convert::Infallible, sync::Arc}; +use url::Url; +use warp::filters::BoxedFilter; +use warp::http::Response; +use warp::Filter; + +/// The userscript section as served to the web UI: the master switch plus every +/// installed script. +#[derive(Debug, Serialize)] +struct UserScriptsResponse { + enabled: bool, + allow_private_network_requests: bool, + scripts: Vec, +} + +/// An installed script, annotated with what its metadata block declares so the +/// UI can show where it runs without parsing JavaScript itself. +#[derive(Debug, Serialize)] +struct UserScriptResponse { + enabled: bool, + title: String, + file_name: String, + url: Option, + version: Option, + description: Option, + run_at: Option, + matches: Vec, + grants: Vec, + no_frames: bool, + /// Set when the stored body could not be read or no longer parses. Such a + /// script is skipped at injection time, so the UI must be able to say so. + error: Option, + /// Non-fatal compile problems, e.g. a `@require` library that could not be + /// fetched. The script still runs, degraded. + warnings: Vec, +} + +#[derive(Debug, Deserialize)] +pub struct UserScriptStatusChangeRequest { + file_name: String, + enabled: bool, +} + +/// `PUT /api/userscripts/enabled` — the engine-level switches. Both fields are +/// optional so a client may change one without knowing the other's value. +#[derive(Debug, Deserialize)] +pub struct EngineSettingsRequest { + #[serde(default)] + enabled: Option, + #[serde(default)] + allow_private_network_requests: Option, +} + +/// Install a script, either from pasted source or by fetching a URL. +#[derive(Debug, Deserialize)] +pub struct AddUserScriptRequest { + /// Script source, pasted directly into the UI. + #[serde(default)] + body: Option, + /// URL to fetch the script from, for installing from e.g. Greasyfork. + #[serde(default)] + url: Option, + #[serde(default = "enabled_by_default")] + enabled: bool, +} + +fn enabled_by_default() -> bool { + true +} + +#[derive(Debug, Deserialize)] +pub struct UpdateUserScriptRequest { + file_name: String, + body: String, +} + +#[derive(Debug, Deserialize)] +pub struct DeleteUserScriptRequest { + file_name: String, +} + +/// `POST /api/userscripts/update` — refresh from upstream on demand rather than +/// waiting out the 24h timer. +#[derive(Debug, Deserialize)] +pub struct RefreshRequest { + /// One script, or every URL-installed script when absent. + #[serde(default)] + file_name: Option, +} + +/// Turn a stored script plus its on-disk body into the UI representation, +/// reporting rather than hiding a body that cannot be read or parsed. +/// +/// Compile warnings (an unreachable `@require`, say) are read from the live +/// store rather than recomputed, so the UI reports exactly what the injection +/// path is actually working with. A disabled script is absent from the store and +/// therefore has no warnings — it isn't being injected at all. +async fn describe_script( + script: &UserScript, + user_script_store: &UserScriptStore, +) -> UserScriptResponse { + let mut response = UserScriptResponse { + enabled: script.enabled, + title: script.title.clone(), + file_name: script.file_name.clone(), + url: script.url.as_ref().map(|url| url.to_string()), + version: None, + description: None, + run_at: None, + matches: Vec::new(), + grants: Vec::new(), + no_frames: false, + error: None, + warnings: user_script_store + .find(&script.file_name) + .map(|compiled| compiled.warnings.clone()) + .unwrap_or_default(), + }; + + let body = match script.read_body().await { + Ok(body) => body, + Err(err) => { + response.error = Some(format!("Unable to read the script body: {err}")); + return response; + } + }; + + match UserScriptMetadata::parse(&body) { + Ok(metadata) => { + response.version = metadata.version.clone(); + response.description = metadata.description.clone(); + response.run_at = Some(metadata.run_at.as_token().to_string()); + response.no_frames = metadata.no_frames; + response.grants = metadata.grants.clone(); + response.matches = metadata + .matches + .iter() + .map(|pattern| pattern.as_str().to_string()) + .chain( + metadata + .includes + .iter() + .map(|pattern| pattern.as_str().to_string()), + ) + .collect(); + } + Err(err) => response.error = Some(err.to_string()), + } + + response +} + +async fn get_userscripts( + user_script_store: UserScriptStore, +) -> Result { + let configuration = match Configuration::read_from_home().await { + Ok(configuration) => configuration, + Err(err) => { + log::error!("Failed to read configuration: {err}"); + return Ok(get_error_response(err)); + } + }; + + let mut scripts = Vec::with_capacity(configuration.userscripts.scripts.len()); + for script in &configuration.userscripts.scripts { + scripts.push(describe_script(script, &user_script_store).await); + } + + let response = UserScriptsResponse { + enabled: configuration.userscripts.enabled, + allow_private_network_requests: configuration.userscripts.allow_private_network_requests, + scripts, + }; + + Ok(Response::builder() + .header(http::header::CONTENT_TYPE, "application/json") + .body(serde_json::to_string(&response).unwrap()) + .unwrap()) +} + +async fn get_userscript_body( + file_name: String, + _user_script_store: UserScriptStore, +) -> Result { + let configuration = match Configuration::read_from_home().await { + Ok(configuration) => configuration, + Err(err) => { + log::error!("Failed to read configuration: {err}"); + return Ok(get_error_response(err)); + } + }; + + let Some(script) = configuration.userscripts.find(&file_name) else { + return Ok(super::get_unprocessable_response("No such userscript")); + }; + + match script.read_body().await { + Ok(body) => Ok(Response::builder() + .header(http::header::CONTENT_TYPE, "text/plain; charset=utf-8") + .body(body) + .unwrap()), + Err(err) => { + log::error!("Failed to read userscript body: {err}"); + Ok(get_error_response(err)) + } + } +} + +/// Map a configuration error to the right status: a rejected script is the +/// caller's problem (422), anything else is ours (500). +fn userscript_error_response(err: ConfigurationError) -> Response { + match err { + ConfigurationError::UserScript(err) => get_unprocessable_response(&err.to_string()), + ConfigurationError::UserScriptFetchError(message) => get_unprocessable_response(&message), + ConfigurationError::UserScriptNotFound(_) => { + get_unprocessable_response("No such userscript") + } + err => { + log::error!("Userscript operation failed: {err}"); + get_error_response(err) + } + } +} + +async fn set_engine_settings( + request: EngineSettingsRequest, + http_client: reqwest::Client, + configuration_save_lock: Arc>, + user_script_store: UserScriptStore, + private_network_access: PrivateNetworkAccess, +) -> Result { + // The lock is held across read-modify-write so a concurrent mutation cannot + // be lost by writing back a configuration read before it landed. + let _guard = configuration_save_lock.lock().await; + + let mut configuration = match Configuration::read_from_home().await { + Ok(configuration) => configuration, + Err(err) => return Ok(get_error_response(err)), + }; + + if let Some(enabled) = request.enabled { + configuration.userscripts.enabled = enabled; + } + if let Some(allowed) = request.allow_private_network_requests { + configuration.userscripts.allow_private_network_requests = allowed; + } + + if let Err(err) = configuration.save().await { + return Ok(userscript_error_response(err)); + } + + // Applied to the live relay switch immediately, so this behaves like every + // other userscript setting rather than waiting for a reload. + private_network_access.set(configuration.userscripts.allow_private_network_requests); + + reload_userscripts(&user_script_store, &configuration, &http_client).await; + + Ok(Response::builder() + .status(http::StatusCode::ACCEPTED) + .body(String::new()) + .unwrap()) +} + +async fn change_userscript_status( + requests: Vec, + http_client: reqwest::Client, + configuration_save_lock: Arc>, + user_script_store: UserScriptStore, +) -> Result { + let _guard = configuration_save_lock.lock().await; + + let mut configuration = match Configuration::read_from_home().await { + Ok(configuration) => configuration, + Err(err) => return Ok(get_error_response(err)), + }; + + for request in requests { + if let Err(err) = configuration + .set_userscript_enabled_status(&request.file_name, request.enabled) + .await + { + return Ok(userscript_error_response(err)); + } + } + + reload_userscripts(&user_script_store, &configuration, &http_client).await; + + Ok(Response::builder() + .status(http::StatusCode::ACCEPTED) + .body(String::new()) + .unwrap()) +} + +async fn add_userscript( + request: AddUserScriptRequest, + http_client: reqwest::Client, + configuration_save_lock: Arc>, + user_script_store: UserScriptStore, +) -> Result { + // Resolve the source before taking the lock: fetching a remote script can + // be slow and must not serialize other configuration writers. + let (body, url) = match (request.body, request.url) { + (_, Some(url)) => { + let Ok(url) = Url::parse(url.trim()) else { + return Ok(get_unprocessable_response("The script URL is not valid")); + }; + match fetch_userscript(&url, &http_client).await { + Ok(body) => (body, Some(url)), + Err(err) => return Ok(userscript_error_response(err)), + } + } + (Some(body), None) => (body, None), + (None, None) => { + return Ok(get_unprocessable_response( + "Provide either a script body or a URL to install from", + )) + } + }; + + let file_name = match &url { + // Keyed by URL so re-installing the same script reuses its body file. + Some(url) => calc_userscript_filename(url.as_str()), + None => calc_local_userscript_filename(), + }; + + let _guard = configuration_save_lock.lock().await; + + let mut configuration = match Configuration::read_from_home().await { + Ok(configuration) => configuration, + Err(err) => return Ok(get_error_response(err)), + }; + + if configuration.userscripts.find(&file_name).is_some() { + return Ok(Response::builder() + .status(http::StatusCode::CONFLICT) + .header(http::header::CONTENT_TYPE, "application/json") + .body( + serde_json::to_string(&super::ApiError { + error: "This userscript is already installed".to_string(), + }) + .unwrap(), + ) + .unwrap()); + } + + let script = UserScript { + enabled: request.enabled, + // Replaced with the parsed `@name` by `add_userscript`. + title: String::new(), + file_name, + url, + }; + + if let Err(err) = configuration.add_userscript(script, &body).await { + return Ok(userscript_error_response(err)); + } + + reload_userscripts(&user_script_store, &configuration, &http_client).await; + + Ok(Response::builder() + .status(http::StatusCode::CREATED) + .body(String::new()) + .unwrap()) +} + +/// Refresh one script or all of them from upstream. +/// +/// Unlike the periodic refresh in `ConfigurationUpdater`, this holds the save +/// lock, so a `@name` or `@version` that changed upstream is persisted rather +/// than being recomputed on every read. +async fn refresh_userscripts( + request: RefreshRequest, + http_client: reqwest::Client, + configuration_save_lock: Arc>, + user_script_store: UserScriptStore, +) -> Result { + let _guard = configuration_save_lock.lock().await; + + let mut configuration = match Configuration::read_from_home().await { + Ok(configuration) => configuration, + Err(err) => return Ok(get_error_response(err)), + }; + + let outcomes = match request.file_name { + Some(file_name) => { + let Some(script) = configuration + .userscripts + .scripts + .iter_mut() + .find(|script| script.file_name == file_name) + else { + return Ok(get_unprocessable_response("No such userscript")); + }; + + if script.url.is_none() { + return Ok(get_unprocessable_response( + "This userscript was pasted rather than installed from a URL, so there is \ + nothing to refresh.", + )); + } + + let outcome = match script.update(&http_client).await { + Ok(UserScriptUpdate::Updated { version }) => RefreshOutcome::Updated { version }, + Ok(UserScriptUpdate::AlreadyCurrent) => RefreshOutcome::AlreadyCurrent, + Err(err) => RefreshOutcome::Failed { + error: err.to_string(), + }, + }; + + vec![UserScriptRefresh { + file_name: script.file_name.clone(), + title: script.title.clone(), + outcome, + }] + } + None => configuration.update_userscripts(&http_client).await, + }; + + // Titles and versions may have moved on; persist them while we hold the lock. + if let Err(err) = configuration.save().await { + return Ok(userscript_error_response(err)); + } + + reload_userscripts(&user_script_store, &configuration, &http_client).await; + + Ok(Response::builder() + .header(http::header::CONTENT_TYPE, "application/json") + .body(serde_json::to_string(&outcomes).unwrap()) + .unwrap()) +} + +async fn update_userscript( + request: UpdateUserScriptRequest, + http_client: reqwest::Client, + configuration_save_lock: Arc>, + user_script_store: UserScriptStore, +) -> Result { + let _guard = configuration_save_lock.lock().await; + + let mut configuration = match Configuration::read_from_home().await { + Ok(configuration) => configuration, + Err(err) => return Ok(get_error_response(err)), + }; + + if let Err(err) = configuration + .replace_userscript_body(&request.file_name, &request.body) + .await + { + return Ok(userscript_error_response(err)); + } + + reload_userscripts(&user_script_store, &configuration, &http_client).await; + + Ok(Response::builder() + .status(http::StatusCode::OK) + .body(String::new()) + .unwrap()) +} + +async fn delete_userscript( + request: DeleteUserScriptRequest, + http_client: reqwest::Client, + configuration_save_lock: Arc>, + user_script_store: UserScriptStore, + gm_storage: GmStorageStore, +) -> Result { + let _guard = configuration_save_lock.lock().await; + + let mut configuration = match Configuration::read_from_home().await { + Ok(configuration) => configuration, + Err(err) => return Ok(get_error_response(err)), + }; + + if let Err(err) = configuration.remove_userscript(&request.file_name).await { + return Ok(userscript_error_response(err)); + } + + // An uninstalled script must not leave its stored values behind to be + // silently inherited if a script with the same file name is installed later. + gm_storage.forget(&request.file_name); + + reload_userscripts(&user_script_store, &configuration, &http_client).await; + + Ok(Response::builder() + .status(http::StatusCode::NO_CONTENT) + .body(String::new()) + .unwrap()) +} + +pub(super) fn create_routes( + configuration_save_lock: Arc>, + http_client: reqwest::Client, + user_script_store: UserScriptStore, + gm_storage: GmStorageStore, + private_network_access: PrivateNetworkAccess, +) -> BoxedFilter<(impl warp::Reply,)> { + // The master switch lives on a fixed sub-path, so it must be matched before + // the `` parameter route below would swallow it. + let engine_enabled_route = warp::path("enabled") + .and(warp::path::end()) + .and(warp::put()) + .and(warp::body::json()) + .and(super::with_http_client(http_client.clone())) + .and(super::with_configuration_save_lock( + configuration_save_lock.clone(), + )) + .and(super::with_arc(user_script_store.clone())) + .and(super::with_arc(private_network_access)) + .and_then(self::set_engine_settings); + + let refresh_route = warp::path("update") + .and(warp::path::end()) + .and(warp::post()) + .and(warp::body::json()) + .and(super::with_http_client(http_client.clone())) + .and(super::with_configuration_save_lock( + configuration_save_lock.clone(), + )) + .and(super::with_arc(user_script_store.clone())) + .and_then(self::refresh_userscripts); + + let body_route = warp::path::param::() + .and(warp::path("body")) + .and(warp::path::end()) + .and(warp::get()) + .and(super::with_arc(user_script_store.clone())) + .and_then(self::get_userscript_body); + + engine_enabled_route + .or(refresh_route) + .or(body_route) + .or(warp::path::end() + .and(warp::get()) + .and(super::with_arc(user_script_store.clone())) + .and_then(self::get_userscripts)) + .or(warp::path::end() + .and(warp::put()) + .and(warp::body::json()) + .and(super::with_http_client(http_client.clone())) + .and(super::with_configuration_save_lock( + configuration_save_lock.clone(), + )) + .and(super::with_arc(user_script_store.clone())) + .and_then(self::change_userscript_status)) + .or(warp::path::end() + .and(warp::post()) + .and(warp::body::json()) + .and(super::with_http_client(http_client.clone())) + .and(super::with_configuration_save_lock( + configuration_save_lock.clone(), + )) + .and(super::with_arc(user_script_store.clone())) + .and_then(self::add_userscript)) + .or(warp::path::end() + .and(warp::patch()) + .and(warp::body::json()) + .and(super::with_http_client(http_client.clone())) + .and(super::with_configuration_save_lock( + configuration_save_lock.clone(), + )) + .and(super::with_arc(user_script_store.clone())) + .and_then(self::update_userscript)) + .or(warp::path::end() + .and(warp::delete()) + .and(warp::body::json()) + .and(super::with_http_client(http_client)) + .and(super::with_configuration_save_lock(configuration_save_lock)) + .and(super::with_arc(user_script_store)) + .and(super::with_arc(gm_storage)) + .and_then(self::delete_userscript)) + .boxed() +} diff --git a/web_frontend/src/main.rs b/web_frontend/src/main.rs index dcc3431..979aaef 100644 --- a/web_frontend/src/main.rs +++ b/web_frontend/src/main.rs @@ -26,6 +26,8 @@ mod settings; mod settings_textarea; mod submit_banner; mod tls_failures; +mod userscript_edit; +mod userscripts; #[derive(Debug, Deserialize, Clone)] pub(crate) struct ApiError { diff --git a/web_frontend/src/settings.rs b/web_frontend/src/settings.rs index 0fd78c0..001114d 100644 --- a/web_frontend/src/settings.rs +++ b/web_frontend/src/settings.rs @@ -6,6 +6,7 @@ use crate::general::GeneralSettings; use crate::pac::PacSettingsPage; use crate::set_title; use crate::settings_textarea::SettingsTextarea; +use crate::userscripts::UserScriptsPage; use yew::prelude::*; use yew::{html, Html}; use yew_router::prelude::*; @@ -20,6 +21,8 @@ pub enum SettingsRoute { Exclusions, #[at("/settings/custom-filters")] CustomFilters, + #[at("/settings/userscripts")] + Userscripts, #[at("/settings/pac")] Pac, #[at("/settings/account")] @@ -89,6 +92,11 @@ pub fn switch_settings(route: SettingsRoute) -> Html { html! {} } + SettingsRoute::Userscripts => { + set_title("Settings - Userscripts"); + + html! { } + } SettingsRoute::Pac => { set_title("Settings - PAC"); @@ -112,6 +120,7 @@ pub fn switch_settings(route: SettingsRoute) -> Html { classes={get_classes(route, SettingsRoute::Filters)} to={SettingsRoute::Filters}> { "Filters" }> classes={get_classes(route, SettingsRoute::Exclusions)} to={SettingsRoute::Exclusions}> { "Exclusions" }> classes={get_classes(route, SettingsRoute::CustomFilters)} to={SettingsRoute::CustomFilters}> { "Custom filters" }> + classes={get_classes(route, SettingsRoute::Userscripts)} to={SettingsRoute::Userscripts}> { "Userscripts" }> classes={get_classes(route, SettingsRoute::Pac)} to={SettingsRoute::Pac}> { "PAC" }> classes={get_classes(route, SettingsRoute::Account)} to={SettingsRoute::Account}> { "Account" }> classes={get_classes(route, SettingsRoute::Debug)} to={SettingsRoute::Debug}> { "Debug" }> diff --git a/web_frontend/src/userscript_edit.rs b/web_frontend/src/userscript_edit.rs new file mode 100644 index 0000000..cc0ee65 --- /dev/null +++ b/web_frontend/src/userscript_edit.rs @@ -0,0 +1,280 @@ +//! Modal for editing or uninstalling a userscript. +//! +//! The body is fetched on open rather than passed in as a property: script +//! bodies are large and the list endpoint deliberately serves only metadata. + +use crate::{failure_banner, ApiError}; +use gloo_net::http::Request; +use serde::Serialize; +use wasm_bindgen_futures::spawn_local; +use web_sys::HtmlTextAreaElement; +use yew::prelude::*; + +const USERSCRIPTS_RESOURCE_URL: &str = "/api/userscripts"; + +#[derive(Debug, Serialize)] +struct UpdateUserScriptRequest { + file_name: String, + body: String, +} + +#[derive(Debug, Serialize)] +struct DeleteUserScriptRequest { + file_name: String, +} + +#[derive(Properties, PartialEq)] +pub struct Props { + /// Identifies the script to the backend; stable across edits, including + /// edits that change its `@name`. + pub file_name: String, + pub title: String, + /// Emitted when the modal is dismissed without a change. + pub on_close: Callback<()>, + /// Emitted after a successful save or delete; the parent should unmount the + /// modal and reload its script list. + pub on_changed: Callback<()>, +} + +pub enum Message { + BodyLoaded(String), + BodyChanged(String), + Save, + Delete, + ConfirmDelete, + CancelDelete, + Succeeded, + Failed(String), + AcknowledgeError, +} + +pub struct UserScriptEditModal { + body: Option, + busy: bool, + confirming_delete: bool, + error: Option, +} + +impl Component for UserScriptEditModal { + type Message = Message; + type Properties = Props; + + fn create(ctx: &Context) -> Self { + Self::fetch_body(ctx); + + Self { + body: None, + busy: false, + confirming_delete: false, + error: None, + } + } + + fn changed(&mut self, ctx: &Context, old_props: &Self::Properties) -> bool { + // Re-fetch when the modal is pointed at a different script without + // being unmounted in between. + if ctx.props().file_name != old_props.file_name { + self.body = None; + self.busy = false; + self.confirming_delete = false; + self.error = None; + Self::fetch_body(ctx); + } + true + } + + fn update(&mut self, ctx: &Context, msg: Self::Message) -> bool { + match msg { + Message::BodyLoaded(body) => self.body = Some(body), + Message::BodyChanged(body) => self.body = Some(body), + Message::Save => { + if self.busy { + return false; + } + let Some(body) = self.body.clone() else { + return false; + }; + + let request_body = serde_json::to_string(&UpdateUserScriptRequest { + file_name: ctx.props().file_name.clone(), + body, + }) + .unwrap(); + + self.dispatch(ctx, Request::patch(USERSCRIPTS_RESOURCE_URL), request_body); + } + Message::Delete => self.confirming_delete = true, + Message::CancelDelete => self.confirming_delete = false, + Message::ConfirmDelete => { + if self.busy { + return false; + } + + let request_body = serde_json::to_string(&DeleteUserScriptRequest { + file_name: ctx.props().file_name.clone(), + }) + .unwrap(); + + self.dispatch(ctx, Request::delete(USERSCRIPTS_RESOURCE_URL), request_body); + } + Message::Succeeded => { + self.busy = false; + ctx.props().on_changed.emit(()); + } + Message::Failed(message) => { + log::error!("Userscript edit failed: {message}"); + self.busy = false; + self.confirming_delete = false; + self.error = Some(message); + } + Message::AcknowledgeError => self.error = None, + } + + true + } + + fn view(&self, ctx: &Context) -> Html { + let on_body_input = ctx.link().callback(|event: InputEvent| { + let textarea: HtmlTextAreaElement = event.target_unchecked_into(); + Message::BodyChanged(textarea.value()) + }); + + let error_banner = match &self.error { + Some(message) => failure_banner!( + true, + ctx.link().callback(|_| Message::AcknowledgeError), + message.clone() + ), + None => html! {}, + }; + + let body_editor = match &self.body { + Some(body) => html! { +