diff --git a/README.md b/README.md index a8a4da4..2743c98 100644 --- a/README.md +++ b/README.md @@ -106,26 +106,20 @@ See the [Steel browser reference](docs/references/steel-browser.md) and the [com ## Endpoint Resolution -For browser sessions, browser actions, and top-level API tools (`scrape`, `screenshot`, `pdf`), endpoint selection is deterministic. - -Self-hosted precedence (highest to lowest): +Every command that calls the Steel API uses one URL, chosen in this order: 1. `--api-url ` -2. `STEEL_BROWSER_API_URL` -3. `STEEL_LOCAL_API_URL` -4. `browser.apiUrl` in `~/.config/steel/config.json` -5. `http://localhost:3000/v1` - -Cloud precedence: +2. `--local`: `browser.apiUrl` in `~/.config/steel/config.json`, or `http://localhost:3000/v1` +3. `STEEL_API_URL` +4. `https://api.steel.dev/v1` -1. `STEEL_API_URL` -2. `https://api.steel.dev/v1` +URLs on `steel.dev` are Steel Cloud and need an API key. Any other URL is treated as self-hosted. ## Output and Runtime Notes -- `steel scrape` defaults to markdown-first output for token efficiency; use `--raw` for the full JSON payload. +- `steel scrape` prints markdown by default for token efficiency; use `--json` for the full API response. - `steel browser start` prints a display-safe connect URL with sensitive query values redacted. -- Piped output auto-switches to JSON for machine-readable workflows. Use `--json` to force it, or `STEEL_FORCE_TTY=1` to disable. +- Output is text by default, including when piped. Pass `--json` for machine-readable output. ## Documentation Map diff --git a/docs/cli-reference.md b/docs/cli-reference.md index c173c7a..c184cbe 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -67,7 +67,6 @@ Steel CLI - browser automation for AI agents. This file is generated from `steel - [steel browser dialog](#steel-browser-dialog) - [steel browser status](#steel-browser-status) - [steel browser cancel](#steel-browser-cancel) -- [steel browser close](#steel-browser-close) - [steel sessions](#steel-sessions) - [steel sessions list](#steel-sessions-list) - [steel sessions get](#steel-sessions-get) @@ -113,7 +112,7 @@ These options are accepted by every command. - `--json` (boolean, optional): Output all results as JSON (structured output for automation) - `--local` (boolean, optional): Use local Steel runtime instead of cloud -- `--api-url` (string, optional): Explicit self-hosted API endpoint URL +- `--api-url` (string, optional): Steel API endpoint URL (overrides --local and STEEL_API_URL) - `-h, --help`: Print help for a command - `-V, --version`: Print the Steel CLI version @@ -225,7 +224,6 @@ steel browser - `dialog`: Accept or dismiss an open dialog - `status`: Wait for a command that is still running - `cancel`: Stop a command that is still running -- `close`: Close the browser session ## steel browser start @@ -241,9 +239,9 @@ steel browser start - `--stealth` (boolean, optional): Enable stealth mode (humanize interactions + auto CAPTCHA) - `-p, --proxy` (string, optional): Use a residential proxy -- `--session-timeout` (string, optional): Session timeout in milliseconds (create-time only) +- `--timeout` (string, optional): Session timeout in milliseconds (create-time only) - `--inactivity-timeout` (string, optional): Inactivity timeout in milliseconds: release the session when no command, CDP message, or remote input arrives for this long. Defaults to 600000 (10 minutes); pass 0 to disable (create-time only) -- `--session-solve-captcha` (boolean, optional): Enable manual CAPTCHA solving on new sessions (create-time only) +- `--solve-captcha` (boolean, optional): Enable manual CAPTCHA solving on new sessions (create-time only) - `--profile` (string, optional): Named profile to persist browser state across sessions - `--update-profile` (boolean, optional): Save session state back to the profile when the session ends - `--namespace` (string, optional): Credential namespace @@ -259,6 +257,8 @@ Stop a browser session steel browser stop ``` +Aliases: `close`, `quit`, `exit` + ### Parameters - `-a, --all` (boolean, optional): Stop all sessions @@ -310,7 +310,6 @@ steel browser captcha solve ### Parameters -- `--session-id` (string, optional): Explicit session ID (overrides --session name lookup) - `--page-id` (string, optional): Page ID - `--url` (string, optional): Page URL for targeted CAPTCHA solving - `--task-id` (string, optional): CAPTCHA task ID for targeted solving @@ -327,7 +326,6 @@ steel browser captcha status ### Parameters -- `--session-id` (string, optional): Explicit session ID (overrides --session name lookup) - `--page-id` (string, optional): Page ID - `-w, --wait` (boolean, optional): Wait for terminal status - `--timeout` (string, optional): Timeout in milliseconds for --wait mode @@ -1089,18 +1087,6 @@ steel browser cancel - `execution` (string, required): Execution id such as x-7 -## steel browser close - -Close the browser session - -### Usage - -```bash -steel browser close -``` - -Aliases: `quit`, `exit` - ## steel sessions Cloud session management and debugging diff --git a/docs/references/agent-troubleshooting.md b/docs/references/agent-troubleshooting.md index bcebc8b..083a288 100644 --- a/docs/references/agent-troubleshooting.md +++ b/docs/references/agent-troubleshooting.md @@ -32,7 +32,7 @@ From `steel browser start` output: Do not assume `connect_url` contains usable raw credentials. -From action output (`--json`, or any piped output): +From action output with `--json`: - `success`: whether the command did what it was asked. - `error_code`: the engine error code on failure, such as `stale_ref`. diff --git a/docs/references/steel-browser-commands.md b/docs/references/steel-browser-commands.md index f0b9373..9d319df 100644 --- a/docs/references/steel-browser-commands.md +++ b/docs/references/steel-browser-commands.md @@ -8,8 +8,7 @@ Every `steel browser` action sends one request to the session's commands API (`P steel browser start [--session ] # create a session (default name: default) steel browser sessions # list local sessions that are still live steel browser live # print the live viewer URL -steel browser stop [--session ] # release the session (-a for all) -steel browser close # same as stop (aliases: quit, exit) +steel browser stop [--session ] # release the session (-a for all; aliases: close, quit, exit) ``` - Actions without `--session` use the `default` session. It starts on first use. @@ -128,14 +127,14 @@ Dialogs stay open until you answer them. A command that opens one prints the dia steel browser batch "fill e3 Seoul" "fill e4 Tokyo" "click e5" "snapshot -i" ``` -Batch sends all commands in one request (at most 50). They run in order and stop at the first failure. `tab new`, `tab switch`, `tab close`, `dialog`, `status`, `cancel`, and `close` must run on their own. +Batch sends all commands in one request (at most 50). They run in order and stop at the first failure. `tab new`, `tab switch`, `tab close`, `dialog`, `status`, and `cancel` must run on their own. `stop` is not a batch command. ## Retries and output - The CLI sends a random `Idempotency-Key` and the last operation id. If the API cannot reach the browser, it retries with the same key, so the command never runs twice. - When a command's response is lost, run the same command again. You get the stored result instead of a second run, with the note `This is the stored result of an earlier run`. - Text output prints the result. Notes about dialogs, new tabs, downloads, and replays go to stderr. -- `--json` (or piped output) prints the engine response under `data`, with any `notices`. +- `--json` prints the engine response under `data`, with any `notices`. - A failed command exits with code 1 and prints the engine error code, such as `stale_ref`, `not_actionable`, or `tab_required`. ## Removed in 0.5 diff --git a/docs/references/steel-browser.md b/docs/references/steel-browser.md index aa36a38..fb3bdb3 100644 --- a/docs/references/steel-browser.md +++ b/docs/references/steel-browser.md @@ -12,27 +12,16 @@ Each `steel browser` action is one HTTP request to the session's commands API: The full command list is in [./steel-browser-commands.md](./steel-browser-commands.md). -## Modes - -- Cloud mode (default): no mode flag required. -- Self-hosted mode: use `--local` or `--api-url `. - -`--api-url` implies self-hosted mode. - ## Endpoint Resolution -Self-hosted endpoint precedence: +The API URL is chosen in this order: 1. `--api-url ` -2. `STEEL_BROWSER_API_URL` -3. `STEEL_LOCAL_API_URL` -4. `browser.apiUrl` in `~/.config/steel/config.json` -5. `http://localhost:3000/v1` +2. `--local`: `browser.apiUrl` in `~/.config/steel/config.json`, or `http://localhost:3000/v1` +3. `STEEL_API_URL` +4. `https://api.steel.dev/v1` -Cloud endpoint precedence: - -1. `STEEL_API_URL` -2. `https://api.steel.dev/v1` +URLs on `steel.dev` are Steel Cloud and need an API key. Any other URL is treated as self-hosted. ## Steel-Owned Command Contracts @@ -47,9 +36,9 @@ Main flags: - `--session ` - `--stealth` - `--proxy ` -- `--session-timeout ` +- `--timeout ` - `--inactivity-timeout ` (default 600000; `0` turns it off) -- `--session-solve-captcha` +- `--solve-captcha` - `--namespace ` - `--credentials` - `--profile ` @@ -61,7 +50,7 @@ Flag semantics: - `stealthConfig.humanizeInteractions = true` - `stealthConfig.autoCaptchaSolving = true` - `solveCaptcha = true` -- `--session-solve-captcha` enables manual CAPTCHA solving for new sessions: +- `--solve-captcha` enables manual CAPTCHA solving for new sessions: - `solveCaptcha = true` - `stealthConfig.autoCaptchaSolving` is not forced on - `--proxy ` sets `proxyUrl` on session creation. The Sessions API may return @@ -135,14 +124,12 @@ Purpose: manually trigger CAPTCHA solving for a target session. Main flags: -- `--session-id ` -- `--session ` +- `--session ` - `--page-id ` - `--url ` - `--task-id ` - `--local` - `--api-url ` -- `--raw` API mapping: @@ -199,7 +186,7 @@ Self-hosted steel-browser serves the commands API only when it runs with - CAPTCHA solving notes: - CAPTCHA solving requires a paid Steel plan. - `--stealth` enables auto solving; wait and monitor with screenshots. - - `--session-solve-captcha` enables manual solving; use `steel browser captcha solve`. + - `--solve-captcha` enables manual solving; use `steel browser captcha solve`. - Default sessions do not solve captchas; restart with CAPTCHA solving enabled and navigate back. - Proxy + CAPTCHA solving together is a strong anti-bot evasion combination. - Need hard reset of stale session mapping: diff --git a/docs/references/steel-cli.md b/docs/references/steel-cli.md index 8e8cb30..94becef 100644 --- a/docs/references/steel-cli.md +++ b/docs/references/steel-cli.md @@ -64,8 +64,8 @@ For generated flags and argument schemas, use [../cli-reference.md](../cli-refer keys redacted. - `steel scrape`, `steel screenshot`, and `steel pdf` share local/cloud endpoint resolution conventions with browser lifecycle commands. -- `steel scrape` defaults to markdown-first output for token efficiency; use - `--raw` to print the full API JSON payload. +- `steel scrape` prints markdown by default for token efficiency; use `--json` + to print the full API response. ## Core Config Paths @@ -75,7 +75,7 @@ For generated flags and argument schemas, use [../cli-reference.md](../cli-refer - Browser session state: `~/.config/steel/browser-sessions/.json` - Profile metadata: `~/.config/steel/profiles/.json` -On the first run where telemetry is enabled, the CLI prints a one-time notice to stderr describing what is collected and how to opt out. The notice is suppressed in JSON/non-TTY output. +On the first run where telemetry is enabled, the CLI prints a one-time notice to stderr describing what is collected and how to opt out. The notice is suppressed with `--json`. Telemetry can be disabled persistently in `config.json` with: @@ -90,13 +90,11 @@ Telemetry can be disabled persistently in `config.json` with: ## Environment Variables (Common) - `STEEL_API_KEY`: cloud auth for the session and commands APIs. -- `STEEL_API_URL`: cloud API endpoint override. -- `STEEL_BROWSER_API_URL`: canonical self-hosted local endpoint override. -- `STEEL_LOCAL_API_URL`: backward-compatible self-hosted alias. +- `STEEL_API_URL`: API endpoint override, cloud or self-hosted. +- `STEEL_WS_URL`: WebSocket endpoint for `steel sessions logs --follow` and `traces --follow`. - `STEEL_CONFIG_DIR`: override config directory root. - `STEEL_TELEMETRY_HOST`: override the PostHog ingest host. - `STEEL_TELEMETRY_DISABLED`: force telemetry off. -- `STEEL_PROFILE`: default profile name for browser sessions. ## Key References diff --git a/scripts/generate-api.mjs b/scripts/generate-api.mjs index c04069f..df07ad6 100644 --- a/scripts/generate-api.mjs +++ b/scripts/generate-api.mjs @@ -218,12 +218,7 @@ ${queryPushes} } function methodArgs(operation) { - const args = [ - "&self", - "base_url: &str", - "mode: ApiMode", - "auth: &Auth", - ]; + const args = ["&self"]; if (operation.pathParams.some((parameter) => parameter.name === "id")) { args.push("session_id: &str"); } @@ -248,15 +243,8 @@ function renderClientMethod(operation) { return ` pub async fn cli_${snakeCase(operation.id)}( ${methodArgs(operation)}, ) -> Result { - self.request( - base_url, - mode, - reqwest::Method::${operation.method}, - ${requestPath}, - None, - auth, - ) - .await + self.request(reqwest::Method::${operation.method}, ${requestPath}, None) + .await }`; } @@ -317,8 +305,6 @@ use serde::{Deserialize, Serialize}; use serde_json::Value; use crate::api::client::{ApiError, SteelClient}; -use crate::config::auth::Auth; -use crate::config::settings::ApiMode; #[derive(Debug, Clone, Copy, Serialize, Deserialize)] pub struct OperationMetadata { diff --git a/src/api/client.rs b/src/api/client.rs index 4b59138..a1a3b2b 100644 --- a/src/api/client.rs +++ b/src/api/client.rs @@ -27,13 +27,6 @@ pub enum ApiError { body: Option, }, - #[error("Steel API request failed ({status}): {message}")] - RetryLater { - status: u16, - message: Cow<'static, str>, - seconds: u64, - }, - #[error(transparent)] Other(#[from] reqwest::Error), } @@ -80,72 +73,88 @@ fn extract_error_message(body: &Value, status_text: &str) -> Cow<'static, str> { pub struct SteelClient { http: Client, + base_url: String, + mode: ApiMode, + auth: Auth, } impl SteelClient { pub fn new() -> Result { + let (mode, base_url, auth) = crate::util::api::resolve_with_auth(); + Self::with(base_url, mode, auth) + } + + pub fn with(base_url: String, mode: ApiMode, auth: Auth) -> Result { let http = Client::builder().user_agent("steel-cli").build()?; - Ok(Self { http }) + Ok(Self { + http, + base_url, + mode, + auth, + }) + } + + pub fn at(&self, base_url: &str, mode: ApiMode) -> Self { + Self { + http: self.http.clone(), + base_url: base_url.to_string(), + mode, + auth: self.auth.clone(), + } + } + + pub fn base_url(&self) -> &str { + &self.base_url + } + + pub const fn mode(&self) -> ApiMode { + self.mode + } + + pub const fn auth(&self) -> &Auth { + &self.auth + } + + pub fn url(&self, path: &str) -> String { + format!("{}{path}", self.base_url) } pub fn authed( &self, method: reqwest::Method, - url: &str, - mode: ApiMode, - auth: &Auth, + path: &str, ) -> Result { - if mode == ApiMode::Cloud && auth.api_key.is_none() { + if self.mode == ApiMode::Cloud && self.auth.api_key.is_none() { return Err(ApiError::MissingAuth); } - let mut req = self.http.request(method, url); - if let Some(key) = &auth.api_key { + let mut req = self.http.request(method, self.url(path)); + if let Some(key) = &self.auth.api_key { req = req.header("Steel-Api-Key", key); } Ok(req) } - /// Make an authenticated API request. - /// Matches the shared request pattern in TS `requestApi()` and `requestTopLevelApi()`. pub async fn request( &self, - base_url: &str, - mode: ApiMode, method: reqwest::Method, path: &str, body: Option, - auth: &Auth, ) -> Result { - // Cloud mode requires auth - if mode == ApiMode::Cloud && auth.api_key.is_none() { - return Err(ApiError::MissingAuth); - } - - let url = format!("{base_url}{path}"); - - let mut req = self.http.request(method, &url); - req = req.header("Content-Type", "application/json"); - - if let Some(key) = &auth.api_key { - req = req.header("Steel-Api-Key", key); - } - + let mut req = self + .authed(method, path)? + .header("Content-Type", "application/json"); if let Some(body) = body { req = req.json(&body); } - let resp = req.send().await.map_err(|e| ApiError::Unreachable { - url: url.clone(), - source: e, + let resp = req.send().await.map_err(|source| ApiError::Unreachable { + url: self.url(path), + source, })?; let status = resp.status(); - let status_code = status.as_u16(); let status_text = status.canonical_reason().unwrap_or("").to_string(); - - // Read body as text, then try to parse as JSON let response_text = resp.text().await.map_err(ApiError::Other)?; - let response_data: Value = if response_text.trim().is_empty() { Value::Null } else { @@ -155,7 +164,7 @@ impl SteelClient { if !status.is_success() { let message = extract_error_message(&response_data, &status_text); return Err(ApiError::RequestFailed { - status: status_code, + status: status.as_u16(), message, body: Some(response_data), }); @@ -163,67 +172,6 @@ impl SteelClient { Ok(response_data) } - - pub async fn request_raw( - &self, - base_url: &str, - mode: ApiMode, - method: reqwest::Method, - path: &str, - body: Option, - auth: &Auth, - ) -> Result { - if mode == ApiMode::Cloud && auth.api_key.is_none() { - return Err(ApiError::MissingAuth); - } - - let url = format!("{base_url}{path}"); - let mut req = self.http.request(method, &url); - req = req.header("Content-Type", "application/json"); - if let Some(key) = &auth.api_key { - req = req.header("Steel-Api-Key", key); - } - if let Some(body) = body { - req = req.json(&body); - } - - let resp = req.send().await.map_err(|e| ApiError::Unreachable { - url: url.clone(), - source: e, - })?; - - let status = resp.status(); - if status.is_success() { - return Ok(resp); - } - - let status_code = status.as_u16(); - let status_text = status.canonical_reason().unwrap_or("").to_string(); - let retry_after = resp - .headers() - .get(reqwest::header::RETRY_AFTER) - .and_then(|v| v.to_str().ok()) - .and_then(|v| v.trim().parse::().ok()); - let response_text = resp.text().await.map_err(ApiError::Other)?; - let response_data: Value = if response_text.trim().is_empty() { - Value::Null - } else { - serde_json::from_str(&response_text).unwrap_or(Value::String(response_text)) - }; - let message = extract_error_message(&response_data, &status_text); - if let Some(seconds) = retry_after { - return Err(ApiError::RetryLater { - status: status_code, - message, - seconds, - }); - } - Err(ApiError::RequestFailed { - status: status_code, - message, - body: Some(response_data), - }) - } } #[cfg(test)] @@ -286,22 +234,13 @@ mod tests { #[tokio::test] async fn missing_auth_in_cloud_mode() { - let client = SteelClient::new().unwrap(); let auth = Auth { api_key: None, source: crate::config::auth::AuthSource::None, }; + let client = SteelClient::with("http://localhost".into(), ApiMode::Cloud, auth).unwrap(); - let result = client - .request( - "http://localhost", - ApiMode::Cloud, - reqwest::Method::GET, - "/test", - None, - &auth, - ) - .await; + let result = client.request(reqwest::Method::GET, "/test", None).await; assert!(matches!(result, Err(ApiError::MissingAuth))); } @@ -309,22 +248,13 @@ mod tests { #[tokio::test] async fn local_mode_allows_no_auth() { // This will fail to connect, but should NOT fail with MissingAuth - let client = SteelClient::new().unwrap(); let auth = Auth { api_key: None, source: crate::config::auth::AuthSource::None, }; + let client = SteelClient::with("http://127.0.0.1:1".into(), ApiMode::Local, auth).unwrap(); - let result = client - .request( - "http://127.0.0.1:1", - ApiMode::Local, - reqwest::Method::GET, - "/test", - None, - &auth, - ) - .await; + let result = client.request(reqwest::Method::GET, "/test", None).await; assert!(matches!(result, Err(ApiError::Unreachable { .. }))); } diff --git a/src/api/commands.rs b/src/api/commands.rs index 54c1318..02c23f3 100644 --- a/src/api/commands.rs +++ b/src/api/commands.rs @@ -3,8 +3,6 @@ use std::time::Duration; use serde_json::{Value, json}; use crate::api::client::{ApiError, SteelClient}; -use crate::config::auth::Auth; -use crate::config::settings::ApiMode; const REQUEST_TIMEOUT: Duration = Duration::from_secs(40); const FILE_TIMEOUT: Duration = Duration::from_secs(300); @@ -46,9 +44,6 @@ impl Reply { pub struct Session<'a> { pub client: &'a SteelClient, - pub base_url: &'a str, - pub mode: ApiMode, - pub auth: &'a Auth, pub id: &'a str, } @@ -63,15 +58,16 @@ const fn backoff(attempt: u32) -> Duration { } impl Session<'_> { - fn url(&self, path: &str) -> String { - format!("{}/sessions/{}{path}", self.base_url, self.id) + fn path(&self, suffix: &str) -> String { + format!("/sessions/{}{suffix}", self.id) } async fn post(&self, body: &Value, key: Option<&str>) -> Result { - let url = self.url("/commands"); + let path = self.path("/commands"); + let url = self.client.url(&path); let mut req = self .client - .authed(reqwest::Method::POST, &url, self.mode, self.auth)? + .authed(reqwest::Method::POST, &path)? .timeout(REQUEST_TIMEOUT) .json(body); if let Some(key) = key { @@ -192,14 +188,15 @@ impl Session<'_> { } pub async fn upload_file(&self, bytes: Vec, name: &str) -> Result { - let url = self.url("/files"); + let path = self.path("/files"); + let url = self.client.url(&path); let part = reqwest::multipart::Part::bytes(bytes).file_name(name.to_string()); let form = reqwest::multipart::Form::new() .part("file", part) .text("path", name.to_string()); let resp = self .client - .authed(reqwest::Method::POST, &url, self.mode, self.auth)? + .authed(reqwest::Method::POST, &path)? .timeout(FILE_TIMEOUT) .multipart(form) .send() @@ -225,10 +222,11 @@ impl Session<'_> { .split('/') .map(|segment| urlencoding::encode(segment).into_owned()) .collect(); - let url = self.url(&format!("/files/{}", encoded.join("/"))); + let path = self.path(&format!("/files/{}", encoded.join("/"))); + let url = self.client.url(&path); let resp = self .client - .authed(reqwest::Method::GET, &url, self.mode, self.auth)? + .authed(reqwest::Method::GET, &path)? .timeout(FILE_TIMEOUT) .send() .await @@ -267,7 +265,8 @@ fn failed(status: reqwest::StatusCode, body: &Value) -> ApiError { #[cfg(test)] mod tests { use super::*; - use crate::config::auth::AuthSource; + use crate::config::auth::{Auth, AuthSource}; + use crate::config::settings::ApiMode; use wiremock::matchers::{body_partial_json, header, header_exists, method, path}; use wiremock::{Mock, MockServer, Request, ResponseTemplate}; @@ -291,14 +290,9 @@ mod tests { } async fn run(server: &MockServer, body: Value) -> Result { - let client = SteelClient::new().unwrap(); - let auth = auth(); - let base = server.uri(); + let client = SteelClient::with(server.uri(), ApiMode::Cloud, auth()).unwrap(); let session = Session { client: &client, - base_url: &base, - mode: ApiMode::Cloud, - auth: &auth, id: ID, }; session.run(body).await @@ -534,16 +528,13 @@ mod tests { #[tokio::test] async fn missing_api_key_fails_before_sending() { - let client = SteelClient::new().unwrap(); let auth = Auth { api_key: None, source: AuthSource::None, }; + let client = SteelClient::with("http://127.0.0.1:1".into(), ApiMode::Cloud, auth).unwrap(); let session = Session { client: &client, - base_url: "http://127.0.0.1:1", - mode: ApiMode::Cloud, - auth: &auth, id: ID, }; let result = session.run(json!({"commands": []})).await; diff --git a/src/api/generated.rs b/src/api/generated.rs index 93fd984..ad9bf4c 100644 --- a/src/api/generated.rs +++ b/src/api/generated.rs @@ -9,8 +9,6 @@ use serde::{Deserialize, Serialize}; use serde_json::Value; use crate::api::client::{ApiError, SteelClient}; -use crate::config::auth::Auth; -use crate::config::settings::ApiMode; #[derive(Debug, Clone, Copy, Serialize, Deserialize)] pub struct OperationMetadata { @@ -336,129 +334,74 @@ fn build_get_session_agent_traces_path( impl SteelClient { pub async fn cli_get_session_agent_logs( &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, session_id: &str, query: &GetSessionAgentLogsQuery, ) -> Result { self.request( - base_url, - mode, reqwest::Method::GET, &build_get_session_agent_logs_path(session_id, query), None, - auth, ) .await } - pub async fn cli_get_session( - &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, - session_id: &str, - ) -> Result { + pub async fn cli_get_session(&self, session_id: &str) -> Result { self.request( - base_url, - mode, reqwest::Method::GET, &build_get_session_path(session_id), None, - auth, ) .await } - pub async fn cli_get_sessions( - &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, - query: &GetSessionsQuery, - ) -> Result { - self.request( - base_url, - mode, - reqwest::Method::GET, - &build_get_sessions_path(query), - None, - auth, - ) - .await + pub async fn cli_get_sessions(&self, query: &GetSessionsQuery) -> Result { + self.request(reqwest::Method::GET, &build_get_sessions_path(query), None) + .await } pub async fn cli_get_session_logs( &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, session_id: &str, query: &GetSessionLogsQuery, ) -> Result { self.request( - base_url, - mode, reqwest::Method::GET, &build_get_session_logs_path(session_id, query), None, - auth, ) .await } - pub async fn cli_release_session( - &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, - session_id: &str, - ) -> Result { + pub async fn cli_release_session(&self, session_id: &str) -> Result { self.request( - base_url, - mode, reqwest::Method::POST, &build_release_session_path(session_id), None, - auth, ) .await } pub async fn cli_release_all_sessions( &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, query: &ReleaseAllSessionsQuery, ) -> Result { self.request( - base_url, - mode, reqwest::Method::POST, &build_release_all_sessions_path(query), None, - auth, ) .await } pub async fn cli_get_session_agent_traces( &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, session_id: &str, query: &GetSessionAgentTracesQuery, ) -> Result { self.request( - base_url, - mode, reqwest::Method::GET, &build_get_session_agent_traces_path(session_id, query), None, - auth, ) .await } diff --git a/src/api/projects.rs b/src/api/projects.rs index 834be10..9ec6ab7 100644 --- a/src/api/projects.rs +++ b/src/api/projects.rs @@ -5,8 +5,6 @@ use serde::Deserialize; use serde_json::Value; use crate::api::client::{ApiError, SteelClient}; -use crate::config::auth::Auth; -use crate::config::settings::ApiMode; #[derive(Debug, Clone, Deserialize)] #[serde(rename_all = "camelCase")] @@ -53,21 +51,8 @@ pub const fn environment_label(is_production: bool) -> &'static str { impl SteelClient { /// Fetch the projects visible to the current API key. - pub async fn get_projects( - &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, - ) -> Result { - self.request( - base_url, - mode, - reqwest::Method::GET, - "/projects", - None, - auth, - ) - .await + pub async fn get_projects(&self) -> Result { + self.request(reqwest::Method::GET, "/projects", None).await } } @@ -189,7 +174,8 @@ mod tests { // --- get_projects against a mock server --- - use crate::config::auth::AuthSource; + use crate::config::auth::{Auth, AuthSource}; + use crate::config::settings::ApiMode; use wiremock::matchers::{header, method, path}; use wiremock::{Mock, MockServer, ResponseTemplate}; @@ -212,16 +198,13 @@ mod tests { .mount(&server) .await; - let client = SteelClient::new().unwrap(); let auth = Auth { api_key: Some("test-key".into()), source: AuthSource::Env, }; + let client = SteelClient::with(server.uri(), ApiMode::Cloud, auth).unwrap(); - let data = client - .get_projects(&server.uri(), ApiMode::Cloud, &auth) - .await - .unwrap(); + let data = client.get_projects().await.unwrap(); let projects = parse_projects(&data); let current = resolve_current_project(&projects).unwrap(); diff --git a/src/api/session.rs b/src/api/session.rs index 9e8d41f..f038106 100644 --- a/src/api/session.rs +++ b/src/api/session.rs @@ -1,13 +1,14 @@ -//! Session API: create, list, get, release, captcha. -//! Ported from: cli/source/utils/browser/lifecycle/api-client.ts - +use anyhow::Context; +use serde::Deserialize; use serde_json::{Value, json}; use crate::api::client::{ApiError, SteelClient}; use crate::config::auth::Auth; use crate::config::settings::ApiMode; -/// Options for creating a session. Matches TS `StartSessionRequestOptions`. +const CLOUD_CONNECT_URL: &str = "wss://connect.steel.dev"; +const CLOUD_VIEWER_URL: &str = "https://app.steel.dev/sessions"; + #[derive(Debug, Default)] pub struct CreateSessionOptions { pub stealth: bool, @@ -23,45 +24,75 @@ pub struct CreateSessionOptions { pub credentials: bool, } -/// Extract session list from response. Matches TS `extractSessionList()`. -fn extract_session_list(data: &Value) -> Vec { - // Direct array - if let Some(arr) = data.as_array() { - return arr.iter().filter(|v| v.is_object()).cloned().collect(); +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct Session { + pub id: String, + pub status: Option, + pub websocket_url: Option, + pub session_viewer_url: Option, + pub timeout: Option, + pub inactivity_timeout: Option, + pub created_at: Option, + pub profile_id: Option, +} + +fn non_empty(value: Option<&str>) -> Option<&str> { + value.filter(|v| !v.trim().is_empty()) +} + +impl Session { + pub fn parse(value: Value) -> anyhow::Result { + serde_json::from_value(value).context("The Steel API returned an unexpected session.") } - // Nested { sessions: [...] } - if let Some(obj) = data.as_object() { - if let Some(Value::Array(arr)) = obj.get("sessions") { - return arr.iter().filter(|v| v.is_object()).cloned().collect(); - } + pub fn is_live(&self) -> bool { + !matches!(self.status.as_deref(), Some("released" | "failed")) + } - // Single session object with id - if obj.contains_key("id") { - return vec![data.clone()]; - } + pub fn timeout_ms(&self) -> Option { + self.timeout.filter(|&ms| ms > 0) } - vec![] -} + pub fn inactivity_timeout_ms(&self) -> Option { + self.inactivity_timeout.filter(|&ms| ms > 0) + } -/// Extract a single session from response. Matches TS `extractSingleSession()`. -fn extract_single_session(data: &Value) -> Result { - if let Some(obj) = data.as_object() { - // Nested { session: {...} } - if let Some(session) = obj.get("session") - && session.is_object() - { - return Ok(session.clone()); - } - return Ok(data.clone()); + pub fn created_at_ms(&self) -> Option { + let created = self + .created_at + .as_deref()? + .parse::() + .ok()?; + u64::try_from(created.as_millisecond()).ok() + } + + pub fn viewer_url(&self, mode: ApiMode) -> Option { + non_empty(self.session_viewer_url.as_deref()) + .map(str::to_string) + .or_else(|| (mode == ApiMode::Cloud).then(|| format!("{CLOUD_VIEWER_URL}/{}", self.id))) } - Err(ApiError::RequestFailed { - status: 0, - message: "Unexpected empty response from Steel session API.".into(), - body: None, - }) + pub fn connect_url(&self, mode: ApiMode, auth: &Auth) -> Option { + let given = non_empty(self.websocket_url.as_deref()).map(str::to_string); + let (ApiMode::Cloud, Some(key)) = (mode, auth.api_key.as_deref()) else { + return given; + }; + let Some(url) = given else { + return Some(format!( + "{CLOUD_CONNECT_URL}?apiKey={key}&sessionId={}", + self.id + )); + }; + if url.contains("apiKey=") { + return Some(url); + } + let Ok(mut parsed) = url::Url::parse(&url) else { + return Some(url); + }; + parsed.query_pairs_mut().append_pair("apiKey", key); + Some(parsed.to_string()) + } } /// Build the JSON body for session creation. @@ -130,94 +161,20 @@ fn build_create_body(options: &CreateSessionOptions) -> Value { } impl SteelClient { - pub async fn list_sessions( - &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, - ) -> Result, ApiError> { - let data = self - .request( - base_url, - mode, - reqwest::Method::GET, - "/sessions", - None, - auth, - ) - .await?; - Ok(extract_session_list(&data)) - } - - pub async fn get_session( - &self, - base_url: &str, - mode: ApiMode, - session_id: &str, - auth: &Auth, - ) -> Result { - let data = self - .request( - base_url, - mode, - reqwest::Method::GET, - &format!("/sessions/{session_id}"), - None, - auth, - ) - .await?; - extract_single_session(&data) - } - - pub async fn create_session( - &self, - base_url: &str, - mode: ApiMode, - options: &CreateSessionOptions, - auth: &Auth, - ) -> Result { + pub async fn create_session(&self, options: &CreateSessionOptions) -> anyhow::Result { let body = build_create_body(options); let data = self - .request( - base_url, - mode, - reqwest::Method::POST, - "/sessions", - Some(body), - auth, - ) + .request(reqwest::Method::POST, "/sessions", Some(body)) .await?; - extract_single_session(&data) - } - - pub async fn release_session( - &self, - base_url: &str, - mode: ApiMode, - session_id: &str, - auth: &Auth, - ) -> Result<(), ApiError> { - self.request( - base_url, - mode, - reqwest::Method::POST, - &format!("/sessions/{session_id}/release"), - None, - auth, - ) - .await?; - Ok(()) + Session::parse(data) } pub async fn solve_captcha( &self, - base_url: &str, - mode: ApiMode, session_id: &str, page_id: Option<&str>, url: Option<&str>, task_id: Option<&str>, - auth: &Auth, ) -> Result { let mut payload = json!({}); let obj = payload.as_object_mut().unwrap(); @@ -242,23 +199,17 @@ impl SteelClient { } self.request( - base_url, - mode, reqwest::Method::POST, &format!("/sessions/{session_id}/captchas/solve"), Some(payload), - auth, ) .await } pub async fn captcha_status( &self, - base_url: &str, - mode: ApiMode, session_id: &str, page_id: Option<&str>, - auth: &Auth, ) -> Result, ApiError> { let path = match page_id { Some(pid) if !pid.trim().is_empty() => { @@ -268,9 +219,7 @@ impl SteelClient { _ => format!("/sessions/{session_id}/captchas/status"), }; - let data = self - .request(base_url, mode, reqwest::Method::GET, &path, None, auth) - .await?; + let data = self.request(reqwest::Method::GET, &path, None).await?; if let Some(arr) = data.as_array() { return Ok(arr.iter().filter(|v| v.is_object()).cloned().collect()); @@ -286,49 +235,105 @@ impl SteelClient { mod tests { use super::*; - // --- extract_session_list --- + fn session(value: Value) -> Session { + Session::parse(value).unwrap() + } + + fn key(api_key: Option<&str>) -> Auth { + Auth { + api_key: api_key.map(str::to_string), + source: crate::config::auth::AuthSource::Env, + } + } #[test] - fn extract_list_from_array() { - let data = json!([{"id": "s1"}, {"id": "s2"}]); - let list = extract_session_list(&data); - assert_eq!(list.len(), 2); + fn parse_needs_an_id() { + assert!(Session::parse(json!({"status": "live"})).is_err()); + assert_eq!(session(json!({"id": "s1"})).id, "s1"); } #[test] - fn extract_list_from_nested() { - let data = json!({"sessions": [{"id": "s1"}]}); - let list = extract_session_list(&data); - assert_eq!(list.len(), 1); + fn released_and_failed_sessions_are_not_live() { + assert!(session(json!({"id": "s1", "status": "live"})).is_live()); + assert!(session(json!({"id": "s1", "status": "idle"})).is_live()); + assert!(session(json!({"id": "s1"})).is_live()); + assert!(!session(json!({"id": "s1", "status": "released"})).is_live()); + assert!(!session(json!({"id": "s1", "status": "failed"})).is_live()); } #[test] - fn extract_list_single_object() { - let data = json!({"id": "s1", "status": "live"}); - let list = extract_session_list(&data); - assert_eq!(list.len(), 1); + fn zero_timeouts_mean_no_limit() { + let s = session(json!({"id": "s1", "timeout": 300000, "inactivityTimeout": 120000})); + assert_eq!(s.timeout_ms(), Some(300000)); + assert_eq!(s.inactivity_timeout_ms(), Some(120000)); + let s = session(json!({"id": "s1", "timeout": 0, "inactivityTimeout": 0})); + assert_eq!(s.timeout_ms(), None); + assert_eq!(s.inactivity_timeout_ms(), None); + assert_eq!(session(json!({"id": "s1"})).timeout_ms(), None); } #[test] - fn extract_list_null() { - let list = extract_session_list(&Value::Null); - assert!(list.is_empty()); + fn created_at_is_read_as_epoch_ms() { + let s = session(json!({"id": "s1", "createdAt": "2025-01-15T10:30:00Z"})); + assert_eq!(s.created_at_ms(), Some(1_736_937_000_000)); + let s = session(json!({"id": "s1", "createdAt": "2025-01-15T10:30:00.123Z"})); + assert_eq!(s.created_at_ms(), Some(1_736_937_000_123)); + assert_eq!(session(json!({"id": "s1"})).created_at_ms(), None); } - // --- extract_single_session --- + #[test] + fn viewer_url_falls_back_to_the_cloud_dashboard() { + let given = session(json!({"id": "s1", "sessionViewerUrl": "https://custom.dev/view"})); + assert_eq!( + given.viewer_url(ApiMode::Local).as_deref(), + Some("https://custom.dev/view") + ); + let bare = session(json!({"id": "s1"})); + assert_eq!( + bare.viewer_url(ApiMode::Cloud).as_deref(), + Some("https://app.steel.dev/sessions/s1") + ); + assert_eq!(bare.viewer_url(ApiMode::Local), None); + } #[test] - fn extract_single_direct() { - let data = json!({"id": "s1"}); - let session = extract_single_session(&data).unwrap(); - assert_eq!(session["id"], "s1"); + fn cloud_connect_urls_carry_the_api_key_once() { + let auth = key(Some("sk-test")); + let fresh = + session(json!({"id": "s1", "websocketUrl": "wss://connect.steel.dev?sessionId=s1"})); + let url = fresh.connect_url(ApiMode::Cloud, &auth).unwrap(); + assert!(url.contains("apiKey=sk-test")); + assert!(url.contains("sessionId=s1")); + + let keyed = session( + json!({"id": "s1", "websocketUrl": "wss://connect.steel.dev?apiKey=old&sessionId=s1"}), + ); + let url = keyed.connect_url(ApiMode::Cloud, &auth).unwrap(); + assert_eq!(url.matches("apiKey").count(), 1); + assert!(url.contains("apiKey=old")); + + let local = + session(json!({"id": "s1", "websocketUrl": "ws://localhost:3000/ws?sessionId=s1"})); + assert!( + !local + .connect_url(ApiMode::Local, &auth) + .unwrap() + .contains("apiKey") + ); } #[test] - fn extract_single_nested() { - let data = json!({"session": {"id": "s1"}}); - let session = extract_single_session(&data).unwrap(); - assert_eq!(session["id"], "s1"); + fn cloud_connect_url_is_built_when_missing() { + let bare = session(json!({"id": "s1"})); + let url = bare + .connect_url(ApiMode::Cloud, &key(Some("sk-test"))) + .unwrap(); + assert_eq!(url, "wss://connect.steel.dev?apiKey=sk-test&sessionId=s1"); + assert_eq!(bare.connect_url(ApiMode::Cloud, &key(None)), None); + assert_eq!( + bare.connect_url(ApiMode::Local, &key(Some("sk-test"))), + None + ); } // --- build_create_body --- @@ -398,58 +403,15 @@ mod tests { // --- Integration tests with wiremock --- use crate::config::auth::AuthSource; - use wiremock::matchers::{header, method, path}; + use wiremock::matchers::{method, path}; use wiremock::{Mock, MockServer, ResponseTemplate}; - fn test_auth() -> Auth { - Auth { + fn client(server: &MockServer) -> SteelClient { + let auth = Auth { api_key: Some("test-key".into()), source: AuthSource::Env, - } - } - - #[tokio::test] - async fn list_sessions_success() { - let server = MockServer::start().await; - - Mock::given(method("GET")) - .and(path("/sessions")) - .and(header("Steel-Api-Key", "test-key")) - .respond_with( - ResponseTemplate::new(200).set_body_json(json!([{"id": "s1", "status": "live"}])), - ) - .mount(&server) - .await; - - let client = SteelClient::new().unwrap(); - let sessions = client - .list_sessions(&server.uri(), ApiMode::Local, &test_auth()) - .await - .unwrap(); - - assert_eq!(sessions.len(), 1); - assert_eq!(sessions[0]["id"], "s1"); - } - - #[tokio::test] - async fn get_session_success() { - let server = MockServer::start().await; - - Mock::given(method("GET")) - .and(path("/sessions/s1")) - .respond_with( - ResponseTemplate::new(200).set_body_json(json!({"id": "s1", "status": "live"})), - ) - .mount(&server) - .await; - - let client = SteelClient::new().unwrap(); - let session = client - .get_session(&server.uri(), ApiMode::Local, "s1", &test_auth()) - .await - .unwrap(); - - assert_eq!(session["id"], "s1"); + }; + SteelClient::with(server.uri(), ApiMode::Local, auth).unwrap() } #[tokio::test] @@ -465,35 +427,12 @@ mod tests { .mount(&server) .await; - let client = SteelClient::new().unwrap(); - let session = client - .create_session( - &server.uri(), - ApiMode::Local, - &CreateSessionOptions::default(), - &test_auth(), - ) + let session = client(&server) + .create_session(&CreateSessionOptions::default()) .await .unwrap(); - assert_eq!(session["id"], "new-sess"); - } - - #[tokio::test] - async fn release_session_success() { - let server = MockServer::start().await; - - Mock::given(method("POST")) - .and(path("/sessions/s1/release")) - .respond_with(ResponseTemplate::new(200).set_body_json(json!({}))) - .mount(&server) - .await; - - let client = SteelClient::new().unwrap(); - client - .release_session(&server.uri(), ApiMode::Local, "s1", &test_auth()) - .await - .unwrap(); + assert_eq!(session.id, "new-sess"); } #[tokio::test] @@ -501,18 +440,14 @@ mod tests { let server = MockServer::start().await; Mock::given(method("GET")) - .and(path("/sessions")) + .and(path("/sessions/s1")) .respond_with( ResponseTemplate::new(403).set_body_json(json!({"message": "Invalid API key"})), ) .mount(&server) .await; - let client = SteelClient::new().unwrap(); - let err = client - .list_sessions(&server.uri(), ApiMode::Local, &test_auth()) - .await - .unwrap_err(); + let err = client(&server).cli_get_session("s1").await.unwrap_err(); let msg = err.to_string(); assert!(msg.contains("403"), "expected 403 in: {msg}"); @@ -532,11 +467,7 @@ mod tests { .mount(&server) .await; - let client = SteelClient::new().unwrap(); - let err = client - .get_session(&server.uri(), ApiMode::Local, "gone", &test_auth()) - .await - .unwrap_err(); + let err = client(&server).cli_get_session("gone").await.unwrap_err(); assert!(err.is_not_found()); } diff --git a/src/api/top_level.rs b/src/api/top_level.rs index e11de37..848aee5 100644 --- a/src/api/top_level.rs +++ b/src/api/top_level.rs @@ -4,8 +4,6 @@ use serde_json::{Value, json}; use crate::api::client::{ApiError, SteelClient}; -use crate::config::auth::Auth; -use crate::config::settings::ApiMode; /// Supported scrape output formats. pub const SCRAPE_FORMATS: &[&str] = &["html", "readability", "cleaned_html", "markdown"]; @@ -78,123 +76,65 @@ pub fn get_hosted_url(data: &Value) -> Option { .map(|s| s.trim().to_string()) } +pub struct Target<'a> { + pub url: &'a str, + pub delay: Option, + pub use_proxy: bool, + pub region: Option<&'a str>, +} + +impl Target<'_> { + fn body(&self) -> Value { + let mut body = json!({"url": self.url}); + if let Some(delay) = self.delay { + body["delay"] = json!(delay); + } + if self.use_proxy { + body["useProxy"] = json!(true); + } + if let Some(region) = self.region { + body["region"] = json!(region); + } + body + } +} + impl SteelClient { pub async fn scrape( &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, - url: &str, + target: &Target<'_>, formats: &[String], - delay: Option, pdf: bool, screenshot: bool, - use_proxy: bool, - region: Option<&str>, ) -> Result { - let mut body = json!({ - "url": url, - "format": formats, - }); - let obj = body.as_object_mut().unwrap(); - - if let Some(d) = delay { - obj.insert("delay".into(), json!(d)); - } + let mut body = target.body(); + body["format"] = json!(formats); if pdf { - obj.insert("pdf".into(), json!(true)); + body["pdf"] = json!(true); } if screenshot { - obj.insert("screenshot".into(), json!(true)); + body["screenshot"] = json!(true); } - if use_proxy { - obj.insert("useProxy".into(), json!(true)); - } - if let Some(r) = region { - obj.insert("region".into(), json!(r)); - } - - self.request( - base_url, - mode, - reqwest::Method::POST, - "/scrape", - Some(body), - auth, - ) - .await + self.request(reqwest::Method::POST, "/scrape", Some(body)) + .await } pub async fn screenshot( &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, - url: &str, - delay: Option, + target: &Target<'_>, full_page: bool, - use_proxy: bool, - region: Option<&str>, ) -> Result { - let mut body = json!({"url": url}); - let obj = body.as_object_mut().unwrap(); - - if let Some(d) = delay { - obj.insert("delay".into(), json!(d)); - } + let mut body = target.body(); if full_page { - obj.insert("fullPage".into(), json!(true)); + body["fullPage"] = json!(true); } - if use_proxy { - obj.insert("useProxy".into(), json!(true)); - } - if let Some(r) = region { - obj.insert("region".into(), json!(r)); - } - - self.request( - base_url, - mode, - reqwest::Method::POST, - "/screenshot", - Some(body), - auth, - ) - .await + self.request(reqwest::Method::POST, "/screenshot", Some(body)) + .await } - pub async fn pdf( - &self, - base_url: &str, - mode: ApiMode, - auth: &Auth, - url: &str, - delay: Option, - use_proxy: bool, - region: Option<&str>, - ) -> Result { - let mut body = json!({"url": url}); - let obj = body.as_object_mut().unwrap(); - - if let Some(d) = delay { - obj.insert("delay".into(), json!(d)); - } - if use_proxy { - obj.insert("useProxy".into(), json!(true)); - } - if let Some(r) = region { - obj.insert("region".into(), json!(r)); - } - - self.request( - base_url, - mode, - reqwest::Method::POST, - "/pdf", - Some(body), - auth, - ) - .await + pub async fn pdf(&self, target: &Target<'_>) -> Result { + self.request(reqwest::Method::POST, "/pdf", Some(target.body())) + .await } } @@ -310,42 +250,67 @@ mod tests { // --- Integration tests with wiremock --- - use crate::config::auth::AuthSource; - use wiremock::matchers::{method, path}; + use crate::config::auth::{Auth, AuthSource}; + use crate::config::settings::ApiMode; + use wiremock::matchers::{body_json, method, path}; use wiremock::{Mock, MockServer, ResponseTemplate}; - fn test_auth() -> Auth { - Auth { + fn client(server: &MockServer) -> SteelClient { + let auth = Auth { api_key: Some("test-key".into()), source: AuthSource::Env, + }; + SteelClient::with(server.uri(), ApiMode::Local, auth).unwrap() + } + + const fn target(url: &str) -> Target<'_> { + Target { + url, + delay: None, + use_proxy: false, + region: None, } } + #[test] + fn target_body_includes_only_set_options() { + assert_eq!( + target("https://example.com").body(), + json!({"url": "https://example.com"}) + ); + let full = Target { + url: "https://example.com", + delay: Some(500), + use_proxy: true, + region: Some("lax"), + }; + assert_eq!( + full.body(), + json!({"url": "https://example.com", "delay": 500, "useProxy": true, "region": "lax"}) + ); + } + #[tokio::test] async fn scrape_success() { let server = MockServer::start().await; Mock::given(method("POST")) .and(path("/scrape")) + .and(body_json( + json!({"url": "https://example.com", "format": ["markdown"]}), + )) .respond_with(ResponseTemplate::new(200).set_body_json(json!({ "content": {"markdown": "# Hello"} }))) .mount(&server) .await; - let client = SteelClient::new().unwrap(); - let data = client + let data = client(&server) .scrape( - &server.uri(), - ApiMode::Local, - &test_auth(), - "https://example.com", + &target("https://example.com"), &["markdown".into()], - None, - false, false, false, - None, ) .await .unwrap(); @@ -360,6 +325,9 @@ mod tests { Mock::given(method("POST")) .and(path("/screenshot")) + .and(body_json( + json!({"url": "https://example.com", "fullPage": true}), + )) .respond_with( ResponseTemplate::new(200) .set_body_json(json!({"url": "https://cdn.steel.dev/shot.png"})), @@ -367,18 +335,8 @@ mod tests { .mount(&server) .await; - let client = SteelClient::new().unwrap(); - let data = client - .screenshot( - &server.uri(), - ApiMode::Local, - &test_auth(), - "https://example.com", - None, - false, - false, - None, - ) + let data = client(&server) + .screenshot(&target("https://example.com"), true) .await .unwrap(); @@ -401,17 +359,8 @@ mod tests { .mount(&server) .await; - let client = SteelClient::new().unwrap(); - let data = client - .pdf( - &server.uri(), - ApiMode::Local, - &test_auth(), - "https://example.com", - None, - false, - None, - ) + let data = client(&server) + .pdf(&target("https://example.com")) .await .unwrap(); diff --git a/src/browser/lifecycle.rs b/src/browser/lifecycle.rs deleted file mode 100644 index 3ba69bb..0000000 --- a/src/browser/lifecycle.rs +++ /dev/null @@ -1,666 +0,0 @@ -//! Browser session lifecycle utilities. -//! Provides helpers for parsing API responses, CAPTCHA operations, and URL handling. - -use serde_json::Value; - -use crate::api::client::SteelClient; -use crate::config::auth::Auth; -use crate::config::settings::ApiMode; - -/// Summary of a browser session, matching TS `BrowserSessionSummary`. -#[derive(Debug)] -pub struct SessionSummary { - pub id: String, - pub mode: ApiMode, - pub name: Option, - pub live: bool, - pub status: Option, - pub connect_url: Option, - pub viewer_url: Option, - pub profile_id: Option, -} - -/// Result of solving a CAPTCHA. -pub struct CaptchaSolveResult { - pub mode: ApiMode, - pub session_id: String, - pub success: bool, - pub message: Option, - pub raw: Value, -} - -/// Result of checking CAPTCHA status. -pub struct CaptchaStatusResult { - pub mode: ApiMode, - pub session_id: String, - pub status: String, - pub types: Vec, - pub raw: Value, -} - -const CLOSED_SESSION_STATUSES: &[&str] = &[ - "closed", - "completed", - "ended", - "failed", - "released", - "stopped", - "terminated", -]; - -/// Extract session ID from API response. Matches TS `getSessionId()`. -pub fn get_session_id(session: &Value) -> Option { - session - .get("id") - .or_else(|| session.get("sessionId")) - .and_then(|v| v.as_str()) - .filter(|s| !s.trim().is_empty()) - .map(|s| s.trim().to_string()) -} - -/// Check if session is live. Matches TS `isSessionLive()`. -pub fn is_session_live(session: &Value) -> bool { - let live_keys = ["isLive", "live", "active"]; - for key in &live_keys { - if let Some(v) = session.get(key) - && let Some(b) = v.as_bool() - { - return b; - } - } - - if let Some(ended) = session.get("endedAt") - && let Some(s) = ended.as_str() - && !s.trim().is_empty() - { - return false; - } - - if let Some(status) = get_session_status(session) { - return !CLOSED_SESSION_STATUSES.contains(&status.to_lowercase().as_str()); - } - - true -} - -fn get_session_status(session: &Value) -> Option { - session - .get("status") - .and_then(|v| v.as_str()) - .filter(|s| !s.trim().is_empty()) - .map(|s| s.trim().to_string()) -} - -/// Extract session timeout (milliseconds) from API response. -/// The API may return it as `timeout`, `sessionTimeout`, or `timeoutMs`. -/// -/// A returned value of `0` is treated as "no expiry" (matches self-hosted -/// servers that signal "session never expires" with `timeout: 0`). Callers -/// that need an actual deadline must treat `None` and `Some(0)` identically. -pub fn get_session_timeout(session: &Value) -> Option { - let keys = ["timeout", "sessionTimeout", "timeoutMs"]; - for key in &keys { - if let Some(v) = session.get(key) { - if let Some(n) = v.as_u64() { - return Some(n).filter(|&n| n > 0); - } - // Handle string numbers - if let Some(s) = v.as_str() - && let Ok(n) = s.trim().parse::() - { - return Some(n).filter(|&n| n > 0); - } - } - } - None -} - -pub fn get_session_inactivity_timeout(session: &Value) -> Option { - let keys = [ - "inactivityTimeout", - "inactivity_timeout", - "inactivityTimeoutMs", - ]; - for key in &keys { - if let Some(v) = session.get(key) { - if let Some(n) = v.as_u64() { - return Some(n).filter(|&n| n > 0); - } - if let Some(s) = v.as_str() - && let Ok(n) = s.trim().parse::() - { - return Some(n).filter(|&n| n > 0); - } - } - } - None -} - -/// Extract session creation timestamp as epoch milliseconds from API response. -/// Looks for `createdAt` or `created_at` as ISO 8601 / RFC 3339 string. -pub fn get_session_created_at_ms(session: &Value) -> Option { - let keys = ["createdAt", "created_at"]; - for key in &keys { - if let Some(v) = session.get(key) - && let Some(s) = v.as_str() - { - let trimmed = s.trim(); - if !trimmed.is_empty() - && let Ok(ts) = jiff::Timestamp::strptime("%Y-%m-%dT%H:%M:%S%.fZ", trimmed) - .or_else(|_| trimmed.parse::()) - { - return Some(ts.as_millisecond() as u64); - } - } - } - None -} - -pub fn get_connect_url(session: &Value) -> Option { - let keys = [ - "websocketUrl", - "wsUrl", - "connectUrl", - "cdpUrl", - "browserWSEndpoint", - "wsEndpoint", - ]; - for key in &keys { - if let Some(v) = session.get(key) - && let Some(s) = v.as_str() - { - let trimmed = s.trim(); - if !trimmed.is_empty() { - return Some(trimmed.to_string()); - } - } - } - None -} - -pub fn get_viewer_url(session: &Value, mode: ApiMode, session_id: &str) -> Option { - let keys = ["sessionViewerUrl", "viewerUrl", "liveViewUrl"]; - for key in &keys { - if let Some(v) = session.get(key) - && let Some(s) = v.as_str() - { - let trimmed = s.trim(); - if !trimmed.is_empty() { - return Some(trimmed.to_string()); - } - } - } - - if mode == ApiMode::Cloud { - return Some(format!("https://app.steel.dev/sessions/{session_id}")); - } - - None -} - -/// Build session summary from API response. Matches TS `toSessionSummary()`. -pub fn to_session_summary( - session: &Value, - mode: ApiMode, - name: Option<&str>, - auth: &Auth, -) -> anyhow::Result { - let session_id = get_session_id(session) - .ok_or_else(|| anyhow::anyhow!("Session response did not contain an id."))?; - - let mut connect_url = get_connect_url(session); - - // Fallback: build connect URL for cloud mode - if connect_url.is_none() - && mode == ApiMode::Cloud - && let Some(ref api_key) = auth.api_key - { - connect_url = Some(format!( - "wss://connect.steel.dev?apiKey={api_key}&sessionId={session_id}" - )); - } - - // Inject apiKey into connect URL for cloud mode - if let Some(ref url) = connect_url - && mode == ApiMode::Cloud - && let Some(ref api_key) = auth.api_key - && !url.contains("apiKey=") - && let Ok(mut parsed) = url::Url::parse(url) - { - parsed.query_pairs_mut().append_pair("apiKey", api_key); - connect_url = Some(parsed.to_string()); - } - - Ok(SessionSummary { - id: session_id.clone(), - mode, - name: name.map(|s| s.to_string()), - live: is_session_live(session), - status: get_session_status(session), - connect_url, - viewer_url: get_viewer_url(session, mode, &session_id), - profile_id: session - .get("profileId") - .and_then(|v| v.as_str()) - .map(|s| s.to_string()), - }) -} - -/// Solve a CAPTCHA for a browser session. -pub async fn solve_captcha( - client: &SteelClient, - base_url: &str, - mode: ApiMode, - auth: &Auth, - session_id: &str, - page_id: Option<&str>, - url: Option<&str>, - task_id: Option<&str>, -) -> anyhow::Result { - let raw = client - .solve_captcha(base_url, mode, session_id, page_id, url, task_id, auth) - .await - .map_err(|e| anyhow::anyhow!("{e}"))?; - - let success = raw - .get("success") - .and_then(|v| v.as_bool()) - .unwrap_or(false); - let message = raw - .get("message") - .and_then(|v| v.as_str()) - .map(|s| s.to_string()); - - Ok(CaptchaSolveResult { - mode, - session_id: session_id.to_string(), - success, - message, - raw, - }) -} - -/// Get CAPTCHA status for a browser session. -pub async fn captcha_status( - client: &SteelClient, - base_url: &str, - mode: ApiMode, - auth: &Auth, - session_id: &str, - page_id: Option<&str>, - wait: bool, - timeout_ms: Option, - interval_ms: Option, -) -> anyhow::Result { - let timeout = timeout_ms.unwrap_or(60_000); - let interval = interval_ms.unwrap_or(1_000); - - let start = std::time::Instant::now(); - - loop { - let pages = client - .captcha_status(base_url, mode, session_id, page_id, auth) - .await - .map_err(|e| anyhow::anyhow!("{e}"))?; - - let (status, types) = normalize_captcha_status(&pages); - - if !wait || is_terminal_captcha_status(&status) { - return Ok(CaptchaStatusResult { - mode, - session_id: session_id.to_string(), - status, - types, - raw: serde_json::json!({"pages": pages}), - }); - } - - if start.elapsed().as_millis() as u64 >= timeout { - anyhow::bail!( - "CAPTCHA status polling timed out after {timeout}ms. Last status: {status}" - ); - } - - tokio::time::sleep(std::time::Duration::from_millis(interval)).await; - } -} - -const KNOWN_CAPTCHA_TYPES: &[&str] = &["recaptchaV2", "recaptchaV3", "turnstile", "image_to_text"]; - -fn normalize_captcha_status(pages: &[Value]) -> (String, Vec) { - if pages.is_empty() { - return ("none".to_string(), vec![]); - } - - let mut all_tasks: Vec<&Value> = Vec::new(); - let mut any_solving = false; - - for page in pages { - if page.get("isSolvingCaptcha").and_then(|v| v.as_bool()) == Some(true) { - any_solving = true; - } - - if let Some(tasks) = page.get("tasks").and_then(|v| v.as_array()) { - for task in tasks { - if task.is_object() { - all_tasks.push(task); - } - } - } - } - - if all_tasks.is_empty() { - let status = if any_solving { "solving" } else { "none" }; - return (status.to_string(), vec![]); - } - - let mut solving_types: Vec = Vec::new(); - let mut failed_types: Vec = Vec::new(); - let mut has_solving = false; - let mut has_failed = false; - let mut has_solved = false; - - for task in &all_tasks { - let task_status = task.get("status").and_then(|v| v.as_str()).unwrap_or(""); - let task_type = task.get("type").and_then(|v| v.as_str()).unwrap_or(""); - let is_known = KNOWN_CAPTCHA_TYPES.contains(&task_type); - - match task_status { - "solving" | "detected" | "validating" => { - has_solving = true; - if is_known && !solving_types.contains(&task_type.to_string()) { - solving_types.push(task_type.to_string()); - } - } - "failed_to_solve" | "failed_to_detect" | "validation_failed" => { - has_failed = true; - if is_known && !failed_types.contains(&task_type.to_string()) { - failed_types.push(task_type.to_string()); - } - } - "solved" => { - has_solved = true; - } - _ => {} - } - } - - if has_solving || any_solving { - return ("solving".to_string(), solving_types); - } - - if has_failed { - return ("failed".to_string(), failed_types); - } - - if has_solved { - return ("solved".to_string(), vec![]); - } - - ("none".to_string(), vec![]) -} - -fn is_terminal_captcha_status(status: &str) -> bool { - matches!(status, "solved" | "failed" | "none") -} - -/// Sanitize connect URL for display (mask API key). -pub fn sanitize_connect_url(url: &str) -> String { - if let Ok(mut parsed) = url::Url::parse(url) { - let pairs: Vec<(String, String)> = parsed - .query_pairs() - .map(|(k, v)| { - if k == "apiKey" && v.len() > 7 { - (k.to_string(), format!("{}...", &v[..7])) - } else { - (k.to_string(), v.to_string()) - } - }) - .collect(); - - if !pairs.is_empty() { - parsed.query_pairs_mut().clear(); - for (k, v) in &pairs { - parsed.query_pairs_mut().append_pair(k, v); - } - } - - parsed.to_string() - } else { - url.to_string() - } -} - -#[cfg(test)] -mod tests { - use super::*; - use serde_json::json; - - #[test] - fn get_session_id_from_id() { - let s = json!({"id": "s1"}); - assert_eq!(get_session_id(&s).as_deref(), Some("s1")); - } - - #[test] - fn get_session_id_from_session_id() { - let s = json!({"sessionId": "s2"}); - assert_eq!(get_session_id(&s).as_deref(), Some("s2")); - } - - #[test] - fn get_session_id_empty() { - let s = json!({"id": " "}); - assert_eq!(get_session_id(&s), None); - } - - #[test] - fn session_live_by_flag() { - assert!(is_session_live(&json!({"isLive": true}))); - assert!(!is_session_live(&json!({"isLive": false}))); - } - - #[test] - fn session_live_by_status() { - assert!(is_session_live(&json!({"status": "running"}))); - assert!(!is_session_live(&json!({"status": "released"}))); - assert!(!is_session_live(&json!({"status": "terminated"}))); - } - - #[test] - fn session_live_ended_at() { - assert!(!is_session_live(&json!({"endedAt": "2025-01-01"}))); - } - - #[test] - fn session_live_default() { - assert!(is_session_live(&json!({"id": "s1"}))); - } - - #[test] - fn viewer_url_cloud_fallback() { - let url = get_viewer_url(&json!({}), ApiMode::Cloud, "s1"); - assert_eq!(url.as_deref(), Some("https://app.steel.dev/sessions/s1")); - } - - #[test] - fn viewer_url_local_none() { - let url = get_viewer_url(&json!({}), ApiMode::Local, "s1"); - assert_eq!(url, None); - } - - #[test] - fn viewer_url_from_response() { - let url = get_viewer_url( - &json!({"sessionViewerUrl": "https://custom.dev/view"}), - ApiMode::Local, - "s1", - ); - assert_eq!(url.as_deref(), Some("https://custom.dev/view")); - } - - #[test] - fn connect_url_from_response() { - let url = get_connect_url(&json!({"websocketUrl": "wss://foo.dev/ws"})); - assert_eq!(url.as_deref(), Some("wss://foo.dev/ws")); - } - - #[test] - fn sanitize_masks_api_key() { - let url = "wss://connect.steel.dev?apiKey=sk-12345678&sessionId=s1"; - let sanitized = sanitize_connect_url(url); - assert!(sanitized.contains("sk-1234...")); - assert!(sanitized.contains("sessionId=s1")); - } - - #[test] - fn session_timeout_from_response() { - assert_eq!( - get_session_timeout(&json!({"timeout": 300000})), - Some(300000) - ); - assert_eq!( - get_session_timeout(&json!({"sessionTimeout": 60000})), - Some(60000) - ); - assert_eq!( - get_session_timeout(&json!({"timeoutMs": 120000})), - Some(120000) - ); - } - - #[test] - fn session_timeout_string_number() { - assert_eq!( - get_session_timeout(&json!({"timeout": "300000"})), - Some(300000) - ); - } - - #[test] - fn session_timeout_missing() { - assert_eq!(get_session_timeout(&json!({"id": "s1"})), None); - } - - #[test] - fn session_timeout_zero_collapses_to_none() { - // Self-hosted server returns `timeout: 0` to mean "no expiry". - // Treating it as None keeps the remaining-time math from reporting - // the session as already expired. - assert_eq!(get_session_timeout(&json!({"timeout": 0})), None); - assert_eq!(get_session_timeout(&json!({"sessionTimeout": 0})), None); - assert_eq!(get_session_timeout(&json!({"timeoutMs": 0})), None); - assert_eq!(get_session_timeout(&json!({"timeout": "0"})), None); - // A field set to 0 must not mask another field with a real value - // in any single response, but per the existing precedence rules - // the first key wins. Document the precedence stays unchanged. - assert_eq!( - get_session_timeout(&json!({"timeout": 0, "sessionTimeout": 60000})), - None, - "precedence keeps the 0 → None semantics, doesn't fall through" - ); - } - - #[test] - fn inactivity_timeout_zero_collapses_to_none() { - assert_eq!( - get_session_inactivity_timeout(&json!({"inactivityTimeout": 0})), - None - ); - assert_eq!( - get_session_inactivity_timeout(&json!({"inactivityTimeout": "0"})), - None - ); - assert_eq!( - get_session_inactivity_timeout(&json!({"inactivityTimeoutMs": 0})), - None - ); - } - - #[test] - fn session_timeout_preserves_nonzero() { - // Sanity: the filter only affects the zero case. - assert_eq!(get_session_timeout(&json!({"timeout": 1})), Some(1)); - assert_eq!( - get_session_timeout(&json!({"timeout": u64::MAX})), - Some(u64::MAX) - ); - } - - #[test] - fn inactivity_timeout_from_response() { - assert_eq!( - get_session_inactivity_timeout(&json!({"inactivityTimeout": 120000})), - Some(120000) - ); - assert_eq!( - get_session_inactivity_timeout(&json!({"inactivityTimeout": "120000"})), - Some(120000) - ); - assert_eq!(get_session_inactivity_timeout(&json!({"id": "s1"})), None); - } - - #[test] - fn session_created_at_iso() { - let s = json!({"createdAt": "2025-01-15T10:30:00Z"}); - let ms = get_session_created_at_ms(&s).unwrap(); - // 2025-01-15T10:30:00Z in epoch ms - assert!(ms > 1_700_000_000_000); - assert!(ms < 1_800_000_000_000); - } - - #[test] - fn session_created_at_with_fractional() { - let s = json!({"createdAt": "2025-01-15T10:30:00.123Z"}); - let ms = get_session_created_at_ms(&s).unwrap(); - assert!(ms > 1_700_000_000_000); - } - - #[test] - fn session_created_at_missing() { - assert_eq!(get_session_created_at_ms(&json!({"id": "s1"})), None); - } - - #[test] - fn session_created_at_snake_case() { - let s = json!({"created_at": "2025-01-15T10:30:00Z"}); - assert!(get_session_created_at_ms(&s).is_some()); - } - - #[test] - fn normalize_captcha_none() { - let (status, types) = normalize_captcha_status(&[]); - assert_eq!(status, "none"); - assert!(types.is_empty()); - } - - #[test] - fn normalize_captcha_solving() { - let pages = vec![json!({ - "isSolvingCaptcha": true, - "tasks": [{"status": "solving", "type": "turnstile"}] - })]; - let (status, types) = normalize_captcha_status(&pages); - assert_eq!(status, "solving"); - assert_eq!(types, vec!["turnstile"]); - } - - #[test] - fn normalize_captcha_solved() { - let pages = vec![json!({ - "tasks": [{"status": "solved", "type": "recaptchaV2"}] - })]; - let (status, _types) = normalize_captcha_status(&pages); - assert_eq!(status, "solved"); - } - - #[test] - fn normalize_captcha_failed() { - let pages = vec![json!({ - "tasks": [{"status": "failed_to_solve", "type": "turnstile"}] - })]; - let (status, types) = normalize_captcha_status(&pages); - assert_eq!(status, "failed"); - assert_eq!(types, vec!["turnstile"]); - } -} diff --git a/src/browser/mod.rs b/src/browser/mod.rs index 87fc9ce..6a368c6 100644 --- a/src/browser/mod.rs +++ b/src/browser/mod.rs @@ -1,4 +1,3 @@ -pub mod lifecycle; pub mod profile_porter; pub mod profile_store; pub mod state; diff --git a/src/browser/profile_porter.rs b/src/browser/profile_porter.rs index 3f83470..1ceeca4 100644 --- a/src/browser/profile_porter.rs +++ b/src/browser/profile_porter.rs @@ -53,7 +53,6 @@ impl BrowserId { pub const fn descriptor(self) -> BrowserDescriptor { match self { Self::Chrome => BrowserDescriptor { - id: self, profile_base_dirs: PlatformStr { darwin: Some("Google/Chrome"), win32: Some("Google/Chrome/User Data"), @@ -71,7 +70,6 @@ impl BrowserId { }, }, Self::Edge => BrowserDescriptor { - id: self, profile_base_dirs: PlatformStr { darwin: Some("Microsoft Edge"), win32: Some("Microsoft/Edge/User Data"), @@ -89,7 +87,6 @@ impl BrowserId { }, }, Self::Brave => BrowserDescriptor { - id: self, profile_base_dirs: PlatformStr { darwin: Some("BraveSoftware/Brave-Browser"), win32: Some("BraveSoftware/Brave-Browser/User Data"), @@ -107,7 +104,6 @@ impl BrowserId { }, }, Self::Arc => BrowserDescriptor { - id: self, profile_base_dirs: PlatformStr { darwin: Some("Arc/User Data"), win32: None, @@ -125,7 +121,6 @@ impl BrowserId { }, }, Self::Opera => BrowserDescriptor { - id: self, profile_base_dirs: PlatformStr { darwin: Some("com.operasoftware.Opera"), win32: Some("Opera Software/Opera Stable"), @@ -144,7 +139,6 @@ impl BrowserId { }, }, Self::Vivaldi => BrowserDescriptor { - id: self, profile_base_dirs: PlatformStr { darwin: Some("Vivaldi"), win32: Some("Vivaldi/User Data"), @@ -195,7 +189,6 @@ pub struct PlatformStrs { } pub struct BrowserDescriptor { - pub id: BrowserId, pub profile_base_dirs: PlatformStr, pub process_names: PlatformStrs, pub keychain_service: PlatformStr, @@ -279,7 +272,6 @@ pub struct BrowserProfile { pub dir_name: String, pub display_name: String, pub email: Option, - pub browser: BrowserId, } pub fn find_browser_profiles(browser: BrowserId) -> Vec { @@ -310,7 +302,6 @@ pub fn find_browser_profiles(browser: BrowserId) -> Vec { dir_name, display_name, email, - browser, }); } profiles.sort_by(|a, b| a.dir_name.cmp(&b.dir_name)); @@ -1212,14 +1203,6 @@ mod tests { assert!("firefox".parse::().is_err()); } - #[test] - fn all_browsers_have_descriptors() { - for id in BrowserId::all() { - let desc = id.descriptor(); - assert_eq!(desc.id, *id); - } - } - #[test] fn minimal_preferences_has_clean_exit() { let data = minimal_preferences(); diff --git a/src/commands/browser/action.rs b/src/commands/browser/action.rs index 72b798f..e36d6ea 100644 --- a/src/commands/browser/action.rs +++ b/src/commands/browser/action.rs @@ -7,14 +7,12 @@ use serde_json::{Map, Value, json}; use crate::api::client::{ApiError, SteelClient}; use crate::api::commands::{Reply, Session}; -use crate::browser::lifecycle::{get_viewer_url, is_session_live}; +use crate::api::session::Session as RemoteSession; use crate::browser::state::{self, BrowserSession, SnapshotInfo}; -use crate::config::auth::Auth; use crate::config::settings::ApiMode; -use crate::util::api; use crate::util::output::{self, SilentExit}; -use super::{DEFAULT_SESSION, start}; +use super::{DEFAULT_SESSION, Target, start}; #[derive(Parser)] pub struct SelectorArg { @@ -124,9 +122,6 @@ pub enum ActionCommand { Status(ExecutionArgs), /// Stop a command that is still running Cancel(ExecutionArgs), - /// Close the browser session - #[command(aliases = ["quit", "exit"])] - Close, #[command(hide = true)] Cookies(RemovedArgs), @@ -184,7 +179,6 @@ impl ActionCommand { Self::Dialog(_) => "dialog", Self::Status(_) => "status", Self::Cancel(_) => "cancel", - Self::Close => "close", Self::Cookies(_) => "cookies", Self::Storage(_) => "storage", Self::Set(_) => "set", @@ -524,7 +518,7 @@ fn parse_tab(value: &str) -> Result { } } -pub(crate) enum Output { +pub enum Output { None, Snapshot, Value, @@ -538,7 +532,7 @@ pub(crate) enum Output { Dialog, } -pub(crate) struct Step { +pub struct Step { pub commands: Vec, pub output: Output, pub uploads: Vec<(PathBuf, String)>, @@ -553,7 +547,7 @@ impl Step { } } - pub(crate) fn batchable(&self) -> bool { + pub fn batchable(&self) -> bool { !matches!(self.output, Output::Switch(_) | Output::TabOpened) && !self .commands @@ -562,10 +556,9 @@ impl Step { } } -pub(crate) enum Plan { +pub enum Plan { Run(Step), Control(Value, Output), - Close, } struct Cmd(Map); @@ -633,7 +626,7 @@ fn image_format(output: &str) -> Option<&'static str> { } impl ActionCommand { - pub(crate) fn plan(self) -> Result { + pub fn plan(self) -> Result { use Output as O; match self { Self::Navigate(a) => { @@ -877,7 +870,6 @@ impl ActionCommand { cmd("cancel").set("execution", a.execution).done(), O::Execution, )), - Self::Close => Ok(Plan::Close), Self::Cookies(_) => Err(removed( "cookies", "Read cookies with `steel browser eval \"document.cookie\"` (HttpOnly cookies are not visible). Keep logins across sessions with `steel browser start --profile `.", @@ -1022,7 +1014,7 @@ fn session_flag(name: &str) -> String { } } -pub(crate) fn fail(code: &str, message: &str, data: Option<&Value>) -> anyhow::Error { +pub fn fail(code: &str, message: &str, data: Option<&Value>) -> anyhow::Error { if output::is_json() { let mut obj = json!({"success": false, "error": message, "error_code": code}); if let Some(data) = data { @@ -1196,69 +1188,48 @@ fn restarted(body: &Value) -> bool { }) } -pub(crate) struct Browser { +pub struct Browser { client: SteelClient, - auth: Auth, pub state: BrowserSession, } -async fn attach( - client: &SteelClient, - mode: ApiMode, - base_url: &str, - auth: &Auth, - id: &str, -) -> Result { - let session = client.get_session(base_url, mode, id, auth).await?; - if !is_session_live(&session) { +async fn attach(client: &SteelClient, id: &str) -> Result { + let remote = RemoteSession::parse(client.cli_get_session(id).await?)?; + if !remote.is_live() { bail!("Session {id} has ended. Start a new one with `steel browser start`."); } - let mut attached = - BrowserSession::new(id.to_string(), id.to_string(), mode, base_url.to_string()); - attached.live_url = get_viewer_url(&session, mode, id); + let mut attached = BrowserSession::new( + id.to_string(), + id.to_string(), + client.mode(), + client.base_url().to_string(), + ); + attached.live_url = remote.viewer_url(client.mode()); state::save(&attached)?; Ok(attached) } impl Browser { - pub(crate) async fn open(name: Option<&str>, start_default: bool) -> Result { - let (mode, base_url, auth) = api::resolve_with_auth(); + pub async fn open(name: Option<&str>, start_default: bool) -> Result { let client = SteelClient::new()?; - let state = match name { - Some(name) => match state::load(name)? { - Some(found) => found, - None if state::looks_like_session_id(name) => { - attach(&client, mode, &base_url, &auth, name).await? - } - None => bail!( - "Session \"{name}\" is not running. It may have expired. Run `steel browser start --session {name}` to start a new one." - ), - }, - None => match state::load(DEFAULT_SESSION)? { - Some(found) => found, - None if start_default => { - eprintln!("warning: session \"default\" is not running; starting a new one"); - let options = start::default_options(); - start::create(&client, mode, &base_url, &auth, DEFAULT_SESSION, &options) - .await? - .0 - } - None => bail!("No active browser session. Start one with `steel browser start`."), - }, + let state = match super::target(name)? { + Some(Target::Saved(found)) => *found, + Some(Target::Id(id)) => attach(&client, &id).await?, + None if name.is_none() && start_default => { + eprintln!("warning: session \"default\" is not running; starting a new one"); + start::create(&client, DEFAULT_SESSION, &start::default_options()) + .await? + .0 + } + None => return Err(super::not_running(name)), }; - Ok(Self { - client, - auth, - state, - }) + let client = client.at(&state.api_url, state.mode); + Ok(Self { client, state }) } fn session(&self) -> Session<'_> { Session { client: &self.client, - base_url: &self.state.api_url, - mode: self.state.mode, - auth: &self.auth, id: &self.state.id, } } @@ -1409,7 +1380,7 @@ impl Browser { } } - pub(crate) async fn execute(&mut self, steps: Vec, diff: bool) -> Result<()> { + pub async fn execute(&mut self, steps: Vec, diff: bool) -> Result<()> { for (path, name) in steps.iter().flat_map(|step| step.uploads.iter()) { let bytes = std::fs::read(path) .with_context(|| format!("Could not read {}", path.display()))?; @@ -1575,7 +1546,7 @@ impl Browser { }) } - pub(crate) async fn control(&mut self, mut command: Value, output: Output) -> Result<()> { + pub async fn control(&mut self, mut command: Value, output: Output) -> Result<()> { if command["command"] == "dialog" && command.get("dialog").is_none() { let Some(id) = self.state.dialog.clone() else { bail!( @@ -1640,7 +1611,6 @@ pub async fn run(action: ActionCommand, session: Option<&str>, diff: bool) -> Re let mut browser = Browser::open(session, false).await?; browser.control(command, output).await } - Plan::Close => super::stop::run(super::stop::Args { all: false }, session).await, } } @@ -1864,7 +1834,6 @@ mod tests { plan(&["status", "x-7"]).unwrap(), Plan::Control(ref c, Output::Execution) if *c == json!({"command": "status", "execution": "x-7"}) )); - assert!(matches!(plan(&["close"]).unwrap(), Plan::Close)); } #[test] diff --git a/src/commands/browser/batch.rs b/src/commands/browser/batch.rs index 3698ebd..a7f4cac 100644 --- a/src/commands/browser/batch.rs +++ b/src/commands/browser/batch.rs @@ -22,7 +22,7 @@ struct ActionParser { const MAX_COMMANDS: usize = 50; -fn split_command_str(s: &str) -> Vec { +pub fn split_command_str(s: &str) -> Vec { let mut args = Vec::new(); let mut current = String::new(); let mut in_quote: Option = None; @@ -127,7 +127,6 @@ mod tests { "tab new", "tab close", "dialog accept", - "close", "status x-1", ] { let err = run( diff --git a/src/commands/browser/captcha.rs b/src/commands/browser/captcha.rs index 977284e..15195c3 100644 --- a/src/commands/browser/captcha.rs +++ b/src/commands/browser/captcha.rs @@ -1,9 +1,9 @@ use clap::{Parser, Subcommand}; -use serde_json::json; +use serde_json::{Value, json}; use crate::api::client::SteelClient; -use crate::browser::lifecycle; -use crate::util::{api, output}; +use crate::browser::state::BrowserSession; +use crate::util::output; #[derive(Subcommand)] pub enum Command { @@ -25,10 +25,6 @@ impl Command { #[derive(Parser)] pub struct SolveArgs { - /// Explicit session ID (overrides --session name lookup) - #[arg(long)] - pub session_id: Option, - /// Page ID #[arg(long)] pub page_id: Option, @@ -44,10 +40,6 @@ pub struct SolveArgs { #[derive(Parser)] pub struct StatusArgs { - /// Explicit session ID (overrides --session name lookup) - #[arg(long)] - pub session_id: Option, - /// Page ID #[arg(long)] pub page_id: Option, @@ -72,57 +64,47 @@ pub async fn run(command: Command, session: Option<&str>) -> anyhow::Result<()> } } -fn resolve_session_id( - explicit_session_id: Option<&str>, - session: Option<&str>, -) -> anyhow::Result { - if let Some(id) = explicit_session_id { - let trimmed = id.trim(); - if !trimmed.is_empty() { - return Ok(trimmed.to_string()); - } - } - Ok(super::existing_session(session)?.id) +fn client_for(session: Option<&str>) -> anyhow::Result<(SteelClient, BrowserSession)> { + let target = super::existing_session(session)?; + let client = SteelClient::new()?.at(&target.api_url, target.mode); + Ok((client, target)) } async fn run_solve(args: SolveArgs, session: Option<&str>) -> anyhow::Result<()> { - let (mode, base_url, auth) = api::resolve_with_auth(); - let client = SteelClient::new()?; - - let session_id = resolve_session_id(args.session_id.as_deref(), session)?; - - let result = lifecycle::solve_captcha( - &client, - &base_url, - mode, - &auth, - &session_id, - args.page_id.as_deref(), - args.url.as_deref(), - args.task_id.as_deref(), - ) - .await?; + let (client, target) = client_for(session)?; + let (mode, session_id) = (target.mode, target.id); + + let raw = client + .solve_captcha( + &session_id, + args.page_id.as_deref(), + args.url.as_deref(), + args.task_id.as_deref(), + ) + .await?; + let success = raw["success"].as_bool().unwrap_or(false); + let message = raw["message"].as_str(); if output::is_json() { let mut data = json!({ - "sessionId": result.session_id, - "mode": result.mode.to_string(), - "success": result.success, + "sessionId": session_id, + "mode": mode.to_string(), + "success": success, }); - if let Some(ref msg) = result.message { + if let Some(msg) = message { data["message"] = json!(msg); } output::success_data(data); } else { - println!("session_id: {}", result.session_id); - println!("mode: {}", result.mode); - println!("success: {}", result.success); - if let Some(ref msg) = result.message { + println!("session_id: {session_id}"); + println!("mode: {mode}"); + println!("success: {success}"); + if let Some(msg) = message { println!("message: {msg}"); } } - if !result.success { + if !success { anyhow::bail!("CAPTCHA solve failed"); } @@ -130,44 +112,169 @@ async fn run_solve(args: SolveArgs, session: Option<&str>) -> anyhow::Result<()> } async fn run_status(args: StatusArgs, session: Option<&str>) -> anyhow::Result<()> { - let (mode, base_url, auth) = api::resolve_with_auth(); - let client = SteelClient::new()?; - - let session_id = resolve_session_id(args.session_id.as_deref(), session)?; - - let result = lifecycle::captcha_status( - &client, - &base_url, - mode, - &auth, - &session_id, - args.page_id.as_deref(), - args.wait, - args.timeout, - args.interval, - ) - .await?; + let (client, target) = client_for(session)?; + let session_id = target.id; + + let timeout = args.timeout.unwrap_or(60_000); + let interval = args.interval.unwrap_or(1_000); + let started = std::time::Instant::now(); + let (status, types) = loop { + let pages = client + .captcha_status(&session_id, args.page_id.as_deref()) + .await?; + let (status, types) = normalize_captcha_status(&pages); + if !args.wait || is_terminal_captcha_status(&status) { + break (status, types); + } + if started.elapsed().as_millis() >= u128::from(timeout) { + anyhow::bail!( + "CAPTCHA status polling timed out after {timeout}ms. Last status: {status}" + ); + } + tokio::time::sleep(std::time::Duration::from_millis(interval)).await; + }; if output::is_json() { let mut data = json!({ - "status": result.status, + "status": status, }); - if !result.types.is_empty() { - data["types"] = json!(result.types); + if !types.is_empty() { + data["types"] = json!(types); } output::success_data(data); } else { - let type_suffix = if result.types.is_empty() { + let type_suffix = if types.is_empty() { String::new() } else { - format!(" {}", result.types.join(",")) + format!(" {}", types.join(",")) }; - println!("{}{type_suffix}", result.status); + println!("{status}{type_suffix}"); } - if result.status != "solved" && result.status != "none" { - anyhow::bail!("CAPTCHA status: {}", result.status); + if status != "solved" && status != "none" { + anyhow::bail!("CAPTCHA status: {}", status); } Ok(()) } + +const KNOWN_CAPTCHA_TYPES: &[&str] = &["recaptchaV2", "recaptchaV3", "turnstile", "image_to_text"]; + +fn normalize_captcha_status(pages: &[Value]) -> (String, Vec) { + if pages.is_empty() { + return ("none".to_string(), vec![]); + } + + let mut all_tasks: Vec<&Value> = Vec::new(); + let mut any_solving = false; + + for page in pages { + if page.get("isSolvingCaptcha").and_then(|v| v.as_bool()) == Some(true) { + any_solving = true; + } + + if let Some(tasks) = page.get("tasks").and_then(|v| v.as_array()) { + for task in tasks { + if task.is_object() { + all_tasks.push(task); + } + } + } + } + + if all_tasks.is_empty() { + let status = if any_solving { "solving" } else { "none" }; + return (status.to_string(), vec![]); + } + + let mut solving_types: Vec = Vec::new(); + let mut failed_types: Vec = Vec::new(); + let mut has_solving = false; + let mut has_failed = false; + let mut has_solved = false; + + for task in &all_tasks { + let task_status = task.get("status").and_then(|v| v.as_str()).unwrap_or(""); + let task_type = task.get("type").and_then(|v| v.as_str()).unwrap_or(""); + let is_known = KNOWN_CAPTCHA_TYPES.contains(&task_type); + + match task_status { + "solving" | "detected" | "validating" => { + has_solving = true; + if is_known && !solving_types.contains(&task_type.to_string()) { + solving_types.push(task_type.to_string()); + } + } + "failed_to_solve" | "failed_to_detect" | "validation_failed" => { + has_failed = true; + if is_known && !failed_types.contains(&task_type.to_string()) { + failed_types.push(task_type.to_string()); + } + } + "solved" => { + has_solved = true; + } + _ => {} + } + } + + if has_solving || any_solving { + return ("solving".to_string(), solving_types); + } + + if has_failed { + return ("failed".to_string(), failed_types); + } + + if has_solved { + return ("solved".to_string(), vec![]); + } + + ("none".to_string(), vec![]) +} + +fn is_terminal_captcha_status(status: &str) -> bool { + matches!(status, "solved" | "failed" | "none") +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn normalize_captcha_none() { + let (status, types) = normalize_captcha_status(&[]); + assert_eq!(status, "none"); + assert!(types.is_empty()); + } + + #[test] + fn normalize_captcha_solving() { + let pages = vec![json!({ + "isSolvingCaptcha": true, + "tasks": [{"status": "solving", "type": "turnstile"}] + })]; + let (status, types) = normalize_captcha_status(&pages); + assert_eq!(status, "solving"); + assert_eq!(types, vec!["turnstile"]); + } + + #[test] + fn normalize_captcha_solved() { + let pages = vec![json!({ + "tasks": [{"status": "solved", "type": "recaptchaV2"}] + })]; + let (status, _types) = normalize_captcha_status(&pages); + assert_eq!(status, "solved"); + } + + #[test] + fn normalize_captcha_failed() { + let pages = vec![json!({ + "tasks": [{"status": "failed_to_solve", "type": "turnstile"}] + })]; + let (status, types) = normalize_captcha_status(&pages); + assert_eq!(status, "failed"); + assert_eq!(types, vec!["turnstile"]); + } +} diff --git a/src/commands/browser/live.rs b/src/commands/browser/live.rs index 35f8b13..92ebc9c 100644 --- a/src/commands/browser/live.rs +++ b/src/commands/browser/live.rs @@ -2,8 +2,7 @@ use clap::Parser; use serde_json::json; use crate::api::client::SteelClient; -use crate::browser::lifecycle::get_viewer_url; -use crate::config::auth; +use crate::api::session::Session; use crate::util::output; #[derive(Parser)] @@ -15,16 +14,11 @@ pub async fn run(_args: Args, session: Option<&str>) -> anyhow::Result<()> { let url = match target.live_url { Some(url) => Some(url), None => { - let client = SteelClient::new()?; - let remote = client - .get_session( - &target.api_url, - target.mode, - &target.id, - &auth::resolve_auth(), - ) + let remote = SteelClient::new()? + .at(&target.api_url, target.mode) + .cli_get_session(&target.id) .await?; - get_viewer_url(&remote, target.mode, &target.id) + Session::parse(remote)?.viewer_url(target.mode) } }; diff --git a/src/commands/browser/mod.rs b/src/commands/browser/mod.rs index 6dc86d1..fbeda43 100644 --- a/src/commands/browser/mod.rs +++ b/src/commands/browser/mod.rs @@ -10,12 +10,12 @@ use clap::{Parser, Subcommand}; use crate::browser::state::{self, BrowserSession}; -pub(crate) const DEFAULT_SESSION: &str = "default"; +pub const DEFAULT_SESSION: &str = "default"; #[derive(Parser)] pub struct BrowserArgs { /// Named session to target, or a session ID - #[arg(long, global = true)] + #[arg(long, global = true, alias = "session-id")] pub session: Option, /// Print what the command changed on the page (a diff against the last snapshot) @@ -32,6 +32,7 @@ pub enum Command { Start(start::Args), /// Stop a browser session + #[command(aliases = ["close", "quit", "exit"])] Stop(stop::Args), /// List active browser sessions @@ -68,28 +69,44 @@ impl Command { } } -pub(crate) fn existing_session(name: Option<&str>) -> anyhow::Result { +pub enum Target { + Saved(Box), + Id(String), +} + +pub fn target(name: Option<&str>) -> anyhow::Result> { let wanted = name.unwrap_or(DEFAULT_SESSION); + if let Some(err) = state::validate_session_name(wanted) { + anyhow::bail!("{err}"); + } if let Some(found) = state::load(wanted)? { - return Ok(found); + return Ok(Some(Target::Saved(Box::new(found)))); } if state::looks_like_session_id(wanted) { - let (mode, base_url) = crate::util::api::resolve(); - return Ok(BrowserSession::new( - wanted.to_string(), - wanted.to_string(), - mode, - base_url, - )); + return Ok(Some(Target::Id(wanted.to_string()))); } + Ok(None) +} + +pub fn not_running(name: Option<&str>) -> anyhow::Error { match name { - Some(name) => anyhow::bail!( - "Session \"{name}\" is not running. Start it with `steel browser start --session {name}`." + Some(name) => anyhow::anyhow!( + "Session \"{name}\" is not running. It may have expired. Start it with `steel browser start --session {name}`." ), - None => anyhow::bail!("No active browser session. Start one with `steel browser start`."), + None => anyhow::anyhow!("No active browser session. Start one with `steel browser start`."), } } +pub fn existing_session(name: Option<&str>) -> anyhow::Result { + Ok(match target(name)?.ok_or_else(|| not_running(name))? { + Target::Saved(found) => *found, + Target::Id(id) => { + let (mode, base_url) = crate::util::api::resolve(); + BrowserSession::new(id.clone(), id, mode, base_url) + } + }) +} + pub async fn run(args: BrowserArgs) -> anyhow::Result<()> { let session = args.session; if let Some(ref name) = session @@ -124,6 +141,38 @@ mod tests { } } + #[test] + fn target_rejects_paths_and_accepts_ids() { + assert!(super::target(Some("../etc/passwd")).is_err()); + let id = "4f7c2b1e-9a3d-4e5f-8b6a-1c2d3e4f5a6b"; + assert!( + matches!(super::target(Some(id)), Ok(Some(super::Target::Id(found))) if found == id) + ); + } + + #[test] + fn old_names_still_parse() { + let args = parse(&[ + "start", + "--session-timeout", + "1000", + "--session-solve-captcha", + ]); + let Command::Start(start) = args.command else { + panic!("expected start"); + }; + assert_eq!(start.timeout, Some(1000)); + assert!(start.solve_captcha); + + let id = "4f7c2b1e-9a3d-4e5f-8b6a-1c2d3e4f5a6b"; + let args = parse(&["captcha", "solve", "--session-id", id]); + assert_eq!(args.session.as_deref(), Some(id)); + + for name in ["close", "quit", "exit"] { + assert!(matches!(parse(&[name]).command, Command::Stop(_)), "{name}"); + } + } + #[test] fn session_after_fill_value_is_a_flag() { let args = parse(&["fill", "@e1", "herman", "miller", "--session", "repro"]); diff --git a/src/commands/browser/sessions.rs b/src/commands/browser/sessions.rs index 9a332a7..fa43c20 100644 --- a/src/commands/browser/sessions.rs +++ b/src/commands/browser/sessions.rs @@ -2,9 +2,8 @@ use clap::Parser; use serde_json::json; use crate::api::client::SteelClient; -use crate::browser::lifecycle::is_session_live; +use crate::api::session::Session; use crate::browser::state::{self, BrowserSession}; -use crate::config::auth; use crate::util::output; #[derive(Parser)] @@ -12,25 +11,26 @@ pub struct Args {} pub async fn run(_args: Args) -> anyhow::Result<()> { let client = SteelClient::new()?; - let auth = auth::resolve_auth(); let mut sessions: Vec<(BrowserSession, String)> = Vec::new(); for session in state::list() { match client - .get_session(&session.api_url, session.mode, &session.id, &auth) + .at(&session.api_url, session.mode) + .cli_get_session(&session.id) .await + .map(Session::parse) { - Ok(remote) if is_session_live(&remote) => { - let status = remote["status"].as_str().unwrap_or("live").to_string(); + Ok(Ok(remote)) if remote.is_live() => { + let status = remote.status.unwrap_or_else(|| "live".to_string()); sessions.push((session, status)); } - Ok(_) => { + Ok(Ok(_)) => { state::remove(&session.name)?; } Err(err) if err.is_not_found() => { state::remove(&session.name)?; } - Err(_) => sessions.push((session, "unknown".to_string())), + Ok(Err(_)) | Err(_) => sessions.push((session, "unknown".to_string())), } } diff --git a/src/commands/browser/start.rs b/src/commands/browser/start.rs index 4a8f956..5e3b306 100644 --- a/src/commands/browser/start.rs +++ b/src/commands/browser/start.rs @@ -3,16 +3,10 @@ use serde_json::json; use crate::api::client::SteelClient; use crate::api::session::CreateSessionOptions; -use crate::browser::lifecycle::{ - get_session_created_at_ms, get_session_inactivity_timeout, get_session_timeout, - sanitize_connect_url, to_session_summary, -}; use crate::browser::profile_store; use crate::browser::state::{self, BrowserSession}; -use crate::config::auth::Auth; -use crate::config::settings::ApiMode; use crate::status; -use crate::util::{api, output}; +use crate::util::output; pub const DEFAULT_INACTIVITY_TIMEOUT_MS: u64 = 600_000; @@ -27,24 +21,24 @@ pub struct Args { pub proxy: Option, /// Session timeout in milliseconds (create-time only) - #[arg(long = "session-timeout")] - pub session_timeout: Option, + #[arg(long, alias = "session-timeout")] + pub timeout: Option, /// Inactivity timeout in milliseconds: release the session when no command, CDP message, or remote input arrives for this long. Defaults to 600000 (10 minutes); pass 0 to disable (create-time only) #[arg(long = "inactivity-timeout")] pub inactivity_timeout: Option, /// Create new sessions in headless mode (create-time only) - #[arg(long = "session-headless", hide = true)] - pub session_headless: Option, + #[arg(long, alias = "session-headless", hide = true)] + pub headless: Option, /// Preferred session region (create-time only) - #[arg(long = "session-region", hide = true)] - pub session_region: Option, + #[arg(long, alias = "session-region", hide = true)] + pub region: Option, /// Enable manual CAPTCHA solving on new sessions (create-time only) - #[arg(long = "session-solve-captcha")] - pub session_solve_captcha: bool, + #[arg(long, alias = "session-solve-captcha")] + pub solve_captcha: bool, /// Named profile to persist browser state across sessions #[arg(long)] @@ -79,33 +73,29 @@ fn now_ms() -> Option { pub async fn create( client: &SteelClient, - mode: ApiMode, - base_url: &str, - auth: &Auth, name: &str, options: &CreateSessionOptions, ) -> anyhow::Result<(BrowserSession, Option)> { - let session = client.create_session(base_url, mode, options, auth).await?; - let summary = to_session_summary(&session, mode, Some(name), auth)?; + let session = client.create_session(options).await?; + let mode = client.mode(); let mut created = BrowserSession::new( name.to_string(), - summary.id.clone(), + session.id.clone(), mode, - base_url.to_string(), + client.base_url().to_string(), ); - created.live_url = summary.viewer_url; - created.profile_id = summary.profile_id; - created.timeout_ms = get_session_timeout(&session).or(options.timeout_ms); - created.inactivity_timeout_ms = - get_session_inactivity_timeout(&session).or(options.inactivity_timeout_ms); - created.created_at_ms = get_session_created_at_ms(&session).or_else(now_ms); + created.live_url = session.viewer_url(mode); + created.profile_id = session.profile_id.clone(); + created.timeout_ms = session.timeout_ms().or(options.timeout_ms); + created.inactivity_timeout_ms = session + .inactivity_timeout_ms() + .or(options.inactivity_timeout_ms); + created.created_at_ms = session.created_at_ms().or_else(now_ms); state::save(&created)?; - Ok((created, summary.connect_url)) + Ok((created, session.connect_url(mode, client.auth()))) } pub async fn run(args: Args, session: Option<&str>) -> anyhow::Result<()> { - let (mode, base_url, auth) = api::resolve_with_auth(); - let session_name = session.unwrap_or("default").to_string(); if state::looks_like_session_id(&session_name) { anyhow::bail!( @@ -131,11 +121,11 @@ pub async fn run(args: Args, session: Option<&str>) -> anyhow::Result<()> { let proxy_enabled = args.proxy.is_some(); let namespace_set = args.namespace.is_some(); let inactivity_timeout_ms = resolve_inactivity_timeout(args.inactivity_timeout); - if let (Some(timeout), Some(inactivity)) = (args.session_timeout, inactivity_timeout_ms) + if let (Some(timeout), Some(inactivity)) = (args.timeout, inactivity_timeout_ms) && inactivity >= timeout { eprintln!( - "warning: --inactivity-timeout ({inactivity}ms) >= --session-timeout ({timeout}ms); inactivity timeout has no effect because the session timeout elapses first" + "warning: --inactivity-timeout ({inactivity}ms) >= --timeout ({timeout}ms); inactivity timeout has no effect because the session timeout elapses first" ); } @@ -143,7 +133,7 @@ pub async fn run(args: Args, session: Option<&str>) -> anyhow::Result<()> { if let Some(existing) = state::load(&session_name)? { status!("Replacing existing session \"{session_name}\"..."); - if let Err(err) = super::stop::release(&client, &auth, &existing).await { + if let Err(err) = super::stop::release(&client, &existing).await { eprintln!("warning: could not release session {}: {err}", existing.id); state::remove(&session_name)?; } @@ -152,19 +142,18 @@ pub async fn run(args: Args, session: Option<&str>) -> anyhow::Result<()> { let options = CreateSessionOptions { stealth: args.stealth, proxy_url: args.proxy, - timeout_ms: args.session_timeout, + timeout_ms: args.timeout, inactivity_timeout_ms, - headless: args.session_headless, - region: args.session_region, - solve_captcha: args.session_solve_captcha, + headless: args.headless, + region: args.region, + solve_captcha: args.solve_captcha, profile_id: resolved_profile_id, persist_profile, namespace: args.namespace, credentials: args.credentials, }; - let (created, connect_url) = - create(&client, mode, &base_url, &auth, &session_name, &options).await?; + let (created, connect_url) = create(&client, &session_name, &options).await?; if let Some(ref profile_name) = args.profile && let Some(ref returned_profile_id) = created.profile_id @@ -184,10 +173,7 @@ pub async fn run(args: Args, session: Option<&str>) -> anyhow::Result<()> { properties.insert("persist_profile".into(), json!(persist_profile)); properties.insert("credentials".into(), json!(args.credentials)); properties.insert("namespace_set".into(), json!(namespace_set)); - properties.insert( - "session_timeout_set".into(), - json!(args.session_timeout.is_some()), - ); + properties.insert("session_timeout_set".into(), json!(args.timeout.is_some())); properties.insert("inactivity_timeout_ms".into(), json!(inactivity_timeout_ms)); crate::telemetry::track_event("browser_session_started", properties); @@ -263,6 +249,32 @@ fn humanize_secs(secs: u64) -> String { } } +fn sanitize_connect_url(url: &str) -> String { + if let Ok(mut parsed) = url::Url::parse(url) { + let pairs: Vec<(String, String)> = parsed + .query_pairs() + .map(|(k, v)| { + if k == "apiKey" && v.len() > 7 { + (k.to_string(), format!("{}...", &v[..7])) + } else { + (k.to_string(), v.to_string()) + } + }) + .collect(); + + if !pairs.is_empty() { + parsed.query_pairs_mut().clear(); + for (k, v) in &pairs { + parsed.query_pairs_mut().append_pair(k, v); + } + } + + parsed.to_string() + } else { + url.to_string() + } +} + const fn resolve_inactivity_timeout(explicit: Option) -> Option { match explicit { None => Some(DEFAULT_INACTIVITY_TIMEOUT_MS), @@ -302,6 +314,20 @@ mod tests { assert!(!options.stealth); } + #[test] + fn sanitize_masks_long_api_keys_only() { + let masked = + sanitize_connect_url("wss://connect.steel.dev?apiKey=sk-12345678&sessionId=s1"); + assert!(masked.contains("sk-1234...")); + assert!(masked.contains("sessionId=s1")); + assert!(!masked.contains("sk-12345678")); + let short = "wss://connect.steel.dev?apiKey=short&sessionId=s1"; + assert!(sanitize_connect_url(short).contains("apiKey=short")); + let local = "ws://localhost:3000/ws?sessionId=sess-1"; + assert_eq!(sanitize_connect_url(local), local); + assert_eq!(sanitize_connect_url("not a valid url"), "not a valid url"); + } + #[test] fn humanize_secs_formats() { assert_eq!(humanize_secs(45), "45s"); diff --git a/src/commands/browser/stop.rs b/src/commands/browser/stop.rs index 48daef8..df76cfd 100644 --- a/src/commands/browser/stop.rs +++ b/src/commands/browser/stop.rs @@ -3,7 +3,6 @@ use serde_json::json; use crate::api::client::SteelClient; use crate::browser::state::{self, BrowserSession}; -use crate::config::auth::{self, Auth}; use crate::util::output; #[derive(Parser)] @@ -13,16 +12,13 @@ pub struct Args { pub all: bool, } -pub async fn release( - client: &SteelClient, - auth: &Auth, - session: &BrowserSession, -) -> anyhow::Result<()> { +pub async fn release(client: &SteelClient, session: &BrowserSession) -> anyhow::Result<()> { match client - .release_session(&session.api_url, session.mode, &session.id, auth) + .at(&session.api_url, session.mode) + .cli_release_session(&session.id) .await { - Ok(()) => {} + Ok(_) => {} Err(err) if err.is_not_found() => {} Err(err) => return Err(err.into()), } @@ -36,7 +32,6 @@ pub async fn run(args: Args, session: Option<&str>) -> anyhow::Result<()> { } let client = SteelClient::new()?; - let auth = auth::resolve_auth(); let stopped: Vec = if args.all { let sessions = state::list(); @@ -50,7 +45,7 @@ pub async fn run(args: Args, session: Option<&str>) -> anyhow::Result<()> { } let mut stopped = Vec::new(); for session in &sessions { - match release(&client, &auth, session).await { + match release(&client, session).await { Ok(()) => stopped.push(session.name.clone()), Err(err) => eprintln!("warning: could not stop \"{}\": {err}", session.name), } @@ -58,7 +53,7 @@ pub async fn run(args: Args, session: Option<&str>) -> anyhow::Result<()> { stopped } else { let target = super::existing_session(session)?; - release(&client, &auth, &target).await?; + release(&client, &target).await?; vec![target.name] }; diff --git a/src/commands/cache.rs b/src/commands/cache.rs index 8792c51..6e09dac 100644 --- a/src/commands/cache.rs +++ b/src/commands/cache.rs @@ -1,6 +1,7 @@ use clap::Parser; +use serde_json::json; -use crate::status; +use crate::util::output; #[derive(Parser)] pub struct Args { @@ -15,13 +16,21 @@ pub async fn run(args: Args) -> anyhow::Result<()> { .join("steel"); if !args.clean { - status!("Steel CLI cache directory: {}", cache_dir.display()); - status!("Use --clean to remove all cached files."); + output::success( + json!({"cacheDir": cache_dir}), + &format!( + "Steel CLI cache directory: {}\nUse --clean to remove all cached files.\n", + cache_dir.display() + ), + ); return Ok(()); } if !cache_dir.exists() { - status!("Cache directory does not exist. Nothing to clean."); + output::success( + json!({"cacheDir": cache_dir, "removed": 0}), + "Cache directory does not exist. Nothing to clean.\n", + ); return Ok(()); } @@ -37,8 +46,13 @@ pub async fn run(args: Args) -> anyhow::Result<()> { count += 1; } - status!("Removed {count} item(s) from cache."); - status!("Cache directory: {}", cache_dir.display()); + output::success( + json!({"cacheDir": cache_dir, "removed": count}), + &format!( + "Removed {count} item(s) from cache.\nCache directory: {}\n", + cache_dir.display() + ), + ); Ok(()) } diff --git a/src/commands/config.rs b/src/commands/config.rs index 29acac7..8b26642 100644 --- a/src/commands/config.rs +++ b/src/commands/config.rs @@ -4,7 +4,7 @@ use clap::Parser; use crate::api::client::SteelClient; use crate::api::projects::{environment_label, parse_projects, resolve_current_project}; -use crate::config::auth::{self, Auth}; +use crate::config::auth; use crate::config::settings::{ApiMode, read_config}; #[derive(Parser)] @@ -29,7 +29,7 @@ pub async fn run(_args: Args) -> anyhow::Result<()> { println!("source: {}", auth.source); } - print_project_info(&auth).await; + print_project_info().await; if let Some(ref cfg) = config && let Some(ref browser) = cfg.browser @@ -48,16 +48,15 @@ pub async fn run(_args: Args) -> anyhow::Result<()> { /// Print the project (environment) the API key is scoped to. /// Best-effort: skipped in local mode, without a key, or on any fetch error, /// so `steel config` stays usable offline. -async fn print_project_info(auth: &Auth) { - let (mode, base_url) = crate::util::api::resolve(); - if mode != ApiMode::Cloud || auth.api_key.is_none() { - return; - } +async fn print_project_info() { let Ok(client) = SteelClient::new() else { return; }; + if client.mode() != ApiMode::Cloud || client.auth().api_key.is_none() { + return; + } - let fetch = client.get_projects(&base_url, mode, auth); + let fetch = client.get_projects(); let Ok(Ok(data)) = tokio::time::timeout(Duration::from_secs(2), fetch).await else { return; }; diff --git a/src/commands/credentials.rs b/src/commands/credentials.rs index b295ae2..d2517e6 100644 --- a/src/commands/credentials.rs +++ b/src/commands/credentials.rs @@ -2,7 +2,7 @@ use clap::{Parser, Subcommand}; use serde_json::json; use crate::api::client::SteelClient; -use crate::util::{api, output}; +use crate::util::output; #[derive(Subcommand)] pub enum Command { @@ -116,7 +116,6 @@ pub async fn run(command: Command) -> anyhow::Result<()> { } async fn run_list(args: ListArgs) -> anyhow::Result<()> { - let (mode, base_url, auth_info) = api::resolve_with_auth(); let client = SteelClient::new()?; let mut query_params = Vec::new(); @@ -134,14 +133,7 @@ async fn run_list(args: ListArgs) -> anyhow::Result<()> { }; let data = client - .request( - &base_url, - mode, - reqwest::Method::GET, - &path, - None, - &auth_info, - ) + .request(reqwest::Method::GET, &path, None) .await .map_err(|e| anyhow::anyhow!("{e}"))?; @@ -179,7 +171,6 @@ async fn run_create(args: CreateArgs) -> anyhow::Result<()> { .filter(|s| !s.is_empty()) .ok_or_else(|| anyhow::anyhow!("Missing required flag: --password"))?; - let (mode, base_url, auth_info) = api::resolve_with_auth(); let client = SteelClient::new()?; let mut value = json!({ @@ -211,14 +202,7 @@ async fn run_create(args: CreateArgs) -> anyhow::Result<()> { } let result = client - .request( - &base_url, - mode, - reqwest::Method::POST, - "/credentials", - Some(body), - &auth_info, - ) + .request(reqwest::Method::POST, "/credentials", Some(body)) .await .map_err(|e| anyhow::anyhow!("{e}"))?; @@ -256,7 +240,6 @@ async fn run_update(args: UpdateArgs) -> anyhow::Result<()> { .filter(|s| !s.is_empty()) .ok_or_else(|| anyhow::anyhow!("Missing required flag: --origin"))?; - let (mode, base_url, auth_info) = api::resolve_with_auth(); let client = SteelClient::new()?; let mut body = json!({"origin": origin}); @@ -300,14 +283,7 @@ async fn run_update(args: UpdateArgs) -> anyhow::Result<()> { } let result = client - .request( - &base_url, - mode, - reqwest::Method::PUT, - "/credentials", - Some(body), - &auth_info, - ) + .request(reqwest::Method::PUT, "/credentials", Some(body)) .await .map_err(|e| anyhow::anyhow!("{e}"))?; @@ -338,7 +314,6 @@ async fn run_delete(args: DeleteArgs) -> anyhow::Result<()> { .filter(|s| !s.is_empty()) .ok_or_else(|| anyhow::anyhow!("Missing required flag: --origin"))?; - let (mode, base_url, auth_info) = api::resolve_with_auth(); let client = SteelClient::new()?; let mut body = json!({"origin": origin}); @@ -350,14 +325,7 @@ async fn run_delete(args: DeleteArgs) -> anyhow::Result<()> { } client - .request( - &base_url, - mode, - reqwest::Method::DELETE, - "/credentials", - Some(body), - &auth_info, - ) + .request(reqwest::Method::DELETE, "/credentials", Some(body)) .await .map_err(|e| anyhow::anyhow!("{e}"))?; diff --git a/src/commands/describe.rs b/src/commands/describe.rs index 5c3de98..e3c2e8b 100644 --- a/src/commands/describe.rs +++ b/src/commands/describe.rs @@ -394,6 +394,91 @@ fn has_visible_subcommands(cmd: &clap::Command) -> bool { .any(|s| !s.is_hide_set() && s.get_name() != "help") } +const OVERVIEW_COLUMN: usize = 40; + +pub fn overview(root: &clap::Command) -> String { + let mut lines = Vec::new(); + for sub in get_visible_subcommands(root) { + overview_lines(&sub, &format!("steel {}", sub.get_name()), &mut lines); + } + lines.join("\n") +} + +fn overview_lines(cmd: &clap::Command, path: &str, lines: &mut Vec) { + let (positionals, flags): (Vec<&clap::Arg>, Vec<&clap::Arg>) = cmd + .get_arguments() + .filter(|a| is_visible_arg(a)) + .partition(|a| a.is_positional()); + let usage = positionals.iter().fold(path.to_string(), |usage, arg| { + format!("{usage} {}", placeholder(arg)) + }); + let mut about = cmd.get_about().map(|a| a.to_string()).unwrap_or_default(); + let aliases = get_aliases(cmd); + if !aliases.is_empty() { + about.push_str(&format!(" (aliases: {})", aliases.join(", "))); + } + lines.push(overview_row(2, &usage, &about)); + for arg in flags { + lines.push(overview_row(4, &flag_usage(arg), &arg_summary(arg))); + } + for sub in get_visible_subcommands(cmd) { + overview_lines(&sub, &format!("{path} {}", sub.get_name()), lines); + } +} + +fn overview_row(indent: usize, left: &str, right: &str) -> String { + let left = format!("{:indent$}{left}", ""); + if right.is_empty() { + return left; + } + let width = OVERVIEW_COLUMN.max(left.len() + 2); + format!("{left: String { + arg.get_value_names() + .and_then(|names| names.first()) + .map_or_else(|| arg.get_id().to_string(), |name| name.to_string()) +} + +fn takes_many(arg: &clap::Arg) -> bool { + arg.get_num_args() + .is_some_and(|range| range.max_values() > 1) +} + +fn placeholder(arg: &clap::Arg) -> String { + let name = value_name(arg); + let many = if takes_many(arg) { "..." } else { "" }; + if arg.is_required_set() { + format!("<{name}>{many}") + } else { + format!("[{name}]{many}") + } +} + +fn flag_usage(arg: &clap::Arg) -> String { + let mut usage = match (arg.get_short(), arg.get_long()) { + (Some(short), Some(long)) => format!("-{short}, --{long}"), + (Some(short), None) => format!("-{short}"), + (None, Some(long)) => format!("--{long}"), + (None, None) => value_name(arg), + }; + if arg.get_action().takes_values() { + usage.push_str(&format!(" <{}>", value_name(arg))); + } + usage +} + +fn arg_summary(arg: &clap::Arg) -> String { + let mut summary = arg.get_help().map(|h| h.to_string()).unwrap_or_default(); + if arg.get_action().takes_values() + && let Some(default) = arg.get_default_values().first() + { + summary.push_str(&format!(" [default: {}]", default.to_string_lossy())); + } + summary +} + // ── Tests ─────────────────────────────────────────────────────────── #[cfg(test)] @@ -600,4 +685,31 @@ mod tests { assert_eq!(ops[0]["streaming"]["transport"], "websocket"); assert_eq!(ops[0]["streaming"]["path"], "/v1/sessions/{id}/logs"); } + + #[test] + fn generated_operation_ids_are_unique() { + let ids: Vec<_> = CLI_OPERATION_METADATA.iter().map(|op| op.id).collect(); + let unique: std::collections::BTreeSet<_> = ids.iter().collect(); + assert_eq!(ids.len(), unique.len()); + } + + #[test] + fn generated_command_paths_exist_in_clap_tree() { + let root = root_cmd(); + for op in CLI_OPERATION_METADATA { + let mut current = root.clone(); + for segment in op + .command + .split_whitespace() + .filter(|s| !s.starts_with('-')) + { + current = current + .find_subcommand(segment) + .unwrap_or_else(|| { + panic!("generated command path does not exist: {}", op.command) + }) + .clone(); + } + } + } } diff --git a/src/commands/doctor.rs b/src/commands/doctor.rs index fc29ef9..2e3d471 100644 --- a/src/commands/doctor.rs +++ b/src/commands/doctor.rs @@ -124,7 +124,7 @@ async fn check_auth_and_api(mode: ApiMode, base_url: &str, auth: &Auth) -> Vec Vec Vec Vec { checks.push(Check { category: "api", diff --git a/src/commands/mod.rs b/src/commands/mod.rs index b1d9daf..ade62a4 100644 --- a/src/commands/mod.rs +++ b/src/commands/mod.rs @@ -18,228 +18,64 @@ pub mod sessions; pub mod skills; pub mod update; -use clap::{Parser, Subcommand, builder::styling::Styles}; - -const LONG_HELP: &str = "\ -Global Flags: - --json Structured JSON output for automation - -l, --local Use local Steel runtime - --api-url Explicit self-hosted API endpoint URL - -Getting Started: - steel init Log in, verify, and install Steel skills into detected agents - --agent Auto-accept prompts and print agent-friendly output - steel skills list List available Steel Skills - steel skills install --all Install all Steel Skills through npx skills - steel skills install Install a Steel Skill through npx skills - steel skills doctor Check skills installer, auth, manifest, and paths - -Quick Actions: - steel scrape Scrape webpage content (markdown by default) - --format Output formats: html, readability, cleaned_html, markdown - -d, --delay Delay before scraping - --pdf / --screenshot Include PDF or screenshot in response - --use-proxy Use Steel-managed residential proxy - -r, --region Region for request execution - -l, --local Use local Steel runtime - steel screenshot Capture a screenshot - -f, --full-page Full-page capture - steel pdf Generate a PDF from a webpage - -Browser Sessions: - steel browser start [--session ] Create a browser session - --stealth Enable stealth mode (humanize + auto CAPTCHA) - -p, --proxy Use residential proxy - --session-timeout Session timeout - --inactivity-timeout Release after this long without commands (default: 10 min) - --session-solve-captcha Enable manual CAPTCHA solving - --profile Named profile to persist browser state - --update-profile Save state back to profile on session end - --namespace / --credentials Inject stored credentials - steel browser stop [--session ] Stop a session (-a for all) - steel browser sessions List active sessions (use --json for structured output) - steel browser live [--session ] Open live session viewer - steel browser batch \"\" \"\" Run several commands in one request - --session Target a named session or a session ID - --diff Print what a command changed on the page - -Cloud Sessions: - steel sessions list List cloud sessions - --status Filter: live, released, failed - --limit Number of sessions to return - steel sessions get Show cloud session details - steel sessions release Release a cloud session - -a, --all Release all active cloud sessions - steel sessions logs Show browser event logs - -f, --follow Follow live logs over WebSocket - --since