diff --git a/.claude/commands/line-analyze.md b/.agents/commands/line-analyze.md similarity index 84% rename from .claude/commands/line-analyze.md rename to .agents/commands/line-analyze.md index 760755b..9855417 100644 --- a/.claude/commands/line-analyze.md +++ b/.agents/commands/line-analyze.md @@ -1,8 +1,9 @@ Parse and analyze a LINE Chrome Extension network capture. -## Arguments +## Input -$ARGUMENTS is the path to a net-log JSON file (default: `/tmp/line-capture/line-net-log.json`). +Use the provided path to a net-log JSON file. If no path is provided, default to +`/tmp/line-chrome-extension-capture/line-net-log.json`. ## What to do @@ -35,7 +36,7 @@ $ARGUMENTS is the path to a net-log JSON file (default: `/tmp/line-capture/line- 5. Highlight any **unimplemented** endpoints or behaviors that differ from what the bridge currently does. Cross-reference with `pkg/line/methods.go` and `pkg/connector/`. -6. After presenting the analysis, ask the user if they want to clean up the capture files (`/tmp/line-capture/`). Warn that the raw net-log and parsed output contain sensitive data (auth tokens in `X-Line-Access` headers, message IDs, etc.) and should not be left on disk or committed to the repository. If the user agrees, delete the entire capture directory. +6. After presenting the analysis, ask the user if they want to clean up the capture files (`/tmp/line-chrome-extension-capture/`). Warn that the raw net-log and parsed output contain sensitive data (auth tokens in `X-Line-Access` headers, message IDs, etc.) and should not be left on disk or committed to the repository. If the user agrees, delete the entire capture directory. ## LINE protocol context diff --git a/.agents/commands/line-capture.md b/.agents/commands/line-capture.md new file mode 100644 index 0000000..b0ecd69 --- /dev/null +++ b/.agents/commands/line-capture.md @@ -0,0 +1,38 @@ +Launch Chrome with network capture enabled for LINE Chrome Extension analysis. + +## What to do + +1. **Close any running Chrome instances first** (Chrome only supports one instance per profile with remote debugging). Warn the user if Chrome is running. + +2. Run the capture script: + ``` + bash scripts/line-chrome-capture.sh /tmp/line-chrome-extension-capture + ``` + This launches Chrome with: + - Remote debugging on port 9222 + - Full network logging to `/tmp/line-chrome-extension-capture/line-net-log.json` + - A persistent capture-only Chrome profile at + `~/.cache/line-chrome-extension-capture/chrome-debug-profile` + - Auto-loaded LINE and Codex extensions staged from the user's normal Chrome + profile into `~/.cache/line-chrome-extension-capture/extensions` + - A LINE extension version preflight: + - repo version from `pkg/line/client.go` `ExtensionVersion` + - installed/staged Chrome extension version from `manifest.json` + - a warning if those versions differ + - First-run Chrome prompts pre-skipped: + - "Make Google Chrome the default browser" stays unchecked + - "Automatically send usage statistics and crash reports" stays unchecked + - Chrome/Sync sign-in is skipped; use "Stay Signed Out" if it appears + - If the LINE extension is missing and the user asked for setup, relaunch with + `LINE_CAPTURE_AUTO_LOAD_EXTENSIONS=0`, open the official Web Store listing + for `ophjlpahpchlmihnnnihgmmeilfjmjjc`, verify LINE / LY Corporation, click + Add to Chrome, and accept the standard "Add extension" confirmation + +3. Tell the user: + - Open the LINE Chrome Extension (browser toolbar icon) + - Log in manually if the capture profile is not already logged in + - Perform whatever flow they want to analyze + - When done, quit Chrome completely (Cmd+Q) + +4. Once Chrome is closed, the net-log file is ready. Continue with the + line-analyze workflow. diff --git a/.agents/commands/line-implement.md b/.agents/commands/line-implement.md new file mode 100644 index 0000000..246da20 --- /dev/null +++ b/.agents/commands/line-implement.md @@ -0,0 +1,93 @@ +Implement a matrix-line feature by capturing and mimicking LINE Chrome Extension behavior. + +## Input + +Use the provided feature goal, for example: + +``` +Support Matrix group avatar changes by matching LINE Chrome Extension traffic. +``` + +## What to do + +1. **Understand the goal and the current bridge first.** + - Read `AGENTS.md`/`CLAUDE.md`, `docs/protocol/README.md`, relevant endpoint docs, and `docs/protocol/gap-analysis.md`. + - Inspect the likely implementation surface in `pkg/line/`, `pkg/connector/`, and existing tests. + - If existing protocol docs already answer the feature, implement from docs and code without a new capture. + +2. **Write a capture contract before launching Chrome.** + Capture only the missing evidence: + - exact user-visible LINE action to perform + - expected Talk/Auth/OBS/SSE endpoints and method names + - request argument shape and ordering + - response wrapper shape, headers, and error behavior + - resulting SSE/operation events, revisions, message metadata, and local state changes + - what values the bridge must persist or expose afterward + +3. **Check LINE Chrome Extension version drift.** + - Compare the installed/staged extension `manifest.json` version against + `pkg/line/client.go` `ExtensionVersion`. + - Also check literal `x-line-application` versions in `pkg/line/client.go` + and the default `clientVersion` in `pkg/runner.go`. + - The capture launcher prints the repo and Chrome extension versions and + warns on mismatch. Do not silently mix capture evidence from a different + LINE Chrome Extension version. + +4. **Security preflight.** + - Treat raw captures as secrets: they can contain access tokens, cookies, MIDs, message contents, contact data, and E2EE metadata. + - Use a fresh `/tmp/line-chrome-extension-capture/-` directory with `umask 077`. + - Do not commit raw capture files, Chrome profiles, screenshots, QR codes, or parsed JSON. + - Do not request, store, or automate the LINE password. + - The user performs LINE login manually, including email/password, QR, PIN, 2FA, and mobile approval. + - Do not run the bridge and LINE Chrome Extension simultaneously unless the user explicitly accepts session invalidation. + - Prefer the persistent capture Chrome profile from `scripts/line-chrome-capture.sh`. Use the user's normal Chrome profile only after explicit approval. + +5. **Run the capture.** + - Check for running Chrome and `matrix-line` processes; warn before closing or stopping anything. + - Start capture: + ``` + umask 077 + CAPTURE_DIR="/tmp/line-chrome-extension-capture/$(date +%Y%m%d-%H%M%S)-" + bash scripts/line-chrome-capture.sh "$CAPTURE_DIR" + ``` + - Use Codex Chrome automation when available to drive the already-logged-in extension UI. If it cannot access the capture profile or extension UI, use Computer Use. + - The launcher stages clean unpacked LINE and Codex extension copies from the user's normal Chrome profile and auto-loads them when possible. + - The launcher pre-skips Chrome's first-run prompts: default browser is off, + usage/crash reporting is off, and Chrome/Sync sign-in should be skipped. + If the sign-in screen still appears, choose "Stay Signed Out". + - If Chrome blocks unpacked loading or LINE is missing from the capture + profile, relaunch setup with `LINE_CAPTURE_AUTO_LOAD_EXTENSIONS=0`, open + the official Web Store listing + `https://chromewebstore.google.com/detail/line/ophjlpahpchlmihnnnihgmmeilfjmjjc`, + verify LINE / LY Corporation, click Add to Chrome, and accept the standard + "Add extension" confirmation when the user has requested setup. + - If Chrome is not logged in, pause and ask the user to complete LINE login manually in the capture browser. + - Perform the smallest LINE UI flow that satisfies the capture contract, then quit Chrome completely. + +6. **Parse, redact, and analyze.** + - Prefer CDP output because it includes SSE messages: + ``` + python3 scripts/parse-line-traffic.py "$CAPTURE_DIR/line-cdp-capture.json" "$CAPTURE_DIR/line-cdp-capture-parsed.json" + python3 scripts/redact-line-capture.py "$CAPTURE_DIR/line-cdp-capture-parsed.json" + ``` + - Read raw files only locally and only when redacted output loses information needed for implementation. + - Quote or document only redacted snippets. + - Compare against `pkg/line/methods.go`, `pkg/line/client.go`, `pkg/line/sse.go`, and the relevant `pkg/connector/` flow. + - Produce an evidence table: Chrome behavior, bridge behavior, implementation delta, files to change. + +7. **Implement religiously.** + - Match the Chrome Extension's endpoint path, service/method name, argument order, headers, request body shape, response handling, and fallback behavior. + - Preserve existing bridge architecture: Thrift/API code in `pkg/line`, Matrix-facing behavior in `pkg/connector`, E2EE behavior in `pkg/e2ee`/`pkg/ltsm`. + - Update protocol docs only with redacted request/response/event evidence. + - Add or update focused tests for parsing, request construction, event handling, persistence, and fallback/error behavior. + +8. **Verify and clean up.** + - Run: + ``` + go fmt ./... + goimports -local "github.com/highesttt/matrix-line-messenger" -w . + go test ./... + go vet $(go list ./... | grep -v /ltsm) + ``` + - If available, run `staticcheck $(go list ./... | grep -v /ltsm)`. + - Before finishing, report where raw captures live and ask whether to delete them. Prefer keeping only redacted notes/docs. diff --git a/.agents/skills/line-implement/REFERENCE.md b/.agents/skills/line-implement/REFERENCE.md new file mode 100644 index 0000000..d163583 --- /dev/null +++ b/.agents/skills/line-implement/REFERENCE.md @@ -0,0 +1,170 @@ +# Line Implement Reference + +## Repo Orientation + +Start with: + +- `AGENTS.md` / `CLAUDE.md` +- `.agents/commands/line-capture.md` +- `.agents/commands/line-analyze.md` +- `.agents/commands/line-implement.md` +- `docs/protocol/README.md` +- `docs/protocol/endpoints/README.md` +- `docs/protocol/gap-analysis.md` +- `pkg/line/methods.go`, `pkg/line/client.go`, `pkg/line/sse.go` +- likely `pkg/connector/*` files for the Matrix-facing feature + +Do not capture if the current protocol docs already provide enough evidence. +Use `.agents` as the repo-local source of truth. `.claude/commands` is a +compatibility symlink for Claude-style command discovery when present. + +## Capture Contract + +Before launching Chrome, write down: + +- the exact LINE UI action to perform +- fixture requirements such as DM/group, `c...`/`r...` chat MID type, media type, + old/new state, or Letter Sealing state +- expected endpoint families: TalkService, AuthService, LoginQrCode, OBS, SSE, + CDN, or secondary APIs +- unknowns needed for implementation: method name, URL, argument order, request + body shape, response wrapper, headers, operation events, revision changes, + persisted IDs, and fallback/error behavior + +Keep the UI flow narrow. Avoid broad exploratory captures unless the protocol +surface is genuinely unknown. + +## Version Preflight + +Before trusting capture evidence, compare the repo's LINE Chrome Extension +version with the installed extension: + +```bash +rg -n 'ExtensionVersion|x-line-application|clientVersion' pkg/line/client.go pkg/runner.go +find "$HOME/.cache/line-chrome-extension-capture/chrome-debug-profile/Default/Extensions/ophjlpahpchlmihnnnihgmmeilfjmjjc" \ + -maxdepth 2 -type f -name manifest.json -print +``` + +The canonical repo version is `pkg/line/client.go` `ExtensionVersion`. Also +check hardcoded `x-line-application` header strings in `pkg/line/client.go` and +the default `clientVersion` in `pkg/runner.go`; those must not drift from the +extension manifest version. The capture launcher prints repo and Chrome LINE +versions at startup and warns on mismatch. If versions differ, do not silently +mix evidence: warn the user and either update the repo version constants/headers +or record that the capture was taken from a different LINE Chrome Extension +version. + +## Security Rules + +Raw LINE captures are secrets. They may include access tokens, cookies, session +IDs, MIDs, contacts, message bodies, media object IDs, QR/login artifacts, and +E2EE metadata. + +- Use `/tmp/line-chrome-extension-capture/-` with `umask 077`. +- Prefer the persistent dedicated Chrome profile created by `scripts/line-chrome-capture.sh` + at `~/.cache/line-chrome-extension-capture/chrome-debug-profile`. +- Ask before using the user's normal Chrome profile. +- Ask before stopping a running bridge or invalidating an active LINE session. +- Keep remote debugging local and close Chrome after capture. +- If the user asks to set up the capture browser, installing the official LINE + Chrome Extension is in scope. Verify extension ID + `ophjlpahpchlmihnnnihgmmeilfjmjjc`, listing name LINE, and publisher/offered + by LY Corporation before clicking Add to Chrome and accepting the standard + "Add extension" confirmation. +- Do not request, store, or automate LINE credentials. +- Never ask the user to paste their LINE password into chat, command arguments, + files, or terminal output. +- If the capture browser is not logged in, pause and ask the user to complete + LINE login manually in Chrome, including email/password, QR, PIN, 2FA, and + mobile approval. +- Never commit raw captures, Chrome profiles, screenshots, QR codes, parsed JSON, + or redacted files unless the user explicitly asks for sanitized docs. +- Quote/document only redacted snippets. + +## Capture Commands + +Use CDP capture first because it includes SSE event bodies: + +```bash +umask 077 +CAPTURE_DIR="/tmp/line-chrome-extension-capture/$(date +%Y%m%d-%H%M%S)-feature-slug" +bash scripts/line-chrome-capture.sh "$CAPTURE_DIR" +``` + +The launcher stages clean unpacked copies of the LINE and Codex Chrome extension +directories from the user's normal Chrome profile when they are installed there, +then auto-loads those staged copies. When LINE is already installed in the +persistent capture profile, the launcher uses that installed copy and only +auto-loads other staged extensions; in that mode it must not pass +`--disable-extensions-except` for only the staged helpers, because that disables +the installed LINE extension. Override detection with +`LINE_CHROME_EXTENSION_DIR` or `CODEX_CHROME_EXTENSION_DIR` when needed. +If unpacked loading is blocked by Chrome, use the persistent capture profile to +open the official LINE Chrome Web Store listing and install it there; start this +setup run with `LINE_CAPTURE_AUTO_LOAD_EXTENSIONS=0` so Chrome is not also +loading a staged unpacked extension with the same ID. Accept the expected +extension confirmation only after the user has requested setup. +It also pre-seeds the dedicated capture profile so Chrome first-run/sign-in +prompts do not interrupt capture: default-browser opt-in is off, usage/crash +reporting is off, and Chrome/Sync stays signed out. If Chrome still shows the +sign-in screen, choose "Stay Signed Out" and continue to LINE login only. + +When Chrome closes: + +```bash +python3 scripts/parse-line-traffic.py "$CAPTURE_DIR/line-cdp-capture.json" "$CAPTURE_DIR/line-cdp-capture-parsed.json" +python3 scripts/redact-line-capture.py "$CAPTURE_DIR/line-cdp-capture-parsed.json" +``` + +Use the net-log only as a backup: + +```bash +python3 scripts/parse-line-traffic.py "$CAPTURE_DIR/line-net-log.json" "$CAPTURE_DIR/line-net-log-parsed.json" +python3 scripts/redact-line-capture.py "$CAPTURE_DIR/line-net-log-parsed.json" +``` + +## Analysis Checklist + +Build an evidence table with: + +- ordered requests and timestamps +- endpoint URL and service/method +- meaningful headers such as `x-lhm`, `x-lpv`, `x-lsr`, OBS headers, and content type +- request argument shape and exact order +- response status, wrapper code/message/data, and important headers +- SSE/operation event type, params, revisions, message metadata, LOC_KEYs, and + localRev/chat revision changes +- Chrome behavior vs bridge behavior +- implementation delta and files to change + +For Thrift payloads, prefer structured decoded data when available. If only +opaque binary/base64 is available, use the URL, service/method, headers, and +observable response/event behavior, then map to existing Go request builders. + +## Implementation Rules + +- Put LINE API/client behavior in `pkg/line`. +- Put Matrix bridge behavior and portal/message handling in `pkg/connector`. +- Keep E2EE behavior in `pkg/e2ee` and avoid editing generated `pkg/ltsm/wbc_generated.go`. +- Match Chrome's method names, endpoint paths, arg order, header semantics, + body shape, retry/fallback behavior, and event interpretation. +- Prefer existing request helpers and local patterns over new abstractions. +- Add protocol docs for new endpoint behavior using only redacted evidence. +- Add focused tests for request construction, response parsing, event handling, + persistence, and user-visible bridge behavior. + +## Verification + +Run the relevant subset, then the broader checks when practical: + +```bash +go fmt ./... +goimports -local "github.com/highesttt/matrix-line-messenger" -w . +go test ./... +go vet $(go list ./... | grep -v /ltsm) +staticcheck $(go list ./... | grep -v /ltsm) +``` + +If a command is unavailable or too slow, report that clearly. + +At the end, state where raw captures remain and ask whether to delete them. diff --git a/.agents/skills/line-implement/SKILL.md b/.agents/skills/line-implement/SKILL.md new file mode 100644 index 0000000..d4d0a8a --- /dev/null +++ b/.agents/skills/line-implement/SKILL.md @@ -0,0 +1,72 @@ +--- +name: line-implement +description: Plans and implements matrix-line bridge features by inspecting bridge code, deriving the missing LINE Chrome Extension evidence, securely capturing/analyzing traffic, and coding behavior to match Chrome. Use when implementing LINE protocol features, mimicking LINE Chrome Extension APIs, or investigating matrix-line behavior from network captures. +--- + +# Line Implement + +## Quick Start + +Use this for the Matrix LINE bridge when a feature goal depends on exact LINE +Chrome Extension behavior. + +1. Confirm the repo is `beeper-line`/`matrix-line-messenger`; read `AGENTS.md`. +2. Treat `.agents` as the repo-local source of truth. Read + `.agents/commands/line-implement.md` if present. `.claude/commands` may exist + only as a compatibility symlink to `.agents/commands`. +3. Load [REFERENCE.md](REFERENCE.md) for the full workflow. +4. Check LINE Chrome Extension version drift before capture: compare the repo + `pkg/line/client.go` `ExtensionVersion` and hardcoded LINE application + header versions against the installed extension `manifest.json`. +5. First inspect code and `docs/protocol`; capture only unknown behavior. +6. Treat all raw captures as secrets and work from redacted outputs whenever possible. + +## Required Loop + +1. Convert the user's feature goal into a capture contract. +2. Compare existing protocol docs and bridge implementation. +3. Use secure Chrome/CDP capture only for missing evidence. +4. Parse and redact captures before quoting, documenting, or sharing details. +5. Implement the bridge to match Chrome's endpoint, headers, argument shape, + response handling, events, and fallback behavior. +6. Run focused tests plus repo formatting/linting commands. + +## Automation + +For Chrome UI work, prefer the Codex Chrome connector when it can access the +needed browser/profile. If it cannot operate the LINE extension UI, use +Computer Use. + +The capture launcher should pre-skip Chrome first-run prompts. Keep the default +browser checkbox off, usage/crash reporting off, and Chrome/Sync signed out. If +Chrome still shows the sign-in page, choose "Stay Signed Out"; only LINE login is +manual. + +Before trusting a capture, check the installed LINE Chrome Extension version +against the repo version. The repo source is `pkg/line/client.go` +`ExtensionVersion`; also scan for literal `x-line-application` versions in +`pkg/line/client.go` and the default client version in `pkg/runner.go`. The +installed extension version comes from +`~/.cache/line-chrome-extension-capture/chrome-debug-profile/Default/Extensions/ophjlpahpchlmihnnnihgmmeilfjmjjc/*/manifest.json` +or the staged extension manifest. If they differ, warn the user and either +update the repo version constants/headers or note that the evidence is from a +different Chrome Extension version. + +The capture launcher stages clean unpacked LINE/Codex extension copies from the +user's normal Chrome profile before loading them, because Chrome Web Store +install directories contain metadata that unpacked-extension loading rejects. +If LINE is already installed in the persistent capture profile, the launcher +uses that installed copy instead of injecting a staged LINE copy. +When installing LINE from the Chrome Web Store into the persistent capture +profile, run the launcher with `LINE_CAPTURE_AUTO_LOAD_EXTENSIONS=0` so the +Web Store is not fighting an already command-line-loaded extension ID. + +If the LINE extension is missing from the capture Chrome profile and the user +has asked to set it up, open the official Chrome Web Store listing for extension +ID `ophjlpahpchlmihnnnihgmmeilfjmjjc` (LINE, offered by LY Corporation), click +Add to Chrome, and accept the standard "Add extension" confirmation. Do not +accept prompts for any other extension or publisher without a new user request. + +Do not request, store, or automate LINE credentials. If the extension is not +logged in, pause and ask the user to complete login manually in the capture +browser, including email/password, QR, PIN, 2FA, and mobile approval. diff --git a/.claude/commands b/.claude/commands new file mode 120000 index 0000000..1ea3574 --- /dev/null +++ b/.claude/commands @@ -0,0 +1 @@ +../.agents/commands \ No newline at end of file diff --git a/.claude/commands/line-capture.md b/.claude/commands/line-capture.md deleted file mode 100644 index 345cca8..0000000 --- a/.claude/commands/line-capture.md +++ /dev/null @@ -1,21 +0,0 @@ -Launch Chrome with network capture enabled for LINE Chrome Extension analysis. - -## What to do - -1. **Close any running Chrome instances first** (Chrome only supports one instance per profile with remote debugging). Warn the user if Chrome is running. - -2. Run the capture script: - ``` - bash scripts/line-chrome-capture.sh /tmp/line-capture - ``` - This launches Chrome with: - - Remote debugging on port 9222 - - Full network logging to `/tmp/line-capture/line-net-log.json` - - The user's default Chrome profile (LINE extension should already be installed) - -3. Tell the user: - - Open the LINE Chrome Extension (browser toolbar icon) - - Perform whatever flow they want to analyze - - When done, quit Chrome completely (Cmd+Q) - -4. Once Chrome is closed, the net-log file is ready. Tell the user to run `/line-analyze` next. diff --git a/scripts/line-cdp-capture.py b/scripts/line-cdp-capture.py index f94d5d4..142befe 100755 --- a/scripts/line-cdp-capture.py +++ b/scripts/line-cdp-capture.py @@ -363,7 +363,7 @@ def wait_for_chrome(timeout: int = 60) -> bool: def main(): - output_path = sys.argv[1] if len(sys.argv) > 1 else "/tmp/line-capture/line-cdp-capture.json" + output_path = sys.argv[1] if len(sys.argv) > 1 else "/tmp/line-chrome-extension-capture/line-cdp-capture.json" # Ensure output directory exists os.makedirs(os.path.dirname(output_path), exist_ok=True) diff --git a/scripts/line-chrome-capture.sh b/scripts/line-chrome-capture.sh index 778c5a7..101967e 100755 --- a/scripts/line-chrome-capture.sh +++ b/scripts/line-chrome-capture.sh @@ -7,9 +7,27 @@ # This starts Chrome with remote debugging and runs a CDP listener that # captures all LINE traffic including SSE EventSource message bodies. # -# Uses a separate Chrome profile at /chrome-debug-profile. -# On first run, you'll need to install the LINE extension from the Chrome -# Web Store and log in. The profile persists across runs. +# Uses a persistent separate Chrome profile at: +# ${LINE_CAPTURE_CHROME_PROFILE:-$HOME/.cache/line-chrome-extension-capture/chrome-debug-profile} +# On first run, you'll need to log in to LINE. The profile persists across +# capture output directories. +# +# The capture profile is pre-seeded to skip Chrome's first-run prompts: it does +# not make Chrome the default browser, does not send usage/crash reports, and +# stays signed out of Chrome/Sync. LINE login is still manual. +# +# If LINE/Codex Chrome extensions are already installed in your normal Chrome +# Default profile, this script stages clean unpacked copies into: +# ${LINE_CAPTURE_EXTENSION_CACHE_DIR:-$HOME/.cache/line-chrome-extension-capture/extensions} +# and auto-loads those copies into the capture profile. If LINE is already +# installed in the persistent capture profile, the script uses that installation +# instead of injecting a staged copy. Override detection with: +# LINE_CHROME_EXTENSION_DIR=/path/to/LINE/3.7.2_0 +# CODEX_CHROME_EXTENSION_DIR=/path/to/Codex/1.1.5_0 +# Disable staged extension auto-loading with: +# LINE_CAPTURE_AUTO_LOAD_EXTENSIONS=0 +# This is useful when installing LINE from the Chrome Web Store into the +# persistent capture profile. # # When done, close Chrome (Cmd+Q). The capture files will be at: # /line-cdp-capture.json — CDP capture (SSE + request bodies) @@ -18,13 +36,221 @@ set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -OUTPUT_DIR="${1:-/tmp/line-capture}" +OUTPUT_DIR="${1:-/tmp/line-chrome-extension-capture}" mkdir -p "$OUTPUT_DIR" NETLOG_FILE="$OUTPUT_DIR/line-net-log.json" CDP_FILE="$OUTPUT_DIR/line-cdp-capture.json" -DEBUG_PROFILE="$OUTPUT_DIR/chrome-debug-profile" +DEBUG_PROFILE="${LINE_CAPTURE_CHROME_PROFILE:-$HOME/.cache/line-chrome-extension-capture/chrome-debug-profile}" +EXTENSION_CACHE_DIR="${LINE_CAPTURE_EXTENSION_CACHE_DIR:-$HOME/.cache/line-chrome-extension-capture/extensions}" +AUTO_LOAD_EXTENSIONS="${LINE_CAPTURE_AUTO_LOAD_EXTENSIONS:-1}" + +seed_chrome_capture_profile() { + local profile_dir="$1" + local default_dir="$profile_dir/Default" + + mkdir -p "$default_dir" + chmod 700 "$profile_dir" 2>/dev/null || true + chmod 700 "$default_dir" 2>/dev/null || true + + # Chrome uses this sentinel for first-run completion. The JSON preferences + # below keep the choices explicit and readable for future maintainers. + touch "$profile_dir/First Run" + + python3 - "$profile_dir" <<'PY' +import json +import sys +from pathlib import Path + +profile = Path(sys.argv[1]) + +def merge(dst, src): + for key, value in src.items(): + if isinstance(value, dict) and isinstance(dst.get(key), dict): + merge(dst[key], value) + else: + dst[key] = value + return dst + +def update_json(path, values): + try: + data = json.loads(path.read_text()) if path.exists() else {} + if not isinstance(data, dict): + data = {} + except json.JSONDecodeError: + data = {} + merge(data, values) + path.write_text(json.dumps(data, indent=2, sort_keys=True) + "\n") + path.chmod(0o600) + +update_json(profile / "Local State", { + "browser": { + "check_default_browser": False, + "has_seen_welcome_page": True, + }, + "distribution": { + "import_bookmarks": False, + "import_history": False, + "make_chrome_default": False, + "skip_first_run_ui": True, + "suppress_first_run_default_browser_prompt": True, + }, + "signin": { + "allowed": False, + "allowed_on_next_startup": False, + }, + "sync": { + "suppress_start": True, + }, + "user_experience_metrics": { + "reporting_enabled": False, + }, +}) + +update_json(profile / "Default" / "Preferences", { + "browser": { + "check_default_browser": False, + "has_seen_welcome_page": True, + }, + "credentials_enable_autosignin": False, + "credentials_enable_service": False, + "distribution": { + "import_bookmarks": False, + "import_history": False, + "make_chrome_default": False, + "skip_first_run_ui": True, + "suppress_first_run_default_browser_prompt": True, + }, + "profile": { + "exit_type": "Normal", + "exited_cleanly": True, + "name": "LINE Capture", + }, + "signin": { + "allowed": False, + "allowed_on_next_startup": False, + }, + "sync": { + "requested": False, + "suppress_start": True, + }, +}) +PY +} + +find_latest_extension_dir() { + local extension_id="$1" + local base="$HOME/Library/Application Support/Google/Chrome/Default/Extensions/$extension_id" + if [ -d "$base" ]; then + find "$base" -mindepth 1 -maxdepth 1 -type d | sort -V | tail -n 1 + fi +} + +find_latest_profile_extension_dir() { + local profile_dir="$1" + local extension_id="$2" + local base="$profile_dir/Default/Extensions/$extension_id" + if [ -d "$base" ]; then + find "$base" -mindepth 1 -maxdepth 1 -type d | sort -V | tail -n 1 + fi +} + +read_manifest_version() { + local extension_dir="$1" + if [ -n "${extension_dir:-}" ] && [ -f "$extension_dir/manifest.json" ]; then + python3 - "$extension_dir/manifest.json" <<'PY' +import json +import sys +from pathlib import Path + +try: + print(json.loads(Path(sys.argv[1]).read_text()).get("version", "")) +except Exception: + print("") +PY + fi +} + +find_repo_extension_version() { + local client_file="$SCRIPT_DIR/../pkg/line/client.go" + if [ -f "$client_file" ]; then + python3 - "$client_file" <<'PY' +import re +import sys +from pathlib import Path + +match = re.search(r'ExtensionVersion\s*=\s*"([^"]+)"', Path(sys.argv[1]).read_text()) +print(match.group(1) if match else "") +PY + fi +} + +stage_extension_dir() { + local extension_id="$1" + local source_dir="$2" + + if [ -z "${source_dir:-}" ] || [ ! -f "$source_dir/manifest.json" ]; then + return 0 + fi + + local version + version="$(python3 - "$source_dir/manifest.json" <<'PY' +import json +import sys +from pathlib import Path + +try: + version = json.loads(Path(sys.argv[1]).read_text()).get("version", "unknown") +except Exception: + version = "unknown" +print("".join(c if c.isalnum() or c in ".-_" else "_" for c in str(version))) +PY +)" -CHROME="/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" + local dest="$EXTENSION_CACHE_DIR/$extension_id/$version" + mkdir -p "$dest" + + # Chrome Web Store extension directories contain _metadata/, which Chrome + # rejects when loading as unpacked. Stage a clean copy for capture runs. + if command -v rsync >/dev/null 2>&1; then + rsync -a --delete --exclude '_metadata' "$source_dir/" "$dest/" + else + local tmp="$dest.tmp" + rm -rf "$tmp" + mkdir -p "$tmp" + (cd "$source_dir" && tar --exclude './_metadata' -cf - .) | (cd "$tmp" && tar -xf -) + rm -rf "$dest" + mv "$tmp" "$dest" + fi + chmod -R go-rwx "$EXTENSION_CACHE_DIR" 2>/dev/null || true + + echo "$dest" +} + +LINE_EXTENSION_DIR="" +CODEX_EXTENSION_DIR="" +LINE_PROFILE_EXTENSION_DIR="$(find_latest_profile_extension_dir "$DEBUG_PROFILE" ophjlpahpchlmihnnnihgmmeilfjmjjc)" +LINE_PROFILE_EXTENSION_VERSION="$(read_manifest_version "$LINE_PROFILE_EXTENSION_DIR")" +if [ "$AUTO_LOAD_EXTENSIONS" != "0" ]; then + if [ -z "${LINE_PROFILE_EXTENSION_DIR:-}" ]; then + LINE_EXTENSION_SOURCE_DIR="${LINE_CHROME_EXTENSION_DIR:-$(find_latest_extension_dir ophjlpahpchlmihnnnihgmmeilfjmjjc)}" + LINE_EXTENSION_DIR="$(stage_extension_dir ophjlpahpchlmihnnnihgmmeilfjmjjc "$LINE_EXTENSION_SOURCE_DIR")" + fi + CODEX_EXTENSION_SOURCE_DIR="${CODEX_CHROME_EXTENSION_DIR:-$(find_latest_extension_dir hehggadaopoacecdllhhajmbjkdcmajg)}" + CODEX_EXTENSION_DIR="$(stage_extension_dir hehggadaopoacecdllhhajmbjkdcmajg "$CODEX_EXTENSION_SOURCE_DIR")" +fi +LINE_STAGED_EXTENSION_VERSION="$(read_manifest_version "$LINE_EXTENSION_DIR")" +LINE_CHROME_EXTENSION_VERSION="${LINE_PROFILE_EXTENSION_VERSION:-$LINE_STAGED_EXTENSION_VERSION}" +REPO_EXTENSION_VERSION="$(find_repo_extension_version)" + +EXTENSION_DIRS=() +if [ -n "${LINE_EXTENSION_DIR:-}" ] && [ -f "$LINE_EXTENSION_DIR/manifest.json" ]; then + EXTENSION_DIRS+=("$LINE_EXTENSION_DIR") +fi +if [ -n "${CODEX_EXTENSION_DIR:-}" ] && [ -f "$CODEX_EXTENSION_DIR/manifest.json" ]; then + EXTENSION_DIRS+=("$CODEX_EXTENSION_DIR") +fi + +CHROME="${CHROME:-/Applications/Google Chrome.app/Contents/MacOS/Google Chrome}" if [ ! -x "$CHROME" ]; then echo "Chrome not found at $CHROME" echo "Set CHROME env var to your Chrome binary path." @@ -36,15 +262,40 @@ if [ ! -d "$DEBUG_PROFILE" ]; then FIRST_RUN=true mkdir -p "$DEBUG_PROFILE" fi +seed_chrome_capture_profile "$DEBUG_PROFILE" echo "==> Starting Chrome with CDP + net-log capture" echo " CDP output: $CDP_FILE" echo " Net-log: $NETLOG_FILE" echo " Profile: $DEBUG_PROFILE" +if [ -n "${LINE_PROFILE_EXTENSION_DIR:-}" ]; then + echo " LINE profile extension: $LINE_PROFILE_EXTENSION_DIR" +fi +if [ -n "${REPO_EXTENSION_VERSION:-}" ]; then + echo " Repo LINE version: $REPO_EXTENSION_VERSION" +fi +if [ -n "${LINE_CHROME_EXTENSION_VERSION:-}" ]; then + echo " Chrome LINE version: $LINE_CHROME_EXTENSION_VERSION" +fi +if [ -n "${REPO_EXTENSION_VERSION:-}" ] && [ -n "${LINE_CHROME_EXTENSION_VERSION:-}" ] && [ "$REPO_EXTENSION_VERSION" != "$LINE_CHROME_EXTENSION_VERSION" ]; then + echo " WARNING: repo ExtensionVersion ($REPO_EXTENSION_VERSION) does not match Chrome extension ($LINE_CHROME_EXTENSION_VERSION)." +fi +if [ ${#EXTENSION_DIRS[@]} -gt 0 ]; then + echo " Extensions:" + for ext_dir in "${EXTENSION_DIRS[@]}"; do + echo " - $ext_dir" + done +else + echo " Extensions: none auto-loaded" +fi echo "" if [ "$FIRST_RUN" = true ]; then - echo " FIRST RUN: Install LINE extension from Chrome Web Store and log in." - echo " The profile persists — you only need to do this once." + echo " FIRST RUN: Chrome first-run/sign-in prompts are pre-skipped." + echo " Log in to LINE manually in the capture browser." + if [ ${#EXTENSION_DIRS[@]} -eq 0 ]; then + echo " LINE extension was not auto-detected; install it from the Chrome Web Store first." + fi + echo " The profile persists across capture dirs — you only need to do this once." echo "" fi echo " 1. Open the LINE Chrome Extension and do your flow" @@ -52,13 +303,40 @@ echo " 2. When done, close Chrome (Cmd+Q)" echo "" # Chrome 146+ requires --user-data-dir for remote debugging. -"$CHROME" \ - --remote-debugging-port=9222 \ - --remote-allow-origins="*" \ - --user-data-dir="$DEBUG_PROFILE" \ - --log-net-log="$NETLOG_FILE" \ - --net-log-capture-mode=IncludeSensitive \ - 2>/dev/null & +CHROME_ARGS=( + --no-first-run + --no-default-browser-check + --disable-sync + --disable-features=SignInPromo,SigninInterception,ChromeSigninIntercept,DiceWebSigninInterception + --remote-debugging-port=9222 + --remote-allow-origins="*" + --user-data-dir="$DEBUG_PROFILE" + --log-net-log="$NETLOG_FILE" + --net-log-capture-mode=IncludeSensitive +) + +if [ ${#EXTENSION_DIRS[@]} -gt 0 ]; then + IFS=, + EXTENSION_ARG="${EXTENSION_DIRS[*]}" + unset IFS + if [ -z "${LINE_PROFILE_EXTENSION_DIR:-}" ]; then + CHROME_ARGS+=(--disable-extensions-except="$EXTENSION_ARG") + fi + CHROME_ARGS+=(--load-extension="$EXTENSION_ARG") +fi + +START_URL="${LINE_CAPTURE_START_URL:-}" +if [ -z "$START_URL" ] && [ -n "${LINE_PROFILE_EXTENSION_DIR:-}" ] && [ -f "$LINE_PROFILE_EXTENSION_DIR/manifest.json" ]; then + START_URL="chrome-extension://ophjlpahpchlmihnnnihgmmeilfjmjjc/index.html" +fi +if [ -z "$START_URL" ] && [ -n "${LINE_EXTENSION_DIR:-}" ] && [ -f "$LINE_EXTENSION_DIR/manifest.json" ]; then + START_URL="chrome-extension://ophjlpahpchlmihnnnihgmmeilfjmjjc/index.html" +fi +if [ -z "$START_URL" ]; then + START_URL="about:blank" +fi + +"$CHROME" "${CHROME_ARGS[@]}" "$START_URL" 2>/dev/null & CHROME_PID=$! # Start the CDP capture (it waits for Chrome to be ready) diff --git a/scripts/redact-line-capture.py b/scripts/redact-line-capture.py new file mode 100755 index 0000000..8c20f48 --- /dev/null +++ b/scripts/redact-line-capture.py @@ -0,0 +1,143 @@ +#!/usr/bin/env python3 +"""Redact sensitive values from LINE capture JSON files. + +Usage: + python3 scripts/redact-line-capture.py input.json [output.json] + +The default output path is input-redacted.json. This helper preserves JSON +structure for protocol analysis while removing credentials, cookies, tokens, +session identifiers, email addresses, and obvious long secrets. +""" + +from __future__ import annotations + +import hashlib +import json +import os +import re +import sys +from typing import Any + + +SENSITIVE_KEYS = { + "authorization", + "cookie", + "set-cookie", + "x-hmac", + "x-line-access", + "x-line-channeltoken", + "x-line-next-access-token", + "x-line-refresh-token", + "x-line-session-id", + "x-obs-params", + "password", + "passwd", + "email", + "mail", + "token", + "access_token", + "refresh_token", + "session", + "certificate", +} + +SENSITIVE_KEY_FRAGMENTS = ( + "auth", + "cookie", + "credential", + "password", + "secret", + "session", + "token", +) + +EMAIL_RE = re.compile(r"(?i)[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,}") +JWT_RE = re.compile(r"eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}") +MID_RE = re.compile(r"\b([ucr][0-9a-f]{20,})\b", re.IGNORECASE) +LONG_SECRET_RE = re.compile(r"^[A-Za-z0-9+/=_-]{96,}$") + + +def marker(kind: str, value: str) -> str: + digest = hashlib.sha256(value.encode("utf-8", errors="ignore")).hexdigest()[:12] + return f"" + + +def key_is_sensitive(key: str) -> bool: + normalized = key.lower() + if normalized in SENSITIVE_KEYS: + return True + return any(fragment in normalized for fragment in SENSITIVE_KEY_FRAGMENTS) + + +def redact_string(value: str, force: str | None = None) -> str: + if not value: + return value + if force: + return marker(force, value) + + value = JWT_RE.sub(lambda match: marker("jwt", match.group(0)), value) + value = EMAIL_RE.sub(lambda match: marker("email", match.group(0)), value) + value = MID_RE.sub(lambda match: match.group(1)[0].lower() + marker("mid", match.group(1)), value) + + stripped = value.strip() + if LONG_SECRET_RE.match(stripped): + return marker("opaque", stripped) + return value + + +def redact(value: Any, parent_key: str = "") -> Any: + if isinstance(value, dict): + redacted = {} + for key, child in value.items(): + key_text = str(key) + if key_is_sensitive(key_text): + if isinstance(child, (dict, list)): + redacted[key] = marker(key_text.lower(), json.dumps(child, sort_keys=True, default=str)) + else: + redacted[key] = redact_string(str(child), key_text.lower()) + else: + redacted[key] = redact(child, key_text) + return redacted + + if isinstance(value, list): + return [redact(item, parent_key) for item in value] + + if isinstance(value, str): + return redact_string(value) + + return value + + +def default_output_path(input_path: str) -> str: + base, ext = os.path.splitext(input_path) + return f"{base}-redacted{ext or '.json'}" + + +def main() -> int: + if len(sys.argv) not in (2, 3): + print(__doc__, file=sys.stderr) + return 1 + + input_path = sys.argv[1] + output_path = sys.argv[2] if len(sys.argv) == 3 else default_output_path(input_path) + + with open(input_path, "r", encoding="utf-8") as infile: + data = json.load(infile) + + redacted = redact(data) + + with open(output_path, "w", encoding="utf-8") as outfile: + json.dump(redacted, outfile, indent=2, ensure_ascii=False) + outfile.write("\n") + + try: + os.chmod(output_path, 0o600) + except OSError: + pass + + print(f"Redacted capture written to: {output_path}", file=sys.stderr) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main())