Skip to content

feat: Session Replay and Bridge Capability Mapping from Recorded User Flows #19

Description

@Delqhi

Summary

OpenSIN-Bridge should not only record sessions but also map them to what the bridge can already automate and where the gaps are.

Problem

We currently lack a feedback loop that answers:

  • which recorded steps are already bridge-native
  • which need new tools
  • which need agent-side orchestration
  • which fail because of shadow DOM, iframe, auth, or anti-bot constraints

Goal

Build a replay/mapping layer that:

  1. replays candidate steps in dry-run / review mode
  2. marks each step as bridge_supported, bridge_partial, or bridge_missing
  3. generates implementation hints for missing capabilities

Suggested Output

  • per-step support classification
  • required bridge tools
  • required new issues / missing tool suggestions
  • replay evidence (DOM diff / screenshot diff / network diff)

Repo Evidence

  • Existing observe, snapshot, page_diff, click_ref, type_ref, vision_* tools
  • Existing recording/video + request log fragments
  • Existing bridge/server split suitable for capability mapping

Acceptance Criteria

  • Recorded session can be replayed in review mode
  • Each step gets a capability mapping status
  • Missing bridge capabilities are surfaced as structured gaps
  • Output is suitable for agent planning / future GitHub issue generation

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions