Skip to content

fix(screen): negotiate Linux capture video - #104

Merged
leepokai merged 3 commits into
leepokai:devfrom
fergusean:fergusean/linux-screen-codecs
Oct 15, 2026
Merged

leepokai merged 3 commits into
leepokai:devfrom
fergusean:fergusean/linux-screen-codecs

Conversation

@fergusean

@fergusean fergusean commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Linux remote screens use the H.264 payload offered by the viewer, feed encoders 8-bit 4:2:0 video, and keep PipeWire capture timestamps on the system clock. Negotiate the answer before starting capture so the encoder accepts a Baseline offer before fixing its profile.

Split these capture/codec corrections from #40 as requested in its review. This PR uses existing complete-SDP negotiation and introduces no trickle signaling, client UI, version or lockfile changes. macOS/Windows capture pipelines are unaffected because these changes are confined to the Linux GStreamer helper.

Validation

  • Rebased onto upstream dev at dd141b99 (2.15.2), including merged fix(host): restore baseline CI checks #103. The feature patch is unchanged and fresh Linux helper CI passes.
  • Native Linux formatting, all-target Clippy, tests and release build pass.
  • Regression coverage accepts a complete-SDP H.264 offer using payload 102, verifies the selected offered payload, and encodes RGB capture as 8-bit 4:2:0.
  • Documentation links/fences and whitespace checks pass.
  • Real-device acceptance passes: a Linux helper with source files matching this PR streamed its 1280×800 display through complete-SDP fallback to the actual iPhone app. The viewer decoded H.264 using the x264 pipeline and displayed test-pattern changes after stationary intervals. Native portal capture/input permission is granted. The exact live RTP payload number was not collected; non-default payload 102 is covered by the SDP regression above.

The broader forced-relay, UDP-blocked TURN and mixed-version acceptance matrix belongs to #40.

What's New

zh-Hant

en-US

@fergusean
fergusean marked this pull request as ready for review October 13, 2026 05:29
@fergusean
fergusean force-pushed the fergusean/linux-screen-codecs branch from 2ddfa58 to 7399d17 Compare October 13, 2026 09:29
@leepokai

leepokai commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

Hi @fergusean, thank you for all your contributions to Codync so far! We just opened a Discord server for Codync: https://discord.gg/JzMXd7gHAJ

It should make it much easier to talk through PRs and plans going forward, so it'd be great to have you there. This link gives you the Contributor role automatically when you join.

@fergusean
fergusean force-pushed the fergusean/linux-screen-codecs branch from 7399d17 to e9340b7 Compare October 13, 2026 18:57
fergusean and others added 2 commits October 14, 2026 15:50
Use the viewer's H.264 payload and 8-bit 4:2:0 encoder input, negotiate
the answer before capture starts, and keep PipeWire timestamps on the
system clock. These fixes work with complete SDP independently of
trickle ICE and are split from PR leepokai#40 as requested.

Co-authored-by: Codex <codex@openai.com>
The codec regressions exercise real GStreamer pipelines, so install
their base, good, bad, ugly and ICE runtime plugins on the clean runner.

Co-authored-by: Codex <codex@openai.com>
@fergusean
fergusean force-pushed the fergusean/linux-screen-codecs branch from e9340b7 to 3cc5a52 Compare October 14, 2026 20:17
Setting pipewiresrc's provide-clock property panicked on every PipeWire
before 1.6 (Ubuntu 24.04 ships 1.0), so the helper never answered and
the phone couldn't connect. The pipeline now picks the system clock
itself, which keeps capture timestamps on it with every version, and a
test builds the capture source on the installed PipeWire (CI now
installs its GStreamer plugin).

Also accept only dynamic payload numbers, the only ones rtph264pay
takes, and make the docs note a list item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDC5WbCSUif2ysBUruDTzi
@leepokai
leepokai merged commit dc1794e into leepokai:dev Oct 15, 2026
1 check passed
@leepokai

Copy link
Copy Markdown
Owner

Merged, thanks @fergusean! The payload, 4:2:0 and answer-before-capture fixes are all right, and it ships in 2.17.2.

Before merging I pushed one fix on top (39dabfd): pipewiresrc only has provide-clock from PipeWire 1.6, and on older ones the element builder panics on the unknown property. On Ubuntu 24.04 (PipeWire 1.0.5) every answer panicked, so the phone never got a reply. I reproduced that in an Ubuntu 24.04 VM. The pipeline now uses the system clock itself (use_clock), which keeps your timestamp fix on every PipeWire version. A new test builds the capture source on the installed PipeWire, and CI now installs gstreamer1.0-pipewire so it runs there. I also limited the payload to the dynamic range (96–127), the only numbers rtph264pay takes, and made the docs note a list item.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants