Skip to content

server: Normalize pixel format depth from client - #165

Merged
any1 merged 1 commit into
any1:masterfrom
alistairreay:fix/cpixel-calculation
Feb 22, 2026
Merged

server: Normalize pixel format depth from client#165
any1 merged 1 commit into
any1:masterfrom
alistairreay:fix/cpixel-calculation

Conversation

@alistairreay

@alistairreay alistairreay commented Feb 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Some VNC clients (e.g. macOS Screen Sharing) report depth=32 in their
SetPixelFormat message for standard RGB888 pixel formats. This causes
calc_bytes_per_cpixel to return 4 instead of 3, corrupting every ZRLE
tile -- the encoder writes 4 bytes per pixel but the client expects 3.

This patch normalizes the depth field using rfb_pixfmt_depth() when
receiving the client's pixel format in on_client_set_pixel_format().
This derives the true depth from the RGB max values (via popcount),
ensuring correct CPIXEL encoding downstream without modifying the
encoding logic itself.

Testing

Tested with macOS Screen Sharing (which sends depth=32) connecting to
wayvnc with ZRLE encoding. Before the fix, the display showed corrupted
flashing tiles. After the fix, the display renders correctly.

Also verified that existing clients sending depth=24 are unaffected --
rfb_pixfmt_depth() returns 24 for standard RGB888, same as what those
clients already report.

I have read and understood CONTRIBUTING.md

@any1

any1 commented Feb 20, 2026

Copy link
Copy Markdown
Owner

Did you see this? #160 (comment)

Some VNC clients (e.g. macOS Screen Sharing) report depth=32 in
their SetPixelFormat message for standard RGB888 pixel formats.
This causes calc_bytes_per_cpixel to return 4 instead of 3,
corrupting ZRLE tile data.

Normalize the depth field using rfb_pixfmt_depth() when receiving
the client's pixel format. This derives the true depth from the
RGB max values, ensuring correct CPIXEL encoding downstream.
@alistairreay
alistairreay force-pushed the fix/cpixel-calculation branch from 74f6621 to c5de84f Compare February 20, 2026 18:12
@alistairreay alistairreay changed the title enc: Fix CPIXEL size calculation per RFB spec server: Normalize pixel format depth from client Feb 20, 2026
@alistairreay

Copy link
Copy Markdown
Contributor Author

Did you see this? #160 (comment)

Ahh yes, thanks for pointing that out. I've reworked the fix to normalize depth using rfb_pixfmt_depth() in on_client_set_pixel_format() instead of modifying the encoding logic. Should be a much cleaner approach.

@any1
any1 merged commit 8c646d0 into any1:master Feb 22, 2026
@any1

any1 commented Feb 22, 2026

Copy link
Copy Markdown
Owner

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants