diff --git a/CHANGELOG.md b/CHANGELOG.md index 10544d89..f20c9e6b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,22 @@ # Changelog +## 0.8.0 + +Use the latest [WhateverXe Graphics](https://github.com/b00t0x/WhateverXe-Graphics/releases) package with this update. + +### Added + +- Add Raptor Lake-U graphics support. +- Add experimental system sleep/wake support, tested on MIPI DSI laptop configurations with Big Sur and Tahoe and model-specific ACPI settings. +- Add an optional 60 Hz mode for matching MIPI DSI panels in MiniBook X N100 and U300. + +### Fixed + +- Fix a regression introduced in 0.7.0 that disabled graphics acceleration on some supported Tiger Lake configurations. +- Fix restart and shutdown with the supported MIPI DSI panels, and accelerated startup with USB-C displays already connected. +- Fix rotated MIPI DSI display-mode selection and improve display restoration after sleep. +- Improve compatibility with supported macOS updates when kernel or system-driver code placement changes. + ## 0.7.0 Use the latest [WhateverXe Graphics](https://github.com/b00t0x/WhateverXe-Graphics/releases) package with this update. diff --git a/README.md b/README.md index bd1de692..d7fad6af 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,7 @@ Sequoia and Tahoe accept compatible updates within their OS family. Future Apple - Twin Lake: 32-EU configurations. - Alder Lake-H: 96-EU configurations, including single-channel operation. - Alder Lake-S: UHD Graphics 770. +- Raptor Lake-U: 48-EU configurations. See [tested features and known issues](#tested-features-and-known-issues) for OS coverage and display configurations. PCI IDs are listed in [device selection](docs/device-selection.md). @@ -32,13 +33,12 @@ See [tested features and known issues](#tested-features-and-known-issues) for OS - Other supported Tiger Lake-U/H35 configurations, including 96 EU, and Tiger Lake-H45. - Other supported Alder Lake-N / Twin Lake configurations. -- Other supported Alder Lake-H/P/U and Raptor Lake-H/P configurations. +- Other supported Alder Lake-H/P/U and Raptor Lake-H/P/U configurations. - Other supported Alder Lake-S/HX and Raptor Lake-S/HX configurations. -- Internal eDP on Alder Lake-N / Twin Lake and Alder Lake-H/P/U / Raptor Lake-H/P is implemented but has not been tested on hardware. +- Internal eDP on Alder Lake-N / Twin Lake and Alder Lake-H/P/U / Raptor Lake-H/P/U is implemented but has not been tested on hardware. ### Further implementation required -- Raptor Lake-U: dedicated display-clock handling is not implemented. - Rocket Lake-S: display control requires additional support. ### No support planned @@ -82,7 +82,7 @@ These settings are unnecessary at 96MB and do not increase firmware-allocated me - Add `SSDT-PNLF` for internal-display brightness control. - GPU selection is automatic; a `device-id` override is normally unnecessary. -- Alder Lake-N / Twin Lake and Alder Lake-H/P/U / Raptor Lake-H/P select tiling automatically. Do not use `-yTileDisable`; `AAPL,GfxYTile` is unnecessary. +- Alder Lake-N / Twin Lake and Alder Lake-H/P/U / Raptor Lake-H/P/U select tiling automatically. Do not use `-yTileDisable`; `AAPL,GfxYTile` is unnecessary. ## Tested features and known issues @@ -93,25 +93,27 @@ The common regression checks cover Metal, OpenGL, OpenCL, hardware H.264 encode/ | Tiger Lake mini PCs | Big Sur–Tahoe: 80 EU, two 1080p60 displays, display off/on, Safari video and Widevine | | Tiger Lake laptops | Big Sur–Tahoe: 48 EU, internal eDP and brightness, HDMI/USB-C, up to three displays and display off/on | | Alder Lake-N mini PCs | Big Sur–Tahoe: 16/24 EU, two 1080p60 HDMI displays and display off/on | -| Alder Lake-N / Twin Lake laptops | MIPI DSI on MicroPC 2 (N250) and MiniBook X N100. Big Sur: brightness, rotation, display off/on and external HDMI/USB-C coexistence; MicroPC 2 also has HiDPI and up to three displays. Tahoe: MicroPC 2 internal/external displays, brightness, rotation and HiDPI | +| Alder Lake-N / Twin Lake laptops | MicroPC 2 and MiniBook X N100 on Big Sur/Tahoe: MIPI DSI, brightness, rotation, display off/on and HDMI/USB-C coexistence. MicroPC 2: HiDPI and up to three displays; MiniBook X N100: optional 60 Hz | +| Raptor Lake-U laptop | MiniBook X U300 on Big Sur/Tahoe: internal MIPI DSI, brightness, rotation, HiDPI and up to two USB-C displays. Internal panel: 50 Hz; optional 60 Hz also tested on Big Sur | | Alder Lake-H | Big Sur–Tahoe: 96 EU, three displays using HDMI and a DP-to-HDMI adapter. Single-channel `4626` graphics acceleration checked on Big Sur | | Alder Lake-S | Big Sur–Tahoe: UHD 770, HDMI/DP/USB-C and up to three displays; Safari video, with Widevine checked on Big Sur/Tahoe | -MIPI DSI support is **experimental and tested only on GPD MicroPC 2 and CHUWI MiniBook X N100**. Do not expect it to work on other devices. +MIPI DSI support is **experimental**, limited to the tested GPD MicroPC 2 and CHUWI MiniBook X N100/U300 panels. Do not expect it to work on other devices. + +System sleep/wake support is **experimental**. Tested configurations are MiniBook X N100/U300 (MIPI DSI) on Big Sur and Tahoe with model-specific ACPI settings, using the internal display alone or up to two external displays. Other models, OS versions and prolonged DarkWake cycles remain unverified. ### Known issues and limits - DisplayPort reconnection can cause black screens or freezes. Tiger Lake USB-C reconnection is also unreliable; Alder Lake-S DP output/reconnection can be slow. - Some image-conversion operations can crash applications; App Store video can freeze the GPU on Sequoia. - Safari video may occasionally drop frames. FairPlay-protected playback is unsupported. -- Up to three displays are supported. Resolutions beyond the tested configurations and full system sleep/wake remain unverified. +- Up to three displays are supported. Resolutions beyond the tested configurations remain unverified. - Some connector layouts and backlight methods, MST and Thunderbolt-managed outputs are unsupported. ### Roadmap -- Implement Raptor Lake-U display-clock support. -- Complete DisplayPort, USB-C and internal-display support on Alder Lake-N / Twin Lake and Alder Lake-H/P/U / Raptor Lake-H/P. -- Improve display reconnection reliability and investigate full system sleep/wake. +- Complete DisplayPort, USB-C and internal-display support on Alder Lake-N / Twin Lake and Alder Lake-H/P/U / Raptor Lake-H/P/U. +- Improve display reconnection reliability and extend system sleep/wake coverage. ## Build and test @@ -132,3 +134,5 @@ See [LICENSE.txt](LICENSE.txt). Lilu and MacKernelSDK are separate dependencies. Thanks to [JulioNectar/XPS13Plus](https://github.com/JulioNectar/Hackintosh-XPS13Plus-IrisXe-Tahoe) for sharing its Xe graphics work. Its compiler adapter and LLVM bridge informed the OS-native AIR conversion used in WhateverXe Graphics. Internal-panel work also draws on [WhateverXe-RaptorLakeP](https://github.com/samranahmad09/WhateverXe-RaptorLakeP), [X1Carbon-G11](https://github.com/dssoftx/Hackintosh-X1Carbon-G11-IrisXe-Tahoe), XPS13Plus and the [Linux i915 display sources](https://github.com/torvalds/linux/tree/v6.12/drivers/gpu/drm/i915/display) for hardware behavior and implementation comparisons. + +The optional MiniBook X 60 Hz panel timing also draws on the [minibookx-ctl display guide](https://github.com/ishad0w/minibookx-ctl/blob/main/docs/guide/display.md). diff --git a/README_ja.md b/README_ja.md index dd365876..493ecc5d 100644 --- a/README_ja.md +++ b/README_ja.md @@ -25,6 +25,7 @@ SequoiaとTahoeは同じOS系列内の互換性のある更新に対応します - Twin Lake：32 EU構成。 - Alder Lake-H：シングルチャネルを含む96 EU構成。 - Alder Lake-S：UHD Graphics 770。 +- Raptor Lake-U：48 EU構成。 OS別の範囲と表示構成は[動作確認済みの機能と既知の問題](#動作確認済みの機能と既知の問題)、PCI IDは[デバイス選択](docs/device-selection.md)を参照してください。 @@ -32,13 +33,12 @@ OS別の範囲と表示構成は[動作確認済みの機能と既知の問題]( - 96 EUを含む対応IDの他のTiger Lake-U/H35構成と、Tiger Lake-H45。 - 対応IDの他のAlder Lake-N / Twin Lake構成。 -- 対応IDの他のAlder Lake-H/P/U・Raptor Lake-H/P構成。 +- 対応IDの他のAlder Lake-H/P/U・Raptor Lake-H/P/U構成。 - 対応IDの他のAlder Lake-S/HX・Raptor Lake-S/HX構成。 -- Alder Lake-N / Twin LakeとAlder Lake-H/P/U / Raptor Lake-H/Pの内蔵eDPは実装済み・実機未検証です。 +- Alder Lake-N / Twin LakeとAlder Lake-H/P/U / Raptor Lake-H/P/Uの内蔵eDPは実装済み・実機未検証です。 ### 対応に追加改修が必要 -- Raptor Lake-U：専用の表示クロック制御が未実装です。 - Rocket Lake-S：表示制御に追加対応が必要です。 ### 対応予定なし @@ -82,7 +82,7 @@ BIOS/UEFIで設定できる場合は、**DVMT Pre-Allocatedを96MB**にします - 内蔵画面の輝度調整には`SSDT-PNLF`を追加します。 - GPUは自動選択するため、通常は`device-id`上書きは不要です。 -- Alder Lake-N / Twin LakeとAlder Lake-H/P/U / Raptor Lake-H/Pではタイル形式を自動選択します。`-yTileDisable`は使わないでください。`AAPL,GfxYTile`は不要です。 +- Alder Lake-N / Twin LakeとAlder Lake-H/P/U / Raptor Lake-H/P/Uではタイル形式を自動選択します。`-yTileDisable`は使わないでください。`AAPL,GfxYTile`は不要です。 ## 動作確認済みの機能と既知の問題 @@ -93,25 +93,27 @@ BIOS/UEFIで設定できる場合は、**DVMT Pre-Allocatedを96MB**にします | Tiger LakeミニPC | Big Sur〜Tahoe：80 EU、1080p60の2画面、消灯復帰、Safari動画・Widevine | | Tiger Lakeラップトップ | Big Sur〜Tahoe：48 EU、内蔵eDPと輝度調整、HDMI/USB-C、最大3画面・消灯復帰 | | Alder Lake-NミニPC | Big Sur〜Tahoe：16/24 EU、1080p60のHDMI 2画面・消灯復帰 | -| Alder Lake-N / Twin Lakeラップトップ | MicroPC 2（N250）とMiniBook X N100のMIPI DSI。Big Sur：輝度・回転・消灯復帰・外部HDMI/USB-C共存。MicroPC 2はHiDPI・最大3画面も確認。Tahoe：MicroPC 2の内蔵／外部表示・輝度・回転・HiDPI | +| Alder Lake-N / Twin Lakeラップトップ | Big Sur・Tahoe：MicroPC 2・MiniBook X N100のMIPI DSI・輝度・回転・消灯復帰・HDMI/USB-C共存。MicroPC 2はHiDPI・最大3画面、MiniBook X N100は任意設定の60 Hzも確認 | +| Raptor Lake-Uラップトップ | Big Sur・Tahoe：MiniBook X U300の内蔵MIPI DSI・輝度・回転・HiDPI・USB-C外部2画面。内蔵は50 Hz、Big Surでは任意設定の60 Hzも確認 | | Alder Lake-H | Big Sur〜Tahoe：96 EU、HDMIとDP→HDMI変換器による3画面。シングルチャネル構成の`4626`でもBig Surの加速を確認 | | Alder Lake-S | Big Sur〜Tahoe：UHD 770、HDMI/DP/USB-C、最大3画面、Safari動画。WidevineはBig Sur/Tahoeで確認 | -MIPI DSIは**GPD MicroPC 2とCHUWI MiniBook X N100のみで検証した試験的対応**です。それ以外の機器での動作は期待しないでください。 +MIPI DSI対応は**試験的**で、検証したGPD MicroPC 2・CHUWI MiniBook X N100/U300の内蔵パネルに限ります。他の機器での動作を期待しないでください。 + +システム全体のスリープ／復帰対応は**試験的**です。実機確認済みの構成は、機種別ACPI設定を使用したMiniBook X N100/U300（MIPI DSI）のBig Sur・Tahoeで、内蔵単独と外部2画面までです。他機種・他OSと長時間のDarkWake反復は未検証です。 ### 既知の問題と制限 - DisplayPortの再接続で黒画面・停止が起きる場合があります。Tiger LakeのUSB-C再接続にも未解決の問題があり、Alder Lake-SのDP表示開始・再接続は時間がかかる場合があります。 - 一部の画像変換でアプリがクラッシュし、SequoiaのApp Store動画ではGPUが停止することがあります。 - Safari動画で時折コマ落ちします。FairPlay保護コンテンツは未対応です。 -- 同時表示は最大3画面です。検証構成を超える解像度とシステム全体のスリープ／復帰は未検証です。 +- 同時表示は最大3画面です。検証構成を超える解像度は未検証です。 - 一部の端子配線・輝度制御方式、MST、Thunderbolt管理の出力は未対応です。 ### ロードマップ -- Raptor Lake-Uの表示クロック対応。 -- Alder Lake-N / Twin LakeとAlder Lake-H/P/U / Raptor Lake-H/PでのDisplayPort・USB-C・内蔵画面対応の拡充。 -- 画面再接続の安定性改善とシステム全体のスリープ／復帰の調査。 +- Alder Lake-N / Twin LakeとAlder Lake-H/P/U / Raptor Lake-H/P/UでのDisplayPort・USB-C・内蔵画面対応の拡充。 +- 画面再接続の安定性改善とシステム全体のスリープ／復帰の対応範囲拡充。 ## ビルドとテスト @@ -131,3 +133,5 @@ WhateverXeはWhateverGreenの著作権表示とBSD-3-Clauseライセンスを継 Xeグラフィックスの取り組みを公開している[JulioNectar/XPS13Plus](https://github.com/JulioNectar/Hackintosh-XPS13Plus-IrisXe-Tahoe)に感謝します。同リポジトリのcompiler adapterとLLVM bridgeを、WhateverXe GraphicsのOS標準AIR変換の参考にしました。 内蔵パネルの対応では、[WhateverXe-RaptorLakeP](https://github.com/samranahmad09/WhateverXe-RaptorLakeP)、[X1Carbon-G11](https://github.com/dssoftx/Hackintosh-X1Carbon-G11-IrisXe-Tahoe)、XPS13Plus、[Linux i915の表示ソース](https://github.com/torvalds/linux/tree/v6.12/drivers/gpu/drm/i915/display)も、ハードウェアの挙動と実装の比較に参照しています。 + +MiniBook Xの任意設定60 Hzのパネルタイミングには、[minibookx-ctlの表示ガイド](https://github.com/ishad0w/minibookx-ctl/blob/main/docs/guide/display.md)も参照しています。 diff --git a/WhateverXe/kern_display13_boot.hpp b/WhateverXe/kern_display13_boot.hpp index 612bba77..cd1175a8 100644 --- a/WhateverXe/kern_display13_boot.hpp +++ b/WhateverXe/kern_display13_boot.hpp @@ -125,13 +125,14 @@ struct NormalizeScratch { }; template Result normalize(IO &io,const AlderLakeN::DisplayReadout &boot,bool admitted, const Display13HDMIWiring &w,const XELPMemory::Layout &memory, - NormalizeScratch &scratch) { + NormalizeScratch &scratch,const XELP::DSI::Readout *dsi=nullptr) { Result r;r.before=boot.activePipes; if(!admitted || &boot==&scratch.after)return r; r.status=Status::Topology; const auto active=boot.activePipes; if(w.panel || !memory.valid || !Display13FBTopology::multiOutputWiring(w) || boot.status!=AlderLakeN::ReadoutStatus::Complete || - !(active&1) || active==1 || (active&~7U) || (active&~boot.poweredPipes))return r; + (dsi ? (w.dsi!=1 || !active || (active&1) || dsi->identity.pipe!=0 || !XELP::DSI::sourceUnchanged(io,*dsi)) + : (w.dsi || !(active&1) || active==1)) || (active&~7U) || (active&~boot.poweredPipes))return r; uint32_t ports[3] {},clocks[3] {},transport[3] {},used=0,dpPipes=0; auto &tc=scratch.tc; for(auto &port:tc)port={}; @@ -144,7 +145,7 @@ template Result normalize(IO &io,const AlderLakeN::DisplayReadout &boo for(unsigned p=0;p<3;++p)if(boot.poweredPipes&(1U< Result normalize(IO &io,const AlderLakeN::DisplayReadout &boo uint32_t ctl=0,trans=0,surface=0,live=0; if(!AlderLakeN::value(boot,0x70180+bank,ctl) || !AlderLakeN::value(boot,0x60400+bank,trans))return r; const auto port=Display13FBTopology::bootDDI(w,trans,true); - if(port>=7 || (used&(1U<=7 || (used&(1U<>24)&7)==2; if(port>=3) { - tc[p]=Display13TC::Boot::capture(io,true,port,p,dp); + const bool alt=dsi && (w.typeC&(1U< Result normalize(IO &io,const AlderLakeN::DisplayReadout &boo const auto peer=[&] { // These are borrowed firmware requests. Keep them unchanged, including // the stopped output's requests; native leases own subsequent changes. - if(io.read(0x45404)!=power || io.read(0x45454)!=ddiPower || io.read(0x45444)!=auxPower)return false; + if(io.read(0x45404)!=power || io.read(0x45454)!=ddiPower || io.read(0x45444)!=auxPower || + (dsi && !XELP::DSI::sourceUnchanged(io,*dsi)))return false; for(unsigned p=0;p<3;++p)if((active&(1U< Result normalize(IO &io,const AlderLakeN::DisplayReadout &boo } } access{io,tc[p],r,peer,write}; const auto owner=reinterpret_cast(&io),object=reinterpret_cast(&tc[p]); - if(access.ownership.acquire(access,owner,object,ports[p],true).status!=Display13TC::Status::Complete || + if(access.ownership.acquire(access,owner,object,ports[p],true,tc[p].mode).status!=Display13TC::Status::Complete || !Display13TC::Clock::disable(access,tc[p].port) || access.ownership.release(access,owner,object,ports[p]).status!=Display13TC::Status::Complete) { if(r.status!=Status::PeerChanged)r.status=Status::Readback;return r; @@ -328,8 +332,72 @@ template Result normalize(IO &io,const AlderLakeN::DisplayReadout &boo } current=UINT32_MAX; auto &after=scratch.after;AlderLakeN::readDisplay(io,true,after);r.after=after.activePipes; - if(after.status!=AlderLakeN::ReadoutStatus::Complete || after.activePipes!=1 || !peer()) {r.status=Status::PeerChanged;return r;} + if(after.status!=AlderLakeN::ReadoutStatus::Complete || after.activePipes!=(dsi?0U:1U) || !peer()) {r.status=Status::PeerChanged;return r;} r.stage=7;r.status=Status::Complete;return r; } +// Preserve the DSI link and the firmware allocation. Once the TC peers have +// drained, remove the primary pipe scaler and present that same (possibly +// smaller) linear plane on the native canvas until the driver owns a full +// panel-sized backing. Never enlarge the firmware plane's memory footprint. +template Result normalizeDSIMirror(IO &io,const AlderLakeN::DisplayReadout &boot, + bool admitted,const Display13HDMIWiring &w,const XELPMemory::Layout &memory, + const XELP::DSI::Profile &profile,NormalizeScratch &scratch) { + Result r;r.before=boot.activePipes; + if(!admitted || !profile.valid() || !memory.reservationKnown || + boot.status!=AlderLakeN::ReadoutStatus::Complete || !(boot.poweredPipes&1) || + !boot.activePipes || (boot.activePipes&~6U) || w.dsi!=1)return r; + r.status=Status::Topology;r.stage=10; + const auto get=[&](uint32_t reg){uint32_t v;return AlderLakeN::value(boot,reg,v)?v:UINT32_MAX;}; + if(!io.readable(0x51004) || !io.readable(0x45404) || io.read(0x45404)!=get(0x45404))return r; + const auto ref=io.read(0x51004)>>29; + const uint32_t refs[]={24000,19200,38400};if(ref>=3)return r; + auto link=XELP::DSI::decodeReadout([&](uint32_t reg,uint32_t &v){ + if(!io.readable(reg))return false;v=io.read(reg);return true; + },refs[ref]); + if(link.identity.pipe!=0 || !link.matches(profile) || !XELP::DSI::sourceUnchanged(io,link) || + !io.readable(0x7B808) || (io.read(0x7B808)&0xC0000000U))return r; + r.stage=11; + const auto control=get(0x70180),surface=get(0x7019C); + const auto g=XELPGeometry::linear(control,get(0x70188),get(0x70190),get(0x7018C),get(0x701A4)); + const auto width=(link.htotal&65535)+1,height=(link.vtotal&65535)+1; + if(!AlderLakeN::bootGeometry(boot,0,g,link.htotal,link.vtotal) || + get(0x6001C)!=((g.width-1)<<16|(g.height-1)) || !memory.range(surface,g.bytes,true) || + surface!=get(0x701AC) || (get(0x60400)&(1U<<31)) || (get(0x70008)&0xC0000000U))return r; + uint32_t scaler=0; + for(unsigned i=0;i<2;++i)if(get(0x68180+i*0x100)&(1U<<31)) { + scaler=0x68170+i*0x100; + if(g.width==width && g.height==height)return r; + } + // Reject other consumers on any powered pipe before touching any of them. + for(unsigned p=0;p<3;++p)if(boot.poweredPipes&(1U<=2 && uint32_t(v-first)<0x10000000U; + },0x70040,r.framesAfter,r.status,Status::PeerChanged,false); + }; + const auto write=[&](uint32_t reg,uint32_t value) { + r.lastReg=reg;r.expected=value;io.write(reg,value);++r.writes; + r.observed=io.read(reg);return r.observed==value; + }; + if(!stable() || !AlderLakeN::bootGeometryUnchanged(io,boot,0) || + io.read(0x70180)!=control || io.read(0x701AC)!=surface || !frames())return r; + if(!write(0x70180,control&~(1U<<31)) || !write(0x7019C,surface) || !frames())return r; + if(!write(scaler+0x10,0) || !write(scaler,0) || !write(scaler+4,0) || + !write(0x6001C,((width-1)<<16)|(height-1)) || !frames() || + !write(0x70180,control) || !write(0x7019C,surface) || !frames() || + io.read(0x701AC)!=surface)return r; + r.stage=13;r.status=Status::Complete;return r; +} + } #endif diff --git a/WhateverXe/kern_display13_clock.hpp b/WhateverXe/kern_display13_clock.hpp index 06449435..47497dd6 100644 --- a/WhateverXe/kern_display13_clock.hpp +++ b/WhateverXe/kern_display13_clock.hpp @@ -1,6 +1,6 @@ -// ADL-N Display D0 clock readout. Linux v6.12 intel_cdclk.c's non-A-step -// adlp_cdclk_table and bxt_get_cdclk are the hardware reference. This does -// keeps the donor Hz ABI separate from the Linux CDCLK frequency. Transitions +// Display13 CDCLK. Linux v6.12 intel_cdclk.c defines adlp_cdclk_table, +// rplu_cdclk_table and the corresponding PCODE voltage levels. +// Keep the donor Hz ABI separate from the Linux CDCLK frequency. Transitions // below require caller-owned clocks, serialized PCODE and qualified consumers. #ifndef kern_display13_clock_hpp #define kern_display13_clock_hpp @@ -8,11 +8,25 @@ namespace Display13Clock { constexpr uint32_t DSSM=0x51004, PLL=0x46070, Control=0x46000, Squash=0x46008; -struct Entry {uint32_t ref,frequency,ratio;}; +enum class Variant {ADLP, RPLU}; +constexpr Variant variantForPCI(uint32_t identity) { + switch(identity) { + case 0xA7218086: case 0xA7A18086: case 0xA7A98086: case 0xA7AC8086: case 0xA7AD8086: + return Variant::RPLU; + default:return Variant::ADLP; + } +} +struct Entry { + uint32_t ref,frequency,ratio; + bool rpluOnly {}; + bool available(uint32_t reference,Variant variant) const { + return ref==reference && (!rpluOnly || variant==Variant::RPLU); + } +}; static constexpr Entry table[] { - {19200,172800,27},{19200,192000,20},{19200,307200,32},{19200,556800,58},{19200,652800,68}, - {24000,176000,22},{24000,192000,16},{24000,312000,26},{24000,552000,46},{24000,648000,54}, - {38400,179200,14},{38400,192000,10},{38400,307200,16},{38400,556800,29},{38400,652800,34} + {19200,172800,27},{19200,192000,20},{19200,307200,32},{19200,480000,50,true},{19200,556800,58},{19200,652800,68}, + {24000,176000,22},{24000,192000,16},{24000,312000,26},{24000,480000,40,true},{24000,552000,46},{24000,648000,54}, + {38400,179200,14},{38400,192000,10},{38400,307200,16},{38400,480000,25,true},{38400,556800,29},{38400,652800,34} }; enum class Status : uint32_t { NotReady, Unavailable, UnsupportedReference, Disabled, Unlocked, @@ -30,7 +44,7 @@ struct Readout { }; static_assert(sizeof(Readout)==12*4,"Display13 CDCLK readout layout"); -inline Readout decode(uint32_t dssm,uint32_t pll,uint32_t control,uint32_t squash) { +inline Readout decode(uint32_t dssm,uint32_t pll,uint32_t control,uint32_t squash,Variant variant=Variant::ADLP) { Readout r; r.dssm=dssm;r.pll=pll;r.control=control;r.squash=squash; if(dssm==UINT32_MAX || pll==UINT32_MAX || control==UINT32_MAX || squash==UINT32_MAX) { r.status=Status::Unavailable;return r; @@ -39,7 +53,7 @@ inline Readout decode(uint32_t dssm,uint32_t pll,uint32_t control,uint32_t squas const uint32_t references[] {24000,19200,38400}; if(r.refIndex>=3) {r.status=Status::UnsupportedReference;return r;} r.refKHz=references[r.refIndex];r.ratio=pll&0xFF; - for(const auto &e:table)if(e.ref==r.refKHz && e.frequency>r.maxKHz)r.maxKHz=e.frequency; + for(const auto &e:table)if(e.available(r.refKHz,variant) && e.frequency>r.maxKHz)r.maxKHz=e.frequency; if(!(pll&(1U<<31))) {r.status=Status::Disabled;return r;} if(!(pll&(1U<<30))) {r.status=Status::Unlocked;return r;} // ADLP's supported table has no squash waveforms. Do not interpret a @@ -50,7 +64,7 @@ inline Readout decode(uint32_t dssm,uint32_t pll,uint32_t control,uint32_t squas const uint32_t vco=r.refKHz*r.ratio; r.cdclkKHz=(vco+r.divider/2)/r.divider; bool found=false; - for(const auto &e:table) if(e.ref==r.refKHz) { + for(const auto &e:table) if(e.available(r.refKHz,variant)) { if(e.frequency>r.maxKHz)r.maxKHz=e.frequency; if(e.ratio==r.ratio && e.frequency==r.cdclkKHz)found=true; } @@ -62,12 +76,12 @@ inline Readout decode(uint32_t dssm,uint32_t pll,uint32_t control,uint32_t squas r.status=Status::Complete;return r; } -template Readout read(IO &io) { +template Readout read(IO &io,Variant variant=Variant::ADLP) { Readout r; const uint32_t registers[] {DSSM,PLL,Control,Squash}; for(auto reg:registers) if(!io.readable(reg))return r; const auto dssm=io.read(DSSM),pll=io.read(PLL),control=io.read(Control),squash=io.read(Squash); - r=decode(dssm,pll,control,squash); + r=decode(dssm,pll,control,squash,variant); // A concurrent clock transition must not publish a mixed native value. if(io.read(DSSM)!=dssm || io.read(PLL)!=pll || io.read(Control)!=control || io.read(Squash)!=squash) r.status=Status::Changed; @@ -94,23 +108,24 @@ struct Conditions { bool owned {}, watermarksReady {}; uint32_t activePipes {}; }; -inline bool plan(uint32_t reference,uint64_t nativeHz,uint32_t &ratio,uint32_t &divisor,uint32_t &control) { - for(const auto &e:table) if(e.ref==reference && uint64_t(e.frequency)*2000==nativeHz) { +inline bool plan(uint32_t reference,uint64_t nativeHz,uint32_t &ratio,uint32_t &divisor,uint32_t &control,Variant variant=Variant::ADLP) { + for(const auto &e:table) if(e.available(reference,variant) && uint64_t(e.frequency)*2000==nativeHz) { ratio=e.ratio;divisor=e.ref*e.ratio/e.frequency; control=(divisor==3 ? 1U<<22:0U)|0x380000|((2*e.frequency+500)/1000-2); return true; } return false; } -inline uint32_t voltage(uint32_t cdclkKHz) { - return cdclkKHz<=312000 ? 0:cdclkKHz<=326400 ? 1:cdclkKHz<=556800 ? 2:3; +inline uint32_t voltage(uint32_t cdclkKHz,Variant variant=Variant::ADLP) { + const uint32_t level1Max=variant==Variant::RPLU ? 480000 : 326400; + return cdclkKHz<=312000 ? 0:cdclkKHz<=level1Max ? 1:cdclkKHz<=556800 ? 2:3; } // calculateCDClockFrequency's HDMI branch requests the maximum table entry. // The other branch takes a pixel-clock lower bound, with a strict comparison // at table thresholds. Return an actual Display D0 entry in the donor's Hz ABI. -inline uint64_t select(uint32_t reference,uint64_t minimumHz,bool external) { +inline uint64_t select(uint32_t reference,uint64_t minimumHz,bool external,Variant variant=Variant::ADLP) { uint64_t selected=0; - for(const auto &e:table)if(e.ref==reference) { + for(const auto &e:table)if(e.available(reference,variant)) { const auto hz=uint64_t(e.frequency)*2000; if(external)selected=hz; else if(hz>minimumHz)return hz; @@ -154,14 +169,14 @@ template bool mailbox(IO &io,Change &r,uint32_t data,unsigned budgetUs if(r.mailboxStatus&0xFF) {r.status=ChangeStatus::MailboxError;return false;} return true; } -template Change set(IO &io,uint64_t nativeHz,Conditions conditions) { - Change r;r.requestedHz=nativeHz;r.before=read(io); +template Change set(IO &io,uint64_t nativeHz,Conditions conditions,Variant variant=Variant::ADLP) { + Change r;r.requestedHz=nativeHz;r.before=read(io,variant); if(r.before.status!=Status::Complete && r.before.status!=Status::Disabled)return r; if(r.before.squash&(1U<<31))return r; uint32_t ratio=0,divisor=0,control=0; - if(!plan(r.before.refKHz,nativeHz,ratio,divisor,control)) {r.status=ChangeStatus::InvalidFrequency;return r;} + if(!plan(r.before.refKHz,nativeHz,ratio,divisor,control,variant)) {r.status=ChangeStatus::InvalidFrequency;return r;} if(r.before.status==Status::Complete && r.before.nativeHz()==nativeHz) { - r.after=read(io); + r.after=read(io,variant); r.status=r.after.status==Status::Complete && r.after.nativeHz()==nativeHz ? ChangeStatus::Complete:ChangeStatus::ReadbackMismatch; return r; // Actual unchanged clock, no invented fixed-frequency success. } @@ -182,7 +197,7 @@ template Change set(IO &io,uint64_t nativeHz,Conditions conditions) { if(r.delayUs-start>=53000) {r.status=ChangeStatus::PrepareTimeout;return r;} delay(io,r,10); } - const auto prepared=read(io); + const auto prepared=read(io,variant); if(prepared.status!=r.before.status || prepared.dssm!=r.before.dssm || prepared.pll!=r.before.pll || prepared.control!=r.before.control || prepared.squash!=r.before.squash) { r.after=prepared;r.status=ChangeStatus::ReadbackMismatch;return r; @@ -211,10 +226,10 @@ template Change set(IO &io,uint64_t nativeHz,Conditions conditions) { write(io,r,Control,control); // Read back before lowering the prepare voltage. Failed PLL/decimal // proof must not lower voltage or attempt an unproven rollback. - r.after=read(io); + r.after=read(io,variant); if(r.after.status!=Status::Complete || r.after.nativeHz()!=nativeHz) {r.status=ChangeStatus::ReadbackMismatch;return r;} - if(!mailbox(io,r,voltage(r.after.cdclkKHz)))return r; - r.after=read(io); + if(!mailbox(io,r,voltage(r.after.cdclkKHz,variant)))return r; + r.after=read(io,variant); r.status=r.after.status==Status::Complete && r.after.nativeHz()==nativeHz ? ChangeStatus::Complete:ChangeStatus::ReadbackMismatch; return r; } @@ -222,8 +237,8 @@ template Change set(IO &io,uint64_t nativeHz,Conditions conditions) { // Caller proves every pipe/plane/cursor/DSB stopped and holds PW1 throughout. // Follow icl_cdclk_pll_disable and the Display >=12 bypass control, including // PCODE prepare/final voltage. Never report a disabled PLL before LOCK clears. -template Change stop(IO &io,Conditions conditions) { - Change r;r.before=read(io); +template Change stop(IO &io,Conditions conditions,Variant variant=Variant::ADLP) { + Change r;r.before=read(io,variant); if(r.before.status!=Status::Complete && r.before.status!=Status::Disabled)return r; if((r.before.squash&(1U<<31)) || (r.before.pll&0xC00000))return r; if(!conditions.owned) {r.status=ChangeStatus::OwnershipRequired;return r;} @@ -241,7 +256,7 @@ template Change stop(IO &io,Conditions conditions) { if(r.delayUs-start>=53000) {r.status=ChangeStatus::PrepareTimeout;return r;} delay(io,r,10); } - const auto prepared=read(io); + const auto prepared=read(io,variant); if(prepared.status!=r.before.status || prepared.dssm!=r.before.dssm || prepared.pll!=r.before.pll || prepared.control!=r.before.control || prepared.squash!=r.before.squash) { r.after=prepared;r.status=ChangeStatus::ReadbackMismatch;return r; @@ -253,13 +268,13 @@ template Change stop(IO &io,Conditions conditions) { } const auto bypass=r.before.refKHz/2; write(io,r,Control,0x380000|((2*bypass+500)/1000-2)); - r.after=read(io); + r.after=read(io,variant); if(r.after.status!=Status::Disabled || (r.after.pll&0xC0000000) || r.after.control!=(0x380000|((2*bypass+500)/1000-2)) || r.after.dssm!=r.before.dssm) { r.status=ChangeStatus::ReadbackMismatch;return r; } if(!mailbox(io,r,voltage(bypass)))return r; - r.after=read(io); + r.after=read(io,variant); r.status=r.after.status==Status::Disabled && !(r.after.pll&0xC0000000) && r.after.control==(0x380000|((2*bypass+500)/1000-2)) && r.after.dssm==r.before.dssm ? ChangeStatus::Complete:ChangeStatus::ReadbackMismatch; diff --git a/WhateverXe/kern_display13_dbuf.hpp b/WhateverXe/kern_display13_dbuf.hpp index 5441d9b9..f7121872 100644 --- a/WhateverXe/kern_display13_dbuf.hpp +++ b/WhateverXe/kern_display13_dbuf.hpp @@ -39,6 +39,8 @@ struct State { struct Released {bool valid {};Range primary,cursor;} released[3]; uint32_t capacity {}; // Optional board capacity: reserve B/C banks before either path attaches. uint32_t selected {},updating {},unretired {}; // Updating is a per-pipe transaction mask. + uint32_t restorePending {}; // Retained allocations whose hardware was lost in system sleep. + uint32_t creditSaved {},credit[3] {}; // Writable MBUS fetch-credit fields of selected, powered pipes. }; inline Status fail(State &s,Status status) {s.faulted=true;return status;} inline uint32_t tracked(const State &s) { @@ -296,6 +298,63 @@ template Status power(IO &io,State &s,uint32_t wanted,uint32_t consume } s.consumers=consumers;return Status::Complete; } +constexpr uint32_t CreditMask=0x01FFDF0F; +// Snapshot while the display owner lock still protects the selected pipes. +// Never read an unpowered bank. Native later enable restores arbitration +// slots separately; these MBUS credits also need the pre-sleep allocation. +template Status captureCredits(IO &io,State &s) { + if(s.faulted)return Status::Faulted; + if(!s.begun)return Status::Unavailable; + s.creditSaved&=s.selected; // Keep banks already powered down by earlier callbacks. + // A probe may get/put PG before native allocation restore. Its reset bank + // is not a new snapshot and must not replace the retained credits. + const auto wells=io.read(0x45404); + if(wells==UINT32_MAX)return fail(s,Status::Power); + for(unsigned p=0;p<3;++p)if((s.selected&(1U< Status restoreAllocation(IO &io,State &s,unsigned index) { + if(s.faulted)return Status::Faulted; + if(!s.begun || index>=3)return Status::Unavailable; + const auto bit=1U< Status bindOffPipe(IO &io,State &s,unsigned index) { if(!s.begun)return Status::Unavailable; if(index>=3 || s.joined)return fail(s,Status::Topology); const auto bit=1U< Status bindOffPipe(IO &io,State &s,unsigned index) { } s.pipe[index]=next;s.released[index]={};s.selected|=bit;syncMasks(s);return Status::Complete; } -// Read-only admission for a retained, powered allocation. Global startup -// preflights every eligible pipe before it makes any shared slice request. +// Admission for a retained, powered allocation. Restore a proven lost bank +// first; ordinary engine restart remains read-only here. Global startup +// checks every eligible pipe before it makes any shared slice request. template Status resumeProof(IO &io,State &s,unsigned index) { if(s.faulted)return Status::Faulted; if(!s.begun || index>=3 || !(s.selected&(1U< Status detach(IO &io,State &s,unsigned index) { const auto &a=s.pipe[index]; if(a.consumers || a.unretired || (s.updating&(1U< Status stop(IO &io,State &s) { if(ctl==UINT32_MAX || (ctl&(Request|1)))return fail(s,Status::Conflict); } } + const auto captured=captureCredits(io,s);if(captured!=Status::Complete)return captured; PipeAllocation old[3]; for(unsigned p=0;p<3;++p) {old[p]=s.pipe[p];s.pipe[p].consumers=s.pipe[p].unretired=0;s.pipe[p].heldPrimary=s.pipe[p].heldCursor={};} syncMasks(s);const auto result=power(io,s,0,0); diff --git a/WhateverXe/kern_display13_dsi_trial.hpp b/WhateverXe/kern_display13_dsi_trial.hpp index 1d1e00e9..ee2708f8 100644 --- a/WhateverXe/kern_display13_dsi_trial.hpp +++ b/WhateverXe/kern_display13_dsi_trial.hpp @@ -18,7 +18,8 @@ struct Result { uint32_t stage=0,writes=0,reg=0,value=0,pll=UINT32_MAX,sourcePLL=UINT32_MAX,frames=0; }; template Result transfer(IO &io,bool requested,const AlderLakeN::DisplayReadout &boot, - const XELPMemory::Layout &memory,unsigned hdmiMask) { + const XELPMemory::Layout &memory,unsigned hdmiMask, + Display13Clock::Variant variant=Display13Clock::Variant::ADLP) { Result r; if(!requested)return r; r.status=Status::Source; @@ -48,7 +49,7 @@ template Result transfer(IO &io,bool requested,const AlderLakeN::Displ for(unsigned i=0;i=2 || (mux&(1U<<10)))return r; diff --git a/WhateverXe/kern_display13_fb_irq.hpp b/WhateverXe/kern_display13_fb_irq.hpp index 48be1ac3..2dc2cad6 100644 --- a/WhateverXe/kern_display13_fb_irq.hpp +++ b/WhateverXe/kern_display13_fb_irq.hpp @@ -263,6 +263,18 @@ template Status enable(IO &io,State &s,uint32_t excludedPCH=0,uint32_t if(!(io.read(Master)&Enable) || !(io.read(GfxMaster)&Enable))return fail(io,s,Status::Readback); s.enabled=true;return Status::Complete; } +// Caller has proved total display power loss and restored its power/clock +// dependencies. Cached armed banks no longer describe hardware. Reinstall +// both pipe interrupts and connector detection, not just the live DSI bank. +template Status restoreAfterPowerLoss(IO &io,State &s,uint32_t allowed) { + if(!s.begun || s.faulted)return Status::Unavailable; + const auto actual=readActivePipes(io,allowed,s.transcoders); + if(actual.status!=Status::Complete)return actual.status; + io.write(Master,0); + if(io.read(Master)&Enable)return fail(io,s,Status::Readback); + s.enabled=false;s.active=0; + return enable(io,s,s.excludedPCH,NativeGT,actual.active,s.ports,s.typeC); +} template Status disable(IO &io,State &s) { if(!s.begun || !io.readable(Master))return Status::Unavailable; io.write(Master,0);s.enabled=false; diff --git a/WhateverXe/kern_display13_fb_memory.hpp b/WhateverXe/kern_display13_fb_memory.hpp index 52c2a0d5..ee5af80d 100644 --- a/WhateverXe/kern_display13_fb_memory.hpp +++ b/WhateverXe/kern_display13_fb_memory.hpp @@ -364,7 +364,7 @@ template Readout inspect(IO &io,const void *controller,size_t controll uint64_t apertureBytes,const XELPMemory::Layout &memory,bool bootScanout=true,uintptr_t expectedOwner=0, unsigned expectedDDI=1,const Display13TC::Boot *tc=nullptr, const AlderLakeN::DisplayReadout *dp=nullptr,bool panel=false, - const XELP::DSI::Readout *dsi=nullptr) { + const XELP::DSI::Readout *dsi=nullptr,uint32_t clippedBootSize=0) { Readout r; if (!associations(controller,controllerBytes,framebuffer,framebufferBytes,expectedOwner)) { r.status=Status::Association;return r; @@ -410,12 +410,19 @@ template Readout inspect(IO &io,const void *controller,size_t controll if(wells==UINT32_MAX || (wells&(bootScanout ? 0x403U : 3U))!=(bootScanout ? 0x403U : 3U)) {r.status=Status::DisplayState;return r;} uint32_t dpt=0,trans=0,conf=0,position=0,offset=0; const uint32_t transReg=dsi?0x6B400U:0x60400U,confReg=dsi?0x7B008U:0x70008U; + const auto clippedCanvas=[&]() { + return dsi && clippedBootSize && r.size==clippedBootSize && + io.readable(0x6001C) && io.readable(0x68180) && io.readable(0x68280) && + io.read(0x6001C)==(((dsi->htotal&0x3fff)<<16)|(dsi->vtotal&0x3fff)) && + !(io.read(0x68180)&(1U<<31)) && !(io.read(0x68280)&(1U<<31)); + }; if(bootScanout) { dpt=io.read(0x42084);trans=io.read(transReg);conf=io.read(confReg); position=io.read(0x7018C);offset=io.read(0x701A4); r.surface=io.read(0x7019C);r.control=io.read(0x70180);r.stride=io.read(0x70188);r.size=io.read(0x70190); const auto geometry=XELPGeometry::linear(r.control,r.stride,r.size,position,offset); - const bool source=dsi ? !(io.read(0x60400)&(1U<<31)) && XELPGeometry::timing(geometry,dsi->htotal,dsi->vtotal) : + const bool source=dsi ? !(io.read(0x60400)&(1U<<31)) && (XELPGeometry::timing(geometry,dsi->htotal,dsi->vtotal) || + (clippedCanvas() && geometry.width<=(dsi->htotal&0x3fff)+1 && geometry.height<=(dsi->vtotal&0x3fff)+1)) : (dpt&(1U<<30)) && ((trans>>27)&15)==expectedDDI+1 && (dp ? ((trans>>24)&7)==2 : ((trans>>24)&7)<=1) && !((trans>>20)&7); if (dpt==UINT32_MAX || !source || !(trans&(1U<<31)) || (conf&0xC0000000)!=0xC0000000 || @@ -447,7 +454,7 @@ template Readout inspect(IO &io,const void *controller,size_t controll io.read(0x70180)!=r.control || io.read(0x70188)!=r.stride || io.read(0x70190)!=r.size || (expectedDDI>=3 && !tc->unchanged(io)) || (dp && !AlderLakeN::dpSourceUnchanged(io,*dp,0,expectedDDI,0,tc,panel)) || - (dsi && (!XELP::DSI::sourceUnchanged(io,*dsi) || (io.read(0x60400)&(1U<<31))))))) { + (dsi && (!XELP::DSI::sourceUnchanged(io,*dsi) || (io.read(0x60400)&(1U<<31)) || (clippedBootSize && !clippedCanvas())))))) { r.status=Status::StateChanged;return r; } r.status=Status::Complete;return r; diff --git a/WhateverXe/kern_display13_framebuffer_power.inc b/WhateverXe/kern_display13_framebuffer_power.inc new file mode 100644 index 00000000..7fe87f29 --- /dev/null +++ b/WhateverXe/kern_display13_framebuffer_power.inc @@ -0,0 +1,254 @@ +// Common IOFramebuffer power dispatch and wake notification contract. +// The controller owns shared power/clock/DBUF/IRQ state. Native HDMI/DP/eDP +// retain their transport lifecycle; DSI substitutes only its panel/host path. +// IOGraphics subscribers include offline framebuffers, not just active pipes. + +uint32_t XE::wrapDisplay13SetPowerState(IOService *fb,uint32_t state) { + bool notifyWake=false; + recordDSIClient(fb,9,state,UINT32_MAX); + // Subscribers can re-enter clients. Serialize this transport's callback, + // without holding its FB mutex across notifications. + auto busy=callbackXE->adlnOutputWiring.dsi && fb && + fb->getMetaClass()->getClassSize()>=0x4B90 && dsiModeTarget(fb) ? &callbackXE->dsiPowerCallbackBusy : nullptr; + if(busy && __atomic_exchange_n(busy,1U,__ATOMIC_ACQUIRE))return kIOReturnNotReady; + struct PowerCallback { + uint32_t *busy; + ~PowerCallback(){if(busy)__atomic_store_n(busy,0U,__ATOMIC_RELEASE);} + } callback{busy}; + // Keep WSAA out of the native queue wait whenever this callback cannot + // finish hardware restoration or its notification obligation. This covers + // early failures too, not only the post-resume cadence diagnostic. A + // startup readiness retry has no sleep/wake obligation and must not + // prevent the initial WindowServer acceleration handshake. + const bool restoring=callbackXE->dsiRuntimePower.sleeping || callbackXE->dsiRuntimePower.failed || + callbackXE->dsiSystemWakeNotify || callbackXE->dsiPowerOffNotified; + if(busy && state==2 && restoring)__atomic_store_n(&callbackXE->dsiWakeIncomplete,1U,__ATOMIC_RELEASE); + auto result=[&]() -> uint32_t { + if(callbackXE->adlnOutputWiring.dsi) { + if(!fb || fb->getMetaClass()->getClassSize()<0x4B90)return kIOReturnNotReady; + DSIModeScope scope(fb);if(!scope.lock)return kIOReturnNotReady; + if(dsiLifecycleTarget(fb,Display13FBMemory::singlePath(fb))) { + if(callbackXE->dsiPowerTest && state<=2 && dsiModeTarget(fb)) { + auto controller=Display13FBMemory::field(fb,0x1D0); + auto path=Display13FBMemory::singlePath(fb); + auto &power=callbackXE->dsiRuntimePower; + if(state<2 && !power.sleeping && !power.failed && dsiNativeUpdateAllowed(fb) && + !callbackXE->dsiPowerOffNotified) { + if(!callbackXE->display13FramebufferEvent)return kIOReturnNotReady; + // Match prepareToExitWake's WillPowerOff notification + // before disabling the display and its completion IRQ. + IORecursiveLockUnlock(scope.lock);scope.lock=nullptr; + callbackXE->dsiPowerOffNotified=true; + ++callbackXE->dsiPowerOffEvents; + const auto event=reinterpret_cast(callbackXE->display13FramebufferEvent); + const auto status=event(fb,7,nullptr); + uint32_t notice[]={1,7,status,callbackXE->dsiPowerOffEvents,callbackXE->dsiPowerOnEvents}; + IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIPowerNotifications",notice,sizeof(notice)); + if(status!=kIOReturnSuccess)return status; + scope.lock=Display13FBMemory::field(fb,0x4220); + if(!scope.lock)return kIOReturnNotReady; + IORecursiveLockLock(scope.lock); + if(!dsiModeTarget(fb) || Display13FBMemory::singlePath(fb)!=path || + Display13FBMemory::field(fb,0x1D0)!=controller)return kIOReturnNotReady; + } + const bool ready=!power.failed && (state==2 ? (power.sleeping ? callbackXE->resumeDSIPath(controller,fb,path) : dsiNativeUpdateAllowed(fb)) : + (power.sleeping ? true : (dsiNativeUpdateAllowed(fb) && callbackXE->quiesceDSIPath(controller,fb,path,true)))); + if(!ready)return kIOReturnNotReady; + if(state==2 && callbackXE->dsiSystemWakeNotify) { + notifyWake=true; + } + *reinterpret_cast(reinterpret_cast(fb)+0x4284)=state; + return kIOReturnSuccess; + } + // One-way runtime stop experiment through the real power callback. + // Recovery is a reboot; state 2 never replays the donor's DDI path. + if(callbackXE->dsiStopTest && state<2 && dsiModeTarget(fb)) { + const bool stopped=callbackXE->dsiMemoryReleased && callbackXE->dsiDDIReleased && + callbackXE->dsiClockStop.complete() && callbackXE->dsiPanelStop.complete; + if(stopped)return kIOReturnSuccess; + if(!callbackXE->dsiNativeWriterHooksReady || !dsiNativeUpdateAllowed(fb))return kIOReturnNotReady; + auto controller=Display13FBMemory::field(fb,0x1D0); + auto path=Display13FBMemory::singlePath(fb); + if(!callbackXE->quiesceDSIPath(controller,fb,path))return kIOReturnNotReady; + *reinterpret_cast(reinterpret_cast(fb)+0x4284)=state; + return kIOReturnSuccess; + } + // Match only the donor's already-Wake no-op. Do not publish a new + // native power state without executing a DSI power transaction. + if(state!=2 || Display13FBMemory::field(fb,0x4284)!=2)return kIOReturnUnsupported; + return callbackXE->dsiNativeWriterHooksReady && dsiNativeUpdateAllowed(fb) ? kIOReturnSuccess : kIOReturnNotReady; + } + } + if(state==2 && !prepareDisplay13Wake(fb))return kIOReturnNotReady; + return FunctionCast(wrapDisplay13SetPowerState,callbackXE->orgDSIAttributes[2])(fb,state); + }(); + // Preserve the notification obligation across a failed TC handshake or + // subscriber event. Later power callbacks may retry it without rerunning + // a completed DSI panel restart. + if(result==kIOReturnSuccess && state==2 && (notifyWake || callbackXE->dsiSystemWakeNotify)) { + if(callbackXE->finishDisplay13SystemWake(fb))callbackXE->dsiSystemWakeNotify=false; + else result=kIOReturnNotReady; + } else if(result==kIOReturnSuccess && state==2 && busy && callbackXE->dsiPowerOffNotified) { + // Display-only doze has no system DidWake obligation. Pair its off + // notification only after the DSI path is usable again. + const auto event=reinterpret_cast(callbackXE->display13FramebufferEvent); + result=event ? event(fb,8,nullptr) : kIOReturnNotReady; + if(result==kIOReturnSuccess){callbackXE->dsiPowerOffNotified=false;++callbackXE->dsiPowerOnEvents;} + uint32_t notice[]={1,8,result,callbackXE->dsiPowerOffEvents,callbackXE->dsiPowerOnEvents}; + IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIPowerNotifications",notice,sizeof(notice)); + } + if(busy && state==2 && result==kIOReturnSuccess)__atomic_store_n(&callbackXE->dsiWakeIncomplete,0U,__ATOMIC_RELEASE); + recordDSIClient(fb,9,state,result); + return result; +} + +bool XE::ownedDisplay13Framebuffer(IOService *fb) const { + return fb && display13FBLifecycleHooksReady && display13FBABIReady && display13PagingLayoutKnown && + !strcmp(fb->getMetaClass()->getClassName(),"AppleIntelFramebuffer") && + fb->getMetaClass()->getClassSize()>=0x4B90 && adlnTopologyController && + Display13FBMemory::field(fb,0x1D0)==adlnTopologyController && + Display13FBMemory::field(fb,0x1DC)<3 && + Display13FBMemory::field(adlnTopologyController,0xD60+8*Display13FBMemory::field(fb,0x1DC))==fb; +} + +bool XE::finishDisplay13SystemWake(IOService *caller) { + if(!display13FramebufferEvent || !caller)return false; + auto controller=Display13FBMemory::field(caller,0x1D0); + if(!controller || controller!=adlnTopologyController || controller->getMetaClass()->getClassSize()<0x1C20)return false; + auto &c=display13FBPower; + if(!c.lock || !display13PagingLayoutKnown)return false; + // Notifications can re-enter native clients. Do not hold a framebuffer + // mutex, Intel workloop gate or owner lock across handleEvent. A second + // completion caller leaves the obligation pending instead of duplicating it. + if(__atomic_exchange_n(&displayWakeEventBusy,1U,__ATOMIC_ACQUIRE))return false; + struct Completion { + uint32_t *busy; + ~Completion(){__atomic_store_n(busy,0U,__ATOMIC_RELEASE);} + } completion{&displayWakeEventBusy}; + IOLockLock(c.lock);const auto epoch=c.restoreEpoch;IOLockUnlock(c.lock); + // Port handshakes run without FB/shared locks. They apply to TC PHYs in + // fixed HDMI/DP mode too; connector shape is not a wake admission rule. + // DidWake releases parked clients, including offline connectors. It is + // not a scanout-success report and must not depend on an unplugged peer + // acquiring a link. Keep unsuccessful TC restoration pending/observable; + // prepareDisplay13Wake requires it for that output if HPD is present. + const bool portsReady=restoreDisplay13Ports(controller); + if(displayWakeEventEpoch!=epoch) { + displayWakeEventEpoch=epoch;displayWakeEventErrors=0; + for(auto &phase:displayWakeEventPhase)phase=0; + } + using Event=uint32_t (*)(IOService *,int,void *); + const auto event=reinterpret_cast(display13FramebufferEvent); + uint32_t events[18]={4,display13PagingLayoutKnown};bool complete=true; + for(unsigned index=0;index<3;++index) { + IOLockLock(c.lock); + auto fb=Display13FBMemory::field(controller,0xD60+8*index); + if(fb)fb->retain(); + IOLockUnlock(c.lock); + if(!fb)continue; + if(fb->getMetaClass()->getClassSize()<0x4B90 || + Display13FBMemory::field(fb,0x1D0)!=controller || + Display13FBMemory::field(fb,0x1DC)!=index){complete=false;fb->release();continue;} + auto &phase=displayWakeEventPhase[index]; + const bool pending=display13PagingLayoutKnown && !(Display13FBMemory::field(fb,0x199)&0x20); + // Native peers already notified themselves if paging is active. The + // DSI adapter bypassed those native calls and owns its own pair. + const bool internal=adlnOutputWiring.dsi && index==0; + const auto nativePower=Display13FBMemory::field(fb,0x4284); + events[14+index]=nativePower; + // Native Sleep(0)->Doze/Wake runs prepareToExitSleep, which owns + // DidWake(true/false). Do not preempt that callback merely because + // its paging bit is still clear while another FB has already resumed. + // Doze(1)->Wake does not run prepareToExitSleep; an otherwise parked + // client in that state still needs the common notification below. + if(!phase && !internal && pending && nativePower==0)phase=3; + if(phase==3)events[17]|=1U<(1)); + // handleEvent returns Success without delivery on the wrong + // power thread. Conversely a subscriber error can follow a + // delivered DidWake: never replay that scope-decrement event. + if(Display13FBMemory::field(fb,0x199)&0x20)phase=1; + if(events[4+2*index]!=kIOReturnSuccess)displayWakeEventErrors|=1U<setProperty("WEXDSIPowerNotifications",notice,sizeof(notice)); + } + if(events[5+2*index]!=kIOReturnSuccess)displayWakeEventErrors|=1U<<(index+3); + } + } + // Phase 3 is delegated, not a claim that the native callback finished. + if(phase==2)events[3]|=1U<release(); + } + IOLockLock(c.lock);const bool sameEpoch=c.restoreEpoch==epoch;IOLockUnlock(c.lock); + complete=complete && sameEpoch && !displayWakeEventErrors; + events[10]=displayWakeEventEpoch;events[11]=displayWakeEventErrors;events[12]=portsReady;events[13]=complete; + IORegistryEntry::getRegistryRoot()->setProperty("WEXDisplay13SystemWakeEvents",events,sizeof(events)); + return complete; +} + +bool XE::processDisplay13NativeAttributes(KernelPatcher &patcher,size_t index,mach_vm_address_t address,size_t size) { + const char *symbols[]={ + "__ZN21AppleIntelFramebuffer12setAttributeEjm", + "__ZN21AppleIntelFramebuffer25setAttributeForConnectionEijm", + "__ZN21AppleIntelFramebuffer24setFramebufferPowerStateEj", + "__ZN31AppleIntelFramebufferController11hwSetStreamEP21AppleIntelFramebufferP21AppleIntelDisplayPathPK29IODetailedTimingInformationV2"}; + const uint32_t offsets[]={0x3331E,0x352D4,0x24DE8,0x50DB0}; + const uint8_t entries[][20]={ + {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x83,0xEC,0x18,0x48,0xFF,0x05}, + {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x83,0xEC,0x18,0x48,0xFF,0x05}, + {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x53,0x50,0x41,0x89,0xF6,0x49,0x89,0xFF,0x48,0xFF,0x05,0xF1}, + {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x81,0xEC,0x68,0x01,0x00,0x00}}; + if(!display13FramebufferRequested || size<0x50DC4)return false; + for(unsigned i=0;i<4;++i) { + const auto fn=patcher.solveSymbol(index,symbols[i],address,size);patcher.clearError(); + if(fn!=address+offsets[i] || memcmp(reinterpret_cast(fn),entries[i],20))return false; + } + // Native prepareToExitSleep / prepareToEnterWake call the same imported + // IOFramebuffer::handleEvent with DidWake(true) and DidPowerOn(nullptr). + // Compare the donor import with the host symbol validated in its Lilu callback. + if(size>0x26F1C) { + const uint8_t wake[]={0xBA,1,0,0,0,0x4C,0x89,0xF7,0xBE,4,0,0,0,0xE8}; + const uint8_t on[]={0x4C,0x89,0xEF,0xBE,8,0,0,0,0x31,0xD2,0xE8}; + if(!memcmp(reinterpret_cast(address+0x26F0A),wake,sizeof(wake)) && + !memcmp(reinterpret_cast(address+0x26809),on,sizeof(on))) { + int32_t wd,od; + memcpy(&wd,reinterpret_cast(address+0x26F18),sizeof(wd)); + memcpy(&od,reinterpret_cast(address+0x26814),sizeof(od)); + const auto imported=address+0x26F1C+wd; + if(wd && od && imported==address+0x26818+od) { + const auto target=XELPCall::importedTarget(imported,hostFramebufferEvent, + [](uintptr_t from,void *to,size_t length) { + return (from>>47)==0x1FFFF && tglVM.nofault && + tglVM.nofault(from,reinterpret_cast(to),length)==length; + }); + display13PagingLayoutKnown=hostFramebufferEvent && target==hostFramebufferEvent; + if(display13PagingLayoutKnown)display13FramebufferEvent=imported; + IORegistryEntry::getRegistryRoot()->setProperty("WEXDisplay13PagingContractReady",display13PagingLayoutKnown); + } + } + } + // A DSI writer replaces the native power callback and therefore owns its + // notifications and failed-wake escape. Refuse incomplete initialization + // here, before any GOP ownership or panel power is changed. + const bool hostReady=display13FramebufferEvent && display13PagingLayoutKnown && hostSleepReady && hostWakeRecoveryReady; + IORegistryEntry::getRegistryRoot()->setProperty("WEXDisplay13HostPowerReady",hostReady); + if(adlnOutputWiring.dsi && !hostReady)return false; + // Standard routes copy 17/17/16/20-byte prologues, before RIP counters. + KernelPatcher::RouteRequest routes[]={ + {symbols[0],wrapDSISetAttribute,orgDSIAttributes[0]}, + {symbols[1],wrapDSISetConnectionAttribute,orgDSIAttributes[1]}, + {symbols[2],wrapDisplay13SetPowerState,orgDSIAttributes[2]}, + {symbols[3],wrapDSISetStream,orgDSIAttributes[3]}}; + const bool ready=patcher.routeMultiple(index,routes,arrsize(routes),address,size);patcher.clearError();return ready; +} diff --git a/WhateverXe/kern_display13_power.hpp b/WhateverXe/kern_display13_power.hpp index d036f4d9..0f146caa 100644 --- a/WhateverXe/kern_display13_power.hpp +++ b/WhateverXe/kern_display13_power.hpp @@ -57,7 +57,7 @@ struct State { uint32_t refs[Count] {}; // Requests set by this object, versus requests found already set. Never // clear firmware/predecessor driver requests without an explicit handoff. - uint32_t owned {}, borrowed {}; + uint32_t owned {}, borrowed {}, deferredTC {}; uint32_t initialDriver {}, initialDDI {}, initialAUX {}, initialFuses {}; }; struct Lease { @@ -188,8 +188,8 @@ template Result acquire(IO &io, State &s, Lease &lease, uint32_t domai if (r.status != Status::Complete) break; } if (!valid(io, control(d), value)) { r.status = Status::Unavailable; break; } - if (s.refs[d] && !(value & request(d))) { r.status = Status::RequestLost; break; } - if(tcAux(d) && !s.refs[d]) { + if (s.refs[d] && !(s.deferredTC & mask) && !(value & request(d))) { r.status = Status::RequestLost; break; } + if(tcAux(d) && (!s.refs[d] || (s.deferredTC&mask))) { r.status=enableTCAux(io,d);if(r.status!=Status::Complete)break; } if (!(value & request(d))) { @@ -211,6 +211,7 @@ template Result acquire(IO &io, State &s, Lease &lease, uint32_t domai } if (r.status == Status::Complete) { for (unsigned d = 0; d < Count; ++d) if (domains & (1U << d)) ++s.refs[d]; + s.deferredTC &= ~domains; lease.owner = &s; lease.domains = domains; r.domain = Count; return r; @@ -252,7 +253,7 @@ template Result release(IO &io, State &s, Lease &lease) { r.status = clear(io, s, d, r); if (r.status != Status::Complete) { s.faulted = true; return r; } } - --s.refs[d]; + if(!--s.refs[d])s.deferredTC &= ~(1U< Result suspend(IO &io,State &s) { // Restore the current held scopes, in dependency order, without duplicate // gets. A request lost during power collapse is acquired anew and becomes // ours; a surviving borrowed request is never adopted merely by this call. -template Result restore(IO &io,State &s) { +template Result restore(IO &io,State &s,bool deferTC,PrepareDDIA prepareDDIA) { Result r; if(!s.ready || s.faulted)return r; uint32_t value; @@ -294,15 +295,25 @@ template Result restore(IO &io,State &s) { for(auto reg : registers)if(!valid(io,reg,value)) { r.status=Status::Unavailable;return r; } - for(unsigned d=0;d Result restore(IO &io,State &s) { else s.faulted=true; // Keep partial restoration and its prerequisites. return r; } +template Result restore(IO &io,State &s,bool deferTC=false) { + return restore(io,s,deferTC,[](uint32_t){return true;}); +} } #endif diff --git a/WhateverXe/kern_display13_power_bridge.hpp b/WhateverXe/kern_display13_power_bridge.hpp index a14d8bf4..ce5f0578 100644 --- a/WhateverXe/kern_display13_power_bridge.hpp +++ b/WhateverXe/kern_display13_power_bridge.hpp @@ -158,6 +158,13 @@ struct NativeBridge { --pgRefs[pg-1]; r.status = Status::Complete; return r; } + // A coalesced native get still has to rearm a retained Type-C leaf + // following power loss. Balance the temporary logical reference. + template Result rearmRetained(IO &io,uint32_t domain) { + Lease temporary; + auto r=acquire(io,state,temporary,domain); + return r.status==Status::Complete ? release(io,state,temporary) : r; + } template Result getDDI(IO &io, unsigned port) { Result r; if (!begun || state.faulted || state.suspended) return r; @@ -167,6 +174,10 @@ struct NativeBridge { r = acquire(io,state,ddiLease[port],ddiDomain(port)); if (r.status != Status::Complete) return r; } + if(state.deferredTC & ddiDomain(port)) { + r=rearmRetained(io,ddiDomain(port)); + if(r.status!=Status::Complete)return r; + } ++ddiRefs[port]; r.status = Status::Complete; return r; } @@ -195,6 +206,10 @@ struct NativeBridge { r = acquire(io,state,auxLease[port],auxDomain(port)); if (r.status != Status::Complete) return r; } + if(state.deferredTC & auxDomain(port)) { + r=rearmRetained(io,auxDomain(port)); + if(r.status!=Status::Complete)return r; + } ++auxRefs[port]; r.status = Status::Complete; return r; } diff --git a/WhateverXe/kern_display13_tc.hpp b/WhateverXe/kern_display13_tc.hpp index 238d443c..4509d411 100644 --- a/WhateverXe/kern_display13_tc.hpp +++ b/WhateverXe/kern_display13_tc.hpp @@ -66,19 +66,20 @@ struct Boot { // left unowned by firmware. Only explicit DP source validation may request // this witness; the later Mode acquisition must still take ownership. bool ownershipOptional {}; + unsigned mode {4}; bool valid() const { const auto powered=pipe<3 ? 5U|(1U<<(10+2*pipe)) : 0U; return port.ddi>=3 && port.ddi<=6 && pipe<3 && power!=UINT32_MAX && (power&powered)==powered && ddiPower!=UINT32_MAX && - (ddiPower&(1U<<(2*port.ddi))) && ready!=UINT32_MAX && (ready&7)==4 && + (ddiPower&(1U<<(2*port.ddi))) && (ready&4) && Port::modeMatches(ready,mode) && (mode==4 || !ownershipOptional) && buffer!=UINT32_MAX && (buffer&0x80000080U)==0x80000000U && (ownershipOptional || (buffer&64)) && clock!=UINT32_MAX && (clock>>28)==8 && pll!=UINT32_MAX && (pll&0xCC000000U)==0xCC000000U && transClock!=UINT32_MAX && (transClock>>28)==port.ddi+3; } - template static Boot capture(IO &io,bool admitted,unsigned ddi,unsigned pipe,bool ownershipOptional=false) { + template static Boot capture(IO &io,bool admitted,unsigned ddi,unsigned pipe,bool ownershipOptional=false,unsigned mode=4) { Boot b; if(!admitted || pipe>=3 || !Port::resolve(ddi,b.port))return b; - b.pipe=pipe;b.ownershipOptional=ownershipOptional; + b.pipe=pipe;b.ownershipOptional=ownershipOptional;b.mode=mode; if(!io.readable(0x45404) || !io.readable(0x45454))return {}; b.power=io.read(0x45404);b.ddiPower=io.read(0x45454); const auto powered=5U|(1U<<(10+2*pipe)); @@ -95,7 +96,7 @@ struct Boot { } template bool unchanged(IO &io) const { if(!valid())return false; - const auto now=capture(io,true,port.ddi,pipe,ownershipOptional); + const auto now=capture(io,true,port.ddi,pipe,ownershipOptional,mode); return now.valid() && now.power==power && now.ddiPower==ddiPower && now.ready==ready && now.buffer==buffer && now.clock==clock && now.pll==pll && now.transClock==transClock; } diff --git a/WhateverXe/kern_display13_tc_mode.hpp b/WhateverXe/kern_display13_tc_mode.hpp index 3b7b1fa0..3866d18b 100644 --- a/WhateverXe/kern_display13_tc_mode.hpp +++ b/WhateverXe/kern_display13_tc_mode.hpp @@ -12,7 +12,7 @@ struct Mode { Display13Power::Result power;uint32_t native {}; }; uintptr_t controller {},object {};unsigned ddi {}; - Phase phase {Phase::Idle};bool disconnectPending {}; + Phase phase {Phase::Idle};bool disconnectPending {},restorePending {}; Ownership ownership; unsigned selectedMode {};Lanes lanes; // ADLP: register/lane access -> ownership -> AUX cold block. @@ -22,6 +22,11 @@ struct Mode { Display13Power::Lease accessLease,auxLease; static uint32_t accessDomains() {return Display13Power::bit(Display13Power::Domain::PW2);} Mode()=default;Mode(const Mode&)=delete;Mode& operator=(const Mode&)=delete; + // Only a caller which observed collapsed display power may invalidate + // the physical ownership witness. Logical leases and native mode survive. + void notePowerLoss() { + if(phase==Phase::Active) {restorePending=true;ownership.active=false;} + } static uint32_t domains(unsigned d) { Port p;return Port::resolve(d,p) ? Display13Power::ddiDomain(d)| Display13Power::auxDomain(d)|Display13Power::bit(Display13Power::Domain::PW2) : 0; @@ -34,9 +39,24 @@ struct Mode { if(phase==Phase::Cleanup){r.status=Status::InUse;return r;} if(phase==Phase::Active) { if(wanted!=selectedMode || io.nativeMode()!=selectedMode || - (wanted==3 && !lanes.unchanged(io,ownership.port))){r.status=Status::Readback;return r;} + (wanted==3 && !restorePending && !lanes.unchanged(io,ownership.port))){r.status=Status::Readback;return r;} r.phy=ownership.acquire(io,owner,identity,d,false,wanted); - r.status=r.phy.status;if(r.status==Status::Complete)disconnectPending=false; + r.status=r.phy.status; + if(r.status==Status::Complete && restorePending) { + // Restore the retained AUX leaf only after PHY ownership. + // A temporary reference rearms hardware without replacing or + // double-counting the native mode's original lease. + Display13Power::Lease temporary; + r.power=io.acquireDomains(temporary,Display13Power::auxDomain(d)); + if(r.power.status==Display13Power::Status::Complete)r.power=io.releaseDomains(temporary); + if(r.power.status!=Display13Power::Status::Complete){r.status=Status::NoPower;return r;} + if(wanted==3) { + lanes=Lanes::capture(io,ownership.port); + if(!lanes.valid()){r.status=Status::NotReady;return r;} + } + restorePending=false; + } + if(r.status==Status::Complete)disconnectPending=false; return r; } if(io.nativeMode()!=0){r.status=Status::NeedsAdoption;return r;} @@ -82,7 +102,7 @@ struct Mode { r.power=io.releaseDomains(accessLease); if(r.power.status!=Display13Power::Status::Complete){r.status=Status::NoPower;return r;} } - controller=object=0;ddi=selectedMode=0;lanes={};phase=Phase::Idle;disconnectPending=false;r.status=Status::Complete;return r; + controller=object=0;ddi=selectedMode=0;lanes={};phase=Phase::Idle;disconnectPending=restorePending=false;r.status=Status::Complete;return r; } }; } diff --git a/WhateverXe/kern_dsi_adapter_backing.inc b/WhateverXe/kern_dsi_adapter_backing.inc index c0817c53..fe968a25 100644 --- a/WhateverXe/kern_dsi_adapter_backing.inc +++ b/WhateverXe/kern_dsi_adapter_backing.inc @@ -266,7 +266,7 @@ bool XE::switchDSINativeSurface() { decltype(owned) &owns; IO(IOMemoryMap *m,decltype(owned) &f):Display13FBBootIO{m},owns(f){} bool ready(){return owns();} - uint64_t nativeClockHz(){Display13FBClockIO clock{map};return Display13Clock::read(clock).nativeHz();} + uint64_t nativeClockHz(){Display13FBClockIO clock{map};return Display13Clock::read(clock,display13ClockVariant()).nativeHz();} uint64_t nowUs(){uint64_t t=0,ns=0;clock_get_uptime(&t);absolutetime_to_nanoseconds(t,&ns);return ns/1000;} void delayUs(unsigned us){if(us>=1000)IOSleep((us+999)/1000);else IODelay(us);} void write(uint32_t reg,uint32_t value) { @@ -297,13 +297,14 @@ bool XE::switchDSINativeSurface() { const Display13WM::Snapshot snapshot{&wm,nullptr,ddb,cursorDDB,static_cast(o.proof.gpu),0,ipc,hz,false,false, Display13WM::pipeLineTime(dsiProfile.mode.htotal(),pixel),pixel,0,c.transcoders}; Surface::Plane current{dsiBoot.control,io.read(0x701CC),dsiBoot.stride,dsiBoot.size,dsiBoot.surface,0,0}; - auto next=current;next.control=Display13WM::arbitrationControl((current.control&~(7U<<10))|(4U<<10)); + auto next=current;next.size=((uint32_t(dsiProfile.mode.height)-1)<<16)|(dsiProfile.mode.width-1); + next.control=Display13WM::arbitrationControl((current.control&~(7U<<10))|(4U<<10)); next.color|=1U<<13;next.stride=static_cast(o.proof.pitch/128);next.surface=static_cast(o.proof.gpu); Surface::Backing backing{reinterpret_cast(o.record),o.proof.gpu,o.proof.bytes,o.proof.pitch}; Capabilities caps;caps.installed=Capabilities::Adoption; if(ddb!=UINT32_MAX && cursorDDB!=UINT32_MAX && Display13DBUF::valid(range) && dsiSession.bind(caps,c.bootDSI,dsiProfile,1,reinterpret_cast(path),reinterpret_cast(&dsiBoot))) { - result=Surface::change(io,dsiSession,c.bootDSI,dsiProfile,current,next,backing,snapshot); + result=Surface::change(io,dsiSession,c.bootDSI,dsiProfile,current,next,backing,snapshot,dsiBoot.clipped); o.armed=result.writes!=0 || dsiSession.state==SessionState::Switching || dsiSession.state==SessionState::Faulted || dsiSession.state==SessionState::Streaming; if(!o.armed)dsiSession={}; diff --git a/WhateverXe/kern_dsi_adapter_boot.inc b/WhateverXe/kern_dsi_adapter_boot.inc index 080ff502..ab31d0cf 100644 --- a/WhateverXe/kern_dsi_adapter_boot.inc +++ b/WhateverXe/kern_dsi_adapter_boot.inc @@ -10,6 +10,29 @@ bool XE::prepareDSIFirmware(IOService *provider) { if(!copy)return false; auto parsed=XELP::DSI::parseProfile({static_cast(copy->getBytesNoCopy()),copy->getLength()},true); if(!parsed.valid()){copy->release();return false;} + // Explicit override for the captured MiniBook X panel tables. Hash the + // declared VBT supplied by the provider, without Linux debugfs padding. + if(checkKernelArgument("-wex-dsi-60hz")) { + static const uint8_t u300[]={ + 0xf3,0x03,0x0f,0xec,0x11,0x17,0x81,0x7b, + 0xf0,0x4d,0xf2,0xfa,0x32,0xa0,0x3b,0xfa, + 0xbd,0x8d,0x8a,0x9b,0x78,0xdf,0x6c,0x44, + 0x30,0x8c,0x79,0x06,0x1c,0x1b,0x06,0xa9}; + static const uint8_t n100[]={ + 0xe0,0x3d,0x64,0x90,0x5f,0x8d,0x0a,0xc1, + 0x6c,0x7c,0x80,0x49,0xfc,0x45,0x4a,0x93, + 0x27,0xb2,0xbd,0xe3,0x6d,0xa2,0x51,0x81, + 0x3c,0xbd,0x9d,0x78,0x45,0x1d,0x35,0x40}; + auto digest=Crypto::hash(static_cast(copy->getBytesNoCopy()),copy->getLength()); + if(digest) { + if(!memcmp(digest,u300,sizeof(u300)) || !memcmp(digest,n100,sizeof(n100))) { + // Keep the firmware blanking and use the 10 kHz EDID clock unit. + const auto hz=uint64_t(parsed.mode.htotal())*parsed.mode.vtotal()*60; + dsi60HzClockKHz=static_cast((hz+5000)/10000*10); + } + Buffer::deleter(digest); + } + } dsiProfile=parsed;dsiFirmware=copy;return true; } @@ -280,6 +303,7 @@ bool XE::bindDSIResources(IOService *controller) { // The live DSI GOP is linear, so this shared mode can be prepared // without disabling its plane. Retain PLL/power on any failure. if(XELP::DSI::prepareGGTT(io,dsiBoot,xelpMemory,dsiGGTT)) { + c.clippedDSIBootSize=dsiBoot.clipped ? dsiBoot.size : 0; c.bootDSI=dsiBoot.source;c.transcoders.dsi0Pipe=0; irq.state.transcoders=c.transcoders;irq.base=c.map->getVirtualAddress();irq.lock=irqLock; __atomic_store_n(&irq.admittedPipes,1U,__ATOMIC_RELEASE); @@ -293,14 +317,14 @@ bool XE::bindDSIResources(IOService *controller) { // before native display users start; this is not an opt-in host trial. bool prepareExternalClock=false; if(ready && !dsiPanelOnly && Display13FBTopology::multiOutputWiring(adlnOutputWiring)) { - const auto current=Display13Clock::read(io); + const auto current=Display13Clock::read(io,display13ClockVariant()); ready=current.status==Display13Clock::Status::Complete; prepareExternalClock=ready && current.nativeHz()!=current.nativeMaxHz(); } IOLockUnlock(c.lock);IORecursiveLockUnlock(clock.lock); if(ready)dsiPublishIRQLive(); if(ready && dsiCommandTest)ready=probeDSICommands(controller); - if(ready && (dsiPanelTest || prepareExternalClock))ready=probeDSIPanel(controller,prepareExternalClock); + if(ready && (dsiPanelTest || prepareExternalClock || dsi60HzClockKHz))ready=probeDSIPanel(controller,prepareExternalClock); if(ready) { IOLockLock(c.lock); using Light=XELP::DSI::Backlight; diff --git a/WhateverXe/kern_dsi_adapter_diagnostics.inc b/WhateverXe/kern_dsi_adapter_diagnostics.inc index af8e1ed5..af8910ef 100644 --- a/WhateverXe/kern_dsi_adapter_diagnostics.inc +++ b/WhateverXe/kern_dsi_adapter_diagnostics.inc @@ -69,7 +69,7 @@ bool XE::probeDSICommands(IOService *controller) { // the optional host trial rebuilds them before panel INIT_OTP. Neither path // releases shared resources or enables runtime power transitions. bool XE::probeDSIPanel(IOService *controller,bool prepareExternalClock) { - const bool restart=dsiHostTest || prepareExternalClock; + const bool restart=dsiHostTest || prepareExternalClock || dsi60HzClockKHz; auto &c=display13FBPower;auto &clock=comboClockTransaction;auto &irq=display13FBIRQ; if(!controller || !c.lock || !clock.lock || !irq.lock)return false; IORecursiveLockLock(clock.lock);IOLockLock(c.lock); @@ -113,7 +113,13 @@ bool XE::probeDSIPanel(IOService *controller,bool prepareExternalClock) { void delayUs(unsigned us){IODelay(us);} } io{c.map,owned,irq.lock,restart,dsiBoot.source.identity.pll}; Display13Clock::Change bootCDClock; + // Adopt and stop the firmware's original link first. Only the restart + // uses the new clock; panel commands and blanking stay firmware-derived. + auto target=dsiProfile; + if(dsi60HzClockKHz)target.mode.clockKHz=dsi60HzClockKHz; + const auto targetPlan=XELP::DSI::Restart::plan(target,dsiBoot.source.referenceKHz); struct Lifecycle { + const XELP::DSI::Restart::Plan &targetPlan; bool enabled,prepareExternal;XELPComboClock::ClockAllocation &allocation;uintptr_t owner; XELP::DSI::Restart::State &state;Display13Clock::Change &cdclk; bool stop(IO &io,const XELP::DSI::Readout &source,const XELP::DSI::Profile &profile) { @@ -128,12 +134,12 @@ bool XE::probeDSIPanel(IOService *controller,bool prepareExternalClock) { if(prepareExternal) { auto &c=display13FBPower; Display13FBClockIO clockIO {c.map}; - const auto current=Display13Clock::read(clockIO); + const auto current=Display13Clock::read(clockIO,display13ClockVariant()); cdclk.before=current;cdclk.requestedHz=current.nativeMaxHz(); const bool stopped=io.ready() && state.stopped && !state.started && !c.engineEnabled && Display13FBIRQ::consumersStopped(io,7,c.transcoders); if(!stopped || current.status!=Display13Clock::Status::Complete)return false; - cdclk=Display13Clock::set(clockIO,current.nativeMaxHz(),{true,true,0}); + cdclk=Display13Clock::set(clockIO,current.nativeMaxHz(),{true,true,0},display13ClockVariant()); if(cdclk.status!=Display13Clock::ChangeStatus::Complete) { adlnFault(c.clockFaulted,__LINE__);return false; } @@ -142,12 +148,13 @@ bool XE::probeDSIPanel(IOService *controller,bool prepareExternalClock) { *reinterpret_cast(reinterpret_cast(controller)+0xE98)=cdclk.after.nativeHz(); *reinterpret_cast(reinterpret_cast(controller)+0xEA0)=cdclk.after.nativeHz(); } + state.config=targetPlan; if(!XELP::DSI::Restart::start(io,allocation,owner,state))return false; source=XELP::DSI::Restart::expected(source,state);return source.complete(); } - } lifecycle{restart,!dsiPanelOnly,clock.allocation,reinterpret_cast(controller),dsiHostRestart,bootCDClock}; + } lifecycle{targetPlan,restart,!dsiPanelOnly,clock.allocation,reinterpret_cast(controller),dsiHostRestart,bootCDClock}; uint64_t record[32] {1}; - record[1]=io.ready() && !dsiPanelTrial.attempted && dsiProfile.valid() && + record[1]=(!restart || targetPlan.valid) && io.ready() && !dsiPanelTrial.attempted && dsiProfile.valid() && !dsiCommandHost.faulted && !dsiCommandHost.pending && c.bridge.outputHeld(io,0,0) && XELP::DSI::bootUnchanged(io,dsiBoot,xelpMemory); if(record[1]) { @@ -165,7 +172,8 @@ bool XE::probeDSIPanel(IOService *controller,bool prepareExternalClock) { auto nextBoot=dsiBoot; if(dsiPanelTrial.complete())nextBoot.source=dsiPanelTrial.source; record[30]=io.ready() && dsiPanelTrial.complete() && XELP::DSI::bootUnchanged(io,nextBoot,xelpMemory); - if(record[30]){dsiBoot=nextBoot;c.bootDSI=nextBoot.source;} + if(record[30] && !nextBoot.source.matches(target))record[30]=0; + if(record[30]){dsiBoot=nextBoot;c.bootDSI=nextBoot.source;dsiProfile.mode.clockKHz=target.mode.clockKHz;} } const auto &r=dsiPanelTrial;const auto &h=r.host; record[2]=static_cast(r.stage);record[3]=static_cast(r.error); @@ -180,7 +188,26 @@ bool XE::probeDSIPanel(IOService *controller,bool prepareExternalClock) { host.reg,host.value,host.writes,host.waitUs,host.stopped,host.started,host.pll, p.pll.cfg0(),p.pll.cfg1(),p.link.clockTiming,p.link.dataTiming,p.hsTimeout,p.lpTimeout,p.taTimeout, record[30],complete}; + uint64_t scan[6] {}; + if(dsi60HzClockKHz && complete && checkKernelArgument("-wex-dsi-audit")) { + // Count physical pipe-A frame boundaries, even before WindowServer starts. + scan[0]=1;scan[1]=dsiBoot.source.afeKHz;scan[2]=dsiProfile.mode.clockKHz; + uint32_t first=io.read(0x70040),edge=first;uint64_t start=0; + for(unsigned us=0;us<140000 && io.ready();us+=100) { + const auto frame=io.read(0x70040); + if(frame==UINT32_MAX || first==UINT32_MAX)break; + if(!start && frame!=first){start=mach_absolute_time();edge=frame;} + if(start && uint32_t(frame-edge)>=5) { + scan[3]=uint32_t(frame-edge); + absolutetime_to_nanoseconds(mach_absolute_time()-start,&scan[4]); + scan[5]=XELP::DSI::sourceUnchanged(io,dsiBoot.source); + break; + } + io.delayUs(100); + } + } IOLockUnlock(c.lock);IORecursiveLockUnlock(clock.lock); + if(scan[0])IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIScanTiming",scan,sizeof(scan)); IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIPanelTrial",record,sizeof(record)); if(restart)IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIHostRestart",hostRecord,sizeof(hostRecord)); if(restart && !dsiPanelOnly) diff --git a/WhateverXe/kern_dsi_adapter_lifecycle.inc b/WhateverXe/kern_dsi_adapter_lifecycle.inc index f258b1f4..522fa0f1 100644 --- a/WhateverXe/kern_dsi_adapter_lifecycle.inc +++ b/WhateverXe/kern_dsi_adapter_lifecycle.inc @@ -74,9 +74,11 @@ int XE::beginDSIFramebufferWriter(IOService *fb) { } uint32_t XE::wrapDSIFlipTransaction(IOService *fb,void *transaction,uint64_t options,void *params) { - const int token=beginDSIFramebufferWriter(fb);if(token<0)return kIOReturnNotReady; + const int token=beginDSIFramebufferWriter(fb);if(token<0) {recordDSIClient(fb,13,options,kIOReturnNotReady);return kIOReturnNotReady;} const auto result=FunctionCast(wrapDSIFlipTransaction,callbackXE->orgDSITransaction[0])(fb,transaction,options,params); - endDSINativeWriter(token);return result; + endDSINativeWriter(token); + const uint64_t flags=params ? Display13FBMemory::field(params,0x34) : 0; + recordDSIClient(fb,13,(options&UINT32_MAX)|(flags<<32),result);return result; } uint32_t XE::wrapDSIGammaTransaction(IOService *fb,void *transaction,void *params) { @@ -116,7 +118,17 @@ uint32_t XE::wrapDSISetAttribute(IOService *fb,uint32_t attribute,uintptr_t valu if(dsiLifecycleTarget(fb,Display13FBMemory::singlePath(fb))) { if(attribute==0x706F7772 || attribute==0x70777273) { // powr / IOGraphics pwrs relay. if(value>UINT32_MAX)return kIOReturnUnsupported; - return wrapDSISetPowerState(fb,static_cast(value)); + // The power wrapper takes its own FB lock. Release ours so + // restored framebuffer clients can re-enter after notification. + IORecursiveLockUnlock(scope.lock);scope.lock=nullptr; + return wrapDisplay13SetPowerState(fb,static_cast(value)); + } + // IOGraphics sends spwr before restart or power-off. Park the adopted + // panel through its own power path while display resources are live. + if(attribute==kIOSystemPowerAttribute && callbackXE->dsiPowerTest && dsiModeTarget(fb) && + (value==kIOMessageSystemWillRestart || value==kIOMessageSystemWillPowerOff)) { + IORecursiveLockUnlock(scope.lock);scope.lock=nullptr; + return wrapDisplay13SetPowerState(fb,0); } if(attribute==0x626B7376)return callbackXE->saveDSIBrightness(fb,value); if(XELP::DSI::AppleAdapter::transportAttribute(attribute))return kIOReturnUnsupported; @@ -227,48 +239,20 @@ uint32_t XE::wrapDSISetConnectionAttribute(IOService *fb,int32_t connection,uint return result; } -uint32_t XE::wrapDSISetPowerState(IOService *fb,uint32_t state) { - recordDSIClient(fb,9,state,UINT32_MAX); - const auto result=[&]() -> uint32_t { - if(callbackXE->adlnOutputWiring.dsi) { - if(!fb || fb->getMetaClass()->getClassSize()<0x4B90)return kIOReturnNotReady; - DSIModeScope scope(fb);if(!scope.lock)return kIOReturnNotReady; - if(dsiLifecycleTarget(fb,Display13FBMemory::singlePath(fb))) { - if(callbackXE->dsiPowerTest && state<=2 && dsiModeTarget(fb)) { - auto controller=Display13FBMemory::field(fb,0x1D0); - auto path=Display13FBMemory::singlePath(fb); - auto &power=callbackXE->dsiRuntimePower; - const bool ready=!power.failed && (state==2 ? (power.sleeping ? callbackXE->resumeDSIPath(controller,fb,path) : dsiNativeUpdateAllowed(fb)) : - (power.sleeping ? true : (dsiNativeUpdateAllowed(fb) && callbackXE->quiesceDSIPath(controller,fb,path,true)))); - if(!ready)return kIOReturnNotReady; - *reinterpret_cast(reinterpret_cast(fb)+0x4284)=state; - return kIOReturnSuccess; - } - // One-way runtime stop experiment through the real power callback. - // Recovery is a reboot; state 2 never replays the donor's DDI path. - if(callbackXE->dsiStopTest && state<2 && dsiModeTarget(fb)) { - const bool stopped=callbackXE->dsiMemoryReleased && callbackXE->dsiDDIReleased && - callbackXE->dsiClockStop.complete() && callbackXE->dsiPanelStop.complete; - if(stopped)return kIOReturnSuccess; - if(!callbackXE->dsiNativeWriterHooksReady || !dsiNativeUpdateAllowed(fb))return kIOReturnNotReady; - auto controller=Display13FBMemory::field(fb,0x1D0); - auto path=Display13FBMemory::singlePath(fb); - if(!callbackXE->quiesceDSIPath(controller,fb,path))return kIOReturnNotReady; - *reinterpret_cast(reinterpret_cast(fb)+0x4284)=state; - return kIOReturnSuccess; - } - // Match only the donor's already-Wake no-op. Do not publish a new - // native power state without executing a DSI power transaction. - if(state!=2 || Display13FBMemory::field(fb,0x4284)!=2)return kIOReturnUnsupported; - return callbackXE->dsiNativeWriterHooksReady && dsiNativeUpdateAllowed(fb) ? kIOReturnSuccess : kIOReturnNotReady; - } - } - return FunctionCast(wrapDSISetPowerState,callbackXE->orgDSIAttributes[2])(fb,state); - }(); - recordDSIClient(fb,9,state,result); - return result; + + +bool XE::failedDSIWakeFor(IOService *fb) const { + if((!__atomic_load_n(&dsiWakeDegraded,__ATOMIC_ACQUIRE) && + !__atomic_load_n(&dsiWakeIncomplete,__ATOMIC_ACQUIRE)) || !fb || + strcmp(fb->getMetaClass()->getClassName(),"AppleIntelFramebuffer") || + fb->getMetaClass()->getClassSize()<0x4B90 || + Display13FBMemory::field(fb,0x1D0)!=adlnTopologyController)return false; + if(__atomic_add_fetch(&callbackXE->dsiWakeRejections,1,__ATOMIC_RELAXED)==1) + IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIWakeRecoveryRejected",true); + return true; } + uint32_t XE::wrapDSISetStream(IOService *controller,IOService *fb,OSObject *path,const void *timing) { if(callbackXE->adlnOutputWiring.dsi) { if(!fb || fb->getMetaClass()->getClassSize()<0x4B90)return kIOReturnNotReady; @@ -278,31 +262,6 @@ uint32_t XE::wrapDSISetStream(IOService *controller,IOService *fb,OSObject *path return FunctionCast(wrapDSISetStream,callbackXE->orgDSIAttributes[3])(controller,fb,path,timing); } -bool XE::processDSINativeAttributes(KernelPatcher &patcher,size_t index,mach_vm_address_t address,size_t size) { - const char *symbols[]={ - "__ZN21AppleIntelFramebuffer12setAttributeEjm", - "__ZN21AppleIntelFramebuffer25setAttributeForConnectionEijm", - "__ZN21AppleIntelFramebuffer24setFramebufferPowerStateEj", - "__ZN31AppleIntelFramebufferController11hwSetStreamEP21AppleIntelFramebufferP21AppleIntelDisplayPathPK29IODetailedTimingInformationV2"}; - const uint32_t offsets[]={0x3331E,0x352D4,0x24DE8,0x50DB0}; - const uint8_t entries[][20]={ - {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x83,0xEC,0x18,0x48,0xFF,0x05}, - {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x83,0xEC,0x18,0x48,0xFF,0x05}, - {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x53,0x50,0x41,0x89,0xF6,0x49,0x89,0xFF,0x48,0xFF,0x05,0xF1}, - {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x81,0xEC,0x68,0x01,0x00,0x00}}; - if(!display13FramebufferRequested || size<0x50DC4)return false; - for(unsigned i=0;i<4;++i) { - const auto fn=patcher.solveSymbol(index,symbols[i],address,size);patcher.clearError(); - if(fn!=address+offsets[i] || memcmp(reinterpret_cast(fn),entries[i],20))return false; - } - // Standard routes copy 17/17/16/20-byte prologues, before RIP counters. - KernelPatcher::RouteRequest routes[]={ - {symbols[0],wrapDSISetAttribute,orgDSIAttributes[0]}, - {symbols[1],wrapDSISetConnectionAttribute,orgDSIAttributes[1]}, - {symbols[2],wrapDSISetPowerState,orgDSIAttributes[2]}, - {symbols[3],wrapDSISetStream,orgDSIAttributes[3]}}; - const bool ready=patcher.routeMultiple(index,routes,arrsize(routes),address,size);patcher.clearError();return ready; -} bool XE::processDSINativeTransactions(KernelPatcher &patcher,size_t index,mach_vm_address_t address,size_t size) { const char *symbols[]={ @@ -489,6 +448,7 @@ bool XE::disposeDSIPath(IOService *fb,OSObject *path) { // Host/IRQ/PLL retirement; the native DDI reference follows the stopped clock. // Keep the real stopped source separate from Session cleanup: it is not a full // Session StopProof. The one-shot journal retains a partial stop for diagnosis. + bool XE::quiesceDSIPath(IOService *controller,IOService *fb,OSObject *path,bool retain) { auto &c=display13FBPower;auto &clock=comboClockTransaction;auto &irq=display13FBIRQ; if(!c.lock || !clock.lock || !controller || controller->getMetaClass()->getClassSize()<0x1C20 || @@ -698,7 +658,22 @@ bool XE::quiesceDSIPath(IOService *controller,IOService *fb,OSObject *path,bool const bool complete=retain ? (parked && restored && power.host.stopped && !power.host.started && power.host.error==XELP::DSI::Restart::Error::None && dsiPanelStop.complete && qualified() && dsiArmedBackingOwned(fb,path) && !c.memoryFaulted && !c.dbuf.faulted) : (stoppedPath && dsiMemoryReleased); - if(retain && complete)power.sleeping=true; + if(retain && complete) { + power.sleeping=true; + // These are pipe A / DSI-owned registers. A peer may still be on at + // this point: its later power callback must not veto our snapshot. + dsiSleepImage.captured=(c.dbuf.selected&1) && !(irq.state.active&1); + if(dsiSleepImage.captured) { + Display13FBDBUFIO dbuf{c.map}; + dsiSleepImage.captured=Display13DBUF::captureCredits(dbuf,c.dbuf)==Display13DBUF::Status::Complete; + for(unsigned i=0;i 4dbc6). Mirror that CPU completion now, @@ -729,7 +704,8 @@ bool XE::quiesceDSIPath(IOService *controller,IOService *fb,OSObject *path,bool } // Runtime display wake keeps the native scanout allocation and every display -// lease from the matching off transaction. System power loss is not covered. +// lease from the matching off transaction. A captured system-sleep image also +// restores collapsed shared power before native peer mode-setting resumes. bool XE::resumeDSIPath(IOService *controller,IOService *fb,OSObject *path) { auto &c=display13FBPower;auto &clock=comboClockTransaction;auto &irq=display13FBIRQ;auto &power=dsiRuntimePower; if(!c.lock || !clock.lock || !controller || controller->getMetaClass()->getClassSize()<0x1C20 || @@ -743,14 +719,14 @@ bool XE::resumeDSIPath(IOService *controller,IOService *fb,OSObject *path) { const bool acquired=boundLoop && !inGate && reinterpret_cast(adlnWorkLoopTryClose)(loop); if(!inGate && !acquired){if(loop)loop->release();IORecursiveLockUnlock(mutex);return false;} IORecursiveLockLock(clock.lock);IOLockLock(c.lock); - const auto owned=[&]() { + const auto bindingOwned=[&]() { if(!power.sleeping || power.failed || !dsiStopAttempted || !dsiIRQParked || !dsiResourcesReady || !dsiStop.complete() || !dsiPanelStop.complete || !dsiNativeWriterHooksReady || !dsiProfile.valid() || controller!=c.controller || controller!=adlnTopologyController || c.workLoop!=loop || Display13FBMemory::field(controller,0xE00)!=loop || adlnOutputWiring.dsi!=1 || c.pipe[0].memoryFramebuffer!=fb || Display13FBMemory::singlePath(fb)!=path || c.transcoders.dsi0Pipe!=0 || c.dsiNativeWriters || c.memoryFaulted || c.clockFaulted || c.watermarkFaulted || - c.bridge.state.faulted || c.bridge.state.suspended || c.dbuf.faulted || c.dbuf.updating || + c.bridge.state.faulted || c.dbuf.faulted || c.dbuf.updating || c.pipe[0].destroyingPath || c.pipe[0].wmProgramming || c.pipe[0].wmCommitting || !c.dsiPort || Display13FBMemory::field(path,0x3648)!=c.dsiPort || Display13FBMemory::field(c.dsiPort,0x550)!=1 || Display13FBMemory::field(path,0x3644)!=2 || @@ -758,7 +734,10 @@ bool XE::resumeDSIPath(IOService *controller,IOService *fb,OSObject *path) { clock.thread || clock.allocation.external[0]!=reinterpret_cast(controller) || clock.allocation.binding[0]!=dsiBoot.source.identity.pll+1 || !c.bootDSI.sameSource(dsiBoot.source) || !dsiPublishedTarget(fb) || !dsiPendingBacking.modeReady || !dsiArmedBackingOwned(fb,path))return false; - Display13FBBootIO io{c.map};return c.bridge.outputHeld(io,0,0); + return true; + }; + const auto owned=[&]() { + Display13FBBootIO io{c.map};return bindingOwned() && c.bridge.outputHeld(io,0,0); }; struct IO : Display13FBBootIO { decltype(owned) &owns;IOSimpleLock *lock;unsigned pll; @@ -779,7 +758,47 @@ bool XE::resumeDSIPath(IOService *controller,IOService *fb,OSObject *path) { uint64_t receipt[24] {1,1,0,power.cycles,power.sleeping},irqReceipt[24] {1}; bool masked=false,restored=false,rearmed=false,cursor=false,complete=false;uint32_t master=0; auto stoppedPlane=power.plane;stoppedPlane.control&=~(1U<<31); - receipt[5]=owned() && power.host.stopped && !power.host.started && !power.panel.attempted && + // A system sleep may clear even PW1. Restore held references before + // reading powered pipe banks. Display-only wake follows the retained path. + uint32_t coldStage=0,coldReg=0,coldValue=0; + uint32_t dbox[3]={},phyPower[4]={}; + const auto restoreLostDisplay=[&]() { + if(!bindingOwned())return false; + if(!restoreDisplay13ResourcesLocked(c.power,true))return false; + if(c.restoreEpoch==dsiRestoredEpoch)return true; + if(!dsiSleepImage.captured)return false; + auto store=[&](uint32_t reg,uint32_t v) { + coldReg=reg; + *reinterpret_cast(c.map->getVirtualAddress()+reg)=v; + coldValue=io.read(reg);return coldValue==v; + }; + if(!owned() || !c.engineEnabled || c.clockStopped)return false; + phyPower[0]=1;phyPower[3]=io.read(0x6B094); + if(phyPower[3]==UINT32_MAX || !(phyPower[3]&1))return false; + coldStage=4;dbox[0]=c.dbuf.credit[0];dbox[1]=io.read(0x7003C); + for(unsigned i=0;i(c.map->getVirtualAddress()+field.reg)=(before&~field.mask)|value; + coldValue=io.read(field.reg);if((coldValue&field.mask)!=value)return false; + } + } + for(unsigned i=0;isetProperty("WEXDSIWakePHYPower",phyPower,sizeof(phyPower)); + receipt[5]=domainReady && owned() && power.host.stopped && !power.host.started && !power.panel.attempted && power.host.error==XELP::DSI::Restart::Error::None && XELP::DSI::Surface::matches(io,stoppedPlane,false); if(receipt[5] && adlnIRQOwner(controller) && irq.lock && irq.lock==Display13FBMemory::field(controller,0x90) && irq.base==c.map->getVirtualAddress()) { @@ -790,9 +809,10 @@ bool XE::resumeDSIPath(IOService *controller,IOService *fb,OSObject *path) { master=irqIO.read(Display13FBIRQ::Master); const bool quiet=irq.scopes.total==irq.handlers && !irq.scopes.display && !irq.softwareHandlers && !(queued(0xE08)&Display13FBIRQ::pipeEvents(0)) && !queued(0xE0C) && !(queued(0xE10)&3); - if(quiet && !irq.quiescing && !irq.blockedPipes && !irq.state.faulted && irq.state.begun && irq.state.enabled && + if(quiet && !irq.quiescing && !irq.blockedPipes && !irq.state.faulted && irq.state.begun && (irq.state.enabled || coldStage==5) && !(irq.state.active&1) && (irq.state.configured&1) && (irq.admittedPipes&1) && irq.state.transcoders.dsi0Pipe==0 && - master!=UINT32_MAX && (master&Display13FBIRQ::Enable)) { + master!=UINT32_MAX && ((master&Display13FBIRQ::Enable) || coldStage==5)) { + if(coldStage==5)master|=Display13FBIRQ::Enable; irq.quiescing=true;irq.blockedPipes=1;irq.state.enabled=false;masked=true; irqIO.write(Display13FBIRQ::Master,0); const auto after=irqIO.read(Display13FBIRQ::Master); @@ -802,15 +822,15 @@ bool XE::resumeDSIPath(IOService *controller,IOService *fb,OSObject *path) { IOSimpleLockUnlockEnableInterrupt(irq.lock,saved); if(canStart) { struct Lifecycle { - XELPComboClock::ClockAllocation &allocation;uintptr_t owner;XELP::DSI::Restart::State &state; + XELPComboClock::ClockAllocation &allocation;uintptr_t owner;XELP::DSI::Restart::State &state;bool modeSelected; bool stop(IO &,const XELP::DSI::Readout &,const XELP::DSI::Profile &){return false;} bool start(IO &io,XELP::DSI::Readout &source) { - if(!XELP::DSI::Restart::start(io,allocation,owner,state))return false; + if(!XELP::DSI::Restart::start(io,allocation,owner,state,modeSelected))return false; source=XELP::DSI::Restart::expected(source,state);return source.complete(); } - } lifecycle{clock.allocation,reinterpret_cast(controller),power.host}; + } lifecycle{clock.allocation,reinterpret_cast(controller),power.host,phyPower[0]!=0}; XELP::DSI::Panel::cycle(io,c.bootDSI,dsiProfile,power.plane.control,power.plane.surface,power.panel,lifecycle,&dsiPanelStop, - dsiBacklight.initialized?&dsiBacklight:nullptr); + dsiBacklight.initialized?&dsiBacklight:nullptr,coldStage==5); if(power.panel.complete() && owned() && XELP::DSI::Surface::matches(io,power.plane,true)) { io.write(0x70080,power.cursorControl);io.write(0x70088,power.cursorPosition);io.write(0x70084,power.cursorBase); const auto before=io.read(0x70040); @@ -831,6 +851,8 @@ bool XE::resumeDSIPath(IOService *controller,IOService *fb,OSObject *path) { const auto wells=irqIO.read(Display13FBIRQ::Driver); if(wells==UINT32_MAX || (wells&3)!=3)irq.state.faulted=true; if(!irq.state.faulted) { + if(coldStage==5 && !(irqIO.read(Display13FBIRQ::GfxMaster)&Display13FBIRQ::Enable)) + irqIO.write(Display13FBIRQ::GfxMaster,Display13FBIRQ::Enable); irqIO.write(Display13FBIRQ::Master,master&Display13FBIRQ::Enable); const auto after=irqIO.read(Display13FBIRQ::Master); restored=after!=UINT32_MAX && (after&Display13FBIRQ::Enable);irq.state.enabled=restored; @@ -844,7 +866,10 @@ bool XE::resumeDSIPath(IOService *controller,IOService *fb,OSObject *path) { irqReceipt[7+3*p]=irqIO.read(Display13FBIRQ::PipeIIR(p)+4); irqReceipt[8+3*p]=irqIO.read(Display13FBIRQ::PipeIIR(p)-4); } - const auto status=Display13FBIRQ::extendActivePipes(irqIO,irq.state,irq.admittedPipes); + const auto status=irq.powerLost ? + Display13FBIRQ::restoreAfterPowerLoss(irqIO,irq.state,irq.admittedPipes) : + Display13FBIRQ::extendActivePipes(irqIO,irq.state,irq.admittedPipes); + if(status==Display13FBIRQ::Status::Complete)irq.powerLost=false; irqReceipt[15]=static_cast(status);irqReceipt[16]=irq.state.active; irqReceipt[17]=static_cast(irq.state.firstFailure); irqReceipt[18]=irq.state.failureRegister;irqReceipt[19]=irq.state.failureBits; @@ -855,10 +880,30 @@ bool XE::resumeDSIPath(IOService *controller,IOService *fb,OSObject *path) { irq.quiescing=false;irq.blockedPipes=0; adlnIRQCache(controller,irqIO);IOSimpleLockUnlockEnableInterrupt(irq.lock,saved); complete=restored && rearmed && cursor && power.panel.complete() && owned(); - if(complete){power.sleeping=false;++power.cycles;dsiStopAttempted=false;dsiIRQParked=false;} + if(complete){power.sleeping=false;++power.cycles;dsiStopAttempted=false;dsiIRQParked=false;dsiFirstWakeLatch=coldStage==5;dsiRestoredEpoch=c.restoreEpoch;} else{power.failed=true;adlnFault(c.memoryFaulted,__LINE__);} } } + uint64_t cadence[6]={}; + if(complete && coldStage==5) { + // Sample while this FB still owns the restored modeset. Distinguish + // an actual slow port from a sticky PIPESTATUS bit or sample jitter. + const auto &mode=dsiProfile.mode; + const uint64_t period=(uint64_t(mode.htotal())*mode.vtotal()*1000+mode.clockKHz-1)/mode.clockKHz; + if(period>=1000 && period<=100000) { + const auto first=io.read(0x70040); + const auto t=mach_absolute_time(); + IOSleep(static_cast((period*8+999)/1000)); + const auto last=io.read(0x70040),port=io.read(0x70058); + uint64_t ns=0;absolutetime_to_nanoseconds(mach_absolute_time()-t,&ns); + cadence[0]=period;cadence[1]=ns/1000;cadence[2]=first;cadence[3]=last;cadence[4]=port; + const bool bad=first!=UINT32_MAX && last!=UINT32_MAX && port!=UINT32_MAX && + (port&(1U<<26)) && cadence[1]>=period*6 && + uint64_t(uint32_t(last-first)+1)*period*2setProperty("WEXDSIRuntimePower",receipt,sizeof(receipt)); IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIResumeIRQ",irqReceipt,sizeof(irqReceipt)); IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIResumeCommands",commandReceipt,sizeof(commandReceipt)); + if(complete && coldStage==5){dsiSystemWakeNotify=true;IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIWakeCadence",cadence,sizeof(cadence));} + uint32_t systemWake[]={coldStage,coldReg,coldValue,dsiSleepImage.captured,domainReady,p.frameBefore,p.frameAfter,p.waitUs,display13FramebufferEvent!=0,dbox[0],dbox[1],dbox[2]}; + IORegistryEntry::getRegistryRoot()->setProperty("WEXDSISystemWake",systemWake,sizeof(systemWake)); publishDSIBacklight(); return complete; } diff --git a/WhateverXe/kern_dsi_adapter_modes.inc b/WhateverXe/kern_dsi_adapter_modes.inc index 6d600ab7..ae7309e2 100644 --- a/WhateverXe/kern_dsi_adapter_modes.inc +++ b/WhateverXe/kern_dsi_adapter_modes.inc @@ -102,6 +102,35 @@ bool XE::processDisplay13DSIModes(KernelPatcher &patcher,size_t index,mach_vm_ad // observe calls before the fixed-mode owner is bound; do not treat the trace // as admission or invoke any driver/hardware operation while collecting it. void XE::recordDSIClient(IOService *fb,unsigned event,uint64_t argument,uint32_t result) { + // Observe peer presentation at the actual framebuffer API boundary. + // Bounded flips per mode/power transition; no alternate rendering path. + if(callbackXE->adlnOutputWiring.dsi && fb && fb->getMetaClass()->getClassSize()>=0x4B90 && + Display13FBMemory::field(fb,0x1D0)==callbackXE->adlnTopologyController) { + const auto index=Display13FBMemory::field(fb,0x1DC); + if(index>0 && index<3 && (event==6 || event==9 || event==10 || event==11 || event==13)) { + DSIModeScope scope(fb);if(!scope.lock)return; + static uint64_t counts[3][14] {}; + if(event==6 || event==9)counts[index][13]=0; + const auto count=++counts[index][event]; + if(event==13 && count>8 && (!result || count>32))return; + auto path=Display13FBMemory::singlePath(fb); + const bool pathReady=path && path->getMetaClass()->getClassSize()>=0x3730; + uint64_t words[]={1,event,count,argument,result,index, + Display13FBMemory::field(fb,0x1E0),Display13FBMemory::field(fb,0x199), + Display13FBMemory::field(fb,0x4004),Display13FBMemory::field(fb,0x4284), + Display13FBMemory::field(fb,0x4270),Display13FBMemory::field(fb,0x4A00), + pathReady,pathReady ? Display13FBMemory::field(path,0x3640) : UINT32_MAX, + pathReady ? Display13FBMemory::field(path,0x3644) : UINT32_MAX, + pathReady ? Display13FBMemory::field(path,0x170) : UINT32_MAX, + pathReady ? Display13FBMemory::field(path,0x32A5) : UINT32_MAX, + pathReady && Display13FBMemory::field(path,0x32B0)==fb, + Display13FBMemory::field(fb,0x4250),Display13FBMemory::field(fb,0x4254)}; + const char *name=event==6 ? "Mode" : event==9 ? "Power" : event==10 ? "Current" : event==11 ? "Online" : "Flip"; + char key[64];snprintf(key,sizeof(key),"WEXPeer%u%s",index,name); + IORegistryEntry::getRegistryRoot()->setProperty(key,words,sizeof(words)); + return; + } + } if(!callbackXE->adlnOutputWiring.dsi || event>=13 || !fb || fb->getMetaClass()->getClassSize()<0x4B90 || Display13FBMemory::field(fb,0x1DC))return; DSIModeScope scope(fb);if(!scope.lock)return; @@ -256,7 +285,8 @@ uint32_t XE::wrapDSISetDetailedTimings(IOService *fb,OSArray *array) { if(array)for(unsigned i=0;igetCount();++i) { auto bytes=OSDynamicCast(OSData,array->getObject(i)); if(!bytes || bytes->getLength()!=sizeof(dsiFixedMode.timing))continue; - const auto &t=*static_cast(bytes->getBytesNoCopy()); + const auto t=XELP::DSI::AppleAdapter::panelOffer( + *static_cast(bytes->getBytesNoCopy()),dsiFixedMode.timing); const bool native=XELP::DSI::AppleAdapter::clientTiming(dsiFixedMode.timing,t,i); if(!native && !XELP::DSI::AppleAdapter::presentationTiming(dsiFixedMode.timing,t,i)) { // Keep the last rejected OS request across later successful @@ -264,7 +294,14 @@ uint32_t XE::wrapDSISetDetailedTimings(IOService *fb,OSArray *array) { IORegistryEntry::getRegistryRoot()->setProperty("WEXDSITimingRejected",bytes); continue; } - ++count;if(native)client=t.reservedA[0];offer[2]=i;offer[3]=client;offer[4]=t.pixelClock; + // A rotated native offer has explicit draw dimensions and a + // transform. Keep an exact wire alias first when both are offered. + const bool swap=(t.scalerFlags&0x10)!=0; + const bool fullSize=!t.horizontalInset && !t.verticalInset && + t.horizontalScaled==(swap?t.verticalActive:t.horizontalActive) && + t.verticalScaled==(swap?t.horizontalActive:t.verticalActive); + if(native || (!client && fullSize))client=t.reservedA[0]; + ++count;offer[2]=i;offer[3]=client;offer[4]=t.pixelClock; offer[5]=t.horizontalActive;offer[6]=t.horizontalBlanking;offer[7]=t.horizontalSyncOffset;offer[8]=t.horizontalSyncWidth; offer[9]=t.verticalActive;offer[10]=t.verticalBlanking;offer[11]=t.verticalSyncOffset;offer[12]=t.verticalSyncWidth; offer[13]=t.scalerFlags;offer[14]=t.horizontalScaled;offer[15]=t.verticalScaled; @@ -285,7 +322,8 @@ uint32_t XE::wrapDSISetDetailedTimings(IOService *fb,OSArray *array) { for(unsigned i=0;igetCount();++i) { auto bytes=OSDynamicCast(OSData,array->getObject(i)); if(!bytes || bytes->getLength()!=sizeof(dsiFixedMode.timing))continue; - const auto &t=*static_cast(bytes->getBytesNoCopy()); + const auto t=XELP::DSI::AppleAdapter::panelOffer( + *static_cast(bytes->getBytesNoCopy()),dsiFixedMode.timing); if(XELP::DSI::AppleAdapter::clientTiming(dsiFixedMode.timing,t,i) || XELP::DSI::AppleAdapter::presentationTiming(dsiFixedMode.timing,t,i)) { if(!replacement->appendBytes(&t,sizeof(t))) { @@ -304,7 +342,7 @@ uint32_t XE::wrapDSISetDetailedTimings(IOService *fb,OSArray *array) { dsiFixedMode.offerCount=count; if(replacement)IORegistryEntry::getRegistryRoot()->setProperty("WEXDSITimingList",replacement); else IORegistryEntry::getRegistryRoot()->removeProperty("WEXDSITimingList"); - if(client)dsiFixedMode.clientMode=client; + dsiFixedMode.clientMode=client ? client : XELP::DSI::AppleAdapter::FixedModeID; OSSafeReleaseNULL(previous); } else { OSSafeReleaseNULL(replacement); diff --git a/WhateverXe/kern_dsi_apple_timing.hpp b/WhateverXe/kern_dsi_apple_timing.hpp index 55c23b0e..75891fe5 100644 --- a/WhateverXe/kern_dsi_apple_timing.hpp +++ b/WhateverXe/kern_dsi_apple_timing.hpp @@ -40,6 +40,12 @@ inline bool annotateMode(const Mode &m,ModeInformation &info,bool rotated=false, else info.flags&=~0x02000004U; // A scaled source is not the native/default mode. return true; } +// A presentation mode reuses the bound DSI transport. CoreDisplay's +// numLinks hint cannot select a different physical link in this path. +inline Timing panelOffer(Timing t,const Timing &fixed) { + t.numLinks=fixed.numLinks; + return t; +} // CoreDisplay can replace the reserved client IDs when rebuilding its mode // list. Treat reserved IDs as opaque: the complete // timing, not an assumed generation/index bit layout, qualifies this alias. diff --git a/WhateverXe/kern_dsi_attributes.hpp b/WhateverXe/kern_dsi_attributes.hpp index cee316ef..760aa962 100644 --- a/WhateverXe/kern_dsi_attributes.hpp +++ b/WhateverXe/kern_dsi_attributes.hpp @@ -10,7 +10,7 @@ inline bool transportAttribute(uint32_t attribute) { // AGPM selectors belong to the shared GPU power implementation and // are not DSI transport requests. Do not filter them here. case 0x68646370U: // hdcp: DisplayPath::initHDCP (DDI/AUX) - case 0x73707772U: // spwr: controller hwShutdown, outside DSI stop + case 0x73707772U: // spwr: never run the donor DDI shutdown on the DSI panel return true; default:return false; } diff --git a/WhateverXe/kern_dsi_boot.hpp b/WhateverXe/kern_dsi_boot.hpp index a22c54d9..62a4f8dc 100644 --- a/WhateverXe/kern_dsi_boot.hpp +++ b/WhateverXe/kern_dsi_boot.hpp @@ -13,6 +13,7 @@ struct Boot { uint32_t version {1};BootStatus status {BootStatus::Unavailable}; Readout source;XELPMemory::Layout memory;XELPGeometry::Linear geometry; uint32_t poweredPipes {},control {},stride {},size {},surface {},dpt {},verifiedPages {}; + bool clipped {}; bool usable() const {return status==BootStatus::Candidate || status==BootStatus::Complete;} bool complete() const {return status==BootStatus::Complete;} }; @@ -50,10 +51,10 @@ template bool bootUnchanged(IO &io,const Boot &boot,const XELPMemory:: const uint32_t expected[]={boot.dpt,boot.control,boot.stride,boot.size,boot.surface,boot.surface,0,0}; for(unsigned i=0;i void captureBoot(IO &io,const AlderLakeN::DisplayReadout &normal, - const Profile &profile,const XELPMemory::Layout &memory,bool admitted,Boot &r) { + const Profile &profile,const XELPMemory::Layout &memory,bool admitted,Boot &r,bool mirrorPrepared=false) { r={};if(!admitted || !profile.valid())return; r.status=BootStatus::Memory; if(!memory.valid || !memory.reservationKnown || !memory.bytes || memory.bytes%4096 || @@ -76,7 +77,9 @@ template void captureBoot(IO &io,const AlderLakeN::DisplayReadout &nor r.dpt=words[0];r.control=words[1];r.stride=words[2];r.size=words[3];r.surface=words[4]; r.geometry=XELPGeometry::linear(r.control,r.stride,r.size,words[6],words[7]); // LINEAR uses byte GGTT offsets even with global DPT-disable clear. - if(!XELPGeometry::timing(r.geometry,r.source.htotal,r.source.vtotal) || r.surface!=words[5] || + r.clipped=mirrorPrepared && !XELPGeometry::timing(r.geometry,r.source.htotal,r.source.vtotal) && + io.readable(0x6001C) && XELPGeometry::insideCanvas(r.geometry,r.source.htotal,r.source.vtotal,io.read(0x6001C)); + if((!r.clipped && !XELPGeometry::timing(r.geometry,r.source.htotal,r.source.vtotal)) || r.surface!=words[5] || !memory.range(r.surface,r.geometry.bytes,true))return; r.status=BootStatus::Candidate; if(!bootUnchanged(io,r,memory))r.status=BootStatus::Changed; diff --git a/WhateverXe/kern_dsi_host.hpp b/WhateverXe/kern_dsi_host.hpp index 6c2e84ae..bc2d6d51 100644 --- a/WhateverXe/kern_dsi_host.hpp +++ b/WhateverXe/kern_dsi_host.hpp @@ -15,7 +15,7 @@ inline Packet packet(uint8_t type,Span data) { uint32_t parameters=0; if(type==0x05 && data.size==1)parameters=data.data[0]; else if(type==0x15 && data.size==2)parameters=data.data[0]|uint32_t(data.data[1])<<8; - else if(type==0x39){parameters=uint32_t(data.size);p.payload=data;} + else if(longWrite(type)){parameters=uint32_t(data.size);p.payload=data;} else return p; // Fresh header: VC0, LP, no stale fence/flush or short-packet payload bit. p.header=(p.payload.data?1U<<31:0)|(1U<<30)|(parameters<<8)|type; diff --git a/WhateverXe/kern_dsi_panel.hpp b/WhateverXe/kern_dsi_panel.hpp index 42624d40..4bd49c51 100644 --- a/WhateverXe/kern_dsi_panel.hpp +++ b/WhateverXe/kern_dsi_panel.hpp @@ -86,7 +86,7 @@ struct Shutdown { // IO::gpio serializes HPD writes; no delays may run under that IRQ lock. // A failed transaction is never retried and must retain all resources. template void cycle(IO &io,const Readout &link,const Profile &profile, - uint32_t planeControl,uint32_t surface,Result &r,Lifecycle &lifecycle,const Shutdown *shutdown=nullptr,Backlight *backlight=nullptr) { + uint32_t planeControl,uint32_t surface,Result &r,Lifecycle &lifecycle,const Shutdown *shutdown=nullptr,Backlight *backlight=nullptr,bool systemWake=false) { Readout current=link; if(r.attempted)return; r.attempted=true; @@ -122,11 +122,16 @@ template void cycle(IO &io,const Readout &link,const P }; const auto frame=[&]() { if(!read(0x70040,r.frameBefore))return false; + bool rebased=false; for(uint32_t us=0;;us+=1000) { if(!read(0x70040,r.frameAfter))return false; const auto n=uint32_t(r.frameAfter-r.frameBefore); if(n && n<0x80000000U)return true; - if(n || us>=3*period+1000){r.error=Error::Timeout;return false;} + // The first frame after a system power loss may reload the counter. + // Require actual forward progress from the new value as well. + if(systemWake && n>=0x80000000U && !rebased){r.frameBefore=r.frameAfter;rebased=true;} + else if(n){r.error=Error::Timeout;return false;} + if(us>=(systemWake?10:3)*period+1000){r.error=Error::Timeout;return false;} io.delayUs(1000);r.waitUs+=1000; } }; @@ -135,8 +140,11 @@ template void cycle(IO &io,const Readout &link,const P bool gpio(unsigned role,unsigned value){return this->io.gpio(role,value);} } port{io}; const auto sequence=[&](unsigned id) { - const bool video=r.stage==Stage::BacklightOff || r.stage==Stage::TranscoderOn || r.stage==Stage::BacklightOn; - const auto budget=video?videoCreditBudgetUs(profile.mode):100; + // INIT can queue a full burst of LP payloads before its first delay. + // Wait for that burst to drain before the next panel operation. + const bool frameBudget=r.stage==Stage::Init || r.stage==Stage::BacklightOff || + r.stage==Stage::TranscoderOn || r.stage==Stage::BacklightOn; + const auto budget=frameBudget?videoCreditBudgetUs(profile.mode):100; if(!r.host.run(port,profile.sequences,id,budget)){r.error=Error::Sequence;return false;}return true; }; const auto delay=[&](unsigned index) { @@ -176,6 +184,16 @@ template void cycle(IO &io,const Readout &link,const P // Sequences drain before returning; never enable video over queued data. if(port.failed || r.host.pending || !held())return; r.stage=Stage::TranscoderOn; + if(systemWake) { + // After cold link training and INIT_OTP, the video timing bank is + // clocked but scanout is still off (icl_dsi.c, pre_enable step 6c). + // Reissue the timing writes here even when the shadow reads match. + if(!wait(0x7B008,0xC0000000U,0,0) || !wait(0x6B030,1U<<20,1U<<20,0))return; + const uint32_t timings[][2]={{0x6B000,current.htotal},{0x6B008,current.hsync}, + {0x6B00C,current.vtotal},{0x6B014,current.vsync},{0x6B028,0},{0x6B010,current.vtotal}}; + for(const auto &entry:timings) + if(!write(entry[0],entry[1]) || !wait(entry[0],UINT32_MAX,entry[1],0))return; + } if(!write(0x7B008,link.conf) || !wait(0x7B008,0xC0000000U,0xC0000000U,10000))return; if(!sequence(3))return; r.stage=Stage::PlaneOn; diff --git a/WhateverXe/kern_dsi_restart.hpp b/WhateverXe/kern_dsi_restart.hpp index b232ef2f..a579442a 100644 --- a/WhateverXe/kern_dsi_restart.hpp +++ b/WhateverXe/kern_dsi_restart.hpp @@ -197,14 +197,15 @@ template bool retainedStopped(IO &io,const Readout &source, // transport derive from the profile; no firmware PLL/PHY register dump replay. // Leaves pixel transfer off, ready for command sequence initialization. The // caller must enable video/plane, validate and publish the new source identity. -template bool start(IO &io,XELPComboClock::ClockAllocation &allocation,uintptr_t owner,State &s) { +template bool start(IO &io,XELPComboClock::ClockAllocation &allocation,uintptr_t owner,State &s,bool modeSelected=false) { if(!s.stopped || s.started || s.stage!=Stage::Stopped || s.error!=Error::None || !s.config.valid)return false; Access a{io,allocation,owner,s};XELPComboClock::PLL pll; if(!XELPComboClock::PLL::resolve(s.pll,pll) || !a.held())return false; if(!a.wait(pll.enable,0xCC000000U,0,0) || !a.wait(0x6B400,1U<<31,0,0) || !a.wait(0x64000,0x80000080U,0x80,0) || !a.wait(0x164280,0x403,0x400|s.pll,0))return false; const auto &p=s.config; - s.stage=Stage::ModeOn;if(!a.rmw(0x6B094,0,1))return false; + s.stage=Stage::ModeOn; + if(!(modeSelected ? a.wait(0x6B094,1,1,0) : a.rmw(0x6B094,0,1)))return false; s.stage=Stage::Escape; if(!a.set(0x6B090,p.link.escapeDivider,0x1FF) || !a.set(0x162190,p.link.escapeDivider,0x1FF) || // Linux ADL_MIPIO_DW8 writes zero (unshifted divider & shifted mask), @@ -247,7 +248,8 @@ template bool start(IO &io,XELPComboClock::ClockAllocation &allocation !a.rmw(0x6B0C0,0xF000,0x4000) || !a.set(0x6B030,p.transport,~(1U<<20)) || !a.set(0x6B400,p.function))return false; s.stage=Stage::Link; - if(!a.wait(0x6B030,1U<<20,1U<<20,2500,10) || !a.wait(0x6B0D8,0x30001,0,2500,10) || + // Some DSI panels need longer than 2.5 ms after the transcoder is configured. + if(!a.wait(0x6B030,1U<<20,1U<<20,10000,10) || !a.wait(0x6B0D8,0x30001,0,2500,10) || !a.wait(0x6B0D0,0x1FFF,0x1040,100,10))return false; s.stage=Stage::Ready;s.started=true;return true; } diff --git a/WhateverXe/kern_dsi_sequence.hpp b/WhateverXe/kern_dsi_sequence.hpp index e450b31f..12c5a9db 100644 --- a/WhateverXe/kern_dsi_sequence.hpp +++ b/WhateverXe/kern_dsi_sequence.hpp @@ -21,6 +21,7 @@ struct Sequences { uint64_t delayUs {}; bool valid() const {return status==SequenceStatus::Valid;} }; +inline bool longWrite(uint8_t type) {return type==0x29 || type==0x39;} inline uint32_t sequenceU32(const uint8_t *p) { return uint32_t(p[0]) | uint32_t(p[1])<<8 | uint32_t(p[2])<<16 | uint32_t(p[3])<<24; } @@ -41,7 +42,7 @@ inline SequenceStatus validateSequence(Span bytes, Sequences &counts) { // flags require a separate host contract, never silent coercion. if(p[0] || length!=n-4)return SequenceStatus::Packet; if(!((p[1]==0x05 && length==1) || (p[1]==0x15 && length==2) || - (p[1]==0x39 && length>=1)))return SequenceStatus::Packet; + (longWrite(p[1]) && length>=1)))return SequenceStatus::Packet; ++counts.packets;break; } case 2: diff --git a/WhateverXe/kern_dsi_sleep_image.hpp b/WhateverXe/kern_dsi_sleep_image.hpp new file mode 100644 index 00000000..a51f2ead --- /dev/null +++ b/WhateverXe/kern_dsi_sleep_image.hpp @@ -0,0 +1,35 @@ +#pragma once +#include +namespace XELP { namespace DSI { +struct SystemSleepImage { +// Explicit writable configuration owned by the parked DSI pipe. DDB and WM +// precede plane arming. Shared DBUF owns DDB/MBUS credits independently. +// No interrupt identity, status or frame counters. +static constexpr uint32_t config[] = { + 0x6001C,0x45270, + 0x70240,0x70244,0x70248,0x7024C,0x70250,0x70254,0x70258,0x7025C,0x70268, + 0x70140,0x70144,0x70148,0x7014C,0x70150,0x70154,0x70158,0x7015C,0x70168, + 0x70188,0x7018C,0x70190,0x701A4,0x701CC,0x70180,0x7019C, + 0xC8254,0xC8250 +}; + // Linux intel_combo_phy.c: PHY-A voltage/process compensation, master + // current reference and lane power-down enable. Retain only RW fields. + struct Field {uint32_t reg, mask;}; + static constexpr Field fields[]={ + {0x162104,0x00FF00FF},{0x162124,UINT32_MAX},{0x162128,UINT32_MAX}, + {0x162120,1U<<24},{0x162100,1U<<31},{0x162014,1U<<4}, + {0x42084,1U<<30},{0xC7204,1U<<1}, + // Plane and cursor controls encode Display13 arbitration slots. + // Select those slots on the pipe before rearming either consumer. + {0x70028,1U<<13}, + // PHY A TX_DW8 stores the source/divider for duty-cycle correction. + // Preserve each data lane's other trim fields. + {0x1628A0,7U<<29},{0x1629A0,7U<<29},{0x162AA0,7U<<29},{0x162BA0,7U<<29} + }; + static constexpr unsigned fieldCount=sizeof(fields)/sizeof(fields[0]); + uint32_t savedFields[fieldCount] {}; + static constexpr unsigned count=sizeof(config)/sizeof(config[0]); + uint32_t regs[count] {}; + bool captured {}; +}; +}} diff --git a/WhateverXe/kern_dsi_surface.hpp b/WhateverXe/kern_dsi_surface.hpp index d69d8bc0..3890ded5 100644 --- a/WhateverXe/kern_dsi_surface.hpp +++ b/WhateverXe/kern_dsi_surface.hpp @@ -21,14 +21,16 @@ constexpr uint32_t Registers[]={0x70188,0x7018C,0x70190,0x701A4,0x701CC,0x70180, inline void values(const Plane &p,uint32_t (&v)[7]) { v[0]=p.stride;v[1]=p.position;v[2]=p.size;v[3]=p.offset;v[4]=p.color;v[5]=p.control;v[6]=p.surface; } -inline bool shape(const Plane &p,const Profile &profile) { +inline bool shape(const Plane &p,const Profile &profile,bool clippedBoot=false) { // The existing Display13 WM backend admits linear and uncompressed Y. // Keep the P2 fixed RGB8888 geometry; rotation/scaling/color modes are not // implicitly implemented by copying the donor's register cache. return profile.valid() && profile.mode.valid() && profile.mode.width<=8192 && profile.mode.height<=8192 && Display13WM::supportedRGBControl(p.control) && (p.control&(1U<<31)) && ((p.control>>23)&31)==8 && !(p.control&((3U<<21)|(1U<<9)|(1U<<8))) && - !p.position && !p.offset && p.size==((uint32_t(profile.mode.height)-1)<<16 | (uint32_t(profile.mode.width)-1)) && + !p.position && !p.offset && + (p.size==((uint32_t(profile.mode.height)-1)<<16 | (uint32_t(profile.mode.width)-1)) || + (clippedBoot && !(p.control&(7U<<10)) && (p.size&65535)>16) bool source(IO &io,const Readout &link,bool stoppingCursor=fa // the lease in Session. This never calls cleanup or reports a successful stop. template Result change(IO &io,Session &session,const Readout &link,const Profile &profile, const Plane ¤t,const Plane &next,const Backing &backing, - const Display13WM::Snapshot &wm) { + const Display13WM::Snapshot &wm,bool clippedBoot=false) { Result r; if(!io.ready() || !link.matches(profile) || !session.identity.same(link.identity) || - !shape(current,profile) || !target(next,profile,backing) || next.surface==current.surface || + !shape(current,profile,clippedBoot) || !target(next,profile,backing) || next.surface==current.surface || ((next.control^current.control)&~((7U<<28)|(7U<<10))) || (next.color!=current.color && next.color!=(current.color|(1U<<13))) || wm.pipeIndex!=0 || wm.transcoders.dsi0Pipe!=0 || wm.cursorActive || !wm.primary || diff --git a/WhateverXe/kern_host.cpp b/WhateverXe/kern_host.cpp index db79068d..d3169eef 100644 --- a/WhateverXe/kern_host.cpp +++ b/WhateverXe/kern_host.cpp @@ -1,27 +1,49 @@ // Copyright © 2018 vit9696. Derived from WhateverGreen (BSD-3-Clause). #include "kern_host.hpp" +#include "kern_donor.hpp" +#include +#include "kern_host_power_contract.hpp" +#include "kern_xelp_native_placement.hpp" +#include #include "kern_xelp_device.hpp" #include "kern_xe_data_access.hpp" #include "kern_backlight_profile.hpp" #include #include #include +static size_t hostInstructionSize(const uint8_t *bytes,size_t length) { + uint8_t code[32] {};memcpy(code,bytes,length(code),&ins); + return !(ins.flags&F_ERROR) && n<=length ? n : 0; +} +static size_t hostFunctionExtent(mach_vm_address_t fn,mach_vm_address_t base,size_t size) { + if(fn=size)return 0; + const auto remaining=size-(fn-base); + return HostPowerContract::extent(reinterpret_cast(fn),remaining<2048?remaining:2048,hostInstructionSize); +} XeHost *XeHost::callbackHost; struct FramebufferViewer : public IOFramebuffer { static IOMemoryMap *&getVramMap(IOFramebuffer *fb) { return static_cast(fb)->fVramMap; } }; static const char *graphicsPath[] { "/System/Library/Extensions/IOGraphicsFamily.kext/IOGraphicsFamily" }; static KernelPatcher::KextInfo graphics { "com.apple.iokit.IOGraphicsFamily",graphicsPath,arrsize(graphicsPath),{true},{},KernelPatcher::KextInfo::Unloaded }; +static const char *accelPath[] { "/System/Library/Extensions/IOAcceleratorFamily2.kext/Contents/MacOS/IOAcceleratorFamily2" }; +static KernelPatcher::KextInfo accelFamily { "com.apple.iokit.IOAcceleratorFamily2",accelPath,arrsize(accelPath),{true},{},KernelPatcher::KextInfo::Unloaded }; static const char *backlightPath[] { "/System/Library/Extensions/AppleBacklight.kext/Contents/MacOS/AppleBacklight" }; static KernelPatcher::KextInfo backlight { "com.apple.driver.AppleBacklight",backlightPath,arrsize(backlightPath),{true},{},KernelPatcher::KextInfo::Unloaded }; +static const char *surfacePath[] { "/System/Library/Extensions/IOSurface.kext/Contents/MacOS/IOSurface" }; +static KernelPatcher::KextInfo surface { "com.apple.iokit.IOSurface",surfacePath,arrsize(surfacePath),{true},{},KernelPatcher::KextInfo::Unloaded }; void XeHost::init() { callbackHost=this; PE_parse_boot_argn("gfxrst",&resetFramebuffer,sizeof(resetFramebuffer)); if(resetFramebuffer>=FB_TOTAL)resetFramebuffer=FB_DETECT; lilu.onPatcherLoadForce([](void *u,KernelPatcher &p){static_cast(u)->processKernel(p);},this); lilu.onKextLoadForce(nullptr,0,[](void *u,KernelPatcher &p,size_t i,mach_vm_address_t a,size_t s){static_cast(u)->processKext(p,i,a,s);},this); - if(resetFramebuffer!=FB_NONE)lilu.onKextLoadForce(&graphics); + lilu.onKextLoadForce(&graphics); + lilu.onKextLoadForce(&accelFamily); lilu.onKextLoadForce(&backlight); + lilu.onKextLoadForce(&surface); xe.init(); XeDataAccess::init(); } @@ -34,7 +56,9 @@ void XeHost::processKernel(KernelPatcher &patcher) { // The host must dispatch framebuffer patches and accelerator rejection // hooks even when the selected profile deliberately has no Render backend. active=profile.displayAdmitted(); + const auto display=XELP::displayAdapter(profile.family); + systemWakeRecovery=active && (display==XELP::DisplayAdapter::ADLP || display==XELP::DisplayAdapter::ADLN); backlightEnabled=active && (display==XELP::DisplayAdapter::TGL || display==XELP::DisplayAdapter::ADLS || display==XELP::DisplayAdapter::ADLP || display==XELP::DisplayAdapter::ADLN) && info->firmwareVendor!=DeviceInfo::FirmwareVendor::Apple; @@ -69,11 +93,96 @@ void XeHost::processKernel(KernelPatcher &patcher) { if(v){consoleVinfo=*v;gotConsoleVinfo=true;}else patcher.clearError(); } } - if(!active || resetFramebuffer==FB_DETECT){resetFramebuffer=FB_NONE;graphics.switchOff();} + if(!active || resetFramebuffer==FB_DETECT){resetFramebuffer=FB_NONE;if(!systemWakeRecovery)graphics.switchOff();} DeviceInfo::deleter(info); } +void XeHost::processLoadedDependency(KernelPatcher &patcher,KernelPatcher::KextInfo &info) { + if(info.loadIndex==KernelPatcher::KextInfo::Unloaded)return; + // A donor's OSBundleLibraries have loaded before its load callback. Some + // KC hosts (including IOSurface on Monterey) do not deliver a Lilu callback. + // Use the same named kmod list as Lilu, then its image/symbol machinery; + // never infer an ABI from an OS/build name or scan arbitrary memory. + auto list=reinterpret_cast(patcher.solveSymbol(KernelPatcher::KernelID,"_kmod")); + patcher.clearError();if(!list)return; + // XNU owns this finite list. Match Lilu's traversal; a module-count cap + // must not reject a valid dependency merely because it is late in the list. + for(auto module=*list;module;module=module->next) { + if(strncmp(module->name,info.id,KMOD_MAX_NAME))continue; + if(!module->address || module->size<32)return; + const auto size=patcher.updateRunningInfo(info.loadIndex,module->address,module->size); + const bool ready=patcher.getError()==KernelPatcher::Error::NoError; + patcher.clearError();if(ready)processKext(patcher,info.loadIndex,module->address,size); + return; + } +} void XeHost::processKext(KernelPatcher &patcher,size_t index,mach_vm_address_t address,size_t size) { if(!active)return; + if(xe.isIntelProvider(index)) { + // Donor imports exist before their native start, but Lilu's already-loaded + // kmod walk need not notify their provider first. Resolve its exports before + // the one-shot framebuffer installer consumes them. Seen is not ABI-ready: + // an incompatible provider still reaches the existing rejection guards. + xe.processKext(patcher,index,address,size); + providerSeen=true; + return; + } + if(surface.loadIndex==index) { + if(surfaceSeen)return; + const char *names[]={"__ZNK9IOSurface14getPixelFormatEv","__ZNK9IOSurface13getPlaneCountEv", + "__ZNK9IOSurface12getSurfaceIDEv","__ZNK9IOSurface14getYCbCrMatrixEv"}; + const uint8_t *functions[4] {};size_t lengths[4] {}; + for(unsigned i=0;i<4;++i) { + const auto fn=patcher.solveSymbol(index,names[i],address,size);patcher.clearError(); + if(fn>=address && fn-address(fn); + const auto remaining=size-(fn-address);lengths[i]=remaining<64?remaining:64; + } + } + const auto fb=XELPNative::surfaceABI(functions,lengths,true); + const auto accel=XELPNative::surfaceABI(functions,lengths,false); + xe.setHostSurfaceABI(static_cast(fb),static_cast(accel)); + surfaceSeen=true; + uint32_t state[]={static_cast(fb),static_cast(accel)}; + IORegistryEntry::getRegistryRoot()->setProperty("WEXHostSurfaceABI",&state,sizeof(state)); + return; + } + if(accelFamily.loadIndex==index) { + if(!systemWakeRecovery || accelSeen)return; + // Only the first loop of system_will_sleep(false) opens mode changes. + // Leave its subsequent orphan cleanup and the hardware sleep phase native. + const auto sym=[&](const char *name) { + auto v=patcher.solveSymbol(index,name,address,size);patcher.clearError();return v; + }; + sleepFBCount=sym("__ZNK21IOAccelDisplayMachine19getFramebufferCountEv"); + const auto sleep=sym("__ZN22IOGraphicsAccelerator217system_will_sleepEib"); + const auto mode=sym("__ZN21IOAccelDisplayMachine24display_mode_will_changeEj"); + const auto sleepLength=hostFunctionExtent(sleep,address,size); + const auto modeLength=hostFunctionExtent(mode,address,size); + const auto site=HostPowerContract::sleepSite(reinterpret_cast(sleep),sleepLength,sleep,sleepFBCount,hostInstructionSize); + bool ready=false; + if(!sleepScopeLock)sleepScopeLock=IOSimpleLockAlloc(); + KernelPatcher::RouteRequest notification("__ZN18IOAccelDisplayPipe22display_change_handlerEPvP13IOFramebufferiS0_", + wrapDisplayChange,orgDisplayChange); + const bool notifications=sleepScopeLock && patcher.routeMultiple(index,¬ification,1,address,size); + patcher.clearError(); + if(notifications && sleepFBCount && site.valid && HostPowerContract::sleepFields( + reinterpret_cast(sleep),sleepLength,reinterpret_cast(mode),modeLength,hostInstructionSize,site.owner)) { + // Pass this accelerator in rdi. The host compiler may keep it in rbx or + // r14; the matched count query establishes which register owns it. + uint8_t code[]={0x48,0x89,0xdf,0xe8,0,0,0,0,0x90,0x90,0x90,0x90,0x90}; + code[0]=0x48|(site.owner>=8 ? 4 : 0);code[2]=0xC7|((site.owner&7)<<3); + const auto target=sleep+site.offset; + const auto distance=static_cast(reinterpret_cast(pendingSleepDisplays)-(target+8)); + if(distance==static_cast(distance)) { + const auto rel=static_cast(distance);memcpy(code+4,&rel,4); + patcher.routeBlock(target,code,site.length); + ready=patcher.getError()==KernelPatcher::Error::NoError && !memcmp(reinterpret_cast(target),code,site.length);patcher.clearError(); + } + } + IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIDarkWakeReady",ready); + xe.setHostSleepReady(ready);accelSeen=true; + return; + } if(backlight.loadIndex==index) { if(backlightEnabled) { KernelPatcher::RouteRequest route("__ZN15AppleIntelPanel10setDisplayEP9IODisplay",wrapPanelSetDisplay,orgPanelSetDisplay); @@ -85,12 +194,117 @@ void XeHost::processKext(KernelPatcher &patcher,size_t index,mach_vm_address_t a } if(XeDataAccess::processKext(patcher,index,address,size))return; if(graphics.loadIndex==index) { + if(systemWakeRecovery && !graphicsSeen) { + const char *sym="__ZN13IOFramebuffer16setWSAAAttributeEjj"; + // Lilu resolves the symbol, verifies the image range, and builds the + // entrance trampoline. This wrapper needs no private implementation ABI. + KernelPatcher::RouteRequest route(sym,wrapWSAAAttribute,orgWSAAAttribute); + const bool ready=patcher.routeMultiple(index,&route,1,address,size);patcher.clearError(); + xe.setHostWakeRecoveryReady(ready); + // Bound the host function by its own image, not by the Intel donor. + const auto event=patcher.solveSymbol(index,"__ZN13IOFramebuffer11handleEventEiPv",address,size); + patcher.clearError(); + const auto eventLength=hostFunctionExtent(event,address,size); + const bool paging=eventLength && HostPowerContract::paging(reinterpret_cast(event),eventLength,hostInstructionSize); + xe.setHostFramebufferEvent(paging?event:0); + IORegistryEntry::getRegistryRoot()->setProperty("WEXHostPagingContractReady",paging); + IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIWakeRecoveryReady",ready); + graphicsSeen=true; + } + if(resetFramebuffer==FB_NONE)return; gIOFBVerboseBootPtr=patcher.solveSymbol(index,"__ZL16gIOFBVerboseBoot",address,size); if(gIOFBVerboseBootPtr){KernelPatcher::RouteRequest route("__ZN13IOFramebuffer6initFBEv",wrapFramebufferInit,orgFramebufferInit);patcher.routeMultiple(index,&route,1,address,size);} patcher.clearError();return; } + if(xe.isIntelDonor(index)) { + if(!surfaceSeen)processLoadedDependency(patcher,surface); + if(xe.isIntelFramebuffer(index) && xe.requiresIntelProvider() && !providerSeen) + processLoadedDependency(patcher,kextAppleIntelTGLCompat); + if(systemWakeRecovery) { + if(!graphicsSeen)processLoadedDependency(patcher,graphics); + if(!accelSeen)processLoadedDependency(patcher,accelFamily); + } + IORegistryEntry::getRegistryRoot()->setProperty("WEXHostDependenciesSeen",dependenciesSeen(index)); + // Always install the donor's probe/start guards in this callback. Returning + // with queued work lets OSKext enter the unpatched native lifecycle. Unknown + // host ABIs remain rejected by the existing donor readiness gates. + } xe.processKext(patcher,index,address,size); } +// Observe the subscriber that owns a sleep scope. Native system_will_sleep +// ignores its event-number argument and loops over every display pipe; a +// controller-wide asleep flag cannot identify which subscriber already woke. +IOReturn XeHost::wrapDisplayChange(void *pipe,void *reference,IOFramebuffer *fb,int event,void *argument) { + auto h=callbackHost; + const bool tracked=(event==3 || event==4) && !argument && h->xe.ownedDisplay13Framebuffer(fb); + if((event!=3 && event!=4) || argument) + return FunctionCast(wrapDisplayChange,h->orgDisplayChange)(pipe,reference,fb,event,argument); + SleepScope scope;scope.thread=reinterpret_cast(current_thread());scope.fb=tracked && event==3 ? fb : nullptr; + auto saved=IOSimpleLockLockDisableInterrupt(h->sleepScopeLock); + scope.next=h->sleepScopes;h->sleepScopes=&scope; + IOSimpleLockUnlockEnableInterrupt(h->sleepScopeLock,saved); + const auto result=FunctionCast(wrapDisplayChange,h->orgDisplayChange)(pipe,reference,fb,event,argument); + saved=IOSimpleLockLockDisableInterrupt(h->sleepScopeLock); + // The list holds stack records only during a synchronous native callback. + // Another thread or a nested subscriber must never inherit this decision. + auto link=&h->sleepScopes; + while(*link && *link!=&scope)link=&(*link)->next; + if(*link)*link=scope.next; + SleepEntry *entry=nullptr; + if(tracked)for(auto &item:h->sleepEntries)if(item.fb==fb){entry=&item;break;} + if(tracked && !entry)for(auto &item:h->sleepEntries)if(!item.pending){entry=&item;item.fb=fb;break;} + if(entry && result==kIOReturnSuccess) { + if(event==3 && scope.reached){entry->pending=true;++h->sleepNotifications;} + if(event==4){entry->pending=false;++h->wakeNotifications;} + } + uint32_t pending=0; + for(auto &item:h->sleepEntries)if(item.pending)++pending; + uint32_t receipt[]={1,h->sleepNotifications,h->wakeNotifications,h->repeatedSleep,pending, + static_cast(event),scope.reached,scope.skipped,static_cast(result)}; + IOSimpleLockUnlockEnableInterrupt(h->sleepScopeLock,saved); + IORegistryEntry::getRegistryRoot()->setProperty("WEXHostSleepScopes",receipt,sizeof(receipt)); + return result; +} + +uint32_t XeHost::pendingSleepDisplays(void *accelerator) { + auto h=callbackHost; + auto bytes=static_cast(accelerator); + auto machine=*reinterpret_cast(bytes+0x378); + const auto count=reinterpret_cast(h->sleepFBCount)(machine); + if(!count || !h->sleepScopeLock)return count; + const auto thread=reinterpret_cast(current_thread()); + auto saved=IOSimpleLockLockDisableInterrupt(h->sleepScopeLock); + SleepScope *scope=h->sleepScopes; + while(scope && scope->thread!=thread)scope=scope->next; + bool duplicate=false; + if(scope && scope->fb) { + scope->reached=true; + for(auto &entry:h->sleepEntries)if(entry.fb==scope->fb && entry.pending){duplicate=true;break;} + scope->skipped=duplicate; + if(duplicate)++h->repeatedSleep; + } + const auto repeated=h->repeatedSleep; + IOSimpleLockUnlockEnableInterrupt(h->sleepScopeLock,saved); + if(!duplicate)return count; + // Skip only the repeated preparation loop. Orphan cleanup, the hardware + // sleep phase and every native wake notification still execute normally. + const auto depth=*reinterpret_cast(machine+0x10c); + uint32_t record[]={repeated,count,depth}; + IORegistryEntry::getRegistryRoot()->setProperty("WEXDSIDarkWakeParked",record,sizeof(record)); + return 0; +} + +IOReturn XeHost::wrapWSAAAttribute(IOFramebuffer *fb,uint32_t attr,uint32_t value) { + // IOGraphics sends WillExitDefer under its global gate before calling the + // vendor attribute setter. Reject a failed scanout before queue-idle can + // sleep in that gate. No flip is acknowledged or GPU allocation released. + const auto state=value&kIOWSAA_StateMask; + if(attr==kIOWindowServerActiveAttribute && + (state==kIOWSAA_Accelerated || state==kIOWSAA_Unaccelerated) && + callbackHost->xe.failedDSIWakeFor(fb))return kIOReturnNotReady; + return FunctionCast(wrapWSAAAttribute,callbackHost->orgWSAAAttribute)(fb,attr,value); +} + bool XeHost::wrapPanelSetDisplay(IOService *panel,IOService *display) { // Add only the missing UID-19 profile for an actual internal 16-bit PWM // display. Keep Apple's/custom profiles and the native selection algorithm. diff --git a/WhateverXe/kern_host.hpp b/WhateverXe/kern_host.hpp index 903b46f8..8b76c9d3 100644 --- a/WhateverXe/kern_host.hpp +++ b/WhateverXe/kern_host.hpp @@ -8,7 +8,27 @@ class XeHost { private: static XeHost *callbackHost; XE xe; - bool active {}; + bool active {},systemWakeRecovery {}; + bool surfaceSeen {},graphicsSeen {},accelSeen {},providerSeen {}; + bool dependenciesSeen(size_t index) const {return surfaceSeen && + (!xe.isIntelFramebuffer(index) || !xe.requiresIntelProvider() || providerSeen) && + (!systemWakeRecovery || (graphicsSeen && accelSeen));} + void processLoadedDependency(KernelPatcher &,KernelPatcher::KextInfo &); + mach_vm_address_t sleepFBCount {}; + uint32_t repeatedSleep {},sleepNotifications {},wakeNotifications {}; + IOSimpleLock *sleepScopeLock {}; + struct SleepEntry {IOService *fb {};bool pending {};}; + SleepEntry sleepEntries[3] {}; + struct SleepScope { + SleepScope *next {};uintptr_t thread {};IOService *fb {}; + bool reached {},skipped {}; + }; + SleepScope *sleepScopes {}; + mach_vm_address_t orgDisplayChange {}; + static IOReturn wrapDisplayChange(void *,void *,IOFramebuffer *,int,void *); + static uint32_t pendingSleepDisplays(void *); + mach_vm_address_t orgWSAAAttribute {}; + static IOReturn wrapWSAAAttribute(IOFramebuffer *,uint32_t,uint32_t); bool backlightEnabled {}; IOPCIDevice *backlightGPU {}; mach_vm_address_t orgPanelSetDisplay {}; diff --git a/WhateverXe/kern_host_power_contract.hpp b/WhateverXe/kern_host_power_contract.hpp new file mode 100644 index 00000000..9cf408b4 --- /dev/null +++ b/WhateverXe/kern_host_power_contract.hpp @@ -0,0 +1,141 @@ +#pragma once +#include +#include + +// Host IOGraphics/IOAccelerator are OS components, not a fixed Intel donor. +// Match instruction operands and call targets; never their image offsets. +namespace HostPowerContract { +using Decode = size_t (*)(const uint8_t *,size_t); +inline bool equal(const uint8_t *a,const uint8_t *b,size_t n) { + for(size_t i=0;i=10 && p[i]==0x41 && p[i+1]==0x83 && p[i+2]==(r12?0xFC:0xFE)) { + // DidWake falls through the != branch; its paging write must be + // in that case. DidPowerOn branches directly to the other write. + if(p[i+3]==4 && p[i+4]==0x0F && p[i+5]==0x85) { + const auto end=int64_t(i+10)+int32_t(word(p+i+6)); + if(end>int64_t(i+10) && end<=int64_t(size)) { + size_t stop=i+10; + while(stop=sizeof(argument) && equal(p+branch,argument,sizeof(argument)))branch+=sizeof(argument); + if(size-branch>=2 && p[branch]==0x74) { + const auto target=int64_t(branch+2)+int8_t(p[branch+1]); + if(target>=0 && target+sizeof(bit)<=size && equal(p+target,bit,sizeof(bit)))++on; + } + } + } + const auto n=decode(p+i,size-i);if(!n)return false; + i+=n; + } + return wake==1 && on==1; +} +struct SleepSite { + size_t offset {}, length {}; + unsigned owner {}, success {}; + bool valid {}; +}; +inline bool savedRegister(unsigned reg) { return reg==3 || (reg>=12 && reg<=15); } +// Decode mov rdi,[callee-saved owner+disp32]. R12 needs an explicit SIB. +inline size_t machineLoad(const uint8_t *p,size_t size,unsigned &owner) { + if(size<7 || (p[0]!=0x48 && p[0]!=0x49) || p[1]!=0x8B || (p[2]&0xF8)!=0xB8)return 0; + owner=(p[2]&7)+((p[0]&1)?8:0); + if(!savedRegister(owner))return 0; + const size_t disp=(owner&7)==4 ? 4 : 3; + if(size=8)),0x89,uint8_t(0xF8|(owner&7))}; + return occurrences(p,size,save,sizeof(save),decode)==1; +} +inline SleepSite sleepSite(const uint8_t *p,size_t size,uintptr_t address,uintptr_t count,Decode decode) { + SleepSite result;unsigned hits=0,calls=0; + for(size_t i=0;isize-i)return {}; + if(n==5 && p[i]==0xE8 && address+i+5+int32_t(word(p+i+1))==count)++calls; + unsigned owner=0; + const auto load=machineLoad(p+i,size-i,owner); + const auto call=i+load; + if(load && size-call>=5 && p[call]==0xE8 && address+call+5+int32_t(word(p+call+1))==count) { + auto next=call+5; + unsigned high=0; + if(next=6 && (p[next]&0xF8)==0xB0 && p[next+1]==1) { + const unsigned success=(p[next]&7)+high; + next+=2; + if(savedRegister(success) && success!=owner && p[next]==0x85 && p[next+1]==0xC0 && p[next+2]==0x74) { + const auto end=int64_t(next+4)+int8_t(p[next+3]); + if(end>int64_t(next+4) && end<=int64_t(size) && ownerFromThis(p,i,owner,decode)) { + unsigned bounds=0; + for(size_t j=next+4;j=len+5 && p[j+len]==0xE8 && + address+j+len+5+int32_t(word(p+j+len+1))==count)++bounds; + const auto k=decode(p+j,size_t(end)-j);if(!k || k>size_t(end)-j)return {}; + j+=k; + } + if(bounds==1){result={i,load+5,owner,success,true};++hits;} + } + } + } + } + i+=n; + } + return hits==1 && calls==2 ? result : SleepSite{}; +} +inline bool sleepFields(const uint8_t *sleep,size_t sleepSize,const uint8_t *mode,size_t modeSize,Decode decode,unsigned owner) { + const uint8_t depthRBX[]={0x8B,0x83,0x0C,1,0,0,0x8D,0x48,1,0x89,0x8B,0x0C,1,0,0}; + const uint8_t depthR12[]={0x41,0x8B,0x84,0x24,0x0C,1,0,0,0x8D,0x48,1,0x41,0x89,0x8C,0x24,0x0C,1,0,0}; + return ownerFromThis(sleep,sleepSize,owner,decode) && + ((ownerFromThis(mode,modeSize,3,decode) && occurrences(mode,modeSize,depthRBX,sizeof(depthRBX),decode)==1) || + (ownerFromThis(mode,modeSize,12,decode) && occurrences(mode,modeSize,depthR12,sizeof(depthR12),decode)==1)); +} +} diff --git a/WhateverXe/kern_tgl_recovery.hpp b/WhateverXe/kern_tgl_recovery.hpp index cf82e536..4da1a6b5 100644 --- a/WhateverXe/kern_tgl_recovery.hpp +++ b/WhateverXe/kern_tgl_recovery.hpp @@ -1,4 +1,4 @@ -// Exact TGL donor layout, with the 20G1443 ICL recovery caller contract. +// Exact TGL donor layout, with the native ICL single-pass recovery contract. // The caller must qualify the binary and select host scheduler 5 first. #ifndef kern_tgl_recovery_hpp #define kern_tgl_recovery_hpp @@ -15,7 +15,7 @@ struct Result { bool resetOK; }; -// ICL 20G1443 resetHardwareAndReplay (4A712): reset once, account a native +// ICL resetHardwareAndReplay: reset once, account a native // failure, resume scheduling, return to IOAccel's error/event processing. // TGL's lower reset still owns hardware reset, context replay and CSB handling. // Do not append a new stamp or wait inside this recovery work-loop callback. diff --git a/WhateverXe/kern_xe.cpp b/WhateverXe/kern_xe.cpp index 7b8316bf..b18630b9 100644 --- a/WhateverXe/kern_xe.cpp +++ b/WhateverXe/kern_xe.cpp @@ -1,5 +1,7 @@ #include "kern_display13_color.hpp" #include +#include +#include // // kern_xe.cpp // WhateverGreen @@ -16,6 +18,7 @@ #include "kern_xelp_call_target.hpp" #include "kern_xelp_native_placement.hpp" #include +#include #include "kern_alderlake_n.hpp" #include "kern_xelp_device.hpp" #include "kern_xelp_memory_layout.hpp" @@ -156,9 +159,16 @@ extern "C" __attribute__((naked,used)) void tglTahoeMatrixBridge() { "1: jmpq *_tglSonomaMatrixZero(%rip)\n" "2: jmpq *_tglSonomaMatrixOther(%rip)\n"); } -static bool tglSonomaSurfaceFramebufferReady=true; +static XELPNative::SurfaceABI tglHostSurfaceABI[2] {}; +void XE::setHostSurfaceABI(unsigned framebuffer,unsigned accelerator) { + tglHostSurfaceABI[0]=static_cast(framebuffer); + tglHostSurfaceABI[1]=static_cast(accelerator); +} +static bool tglSonomaSurfaceFramebufferReady=false; static bool tglInstallSonomaSurfaceABI(KernelPatcher &patcher,size_t index,mach_vm_address_t address,size_t size,bool framebuffer) { - if((XELP::kernelOS(version)!=XELP::OS::Sonoma) && (XELP::kernelOS(version)!=XELP::OS::Sequoia) && (XELP::kernelOS(version)!=XELP::OS::Tahoe))return true; + const auto layout=tglHostSurfaceABI[framebuffer?0:1]; + if(layout==XELPNative::SurfaceABI::Unknown)return false; + if(layout==XELPNative::SurfaceABI::Original)return true; struct Site {size_t offset,length;uint8_t before[6],after[6];}; const Site fb[] { {0x2B5E8,6,{0x8B,0x80,0xA4,0,0,0},{0x8B,0x80,0x80,0,0,0}}, // pixelFormat @@ -186,7 +196,7 @@ static bool tglInstallSonomaSurfaceABI(KernelPatcher &patcher,size_t index,mach_ if(!framebuffer) { tglSonomaMatrixZero=address+0x8130E;tglSonomaMatrixOne=address+0x81305;tglSonomaMatrixOther=address+0x81361; uint8_t jump[15] {0xFF,0x25,0,0,0,0}; - const auto target=reinterpret_cast((XELP::kernelOS(version)==XELP::OS::Tahoe) ? tglTahoeMatrixBridge : tglSonomaMatrixBridge); + const auto target=reinterpret_cast((layout==XELPNative::SurfaceABI::Attachment6E) ? tglTahoeMatrixBridge : tglSonomaMatrixBridge); memcpy(jump+6,&target,sizeof(target));jump[14]=0x90; patcher.routeBlock(address+0x812F6,jump,sizeof(jump)); ready=patcher.getError()==KernelPatcher::Error::NoError && !memcmp(reinterpret_cast(address+0x812F6),jump,sizeof(jump)); @@ -228,6 +238,9 @@ static bool xelpInstallNVRAMABI(KernelPatcher &patcher,size_t index,mach_vm_addr XE *XE::callbackXE; static IOPCIDevice *xelpSelectedGPU; static XELP::Profile xelpSelectedProfile; +static Display13Clock::Variant display13ClockVariant() { + return Display13Clock::variantForPCI(xelpSelectedProfile.identity); +} static bool xelpSelectedDevice(IOPCIDevice *pci) { return pci && pci==xelpSelectedGPU && xelpSelectedProfile.displayAdmitted() && XELP::same(xelpSelectedProfile,XELP::deviceProfile(pci)); @@ -1918,38 +1931,76 @@ static bool tglPrepareMMIOBAR(IOMemoryDescriptor *bar) { tglPmapBARFirstPage=first;tglPmapBAREndPage=end; return true; } +static size_t tglNativeInstructionSize(const uint8_t *bytes,size_t length) { + // HDE may read ahead by 32 bytes. Decode a padded local copy, then require + // the complete instruction to lie in the nofault-read native span. + uint8_t code[32] {};memcpy(code,bytes,length(code),&ins); + return !(ins.flags&F_ERROR) && size<=length ? size : 0; +} +static bool tglNativeProtectCall(uintptr_t protect,uintptr_t pmap,XELPNative::ProtectABI abi) { + uint8_t code[512] {}; + return tglVM.nofault && protect && pmap && + tglVM.nofault(protect,reinterpret_cast(code),sizeof(code))==sizeof(code) && + XELPNative::protectCall(code,sizeof(code),protect,pmap,abi,tglNativeInstructionSize); +} +// The kernel can reorder basic blocks and symbols. Lilu locates the +// function and searches its code; our only extra constraints are the live +// instruction boundaries and the stack/register ABI used by each bridge. +struct TGLNativePmapCode { + static constexpr size_t capacity=0x4000; + uint8_t *storage {};size_t length {}; + TGLNativePmapCode(KernelPatcher &patcher,uintptr_t fn) { + const auto panic=patcher.solveSymbol(KernelPatcher::KernelID,"_panic");patcher.clearError(); + if(!tglVM.nofault || !fn || !panic)return; + storage=Buffer::create(capacity*4);if(!storage)return; + const auto copied=tglVM.nofault(fn,reinterpret_cast(storage),capacity); + if(copied)length=XELPNative::reachable(storage,copied,storage+capacity, + reinterpret_cast(storage+capacity*2),fn,panic,tglNativeInstructionSize); + } + ~TGLNativePmapCode(){if(storage)Buffer::deleter(storage);} + template bool find(const uint8_t (&pattern)[N],size_t &offset,bool requireUnique=true) const { + return length && XELPNative::findInstruction(storage,length,storage+capacity, + pattern,N,offset,KernelPatcher::findPattern,requireUnique); + } +}; static void tglPrepareMontereyMMIOUC(KernelPatcher &patcher,IOPCIDevice *pci) { tglMMIOUCReady=false; auto bar=pci->getDeviceMemoryWithRegister(0x10); auto fn=patcher.solveSymbol(KernelPatcher::KernelID,"_pmap_enter_options"); auto protect=patcher.solveSymbol(KernelPatcher::KernelID,"_IOProtectCacheMode"); auto kernelMap=patcher.solveSymbol(KernelPatcher::KernelID,"_kernel_map"); - const uint8_t frame[] {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x81,0xEC,0x08,0x01,0,0}; + const uint8_t frame[] {0x55,0x48,0x89,0xE5}; // RBP frame; total stack allocation is not bridge ABI const uint8_t args[] {0x44,0x89,0x4D,0x88,0x89,0x95,0x70,0xFF,0xFF,0xFF,0x89,0xD7,0x49,0x8B,0x77,0x40}; const uint8_t nonEPT[] {0x48,0x89,0x75,0x90}; const uint8_t protection[] {0x89,0x4D,0x8C}; const uint8_t create[] {0x8B,0x4D,0x8C,0x89,0xCB,0x83,0xE3,1,0x48,0x09,0xD3,0xF6,0xC1,2}; const uint8_t update[] {0x8B,0x4D,0x8C,0x41,0x89,0xCE,0x41,0x83,0xE6,1,0x49,0x09,0xC6,0xF6,0xC1,2}; - const uint8_t protectFrame[] {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x83,0xEC,0x28}; - const uint8_t protectCall[] {0xE8,0x1A,0x99,0x02,0}; + TGLNativePmapCode native(patcher,fn); + size_t frameAt=0;const bool frameReady=native.find(frame,frameAt,false); + size_t argsAt=0;const bool argsReady=native.find(args,argsAt,false); + size_t protectionAt=0;const bool protectionReady=native.find(protection,protectionAt,false); + size_t nonEPTAt=0;const bool nonEPTReady=native.find(nonEPT,nonEPTAt,false); + size_t createAt=0;const bool createReady=native.find(create,createAt); + size_t updateAt=0;const bool updateReady=native.find(update,updateAt); + const bool protectCallReady=tglNativeProtectCall(protect,fn,XELPNative::ProtectABI::Legacy); uint32_t lo,hi;asm volatile("rdmsr" : "=a"(lo),"=d"(hi) : "c"(0x277)); const uint64_t pat=static_cast(hi)<<32|lo; bool ok=tglVM.enabled && tglPrepareMMIOBAR(bar) && - pat==0x1040600070406ULL && fn && protect && kernelMap && - !memcmp(reinterpret_cast(fn),frame,sizeof(frame)) && - !memcmp(reinterpret_cast(fn+0x51),args,sizeof(args)) && - !memcmp(reinterpret_cast(fn+0x78),protection,sizeof(protection)) && - !memcmp(reinterpret_cast(fn+0x116),nonEPT,sizeof(nonEPT)) && - !memcmp(reinterpret_cast(fn+0x15F9),create,sizeof(create)) && - !memcmp(reinterpret_cast(fn+0x12CE),update,sizeof(update)) && - !memcmp(reinterpret_cast(protect),protectFrame,sizeof(protectFrame)) && - !memcmp(reinterpret_cast(protect+0xC1),protectCall,sizeof(protectCall)) && + pat==0x1040600070406ULL && fn && protect && kernelMap && protectCallReady && + frameReady && + argsReady && + protectionReady && + nonEPTReady && + createReady && + updateReady && tglVMRead(kernelMap,tglKernelMap) && (tglKernelMap>>47)==0x1FFFF; if(ok) { tglProtectCacheMode=reinterpret_cast(protect); - tglPmapNewResume=fn+0x1607;tglPmapUpdateResume=fn+0x12DE; + tglPmapNewResume=fn+createAt+sizeof(create);tglPmapUpdateResume=fn+updateAt+sizeof(update); const uintptr_t targets[] {reinterpret_cast(tglMontereyPmapNewBridge),reinterpret_cast(tglMontereyPmapUpdateBridge)}; - const uintptr_t sites[] {fn+0x15F9,fn+0x12CE}; + const uintptr_t sites[] {fn+createAt,fn+updateAt}; const size_t lengths[] {sizeof(create),sizeof(update)}; for(unsigned i=0;i<2 && ok;++i) { uint8_t jump[16] {0xFF,0x25,0,0,0,0};jump[14]=jump[15]=0x90; @@ -1969,34 +2020,39 @@ static void tglPrepareVenturaMMIOUC(KernelPatcher &patcher,IOPCIDevice *pci) { auto fn=patcher.solveSymbol(KernelPatcher::KernelID,"_pmap_enter_options"); auto protect=patcher.solveSymbol(KernelPatcher::KernelID,"_IOProtectCacheMode"); auto kernelMap=patcher.solveSymbol(KernelPatcher::KernelID,"_kernel_map"); - const uint8_t frame[] {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x81,0xEC,0xC8,0x00,0x00,0x00}; + const uint8_t frame[] {0x55,0x48,0x89,0xE5}; // RBP frame; total stack allocation is not bridge ABI const uint8_t flags[] {0x44,0x89,0x4D,0xBC}; const uint8_t args[] {0x89,0x55,0x94,0x89,0xD7,0x49,0x8B,0x75,0x40}; const uint8_t nonEPT[] {0x48,0x89,0x75,0xB0}; const uint8_t protection[] {0x89,0x4D,0xAC}; const uint8_t create[] {0x8B,0x75,0xAC,0x89,0xF1,0x83,0xE1,0x01,0x48,0x09,0xD1,0x40,0xF6,0xC6,0x02}; const uint8_t update[] {0x8B,0x55,0xAC,0x89,0xD0,0x83,0xE0,0x01,0x48,0x09,0xC8,0xF6,0xC2,0x02}; - const uint8_t protectFrame[] {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x83,0xEC,0x28}; - const uint8_t protectCall[] {0xE8,0xBA,0x51,0x02,0x00}; + TGLNativePmapCode native(patcher,fn); + size_t frameAt=0;const bool frameReady=native.find(frame,frameAt,false); + size_t flagsAt=0;const bool flagsReady=native.find(flags,flagsAt,false); + size_t argsAt=0;const bool argsReady=native.find(args,argsAt,false); + size_t protectionAt=0;const bool protectionReady=native.find(protection,protectionAt,false); + size_t nonEPTAt=0;const bool nonEPTReady=native.find(nonEPT,nonEPTAt,false); + size_t createAt=0;const bool createReady=native.find(create,createAt); + size_t updateAt=0;const bool updateReady=native.find(update,updateAt); + const bool protectCallReady=tglNativeProtectCall(protect,fn,XELPNative::ProtectABI::Legacy); uint32_t lo,hi;asm volatile("rdmsr" : "=a"(lo),"=d"(hi) : "c"(0x277)); const uint64_t pat=static_cast(hi)<<32|lo; bool ok=tglVM.enabled && tglPrepareMMIOBAR(bar) && - pat==0x1040600070406ULL && fn && protect && kernelMap && - !memcmp(reinterpret_cast(fn),frame,sizeof(frame)) && - !memcmp(reinterpret_cast(fn+0x14),flags,sizeof(flags)) && - !memcmp(reinterpret_cast(fn+0x47),args,sizeof(args)) && - !memcmp(reinterpret_cast(fn+0x64),protection,sizeof(protection)) && - !memcmp(reinterpret_cast(fn+0xC1),nonEPT,sizeof(nonEPT)) && - !memcmp(reinterpret_cast(fn+0x13F4),create,sizeof(create)) && - !memcmp(reinterpret_cast(fn+0x1097),update,sizeof(update)) && - !memcmp(reinterpret_cast(protect),protectFrame,sizeof(protectFrame)) && - !memcmp(reinterpret_cast(protect+0xC1),protectCall,sizeof(protectCall)) && + pat==0x1040600070406ULL && fn && protect && kernelMap && protectCallReady && + frameReady && + flagsReady && + argsReady && + protectionReady && + nonEPTReady && + createReady && + updateReady && tglVMRead(kernelMap,tglKernelMap) && (tglKernelMap>>47)==0x1FFFF; if(ok) { tglProtectCacheMode=reinterpret_cast(protect); - tglPmapNewResume=fn+0x1403;tglPmapUpdateResume=fn+0x10A5; + tglPmapNewResume=fn+createAt+sizeof(create);tglPmapUpdateResume=fn+updateAt+sizeof(update); const uintptr_t targets[] {reinterpret_cast(tglVenturaPmapNewBridge),reinterpret_cast(tglVenturaPmapUpdateBridge)}; - const uintptr_t sites[] {fn+0x13F4,fn+0x1097}; + const uintptr_t sites[] {fn+createAt,fn+updateAt}; const size_t lengths[] {sizeof(create),sizeof(update)}; for(unsigned i=0;i<2 && ok;++i) { uint8_t jump[16] {0xFF,0x25,0,0,0,0};jump[14]=jump[15]=0x90; @@ -2010,64 +2066,51 @@ static void tglPrepareVenturaMMIOUC(KernelPatcher &patcher,IOPCIDevice *pci) { auto evidence=OSSerializer::forTarget(nullptr,tglMMIOEvidenceSerialize); if(evidence) {root->setProperty("TGLMMIOMappingEvidence",evidence);evidence->release();} } -static size_t tglNativeInstructionSize(const uint8_t *bytes,size_t length) { - // HDE may read ahead by 32 bytes. Decode a padded local copy, then require - // the complete instruction to lie in the nofault-read native span. - uint8_t code[32] {};memcpy(code,bytes,length(code),&ins); - return !(ins.flags&F_ERROR) && size<=length ? size : 0; -} -static bool tglNativeProtectCall(uintptr_t protect,uintptr_t pmap,bool tahoe) { - uint8_t code[512] {}; - return tglVM.nofault && protect && pmap && - tglVM.nofault(protect,reinterpret_cast(code),sizeof(code))==sizeof(code) && - XELPNative::protectCall(code,sizeof(code),protect,pmap,tahoe,tglNativeInstructionSize); -} static void tglPrepareTahoeMMIOUC(KernelPatcher &patcher,IOPCIDevice *pci) { tglMMIOUCReady=false; auto bar=pci->getDeviceMemoryWithRegister(0x10); auto fn=patcher.solveSymbol(KernelPatcher::KernelID,"_pmap_enter_options"); auto protect=patcher.solveSymbol(KernelPatcher::KernelID,"_IOProtectCacheMode"); auto kernelMap=patcher.solveSymbol(KernelPatcher::KernelID,"_kernel_map"); - const uint8_t frame[] {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x81,0xEC,0xC8,0x00,0x00,0x00}; + const uint8_t frame[] {0x55,0x48,0x89,0xE5}; // RBP frame; total stack allocation is not bridge ABI const uint8_t flags[] {0x44,0x89,0x6D,0xB0}; const uint8_t args[] {0x89,0x55,0x9C,0x89,0xD7,0x49,0x8B,0x76,0x40}; const uint8_t nonEPT[] {0x48,0x89,0x75,0xC0}; const uint8_t protection[] {0x89,0x4D,0xB4}; const uint8_t create[] {0x8B,0x75,0xB4,0x89,0xF1,0x83,0xE1,0x01,0x48,0x09,0xD1,0x40,0xF6,0xC6,0x02}; const uint8_t update[] {0x8B,0x55,0xB4,0x89,0xD0,0x83,0xE0,0x01,0x48,0x09,0xC8,0xF6,0xC2,0x02}; - const uint8_t protectFrame[] {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x83,0xEC,0x28}; - const bool protectCallReady=tglNativeProtectCall(protect,fn,true); + TGLNativePmapCode native(patcher,fn); + size_t frameAt=0;const bool frameReady=native.find(frame,frameAt,false); + size_t flagsAt=0;const bool flagsReady=native.find(flags,flagsAt,false); + size_t argsAt=0;const bool argsReady=native.find(args,argsAt,false); + size_t protectionAt=0;const bool protectionReady=native.find(protection,protectionAt,false); + size_t nonEPTAt=0;const bool nonEPTReady=native.find(nonEPT,nonEPTAt,false); + size_t createAt=0;const bool createReady=native.find(create,createAt); + size_t updateAt=0;const bool updateReady=native.find(update,updateAt); + const bool protectCallReady=tglNativeProtectCall(protect,fn,XELPNative::ProtectABI::Tahoe); uint32_t lo,hi;asm volatile("rdmsr" : "=a"(lo),"=d"(hi) : "c"(0x277)); const uint64_t pat=static_cast(hi)<<32|lo; bool ok=tglVM.enabled && tglPrepareMMIOBAR(bar) && pat==0x1040600070406ULL && fn && protect && kernelMap && - !memcmp(reinterpret_cast(fn),frame,sizeof(frame)) && - !memcmp(reinterpret_cast(fn+0x15A),flags,sizeof(flags)) && - !memcmp(reinterpret_cast(fn+0x44),args,sizeof(args)) && - !memcmp(reinterpret_cast(fn+0x61),protection,sizeof(protection)) && - !memcmp(reinterpret_cast(fn+0xAA),nonEPT,sizeof(nonEPT)) && - !memcmp(reinterpret_cast(fn+0x10C1),create,sizeof(create)) && - !memcmp(reinterpret_cast(fn+0x1197),update,sizeof(update)) && - !memcmp(reinterpret_cast(protect),protectFrame,sizeof(protectFrame)) && + frameReady && + flagsReady && + argsReady && + protectionReady && + nonEPTReady && + createReady && + updateReady && protectCallReady && tglVMRead(kernelMap,tglKernelMap) && (tglKernelMap>>47)==0x1FFFF; uint64_t guard[20] {1,tglVM.enabled,tglVM.pciBAR,bar ? bar->getLength() : 0,pat,bool(fn),bool(protect),bool(kernelMap)}; - if(fn && protect) { - const uintptr_t sites[] {fn,fn+0x15A,fn+0x44,fn+0x61,fn+0xAA,fn+0x10C1,fn+0x1197,protect}; - const uint8_t *bytes[] {frame,flags,args,protection,nonEPT,create,update,protectFrame}; - const size_t lengths[] {sizeof(frame),sizeof(flags),sizeof(args),sizeof(protection),sizeof(nonEPT),sizeof(create),sizeof(update),sizeof(protectFrame)}; - for(unsigned i=0;i<8;++i)guard[8+i]=!memcmp(reinterpret_cast(sites[i]),bytes[i],lengths[i]); - guard[16]=protectCallReady; - } + const bool checks[] {frameReady,flagsReady,argsReady,protectionReady,nonEPTReady,createReady,updateReady,protectCallReady}; + for(unsigned i=0;isetProperty("TGLTahoeMMIOGuard",guard,sizeof(guard)); if(ok) { tglProtectCacheMode=reinterpret_cast(protect); - tglPmapNewResume=fn+0x10D0;tglPmapUpdateResume=fn+0x11A5; + tglPmapNewResume=fn+createAt+sizeof(create);tglPmapUpdateResume=fn+updateAt+sizeof(update); const uintptr_t targets[] {reinterpret_cast(tglTahoePmapNewBridge),reinterpret_cast(tglTahoePmapUpdateBridge)}; - const uintptr_t sites[] {fn+0x10C1,fn+0x1197}; + const uintptr_t sites[] {fn+createAt,fn+updateAt}; const size_t lengths[] {sizeof(create),sizeof(update)}; for(unsigned i=0;i<2 && ok;++i) { uint8_t jump[16] {0xFF,0x25,0,0,0,0};jump[14]=jump[15]=0x90; @@ -2087,44 +2130,45 @@ static void tglPrepareSequoiaMMIOUC(KernelPatcher &patcher,IOPCIDevice *pci) { auto fn=patcher.solveSymbol(KernelPatcher::KernelID,"_pmap_enter_options"); auto protect=patcher.solveSymbol(KernelPatcher::KernelID,"_IOProtectCacheMode"); auto kernelMap=patcher.solveSymbol(KernelPatcher::KernelID,"_kernel_map"); - const uint8_t frame[] {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x81,0xEC,0xC8,0x00,0x00,0x00}; + const uint8_t frame[] {0x55,0x48,0x89,0xE5}; // RBP frame; total stack allocation is not bridge ABI const uint8_t flags[] {0x44,0x89,0x65,0xB0}; const uint8_t args[] {0x89,0x55,0x94,0x89,0xD7,0x49,0x8B,0x76,0x40}; const uint8_t nonEPT[] {0x48,0x89,0x75,0xB8}; const uint8_t protection[] {0x89,0x4D,0xB4}; const uint8_t create[] {0x8B,0x75,0xB4,0x89,0xF1,0x83,0xE1,0x01,0x48,0x09,0xD1,0x40,0xF6,0xC6,0x02}; const uint8_t update[] {0x8B,0x55,0xB4,0x89,0xD0,0x83,0xE0,0x01,0x48,0x09,0xC8,0xF6,0xC2,0x02}; - const uint8_t protectFrame[] {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x83,0xEC,0x28}; - const bool protectCallReady=tglNativeProtectCall(protect,fn,false); + TGLNativePmapCode native(patcher,fn); + size_t frameAt=0;const bool frameReady=native.find(frame,frameAt,false); + size_t flagsAt=0;const bool flagsReady=native.find(flags,flagsAt,false); + size_t argsAt=0;const bool argsReady=native.find(args,argsAt,false); + size_t protectionAt=0;const bool protectionReady=native.find(protection,protectionAt,false); + size_t nonEPTAt=0;const bool nonEPTReady=native.find(nonEPT,nonEPTAt,false); + size_t createAt=0;const bool createReady=native.find(create,createAt); + size_t updateAt=0;const bool updateReady=native.find(update,updateAt); + const bool protectCallReady=tglNativeProtectCall(protect,fn,XELPNative::ProtectABI::Extended); uint32_t lo,hi;asm volatile("rdmsr" : "=a"(lo),"=d"(hi) : "c"(0x277)); const uint64_t pat=static_cast(hi)<<32|lo; bool ok=tglVM.enabled && tglPrepareMMIOBAR(bar) && pat==0x1040600070406ULL && fn && protect && kernelMap && - !memcmp(reinterpret_cast(fn),frame,sizeof(frame)) && - !memcmp(reinterpret_cast(fn+0x159),flags,sizeof(flags)) && - !memcmp(reinterpret_cast(fn+0x35),args,sizeof(args)) && - !memcmp(reinterpret_cast(fn+0x52),protection,sizeof(protection)) && - !memcmp(reinterpret_cast(fn+0xB8),nonEPT,sizeof(nonEPT)) && - !memcmp(reinterpret_cast(fn+0x1390),create,sizeof(create)) && - !memcmp(reinterpret_cast(fn+0x1030),update,sizeof(update)) && - !memcmp(reinterpret_cast(protect),protectFrame,sizeof(protectFrame)) && + frameReady && + flagsReady && + argsReady && + protectionReady && + nonEPTReady && + createReady && + updateReady && protectCallReady && tglVMRead(kernelMap,tglKernelMap) && (tglKernelMap>>47)==0x1FFFF; uint64_t guard[20] {1,tglVM.enabled,tglVM.pciBAR,bar ? bar->getLength() : 0,pat,bool(fn),bool(protect),bool(kernelMap)}; - if(fn && protect) { - const uintptr_t sites[] {fn,fn+0x159,fn+0x35,fn+0x52,fn+0xB8,fn+0x1390,fn+0x1030,protect}; - const uint8_t *bytes[] {frame,flags,args,protection,nonEPT,create,update,protectFrame}; - const size_t lengths[] {sizeof(frame),sizeof(flags),sizeof(args),sizeof(protection),sizeof(nonEPT),sizeof(create),sizeof(update),sizeof(protectFrame)}; - for(unsigned i=0;i<8;++i)guard[8+i]=!memcmp(reinterpret_cast(sites[i]),bytes[i],lengths[i]); - guard[16]=protectCallReady; - } + const bool checks[] {frameReady,flagsReady,argsReady,protectionReady,nonEPTReady,createReady,updateReady,protectCallReady}; + for(unsigned i=0;isetProperty("TGLSequoiaMMIOGuard",guard,sizeof(guard)); if(ok) { tglProtectCacheMode=reinterpret_cast(protect); - tglPmapNewResume=fn+0x139F;tglPmapUpdateResume=fn+0x103E; + tglPmapNewResume=fn+createAt+sizeof(create);tglPmapUpdateResume=fn+updateAt+sizeof(update); const uintptr_t targets[] {reinterpret_cast(tglSequoiaPmapNewBridge),reinterpret_cast(tglSequoiaPmapUpdateBridge)}; - const uintptr_t sites[] {fn+0x1390,fn+0x1030}; + const uintptr_t sites[] {fn+createAt,fn+updateAt}; const size_t lengths[] {sizeof(create),sizeof(update)}; for(unsigned i=0;i<2 && ok;++i) { uint8_t jump[16] {0xFF,0x25,0,0,0,0};jump[14]=jump[15]=0x90; @@ -2144,43 +2188,44 @@ static void tglPrepareSonomaMMIOUC(KernelPatcher &patcher,IOPCIDevice *pci) { auto fn=patcher.solveSymbol(KernelPatcher::KernelID,"_pmap_enter_options"); auto protect=patcher.solveSymbol(KernelPatcher::KernelID,"_IOProtectCacheMode"); auto kernelMap=patcher.solveSymbol(KernelPatcher::KernelID,"_kernel_map"); - const uint8_t frame[] {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x81,0xEC,0xC8,0x00,0x00,0x00}; + const uint8_t frame[] {0x55,0x48,0x89,0xE5}; // RBP frame; total stack allocation is not bridge ABI const uint8_t flags[] {0x44,0x89,0x4D,0xB4}; const uint8_t args[] {0x89,0x55,0x9C,0x89,0xD6,0x4C,0x8B,0x6F,0x40}; const uint8_t nonEPT[] {0x4C,0x89,0x6D,0xB8}; const uint8_t protection[] {0x89,0x4D,0xB0}; const uint8_t create[] {0x8B,0x75,0xB0,0x89,0xF1,0x83,0xE1,0x01,0x48,0x09,0xD1,0x40,0xF6,0xC6,0x02}; const uint8_t update[] {0x8B,0x55,0xB0,0x89,0xD0,0x83,0xE0,0x01,0x48,0x09,0xC8,0xF6,0xC2,0x02}; - const uint8_t protectFrame[] {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x83,0xEC,0x28}; - const uint8_t protectCall[] {0xE8,0xDC,0x8F,0x02,0x00}; + TGLNativePmapCode native(patcher,fn); + size_t frameAt=0;const bool frameReady=native.find(frame,frameAt,false); + size_t flagsAt=0;const bool flagsReady=native.find(flags,flagsAt,false); + size_t argsAt=0;const bool argsReady=native.find(args,argsAt,false); + size_t protectionAt=0;const bool protectionReady=native.find(protection,protectionAt,false); + size_t nonEPTAt=0;const bool nonEPTReady=native.find(nonEPT,nonEPTAt,false); + size_t createAt=0;const bool createReady=native.find(create,createAt); + size_t updateAt=0;const bool updateReady=native.find(update,updateAt); + const bool protectCallReady=tglNativeProtectCall(protect,fn,XELPNative::ProtectABI::Extended); uint32_t lo,hi;asm volatile("rdmsr" : "=a"(lo),"=d"(hi) : "c"(0x277)); const uint64_t pat=static_cast(hi)<<32|lo; bool ok=tglVM.enabled && tglPrepareMMIOBAR(bar) && - pat==0x1040600070406ULL && fn && protect && kernelMap && - !memcmp(reinterpret_cast(fn),frame,sizeof(frame)) && - !memcmp(reinterpret_cast(fn+0x14),flags,sizeof(flags)) && - !memcmp(reinterpret_cast(fn+0x48),args,sizeof(args)) && - !memcmp(reinterpret_cast(fn+0x65),protection,sizeof(protection)) && - !memcmp(reinterpret_cast(fn+0x103),nonEPT,sizeof(nonEPT)) && - !memcmp(reinterpret_cast(fn+0x12F2),create,sizeof(create)) && - !memcmp(reinterpret_cast(fn+0xFE1),update,sizeof(update)) && - !memcmp(reinterpret_cast(protect),protectFrame,sizeof(protectFrame)) && - !memcmp(reinterpret_cast(protect+0xBF),protectCall,sizeof(protectCall)) && + pat==0x1040600070406ULL && fn && protect && kernelMap && protectCallReady && + frameReady && + flagsReady && + argsReady && + protectionReady && + nonEPTReady && + createReady && + updateReady && tglVMRead(kernelMap,tglKernelMap) && (tglKernelMap>>47)==0x1FFFF; uint64_t guard[20] {1,tglVM.enabled,tglVM.pciBAR,bar ? bar->getLength() : 0,pat,bool(fn),bool(protect),bool(kernelMap)}; - if(fn && protect) { - const uintptr_t sites[] {fn,fn+0x14,fn+0x48,fn+0x65,fn+0x103,fn+0x12F2,fn+0xFE1,protect,protect+0xBF}; - const uint8_t *bytes[] {frame,flags,args,protection,nonEPT,create,update,protectFrame,protectCall}; - const size_t lengths[] {sizeof(frame),sizeof(flags),sizeof(args),sizeof(protection),sizeof(nonEPT),sizeof(create),sizeof(update),sizeof(protectFrame),sizeof(protectCall)}; - for(unsigned i=0;i<9;++i)guard[8+i]=!memcmp(reinterpret_cast(sites[i]),bytes[i],lengths[i]); - } + const bool checks[] {frameReady,flagsReady,argsReady,protectionReady,nonEPTReady,createReady,updateReady,protectCallReady}; + for(unsigned i=0;isetProperty("TGLSonomaMMIOGuard",guard,sizeof(guard)); if(ok) { tglProtectCacheMode=reinterpret_cast(protect); - tglPmapNewResume=fn+0x1301;tglPmapUpdateResume=fn+0xFEF; + tglPmapNewResume=fn+createAt+sizeof(create);tglPmapUpdateResume=fn+updateAt+sizeof(update); const uintptr_t targets[] {reinterpret_cast(tglSonomaPmapNewBridge),reinterpret_cast(tglSonomaPmapUpdateBridge)}; - const uintptr_t sites[] {fn+0x12F2,fn+0xFE1}; + const uintptr_t sites[] {fn+createAt,fn+updateAt}; const size_t lengths[] {sizeof(create),sizeof(update)}; for(unsigned i=0;i<2 && ok;++i) { uint8_t jump[16] {0xFF,0x25,0,0,0,0};jump[14]=jump[15]=0x90; @@ -2217,28 +2262,36 @@ static void tglPrepareMMIOUC(KernelPatcher &patcher,IOPCIDevice *pci) { auto fn=patcher.solveSymbol(KernelPatcher::KernelID,"_pmap_enter_options"); auto protect=patcher.solveSymbol(KernelPatcher::KernelID,"_IOProtectCacheMode"); auto kernelMap=patcher.solveSymbol(KernelPatcher::KernelID,"_kernel_map"); - const uint8_t protectFrame[] {0x55,0x48,0x89,0xE5,0x41,0x57,0x41,0x56,0x41,0x55,0x41,0x54,0x53,0x48,0x83,0xEC,0x28}; - const uint8_t protectCall[] {0xE8,0xC9,0x66,0x02,0x00}; const uint8_t frame[] {0x89,0x55,0x98,0x89,0xD0,0x48,0x89,0x85,0xF0,0xFE,0xFF,0xFF,0x48,0xC1,0xE0,0x0C}; + const uint8_t flags[] {0x44,0x89,0x4D,0xB4}; + const uint8_t protection[] {0x89,0x4D,0x9C}; + const uint8_t rootSource[] {0x48,0x8B,0x4F,0x40}; const uint8_t nonEPT[] {0x48,0x89,0x4D,0xA8,0x4C,0x89,0x7D,0xC0}; const uint8_t create[] {0x8B,0x75,0x9C,0x89,0xF1,0x83,0xE1,1,0x48,0x09,0xD1,0x40,0xF6,0xC6,2}; const uint8_t update[] {0x8B,0x75,0x9C,0x89,0xF0,0x83,0xE0,1,0x48,0x09,0xD0,0x40,0xF6,0xC6,2}; + TGLNativePmapCode native(patcher,fn); + size_t frameAt=0;const bool frameReady=native.find(frame,frameAt); + size_t flagsAt=0;const bool flagsReady=native.find(flags,flagsAt,false); + size_t protectionAt=0;const bool protectionReady=native.find(protection,protectionAt,false); + size_t rootSourceAt=0;const bool rootSourceReady=native.find(rootSource,rootSourceAt,false); + size_t nonEPTAt=0;const bool nonEPTReady=native.find(nonEPT,nonEPTAt,false); + size_t createAt=0;const bool createReady=native.find(create,createAt); + size_t updateAt=0;const bool updateReady=native.find(update,updateAt); + const bool protectCallReady=tglNativeProtectCall(protect,fn,XELPNative::ProtectABI::Legacy); uint32_t lo,hi;asm volatile("rdmsr" : "=a"(lo),"=d"(hi) : "c"(0x277)); const uint64_t pat=static_cast(hi)<<32|lo; bool ok=native20G && tglPrepareMMIOBAR(bar) && - pat==0x1040600070406ULL && fn && protect && kernelMap && - !memcmp(reinterpret_cast(protect),protectFrame,sizeof(protectFrame)) && - !memcmp(reinterpret_cast(protect+0xD2),protectCall,sizeof(protectCall)) && + pat==0x1040600070406ULL && fn && protect && kernelMap && protectCallReady && tglVMRead(kernelMap,tglKernelMap) && (tglKernelMap>>47)==0x1FFFF && - !memcmp(reinterpret_cast(fn+0x48),frame,sizeof(frame)) && - !memcmp(reinterpret_cast(fn+0x157),nonEPT,sizeof(nonEPT)) && - !memcmp(reinterpret_cast(fn+0x15FA),create,sizeof(create)) && - !memcmp(reinterpret_cast(fn+0x1850),update,sizeof(update)); + frameReady && flagsReady && protectionReady && rootSourceReady && + nonEPTReady && + createReady && + updateReady; if (ok) { tglProtectCacheMode=reinterpret_cast(protect); - tglPmapEntryResume=fn+0x58; tglPmapNewResume=fn+0x1609; tglPmapUpdateResume=fn+0x185F; + tglPmapEntryResume=fn+frameAt+sizeof(frame); tglPmapNewResume=fn+createAt+sizeof(create); tglPmapUpdateResume=fn+updateAt+sizeof(update); const uintptr_t targets[] {reinterpret_cast(tglPmapEntryBridge),reinterpret_cast(tglPmapNewBridge),reinterpret_cast(tglPmapUpdateBridge)}; - const uintptr_t sites[] {fn+0x48,fn+0x15FA,fn+0x1850}; + const uintptr_t sites[] {fn+frameAt,fn+createAt,fn+updateAt}; for (unsigned i=0;i<3 && ok;i++) { uint8_t jump[16] {0xFF,0x25,0,0,0,0}; jump[14]=jump[15]=0x90; const size_t length=i ? 15 : 16; @@ -2264,30 +2317,14 @@ void tglVMPrepare(KernelPatcher &patcher,IOPCIDevice *pci) { auto nf=patcher.solveSymbol(KernelPatcher::KernelID,"_ml_nofault_copy"); patcher.clearError(); if (!verify || !kp || !pb || !pe || !nf) return; - const bool dp1=(XELP::kernelOS(version)==XELP::OS::Donor); - const bool monterey=(XELP::kernelOS(version)==XELP::OS::Monterey); - const bool ventura=(XELP::kernelOS(version)==XELP::OS::Ventura); - const bool sonoma=(XELP::kernelOS(version)==XELP::OS::Sonoma); - const bool sequoia=(XELP::kernelOS(version)==XELP::OS::Sequoia); - const bool tahoe=(XELP::kernelOS(version)==XELP::OS::Tahoe); - if(sequoia || tahoe) { - const auto store=patcher.solveSymbol(KernelPatcher::KernelID,"_kernel_pmap_store"); - patcher.clearError(); - uint8_t caller[96] {},accessor[64] {}; - const auto copy=reinterpret_cast(nf); - if(!store || copy(verify,reinterpret_cast(caller),sizeof(caller))!=sizeof(caller))return; - const auto target=XELPNative::rootCall(caller,sizeof(caller),verify,store); - if((target>>47)!=0x1FFFF || copy(target,reinterpret_cast(accessor),sizeof(accessor))!=sizeof(accessor) || - !XELPNative::rootAccessor(accessor,sizeof(accessor)))return; - } else { - const uint8_t callPrefix[] {0x48,0x89,0xDE,0xE8}; - if (memcmp(reinterpret_cast(verify+0x10),callPrefix,sizeof(callPrefix))) return; - int32_t displacement;memcpy(&displacement,reinterpret_cast(verify+0x14),4); - const auto target=verify+0x18+displacement; - if (target!=verify-(dp1 ? 0x3930 : monterey ? 0x4190 : ventura ? 0x3D70 : sonoma ? 0x3EF0 : 0x3AB0)) return; - const uint8_t rootLoad[] {0x48,0x8B,0x47,0x50}; - if (memcmp(reinterpret_cast(target+0x20),rootLoad,sizeof(rootLoad))) return; - } + const auto store=patcher.solveSymbol(KernelPatcher::KernelID,"_kernel_pmap_store"); + patcher.clearError(); + uint8_t caller[96] {},accessor[64] {}; + const auto copy=reinterpret_cast(nf); + if(!store || copy(verify,reinterpret_cast(caller),sizeof(caller))!=sizeof(caller))return; + const auto target=XELPNative::rootCall(caller,sizeof(caller),verify,store,kp); + if((target>>47)!=0x1FFFF || copy(target,reinterpret_cast(accessor),sizeof(accessor))!=sizeof(accessor) || + !XELPNative::rootAccessor(accessor,sizeof(accessor)))return; tglVM.nofault=reinterpret_cast(nf); if (!tglVMRead(kp,tglVM.kernelPmap) || !tglVMRead(pb,tglVM.directBase) || !tglVMRead(pe,tglVM.directEnd)) return; @@ -4029,7 +4066,7 @@ void XE::adlnBootstrap(IOPCIDevice *pci) { } auditDSIBoot(pci,io.base,io.length,result,barReady && reservationReady); if(checkKernelArgument("-wex-dsi-hdmi-test")) { - auto handoff=Display13DSITrial::transfer(io,barReady && xelpMemoryMatches(pci),result,xelpMemory,board.hdmi); + auto handoff=Display13DSITrial::transfer(io,barReady && xelpMemoryMatches(pci),result,xelpMemory,board.hdmi,display13ClockVariant()); pci->setProperty("WEXDisplay13DSIHandoff",&handoff,sizeof(handoff)); AlderLakeN::readDisplay(io,barReady,result); pci->setProperty("WEXDisplay13DSIAfter",&result,sizeof(result)); @@ -4048,8 +4085,17 @@ void XE::adlnBootstrap(IOPCIDevice *pci) { pci->setProperty("WEXDisplay13TCBootPorts",bootTC,sizeof(bootTC)); } + bool mirrorPrepared=false,mirrorFailed=false; + if(displayRequested && linearTest && board.dsi && result.activePipes) { + auto parked=Display13Boot::normalizeDSIMirror(io,result,barReady && reservationReady && xelpMemoryMatches(pci), + board,xelpMemory,dsiProfile,snapshot->normalize); + pci->setProperty("WEXDSIMirrorNormalize",&parked,sizeof(parked)); + mirrorPrepared=parked.status==Display13Boot::Status::Complete; + mirrorFailed=!mirrorPrepared; + AlderLakeN::readDisplay(io,barReady,result); + } if(displayRequested && board.dsi) - XELP::DSI::captureBoot(io,result,dsiProfile,xelpMemory,barReady && reservationReady,dsiBoot); + XELP::DSI::captureBoot(io,result,dsiProfile,xelpMemory,barReady && reservationReady && !mirrorFailed,dsiBoot,mirrorPrepared); bool bootstrapReady=!board.dsi || dsiBoot.usable(); if(!board.dsi && displayRequested && linearTest && (result.activePipes&(result.activePipes-1))) { auto parked=Display13Boot::normalize(io,result,barReady && Display13FBTopology::multiOutputWiring(board) && xelpMemoryMatches(pci),board,xelpMemory,snapshot->normalize); @@ -4508,6 +4554,23 @@ void XE::processKernel(KernelPatcher &patcher, DeviceInfo *info) { } } +bool XE::isIntelDonor(size_t index) const { + return (currentFramebuffer && currentFramebuffer->loadIndex==index) || + (currentGraphics && currentGraphics->loadIndex==index); +} + +bool XE::isIntelFramebuffer(size_t index) const { + return currentFramebuffer && currentFramebuffer->loadIndex==index; +} + +bool XE::requiresIntelProvider() const { + return !adlpLoadOnly && (tglRequested || display13FramebufferRequested); +} + +bool XE::isIntelProvider(size_t index) const { + return requiresIntelProvider() && kextAppleIntelTGLCompat.loadIndex==index; +} + bool XE::processKext(KernelPatcher &patcher, size_t index, mach_vm_address_t address, size_t size) { // No full donor patch set or native display/GT dispatcher in this stage. if(adlpLoadOnly) { @@ -4515,7 +4578,7 @@ bool XE::processKext(KernelPatcher &patcher, size_t index, mach_vm_address_t add return processADLPLoadOnly(patcher,index,address,size); return false; } - if((tglRequested || display13FramebufferRequested) && kextAppleIntelTGLCompat.loadIndex==index) { + if(isIntelProvider(index)) { const auto probe=patcher.solveSymbol(index,"__ZN17AppleIntelPortHAL13probePortModeEv",address,size); patcher.clearError(); const auto configure=patcher.solveSymbol(index,"_xelpConfigureDisplay",address,size); @@ -4613,6 +4676,7 @@ struct Display13FBPowerContext { // Publish with IRQ state only under a quiesced adoption/retirement boundary. // No DSI binding is installed until the complete adoption backend exists. Display13Transcoder::Bindings transcoders; + uint32_t clippedDSIBootSize {}; XELP::DSI::Readout bootDSI; // Captured before handoff; not an implicit admission flag. OSObject *dsiPort {}; // Retained after source-qualified native mode binding. uint32_t dsiNativeWriters {}; // Owner-lock count; cursor bodies may wait without the FB mutex. @@ -4636,6 +4700,9 @@ struct Display13FBPowerContext { Display13WM::Latency latency; bool watermarkFaulted {}; bool engineEnabled {}; + // Shared reset generation; output callbacks consume it independently. + uint32_t restoreEpoch {}; + bool restorePending {}; volatile uintptr_t arbitrationWriterBase {}; // Qualified in sleepable engine context. IOLock *lock {}; // Borrow only while lock is held, and finish consuming the snapshot @@ -4723,7 +4790,7 @@ struct Display13FBIRQContext { IOSimpleLock *lock {}; IOService *controller {}; Display13FBIRQ::State state; - unsigned handlers {},softwareHandlers {};bool quiescing {};uint32_t blockedPipes {}; + unsigned handlers {},softwareHandlers {};bool quiescing {},powerLost {};uint32_t blockedPipes {}; uintptr_t softwareThread {}; Display13FBIRQ::HandlerScopes scopes; volatile uint32_t admittedPipes {}; // Release-published after actual path/backing proof. @@ -5072,6 +5139,11 @@ struct Display13FBDBUFIO { uint32_t read(uint32_t reg) const {return *reinterpret_cast(map->getVirtualAddress()+reg);} void write(uint32_t reg,uint32_t value) { for(auto allowed:Display13DBUF::regs)if(reg==allowed) {*reinterpret_cast(map->getVirtualAddress()+reg)=value;return;} + // Cold restore writes retained primary/cursor allocation and fetch + // credits after restoreAllocation proves this bank is stopped/owned. + for(unsigned pipe=0;pipe<3;++pipe)if(reg==0x7027C+0x1000*pipe || reg==0x7017C+0x1000*pipe || reg==0x7003C+0x1000*pipe) { + *reinterpret_cast(map->getVirtualAddress()+reg)=value;return; + } } void delayUs(unsigned us) {IODelay(us);} }; @@ -5278,7 +5350,7 @@ static Display13FBMemory::Readout display13FBInspectMemory(IOService *fb,bool bo return Display13FBMemory::inspect(io,controller,controller->getMetaClass()->getClassSize(), fb,fb->getMetaClass()->getClassSize(),aperture->getLength(),xelpMemory,bootScanout,display13FBExpectedOwner(controller),c.bridge.bootPort, c.bridge.bootPort>=3 ? &tc : nullptr,dp?&source:nullptr, - c.bootPanel,bootScanout && c.transcoders.dsi(0) ? &c.bootDSI : nullptr); + c.bootPanel,bootScanout && c.transcoders.dsi(0) ? &c.bootDSI : nullptr,c.clippedDSIBootSize); } } @@ -5505,8 +5577,9 @@ bool XE::processDisplay13Framebuffer(KernelPatcher &patcher,size_t index,mach_vm IORegistryEntry::getRegistryRoot()->setProperty("N97FBGTTHooksReady",display13FBGTTHooksReady); display13FBTopologyHooksReady=processDisplay13FramebufferTopology(patcher,index,address,size); display13DSIModeHooksReady=processDisplay13DSIModes(patcher,index,address,size); - dsiNativeWriterHooksReady=!adlnOutputWiring.dsi || (processDSINativeWriters(patcher,index,address,size) && - processDSINativeTransactions(patcher,index,address,size) && processDSINativeAttributes(patcher,index,address,size)); + const bool displayPowerHooks=processDisplay13NativeAttributes(patcher,index,address,size); + dsiNativeWriterHooksReady=displayPowerHooks && (!adlnOutputWiring.dsi || (processDSINativeWriters(patcher,index,address,size) && + processDSINativeTransactions(patcher,index,address,size))); dsiBootProbeReady=processDisplay13DSIBoot(patcher,index,address,size); dsiCacheHooksReady=processDisplay13DSICache(patcher,index,address,size); IORegistryEntry::getRegistryRoot()->setProperty("N97FBTopologyHooksReady",display13FBTopologyHooksReady); @@ -6035,8 +6108,11 @@ IOReturn XE::wrapDisplay13FBEnableIRQ(IOService *controller) { check[17]=io.read(0xC400C);check[18]=io.read(0xC4004); constexpr uint32_t otherBanks[] {0x44448,0x44468,0x44478,0x44488}; for(unsigned i=0;i<4;++i)check[19+i]=io.read(otherBanks[i]+4)&~io.read(otherBanks[i]-4); - if(Display13FBTopology::multiOutputWiring(callbackXE->adlnOutputWiring)) { - const auto admitted=__atomic_load_n(&c.admittedPipes,__ATOMIC_ACQUIRE); + if(c.powerLost && !c.quiescing) { + c.status=Display13FBIRQ::restoreAfterPowerLoss(io,c.state,c.admittedPipes); + if(c.status==Display13FBIRQ::Status::Complete)c.powerLost=false; + } else if(Display13FBTopology::multiOutputWiring(callbackXE->adlnOutputWiring)) { + const auto admitted=__atomic_load_n(&c.admittedPipes,__ATOMIC_ACQUIRE); const auto actual=Display13FBIRQ::readActivePipes(io,admitted,c.state.transcoders); c.status=c.quiescing ? Display13FBIRQ::Status::Power : actual.status; if(c.status==Display13FBIRQ::Status::Complete)c.status=c.state.begun && c.state.enabled ? @@ -9029,7 +9105,7 @@ Display13Clock::Readout XE::adlnClockReadout(IOService *controller) { } uint32_t read(uint32_t reg) const {return *reinterpret_cast(base+reg);} } io {static_cast(map->getVirtualAddress())}; - r=Display13Clock::read(io); + r=Display13Clock::read(io,display13ClockVariant()); } OSSafeReleaseNULL(map); } @@ -9075,7 +9151,7 @@ void XE::wrapAlderLakeNClockSet(IOService *controller,uint64_t nativeHz) { Display13FBClockGuard guard; const bool idle=!Display13WM::busyPipes(c.pipe) && !c.dbuf.updating; const bool cold=idle && !active && adlnBeginClockGuard(guard); - const auto current=Display13Clock::read(io); + const auto current=Display13Clock::read(io,display13ClockVariant()); const bool changing=current.nativeHz()!=nativeHz; if(changing && idle && active==1 && mutex && !c.pipe[0].wmPending && !c.dbuf.unretired && !c.dbuf.faulted && current.status==Display13Clock::Status::Complete && c.pipe[0].wmLatch.status==Display13WM::Status::Complete && @@ -9087,7 +9163,7 @@ void XE::wrapAlderLakeNClockSet(IOService *controller,uint64_t nativeHz) { // the current Hz plus target bandwidth qualifies it BEFORE crawl; // a second latch at the actual new Hz qualifies the new certificate. if(changing && activeReady)c.pipe[0].watermarkClockHz=0; - result=Display13Clock::set(io,nativeHz,{c.bridge.bootAdopted && (active || cold),wmReady,active}); + result=Display13Clock::set(io,nativeHz,{c.bridge.bootAdopted && (active || cold),wmReady,active},display13ClockVariant()); if(cold && result.status==Display13Clock::ChangeStatus::Complete)adlnInvalidateWM(); if(changing && activeReady && result.status==Display13Clock::ChangeStatus::Complete) { afterWM=adlnVerifyWMLatch(fb,result.after.nativeHz(),nativeHz);c.pipe[0].wmLatch=afterWM; @@ -9126,12 +9202,12 @@ void XE::adlnSetDualClock(IOService *controller,uint64_t nativeHz) { !c.bridge.state.faulted && !c.clockFaulted && !c.watermarkFaulted && Display13Power::objectAssociations(c.power,0x80,controller,0xC60); Display13FBClockIO clockIO {c.map}; - auto current=valid ? Display13Clock::read(clockIO) : Display13Clock::Readout {}; + auto current=valid ? Display13Clock::read(clockIO,display13ClockVariant()) : Display13Clock::Readout {}; evidence[2]=current.nativeHz(); // Core independently re-reads the actual clock for an unchanged request. // No synthetic unchanged success, gate or peer-lock dependency is added. if(valid && current.status==Display13Clock::Status::Complete && current.nativeHz()==nativeHz) - result=Display13Clock::set(clockIO,nativeHz,{false,false,0}); + result=Display13Clock::set(clockIO,nativeHz,{false,false,0},display13ClockVariant()); else { auto measure=[&](uint32_t &mask) { Display13FBPowerIO io {c.map};const auto wells=io.read(Display13Power::Driver);mask=0; @@ -9189,7 +9265,7 @@ void XE::adlnSetDualClock(IOService *controller,uint64_t nativeHz) { TGLFBBinding::publish(paths[p],reinterpret_cast(fb),reinterpret_cast(path),binding,p); } } - const auto now=ready ? Display13Clock::read(clockIO) : Display13Clock::Readout {}; + const auto now=ready ? Display13Clock::read(clockIO,display13ClockVariant()) : Display13Clock::Readout {}; ready=ready && now.status==current.status && now.nativeHz()==current.nativeHz() && !c.bridge.state.faulted && !c.clockFaulted && !c.watermarkFaulted; evidence[10]=ready; @@ -9207,7 +9283,7 @@ void XE::adlnSetDualClock(IOService *controller,uint64_t nativeHz) { } const bool activeReady=active && evidence[12]==active; if(activeReady)for(unsigned p=0;p<3;++p)if(active&(1U<display13FBLifecycleHooksReady && callbackXE->adlnHasExternalDisplay && controller==c.controller && c.bridge.begun && !c.bridge.state.faulted && !c.clockFaulted) { - Display13FBClockIO io {c.map};const auto clock=Display13Clock::read(io); + Display13FBClockIO io {c.map};const auto clock=Display13Clock::read(io,display13ClockVariant()); if(clock.status==Display13Clock::Status::Complete || (c.clockStopped && clock.status==Display13Clock::Status::Disabled)) { const bool external=reinterpret_cast(callbackXE->adlnHasExternalDisplay)(controller); - selected=Display13Clock::select(clock.refKHz,minimumHz,external); + selected=Display13Clock::select(clock.refKHz,minimumHz,external,display13ClockVariant()); record[3]=external;record[4]=clock.refKHz; if(selected)*reinterpret_cast(reinterpret_cast(controller)+0xEA0)=selected; } @@ -9661,7 +9737,7 @@ Display13WM::Latch XE::adlnVerifyWMLatch(IOService *fb,uint64_t nativeHz,uint64_ IOMemoryMap *map; bool readable(uint32_t reg) const {return map && map->getLength()==0x1000000 && map->getVirtualAddress() && !(reg&3) && reg<=0xFFFFFC;} uint32_t read(uint32_t reg) const {return *reinterpret_cast(map->getVirtualAddress()+reg);} - uint64_t nativeClockHz() {Display13FBClockIO clockIO {map};return Display13Clock::read(clockIO).nativeHz();} + uint64_t nativeClockHz() {Display13FBClockIO clockIO {map};return Display13Clock::read(clockIO,display13ClockVariant()).nativeHz();} uint64_t nowUs() {uint64_t t=0,ns=0;clock_get_uptime(&t);absolutetime_to_nanoseconds(t,&ns);return ns/1000;} void delayUs(unsigned us) {IOSleep(us/1000);} } io {c.map}; @@ -9858,7 +9934,7 @@ void XE::display13WMArmUpdate(IOService *fb) { uint64_t record[16] {};IOLockLock(c.lock); Display13WM::Latch result;Display13DBUF::Status retired=Display13DBUF::Status::Unavailable; const auto pending=c.pipe[pipe].wmPending,old=c.dbuf.unretired; - Display13FBClockIO clockIO {c.map};const auto actualHz=Display13Clock::read(clockIO).nativeHz(); + Display13FBClockIO clockIO {c.map};const auto actualHz=Display13Clock::read(clockIO,display13ClockVariant()).nativeHz(); trigger[9]=path ? Display13FBMemory::field(path,0x35F0) : UINT32_MAX; if(intent.valid && !trigger[9])result=adlnVerifyWMLatch(fb,actualHz,actualHz); else result.status=Display13WM::Status::Conflict; @@ -9870,6 +9946,17 @@ void XE::display13WMArmUpdate(IOService *fb) { if(retired==Display13DBUF::Status::Complete) {c.pipe[pipe].watermarkClockHz=actualHz;c.pipe[pipe].wmPending=0;} else result.status=Display13WM::Status::Conflict; } + // A cold DSI restart can pause the frame counter during the first + // native flip. Keep both DDB allocations pinned for the next real flush. + // No clock qualification or retirement is granted until a frame advances. + if(pipe==0 && callbackXE->dsiFirstWakeLatch && c.transcoders.dsi0Pipe==0) { + if(result.status==Display13WM::Status::CommitTimeout && + result.check==Display13WM::LatchCheck::Registers && result.frameAfter==result.frameBefore) { + result.status=Display13WM::Status::Unavailable; + result.check=Display13WM::LatchCheck::AwaitingFlip; + callbackXE->dsiFirstWakeLatch=false; + } else if(result.status==Display13WM::Status::Complete)callbackXE->dsiFirstWakeLatch=false; + } c.pipe[pipe].wmLatch=result;c.pipe[pipe].wmCommitting=false; if(result.status!=Display13WM::Status::Complete && result.status!=Display13WM::Status::Unavailable)adlnFault(c.watermarkFaulted,__LINE__); record[0]=1;record[1]=static_cast(result.status);record[2]=pending;record[3]=c.pipe[pipe].wmPending; @@ -9946,10 +10033,10 @@ void XE::display13WMArmUpdate(IOService *fb) { snprintf(property,sizeof(property),"N97FBPipe%uWMLatch",pipe);root->setProperty(property,record,sizeof(record)); } -void XE::wrapAlderLakeNEngineEnable(void *power) { +bool XE::enableDisplay13EngineLocked(void *power) { auto &c=display13FBPower; - if(!c.lock || power!=c.power || !c.controller)return; - uint64_t record[32] {};IOLockLock(c.lock); + if(!c.lock || power!=c.power || !c.controller)return false; + uint64_t record[32] {}; Display13Power::Result acquired; if(callbackXE->display13FBEngineEnableHookReady && c.bridge.begun && !c.bridge.state.faulted && !c.bridge.state.suspended && !c.clockFaulted && Display13Power::objectAssociations(power,0x80,c.controller,0x1C20)) { @@ -9957,23 +10044,24 @@ void XE::wrapAlderLakeNEngineEnable(void *power) { else {Display13FBPowerIO io {c.map};acquired=Display13Power::acquire(io,c.bridge.state,c.engine,Display13Power::bit(Display13Power::Domain::PW1));} } c.writes+=acquired.writes; - Display13FBClockIO clockIO {c.map};auto clock=Display13Clock::read(clockIO); - if(clock.status==Display13Clock::Status::Disabled && c.clockStopped && c.resumeClockHz && + Display13FBClockIO clockIO {c.map};auto clock=Display13Clock::read(clockIO,display13ClockVariant()); + bool clockReady=!c.restorePending; + if((c.restorePending || (clock.status==Display13Clock::Status::Disabled && c.clockStopped)) && c.resumeClockHz && !c.bridge.state.faulted && !c.bridge.state.suspended && !c.clockFaulted && !c.dbuf.updating && !Display13WM::busyPipes(c.pipe)) { Display13FBClockGuard guard; if(adlnBeginClockGuard(guard)) { - const auto changed=Display13Clock::set(clockIO,c.resumeClockHz,{true,true,0}); + const auto changed=Display13Clock::set(clockIO,c.resumeClockHz,{true,true,0},display13ClockVariant()); if(changed.status==Display13Clock::ChangeStatus::Complete) { - clock=changed.after;c.clockStopped=false;adlnInvalidateWM(); + clock=changed.after;clockReady=true;c.clockStopped=false;adlnInvalidateWM(); *reinterpret_cast(reinterpret_cast(c.controller)+0xE98)=clock.nativeHz(); - } else adlnFault(c.clockFaulted,__LINE__); + } else if(changed.writes)adlnFault(c.clockFaulted,__LINE__); adlnEndClockGuard(guard,changed.status==Display13Clock::ChangeStatus::Complete || !changed.writes); - } else adlnFault(c.clockFaulted,__LINE__); + } } Display13DBUF::Status status=Display13DBUF::Status::Unavailable; if(callbackXE->display13FBEngineEnableHookReady && c.bridge.begun && !c.bridge.state.faulted && - !c.bridge.state.suspended && acquired.status==Display13Power::Status::Complete && !c.clockFaulted && clock.status==Display13Clock::Status::Complete && + !c.bridge.state.suspended && acquired.status==Display13Power::Status::Complete && clockReady && !c.clockFaulted && clock.status==Display13Clock::Status::Complete && Display13Power::objectAssociations(power,0x80,c.controller,0x1C20)) { Display13FBDBUFIO io {c.map}; status=Display13DBUF::begin(io,c.dbuf,__builtin_popcount(callbackXE->adlnOutputWiring.selected())>=3 ? 7U : 0U,c.transcoders); @@ -9988,8 +10076,8 @@ void XE::wrapAlderLakeNEngineEnable(void *power) { // binding here, then publish only the native VA for the nonblocking writer. const auto writerBase=c.engineEnabled && adlnWriterMappingBound() ? Display13FBMemory::field(c.access,0x50) : 0; __atomic_store_n(&c.arbitrationWriterBase,writerBase,__ATOMIC_RELEASE); - if(c.engineEnabled) {c.clock=clock;c.resumeClockHz=clock.nativeHz();} - if(!c.engineEnabled)adlnFault(c.bridge.state.faulted,__LINE__); + if(c.engineEnabled) {c.clock=clock;c.resumeClockHz=clock.nativeHz();c.clockStopped=false;} + if(!c.engineEnabled && (c.clockFaulted || c.dbuf.faulted))adlnFault(c.bridge.state.faulted,__LINE__); auto &d=c.dbuf; record[0]=1;record[1]=static_cast(status);record[2]=c.engineEnabled;record[3]=c.engine.domains; record[4]=clock.nativeHz();record[5]=d.begun;record[6]=d.faulted;record[7]=d.mbus;record[8]=d.joined; @@ -9998,13 +10086,22 @@ void XE::wrapAlderLakeNEngineEnable(void *power) { for(unsigned i=0;i<4;++i) {record[17+i]=d.initial[i];record[21+i]=d.raw[i];} record[25]=writerBase!=0; record[26]=d.selected;record[27]=static_cast(acquired.status);record[28]=static_cast(clock.status); - IOLockUnlock(c.lock); auto root=IORegistryEntry::getRegistryRoot();root->setProperty("N97FBEngine",record,sizeof(record)); if(!record[26] && !root->getProperty("N97FBEngineFirstEmptyResume")) root->setProperty("N97FBEngineFirstEmptyResume",record,sizeof(record)); + return c.engineEnabled; +} + +void XE::wrapAlderLakeNEngineEnable(void *power) { + auto &c=display13FBPower;if(!c.lock)return; + IOLockLock(c.lock);enableDisplay13EngineLocked(power);IOLockUnlock(c.lock); } void XE::wrapAlderLakeNEngineDisable(void *power) { + stopDisplay13Engine(power,false); +} + +void XE::stopDisplay13Engine(void *power,bool systemSuspend) { auto &c=display13FBPower;if(!c.lock || power!=c.power || !c.controller)return; uint64_t record[12] {},hotplug[16] {},irqRelease[12] {};Display13Power::Result released;Display13Clock::Change clock; IOWorkLoop *acquiredGate=nullptr;bool gateAttempted=false,gateReady=false;uint64_t gateEvidence[8] {}; @@ -10058,7 +10155,7 @@ void XE::wrapAlderLakeNEngineDisable(void *power) { if(ensureGate() && adlnBeginClockGuard(guard)) { Display13FBDBUFIO io {c.map};status=Display13DBUF::stop(io,c.dbuf); if(status==Display13DBUF::Status::Complete) { - Display13FBClockIO clockIO {c.map};clock=Display13Clock::stop(clockIO,{true,false,0}); + Display13FBClockIO clockIO {c.map};clock=Display13Clock::stop(clockIO,{true,false,0},display13ClockVariant()); if(clock.status!=Display13Clock::ChangeStatus::Complete) {status=Display13DBUF::Status::Power;adlnFault(c.clockFaulted,__LINE__);} else { if(clock.before.nativeHz())c.resumeClockHz=clock.before.nativeHz(); @@ -10092,7 +10189,7 @@ void XE::wrapAlderLakeNEngineDisable(void *power) { } } if(status==Display13DBUF::Status::Complete)c.engineEnabled=false; - if(status==Display13DBUF::Status::Complete && c.bridge.bootAdopted && (c.bridge.state.borrowed&1) && + if(!systemSuspend && status==Display13DBUF::Status::Complete && c.bridge.bootAdopted && (c.bridge.state.borrowed&1) && (dual || (c.pipe[0].memoryFramebuffer && !Display13FBMemory::field(c.pipe[0].memoryFramebuffer,0x1E0))))restoreHPD(); adlnEndClockGuard(guard,status!=Display13DBUF::Status::Complete && !clock.writes && !c.clockStopped); @@ -10101,11 +10198,11 @@ void XE::wrapAlderLakeNEngineDisable(void *power) { } // A repeated native IRQ disable can precede an idempotent engine stop. // Revalidate/re-arm HPD even when no shared resource mutation was needed. - if(dual && status==Display13DBUF::Status::Complete && !hotplug[0] && !c.bridge.state.suspended && + if(!systemSuspend && dual && status==Display13DBUF::Status::Complete && !hotplug[0] && !c.bridge.state.suspended && c.bridge.bootAdopted && (c.bridge.state.borrowed&1) && c.clockStopped && !c.engine.domains)restoreHPD(); if(status==Display13DBUF::Status::Complete)c.engineEnabled=false; - else adlnFault(c.bridge.state.faulted,__LINE__); + else if(c.dbuf.faulted || c.clockFaulted || released.writes || clock.writes)adlnFault(c.bridge.state.faulted,__LINE__); c.writes+=released.writes; record[0]=1;record[1]=static_cast(status);record[2]=c.engineEnabled;record[3]=c.engine.domains; record[4]=static_cast(clock.status);record[5]=c.clockStopped;record[6]=c.resumeClockHz; @@ -10128,41 +10225,208 @@ void XE::wrapAlderLakeNPowerOverride(void *power,bool enable) { finishDisplay13TCShutdown(c.controller); // override(true) is only reached by the donor's simulator backend; our // physical object has simulator=0. Do not force unrelated DDI/AUX domains. - if(!enable)wrapAlderLakeNEngineDisable(power); - uint64_t record[32] {};Display13Power::Result result;IOLockLock(c.lock); + if(!enable)stopDisplay13Engine(power,true); + uint64_t record[32] {},suspendIRQ[24] {};Display13Power::Result result; + IOWorkLoop *acquiredGate=nullptr;IOLockLock(c.lock); if(callbackXE->display13FBLifecycleHooksReady && !enable && c.bridge.begun && !c.bridge.state.faulted && !c.engineEnabled && c.clockStopped && !c.dbuf.owned && !Display13WM::busyPipes(c.pipe)) { if(c.bridge.state.suspended)result.status=Display13Power::Status::Complete; else { auto &irq=display13FBIRQ;Display13FBIRQIO irqIO {irq.base}; - const auto saved=IOSimpleLockLockDisableInterrupt(irq.lock); - const bool quiet=!irq.quiescing && Display13FBIRQ::quiescent(irqIO,irq.state,c.bridge.state.allowed,irq.handlers); - if(quiet) {irq.quiescing=true;irq.blockedPipes=7;} - IOSimpleLockUnlockEnableInterrupt(irq.lock,saved); - if(quiet) { - Display13FBPowerIO io {c.map};result=c.bridge.suspend(io); + const bool multi=Display13FBTopology::multiOutputWiring(callbackXE->adlnOutputWiring); + bool gated=!multi; + suspendIRQ[0]=1;suspendIRQ[1]=multi;suspendIRQ[2]=c.bridge.state.allowed; + if(multi && c.workLoop && Display13FBMemory::field(c.controller,0xE00)==c.workLoop && + callbackXE->adlnWorkLoopInGate && callbackXE->adlnWorkLoopTryClose && callbackXE->adlnWorkLoopOpen) { + suspendIRQ[3]=reinterpret_cast(callbackXE->adlnWorkLoopInGate)(c.workLoop); + if(suspendIRQ[3])gated=true; + else if(reinterpret_cast(callbackXE->adlnWorkLoopTryClose)(c.workLoop)) { + acquiredGate=c.workLoop;suspendIRQ[4]=1;gated=true; + } + } + suspendIRQ[5]=gated; + Display13FBIRQ::PowerReleaseScope scope;bool quiet=false; + if(gated && irq.lock && irq.base && adlnIRQOwner(static_cast(c.controller))) { + const auto saved=IOSimpleLockLockDisableInterrupt(irq.lock); + suspendIRQ[6]=irq.handlers;suspendIRQ[7]=irq.scopes.display;suspendIRQ[8]=irq.softwareHandlers; + suspendIRQ[9]=irq.state.enabled;suspendIRQ[10]=irqIO.read(Display13FBIRQ::Master); + suspendIRQ[11]=irq.state.active;suspendIRQ[12]=irq.quiescing; + for(unsigned i=0;i<3;++i)suspendIRQ[13+i]=__atomic_load_n(reinterpret_cast(reinterpret_cast(c.controller)+0xE08+4*i),__ATOMIC_ACQUIRE); + // Engine idle keeps HPD alive. Full suspend must explicitly park + // that IRQ ownership, not reject the master enable we just armed. + // Require all consumers and competing work to retire first. + if(multi) { + const auto status=!irq.quiescing && irq.scopes.total==irq.handlers ? + Display13FBIRQ::beginPowerRelease(irqIO,irq.state,scope,c.bridge.state.allowed,7, + irq.scopes.display,irq.softwareHandlers,suspendIRQ[13],suspendIRQ[14],suspendIRQ[15]) : Display13FBIRQ::Status::Unavailable; + suspendIRQ[16]=static_cast(status);quiet=status==Display13FBIRQ::Status::Complete; + } else quiet=!irq.quiescing && Display13FBIRQ::quiescent(irqIO,irq.state,c.bridge.state.allowed,irq.handlers); + if(quiet || scope.held) {irq.quiescing=true;irq.blockedPipes=7;} + IOSimpleLockUnlockEnableInterrupt(irq.lock,saved); + } + suspendIRQ[17]=quiet;suspendIRQ[18]=c.bridge.auxBusy(); + if(quiet) {Display13FBPowerIO io {c.map};result=c.bridge.suspend(io);} + if(quiet || scope.held) { const auto done=IOSimpleLockLockDisableInterrupt(irq.lock); + suspendIRQ[19]=scope.parked; + if(scope.held && result.status!=Display13Power::Status::Complete) + suspendIRQ[20]=static_cast(Display13FBIRQ::endPowerRelease(irqIO,irq.state,scope)); + // Successful suspend consumes the scope with DE masked. Re-arming + // HPD here would undo suspend when firmware still owns PW1. + scope.held=false;suspendIRQ[21]=irq.state.enabled; irq.quiescing=false;irq.blockedPipes=0;IOSimpleLockUnlockEnableInterrupt(irq.lock,done); } + suspendIRQ[22]=static_cast(result.status);suspendIRQ[23]=c.bridge.state.suspended; } } - if(result.status!=Display13Power::Status::Complete)adlnFault(c.bridge.state.faulted,__LINE__); + if(result.status!=Display13Power::Status::Complete && result.writes)adlnFault(c.bridge.state.faulted,__LINE__); display13FBPowerRecord(record,8,enable,result);IOLockUnlock(c.lock); + if(acquiredGate)reinterpret_cast(callbackXE->adlnWorkLoopOpen)(acquiredGate); + if(suspendIRQ[0])IORegistryEntry::getRegistryRoot()->setProperty("N97FBSuspendIRQ",suspendIRQ,sizeof(suspendIRQ)); adlnPublishPowerRecord("N97FBPowerOverride",record); } + +// Called with the display owner lock. No framebuffer or connector owns this +// transition. Native global wake and a transport that bypasses native wake +// both enter here; a second caller must not reset an already running peer. +bool XE::restoreDisplay13ResourcesLocked(void *power,bool waking) { + auto &c=display13FBPower; + uint64_t record[32] {},cold[12] {};Display13Power::Result result; + if(power!=c.power || !c.controller || !callbackXE->display13FBLifecycleHooksReady || + !c.bridge.begun || c.bridge.state.faulted || c.clockFaulted || + !Display13Power::objectAssociations(power,0x80,c.controller,0x1C20))return false; + // Serialize against hotplug/port work. Try-close while holding the owner + // lock: another native callback may own the gate and be waiting for us. + auto loop=c.workLoop; + if(!loop || Display13FBMemory::field(c.controller,0xE00)!=loop || + !callbackXE->adlnWorkLoopInGate || !callbackXE->adlnWorkLoopTryClose || !callbackXE->adlnWorkLoopOpen)return false; + struct Gate { + IOWorkLoop *loop;mach_vm_address_t open;bool acquired; + ~Gate(){if(acquired)reinterpret_cast(open)(loop);} + } gate{loop,callbackXE->adlnWorkLoopOpen,false}; + if(!reinterpret_cast(callbackXE->adlnWorkLoopInGate)(loop)) { + gate.acquired=reinterpret_cast(callbackXE->adlnWorkLoopTryClose)(loop); + if(!gate.acquired)return false; + } + Display13FBBootIO io{c.map}; + const auto driver=io.read(Display13Power::Driver); + if(driver==UINT32_MAX)return false; + // The donor also restores from its low-power exit with waking=false. + // Hardware state, not that hint, determines whether shared state was lost. + const bool lost=!(driver&3) || c.bridge.state.suspended; + if(!lost && !c.restorePending && !c.bridge.state.suspended && c.engineEnabled && !c.clockStopped)return true; + if(lost && !c.restorePending) { + ++c.restoreEpoch;c.restorePending=true; + c.dbuf.restorePending|=c.dbuf.selected; + for(unsigned tc=0;tc<4;++tc) { + c.tcMode[tc].notePowerLoss();c.tcSignalReady[tc]=false;c.tcDPLink[tc]={}; + } + c.bridge.state.suspended=true; + c.engineEnabled=false;c.clockStopped=true; + if(!c.resumeClockHz)c.resumeClockHz=c.clock.nativeHz(); + auto &irq=display13FBIRQ; + const auto saved=IOSimpleLockLockDisableInterrupt(irq.lock); + irq.powerLost=true; + IOSimpleLockUnlockEnableInterrupt(irq.lock,saved); + } + cold[0]=c.restorePending;cold[1]=c.restoreEpoch; + // This is a PHY/protocol prerequisite, not a condition for shared wake. + // Combo HDMI/DP/eDP and TC legacy/DP-alt outputs use the same shared path. + const auto selectDSI=[&](uint32_t ddi) { + if(!c.restorePending || c.transcoders.dsi0Pipe>=3)return true; + constexpr uint32_t reg=0x6B094; + const auto before=io.read(reg);cold[2]=before; + if(before==UINT32_MAX)return false; + if(!(before&1)) { + if(ddi&3)return false; + *reinterpret_cast(c.map->getVirtualAddress()+reg)=before|1; + } + cold[3]=io.read(reg);return cold[3]!=UINT32_MAX && (cold[3]&1); + }; + result=Display13Power::restore(io,c.bridge.state,c.restorePending,selectDSI); + if(result.status==Display13Power::Status::Complete && c.restorePending) { + // No clock/MBUS writes after any output starts scanning out. + if(!Display13FBIRQ::consumersStopped(io,7,c.transcoders))result.status=Display13Power::Status::NotReady; + else { + cold[4]=io.read(0x4438C); + if(cold[4]!=c.dbuf.mbus) + *reinterpret_cast(c.map->getVirtualAddress()+0x4438C)=c.dbuf.mbus; + cold[5]=io.read(0x4438C); + if(cold[5]!=c.dbuf.mbus)result.status=Display13Power::Status::NotReady; + else for(unsigned i=0;i<4;++i) + if(!(io.read(Display13DBUF::regs[i])&Display13DBUF::Request))c.dbuf.owned&=~(1U<(result.status);cold[7]=c.bridge.state.deferredTC; + const bool ready=result.status==Display13Power::Status::Complete && + ((c.engineEnabled && !c.restorePending && !c.clockStopped) || enableDisplay13EngineLocked(power)); + if(ready)c.restorePending=false; + else if(result.status==Display13Power::Status::Complete)result.status=Display13Power::Status::NotReady; + // A read/preflight rejection retains the pending generation. Partial + // hardware failures are already latched by the resource that owns them. + cold[8]=c.engineEnabled;cold[9]=c.clockStopped;cold[10]=c.dbuf.restorePending;cold[11]=c.bridge.state.faulted; + display13FBPowerRecord(record,9,waking,result); + adlnPublishPowerRecord("N97FBPowerRestore",record); + IORegistryEntry::getRegistryRoot()->setProperty("WEXDisplay13SharedResume",cold,sizeof(cold)); + return ready; +} + +bool XE::restoreDisplay13Ports(IOService *controller,unsigned onlyDDI) { + auto &c=display13FBPower;bool ready=true;uint32_t ports[10]={1}; + for(unsigned tc=0;c.lock && tc<4;++tc) { + if(onlyDDI!=UINT32_MAX && tc+3!=onlyDDI)continue; + IOLockLock(c.lock); + const bool pending=c.controller==controller && c.tcMode[tc].restorePending; + auto port=pending ? c.tcObject[tc] : nullptr; + const auto mode=c.tcMode[tc].selectedMode; + if(port)port->retain(); + IOLockUnlock(c.lock); + if(pending && !port)ready=false; + if(port) { + ports[1+2*tc]=mode;ports[2+2*tc]=wrapDisplay13TCMode(port,mode); + if(ports[2+2*tc]!=kIOReturnSuccess)ready=false; + port->release(); + } + } + ports[9]=ready;IORegistryEntry::getRegistryRoot()->setProperty("WEXDisplay13ResumePorts",ports,sizeof(ports)); + return ready; +} + +// prepareToExitSleep invokes a void restorePowerWellsState and otherwise +// continues on failure. Intercept at the enclosing status-returning FB entry +// so a busy/partial shared transition cannot fall through into native writers. +bool XE::prepareDisplay13Wake(IOService *fb) { + auto &c=display13FBPower; + if(!c.lock || !fb || fb->getMetaClass()->getClassSize()<0x4B90)return true; + auto controller=Display13FBMemory::field(fb,0x1D0); + IOLockLock(c.lock); + if(controller!=c.controller || !c.bridge.begun){IOLockUnlock(c.lock);return true;} + Display13FBBootIO io{c.map};const auto driver=io.read(Display13Power::Driver); + const bool needed=c.restorePending || c.bridge.state.suspended || !(driver&3); + const bool ready=driver!=UINT32_MAX && (!needed || restoreDisplay13ResourcesLocked(c.power,true)); + unsigned ddi=UINT32_MAX;bool connected=false; + const auto pipe=Display13FBMemory::boundPipe(c.pipe,fb); + if(ready && pipe<3 && c.pipe[pipe].powerPath.binding.status==TGLFBBinding::Status::Complete) { + ddi=c.pipe[pipe].powerPath.binding.ddi; + Display13TC::Port port; + if(Display13TC::Port::resolve(ddi,port) && c.tcMode[ddi-3].restorePending) { + const auto sde=io.read(0xC4000),north=io.read(0x44470); + connected=sde==UINT32_MAX || north==UINT32_MAX || port.legacyHPD(sde) || port.altHPD(sde,north); + } + } + IOLockUnlock(c.lock); + // A disconnected peer must reach native probing/teardown, not prevent + // every other FB from waking. Only this output's live TC link is required. + return ready && (!connected || restoreDisplay13Ports(controller,ddi)); +} + void XE::wrapAlderLakeNPowerRestore(void *power,bool waking) { - auto &c=display13FBPower;if(!c.lock || power!=c.power || !c.controller)return; - uint64_t record[32] {};Display13Power::Result result;IOLockLock(c.lock); - if(callbackXE->display13FBLifecycleHooksReady && c.bridge.begun && !c.bridge.state.faulted && !c.clockFaulted && - Display13Power::objectAssociations(power,0x80,c.controller,0x1C20)) { - Display13FBPowerIO io {c.map};result=Display13Power::restore(io,c.bridge.state); - if(result.status==Display13Power::Status::Complete && - !Display13Power::publishObject(c.bridge,power,0x80,c.controller,0x1C20))result.status=Display13Power::Status::NotReady; - } - if(result.status!=Display13Power::Status::Complete)adlnFault(c.bridge.state.faulted,__LINE__); - display13FBPowerRecord(record,9,waking,result);IOLockUnlock(c.lock); - adlnPublishPowerRecord("N97FBPowerRestore",record); - if(result.status==Display13Power::Status::Complete)wrapAlderLakeNEngineEnable(power); + auto &c=display13FBPower;if(!c.lock || power!=c.power || !c.controller)return; + IOLockLock(c.lock);const bool ready=restoreDisplay13ResourcesLocked(power,waking); + auto controller=static_cast(c.controller);IOLockUnlock(c.lock); + if(ready)restoreDisplay13Ports(controller); } void XE::wrapAlderLakeNPrimaryDBUF(OSObject *plane,uint32_t start,uint32_t end,bool wait) { @@ -10554,7 +10818,7 @@ bool XE::adlnPowerReference(void *power,uint32_t operation,uint32_t domain) { } uint64_t record[32] {},check[21] {2,operation,domain},auxRecord[10] {},context[24] {},gateEvidence[8] {}; Display13Power::Result result; IOWorkLoop *acquiredGate=nullptr; - Display13FBIRQ::PowerReleaseScope releaseScope;uint64_t irqRelease[12] {}; + Display13FBIRQ::PowerReleaseScope releaseScope;uint64_t irqRelease[16] {}; bool previousQuiescing=false;uint32_t previousBlocked=0; bool disableEngine=false; IOLockLock(c.lock); @@ -10582,6 +10846,14 @@ bool XE::adlnPowerReference(void *power,uint32_t operation,uint32_t domain) { ++h[13+(h[7]&3)+((h[7]&8)?4:0)]; } else {++h[21];h[8]=h[9]=h[10]=h[11]=h[12]=0;} } + // A bank may be off by the time global STOP runs. Capture it before + // its last native PG put; retain snapshots of already parked peers. + if(operation==2 && c.dbuf.begun) { + Display13FBDBUFIO snapshot{c.map}; + if(Display13DBUF::captureCredits(snapshot,c.dbuf)!=Display13DBUF::Status::Complete) { + IOLockUnlock(c.lock);return false; + } + } Display13FBPowerIO io {c.map}; uint32_t released=0; if(operation==2 && domain>=1 && domain<=4 && c.bridge.pgRefs[domain-1]==1) @@ -10635,9 +10907,21 @@ bool XE::adlnPowerReference(void *power,uint32_t operation,uint32_t domain) { irqRelease[0]=1;irqRelease[1]=released;irqRelease[2]=affected;irqRelease[3]=irq.handlers; irqRelease[4]=irq.scopes.display;irqRelease[5]=irq.softwareHandlers; irqRelease[6]=queued;irqRelease[7]=aux;irqRelease[8]=hpd; + // Native short-pulse/hotplug handlers get PG before AUX work + // and put it before returning to SWInterruptHandler. That same + // gated stack is the releaser, not a competing SW consumer. + // Permit only stopped leaf pipe wells: shared PW1/PW2 teardown + // still requires the entire software handler to have returned. + constexpr uint32_t pipeWells=Display13Power::bit(Display13Power::Domain::PipeA)| + Display13Power::bit(Display13Power::Domain::PipeB)|Display13Power::bit(Display13Power::Domain::PipeC); + const bool ownSoftware=operation==2 && !(released&~pipeWells) && gateEvidence[4] && + irq.softwareHandlers==1 && irq.softwareThread==reinterpret_cast(current_thread()); + const unsigned competingSoftware=ownSoftware ? 0 : irq.softwareHandlers; + irqRelease[12]=ownSoftware;irqRelease[13]=competingSoftware; + irqRelease[14]=irq.softwareThread==reinterpret_cast(current_thread());irqRelease[15]=gateEvidence[4]; const auto status=!irq.quiescing && irq.scopes.total==irq.handlers ? Display13FBIRQ::beginPowerRelease(irqIO,irq.state,releaseScope,released,affected, - irq.scopes.display,irq.softwareHandlers,queued,aux,hpd) : Display13FBIRQ::Status::Unavailable; + irq.scopes.display,competingSoftware,queued,aux,hpd) : Display13FBIRQ::Status::Unavailable; irqRelease[9]=static_cast(status);releaseAllowed=status==Display13FBIRQ::Status::Complete; if(releaseScope.held) {irq.quiescing=true;irq.blockedPipes=affected;} } else { @@ -10671,7 +10955,7 @@ bool XE::adlnPowerReference(void *power,uint32_t operation,uint32_t domain) { } if(result.status==Display13Power::Status::Complete && !Display13Power::publishObject(c.bridge,power,0x80,c.controller,0xC60)) result.status=Display13Power::Status::NotReady; - if(result.status!=Display13Power::Status::Complete)adlnFault(c.bridge.state.faulted,__LINE__); + if(result.status!=Display13Power::Status::Complete && (releaseAllowed || display13FBIRQ.state.faulted))adlnFault(c.bridge.state.faulted,__LINE__); disableEngine=operation==2 && result.status==Display13Power::Status::Complete && !c.bridge.nativeRefs(1) && c.engineEnabled; } else if(power==c.power && c.controller)adlnFault(c.bridge.state.faulted,__LINE__); @@ -10688,6 +10972,7 @@ bool XE::adlnPowerReference(void *power,uint32_t operation,uint32_t domain) { if(display13TraceEnabled)IORegistryEntry::getRegistryRoot()->setProperty("N97FBPowerContext",context,sizeof(context)); if(gateEvidence[0])IORegistryEntry::getRegistryRoot()->setProperty("N97FBPowerReleaseGate",gateEvidence,sizeof(gateEvidence)); if(irqRelease[0])IORegistryEntry::getRegistryRoot()->setProperty("N97FBPowerReleaseIRQ",irqRelease,sizeof(irqRelease)); + if(irqRelease[12])IORegistryEntry::getRegistryRoot()->setProperty("N97FBOwnSoftwarePGRelease",irqRelease,sizeof(irqRelease)); IORegistryEntry::getRegistryRoot()->setProperty("N97FBAUXPower",auxRecord,sizeof(auxRecord)); if(check[14])IORegistryEntry::getRegistryRoot()->setProperty("N97FBPowerReleaseMap",check,sizeof(check)); if(record[4]!=static_cast(Display13Power::Status::Complete) && record[7]==1) @@ -11894,12 +12179,11 @@ bool XE::processTGLAccelerator(KernelPatcher &patcher, size_t index, mach_vm_add // target returns after native reset/resume; its lower-level reset still // performs context replay. Port that CALLER contract only. No changed // completion values, queue drops, hardware reset or panic entry points. - // The same donor caller also stalls the accelerator resource lock on - // this Tahoe kernel. Keep the OS-family and exact donor guards: only the - // recovery caller changes; native reset/replay still owns completion. - const bool recoveryRequested=checkKernelArgument("-disablegfxfirmware") && - ((XELP::kernelOS(version)==XELP::OS::BigSur) || - (XELP::kernelOS(version)==XELP::OS::Tahoe)); + // IOAccel on Darwin 20-25 releases its recovery locks after this void + // callback returns. Native ICL uses the same single-pass caller; no host + // requires the TGL donor's extra stamp wait. Select by the firmware mode, + // exact donor and scheduler ABI, not by the OS used to observe a hang. + const bool recoveryRequested=checkKernelArgument("-disablegfxfirmware"); bool recoveryOK=!recoveryRequested; if (recoveryRequested) { const char *recovery="__ZN18IGAccelFIFOChannel22resetHardwareAndReplayEv"; @@ -11925,7 +12209,12 @@ bool XE::processTGLAccelerator(KernelPatcher &patcher, size_t index, mach_vm_add } IORegistryEntry::getRegistryRoot()->setProperty("TGLRecoverySinglePassRequested",recoveryRequested); IORegistryEntry::getRegistryRoot()->setProperty("TGLRecoverySinglePassReady",recoveryRequested && recoveryOK); - tglAcceleratorReady = surfaceABIReady && adlnObserved && skuReady && urbReady && ccsReady && hizReady && ccsPSReady && ggttOK && observed && ok && gucOK && dmaWriteOK && hostOK && scratchOK && blitInitOK && tglMMIOUCReady && copyOK && auxAlignOK && iosCCSOK && ffModeOK && cpsOK && flushOK && tlbPreparserOK && semaphoreTLBOK && hangOK && recoveryOK && completionCacheOK && contextCacheOK; + // These resource checks also precede probe/start. Include them in + // admission reporting so an installed patch set cannot hide an + // accelerator rejected before either native lifecycle callback. + const bool resourcesReady=(!tglGT2Detected || tglGT2Ready) && + (!display12SmallGTDetected || display12SmallGTReady); + tglAcceleratorReady = resourcesReady && surfaceABIReady && adlnObserved && skuReady && urbReady && ccsReady && hizReady && ccsPSReady && ggttOK && observed && ok && gucOK && dmaWriteOK && hostOK && scratchOK && blitInitOK && tglMMIOUCReady && copyOK && auxAlignOK && iosCCSOK && ffModeOK && cpsOK && flushOK && tlbPreparserOK && semaphoreTLBOK && hangOK && recoveryOK && completionCacheOK && contextCacheOK; // r201: resource/copy diagnostics are absent, not admission requirements. IORegistryEntry::getRegistryRoot()->setProperty("TGLResourceDiagnosticsRemoved",true); IORegistryEntry::getRegistryRoot()->setProperty("TGLAcceleratorAdmissionReady", tglAcceleratorReady); @@ -20080,4 +20369,5 @@ void XE::wrapTGLWriteDekel(void *port, uint32_t index, uint32_t value) { #include "kern_dsi_adapter_backing.inc" #include "kern_dsi_adapter_modes.inc" #include "kern_dsi_adapter_lifecycle.inc" +#include "kern_display13_framebuffer_power.inc" #include "kern_dsi_adapter_diagnostics.inc" diff --git a/WhateverXe/kern_xe.hpp b/WhateverXe/kern_xe.hpp index b35a3d45..d333a30b 100644 --- a/WhateverXe/kern_xe.hpp +++ b/WhateverXe/kern_xe.hpp @@ -8,6 +8,7 @@ #ifndef kern_xe_hpp #define kern_xe_hpp +#include "kern_dsi_sleep_image.hpp" #include "kern_tgl_tc_boot.hpp" #include "kern_dsi_boot.hpp" #include "kern_dsi_apple_timing.hpp" @@ -72,6 +73,16 @@ class XE { * @return true if patched anything */ bool processKext(KernelPatcher &patcher, size_t index, mach_vm_address_t address, size_t size); + bool ownedDisplay13Framebuffer(IOService *fb) const; + bool failedDSIWakeFor(IOService *fb) const; + bool isIntelDonor(size_t index) const; + bool isIntelFramebuffer(size_t index) const; + bool requiresIntelProvider() const; + bool isIntelProvider(size_t index) const; + void setHostSurfaceABI(unsigned framebuffer,unsigned accelerator); + void setHostSleepReady(bool ready) { hostSleepReady=ready; } + void setHostWakeRecoveryReady(bool ready) { hostWakeRecoveryReady=ready; } + void setHostFramebufferEvent(mach_vm_address_t event) { hostFramebufferEvent=event; } private: bool adlpLoadOnly {}; @@ -145,6 +156,7 @@ class XE { bool display13BootReady {false}; OSData *dsiFirmware {}; // Private immutable copy backing profile/sequence spans. XELP::DSI::Profile dsiProfile; + uint32_t dsi60HzClockKHz {}; XELP::DSI::Boot dsiBoot; XELP::DSI::GGTTMode dsiGGTT; bool prepareDSIFirmware(IOService *); @@ -243,12 +255,22 @@ class XE { static bool dsiSharedWriterStopped(IOService *,unsigned); bool processDSINativeWriters(KernelPatcher &,size_t,mach_vm_address_t,size_t); bool processDSINativeTransactions(KernelPatcher &,size_t,mach_vm_address_t,size_t); - bool processDSINativeAttributes(KernelPatcher &,size_t,mach_vm_address_t,size_t); + bool processDisplay13NativeAttributes(KernelPatcher &,size_t,mach_vm_address_t,size_t); mach_vm_address_t orgDSIAttributes[4] {}; + mach_vm_address_t display13FramebufferEvent {},hostFramebufferEvent {}; + uint32_t dsiWakeDegraded {},dsiWakeIncomplete {},dsiWakeRejections {}; + bool hostSleepReady {},hostWakeRecoveryReady {}; + bool dsiSystemWakeNotify {},display13PagingLayoutKnown {},dsiFirstWakeLatch {}; + bool dsiPowerOffNotified {}; + uint32_t dsiPowerCallbackBusy {},dsiPowerOffEvents {},dsiPowerOnEvents {}; + uint32_t dsiRestoredEpoch {}; // Last shared reset consumed by this transport. + bool finishDisplay13SystemWake(IOService *); + uint32_t displayWakeEventEpoch {},displayWakeEventBusy {},displayWakeEventErrors {}; + uint8_t displayWakeEventPhase[3] {}; static uint32_t wrapDSISetStream(IOService *,IOService *,OSObject *,const void *); static uint32_t wrapDSISetAttribute(IOService *,uint32_t,uintptr_t); static uint32_t wrapDSISetConnectionAttribute(IOService *,int32_t,uint32_t,uintptr_t); - static uint32_t wrapDSISetPowerState(IOService *,uint32_t); + static uint32_t wrapDisplay13SetPowerState(IOService *,uint32_t); mach_vm_address_t orgDSITransaction[5] {}; static int beginDSIFramebufferWriter(IOService *); static uint32_t wrapDSIFlipTransaction(IOService *,void *,uint64_t,void *); @@ -264,6 +286,7 @@ class XE { bool dsiHostTest {}; bool dsiStopTest {}; bool dsiPowerTest {}; + XELP::DSI::SystemSleepImage dsiSleepImage; struct { bool sleeping {},failed {};uint32_t cycles {}; XELP::DSI::Surface::Plane plane; @@ -396,6 +419,11 @@ class XE { static void wrapAlderLakeNPowerInit(void *power, void *controller); static void wrapAlderLakeNEngineEnable(void *power); static void wrapAlderLakeNEngineDisable(void *power); + static void stopDisplay13Engine(void *power,bool systemSuspend); + static bool enableDisplay13EngineLocked(void *power); + static bool restoreDisplay13ResourcesLocked(void *power,bool waking); + static bool restoreDisplay13Ports(IOService *controller,unsigned onlyDDI=UINT32_MAX); + static bool prepareDisplay13Wake(IOService *fb); static void wrapAlderLakeNPowerOverride(void *power,bool enable); static void wrapAlderLakeNPowerRestore(void *power,bool waking); __attribute__((noinline)) bool processDisplay13FramebufferLifecycle(KernelPatcher &patcher,size_t index,mach_vm_address_t address,size_t size); diff --git a/WhateverXe/kern_xelp_call_target.hpp b/WhateverXe/kern_xelp_call_target.hpp index ca168842..96029042 100644 --- a/WhateverXe/kern_xelp_call_target.hpp +++ b/WhateverXe/kern_xelp_call_target.hpp @@ -19,5 +19,17 @@ inline bool targets(const uint8_t *bytes,size_t length,uintptr_t site,uintptr_t if(!bytes || length<5 || !site || !target || bytes[0]!=0xE8 || site>UINTPTR_MAX-5)return false; return relativeTarget(bytes+1,site+5)==target; } +// A KC may place branch stubs and GOT slots outside the calling kext. +// Read through the caller's nofault accessor; validate the destination identity. +template uintptr_t importedTarget(uintptr_t entry,uintptr_t expected,Read read) { + if(!entry || !expected)return 0; + if(entry==expected)return entry; + uint8_t stub[6] {};uintptr_t target=0; + if(entry>UINTPTR_MAX-sizeof(stub) || !read(entry,stub,sizeof(stub)) || + stub[0]!=0xFF || stub[1]!=0x25)return 0; + const auto slot=relativeTarget(stub+2,entry+sizeof(stub)); + if(!slot || slot>UINTPTR_MAX-sizeof(target) || !read(slot,&target,sizeof(target)))return 0; + return target==expected ? target : 0; +} } #endif diff --git a/WhateverXe/kern_xelp_geometry.hpp b/WhateverXe/kern_xelp_geometry.hpp index 35a25c1f..e3cefbc5 100644 --- a/WhateverXe/kern_xelp_geometry.hpp +++ b/WhateverXe/kern_xelp_geometry.hpp @@ -15,6 +15,12 @@ inline Linear linear(uint32_t control,uint32_t stride,uint32_t size,uint32_t pos if(r.pitch size_t reachable(const uint8_t *bytes,size_t length,uint8_t *marks, + uint16_t *pending,uintptr_t address,uintptr_t panic,Decode decode) { + if(!bytes || !length || length>0x4000)return 0; + memset(marks,0,length);size_t queued=1,end=0;pending[0]=0; + while(queued) { + size_t pos=pending[--queued]; + while(pos=2 && bytes[trap]==0x0F && (bytes[trap+1]==0xB9 || bytes[trap+1]==0x0B)) { + if(trap+2>end)end=trap+2; + break; + } + const auto n=decode(bytes+pos,length-pos); + if(!n || n>15 || n>length-pos)return 0; + for(size_t k=1;kend)end=pos+n; + const auto op=bytes[pos]; + if(n==5 && op==0xE8 && XELPCall::targets(bytes+pos,n,address+pos,panic))break; + if(op==0xC3 || (op==0xC2 && n==3) || (n==2 && op==0x0F && bytes[pos+1]==0x0B))break; + bool branch=false,conditional=false;int32_t delta=0; + if(n==2 && (op==0xEB || (op>=0x70 && op<=0x7F))) { + branch=true;conditional=op!=0xEB;delta=int8_t(bytes[pos+1]); + } else if(n==5 && op==0xE9) { + branch=true;memcpy(&delta,bytes+pos+1,4); + } else if(n==6 && op==0x0F && (bytes[pos+1]&0xF0)==0x80) { + branch=conditional=true;memcpy(&delta,bytes+pos+2,4); + } + if(branch) { + const auto target=int64_t(pos+n)+delta; + if(target<0 || target>=int64_t(length)) { + if(conditional)return 0; + break; // external tail call; do not scan the following function + } + if(queued>=length)return 0; + pending[queued++]=uint16_t(target); + if(!conditional)break; + } + // Do not infer jump-table targets or mark the following bytes as code. + // A patch site must be reachable through resolved direct edges. + // Indirect CALLs do have a fallthrough. + size_t opcode=pos; + if((op&0xF0)==0x40)++opcode; + if(opcode+1 bool findInstruction(const uint8_t *bytes,size_t length,const uint8_t *marks, + const uint8_t *pattern,size_t count,size_t &offset,Find find,bool requireUnique=true) { + size_t cursor=0,hits=0; + while(find(pattern,nullptr,count,bytes,length,&cursor)) { + if(marks[cursor]==1 && (cursor+count==length || marks[cursor+count]==1)) {offset=cursor;++hits;} + ++cursor; + } + return requireUnique ? hits==1 : hits>0; +} // Only canonical, side-effect-free alignment NOPs may separate the expected // instructions. Never scan past an unrecognised instruction to find a suffix. struct Cursor { @@ -18,15 +85,50 @@ struct Cursor { } bool skip(size_t count) {if(count>length-pos)return false;pos+=count;return true;} }; +enum class SurfaceABI { Unknown, Original, Attachment6A, Attachment6E, RelocatedFields }; +// These exported getters are the meanings of the fields inlined by the fixed +// donor. OS family names do not establish the host object layout. +inline bool surfaceGetter(const uint8_t *bytes,size_t length,const char *load,size_t size) { + if(!bytes)return false; + Cursor c{bytes,length,0}; + // Frame omission and alignment do not change this leaf accessor's ABI. + auto framed=c;const bool frame=framed.take("\x55",1) && framed.take("\x48\x89\xe5",3); + if(frame)c=framed; + return c.take(load,size) && (!frame || c.take("\x5d",1)) && c.take("\xc3",1); +} +inline SurfaceABI surfaceABI(const uint8_t *const functions[4],const size_t lengths[4],bool framebuffer) { + const auto getter=[&](unsigned i,const char *load,size_t n){return surfaceGetter(functions[i],lengths[i],load,n);}; + // Qualify only the fields this consumer reads. A matrix/plane change must + // not disable the framebuffer, nor an ID change disable the accelerator. + if(framebuffer) { + if(getter(0,"\x8b\x87\xa4\0\0\0",6) && getter(2,"\x8b\x47\x0c",3))return SurfaceABI::Original; + if(getter(0,"\x8b\x87\x80\0\0\0",6) && getter(2,"\x8b\x47\x10",3))return SurfaceABI::RelocatedFields; + return SurfaceABI::Unknown; + } + if(getter(0,"\x8b\x87\xa4\0\0\0",6) && getter(1,"\x8b\x87\xb0\0\0\0",6) && + getter(3,"\x8b\x87\xb4\0\0\0",6))return SurfaceABI::Original; + if(!getter(0,"\x8b\x87\x80\0\0\0",6) || !getter(1,"\x8b\x87\x98\0\0\0",6))return SurfaceABI::Unknown; + if(getter(3,"\x48\x8b\x87\x78\x01\0\0\x0f\xb6\x40\x6a",11))return SurfaceABI::Attachment6A; + if(getter(3,"\x48\x8b\x87\x78\x01\0\0\x0f\xb6\x40\x6e",11))return SurfaceABI::Attachment6E; + return SurfaceABI::Unknown; +} // Decode the verifier's first call only after proving both argument sources: // original VA in RSI and symbol-resolved kernel_pmap_store in RDI. -inline uintptr_t rootCall(const uint8_t *bytes,size_t length,uintptr_t base,uintptr_t store) { +inline uintptr_t rootCall(const uint8_t *bytes,size_t length,uintptr_t base,uintptr_t store,uintptr_t global=0) { if(!bytes || !base || !store || length>UINTPTR_MAX-base)return 0; Cursor c {bytes,length,0}; if(!c.take("\x55",1) || !c.take("\x48\x89\xe5",3) || !c.take("\x53",1) || - !c.take("\x50",1) || !c.take("\x48\x89\xfb",3) || !c.take("\x48\x8d\x3d",3) || - !c.skip(4) || XELPCall::relativeTarget(bytes+c.pos-4,base+c.pos)!=store || - !c.take("\x48\x89\xde",3) || !c.take("\xe8",1) || !c.skip(4))return 0; + !c.take("\x50",1) || !c.take("\x48\x89\xfb",3))return 0; + auto argument=c; + if(argument.take("\x48\x8d\x3d",3) && argument.skip(4) && + XELPCall::relativeTarget(bytes+argument.pos-4,base+argument.pos)==store)c=argument; + else { + argument=c; + if(!global || !argument.take("\x48\x8b\x3d",3) || !argument.skip(4) || + XELPCall::relativeTarget(bytes+argument.pos-4,base+argument.pos)!=global)return 0; + c=argument; + } + if(!c.take("\x48\x89\xde",3) || !c.take("\xe8",1) || !c.skip(4))return 0; const auto target=XELPCall::relativeTarget(bytes+c.pos-4,base+c.pos); // The caller consumes the returned PTE pointer; an unrelated call is rejected. return c.take("\x48\x85\xc0",3) ? target : 0; @@ -42,19 +144,21 @@ inline bool rootAccessor(const uint8_t *bytes,size_t length) { // Scan a bounded native function on decoded instruction boundaries. Require // exactly one call to pmap, its complete register/stack argument setup, stack // cleanup, and the function's terminating RET. Embedded bytes are not calls. +enum class ProtectABI { Legacy, Extended, Tahoe }; template bool protectCall(const uint8_t *bytes,size_t length,uintptr_t base, - uintptr_t pmap,bool tahoe,Decode decode) { + uintptr_t pmap,ProtectABI abi,Decode decode) { if(!bytes || !base || !pmap || length>UINTPTR_MAX-base)return false; + const bool legacy=abi==ProtectABI::Legacy, extra=abi==ProtectABI::Extended; unsigned found=0; for(size_t pos=0;pos15 || size>length-pos)return false; if(size==1 && bytes[pos]==0xc3)return found==1; Cursor c {bytes,length,pos}; - bool args=(!tahoe || c.take("\x48\x83\xec\x08",4)) && - c.take("\x4c\x89\xff",3) && c.take("\x4c\x89\xee",3) && c.take("\x89\xc2",2) && - c.take("\x8b\x4d\xd4",3) && c.take("\x44\x8b\x4d\xd0",4) && - (tahoe || c.take("\x6a\0",2)) && c.take("\x48\x8d\x45\xb8",4) && + bool args=(extra || c.take("\x48\x83\xec\x08",4)) && + c.take(legacy?"\x48\x89\xdf":"\x4c\x89\xff",3) && c.take(legacy?"\x4c\x89\xe6":"\x4c\x89\xee",3) && c.take("\x89\xc2",2) && + c.take(legacy?"\x8b\x4d\xd0":"\x8b\x4d\xd4",3) && c.take(legacy?"\x44\x8b\x4d\xd4":"\x44\x8b\x4d\xd0",4) && + (!extra || c.take("\x6a\0",2)) && c.take("\x48\x8d\x45\xb8",4) && c.take("\x50",1) && c.take("\x6a\x20",2) && c.take("\x6a\x01",2) && c.take("\xe8",1); if(args && c.skip(4) && XELPCall::relativeTarget(bytes+c.pos-4,base+c.pos)==pmap && c.take("\x48\x83\xc4\x20",4)) { @@ -69,5 +173,10 @@ template bool protectCall(const uint8_t *bytes,size_t length,uintp } return false; } +template bool protectCall(const uint8_t *bytes,size_t length,uintptr_t base, + uintptr_t pmap,bool tahoe,Decode decode) { + return protectCall(bytes,length,base,pmap,tahoe?ProtectABI::Tahoe:ProtectABI::Extended,decode); +} + } #endif diff --git a/WhateverXe/kern_xelp_os.hpp b/WhateverXe/kern_xelp_os.hpp index ad85a66e..9ee4ca59 100644 --- a/WhateverXe/kern_xelp_os.hpp +++ b/WhateverXe/kern_xelp_os.hpp @@ -8,19 +8,13 @@ inline bool hostOS(OS os) { return os>=OS::BigSur && os<=OS::Tahoe; } inline OS kernelOS(const char *version) { - static const char *const versions[] = { - "Darwin Kernel Version 20.0.0: Sat Jun 13 17:58:16 PDT 2020; root:xnu-7090.0.0.111.5~1/RELEASE_X86_64", - "Darwin Kernel Version 20.6.0: Thu Jul 6 22:12:47 PDT 2023; root:xnu-7195.141.49.702.12~1/RELEASE_X86_64", - "Darwin Kernel Version 21.6.0: Mon Jun 24 00:56:10 PDT 2024; root:xnu-8020.240.18.709.2~1/RELEASE_X86_64", - "Darwin Kernel Version 22.6.0: Tue Jul 15 08:22:28 PDT 2025; root:xnu-8796.141.3.713.2~2/RELEASE_X86_64", - "Darwin Kernel Version 23.6.0: Tue Jul 21 21:56:37 PDT 2026; root:xnu-10063.141.1.713.39~1/RELEASE_X86_64", - }; + // The original DP1 execution environment is a separate exact identity. + // Supported host families below never depend on XNU revision/date text. + static const char donor[]="Darwin Kernel Version 20.0.0: Sat Jun 13 17:58:16 PDT 2020; root:xnu-7090.0.0.111.5~1/RELEASE_X86_64"; if (!version) return OS::Unknown; - for (unsigned i=0;i(i+1); - } + const char *a=version,*b=donor; + while (*a && *a==*b) {++a;++b;} + if (*a==*b) return OS::Donor; // Servicing updates must not silently bypass ABI setup because the build // date/XNU revision changed. Darwin 24.6 cannot distinguish macOS 15.7 // from 15.8: the package enforces 15.8.x. Darwin 25.6+ covers 26.7+. @@ -43,7 +37,10 @@ inline OS kernelOS(const char *version) { const char *tail=end-(sizeof(suffix)-1); for (unsigned i=0;i=20 && numbers[0]<=24 && numbers[1]==6) + return static_cast(unsigned(OS::BigSur)+numbers[0]-20); if (numbers[0]==25 && numbers[1]>=6) return OS::Tahoe; return OS::Unknown; } diff --git a/docs/device-selection.md b/docs/device-selection.md index 163ccebf..ae349bd2 100644 --- a/docs/device-selection.md +++ b/docs/device-selection.md @@ -10,11 +10,12 @@ WEX selects the GPU from its physical PCI ID and detects its actual resources au | Alder Lake-N / Twin Lake | `46D0`–`46D4` | | Alder Lake-H / P / U | `4626`, `4628`, `462A`, `46A0`–`46A3`, `46A6`, `46A8`, `46AA`, `46B0`–`46B3`, `46C0`–`46C3` | | Raptor Lake-H / P | `A720`, `A7A0`, `A7A8`, `A7AA`, `A7AB` | +| Raptor Lake-U | `A721`, `A7A1`, `A7A9`, `A7AC`, `A7AD` | | Alder Lake-S / HX | `4680`, `4682`, `4688`, `468A`, `468B`, `4690`, `4692`, `4693` | | Raptor Lake-S / HX | `A780`, `A781`, `A782`, `A783`, `A788`, `A789`, `A78A`, `A78B` | An implemented backend does not imply hardware validation for every ID or connector. See [compatibility and tested configurations](../README.md#compatibility). -Raptor Lake-U IDs `A721`, `A7A1`, `A7A9`, `A7AC` and `A7AD` are recognized but unsupported: their dedicated display-clock handling is not implemented. Rocket Lake is also unsupported. +Raptor Lake-U internal DSI and acceleration have been verified on MiniBook X U300 (`A7A9`) with Big Sur and Tahoe. Rocket Lake remains unsupported. Memory configuration can change the PCI ID, such as `46A6` to `4626` in a single-channel Alder Lake-H configuration. No override is needed; use updated WEX and Graphics components together. diff --git a/tests/accelerator_resources.py b/tests/accelerator_resources.py new file mode 100644 index 00000000..b4914120 --- /dev/null +++ b/tests/accelerator_resources.py @@ -0,0 +1,104 @@ +#!/usr/bin/env python3 +"""Replay fuse qualification through admission reporting and native probe. + +Resource expectations are explicit Gen12 capacity examples, including a retained +physical 80-EU/six-bank readout. EU and cache populations are independent. A +failed ownership/readout prerequisite must still prevent native entry. +""" +import re, subprocess, tempfile +from pathlib import Path + +root=Path(__file__).resolve().parents[1] +source=(root/'WhateverXe/kern_xe.cpp').read_text() +qualification=source.split(' const bool readoutReady=')[1].split(' if(tglGT2Detected)')[0] +qualification='const bool readoutReady='+qualification +probe=re.search(r'IOService \*XE::wrapTGLAcceleratorProbe\(.*?\n\}',source,re.S)[0] +start=re.search(r'bool XE::wrapTGLAcceleratorStart\(.*?\n\}',source,re.S)[0] +start_guard=start.split('{',1)[1].split(' if (callbackXE->adlnDetected')[0] +aggregate=re.search(r'tglAcceleratorReady = ([^;]+);',source)[0] +resources=re.search(r'const bool resourcesReady=.*?;',source,re.S) +operands=set(re.findall(r'\b\w+\b',aggregate.split('=',1)[1]))-{'resourcesReady'} +code=r''' +#include "kern_xelp_gt_resources.hpp" +#include "kern_alderlake_n.hpp" +#include +#include +#include +using SInt32=int;using mach_vm_address_t=uintptr_t; +struct IOService {virtual ~IOService()=default;}; +struct IOPCIDevice:IOService {uint32_t configRead32(unsigned){return 0x9a498086;}}; +struct IORegistryEntry {static IORegistryEntry *getRegistryRoot(){static IORegistryEntry r;return &r;} + templatevoid setProperty(T...){};}; +#define OSDynamicCast(T,p) dynamic_cast(p) +#define FunctionCast(fn,p) reinterpret_cast(p) +bool selected=true; +bool xelpSelectedDevice(IOPCIDevice *p){return p && selected;} +bool xelpDisplay13RenderDevice(IOPCIDevice *p){return p && selected;} +bool checkKernelArgument(const char*){return false;} +templatevoid tglObserveAccelerator(T...){} +unsigned nativeCalls=0; +IOService *nativeProbe(IOService *that,IOService*,SInt32*){++nativeCalls;return that;} +struct XE { + static XE *callbackXE; + bool tglGT2Detected=false,tglGT2Ready=false,display12SmallGTDetected=false,display12SmallGTReady=false; + bool adlsDisplaySelected=false,adlsDisplayHooksReady=true,adlsBootConsumersFailed=false; + bool adlnDetected=false,tglAcceleratorRequested=true,tglAcceleratorReady=false; + XELP::GTResources tglGT2Resources,display12SmallGT; + OPERANDS + uintptr_t orgTGLAcceleratorProbe=reinterpret_cast(&nativeProbe); + void qualify(AlderLakeN::GTReadout gt,uint32_t edram){QUALIFICATION} + void report(){RESOURCES AGGREGATE} + static IOService *wrapTGLAcceleratorProbe(IOService*,IOService*,SInt32*); + static bool startResourceGate(){START_GUARD return true;} +}; +XE *XE::callbackXE; +PROBE +struct Capacity {bool small;uint32_t fuses[5];unsigned dss,eus,banks;}; +// The cache-disable mask is not derived from the EU count. Keep both TGL +// 80-EU layouts, and independent donor-supported capacities as separate input. +const Capacity inputs[]={ + {false,{1,0x1f,0,0x88,0xe00fa},5,80,6}, + {false,{1,0x1f,0,0,0xe00fa},5,80,8}, + {false,{1,0x07,0,0xc0,0xe00fe},3,48,6}, + {false,{1,0x3f,0,0,0xe00fa},6,96,8}, + {false,{1,0x3f,0xc0,0xf0,0xe00fa},6,72,4}, + {true,{1,0x03,0,0xf0,0xe00fa},2,32,4}, + {true,{1,0x03,0xc0,0xf0,0xe00fe},2,24,4}, + {true,{1,0x01,0,0xf0,0xe00fa},1,16,4} +}; +int main(){IOPCIDevice pci;IOService accelerator;SInt32 score=0; + for(const auto &input:inputs)for(unsigned failure=0;failure<8;++failure){ + XE xe;XE::callbackXE=&xe;xe.tglGT2Detected=!input.small;xe.display12SmallGTDetected=input.small; + AlderLakeN::GTReadout gt;gt.status=AlderLakeN::GTStatus::ProfileMismatch; + gt.requestedMask=gt.releasedMask=3;memcpy(gt.fuses,input.fuses,sizeof(gt.fuses)); + if(failure==1)gt.status=AlderLakeN::GTStatus::AckTimeout; + if(failure==2)gt.releasedMask=1; + if(failure==3)gt.releaseFailedMask=2; + if(failure==4)gt.fuses[0]=UINT32_MAX; + xe.qualify(gt,failure==5?1U:0U); + const bool capacityReady=failure==0 || failure>=6; + const auto &capacity=input.small?xe.display12SmallGT:xe.tglGT2Resources; + assert((input.small?xe.display12SmallGTReady:xe.tglGT2Ready)==capacityReady); + if(capacityReady)assert(capacity.dss==input.dss && capacity.eus==input.eus && + capacity.nativeSubslices==2*input.dss && capacity.l3Banks==input.banks && capacity.urbKiB==64*input.banks); + if(failure==6)xe.surfaceABIReady=false; + selected=failure!=7;nativeCalls=0;xe.report(); + // Admission reporting must include pre-probe resource rejection. This + // caught the misleading Ready=true report independently of decoder tests. + assert(xe.tglAcceleratorReady==(capacityReady && failure!=6)); + const auto result=XE::wrapTGLAcceleratorProbe(&accelerator,&pci,&score); + assert((result==&accelerator)==(failure==0)); + assert(nativeCalls==(failure==0?1U:0U)); + assert(XE::startResourceGate()==capacityReady); + } + puts("PASS independent capacities reach native probe; failed ownership/readout is rejected and reported"); +} +''' +for key,value in {'OPERANDS':'\n'.join('bool '+x+'=true;' for x in sorted(operands)), + 'QUALIFICATION':qualification,'RESOURCES':resources[0] if resources else '', + 'AGGREGATE':aggregate,'START_GUARD':start_guard,'PROBE':probe}.items(): + code=code.replace(key,value) +with tempfile.TemporaryDirectory(prefix='wex-accelerator-resources-') as t: + p=Path(t);(p/'test.cpp').write_text(code) + subprocess.run(['clang++','-std=c++17','-fsanitize=address,undefined','-I'+str(root/'WhateverXe'),str(p/'test.cpp'),'-o',str(p/'test')],check=True) + subprocess.run([str(p/'test')],check=True) diff --git a/tests/adlp_load_only.py b/tests/adlp_load_only.py index 17262154..709fa182 100644 --- a/tests/adlp_load_only.py +++ b/tests/adlp_load_only.py @@ -6,7 +6,7 @@ a=s.index('// ADLP_LOAD_ONLY_HOOKS:');b=s.index('// ADLP_LOAD_ONLY_HOOKS_END',a);body=s[a:b] a=s.index(' // ADLP_LOAD_ONLY_KERNEL:');b=s.index(' // ADLP_LOAD_ONLY_KERNEL_END',a);selection=s[a:b] a=s.index(' if(xeProfile.mobileDisplay13() && physicalGPU && checkKernelArgument("-wex-adlp-readout")');b=s.index(' if(((display12SmallGTDetected',a);diagnostic=s[a:b] -dispatch=s[s.index('bool XE::processKext('):s.index(' if((tglRequested',s.index('bool XE::processKext('))] +dispatch=s[s.index('bool XE::processKext('):s.index(' if(isIntelProvider(index))',s.index('bool XE::processKext('))] assert dispatch.index('if(adlpLoadOnly)') diff --git a/tests/bigsur_pmap.py b/tests/bigsur_pmap.py index 569fd5f9..4902ad50 100644 --- a/tests/bigsur_pmap.py +++ b/tests/bigsur_pmap.py @@ -5,11 +5,14 @@ or GPU coherency. """ from pathlib import Path -import argparse,hashlib,re,struct,subprocess,tempfile +import argparse,re,subprocess,tempfile R=Path(__file__).resolve().parents[1] s=(R/'WhateverXe/kern_xe.cpp').read_text() -a=argparse.ArgumentParser();a.add_argument('--kernel',type=Path);a=a.parse_args() +a=argparse.ArgumentParser();a.add_argument('--kernel',type=Path);a.add_argument('--hde-source',type=Path);a=a.parse_args() bridges='\n'.join(re.search(r'extern "C" __attribute__\(\(naked,used\)\) void tglPmap'+kind+r'Bridge\(\) \{.*?\n\}',s,re.S)[0] for kind in ['New','Update']) +if a.kernel: + from pmap_kernel_contract import check + check(a.kernel,s,"BigSur",a.hde_source) cpp=r'''#include #include #include @@ -80,7 +83,7 @@ capture='\n'.join('"movq %'+r+', _out+'+str(8*i)+'(%rip)\\n"' for i,r in enumerate(regs)) load='\n'.join('"movq _in+'+str(8*i)+'(%rip), %'+r+'\\n"' for i,r in enumerate(regs) if r!='rsp') cpp=cpp.replace('BRIDGES',bridges).replace('CAPTURE',capture).replace('LOAD',load) -with tempfile.TemporaryDirectory(prefix='ventura-pmap-') as temp: +with tempfile.TemporaryDirectory(prefix='bigsur-pmap-') as temp: p=Path(temp);(p/'test.cpp').write_text(cpp) subprocess.run(['clang++','-arch','x86_64','-std=c++17','-O2',str(p/'test.cpp'),'-o',str(p/'test')],check=True) subprocess.run([str(p/'test')],check=True) diff --git a/tests/display13_boot.cpp b/tests/display13_boot.cpp index 3a70ffdb..a02b4d75 100644 --- a/tests/display13_boot.cpp +++ b/tests/display13_boot.cpp @@ -1,4 +1,5 @@ #include "kern_display13_boot.hpp" +#include "kern_dsi_boot.hpp" #include "kern_display13_tc_mode.hpp" #include "kern_display13_tc_dp.hpp" #include @@ -350,7 +351,40 @@ static void unusedTCBoot() { assert(r.status==Display13Boot::Status::Readback && failed.writes.size()==1); puts("PASS unused fixed TC firmware ownership: cold-only release, peers, live-resource refusal and readback"); } +static void dsiMirror() { + const unsigned ports[]={0,3,4}; + XELP::DSI::Profile profile;profile.status=XELP::DSI::ProfileStatus::Valid; + profile.lanes=4;profile.mode={135255,1080,46,22,2,1920,82,40,2,86,154}; + for(unsigned count:{2U,3U})for(unsigned fault=0;fault<5;++fault) { + IO io; + auto w=multiGOP(io,ports,count,count==2 ? 8 : 24); + w.hdmi=0;w.ddc[0]=0;w.dsi=1;w.typeC=w.dp; + assert(XELPMemory::reservation(io.memory,io.memory.base+io.memory.bytes-0x100000)); + for(unsigned p=1;p960000000); + } + { + IO io; + assert(set(io,960000000,ready,Variant::ADLP).status==ChangeStatus::InvalidFrequency); + assert(io.writes.empty()); + auto r=set(io,960000000,ready,Variant::RPLU); + assert(r.status==ChangeStatus::Complete && r.after.cdclkKHz==480000 && + r.after.ratio==50 && io.regs[Data]==1); + } assert(select(38400,0,true)==1305600000 && select(24000,0,true)==1296000000); assert(select(38400,345600000,false)==358400000 && select(38400,358400000,false)==384000000); assert(!select(38400,1305600000,false) && !select(0,0,true)); diff --git a/tests/display13_dbuf.cpp b/tests/display13_dbuf.cpp index 6551e7eb..71315212 100644 --- a/tests/display13_dbuf.cpp +++ b/tests/display13_dbuf.cpp @@ -18,6 +18,7 @@ struct IO { r[0x60400+pipe*0x1000]=0; if(pipe)r[0x7027C+pipe*0x1000]=r[0x7017C+pipe*0x1000]=0; r[0x70080+pipe*0x1000]=0; + r[0x7003C+pipe*0x1000]=0; for(unsigned d=0;d<3;++d)r[0x70B08+pipe*0x1000+d*0x100]=0; for(unsigned plane=pipe ? 0 : 1;plane<5;++plane)r[0x70180+pipe*0x1000+plane*0x100]=0; } @@ -31,6 +32,50 @@ struct IO { void delayUs(unsigned us) {delays+=us;} }; int main() { + // Port probing may briefly power a reset bank before the next modeset. + // A snapshot during that get/put must preserve the pre-sleep credits. + { + IO io;State s;assert(begin(io,s,7)==Status::Complete); + io.r[0x7003C]=0x01A50A05; + assert(captureCredits(io,s)==Status::Complete); + const auto saved=s.credit[0];assert(saved); + s.restorePending=1;io.r[0x7003C]=0; + assert(captureCredits(io,s)==Status::Complete && s.credit[0]==saved); + io.r[0x45404]&=~(3U<<10); + assert(captureCredits(io,s)==Status::Complete && (s.creditSaved&1) && s.credit[0]==saved); + } + // Reproduce system sleep with a retained external allocation but zeroed + // DDB shadows. Do not invent new ownership or drop pending retirements. + for(unsigned p:{1U,2U})for(unsigned variant=0;variant<8;++variant) { + IO io;State s;assert(begin(io,s,7)==Status::Complete); + assert(bindOffPipe(io,s,p)==Status::Complete); + const auto plan=plannedLayout(7,p);auto &a=s.pipe[p]; + a.primary=plan.primary;a.cursor=plan.cursor;a.consumers=slices(a.primary)|slices(a.cursor); + syncMasks(s);s.restorePending=1U< +#include +struct IOMemoryMap { + std::vector words=std::vector(0x1000000/4); + uintptr_t getVirtualAddress(){return reinterpret_cast(words.data());} + unsigned getLength(){return 0x1000000;} + uint32_t ®(unsigned r){return words[r/4];} +}; +void IODelay(unsigned){} +ADAPTER +int main(){ + using namespace Display13DBUF; + for(unsigned pipe=0;pipe<3;++pipe){ + IOMemoryMap map;Display13FBDBUFIO io{&map};State s; + s.begun=true;s.selected=1U<second;return true; },38400); }; - const auto inspect=[](Fixture &f,const XELP::DSI::Readout &source) { + const auto inspect=[](Fixture &f,const XELP::DSI::Readout &source,uint32_t cropped=0) { return Display13FBMemory::inspect(f,f.controller,sizeof(f.controller),f.framebuffer,sizeof(f.framebuffer), - 0x10000000,memory,true,0,0,nullptr,nullptr,false,&source); + 0x10000000,memory,true,0,0,nullptr,nullptr,false,&source,cropped); }; Fixture good;const auto source=setup(good);assert(source.complete()); auto result=inspect(good,source);assert(result.status==Status::Complete && result.verifiedPages==good.ptes.size()); + for(bool drift:{false,true}) { + Fixture crop;setup(crop);crop.registers[0x70190]=0x04370437; + crop.registers[0x6001C]=0x0437077F;crop.registers[0x68180]=crop.registers[0x68280]=0; + if(drift)crop.changeRegister=0x6001C; + assert(inspect(crop,source,0x04370437).status==(drift?Status::StateChanged:Status::Complete)); + } // The GOP has global DPT enabled, but its linear plane still addresses GGTT. // No acceptance of tiled/rotated geometry or a shortened pool follows. for(auto pair:std::map{{0x70180,0x94000408},{0x70188,67}, diff --git a/tests/display13_power.cpp b/tests/display13_power.cpp index 0a802c51..20d83cae 100644 --- a/tests/display13_power.cpp +++ b/tests/display13_power.cpp @@ -253,6 +253,16 @@ int main() { assert(ok(acquire(io,s,a,BootHDMIA)) && ok(acquire(io,s,b,bit(Domain::PipeB)|bit(Domain::DDIB)))); assert(ok(suspend(io,s)) && s.suspended);assert(ok(restore(io,s)) && !s.suspended && s.refs[0]==2 && io.regs[DDI]==0xf); assert(ok(release(io,s,b)) && ok(release(io,s,a)));empty(s);} + // The DSI selector runs with PW1 on and before the DDI A request. + for(bool select : {false,true}) { + IO io;State s;Lease a;assert(initialize(io,s,All)); + assert(ok(acquire(io,s,a,BootHDMIA)) && ok(suspend(io,s))); + unsigned calls=0; + auto r=restore(io,s,true,[&](uint32_t ddi) { + ++calls;assert((io.regs[Driver]&3)==3 && !(ddi&3));return select; + }); + assert(calls==1 && ok(r)==select && bool(io.regs[DDI]&2)==select); + } // A state failure rolls back A/PW_A/PW1, never a borrowed B request. {IO io;io.regs[DDI]=0xc;io.failedEnableReg=DDI;io.failedEnable=1; State s;Lease a;assert(initialize(io,s,All));auto r=acquire(io,s,a,BootHDMIA); diff --git a/tests/display13_shutdown.py b/tests/display13_shutdown.py index a52b9ccf..c64198bb 100644 --- a/tests/display13_shutdown.py +++ b/tests/display13_shutdown.py @@ -64,7 +64,7 @@ subprocess.run(['clang++','-std=c++17','-O1','-fsanitize=address,undefined',str(p/'test.cpp'),'-o',str(p/'test')],check=True) subprocess.run([str(p/'test')],check=True) override=re.search(r'void XE::wrapAlderLakeNPowerOverride\(.*?\n\}',s,re.S)[0] -assert override.index('display13ShutdownPowerCaller')1: b=Path(sys.argv[1]).read_bytes() install=re.search(r'bool XE::processDisplay13Shutdown\(.*?\n\}',s,re.S)[0] diff --git a/tests/display13_tc.cpp b/tests/display13_tc.cpp index c3335b68..d876e7e8 100644 --- a/tests/display13_tc.cpp +++ b/tests/display13_tc.cpp @@ -70,6 +70,14 @@ int main(){ assert(b.putAUX(io,port).status==Status::Complete && b.state.refs[1]==2); assert(suspend(io,b.state).status==Status::Complete); assert(restore(io,b.state).status==Status::Complete && io.prepares==2); + assert(suspend(io,b.state).status==Status::Complete); + const auto priorPW2=b.state.refs[1]; + assert(restore(io,b.state,true).status==Status::Complete && io.prepares==2); + assert(b.state.deferredTC==(auxDomain(port)|ddiDomain(port))); + assert(b.getAUX(io,port).status==Status::Complete && io.prepares==3 && b.auxHeld(io,port)); + assert(b.putAUX(io,port).status==Status::Complete && b.state.refs[1]==priorPW2); + assert(b.getDDI(io,port).status==Status::Complete && b.ddiHeld(io,port) && !b.state.deferredTC); + assert(b.putDDI(io,port).status==Status::Complete && b.state.refs[1]==priorPW2); assert(b.putAUX(io,port).status==Status::Complete && b.state.refs[1]==1 && (io.regs[Driver]&0xC)==0xC); assert(b.putDDI(io,port).status==Status::Complete && !b.nativeDDIRefs(port)); assert((io.regs[0x45404]&0xA80A)==0x802 && (io.regs[0x45454]&0x2AAA)==2); diff --git a/tests/display13_tc_mode.cpp b/tests/display13_tc_mode.cpp index 327f72db..045c3ae6 100644 --- a/tests/display13_tc_mode.cpp +++ b/tests/display13_tc_mode.cpp @@ -43,6 +43,21 @@ struct IO { bool quiescent(uintptr_t,uintptr_t,unsigned d){return quiet && !(regs[0x64000+0x100*d]&0x80000000U);} }; int main() { + // System power loss keeps leases but erases the PHY and AUX request. + // Reclaim PHY before AUX without accumulating logical references. + for(unsigned d=3;d<=6;++d) { + IO io(d);Mode m;assert(m.connect(io,1,2,d).status==Status::Complete); + const auto before=io.power; + io.regs[0x45404]=io.regs[0x45454]=io.regs[0x45444]=0; + io.regs[io.port.buffer]=0;m.notePowerLoss();io.power.suspended=true; + assert(Display13Power::restore(io,io.power,true).status==Display13Power::Status::Complete); + assert(!m.ownership.active && m.restorePending && !(io.regs[0x45444]&0xc0)); + assert(m.connect(io,1,2,d).status==Status::Complete); + assert(m.ownership.active && !m.restorePending); + for(unsigned n=0;n Doze/Wake contract is from prepareToExitSleep in the +identified framebuffer donor. This fixture does not emulate native MMIO or +IOGraphics thread scheduling; those remain live validation requirements. +""" +from pathlib import Path +import re, subprocess, tempfile + +root = Path(__file__).resolve().parents[1] +source = (root/'WhateverXe/kern_display13_framebuffer_power.inc').read_text() +function = re.search(r'bool XE::finishDisplay13SystemWake\([^;{]*\{.*?\n\}', source, re.S)[0] +code = r''' +#include "kern_display13_fb_memory.hpp" +#include +#include +struct IOService { + alignas(8) uint8_t bytes[0x4B90]{}; unsigned refs=1; + IOService *getMetaClass(){return this;} + unsigned getClassSize(){return sizeof(bytes);} + void retain(){++refs;} void release(){assert(refs>1);--refs;} +}; +using IOLock=int; unsigned held=0; +void IOLockLock(IOLock*){assert(!held);++held;} +void IOLockUnlock(IOLock*){assert(held==1);--held;} +struct IORegistryEntry { + static IORegistryEntry *getRegistryRoot(){static IORegistryEntry r;return &r;} + void setProperty(const char*,const void*,unsigned){} +}; +constexpr uint32_t kIOReturnSuccess=0; +struct XE { + uintptr_t display13FramebufferEvent=0; IOService *adlnTopologyController=nullptr; + bool display13PagingLayoutKnown=true,dsiPowerOffNotified=true; + uint32_t dsiPowerOffEvents=1,dsiPowerOnEvents=0; + struct {IOLock *lock=nullptr;uint32_t restoreEpoch=1;}display13FBPower; + struct {bool dsi=true;}adlnOutputWiring; + uint32_t displayWakeEventBusy=0,displayWakeEventEpoch=0,displayWakeEventErrors=0; + uint32_t displayWakeEventPhase[3]{}; + bool restoreDisplay13Ports(IOService*){assert(!held);return true;} + bool finishDisplay13SystemWake(IOService*); +}; +FUNCTION +template void put(IOService &fb,unsigned offset,T value){memcpy(fb.bytes+offset,&value,sizeof(value));} +unsigned wakes[3]{},ons[3]{}; bool powerThread=true; +uint32_t event(IOService *fb,int what,void *arg){ + assert(!held); + if(!powerThread)return 0; // handleEvent's successful no-delivery path + const auto i=Display13FBMemory::field(fb,0x1DC); + if(what==4){assert(arg==reinterpret_cast(1));++wakes[i];fb->bytes[0x199]|=0x20;} + else {assert(what==8 && !arg);++ons[i];} + return 0; +} +int main(){ + IOService ctl,fb[3]; IOLock lock=0;XE xe; + xe.adlnTopologyController=&ctl;xe.display13FBPower.lock=&lock; + xe.display13FramebufferEvent=reinterpret_cast(&event); + for(unsigned i=0;i<3;++i){put(ctl,0xD60+8*i,&fb[i]);put(fb[i],0x1D0,&ctl);put(fb[i],0x1DC,i);} + auto reset=[&](bool nativeFirst){ + ++xe.display13FBPower.restoreEpoch;xe.dsiPowerOffNotified=true; + for(unsigned i=0;i<3;++i){wakes[i]=ons[i]=0;fb[i].bytes[0x199]=0;} + put(fb[0],0x4284,2U);put(fb[1],0x4284,0U);put(fb[2],0x4284,1U); + if(nativeFirst){event(&fb[1],4,reinterpret_cast(1));event(&fb[1],8,nullptr);put(fb[1],0x4284,2U);} + }; + // Same topology, either callback order: each client receives one wake. + // FB2 is parked in Doze; native Doze -> Wake emits no DidWake. + for(bool nativeFirst:{false,true}){ + reset(nativeFirst);assert(xe.finishDisplay13SystemWake(&fb[0])); + assert(wakes[1]==unsigned(nativeFirst)); + if(!nativeFirst){event(&fb[1],4,reinterpret_cast(1));event(&fb[1],8,nullptr);put(fb[1],0x4284,2U);} + assert(xe.finishDisplay13SystemWake(&fb[1])); + for(unsigned i=0;i<3;++i)assert(wakes[i]==1 && ons[i]==1 && fb[i].refs==1); + } + // An inactive native client already parked in Wake also needs one wake; + // no new native transition will notify it. + reset(false);put(fb[1],0x4284,2U);assert(xe.finishDisplay13SystemWake(&fb[0]));assert(wakes[1]==1); + // No-DSI controller retains native ownership, including its FB0. + reset(false);xe.adlnOutputWiring.dsi=false;put(fb[0],0x4284,0U); + assert(xe.finishDisplay13SystemWake(&fb[1]));assert(!wakes[0] && !wakes[1] && wakes[2]==1); + // A successful return without paging activation must remain retryable. + reset(false);xe.adlnOutputWiring.dsi=true;powerThread=false; + assert(!xe.finishDisplay13SystemWake(&fb[0]));assert(!wakes[0]); + powerThread=true;assert(xe.finishDisplay13SystemWake(&fb[0]));assert(wakes[0]==1); + assert(!xe.displayWakeEventBusy && !held); + puts("production wake notification ownership: OK"); +} +'''.replace('FUNCTION',function).replace('#include ', '#include \n#include ') +with tempfile.TemporaryDirectory() as temp: + path=Path(temp);(path/'fixture.cpp').write_text(code) + subprocess.run(['clang++','-std=c++17','-fsanitize=address,undefined','-I'+str(root/'WhateverXe'),str(path/'fixture.cpp'),'-o',str(path/'fixture')],check=True) + subprocess.run([path/'fixture'],check=True) diff --git a/tests/dsi_apple_adapter.py b/tests/dsi_apple_adapter.py index 4b7566d1..95343c34 100644 --- a/tests/dsi_apple_adapter.py +++ b/tests/dsi_apple_adapter.py @@ -208,6 +208,7 @@ data->release();replacement->release(); auto offered=dsiFixedMode.timing;offered.reservedA[0]=0x80001001;offered.signalLevels=1; offered.pixelClock=offered.minPixelClock=offered.maxPixelClock=135260000; + offered.numLinks=2; // CoreDisplay hint on a verified single-link panel auto list=OSArray::withCapacity(1);auto wrong=offered;wrong.horizontalActive=640; auto wrongData=OSData::withBytes(&wrong,sizeof(wrong));auto offeredData=OSData::withBytes(&offered,sizeof(offered)); list->setObject(wrongData);list->setObject(offeredData); @@ -237,6 +238,7 @@ assert(XE::wrapDSIGetModeCount(fb)==2);int32_t pair[2];assert(!XE::wrapDSIGetModes(fb,pair)); assert(static_cast(pair[1])==rotated.reservedA[0]); assert(!XE::wrapDSIValidateMode(fb,pair[1],nullptr,&timing)); + assert(static_cast(timing)->numLinks==1 && rotated.numLinks==2); assert(static_cast(timing)->scalerFlags==0x30 && !dsiFixedMode.timing.scalerFlags); assert(!XE::wrapDSIGetModeInformation(fb,pair[1],&info) && info.width==1920 && info.height==1080); // A large OS list with many distinct scaled dimensions must not be @@ -260,7 +262,15 @@ assert(static_cast(timing)->horizontalScaled==1280+2*i); assert(!XE::wrapDSIGetModeInformation(fb,modes[i],&info) && info.width==1280+2*i && info.height==720+i);} many->release();assert(!XE::wrapDSISetDetailedTimings(fb,rotationList)); - rotationData->release();rotationList->release(); + // A new quarter-turn generation can contain no unrotated wire alias. + auto landscape=OSArray::withCapacity(2);auto scaled=rotated; + scaled.reservedA[0]=0x80005003;scaled.horizontalScaled=1638;scaled.verticalScaled=1024; + auto scaledData=OSData::withBytes(&scaled,sizeof(scaled));landscape->setObject(scaledData);landscape->setObject(rotationData); + assert(!XE::wrapDSISetDetailedTimings(fb,landscape) && dsiFixedMode.clientMode==rotated.reservedA[0]); + assert(!XE::wrapDSIGetCurrentMode(fb,¤t,&depth) && static_cast(current)==rotated.reservedA[0]); + assert(XE::wrapDSIValidateMode(fb,static_cast(offered.reservedA[0]),nullptr,&timing)==kIOReturnBadArgument); + assert(!XE::wrapDSISetDetailedTimings(fb,rotationList) && dsiFixedMode.clientMode==offered.reservedA[0]); + scaledData->release();landscape->release();rotationData->release();rotationList->release(); // Legacy UI attributes alone must never fabricate a presentation timing. auto orientation=new OSNumber;fb->setProperty("IOFBTransform",orientation); for(auto transform:{3ULL,5ULL,0ULL}) { diff --git a/tests/dsi_apple_timing.cpp b/tests/dsi_apple_timing.cpp index 0931d659..dcc08646 100644 --- a/tests/dsi_apple_timing.cpp +++ b/tests/dsi_apple_timing.cpp @@ -14,6 +14,13 @@ int main() { auto offered=t;offered.reservedA[0]=0x80001000;offered.signalLevels=1; offered.pixelClock=offered.minPixelClock=offered.maxPixelClock=135260000; assert(AppleAdapter::clientTiming(t,offered,0)); + { + auto fixed=t;fixed.pixelClock=fixed.minPixelClock=fixed.maxPixelClock=165920000; + auto os=fixed;os.numLinks=2;os.reservedA[0]=0x8000300E; + assert(AppleAdapter::clientTiming(fixed,AppleAdapter::panelOffer(os,fixed),0)); + os.horizontalBlanking++; + assert(!AppleAdapter::clientTiming(fixed,AppleAdapter::panelOffer(os,fixed),0)); + } for(unsigned offset=4;offset(&corrupt)[offset]^=0x80; assert(!AppleAdapter::clientTiming(t,corrupt,0)); diff --git a/tests/dsi_boot_adapter.py b/tests/dsi_boot_adapter.py index 1d511168..6b2b3a24 100644 --- a/tests/dsi_boot_adapter.py +++ b/tests/dsi_boot_adapter.py @@ -2,7 +2,7 @@ """Execute production firmware-copy and DSI pre-native handoff with mock IOKit.""" from pathlib import Path from production_source import read_source -import re, subprocess, tempfile +import re, subprocess, sys, tempfile root=Path(__file__).resolve().parents[1] source=read_source(root/'WhateverXe/kern_xe.cpp') # Observation must precede the destructive external-output trial and must @@ -23,11 +23,18 @@ #include #include #include +#include +#include +#include "kern_dsi_restart.hpp" using namespace XELP::DSI; FIXTURE bool copyOK=true,deviceOK=true,memoryOK=true,auditEnabled=false;unsigned alive=0,releases=0,audits=0,allocations=0; BootAudit lastAudit; -bool checkKernelArgument(const char *key){assert(!strcmp(key,"-wex-dsi-audit"));return auditEnabled;} +bool request60=false; +namespace Crypto {uint8_t *hash(const uint8_t *p,unsigned n){ + auto out=static_cast(malloc(CC_SHA256_DIGEST_LENGTH));return CC_SHA256(p,n,out);}} +namespace Buffer {void deleter(uint8_t *p){free(p);}} +bool checkKernelArgument(const char *key){if(!strcmp(key,"-wex-dsi-60hz"))return request60;assert(!strcmp(key,"-wex-dsi-audit"));return auditEnabled;} void *IOMalloc(size_t bytes){++allocations;return calloc(1,bytes);} void IOFree(void *p,size_t){--allocations;free(p);} struct OSObject {OSObject(){++alive;}virtual ~OSObject(){--alive;}void release(){delete this;}}; @@ -56,7 +63,7 @@ #define SYSLOG(...) #define OSDynamicCast(T,p) dynamic_cast(p) struct XE { - OSData *dsiFirmware=nullptr;Profile dsiProfile;Boot dsiBoot;bool display13BootReady=true; + OSData *dsiFirmware=nullptr;Profile dsiProfile;Boot dsiBoot;bool display13BootReady=true;uint32_t dsi60HzClockKHz=0; XELPMemory::Layout xelpMemory;Display13OutputWiring adlnOutputWiring; bool xelpDisplay13Device(IOPCIDevice *){return deviceOK;} bool xelpMemoryMatches(IOPCIDevice *){return memoryOK;} @@ -72,11 +79,33 @@ uint64_t readPTE(uint32_t r){uint64_t v;memcpy(&v,map.bytes.data()+r,8);return v;} void word(uint32_t r,uint32_t v){memcpy(map.bytes.data()+r,&v,4);} }; -int main(){ +int main(int argc,char **argv){ + for(int i=1;i(in),{}}; + assert(vbt.size()>=28); + const unsigned declared=profileU16(vbt.data()+24);assert(declared<=vbt.size());vbt.resize(declared); + const bool n100=declared==11621;const unsigned clock=n100?164260:165920; + assert(n100 || declared==10269); + for(unsigned scenario=0;scenario<3;++scenario){ + XE x;IOService provider;request60=scenario!=0; + if(scenario==2){vbt[18]^=1;vbt[26]+=vbt[18]&1?-1:1;} // same panel timing, different table + provider.property=OSData::withBytes(vbt.data(),vbt.size()); + assert(x.prepareDSIFirmware(&provider) && x.dsi60HzClockKHz==(scenario==1?clock:0)); + assert(x.dsiProfile.mode.clockKHz==(n100?136887:138267)); // firmware handoff remains 50 Hz + if(x.dsi60HzClockKHz){ + auto target=x.dsiProfile;target.mode.clockKHz=x.dsi60HzClockKHz; + const auto plan=Restart::plan(target,38400); + assert(plan.valid && plan.link.afeKHz==clock*6); + assert(target.mode.htotal()==(n100?1268:1272) && target.mode.vtotal()==(n100?2159:2174)); + } + x.dsiFirmware->release();provider.property->release(); + } + } + request60=true; XE xe;IOService provider;assert(!xe.prepareDSIFirmware(&provider)); auto bytes=fixture();provider.property=OSData::withBytes(bytes.data(),bytes.size()); copyOK=false;assert(!xe.prepareDSIFirmware(&provider) && !xe.dsiFirmware && alive==1);copyOK=true; - assert(xe.prepareDSIFirmware(&provider) && xe.dsiProfile.valid() && alive==2); + assert(xe.prepareDSIFirmware(&provider) && xe.dsiProfile.valid() && !xe.dsi60HzClockKHz && alive==2); assert(xe.dsiFirmware!=provider.property && xe.prepareDSIFirmware(&provider) && alive==2); auto start=static_cast(xe.dsiFirmware->getBytesNoCopy()); for(auto span:xe.dsiProfile.sequences.sequence)if(span.data)assert(span.data>=start && span.data+span.size<=start+bytes.size()); @@ -125,4 +154,4 @@ with tempfile.TemporaryDirectory(prefix='dsi-boot-adapter-') as directory: p=Path(directory);(p/'test.cpp').write_text(code) subprocess.run(['clang++','-std=c++17','-O1','-fsanitize=address,undefined','-I'+str(root/'WhateverXe'),str(p/'test.cpp'),'-o',str(p/'test')],check=True) - subprocess.run([str(p/'test')],check=True) + subprocess.run([str(p/'test'),*sys.argv[1:]],check=True) diff --git a/tests/dsi_host.cpp b/tests/dsi_host.cpp index b160a47e..6e32659b 100644 --- a/tests/dsi_host.cpp +++ b/tests/dsi_host.cpp @@ -29,6 +29,10 @@ int main(int argc,char **argv) { assert(packet(0x15,{data,2}).header==0x40122915); assert(packet(0x39,{data,5}).header==0xC0000539); assert(!packet(6,{data,1}).valid&&!packet(5,{nullptr,1}).valid&&!packet(0x39,{data,257}).valid); + const uint8_t generic[]={0xde,0}; + Mock tx;Host genericHost(true); + assert(genericHost.send(tx,0x29,{generic,2})); + assert(tx.h.size()==1 && tx.h[0]==0xc0000229 && tx.p.size()==1 && tx.p[0]==0xde); Mock io;Host off;assert(!off.send(io,5,{data,1})&&off.error==HostError::Disabled&&io.h.empty()); Host ok(true);assert(ok.send(io,0x39,{data,5})&&ok.pending); assert(io.p.size()==2&&io.p[0]==0x56341229&&io.p[1]==0x78); diff --git a/tests/dsi_panel.py b/tests/dsi_panel.py index e9a39b34..a733d45a 100644 --- a/tests/dsi_panel.py +++ b/tests/dsi_panel.py @@ -22,7 +22,7 @@ struct Sim:IO { uint32_t expectedDuty=UINT32_MAX; bool owned=true,freezeFrame=false,stuckStop=false,stuckStart=false,stuckCommand=false; - uint32_t drop=UINT32_MAX,elapsed=0,headers=0,payload=0,videoCommandDelay=0,creditRelease=0;unsigned irqDepth=0; + uint32_t drop=UINT32_MAX,elapsed=0,headers=0,payload=0,videoCommandDelay=0,initCommandDelay=0,creditRelease=0;unsigned irqDepth=0; std::vector packets,gpioTrace; bool ready(){return owned && (regs[0x45404]&0x403)==0x403;} uint32_t read(uint32_t reg){ @@ -38,6 +38,7 @@ assert(!irqDepth); if(reg==0x6B100){++headers;packets.push_back(value); if(regs[0x7B008]&(1U<<31))creditRelease=elapsed+videoCommandDelay; + else if((value&0x3f)==0x39)creditRelease=elapsed+initCommandDelay; return;} if(reg==0x6B104){++payload;return;} if(reg==0x7B008){regs[reg]=(value&~(1U<<30))|(regs[reg]&(1U<<30));return;} @@ -89,6 +90,11 @@ assert(f.io.regs[0x71180]==0x94000000 && f.io.regs[0x64100]==0x80000000); } int main(int argc,char **argv){ + for(bool stalled:{false,true}){ + Fixture f;f.io.initCommandDelay=stalled?200000:1500; + f.run();assert(f.result.complete()==!stalled); + if(stalled)assert(f.result.stage==Panel::Stage::Init && f.result.host.error==HostError::CreditTimeout); + } for(bool stalled:{false,true}) { Fixture f;f.io.videoCommandDelay=stalled?200000:1000; f.seq[3]={1,5,0,5,1,0,0x11,2,4,0xA0,0x86,1,0,1,5,0,5,1,0,0x29,2,4,0xA0,0x86,1,0,0}; diff --git a/tests/dsi_quiesce_adapter.py b/tests/dsi_quiesce_adapter.py index 6c3fff82..210b5045 100644 --- a/tests/dsi_quiesce_adapter.py +++ b/tests/dsi_quiesce_adapter.py @@ -9,6 +9,9 @@ b=(root/'tests/dsi_boot.cpp').read_text();fixture=b[b.index('struct IO {'):b.index('Boot capture(')] code=r''' #include "kern_dsi_quiesce.hpp" +#include "kern_dsi_sleep_image.hpp" +#include "kern_display13_clock.hpp" +static constexpr auto display13ClockVariant(){return Display13Clock::Variant::RPLU;} #include "kern_dsi_clock.hpp" #include "kern_dsi_restart.hpp" #include "kern_dsi_panel.hpp" @@ -81,7 +84,8 @@ struct {bool ready=true;}dsiFixedMode; #ifndef DSI_SURFACE_REPLAY struct {NativeBacking::Record *record=nullptr;bool modeReady=false;} dsiPendingBacking; -bool dsiPublishedTarget(IOService *){return false;} // This fixture retires backing; it does not retain it for wake. +bool sleepPublished=false; +bool dsiPublishedTarget(IOService *fb){return sleepPublished && fb==pendingFB;} #endif bool dsiModeTarget(IOService *fb){return fb==display13FBPower.pipe[0].memoryFramebuffer;} bool mappingBound=true,failDDIPut=false,failMemoryFree=false,lieMemoryFree=false,loseDBUF=false,loseIRQRestore=false,losePowerFree=false;unsigned memoryFrees=0; @@ -110,6 +114,7 @@ struct {unsigned dsi=1;operator unsigned()const{return dsi;}}adlnOutputWiring;void *adlnPlatform=nullptr; bool dsiNativeWriterHooksReady=true,dsiResourcesReady=true,dsiStopAttempted=false,dsiIRQParked=false,dsiDDIReleased=false,dsiMemoryReleased=false; bool dsiStopTest=false,dsiPowerTest=false; + SystemSleepImage dsiSleepImage; Panel::Shutdown dsiPanelStop; struct {bool sleeping=false,failed=false;unsigned cycles=0;Surface::Plane plane; uint32_t cursorControl=0,cursorPosition=0,cursorBase=0;Restart::State host;Panel::Result panel;}dsiRuntimePower; @@ -189,6 +194,20 @@ } }; int main(){ + // The DSI snapshot must survive either peer's still-active stop ordering. + for(unsigned peer:{0U,2U,4U,6U}) {Case c; + static const uint8_t end[]={0};c.xe.dsiProfile.sequences.status=SequenceStatus::Valid; + for(unsigned id:{1U,2U,4U,5U,6U,7U,10U,11U})c.xe.dsiProfile.sequences.sequence[id]={end,sizeof(end)}; + pendingFB=&c.fb;armedOwner=true;sleepPublished=true;dsiPendingBacking.modeReady=true; + display13FBPower.dbuf.selected=1|peer; + display13FBIRQ.state.active=1|peer; + const auto peerPlane=c.map.reg(0x71180); + assert(c.xe.quiesceDSIPath(&c.controller,&c.fb,&c.path,true)); + assert(c.xe.dsiRuntimePower.sleeping && c.xe.dsiSleepImage.captured); + assert(display13FBIRQ.state.active==peer && c.map.reg(0x71180)==peerPlane); + assert(!memoryFrees && pendingFB==&c.fb && locks.empty()); + pendingFB=nullptr;armedOwner=sleepPublished=false;dsiPendingBacking.modeReady=false; + } {Case c;pendingFB=&c.fb;armedOwner=true;nativeRetires=0; assert(c.run() && nativeRetires==1 && !pendingFB && memoryFrees==1);armedOwner=false;} {Case c;pendingFB=&c.fb;armedOwner=true;failNativeRetire=true;nativeRetires=0; diff --git a/tests/dsi_resources.py b/tests/dsi_resources.py index 489b45a2..68c411d7 100644 --- a/tests/dsi_resources.py +++ b/tests/dsi_resources.py @@ -21,6 +21,7 @@ #include #include using namespace XELP::DSI; +Display13Clock::Variant display13ClockVariant(){return Display13Clock::Variant::ADLP;} IO_FIXTURE unsigned lockLevel=0; struct IORecursiveLock{};struct IOLock{};struct IOSimpleLock{}; @@ -46,7 +47,7 @@ }; struct {IORecursiveLock *lock;uintptr_t thread=0;XELPComboClock::ClockAllocation allocation;} comboClockTransaction; struct {IOLock *lock;IOService *controller;Map *map;Display13Transcoder::Bindings transcoders; - Readout bootDSI;Display13Power::NativeBridge bridge; + uint32_t clippedDSIBootSize {};Readout bootDSI;Display13Power::NativeBridge bridge; bool engineEnabled=false,memoryFaulted=false,clockFaulted=false;struct {bool begun=false;} dbuf;} display13FBPower; struct {uintptr_t base=0;IOSimpleLock *lock=nullptr;IOService *controller=nullptr;Display13FBIRQ::State state; unsigned handlers=0,softwareHandlers=0,admittedPipes=0;bool quiescing=false;} display13FBIRQ; @@ -54,7 +55,7 @@ unsigned published=0; void dsiPublishIRQLive(){assert(lockLevel==0 && display13FBIRQ.controller && display13FBIRQ.lock);++published;} struct XE { - bool dsiPanelTest=false,dsiPanelOnly=false,cycleOK=true;unsigned cycles=0;bool externalPreparation=false; + uint32_t dsi60HzClockKHz=0;bool dsiPanelTest=false,dsiPanelOnly=false,cycleOK=true;unsigned cycles=0;bool externalPreparation=false; bool probeDSIPanel(IOService*,bool prepare){assert(lockLevel==0);++cycles;externalPreparation=prepare;return cycleOK;} bool dsiCommandTest=false;bool probeDSICommands(IOService*){assert(lockLevel==0);return true;} bool display13FBIRQReady=true,dsiNativeWriterHooksReady=true,dsiResourcesReady=false;IOService *adlnTopologyController; diff --git a/tests/dsi_restart.py b/tests/dsi_restart.py index 98283ca5..b592f256 100644 --- a/tests/dsi_restart.py +++ b/tests/dsi_restart.py @@ -22,7 +22,7 @@ struct Sim:IO { bool owned=true,dropPLLDisable=false,dropPLLPower=false,dropPLLLock=false,dropBuffer=false,dropLink=false; bool ulpsRequest=false; - uint32_t drop=UINT32_MAX,elapsed=0; + uint32_t drop=UINT32_MAX,elapsed=0,linkReadyAt=0; std::vector gpioTrace; bool gpio(unsigned role,unsigned value){const auto r=Panel::gpio(*this,role,value);if(r.complete)gpioTrace.push_back(role*2+value);return r.complete;} bool ready(){return owned;} @@ -50,7 +50,7 @@ if(pll&(1U<<31)){if(!dropPLLLock)pll|=1U<<30;}else if(!dropPLLDisable)pll&=~(1U<<30); if(!dropBuffer){if(regs[0x64000]&(1U<<31))regs[0x64000]&=~0x80U;else regs[0x64000]|=0x80;} if(ulpsRequest)regs[0x6B0D8]|=1U<<16; - if(!dropLink && (regs[0x6B400]&(1U<<31)))regs[0x6B030]|=1U<<20; + if(!dropLink && elapsed>=linkReadyAt && (regs[0x6B400]&(1U<<31)))regs[0x6B030]|=1U<<20; } }; struct Fixture { @@ -89,7 +89,7 @@ assert(f.io.regs[0x70180]==0x94000008 && f.io.regs[0x7B008]==0xC0000000); assert(f.allocation.external[0]==1 && f.allocation.binding[0]==2); } -void retainedCycle(Fixture &f,unsigned fault=0){ +void retainedCycle(Fixture &f,unsigned fault=0,bool cold=false){ f.io.regs[0x7B008]=0xC0000000;f.io.regs[0x70180]=0x94001000;f.io.regs[0x7019C]=f.io.regs[0x701AC]=0x44E00000; f.io.regs[0x42084]=1U<<30;f.io.regs[0x6001C]=0x0437077F;f.io.regs[0x6B010]=f.io.regs[0x6B00C]; f.io.regs[0xC7204]=15;f.io.regs[0xC4030]=0x84;f.source.mux=f.io.regs[0x164280]; @@ -105,7 +105,16 @@ assert(Restart::stopQuiesced(f.io,f.source,f.p,stopped,f.allocation,1,f.state)); assert(f.state.stopped && !f.io.regs[0x46014] && f.allocation.external[0]==1 && f.allocation.binding[0]==2); Panel::Result panel;Lifecycle lifecycle{f.allocation,f.state}; - Panel::cycle(f.io,f.source,f.p,0x94001000,0x44E00000,panel,lifecycle,&shutdown); + Panel::cycle(f.io,f.source,f.p,0x94001000,0x44E00000,panel,lifecycle,&shutdown,nullptr,cold); + if(cold){ + size_t ready=0,timing=0,video=0; + for(size_t i=count;i=f.io.linkReadyAt);} for(unsigned fault=0;fault<4;++fault){Fixture f;retainedCycle(f,fault);} {Fixture f;panelCycle(f);} {Fixture f;panelCycle(f,true);} @@ -195,7 +212,7 @@ assert(!f.start() && f.state.error!=Restart::Error::None); const auto count=f.io.writes.size();assert(!f.start() && f.io.writes.size()==count); assert(f.allocation.external[0]==1 && f.allocation.binding[0]==2 && !f.io.regs[0x7B008]); - assert(f.io.elapsed<5000); + assert(f.io.elapsed<(fault==3?12500U:5000U)); } if(argc==2){ {Fixture f;std::ifstream in(argv[1],std::ios::binary);assert(in);f.vbt=Bytes{std::istreambuf_iterator(in),{}}; diff --git a/tests/dsi_sequence.cpp b/tests/dsi_sequence.cpp index 3e42d96d..2a44bf26 100644 --- a/tests/dsi_sequence.cpp +++ b/tests/dsi_sequence.cpp @@ -22,6 +22,8 @@ int main(int argc,char **argv) { } for(auto op:Bytes{4,5,6,7,255}){auto bad=ops;bad[0]=op;rejected(block(panel(bad)));} for(auto flags:Bytes{1,2,8,32,255}){auto bad=ops;bad[2]=flags;rejected(block(panel(bad)));} + assert(parse(block(panel(Bytes{1,6,0,0x29,2,0,0xde,0,0}))).valid()); + rejected(block(panel(Bytes{1,4,0,0x29,0,0,0}))); auto bad=ops;bad[3]=6;rejected(block(panel(bad))); // read unsupported bad=ops;bad[4]=2;rejected(block(panel(bad))); // wrong packet length bad=ops;bad.pop_back();rejected(block(panel(bad))); // no operation END diff --git a/tests/dsi_surface.py b/tests/dsi_surface.py index 212cc47a..ffc4b14e 100644 --- a/tests/dsi_surface.py +++ b/tests/dsi_surface.py @@ -72,6 +72,12 @@ assert(!munmap(image,size));puts("PASS exact donor setupPlane cache feeds the DSI Y surface transaction; X rejected before MMIO"); } int main(int argc,char **argv){ + {Fixture f;f.current.size=f.io.regs[0x70190]=0x04370437; + assert(f.run().status==Surface::Status::Input); + const auto result=Surface::change(f.io,f.session,f.link,f.p,f.current,f.next,f.backing,f.snapshot,true); + assert(result.status==Surface::Status::Complete && f.io.regs[0x70190]==0x077F0437); + } + {Fixture f;const auto result=f.run();assert(result.status==Surface::Status::Complete && result.writes==17); assert(result.latch.status==Display13WM::Status::Complete && result.latch.frameBefore==100 && result.latch.frameAfter==101); assert(f.session.state==SessionState::Streaming && f.session.currentBacking==201 && f.session.retiredBacking==200 && f.session.lease==100); diff --git a/tests/dsi_surface_adapter.py b/tests/dsi_surface_adapter.py index 5f30f69b..d14111d7 100644 --- a/tests/dsi_surface_adapter.py +++ b/tests/dsi_surface_adapter.py @@ -14,7 +14,8 @@ q=(root/'tests/dsi_quiesce_adapter.py').read_text() q=q.split("code=r'''",1)[1].split("'''.replace",1)[0] prelude='#define DSI_SURFACE_REPLAY\n'+q[:q.index('struct XE {')] -prelude=prelude.replace('#include "kern_dsi_boot.hpp"','#include "kern_dsi_boot.hpp"\n#include "kern_dsi_surface.hpp"\n#include "kern_dsi_native_backing.hpp"\n#include "kern_display13_clock.hpp"') +prelude=prelude.replace('return Display13Clock::Variant::RPLU;', 'return Display13Clock::Variant::ADLP;') +prelude=prelude.replace('#include "kern_dsi_boot.hpp"','#include "kern_dsi_boot.hpp"\n#include "kern_dsi_surface.hpp"\n#include "kern_dsi_native_backing.hpp"') b=(root/'tests/dsi_boot.cpp').read_text();prelude=prelude.replace('FIXTURE',b[b.index('struct IO {'):b.index('Boot capture(')]) prelude=prelude.replace('bool armedOwner=false,failNativeRetire=false;unsigned nativeRetires=0;IOService *pendingFB=nullptr;bool dsiPendingTarget(IOService *fb){return fb && pendingFB==fb;}',state+'\nDSIPendingBackingOwner dsiPendingBacking;\nbool dsiPendingTarget(IOService *fb){return fb && dsiPendingBacking.framebuffer==fb;}') prelude=prelude.replace('#include "kern_dsi_native_backing.hpp"','#include "kern_dsi_native_backing.hpp"\n#include "kern_dsi_native_mode.hpp"') diff --git a/tests/dsi_update_dispatch.py b/tests/dsi_update_dispatch.py index 3450874a..afae756a 100644 --- a/tests/dsi_update_dispatch.py +++ b/tests/dsi_update_dispatch.py @@ -5,13 +5,13 @@ import hashlib,re,subprocess,sys,tempfile root=Path(__file__).resolve().parents[1];s=read_source(root/'WhateverXe/kern_xe.cpp') functions='\n'.join(re.search(r'(?:static bool|bool|uint32_t|void|int) (?:XE::)?'+name+r'\([^;{]*\{.*?\n\}',s,re.S)[0] - for name in ['dsiCachePath','dsiLifecycleTarget','wrapDSISetDisplayMode','dsiNativeUpdateAllowed','dsiSharedWriterStopped','wrapDisplay13WMArmUpdate','beginDSINativeWriter','endDSINativeWriter','wrapDSICursorImage','wrapDSICursorState','beginDSIFramebufferWriter','wrapDSIFlipTransaction','wrapDSIGammaTransaction','wrapDSICscTransaction','wrapDSIPlaneGammaTransaction','wrapDSISetGammaTable','wrapDSISetAttribute','wrapDSISetConnectionAttribute','wrapDSISetPowerState','wrapDSISetStream']) + for name in ['dsiCachePath','dsiLifecycleTarget','wrapDSISetDisplayMode','dsiNativeUpdateAllowed','dsiSharedWriterStopped','wrapDisplay13WMArmUpdate','beginDSINativeWriter','endDSINativeWriter','wrapDSICursorImage','wrapDSICursorState','beginDSIFramebufferWriter','wrapDSIFlipTransaction','wrapDSIGammaTransaction','wrapDSICscTransaction','wrapDSIPlaneGammaTransaction','wrapDSISetGammaTable','wrapDSISetAttribute','wrapDSISetConnectionAttribute','wrapDisplay13SetPowerState','wrapDSISetStream']) functions=functions.replace('void XE::endDSINativeWriter(int token) {', 'void XE::endDSINativeWriter(int token) {\n struct EndScope {EndScope(){endingWriter=true;}~EndScope(){endingWriter=false;}} endScope;') scope=re.search(r'struct DSIModeScope \{.*?\n\};',s,re.S)[0] if len(sys.argv)==2: binary=Path(sys.argv[1]).read_bytes() assert hashlib.sha256(binary).hexdigest()=='c004fc1ac9ee4157e2e12736f40fa6639bde025a279326beeb9dd48c9b445731' - for name,width,count in [('processDSINativeWriters',20,2),('processDSINativeTransactions',17,5),('processDSINativeAttributes',20,4)]: + for name,width,count in [('processDSINativeWriters',20,2),('processDSINativeTransactions',17,5),('processDisplay13NativeAttributes',20,4)]: installer=re.search(r'bool XE::'+name+r'\([^;{]*\{.*?\n\}',s,re.S)[0] offsets=[int(x,0) for x in re.search(r'offsets\[\]=\{([^}]+)',installer)[1].split(',')] entries=re.search(r'entries\[\]\['+str(width)+r'\]=\{(.*?)\};',installer,re.S)[1] @@ -31,6 +31,8 @@ using IOService=OSObject; struct IORegistryEntry {static IORegistryEntry *getRegistryRoot(){static IORegistryEntry r;return &r;} void setProperty(const char *name,const void *data,unsigned bytes){ + if(!strcmp(name,"WEXDSISystemWakeEvents")){assert(bytes==8);return;} + if(!strcmp(name,"WEXDSIPowerNotifications")){assert(bytes==20);return;} assert(!strcmp(name,"WEXDSIModeRequest") && bytes==64); const auto words=static_cast(data);assert(words[0]==1); }}; @@ -52,6 +54,7 @@ return XELP::DSI::AppleAdapter::acceptsMode(mode,dsiFixedMode.clientMode)?&t:nullptr; } IOService *panel=nullptr,*nativeFB=nullptr;bool dsiModeTarget(IOService *f){return f && f==panel;} +constexpr uint32_t kIOSystemPowerAttribute=0x73707772U, kIOMessageSystemWillRestart=0xe0000310U, kIOMessageSystemWillPowerOff=0xe0000250U; constexpr uint32_t kIOReturnNotReady=0xe00002d8U,kIOReturnUnsupported=0xe00002c7U,kIOReturnSuccess=0; #define FunctionCast(f,p) reinterpret_cast(p) uint32_t nativeMode(IOService*,int32_t m,int32_t d){assert(!fbLock.depth && !owner);assert(m==0x10000 && d==2);++modeCalls;return 7;} @@ -74,7 +77,13 @@ unsigned streamCalls=0;OSObject *expectedPath=nullptr; uint32_t nativeStream(IOService*,IOService*,OSObject*,const void*); struct XE { + static bool prepareDisplay13Wake(IOService*){return true;} struct {unsigned dsi=1;}adlnOutputWiring;bool dsiStopAttempted=false,dsiResourcesReady=true; + uintptr_t display13FramebufferEvent=0;bool dsiSystemWakeNotify=false,dsiPowerOffNotified=false; + uint32_t dsiWakeIncomplete=0; + uint32_t dsiPowerCallbackBusy=0,dsiPowerOffEvents=0,dsiPowerOnEvents=0; + bool notifyOK=true; + bool finishDisplay13SystemWake(IOService *fb){if(!notifyOK)return false;using Event=uint32_t(*)(IOService*,int,void*);auto event=reinterpret_cast(display13FramebufferEvent);event(fb,4,reinterpret_cast(1));event(fb,8,nullptr);if(dsiPowerOffNotified){dsiPowerOffNotified=false;++dsiPowerOnEvents;}return true;} bool dsiPowerTest=false;struct {bool sleeping=false,failed=false;}dsiRuntimePower;unsigned wakeCalls=0;bool wakeOK=false; bool dsiStopTest=false,dsiMemoryReleased=false,dsiDDIReleased=false,stopOK=false;unsigned stopCalls=0; struct {bool done=false;bool complete(){return done;}}dsiClockStop; @@ -105,7 +114,7 @@ uint32_t saveDSIBrightness(IOService*,uintptr_t value){assert(fbLock.depth && !owner);return value<=UINT32_MAX?0U:kIOReturnUnsupported;} uint32_t setDSIBrightness(IOService*,uintptr_t value){assert(fbLock.depth && !owner);return value<=65535?0U:kIOReturnUnsupported;} static uint32_t wrapDSISetConnectionAttribute(IOService*,int32_t,uint32_t,uintptr_t); - static uint32_t wrapDSISetPowerState(IOService*,uint32_t); + static uint32_t wrapDisplay13SetPowerState(IOService*,uint32_t); uintptr_t orgDSITransaction[5]{}; static int beginDSIFramebufferWriter(IOService*); static uint32_t wrapDSIFlipTransaction(IOService*,void*,uint64_t,void*); @@ -183,9 +192,11 @@ ++streamCalls;return 0xe0000124U; } templatevoid put(void *p,unsigned o,T v){memcpy(static_cast(p)+o,&v,sizeof(v));} +static uint32_t systemEvent(IOService*,int,void*); struct Case { IOService fb,ctl,peer;OSObject path,port;IOLock lock; Case(){assert(!fbLock.depth && !owner);expectedWriters=1;nestedCursor=nestedTx=nestedAttribute=false;xe={};dsiFixedMode={}; + xe.display13FramebufferEvent=reinterpret_cast(&systemEvent); xe.orgDSIAttributes[0]=reinterpret_cast(nativeAttribute); xe.orgDSIAttributes[1]=reinterpret_cast(nativeConnectionAttribute); xe.orgDSIAttributes[2]=reinterpret_cast(nativePower); @@ -208,8 +219,38 @@ } void refuse(){const auto calls=armCalls;XE::wrapDisplay13WMArmUpdate(&fb);assert(calls==armCalls && !fbLock.depth && !owner);} }; +static unsigned systemEvents; +static uint32_t systemEvent(IOService*,int event,void *arg){ + assert(!fbLock.depth && !owner); + assert((event==4 && arg==reinterpret_cast(1)) || ((event==7 || event==8) && !arg));++systemEvents;return 0; +} int main(){ + // Real power wrapper must notify without a retained outer attribute lock, + // preserve the off obligation across failed stop/wake, and avoid a false + // system DidWake for display-only doze. + {Case c;xe.dsiPowerTest=true;put(&c.fb,0x4284,2U); + assert(XE::wrapDSISetAttribute(&c.fb,0x706F7772,1)==kIOReturnNotReady); + assert(xe.dsiPowerOffNotified && xe.dsiPowerOffEvents==1 && !xe.dsiPowerCallbackBusy); + xe.stopOK=true;assert(XE::wrapDSISetAttribute(&c.fb,0x706F7772,1)==0); + assert(xe.dsiPowerOffEvents==1); + assert(XE::wrapDSISetAttribute(&c.fb,0x706F7772,2)==kIOReturnNotReady); + assert(!xe.dsiPowerOnEvents && xe.dsiPowerOffNotified); + xe.wakeOK=true;auto n=systemEvents; + assert(XE::wrapDSISetAttribute(&c.fb,0x706F7772,2)==0); + assert(systemEvents==n+1 && xe.dsiPowerOnEvents==1 && !xe.dsiPowerOffNotified); + } + + for(auto message:{kIOMessageSystemWillRestart,kIOMessageSystemWillPowerOff}){Case c;put(&c.fb,0x4284,2U); + auto notify=[&]{return XE::wrapDSISetAttribute(&c.fb,kIOSystemPowerAttribute,message);}; + assert(notify()==kIOReturnUnsupported && !xe.stopCalls); + xe.dsiPowerTest=true; + assert(XE::wrapDSISetAttribute(&c.fb,kIOSystemPowerAttribute,0xe0000255U)==kIOReturnUnsupported); + assert(notify()==kIOReturnNotReady && Display13FBMemory::field(&c.fb,0x4284)==2); + xe.stopOK=true;assert(notify()==kIOReturnSuccess && xe.dsiRuntimePower.sleeping); + assert(Display13FBMemory::field(&c.fb,0x4284)==0); + auto stops=xe.stopCalls;assert(notify()==kIOReturnSuccess && xe.stopCalls==stops); + } for(unsigned attr:{0x706F7772U,0x70777273U}){Case c;xe.dsiPowerTest=true;put(&c.fb,0x4284,2U); assert(XE::wrapDSISetAttribute(&c.fb,attr,1)==kIOReturnNotReady && xe.stopCalls==1); assert(Display13FBMemory::field(&c.fb,0x4284)==2); @@ -221,10 +262,17 @@ assert(Display13FBMemory::field(&c.fb,0x4284)==0 && !xe.dsiMemoryReleased && !xe.dsiDDIReleased); c.refuse();c.refuseCursor();xe.wakeOK=false; assert(XE::wrapDSISetAttribute(&c.fb,attr,2)==kIOReturnNotReady); - assert(Display13FBMemory::field(&c.fb,0x4284)==0); - xe.wakeOK=true;assert(XE::wrapDSISetAttribute(&c.fb,attr,2)==kIOReturnSuccess); + assert(Display13FBMemory::field(&c.fb,0x4284)==0 && xe.dsiWakeIncomplete); + xe.dsiSystemWakeNotify=true;xe.display13FramebufferEvent=reinterpret_cast(&systemEvent); + const auto eventsBefore=systemEvents; + xe.wakeOK=true;xe.notifyOK=false; + assert(XE::wrapDSISetAttribute(&c.fb,attr,2)==kIOReturnNotReady); + assert(xe.dsiWakeIncomplete && xe.dsiSystemWakeNotify && !xe.dsiRuntimePower.sleeping); + const auto completedWakeCalls=xe.wakeCalls;xe.notifyOK=true; + assert(XE::wrapDSISetAttribute(&c.fb,attr,2)==kIOReturnSuccess); + assert(!xe.dsiWakeIncomplete && xe.wakeCalls==completedWakeCalls); assert(!xe.dsiRuntimePower.sleeping && !xe.dsiStopAttempted && Display13FBMemory::field(&c.fb,0x4284)==2); - auto wakes=xe.wakeCalls;assert(XE::wrapDSISetAttribute(&c.fb,attr,2)==kIOReturnSuccess && xe.wakeCalls==wakes); + auto wakes=xe.wakeCalls;assert(XE::wrapDSISetAttribute(&c.fb,attr,2)==kIOReturnSuccess && xe.wakeCalls==wakes && systemEvents==eventsBefore+2); } xe.dsiRuntimePower.failed=true;auto calls=xe.stopCalls; assert(XE::wrapDSISetAttribute(&c.fb,attr,1)==kIOReturnNotReady && xe.stopCalls==calls); @@ -359,7 +407,8 @@ assert(attrCalls[1]==before && !display13FBPower.dsiNativeWriters);} for(unsigned old=0;old<3;++old)for(unsigned state=0;state<4;++state){Case c;put(&c.fb,0x4284,old);const auto before=attrCalls[2]; const auto expected=old==2 && state==2?kIOReturnSuccess:kIOReturnUnsupported; - assert(XE::wrapDSISetPowerState(&c.fb,state)==expected && attrCalls[2]==before); + assert(XE::wrapDisplay13SetPowerState(&c.fb,state)==expected && attrCalls[2]==before); + assert(xe.dsiWakeIncomplete==0); // no prior sleep or pending notification assert(Display13FBMemory::field(&c.fb,0x4284)==old); assert(XE::wrapDSISetAttribute(&c.fb,0x706F7772,state)==expected);} {Case c;put(&c.fb,0x4284,2U);assert(XE::wrapDSISetAttribute(&c.fb,0x706F7772,0x100000002ULL)==kIOReturnUnsupported);} @@ -368,7 +417,11 @@ if(bad==2)xe.dsiNativeWriterHooksReady=false;if(bad==3)display13FBPower.memoryFaulted=true; assert(XE::wrapDSISetAttribute(&c.fb,0x77737276,1)==kIOReturnNotReady); assert(XE::wrapDSISetConnectionAttribute(&c.fb,0,0x72677363,1)==kIOReturnNotReady); - assert(XE::wrapDSISetPowerState(&c.fb,2)==kIOReturnNotReady); + assert(XE::wrapDisplay13SetPowerState(&c.fb,2)==kIOReturnNotReady); + assert(xe.dsiWakeIncomplete==0); // startup NotReady is not failed wake + xe.dsiRuntimePower.sleeping=true; + assert(XE::wrapDisplay13SetPowerState(&c.fb,2)==kIOReturnNotReady); + assert(xe.dsiWakeIncomplete==1); // same early failure during real restoration assert(attrCalls[0]+attrCalls[1]+attrCalls[2]==before && !display13FBPower.dsiNativeWriters);} for(bool disabled:{false,true}){Case c;expectedWriters=0;expectedValue=txOptions;expectedAttribute=0x706F7772;expectedConnection=4;expectedPower=0; IOService *fb=&c.fb; @@ -376,7 +429,7 @@ else{panel=nullptr;put(&c.path,0x3640,1U);put(&c.port,0x120,3U);xe.dsiStopAttempted=true;} assert(XE::wrapDSISetAttribute(fb,expectedAttribute,expectedValue)==0xe0000121U); assert(XE::wrapDSISetConnectionAttribute(fb,expectedConnection,expectedAttribute,expectedValue)==0xe0000122U); - assert(XE::wrapDSISetPowerState(fb,expectedPower)==0xe0000123U);} + assert(XE::wrapDisplay13SetPowerState(fb,expectedPower)==0xe0000123U);} {Case c;auto before=streamCalls;assert(XE::wrapDSISetStream(&c.ctl,&c.fb,&c.path,¶mAnchor)==kIOReturnNotReady); assert(streamCalls==before && !fbLock.depth && !owner);} {Case c;panel=nullptr;put(&c.path,0x3640,1U);put(&c.port,0x120,3U);expectedPath=&c.path; diff --git a/tests/fixtures/boot_argument_names.json b/tests/fixtures/boot_argument_names.json index 9bef982c..f6a6c31a 100644 --- a/tests/fixtures/boot_argument_names.json +++ b/tests/fixtures/boot_argument_names.json @@ -1,5 +1,6 @@ { "-wex-dsi-audit": [], + "-wex-dsi-60hz": [], "-wex-dsi-adopt": [], "-wex-dsi-panel-only": [], "-wex-dsi-command-test": [], diff --git a/tests/fixtures/released_physical_admission.json b/tests/fixtures/released_physical_admission.json new file mode 100644 index 00000000..2892d001 --- /dev/null +++ b/tests/fixtures/released_physical_admission.json @@ -0,0 +1,327 @@ +{ + "schema": 1, + "scope": "Physical PCI/backend admission only; not a hardware validation claim.", + "releases": [ + { + "tag": "0.2.2", + "commit": "c3fec934e7ac716d7ae52a70914403d63c802c01", + "physical_ids": [ + { + "devices": [ + "46d0", + "46d1", + "46d2", + "46d3", + "46d4" + ], + "revision_min": 0, + "revision_max": 0 + }, + { + "devices": [ + "9a40", + "9a49", + "9a78" + ], + "revision_min": 0, + "revision_max": 255 + } + ] + }, + { + "tag": "0.2.3", + "commit": "61a449255d385aeb468f64e5d5042e126f387b90", + "physical_ids": [ + { + "devices": [ + "46d0", + "46d1", + "46d2", + "46d3", + "46d4" + ], + "revision_min": 0, + "revision_max": 0 + }, + { + "devices": [ + "9a40", + "9a49", + "9a78" + ], + "revision_min": 0, + "revision_max": 255 + } + ] + }, + { + "tag": "0.3.0", + "commit": "c4881e36f28789a8103eeadcc7c3462e48acb37e", + "physical_ids": [ + { + "devices": [ + "46d0", + "46d1", + "46d2", + "46d3", + "46d4" + ], + "revision_min": 0, + "revision_max": 0 + }, + { + "devices": [ + "9a40", + "9a49", + "9a78" + ], + "revision_min": 0, + "revision_max": 255 + } + ] + }, + { + "tag": "0.4.0", + "commit": "59bbe3cf436272d3a9aa575bdb178afc8fb5cbe0", + "physical_ids": [ + { + "devices": [ + "46d0", + "46d1", + "46d2", + "46d3", + "46d4" + ], + "revision_min": 0, + "revision_max": 0 + }, + { + "devices": [ + "9a40", + "9a49", + "9a78" + ], + "revision_min": 0, + "revision_max": 255 + } + ] + }, + { + "tag": "0.5.0", + "commit": "c6cd8ec5ec5bd64ad3c8a852a30d1f537d819fb1", + "physical_ids": [ + { + "devices": [ + "4680", + "4682", + "4688", + "468a", + "468b", + "4690", + "4692", + "4693", + "46d0", + "46d1", + "46d2", + "46d3", + "46d4", + "9a40", + "9a49", + "9a60", + "9a68", + "9a70", + "9a78", + "a780", + "a781", + "a782", + "a783", + "a788", + "a789", + "a78a", + "a78b" + ], + "revision_min": 0, + "revision_max": 255 + } + ] + }, + { + "tag": "0.6.0", + "commit": "7b6b246dd425ff4847ba31dd66c89740e9c712c0", + "physical_ids": [ + { + "devices": [ + "4680", + "4682", + "4688", + "468a", + "468b", + "4690", + "4692", + "4693", + "46a3", + "46a6", + "46a8", + "46aa", + "46b3", + "46c3", + "46d0", + "46d1", + "46d2", + "46d3", + "46d4", + "9a40", + "9a49", + "9a60", + "9a68", + "9a70", + "9a78", + "a780", + "a781", + "a782", + "a783", + "a788", + "a789", + "a78a", + "a78b", + "a7a0", + "a7a1", + "a7a8", + "a7a9", + "a7aa", + "a7ab", + "a7ac", + "a7ad" + ], + "revision_min": 0, + "revision_max": 255 + } + ] + }, + { + "tag": "0.6.1", + "commit": "68a24273ea9ba668e7bb029ed60fd775c45b16d1", + "physical_ids": [ + { + "devices": [ + "4680", + "4682", + "4688", + "468a", + "468b", + "4690", + "4692", + "4693", + "46a3", + "46a6", + "46a8", + "46aa", + "46b3", + "46c3", + "46d0", + "46d1", + "46d2", + "46d3", + "46d4", + "9a40", + "9a49", + "9a60", + "9a68", + "9a70", + "9a78", + "a780", + "a781", + "a782", + "a783", + "a788", + "a789", + "a78a", + "a78b", + "a7a0", + "a7a1", + "a7a8", + "a7a9", + "a7aa", + "a7ab", + "a7ac", + "a7ad" + ], + "revision_min": 0, + "revision_max": 255 + } + ] + }, + { + "tag": "0.7.0", + "commit": "d0b7fb6275ca51fde088fa7e91595462bd457ec9", + "physical_ids": [ + { + "devices": [ + "4626", + "4628", + "462a", + "4680", + "4682", + "4688", + "468a", + "468b", + "4690", + "4692", + "4693", + "46a0", + "46a1", + "46a2", + "46a3", + "46a6", + "46a8", + "46aa", + "46b0", + "46b1", + "46b2", + "46b3", + "46c0", + "46c1", + "46c2", + "46c3", + "46d0", + "46d1", + "46d2", + "46d3", + "46d4", + "9a40", + "9a49", + "9a59", + "9a60", + "9a68", + "9a70", + "9a78", + "9ac0", + "9ac9", + "9ad9", + "9af8", + "a720", + "a721", + "a780", + "a781", + "a782", + "a783", + "a788", + "a789", + "a78a", + "a78b", + "a7a0", + "a7a1", + "a7a8", + "a7a9", + "a7aa", + "a7ab", + "a7ac", + "a7ad" + ], + "revision_min": 0, + "revision_max": 255 + } + ] + } + ] +} diff --git a/tests/fixtures/released_topology.json b/tests/fixtures/released_topology.json new file mode 100644 index 00000000..91621d15 --- /dev/null +++ b/tests/fixtures/released_topology.json @@ -0,0 +1,14 @@ +[ + { + "file": "released_topology_022.cpp", + "commit": "c3fec934e7ac716d7ae52a70914403d63c802c01", + "tag": "0.2.2", + "sha256": "e016a6a1352578c258d20a3505128fb4f4ebb467e36fe801b672ba272998521d" + }, + { + "file": "released_topology_050.cpp", + "commit": "c6cd8ec5ec5bd64ad3c8a852a30d1f537d819fb1", + "tag": "0.5.0", + "sha256": "32d02829260873b2b8e190385cb007bb4c124f88f6a95909a824ed947defbb1c" + } +] diff --git a/tests/fixtures/released_topology_022.cpp b/tests/fixtures/released_topology_022.cpp new file mode 100644 index 00000000..9a4bf4ac --- /dev/null +++ b/tests/fixtures/released_topology_022.cpp @@ -0,0 +1,97 @@ +#include "kern_display13_fb_topology.hpp" +#include +#include +using namespace Display13FBTopology; +struct Platform { + uint64_t words[Bytes/8] {}; + uint8_t *data() {return reinterpret_cast(words);} + uint32_t *config(unsigned d) {return reinterpret_cast(data()+0x20+24*d);} + Platform() { + data()[0x12]=Ports;data()[0x13]=3; + const uint32_t panel[] {0,0,0,0,2,0x18};memcpy(config(0),panel,24); + for(unsigned d=1;d=1 && selector<=3 && + (mask&(1U<<(selector-1))) ? selector-1 : UINT32_MAX; + assert(bootDDI(w,trans)==expected); + } + assert(bootDDI(w,UINT32_MAX)==UINT32_MAX); + assert(bootstrapDDI(w,true,3,1,0x89030100)==((mask&1)?0U:UINT32_MAX)); + assert(bootstrapDDI(w,true,3,1,0x91030100)==((mask&2)?1U:UINT32_MAX)); + assert(bootstrapDDI(w,true,3,3,0x89030100)==UINT32_MAX); // No silent second-GOP takeover. + assert(bootstrapDDI(w,true,2,1,0x89030100)==UINT32_MAX); + assert(bootstrapDDI(w,false,3,1,0x89030100)==UINT32_MAX); + assert(bootstrapDDI(w,true,3,2,0x91030100)==UINT32_MAX); // Boot backend still adopts pipe A. + for(unsigned d=0;d(1),w)); + assert(!memcmp(out.data(),in.data(),0x20) && !memcmp(out.data()+0xF8,in.data()+0xF8,Bytes-0xF8)); + auto malformed=w;malformed.ddc[0]=4;auto keep=out; + assert(!apply(out.data(),Bytes,in.data(),malformed) && !memcmp(out.data(),keep.data(),Bytes)); + if(mask==3){auto broken=out;broken.config(0)[5]=0x18;assert(!excludedTypeC(broken.data(),Bytes,w));} + } + assert(singleOutputWiring(wiring)); + auto invalid=wiring;invalid.ddc[1]=4;assert(bootDDI(invalid,0x90030100)==UINT32_MAX); + puts("PASS AlderLakeN connector policy: stable identities/counts/pool, A/DDC1, B/DDC2 and A+B; measured single-GOP bootstrap, excluded connectors, malformed wiring and foreign config rejection. CPU only."); +} diff --git a/tests/fixtures/released_topology_050.cpp b/tests/fixtures/released_topology_050.cpp new file mode 100644 index 00000000..9f4f0e62 --- /dev/null +++ b/tests/fixtures/released_topology_050.cpp @@ -0,0 +1,129 @@ +#include "kern_display13_fb_topology.hpp" +#include +#include +using namespace Display13FBTopology; +struct Platform { + uint64_t words[Bytes/8] {}; + uint8_t *data() {return reinterpret_cast(words);} + uint32_t *config(unsigned d) {return reinterpret_cast(data()+0x20+24*d);} + Platform() { + data()[0x12]=Ports;data()[0x13]=3; + const uint32_t panel[] {0,0,0,0,2,0x18};memcpy(config(0),panel,24); + for(unsigned d=1;d=1 && selector<=3 && + (mask&(1U<<(selector-1))) ? selector-1 : UINT32_MAX; + assert(bootDDI(w,trans)==expected); + } + assert(bootDDI(w,UINT32_MAX)==UINT32_MAX); + assert(bootstrapDDI(w,true,3,1,0x89030100)==((mask&1)?0U:UINT32_MAX)); + assert(bootstrapDDI(w,true,3,1,0x91030100)==((mask&2)?1U:UINT32_MAX)); + assert(bootstrapDDI(w,true,3,3,0x89030100)==UINT32_MAX); // No silent second-GOP takeover. + assert(bootstrapDDI(w,true,2,1,0x89030100)==UINT32_MAX); + assert(bootstrapDDI(w,false,3,1,0x89030100)==UINT32_MAX); + assert(bootstrapDDI(w,true,3,2,0x91030100)==UINT32_MAX); // Boot backend still adopts pipe A. + for(unsigned d=0;d(1),w)); + assert(!memcmp(out.data(),in.data(),0x20) && !memcmp(out.data()+0xF8,in.data()+0xF8,Bytes-0xF8)); + auto malformed=w;malformed.ddc[0]=4;auto keep=out; + assert(!apply(out.data(),Bytes,in.data(),malformed) && !memcmp(out.data(),keep.data(),Bytes)); + if(mask==3){auto broken=out;broken.config(0)[5]=0x18;assert(!excludedTypeC(broken.data(),Bytes,w));} + } + // All fixed Combo HDMI/DP assignments compose CPU topology/resources. + // DP runtime/boot admission stays closed until the hardware hooks exist. + for(unsigned assignment=1;assignment<9;++assignment) { + Display13OutputWiring w;w.valid=true;unsigned code=assignment; + for(unsigned d=0;d<2;++d){unsigned kind=code%3;code/=3; + if(kind==1){w.hdmi|=1U<1)assert(!memcmp(image.data()+span.first,span.second.data(),span.second.size())); else memcpy(image.data()+span.first,span.second.data(),span.second.size()); } - for(bool display13:{false,true})for(unsigned failure=0;failure<14;++failure){ + for(bool display13:{false,true})for(unsigned failure=0;failure<16;++failure){ shutdownCalls=0; auto b=image;uintptr_t pure=0x1234;memcpy(b.data()+0xccad0+0x188,&pure,sizeof(pure)); XE x;callbackXE=&x;x.display13FramebufferRequested=display13;KernelPatcher p; + // The provider must be resolved before this one-shot installation. These + // are the real consumer guards behind the host notification-order fixture. + if(failure==14)x.xelpProviderProbe=0; + if(failure==15)x.xelpProviderConfigureProbe=0; surfaceOK=failure!=5;nvramOK=failure!=6;x.runtimeOK=failure!=7;x.agdcOK=failure!=8;x.backendOK=failure!=9;x.trainingOK=failure!=10;x.stopDuringStart=failure==11;x.nativeOK=failure!=12; if(failure>=1&&failure<=4)p.failAt=failure; bool handled=x.processFramebufferKext(p,1,(uintptr_t)b.data(),b.size()); diff --git a/tests/host_power_contract.py b/tests/host_power_contract.py new file mode 100644 index 00000000..d8f8ee74 --- /dev/null +++ b/tests/host_power_contract.py @@ -0,0 +1,71 @@ +#!/usr/bin/env python3 +"""Replay production host matchers against supplied extracted host functions. + +Inputs: directories containing -.bin and +-symbols.json (symbol -> virtual address), plus Lilu's hde64.c. +No Apple binaries or machine paths are embedded in this test. +""" +import argparse, json, re, subprocess, tempfile +from pathlib import Path + +p=argparse.ArgumentParser(description=__doc__) +p.add_argument('--host',type=Path,action='append',required=True) +p.add_argument('--hde-source',type=Path,required=True) +a=p.parse_args() +r=Path(__file__).resolve().parents[1] +source=(r/'WhateverXe/kern_host.cpp').read_text() +decoder=re.search(r'static size_t hostInstructionSize\(.*?\n\}',source,re.S)[0] +decoder=decoder.replace('Disassembler::hde_t','hde64s').replace('Disassembler::hdeDisasm(reinterpret_cast(code),','hde64_disasm(code,') +code=r''' +#include +#include +#include +#include +#include +#include "hde64.h" +#include "kern_host_power_contract.hpp" +DECODER +using namespace HostPowerContract; +std::vector read(const char *path){std::ifstream f(path,std::ios::binary);assert(f);return {std::istreambuf_iterator(f),{}};} +int main(int argc,char **argv){ + assert(argc==7);auto event=read(argv[1]),sleep=read(argv[2]),mode=read(argv[3]),wsaa=read(argv[4]); + const auto address=strtoull(argv[5],nullptr,0),count=strtoull(argv[6],nullptr,0); + auto decode=hostInstructionSize; + auto ne=extent(event.data(),event.size(),decode),ns=extent(sleep.data(),sleep.size(),decode),nm=extent(mode.data(),mode.size(),decode); + assert(ne && ns && nm && extent(wsaa.data(),wsaa.size(),decode)); + assert(paging(event.data(),ne,decode)); + const auto site=sleepSite(sleep.data(),ns,address,count,decode);assert(site.valid); + assert(sleepFields(sleep.data(),ns,mode.data(),nm,decode,site.owner)); + assert(!sleepSite(sleep.data(),ns,address,count+1,decode).valid); + auto shifted=sleep;shifted.insert(shifted.begin(),17,0x90); + const auto moved=sleepSite(shifted.data(),ns+17,address-17,count,decode); + assert(moved.valid && moved.offset==site.offset+17 && moved.owner==site.owner); + const auto relocated=sleepSite(sleep.data(),ns,address+0x123400,count+0x123400,decode); + assert(relocated.valid && relocated.offset==site.offset); + // Register allocation of the loop flag is independent of accelerator owner. + for(unsigned flag=12;flag<=15;++flag)if(flag!=site.owner){ + auto changed=sleep;assert(changed[site.offset+site.length]==0x41); + changed[site.offset+site.length+1]=0xB0+(flag-8); + auto alternative=sleepSite(changed.data(),ns,address,count,decode); + assert(alternative.valid && alternative.success==flag && alternative.owner==site.owner); + } + auto wrong=sleep;wrong[site.offset+3]^=8; + assert(!sleepSite(wrong.data(),ns,address,count,decode).valid); + for(size_t n=0;n #include @@ -12,9 +17,17 @@ #include #include #include +#include +#define arrsize(x) (sizeof(x)/sizeof((x)[0])) +#define SYSLOG(...) ((void)0) #include "kern_xelp_platform.hpp" +#include "kern_host_power_contract.hpp" +#include "kern_xelp_native_placement.hpp" +size_t hostInstructionSize(const uint8_t*,size_t){return 0;} +size_t hostFunctionExtent(uintptr_t,uintptr_t,size_t){return 0;} using mach_vm_address_t=uintptr_t; -struct IORegistryEntry {static IORegistryEntry *getRegistryRoot(){static IORegistryEntry r;return &r;}void setProperty(const char*,bool){}}; +void clock_get_uptime(uint64_t *value){*value=0;} +struct IORegistryEntry {static IORegistryEntry *getRegistryRoot(){static IORegistryEntry r;return &r;}void setProperty(const char*,bool){}void setProperty(const char*,const void*,size_t){}}; std::vector events;bool publish=true,published=false;uint32_t identity=0x9a498086;uint8_t revision=0; struct Object {virtual ~Object()=default;}; struct OSData:Object {uint32_t value=0;void *getBytesNoCopy(){return &value;}size_t getLength(){return 4;}}; @@ -35,33 +48,206 @@ };DeviceInfo DeviceInfo::value; namespace WIOKit {bool awaitPublishing(IOPCIDevice *){events.push_back(1);published=publish;return publish;}void renameDevice(IOPCIDevice *device,const char *name){device->name=name;++device->renames;events.push_back(5);}} // DEVICE_ADAPTER +#define KMOD_MAX_NAME 64 +struct kmod_info_t {const char *name;uintptr_t address=0x1000;size_t size=0x1000;kmod_info_t *next=nullptr;}; +kmod_info_t *loadedHead=nullptr;bool validImage=true;unsigned imageUpdates=0; struct KernelPatcher { + struct KextInfo {static constexpr size_t Unloaded=0;const char *id=nullptr;size_t loadIndex=0;}; + enum class Error {NoError,BadImage};Error error=Error::NoError;Error getError(){return error;} static constexpr int KernelID=0; - uintptr_t solveSymbol(int,const char *){return 0;} - templateT solveSymbol(size_t,const char*,mach_vm_address_t,size_t){return nullptr;} + uintptr_t solveSymbol(int,const char *name){return !strcmp(name,"_kmod") && loadedHead ? reinterpret_cast(&loadedHead):0;} + size_t updateRunningInfo(size_t,uintptr_t,size_t size){++imageUpdates;error=validImage?Error::NoError:Error::BadImage;return size;} + templateT solveSymbol(size_t,const char*,mach_vm_address_t,size_t){return T{};} struct RouteRequest {templateRouteRequest(const char*,T,uintptr_t&) {}}; bool routeMultiple(size_t,RouteRequest*,unsigned,mach_vm_address_t,size_t){return true;} - void clearError(){} + void routeBlock(uintptr_t,const uint8_t*,size_t){} + void clearError(){error=Error::NoError;} }; -// The optional observer is inactive in this host-publication fixture. namespace XeDataAccess {bool processKext(KernelPatcher&,size_t,mach_vm_address_t,size_t){return false;}void processKernel(KernelPatcher &){assert(published);events.push_back(7);}} -struct Graphics {size_t loadIndex=4;bool off=false;void switchOff(){off=true;}} graphics; -struct {size_t loadIndex=5;}backlight; +struct Graphics:KernelPatcher::KextInfo {Graphics(size_t i=4){loadIndex=i;id="com.apple.iokit.IOGraphicsFamily";}bool off=false;void switchOff(){off=true;}} graphics; +KernelPatcher::KextInfo surface{"com.apple.iokit.IOSurface",6},backlight{"com.apple.driver.AppleBacklight",5},accelFamily{"com.apple.iokit.IOAcceleratorFamily2",7},kextAppleIntelTGLCompat{"org.b00t0x.AppleIntelTGLCompat",3}; +using IOSimpleLock=int;int scopeLock=0;uintptr_t fakeThread=1; +IOSimpleLock *IOSimpleLockAlloc(){return &scopeLock;} +int IOSimpleLockLockDisableInterrupt(IOSimpleLock*){return 0;} +void IOSimpleLockUnlockEnableInterrupt(IOSimpleLock*,int){} +void *current_thread(){return reinterpret_cast(fakeThread);} +using IOService=IOPCIDevice;using IOReturn=int;using IOFramebuffer=IOPCIDevice; +constexpr uint32_t kIOWindowServerActiveAttribute=1,kIOWSAA_StateMask=15,kIOWSAA_Accelerated=1,kIOWSAA_Unaccelerated=0; +constexpr IOReturn kIOReturnSuccess=0,kIOReturnNotReady=-1; +#define FunctionCast(fn,addr) reinterpret_cast(addr) struct XeHost { - bool active=false,backlightEnabled=false;enum {FB_DETECT,FB_RESET,FB_COPY,FB_ZEROFILL,FB_NONE,FB_TOTAL};uint32_t resetFramebuffer=FB_NONE; + static XeHost *callbackHost; + HOST_STATE + void processLoadedDependency(KernelPatcher&,KernelPatcher::KextInfo&); + uintptr_t sleepFBCount=0,sleepFBAt=0;uint32_t repeatedSleep=0,sleepNotifications=0,wakeNotifications=0; + IOSimpleLock *sleepScopeLock=nullptr; + struct SleepEntry {IOService *fb=nullptr;bool pending=false;};SleepEntry sleepEntries[3]{}; + struct SleepScope {SleepScope *next=nullptr;uintptr_t thread=0;IOService *fb=nullptr;bool reached=false,skipped=false;};SleepScope *sleepScopes=nullptr; + uintptr_t orgDisplayChange=0; + static IOReturn wrapDisplayChange(void*,void*,IOFramebuffer*,int,void*); + static uint32_t pendingSleepDisplays(void*); + uintptr_t orgWSAAAttribute=0; + static IOReturn wrapWSAAAttribute(IOFramebuffer*,uint32_t,uint32_t); + bool backlightEnabled=false;enum {FB_DETECT,FB_RESET,FB_COPY,FB_ZEROFILL,FB_NONE,FB_TOTAL};uint32_t resetFramebuffer=FB_NONE; IOPCIDevice *backlightGPU=nullptr; struct vc_info{};vc_info consoleVinfo;bool gotConsoleVinfo=false; struct XE { + bool defer=false;bool isIntelDonor(size_t i)const{return defer && (i==1 || i==2);} + bool isIntelProvider(size_t index)const; + bool requiresIntelProvider()const; + bool isIntelFramebuffer(size_t)const;Graphics frame{1};Graphics *currentFramebuffer=&frame; + bool requireProvider=false,providerReady=false,providerOK=true;unsigned rejectedDonors=0; + bool tglRequested=false,display13FramebufferRequested=false,adlpLoadOnly=false; + void setHostSurfaceABI(unsigned,unsigned){} + void setHostFramebufferEvent(uintptr_t){} + void setHostSleepReady(bool){} + void setHostWakeRecoveryReady(bool){} + std::vector owned;bool ownedDisplay13Framebuffer(IOService *f)const{return std::find(owned.begin(),owned.end(),f)!=owned.end();} + IOFramebuffer *degraded=nullptr; bool failedDSIWakeFor(IOFramebuffer *f) const {return f && f==degraded;} void processKernel(KernelPatcher &,DeviceInfo *){events.push_back(8);} - void processKext(KernelPatcher&,size_t index,mach_vm_address_t,size_t){events.push_back(100+index);} + void processKext(KernelPatcher&,size_t index,mach_vm_address_t,size_t){ + if(index==3){providerReady=providerOK;return;} + if(requireProvider && isIntelFramebuffer(index) && !providerReady)++rejectedDonors; + events.push_back(100+index); + } } xe; void processKernel(KernelPatcher &); void processKext(KernelPatcher &,size_t,mach_vm_address_t,size_t); static void wrapPanelSetDisplay(){}static void wrapFramebufferInit(){} uintptr_t orgPanelSetDisplay=0,orgFramebufferInit=0;uint8_t *gIOFBVerboseBootPtr=nullptr; }; -'''.replace('// DEVICE_ADAPTER',adapter)+body+r''' +'''.replace('// DEVICE_ADAPTER',adapter).replace('HOST_STATE',host_state)+provider_classifier.replace('XE::','XeHost::XE::')+body+r''' +XeHost *XeHost::callbackHost; +static IOReturn vendor(IOFramebuffer*,uint32_t,uint32_t){return 42;} int main(){KernelPatcher patcher;IOPCIDevice gpu,mei; +XeHost exercised;XeHost::callbackHost=&exercised;exercised.orgWSAAAttribute=reinterpret_cast(&vendor); +assert(XeHost::wrapWSAAAttribute(&gpu,1,1)==42);exercised.xe.degraded=&gpu; +assert(XeHost::wrapWSAAAttribute(&gpu,1,0x101)==kIOReturnNotReady && XeHost::wrapWSAAAttribute(&gpu,1,0)==kIOReturnNotReady); +assert(XeHost::wrapWSAAAttribute(&mei,1,1)==42 && XeHost::wrapWSAAAttribute(&gpu,2,1)==42 && XeHost::wrapWSAAAttribute(&gpu,1,3)==42); +// Replay mixed DarkWake with every subscriber ordering and awake subset. +alignas(8) uint8_t accel[0xd00]{},machine[0x140]{}; +*reinterpret_cast(accel+0x378)=machine; +exercised.sleepFBCount=reinterpret_cast(+[](void*)->uint32_t{return 3;}); +exercised.sleepScopeLock=&scopeLock; +IOPCIDevice fb[3],foreign;exercised.xe.owned={fb,fb+1,fb+2}; +for(unsigned i=0;i<3;++i)reinterpret_cast(machine)[i]=fb+i; +exercised.sleepFBAt=reinterpret_cast(+[](void *m,unsigned i)->IOService*{assert(i<3);return static_cast(m)[i];}); +static uint8_t *testAccel;testAccel=accel;static unsigned cleanups,hardware;static bool reject; +exercised.orgDisplayChange=reinterpret_cast(+[](void*,void*,IOFramebuffer*,int event,void *argument)->IOReturn{ + auto machine=*reinterpret_cast(testAccel+0x378);auto &depth=*reinterpret_cast(machine+0x10c); + if(reject)return -1; + if(event==3 && !argument){depth+=XeHost::pendingSleepDisplays(testAccel);++cleanups;} + if(event==4 && !argument){assert(depth>=3);depth-=3;} + if(argument)++hardware; + return 0; +}); +unsigned ordering[]={0,1,2};do{for(unsigned mask=0;mask<8;++mask){ + for(auto &entry:exercised.sleepEntries)entry={};exercised.repeatedSleep=0; + auto &depth=*reinterpret_cast(machine+0x10c);depth=0;cleanups=hardware=0; + auto notify=[&](unsigned i,int event){assert(XeHost::wrapDisplayChange(nullptr,nullptr,fb+i,event,nullptr)==0);assert(XeHost::wrapDisplayChange(nullptr,nullptr,fb+i,event,reinterpret_cast(1))==0);}; + for(auto i:ordering)notify(i,3);assert(depth==9); + for(unsigned cycle=0;cycle<4;++cycle){ + for(auto i:ordering)if(mask&(1U<(accel+0xc78)=0; + for(auto i:ordering)notify(i,3); + assert(depth==9); + } + for(auto i:ordering)notify(i,4); + assert(depth==0 && exercised.repeatedSleep==4*(3-__builtin_popcount(mask))); + assert(cleanups==15 && hardware==18+4*__builtin_popcount(mask)); + assert(!exercised.sleepScopes); +}} +while(std::next_permutation(ordering,ordering+3)); +// Failed notifications do not establish a completed preparation scope. +for(auto &entry:exercised.sleepEntries)entry={};reject=true; +assert(XeHost::wrapDisplayChange(nullptr,nullptr,fb,3,nullptr)==-1);reject=false; +assert(XeHost::wrapDisplayChange(nullptr,nullptr,fb,3,nullptr)==0); +assert(*reinterpret_cast(machine+0x10c)==3); +// Other threads and direct calls cannot inherit an enclosing skip decision. +XeHost::SleepScope outer;outer.thread=2;outer.fb=fb;exercised.sleepScopes=&outer; +assert(XeHost::pendingSleepDisplays(accel)==3);exercised.sleepScopes=nullptr; +assert(XeHost::pendingSleepDisplays(accel)==3); +// A foreign nested subscriber on this thread is a barrier, not the owner +// of the outer subscriber's pending preparation. +outer.thread=1;outer.fb=fb;exercised.sleepScopes=&outer; +const auto before=*reinterpret_cast(machine+0x10c); +assert(XeHost::wrapDisplayChange(nullptr,nullptr,&foreign,3,nullptr)==0); +assert(*reinterpret_cast(machine+0x10c)==before+3); +assert(exercised.sleepScopes==&outer && !outer.reached); +assert(XeHost::wrapDisplayChange(nullptr,nullptr,&foreign,4,nullptr)==0); +assert(*reinterpret_cast(machine+0x10c)==before); +exercised.sleepScopes=nullptr; +// A donor imports loaded hosts, but a KC host can have no load notification. +// All native dependencies exist before each donor callback. Notification order +// must not defer entry guards past that callback or require an unrelated provider. +kmod_info_t modules[]={ + {surface.id,0x1000,0x1000,modules+1}, + {kextAppleIntelTGLCompat.id,0x2000,0x1000,modules+2}, + {graphics.id,0x3000,0x1000,modules+3}, + {accelFamily.id,0x4000,0x1000,nullptr}}; +loadedHead=modules; +unsigned order[]={1,2,3,4,6,7};do {for(bool display13:{false,true})for(bool validProvider:{false,true}) { + XeHost host;host.active=host.xe.defer=true;events.clear(); + host.systemWakeRecovery=display13; + host.xe.tglRequested=!display13;host.xe.display13FramebufferRequested=display13; + host.xe.providerOK=validProvider;host.xe.requireProvider=true; + std::vector expected; + for(auto id:order){ + if(id==1 || id==2)expected.push_back(100+id); + host.processKext(patcher,id,0,0); + assert(events==expected); + if(id==1 || id==2){assert(host.surfaceSeen);if(display13)assert(host.graphicsSeen && host.accelSeen);} + } + assert(events==expected && events.size()==2); + assert(host.xe.rejectedDonors==(validProvider?0U:1U)); + assert(host.providerSeen && host.xe.providerReady==validProvider); + } +}while(std::next_permutation(order,order+6)); +// No notifications for any dependency: both lifecycle installers still run +// within the donor callbacks, after resolving loaded hosts through Lilu. +for(bool display13:{false,true}){ + XeHost host;host.active=host.xe.defer=true;host.xe.tglRequested=!display13;host.xe.display13FramebufferRequested=display13; + host.systemWakeRecovery=display13;host.xe.requireProvider=true;events.clear(); + host.processKext(patcher,2,0,0);assert((events==std::vector{102})); + assert(host.surfaceSeen && !host.providerSeen); + host.processKext(patcher,1,0,0);assert((events==std::vector{102,101})); + assert(host.providerSeen && !host.xe.rejectedDonors); +} +// Missing/invalid metadata must not be used to fabricate an observed ABI. +// The installer is still dispatched, so its existing fail-closed guard is +// installed instead of leaving the native unpatched lifecycle reachable. +for(unsigned failure=0;failure<4;++failure){ + XeHost host;host.active=host.xe.defer=true;host.xe.tglRequested=true;host.xe.requireProvider=true;events.clear(); + loadedHead=failure==0?nullptr:modules;validImage=failure!=1; + auto savedSize=modules[0].size;auto savedIndex=surface.loadIndex; + if(failure==2)modules[0].size=0;if(failure==3)surface.loadIndex=0; + host.processKext(patcher,2,0,0);assert((events==std::vector{102}) && !host.surfaceSeen); + modules[0].size=savedSize;surface.loadIndex=savedIndex; +} +validImage=true;loadedHead=nullptr; +for(bool loadOnly:{false,true})for(bool requested:{false,true}) { + XeHost host;host.xe.adlpLoadOnly=loadOnly;host.xe.display13FramebufferRequested=requested; + assert(host.xe.requiresIntelProvider()==(requested && !loadOnly)); + assert(host.xe.isIntelProvider(3)==(requested && !loadOnly)); + assert(!host.xe.isIntelProvider(99)); +} + +// The native loader may probe/start a donor as soon as this callback returns. +// No later host notification is available to repair a missed installation. +// Dependencies near the end of a valid kmod list are still loaded dependencies. +for(unsigned prefix:{0U,510U,511U,512U,600U}){ + std::vector peers(prefix);std::vector> ids(prefix,std::vector(64)); + for(unsigned i=0;i{101})); +} +loadedHead=nullptr; const char *names[]={nullptr,"IMEI","HECI","MEI"}; for(unsigned firmware=0;firmware<2;++firmware)for(unsigned me=0;me<5;++me)for(unsigned test=0;test<6;++test){ mei.name=names[me%4];mei.renames=0;gpu.name="GFX0"; diff --git a/tests/kernel_os_updates.cpp b/tests/kernel_os_updates.cpp index de642ecf..3865224c 100644 --- a/tests/kernel_os_updates.cpp +++ b/tests/kernel_os_updates.cpp @@ -15,14 +15,14 @@ int main() { assert(kernelOS("Darwin Kernel Version 25.7.0: update/RELEASE_X86_64")==OS::Tahoe); // Metamorphic family acceptance: vary servicing minor/patch and arbitrary // build metadata, without adding known Apple builds to an allowlist. - for(unsigned major=24;major<=25;++major) for(unsigned minor=6;minor<(major==24 ? 7U : 9U);++minor) + for(unsigned major=20;major<=25;++major) for(unsigned minor=6;minor<(major<25 ? 7U : 9U);++minor) for(unsigned patch=0;patch<100;patch+=7) { char version[160]; snprintf(version,sizeof(version),"Darwin Kernel Version %u.%u.%u: synthetic-%u/RELEASE_X86_64",major,minor,patch,minor*109+patch); - assert(kernelOS(version)==(major==24 ? OS::Sequoia : OS::Tahoe)); + assert(kernelOS(version)==static_cast(unsigned(OS::BigSur)+major-20)); } const char *rejected[] { - "Darwin Kernel Version 23.6.0: update/RELEASE_X86_64", + "Darwin Kernel Version 23.5.0: update/RELEASE_X86_64", "Darwin Kernel Version 24.5.0: update/RELEASE_X86_64", "Darwin Kernel Version 24.7.0: update/RELEASE_X86_64", "Darwin Kernel Version 25.5.0: update/RELEASE_X86_64", diff --git a/tests/monterey_pmap.py b/tests/monterey_pmap.py index 313b110d..db89586f 100644 --- a/tests/monterey_pmap.py +++ b/tests/monterey_pmap.py @@ -5,31 +5,14 @@ or GPU coherency. Optional --kernel checks all source guards against exact bytes. """ from pathlib import Path -import argparse,hashlib,re,struct,subprocess,tempfile +import argparse,re,subprocess,tempfile R=Path(__file__).resolve().parents[1] s=(R/'WhateverXe/kern_xe.cpp').read_text() -a=argparse.ArgumentParser();a.add_argument('--kernel',type=Path);a=a.parse_args() +a=argparse.ArgumentParser();a.add_argument('--kernel',type=Path);a.add_argument('--hde-source',type=Path);a=a.parse_args() bridges='\n'.join(re.search(r'extern "C" __attribute__\(\(naked,used\)\) void tglMontereyPmap'+kind+r'Bridge\(\) \{.*?\n\}',s,re.S)[0] for kind in ['New','Update']) if a.kernel: - d=a.kernel.read_bytes() - assert hashlib.sha256(d).hexdigest()=='587b17cd2f01663b3492dc11c59dc91787fdcbb1bbe542e7e7684a0d37e0a773' - segments=[];pos=32 - for _ in range(struct.unpack_from(' #include #include diff --git a/tests/native_surface.cpp b/tests/native_surface.cpp new file mode 100644 index 00000000..9d2474df --- /dev/null +++ b/tests/native_surface.cpp @@ -0,0 +1,32 @@ +#include "kern_xelp_native_placement.hpp" +#include +#include +using Bytes=std::vector; +int main() { + using A=XELPNative::SurfaceABI; + const Bytes old[]={{0x8b,0x87,0xa4,0,0,0},{0x8b,0x87,0xb0,0,0,0},{0x8b,0x47,0x0c},{0x8b,0x87,0xb4,0,0,0}}; + const Bytes recent[]={{0x8b,0x87,0x80,0,0,0},{0x8b,0x87,0x98,0,0,0},{0x8b,0x47,0x10},{0x48,0x8b,0x87,0x78,1,0,0,0x0f,0xb6,0x40,0x6a}}; + for(auto expected:{A::Original,A::Attachment6A,A::Attachment6E})for(bool frame:{false,true})for(unsigned padding:{0U,17U}) { + Bytes functions[4];const uint8_t *ptr[4];size_t length[4]; + for(unsigned i=0;i<4;++i) { + auto &f=functions[i];f.assign(padding,0x90); + if(frame)f.insert(f.end(),{0x55,0x48,0x89,0xe5}); + auto load=expected==A::Original?old[i]:recent[i]; + if(i==3 && expected==A::Attachment6E)load.back()=0x6e; + f.insert(f.end(),load.begin(),load.end());if(frame)f.push_back(0x5d);f.push_back(0xc3); + ptr[i]=f.data();length[i]=f.size(); + } + for(bool framebuffer:{false,true}) { + const auto result=framebuffer && expected!=A::Original ? A::RelocatedFields : expected; + assert(XELPNative::surfaceABI(ptr,length,framebuffer)==result); + for(unsigned i=0;i<4;++i) { + const bool used=framebuffer ? (i==0 || i==2) : i!=2; + const auto changed=used?A::Unknown:result; + auto &f=functions[i];const auto at=padding+(frame?4:0)+2; + f[at]^=1;assert(XELPNative::surfaceABI(ptr,length,framebuffer)==changed);f[at]^=1; + --length[i];assert(XELPNative::surfaceABI(ptr,length,framebuffer)==changed);++length[i]; + ptr[i]=nullptr;assert(XELPNative::surfaceABI(ptr,length,framebuffer)==changed);ptr[i]=f.data(); + } + } + } +} diff --git a/tests/pmap_kernel_contract.py b/tests/pmap_kernel_contract.py index fa5f763e..4809ebe5 100644 --- a/tests/pmap_kernel_contract.py +++ b/tests/pmap_kernel_contract.py @@ -1,4 +1,4 @@ -"""Check runtime guards in any supplied x86_64 kernel/KC, without a build list.""" +"""Replay production pmap location/ABI guards in a supplied x86_64 kernel.""" import re import struct import subprocess @@ -31,45 +31,68 @@ def read(address, length): return data[offset + address - base:offset + address - base + length] raise ValueError(hex(address)) - fn, protect = (symbols[x] for x in ('_pmap_enter_options', '_IOProtectCacheMode')) - block = source.split('static void tglPrepare' + family + 'MMIOUC')[1].split('static void ')[0] - checks = re.findall(r'!memcmp\(reinterpret_cast\((fn|protect)(\+0x[0-9A-Fa-f]+)?\),([A-Za-z]+),sizeof', block) - assert len(checks) == 8, checks - for base, offset, name in checks: - address = (fn if base == 'fn' else protect) + (int(offset[1:], 16) if offset else 0) - values = re.search(r'const uint8_t ' + name + r'\[\] \{([^}]+)\}', block)[1] - expected = bytes(int(v.strip(), 0) for v in values.split(',')) - assert read(address, len(expected)) == expected, name - assert 'tglNativeProtectCall(protect,fn,' + ('true' if family == 'Tahoe' else 'false') + ')' in block - protect_code = read(protect, 512) - verify = symbols['_pmap_verify_noncacheable'] - caller = read(verify, 96) - store = symbols['_kernel_pmap_store'] - # Resolve the call with the production matcher below before fetching its - # callee. No verifier/call/field offsets or particular relative distance. + fn, protect, verify, store, panic = (symbols[x] for x in ( + '_pmap_enter_options', '_IOProtectCacheMode', '_pmap_verify_noncacheable', '_kernel_pmap_store', '_panic')) + name = '' if family == 'BigSur' else family + block = source.split('static void tglPrepare' + name + 'MMIOUC(')[1].split('\n}')[0] + patterns = re.findall(r'const bool (\w+)Ready=native.find\(\1,\1At(?:,false)?\);', block) + assert len(patterns) >= 4 + abi = re.search(r'tglNativeProtectCall\(protect,fn,XELPNative::ProtectABI::(\w+)\)', block)[1] + assert not re.search(r'fn\+0x|protect\+0x', block), 'fixed native instruction placement' include = Path(__file__).resolve().parents[1] / 'WhateverXe' + decoder = re.search(r'static size_t tglNativeInstructionSize\(.*?\n\}',source,re.S)[0] + decoder = decoder.replace('Disassembler::hde_t','hde64s').replace('Disassembler::hdeDisasm(reinterpret_cast(code),','hde64_disasm(code,') + locator = re.search(r'struct TGLNativePmapCode \{.*?\n\};',source,re.S)[0] + lilu = (hde_source.parent.parent/'Lilu/Sources/kern_patcher.cpp').read_text() + find = re.search(r'bool KernelPatcher::findPattern\(.*?\n\}',lilu,re.S)[0] + caller = read(verify,96) + # Fetch the accessor from the production resolver; no caller/callee offset. + def literal(value): + return ','.join(str(x) for x in value) with tempfile.TemporaryDirectory(prefix='xe-root-target-') as directory: - path = Path(directory) - (path / 'target.cpp').write_text('#include "kern_xelp_native_placement.hpp"\n#include \nint main(){unsigned char b[]={' + ','.join(map(str,caller)) + '};printf("%llx",(unsigned long long)XELPNative::rootCall(b,sizeof(b),' + str(verify) + 'ULL,' + str(store) + 'ULL));}') + path=Path(directory) + (path/'target.cpp').write_text('#include "kern_xelp_native_placement.hpp"\n#include \nint main(){unsigned char b[]={' + literal(caller) + '};printf("%llx",(unsigned long long)XELPNative::rootCall(b,sizeof(b),' + str(verify) + 'ULL,' + str(store) + 'ULL,'+str(symbols['_kernel_pmap'])+'ULL));}') subprocess.run(['clang++','-std=c++17','-I'+str(include),str(path/'target.cpp'),'-o',str(path/'target')],check=True) - target = int(subprocess.check_output([str(path/'target')],text=True),16) + target=int(subprocess.check_output([str(path/'target')],text=True),16) assert target - accessor = read(target, 64) - # Execute the production decoder/field contract on the real native bytes. - def literal(value): - return ','.join(str(x) for x in value) - # Compile the exact production decoder wrapper against Lilu's HDE source. - decoder = re.search(r'static size_t tglNativeInstructionSize\(.*?\n\}',source,re.S)[0] - decoder = decoder.replace('Disassembler::hde_t','hde64s').replace('Disassembler::hdeDisasm(reinterpret_cast(code),','hde64_disasm(code,') - cpp = '#include "kern_xelp_native_placement.hpp"\n#include "hde64.h"\n#include \n' + decoder + '\nint main(){\n' - cpp += 'unsigned char protect[]={' + literal(protect_code) + '}, accessor[]={' + literal(accessor) + '};\n' - cpp += f'assert(XELPNative::protectCall(protect,sizeof(protect),{protect}ULL,{fn}ULL,' + ('true' if family=='Tahoe' else 'false') + ',tglNativeInstructionSize));\n' - cpp += 'assert(XELPNative::rootAccessor(accessor,sizeof(accessor)));}\n' + cpp=r''' +#include "kern_xelp_native_placement.hpp" +#include "hde64.h" +#include +#include +#include +using Bytes=std::vector; +namespace Buffer { templateT*create(size_t n){return static_cast(malloc(n*sizeof(T)));} templatevoid deleter(T*p){free(p);} } +struct KernelPatcher { + static constexpr int KernelID=0; uintptr_t panic; + uintptr_t solveSymbol(int,const char*){return panic;} void clearError(){} + static bool findPattern(const void*,const void*,size_t,const void*,size_t,size_t*); +}; +struct {uint64_t(*nofault)(uint64_t,uint64_t,uint64_t);}tglVM; +uintptr_t mapped=0;Bytes input; +uint64_t copy(uint64_t from,uint64_t to,uint64_t n){if(from!=mapped)return 0;auto len=n(to),input.data(),len);return len;} +'''+find+'\n'+decoder+'\n'+locator+'\nint main(){\n' + cpp+=f'KernelPatcher patcher{{{panic}ULL}};tglVM.nofault=copy;mapped={fn}ULL;\n' + cpp+='const Bytes original={'+literal(read(fn,0x4000))+'};input=original;\n' + cpp+='for(unsigned shift:{0U,17U}){input=original;input.insert(input.begin(),shift,0x90);input.resize(original.size());mapped='+str(fn)+'ULL-shift;TGLNativePmapCode code(patcher,mapped);assert(code.length);\n' + for key in patterns: + values=re.search(r'const uint8_t '+key+r'\[\]\s*\{([^}]+)\}',block)[1] + unique='false' if 'native.find('+key+','+key+'At,false)' in block else 'true' + cpp+='{const uint8_t pattern[]={'+values+'};size_t at=0;assert(code.find(pattern,at,'+unique+'));\n' + cpp+='auto saved=input;input[at+sizeof(pattern)-1]^=1;{TGLNativePmapCode wrong(patcher,mapped);size_t ignored=0;assert(!wrong.find(pattern,ignored,'+unique+'));}input=saved;}\n' + cpp+='}\n' + cpp+='uint8_t protection[]={'+literal(read(protect,512))+'},accessor[]={'+literal(read(target,64))+'};\n' + cpp+=f'assert(XELPNative::protectCall(protection,sizeof(protection),{protect}ULL,{fn}ULL,XELPNative::ProtectABI::{abi},tglNativeInstructionSize));\n' + cpp+=f'assert(!XELPNative::protectCall(protection,sizeof(protection),{protect}ULL,{fn+1}ULL,XELPNative::ProtectABI::{abi},tglNativeInstructionSize));\n' + cpp+='assert(XELPNative::rootAccessor(accessor,sizeof(accessor)));}\n' with tempfile.TemporaryDirectory(prefix='xe-kernel-contract-') as directory: - path = Path(directory) - (path / 'test.cpp').write_text(cpp) - headers = include.parent / 'Lilu.kext/Contents/Resources/Headers' - subprocess.run(['clang','-I'+str(headers),'-c',str(hde_source),'-o',str(path/'hde.o')],check=True) - subprocess.run(['clang++', '-std=c++17', '-I' + str(include), '-I'+str(headers), str(path / 'test.cpp'), str(path/'hde.o'), '-o', str(path / 'test')], check=True) - subprocess.run([str(path / 'test')], check=True) - print('PASS: 8 instruction/stack guards, decoded native protect/verifier calls and private pmap accessor; no build/hash/call-offset admission') + path=Path(directory);(path/'test.cpp').write_text(cpp) + subprocess.run(['clang','-I'+str(hde_source.parent),'-c',str(hde_source),'-o',str(path/'hde.o')],check=True) + subprocess.run(['clang++','-std=c++17','-fsanitize=address,undefined','-I'+str(include),'-I'+str(hde_source.parent),str(path/'test.cpp'),str(path/'hde.o'),'-o',str(path/'test')],check=True) + subprocess.run([str(path/'test')],check=True) + print('PASS: '+family+' production locator with Lilu search/HDE, placement shift, ABI mutation, resolved protect/verifier calls') + +if __name__=='__main__': + import argparse + p=argparse.ArgumentParser();p.add_argument('--kernel',type=Path,required=True);p.add_argument('--family',choices=['BigSur','Monterey','Ventura','Sonoma','Sequoia','Tahoe'],required=True);p.add_argument('--hde-source',type=Path,required=True);a=p.parse_args() + check(a.kernel,(Path(__file__).resolve().parents[1]/'WhateverXe/kern_xe.cpp').read_text(),a.family,a.hde_source) diff --git a/tests/provider_binding_fixture.cpp b/tests/provider_binding_fixture.cpp index 54ff4102..4ba67668 100644 --- a/tests/provider_binding_fixture.cpp +++ b/tests/provider_binding_fixture.cpp @@ -4,6 +4,7 @@ #include #include #include "AppleIntelTGLCompatPortProbe.hpp" +#include "kern_display13_clock.hpp" struct Meta {unsigned size=0x1C20;unsigned getClassSize(){return size;}}; struct OSObject { virtual ~OSObject()=default; diff --git a/tests/released_physical_admission.py b/tests/released_physical_admission.py new file mode 100644 index 00000000..dd88fbb5 --- /dev/null +++ b/tests/released_physical_admission.py @@ -0,0 +1,44 @@ +#!/usr/bin/env python3 +"""Preserve physical selections accepted by each released implementation. + +The frozen corpus comes from the referenced releases' own headers, not the +current DeviceTable. This checks backend selection, not live functionality. +New support may expand the set without rewriting older positive expectations. +""" +import json, subprocess, tempfile +from pathlib import Path + +root=Path(__file__).resolve().parents[1] +corpus=json.loads((root/'tests/fixtures/released_physical_admission.json').read_text()) +assert corpus['schema']==1 and corpus['releases'] +rows=[] +for release in corpus['releases']: + assert len(release['commit'])==40 and release['physical_ids'] + for entry in release['physical_ids']: + for device in entry['devices']: + rows.append('{0x%s,%d,%d,"%s"}' % (device,entry['revision_min'],entry['revision_max'],release['tag'])) +code=r''' +#include "kern_xelp_platform.hpp" +#include +#include +struct Released {unsigned device,first,last;const char *tag;}; +const Released inputs[]={INPUTS}; +int main(){ + for(const auto &input:inputs)for(unsigned rev=input.first;rev<=input.last;++rev){ + const uint32_t physical=(input.device<<16)|0x8086; + auto p=XELP::identify(physical,rev); + if(!p.admitted() || !p.displayAdmitted()){ + fprintf(stderr,"lost released physical selection: %s %04x rev %u\n",input.tag,input.device,rev); + return 1; + } + assert(p.identity==physical && p.effectiveIdentity==physical); + uint32_t property=input.device; + assert(XELP::same(p,XELP::select(physical,rev,true,&property,sizeof(property)))); + } + puts("PASS frozen released physical selections, including explicit unchanged device-id properties"); +} +'''.replace('INPUTS',',\n'.join(rows)) +with tempfile.TemporaryDirectory(prefix='wex-released-selection-') as t: + p=Path(t);(p/'test.cpp').write_text(code) + subprocess.run(['clang++','-std=c++17','-fsanitize=address,undefined','-I'+str(root/'WhateverXe'),str(p/'test.cpp'),'-o',str(p/'test')],check=True) + subprocess.run([str(p/'test')],check=True) diff --git a/tests/released_topology.py b/tests/released_topology.py new file mode 100644 index 00000000..77d6611f --- /dev/null +++ b/tests/released_topology.py @@ -0,0 +1,51 @@ +#!/usr/bin/env python3 +"""Replay frozen release topology assertions through current production callers. + +The removed APIs had narrower return values: TC interrupt bits only, and whether +a false native HDMI lane result was corrected. Adapters project those meanings +from the current hotplug policy and actual common lane wrapper. They do not +reimplement topology validation or HDMI admission. No hardware is accessed. +""" +from pathlib import Path +import hashlib,json,re,subprocess,tempfile + +root=Path(__file__).resolve().parents[1] +source=(root/'WhateverXe/kern_xe.cpp').read_text() +header=(root/'WhateverXe/kern_xe.hpp').read_text() +fields=header[header.index(' TGLHDMIWiring tglHDMIWiring;'):header.index('\tbool tglGMBusReady')] +functions='\n'.join(re.search(pattern,source,re.S).group() for pattern in [ + r'bool XE::tglLoadHDMIWiring\(.*?\n\}',r'bool XE::wrapTGLGetOSInformation\(.*?\n\}', + r'uint32_t XE::wrapTGLGMBus\(.*?\n\}',r'bool XE::wrapFramebufferComputeLanes\(.*?\n\}', + r'bool XE::framebufferHDMIEligible\(.*?\n\}']) +observations=source[source.index('struct TGLGMBusObservation {'):source.index('struct TGLMEIRecord {')] +fixture=(root/'tests/tgl_hdmi_wiring_fixture.cpp').read_text() +prefix=fixture[:fixture.index('int main(){')].replace('// FIELDS',fields).replace('// OBSERVATIONS',observations).replace('// FUNCTIONS',functions) +adapter=r''' +namespace Display13FBTopology { +template uint32_t excludedTypeC(const void *p,uint32_t bytes,const Wiring &w) { + return excludedHotplug(p,bytes,w)&0x3f000000U; +} +template bool skipDPLanes(const void *p,const void *c,const Wiring &w, + bool exact,bool native,uint32_t available,uint32_t lanes) { + setup();object.display13FramebufferRequested=true; + object.adlnPlatform=const_cast(p);object.adlnOutputWiring=w; + testCaller=exact?0x100:0x101;nativeLanesResult=native; + memcpy(port+0x548,&c,sizeof(c));const auto before=lanes; + const bool result=XE::wrapFramebufferComputeLanes(port,nullptr,24,available,&lanes); + assert(nativeLanesCalls==1 && lanes==before); + return result && !native; +} +} +''' +records=json.loads((root/'tests/fixtures/released_topology.json').read_text()) +with tempfile.TemporaryDirectory(prefix='xe-released-topology-') as tmp: + tmp=Path(tmp) + for record in records: + original=(root/'tests/fixtures'/record['file']).read_bytes() + assert hashlib.sha256(original).hexdigest()==record['sha256'] + cpp=tmp/'test.cpp';exe=tmp/'test' + cpp.write_text(prefix+adapter+original.decode()) + subprocess.run(['clang++','-std=c++14','-O1','-g','-fsanitize=address,undefined', + '-fno-omit-frame-pointer','-I'+str(root/'WhateverXe'),str(cpp),'-o',str(exe)],check=True) + subprocess.run([str(exe)],check=True) + print('PASS frozen topology',record['tag'],record['commit'],flush=True) diff --git a/tests/sonoma_pmap.py b/tests/sonoma_pmap.py index 37a0f367..86a347f6 100644 --- a/tests/sonoma_pmap.py +++ b/tests/sonoma_pmap.py @@ -5,35 +5,14 @@ or GPU coherency. Optional --kernel checks all source guards against exact bytes. """ from pathlib import Path -import argparse,hashlib,re,struct,subprocess,tempfile +import argparse,re,subprocess,tempfile R=Path(__file__).resolve().parents[1] s=(R/'WhateverXe/kern_xe.cpp').read_text() -a=argparse.ArgumentParser();a.add_argument('--kernel',type=Path);a=a.parse_args() +a=argparse.ArgumentParser();a.add_argument('--kernel',type=Path);a.add_argument('--hde-source',type=Path);a=a.parse_args() bridges='\n'.join(re.search(r'extern "C" __attribute__\(\(naked,used\)\) void tglSonomaPmap'+kind+r'Bridge\(\) \{.*?\n\}',s,re.S)[0] for kind in ['New','Update']) if a.kernel: - d=a.kernel.read_bytes() - assert hashlib.sha256(d).hexdigest()=='095754b19dd533699fa9370d68eb8f31a2485060af57ce60644eafa006b455ab' - segments=[];pos=32 - for _ in range(struct.unpack_from('