Skip to content

[WIP] [CI] test Windows inference with CUDA 13.3#79381

Open
gouzil wants to merge 24 commits into
PaddlePaddle:developfrom
gouzil:codex/win-infer-test-vs2022
Open

[WIP] [CI] test Windows inference with CUDA 13.3#79381
gouzil wants to merge 24 commits into
PaddlePaddle:developfrom
gouzil:codex/win-infer-test-vs2022

Conversation

@gouzil

@gouzil gouzil commented Jun 26, 2026

Copy link
Copy Markdown
Member

PR Category

Environment Adaptation

PR Types

Improvements

Description

本 PR 调整 Windows-Inference CI,用于在 win-infer-test runner 上验证 VS2022 + CUDA 13.3 的 Windows inference 构建与测试。

主要改动:

  • Windows-Inference runner group 从 win-infer 切换为 win-infer-test
  • Windows-Inference 使用 VS2022 vcvars64.bat
  • Windows-Inference 的 CUDA_TOOLKIT_ROOT_DIR 从 CUDA 11.7 切换到 CUDA 13.3 默认安装路径。
  • Windows-Inference 将 CUDA 13.3 的 bin\x64 加入 PATH,并让 dynload 额外搜索该目录,用于加载 cublas64_13.dll 等运行时 DLL。
  • Windows-Inference 暂时关闭 WITH_TENSORRT,避免当前 win-infer-test runner 缺失 TensorRT 安装目录导致 CMake 阶段失败。
  • extern_warprnnt 作为独立 CMake 外部工程时显式继承顶层 CUDA_TOOLKIT_ROOT_DIR,避免 legacy FindCUDA 在 runner 环境中回落到 CUDA 11.7。
  • 新增 phi::dtype::is_floating_point,并让 float16_test / bfloat16_test 以及 CPU isfinite 路径使用 Paddle 自己的 dtype trait。VS2022 / MSVC STL 的 std::is_floating_point_v 只按标准内建浮点类型计算,用户对 std::is_floating_point<phi::dtype::float16> 的特化不会影响变量模板路径,因此在 C++20 + MSVC 19.44 下会得到 false;这里避免继续依赖标准库 type traits 的用户特化。

参考链接:

关联拆分 PR:

  • [Windows] export operator kernel registry #79419:将 OperatorWithKernel::AllOpKernels() 从 header inline 改为 operator.cc 中的 PADDLE_API 导出函数,确保 Windows DLL / EXE 共用同一份 Fluid op kernel registry,避免 operator_test 中测试注册的 kernel 在 ChooseKernel() 查询时不可见。
  • [Profiler] Export common host event gather #79426:将 host trace level 设置和 common host event gather 路径改为 paddle/phi/core/platform/profiler.cc 中的 PADDLE_API 导出函数,确保 Windows shared-DLL 下 profiler 设置侧、记录侧和收集侧使用同一份 HostTraceLevelHostEventRecorder<CommonEvent> / HostEventRecorder<CommonMemEvent> 实例;同时放开 Windows skip 列表里的 new_profiler_test,让该修复在 Windows CI 中被覆盖。

Windows-GPU、Windows-OPENBLAS、cmake/flags.cmake 保持不变。

是否引起精度变化

Copilot AI review requested due to automatic review settings June 26, 2026 17:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Windows inference GitHub Actions workflow to run on a different self-hosted runner group intended for VS2022 validation, and adjusts the Visual Studio toolchain initialization to use VS2022 Community’s vcvars64.bat instead of VS2019.

Changes:

  • Switch runs-on.group from win-infer to win-infer-test for the Windows inference job.
  • Update vcvars64_dir to point to VS2022 Community’s vcvars64.bat to avoid using the VS2019 toolchain after runner migration.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

代码改动本身未发现需要阻塞合入的问题。当前 Windows-Inference / Build and test 仍在运行,建议等该 job 在新的 win-infer-test runner 上完成后再合入。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

@paddle-bot paddle-bot Bot added the contributor External developers label Jun 26, 2026
PaddlePaddle-bot

This comment was marked as outdated.

@PaddlePaddle-bot

PaddlePaddle-bot commented Jun 26, 2026

Copy link
Copy Markdown

🤖 Paddle-CI-Agent | ci_status_monitor | 2026-07-17 01:35:41 UTC+08:00

CI报告基于以下代码生成(30分钟更新一次):
PR commit: efeb24d | Merge base: b6fb7ea (branch: develop)


1 Required任务 : 38/48 通过

总执行(rerun次数) 总任务 ✅ 通过 ❌ 失败 ⏳ 运行中 ⏸️ 等待中 跳过
90(8) 82 71 6 0 0 4
任务 错误类型 置信度 日志
DeepMD-Kit-Test / DeepMD-Kit Integration Test 环境问题 Job
Coverage build 环境问题 Job
Windows-Inference / Build and test PR问题 Job
Check approval 需要 Approval Job
Coverage test 未知 Job

2 失败详情

🔴 DeepMD-Kit-Test / DeepMD-Kit Integration Test — 环境问题(置信度: 高)

分析器: 通用分析(fallback)

失败用例:

用例 错误摘要
Install PaddlePaddle 下载 PR wheel 时网络不可达

关键日志:

Failed to establish a new connection: [Errno 101] Network is unreachable
Max retries exceeded with url: /PR/build/79381/efeb24df.../paddlepaddle_gpu-0.0.0-cp310-cp310-linux_x86_64.whl
ERROR: Could not install packages due to an OSError
  • 根因摘要: bcebos wheel 下载网络失败
    失败发生在 Install PaddlePaddle,pip 多次访问 paddle-github-action.bj.bcebos.com 下载 PR wheel 均返回网络不可达,DeepMD 依赖安装和 pytest 均未执行。

修复建议:

  1. 环境问题,请 rerun。

关联变更: 未发现与本 PR 改动文件相关。

🔴 Coverage build — 环境问题(置信度: 高)

分析器: 通用分析(fallback)

失败用例:

用例 错误摘要
Coverage build self-hosted runner 与 GitHub 失联

关键日志:

The self-hosted runner lost communication with the server.
Verify the machine is running and has a healthy network connection.
Build step remained in_progress; no compiler error was reported.
  • 根因摘要: runner 通信中断
    深度日志停在 FlashAttention CUDA 编译输出,workflow 中 Build 步骤仍为 in_progress,没有出现 CMake/编译失败结论。GitHub 返回的失败原因是 self-hosted runner lost communication,属于基础设施或网络异常。

修复建议:

  1. 环境问题,请 rerun。

关联变更: 未发现与本 PR 改动文件相关。

🔴 Windows-Inference / Build and test — PR问题(置信度: 高)

分析器: 通用分析(fallback)

失败用例:

用例 错误摘要
test_svd_op Windows CUDA 13.3 下连续多轮 80s 超时

关键日志:

10/10 Test #2044: test_svd_op ......................***Timeout  80.00 sec
This is the second time to re-run
1/1 Test #2044: test_svd_op ......................***Timeout  80.03 sec
This is the third time to re-run
The following tests FAILED:
        2044 - test_svd_op (Timeout)
Running unit tests failed, will exit
  • 根因摘要: CUDA13.3 skip-list 未覆盖 test_svd_op
    PR 新增 disable_wingpu_cuda133_test 后,在 CUDA 13.3 分支把 disable_wingpu_test 覆盖为新列表;该列表只跳过 FA/MEA 等用例,没有包含 CUDA12 列表中已有的 test_svd_op。因此 Windows-Inference 切到 CUDA 13.3 后执行了 test_svd_op,该用例在新环境中连续超时。

修复建议:

  1. tools/windows/run_unittests.shdisable_wingpu_cuda133_test 中加入 ^test_svd_op$,或改为在 CUDA 13.3 分支追加已有 Windows GPU/CUDA12 禁用列表,避免覆盖掉既有不兼容用例。

关联变更: tools/windows/run_unittests.sh 新增 CUDA 13.3 skip-list 和分支逻辑。

🔴 Check approval — 需要 Approval(置信度: 高)

该 Job 需要人工 Approval,完成审批后 CI 才会继续执行。

修复建议:

  1. 请通过人工审批。
🔴 Coverage test — 未知(置信度: 中)

分析器: 通用分析(fallback)

失败用例:

用例 错误摘要
test_enable_cinn_kernel_cache_2 CINN 生成 fatbin 时发现目标文件已存在

关键日志:

25/135 Test #268: test_enable_cinn_kernel_cache_2 ...........***Failed 6.87 sec
F0716 04:54:51.663448 compiler.cc:980] Internal error: Object file already exists.
This indicates a logic error in hash or kernel naming.
File: /tmp/cinn//0/2589847229578688145/cinn_cuda_kernel.fatbin
  • 根因摘要: CINN kernel cache fatbin 冲突
    失败来自 test_enable_cinn_kernel_cache_2,触发 paddle/cinn/backends/compiler.cc 中的 fatbin 已存在保护。PR 改动文件未涉及 CINN kernel cache 或该测试,当前只能判定为与 PR 直接关联不足的已有/偶发问题。

修复建议:

  1. 先 rerun 验证是否复现;若稳定复现,需要 CINN 相关同学检查 kernel cache hash/命名和 test_enable_cinn_kernel_cache_2test_enable_cinn_kernel_cache 的依赖清理逻辑。

关联变更: 未分析到本 PR 改动关联;相关位置为 test/ir/pir/cinn/test_enable_cinn_kernel_cache_2.pytest/ir/pir/cinn/CMakeLists.txtpaddle/cinn/backends/compiler.cc

@gouzil gouzil changed the title [WIP] [CI] test Windows inference with VS2022 runner [WIP] [CI] enable Windows C++20 toolchain Jun 27, 2026
@gouzil
gouzil force-pushed the codex/win-infer-test-vs2022 branch from 21005eb to 72bc2f3 Compare June 27, 2026 03:17
@gouzil gouzil changed the title [WIP] [CI] enable Windows C++20 toolchain [WIP] [CI] use VS2022 for Windows CI Jun 27, 2026
PaddlePaddle-bot

This comment was marked as outdated.

PaddlePaddle-bot

This comment was marked as outdated.

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查新的 VS2022 Windows CI 配置。当前有需要先修复的阻塞问题:Windows-GPUWindows-OPENBLAS 已经因为新增的 VS2022 vcvars64.bat 固定路径在 runner 上不存在而失败,具体证据和建议见行内评论。

此前我只建议等待 Windows-Inference job 完成;这次新增的失败覆盖了更广的 Windows job,需要先让相关 runner 路径/镜像与 workflow 配置一致后再继续验证。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

Comment thread .github/workflows/_Windows-GPU.yml Outdated
PRECISION_TEST: "OFF"
PYTHON_ROOT: C:\Python310
vcvars64_dir: 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvars64.bat'
vcvars64_dir: 'C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1
这里切到 VS2022 Community 固定路径后,当前 win-gpu runner 上该文件不存在,已经导致 Windows-GPU / Build and test 在 Build paddle 阶段失败:日志打印 C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat 后立即出现 The system cannot find the path specified.,随后 CMake 报 No CMAKE_CXX_COMPILER could be foundhttps://github.com/PaddlePaddle/Paddle/actions/runs/28277012818/job/83785864387)。请先把 win-gpu runner 镜像更新到包含该路径的 VS2022,或改成该 runner 实际存在的 VS2022 vcvars64.bat 路径;否则该必跑 Windows job 无法进入编译。

修复形态例如:

vcvars64_dir: '<win-gpu runner 上实际存在的 VS2022 vcvars64.bat 绝对路径>'

Comment thread .github/workflows/_Windows-OPENBLAS.yml Outdated
WITH_TESTING: "ON"
PRECISION_TEST: "OFF"
PYTHON_ROOT: C:\Python310
vcvars64_dir: 'C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1
新增的 VS2022 Community 路径在当前 win-openblas runner 上也不存在,已经导致 Windows-OPENBLAS / Build and test 在 Build paddle 阶段失败:日志打印 C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat 后出现 The system cannot find the path specified.,随后 CMake 报 No CMAKE_CXX_COMPILER could be foundhttps://github.com/PaddlePaddle/Paddle/actions/runs/28277012818/job/83785863898)。请先更新 win-openblas runner 镜像,或把这里配置为该 runner 实际存在的 VS2022 vcvars64.bat 路径;否则该 Windows OPENBLAS job 会稳定失败。

修复形态例如:

vcvars64_dir: '<win-openblas runner 上实际存在的 VS2022 vcvars64.bat 绝对路径>'

@gouzil
gouzil force-pushed the codex/win-infer-test-vs2022 branch from 72bc2f3 to 8ea483e Compare June 30, 2026 03:36
@gouzil gouzil changed the title [WIP] [CI] use VS2022 for Windows CI [WIP] [CI] test Windows inference with CUDA 13.3 Jun 30, 2026
PaddlePaddle-bot

This comment was marked as outdated.

@gouzil
gouzil force-pushed the codex/win-infer-test-vs2022 branch from 8ea483e to 35c24e2 Compare June 30, 2026 05:41
PaddlePaddle-bot

This comment was marked as outdated.

@gouzil
gouzil force-pushed the codex/win-infer-test-vs2022 branch from 35c24e2 to 49c5fbe Compare June 30, 2026 05:48
PaddlePaddle-bot

This comment was marked as outdated.

@gouzil
gouzil force-pushed the codex/win-infer-test-vs2022 branch from 49c5fbe to e4f76f8 Compare June 30, 2026 09:03
PaddlePaddle-bot

This comment was marked as outdated.

@gouzil
gouzil force-pushed the codex/win-infer-test-vs2022 branch from e4f76f8 to 023bf0b Compare June 30, 2026 09:50
PaddlePaddle-bot

This comment was marked as outdated.

PaddlePaddle-bot

This comment was marked as outdated.

PaddlePaddle-bot

This comment was marked as outdated.

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查最新提交。此前 CUDA 13.3 skip 过滤逻辑的问题仍由原行级线程覆盖;本轮新增的诊断逻辑还会改变 Step4 的 errexit 状态,可能破坏 Windows UT 的失败收集和重试流程,细节已放在新的行级评论中。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

Comment thread tools/windows/run_unittests.sh Outdated

echo "End diagnose Windows loader state for test_kernel_factory"
echo "============================================================"
set -e

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1

这个诊断函数是在 Step4 已经 set +e 之后由 run_unittest_gpu 调用的;Step4 依赖关闭 errexit,让每个 ctest | tee 失败后继续执行,最后再通过 collect_failed_tests/retry 汇总失败。这里返回前改成 set -e 后,后续第一个失败的 wait 就会直接终止脚本,导致剩余分组、失败收集和重试逻辑都不再运行。请保持 Step4 的失败收集模式,或保存/恢复调用方原来的 errexit 状态。

Suggested change
set -e
set +e

PaddlePaddle-bot

This comment was marked as outdated.

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查最新提交。此前关于诊断函数改变 Step4 errexit 状态的问题已随诊断逻辑删除而解决,本轮没有新增行级评论。

CUDA 13.3 skip 过滤仍未合并 disable_win_inference_test,该阻塞点继续以原行级线程为准。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

PaddlePaddle-bot

This comment was marked as outdated.

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查 PR 描述更新和当前 CI 结果。Windows-Inference / Build and test 已在当前提交上通过;此前 CUDA 13.3 skip 过滤线程中的阻塞判断已根据最新日志撤回,细节已补充在原线程中,本轮没有新增行级评论。

当前仍有 Coverage test、Check approval 和部分 clone 相关 checks 未通过,建议按 CI 面板继续处理。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

…test-vs2022

# Conflicts:
#	paddle/fluid/framework/operator.cc
#	paddle/fluid/framework/operator.h
PaddlePaddle-bot

This comment was marked as outdated.

@paddle-bot

paddle-bot Bot commented Jul 14, 2026

Copy link
Copy Markdown

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

…test-vs2022

# Conflicts:
#	paddle/phi/core/platform/profiler.cc
PaddlePaddle-bot

This comment was marked as outdated.

@gouzil

gouzil commented Jul 16, 2026

Copy link
Copy Markdown
Member Author

@ShigureNyako 来 review 一下

PaddlePaddle-bot

This comment was marked as outdated.

@ShigureNyako ShigureNyako left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

请求修改。

我按当前 head 2c228e9c 完整复核了 14 个变更文件。当前仍有三个阻塞点:

  1. GetDsoHandleFromSearchPath 的 extra-path 循环在 bin 已加载成功后仍继续查找 bin/x64,后一次失败会把有效 handle 覆盖为 nullptr;现有行级讨论仍适用于当前代码:discussion_r3499296433
  2. CUDA 13 的非 pip Windows 路径只补了 cuSPARSE 12.x fallback,cuSOLVER/cuFFT 仍缺少 cusolver64_12.dll / cufft64_12.dll;现有行级讨论仍未修复:discussion_r3523269908
  3. 本轮新增的行级评论:CUDA 13.3 skip 列表重新禁用了 new_profiler_test,与已合入的 PaddlePaddle/Paddle#79426 及本 PR 描述中的覆盖目标相冲突。

CI 方面,提交本 review 时当前 head 的 Windows-Inference / Build and test 及 Coverage 等实质检查仍在运行;已完成的两个失败项 Check approval / Require review-bot approval 属于审批门禁。上一 head 422e513 的 Windows-Inference 曾通过,但 Coverage test 因 diff coverage 57.1% 失败,当前 head 尚无完整的新一轮结果。

因此当前证据不足以 approve;请先修复上述代码/覆盖缺口并完成本轮 Windows-Inference 与 Coverage 验证。

Comment thread tools/windows/run_unittests.sh Outdated
disable_wingpu_cuda133_test="^test_convert_mea_2_fa_pass$|\
^test_sequence_pool$|\
^test_cudnn_bn_add_relu$|\
^new_profiler_test$|\

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里会把刚合入的 profiler 修复重新屏蔽。PR 描述关联的 PaddlePaddle/Paddle#79426 已于 7 月 14 日合入,并明确从 Windows skip 列表移除 new_profiler_test,以覆盖 Windows shared-DLL 下的 recorder 单例修复;当前 merge base b6fb7ea 也已不再禁用该测试。但 CUDA 13.3 路径会在本文件 L754 选用这个列表,因此本 PR 的目标 Windows-Inference job 实际不会运行 new_profiler_test

请从 CUDA 13.3 skip 中移除这一项;若它在 CUDA 13.3 上仍有独立失败,请附当前 head 2c228e9c 的失败日志和根因后再保留。

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查最新提交。cuSOLVER/cuFFT 的 CUDA 13 Windows fallback、extra path 查找成功后被后续路径覆盖的问题,以及 CUDA 13.3 skip 列表里的 new_profiler_test 已按当前代码修复;此前由 risemeup1111 提出的阻塞点要么已由后续提交解决,要么已在原线程撤回。

当前未发现需要阻塞合入的代码问题,本轮没有新增行级评论。Windows 相关和 Coverage/clone 任务仍有运行中或排队项,合入前请以最新 CI 面板为准。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

PaddlePaddle-bot

This comment was marked as outdated.

@PaddlePaddle-bot PaddlePaddle-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Paddle-CI-Agent | pr_review | 2026-07-16 11:52:20

📋 Review 摘要

PR 概述:将 Windows-Inference CI 切到 win-infer-test/VS2022/CUDA 13.3,并配套调整 CUDA DLL 搜索、warprnnt CUDA root、dtype trait 与相关测试。
变更范围:Windows CI/build、dynload、Phi dtype trait/isfinite、C++ 测试。
影响面 Tag[Environment Adaptation] [Inference] [Execute Infrastructure]

问题

未发现新的阻塞性问题。PR 规范问题在下面章节报,不在这里重复。

历史 Findings 修复情况

Finding 问题 状态
F1 CUDA 13.3 分支覆盖 disable_wingpu_test,导致原有 Windows-Inference 禁用列表不再生效 ⚠️ 仍存在

📝 PR 规范检查

符合规范。

总体评价

本轮基于完整 PR diff 回溯了 Windows CI/CMake、dynload、dtype trait/isfinite 和 C++ test 变更,未发现新的确定性问题。历史 F1 仍需修复:CUDA 13.3 禁用列表应与 Windows-Inference 既有禁用列表合并,而不是替换。未运行构建或单测。

@ShigureNyako ShigureNyako left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复核当前 head efeb24df 相对上轮 2c228e9c 的完整增量和当前 14 文件 diff。上轮三项代码阻塞均已修复:

  • GetDsoHandleFromSearchPath 现在会在 extra path 加载成功后立即 break,不会再被后续 bin/x64 miss 覆盖。
  • Windows fallback 列表已补入 cusolver64_12.dllcufft64_12.dll,并统一保留 cuSPARSE 12.x fallback。
  • CUDA 13.3 skip 列表已移除 new_profiler_test;当前脚本中已无该测试名。

本轮未发现新的代码阻塞点,但暂不 approve:

  • 当前 Windows-Inference / Build and test 仍停留在 Test paddle,Coverage 主流程也仍在构建。
  • CI-H / Coverage build 已失败;日志在 FlashAttention ptxas 构建约 49% 处直接中断,job 元数据仍把 Build 标为 in-progress、cleanup/upload 标为 pending,没有编译报错,当前证据更符合 runner/基础设施中断,仍需成功重跑闭环。
  • Check approval 明确还有 3 组审批要求:cc_testrisemeup1/zhangbo9674PADDLE_APIwanghuancoder/SigureMo/sunzhongkai588paddle/phiwanghuancoder/zrr1999/DanielSun11
  • Require review-bot approval 的失败发生在 03:33:49Z,早于 risemeup1111 对当前 head 的 approve(03:34:19Z),尚未刷新为成功。

结论:代码 review 阻塞已清除;交付状态仍为等待当前 Windows-Inference/Coverage 成功及 required approvals 完成,当前不可合入。

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查最新提交。test_svd_op 已按当前 Windows-Inference 的超时结果加入 CUDA 13.3 skip 列表,前一轮 CUDA 13.3 失败点已覆盖;未发现新的阻塞问题。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

@risemeup1111 risemeup1111 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已复查最新提交。此前由 risemeup1111 提出的阻塞点维持上一轮结论:已由后续提交修复或已在原线程撤回,本轮没有重复打开旧讨论。

这次新增的 dynamic loader 测试里发现一个新的 GCC 8.x 构建阻塞,细节已放在行级评论中;请修复并推送新的 commit 后再继续复查。当前还有部分 CI/审批检查未完成或失败,合入前也请以最新检查面板为准。

Powered by Nyanpasu with gpt-5.5 xhigh, please check the suggestions carefully.

Comment on lines +1 to +6
if(NOT WIN32)
paddle_test(dynamic_loader_test SRCS dynamic_loader_test.cc)
target_compile_definitions(
dynamic_loader_test
PRIVATE DYNAMIC_LOADER_TEST_DSO_PATH="$<TARGET_FILE:common>")
endif()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1
处理要求:请针对该评论修复并提交新的 commit。

这个新增测试使用了 <filesystem>,但 non-Windows 下没有为 GCC 8.x 链接 stdc++fs。当前仓库仍允许 GCC 8.2(cmake/flags.cmakeGCC >= 8.2 required),CI/docker 里也保留 gcc82 镜像;仓库已有 std::filesystem 相关目标会显式链接 stdc++fs。在 GCC 8.2/C++17 下这里会出现 std::filesystem 符号未定义,导致 dynamic_loader_test 链接失败。请在 GNU<9 时给该目标补上 stdc++fs,或改成不依赖 std::filesystem 的实现。

Suggested change
if(NOT WIN32)
paddle_test(dynamic_loader_test SRCS dynamic_loader_test.cc)
target_compile_definitions(
dynamic_loader_test
PRIVATE DYNAMIC_LOADER_TEST_DSO_PATH="$<TARGET_FILE:common>")
endif()
if(NOT WIN32)
paddle_test(dynamic_loader_test SRCS dynamic_loader_test.cc)
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU" AND CMAKE_CXX_COMPILER_VERSION
VERSION_LESS 9)
target_link_libraries(dynamic_loader_test stdc++fs)
endif()
target_compile_definitions(
dynamic_loader_test
PRIVATE DYNAMIC_LOADER_TEST_DSO_PATH="$<TARGET_FILE:common>")
endif()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

补充当前 head 191a58d7 的终态证据:Linux-DCU / Build 使用 GNU 8.2.0/C++17;新增的空测试 DSO 已成功构建,但链接 dynamic_loader_test 时仍对 std::filesystem::temp_directory_pathcreate_symlinkremove_all 等符号报 undefined reference,随后 collect2 退出。

因此独立 DSO 已解决另一线程的重复 flags 初始化问题,但没有改变本线程的 GNU 8 链接要求;这里仍需补充 stdc++fs 或移除该依赖。

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

收到,这条 Linux-DCU 日志把本线程的判断补实了:dynamic_loader_test 在 GNU 8.2 / C++17 下仍然需要 stdc++fs,当前头没有改动这个链接点,所以链接失败是预期内的。

如果后续改成不依赖 <filesystem>,或者在 GNU < 9 时给这个目标补上 stdc++fs,我再继续复查。

// See the License for the specific language governing permissions and
// limitations under the License.

#include "paddle/phi/backends/dynload/dynamic_loader.cc" // NOLINT(build/include)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级:P1
处理要求:请修复并提交新的 commit。

这里直接 #includedynamic_loader.cc,但 paddle_test 还会链接 Paddle/phi 中已经编译过的同一实现,因此测试进程会重复执行该文件里的 flags 注册。

当前 head 的 Linux-XPU / Test 已给出确定性证据:dynamic_loader_test 首次执行及 3 次重跑都在进入测试体前失败,错误为:

paddle flags error: flag multiple definition, flag "xpti_dir" was defined both in
paddle/phi/backends/dynload/dynamic_loader.cc and
paddle/phi/backends/dynload/dynamic_loader.cc

请避免在测试 translation unit 中直接包含实现 .cc;可将待测逻辑抽到可复用的内部接口,或让测试目标只编译/链接一份实现并避免重复注册。该失败与本次新增测试直接相关,重跑不能解决。

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里直接导入 .cc 是为了测试 GetDsoHandleFromSearchPath

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

理解,这个 helper 当前是 translation unit 内的 static inline,测试需要一个可访问的 seam。7c600408 在包含实现前移除了 PADDLE_WITH_FLAGCXPADDLE_WITH_HIPPADDLE_WITH_XPU;这些宏覆盖了该文件现有的 backend-specific flags 定义,同时在 NOT WIN32 下不会改变待测 GetDsoHandleFromSearchPath 的实现。因此从源码看,本线程的重复注册问题已按等价方案处理;后续以当前 head 的 XPU/DCU 结果确认。

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

补充当前 head 的实测结果:宏隔离仍不足以解决直接包含实现文件带来的重复注册。Linux-XPU / Test 中,xpti_dir 冲突消失了,但 dynamic_loader_test 首次执行及 3 次重跑仍全部失败,新的确定性错误是:

paddle flags error: flag multiple definition, flag "inner_op_parallelism" was defined both in
paddle/paddle/common/flags.cc and paddle/paddle/common/flags.cc

这说明问题不只来自本文件里受三个 backend 宏保护的 flags;在 paddle_test 的现有链接关系下,直接编入整份 dynamic_loader.cc 仍会让测试进程带入重复的注册/实现依赖。请改为只暴露并测试目标 helper,或让测试目标确保相关实现与 flags 仅链接一份。该用例连续重跑保持同一指纹,不能按偶发失败处理。

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

补充 191a58d7 的结果:该提交不再把 common 作为测试中二次 dlopen 的目标,而是改用无 Paddle 静态初始化器的独立空 DSO;这样既保留了 missing/success/missing 路径序列对 GetDsoHandleFromSearchPath 的真实覆盖,也避免通过别名再次加载 common 触发 flags 初始化。

当前 head 的 Linux-XPU / Test 已完整通过,先前的 xpti_dir / inner_op_parallelism 重复注册均未再出现。因此本线程的直接包含/重复注册阻塞可以关闭;GNU 8.2 的 stdc++fs 链接问题仍由另一线程单独跟踪。

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ShigureNyako 笑死了,你从哪学的,怎么完全模仿 nyanpasu 的回复模板啊

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

确实是我这边用了固定的 review 表达,模板味太重了 😂。技术结论不是照着模板下的,前后判断分别来自 XPU 的重复 flags 日志和后续通过结果。之后我会直接写“结论 + 证据”,少用这种套话。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants