Skip to content

compilers/pgi: generate header dependency args for NVHPC/PGI - #16161

Closed
stoiver wants to merge 1 commit into
mesonbuild:masterfrom
stoiver:nvhpc-depfile-deps
Closed

compilers/pgi: generate header dependency args for NVHPC/PGI#16161
stoiver wants to merge 1 commit into
mesonbuild:masterfrom
stoiver:nvhpc-depfile-deps

Conversation

@stoiver

@stoiver stoiver commented Aug 27, 2026

Copy link
Copy Markdown

Fixes #16160.

PGICompiler did not implement get_dependency_gen_args(), so it inherited the base
class stub returning []. Meson still emitted deps = gcc and a depfile into the ninja
rule, but the compile line never received -MD/-MF, so no depfile was ever written:
ninja recorded zero dependencies for every object and a changed header never triggered
a rebuild.

Same defect as #11969 (CUDA), fixed in #12665, in a different compiler class.

This is not just a stale-build annoyance. An incremental build can silently produce a
binary whose translation units disagree about struct layout — no compile error, no
warning. In the project where I hit this, a member in a shared header shifted, only some
objects were rebuilt, and two struct members aliased: the program printed a nonsense value
and hung.

-MQ must not be used here

nvc accepts -MQ but ignores its argument, writing the literal string mttarget as
the depfile target, which ninja then silently discards. That failure looks identical to
having no fix at all, so it is worth stating explicitly. Verified on nvc 26.3:

$ nvc -MD -MQ 'ZZZ_unique_target' -MF q.d -c a.c -o q.o && head -1 q.d
mttarget : a.c \

$ nvc -MD -MT 'ZZZ_unique_target' -MF t.d -c a.c -o q.o && head -1 t.d
ZZZ_unique_target : a.c \

So -MT is used, as for nvcc.

Escaping

-MT does not Make-escape the target, so this needs the escaping #12665 added for CUDA.
That was gated on langname == 'cuda' / compiler.get_language() == 'cuda'. Rather than
add a second special case, this adds a compiler predicate:

def needs_escaped_depfile_target(self) -> bool:
    """Whether the depfile target must be Make-escaped by the backend."""
    return False

returning True for CUDA and PGI, with both ninjabackend.py gates switched to it. The
ninja variable CUDA_ESCAPED_TARGET is renamed ESCAPED_DEPFILE_TARGET since it is no
longer CUDA-specific. No behaviour change for CUDA.

Tests

Two tests in unittests/internaltests.py:

  • test_pgi_dependency_gen_args — the flags themselves.
  • test_depfile_target_escaping_is_declared — ties get_dependency_gen_args to
    needs_escaped_depfile_target as an invariant across GCC, Clang and NVHPC: emitting
    -MT requires True, emitting -MQ requires False. This is the failure a future
    compiler addition is most likely to hit, and it only shows up for object paths
    containing a space, $ or #.

Both fail on unpatched master ([] != ['-MD', '-MT', 'foo.o', '-MF', 'foo.o.d']) and pass
with the change. Full internaltests.py: 78 passed, 129 subtests.

Real-world verification

On an NVHPC project (~34 objects, C plus Cython-generated C, -mp=gpu):

$ grep -A2 '^rule c_COMPILER' build/build.ninja
 command = .../nvc $ARGS -MD -MT $ESCAPED_DEPFILE_TARGET -MF $DEPFILE -o $out -c $in

$ ninja -C build -t deps | grep -cE '^\S+\.o: #deps [1-9]'
34                                  # was 0
$ touch src/sw_domain.h && ninja -C build -n | grep -c 'Compiling C object'
13                                  # was 0

The 13 are exactly the objects including that header, across both extension modules. That
project's own test suites pass unchanged.

Open questions

  1. Version gating. Add support for dumping dependent headers in nvcc #12665 gated CUDA on >= 10.2. I only have nvc 26.3 and do not know
    the earliest NVHPC/PGI release supporting -MD/-MT. Happy to add a bound if someone
    with older toolchains can name one.
  2. Classic PGI vs NVHPC. PGICompiler backs both PGICCompiler (pgcc) and the
    NvidiaHPC_* classes. I verified NVHPC only. If legacy pgcc differs, the override
    may belong on the NVHPC classes instead of the shared mixin.
  3. Fortran. The mixin is shared with the Fortran compilers, which I have not tested.

@bonzini

bonzini commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Unreadable LLM output. Remove the test and the useless comments and there will be a chance of the patch being reviewed.

@bonzini

bonzini commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Example

        # NVHPC/PGI previously inherited the base-class stub, which returns [].
        # Meson still wrote "deps = gcc" and a depfile into the ninja rule, but
        # the compile line never got -MD/-MF, so no depfile was produced, ninja
        # recorded zero dependencies for every object, and a changed header
        # never triggered a rebuild -- silently producing binaries whose
        # translation units disagreed about struct layout.
        #

Why would I care about past wrong behavior? The place for that is the commit messages, not comments.

@bonzini

bonzini commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

The mixin is shared with the Fortran compilers, which I have not tested

Please do it then.

The change is not awful—it's obviously useful and doesn't have blatant algorithmic issues—so it's a good starting point. What's wrong is that you're asking others to complete the work for you when it wasn't even you who did the work in the first place, since you had a machine doing it for you.

Meson is run by volunteers. Asking for anything more than this cursory check will require you to put in some effort: make our job easier as much as the LLM made it easier for you.

PGICompiler did not implement get_dependency_gen_args(), so it inherited the
base class stub returning []. Meson still emitted 'deps = gcc' and a depfile
into the ninja rule, but the compile line never received -MD/-MF, so no depfile
was written: ninja recorded zero dependencies for every object and a changed
header never triggered a rebuild. Incremental builds could then silently
produce binaries whose translation units disagreed about struct layout.

Same defect as mesonbuild#11969 (CUDA), fixed in mesonbuild#12665, in a different compiler class.

C and C++ only. nvfortran accepts -MD/-MT/-MF and exits 0 but writes no
depfile; NVHPC's help describes -M as 'Print dependencies to stdout in C++'.
Putting the override on the shared PGICompiler mixin would therefore have left
Fortran builds emitting the flags with ninja still waiting for a depfile that
never arrives. Hence a separate PGIDependencyMixin, used by the four C/C++
classes and not by the Fortran ones.

-MT rather than -MQ: nvc accepts -MQ but ignores its argument and writes the
literal string 'mttarget' as the depfile target, in both the -MQ x and -MQ=x
forms. Since -MT does not escape the target for Make, this also needs the
escaping added for CUDA in mesonbuild#12665, which was gated on langname == 'cuda'.
Rather than add a second special case, Compiler.needs_escaped_depfile_target()
now selects it, returning True for CUDA and for PGI C/C++. The ninja variable
CUDA_ESCAPED_TARGET is renamed ESCAPED_DEPFILE_TARGET as it is no longer
CUDA-specific. No behaviour change for CUDA.

Tested with nvc, nvc++ and nvfortran 26.3, and on an NVHPC project of 34
objects: all 34 gain recorded dependencies where they previously had none, and
touching a shared header rebuilds the 13 objects that include it.

Fixes mesonbuild#16160
@stoiver
stoiver force-pushed the nvhpc-depfile-deps branch from 29c1e71 to 8a245e5 Compare August 28, 2026 01:58
@stoiver

stoiver commented Aug 28, 2026

Copy link
Copy Markdown
Author

Thanks — the Fortran question was the right one, and it found a bug.

nvfortran accepts -MD -MT -MF and exits 0, but writes no depfile. NVHPC's own
help describes -M as "Print dependencies to stdout in C++": these are C/C++
driver flags that the Fortran driver takes and ignores. So putting the override
on the shared PGICompiler mixin, as the first version did, would have left
Fortran builds passing the flags and ninja still waiting for a depfile that never
arrives — the same silent failure, just moved.

v2 puts it on a separate PGIDependencyMixin used only by the C and C++
compilers, with the reason in the docstring. Verified by resolution:
PGICCompiler, NvidiaHPC_CCompiler, PGICPPCompiler and
NvidiaHPC_CPPCompiler return the flags; PGIFortranCompiler and
NvidiaHPC_FortranCompiler fall back to the base [].

Tests removed, and the comments cut to what a future reader needs — why -MT
rather than -MQ, and why Fortran is excluded. The history is in the commit
message.

Tested on nvc/nvc++/nvfortran 26.3 and on a real NVHPC project: all 34 objects
gain recorded dependencies where they previously had none, and touching a shared
header now rebuilds the 13 objects that include it instead of nothing. I have no
access to classic pgcc/pgc++; they share the mixin on the grounds that NVHPC
is the same compiler line renamed, which I mention so it is on the record rather
than assumed.

The one red CI job, Ubuntu Rolling (clang, clang++), fails identically on master
at the same base commit (ee6dfdc): the same eleven coverage, sanitizer and
env-cflags tests, none of them related to this change. The CUDA job, which is the
one that exercises the ninja variable rename, passes.

On your wider point — fair. The patch was AI-assisted. We should have answered the
Fortran question instead of listing it for someone else to pick up,
especially as it turned out to be important.

@bonzini bonzini closed this Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NVHPC/PGI builds generate no header dependencies (get_dependency_gen_args not implemented)

3 participants