Generated source code not in the location expected by an extension_module.

[WarrenWeckesser]

I have a project called numtypesx with files organized like this:

numtypesx
├── meson.build
├── numtypesx
│   ├── __init__.py
│   └── tests
│       ├── __init__.py
│       ├── test_logfloat_scalars.py
│       └── test_logfloat_ufuncs.py
├── pyproject.toml
├── README.md
├── src
│   ├── logfloat_casts.c.src
│   ├── logfloat_casts.h
│   ├── logfloat_dtype.c.src
│   ├── logfloat_dtype.h
│   ├── logfloat_scalar.c.src
│   ├── logfloat_scalar.h
│   ├── logfloat_umath.c.src
│   ├── logfloat_umath.h
│   ├── meson.build
│   └── numtypesx_module.c
└── tools
    ├── conv_template.py
    └── make_version_file.py

The top-level meson.build file has some boilerplate code to find the include directory and libs for NumPy, and ends with subdir('src'). The file src/meson.build contains

gen = generator(py,
                output : '@BASENAME@',
                arguments : ['../tools/conv_template.py', '@INPUT@', '.'])

processed_src = gen.process('logfloat_dtype.c.src', 'logfloat_casts.c.src',
                            'logfloat_scalar.c.src', 'logfloat_umath.c.src')

headers = files('logfloat_casts.h', 'logfloat_dtype.h', 'logfloat_scalar.h', 'logfloat_umath.h')

py.extension_module(
    'logfloat',
    ['numtypesx_module.c', processed_src, headers],
    include_directories : [inc_np],
    dependencies : [np_dep, npymath_lib],
    link_args : ['-lm'],
    install : true,
    subdir : 'numtypesx'
)

The python script tools/conv_template.py accepts a filename such as foo.c.src and writes the file foo.c to the directory given as the last command line argument. I use a generator to process the .c.src files.

When I run meson setup build followed by ninja -v -j 1 -C build, I get this output:

$ ninja -v -j 1 -C build
ninja: Entering directory `build'
[1/11] /home/warren/py3.11.5/bin/python3.11 ../tools/make_version_file.py ../pyproject.toml _version.py
[2/11] /home/warren/py3.11.5/bin/python3.11 ../tools/conv_template.py ../src/logfloat_dtype.c.src .
[3/11] /home/warren/py3.11.5/bin/python3.11 ../tools/conv_template.py ../src/logfloat_casts.c.src .
[4/11] /home/warren/py3.11.5/bin/python3.11 ../tools/conv_template.py ../src/logfloat_scalar.c.src .
[5/11] /home/warren/py3.11.5/bin/python3.11 ../tools/conv_template.py ../src/logfloat_umath.c.src .
[6/11] cc -Isrc/logfloat.cpython-311-x86_64-linux-gnu.so.p -Isrc -I../src -I/home/warren/py3.11.5/lib/python3.11/site-packages/numpy/core/include -I/home/warren/py3.11.5/include/python3.11 -fvisibility=hidden -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c11 -O0 -g -fPIC -MD -MQ src/logfloat.cpython-311-x86_64-linux-gnu.so.p/meson-generated_logfloat_dtype.c.o -MF src/logfloat.cpython-311-x86_64-linux-gnu.so.p/meson-generated_logfloat_dtype.c.o.d -o src/logfloat.cpython-311-x86_64-linux-gnu.so.p/meson-generated_logfloat_dtype.c.o -c src/logfloat.cpython-311-x86_64-linux-gnu.so.p/logfloat_dtype.c
FAILED: src/logfloat.cpython-311-x86_64-linux-gnu.so.p/meson-generated_logfloat_dtype.c.o 
cc -Isrc/logfloat.cpython-311-x86_64-linux-gnu.so.p -Isrc -I../src -I/home/warren/py3.11.5/lib/python3.11/site-packages/numpy/core/include -I/home/warren/py3.11.5/include/python3.11 -fvisibility=hidden -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c11 -O0 -g -fPIC -MD -MQ src/logfloat.cpython-311-x86_64-linux-gnu.so.p/meson-generated_logfloat_dtype.c.o -MF src/logfloat.cpython-311-x86_64-linux-gnu.so.p/meson-generated_logfloat_dtype.c.o.d -o src/logfloat.cpython-311-x86_64-linux-gnu.so.p/meson-generated_logfloat_dtype.c.o -c src/logfloat.cpython-311-x86_64-linux-gnu.so.p/logfloat_dtype.c
cc1: fatal error: src/logfloat.cpython-311-x86_64-linux-gnu.so.p/logfloat_dtype.c: No such file or directory
compilation terminated.
ninja: build stopped: subcommand failed.

The .c.src files have been processed and written to .c files in the build/ directory. Apparently the extension_module is looking for those files in build/src/logfloat.cpython-311-x86_64-linux-gnu.so.p/. How can I modify the parameters of the various meson objects or the arguments to the conversion script tools/conv_template.py so that the extension_module finds the generated files?

[dnicolodi]

The python script tools/conv_template.py accepts a filename such as foo.c.src and writes the file foo.c to the directory given as the last command line argument.

This is not how generator are supposed to be used. See https://mesonbuild.com/Generating-sources.html#using-generator The output of the generator needs to be written in @BUILD_DIR@. The simplest thing is to pass both @INPUT@ and @OUTPUT@ as arguments to your script.

[WarrenWeckesser]

Got it, thanks!

[dnicolodi]

headers = files('logfloat_casts.h', 'logfloat_dtype.h', 'logfloat_scalar.h', 'logfloat_umath.h')

py.extension_module(
'logfloat',
['numtypesx_module.c', processed_src, headers],
include_directories : [inc_np],
dependencies : [np_dep, npymath_lib],
link_args : ['-lm'],
install : true,
subdir : 'numtypesx'
)

Passing the headers as source files is not necessary: Meson uses the compiler dependency tracking to keep track of this kind of dependencies between build artifact and source files.