Fix building the SDK and Samba from scratch on a clean NetBSD host

[artem-from-ua]

Building this project on a freshly installed NetBSD guest fails at five separate points before producing a binary. Each failure is fixed here in its own commit, and every fix is motivated by an error message the build actually printed — not by anticipation.

Verified end to end: a stock NetBSD 10.1/aarch64 image under QEMU, nothing preinstalled, no manual steps beyond a root password and network setup, through to a working earmv4 smbd.

What went wrong, in the order it happened

1. No git, and no pkgin to install it

_download_sdk.sh clones the NetBSD source tree, but a stock guest has no git, and no pkgin either to install one. The bootstrap uses pkg_add from the base system, deriving the repository URL with the formula NetBSD ships commented out in /root/.profile.

2. The host toolchain packages are missing

gmake, python27 and gcc7 are all absent from a stock guest, and the preflight check reported the shortfall without saying which package would supply it:

Missing required host tool: gmake
Missing required host tool: python2.7
Missing required host tool: /usr/pkg/gcc7/bin/gcc

These are now installed rather than merely reported. gcc7 is conditional: HOST_CC/HOST_CXX are overridable, and an arbitrary path says nothing about which package provides it, so gcc7 is installed only while one of them still holds the default it owns — compared exactly, so a lookalike such as /opt/usr/pkg/gcc7/bin/gcc is not mistaken for it.

3. The distribution phase cannot link

rescue/list requires mount_smbfs unconditionally and resolves libsmb.a against the primary objdir, which MKCOMPAT=yes leaves empty on a from-scratch build. We need only libc and headers to link one static earmv4 smbd, so MKCOMPAT defaults to no.

4. Python 3 and pkg-config are missing

Samba's waf needs Python 3; without pkg-config the GnuTLS configure step reports a misleading

Libnettle 3.6 was not found

when nettle is present and merely undiscoverable.

5. The NetBSD tree's GMP is too old for nettle

install_samba4x_target_gmp() adopted it because the file existed. nettle then probes for __gmpn_zero_p, which that GMP predates:

checking for __gmpn_zero_p in -lgmp... no
    Support for public key algorithms will be unavailable.
  Public key crypto: no

The build carried on and failed later with Libhogweed 3.6 was not found, which points nowhere near the cause. Now the symbol is checked up front and an unusable GMP is rejected, so the bundled one gets built instead.

Verification

A full run from a stock NetBSD 10.1/aarch64 image under QEMU with hvf acceleration, -smp 4, 4 GB RAM, on an Apple M1 Pro. Nothing preinstalled — the bare starting point was asserted rather than assumed:

pkgin absent

Every package the build needs was then installed by the build scripts themselves, each reporting warnings: 0, errors: 0:

Installing pkgin on the VM.      (via pkg_add — the bootstrap)
Installing git-base on the VM.
Installing gcc7 on the VM.
Installing gmake on the VM.
Installing python27 on the VM.

All four stages completed rc=0, ending in a working binary:

/root/tc-netbsd7/sbin/smbd.stripped
ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked,
for NetBSD 7.2, compiled for: earmv4, stripped

Before these fixes the same sequence could not reach a binary at all: it stopped at the first missing package, and each fix only exposed the next failure.

[jamesyc]

The image i used had pkgin, and I consider git etc to be prerequsites. Probably a good filter to have.

But 3 and 5 are valid issues; you can remove the rest. 3 looks good, 5 needs some changes.

Switching GMP should invalidate downstream nettle/GnuTLS artifacts. After replacing libgmp.a, build_samba4x_nettle() can still report already built and reuse them. The nettle stamp is also named system-gmp even when bundled GMP is selected. Please include the GMP provider/version in the dependency stamp or explicitly rebuild nettle and GnuTLS whenever the selected GMP changes.
Tests should cover compatible GMP, missing symbol, failed inspection, and stale downstream dependency stamps.

[artem-from-ua]

The image was the stock NetBSD 10.1/aarch64 gzimg:
https://cdn.netbsd.org/pub/NetBSD/NetBSD-10.1/evbarm-aarch64/binary/gzimg/arm64.img.gz

It boots straight into a working system, but it carries no pkgsrc at all — the
package database under /usr/pkg/pkgdb is empty, and pkgin itself is absent, so
the first step had to come from pkg_add in the base system. Of the tools the
build needs, the base system provides only make and gcc, and neither is the one
required: the scripts want GNU make and gcc7, while NetBSD 10.1 ships its own
BSD make and gcc 10.5. git, gmake, python2.7, python3, and pkg-config are not
present under any path.

That is what those three commits were reacting to, and they are reverted now.
Since your image had pkgin, it is a different starting point — where does it
come from? Writing a gzimg skips sysinst entirely, and
INSTALL.html
offers pkgin from the sysinst post-install menu rather than from every install
method, which is probably where the two starting points diverge.

The GMP stamp is fixed. Both nettle and GnuTLS stamps now carry the provider
and version of the GMP they were built against, so switching providers no
longer matches an existing stamp. The same applied to GMP itself: both
providers install to the same deps/lib/libgmp.a, which is all a bundled stamp
checks for, so adopting the target GMP now retires every bundled stamp. Tests
cover a usable target GMP, a missing __gmpn_zero_p, an nm that cannot be run,
and stale downstream stamps. Verified on a NetBSD 10.1 VM with an existing
build tree: the target GMP is rejected for the missing symbol, the bundled GMP
is selected, and nettle and GnuTLS rebuild instead of reporting "already built".

Separate question, not part of this PR: nettle, libtasn1, and GnuTLS are all
pinned to fetched versions; GMP is the only dependency taken from the NetBSD
tree, and it is the one that caused this. Its version is also assumed rather
than read — gmp.pc gets a hardcoded 6.1.0. Always using the bundled GMP would
drop install_samba4x_target_gmp, find_samba4x_gmp_header, the nm probe, and the
provider-id stamps. The costs are a mandatory network fetch, since the tree GMP
is the only dependency that needs none, and a tarball we do not checksum today.
Worth a separate PR, or do you prefer the tree GMP?