Skip to content

coverage: run the daemons against a declared /etc/nvme #518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
martin-belanger merged 1 commit into from
Sep 2, 2026
Merged

coverage: run the daemons against a declared /etc/nvme #518

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 30 additions & 16 deletions TESTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,26 +199,40 @@ To run the coverage test, from the root of the `nvme-stas` git repo:
$ make coverage
```

This will start `stafd`, `stacd`, and the `nvmet` target. At the end, if all goes well, you should get an output similar to this:

```bash
Name Stmts Miss Cover
----------------------------------------
stacctl 53 0 100%
stacd 190 3 98%
stafctl 75 0 100%
stafd 246 21 91%
staslib/avahi.py 185 19 90%
staslib/defs.py 22 0 100%
staslib/stas.py 858 51 94%
staslib/version.py 31 0 100%
----------------------------------------
TOTAL 1660 94 94%
This will start `stafd`, `stacd`, and the `nvmet` target. The daemons do not read the machine's `/etc/nvme`: they run in a mount namespace where a directory the script writes (host NQN, host ID, and libnvme's exclusion lists) is bind-mounted over it, so the same phases connect the same controllers on any machine. The `nvme` commands the script invokes run outside that namespace and act on the live system. The script tries the bind mount once on a throw-away unit before it starts anything; where that does not work — a systemd too old for `BindPaths=`, or a machine that refuses mount namespacing — it says so and falls back to the real directory.

At the end, if all goes well, you should get an output similar to this:

```bash
Name Stmts Miss Cover
------------------------------------------
staslib/avahi.py 297 30 90%
staslib/conf.py 370 30 92%
staslib/service.py 375 29 92%
staslib/gutil.py 264 19 93%
staslib/stas.py 340 24 93%
staslib/udev.py 243 11 95%
staslib/ctrl.py 473 14 97%
staslib/timeparse.py 83 1 99%
stacctl 56 0 100%
stacd 49 0 100%
stafctl 79 0 100%
stafd 62 0 100%
staslib/__init__.py 2 0 100%
staslib/defs.py 29 0 100%
staslib/iputil.py 133 0 100%
staslib/log.py 25 0 100%
staslib/nbft.py 9 0 100%
staslib/singleton.py 21 0 100%
staslib/trid.py 55 0 100%
staslib/version.py 33 0 100%
------------------------------------------
TOTAL 2998 158 95%
```

Note that the Python coverage package has trouble tracking code executed in threads. And since nvme-stas uses threads, some of the code will not be accounted for (in other words, you'll never get 100% coverage).

Also note, that some of the code (e.g. explicit registration per TP8010) only gets executed when connected to a CDC (not a DDC). So, depending on your environment you will most likely get different coverage result. The above test was done on a system where mDNS discovery with a CDC was available, which provides more coverage than using the `nvmet` driver alone.
Also note, that some of the code (e.g. explicit registration per TP8010) only gets executed when connected to a CDC (not a DDC). So, depending on your environment you will most likely get different coverage result.

An HTML output is also available where you can click on each file and which lines of code got executed and which ones were missed. In your web browser, simply type `file:///[$STAS_DIR]/.build/coverage/index.html` (you must replace `[$STAS_DIR]` by the actual location of the nvme-stas repo where `make coverage` was run) . You should get something like this:

Expand Down
90 changes: 88 additions & 2 deletions coverage.sh.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@ stacd_conf_fname=$(mktemp $file)
file=/tmp/nvme-stas.conf.XXXXXX
stas_conf_fname=$(mktemp $file)

# Everything the daemons read from @ETC@/nvme is created here, and the units
# get it bind-mounted over the real directory. Without it the run depends on
# undeclared machine state: a stale exclusions.conf once changed what the run
# connected, for an unknown number of runs, with nothing in the script saying
# so. Results were therefore not comparable between machines, and a runner
# with no @ETC@/nvme at all was a third answer again.
ETC_NVME_DIR=$(mktemp -d /tmp/stas-etc-nvme.XXXXXX)

CYAN="[1;36m"
RED="[1;31m"
YELLOW="[1;33m"
Expand Down Expand Up @@ -56,6 +64,70 @@ systemctl-exists() {
[ $(systemctl list-unit-files "${unit}" | wc -l) -gt 3 ]
}

# Write the @ETC@/nvme the daemons will see. Nothing here is copied from the
# machine: the point is that a phase behaves the same way wherever the run
# happens.
etc_nvme_populate() {
log "Populate ${ETC_NVME_DIR}, which the daemons see as @ETC@/nvme"

# The system identity. A connectivity configuration naming its own host
# NQN is a persona and overrides both of these; every phase that names
# none gets exactly this host.
echo 'nqn.2014-08.org.nvmexpress:uuid:a1b2c3d4-0000-4000-8000-000000000001' > "${ETC_NVME_DIR}/hostnqn"
echo 'a1b2c3d4-0000-4000-8000-000000000001' > "${ETC_NVME_DIR}/hostid"

# libnvme's host-wide exclusion list, which stas re-reads on every
# connection attempt. "starfleet" is one of the three subsystems nvmet
# serves and is named nowhere else in this script, so excluding it here
# is what exercises the list against a live target: stacd must connect
# the other two and leave this one alone.
cat > "${ETC_NVME_DIR}/exclusions.conf" <<'EOF'
[exclusions]
exclusion = nqn=nqn.1988-11.com.dell:starfleet
EOF

# A named drop-in list. stas enumerates those separately from the main
# file, so one has to exist for that path to run at all. It matches
# nothing the target serves.
mkdir -p "${ETC_NVME_DIR}/exclusions.conf.d"
cat > "${ETC_NVME_DIR}/exclusions.conf.d/coverage.conf" <<'EOF'
[exclusions]
exclusion = transport=tcp;traddr=3.3.3.3
EOF

# Deliberately no stafd.conf, stacd.conf or nvme-stas.conf. Every daemon
# start below passes -f and -c, except the last stafd start, which takes
# neither on purpose: it must find nothing here and fall back to the
# built-in defaults.

chmod -R a+rX "${ETC_NVME_DIR}"

log_file_contents 0 "${ETC_NVME_DIR}/exclusions.conf"
log_file_contents 0 "${ETC_NVME_DIR}/exclusions.conf.d/coverage.conf"
}

# BindPaths= gives a unit its own mount namespace with our directory over
# @ETC@/nvme - visible to the daemons and to libnvme's C code, invisible to
# everything else on the machine. It covers processes started inside units
# only: the "nvme" invocations below run outside, and act on the live system
# with the machine's own identity.
#
# Rather than compare systemd versions, try the property once on a throw-away
# unit that reads a file only our directory has. That catches a systemd too
# old to know BindPaths= as well as a machine where mount namespacing is
# unavailable, which would otherwise fail every daemon start with
# 226/NAMESPACE and leave the run with nothing to measure.
etc_nvme_bind_supported() {
unit=stas-bind-probe-cov.service
sudo systemctl reset-failed "${unit}" >/dev/null 2>&1
sudo systemd-run --unit="${unit}" --wait --property=Type=oneshot \
--property=BindPaths="${ETC_NVME_DIR}:@ETC@/nvme" \
/bin/cat "@ETC@/nvme/exclusions.conf.d/coverage.conf" >/dev/null 2>&1
rc=$?
sudo systemctl reset-failed "${unit}" >/dev/null 2>&1
return ${rc}
}

sd_stop() {
app="$1"
unit="${app}"-cov.service
Expand Down Expand Up @@ -96,7 +168,9 @@ sd_start() {
sudo systemctl reset-failed "${unit}" >/dev/null 2>&1

log "Start ${app}"
sudo systemd-run --unit="${unit}" --working-directory=. --property=Type=dbus --property=BusName="${dbus}" --property="SyslogIdentifier=${app}" --setenv=PYTHONPATH=${PYTHON_PATH} --setenv=RUNTIME_DIRECTORY=${RUNTIME_DIRECTORY} coverage run --rcfile=.coveragerc ${cmd} >${SCRATCH_FILE} 2>&1
# ETC_NVME_PROP is deliberately unquoted: it is one word, or nothing at
# all on a systemd without BindPaths=.
sudo systemd-run --unit="${unit}" --working-directory=. ${ETC_NVME_PROP} --property=Type=dbus --property=BusName="${dbus}" --property="SyslogIdentifier=${app}" --setenv=PYTHONPATH=${PYTHON_PATH} --setenv=RUNTIME_DIRECTORY=${RUNTIME_DIRECTORY} coverage run --rcfile=.coveragerc ${cmd} >${SCRATCH_FILE} 2>&1
log_file_contents $? ${SCRATCH_FILE}
printf "\n"
sleep 1
Expand Down Expand Up @@ -213,10 +287,11 @@ postrun_cleanup() {
log_file_contents $? ${SCRATCH_FILE}
printf "\n"

log "Remove ${stafd_conf_fname}, ${stacd_conf_fname} and ${stas_conf_fname}"
log "Remove ${stafd_conf_fname}, ${stacd_conf_fname}, ${stas_conf_fname} and ${ETC_NVME_DIR}"
rm "${stafd_conf_fname}"
rm "${stacd_conf_fname}"
rm "${stas_conf_fname}"
rm -rf "${ETC_NVME_DIR}"
rm -f "${SCRATCH_FILE}"
printf "\n"

Expand Down Expand Up @@ -259,6 +334,17 @@ fi

prerun_setup

#*******************************************************************************
# Declare the @ETC@/nvme the daemons run against
etc_nvme_populate
if etc_nvme_bind_supported; then
ETC_NVME_PROP="--property=BindPaths=${ETC_NVME_DIR}:@ETC@/nvme"
else
ETC_NVME_PROP=""
log "WARNING: BindPaths= is unavailable here, so the daemons will read the machine's own @ETC@/nvme"
fi
printf "\n"

#*******************************************************************************
# Load nvme kernel module
log "modprobe nvme_tcp"
Expand Down