Skip to content

Commit baeb6ff

Browse files
committed
CI: require the whole coordinate to be at one build, not merely present
Review of #525. The presence check missed the case that matters more than a first publication: an overwrite that failed part-way. Every <snapshotVersion> in a snapshot's maven-metadata.xml carries its own <value>, updated as that file lands. After the first complete publication all three extensions are listed and stay listed, so an overwrite that replaced the POM and then failed leaves the POM at build N+1 with the module metadata and the binary still at N. The old check saw three extensions, read the N+1 POM, found the pin in its stamp, and skipped rebuilding — permanently, and with a coordinate split across two JNI builds. So the check is now agreement rather than presence: the unclassified pom, module and binary entries must all name the same <timestamp>-<buildNumber>, and the POM is fetched by that name, so a metadata entry pointing at a file that never landed reads as no stamp and rebuilds. `timestamped_name` became `snapshot_value`, since the build identifier rather than one file name is what is being compared. The self-test gains the split case, which the previous fixtures could not express. Checked against the published 1.9.0-rc8-SNAPSHOT: all three coordinates whole, at 1.9.0-rc8-20260810.012355-1.
1 parent fd17b7e commit baeb6ff

1 file changed

Lines changed: 65 additions & 42 deletions

File tree

ci/scripts/flat-jni-copy.bash

Lines changed: 65 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -34,11 +34,12 @@ readonly qualifier=${FLAT_JNI_QUALIFIER:-java}
3434
#
3535
# Each with the extension of its binary, because a matching stamp is not on its
3636
# own proof of a finished publication. The POM goes up before the Gradle module
37-
# metadata and before the jar or aar, so a run that died in between would leave
38-
# three readable stamps, no module metadata for Gradle to resolve a variant
39-
# against, and a decision to skip rebuilding — forever, since the next run reads
40-
# the same three stamps. Nothing downstream would catch it either: the consumer
41-
# smoke test runs on Linux, and the broken variant could be the Android one.
37+
# metadata and before the jar or aar, so a publication that died in between —
38+
# whether it was the first or an overwrite — leaves a readable stamp with no
39+
# module metadata for Gradle to resolve a variant against, and a decision to skip
40+
# rebuilding. Forever, since the next run reads the same stamp. Nothing
41+
# downstream would catch it either: the consumer smoke test runs on Linux, and
42+
# the broken variant could be the Android one.
4243
readonly artifacts=(
4344
zenoh-flat-jni:jar
4445
zenoh-flat-jni-jvm:jar
@@ -51,44 +52,55 @@ pinned_commit() {
5152
grep -Eom1 'zenoh-flat-jni\.git[^#"]*#[0-9a-f]{40}' | grep -Eo '[0-9a-f]{40}$'
5253
}
5354

54-
# The file name the newest timestamped build of a snapshot has, from that
55-
# version's maven-metadata.xml on stdin:
56-
# <artifact>-<version without -SNAPSHOT>-<timestamp>-<buildNumber>.<ext>
57-
timestamped_name() { # <artifact> <version> <ext>
55+
# Which timestamped build a snapshot currently resolves to, from that version's
56+
# maven-metadata.xml on stdin:
57+
# <version without -SNAPSHOT>-<timestamp>-<buildNumber>
58+
# It is also the middle of every file name in that build:
59+
# <artifact>-<value>.<ext>
60+
snapshot_value() { # <version>
5861
local metadata timestamp build
5962
metadata=$(cat)
6063
timestamp=$(sed -n 's:.*<timestamp>\(.*\)</timestamp>.*:\1:p' <<<"$metadata" | head -1)
6164
build=$(sed -n 's:.*<buildNumber>\(.*\)</buildNumber>.*:\1:p' <<<"$metadata" | head -1)
6265
[[ -n $timestamp && -n $build ]] || return 1
63-
printf '%s-%s-%s-%s.%s' "$1" "${2%-SNAPSHOT}" "$timestamp" "$build" "$3"
66+
printf '%s-%s-%s' "${1%-SNAPSHOT}" "$timestamp" "$build"
6467
}
6568

6669
# The commit a published POM on stdin was built from; empty when it has no stamp.
6770
pom_commit() {
6871
sed -n 's:.*<zenoh\.flatJniCommit>\(.*\)</zenoh\.flatJniCommit>.*:\1:p' | head -1
6972
}
7073

71-
# Whether the metadata on stdin advertises an unclassified artifact of each given
72-
# extension. Maven appends an entry as each file lands, so a publication that
73-
# stopped part-way lists fewer than a finished one. The anchor is what excludes
74-
# the sources and javadoc jars: the schema puts <classifier> before <extension>,
75-
# so only a main artifact starts its entry with the extension.
76-
advertises() { # <extension>…
77-
local blocks ext
74+
# Whether the metadata on stdin says every one of the given extensions is at the
75+
# given build. Each <snapshotVersion> carries its own <value>, updated as that
76+
# file lands, so an overwrite that failed part-way leaves the POM at build N+1
77+
# while the module metadata and the binary are still at N — which is the case a
78+
# presence check cannot see, since all three entries exist either way and have
79+
# since the first publication.
80+
#
81+
# The anchor is what excludes the sources and javadoc jars: the schema puts
82+
# <classifier> before <extension>, so only a main artifact starts its entry with
83+
# the extension.
84+
all_at() { # <value> <extension>…
85+
local blocks value ext
7886
blocks=$(tr -d '[:space:]' | sed 's:<snapshotVersion>:\n:g')
87+
value=${1//./\\.}
88+
shift
7989
for ext in "$@"; do
80-
grep -q "^<extension>$ext</extension>" <<<"$blocks" || return 1
90+
grep -q "^<extension>$ext</extension><value>$value</value>" <<<"$blocks" || return 1
8191
done
8292
}
8393

8494
# The stamp of the published copy of one coordinate; empty if it is not there, or
85-
# not all of it is.
95+
# not all of it is at the same build.
8696
published_commit() { # <artifact> <version> <binary extension>
87-
local base_url="$repository/$group_path/$1/$2" metadata name
97+
local base_url="$repository/$group_path/$1/$2" metadata value
8898
metadata=$(curl -sf "$base_url/maven-metadata.xml") || return 0
89-
advertises pom module "$3" <<<"$metadata" || return 0
90-
name=$(timestamped_name "$1" "$2" pom <<<"$metadata") || return 0
91-
{ curl -sf "$base_url/$name" || true; } | pom_commit
99+
value=$(snapshot_value "$2" <<<"$metadata") || return 0
100+
all_at "$value" pom module "$3" <<<"$metadata" || return 0
101+
# The POM is fetched by that name, so a metadata entry naming a file that
102+
# never landed reads as no stamp — and rebuilds.
103+
{ curl -sf "$base_url/$1-$value.pom" || true; } | pom_commit
92104
}
93105

94106
self_test() {
@@ -101,7 +113,7 @@ self_test() {
101113
got=$(pinned_commit <Cargo.lock)
102114
[[ $got =~ ^[0-9a-f]{40}$ ]] || { echo "pinned_commit(Cargo.lock): $got" >&2; exit 1; }
103115

104-
got=$(timestamped_name zenoh-flat-jni 1.9.0-java-SNAPSHOT pom <<'EOF'
116+
got=$(snapshot_value 1.9.0-java-SNAPSHOT <<'EOF'
105117
<versioning>
106118
<snapshot>
107119
<timestamp>20260810.012355</timestamp>
@@ -110,11 +122,11 @@ self_test() {
110122
</versioning>
111123
EOF
112124
)
113-
[[ $got == zenoh-flat-jni-1.9.0-java-20260810.012355-1.pom ]] || { echo "timestamped_name: $got" >&2; exit 1; }
125+
[[ $got == 1.9.0-java-20260810.012355-1 ]] || { echo "snapshot_value: $got" >&2; exit 1; }
114126

115-
# A release-style metadata carries no <snapshot> block: no name to build.
116-
if timestamped_name zenoh-flat-jni 1.9.0 pom <<<'<versioning><latest>1.9.0</latest></versioning>' >/dev/null; then
117-
echo "timestamped_name accepted metadata with no snapshot block" >&2
127+
# A release-style metadata carries no <snapshot> block: no build to name.
128+
if snapshot_value 1.9.0 <<<'<versioning><latest>1.9.0</latest></versioning>' >/dev/null; then
129+
echo "snapshot_value accepted metadata with no snapshot block" >&2
118130
exit 1
119131
fi
120132

@@ -124,23 +136,34 @@ EOF
124136
got=$(pom_commit <<<'<project><version>1.9.0-java-SNAPSHOT</version></project>')
125137
[[ -z $got ]] || { echo "pom_commit on an unstamped POM: $got" >&2; exit 1; }
126138

127-
# A finished publication, then the two ways one can be unfinished: stopped
128-
# after the POM, and carrying a sources jar but no main jar. The layout is
129-
# what zenoh-flat-jni really publishes — checked against 1.9.0-rc8-SNAPSHOT.
139+
# A finished publication of build -1, in the layout zenoh-flat-jni really
140+
# produces — checked against the published 1.9.0-rc8-SNAPSHOT.
141+
local n=1.9.0-java-20260810.012355-1
130142
local finished="
131-
<snapshotVersion><extension>pom</extension></snapshotVersion>
132-
<snapshotVersion><extension>module</extension></snapshotVersion>
133-
<snapshotVersion><classifier>sources</classifier><extension>jar</extension></snapshotVersion>
134-
<snapshotVersion><extension>jar</extension></snapshotVersion>"
135-
advertises pom module jar <<<"$finished" || { echo "advertises: finished rejected" >&2; exit 1; }
136-
if advertises pom module aar <<<"$finished"; then
137-
echo "advertises: accepted a jar publication as an aar one" >&2; exit 1
143+
<snapshotVersion><extension>pom</extension><value>$n</value></snapshotVersion>
144+
<snapshotVersion><extension>module</extension><value>$n</value></snapshotVersion>
145+
<snapshotVersion><classifier>sources</classifier><extension>jar</extension><value>$n</value></snapshotVersion>
146+
<snapshotVersion><extension>jar</extension><value>$n</value></snapshotVersion>"
147+
all_at "$n" pom module jar <<<"$finished" || { echo "all_at: finished rejected" >&2; exit 1; }
148+
if all_at "$n" pom module aar <<<"$finished"; then
149+
echo "all_at: accepted a jar publication as an aar one" >&2; exit 1
150+
fi
151+
if all_at "$n" pom module jar <<<"<snapshotVersion><extension>pom</extension><value>$n</value></snapshotVersion>"; then
152+
echo "all_at: accepted a publication that stopped after the POM" >&2; exit 1
138153
fi
139-
if advertises pom module jar <<<'<snapshotVersion><extension>pom</extension></snapshotVersion>'; then
140-
echo "advertises: accepted a publication that stopped after the POM" >&2; exit 1
154+
if all_at "$n" jar <<<"<snapshotVersion><classifier>sources</classifier><extension>jar</extension><value>$n</value></snapshotVersion>"; then
155+
echo "all_at: took the sources jar for the main one" >&2; exit 1
141156
fi
142-
if advertises jar <<<'<snapshotVersion><classifier>sources</classifier><extension>jar</extension></snapshotVersion>'; then
143-
echo "advertises: took the sources jar for the main one" >&2; exit 1
157+
158+
# The case a presence check cannot see: an overwrite that replaced the POM
159+
# and then failed, leaving the module metadata and the binary at the build
160+
# before it. Every extension is still listed; only the values disagree.
161+
local m=1.9.0-java-20260811.030000-2
162+
if all_at "$m" pom module jar <<<"
163+
<snapshotVersion><extension>pom</extension><value>$m</value></snapshotVersion>
164+
<snapshotVersion><extension>module</extension><value>$n</value></snapshotVersion>
165+
<snapshotVersion><extension>jar</extension><value>$n</value></snapshotVersion>"; then
166+
echo "all_at: accepted a coordinate split across two builds" >&2; exit 1
144167
fi
145168

146169
echo "flat-jni-copy.bash self-test OK"

0 commit comments

Comments
 (0)