Skip to content
Flow-IPC pipeline

Flow-IPC pipeline #1049

Sign in to view logs

Flow-IPC pipeline

Flow-IPC pipeline #1049

Workflow file for this run

.github/workflows/main.yml at 1d4799d
# Flow-IPC
# Copyright 2023 Akamai Technologies, Inc.
#
# Licensed under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in
# compliance with the License. You may obtain a copy
# of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in
# writing, software distributed under the License is
# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
# CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing
# permissions and limitations under the License.
name: Flow-IPC pipeline
# Discussion on the philosophy behind the workflow (pipeline) below. The steps themselves should be easy enough to
# follow, but what's the bigger picture? Answer: The pipeline below is an algorithm, as coded below, invoked
# by GitHub on many runners (machines); and the inputs to this algorithm are:
# - a branch (`main`, or a feature/PR branch) or tag (usually release tag v<version>);
# - hence the current contents of that branch/tag;
# - some trigger event that causes the workflow (pipeline) to run in the first place:
# push to the branch/tag; pull request (PR) creation; or manual click by human (as of this writing usually
# for testing or debugging only).
# So given those inputs, what are the outputs or side effects? To answer, consider what jobs are actually run
# (as parallel as possible). There's `setup` and `set-vars` as of this writing, but these are
# purely tactical prerequisites and not worth discussing here; once they complete then the "real" jobs will run:
# build-and-test and doc-and-release. Here are their goals, meaning outputs/side effects given the above
# inputs:
# - build-and-test:
# - Summary: Test building and running the product; and bullet-proof it further via run-time sanitizers.
# - Build the code, including demo/test code, at the given branch/tag in wide variety of compiler/build-type
# combinations. The ability to build (including linking into test binaries) successfully is, of course,
# a test in and of itself.
# - Test that code by running the available unit test and integration test/demo binaries built in previous bullet.
# If something fails, help (within reason) make it clear what failed and why.
# - Output: Make any logs available, whether on success or failure. If console output is sufficient,
# great: it'll be visible via GitHub Actions UI. If not, save the non-console log output in a log tarball
# available as workflow artifact for download via GitHub Actions UI.
# - All of the above but with certain run-time sanitizers (as of this writing ASAN/LSAN, UBSAN, TSAN,
# and possibly clang-MSAN) enabled at build time as well. (RelWithDebInfo is a sufficient base build type
# for a sanitizer-enabled build; a subset of compilers -- as opposed to all of them -- is also sufficient
# for pragamtic reasons.)
# - doc-and-release:
# - Summary: 1, generate documentation from the source code (using Doxygen et al) and make it conveniently
# available. 2, update GitHub Release and GitHub Pages web site automatically with any relevant info, namely
# with the generated docs and source code tarball/zip.
# - Generate the documentation using Doxygen et al. The goal here is the *canonical generated documentation*, not
# *testing the ability to generate it*. It's a subtle difference, but it's important to note it, because
# if we wanted to test the ins and outs of doc generation in various environments then we could have a much more
# complex and longer pipeline -- and perhaps we should... but that's not the goal *here*.
# - Output: Make it available as workflow artfiact for download via GitHub Actions UI.
# Hence human can simply download it, whether it's for the `main` tip or a specific release version.
# - Side effect: (On pushes/merges to `main` branch *only*) Check-in the *generated docs* back into `main`
# itself, so that (1) they're available for local perusal by any person cloning/pulling `main` and (2) ditto
# for any downstream releases (which are mere tags of `main`).
# - Side effect: Having done so, signal GitHub Pages web site for the org, so that the new `main` docs
# are reflected at this web site. (It'll automatically clone `main`, copy the generated docs to web site
# replacing the existing `main` docs if any, and stage this to the web.)
# - (If the branch/tag input is, in fact, tag v* -- a release tag) The mere fact that a push to a release tag X
# is being observed means we want:
# - Side effect: Ensure that the GitHub Release X has the full source code available for human download
# as a package (tarball/zip). I.e., upload it there as needed.
# - Side effect: Ensure that the GitHub Pages web site actually lists this Release and its associated
# generated documentation. I.e., signal Pages that there's a Release X now: it'll then do the right
# thing (namely checkout the docs at the tag and copy them over; and add a link to the release and docs
# in the appropriate web page).
#
# So, roughly speaking,
# - build-and-test is about building and testing the product, including bullet-proofing it with sanitizers.
# - doc-and-release is about official documentation-based-on-source being available in various customary ways;
# with a side of official releases being accessible and comprehensively presented in customary ways.
# - TODO: It *could* be argued that those 2 goals are related but separate, and perhaps they should be 2 separate
# jobs. However, at least as of this writing, there's definite overlap between them, and combining the 2
# makes pragamtic sense. It's worth revisiting periodically perhaps.
on:
# Want to merge to development tip? Should probably pass these builds/tests and doc generation first (the latter
# in case the source change adds a Doxygen error or bad-looking docs). Note this runs on PR creation *and* update.
pull_request:
branches:
- main
push:
# Was able to merge PR to `main` tip? Should ensure these builds/tests/doc generation still pass after the fact
# (post-merge `main` does not always equal the PR-branch state); plus:
# auto-check-in generated docs into `main` branch tip (and signal web site to update accordingly).
branches:
- main
# Created release tag? Just in case, should ensure these builds/tests pass (just in case, but it's redundant
# against the branch=main trigger above); generate docs and make them available as artifact (nice: docs for
# a particular product version available online); plus:
# signal web site to update accordingly (there's probably a new release to present, with full source code attached;
# and there are docs for it; nice: docs for a particular product version available online).
tags:
- v*
# To create the button that runs a workflow manually: testing, debugging, demoing.
workflow_dispatch:
inputs:
job-filter:
description: >-
If not blank: only jobs whose name contains this substring shall really run (e.g.:
"clang-15-relwithdebinfo-asan-cxx17" = 1 cell; "asan" = the ASAN cells; "doc" = the doc job).
Others complete ~instantly with all steps skipped.
required: false
default: ''
type: string
# XXX Temporary validation aid; remove (with its 3 consumers below) when the sanitizer-validation
# campaign concludes. Lets 2 parallel runs split the test load of one cell.
test-subset:
description: >-
If 'u-t-only': the transport_test runs no-op. If 't-t-only': the unit-tests run no-ops.
Blank: everything runs.
required: false
default: ''
type: string
jobs:
# Impetus behind this is to set up at least one magic string used in 2+ places.
# Unfortunately simply using `env:` does not work, as jobs.<id>.if refuses to access the workflow-global `env.<id>`.
# Ridiculous. TODO: Revisit.
#
# Folding this into `setup` would have been fine, but as of this writing `setup` needs at least one constant
# from here, and getting the order of operations correct within that one job is non-trivial at best.
set-vars:
runs-on: ubuntu-latest
steps:
- name: Set variables/constants for subsequent use
run: |
# Set variables/constants for subsequent use.
outputs:
doc-commit-message: (Commit by workflow script) Update generated documentation.
# Impetus behind this is to gate whether the real jobs below actually need to run. Can of course also compute
# other things as needed.
setup:
needs: set-vars
runs-on: ubuntu-latest
steps:
- name: Checkout VERSION file and tag history
uses: actions/checkout@v6
with:
sparse-checkout: VERSION
fetch-depth: 0 # Get all history regarding tags. We'll need that for VERSION checking below.
- name: Grab repository version from `VERSION` file
id: repo_version
run: |
# Grab repository version from `VERSION` file.
if [ ! -e VERSION ]; then
echo '::error ::No VERSION file in repository root. One must exist.'
exit 1
fi
VERSION=`cat VERSION`
echo "Determined version: [$VERSION]."
echo "version=$VERSION" >> $GITHUB_OUTPUT
- name: (On release creation only) Check repository version against release tag name
if: success() && (!cancelled()) && startsWith(github.ref, 'refs/tags/v')
run: |
# Check repository version against release tag name.
VERSION=${{ steps.repo_version.outputs.version }}
TAG_VERSION=${GITHUB_REF#refs/tags/v}
if [ "$VERSION" = "$TAG_VERSION" ]; then
echo ":notice ::Tag name and repo version are an exact match. Looks good."
elif [[ "$TAG_VERSION" == "$VERSION-"* ]]; then
echo "::notice ::Tag name [v$TAG_VERSION] starts with repo version (pre-release). Allowed."
else
echo "::error ::VERSION file version v[$VERSION] must = release tag name [v$TAG_VERSION]."
echo "::error ::Suggest deleting release and tag, fixing VERSION if needed, and re-creating release/tag."
exit 1
fi
# TODO: Would be nice to also grab release name and ensure it equals tag name v<...>.
# Could be a separate step or just execute here if the above succeeded so far. Requires API fetch, so it's
# a few lines.
- name: (Except on release creation) Check repository version against already-existing release tag names
if: success() && (!cancelled()) && (!startsWith(github.ref, 'refs/tags/v'))
run: |
# Check repo version against release tag name.
# Do not fail on account of VERSION in this path however... just warn or gently inform.
VERSION=${{ steps.repo_version.outputs.version }}
if git rev-parse "v$VERSION" >/dev/null 2>&1; then
echo "::warning ::Tag [v$VERSION] already exists. The repo/VERSION file will require "\
"a bump, before the next release. However this should be done *just* ahead of release creation and must "\
"follow semantic versioning conventions, especially regarding when to bump major/minor/patch components. "\
"When ready to do this consult [https://semver.org/] as needed."
elif git tag | grep -q "^v$VERSION-"; then
echo "::notice ::Since pre-release tag(s) [v$VERSION-*] already exist(s), but no tag "\
"[v$VERSION] exists, this repo state is likely intended for a later pre-release or official release. "\
"Cool! However, reminder: "\
"The repo/VERSION version should be decided/finalized *just* ahead of release creation and must "\
"follow semantic versioning conventions, especially regarding when to bump major/minor/patch components. "\
"When ready to do this consult [https://semver.org/] as needed."
else
echo "::notice ::Since no tag [v$VERSION] or [v$VERSION-*] exists, this repo state "\
"is likely intended for a future release. Cool! However, reminder: "\
"The repo/VERSION version should be decided/finalized *just* ahead of release creation and must "\
"follow semantic versioning conventions, especially regarding when to bump major/minor/patch components. "\
"When ready to do this consult [https://semver.org/] as needed."
fi
- id: compute_proceed_else_not
name: Compute whether for main jobs to proceed or not
# For checking whether github.event.head_commit.message starts with needs.set-vars.outputs.doc-commit-message
# it is tempting to just use:
# '${{ github.event.head_commit.message }}' != '${{ needs.set-vars.outputs.doc-commit-message }}'*
# This works usually but is unsafe: If the head commit message contains, for example, a single-quote,
# then the shell syntax breaks down. One approach would be to use pipeline startsWith() before `run`,
# but apparently it cannot be done directly inside `run`; so it is a pain. Staying with shell scripting
# then we can just use here-doc syntax and a temp file, so if the commit message does not have the
# here-doc terminator token, then we're fine.
run: |
# Compute whether for main jobs to proceed or not.
TMP_MSG=/tmp/flow-ipc-pipeline-head-cmt-msg.txt
cat <<'FLOW_IPC_PIPELINE_HEAD_CMD_MSG_EOF' > $TMP_MSG
${{ github.event.head_commit.message }}
FLOW_IPC_PIPELINE_HEAD_CMD_MSG_EOF
if [ '${{ github.ref }}' != 'refs/heads/main' ] || \
[ '${{ github.event_name }}' != 'push' ] || \
! { head --lines=1 $TMP_MSG | fgrep -xq '${{ needs.set-vars.outputs.doc-commit-message }}'; }; then
echo 'proceed-else-not=true' >> $GITHUB_OUTPUT
else
echo 'proceed-else-not=false' >> $GITHUB_OUTPUT
echo 'The real jobs will not run: earlier `doc-and-release` job checked-in generated docs to `main`.'
echo 'That is not a source change and requires no actual pipeline to execute.'
fi
outputs:
proceed-else-not: ${{ steps.compute_proceed_else_not.outputs.proceed-else-not }}
build-and-test:
needs: [setup, set-vars]
if: |
needs.setup.outputs.proceed-else-not == 'true'
strategy:
fail-fast: false
# Please read notes in similar place in Flow's main.yml regarding the matrix's setup, especially
# the bulk of it that discusses how the C++ standard setting (17, 20, etc.) is handled.
# We omit that large comment here and related comments inline inside the `matrix`. Same deal as for Flow.
matrix:
cxx-std:
- id: cxx17
- id: cxx20
compiler:
- id: gcc-9
name: gcc
version: 9
c-path: /usr/bin/gcc-9
cpp-path: /usr/bin/g++-9
install: True
# Very-RAM-hungry when generating debug info (see also high-mem-build in build-test-cfg axis; as of this
# writing if both are true then some step(s) may tweak some knobs to reduce RAM use).
high-mem-build: True
- id: gcc-10
name: gcc
version: 10
c-path: /usr/bin/gcc-10
cpp-path: /usr/bin/g++-10
# See comment on first occurrence.
high-mem-build: True
- id: gcc-11
name: gcc
version: 11
c-path: /usr/bin/gcc-11
cpp-path: /usr/bin/g++-11
# See comment on first occurrence.
high-mem-build: True
- id: gcc-13
name: gcc
version: 13
c-path: /usr/bin/gcc-13
cpp-path: /usr/bin/g++-13
install: True
# See comment on first occurrence.
high-mem-build: True
- id: clang-13
name: clang
version: 13
c-path: /usr/bin/clang-13
cpp-path: /usr/bin/clang++-13
- id: clang-15
name: clang
version: 15
c-path: /usr/bin/clang-15
cpp-path: /usr/bin/clang++-15
- id: clang-16
name: clang
version: 16
c-path: /usr/bin/clang-16
cpp-path: /usr/bin/clang++-16
install: True
- id: clang-17
name: clang
version: 17
c-path: /usr/bin/clang-17
cpp-path: /usr/bin/clang++-17
install: True
build-test-cfg:
- id: debug
conan-profile-build-type: Debug
conan-profile-jemalloc-build-type: Debug
# Debug-info generation makes compiles much more RAM-hungry; see the profile-creation step, which
# *may* (depending on possibly other cfg such as compiler) react to this flag by, say, reducing build
# parallelism. Other consumers may arise over time. This is a hint to the consumer steps which
# are responsible for the ultimate behavior.
high-mem-build: True
# In any case Debug, at the CMake script (in meta-project ./, and in flow/, ipc_*/) level,
# means LTO will be ignored (only *Rel* build-types enable LTO, if so instructed).
# Still keeping this here to make that clear to the reader/maintainer. Could remove it though.
no-lto: True
- id: release
conan-profile-build-type: Release
conan-profile-jemalloc-build-type: Release
# Leaving no-lto at default (false); full-on-optimized-no-debug is the quentessential LTO use case.
- id: relwithdebinfo
conan-profile-build-type: RelWithDebInfo
conan-profile-jemalloc-build-type: Release
# See comment on first occurrence.
high-mem-build: True
# As of this writing RelWithDebInfo (in CMake, and Conan in our case at least doesn't override it)
# defaults to -O2 (not -O3), which isn't a super-effective way of deploying LTO. Plus
# we can use a test of non-LTO building.
# TODO: Perhaps this should be a separate matrix dimension (LTO on, LTO off). Number of configs will
# jump up, but it is more methodical and nice.
no-lto: True
- id: minsizerel
conan-profile-build-type: MinSizeRel
conan-profile-jemalloc-build-type: Release
# Leaving no-lto at default (false); -Os (size-optimizing) with LTO on is pretty realistic.
- id: relwithdebinfo-asan
conan-profile-build-type: RelWithDebInfo
conan-profile-jemalloc-build-type: Release
conan-profile-custom-conf: |
# no-omit-frame-pointer recommended in (A|UB|M)SAN docs for nice stack traces.
tools.build:cflags = ["-fsanitize=address", "-fno-omit-frame-pointer"]
tools.build:cxxflags = ["-fsanitize=address", "-fno-omit-frame-pointer"]
tools.build:sharedlinkflags = ["-fsanitize=address"]
tools.build:exelinkflags = ["-fsanitize=address"]
# jemalloc recipe needs this as of this writing (even though it dupes the stuff just-above conceptually).
conan-profile-custom-buildenv: |
CXXFLAGS = -fsanitize=address -fno-omit-frame-pointer
CFLAGS = -fsanitize=address -fno-omit-frame-pointer
LDFLAGS = -fsanitize=address
conan-profile-custom-settings: |
compiler.sanitizer = address
conan-custom-settings-defs: | # Could we not copy/paste these 4x?
data['compiler']['gcc']['sanitizer'] = ['None', 'address', 'thread', 'memory', 'undefined']
data['compiler']['clang']['sanitizer'] = ['None', 'address', 'thread', 'memory', 'undefined']
sanitizer-name: asan # Used as internal enum of sorts + name of sanitizer-related dirs.
# At least ASAN with clang + LTO => cryptic link error.
# Regardless regular RelWithDebInfo already has no-lto=true; so we would follow suit. Just be aware
# that changing it to false for whatever reason => ASAN probably breaks.
no-lto: True
- id: relwithdebinfo-ubsan
conan-profile-build-type: RelWithDebInfo
conan-profile-jemalloc-build-type: Release
conan-profile-custom-conf: |
tools.build:cflags = ["-fsanitize=undefined", "-fno-omit-frame-pointer"]
tools.build:cxxflags = ["-fsanitize=undefined", "-fno-omit-frame-pointer"]
tools.build:sharedlinkflags = ["-fsanitize=undefined"]
tools.build:exelinkflags = ["-fsanitize=undefined"]
# jemalloc's internals feature certain known, benign, upstream UB (int-overflowing shifts et al):
# so build it -- and only it -- with UBSAN off, by overriding the above flags for that one package.
# (UBSAN is per-translation-unit -- no TSAN-style all-code-must-be-instrumented requirement -- so
# this is safe. Syntax note: this section's scoping fnmatches the full `jemalloc/1.2.3`-style
# reference -- hence the `*`.)
jemalloc*:tools.build:cflags = ["-fno-omit-frame-pointer"]
jemalloc*:tools.build:cxxflags = ["-fno-omit-frame-pointer"]
jemalloc*:tools.build:sharedlinkflags = []
jemalloc*:tools.build:exelinkflags = []
conan-profile-custom-buildenv: |
CXXFLAGS = -fsanitize=undefined -fno-omit-frame-pointer
CFLAGS = -fsanitize=undefined -fno-omit-frame-pointer
LDFLAGS = -fsanitize=undefined
# (Ditto the jemalloc UBSAN-off exemption in the conf section just above.)
jemalloc*:CXXFLAGS = -fno-omit-frame-pointer
jemalloc*:CFLAGS = -fno-omit-frame-pointer
jemalloc*:LDFLAGS =
conan-profile-custom-settings: |
compiler.sanitizer = undefined
conan-custom-settings-defs: |
data['compiler']['gcc']['sanitizer'] = ['None', 'address', 'thread', 'memory', 'undefined']
data['compiler']['clang']['sanitizer'] = ['None', 'address', 'thread', 'memory', 'undefined']
sanitizer-name: ubsan
# While UBSAN might work with LTO, I do not want the aggravation/entropy. Turn it off.
# Plus inheriting from regular RelWithDebInfo anyway.
no-lto: True
- id: relwithdebinfo-tsan
conan-profile-build-type: RelWithDebInfo
conan-profile-jemalloc-build-type: Release
conan-profile-custom-conf: |
# no-omit-frame-pointer recommended in (A|UB|M)SAN docs for nice stack traces; TSAN docs do not
# mention it. Given the various symbolizer problems mentioned elsewhere in comments in this file,
# it seemed prudent to keep this consistent with those other *SAN.
tools.build:cflags = ["-fsanitize=thread", "-fno-omit-frame-pointer"]
tools.build:cxxflags = ["-fsanitize=thread", "-fno-omit-frame-pointer"]
tools.build:sharedlinkflags = ["-fsanitize=thread", "-fno-omit-frame-pointer"]
tools.build:exelinkflags = ["-fsanitize=thread", "-fno-omit-frame-pointer"]
conan-profile-custom-buildenv: |
CXXFLAGS = -fsanitize=thread -fno-omit-frame-pointer
CFLAGS = -fsanitize=thread -fno-omit-frame-pointer
LDFLAGS = -fsanitize=thread -fno-omit-frame-pointer
conan-profile-custom-settings: |
compiler.sanitizer = thread
conan-custom-settings-defs: |
data['compiler']['gcc']['sanitizer'] = ['None', 'address', 'thread', 'memory', 'undefined']
data['compiler']['clang']['sanitizer'] = ['None', 'address', 'thread', 'memory', 'undefined']
sanitizer-name: tsan
# While TSAN might work with LTO, I do not want the aggravation/entropy. Turn it off.
# Also, for some clangs, there are TSAN WARNINGs at times about too-small symbolizer buffer or something;
# throwing LTO into the mix seems like unnecessary entropy.
# Plus inheriting from regular RelWithDebInfo anyway.
no-lto: True
- id: relwithdebinfo-msan
conan-profile-build-type: RelWithDebInfo
conan-profile-jemalloc-build-type: Release
conan-profile-custom-conf: |
tools.build:cflags = ["-fsanitize=memory", "-fno-omit-frame-pointer", "-fsanitize-ignorelist=/tmp/msan_ignore_list.cfg"]
tools.build:cxxflags = ["-fsanitize=memory", "-fno-omit-frame-pointer", "-fsanitize-ignorelist=/tmp/msan_ignore_list.cfg"]
tools.build:sharedlinkflags = ["-fsanitize=memory"]
tools.build:exelinkflags = ["-fsanitize=memory"]
conan-profile-custom-buildenv: |
CXXFLAGS = -fsanitize=memory -fno-omit-frame-pointer -fsanitize-ignorelist=/tmp/msan_ignore_list.cfg
CFLAGS = -fsanitize=memory -fno-omit-frame-pointer -fsanitize-ignorelist=/tmp/msan_ignore_list.cfg
LDFLAGS = -fsanitize=memory
conan-profile-custom-settings: |
compiler.sanitizer = memory
conan-custom-settings-defs: |
data['compiler']['gcc']['sanitizer'] = ['None', 'address', 'thread', 'memory', 'undefined']
data['compiler']['clang']['sanitizer'] = ['None', 'address', 'thread', 'memory', 'undefined']
sanitizer-name: msan
# While MSAN might work with LTO, I do not want the aggravation/entropy. Turn it off.
# Plus inheriting from regular RelWithDebInfo anyway.
no-lto: True
# We concentrate on clang sanitizers; they are newer/nicer; also MSAN is clang-only. So gcc ones excluded.
# Attention! Excluding some sanitizer job(s) (with these reasons):
# - MSAN: MSAN protects against reads of ununitialized memory; it is clang-only (not gcc), unlike the other
# *SAN. Its mission overlaps at least partially with UBSAN's; for example for sure there were a couple of
# uninitialized reads in test code which UBSAN caught. Its current state -- if not excluded -- is as
# follows: 1, due to (as of this writing) building dependencies, including the capnp compiler binary used
# during our build process, with the same compiler build config as the real code, ignore-list entires had
# to be added to get past these problems. 2, before main() in *all* our demos/tests Boost was doing some
# global init which MSAN did not like and hence aborted before main(); these are now ignored as well.
# 3, this got us into main() at least, but immediately cryptic aborts started, seemingly again originating
# in Boost (but requires detailed investigation to really understand). At this point I (ygoldfel)
# disabled MSAN and filed a ticket. The overall status: MSAN is said to be useful, even with UBSAN active,
# but all resources including official docs indicate that it is a high-maintenance tool:
# *All* linked code, including libc and libstdc++/libc++ (the former in our case as of this writing),
# must be instrumented to avoid cryptic false positives. The good news is we do build other things
# instrumented (Boost libs, jemalloc, capnp/kj, gtest); but not libstdc++ (which official docs recommend);
# this can be done but requires more work (I would suggest switching to libc++ in that case from the
# start, as the clang people made it and themselves build it instrumented for their testing).
# So the bottom line: MSAN is a tough cookie and requires more work before enabling. In the meantime we
# have many layers of protection, including ASAN and UBSAN and lots of unit and integration tests.
# So this status quo is pretty good. TODO: Do the work/get MSAN functional/useful; un-exclude it then.
# - *SAN with gcc: We concentrate on clang sanitizers; they are newer/nicer; having to worry about differences
# between them is an excessive burden. So excluding gcc ASAN/UBSAN/TSAN.
# - TODO: Consider reducing to the newest or most stable clang. Generally they just keep improving; the
# chances of something being uncaught (incorrectly) with a later version are slim. Not impossible though.
# Look into it. Update: As of this writing transport_test exercise mode/SHM-jemalloc sub-mode is
# disabled for TSAN clang-17 due to instability of TSAN itself. So clearly when it comes to TSAN
# (which is officially in beta as of clang-17), higher version does not mean everything is better.
# Hence perhaps this to-do is more of a longer-term thing, when all the sanitizers stabilize, or when
# we find a single very-stable config. Just remember we aren't testing the sanitizer or our code's
# ability to be sanitized; we just want to detect any problems in the code -- however we get there.
# - *SAN with clang-13 (but not 15+): As of this writing clang-13 produces some additional warnings,
# at least in TSAN, which appear to be related to nearby non-race messages like
# `==75454==WARNING: Symbolizer buffer too small`. As a result, we either have to eliminate the
# latter problem (TODO: look into it) or suppress false-positive race warnings that might only look
# like added problems due to the incomplete stack traces. In general scanning through clang-13-produced
# TSAN output, there is a chaotic feel to it, when it comes to stack output. Meanwhile so far
# there has been zero evidence that a lower-version clang has actual added problems versus the other
# compiler-versions, due to generating code differently. Since we have at least 3 other compiler-versions
# running all the *SAN, I (ygoldfel) decided to exclude clang-13 *SAN, until the chaotic *SAN messages can
# be at least reduced. In the meantime our *SAN coverage is still quite good.
exclude:
- build-test-cfg: { id: relwithdebinfo-msan }
- compiler: { id: gcc-9 }
build-test-cfg: { id: relwithdebinfo-asan }
- compiler: { id: gcc-10 }
build-test-cfg: { id: relwithdebinfo-asan }
- compiler: { id: gcc-11 }
build-test-cfg: { id: relwithdebinfo-asan }
- compiler: { id: gcc-13 }
build-test-cfg: { id: relwithdebinfo-asan }
- compiler: { id: clang-13 }
build-test-cfg: { id: relwithdebinfo-asan }
- compiler: { id: gcc-9 }
build-test-cfg: { id: relwithdebinfo-ubsan }
- compiler: { id: gcc-10 }
build-test-cfg: { id: relwithdebinfo-ubsan }
- compiler: { id: gcc-11 }
build-test-cfg: { id: relwithdebinfo-ubsan }
- compiler: { id: gcc-13 }
build-test-cfg: { id: relwithdebinfo-ubsan }
- compiler: { id: clang-13 }
build-test-cfg: { id: relwithdebinfo-ubsan }
- compiler: { id: gcc-9 }
build-test-cfg: { id: relwithdebinfo-tsan }
- compiler: { id: gcc-10 }
build-test-cfg: { id: relwithdebinfo-tsan }
- compiler: { id: gcc-11 }
build-test-cfg: { id: relwithdebinfo-tsan }
- compiler: { id: gcc-13 }
build-test-cfg: { id: relwithdebinfo-tsan }
- compiler: { id: clang-13 }
build-test-cfg: { id: relwithdebinfo-tsan }
# The rest of these deal with C++20; again please see Flow's main.yml commentary.
# TODO: I am sure there's an overarching TODO somewhere around here to the same effect but... to
# reiterate... it's unpleasant so much is copy-pasted from one .yml to the other, and even accepting
# that it is odd that for some items the real comment is in A, while B says see A; while for others
# it is the opposite. The info is there, and it's not redundant (only the code is), but still.
- cxx-std: { id: cxx20 }
compiler: { id: gcc-9 }
- cxx-std: { id: cxx20 }
compiler: { id: gcc-10 }
- cxx-std: { id: cxx20 }
compiler: { id: gcc-11 }
- cxx-std: { id: cxx20 }
compiler: { id: clang-13 }
- cxx-std: { id: cxx20 }
compiler: { id: clang-15 }
- cxx-std: { id: cxx20 }
compiler: { id: clang-16 }
- cxx-std: { id: cxx20 }
build-test-cfg: { id: release }
- cxx-std: { id: cxx20 }
build-test-cfg: { id: minsizerel }
- cxx-std: { id: cxx20 }
build-test-cfg: { id: relwithdebinfo-asan }
- cxx-std: { id: cxx20 }
build-test-cfg: { id: relwithdebinfo-ubsan }
- cxx-std: { id: cxx20 }
build-test-cfg: { id: relwithdebinfo-tsan }
# Not using ubuntu-latest, so as to avoid surprises with OS upgrades and such.
runs-on: ubuntu-22.04
# A healthy job as of this writing (8/2026) takes at most 1.25 hours; meanwhile certain sanitizer failure modes
# (TSAN deadlock-detector issues at least) hang a process indefinitely, and the GitHub default job timeout is
# 6 hours. We bound it for reasonabless and visibility of these hangs, particularly during debug/test cycles.
timeout-minutes: 120
name: ${{ matrix.compiler.id }}-${{ matrix.build-test-cfg.id }}-${{ matrix.cxx-std.id }}
env:
# Whether this matrix cell shall really run, per the workflow_dispatch job-filter input. See comments in
# Flow workflow counterpart. We use same techniques.
cell-selected: ${{ (github.event.inputs.job-filter || '') == '' || contains(format('{0}-{1}-{2}', matrix.compiler.id, matrix.build-test-cfg.id, matrix.cxx-std.id), github.event.inputs.job-filter) }}
build-dir: ${{ github.workspace }}/build/${{ matrix.build-test-cfg.conan-profile-build-type }}
install-root-dir: ${{ github.workspace }}/install/${{ matrix.build-test-cfg.conan-profile-build-type }}
# (Unfortunately cannot refer to earlier-assigned `env.` entries within subsequent ones.)
install-dir: ${{ github.workspace }}/install/${{ matrix.build-test-cfg.conan-profile-build-type }}/usr/local
# Target file, as read by sanitized executable being invoked.
san-suppress-cfg-file: ${{ github.workspace }}/install/${{ matrix.build-test-cfg.conan-profile-build-type }}/usr/local/bin/san_suppressions.cfg
# Relative path (including file name) to suppressions file in a given context (which is given as dir name off
# which this path works). Possible contexts as of this writing: ${{ github.workspace }}/<module>/src
# (suppressions endemic to lib<module>); ${{ github.workspace }}/.../<test's source code dir>
# (suppressions endemic to that test specifically). This suppressions file holds the compiler-version-independent
# entries. (Malformed and ignored if matrix.build-test-cfg is not `*san`, meaning not sanitizer-enabled.)
san-suppress-cfg-in-file1: sanitize/${{ matrix.build-test-cfg.sanitizer-name }}/suppressions_${{ matrix.compiler.name }}.cfg
# Same but contains compiler version-specific entries (on top of those in san-suppress-cfg-file).
san-suppress-cfg-in-file2: sanitize/${{ matrix.build-test-cfg.sanitizer-name }}/suppressions_${{ matrix.compiler.name }}_${{ matrix.compiler.version }}.cfg
# Run-time controls for various sanitizers. Invoke before running test but *after* assembling suppressions
# file ${{ env.san-suppress-cfg-file}} if any (clear it if needed, as it might be left-over from a previous
# test). The proper technique is: 1, which of the suppression contexts (see above) are relevant?
# (Safest is to specify all contexts; as you'll see just below, it's fine if there are no files in a given
# context. However it would make code tedious to specify that way everywhere; so it's fine to skip contexts where
# we know that these days there are no suppresisons.) Let the contexts' dirs be $DIR_A, $DIR_B, .... Then:
# 2, `{ cat $DIR_A/${{ env.san-suppress-cfg-in-file1 }} $DIR_A/${{ env.san-suppress-cfg-in-file2 }} \
# $DIR_B/${{ env.san-suppress-cfg-in-file1 }} $DIR_B/${{ env.san-suppress-cfg-in-file2 }} \
# ... \
# > ${{ env.san-suppress-cfg-file }} 2> /dev/null; } || true`
# 3, {{ env.setup-tests-env }}.
#
# Notes about items in *SAN_OPTIONS:
#
# Unclear if we need disable_coredump=0; it might be a gcc thing due to historic
# issues with core size with ASAN; it is not documented in the clang *SAN docs for any
# sanitizers; but it is accepted for ASAN and TSAN, seemingly, and maybe is at worst
# harmless with our compiler versions. TODO: Maybe revisit.
#
# print_stacktrace=1 for UBSAN is recommended by documentation for nice stack traces
# (also that is why we apply sanitizers to RelWithDebInfo; DebInfo part for the nice
# stack traces/etc.; Rel part to reduce the considerable slowdown from some of the
# sanitizers).
#
# second_deadlock_stack=1 for TSAN: TODO: Explain. Don't see it in clang TSAN docs.
#
# detect_deadlocks=1 for TSAN: This is already default, but for clarity we make it explicit; read on.
# It detects deadlocks (lock-order inversion) which is useful. However there is a potential problem:
# The detector has an internal limit of 64 held mutexes; once reached some TSAN-internal assertion fails,
# and the program hangs. When needed -- as of this writing with heavy jemalloc use this tends to happen --
# override detect_deadlocks=0 for the duration of the problematic test/demo. (It's a useful check though;
# so this should be done only if unavoidable.)
#
# Caution! Setting multiple *SAN_OPTIONS (while invoking only 1 sanitizer, as we do)
# seems like it would be harmless and avoids the `if/elif`s... but is actually a
# nightmare; there is some interaction in clang (at least 13-17) which causes the
# wrong sanitizer reading suppressions from another one => suppression parse error =>
# none of the demos/tests get anywhere at all. So set just the right one!
#
# Update: ASAN is somewhat different from the others, as it's really two sanitizers with
# separate suppression files -- ASAN (actual safety checks) and LSAN (leak checks). The "Update" is
# adding the LSAN support. Since it was added to Flow's main.yml first, please see notes in the
# similar location in that file. TODO: Probably should have all *SAN-related comments here or all there,
# not most here, some (LSAN) there....
setup-tests-env: |
if [ '${{ matrix.build-test-cfg.sanitizer-name }}' = asan ]; then
export SAN_SUPP=1
export SAN_SUPP_CFG=${{ github.workspace }}/install/${{ matrix.build-test-cfg.conan-profile-build-type }}/usr/local/bin/san_suppressions.cfg
export ASAN_OPTIONS='disable_coredump=0'
export LSAN_OPTIONS="suppressions=$SAN_SUPP_CFG"
echo "ASAN_OPTIONS = [$ASAN_OPTIONS]."
echo "LSAN_OPTIONS = [$LSAN_OPTIONS]."
elif [ '${{ matrix.build-test-cfg.sanitizer-name }}' = ubsan ]; then
export SAN_SUPP=1
export SAN_SUPP_CFG=${{ github.workspace }}/install/${{ matrix.build-test-cfg.conan-profile-build-type }}/usr/local/bin/san_suppressions.cfg
export UBSAN_OPTIONS="disable_coredump=0 print_stacktrace=1 suppressions=$SAN_SUPP_CFG"
echo "UBSAN_OPTIONS = [$UBSAN_OPTIONS]."
elif [ '${{ matrix.build-test-cfg.sanitizer-name }}' = tsan ]; then
export SAN_SUPP=1
export SAN_SUPP_CFG=${{ github.workspace }}/install/${{ matrix.build-test-cfg.conan-profile-build-type }}/usr/local/bin/san_suppressions.cfg
export TSAN_OPTIONS="disable_coredump=0 detect_deadlocks=1 second_deadlock_stack=1 suppressions=$SAN_SUPP_CFG"
if [ '${{ matrix.compiler.id }}' = 'clang-17' ]; then
# clang-17's llvm-symbolizer-17 crashes ("LLVM ERROR: Sections with relocations should have an
# address of 0") when symbolizing our binaries' TSAN reports, stalling the reporting thread 10+
# seconds each time (leading to, e.g., test-internal timeouts). A sibling version's symbolizer
# handles the same binaries fine -- and nothing requires it to version-match the compiler.
for SYMB_VER in 16 18 15; do
if command -v llvm-symbolizer-$SYMB_VER > /dev/null; then
export TSAN_OPTIONS="$TSAN_OPTIONS external_symbolizer_path=$(command -v llvm-symbolizer-$SYMB_VER)"
echo "To avoid clang-17 symbolizer bugginess overrode symbolizer to [llvm-symbolizer-$SYMB_VER]."
break
fi
done
fi
echo "TSAN_OPTIONS = [$TSAN_OPTIONS]."
fi
if [ "$SAN_SUPP" != '' ]; then
echo 'Sanitizer [${{ matrix.build-test-cfg.sanitizer-name }}] suppressions cfg contents:'
echo '[[[ file--'
cat $SAN_SUPP_CFG
echo '--file ]]]'
fi
# Must execute this ahead of any command that might at any point execute any binary built using the
# build-and-test build config. In addition, must then preface any such command with: $BUILT_CMD_PREFIX.
# Rationale: For most situations this is unnecessary, and the script will do nothing, while $BUILT_CMD_PREFIX
# will be empty as a result. It *is* necessary only when the "target" command would execute at least one
# program that was built with TSAN or ASAN sanitizer (TODO: MSAN too? unknown at the moment) flags.
# Unfortunately, in that case, ASLR (virtual address randomization) performed by at least Linux kernel
# can randomly tickle a bug/problem in clang sanitizers (apparently fixed in clang-18 in late 2023) which
# causes a message like `FATAL: ThreadSanitizer: unexpected memory mapping 0x767804872000-0x767804d00000`
# to be immediately printed, with program instantly exiting failingly. This would at least be obvious, if
# it's our actual code (a demo/test) failing: we can see it in the logs; but since we build deps (like jemalloc)
# with the same compile/link settings (e.g., -fsanitizer=thread), the problem can manifest as early as
# the *configure* step of something like jemalloc or m4; so configure will mysteriously fail with messages
# such as `checking size of void *... 0 / configure: error: Unsupported pointer size: 0`. Really, though,
# it is a `configure`-created test C program that errors-out as shown before (can be seen in config.log which
# has to be uploaded as an artifact to even see it... a pain). Anyway! The easiest work-around was
# suggested in https://stackoverflow.com/questions/5194666/disable-randomization-of-memory-addresses:
# disable ASLR by using `setarch` which then invokes whatever command you wanted in the first place.
# This should not affect our test coverage (in fact it perhaps make it more repeatable which is good);
# and the security aspects of ASLR are not a real factor in our context.
#
# TODO: Revisit, particularly when adding more `clang`s and/or other compilers with which TSAN and ASAN
# are enabled. Also see about MSAN (as of this writing it is disabled for orthogonal reasons, so it's moot).
setup-run-env: |
if [ '${{ matrix.build-test-cfg.sanitizer-name }}' = asan ]; then
export BUILT_CMD_PREFIX='setarch -R'
echo 'Shall use `setarch -R` to disable ASLR to avoid ASAN-built binary misbehavior.'
elif [ '${{ matrix.build-test-cfg.sanitizer-name }}' = tsan ]; then
export BUILT_CMD_PREFIX='setarch -R'
echo 'Shall use `setarch -R` to disable ASLR to avoid TSAN-built binary misbehavior.'
fi
# TODO: Ideally would go, like other things, under github.workspace somewhere (maybe build/),
# but that var isn't available in `strategy` up above, where we specify the compiler option
# pointing to this file. Surely we could cook something up; but meanwhile /tmp works fine.
msan-ignore-list-cfg-file: /tmp/msan_ignore_list.cfg
steps:
- name: Update available software list for apt-get
if: env.cell-selected == 'true'
run: |
# Update available software list for apt-get.
lsb_release -a
sudo apt-get update
- name: Install clang compiler
if: |
(env.cell-selected == 'true')
&& matrix.compiler.install && (matrix.compiler.name == 'clang')
run: |
# Install clang compiler.
wget https://apt.llvm.org/llvm.sh
chmod u+x llvm.sh
sudo ./llvm.sh ${{ matrix.compiler.version }}
- name: Install gcc compiler
if: |
(env.cell-selected == 'true')
&& matrix.compiler.install && (matrix.compiler.name == 'gcc')
run: |
# Install gcc compiler.
sudo apt-get install -y software-properties-common
sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y
sudo apt-get update
sudo apt-get install -y gcc-${{ matrix.compiler.version }} g++-${{ matrix.compiler.version }}
# We get highest 1.x, instead of 2+, because there are certain dependency recipe issues (namely for jemalloc)
# with 2+. TODO: 1, expound more clearly here or elsewhere what those issues are; 2, solve those issues and
# thus move to Conan 2+; Conan 1.x is officially on "deprecation path." This will also help resolve some other
# to-dos more easily (targeting different build settings at built product source versus tools used to build it
# = prime example). 3, when doing (2): also update the Conan-cache steps below (and in Flow counterpart) --
# `path` becomes ~/.conan2/p; bump the key prefixes (conan-data- -> conan2-data-), so stale Conan-1 caches
# cannot be restored.
- name: Install the latest version of Conan which is less than 2
if: env.cell-selected == 'true'
run: pip install 'conan<2'
- name: Checkout `ipc` repository and submodules (`flow`, `ipc_*`)
if: env.cell-selected == 'true'
uses: actions/checkout@v6
with:
submodules: true
- name: Add custom settings for Conan packages
if: |
(env.cell-selected == 'true')
&& matrix.build-test-cfg.conan-custom-settings-defs
run: |
# Add custom settings for Conan packages.
conan config init
pip install PyYAML
CONAN_SETTINGS_PATH=$(conan config home)/settings.yml
python -c "
import yaml
with open('$CONAN_SETTINGS_PATH', 'r') as file:
data = yaml.safe_load(file)
${{ matrix.build-test-cfg.conan-custom-settings-defs }}
with open('$CONAN_SETTINGS_PATH', 'w') as file:
yaml.dump(data, file)
"
# Important info/somewhat important TODO: The C[XX]FLAGS and linker-flags are supplied via
# ${{ matrix.build-test-cfg.conan-profile-custom-conf }} and
# ${{ matrix.build-test-cfg.conan-profile-custom-buildenv }}. These affect not just our objects/libs/executables;
# but also at least: 3rd party libs (jemalloc, capnp/kj, boost, gtest), 3rd party executables
# (capnp compiler binary; temp binaries created by auto-tools configure scripts to test features).
# In the case of the libs that's usually good; in particular things like
# -fsanitize=address (ASAN) (for build-type relwithdebinfo-asan) ideally are applied to all built code
# uniformly. (There are other libs being built, one can see: at least openssl and zlib; they're not really
# involved in our actual product, at least not in a core way, so for those it arguably matters less either way.)
# In the case of the binaries it is usually bad; we want the fastest, most normal tools possible, not ones
# built weirdly with whatever settings we're trying to test with our own software. For one it makes those
# tools slower -- perhaps not the hugest deal in practice here -- but it also generally increases entropy; and
# it has also resulted in various pain:
# - Cannot use -fno-sanitize-recover with UBSAN (abort program on warning) if we wanted to: it causes some
# hidden configure-script-generated binary abort => capnp binary build fails.
# - capnp binary fails MSAN at startup; hence capnp-compilation of our .capnp schemas fails.
# We have worked around all these, so the thing altogether works. It's just somewhat odd and entropy-ridden;
# and might cause maintanability problems over time, as it has already in the past.
# The TODO is to be more judicious about it
# and only apply these things to the libs/executables we want it applied. It is probably not so simple;
# but worst-case it should be possible to use something like build-type-cflags-override to target our code;
# and per-recipe (Boost, jemalloc, gtest...) techniques to target others; and not use the aggressive
# conan-profile-custom-conf and conan-profile-custom-buildenv. That said, there will be interesting subtleties
# like: libcapnp/kj are linked into capnp compiler binary; *and* to our code. So we should instrument it with
# sanitizer (when applicable); now some of the capnp-compiler-binary is instrumented; some isn't. Would that
# work, or would it be better to build capnp twice then (once for the binary, once for the linked libraries)?
- name: Create Conan profile
if: env.cell-selected == 'true'
run: |
# Create Conan profile.
cat <<'EOF' > conan_profile
[settings]
compiler = ${{ matrix.compiler.name }}
compiler.version = ${{ matrix.compiler.version }}
compiler.cppstd = ${{ ((matrix.cxx-std.id == 'cxx20') && '20') || '17' }}
# TODO: Consider testing with LLVM-libc++ also (with clang anyway).
compiler.libcxx = libstdc++11
arch = x86_64
os = Linux
jemalloc:build_type = ${{ matrix.build-test-cfg.conan-profile-jemalloc-build-type }}
build_type = ${{ matrix.build-test-cfg.conan-profile-build-type }}
${{ matrix.build-test-cfg.conan-profile-custom-settings }}
[conf]
tools.env.virtualenv:auto_use = True
tools.build:compiler_executables = {"c": "${{ matrix.compiler.c-path }}", "cpp": "${{ matrix.compiler.cpp-path }}"}
${{ matrix.build-test-cfg.conan-profile-custom-conf }}
[buildenv]
CC = ${{ matrix.compiler.c-path }}
CXX = ${{ matrix.compiler.cpp-path }}
${{ matrix.build-test-cfg.conan-profile-custom-buildenv }}
[options]
# TODO: Are the next 2 lines needed, since we are building via `ipc` conanfile.py?
# I (ygoldfel) am admittedly Conan-ignorant.
flow:doc = False
flow:build = True
ipc:doc = False
ipc:build = True
# `0` here as of this writing would effectively mean `17` too. Specifying `17` explicitly has the same
# effect in the end but goes down a slightly different code-path in FlowLikeCodeGenerate.cmake. It's not
# a huge deal... but it's a little nice to cover more paths, so let's do the `0` path when applicable.
ipc:build_cxx_std = ${{ ((matrix.cxx-std.id == 'cxx20') && '20') || '0' }}
EOF
# (Oddly enough `no-lto: True` and `...: False` do still evaluate as `true` and `false` respectively.
# Hence why this compares against `false` and not `False`....)
if [ '${{ matrix.build-test-cfg.no-lto }}' != '' ] && [ '${{ matrix.build-test-cfg.no-lto }}' != 'false' ]; then
echo 'ipc:build_no_lto = True' >> conan_profile
fi
if [ '${{ matrix.build-test-cfg.build-type-cflags-override }}' != '' ]; then
echo 'ipc:build_type_cflags_override = ${{ matrix.build-test-cfg.build-type-cflags-override }}' >> conan_profile
fi
# high-mem-build config x high-mem-build compiler: several jumbo test
# translation units (.cpp files) compiling in parallel, at the default
# parallelism (= logical cores), can exceed runner RAM; observed as sporadic whole-VM-OOM
# (runner-shutdown) job kills. So use physical cores instead: measured (via `free` sampling) to leave
# ample headroom while costing only the SMT throughput bonus (perhaps 15-25% of the compile phase).
# TODO: Splitting up heavy test `.cpp`s into less memory-hungry chunks, such as by instantiating certain
# key template groups (<=> ipc::session::Session compile-time config, possibly others) in separate
# `.cpp`s, would benefit other environments, including user environments, and be less reliant on fairly
# dynamic hardware conditions from GitHub/whomever. Revisit that; ideally get rid of this as a result.
# Update: The below trick is likely insufficient (from numbers we've seen), so we split the handful of
# heaviest unit_test `.cpp`s per this to-do. Combined with the below trick (but not without it, notably),
# 8/2026 GitHub build was OK. The to-do remains: Ideally split more stuff, so that this can be removed
# (among other benefits). transport_test is likely the guy to attack.
# XXXrevisit soon, if there's time; or delete this line and leave TODO; whatever seems best at the end.
if [ '${{ matrix.compiler.high-mem-build }}' != '' ] \
&& [ '${{ matrix.compiler.high-mem-build }}' != 'false' ] \
&& [ '${{ matrix.build-test-cfg.high-mem-build }}' != '' ] \
&& [ '${{ matrix.build-test-cfg.high-mem-build }}' != 'false' ]; then
N_PHYS_CORES=$(lscpu -p=Core,Socket | grep -v '^#' | sort -u | wc -l)
echo "Reducing build parallelism to N-physical-cores [$N_PHYS_CORES]."
# Must land inside the `[conf]` section. (A plain append would land in `[options]`, the file's
# last section, where it would parse as never-matching-any-package option `tools.build:jobs` --
# a silent no-op.)
sed -i "/^\[conf\]$/a tools.build:jobs = $N_PHYS_CORES" conan_profile
fi
# We need to prepare a sanitizer ignore-list in MSAN mode. Background for this is subtle and annoying:
# As it stands, whatever matrix compiler/build-type is chosen applies not just to our code (correct)
# and 3rd party libraries we link like lib{boost_*|capnp|kj|jemalloc} (semi-optional but good) but also
# unfortunately any items built from source during "Install Flow-IPC dependencies" step that we then
# use during during the build step for our own code subsequently. At a minimum this will slow down
# such programs. (For the time being we accept this as not-so-bad; to target this config at some things
# but not others is hard/a ticket.) In particular, though, the capnp compiler binary is built this way;
# and our "Build targets" step uses it to build key things (namely convert .capnp schemas into .c++
# and .h sources which are then themselves compiled/used in compilation). In the case of MSAN, this
# version of capnp compiler happens to trigger several MSAN failures (presumably they are not a true
# problem, and it's not our job really to sanitize capnp -- though we can file tickets for them and/or
# issue PRs; but I digress). So in MSAN mode our build step fails, when capnp compiler itself aborts
# with MSAN failures. One approach is to not apply MSAN to capnp compiler (no-go for now; it's a ticket
# as mentioned); another is to uses Conan to patch capnp package for them (not a bad idea; ticket filed);
# and lastly we can put the specific failures on the ignore-list for MSAN. For now we do that; while
# unpleasant it does get the job done. TODO: Revisit/resolve tickets/improve (see above).
- name: Prepare MSAN sanitizer compile-time config file(s)
if: |
(env.cell-selected == 'true')
&& (!cancelled()) && (matrix.build-test-cfg.sanitizer-name == 'msan')
run: |
# Prepare MSAN sanitizer compile-time config file(s).
cat <<'EOF' > $${ env.msan-ignore-list-cfg-file }}
[memory]
# Warning: In clang-18 there are breaking changes in how globs/regexes are interpreted. See docs.
# Currently assuming clang-17 or lower.
#
# capnp compiler MSAN failures suppressed:
src:*/kj/filesystem-disk-unix.*
src:*/bits/stl_tree.h
EOF
# Append to that the suppressed items from the source tree. These apply to the real code being
# built below, so just stylistically it is nicer to keep them there along with other
# sanitizers' suppressions. (Note, though, that MSAN does not have a run-time suppression
# system; only these ignore-lists. The others do also have ignore-lists though.
# The format is totally different between the 2 types of suppression.)
# Our MSAN support is budding compared to UBSAN/ASAN/TSAN; so just specify the one ingore-list file
# we have now. TODO: If/when MSAN support gets filled out like the others', then use a context system
# a-la env.setup-tests-env.
cat ${{ github.workspace }}/flow/src/sanitize/msan/ignore_list_${{ matrix.compiler.name }}.cfg \
>> $${ env.msan-ignore-list-cfg-file }}
echo 'The combined MSAN ignore-list config file follows:'
cat $${ env.msan-ignore-list-cfg-file }}
# Cache the Conan-built dependencies (Boost et al): otherwise a fresh runner rebuilds them from source every
# run. See explanatory comments atop the analogous steps in Flow workflow counterpart. We use same
# techniques (including the restore/save split, with the save placed right after the dep-install step, so
# even a run whose later steps fail still primes the cache).
#
# Caution! The restore-keys fallback below can restore binaries built under *different* compile/link
# flags than the current profile's -- and Conan cannot detect that (its package-id ignores flags arriving
# via [conf]/[buildenv]; only [settings]/[options] are keyed). The flag-snapshot guard in the dep-install
# step below is what makes the fallback safe; if you remove or change one, mind the other.
- name: Restore Conan package cache
if: env.cell-selected == 'true'
id: conan-cache-restore
uses: actions/cache/restore@v6
with:
path: ~/.conan/data
key: conan-data-${{ matrix.compiler.id }}-${{ matrix.build-test-cfg.id }}-${{ matrix.cxx-std.id }}-${{ hashFiles('conan_profile', 'conanfile.py') }}
restore-keys: |
conan-data-${{ matrix.compiler.id }}-${{ matrix.build-test-cfg.id }}-${{ matrix.cxx-std.id }}-
- name: Install Flow-IPC dependencies with Conan using the profile
if: env.cell-selected == 'true'
run: |
# Install Flow-IPC dependencies with Conan using the profile.
${{ env.setup-run-env }}
# Flag-snapshot guard (see conan-cache-restore comment above). An exact-key restore is
# trusted by definition: the key hashes the whole profile. Otherwise -- fallback restore, or none --
# compare the flag-containing profile sections against the snapshot recorded inside the cache (see end of
# this step); on mismatch (or no snapshot) evict everything, so dependencies rebuild with current flag values.
# (Do this before the `conan export` just below: the eviction would delete its result too.)
if [ '${{ steps.conan-cache-restore.outputs.cache-hit }}' != 'true' ]; then
flag_sections() { awk '/^\[/{on=0} /^\[(conf|buildenv)\]/{on=1} on' "$1"; }
SNAPSHOT=~/.conan/data/.profile-snapshot
if [ ! -f $SNAPSHOT ] || ! diff <(flag_sections conan_profile) <(flag_sections $SNAPSHOT); then
echo 'Restored cache (if any) lacks a flag-match guarantee; evicting its packages (they will rebuild).'
conan remove --force '*' 2> /dev/null || true
fi
fi
# Next 3 lines: Must do same as flow's main.yml; see that code.
# TODO: See the TODO there which applies to these lines.
NON_REMOTE_BOOST_VER=1.87
conan export ./flow/conan/boost/$NON_REMOTE_BOOST_VER boost/$NON_REMOTE_BOOST_VER.0@
FLOW_VERSION=`cat flow/VERSION`
conan editable add flow flow/$FLOW_VERSION
# At least `configure` mini-programs are built with our build-settings, so $BUILT_CMD_PREFIX is required.
$BUILT_CMD_PREFIX \
conan install . \
--profile:build conan_profile --profile:host conan_profile --build missing || FAILED=yep
# Dep build failure is easy to diagnose from stdout/err, unless it happens during a `configure`;
# then it'll often print something insane like "checking size of void *... 0" and suggest looking at
# config.log... which will usually reveal the true problem, like a seg-fault or TSAN-triggered problem
# running the configure-generated test program. So let's get all the `config.log`s we can find
# and put them where later the log artifact tarball will be sourced; recreate the same dir structure as
# in the Conan dep build area, but include only the `config.log` files. Note: This is probably only truly
# useful if indeed the above command fails; but can't hurt to include it regardless.
SRC=/home/runner/.conan/data
DST=${{ env.install-dir }}/bin/logs/dep_build/conan-data
find $SRC -name config.log | while read -r config_log; do
# Remove the $SRC part of the path to get a relative path.
REL_PATH="${config_log#$SRC/}"
# Determine the directory path within $DST where the config.log should be copied.
DST_DIR="$DST/${REL_PATH%/*}"
mkdir -p "$DST_DIR"
cp -v "$config_log" "$DST_DIR"
done
# Prune the source/build trees, which Conan retains after building from source; only the
# built packages and recipes are needed by this (or later) runs. Keeps cache size down in next step;
# cache is LRU-based but limited enough to matter in practice as of this writing.
conan remove '*' --builds --src --force
if [ "$FAILED" != yep ]; then
# Record the flags this cache's contents were built under, for the flag-snapshot guard (see above).
cp conan_profile ~/.conan/data/.profile-snapshot
fi
[ "$FAILED" != yep ]
# (See comment atop the Restore counterpart above.) Skipped on an exact-key restore: the cache would be
# bit-identical to what is already stored.
- name: Save Conan package cache
if: |
(env.cell-selected == 'true')
&& (steps.conan-cache-restore.outputs.cache-hit != 'true')
uses: actions/cache/save@v6
with:
path: ~/.conan/data
key: ${{ steps.conan-cache-restore.outputs.cache-primary-key }}
- name: Build libraries and demos/tests with Conan
if: env.cell-selected == 'true'
run: |
# Build libraries and demos/tests with Conan.
${{ env.setup-run-env }}
# At least capnp compiler binary is built with our build-settings, so $BUILT_CMD_PREFIX is required.
$BUILT_CMD_PREFIX conan build .
- name: Install built targets with Makefile
if: env.cell-selected == 'true'
run: |
make install \
--directory ${{ env.build-dir }} DESTDIR=${{ env.install-root-dir }}
# Save runner space: blow away build dir after install.
rm -rf ${{ env.build-dir }}
# From now on use !cancelled() to try to run any test/demo that exists regardless
# of preceding failures if any. Same-ish (always()) with the log-upload at the end.
# Worst-case they'll all fail in super-basic ways and immediately; no extra harm done.
# From now on save logs in install-dir/bin/logs. They will be tarred up and uploaded
# as artifacts at the end. For those tests below that produce separate logs as files
# to begin-with, this is a no-brainer. For those (as of this writing the main one
# is unit_test; but also the link tests produce short logs too) who use stdout/stderr
# the reason to do this is 2-fold.
# - UBSAN in particular produces non-fatal error output about problems it detects.
# We need to analyze them after the fact and fail a step below in that case to
# alert developers who would then fix such new problems.
# - One can also build UBSAN-mode things with -fno-sanitize-recover which would
# cause abnormal program exit on *first* error. We do not do this for these
# reasons:
# - It is more convenient to let it continue and thus show all problems in one
# shot.
# - There is a side problem: At this time compile settings, including
# in UBSAN case `-fsanitize=undefined -fno-sanitize-recover`, are applied not
# just to our code or 3rd party libraries but also any other stuff built
# (due to setting C[XX]FLAGS in Conan profile). Targeting just the exact
# stuff we want with those is hard and a separate project/ticket.
# In the meantime -fno-sanitize-recover causes completely unrealted program
# halts during the very-early step, when building dependencies including
# capnp; some autotools configure.sh fails crazily, and nothing can work
# from that point on due to dependencies-install step failing. So at this
# time -fno-sanitize-recover is a no-go; it affects too much unrelated stuff.
# - It is more consistent/convenient to get all the bulky logs in one place as an
# artifact, rather than some in the pipeline output, others in the artifacts.
# (This preference is subjective, yes.)
# Here, as in all other tests below, we assemble a suppressions file in case this is a sanitized
# run; and we follow the procedure explained near setup-tests-env definition. To reiterate: to avoid
# tedium, but at the cost of mantainability of this file (meaning if a suppressions context is added then
# a few lines would need to be added here), we only list those contexts where *any* sanitizer has
# *any* suppression; otherwise we skip it for brevity. `find . -name 'suppressions*.cfg` is pretty useful
# to determine their presence in addition to whether the test itself has its specific suppressions of any kind.
- name: Run link test [`ipc_core` - Flow-IPC Core]
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: |
# Run link test [`ipc_core` - Flow-IPC Core].
cd ${{ env.install-dir }}/bin
OUT_DIR=logs/ipc_core_link_test
mkdir -p $OUT_DIR
SUPP_DIR_A=${{ github.workspace }}/flow/src
{ cat $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file2 }} \
> ${{ env.san-suppress-cfg-file }} 2> /dev/null; } || true
${{ env.setup-tests-env }}
${{ env.setup-run-env }}
$BUILT_CMD_PREFIX \
./ipc_core_link_test.exec $OUT_DIR/log.log > $OUT_DIR/console.log 2>&1
- name: Run link test [`ipc_transport_structured` - Flow-IPC Structured Transport]
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: |
# Run link test [`ipc_transport_structured` - Flow-IPC Structured Transport].
cd ${{ env.install-dir }}/bin
OUT_DIR=logs/ipc_transport_structured_link_test
mkdir -p $OUT_DIR
SUPP_DIR_A=${{ github.workspace }}/flow/src
{ cat $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file2 }} \
> ${{ env.san-suppress-cfg-file }} 2> /dev/null; } || true
${{ env.setup-tests-env }}
${{ env.setup-run-env }}
$BUILT_CMD_PREFIX \
./ipc_transport_structured_link_test.exec $OUT_DIR/log.log > $OUT_DIR/console.log 2>&1
- name: Run link test [`ipc_session` - Flow-IPC Sessions]
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: |
# Run link test [`ipc_session` - Flow-IPC Sessions].
cd ${{ env.install-dir }}/bin
OUT_DIR=logs/ipc_session_link_test
mkdir -p $OUT_DIR
SUPP_DIR_A=${{ github.workspace }}/flow/src
SUPP_DIR_B=${{ github.workspace }}/ipc_session/src
{ cat $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file2 }} \
$SUPP_DIR_B/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_B/${{ env.san-suppress-cfg-in-file2 }} \
> ${{ env.san-suppress-cfg-file }} 2> /dev/null; } || true
${{ env.setup-tests-env }}
${{ env.setup-run-env }}
$BUILT_CMD_PREFIX \
./ipc_session_link_test_srv.exec $OUT_DIR/srv.log > $OUT_DIR/srv.console.log 2>&1 &
SRV_PID=$!
sleep 5
$BUILT_CMD_PREFIX \
./ipc_session_link_test_cli.exec $OUT_DIR/cli.log > $OUT_DIR/cli.console.log 2>&1
if wait $SRV_PID; then SRV_EC=0; else SRV_EC=$?; fi
[ $SRV_EC -eq 0 ]
- name: Run link test [`ipc_shm` - Flow-IPC Shared Memory]
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: |
# Run link test [`ipc_shm` - Flow-IPC Shared Memory].
cd ${{ env.install-dir }}/bin
OUT_DIR=logs/ipc_shm_link_test
mkdir -p $OUT_DIR
SUPP_DIR_A=${{ github.workspace }}/flow/src
SUPP_DIR_B=${{ github.workspace }}/ipc_session/src
{ cat $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file2 }} \
$SUPP_DIR_B/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_B/${{ env.san-suppress-cfg-in-file2 }} \
> ${{ env.san-suppress-cfg-file }} 2> /dev/null; } || true
${{ env.setup-tests-env }}
${{ env.setup-run-env }}
$BUILT_CMD_PREFIX \
./ipc_shm_link_test_srv.exec $OUT_DIR/srv.log > $OUT_DIR/srv.console.log 2>&1 &
SRV_PID=$!
sleep 5
$BUILT_CMD_PREFIX \
./ipc_shm_link_test_cli.exec $OUT_DIR/cli.log > $OUT_DIR/cli.console.log 2>&1
if wait $SRV_PID; then SRV_EC=0; else SRV_EC=$?; fi
[ $SRV_EC -eq 0 ]
- name: Run link test [`ipc_shm_arena_lend` - Flow-IPC SHM-jemalloc]
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: |
# Run link test [`ipc_shm_arena_lend` - Flow-IPC SHM-jemalloc].
cd ${{ env.install-dir }}/bin
OUT_DIR=logs/ipc_shm_arena_lend_link_test
mkdir -p $OUT_DIR
SUPP_DIR_A=${{ github.workspace }}/flow/src
SUPP_DIR_B=${{ github.workspace }}/ipc_session/src
SUPP_DIR_C=${{ github.workspace }}/ipc_shm_arena_lend/src
{ cat $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file2 }} \
$SUPP_DIR_B/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_B/${{ env.san-suppress-cfg-in-file2 }} \
$SUPP_DIR_C/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_C/${{ env.san-suppress-cfg-in-file2 }} \
> ${{ env.san-suppress-cfg-file }} 2> /dev/null; } || true
${{ env.setup-tests-env }}
${{ env.setup-run-env }}
$BUILT_CMD_PREFIX \
./ipc_shm_arena_lend_link_test_srv.exec $OUT_DIR/srv.log > $OUT_DIR/srv.console.log 2>&1 &
SRV_PID=$!
sleep 5
$BUILT_CMD_PREFIX \
./ipc_shm_arena_lend_link_test_cli.exec $OUT_DIR/cli.log > $OUT_DIR/cli.console.log 2>&1
if wait $SRV_PID; then SRV_EC=0; else SRV_EC=$?; fi
[ $SRV_EC -eq 0 ]
- name: Run unit tests
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: |
# Run unit tests.
# XXX Temporary; see test-subset input.
if [ '${{ github.event.inputs.test-subset }}' = 't-t-only' ]; then echo 'Skipped per test-subset.'; exit 0; fi
cd ${{ env.install-dir }}/bin
# Some newline issues with the possible additional args; so need to make a wrapper script
# and then redirect, as desired, its output.
cat <<'EOF' > ${{ env.install-dir }}/bin/run_unit_test.sh
$BUILT_CMD_PREFIX ./libipc_unit_test.exec "$@"
EOF
RUN_IT='/usr/bin/bash -e ${{ env.install-dir }}/bin/run_unit_test.sh'
OUT_DIR=logs/libipc_unit_test
mkdir -p $OUT_DIR
SUPP_DIR_A=${{ github.workspace }}/flow/src
SUPP_DIR_B=${{ github.workspace }}/ipc_session/src
SUPP_DIR_C=${{ github.workspace }}/ipc_shm_arena_lend/src
# As of this writing there are TSAN suppressions for this test specifically. TODO: Revisit them; and then this.
SUPP_DIR_OWN=${{ github.workspace }}/test/suite/unit_test
{ cat $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file2 }} \
$SUPP_DIR_B/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_B/${{ env.san-suppress-cfg-in-file2 }} \
$SUPP_DIR_C/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_C/${{ env.san-suppress-cfg-in-file2 }} \
$SUPP_DIR_OWN/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_OWN/${{ env.san-suppress-cfg-in-file2 }} \
> ${{ env.san-suppress-cfg-file }} 2> /dev/null; } || true
${{ env.setup-tests-env }}
${{ env.setup-run-env }}
if [ '${{ matrix.build-test-cfg.sanitizer-name }}' != tsan ]; then
# This is what we want to do normally.
$RUN_IT > $OUT_DIR/console.log 2>&1
else # if sanitizer-name is tsan:
# First please see comment in setup-tests-env regarding detect_deadlocks=1 (the default) and the
# potential problem in some conditions. In short: heavy jemalloc use => d_d=1 trips 64-mutex cap in TSAN
# => hang. As noted in that comment: we keep the default where possible (no heavy [SHM-]jemalloc);
# temporarily disable detect_deadlocks otherwise.
#
# If some future test ends up in the wrong batch, `timeout` causes an explicit failure without waiting
# for the whole job to time out.
# TODO: Failing to add a test here should already show a timeout... nevertheless this
# setup just is a bit flimsy-feeling. Ideally this list would be auto-generated from some
# mandatory-to-maintain list in the (test/) source somewhere; like maybe any "heavy jemalloc user" test
# would be marked in some consistent way.
SHMJ_TESTS='*Ipc_arena*:*Jemalloc*:*Memory_manager*:*Shm_pool*:*shm_pool*:*Shm_session*:*Lend*:\
*Borrower*:*Owner*:Thread_cache_test.*:Session_persistent_cleanup_test.*:\
*shmJemalloc*:*ShmJemalloc*:*shm_jemalloc*'
# Get rid of newlines/backslashes....
SHMJ_TESTS="$(echo "$SHMJ_TESTS" | tr -d '[:space:]\\')"
if timeout 30m $RUN_IT --gtest_filter=-$SHMJ_TESTS > $OUT_DIR/console.minus-shm-j.log 2>&1; \
then EC_A=0; else EC_A=$?; fi
# (In the sanitizer-flags mini-language a repeated flag = last occurrence wins.)
if TSAN_OPTIONS="$TSAN_OPTIONS detect_deadlocks=0" \
timeout 30m $RUN_IT --gtest_filter=$SHMJ_TESTS > $OUT_DIR/console.shm-j.log 2>&1; \
then EC_B=0; else EC_B=$?; fi
echo "Exit codes: all-minus-shm-j = $EC_A; shm-j = $EC_B. (124 = timeout -- probably a test in the"
echo 'wrong batch; see comment above.)'
[ $EC_A -eq 0 ] && [ $EC_B -eq 0 ]
fi
- name: Prepare run script for [perf_demo] variations below
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: |
# Prepare run script for [perf_demo] variations below.
cat <<'EOF' > ${{ env.install-dir }}/bin/run_perf_demo.sh
# Script created by pipeline during job.
echo "Zero-copy backing type: [$1]."
cd ${{ env.install-dir }}/bin
OUT_DIR=logs/perf_demo/$1
mkdir -p $OUT_DIR
SUPP_DIR_A=${{ github.workspace }}/flow/src
SUPP_DIR_B=${{ github.workspace }}/ipc_session/src
SUPP_DIR_C=${{ github.workspace }}/ipc_shm_arena_lend/src
{ cat $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file2 }} \
$SUPP_DIR_B/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_B/${{ env.san-suppress-cfg-in-file2 }} \
$SUPP_DIR_C/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_C/${{ env.san-suppress-cfg-in-file2 }} \
> ${{ env.san-suppress-cfg-file }} 2> /dev/null; } || true
${{ env.setup-tests-env }}
${{ env.setup-run-env }}
$BUILT_CMD_PREFIX \
./perf_demo_srv_$1.exec 1000 $OUT_DIR/srv.log > $OUT_DIR/srv.console.log 2>&1 &
SRV_PID=$!
sleep 40 # It fills up a large structure before listening, so give it some time (mostly for slow build types).
$BUILT_CMD_PREFIX \
./perf_demo_cli_$1.exec $OUT_DIR/cli.log > $OUT_DIR/cli.console.log 2>&1
if wait $SRV_PID; then SRV_EC=0; else SRV_EC=$?; fi
[ $SRV_EC -eq 0 ]
EOF
- name: Run integration test [perf_demo - SHM-classic mode]
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: /usr/bin/bash -e ${{ env.install-dir }}/bin/run_perf_demo.sh shm_classic
- name: Run integration test [perf_demo - SHM-jemalloc mode]
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: /usr/bin/bash -e ${{ env.install-dir }}/bin/run_perf_demo.sh shm_jemalloc
# For tests below where on failure it can be very helpful to look at higher-verbosity logs,
# any failing step is repeated with logging hiked up. Note that formally (and practically)
# higher-than-INFO verbosity is allowed to affect performance and thus is *not* right for
# the main integration test run. However on failure all bets are off, and we just want the
# info we can get. (Also higher-verbosity runs can take significantly longer; no one wants that.)
#
# As of this writing unit_test already runs with high verbosity, so we do not re-run on failure
# like these. The `link_test`s above are very basic tests, essentially there to ensure all built okay
# and run a sanity-check of a core feature of each of ours libs; so no re-run on failure there either.
# Lastly perf_demo is about performance and as of this writing happen to lack that knob; so
# perhaps TODO: Add the verbosity knob and the re-run-on-failure step for that one. It's about perf though,
# so does not seem like a high priority.
# This follows the instructions in bin/transport_test/README.txt.
- name: Prepare run script for [transport_test - Scripted mode] variations below
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: |
# Prepare run script for [transport_test - Scripted mode] variations below.
cat <<'EOF' > ${{ env.install-dir }}/bin/run_transport_test_sc.sh
# XXX Temporary; see test-subset input.
if [ '${{ github.event.inputs.test-subset }}' = 'u-t-only' ]; then echo 'Skipped per test-subset.'; exit 0; fi
echo "Log level: [$1]."
cd ${{ env.install-dir }}/bin/transport_test
OUT_DIR_NAME=log_level_$1
OUT_DIR=../logs/transport_test/scripted/$OUT_DIR_NAME
mkdir -p $OUT_DIR
SUPP_DIR_A=${{ github.workspace }}/flow/src
# In scripted mode ipc_session, while linked, is not invoked in any way; so skip its suppressions.
{ cat $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file2 }} \
> ${{ env.san-suppress-cfg-file }} 2> /dev/null; } || true
${{ env.setup-tests-env }}
${{ env.setup-run-env }}
$BUILT_CMD_PREFIX \
./transport_test.exec scripted $OUT_DIR/srv.log info $1 \
< srv-script.txt > $OUT_DIR/srv.console.log 2>&1 &
SRV_PID=$!
sleep 5
$BUILT_CMD_PREFIX \
./transport_test.exec scripted $OUT_DIR/cli.log info $1 \
< cli-script.txt > $OUT_DIR/cli.console.log 2>&1 &
CLI_PID=$!
if wait $SRV_PID; then SRV_EC=0; else SRV_EC=$?; fi
echo "Server finished with code [$SRV_EC]."
if wait $CLI_PID; then CLI_EC=0; else CLI_EC=$?; fi
echo "Client finished with code [$CLI_EC]."
[ $SRV_EC -eq 0 ] && [ $CLI_EC -eq 0 ]
EOF
- name: Run integration test [transport_test - Scripted mode]
id: transport_test_scripted
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: /usr/bin/bash -e ${{ env.install-dir }}/bin/run_transport_test_sc.sh info
- name: Re-run with increased logging, on failure only
if: |
(env.cell-selected == 'true')
&& (!cancelled()) && (steps.transport_test_scripted.outcome == 'failure')
run: /usr/bin/bash -e ${{ env.install-dir }}/bin/run_transport_test_sc.sh data
# The following [Exercise mode] tests follow the instructions in bin/transport_test/README.txt.
# Note that the creation of ~/bin/ex_..._run and placement of executables there, plus
# /tmp/var/run for run-time files (PID files and similar), is a necessary consequence of
# the ipc::session safety model for estabshing IPC conversations (sessions).
- name: Prepare IPC-session safety-friendly run-time environment for [transport_test - Exercise mode]
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: |
# Prepare IPC-session safety-friendly run-time environment for [transport_test - Exercise mode].
mkdir -p ~/bin/ex_srv_run ~/bin/ex_cli_run
mkdir -p /tmp/var/run
cp -v ${{ env.install-dir }}/bin/transport_test/transport_test.exec \
~/bin/ex_srv.exec
cp -v ~/bin/ex_srv.exec ~/bin/ex_cli.exec
- name: Prepare run script for [transport_test - Exercise mode] variations below
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: |
# Prepare run script for [transport_test - Exercise mode] variations below.
cat <<'EOF' > ${{ env.install-dir }}/bin/run_transport_test_ex.sh
# Script created by pipeline during job.
# XXX Temporary; see test-subset input.
if [ '${{ github.event.inputs.test-subset }}' = 'u-t-only' ]; then echo 'Skipped per test-subset.'; exit 0; fi
echo "Log level: [$1]."
echo "Exercise sub-mode: [$2]."
echo "Sub-mode snippet (none or '-shm-?'): [$3]."
OUT_DIR=${{ env.install-dir }}/bin/logs/transport_test/exercise/$2
mkdir -p $OUT_DIR
SUPP_DIR_A=${{ github.workspace }}/flow/src
if [ "$3" == '-shm-j' ]; then
# SHM-jemalloc mode => jemalloc is in fact exercised => suppress libjemalloc stuff.
# But, nicely, no transport_test-specific suppressions needed as of this writing.
SUPP_DIR_B=${{ github.workspace }}/ipc_shm_arena_lend/src
elif [ "$3" == '-shm-c' ]; then
SUPP_DIR_B=${{ github.workspace }}/test/suite/transport_test/shm-c
else # if [ "$3" == '' ]; then
SUPP_DIR_B=${{ github.workspace }}/test/suite/transport_test/heap
fi
SUPP_DIR_C=${{ github.workspace }}/ipc_session/src # Session stuff invoked regardless of sub-mode.
{ cat $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_A/${{ env.san-suppress-cfg-in-file2 }} \
$SUPP_DIR_B/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_B/${{ env.san-suppress-cfg-in-file2 }} \
$SUPP_DIR_C/${{ env.san-suppress-cfg-in-file1 }} $SUPP_DIR_C/${{ env.san-suppress-cfg-in-file2 }} \
> ${{ env.san-suppress-cfg-file }} 2> /dev/null; } || true
${{ env.setup-tests-env }}
${{ env.setup-run-env }}
$BUILT_CMD_PREFIX \
~/bin/ex_srv.exec exercise-srv$3 $OUT_DIR/srv.log info $1 \
> $OUT_DIR/srv.console.log 2>&1 &
SRV_PID=$!
sleep 5
$BUILT_CMD_PREFIX \
~/bin/ex_cli.exec exercise-cli$3 $OUT_DIR/cli.log info $1 \
> $OUT_DIR/cli.console.log 2>&1 &
CLI_PID=$!
if wait $SRV_PID; then SRV_EC=0; else SRV_EC=$?; fi
echo "Server finished with code [$SRV_EC]."
if wait $CLI_PID; then CLI_EC=0; else CLI_EC=$?; fi
echo "Client finished with code [$CLI_EC]."
[ $SRV_EC -eq 0 ] && [ $CLI_EC -eq 0 ]
EOF
- name: Run integration test [transport_test - Exercise mode - Heap sub-mode]
id: transport_test_ex_heap
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: /usr/bin/bash -e ${{ env.install-dir }}/bin/run_transport_test_ex.sh info heap
- name: Re-run with increased logging, on failure only
if: |
(env.cell-selected == 'true')
&& (!cancelled()) && (steps.transport_test_ex_heap.outcome == 'failure')
run: /usr/bin/bash -e ${{ env.install-dir }}/bin/run_transport_test_ex.sh data heap_log_level_data
- name: Run integration test [transport_test - Exercise mode - SHM-classic sub-mode]
id: transport_test_ex_shm_c
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: /usr/bin/bash -e ${{ env.install-dir }}/bin/run_transport_test_ex.sh info shm_classic -shm-c
- name: Re-run with increased logging, on failure only
if: |
(env.cell-selected == 'true')
&& (!cancelled()) && (steps.transport_test_ex_shm_c.outcome == 'failure')
run: /usr/bin/bash -e ${{ env.install-dir }}/bin/run_transport_test_ex.sh data shm_classic_log_level_data -shm-c
# clang-17 + TSAN only:
# In this config this run would semi-reliably (50%+) fail: TSAN reports during the run
# invoke llvm-symbolizer-17 which crashes on our binaries ("LLVM ERROR: Sections with relocations should
# have an address of 0" -- an LLVM bug; ticket filed), stalling the reporting thread 10+ seconds each
# time; and a certain giant blocking operation in a particular thread then delays certain processing,
# causing an internal timeout, and the test exits/fails. Worked around by symbolizing with a sibling
# llvm-symbolizer version in that config; see `setup-tests-env`. If this becomes insufficient, or some other
# such problem arises, it is defensible to disable this for the offending compiler, as there are as of this
# writing a few TSAN configs that would remain (different compiler version(s)).
- name: Run integration test [transport_test - Exercise mode - SHM-jemalloc sub-mode]
id: transport_test_ex_shm_j
if: |
(env.cell-selected == 'true')
&& (!cancelled())
run: /usr/bin/bash -e ${{ env.install-dir }}/bin/run_transport_test_ex.sh info shm_jemalloc -shm-j
- name: Re-run with increased logging, on failure only
if: |
(env.cell-selected == 'true')
&& (!cancelled()) && (steps.transport_test_ex_shm_j.outcome == 'failure')
run: /usr/bin/bash -e ${{ env.install-dir }}/bin/run_transport_test_ex.sh data shm_jemalloc_log_level_data -shm-j
# See earlier comment block about why we saved all the logs including for console-output-only tests/demos.
- name: Check test/demo logs for non-fatal sanitizer error(s)
if: |
(env.cell-selected == 'true')
&& (!cancelled()) && (matrix.build-test-cfg.sanitizer-name == 'ubsan')
run: |
cd ${{ env.install-dir }}/bin/logs
# grep returns 0 if 1+ found, 1 if none found, 2+ on error. So check results explicitly instead of -e.
# Namely, for us, the only OK result is an empty stdout and empty stderr.
# Otherwise either grep failed (unlikely) or found 1+ problems; either way redirection target will
# be not-empty, and we will force failure.
{ grep 'SUMMARY: UndefinedBehaviorSanitizer:' `find . -type f` > san_failure_summaries.txt 2>&1; } || true
if [ -s san_failure_summaries.txt ]; then
echo 'Error(s) found. Pipeline will fail. Failures summarized below.'
echo 'Please peruse uploaded log artifacts, resolve the issues, and run pipeline again.'
echo '[[[ file--'
cat san_failure_summaries.txt
echo '--file ]]]'
false
fi
echo 'No errors found in logs.'
- name: Package test/demo logs tarball
if: |
(env.cell-selected == 'true')
&& always()
run: |
# Package test/demo logs tarball.
cd ${{ env.install-dir }}/bin
tar cvzf logs.tgz logs
rm -rf logs # Save runner space.
- name: Upload test/demo logs (please inspect if failure(s) seen above)
if: |
(env.cell-selected == 'true')
&& always()
uses: actions/upload-artifact@v7
with:
name: ipc-test-logs-${{ matrix.compiler.id }}-${{ matrix.build-test-cfg.id }}-${{ matrix.cxx-std.id }}
path: ${{ env.install-dir }}/bin/logs.tgz
# TODO: Look into the topic of debuggability in case of a crash. Is a core generated? Is it saved?
# Do we need to manually save it as an artifact? For that matter we would then need the binary and
# ideally the source. For now we save any logs that are not printed to console, and where possible
# our tests/demos keep it the console exclusively (but in some cases, such as transport_test, it
# is not practical due to parallel execution and other aspects). In general it is always better that
# logs are sufficient; but look into situations where they are not.
#
# Don't forget situation where program exits due to a sanitizer reporting problem (ASAN, etc.); is
# there a core? Do we need one? Etc.
#
# Possibly this is all handled beautifully automatically; then this should be deleted.
doc-and-release:
needs: [setup, set-vars]
# Re. job-filter: this job's matrix has exactly 1 cell, so we can just use its (constant) name directly --
# and at job level, unlike in build-and-test. (Keep the literal in sync with `name` below.)
if: |
(needs.setup.outputs.proceed-else-not == 'true')
&& (((github.event.inputs.job-filter || '') == '')
|| contains('doc-clang-15-release', github.event.inputs.job-filter))
strategy:
fail-fast: false
matrix:
compiler:
# Pick a reasonably modern but pre-installed compiler for building Doxygen/etc.
- id: clang-15
name: clang
version: 15
c-path: /usr/bin/clang-15
cpp-path: /usr/bin/clang++-15
build-cfg:
- id: release
conan-profile-build-type: Release
conan-preset: release
runs-on: ubuntu-22.04
name: doc-${{ matrix.compiler.id }}-${{ matrix.build-cfg.id }}
steps:
- name: Update available software list for apt-get
run: sudo apt-get update
- name: Install Flow-IPC dependencies (like Graphviz) with apt-get
run: sudo apt-get install -y graphviz
- name: Install the latest version of Conan which is less than 2
run: pip install 'conan<2'
- name: Checkout `ipc` repository and submodules (like `flow`)
uses: actions/checkout@v6
with:
submodules: true
# Avoids the duplicate-authorization-error in (not-frequently-invoked) doc_check_in step later down.
persist-credentials: false
# In our process (and this is typical, though it is also possible to create tag without release) we
# hit Publish on a new Release in GitHub; and this creates both the Release and tag, by convention both named
# v<something>, at ~the same time (certainly within seconds of each other, empirically speaking).
# We want a source tarball and/or zip to be attached to the Release. Delightfully, GitHub does this
# automatically. Less delightfully, we have submodules -- where most of the code is in fact -- which GitHub
# will not include (they'll just be empty dirs instead). So we create and upload a tarball, etc., manually.
# (It is not, from what we can tell, possible to then delete the auto-attached "Source code" archives; they
# are not listable/deletable/uploadable assets in the same way but a built-in GitHub thing. Hopefully there
# will be no issue.)
#
# Regarding the actual upload request (x2): We do it via curl, following the documented GitHub API.
# Some resources recommend doing this (x2) instead:
# uses: actions/upload-release-asset@v1.0.2
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# with:
# upload_url: ${{ steps.prep_release_pkgs.outputs.upload-url }}
# asset_path: ${{ github.workspace }}/../${{ steps.prep_release_pkgs.outputs.tgz-name }}
# asset_name: ${{ steps.prep_release_pkgs.outputs.tgz-name }}
# asset_content_type: application/gzip
# For me (ygoldfel), despite a few hours of trying this and that and attempting to get debug output, it kept
# yielding `Invalid name for request` error with no added info. Reasons might be: some silly error on my part
# (but it is very vanilla, and I reduced it down to the bare essentials with no success, while also eliminating
# possibilities like it being unable to find the local file to upload and other stupidities); a change in API
# or some other behavior that was not handled by the upload-release-asset action package; possibly a break
# in a dependency of upload-release-asset (octokat?). upload-release-asset was last updated in 2020 and is
# officially now unmaintained. So I fought valiantly but eventually moved onto just using curl which worked
# almost immediately. Since the resulting code isn't even longer, and we are using a documented public API and
# standard tool (curl) which we also use elsewhere, it seemed a reasonable approach. TODO: Revisit perhaps.
- name: (On release creation only) Attach full source package(s) to Release
if: success() && (!cancelled()) && startsWith(github.ref, 'refs/tags/v')
run: |
# Attach full source package(s) to Release.
REPO=ipc
VERSION=`echo ${{ github.ref }} | cut -c 12-` # E.g., refs/tags/v1.2.3-rc1 => 1.2.3-rc1.
ARCHIVE_NAME=$REPO-${VERSION}_full
TGZ_NAME=$ARCHIVE_NAME.tar.gz
ZIP_NAME=$ARCHIVE_NAME.zip
cd ..
pwd
mv -v $REPO $ARCHIVE_NAME
# When making archives exclude .git directories and files. (.gitignore and .gitmodules, among others,
# must stay.) tar --exclude-vcs sounds delightful, but it'll exclude .gitignore and .gitmodules at least;
# but they are legit code.)
tar cvzf $TGZ_NAME --exclude=.git $ARCHIVE_NAME
zip -r $ZIP_NAME $ARCHIVE_NAME --exclude '*/.git/*' '*/.git'
mv -v $ARCHIVE_NAME $REPO
# Soon we will need the upload URL, but unfortunately github.event.release.upload_url is only auto-populated
# on `release` trigger; whereas we want to be triggerable manually on `push` to tag v* (so a superset
# of `release` trigger). So we must figure it out via API call. Also, if workflow is invoked a 2nd (etc.)
# time after succeeding once, then asset might already be attached to Release; then that upload will fail.
# There is still utility in invoking us manually to, say, signal the web site to re-update; so we do not
# want to overall-fail in that case. So detect that situation with another API call.
RELEASE_DATA=`curl -H 'Authorization: token ${{ secrets.GITHUB_TOKEN }}' \
-H 'Accept: application/vnd.github.v3+json' \
https://api.github.com/repos/Flow-IPC/$REPO/releases/tags/v$VERSION`
echo "For release [v$VERSION]: release-data API request yielded: [$RELEASE_DATA]."
UPLOAD_URL=`echo $RELEASE_DATA | jq -r .upload_url | sed 's/{.*}$//'`
echo "For release [v$VERSION]: upload-URL was computed to be: [$UPLOAD_URL]."
ASSETS_URL=`echo $RELEASE_DATA | jq -r .assets_url`
echo "For release [v$VERSION]: assets-URL was computed to be: [$ASSETS_URL]."
ASSET_NAMES=`curl -H 'Authorization: token ${{ secrets.GITHUB_TOKEN }}' \
-H 'Accept: application/vnd.github.v3+json' \
$ASSETS_URL \
| jq -r '.[].name'`
if echo "$ASSET_NAMES" | fgrep -xq $TGZ_NAME; then
echo "Asset named [$TGZ_NAME] is already attached to Release; upload would fail; skipping."
else
curl --fail-with-body -X POST \
-H 'Authorization: token ${{ secrets.GITHUB_TOKEN }}' \
-H 'Content-Type: application/gzip' -H "Content-Length: $(wc -c < $TGZ_NAME)" \
--data-binary @$TGZ_NAME "$UPLOAD_URL?name=$TGZ_NAME"
echo
echo "Uploaded [$TGZ_NAME] to Release."
fi
if echo "$ASSET_NAMES" | fgrep -xq $ZIP_NAME; then
echo "Asset named [$ZIP_NAME] is already attached to Release; upload would fail; skipping."
else
curl --fail-with-body -X POST \
-H 'Authorization: token ${{ secrets.GITHUB_TOKEN }}' \
-H 'Content-Type: application/zip' -H "Content-Length: $(wc -c < $ZIP_NAME)" \
--data-binary @$ZIP_NAME "$UPLOAD_URL?name=$ZIP_NAME"
echo
echo "Uploaded [$ZIP_NAME] to Release."
fi
- name: (On release creation only) Signal Pages about release (web site should update)
if: success() && (!cancelled()) && startsWith(github.ref, 'refs/tags/v')
# This is pretty similar (not identical) to the main-branch signaling step below. TODO: Code reuse?
run: |
# Signal Pages about release (web site should update).
curl --fail-with-body -X POST \
-H 'Accept: application/vnd.github.v3+json' \
-H 'Authorization: token ${{ secrets.GIT_BOT_PAT }}' \
'https://api.github.com/repos/Flow-IPC/flow-ipc.github.io/dispatches' \
-d '{"event_type": "flow-ipc-sync-doc-event", "client_payload": {"version": "${{ github.ref_name }}"}}'
- name: Create Conan profile
run: |
# Create Conan profile.
cat <<'EOF' > conan_profile
[settings]
compiler = ${{ matrix.compiler.name }}
compiler.version = ${{ matrix.compiler.version }}
compiler.cppstd = 17
compiler.libcxx = libstdc++11
arch = x86_64
os = Linux
build_type = ${{ matrix.build-cfg.conan-profile-build-type }}
[conf]
tools.build:compiler_executables = {"c": "${{ matrix.compiler.c-path }}", "cpp": "${{ matrix.compiler.cpp-path }}"}
[buildenv]
CC = ${{ matrix.compiler.c-path }}
CXX = ${{ matrix.compiler.cpp-path }}
[options]
ipc:doc = True
ipc:build = False
EOF
# Cache Conan-built deps (here: mainly Doxygen). See comments atop the analogous steps in the
# build-and-test job; and the longer explanation in Flow workflow counterpart.
# (The build-and-test job's flag-snapshot guard is deliberately not replicated here: this profile
# carries no meaningful compile/link flags, and the deps are stand-alone tools, so the stale-flags
# hazard the guard addresses cannot meaningfully arise.)
- name: Restore Conan package cache
id: conan-cache-restore
uses: actions/cache/restore@v6
with:
path: ~/.conan/data
key: conan-data-doc-${{ hashFiles('conan_profile', 'conanfile.py') }}
restore-keys: |
conan-data-doc-
- name: Install Flow-IPC dependencies (like Doxygen) with Conan using the profile
run: |
conan install . --profile:build conan_profile --profile:host conan_profile --build missing
# (See comment in the build-and-test job's analogous step.)
conan remove '*' --builds --src --force
- name: Save Conan package cache
if: steps.conan-cache-restore.outputs.cache-hit != 'true'
uses: actions/cache/save@v6
with:
path: ~/.conan/data
key: ${{ steps.conan-cache-restore.outputs.cache-primary-key }}
- name: Generate code documentation using Conan and Doxygen
run: conan build .
- name: Create documentation tarball (full docs, API-only docs, landing page)
run: |
# Create documentation tarball (full docs, API-only docs, landing page).
cd ${{ github.workspace }}/doc/ipc_doc
${{ github.workspace }}/tools/doc/stage_generated_docs.sh \
${{ github.workspace }}/build/${{ matrix.build-cfg.conan-profile-build-type }}
- name: Upload documentation tarball
uses: actions/upload-artifact@v7
with:
name: ipc-doc
path: ${{ github.workspace }}/doc/ipc_doc.tgz
- name: (`main` branch only) Check-in generated documentation directly into source control
id: doc_check_in
if: success() && (!cancelled()) && (github.ref == 'refs/heads/main')
run: |
# Check-in generated documentation directly into source control.
echo 'generated/ docs have been added or replaced locally; mirroring this into checked-in tree.'
# These values informally recommended in:
# https://github.com/actions/checkout#push-a-commit-using-the-built-in-token
git config user.name github-actions
git config user.email github-actions@github.com
# We are forced to use a Personal Access Token attached to a special bot user such that in repo Settings
# we've configured that "guy" as allowed to bypass the requirement to merge via PR. As of this writing
# there's no way to configure the default token to be able to do this.
# TODO: Keep an eye on that in case they provide for a better way:
# https://github.com/orgs/community/discussions/25305
# `-w 0` to avoid issues due to long tokens.
git config --local http.https://github.com/.extraheader \
"AUTHORIZATION: basic $(echo -n x-access-token:${{ secrets.GIT_BOT_PAT }} | base64 -w 0)"
# Avoid HTTP/2 network errors.
git config --local http.version HTTP/1.1
cd ${{ github.workspace }}/doc/ipc_doc
git rm -r --cached generated || echo 'No generated/ currently checked in; no problem.'
git add generated
git commit -m '${{ needs.set-vars.outputs.doc-commit-message }}'
git push origin main
- name: (`main` branch only) Signal Pages that we have updated the generated docs (web site should update)
if: success() && (!cancelled()) && (steps.doc_check_in.outcome != 'skipped')
run: |
# Signal Pages that we have updated the generated docs (web site should update).
curl --fail-with-body -X POST \
-H 'Accept: application/vnd.github.v3+json' \
-H 'Authorization: token ${{ secrets.GIT_BOT_PAT }}' \
'https://api.github.com/repos/Flow-IPC/flow-ipc.github.io/dispatches' \
-d '{"event_type": "flow-ipc-sync-doc-event"}'