fix(test): wait for config-changed hook after juju.config() in set_app_runner_amount - #810
Draft
yanksyoon wants to merge 29 commits into
Draft
fix(test): wait for config-changed hook after juju.config() in set_app_runner_amount#810yanksyoon wants to merge 29 commits into
yanksyoon wants to merge 29 commits into
Conversation
Add a script that deletes dangling servers, keypairs, images and test security groups left by integration tests, plus a 6-hourly workflow on the private-endpoint runner.
Drop the scheduled cleanup workflow. Force-cancelled runs leave resources behind; the next integration suite now deletes CI-named OpenStack leftovers older than 6h before creating new ones.
Public cleanup entry point first; private helpers below. Module-level imports in conftest instead of inline.
Keypairs often lack created_at; treating unknown age as stale risked deleting in-progress CI resources. Skip those instead.
Wait for server delete before SGs, keep OpenStack teardown inside the connection context, best-effort start-up reaping, and tighten test-id regexes to the 8-char CI ids we generate.
Align OpenStack teardown/orphan helpers with existing integration suite call conventions.
Keep suite call style consistent; openstacksdk accepts the name positionally as used elsewhere in this repository.
Drop wait= on image delete (not supported), materialize list_servers, and log orphan cleanup failures with exc_info.
--use-existing-app-suffix is for local runs that keep long-lived resources; do not reap them at suite start.
Single source of truth for test resource names (charm naming.py, manager factories). Orphan cleanup consumes those matchers, no longer redefines regexes, deletes SGs, or uses future annotations. Only servers/images/keypairs that round-trips create are reaped.
Replace fuzzy shorthand (manager-IT, CI-named) with explicit references to github-runner-manager vs charm integration suites and what they create.
Drop matching algorithms and implementation notes from interface comments; the code already encodes those details.
Longer prefixes first is for readability, not early-exit matching.
Charm app suffixes always start with a lowercase letter; keep manager test-runner- ids free to begin with a digit.
Exact name filters miss runner VMs named test-<suffix>-N. Also soften naming module wording around suite scope and random suffixes.
Catch only ResourceNotFound and ConflictException instead of broad Exception. Unexpected failures (auth, bugs) now propagate. Inline the try/except at each resource loop instead of a shared helper.
Satisfies flake8 complexity (C901) and docstring (DCO010) rules.
…n arg - Extract suite teardown from openstack_connection_fixture to reduce flake8 complexity below 10. - Add Args/Returns docstring sections to satisfy DCO020/DCO030. - Remove unused connection param from _delete_resource helper.
jubilant.Juju.config() is fire-and-forget — it sets the config and returns before the config-changed hook fires. The test then polls check-runners while the service is still running with the previous base_virtual_machines value, causing a timeout waiting for runners. Three changes: 1. Verify config was applied by reading back after juju.config() 2. Wait for model to settle (juju.wait(all_active)) after config change so the config-changed hook completes and the service restarts with the new base_virtual_machines value 3. Add a 30s pre-poll grace period in wait_for_runner_ready to give the service time to start creating runners before the first check
Member
Author
|
Re-enabled tmate debug for the integration test jobs to inspect the runner and OpenStack VM state during failures. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
test_check_runnerintest_charm_runner.pyfails withTimeout waiting for 2 runner(s) to be ready. Investigation on the debug tmate session revealed:base_virtual_machines: 0throughout the testdesired=0 current=0every reconcile cycleconfig-changedhook fired (at 15:34:20), none after the test calledjuju.config()jubilant.Juju.config()is fire-and-forget — it returns before the hook firesThe test polls
check-runnerswhile the service is still running with the previousbase_virtual_machinesvalue.Fix
Three changes in two files:
juju.config()and assertjuju.wait(all_active)after config change so the hook completescheck-runnerspoll inwait_for_runner_readyTesting