Skip to content

ci: exercise the driver through Test Kitchen on every pull request - #46

Open
tas50 wants to merge 2 commits into
mainfrom
ci/kitchen-integration-smoke
Open

tas50 wants to merge 2 commits into
mainfrom
ci/kitchen-integration-smoke

Conversation

@tas50

@tas50 tas50 commented Aug 30, 2026

Copy link
Copy Markdown
Member

Why

Nothing in CI ran this plugin through Test Kitchen. The unit tests instantiate Kitchen::Driver::Vro directly, which leaves a whole class of breakage invisible:

  • Test Kitchen failing to discover the plugin by the name people write in their kitchen.yml
  • a gemspec that stopped shipping a file
  • a bad or missing require
  • required_config or default_config coming unwired
  • a driver API change in a new test-kitchen release

There is no way to run a real create/converge here. This driver runs workflows on a vRealize Orchestrator appliance, and there is no such thing as a disposable one — no container image, no free tier, nothing a fork's PR could reach. Everything up to that point does run without an appliance, and that is exactly where the breakage above shows up.

What

A kitchen.yml in the repo root with one platform per documented configuration shape:

Platform Exercises
defaults required options only, so the defaults have to apply
workflow-ids create_workflow_id / destroy_workflow_id
workflow-parameters both parameter hashes, including non-string values
slow-workflows a per-platform request_timeout override
self-signed-cert vro_disable_ssl_verify

And bundle exec rake integration, which builds every instance that file describes the same way the kitchen command does, then asserts:

  • Test Kitchen resolves name: vro to Kitchen::Driver::Vro as a v2 plugin reporting the gem's own version
  • every default matches what the README documents — if someone changes default_config :request_timeout without touching the README, this fails
  • workflow ids, workflow parameters, and per-platform overrides survive the trip through kitchen.yml
  • a configuration missing a required option is rejected, with an error naming the option (test/fixtures/kitchen.missing-required-config.yml)
  • kitchen list and kitchen diagnose --all succeed against the driver

The same kitchen.yml points at a real appliance when VRO_BASE_URL, VRO_USERNAME, VRO_PASSWORD, VRO_CREATE_WORKFLOW, and VRO_DESTROY_WORKFLOW are set, so it doubles as the starting point for manual testing rather than being CI-only scaffolding. CONTRIBUTING.md documents both uses.

CI runs it on Ruby 3.1 and 3.4 after the existing lint-and-unit job.

Verification

$ bundle exec rake integration
Test Kitchen loads and configures the driver
  ok      every configured instance is built
  ok      the plugin is discovered by the name people write in kitchen.yml
  ok      request_timeout defaults to 300, as documented
  ok      vro_disable_ssl_verify defaults to false, as documented
  ok      create_workflow_id defaults to nil, as documented
  ok      destroy_workflow_id defaults to nil, as documented
  ok      create_workflow_parameters defaults to {}, as documented
  ok      destroy_workflow_parameters defaults to {}, as documented
  ok      workflow ids set in kitchen.yml reach the driver
  ok      workflow parameters survive the trip through kitchen.yml
  ok      a per-platform request_timeout overrides the default
  ok      vro_disable_ssl_verify turns TLS verification off
  ok      a config missing a required option is rejected

The kitchen command line works against the driver
  ok      kitchen list succeeds
  ok      kitchen diagnose succeeds

All checks passed.

Confirmed it actually catches a regression — flipping default_config :request_timeout to 600 fails the run with request_timeout defaults to 300, as documented: resolved to 600 and exits 1.

$ bundle exec cookstyle --chefstyle
Inspecting 6 files
......

6 files inspected, no offenses detected

$ bundle exec rake test
51 examples, 0 failures

@tas50

tas50 commented Aug 30, 2026

Copy link
Copy Markdown
Member Author

Carries the same one-line commit as #44 dropping the unused webmock dev dependency. cookstyle 9.0.0 (released since the last merge here) flags the extra spacing on that Gemfile line under Layout/ExtraSpacing, so main is red on its own right now and every PR inherits it. It is duplicated across both PRs deliberately so neither depends on the other's merge order; git merges the identical change cleanly.

tas50 added 2 commits August 29, 2026 18:56
Nothing in CI ran this plugin through Test Kitchen. The unit tests
instantiate Kitchen::Driver::Vro directly, so a whole class of breakage was
invisible: Test Kitchen failing to discover the plugin by the name people
write in their kitchen.yml, a gemspec that stopped shipping a file, a bad
require, required_config or default_config coming unwired, or a driver API
change in a new test-kitchen release.

There is no way to run a real create/converge in CI -- this driver runs
workflows on a vRealize Orchestrator appliance, and there is no such thing as
a disposable one. Everything up to that point can run, though, and that is
where the breakage above shows up.

Adds a kitchen.yml with one platform per documented configuration shape
(defaults only, workflows addressed by id, workflow input parameters, a
longer request_timeout, a self-signed certificate) and `rake integration`,
which builds every instance that file describes and asserts:

- Test Kitchen resolves `name: vro` to Kitchen::Driver::Vro as a v2 plugin
  reporting the gem's own version
- each default matches what the README documents
- workflow ids, workflow parameters, and per-platform overrides survive the
  trip through kitchen.yml
- a configuration missing a required option is rejected, with an error naming
  the option
- `kitchen list` and `kitchen diagnose --all` succeed against the driver

The same kitchen.yml points at a real appliance when VRO_BASE_URL,
VRO_USERNAME, VRO_PASSWORD, VRO_CREATE_WORKFLOW, and VRO_DESTROY_WORKFLOW are
set, so it doubles as the starting point for manual testing.

CI runs it on Ruby 3.1 and 3.4 after the lint and unit job.

Signed-off-by: Tim Smith <tim@mondoo.com>
webmock has never been required or used by anything in spec/ -- the specs
stub the vcoworkflows client rather than the HTTP layer underneath it, so
nothing ever reaches a socket for webmock to intercept.

Removing it also clears the one thing standing between this repo and a green
lint run: cookstyle 9.0.0 flags the extra spacing in that Gemfile line under
Layout/ExtraSpacing, which cookstyle 8 did not, so `main` went red on its own
the moment cookstyle 9 shipped.

Signed-off-by: Tim Smith <tim@mondoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant