Vulnenv week9

docs/test_env/architecture/04-environment-schema.md

@@ -263,7 +263,52 @@ shared:
       expected_output: "uid="
       timeout: 120
 ```
-**NOTE:** LHOST is deliberately NOT set here. The target runs inside a container network namespace, so a hardcoded 127.0.0.1 resolves to the container itself, not the host - the payload can never call back. Metasploit's own outbound-interface auto-detection (used when LHOST is left unset) correctly picks the host's real reachable IP, which is what actually works.
+
+**`ci.exploit`** — read and applied automatically by both `test_env build`
+and `test_env exec`. If `payload` is set and differs from the module's
+current `PAYLOAD`, it's applied via `set PAYLOAD ...` before the module
+runs (this is what lets a definition steer around a module's own default
+payload when that default is known not to work against the image).
+Any keys under `options` are applied the same way.
+
+Do not set `LHOST` here. The target runs inside a container network
+namespace; a hardcoded `LHOST` (especially `127.0.0.1`) resolves to the
+container itself, not the host, so the payload can never call back or be
+fetched. Leaving `LHOST` unset lets Metasploit's own outbound-interface
+auto-detection supply the host's real reachable address, which is what
+actually works from inside a container.
+
+| Key | Type | Required | Description |
+|-----|------|----------|--------------|
+| `payload` | String | No | Payload to select for this environment, if the module's default is unsuitable |
+| `options` | Hash | No | Additional datastore keys to set (e.g. `LPORT`). Do not include `LHOST` |
+
+**`ci.validation`** — read and checked by `test_env validate <ID>`, run
+after `test_env exec <ID>`. This is the single definition of "did this
+environment's exploit actually work," used identically whether a human
+runs `validate` interactively or a future headless CI runner calls the
+same resolution + check path - there is one source of truth, not a
+YAML description alongside a separately-hand-maintained CI script.
+
+| Key | Type | Required | Description |
+|-----|------|----------|--------------|
+| `expected_session` | Boolean | No | Default `true`. If `false`, `validate` passes without checking for a session at all |
+| `session_type` | String | No | `meterpreter` or `shell`. If set, the created session's type must match |
+| `expected_output` | String | No | Substring that running `id` on the session must contain, e.g. `"uid="` |
+| `timeout` | Integer | No | Seconds to wait for a session to appear before failing. Default: 120 |
+
+`validate` reports `PASS` or `FAIL` with a specific reason. If no session
+exists yet, run `test_env exec <ID>` first.
+
+**Known limitation:** `validate` looks for the session in the current
+process's `framework.sessions`. Metasploit sessions are process-local -
+they exist only in the msfconsole process that opened them, with no
+automatic cross-process visibility. `exec` and `validate` must therefore
+run in the *same* msfconsole process/window. A future headless CI runner
+would need to invoke both within a single `msfconsole -x` script (or
+equivalent single-process automation), not as two independent CLI
+invocations - this is a real constraint on any CI-alignment design here,
+not just an interactive-usage quirk.
 
 ### profiles Section
 
@@ -279,6 +324,8 @@ Each profile is a key-value pair:
 | `credentials` | Hash | No | Overrides base `shared.credentials` |
 | `volumes` | Hash | No | Overrides base `shared.volumes` |
 | `ci` | Hash | No | Overrides base `shared.ci` |
+| `provision` | Hash | No | Overrides base `shared.provision` |
+| `verify` | Hash | No | Overrides base `shared.verify` |
 
 **Profile names** must match `[a-z0-9-]+`.
 
@@ -397,10 +444,9 @@ Environment definitions are loaded by the plugin and used to:
 1. Build/pull container images
 2. Map container ports to host ports
 3. Configure health checks
-4. Set module datastore defaults
-5. Apply profile-specific overrides
-6. Apply module-level overrides (if any)
-
-See [03-database-schema.md](https://github.com/Nayeraneru/metasploit-framework/blob/vulnenv-week1/docs/architecture/03-database-schema.md) for registry design.
-
+4. Run one-time provisioning and post-provision verification, if defined
+5. Set module datastore defaults
+6. Apply profile-specific overrides
+7. Apply module-level overrides (if any)
 
+See [03-database-schema.md](https://github.com/Nayeraneru/metasploit-framework/blob/vulnenv-week1/docs/architecture/03-database-schema.md) for registry design.

plugins/test_env.rb

@@ -1506,6 +1506,8 @@ def cmd_test_env(*args)
           cmd_test_env_remove_all(args)
         when 'exec'
           cmd_test_env_exec(args)
+        when 'validate'
+          cmd_test_env_validate(args)
         when 'status'
           cmd_test_env_status(args)
         when 'help'
@@ -1891,6 +1893,7 @@ def cmd_test_env_help
         print_line("  remove <ID> Tear down an environment")
         print_line("  remove-all Tear down all environments")
         print_line("  exec <ID>  Execute exploit against environment")
+        print_line("  validate <ID> Check session/output against ci.validation")
         print_line("  status     Show runtime status")
         print_line("  help       Show this help")
         print_line
@@ -2083,6 +2086,209 @@ def free_local_port
         port
       end
 
+      # Runs a lightweight "who am I" check on a session and returns
+      # output in the same shape a raw 'id' command would (so it can be
+      # matched against a shell-style expected_output like "uid=").
+      #
+      # Deliberately NOT using shell_command_token for Meterpreter: that
+      # opens a process channel (sys.process.execute under the hood), and
+      # in testing that channel path failed deterministically - every
+      # attempt, every session, same "core_channel_write: Operation
+      # failed: 9" - which points to the channel mechanism itself being
+      # unreliable in this environment, not a one-off timing issue.
+      # sys.config.getuid is a plain TLV request/response call with no
+      # channel involved, so it avoids that failure mode entirely.
+      #
+      # Shell sessions have no such alternative - shell_command_token IS
+      # the interface for them, and it worked without issue in testing
+      # (see the WordPress php/reverse_php run), so it stays as-is there.
+      def run_verification_command(session)
+        if session.type.to_s == 'meterpreter' && session.respond_to?(:sys)
+          uid = session.sys.config.getuid
+          "uid=#{uid}"
+        else
+          session.shell_command_token('id')
+        end
+      end
+
+      # Checks a built (and typically already-'exec'd) environment against
+      # its definition's ci.validation block: was a session created, is it
+      # the expected type, and does it produce the expected output. Prints
+      # a clear PASS/FAIL.
+      #
+      # This is the concrete "align shared definitions with CI and local
+      # execution workflows" deliverable: ci.validation was previously
+      # documented in the YAML schema but never actually read by any code
+      # path. Because this method only reads from the resolved definition
+      # (never anything console-session-specific beyond the session list
+      # itself), the exact same check a human runs interactively here is
+      # what a headless CI runner would perform against the same YAML -
+      # there is one definition of "did this pass," not two.
+      def cmd_test_env_validate(args)
+        if args.empty? || args.first == '-h' || args.first == '--help'
+          print_line("Usage: test_env validate <ID>")
+          print_line
+          print_line("Checks session(s) opened against this environment's")
+          print_line("module against the definition's ci.validation")
+          print_line("expectations (session type + expected output). If")
+          print_line("multiple sessions exist (some modules open duplicates),")
+          print_line("each is tried until one responds correctly. Prints PASS")
+          print_line("or FAIL. Run 'test_env exec <ID>' first if no session")
+          print_line("exists yet.")
+          print_line
+          print_line("IMPORTANT: sessions are process-local. Run 'exec' and")
+          print_line("'validate' in the SAME msfconsole window - a session")
+          print_line("opened in one msfconsole process is invisible to another.")
+          return
+        end
+
+        id = args.shift.to_i
+        target = self.class.registry.get(id)
+
+        unless target
+          print_error("Environment #{id} not found. Run 'test_env list' to see tracked environments.")
+          return
+        end
+
+        # Load the module read-only - validate doesn't change console
+        # context (unlike exec, which needs 'use' for driver.run_single
+        # to apply datastore/run the exploit).
+        mod = framework.modules.create(target.module_fullname)
+        unless mod
+          print_error("Could not load module '#{target.module_fullname}'. It may have been removed or renamed since this environment was built.")
+          return
+        end
+
+        raw = mod.send(:module_info)['VulnerableEnvironment'] rescue nil
+        unless raw
+          print_error("Module '#{target.module_fullname}' does not define a VulnerableEnvironment.")
+          return
+        end
+
+        env_meta = VulnerableEnvironment.new(raw)
+        loader = EnvironmentDefinitionLoader.new(Msf::Config.data_directory)
+        config = loader.resolve(env_meta.definition, target.env_version, env_meta.profile, env_meta.overrides) rescue nil
+
+        unless config
+          print_error("Could not resolve environment definition '#{env_meta.definition}'.")
+          return
+        end
+
+        validation = config.dig('ci', 'validation')
+        unless validation
+          print_error("Definition '#{env_meta.definition}' has no ci.validation block - nothing to check against.")
+          return
+        end
+
+        expected_session = validation.fetch('expected_session', true)
+        expected_type    = validation['session_type']
+        expected_output  = validation['expected_output']
+        wait_timeout     = validation['timeout'] || 120
+
+        print_status("Validating environment #{id} (#{target.module_fullname}) against #{env_meta.definition}'s ci.validation...")
+
+        unless expected_session
+          print_good("PASS: ci.validation does not require a session.")
+          return
+        end
+
+        # Session creation can lag slightly behind 'exploit' returning
+        # (staged payloads in particular), so poll for a short window
+        # rather than checking exactly once.
+        candidates = []
+        begin
+          Timeout.timeout(wait_timeout) do
+            until candidates.any?
+              candidates = framework.sessions.values
+                                     .select { |s| s.via_exploit == target.module_fullname }
+                                     .sort_by(&:sid)
+              sleep 2 if candidates.empty?
+            end
+          end
+        rescue Timeout::Error
+          # handled by the empty check below
+        end
+
+        if candidates.empty?
+          print_error("FAIL: no session was created within #{wait_timeout}s (expected_session: true) in this console session.")
+          print_status("Sessions are process-local: they only exist in the msfconsole process that opened them.")
+          print_status("If you ran 'test_env exec #{id}' in a different msfconsole window, run 'test_env validate #{id}' there too - not here.")
+          print_status("Otherwise, run 'test_env exec #{id}' first, then 'test_env validate #{id}' in the same console.")
+          return
+        end
+
+        print_status("Found #{candidates.length} session(s) for this module: #{candidates.map(&:sid).join(', ')}")
+
+        if expected_type
+          candidates = candidates.select { |s| s.type.to_s == expected_type.to_s }
+          if candidates.empty?
+            print_error("FAIL: no session of type '#{expected_type}' found.")
+            return
+          end
+        end
+
+        # Some modules (notably this ActiveMQ one, which fires its Spring
+        # XML payload multiple times) can leave more than one session -
+        # some real, some stale/half-connected duplicates. The most
+        # recent SID is not reliably the working one (observed directly:
+        # the session actually interacted with was SID 1, while SID 2 -
+        # a duplicate opened moments later - consistently failed channel
+        # writes). So rather than guessing based on recency, try each
+        # candidate oldest-first and use the first one that actually
+        # responds. If expected_output isn't set, there's nothing to
+        # distinguish working from broken, so just take the oldest.
+        session = nil
+        output = nil
+
+        if expected_output
+          candidates.each do |candidate|
+            if candidate.respond_to?(:load_stdapi)
+              begin
+                candidate.load_stdapi
+              rescue => e
+                print_status("Could not explicitly load stdapi on session #{candidate.sid} (#{e.message}), continuing anyway...")
+              end
+            end
+
+            result = nil
+            attempts = 0
+            begin
+              attempts += 1
+              result = run_verification_command(candidate)
+            rescue => e
+              if attempts < 3
+                print_status("Session #{candidate.sid}: verification command failed on attempt #{attempts}/3 (#{e.message}), retrying...")
+                sleep 3
+                retry
+              else
+                print_status("Session #{candidate.sid} did not respond after #{attempts} attempts (#{e.message}) - trying next candidate, if any.")
+              end
+            end
+
+            if result&.include?(expected_output)
+              session = candidate
+              output = result
+              break
+            end
+          end
+
+          unless session
+            print_error("FAIL: none of #{candidates.length} candidate session(s) (#{candidates.map(&:sid).join(', ')}) produced the expected output '#{expected_output}'.")
+            return
+          end
+        else
+          session = candidates.first
+        end
+
+        print_status("Using session #{session.sid} (#{session.type})")
+
+        print_good("PASS: environment #{id} validated successfully against #{env_meta.definition}'s ci.validation.")
+      rescue => e
+        print_error("test_env validate failed: #{e.class} - #{e.message}")
+        elog("test_env validate error: #{e.class} - #{e.message}")
+        elog(e.backtrace.join("\n"))
+      end
+
       def cmd_test_env_start(args)
         if args.empty?
           print_error("Usage: test_env start <ID>")
@@ -2342,7 +2548,7 @@ def cmd_test_env_modules(args)
 
       def cmd_test_env_tabs(str, words)
         if words.length == 1
-          return %w[build list modules stop start remove remove-all exec status help]
+          return %w[build list modules stop start remove remove-all exec validate status help]
         end
 
         if words.length == 2 && words[0] == 'build'
@@ -2351,7 +2557,7 @@ def cmd_test_env_tabs(str, words)
 
         if words.length == 2
           case words[0]
-          when 'stop', 'start', 'remove', 'exec'
+          when 'stop', 'start', 'remove', 'exec', 'validate'
             return self.class.registry.list.map(&:local_id).map(&:to_s)
           end
         end