Skip to content

Vulnenv week7 #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Nayeraneru merged 11 commits into from
Aug 1, 2026
Merged

Vulnenv week7 #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
9cc206b
Make the docs consistent with week 6 code change
Jul 24, 2026
ab3c1b3
Environment Management Commands
Jul 25, 2026
17144ee
minor fix
Jul 25, 2026
d422d02
doc refactor
Jul 25, 2026
3e01595
minor refactor
Jul 25, 2026
707044c
code refactor
Jul 26, 2026
2d6615f
Inject credentials into datastore
Jul 29, 2026
2f44f56
fix Podman behavior
Jul 29, 2026
875ab85
Resolved table formatting
Jul 30, 2026
502a661
Automatic provisioning if installation required
Jul 30, 2026
41578c6
update docs
Aug 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 35 additions & 1 deletion data/vuln_envs/wordpress.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,46 @@ shared:
datastore_defaults:
TARGETURI: /

# Liveness only: confirms Apache/PHP are up and responding.
# Does NOT confirm WordPress is installed - a redirect to
# /wp-admin/install.php is a valid 302 here and is expected
# on first boot, since this image ships with wp-config.php
# pointed at a database but never runs the install step itself.
health_check:
type: http
path: /
expected_status: 302
interval: 3
timeout: 2
retries: 15

# One-time setup: drives the WordPress install wizard
# (creates the DB schema + admin account) using the
# credentials defined above. Runs once, after health_check
# passes and before the environment is considered ready.
provision:
type: http_post
path: /wp-admin/install.php?step=2
body:
weblog_title: "Vulnerable WP"
user_name: "{{ credentials.default.username }}"
admin_password: "{{ credentials.default.password }}"
admin_password2: "{{ credentials.default.password }}"
pw_weak: 1
admin_email: "admin@example.com"
blog_public: 0
Submit: "Install WordPress"
run_once: true

# Confirms provisioning actually succeeded: the login form
# (not the installer) must now be served.
verify:
type: http
path: /wp-login.php
expected_status: 200
match: "user_login"
interval: 3
timeout: 2
retries: 10

ci:
Expand All @@ -41,4 +75,4 @@ shared:

profiles:
default:
description: Standard vulnerable WordPress
description: Standard vulnerable WordPress
2 changes: 1 addition & 1 deletion docs/test_env/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This directory contains the architecture and workflow design for the `test_env`
|----------|-------------|
| [01-command-dispatcher.md](https://github.com/Nayeraneru/metasploit-framework/blob/vulnenv-week1/docs/test_env/architecture/01-command-dispatcher.md) | How `test_env` is added to msfconsole via plugin dispatcher |
| [02-module-metadata.md](https://github.com/Nayeraneru/metasploit-framework/blob/vulnenv-week1/docs/test_env/architecture/02-module-metadata.md) | How modules expose `VulnEnv` metadata and how the plugin reads it |
| [03-database-schema.md](https://github.com/Nayeraneru/metasploit-framework/blob/vulnenv-week1/docs/test_env/architecture/03-database-schema.md) | Registry persistence: in-memory Phase 1, PostgreSQL Phase 2 |
| [03-database-schema.md](https://github.com/Nayeraneru/metasploit-framework/blob/vulnenv-week1/docs/test_env/architecture/03-database-schema.md) | Registry persistence: in-memory Phase 1, YAML Persistence with ActiveModel Phase 2 |
| [04-environment-schema.md](https://github.com/Nayeraneru/metasploit-framework/blob/vulnenv-week1/docs/test_env/architecture/04-environment-schema.md) | YAML schema for shared environment definitions in `data/vuln_envs/` |
| [05-runtime-adapter.md](https://github.com/Nayeraneru/metasploit-framework/blob/vulnenv-week1/docs/architecture/05-runtime-adapter.md) | Docker/Podman abstraction, port allocation, container labels |

Expand Down
212 changes: 25 additions & 187 deletions docs/test_env/architecture/03-database-schema.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,204 +182,42 @@ docker run -d \
| Schema evolution | Adding a new label is simpler than versioning a JSON schema |
| No encoding/decoding complexity | No Base64, no JSON parsing errors |

### State Reconstruction From Labels (Future Enhancement)

```ruby
def reconstruct_from_labels(runtime)
# Step 1: Discover all framework-managed containers via native filter
containers = runtime.list(filters: { 'label' => 'msf.vulnenv.managed_by=test_env' })

containers.each do |container|
labels = container['Config']['Labels'] || {}

# Step 2: Skip containers from other msfconsole instances
instance_id = labels['msf.vulnenv.instance_id']
next unless instance_id == current_instance_id

# Step 3: Extract minimal dynamic data from labels
module_fullname = labels['msf.vulnenv.module']
env_id = labels['msf.vulnenv.env_id'].to_i
version = labels['msf.vulnenv.version']

# Parse port mapping: "8081:8080,9090:61616" -> {8080=>8081, 61616=>9090}
ports = parse_port_label(labels['msf.vulnenv.ports'])
## Phase 2: YAML Persistence with ActiveModel (Week 6+)

# Step 4: Load module and resolve its VulnerableEnvironment definition
mod = framework.modules.create(module_fullname)
next unless mod
Phase 2 replaces purely in-memory storage with **ActiveModel-backed YAML persistence** in `~/.msf4/test_env_registry.yml`. This provides cross-session state sharing without requiring any framework database changes.

vuln_env_meta = mod.send(:module_info)['VulnerableEnvironment']
next unless vuln_env_meta

definition_name = vuln_env_meta['definition']
profile = vuln_env_meta['profile'] || 'default'
overrides = vuln_env_meta['overrides'] || {}

# Step 5: Resolve environment config from YAML
loader = EnvironmentDefinitionLoader.new(Msf::Config.data_directory)
config = loader.resolve(definition_name, version, profile, overrides)

# Step 6: Build datastore from port_mapping + allocated ports
datastore = { 'RHOSTS' => '127.0.0.1' }
vuln_env_meta['port_mapping'].each do |container_port, ds_option|
datastore[ds_option] = ports[container_port]
end
### Why ActiveModel + YAML?

# Step 7: Reconstruct registry entry
@environments[env_id] = {
local_id: env_id,
container_id: container['Id'],
module_fullname: module_fullname,
env_version: version,
rhost: '127.0.0.1',
rport: ports.values.first,
runtime: runtime.name,
image_ref: container['Config']['Image'],
status: container['State']['Status'],
exploit_command: build_exploit_command(datastore),
datastore: datastore,
created_at: Time.parse(container['Created']),
started_at: Time.parse(container['State']['StartedAt'])
}

@next_id = [@next_id, env_id + 1].max
end
end

# Parse "8081:8080,9090:61616" into {8080=>8081, 61616=>9090}
def parse_port_label(label_value)
return {} unless label_value

label_value.split(',').each_with_object({}) do |pair, hash|
host_port, container_port = pair.split(':')
hash[container_port.to_i] = host_port.to_i
end
end
| Concern | Resolution |
|---------|-----------|
| **Framework coupling** | No dependency on `framework.db.active` or PostgreSQL |
| **Upstream friction** | No migration files, no `metasploit-data-models` coordination |
| **Portability** | Works in any msfconsole session, even without `msfdb` |
| **Concurrency** | `flock` file locking enables safe multi-process access |
| **Security** | `Psych.safe_load` with permitted classes prevents arbitrary code execution |

# Build exploit command from datastore
def build_exploit_command(datastore)
cmds = datastore.map { |k, v| "set #{k} #{v}" }
cmds.join('; ') + '; exploit'
end
```
### Design Overview

- **`VulnTarget`** — `ActiveModel::Model` representing a single provisioned container
- **`VulnEnvironment`** — `ActiveModel::Validations` collection of all targets
- **`VulnEnvironmentStore`** — Atomic load/save via `File::LOCK_SH` / `File::LOCK_EX`

## Phase 2: Database Integration (Week 6+)
### Cross-Session Behavior

When adding PostgreSQL persistence:
The YAML file acts as a **single shared registry** across all msfconsole instances:

### Migration File
```ruby
# db/migrate/20240624000001_create_vuln_environments.rb
class CreateVulnEnvironments < ActiveRecord::Migration[8.0]
def change
create_table :vuln_environments, id: :serial do |t|
t.string :container_id, null: false
t.string :image_ref, null: false
t.string :module_fullname, null: false
t.string :env_version
t.string :rhost, default: '127.0.0.1'
t.integer :rport, null: false
t.text :datastore
t.string :runtime, default: 'docker', null: false
t.string :msf_instance_id
t.string :status, null: false, default: 'running'
t.text :exploit_command
t.timestamps
t.datetime :started_at
t.datetime :stopped_at
t.datetime :removed_at
end

add_index :vuln_environments, :module_fullname
add_index :vuln_environments, :status
add_index :vuln_environments, :container_id, unique: true
add_index :vuln_environments, :msf_instance_id
add_index :vuln_environments, [:status, :module_fullname]
end
end
```
- **Sparse monotonic IDs**: IDs are allocated sequentially but never renumbered. Removing environment 2 from `[1, 2, 3]` yields `[1, 3]`, not `[1, 2]`. This matches the Metasploit `sessions` table behavior and prevents identity-shift bugs during batch operations.
- **Pruning on startup**: Dead entries (containers that no longer exist) are automatically removed. IDs are **not** compacted after pruning — sparse IDs remain stable.

### ActiveRecord Model
```ruby
class VulnEnvironment < ActiveRecord::Base
self.table_name = 'vuln_environments'
serialize :datastore, JSON

scope :active, -> { where(status: ['running', 'stopped']) }
scope :running, -> { where(status: 'running') }
scope :by_module, ->(name) { where(module_fullname: name) }

validates :container_id, presence: true, uniqueness: true
validates :module_fullname, presence: true
validates :rport, presence: true, numericality: { only_integer: true }
validates :status, inclusion: { in: %w[running stopped removed orphaned error] }
end
```

### Integration With In-Memory Registry

```ruby
class BuiltEnvironmentRegistry
def initialize(framework)
@framework = framework
@environments = {}
@next_id = 1
load_from_database if database_available?
end

private

def database_available?
framework.db.active && defined?(VulnEnvironment)
end

def load_from_database
VulnEnvironment.active.each do |db_env|
@environments[@next_id] = {
local_id: @next_id,
db_id: db_env.id,
container_id: db_env.container_id,
# ... map all fields ...
}
@next_id += 1
end
end

def persist_to_database(record)
VulnEnvironment.create!(...)
end
end
```

## Reference: sessions Table Pattern

From `db/schema.rb`:
```ruby
create_table "sessions", id: :serial, force: :cascade do |t|
t.integer "host_id"
t.string "stype"
t.string "via_exploit" # Module association
t.string "via_payload"
t.string "desc"
t.integer "port"
t.string "platform"
t.text "datastore" # Serialized hash
t.datetime "opened_at", precision: nil, null: false
t.datetime "closed_at", precision: nil
t.string "close_reason"
t.integer "local_id" # In-memory mapping
t.datetime "last_seen", precision: nil
t.integer "module_run_id"
t.index ["module_run_id"], name: "index_sessions_on_module_run_id"
end
```
### State Reconstruction

My `vuln_environments` table follows this exact pattern:
- `id: :serial` primary key
- `module_fullname` like `via_exploit`
- `datastore` serialized text
- `local_id` equivalent via `env_id` label
- Lifecycle timestamps (`created_at`, `started_at`, `stopped_at`, `removed_at`)
On startup, the plugin:

1. Loads the shared YAML registry
2. Queries the runtime for all containers with `label=msf.vulnenv.managed_by=test_env`
3. Prunes registry entries whose `container_id` no longer exists
4. Reconstructs any running containers missing from the registry by reading their labels and assigning the next monotonic ID

**Important:** Reconstruction uses `container_id` as the ground truth for matching, not the `env_id` label. Because IDs are never compacted, `env_id` labels remain stable references, but `container_id` is still the authoritative identifier.
Loading
Loading