Skip to content

Run learner code in a bounded local execution environment #64

Description

@rosshd

Parent: #56

Depends on: #53 and the activity-contract child of #56.

Blocks: the solution-evaluator child of #56.

Problem

The current drill checker executes learner Python in the main openLearn environment with pytest and no explicit wall-clock timeout.

An accidental infinite loop, memory blow-up, import side effect, filesystem write, environment read, or subprocess launch can hang or affect the learner's machine.

Issue #54 narrows tutor tool access, but learner-authored code still needs an execution boundary.

Goal

Add a bounded local code runner for interview activities, starting with Python, that limits damage, reports failures accurately, and works across supported platforms.

Runner contract

  • Accept only a validated workspace, language adapter, problem interface, test bundle, and resource policy.
  • Execute without shell interpolation.
  • Use a scrubbed environment containing only required variables.
  • Set an explicit working directory owned by the attempt.
  • Enforce wall-clock timeout.
  • Enforce CPU, memory, process-count, output-size, and file-size limits where the platform supports them.
  • Disable network access when a reliable supported mechanism is available.
  • Prevent reads and writes outside the owned workspace to the strongest practical degree on each platform.
  • Capture stdout, stderr, exit status, signal or termination reason, duration, and resource-limit reason.
  • Kill the process tree on cancellation or timeout.
  • Keep runner failure distinct from compile, runtime, test, and learner-solution failure.

Platform strategy

  • Define one shared runner result contract.
  • Implement Python first.
  • Use platform adapters where macOS, Linux, and Windows isolation primitives differ.
  • Fail closed for a protection that the selected mode claims to provide.
  • Document the actual security boundary and residual risks honestly.
  • Provide a clearly labeled reduced-isolation fallback only when the learner explicitly opts in.

Test handling

  • Keep hidden tests outside the learner-editable workspace where possible.
  • Do not rewrite the learner's solution to enable tests.
  • Mount or copy only the minimum problem harness required.
  • Bound captured output so print loops cannot exhaust memory or terminal rendering.
  • Make cancellation responsive from the CLI.

Acceptance criteria

  • Infinite loops terminate within the configured wall-clock budget.
  • Process trees are terminated on timeout and cancellation.
  • Excessive output is truncated with an explicit result.
  • Environment fixtures prove secret-like variables are unavailable.
  • Workspace escape, subprocess, network, and resource-limit fixtures exercise the documented boundary on each supported platform.
  • Unsupported protections are reported and never silently claimed.
  • Hidden tests are not readable from the normal learner workspace under the documented mode.
  • Runner infrastructure failures never count as incorrect learner answers.
  • The existing trusted development test suite continues to run outside this runner.
  • Python drill and /check behavior migrate without losing targeted feedback.
  • make check passes on the supported OS matrix.

Escalation

  • If strong local isolation cannot be implemented portably without a container or VM dependency, document the tradeoff and request a product decision before claiming sandbox security.
  • Any default that executes with materially reduced isolation requires explicit product approval.

Out of scope

  • Arbitrary package installation from learner code.
  • General shell-command execution.
  • Production malware analysis.
  • Multi-language adapters beyond the initial Python runner.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions