Skip to content

[Bug]: Python SDK small correctness and ergonomics findings (tracking) #4048

Description

@david-long1

Package

policy-engine

Description

Four small findings from using the agent_control_specification Python SDK as a host, consolidated into one issue per CONTRIBUTING's guidance for low-severity items. Each is independent; I can send a PR for (a), (b) and (d).

Where (main @ e7f5d2b) Symptom Suggested fix Size
a policy-engine/sdk/python/src/lib.rs:84-85 runtime_error() Every engine RuntimeError reaches Python as a bare RuntimeError carrying only the display string, so a host that needs to branch on the reason (runtime_error:path_missing vs policy_invocation_failed vs an annotation timeout) has to string-match the message. Raise a small exception hierarchy, e.g. AcsRuntimeError(RuntimeError) with a .reason attribute set to the reserved reason code; keep the message unchanged so existing callers do not break. small
b policy-engine/sdk/python/agent_control_specification/_types.py:327-337 ApprovalResolution The dataclass has outcome, handle and action_identity but no reason, so a resolver that refuses cannot say why and every refusal collapses to one string in evidence and telemetry. Add `reason: str None = Noneand carry it into the denial verdict's reason/message the same wayVerdict.reason` is carried today.
c policy-engine/sdk/python/agent_control_specification/_types.py:72-93 Decision.applies_effects The deprecated property emits DeprecationWarning on access. A consumer running its suite with -W error (common in CI) fails on any code path that still touches the property, including third-party code they do not control. Either drop the property now (it has been deprecated since the D1 change) or emit the warning once per process. tiny
d responsibleai/agent-control-spec engine/src/manifest.rs:30 vs spec/schema/manifest.schema.json:52-55 The schema declares metadata as {"type": "object", "additionalProperties": true}, but the Rust struct types it as any JsonValue, so a manifest with metadata: "string" or metadata: 3 parses in the engine and only fails if schema validation happens to run first. Type the field as Map<String, JsonValue> (or validate object-ness in the loader) so the two agree. This one lives in the agent-control-spec repo; listing it here for completeness, happy to split it out. tiny

How does this impact your work?

(a) is the one that costs real code: my host keys retry and audit behaviour on the reason code and today does so by parsing the exception text. (b) means a refused approval is indistinguishable from a resolver failure in the audit record. (c) and (d) are hygiene.

Timeline

Not blocking.

Steps to Reproduce

(a) AgentControl.from_native(...), evaluate a point whose policy_target path is absent from the snapshot, catch the exception: it is RuntimeError with no attribute beyond args[0].
(b) Return ApprovalResolution(outcome=ApprovalOutcome.DENIED) from a resolver; inspect the resulting verdict: no field carries a resolver-supplied reason.
(c) python -W error -c "from agent_control_specification import Decision; Decision(...).applies_effects" raises.
(d) Feed the engine a manifest with metadata: "not-an-object"; parse_manifest succeeds.

Environment

agent-control-specification at main e7f5d2b (also 0.3.1b1 from PyPI), Python 3.12, macOS.

Logs / Error Output

Not applicable.

Code of Conduct

  • I agree to follow the Microsoft Open Source Code of Conduct

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingneeds-review:MEDIUMContributor check flagged MEDIUM risktriageNeeds triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions