Skip to content

Enforce patch-only LLM hints and strict validation - #38

Merged
melonattacker merged 3 commits into
mainfrom
fix/hint-processor
Dec 25, 2025
Merged

melonattacker merged 3 commits into
mainfrom
fix/hint-processor

Conversation

@melonattacker

Copy link
Copy Markdown
Owner

This pull request refines how LLM-generated hints are processed and validated, shifting to a strict "patch-only" approach: hints can only update attributes of existing nodes and edges, and cannot add new graph elements. The changes update prompt instructions, validation logic, hint merging behavior, and corresponding tests to enforce this model. Additional improvements include raising token limits for LLM inference and expanding schema validation.

Patch-only hinting and validation:

  • Updated HINT_INSTRUCTIONS in constants.py to clarify that hints must be patch-form, only updating existing nodes/edges and omitting unchanged elements. Rules now explicitly forbid adding or removing graph elements and require empty patches if there are no updates. [1] [2]
  • Modified _validate_hints_payload in llm/inference.py to allow payloads with only nodes, only edges, or only policies, and to validate types only when present. Added stricter checks for the types of nodes, edges, and policies.
  • Removed the "required": ["nodes"] constraint from the hints JSON schema, allowing for patch payloads that only update edges or policies.

Hint merging logic:

  • Changed merge_llm_hints in hint_processor.py to ignore hints for nodes and edges that do not already exist in the graph, preventing accidental creation of new elements. [1] [2]

Testing and documentation:

  • Updated and added tests in test_hint_processor.py to verify that hints for new nodes/edges are ignored and that patching is strictly enforced. [1] [2] [3] [4] [5]
  • Added new tests in test_inference.py to cover validation of patch payloads for nodes, edges, and error cases.

LLM inference improvements:

  • Increased the token limit for hint inference responses to better accommodate larger JSON payloads.

@melonattacker
melonattacker merged commit f5a025f into main Dec 25, 2025
2 checks passed
@melonattacker
melonattacker deleted the fix/hint-processor branch December 25, 2025 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant