Enforce patch-only LLM hints and strict validation - #38
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request refines how LLM-generated hints are processed and validated, shifting to a strict "patch-only" approach: hints can only update attributes of existing nodes and edges, and cannot add new graph elements. The changes update prompt instructions, validation logic, hint merging behavior, and corresponding tests to enforce this model. Additional improvements include raising token limits for LLM inference and expanding schema validation.
Patch-only hinting and validation:
HINT_INSTRUCTIONSinconstants.pyto clarify that hints must be patch-form, only updating existing nodes/edges and omitting unchanged elements. Rules now explicitly forbid adding or removing graph elements and require empty patches if there are no updates. [1] [2]_validate_hints_payloadinllm/inference.pyto allow payloads with only nodes, only edges, or only policies, and to validate types only when present. Added stricter checks for the types ofnodes,edges, andpolicies."required": ["nodes"]constraint from the hints JSON schema, allowing for patch payloads that only update edges or policies.Hint merging logic:
merge_llm_hintsinhint_processor.pyto ignore hints for nodes and edges that do not already exist in the graph, preventing accidental creation of new elements. [1] [2]Testing and documentation:
test_hint_processor.pyto verify that hints for new nodes/edges are ignored and that patching is strictly enforced. [1] [2] [3] [4] [5]test_inference.pyto cover validation of patch payloads for nodes, edges, and error cases.LLM inference improvements: