feat(dsql): enhance query plan explainability with type coercion detection, rewrites, and workflow extraction#162
Conversation
8e33741 to
8261713
Compare
amaksimo
left a comment
There was a problem hiding this comment.
I have a few general commets:
- We should use positive language throughout (llm can confuse DO with DO NOT when we trim context)
- We should try to use RFC language more frequently throughout
- We should break up the references in the query-plan folder as some of the files are very long
07b6baa to
6f97294
Compare
|
|
||
| **Fallback:** If `awsknowledge` is unavailable, use the defaults above and flag that limits should be verified against [DSQL documentation](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/). | ||
| **Fallback:** If `awsknowledge` is unavailable, use the defaults above and note to the user | ||
| that limits should be verified against [DSQL documentation](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/). |
There was a problem hiding this comment.
why did we end up adding line breaks?
|
|
||
| **When:** MUST load all four at Workflow 8 Phase 0 — [query-plan/plan-interpretation.md](references/query-plan/plan-interpretation.md), [query-plan/catalog-queries.md](references/query-plan/catalog-queries.md), [query-plan/guc-experiments.md](references/query-plan/guc-experiments.md), [query-plan/report-format.md](references/query-plan/report-format.md) | ||
| **Contains:** DSQL node types + Node Duration math + estimation-error bands, pg_class/pg_stats/pg_indexes SQL + correlated-predicate verification, GUC experiment procedures + 30-second skip protocol, required report structure + element checklist + support request template | ||
| **When:** MUST load [query-plan/workflow.md](references/query-plan/workflow.md) at Workflow 8 entry — it gates the remaining files |
There was a problem hiding this comment.
why remove the bullet structure?
|
|
||
| **SHOULD apply when:** The WHERE clause rejects NULLs from the right-hand side of a LEFT JOIN (e.g., `IS NOT NULL`, equality comparisons, or any predicate that cannot be true for NULL). | ||
|
|
||
| **Skip when:** NULLs from the right-hand side are intentionally preserved in the result. |
There was a problem hiding this comment.
is this an always? contextualize when/how often? Should this be an SHOULD Skip when?
|
|
||
| When a query uses LEFT JOIN but the WHERE clause rejects NULLs on the joined table, rewrite as INNER JOIN. This enables a simpler, more efficient join plan. | ||
|
|
||
| **SHOULD apply when:** The WHERE clause rejects NULLs from the right-hand side of a LEFT JOIN (e.g., `IS NOT NULL`, equality comparisons, or any predicate that cannot be true for NULL). |
There was a problem hiding this comment.
is this a should or a must? should gives the model a directive to bypass when told something like to rush
| @@ -0,0 +1,48 @@ | |||
| # Rewrite: Propagate Filter to JOIN Columns | |||
There was a problem hiding this comment.
since so many of these are deterministic, I question the meta structure of if we should instead leverage something like a python script converter of sorts that can parse and replace the SQL and the reference files just execute them?
There was a problem hiding this comment.
I think I still hold to this? Why are we relying on markdown applications alone? Why are these transformations not written into script patterns/tooling?
PR #162 — Review Summary
Reviewed at head SHA
Reviewer scope. This review covered the diff at the head SHA (21 files, +1131 / −36) — the new query-plan workflow extraction, type-coercion detection, 11-pattern rewrite library, and eval pair. Prior
|
…vals Correctness fixes (review items 1-5): - awslabs#1: push-computation-to-constant — use NUMERIC column 'amount' to avoid integer division non-equivalence - awslabs#2: not-in-to-not-exists — add NULL semantics warning (NOT EXISTS does not preserve NOT IN's NULL-propagation; MUST confirm with user) - awslabs#3/awslabs#4: subquery-unnesting — prefer EXISTS form (true semi-join); document uniqueness precondition for JOIN+DISTINCT alternative - awslabs#5: subquery-unnesting-scalar — add COALESCE(s_count, 0) for COUNT/SUM (LEFT JOIN returns NULL, scalar returns 0) Dangling reference fixes (review items 6-8): - awslabs#6: workflow.md trigger table — "Phase 5" → reassessment re-entry - awslabs#7: Replace all "implicit cast compatibility matrix" references with "pg_amop query in catalog-queries.md" - awslabs#8: plan-interpretation.md L202 — fix cast-vs-operator contradiction Structural fixes (review items 9-14, 24): - awslabs#9: Hedge "integer family" claim with "at time of writing" + verify - awslabs#10: amopmethod=10003 — add provenance comment and verification SQL - awslabs#11: catalog-queries.md TOC — add 3 missing sections - awslabs#12: plan-interpretation.md TOC — add Type Coercion section - awslabs#13: SKILL.md — explicitly delegate routing to workflow.md - awslabs#24: workflow.md — remove em dashes from headings for clean anchors Other fixes (review items 21-23): - awslabs#21: reltuples-estimate — add staleness warning (MUST warn user) - awslabs#22: catalog-queries — add safe_query.build() note for placeholders - awslabs#23: "Skip when" → "SHOULD skip when" in all rewrite files Eval improvements (review items 14, 16): - awslabs#14: README — add query_plan_rewrite_evals to directory tree and eval section - awslabs#16: Add evals 206-210 covering LEFT JOIN, computation push, NOT IN with NULL warning, nested UNION ALL, and negative case (OR across different columns) - awslabs#7 (eval): Update eval 201 expectation — pg_amop instead of matrix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- awslabs#17: Downgrade eval results to qualitative comparison, record model and version, note n=1 and recommend n>=3 for production confidence - awslabs#18: SKILL.md is 281 lines (will update PR body) - awslabs#20: Strengthen awsknowledge fallback to MUST — refuse fallback when recommendation depends on exact limit value - awslabs#21: Already addressed in prior commit (reltuples staleness) - awslabs#15: Document manual-only status and future Python converter direction (per anwesham-lab's suggestion for deterministic rewrites) - awslabs#19: MCP mirror PR noted as follow-up in PR body Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…vals Correctness fixes (review items 1-5): - awslabs#1: push-computation-to-constant — use NUMERIC column 'amount' to avoid integer division non-equivalence - awslabs#2: not-in-to-not-exists — add NULL semantics warning (NOT EXISTS does not preserve NOT IN's NULL-propagation; MUST confirm with user) - awslabs#3/awslabs#4: subquery-unnesting — prefer EXISTS form (true semi-join); document uniqueness precondition for JOIN+DISTINCT alternative - awslabs#5: subquery-unnesting-scalar — add COALESCE(s_count, 0) for COUNT/SUM (LEFT JOIN returns NULL, scalar returns 0) Dangling reference fixes (review items 6-8): - awslabs#6: workflow.md trigger table — "Phase 5" → reassessment re-entry - awslabs#7: Replace all "implicit cast compatibility matrix" references with "pg_amop query in catalog-queries.md" - awslabs#8: plan-interpretation.md L202 — fix cast-vs-operator contradiction Structural fixes (review items 9-14, 24): - awslabs#9: Hedge "integer family" claim with "at time of writing" + verify - awslabs#10: amopmethod=10003 — add provenance comment and verification SQL - awslabs#11: catalog-queries.md TOC — add 3 missing sections - awslabs#12: plan-interpretation.md TOC — add Type Coercion section - awslabs#13: SKILL.md — explicitly delegate routing to workflow.md - awslabs#24: workflow.md — remove em dashes from headings for clean anchors Other fixes (review items 21-23): - awslabs#21: reltuples-estimate — add staleness warning (MUST warn user) - awslabs#22: catalog-queries — add safe_query.build() note for placeholders - awslabs#23: "Skip when" → "SHOULD skip when" in all rewrite files Eval improvements (review items 14, 16): - awslabs#14: README — add query_plan_rewrite_evals to directory tree and eval section - awslabs#16: Add evals 206-210 covering LEFT JOIN, computation push, NOT IN with NULL warning, nested UNION ALL, and negative case (OR across different columns) - awslabs#7 (eval): Update eval 201 expectation — pg_amop instead of matrix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
122b2a3 to
1178334
Compare
- awslabs#17: Downgrade eval results to qualitative comparison, record model and version, note n=1 and recommend n>=3 for production confidence - awslabs#18: SKILL.md is 281 lines (will update PR body) - awslabs#20: Strengthen awsknowledge fallback to MUST — refuse fallback when recommendation depends on exact limit value - awslabs#21: Already addressed in prior commit (reltuples staleness) - awslabs#15: Document manual-only status and future Python converter direction (per anwesham-lab's suggestion for deterministic rewrites) - awslabs#19: MCP mirror PR noted as follow-up in PR body Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
PR #162 — Multi-agent review (revised after empirical validation on a live DSQL cluster)Reviewed at head SHA
Findings dropped after re-validation (full transcripts available on request):
Empirical context (run on live DSQL cluster):
🤖 Generated with Claude Code If this code review was useful, please react with 👍. Otherwise, react with 👎. |
| @@ -0,0 +1,48 @@ | |||
| # Rewrite: Propagate Filter to JOIN Columns | |||
There was a problem hiding this comment.
I think I still hold to this? Why are we relying on markdown applications alone? Why are these transformations not written into script patterns/tooling?
| | Supported column data types | See docs | `aurora dsql supported data types` | | ||
|
|
||
| **When to verify:** Before recommending batch sizes, connection pool settings, or schema designs where hitting a limit would cause failures; any time the exact number can affect user decision. | ||
| **When to verify:** Before recommending batch sizes, connection pool settings, or schema designs |
There was a problem hiding this comment.
No need to verify for general guidance or when the exact number doesn't affect the user's decision.
rephrase to be prescriptive rather than avoidant prohibition?
|
|
||
| **When to verify:** Before recommending batch sizes, connection pool settings, or schema designs where hitting a limit would cause failures; any time the exact number can affect user decision. | ||
| **When to verify:** Before recommending batch sizes, connection pool settings, or schema designs | ||
| where hitting a limit would cause failures. No need to verify for general guidance or when |
There was a problem hiding this comment.
unnecessary and bad format/practice in skill linebreaks again? LOC consumption occupies additional token usage in skills frequently?
| **Recovery — batch fails midway:** Rows already updated keep their new value (each batch committed | ||
| in its own transaction). Resume by filtering on the unset state — e.g. add | ||
| `WHERE new_column IS NULL` (or the sentinel value) to the next UPDATE — and continue from there. | ||
| Re-running the entire migration is safe because the filter naturally excludes completed rows. |
|
I think it's worth using our self-review skill and doing a couple of explicit passes with subagents deployed from the code-review and pr-toolkit-review plugins to get to an explicit convergence state that you can audit and post to log changes being made. |
|
This pull request is now marked as stale because it hasn't seen activity for a while. Add a comment or it will be closed soon. If you wish to exclude this issue from being marked as stale, add the "backlog" label. |
…vals Correctness fixes (review items 1-5): - awslabs#1: push-computation-to-constant — use NUMERIC column 'amount' to avoid integer division non-equivalence - awslabs#2: not-in-to-not-exists — add NULL semantics warning (NOT EXISTS does not preserve NOT IN's NULL-propagation; MUST confirm with user) - awslabs#3/awslabs#4: subquery-unnesting — prefer EXISTS form (true semi-join); document uniqueness precondition for JOIN+DISTINCT alternative - awslabs#5: subquery-unnesting-scalar — add COALESCE(s_count, 0) for COUNT/SUM (LEFT JOIN returns NULL, scalar returns 0) Dangling reference fixes (review items 6-8): - awslabs#6: workflow.md trigger table — "Phase 5" → reassessment re-entry - awslabs#7: Replace all "implicit cast compatibility matrix" references with "pg_amop query in catalog-queries.md" - awslabs#8: plan-interpretation.md L202 — fix cast-vs-operator contradiction Structural fixes (review items 9-14, 24): - awslabs#9: Hedge "integer family" claim with "at time of writing" + verify - awslabs#10: amopmethod=10003 — add provenance comment and verification SQL - awslabs#11: catalog-queries.md TOC — add 3 missing sections - awslabs#12: plan-interpretation.md TOC — add Type Coercion section - awslabs#13: SKILL.md — explicitly delegate routing to workflow.md - awslabs#24: workflow.md — remove em dashes from headings for clean anchors Other fixes (review items 21-23): - awslabs#21: reltuples-estimate — add staleness warning (MUST warn user) - awslabs#22: catalog-queries — add safe_query.build() note for placeholders - awslabs#23: "Skip when" → "SHOULD skip when" in all rewrite files Eval improvements (review items 14, 16): - awslabs#14: README — add query_plan_rewrite_evals to directory tree and eval section - awslabs#16: Add evals 206-210 covering LEFT JOIN, computation push, NOT IN with NULL warning, nested UNION ALL, and negative case (OR across different columns) - awslabs#7 (eval): Update eval 201 expectation — pg_amop instead of matrix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
b4db0a3 to
d623705
Compare
- awslabs#17: Downgrade eval results to qualitative comparison, record model and version, note n=1 and recommend n>=3 for production confidence - awslabs#18: SKILL.md is 281 lines (will update PR body) - awslabs#20: Strengthen awsknowledge fallback to MUST — refuse fallback when recommendation depends on exact limit value - awslabs#21: Already addressed in prior commit (reltuples staleness) - awslabs#15: Document manual-only status and future Python converter direction (per anwesham-lab's suggestion for deterministic rewrites) - awslabs#19: MCP mirror PR noted as follow-up in PR body Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
I do not see this? Was it accidentally left out of the PR? The Workflow 9 section currently only links the four Phase-0 files, nothing reaches workflow.md or the rewrites, so validate-references flags all 16 new query-plan files as orphans and the agent never loads them. Curious how the evals work then? We can add to the Workflow 9 section to query-plan/workflow.md to make all 16 reachable (the rewrite indexes chain from there). |
|
|
||
| When a query contains a scalar subquery in the SELECT clause computing an aggregate correlated by equality, rewrite it as a LEFT JOIN with GROUP BY. This reduces repeated subquery executions and enables better join planning. | ||
|
|
||
| **SHOULD apply when:** The scalar subquery is correlated via equality and contains an aggregate function (MAX, MIN, COUNT, SUM). For COUNT and SUM, MUST wrap with `COALESCE(..., 0)` because the LEFT JOIN returns NULL (not 0) for unmatched rows — the scalar subquery returns 0. |
There was a problem hiding this comment.
I think there is a mistake here, the examples look right, but the wording here should probably be:
For COUNT, MUST wrap with COALESCE(..., 0) because COUNT returns 0 for an empty input while LEFT JOIN produces NULL for missing groups. Other aggregates (SUM, MIN, MAX, AVG) MUST NOT be wrapped with COALESCE because scalar aggregate subqueries already return NULL for empty inputs.
What is written is backwards for SUM. Eg. (SELECT SUM(y) FROM S WHERE S.x = R.x) returns NULL over no matching rows which is same as the LEFT JOIN thus wrapping it in COALESCE(..., 0) changes the result (NULL→0).
Only COUNT returns 0 for an empty set.
Tested like:
SUM over no rows : None # NULL matches LEFT JOIN, no COALESCE needed
COUNT over no rows: 0 # COALESCE needed
MAX over no rows : None
So MUST COALESCE should apply to COUNT only.... SUM/MIN/MAX/AVG must not be coalesced or the rewrite diverges from the original.
| "hooks": { | ||
| "PostToolUse": [ | ||
| { | ||
| "matcher": "mcp__aurora-dsql.*__transact", |
There was a problem hiding this comment.
this branch is on the transact hook, but the workflow requires EXPLAIN to run via readonly_query (transact is explicitly forbidden for plan capture).... so it'll never fire. either add a readonly_query matcher or drop it. (also missing a period at the end.)
btw, we should re-run evals after this, I would expect different results.
…ction and rewrite references - Add structured trigger phrases and routing criteria for query plan diagnosis - Add type coercion index bypass detection (implicit cast compatibility matrix) - Extend catalog queries with indexed column type retrieval - Add generic SQL rewrite reference (11 patterns: OR-to-IN, subquery unnesting, etc.) - Add DSQL-specific rewrite reference (reltuples estimate, split large joins for DP threshold) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Extract Workflow 8 (query plan explainability) from SKILL.md into references/query-plan/workflow.md to stay under the 300 LOC limit - Wire query-rewrites-generic.md and query-rewrites-dsql-specific.md into the workflow (Phase 0 load list + Phase 2 evidence gathering) - Add behavioral evals (query_plan_rewrite_evals.json) covering type coercion detection, subquery unnesting, OR-to-IN, GROUP BY pushdown, large join splitting, and reltuples estimation - Add eval results (query_plan_rewrite_eval_results.md) with with-skill vs baseline comparison Validation: - validate-size.py: 275 lines (good) - validate-references.py: 0 broken links, 0 new orphans Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…, RFC keywords
Review feedback from amaksimo:
- Split query-rewrites-generic.md into 11 individual files under
query-rewrites/ subdirectory to reduce context consumption
- Split query-rewrites-dsql-specific.md into individual files
- Convert monolithic files to index tables pointing to sub-files
- Fix DATEADD() SQL Server syntax → PostgreSQL NOW() - INTERVAL
- Flip negative language ("Do not apply") to positive ("Skip when")
- Add RFC keywords (MUST, SHOULD, MAY) throughout
- Remove psql fallback from workflow.md (enforce MCP usage)
- Update plan-interpretation.md recommendation template with RFC language
- Make Phase 0 loading explicit: MUST for core refs, SHOULD for rewrites
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…vals Correctness fixes (review items 1-5): - awslabs#1: push-computation-to-constant — use NUMERIC column 'amount' to avoid integer division non-equivalence - awslabs#2: not-in-to-not-exists — add NULL semantics warning (NOT EXISTS does not preserve NOT IN's NULL-propagation; MUST confirm with user) - awslabs#3/awslabs#4: subquery-unnesting — prefer EXISTS form (true semi-join); document uniqueness precondition for JOIN+DISTINCT alternative - awslabs#5: subquery-unnesting-scalar — add COALESCE(s_count, 0) for COUNT/SUM (LEFT JOIN returns NULL, scalar returns 0) Dangling reference fixes (review items 6-8): - awslabs#6: workflow.md trigger table — "Phase 5" → reassessment re-entry - awslabs#7: Replace all "implicit cast compatibility matrix" references with "pg_amop query in catalog-queries.md" - awslabs#8: plan-interpretation.md L202 — fix cast-vs-operator contradiction Structural fixes (review items 9-14, 24): - awslabs#9: Hedge "integer family" claim with "at time of writing" + verify - awslabs#10: amopmethod=10003 — add provenance comment and verification SQL - awslabs#11: catalog-queries.md TOC — add 3 missing sections - awslabs#12: plan-interpretation.md TOC — add Type Coercion section - awslabs#13: SKILL.md — explicitly delegate routing to workflow.md - awslabs#24: workflow.md — remove em dashes from headings for clean anchors Other fixes (review items 21-23): - awslabs#21: reltuples-estimate — add staleness warning (MUST warn user) - awslabs#22: catalog-queries — add safe_query.build() note for placeholders - awslabs#23: "Skip when" → "SHOULD skip when" in all rewrite files Eval improvements (review items 14, 16): - awslabs#14: README — add query_plan_rewrite_evals to directory tree and eval section - awslabs#16: Add evals 206-210 covering LEFT JOIN, computation push, NOT IN with NULL warning, nested UNION ALL, and negative case (OR across different columns) - awslabs#7 (eval): Update eval 201 expectation — pg_amop instead of matrix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- awslabs#17: Downgrade eval results to qualitative comparison, record model and version, note n=1 and recommend n>=3 for production confidence - awslabs#18: SKILL.md is 281 lines (will update PR body) - awslabs#20: Strengthen awsknowledge fallback to MUST — refuse fallback when recommendation depends on exact limit value - awslabs#21: Already addressed in prior commit (reltuples staleness) - awslabs#15: Document manual-only status and future Python converter direction (per anwesham-lab's suggestion for deterministic rewrites) - awslabs#19: MCP mirror PR noted as follow-up in PR body Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…evals - Revert broken relative links in SKILL.md (mcp/.mcp.json, scripts/) back to correct ../../ paths - Rename blacklisted_customers → excluded_customers and blacklist → exclusion_list for inclusive language compliance - Fix stale 'implicit cast compatibility matrix' → pg_amop in eval results - Add eval results for 206–210 (LEFT JOIN, computation push, NOT IN NULL warning, nested UNION ALL, negative OR case) - Include hooks.json update
Addresses amaksimo's review comment: after rebase onto main, the Workflow 9 section lost its link to workflow.md and the rewrite index files, making all 16 new query-plan files unreachable orphans. - Add workflow.md as the entry gate in the reference table - Add query-rewrites-generic.md and query-rewrites-dsql-specific.md - Update Workflow 9 section to load workflow.md instead of listing the 4 Phase-0 files directly (workflow.md handles that routing)
…wslabs#8, awslabs#9 - awslabs#4: COALESCE rule in subquery-unnesting-scalar.md restricted to COUNT only; SUM/MAX/MIN return NULL on empty sets in both forms - awslabs#5: verify-comment in catalog-queries.md changed from amname='btree' to amname='btree_index' (DSQL uses btree_index AM) - awslabs#7: workflow.md context disambiguation removes psql fallback offer, now says no MCP means no plan capture (consistent with Safety) - awslabs#8: split-large-joins.md example expanded from 7 to 11 tables, exceeding the stated DP threshold of 10; CTEs project explicit cols - awslabs#9: plan-interpretation.md recommendation template changed ::float to ::integer (only integer-family cross-type operators registered)
Lift the substitution rule to the file preamble with per-position
guidance: identifier positions (FROM, GROUP BY) use ident(); string-
literal positions (WHERE = '{schema}', IN ('{table}')) use allow()
or regex(). The prior note incorrectly prescribed ident() for all
positions, which would produce invalid SQL in WHERE clauses.
381104b to
cb8f81b
Compare
…ncorrelated Delete redundant in-subquery-to-exists.md — same input shape and same rewrite as subquery-unnesting-uncorrelated.md. Merge the 'large result set' trigger and 'small static set' skip condition into the canonical file. Update index and eval 200 to route exclusively there.
Summary
references/query-plan/workflow.md(SKILL.md: 334 → 281 LOC)plan-interpretation.md, indexed column type queries incatalog-queries.mdquery-rewrites/, plus 2 DSQL-specific rewrites (reltuples estimate, split large joins)Validation
validate-size.py: 281 lines (good, under 300 limit)validate-references.py: 0 broken links, 0 new orphansEval Results
Manual qualitative comparison (n=1, Claude Opus 4.6). Full results in
tools/evals/databases-on-aws/dsql/query_plan_rewrite_eval_results.md:Follow-ups
awslabs/mcpsrc/aurora-dsql-mcp-server/skills/dsql-skill/needs to be synced with these changes (workflow.md, query-rewrites/ split, updated catalog-queries.md, plan-interpretation.md). Will open companion PR after this merges.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.
🤖 Generated with Claude Code