Let HANA datasets run DC/DQD on the source and fail on incomplete cache builds - #3286
Open
p-hoffmann wants to merge 1 commit into
Open
Let HANA datasets run DC/DQD on the source and fail on incomplete cache builds#3286p-hoffmann wants to merge 1 commit into
p-hoffmann wants to merge 1 commit into
Conversation
p-hoffmann
requested review from
LSriragavan,
SantanM,
Zhimin-arya,
alicia-koh,
brandantck,
csafreen,
hengxian-jiang,
jerome-ng,
khairul-syazwan,
maggie-li-yd and
suwarnoong
as code owners
September 7, 2026 19:58
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Source routing has dialect edge cases, the FTS completeness check can miss a missing macro, and new jobplugin tests are not discovered by CI.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds source-database execution for HANA DC/DQD flows and improves cache-build failure diagnostics.
Changes:
- Propagates source-connection options through DC and DQD.
- Adds FTS verification and DuckDB checkpointing.
- Improves CDM and credential validation errors.
File summaries
| File | Description |
|---|---|
plugins/functions/jobplugins/src/types.ts |
Adds the DC source-connection option. |
plugins/functions/jobplugins/src/services/DqdService.ts |
Passes source routing to DQD. |
plugins/functions/jobplugins/src/services/dcTarget.ts |
Resolves DC execution targets. |
plugins/functions/jobplugins/src/services/dcTarget.test.ts |
Adds target-resolution tests. |
plugins/functions/jobplugins/src/services/DataCharacterizationService.ts |
Applies DC source routing. |
plugins/functions/analytics-svc/src/api/controllers/dbsvc.ts |
Improves CDM version errors. |
plugins/flows/base/dqd_plugin/types.py |
Maps source routing to connection mode. |
plugins/flows/base/dqd_plugin/tests/test_types.py |
Tests DQD connection selection. |
plugins/flows/base/dqd_plugin/tests/__init__.py |
Initializes the test package. |
plugins/flows/base/create_cachedb_file_plugin/utils.py |
Adds DuckDB checkpoint support. |
plugins/flows/base/create_cachedb_file_plugin/fts.py |
Verifies and checkpoints FTS indexes. |
plugins/flows/_shared_flow_utils/dao/daobase.py |
Rejects duplicate database credentials. |
Review details
- Files reviewed: 11/12 changed files
- Comments generated: 4
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+175
to
+180
| if not write_conn.fetchall(): | ||
| raise RuntimeError( | ||
| f"FTS index '{database_name}.{index_schema}' is incomplete: its " | ||
| f"'stats' table is missing, so {index_schema}.match_bm25 does not " | ||
| f"exist. The index build for '{schema_name}.{table_name}' did not " | ||
| "finish; re-run it rather than treating the schema as built." |
| cohortDefinitionId, | ||
| releaseId, | ||
| releaseDate, | ||
| useSourceConnection: useSourceConnection ?? false, |
| // (which already uppercases for HANA and lowercases for postgres) -- only | ||
| // webapi datasets need the verbatim registered name. | ||
| return { | ||
| useSourceConnection: requestedUseSourceConnection, |
| assertEquals(target, { useSourceConnection: false, resultsSchema: null }); | ||
| }); | ||
|
|
||
| Deno.test("a non-webapi dataset can opt into the source connection", () => { |
p-hoffmann
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.