Skip to content

fix: Chunk IN clause to <1000 items for oracle compatibility - #2713

Merged
blizzz merged 3 commits into
mainfrom
fix/noid/oracleInClauseFix
Jun 9, 2026
Merged

fix: Chunk IN clause to <1000 items for oracle compatibility#2713
blizzz merged 3 commits into
mainfrom
fix/noid/oracleInClauseFix

Conversation

@AndyScherzinger

Copy link
Copy Markdown
Member

Assisted-by: Claude Code:claude-sonnet-4-6

馃弫 Checklist

  • 鉀戯笍 Tests (unit and/or integration) are included or not needed
  • 馃敊 Backport requests are created or not needed: /backport to stableX.X
  • 馃搮 Milestone is set
  • 馃尭 PR title is meaningful (if it should be in the changelog: is it meaningful to users?)

馃 AI (if applicable)

  • The content of this PR was partly or fully generated using AI

@AndyScherzinger AndyScherzinger added the bug Something isn't working label Jun 8, 2026
Comment thread lib/Db/Row2Mapper.php Outdated
}

$allRows = [];
foreach (array_chunk($rowIds, 997) as $rowIdChunk) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doubtful about this hardcoded number. The query consumes two arrays. ColumnId is unlikely to have more than 1k entries, but definitely more than 3. So should be 1k - count($columnIds). Then I am not sure how Oracle treats the join, but reducing the other 2 seems safe then: 1k - count($columnIds) -2.

The bad thing is that the -2 cannot be computed from here, as it is a detail of the called method.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the triggered query has 2 INs while each can have up to 1K items, so we could probably chunk by 1000 if you like. The columns are not relevant for chunking as you said and currently not chunked, just the row IDs, no?

Else if unsure about how Oracle treats multiple INs in a query and if all of them need to be max 1k in total than we could follow your approach of 1k - count($columnIds) -2

https://stackoverflow.com/questions/2401066/oracle-sql-how-to-use-more-than-1000-items-inside-an-in-clause suggests that many INS are fine as long each is max 1K items.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are passed here:

->where($qb->expr()->in('column_id', $qb->createNamedParameter($columnIds, IQueryBuilder::PARAM_INT_ARRAY, ':columnIds')))
->andWhere($qb->expr()->in('row_id', $qb->createNamedParameter($rowIds, IQueryBuilder::PARAM_INT_ARRAY, ':rowsIds')));

IIRC the 1k is a global threshold for the query. But I might mix it up with another DB. Quick search shows a global limit of 32767 arguments in Postgres, while Oracle should have the 1k limit per IN clause.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the info based on your updated fix via b37f358

Assisted-by: Claude Code:claude-sonnet-4-6
Signed-off-by: Andy Scherzinger <info@andy-scherzinger.de>
@AndyScherzinger
AndyScherzinger force-pushed the fix/noid/oracleInClauseFix branch from f79864d to 84caa49 Compare June 8, 2026 14:34
blizzz and others added 2 commits June 9, 2026 00:07
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
Assisted-by: Claude Code:claude-sonnet-4-6
Signed-off-by: Andy Scherzinger <info@andy-scherzinger.de>
@AndyScherzinger
AndyScherzinger force-pushed the fix/noid/oracleInClauseFix branch from 20be00e to b37f358 Compare June 8, 2026 22:12
@AndyScherzinger AndyScherzinger added this to the v2.3.0 milestone Jun 8, 2026
@blizzz
blizzz merged commit e003740 into main Jun 9, 2026
75 checks passed
@blizzz
blizzz deleted the fix/noid/oracleInClauseFix branch June 9, 2026 06:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI assisted bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants