fix: SQLite 大批次查询与写入回滚 / SQLite chunked queries and rollback - #1217
Open
mfurkanakinci wants to merge 7 commits into
Open
mfurkanakinci wants to merge 7 commits into
mfurkanakinci wants to merge 7 commits into
Conversation
- IN 查询按块执行,避免超过 SQLite 999 变量上限 - Chunk IN queries so batches stay under SQLite's 999-variable limit - 写入失败时回滚,并以 StorageError 抛出不可忽略的存储错误 - Roll back failed writes and raise StorageError for storage failures that must not be ignored
- IN 查询按块执行,避免超过 SQLite 999 变量上限 - Chunk IN queries so batches stay under SQLite's 999-variable limit - 写入失败时回滚,并以 StorageError 抛出不可忽略的存储错误 - Roll back failed writes and raise StorageError for storage failures that must not be ignored
- IN 查询按块执行,避免超过 SQLite 999 变量上限 - Chunk IN queries so batches stay under SQLite's 999-variable limit - 写入失败时回滚,并以 StorageError 抛出不可忽略的存储错误 - Roll back failed writes and raise StorageError for storage failures that must not be ignored
- IN 查询按块执行,避免超过 SQLite 999 变量上限 - Chunk IN queries so batches stay under SQLite's 999-variable limit - 写入失败时回滚,并以 StorageError 抛出不可忽略的存储错误 - Roll back failed writes and raise StorageError for storage failures that must not be ignored
- IN 查询按块执行,避免超过 SQLite 999 变量上限 - Chunk IN queries so batches stay under SQLite's 999-variable limit - 写入失败时回滚,并以 StorageError 抛出不可忽略的存储错误 - Roll back failed writes and raise StorageError for storage failures that must not be ignored
- IN 查询按块执行,避免超过 SQLite 999 变量上限 - Chunk IN queries so batches stay under SQLite's 999-variable limit - 写入失败时回滚,并以 StorageError 抛出不可忽略的存储错误 - Roll back failed writes and raise StorageError for storage failures that must not be ignored
- IN 查询按块执行,避免超过 SQLite 999 变量上限 - Chunk IN queries so batches stay under SQLite's 999-variable limit - 写入失败时回滚,并以 StorageError 抛出不可忽略的存储错误 - Roll back failed writes and raise StorageError for storage failures that must not be ignored
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
中文
大批次写入时,SQLite 的
IN查询会超过 999 个变量的上限并导致整批失败。写入失败时也没有回滚,错误被宽泛地吞掉。trendradar/utils/sqlite.py,把IN ({placeholders})按块执行(每块 500 个值)IN查询改走这条路径StorageError抛出不可忽略的存储错误tests/test_sqlite_chunked_in.py覆盖超过旧上限的IN查询,tests/test_sqlite_mixin.py覆盖写入、合并、排名历史和 AI 标签生命周期。English
Large batches can exceed SQLite's 999-variable limit on
INqueries and crash the write. Failed writes were also left unrolled back, and the error was swallowed.trendradar/utils/sqlite.pyand runIN ({placeholders})in chunks of 500 valuesINqueries in hotlist storage, RSS storage, and the MCP parserStorageErrorfor storage failures that must not be ignoredtests/test_sqlite_chunked_in.pycovers anINquery past the old limit.tests/test_sqlite_mixin.pycovers writes, merges, rank history, and the AI-tag lifecycle.