refactor: move file reading into lint workers#74
Merged
Conversation
d2aec60 to
71d2884
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
摘要
将
batchLint从「主线程预读取全部文件 → 静态分组 → 发送 contentList 给 worker」改为「主线程只发送单文件路径 → worker 自行读取并 lint」,消除主线程对全部文件内容的驻留,并删除有缺陷的averaged-group.ts。依赖:PR1(#69,已合并)建立的内存基准。
跟踪 issue:#73
变更
src/types.ts—LintWorkerOptions.contentList替换为filePath;BatchLintItem补fixedResult字段以保持现有 fix 模式行为不变。src/utils/lint-worker.ts— worker 接收filePath,自行readFile后lintMarkdown,返回{ path, lintResult, fixedResult }。src/utils/batch-lint.ts— 删除fileContentList预读取和averagedGroup分组;用runTasksWithLimit派发单文件任务;try/finally中调用lintWorkerPool.destroy();worker 路径解析兼容测试/生产环境。src/utils/averaged-group.ts— 删除。__tests__/batch-lint.spec.ts— 新增 10 个 PR2 专项测试。验收条件
contentListfilePath)threadsrunTasksWithLimit(tasks, concurrency))try/finally)lintMarkdown结果)runTasksWithLimit保持顺序)PR1 基线 vs PR2 复测
测试环境:Node.js 24.15.0,16 个文件、每个约 1,000,010 字节(总输入约 16 MB),刻意构造的超长英文段落,
tsx启动 CLI。每组 3 次取中位数。threads=1 时 RSS 下降来自主线程不再保留
fileContentList和averagedGroup分组结构。threads=4 时 RSS 持平印证了 PR1 结论:剩余峰值由解析器 AST/临时对象贡献,PR5(基准驱动)再评估。wall time 改善来自去掉主线程readFile串行阶段和averagedGroup排序/分组过程。报告顺序变化
旧实现按
averagedGroup分配的线程组顺序返回结果(组内按文件输入顺序,组间按组序号);新实现按文件输入顺序返回。已被__tests__/batch-lint.spec.ts中「报告顺序」测试断言。测试
__tests__/batch-lint.spec.ts现有 14 个测试(原 4 个 + PR2 新增 10 个),按验收条件分组:contentListthreadsbatch-lint.ts行覆盖率从 42.42% 提升到 96.42%。验证
下一步
PR3(可选输入字节上限)、PR4(fix 结果在 worker 内立即安全写入)在 PR2 合并后依次实施。不实施: