Enhance performance and memory efficiency in ExcelReader and refactor async processing - #27
Merged
Conversation
added 3 commits
July 11, 2026 07:46
- Refactored XlsxReader to optimize XML head classification, reducing unnecessary span comparisons. - Introduced FastDouble for efficient parsing of plain decimal numbers, improving numeric text handling. - Updated shared string parsing to use pooled byte arrays, minimizing memory allocations. - Adjusted CsvWriter and SheetWriter to lower flush thresholds, optimizing memory usage during large writes. - Enhanced cell formatting methods to streamline XML generation for various data types. - Added tests for FastDouble to ensure accuracy and reliability in parsing. - Improved benchmarks by pre-sizing MemoryStreams to reduce garbage collection impact.
- Improved performance of MoveNextAsync methods across various readers (ExcelEnumerable, CsvReader, XlsbReader, XlsxReader) by implementing non-async fast paths that handle most cases synchronously, reducing overhead from async state machines. - Introduced new methods to handle buffer misses more efficiently, allowing for immediate returns when data is available. - Enhanced Biff12RecordWriter to support fixed record writing with a single reservation for the entire record, optimizing memory usage. - Updated RowWriter to implement IDisposable, allowing for synchronous disposal and improved resource management. - Added synchronous counterparts for StartRow in SheetWriter and corresponding extensions to streamline row writing without async overhead. - Benchmarks updated to measure performance improvements with synchronous row writing methods.
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #27 +/- ##
==========================================
- Coverage 90.79% 89.22% -1.57%
==========================================
Files 76 77 +1
Lines 5061 5345 +284
Branches 880 956 +76
==========================================
+ Hits 4595 4769 +174
- Misses 294 377 +83
- Partials 172 199 +27 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Benchmark ResultsMeasured on ExcelReader.Benchmarks.CsvParseBenchmark
ExcelReader.Benchmarks.CsvReadBenchmark
ExcelReader.Benchmarks.CsvWriteBenchmark
ExcelReader.Benchmarks.ParseBenchmark
ExcelReader.Benchmarks.ReadBenchmark
ExcelReader.Benchmarks.WriteBenchmark
ExcelReader.Benchmarks.XlsReadBenchmark
ExcelReader.Benchmarks.XlsWriteBenchmark
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant optimizations to the CSV and Excel row enumeration code, focusing on reducing allocations and improving performance for both synchronous and asynchronous reading paths. The most important changes are:
Performance and Allocation Optimizations
CsvReader.Enumeratorto minimize per-field allocations and instance field accesses by introducing a localFieldStatestruct, and added a fast path for unquoted lines to avoid unnecessary machinery. [1] [2]ExcelEnumerable.AsyncEnumeratorandCsvEnumerable.AsyncEnumeratorby implementing a fast path that avoids unnecessary state machines for the common case where rows are already buffered, only falling back to async/await when truly needed. [1] [2] [3]Documentation and Benchmark Updates
README.mdto reflect new benchmark results, showing improved performance and reduced allocations forExcelReadercompared to other libraries, and revised the summary analysis to match the new results.These changes together result in faster CSV/XLSX reading and writing, lower memory allocations, and more accurate documentation of performance characteristics.