Repository navigation
PivotTables: files with TRUE/FALSE sources open in Excel, spaced headers keep their fields, report filters, date groups, PivotCharts and Show Values As match Excel 365 - #1067
Closed
iluebbe wants to merge 13 commits into
Conversation
…GridCraft lays them out With two or more value fields and no column field, GridCraft lays out the value captions as the header row. Excel 365 does the same for its own PivotTables, because they turn "Show the Values row" off (x14:pivotTableDefinition hideValuesRow="1"). GridCraft's files didn't say so, so Excel added a "Values" caption row on refresh: the report moved down a row, and anything in the row below GridCraft's report left Excel showing only #SPILL! in its place. The writer now turns the Values row off as Excel does, and writes the location Excel writes for such a report (firstHeaderRow 0, firstDataRow 1). With headers off, the caption row above column fields is left out when it would be empty, as in Excel (firstHeaderRow 0). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
… in Excel When every value of an item is blank, Excel 365 leaves its value cells empty for every function. GridCraft showed 0 for Sum, Count, Max, Min and Product and #DIV/0! for Average, StdDev and Var, which reads as data that is 0 or as an error. Shown as a share of a total or as a running total, such an item still counts as 0, as Excel shows it (0 % and the carried running total). Calculated fields keep reading a blank sum as 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
…a, as in Excel Where a row item and a column item have no records together, Excel 365 shows 0 % for % of Grand Total, % of Column Total and % of Row Total, and carries the running total into the cell (0 when nothing came before). GridCraft left these cells empty, because the Show Values As rules ran only for intersections with an aggregate. For Running Total, Excel also leaves the Grand Total line empty; GridCraft filled it with the plain totals. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
Every row and column field was written with sortType="ascending", the default of a new field. For a field grouped by days, Excel 365 reads that as an A-to-Z sort of the item labels, so a GridCraft pivot opened and refreshed in Excel showed 01. Aug, 01. Feb, 01. Jan, 15. Jan. Excel's own date groups have no sortType and keep the items' date order. A date group sorted ascending is now written without sortType (its items are written in date order), and a date group without sortType reads back as ascending. Descending and the sort of other fields are written as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
A PivotChart names its PivotTable in <c:pivotSource><c:name>. GridCraft wrote it as Sheet2!PivotTable1; Excel 365 binds the chart only when the name starts with a workbook in brackets ([Book.xlsx]Sheet2!PivotTable1), and otherwise opens it as an ordinary chart over fixed cells (Chart.PivotLayout is Nothing). The writer now puts the saved file's name in front (passed down from the save as write_xlsx_named), or a placeholder when there is none: Excel accepts any bracketed name. The reader drops the workbook part, so the model keeps Sheet2!PivotTable1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
…on rows, and in outline, as in Excel Laid out as Excel 365 refreshes them: - Column fields and one value field, no row field: the value's caption labels the data line (A5), not the top-left cell (A3). - Column fields and two or more value fields, no row field: there is no label column; the report starts with the first column item (firstDataCol 0). - Σ Values on rows without a column field: the data column has no header. GridCraft wrote "Total" there; Excel leaves it empty after a refresh. - Outline and tabular: the caption row names every column field, Σ Values as "Values" (B3 Product, C3 Values); GridCraft named only the first. - With headers off, a caption row that would be empty is left out (columns only, one value). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
…rror values A pivot cache field's <sharedItems> flags said containsString="0" for a column holding booleans or errors, and containsMixedTypes="1" for text mixed with booleans. Excel 365 counts booleans and errors as strings in these flags and can't open a file that contradicts its own items, not even with repair; any such column in the source was enough, used in the pivot or not. The flags are now written as Excel writes them: strings are text, booleans and errors; containsString="0" and containsSemiMixedTypes="0" only without them, and mixed types only for strings together with numbers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
… on refresh or save
A source header with leading or trailing spaces ("Region ") names its field with the spaces in
Excel, in the cache and in the layout. GridCraft trimmed headers when reading the source and when
writing the cache, while the fields read from an Excel file kept the spaces, so the layout no
longer found them: a refresh dropped the field, and even a plain open and save wrote the pivot
without its row fields and value fields.
Headers are now taken as they are, as Excel does. A name typed without the spaces
(pivot.addField {"field": "Region"}) still finds the field when only one header matches that way,
and is stored with the header's spelling.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
pageField@item is an index into the pivot field's <items>; the item there names the shared item with its x. GridCraft took it as an index into the cache's shared items, so a filter on Qty = 3 read back as Qty = 2 whenever Excel's items weren't in shared-item order, and an open and save wrote that wrong item back, changing the report's values in Excel. The reader now goes through the field's items. The writer was right already: it writes the items in shared-item order, so its index and the shared item agree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
…in Excel The cache field of a date grouping was always written with containsSemiMixedTypes="0", also next to containsBlank="1". Excel 365 then dropped the grouping when it refreshed a GridCraft file whose date column had a blank cell: a pivot by months showed every single date instead. As Excel writes it, a date column with blanks is semi-mixed: containsSemiMixedTypes="0" is now written only when the column has no blank cells. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
…, as in Excel Excel 365 ranks densely: values 5, 4, 3, 3, 4 rank 1, 2, 3, 3, 2. GridCraft counted every larger sibling, so the same values ranked 1, 2, 4, 4, 2. A value's rank is now one more than the number of distinct larger values among its siblings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
…t Numbers → "Count of") The caption word and the names of the summary functions lived in three places: the engine's func_caption, the XLSX writer's func_caption/func_name/func_from, and the commands' func_param/ func_name. They had drifted: a new Count Numbers value field was captioned "Count Numbers of Units" by the engine, while Excel 365 (and the XLSX reader's fallback) say "Count of Units" (German Excel: "Anzahl von Units", the same as for Count). PivotFunc now holds them: caption() with Excel's words for all 11 functions, name() for commands and listings, file_name() for dataField@subtotal, and parse() for all of these plus the aliases the commands accepted. The engine and the XLSX reader and writer use them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
gaps.md §1 and §8 and the PivotTables row of target-app-parity.md said the PivotTable round trip was never checked in Excel. PivotTables made in GridCraft, one feature per file, have now been opened and refreshed in Excel 365; the differences that turned up are fixed in this branch. No score change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
I checked PivotTables against Excel 365 on Windows (German UI) in two directions:
I found no existing issue or PR for the bugs below; overlaps with open PRs are listed at the end. Labels are English here; German Excel shows
Zeilenbeschriftungen,Gesamtergebnisand so on.Excel can't open the file, or fields disappear:
range.setValuesofName, Flag, Sales/a, true, 1/b, false, 2/a, true, 3, theninsert.pivotTable {"source":"Sheet1!A1:C4","rows":["Name"],"values":["Sales"]}. The same file withoutFlagopens.containsString="0"on a booleans or errors field, andcontainsMixedTypes="1"on text plus booleans. Excel writes neither."Region ","Sales ").Regionand valuesS= Sum ofSalesandQ= Sum ofQtyrefreshes to justQ / 6.LIBRE_OFFICE-116306-0.xlsxkeeps 8 of its 14 value fields afterconvert; the 6 dropped ones end in a space.Wrong numbers or layout after a round trip:
Qty= 3; GridCraft reads["2"], so the report shows other values (North 80 → east 87.5).A3:B17grows toA3:B24.01. Aug, 01. Feb, 01. Jan, 15. Janafter a refresh in Excel; GridCraft shows date order.Valuescaption row that GridCraft's saved layout doesn't have, so the report grows by a row when Excel lays it out.#SPILL!cell.insert.pivotChartwrote<c:pivotSource><c:name>Sheet2!PivotTable1</c:name>. Excel binds a chart only to a name that starts with a workbook in brackets ([Book.xlsx]Sheet2!PivotTable1), soChart.PivotLayoutwas Nothing.Values that differ from Excel's own refresh:
#DIV/0!for Average, StdDev and Var.Total.Cause and fix
One commit per item:
ebe4742sharedItemsflags counted booleans and errors as kinds of their own (kinds()). They now count as string-like, as in Excel's caches.49a4f72read_source,header_text), but field names from the file kept their spaces, so nothing matched. Headers now keep their spaces; typing a field name without them still finds the field.718e13bpageField@itemas an index into the cache's shared items. It indexes the pivot field's<items>, whosexis the shared item. The writer was already right.b46332acontainsSemiMixedTypes="0"was written on grouped date fields even next tocontainsBlank="1"; now only when the field has no blanks.86e2b70sortType="ascending", which Excel applies to day labels as text. Ascending date groups are now written without it, as Excel writes them, and read back as ascending.fa844acx14:pivotTableDefinition hideValuesRow="1"in its own new pivots and doesn't show theValuesrow then. GridCraft now writes that flag and thelocationoffsets Excel writes, so Excel lays the pivot out as GridCraft does. GridCraft's layout is unchanged.cee9a27[name.xlsx]Sheet!PivotTable, fromwrite_xlsx_named); the reader still drops it.6e27bb4Acc::resultreturned 0 or#DIV/0!for an accumulator that saw no value. Such cells are now empty; as a percentage or running total they count as 0.fc994836e4e1306c65d3d5f9581dmodel::PivotFunc.e87e250gaps.md§1 / §8 andtarget-app-parity.mdnote that GridCraft-made pivots are now checked in Excel 365.Testing
crates/engineandcrates/xlsx, using the cases above.cargo fmt --all -- --check.cargo clippy -p gridcraft-model -p gridcraft-xlsx -p gridcraft-engine -p gridcraft-chart -p gridcraft-cli --all-targets -- -D warnings.cargo test -p gridcraft-model -p gridcraft-xlsx -p gridcraft-engine: 527 pass.cargo xtask layers,assetsandwasmpass.cargo checkpasses for each commit.#SPILL!case included).PivotLayoutset.Not fixed here:
selects, numbers also match by their plain text), so I left it to PivotTables: slicers from XLSX files are shown, filter their PivotTables and are saved #719.<n>,numFmtId 0) where Excel writes<d>. The same happens on main.<15.01.2023(its first group); after a GridCraft save Excel shows(blank).insertBlankRow) is still dropped on refresh and save. That's a missing feature.Overlaps
#722, #474, #498 and #719 change the same five files (
engine/src/cmd/pivot.rs,engine/src/pivot.rs,model/src/features.rs,xlsx/src/pivot.rs,xlsx/src/tests/pivot_tests.rs). #684 changesengine/src/pivot.rs,model/src/features.rsandxlsx/src/chart.rs.#722 also reads and writes
hideValuesRow(asshow_values_row), and it adds theValuesrow for outline and tabular layouts. Whichever PR lands second should keep a singleshow_values_rowthat this PR's writer sets to hidden for new pivots.🤖 Generated with Claude Code
https://claude.ai/code/session_01SAgnkqJiy3v24Hp25dQLRj