diff --git a/ROADMAP.md b/ROADMAP.md index 070552d07..f0c860322 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,8 +1,8 @@ # GridCraft roadmap -**Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~160–240 h away +**Stage: alpha** · next: beta, ~25 points of "ready for real work" and ~150–230 h away -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-12 · **Change:** minor (rows auto-fit oversized fonts on open and whole-row/column formatting) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** minor (PivotTables: Group Selection, time grouping, calculated items, timelines, label and value filter menus; GETPIVOTDATA and the PivotTable tabs) · **Target:** Microsoft Excel (Microsoft 365) GridCraft aims at full Microsoft Excel parity, and at being better in places: faster, open (XLSX is the native format), fully scriptable by agents (MCP, control channel, CLI), and on the web. @@ -21,11 +21,11 @@ This page is the summary. The detail is in: | Number | Value | Kind | |---|---|---| -| **Feature breadth** (weighted) | **~75%** | estimated, anchored on two measurements: ribbon catalog 274 / 290 (94.5%), worksheet functions 513 / 523 (98.1%; 504 working, 96.4%) | -| **Ready for real work** | **~50%** | estimated from written weights (51.6); [target-app-parity.md](docs/target-app-parity.md) | +| **Feature breadth** (weighted) | **~75%** | estimated, anchored on two measurements: ribbon catalog 292 / 307 (95.1%), worksheet functions 514 / 523 (98.3%; 505 working, 96.6%) | +| **Ready for real work** | **~50%** | estimated from written weights (52.0); [target-app-parity.md](docs/target-app-parity.md) | | **Mainstream practitioner** | **~50%** | estimated: typical analyst/finance user, weekly features, × 0.90 interaction × 0.92 stability × 0.88 file exchange; [method](docs/target-app-parity.md#mainstream-practitioner) | | **Essentials user** | **~63%** | estimated: core features only, × 0.92 launch/stability × 0.95 clarity × 0.90 opening sent files; [method](docs/target-app-parity.md#essentials-user) | -| Remaining to **beta** | **~160–240 Opus 5.5 agent-hours** | estimated; itemized in [roadmap.md](docs/roadmap.md#beta-gates) | +| Remaining to **beta** | **~150–230 Opus 5.5 agent-hours** | estimated; itemized in [roadmap.md](docs/roadmap.md#beta-gates) | | Remaining to **full parity** | **~700–1,100 Opus 5.5 agent-hours** | estimated; sum of the dimension rows below | | Audience | Ready % | Opus 5.5 agent wall-clock hours to ~95% | Work that dominates | @@ -78,7 +78,7 @@ collaboration, mobile, AI). | Editing, navigation, clipboard, fill | ~90% | ~70% | 15–25 | [ui-parity.md](docs/ui-parity.md) | | Cell formatting, styles, conditional formatting | ~95% | ~80% | 8–14 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | | Sort, filter, tables, validation, outline | ~82% | ~70% | 10–16 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | -| PivotTables | ~65% | ~45% | 25–40 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | +| PivotTables | ~78% | ~58% | 10–16 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | | Charts and sparklines | ~50% | ~30% | 40–60 | [chart-parity.md](docs/chart-parity.md) | | Page layout, print, PDF | ~80% | ~55% | 12–20 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | | What-if and analysis (Goal Seek, Solver, ToolPak, Forecast) | ~50% | ~40% | 22–35 | [target-app-parity.md](docs/target-app-parity.md#feature-areas) | @@ -132,6 +132,11 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | What landed | |---|---| +| 2026-10-14 | PivotTables: timelines (Insert › Timeline, PivotTable Analyze › Insert Timeline, the Timeline tab) filter PivotTables by a period of a date field and round-trip through XLSX timeline parts (Excel shows the timeline but does not apply its saved selection yet) | +| 2026-10-14 | PivotTables: calculated items (Fields, Items & Sets › Calculated Item, Solve Order, List Formulas) with Excel's totals and restrictions, loaded from and saved to XLSX (Excel itself does not load the saved items yet) | +| 2026-10-14 | PivotTables: Group Selection and Ungroup (`Region2` / `Group1`), date and time grouping at several levels with the Grouping dialog's Starting at, Ending at and Number of days, with XLSX round-trip | +| 2026-10-14 | PivotTables: the report's field buttons and the field list open Label Filters ▸ / Value Filters ▸ and a Top 10 dialog; Top N keeps items tied at the cut-off (#498 review) | +| 2026-10-14 | PivotTables: GETPIVOTDATA (value field by caption or source field, row, column and report filter items, Excel's rules for totals and hidden items, recalculated on refresh, evaluated on open), Generate GetPivotData (clicking or arrowing onto a PivotTable value while typing a formula inserts it; on by default, also in File › Options › Formulas), and the PivotTable Analyze and Design contextual tabs in Excel 365's layout with German labels and key tips JT/JY | | 2026-10-12 | Rows auto-fit oversized fonts: opening a file and whole-row/column formatting grow non-custom rows to their tallest font (#833) | | 2026-10-12 | Number formats: parsing a format code is linear in its length (long runs of `[`, `,` or `/` no longer take quadratic time), and an unclosed `[` is a literal when splitting sections as well as when tokenizing (#810) | | 2026-10-12 | Number formats: a fraction denominator with 65,536 zeros no longer crashes; format codes are capped at 255 characters like Excel (longer codes display as General, `TEXT` gives `#VALUE!`) (#807) | @@ -148,6 +153,9 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | 2026-10-12 | Ink strokes save as unfilled DrawingML paths and reopen as editable ink with their points, colour and width; icons in #551 remain a follow-up | | 2026-10-12 | The selection outline stays visible when extended to the last row (Ctrl+Shift+Down): its sides are cut to the viewport instead of stroking a 20-million-point rectangle (#687) | | 2026-10-12 | File › Print selects a system printer, copies, duplex and grayscale, submits PDF jobs through craft-print on a worker and shows spooler errors; browser printing and PDF export remain available | +| 2026-10-12 | PivotTables: label filters (with wildcards), value filters and Top 10 (items, percent, sum) on row and column fields, with XLSX round-trip | +| 2026-10-11 | PivotTables: number fields group into ranges (Starting at, Ending at, By) with XLSX round-trip | +| 2026-10-11 | PivotTables: Show Values As covers all 15 calculations (% of, difference from, % of parent totals, % running total, ranks, index), with base field and base item, and XLSX round-trip | | 2026-10-11 | Command-sequence fuzzing: random command sequences checked against undo/redo, XLSX round-trip and full recalculation; fixes for the table, legacy-array, spill, AutoSum, Watch Window and sheet-copy bugs it found | | 2026-10-11 | Password-protected workbooks open: GridCraft asks for the password and decrypts Agile and Standard encryption (#11) | | 2026-10-11 | Text boxes keep their formatting: run size, bold, italic, typeface, text colour and outline, alignment, vertical anchor and "no outline" (#377) | @@ -193,6 +201,8 @@ Ranked; estimates in Opus 5.5 agent-hours. Detail and the full beta-gate list in | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | PivotTables ~65/~45% → ~78/~58% (Group Selection, time grouping, calculated items, timelines, filter menus); full number 51.6 → 52.0; beta ~160–240 → ~150–230 h (gate #8 20–30 → 10–16 h); four progress log rows | +| 2026-10-14 | minor | Progress log row for GETPIVOTDATA and the PivotTable tabs | | 2026-10-12 | minor | Rows auto-fit oversized fonts on open and whole-row/column formatting (#833); progress log row | | 2026-10-12 | minor | Progress log: multi-area row/column delete (#758) | | 2026-10-12 | trivial | Progress log: XLSX chart sheets kept on save (#727) | diff --git a/crates/calc/src/eval.rs b/crates/calc/src/eval.rs index 0f9cfb03d..3dd52af3f 100644 --- a/crates/calc/src/eval.rs +++ b/crates/calc/src/eval.rs @@ -18,6 +18,8 @@ pub const MAX_CELLS: u64 = 16_000_000; /// formulas over them stay fast. const FULL_SIZE_CELLS: u64 = 16_384; const MAX_DEPTH: usize = 400; +/// GETPIVOTDATA takes a value field, a PivotTable and up to 126 field/item pairs. +const MAX_GETPIVOTDATA_ARGS: usize = 254; #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] pub struct Area { @@ -732,6 +734,7 @@ impl<'h> Evaluator<'h> { "INDEX" => self.index(args), "XLOOKUP" => self.xlookup(args), "CELL" => self.cell_info(args), + "GETPIVOTDATA" => self.getpivotdata(args), "TRIMRANGE" => self.trimrange(args), "SUBTOTAL" | "AGGREGATE" => self.subtotal(name, args), "TEXT" => { @@ -1344,6 +1347,63 @@ impl<'h> Evaluator<'h> { self.builtin("XLOOKUP", args) } + /// GETPIVOTDATA(data_field, pivot_table, [field1, item1], …): a value shown in a PivotTable + /// report. `pivot_table` is any cell or range in the report; the report's index, written by + /// the engine with the report, says which cell the value field and items name + /// ([`gridcraft_model::PivotReport::lookup`]). An empty value cell counts as 0, as in Excel. + fn getpivotdata(&mut self, args: &[Expr]) -> Ev { + if args.len() < 2 || args.len() > MAX_GETPIVOTDATA_ARGS { + return err(CellError::Value); + } + let text = |v: Value| -> Result { v.scalar().to_text() }; + let data = match text(self.value(&args[0])) { + Ok(t) => t, + Err(e) => return err(e), + }; + let area = match self.eval(&args[1]) { + Ev::R(areas) => match areas.first() { + Some(a) => *a, + None => return err(CellError::Ref), + }, + Ev::V(Value::Error(e)) => return err(e), + _ => return err(CellError::Ref), + }; + let mut pairs: Vec<(String, Value)> = Vec::with_capacity(args.len() / 2); + for pair in args.get(2..).unwrap_or_default().chunks(2) { + let field = match pair.first().map(|f| text(self.value(f))) { + Some(Ok(f)) => f, + Some(Err(e)) => return err(e), + None => return err(CellError::Ref), + }; + // A field without its item names nothing. + let Some(item) = pair.get(1).map(|i| self.value(i).scalar()) else { return err(CellError::Ref) }; + if let Value::Error(e) = item { + return err(e); + } + pairs.push((field, item)); + } + let found = self.host.workbook().sheet(area.sheet).and_then(|sh| { + sh.pivots.iter().find_map(|p| { + let r = p.last_range.filter(|r| r.intersects(&area.range))?; + Some((r.start, p.report.0.clone()?)) + }) + }); + let Some((origin, report)) = found else { return err(CellError::Ref) }; + let named: Vec<(&str, &Value)> = pairs.iter().map(|(f, v)| (f.as_str(), v)).collect(); + let (r, c) = match report.lookup(&data, &named) { + Ok(at) => at, + Err(e) => return err(e), + }; + let (Some(row), Some(col)) = (origin.row.checked_add(r), origin.col.checked_add(c)) else { return err(CellError::Ref) }; + if row >= MAX_ROWS || col >= MAX_COLS { + return err(CellError::Ref); + } + match self.host.cell_value(area.sheet, CellRef::new(row, col)) { + Value::Empty => Ev::V(Value::Number(0.0)), + v => Ev::V(v), + } + } + fn cell_info(&mut self, args: &[Expr]) -> Ev { let kind = match self.arg_val(args, 0).map(|v| v.scalar().to_text()) { Some(Ok(t)) => t.to_ascii_lowercase(), @@ -1763,6 +1823,24 @@ pub fn precedents(wb: &Workbook, sheet: usize, e: &Expr) -> (Vec, bool) { (_, Some(Expr::Number(_) | Expr::Text(_) | Expr::Bool(_) | Expr::Error(_) | Expr::Array(_) | Expr::Missing)) => {} _ => dynamic = true, }, + // GETPIVOTDATA reads a cell of the report its reference points into. Refreshing a + // PivotTable recalculates everything; an edit inside the report must reach it too. + Expr::Call(n, args) if n == "GETPIVOTDATA" => { + if let Some(Expr::Ref(r)) = args.get(1) { + let range = r.range(); + let mut reports = Vec::new(); + push(&mut reports, r, range); + for a in reports { + let hits = wb + .sheet(a.sheet) + .into_iter() + .flat_map(|sh| sh.pivots.iter()) + .filter_map(|p| p.last_range) + .filter(|pr| pr.intersects(&a.range)); + out.extend(hits.map(|range| Area { sheet: a.sheet, range })); + } + } + } Expr::Call(n, _) if matches!(n.as_str(), "INDIRECT" | "OFFSET" | "NOW" | "TODAY" | "RAND" | "RANDBETWEEN" | "RANDARRAY" | "CELL" | "INFO") => { @@ -1811,6 +1889,7 @@ fn is_special(n: &str) -> bool { | "INDIRECT" | "INDEX" | "CELL" + | "GETPIVOTDATA" | "TRIMRANGE" | "SUBTOTAL" | "AGGREGATE" @@ -1849,6 +1928,7 @@ pub const SPECIAL_FUNCTIONS: &[&str] = &[ "INDIRECT", "INDEX", "CELL", + "GETPIVOTDATA", "TRIMRANGE", "SUBTOTAL", "AGGREGATE", diff --git a/crates/calc/src/tests.rs b/crates/calc/src/tests.rs index 196c57de7..f07ef8c4d 100644 --- a/crates/calc/src/tests.rs +++ b/crates/calc/src/tests.rs @@ -1266,3 +1266,46 @@ fn offset_extreme_arguments_return_ref() { assert_eq!(crate::evaluate(&wb, 0, c("D1"), formula), Value::Number(expected), "{formula}"); } } + +#[test] +fn getpivotdata_reads_the_report_index() { + use gridcraft_model::{PivotReport, PivotReportIndex, PivotTable, ReportDataField, ReportField, ReportItem, ReportKey}; + // A report at D1:E4 as the engine writes it: a header, East and West, a grand total. + let mut t = T::new(); + let item = |l: &str| ReportItem { value: Value::text(l), label: l.into() }; + let key = |offset, items: &[Option]| ReportKey { offset, data: None, items: items.to_vec() }; + let report = PivotReport { + data_fields: vec![ReportDataField { name: "Sum of Sales".into(), source: "Sales".into() }], + row_fields: vec![ReportField { name: "Region".into(), dates: false, items: vec![item("East"), item("West")] }], + rows: vec![key(1, &[Some(0)]), key(2, &[Some(1)]), key(3, &[None])], + cols: vec![ReportKey { offset: 1, data: Some(0), items: vec![] }], + label_cols: 1, + compact: true, + ..PivotReport::default() + }; + let pt = PivotTable { + anchor: c("D1"), + last_range: Some(gridcraft_core::RangeRef::parse("D1:E4").unwrap()), + report: PivotReportIndex(Some(std::sync::Arc::new(report))), + ..PivotTable::default() + }; + t.wb.sheet_mut(0).unwrap().pivots.push(pt); + for (at, v) in [("E2", "30"), ("E3", "12"), ("E4", "42")] { + t.set(at, v); + } + t.set("A1", r#"=GETPIVOTDATA("Sales",D1,"Region","west")"#); + t.set("A2", r#"=GETPIVOTDATA("Sum of Sales",E4)"#); + t.set("A3", r#"=GETPIVOTDATA("Sales",D1,"Region","North")"#); + assert_eq!(t.num("A1"), 12.0); + assert_eq!(t.num("A2"), 42.0); + assert_eq!(t.get("A3"), Value::Error(CellError::Ref)); + // A changed report cell reaches the formulas that read it. + t.set("E3", "13"); + assert_eq!(t.num("A1"), 13.0); + // An empty value cell reads as 0; without an index there is nothing to read. + t.set("E3", ""); + assert_eq!(t.num("A1"), 0.0); + t.wb.sheet_mut(0).unwrap().pivots[0].report = PivotReportIndex::default(); + t.set("A1", r#"=GETPIVOTDATA("Sales",D1,"Region","West")"#); + assert_eq!(t.get("A1"), Value::Error(CellError::Ref)); +} diff --git a/crates/engine/src/catalog.rs b/crates/engine/src/catalog.rs index d6b67e1b4..11aa11e91 100644 --- a/crates/engine/src/catalog.rs +++ b/crates/engine/src/catalog.rs @@ -300,6 +300,7 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[ ("chart.shapeFill", "Format", "Shape Styles", "Shape Fill"), ("chart.shapeOutline", "Format", "Shape Styles", "Shape Outline"), // PivotTable Analyze / Design + ("pivot.generateGetPivotData", "PivotTable Analyze", "PivotTable", "Generate GetPivotData"), ("pivot.fieldList", "PivotTable Analyze", "Show", "Field List"), ("pivot.refresh", "PivotTable Analyze", "Data", "Refresh"), ("pivot.changeSource", "PivotTable Analyze", "Data", "Change Data Source"), @@ -315,8 +316,25 @@ pub const CATALOG: &[(&str, &str, &str, &str)] = &[ ("pivot.filter", "PivotTable Analyze", "Filter", "Filter"), ("pivot.sort", "PivotTable Analyze", "Sort", "Sort"), ("pivot.group", "PivotTable Analyze", "Group", "Group Field"), + ("pivot.groupSelection", "PivotTable Analyze", "Group", "Group Selection"), + ("pivot.ungroup", "PivotTable Analyze", "Group", "Ungroup"), + ("pivot.calculatedItem", "PivotTable Analyze", "Calculations", "Calculated Item"), + ("pivot.solveOrder", "PivotTable Analyze", "Calculations", "Solve Order"), + ("pivot.listFormulas", "PivotTable Analyze", "Calculations", "List Formulas"), + ("pivot.insertTimeline", "PivotTable Analyze", "Filter", "Insert Timeline"), + // Timeline (contextual) + ("timeline.caption", "Timeline", "Timeline", "Timeline Caption"), + ("timeline.reportConnections", "Timeline", "Timeline", "Report Connections"), + ("timeline.style", "Timeline", "Timeline Styles", "Timeline Styles"), + ("timeline.size", "Timeline", "Size", "Size"), + ("timeline.showHeader", "Timeline", "Show", "Header"), + ("timeline.showSelectionLabel", "Timeline", "Show", "Selection Label"), + ("timeline.showScrollbar", "Timeline", "Show", "Scrollbar"), + ("timeline.showTimeLevel", "Timeline", "Show", "Time Level"), ("pivot.collapse", "PivotTable Analyze", "Active Field", "Collapse Field"), - ("pivot.delete", "PivotTable Analyze", "Actions", "Clear All"), + ("pivot.clearAll", "PivotTable Analyze", "Actions", "Clear All"), + ("pivot.clearFilters", "PivotTable Analyze", "Actions", "Clear Filters"), + ("pivot.delete", "PivotTable Analyze", "Actions", "Delete PivotTable"), ]; /// Distinct catalog ids. diff --git a/crates/engine/src/cmd/formulas.rs b/crates/engine/src/cmd/formulas.rs index 40df72df0..589d4f747 100644 --- a/crates/engine/src/cmd/formulas.rs +++ b/crates/engine/src/cmd/formulas.rs @@ -192,6 +192,12 @@ pub fn function_list() -> Vec { ("ISREF", "Information", "ISREF(value)", "TRUE for references."), ("ISFORMULA", "Information", "ISFORMULA(reference)", "TRUE when the cell has a formula."), ("FORMULATEXT", "Lookup", "FORMULATEXT(reference)", "The formula of a cell as text."), + ( + "GETPIVOTDATA", + "Lookup", + "GETPIVOTDATA(data_field, pivot_table, [field1, item1], ...)", + "Returns a value shown in a PivotTable report, named by its value field and items.", + ), ("SHEET", "Information", "SHEET([value])", "Sheet number."), ("SHEETS", "Information", "SHEETS([reference])", "Number of sheets."), ("AREAS", "Lookup", "AREAS(reference)", "Number of areas in a reference."), diff --git a/crates/engine/src/cmd/insert.rs b/crates/engine/src/cmd/insert.rs index d878e4469..a5e0a52a1 100644 --- a/crates/engine/src/cmd/insert.rs +++ b/crates/engine/src/cmd/insert.rs @@ -1099,6 +1099,7 @@ fn delete_object(s: &mut Session, p: &Json) -> Result { "chart" => sh.charts.retain(|c| c.id != id), "image" => sh.images.retain(|c| c.id != id), "slicer" => sh.slicers.retain(|c| c.id != id), + "timeline" => sh.timelines.retain(|c| c.id != id), _ => sh.shapes.retain(|c| c.id != id), } Ok(Json::Null) @@ -1116,6 +1117,7 @@ fn move_object(s: &mut Session, p: &Json) -> Result { "chart" => sh.charts.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), "image" => sh.images.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), "slicer" => sh.slicers.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), + "timeline" => sh.timelines.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), "shape" => sh.shapes.iter_mut().find(|c| c.id == id).map(|c| &mut c.anchor), _ => None, }; diff --git a/crates/engine/src/cmd/mod.rs b/crates/engine/src/cmd/mod.rs index 50865a2b7..d1d5401cc 100644 --- a/crates/engine/src/cmd/mod.rs +++ b/crates/engine/src/cmd/mod.rs @@ -13,12 +13,14 @@ pub mod formulas; pub mod insert; pub mod inspect; pub mod pivot; +pub mod pivot_tools; pub mod print; pub mod review; pub mod sheet; pub mod slicer; pub mod solver; pub mod spelling; +pub mod timeline; pub mod view; pub mod whatif; @@ -139,7 +141,9 @@ pub fn command_specs() -> &'static [CommandSpec] { v.extend(spelling::specs()); v.extend(solver::specs()); v.extend(pivot::specs()); + v.extend(pivot_tools::specs()); v.extend(slicer::specs()); + v.extend(timeline::specs()); v.extend(whatif::specs()); v.extend(forecast::specs()); v.extend(analysis::specs()); diff --git a/crates/engine/src/cmd/pivot.rs b/crates/engine/src/cmd/pivot.rs index 8a441fa1b..43f41feb5 100644 --- a/crates/engine/src/cmd/pivot.rs +++ b/crates/engine/src/cmd/pivot.rs @@ -96,18 +96,62 @@ pub fn specs() -> Vec { "Value Field Settings", ["PivotTable Analyze", "Active Field"], None, - "{pivot?, field: caption or source field | index: n, func?, name?, showAs?: normal|percentOfGrandTotal|percentOfColumnTotal|percentOfRowTotal|runningTotal|rank, numberFormat?: \"#,##0\" | null, replace?}", + "{pivot?, field: caption or source field | index: n, func?, name?, showAs?: normal|percentOfGrandTotal|percentOfColumnTotal|percentOfRowTotal|percentOf|percentOfParentRowTotal|percentOfParentColumnTotal|percentOfParentTotal|differenceFrom|percentDifferenceFrom|runningTotal|percentRunningTotal|rankAscending|rankDescending|index, baseField?: row or column field, baseItem?: item label | \"(previous)\" | \"(next)\", numberFormat?: \"#,##0\" | null, replace?}", has_doc, value_settings ), - cmd!("pivot.filter", "Filter Items", [], None, "{pivot?, field, selected: [\"East\", ..] | null (all), replace?}", has_doc, filter), + cmd!( + "pivot.filter", + "Filter Items", + [], + None, + "{pivot?, field, selected: [\"East\", ..] | null (all), replace?} (clears the field's label, value or Top 10 filter)", + has_doc, + filter + ), + cmd!( + "pivot.labelFilter", + "Label Filter", + [], + None, + "{pivot?, field: row or column field, op: equals|doesNotEqual|beginsWith|doesNotBeginWith|endsWith|doesNotEndWith|contains|doesNotContain|greaterThan|greaterThanOrEqual|lessThan|lessThanOrEqual|between|notBetween, value, value2? (between), replace?} keeps the items whose caption passes (* and ? are wildcards); replaces the field's other filters", + has_doc, + label_filter + ), + cmd!( + "pivot.valueFilter", + "Value Filter", + [], + None, + "{pivot?, field: row or column field, op: equals|doesNotEqual|greaterThan|greaterThanOrEqual|lessThan|lessThanOrEqual|between|notBetween, value: number, value2? (between), valueField?: caption (default the first value field), replace?} keeps the items whose total passes, within each parent item; replaces the field's other filters", + has_doc, + value_filter + ), + cmd!( + "pivot.top10", + "Top 10", + [], + None, + "{pivot?, field: row or column field, top?: true (false: bottom), count?: 10, by?: items|percent|sum, valueField?: caption (default the first value field), replace?} keeps the top or bottom items by their totals, within each parent item; replaces the field's other filters", + has_doc, + top10 + ), + cmd!( + "pivot.clearFilter", + "Clear Filter", + [], + None, + "{pivot?, field?, replace?} clears the field's item, label, value and Top 10 filters (report filters show all); without `field`, every field's", + has_doc, + clear_filter + ), cmd!("pivot.sort", "Sort Field", [], None, "{pivot?, field, order: asc|desc|none, replace?}", has_doc, sort), cmd!( "pivot.group", "Group Field", ["PivotTable Analyze", "Group"], None, - "{pivot?, field, by: years|quarters|months|days|none, replace?}", + "{pivot?, field, by: years|quarters|months|days|hours|minutes|seconds|range|none | [levels] (several date and time levels make one field each, coarsest first), start?, end?: number or date (Starting at / Ending at; default automatic), interval? (by: range; default 10), days?: 1 (Number of days; by: days only), replace?}; without `field` and `by` opens the Grouping dialog for the field at the active cell", has_doc, group ), @@ -188,6 +232,28 @@ pub fn specs() -> Vec { has_doc, delete ), + cmd!(query "pivot.generateGetPivotData", "Generate GetPivotData", ["PivotTable Analyze", "PivotTable", "Options"], None, "{on?: bool} (toggles without it) → {on}: pointing at a PivotTable value while typing a formula inserts GETPIVOTDATA (an application setting, on by default)", always, |s, p| { + s.prefs.generate_get_pivot_data = bool_param(p, "on").unwrap_or(!s.prefs.generate_get_pivot_data); + Ok(json!({"on": s.prefs.generate_get_pivot_data})) + }), + cmd!( + "pivot.clearAll", + "Clear All", + ["PivotTable Analyze", "Actions", "Clear"], + None, + "{pivot?, replace?} takes every field out of Rows, Columns, Values and Filters (the PivotTable stays, empty; calculated fields and layout options are kept)", + has_doc, + clear_all + ), + cmd!( + "pivot.clearFilters", + "Clear Filters", + ["PivotTable Analyze", "Actions", "Clear"], + None, + "{pivot?, replace?} shows every item again: report filters go back to (All) and hidden items of row and column fields come back", + has_doc, + clear_filters + ), ] } @@ -197,14 +263,14 @@ fn with(p: &Json, k: &str, v: Json) -> Json { Json::Object(m) } -fn other(msg: impl Into) -> EngineError { +pub(super) fn other(msg: impl Into) -> EngineError { EngineError::Other(msg.into()) } // ---------------------------------------------------------------- lookup /// (sheet, index) of the pivot a call targets. -fn find_pivot(s: &Session, p: &Json) -> Result<(usize, usize)> { +pub(super) fn find_pivot(s: &Session, p: &Json) -> Result<(usize, usize)> { let d = s.doc()?; let active = d.wb.active_sheet; if let Some(v) = p.get("pivot").filter(|v| !v.is_null()) { @@ -278,27 +344,35 @@ fn func_name(f: PivotFunc) -> &'static str { } } +/// Show Values As modes: (command name, mode). +const SHOW_AS: &[(&str, PivotShowAs)] = &[ + ("normal", PivotShowAs::Normal), + ("percentOfGrandTotal", PivotShowAs::PercentOfGrandTotal), + ("percentOfColumnTotal", PivotShowAs::PercentOfColumnTotal), + ("percentOfRowTotal", PivotShowAs::PercentOfRowTotal), + ("percentOf", PivotShowAs::PercentOf), + ("percentOfParentRowTotal", PivotShowAs::PercentOfParentRowTotal), + ("percentOfParentColumnTotal", PivotShowAs::PercentOfParentColumnTotal), + ("percentOfParentTotal", PivotShowAs::PercentOfParentTotal), + ("differenceFrom", PivotShowAs::DifferenceFrom), + ("percentDifferenceFrom", PivotShowAs::PercentDifferenceFrom), + ("runningTotal", PivotShowAs::RunningTotal), + ("percentRunningTotal", PivotShowAs::PercentRunningTotal), + ("rankAscending", PivotShowAs::RankAscending), + ("rankDescending", PivotShowAs::Rank), + ("index", PivotShowAs::Index), +]; + fn show_as_param(v: &str) -> Option { - Some(match v.to_ascii_lowercase().as_str() { - "normal" | "none" => PivotShowAs::Normal, - "percentofgrandtotal" => PivotShowAs::PercentOfGrandTotal, - "percentofcolumntotal" => PivotShowAs::PercentOfColumnTotal, - "percentofrowtotal" => PivotShowAs::PercentOfRowTotal, - "runningtotal" => PivotShowAs::RunningTotal, - "rank" => PivotShowAs::Rank, - _ => return None, - }) + match v.to_ascii_lowercase().as_str() { + "none" => Some(PivotShowAs::Normal), + "rank" => Some(PivotShowAs::Rank), + l => SHOW_AS.iter().find(|(n, _)| n.eq_ignore_ascii_case(l)).map(|(_, m)| *m), + } } fn show_as_name(v: PivotShowAs) -> &'static str { - match v { - PivotShowAs::Normal => "normal", - PivotShowAs::PercentOfGrandTotal => "percentOfGrandTotal", - PivotShowAs::PercentOfColumnTotal => "percentOfColumnTotal", - PivotShowAs::PercentOfRowTotal => "percentOfRowTotal", - PivotShowAs::RunningTotal => "runningTotal", - PivotShowAs::Rank => "rank", - } + SHOW_AS.iter().find(|(_, m)| *m == v).map(|(n, _)| *n).unwrap_or("normal") } fn layout_name(l: PivotLayout) -> &'static str { @@ -316,9 +390,20 @@ fn group_name(g: PivotDateGroup) -> &'static str { PivotDateGroup::Quarters => "quarters", PivotDateGroup::Months => "months", PivotDateGroup::Days => "days", + PivotDateGroup::Hours => "hours", + PivotDateGroup::Minutes => "minutes", + PivotDateGroup::Seconds => "seconds", } } +/// A row or column field as `pivot.list` and `pivot.fields` report it. +fn axis_json(f: &PivotField) -> Json { + let group = if f.number_group.is_some() { "range" } else { group_name(f.date_group) }; + let range = f.number_group.map(|g| json!({"start": g.start, "end": g.end, "interval": g.by})); + let dates = f.date_range.map(|r| json!({"start": r.start, "end": r.end, "days": r.days})); + json!({"field": f.source_col, "sort": sort_name(f.sort), "group": group, "range": range, "dates": dates, "collapsed": f.collapsed_items}) +} + fn sort_name(o: PivotSort) -> &'static str { match o { PivotSort::Asc => "asc", @@ -329,7 +414,7 @@ fn sort_name(o: PivotSort) -> &'static str { // ---------------------------------------------------------------- layout editing (pure) -fn in_axes(pt: &PivotTable, field: &str) -> bool { +pub(super) fn in_axes(pt: &PivotTable, field: &str) -> bool { pt.rows.iter().chain(pt.columns.iter()).any(|f| f.source_col.eq_ignore_ascii_case(field)) } @@ -360,7 +445,7 @@ fn shift_values(pt: &mut PivotTable, rows: bool, i: usize, inserted: bool) { } /// Takes `field` out of Rows and Columns (Σ Values keeps its place); returns its first entry. -fn take_from_axes(pt: &mut PivotTable, field: &str) -> Option { +pub(super) fn take_from_axes(pt: &mut PivotTable, field: &str) -> Option { let mut first = None; for rows in [true, false] { loop { @@ -375,7 +460,7 @@ fn take_from_axes(pt: &mut PivotTable, field: &str) -> Option { } /// Inserts `f` into Rows (`rows`) or Columns at `position` (default: the end). -fn insert_axis_field(pt: &mut PivotTable, rows: bool, f: PivotField, position: Option) { +pub(super) fn insert_axis_field(pt: &mut PivotTable, rows: bool, f: PivotField, position: Option) { let list = if rows { &mut pt.rows } else { &mut pt.columns }; let at = position.unwrap_or(list.len()).min(list.len()); list.insert(at, f); @@ -408,12 +493,17 @@ fn calc_field<'a>(pt: &'a PivotTable, field: &str) -> Option<&'a PivotCalcField> pt.calculated_fields.iter().find(|c| c.name.eq_ignore_ascii_case(field.trim())) } -/// The canonical name of a source or calculated field. +/// The canonical name of a source, calculated or grouped (Group Selection) field. fn canonical_any(cmd: &str, pt: &PivotTable, src: &pv::Source, field: &str) -> Result { - match calc_field(pt, field) { - Some(c) if src.col(field).is_none() => Ok(c.name.clone()), - _ => canonical(cmd, src, field), + if src.col(field).is_none() { + if let Some(c) = calc_field(pt, field) { + return Ok(c.name.clone()); + } + if let Some(g) = pt.group_field(field) { + return Ok(g.name.clone()); + } } + canonical(cmd, src, field) } fn unique_value_name(pt: &PivotTable, base: &str, except: Option) -> String { @@ -443,7 +533,13 @@ fn add_to_area( if is_calc && area != "values" { return Err(other("A calculated field can only be placed in the Values area.")); } + if src.col(field).is_none() && pt.group_field(field).is_some() && area == "values" { + return Err(other("A field made by Group Selection can't be placed in the Values area.")); + } let field = canonical_any(cmd, pt, src, field)?; + if matches!(area, "values" | "filters") && pt.calculated_items.iter().any(|c| c.field.eq_ignore_ascii_case(&field)) { + return Err(other(super::pivot_tools::CALC_ITEM_AREA)); + } match area { "rows" | "columns" => { if pt.rows.len() + pt.columns.len() >= pv::MAX_AXIS_FIELDS && !in_axes(pt, &field) { @@ -451,8 +547,8 @@ fn add_to_area( } let existing = take_from_axes(pt, &field).unwrap_or_else(|| PivotField { source_col: field.clone(), ..Default::default() }); // A report filter that selects nothing in particular goes away; a selection stays - // as the field's hidden items. - pt.filters.retain(|f| !(f.source_col.eq_ignore_ascii_case(&field) && f.selected.is_none())); + // as the field's hidden items, and a dormant rule becomes active again. + pt.filters.retain(|f| !(f.source_col.eq_ignore_ascii_case(&field) && f.selected.is_none() && f.rule.is_none())); insert_axis_field(pt, area == "rows", existing, position); } "values" => { @@ -461,6 +557,14 @@ fn add_to_area( } let numeric = is_calc || src.col(&field).and_then(|c| src.numeric.get(c).copied()).unwrap_or(false); let f = func.unwrap_or(if numeric { PivotFunc::Sum } else { PivotFunc::Count }); + if !pt.calculated_items.is_empty() { + if super::pivot_tools::averages(f) { + return Err(other(super::pivot_tools::NO_AVERAGES)); + } + if pt.values.iter().any(|v| v.source_col.eq_ignore_ascii_case(&field)) { + return Err(other(super::pivot_tools::NO_DUPLICATE_VALUES)); + } + } let base = match name { Some(n) if !n.trim().is_empty() => { if src.col(n).is_some() { @@ -471,7 +575,7 @@ fn add_to_area( _ => auto_name(f, &field), }; let nm = unique_value_name(pt, &base, None); - let v = PivotValue { source_col: field, func: f, name: nm, show_as: PivotShowAs::Normal, number_format: None }; + let v = PivotValue { source_col: field, func: f, name: nm, ..PivotValue::default() }; let at = position.unwrap_or(pt.values.len()).min(pt.values.len()); pt.values.insert(at, v); } @@ -479,8 +583,9 @@ fn add_to_area( take_from_axes(pt, &field); let existing = pt.filters.iter().position(|f| f.source_col.eq_ignore_ascii_case(&field)); let entry = match existing { - Some(i) => pt.filters.remove(i), - None => PivotFilter { source_col: field, selected: None }, + // Label, value and Top 10 filters apply to row and column fields only. + Some(i) => PivotFilter { rule: None, ..pt.filters.remove(i) }, + None => PivotFilter { source_col: field, ..PivotFilter::default() }, }; let at = position.unwrap_or(pt.filters.len()).min(pt.filters.len()); pt.filters.insert(at, entry); @@ -511,13 +616,15 @@ fn remove_from_area(cmd: &str, pt: &mut PivotTable, field: &str, area: &str) -> .position(|v| v.name.eq_ignore_ascii_case(field)) .or_else(|| pt.values.iter().position(|v| v.source_col.eq_ignore_ascii_case(field))) .ok_or_else(|| bad(cmd, format!("`{field}` isn't in the values area")))?; - Ok(pt.values.remove(i).source_col) + let v = pt.values.remove(i); + retarget_rules(pt, &v.name, None); + Ok(v.source_col) } _ => { let i = pt .filters .iter() - .position(|f| f.source_col.eq_ignore_ascii_case(field) && !in_axes(pt, &f.source_col)) + .position(|f| f.source_col.eq_ignore_ascii_case(field) && f.rule.is_none() && !in_axes(pt, &f.source_col)) .ok_or_else(|| bad(cmd, format!("`{field}` isn't in the filters area")))?; Ok(pt.filters.remove(i).source_col) } @@ -527,7 +634,7 @@ fn remove_from_area(cmd: &str, pt: &mut PivotTable, field: &str, area: &str) -> // ---------------------------------------------------------------- shared update /// Applies `f` to the targeted pivot and re-renders it, as one undo step. -fn update(s: &mut Session, p: &Json, f: impl FnOnce(&mut PivotTable, &pv::Source, &Workbook) -> Result) -> Result { +pub(super) fn update(s: &mut Session, p: &Json, f: impl FnOnce(&mut PivotTable, &pv::Source, &Workbook) -> Result) -> Result { let (si, pi) = find_pivot(s, p)?; let replace = bool_param(p, "replace").unwrap_or(false); edit(s, |cx| { @@ -744,10 +851,10 @@ fn pivot_json(wb: &Workbook, si: usize, pt: &PivotTable) -> Json { "anchor": pt.anchor.a1(), "range": pt.last_range.map(|r| r.a1()), "source": pt.source, - "rows": pt.rows.iter().map(|f| json!({"field": f.source_col, "sort": sort_name(f.sort), "group": group_name(f.date_group), "collapsed": f.collapsed_items})).collect::>(), - "columns": pt.columns.iter().map(|f| json!({"field": f.source_col, "sort": sort_name(f.sort), "group": group_name(f.date_group), "collapsed": f.collapsed_items})).collect::>(), - "values": pt.values.iter().map(|v| json!({"field": v.source_col, "func": func_name(v.func), "name": v.name, "showAs": show_as_name(v.show_as), "numberFormat": v.number_format})).collect::>(), - "filters": pt.filters.iter().map(|f| json!({"field": f.source_col, "selected": f.selected, "report": !in_axes(pt, &f.source_col)})).collect::>(), + "rows": pt.rows.iter().map(axis_json).collect::>(), + "columns": pt.columns.iter().map(axis_json).collect::>(), + "values": pt.values.iter().map(|v| json!({"field": v.source_col, "func": func_name(v.func), "name": v.name, "showAs": show_as_name(v.show_as), "baseField": v.base_field, "baseItem": v.base_item, "numberFormat": v.number_format})).collect::>(), + "filters": pt.filters.iter().map(|f| json!({"field": f.source_col, "selected": f.selected, "rule": f.rule.as_ref().map(rule_json), "report": f.rule.is_none() && !in_axes(pt, &f.source_col)})).collect::>(), "layout": layout_name(pt.layout), "grandTotalsRows": pt.grand_totals_rows, "grandTotalsCols": pt.grand_totals_cols, @@ -757,6 +864,8 @@ fn pivot_json(wb: &Workbook, si: usize, pt: &PivotTable) -> Json { "calculatedFields": pt.calculated_fields.iter().map(|c| json!({"name": c.name, "formula": c.formula})).collect::>(), "valuesOnRows": pt.values_on_rows, "valuesPosition": pt.values_position, + "groupFields": pt.group_fields.iter().map(|g| json!({"name": g.name, "base": g.base, "groups": g.groups.iter().map(|x| json!({"name": x.name, "items": x.items})).collect::>()})).collect::>(), + "calculatedItems": pt.calculated_items.iter().map(|c| json!({"field": c.field, "name": c.name, "formula": c.formula})).collect::>(), }) } @@ -782,14 +891,14 @@ fn fields(s: &mut Session, p: &Json) -> Result { if pt.values.iter().any(|f| f.source_col.eq_ignore_ascii_case(h)) { areas.push("values"); } - if pt.filters.iter().any(|f| f.source_col.eq_ignore_ascii_case(h)) && !in_axes(pt, h) { + if pt.filters.iter().any(|f| f.source_col.eq_ignore_ascii_case(h) && f.rule.is_none()) && !in_axes(pt, h) { areas.push("filters"); } json!({ "name": h, "numeric": src.numeric.get(c).copied().unwrap_or(false), "date": src.dates.get(c).copied().unwrap_or(false), - "samples": pv::distinct_labels(&d.wb, src, c, PivotDateGroup::None, 5), + "samples": pv::distinct_labels(&d.wb, src, c, pv::Grouping::default(), 5), "areas": areas, }) }) @@ -801,6 +910,19 @@ fn fields(s: &mut Session, p: &Json) -> Result { let areas: Vec<&str> = if pt.values.iter().any(|v| v.source_col.eq_ignore_ascii_case(&c.name)) { vec!["values"] } else { vec![] }; list.push(json!({"name": c.name, "numeric": true, "date": false, "samples": [], "areas": areas, "calculated": true, "formula": c.formula})); } + for g in &pt.group_fields { + let mut areas = vec![]; + for (key, area) in [("rows", &pt.rows), ("columns", &pt.columns)] { + if area.iter().any(|f| f.source_col.eq_ignore_ascii_case(&g.name)) { + areas.push(key); + } + } + if pt.filters.iter().any(|f| f.source_col.eq_ignore_ascii_case(&g.name) && f.rule.is_none()) && !in_axes(pt, &g.name) { + areas.push("filters"); + } + let samples: Vec<&str> = g.groups.iter().take(5).map(|x| x.name.as_str()).collect(); + list.push(json!({"name": g.name, "numeric": false, "date": false, "samples": samples, "areas": areas, "grouped": true, "base": g.base})); + } if let Some(m) = out.as_object_mut() { m.insert("fields".into(), Json::Array(list)); if let Ok(src) = &src { @@ -999,7 +1121,7 @@ fn move_field(s: &mut Session, p: &Json) -> Result { if let (Some(k), Some(slot)) = (kept, list.iter_mut().find(|f| f.source_col.eq_ignore_ascii_case(&src_field))) { *slot = k; } - if let Some(h) = hidden.filter(|h| h.selected.is_some()) + if let Some(h) = hidden.filter(|h| h.selected.is_some() || h.rule.is_some()) && !pt.filters.iter().any(|f| f.source_col.eq_ignore_ascii_case(&src_field)) { pt.filters.push(h); @@ -1043,6 +1165,9 @@ fn value_settings(s: &mut Session, p: &Json) -> Result { let cur = pt.values.get(i).cloned().ok_or_else(|| bad(C, "no value field at that index"))?; let mut v = cur.clone(); if let Some(f) = func { + if super::pivot_tools::averages(f) && !pt.calculated_items.is_empty() { + return Err(other(super::pivot_tools::NO_AVERAGES)); + } v.func = f; // An automatic caption follows the function. let auto = auto_name(cur.func, &cur.source_col); @@ -1066,6 +1191,21 @@ fn value_settings(s: &mut Session, p: &Json) -> Result { if let Some(sa) = show { v.show_as = sa; } + if let Some(b) = str_param(p, "baseField") { + if !in_axes(pt, b) { + return Err(bad(C, format!("the base field `{b}` must be a row or column field"))); + } + v.base_field = Some(b.to_string()); + } + if let Some(b) = str_param(p, "baseItem") { + v.base_item = Some(b.to_string()); + } + if v.show_as.needs_base_field() && v.base_field.is_none() { + v.base_field = pt.rows.first().or(pt.columns.first()).map(|f| f.source_col.clone()); + } + if v.show_as.needs_base_item() && v.base_item.is_none() { + v.base_item = Some(BASE_PREVIOUS.to_string()); + } match &fmt { Some(Json::String(f)) if !f.is_empty() && f.len() <= 255 => v.number_format = Some(f.clone()), Some(Json::Null) => v.number_format = None, @@ -1074,6 +1214,9 @@ fn value_settings(s: &mut Session, p: &Json) -> Result { None => {} } let name = v.name.clone(); + if !name.eq_ignore_ascii_case(&cur.name) { + retarget_rules(pt, &cur.name, Some(&name)); + } if let Some(slot) = pt.values.get_mut(i) { *slot = v; } @@ -1107,11 +1250,12 @@ fn filter(s: &mut Session, p: &Json) -> Result { Some(_) => return Err(bad(C, "`selected` is a list of item labels or null")), }; update(s, p, |pt, src, _| { - let field = canonical(C, src, &field)?; + let field = canonical_any(C, pt, src, &field)?; let i = pt.filters.iter().position(|f| f.source_col.eq_ignore_ascii_case(&field)); match (i, selected) { (Some(i), None) => { - if in_axes(pt, &field) { + // An axis field's filter with no rule left has nothing to keep. + if in_axes(pt, &field) && pt.filters.get(i).is_some_and(|f| f.rule.is_none()) { pt.filters.remove(i); } else if let Some(f) = pt.filters.get_mut(i) { f.selected = None; @@ -1120,15 +1264,213 @@ fn filter(s: &mut Session, p: &Json) -> Result { (Some(i), Some(sel)) => { if let Some(f) = pt.filters.get_mut(i) { f.selected = Some(sel); + f.rule = None; } } - (None, Some(sel)) => pt.filters.push(PivotFilter { source_col: field.clone(), selected: Some(sel) }), + (None, Some(sel)) => pt.filters.push(PivotFilter { source_col: field.clone(), selected: Some(sel), rule: None }), (None, None) => {} } Ok(json!({"field": field})) }) } +// ---------------------------------------------------------------- label, value and Top 10 filters + +fn compare_param(cmd: &str, v: Option<&str>) -> Result { + let raw = v.ok_or_else(|| bad(cmd, "missing `op`"))?; + PivotCompare::ALL.into_iter().find(|c| compare_name(*c).eq_ignore_ascii_case(raw)).ok_or_else(|| bad(cmd, format!("unknown comparison `{raw}`"))) +} + +fn compare_name(c: PivotCompare) -> &'static str { + match c { + PivotCompare::Equals => "equals", + PivotCompare::NotEquals => "doesNotEqual", + PivotCompare::BeginsWith => "beginsWith", + PivotCompare::NotBeginsWith => "doesNotBeginWith", + PivotCompare::EndsWith => "endsWith", + PivotCompare::NotEndsWith => "doesNotEndWith", + PivotCompare::Contains => "contains", + PivotCompare::NotContains => "doesNotContain", + PivotCompare::Greater => "greaterThan", + PivotCompare::GreaterOrEqual => "greaterThanOrEqual", + PivotCompare::Less => "lessThan", + PivotCompare::LessOrEqual => "lessThanOrEqual", + PivotCompare::Between => "between", + PivotCompare::NotBetween => "notBetween", + } +} + +fn top_by_name(b: PivotTopBy) -> &'static str { + match b { + PivotTopBy::Items => "items", + PivotTopBy::Percent => "percent", + PivotTopBy::Sum => "sum", + } +} + +/// A filter rule as `pivot.list` and `pivot.fields` report it, in the parameters of the command +/// that sets it. +fn rule_json(r: &PivotRule) -> Json { + match r { + PivotRule::Label { op, value, value2 } => json!({"type": "label", "op": compare_name(*op), "value": value, "value2": value2}), + PivotRule::Value { op, field, value, value2 } => { + json!({"type": "value", "op": compare_name(*op), "valueField": field, "value": value, "value2": value2}) + } + PivotRule::Top { top, count, by, field } => json!({"type": "top10", "top": top, "count": count, "by": top_by_name(*by), "valueField": field}), + } +} + +/// A label filter value: text, or a number or logical written as text. +fn text_param(cmd: &str, p: &Json, k: &str) -> Result> { + match p.get(k) { + None | Some(Json::Null) => Ok(None), + Some(Json::String(t)) if t.len() <= 255 => Ok(Some(t.clone())), + Some(Json::Number(n)) => Ok(Some(n.to_string())), + Some(Json::Bool(b)) => Ok(Some(if *b { "TRUE" } else { "FALSE" }.into())), + Some(_) => Err(bad(cmd, format!("`{k}` is text of at most 255 characters"))), + } +} + +fn number_param(cmd: &str, p: &Json, k: &str) -> Result> { + match p.get(k) { + None | Some(Json::Null) => Ok(None), + Some(v) => v.as_f64().filter(|x| x.is_finite()).map(Some).ok_or_else(|| bad(cmd, format!("`{k}` must be a number"))), + } +} + +/// The caption of the value field a value filter or Top 10 uses: the one named, else the first. +fn value_field_param(cmd: &str, pt: &PivotTable, p: &Json) -> Result { + match str_param(p, "valueField") { + Some(n) => pt + .values + .iter() + .find(|v| v.name.eq_ignore_ascii_case(n)) + .or_else(|| pt.values.iter().find(|v| v.source_col.eq_ignore_ascii_case(n))) + .map(|v| v.name.clone()) + .ok_or_else(|| bad(cmd, format!("`{n}` isn't in the values area"))), + None => pt.values.first().map(|v| v.name.clone()).ok_or_else(|| other("A value filter needs a field in the Values area.")), + } +} + +/// Sets the filter rule of a row or column field, replacing its other filters. +fn set_rule(cmd: &str, pt: &mut PivotTable, field: &str, rule: PivotRule) -> Result { + let field = pt + .rows + .iter() + .chain(pt.columns.iter()) + .find(|f| f.source_col.eq_ignore_ascii_case(field)) + .map(|f| f.source_col.clone()) + .ok_or_else(|| bad(cmd, format!("`{field}` isn't in the rows or columns area")))?; + pt.filters.retain(|f| !f.source_col.eq_ignore_ascii_case(&field)); + pt.filters.push(PivotFilter { source_col: field.clone(), selected: None, rule: Some(rule) }); + Ok(field) +} + +/// Follows a value field's change of caption in the rules that use it, or drops those rules +/// (`to: None`) when the value field is removed. +fn retarget_rules(pt: &mut PivotTable, from: &str, to: Option<&str>) { + pt.filters.retain_mut(|f| { + let Some(rule) = f.rule.as_mut() else { return true }; + let (PivotRule::Value { field, .. } | PivotRule::Top { field, .. }) = rule else { return true }; + if !field.eq_ignore_ascii_case(from) { + return true; + } + match to { + Some(n) => *field = n.to_string(), + None => return false, + } + true + }); + let axis: Vec = pt.rows.iter().chain(pt.columns.iter()).map(|f| f.source_col.clone()).collect(); + pt.filters.retain(|f| f.selected.is_some() || f.rule.is_some() || !axis.iter().any(|a| a.eq_ignore_ascii_case(&f.source_col))); +} + +fn label_filter(s: &mut Session, p: &Json) -> Result { + const C: &str = "pivot.labelFilter"; + let field = str_param(p, "field").ok_or_else(|| bad(C, "missing `field`"))?.to_string(); + let op = compare_param(C, str_param(p, "op"))?; + let value = text_param(C, p, "value")?.ok_or_else(|| bad(C, "missing `value`"))?; + let value2 = text_param(C, p, "value2")?; + if op.is_range() && value2.is_none() { + return Err(bad(C, "`between` and `notBetween` need `value2`")); + } + let rule = PivotRule::Label { op, value, value2: value2.unwrap_or_default() }; + update(s, p, |pt, _, _| Ok(json!({"field": set_rule(C, pt, &field, rule)?}))) +} + +fn value_filter(s: &mut Session, p: &Json) -> Result { + const C: &str = "pivot.valueFilter"; + let field = str_param(p, "field").ok_or_else(|| bad(C, "missing `field`"))?.to_string(); + let op = compare_param(C, str_param(p, "op"))?; + if op.is_text_only() { + return Err(bad(C, "a value filter compares numbers")); + } + let value = number_param(C, p, "value")?.ok_or_else(|| bad(C, "missing `value`"))?; + let value2 = number_param(C, p, "value2")?; + if op.is_range() && value2.is_none() { + return Err(bad(C, "`between` and `notBetween` need `value2`")); + } + update(s, p, |pt, _, _| { + let rule = PivotRule::Value { op, field: value_field_param(C, pt, p)?, value, value2: value2.unwrap_or(0.0) }; + Ok(json!({"field": set_rule(C, pt, &field, rule)?})) + }) +} + +fn top10(s: &mut Session, p: &Json) -> Result { + const C: &str = "pivot.top10"; + let field = str_param(p, "field").ok_or_else(|| bad(C, "missing `field`"))?.to_string(); + let top = match p.get("top") { + None | Some(Json::Null) => true, + Some(Json::Bool(b)) => *b, + Some(_) => return Err(bad(C, "`top` is true (top) or false (bottom)")), + }; + let by = match str_param(p, "by").unwrap_or("items") { + b if b.eq_ignore_ascii_case("items") => PivotTopBy::Items, + b if b.eq_ignore_ascii_case("percent") => PivotTopBy::Percent, + b if b.eq_ignore_ascii_case("sum") => PivotTopBy::Sum, + b => return Err(bad(C, format!("unknown `by` `{b}` (items|percent|sum)"))), + }; + let count = number_param(C, p, "count")?.unwrap_or(10.0); + let ok = match by { + PivotTopBy::Items => (1.0..=2_147_483_647.0).contains(&count) && count.fract() == 0.0, + PivotTopBy::Percent => count > 0.0 && count <= 100.0, + PivotTopBy::Sum => true, + }; + if !ok { + return Err(other(match by { + PivotTopBy::Items => "Enter a whole number of items, 1 or more.", + PivotTopBy::Percent => "Enter a percent greater than 0 and at most 100.", + PivotTopBy::Sum => "Enter a number.", + })); + } + update(s, p, |pt, _, _| { + let rule = PivotRule::Top { top, count, by, field: value_field_param(C, pt, p)? }; + Ok(json!({"field": set_rule(C, pt, &field, rule)?})) + }) +} + +fn clear_filter(s: &mut Session, p: &Json) -> Result { + const C: &str = "pivot.clearFilter"; + let field = str_param(p, "field").map(str::to_string); + update(s, p, |pt, _, _| { + if let Some(n) = &field + && !pt.filters.iter().any(|f| f.source_col.eq_ignore_ascii_case(n)) + && !in_axes(pt, n) + { + return Err(bad(C, format!("`{n}` isn't a field of the PivotTable"))); + } + let hit = |f: &PivotFilter| field.as_ref().is_none_or(|n| f.source_col.eq_ignore_ascii_case(n)); + // Axis and dormant rules go; a report filter stays in its area, showing all. + let axis: Vec = pt.rows.iter().chain(pt.columns.iter()).map(|f| f.source_col.clone()).collect(); + pt.filters.retain(|f| !(hit(f) && (f.rule.is_some() || axis.iter().any(|a| a.eq_ignore_ascii_case(&f.source_col))))); + for f in pt.filters.iter_mut().filter(|f| hit(f)) { + f.selected = None; + f.rule = None; + } + Ok(json!({})) + }) +} + fn axis_field<'a>(cmd: &str, pt: &'a mut PivotTable, field: &str) -> Result<&'a mut PivotField> { pt.rows .iter_mut() @@ -1153,31 +1495,199 @@ fn sort(s: &mut Session, p: &Json) -> Result { } fn group_param(v: Option<&str>) -> Option { - Some(match v?.to_ascii_lowercase().as_str() { + Some(match v?.trim().to_ascii_lowercase().as_str() { "years" | "year" => PivotDateGroup::Years, "quarters" | "quarter" => PivotDateGroup::Quarters, "months" | "month" => PivotDateGroup::Months, "days" | "day" => PivotDateGroup::Days, + "hours" | "hour" => PivotDateGroup::Hours, + "minutes" | "minute" => PivotDateGroup::Minutes, + "seconds" | "second" => PivotDateGroup::Seconds, "none" | "ungroup" => PivotDateGroup::None, _ => return None, }) } -fn group(s: &mut Session, p: &Json) -> Result { +/// Sets the grouping of every Rows or Columns entry of `field` on the axis it is first on: date +/// and time `levels` (one entry per level, coarsest first, at the first entry's place), a number +/// range grouping, or none (no levels and no number). The field's item selections and label +/// filters no longer apply and go. +pub(super) fn set_grouping( + pt: &mut PivotTable, + field: &str, + levels: &[PivotDateGroup], + number: Option, + range: Option, +) -> Result<()> { + let rows = pt.rows.iter().any(|f| f.source_col.eq_ignore_ascii_case(field)); + let list = if rows { &pt.rows } else { &pt.columns }; + let at = list + .iter() + .position(|f| f.source_col.eq_ignore_ascii_case(field)) + .ok_or_else(|| bad("pivot.group", format!("`{field}` isn't in the rows or columns area")))?; + let base = list.get(at).cloned().unwrap_or_default(); + loop { + let list = if rows { &mut pt.rows } else { &mut pt.columns }; + let Some(i) = list.iter().position(|f| f.source_col.eq_ignore_ascii_case(field)) else { break }; + list.remove(i); + shift_values(pt, rows, i, false); + } + let mut levels: Vec = levels.iter().copied().filter(|g| *g != PivotDateGroup::None).collect(); + levels.sort_by_key(|g| std::cmp::Reverse(g.fineness())); + levels.dedup(); + let plain = PivotField { date_group: PivotDateGroup::None, number_group: None, date_range: None, collapsed_items: vec![], ..base }; + let entries: Vec = if levels.is_empty() { + vec![PivotField { number_group: number, ..plain }] + } else { + levels.iter().map(|g| PivotField { date_group: *g, date_range: range, ..plain.clone() }).collect() + }; + for (k, e) in entries.into_iter().enumerate() { + insert_axis_field(pt, rows, e, Some(at + k)); + } + for x in pt.filters.iter_mut().filter(|x| x.source_col.eq_ignore_ascii_case(field)) { + x.selected = None; + if matches!(x.rule, Some(PivotRule::Label { .. })) { + x.rule = None; + } + } + Ok(()) +} + +/// A number or date parameter (`"1/1/2026"`, `"2026-01-01"` or a serial). +fn date_or_number(cmd: &str, p: &Json, key: &str, sys: gridcraft_core::DateSystem) -> Result> { + match p.get(key) { + None | Some(Json::Null) => Ok(None), + Some(Json::String(t)) if !t.trim().is_empty() => match gridcraft_core::parse::parse_input(t.trim(), sys).value { + gridcraft_core::Value::Number(n) if n.is_finite() => Ok(Some(n)), + _ => Err(bad(cmd, format!("`{key}` must be a number or a date"))), + }, + Some(v) => v.as_f64().filter(|x| x.is_finite()).map(Some).ok_or_else(|| bad(cmd, format!("`{key}` must be a number or a date"))), + } +} + +/// The smallest and largest number of source column `c` (infinite when it has none). +fn number_bounds(src: &pv::Source, c: usize) -> (f64, f64) { + src.rows + .iter() + .filter_map(|r| match r.get(c) { + Some(gridcraft_core::Value::Number(n)) => Some(*n), + _ => None, + }) + .fold((f64::INFINITY, f64::NEG_INFINITY), |(lo, hi), n| (lo.min(n), hi.max(n))) +} + +/// The Grouping dialog for the field at the active cell (Group Field without parameters). +fn grouping_dialog(s: &mut Session, p: &Json) -> Result { const C: &str = "pivot.group"; - let field = str_param(p, "field").ok_or_else(|| bad(C, "missing `field`"))?.to_string(); - let raw = str_param(p, "by").unwrap_or("months"); - let by = group_param(Some(raw)).ok_or_else(|| bad(C, format!("unknown grouping `{raw}` (years|quarters|months|days|none)")))?; + let (si, pi) = find_pivot(s, p)?; + let field = match str_param(p, "field") { + Some(f) => f.trim().to_string(), + None => super::pivot_tools::active_item(s, si, pi) + .map(|(f, _)| f) + .ok_or_else(|| EngineError::Disabled(C.into(), "select an item of a row or column field to group".into()))?, + }; + let d = s.doc()?; + let pt = d.wb.sheet(si).and_then(|sh| sh.pivots.get(pi)).ok_or_else(|| other("There's no such PivotTable."))?; + let src = pv::read_source(&d.wb, &pt.source, si).map_err(other)?; + let c = src.col(&field).filter(|c| src.numeric.get(*c).copied().unwrap_or(false)).ok_or_else(|| other(super::pivot_tools::CANNOT_GROUP))?; + let (lo, hi) = number_bounds(&src, c); + let entries: Vec<&PivotField> = pt.rows.iter().chain(pt.columns.iter()).filter(|f| f.source_col.eq_ignore_ascii_case(&field)).collect(); + let date = src.dates.get(c).copied().unwrap_or(false) || entries.iter().any(|f| f.date_group != PivotDateGroup::None); + let mut levels: Vec<&str> = entries.iter().filter(|f| f.date_group != PivotDateGroup::None).map(|f| group_name(f.date_group)).collect(); + if levels.is_empty() && date { + levels.push("months"); + } + let range = entries.iter().find_map(|f| f.date_range); + let number = entries.iter().find_map(|f| f.number_group); + let pivot = pt.name.clone(); + let params = if date { + json!({ + "pivot": pivot, "field": field, "kind": "date", "levels": levels, + "start": range.and_then(|r| r.start).unwrap_or(lo.floor()), "end": range.and_then(|r| r.end).unwrap_or(hi.floor() + 1.0), + "autoStart": range.is_none_or(|r| r.start.is_none()), "autoEnd": range.is_none_or(|r| r.end.is_none()), + "days": range.map_or(1, |r| r.days), + }) + } else { + json!({ + "pivot": pivot, "field": field, "kind": "number", + "start": number.map_or(lo, |g| g.start), "end": number.map_or(hi, |g| g.end), "interval": number.map_or(10.0, |g| g.by), + "autoStart": number.is_none(), "autoEnd": number.is_none(), + }) + }; + s.ui_requests.push(crate::UiRequest::Dialog("pivotGrouping".into(), params)); + ok() +} + +pub(super) fn group(s: &mut Session, p: &Json) -> Result { + const C: &str = "pivot.group"; + if p.get("by").is_none() && p.get("levels").is_none() { + return grouping_dialog(s, p); + } + let field = str_param(p, "field").ok_or_else(|| bad(C, "missing `field`"))?.trim().to_string(); + // `by`: one level name, `range`, `none`, or a list of date and time levels. + let raw: Vec = match p.get("by").or_else(|| p.get("levels")) { + Some(Json::String(b)) => vec![b.clone()], + Some(Json::Array(a)) => a.iter().take(16).map(|x| x.as_str().unwrap_or("?").to_string()).collect(), + _ => return Err(bad(C, "`by` is years|quarters|months|days|hours|minutes|seconds|range|none or a list of levels")), + }; + let range = raw.iter().any(|b| b.trim().eq_ignore_ascii_case("range")); + let mut levels: Vec = Vec::new(); + if !range { + for b in &raw { + levels.push( + group_param(Some(b)) + .ok_or_else(|| bad(C, format!("unknown grouping `{b}` (years|quarters|months|days|hours|minutes|seconds|range|none)")))?, + ); + } + levels.retain(|g| *g != PivotDateGroup::None); + } else if raw.len() > 1 { + return Err(bad(C, "`range` can't be combined with date levels")); + } + let sys = s.doc()?.wb.date_system; + let (start, end) = (date_or_number(C, p, "start", sys)?, date_or_number(C, p, "end", sys)?); + let interval = match p.get("interval") { + None | Some(Json::Null) => None, + Some(v) => Some(v.as_f64().filter(|x| x.is_finite()).ok_or_else(|| bad(C, "`interval` must be a number"))?), + }; + let days = match p.get("days") { + None | Some(Json::Null) => 1, + Some(v) => v + .as_u64() + .filter(|d| (1..=u64::from(PivotDateRange::MAX_DAYS)).contains(d)) + .ok_or_else(|| bad(C, "`days` is a whole number from 1 to 32767"))? as u32, + }; + if days > 1 && levels != [PivotDateGroup::Days] { + return Err(other("Number of days applies only when grouping by Days alone.")); + } update(s, p, |pt, src, _| { - if by != PivotDateGroup::None { - let c = src.col(&field).ok_or_else(|| bad(C, format!("no field `{field}` in the source data")))?; + let grouping = range || !levels.is_empty(); + if grouping { + if !pt.calculated_items.is_empty() { + return Err(other(super::pivot_tools::GROUP_WITH_CALC_ITEMS)); + } + let c = src.col(&field).ok_or_else(|| other(super::pivot_tools::CANNOT_GROUP))?; if !src.numeric.get(c).copied().unwrap_or(false) { - return Err(other("Cannot group that selection.")); + return Err(other(super::pivot_tools::CANNOT_GROUP)); + } + if range { + // Excel's defaults: from the smallest to the largest number, in tens. + let (lo, hi) = number_bounds(src, c); + let g = PivotNumberGroup::checked(start.unwrap_or(lo), end.unwrap_or(hi), interval.unwrap_or(10.0)).ok_or_else(|| { + other("Enter a starting number no greater than the ending number and a positive By value (at most 100,000 groups).") + })?; + set_grouping(pt, &field, &[], Some(g), None)?; + return Ok(json!({})); + } + if let (Some(a), Some(b)) = (start, end) + && a > b + { + return Err(other("The ending date must be after the starting date.")); } + let dr = (start.is_some() || end.is_some() || days > 1).then_some(PivotDateRange { start, end, days }); + set_grouping(pt, &field, &levels, None, dr)?; + } else { + set_grouping(pt, &field, &[], None, None)?; } - let f = axis_field(C, pt, &field)?; - f.date_group = by; - f.collapsed_items.clear(); Ok(json!({})) }) } @@ -1191,14 +1701,10 @@ fn collapse(s: &mut Session, p: &Json) -> Result { update(s, p, |pt, src, wb| { let all = match &items { Some(v) => v.clone(), - None => { - let f = pt.rows.iter().chain(pt.columns.iter()).find(|f| f.source_col.eq_ignore_ascii_case(&field)); - let g = f.map(|f| f.date_group).unwrap_or_default(); - match src.col(&field) { - Some(c) => pv::distinct_labels(wb, src, c, g, 100_000), - None => vec![], - } - } + None => match pv::resolve_name(src, pt, &field) { + Some((c, g)) => pv::distinct_labels(wb, src, c, g, 100_000), + None => vec![], + }, }; let f = axis_field(C, pt, &field)?; if on { @@ -1325,8 +1831,12 @@ fn change_source(s: &mut Session, p: &Json) -> Result { update(s, p, move |pt, _, _| { pt.source = source.clone(); let mut dropped: Vec = vec![]; + // Grouped fields and calculated items go with the source field they are made from. + pt.group_fields.retain(|g| src.col(&g.base).is_some()); + pt.calculated_items.retain(|c| src.col(&c.field).is_some()); + let groups: Vec = pt.group_fields.iter().map(|g| g.name.clone()).collect(); let keep = |name: &str, dropped: &mut Vec| { - let ok = src.col(name).is_some(); + let ok = src.col(name).is_some() || groups.iter().any(|g| g.eq_ignore_ascii_case(name)); if !ok { dropped.push(name.to_string()); } @@ -1368,6 +1878,16 @@ fn delete(s: &mut Session, p: &Json) -> Result { } } pv::clear_output(&mut cx.wb, si, pi); + // Its timelines no longer filter it. + if let Some(id) = cx.wb.sheet(si).and_then(|sh| sh.pivots.get(pi)).map(|pt| pt.id) { + for i in 0..cx.wb.sheets.len() { + if cx.wb.sheet(i).is_some_and(|sh| sh.timelines.iter().any(|t| t.pivots.contains(&id))) { + for t in cx.sheet_mut(i)?.timelines.iter_mut() { + t.pivots.retain(|x| *x != id); + } + } + } + } let sh = cx.sheet_mut(si)?; if pi < sh.pivots.len() { sh.pivots.remove(pi); @@ -1377,6 +1897,33 @@ fn delete(s: &mut Session, p: &Json) -> Result { }) } +/// PivotTable Analyze › Clear › Clear All: an empty PivotTable on the same source, as Excel +/// leaves it. Calculated fields are definitions rather than layout, so they stay available. +fn clear_all(s: &mut Session, p: &Json) -> Result { + update(s, p, |pt, _, _| { + pt.rows.clear(); + pt.columns.clear(); + pt.values.clear(); + pt.filters.clear(); + pt.values_on_rows = false; + pt.values_position = None; + Ok(json!({})) + }) +} + +/// PivotTable Analyze › Clear › Clear Filters: report filters show (All) again and row and +/// column fields lose their hidden-items filters. +fn clear_filters(s: &mut Session, p: &Json) -> Result { + update(s, p, |pt, _, _| { + let axes: Vec = pt.rows.iter().chain(pt.columns.iter()).map(|f| f.source_col.to_ascii_lowercase()).collect(); + pt.filters.retain(|f| f.rule.is_none() && !axes.contains(&f.source_col.to_ascii_lowercase())); + for f in &mut pt.filters { + f.selected = None; + } + Ok(json!({})) + }) +} + // ---------------------------------------------------------------- PivotChart /// The PivotTable a PivotChart call targets: `pivot` may also be a cell of its report. @@ -1457,7 +2004,7 @@ fn insert_pivot_chart(s: &mut Session, p: &Json) -> Result { fn suggestions(wb: &Workbook, src: &pv::Source) -> Vec { let n = src.headers.len(); let h = |i: usize| src.headers.get(i).cloned().unwrap_or_default(); - let distinct = |i: usize| pv::distinct_labels(wb, src, i, PivotDateGroup::None, 1001).len(); + let distinct = |i: usize| pv::distinct_labels(wb, src, i, pv::Grouping::default(), 1001).len(); let records = src.rows.len().max(1); let is_num = |i: usize| src.numeric.get(i).copied().unwrap_or(false); let is_date = |i: usize| src.dates.get(i).copied().unwrap_or(false); @@ -1552,6 +2099,7 @@ fn recommended(s: &mut Session, p: &Json) -> Result { #[cfg(test)] mod tests { use gridcraft_core::{CellError, CellRef, Value}; + use gridcraft_model::{PivotCompare, PivotFilter, PivotRule, PivotTopBy}; use serde_json::{Value as Json, json}; use crate::Session; @@ -1615,6 +2163,67 @@ mod tests { assert_eq!(s.doc().unwrap().wb.sheets.len(), 1); } + #[test] + fn every_show_values_as_mode() { + // Sum of Sales, Region down (East, North, West), Product across (Apples, Pears, Plums): + // Apples Pears Plums Total + // East 160 195 60 415 + // North 90 30 15 135 + // West 170 45 70 285 + // Total 420 270 145 835 + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "columns": ["Product"], "values": ["Sales"]})).unwrap(); + let close = |s: &Session, a: &str, want: f64| { + let got = val(s, a).as_f64().unwrap_or_else(|| panic!("{a} = {:?}", val(s, a))); + assert!((got - want).abs() < 1e-12, "{a}: {got} vs {want}"); + }; + let set = |s: &mut Session, p: serde_json::Value| { + let mut p = p; + p["field"] = json!("Sum of Sales"); + s.execute("pivot.valueSettings", p).unwrap(); + }; + set(&mut s, json!({"showAs": "percentOf", "baseField": "Region", "baseItem": "East"})); + close(&s, "B6", 90.0 / 160.0); + close(&s, "B5", 1.0); + assert_eq!(val(&s, "B8"), Value::Empty, "the grand total row has no Region item"); + let st = s.execute("cell.get", json!({"cell": "B6"})).unwrap(); + assert_eq!(st["style"]["num_fmt"], "0.00%"); + set(&mut s, json!({"showAs": "differenceFrom", "baseItem": "(previous)"})); + close(&s, "B6", 90.0 - 160.0); + assert_eq!(val(&s, "B5"), Value::Empty, "East has no previous item"); + set(&mut s, json!({"showAs": "percentDifferenceFrom", "baseItem": "East"})); + close(&s, "C7", (45.0 - 195.0) / 195.0); + set(&mut s, json!({"showAs": "runningTotal"})); + close(&s, "B7", 160.0 + 90.0 + 170.0); + set(&mut s, json!({"showAs": "percentRunningTotal"})); + close(&s, "B6", (160.0 + 90.0) / 420.0); + set(&mut s, json!({"showAs": "rankAscending"})); + close(&s, "B6", 1.0); + close(&s, "B7", 3.0); + set(&mut s, json!({"showAs": "rankDescending"})); + close(&s, "B7", 1.0); + set(&mut s, json!({"showAs": "index"})); + close(&s, "B5", 160.0 * 835.0 / (415.0 * 420.0)); + set(&mut s, json!({"showAs": "percentOfParentRowTotal"})); + close(&s, "B5", 160.0 / 420.0); + set(&mut s, json!({"showAs": "percentOfParentColumnTotal"})); + close(&s, "B5", 160.0 / 415.0); + // % of Parent Total over nested rows: Region > Product, each product as a share of its + // region. + s.execute("pivot.moveField", json!({"field": "Product", "from": "columns", "to": "rows"})).unwrap(); + set(&mut s, json!({"showAs": "percentOfParentTotal", "baseField": "Region"})); + let row = find_row(&s, "Apples").unwrap(); + close(&s, &CellRef::new(row, 1).a1(), 160.0 / 415.0); + assert!(s.execute("pivot.valueSettings", json!({"field": "Sum of Sales", "baseField": "Units"})).is_err()); + // Through xlsx. + set(&mut s, json!({"showAs": "percentOf", "baseField": "Region", "baseItem": "North"})); + let want = pt(&s).values; + let b64 = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap()["base64"].as_str().unwrap().to_string(); + s.execute("file.open", json!({"name": "p.xlsx", "base64": b64})).unwrap(); + s.execute("sheet.activate", json!({"sheet": 0})).unwrap(); + assert_eq!(pt(&s).values, want); + } + #[test] fn region_by_product_sum_of_sales() { let mut s = sales(); @@ -1676,6 +2285,355 @@ mod tests { assert_eq!(val(&s, &CellRef::new(last.end.row, 7).a1()), Value::Number(TOTAL)); } + fn grand_total(s: &Session) -> Value { + let row = find_row(s, "Grand Total").expect("a Grand Total row"); + val(s, &CellRef::new(row, pt(s).last_range.unwrap().start.col + 1).a1()) + } + + fn dormant_product_rules() -> [(PivotRule, f64); 3] { + [ + (PivotRule::Label { op: PivotCompare::Equals, value: "Pears".into(), value2: String::new() }, 270.0), + (PivotRule::Value { op: PivotCompare::GreaterOrEqual, field: "Sum of Sales".into(), value: 270.0, value2: 0.0 }, 690.0), + (PivotRule::Top { top: true, count: 1.0, by: PivotTopBy::Items, field: "Sum of Sales".into() }, 420.0), + ] + } + + fn add_dormant_product_rule(s: &mut Session, rule: PivotRule) -> Vec { + // XLSX can retain a rule on a pivotField that has no axis. Build that state without + // an external workbook or using a command that requires the field to be on an axis. + let d = s.doc_mut().unwrap(); + let active = d.wb.active_sheet; + let p = std::sync::Arc::make_mut(&mut d.wb).sheet_mut(active).unwrap().pivots.last_mut().unwrap(); + p.filters.push(PivotFilter { source_col: "Product".into(), selected: None, rule: Some(rule) }); + p.filters.clone() + } + + fn pivot_report_values(s: &Session) -> Vec { + // Include the report-filter rows above the anchor and the empty cells around the report. + (0..20).flat_map(|row| (0..8).map(move |col| val(s, &CellRef::new(row, col).a1()))).collect() + } + + #[test] + fn dormant_filter_rules_do_not_change_report_on_refresh() { + for (rule, _) in dormant_product_rules() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap(); + let before = pt(&s); + let cells = s.doc().unwrap().wb.active().unwrap().cells.clone(); + let filters = add_dormant_product_rule(&mut s, rule); + s.execute("pivot.refresh", json!({})).unwrap(); + assert_eq!(pt(&s).filters, filters); + assert_eq!(pt(&s).anchor, before.anchor, "a dormant rule is not a report-filter row"); + assert_eq!(pt(&s).last_range, before.last_range); + assert_eq!(s.doc().unwrap().wb.active().unwrap().cells, cells, "a dormant rule leaves the rendered cells and styles unchanged"); + assert_eq!(grand_total(&s), Value::Number(TOTAL)); + let info = s.execute("pivot.fields", json!({})).unwrap(); + assert_eq!(info["fields"][1]["name"], "Product"); + assert_eq!(info["fields"][1]["areas"], json!([]), "a dormant rule does not place its field in Filters"); + assert_eq!(info["filters"][0]["report"], false); + } + } + + #[test] + fn dormant_filter_rules_round_trip_and_activate_in_rows_or_columns() { + for area in ["rows", "columns"] { + for (rule, filtered_total) in dormant_product_rules() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"values": ["Sales"]})).unwrap(); + let before = pt(&s); + let cells = pivot_report_values(&s); + let filters = add_dormant_product_rule(&mut s, rule); + // A second save proves the imported rule is modelled, rather than only retained + // in opaque XML from the first file. + for _ in 0..2 { + let b64 = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap()["base64"].as_str().unwrap().to_string(); + s.execute("file.open", json!({"name": "dormant-filters.xlsx", "base64": b64})).unwrap(); + s.execute("sheet.activate", json!({"sheet": 0})).unwrap(); + assert_eq!(pt(&s).filters, filters, "the rule must survive XLSX while its field has no axis"); + assert_eq!(pt(&s).anchor, before.anchor); + assert_eq!(pt(&s).last_range, before.last_range); + assert_eq!(pivot_report_values(&s), cells); + s.execute("pivot.refresh", json!({})).unwrap(); + assert_eq!(pt(&s).anchor, before.anchor); + assert_eq!(pt(&s).last_range, before.last_range); + assert_eq!(pivot_report_values(&s), cells); + } + s.execute("pivot.addField", json!({"field": "Product", "area": area})).unwrap(); + assert_eq!(pt(&s).filters, filters, "adding the field to an axis must activate its saved rule"); + let report = pt(&s); + assert_eq!(val(&s, &report.last_range.unwrap().end.a1()), Value::Number(filtered_total)); + assert!(report.rows.iter().chain(report.columns.iter()).any(|f| f.source_col == "Product")); + } + } + } + + #[test] + fn clear_filter_removes_dormant_rules_and_keeps_report_filters() { + for (command, params) in + [("pivot.clearFilter", json!({"field": "Product"})), ("pivot.clearFilter", json!({})), ("pivot.clearFilters", json!({}))] + { + for (rule, _) in dormant_product_rules() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"], "filters": ["Date"]})).unwrap(); + let before = pt(&s); + let cells = s.doc().unwrap().wb.active().unwrap().cells.clone(); + add_dormant_product_rule(&mut s, rule); + s.execute(command, params.clone()).unwrap(); + assert_eq!(pt(&s).filters, before.filters, "clearing the rule must not leave a Product report filter"); + assert_eq!(pt(&s).anchor, before.anchor); + assert_eq!(pt(&s).last_range, before.last_range); + assert_eq!(s.doc().unwrap().wb.active().unwrap().cells, cells); + } + } + } + + #[test] + fn dormant_value_rules_follow_value_field_rename_and_removal() { + for (rule, _) in dormant_product_rules().into_iter().skip(1) { + let mut s = sales(); + let mut control = sales(); + for session in [&mut s, &mut control] { + session.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales", "Units"], "filters": ["Date"]})).unwrap(); + } + add_dormant_product_rule(&mut s, rule); + for session in [&mut s, &mut control] { + session.execute("pivot.valueSettings", json!({"field": "Sum of Sales", "name": "Revenue"})).unwrap(); + } + let renamed = pt(&s); + assert_eq!(renamed.filters.len(), 2); + assert_eq!(renamed.filters[1].rule.as_ref().and_then(PivotRule::value_field), Some("Revenue")); + assert_eq!(renamed.filters[1].selected, None); + assert_eq!(renamed.anchor, pt(&control).anchor); + assert_eq!(renamed.last_range, pt(&control).last_range); + assert_eq!(s.doc().unwrap().wb.active().unwrap().cells, control.doc().unwrap().wb.active().unwrap().cells); + let info = s.execute("pivot.fields", json!({})).unwrap(); + assert_eq!(info["fields"][1]["areas"], json!([])); + assert_eq!(info["filters"][1]["report"], false); + for session in [&mut s, &mut control] { + session.execute("pivot.removeField", json!({"field": "Revenue", "area": "values"})).unwrap(); + } + assert_eq!( + pt(&s).filters, + pt(&control).filters, + "removing the value field removes its dormant rule without creating a Product report filter" + ); + assert_eq!(pt(&s).anchor, pt(&control).anchor); + assert_eq!(pt(&s).last_range, pt(&control).last_range); + assert_eq!(s.doc().unwrap().wb.active().unwrap().cells, control.doc().unwrap().wb.active().unwrap().cells); + assert_eq!(grand_total(&s), Value::Number(72.0)); + } + } + + #[test] + fn label_filters() { + // Sum of Sales by Region: East 415, North 135, West 285. + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap(); + let mut label = |op: &str, value: &str, value2: Option<&str>| { + let mut p = json!({"field": "Region", "op": op, "value": value}); + if let Some(v) = value2 { + p["value2"] = json!(v); + } + s.execute("pivot.labelFilter", p).unwrap(); + grand_total(&s) + }; + assert_eq!(label("beginsWith", "E", None), Value::Number(415.0)); + assert_eq!(label("contains", "ST", None), Value::Number(700.0), "case does not matter"); + assert_eq!(label("equals", "?orth", None), Value::Number(135.0), "wildcards"); + assert_eq!(label("doesNotEqual", "e*t", None), Value::Number(420.0)); + assert_eq!(label("endsWith", "h", None), Value::Number(135.0)); + assert_eq!(label("doesNotBeginWith", "w", None), Value::Number(550.0)); + assert_eq!(label("greaterThan", "N", None), Value::Number(420.0)); + assert_eq!(label("between", "Z", Some("M")), Value::Number(420.0), "bounds in either order"); + assert_eq!(label("notBetween", "M", Some("Z")), Value::Number(415.0)); + assert!(find_row(&s, "West").is_none()); + // One filter per field: selecting items clears the label filter, and the reverse. + s.execute("pivot.filter", json!({"field": "Region", "selected": ["North", "West"]})).unwrap(); + assert_eq!(pt(&s).filters.len(), 1); + assert!(pt(&s).filters[0].rule.is_none()); + assert_eq!(grand_total(&s), Value::Number(420.0)); + s.execute("pivot.labelFilter", json!({"field": "Region", "op": "equals", "value": "East"})).unwrap(); + assert_eq!(pt(&s).filters[0].selected, None); + assert_eq!(grand_total(&s), Value::Number(415.0)); + // Selecting all items keeps the label filter. + s.execute("pivot.filter", json!({"field": "Region", "selected": null})).unwrap(); + assert_eq!(grand_total(&s), Value::Number(415.0)); + let info = s.execute("pivot.list", json!({})).unwrap(); + assert_eq!(info[0]["filters"][0]["rule"], json!({"type": "label", "op": "equals", "value": "East", "value2": ""})); + s.execute("pivot.clearFilter", json!({"field": "Region"})).unwrap(); + assert!(pt(&s).filters.is_empty()); + assert_eq!(grand_total(&s), Value::Number(TOTAL)); + // Numbers compare as numbers. + s.execute("pivot.addField", json!({"field": "Units", "area": "rows", "position": 0})).unwrap(); + s.execute("pivot.removeField", json!({"field": "Region"})).unwrap(); + s.execute("pivot.labelFilter", json!({"field": "Units", "op": "greaterThanOrEqual", "value": 10})).unwrap(); + assert_eq!(grand_total(&s), Value::Number(220.0)); + } + + #[test] + fn value_and_top10_filters() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales", "Units"]})).unwrap(); + let mut run = |c: &str, p: Json| { + s.execute(c, p).unwrap(); + grand_total(&s) + }; + assert_eq!(run("pivot.valueFilter", json!({"field": "Region", "op": "greaterThan", "value": 200})), Value::Number(700.0)); + assert_eq!(run("pivot.valueFilter", json!({"field": "Region", "op": "between", "value": 300, "value2": 100})), Value::Number(420.0)); + // By another value field: Units are East 33, North 12, West 27. + let units = json!({"field": "Region", "op": "lessThan", "value": 20, "valueField": "Sum of Units"}); + assert_eq!(run("pivot.valueFilter", units), Value::Number(135.0)); + assert_eq!(run("pivot.top10", json!({"field": "Region", "count": 1})), Value::Number(415.0)); + assert_eq!(run("pivot.top10", json!({"field": "Region", "top": false, "count": 1})), Value::Number(135.0)); + assert_eq!(run("pivot.top10", json!({"field": "Region", "count": 50, "by": "percent"})), Value::Number(700.0)); + assert_eq!(run("pivot.top10", json!({"field": "Region", "count": 400, "by": "sum"})), Value::Number(415.0)); + assert_eq!(run("pivot.top10", json!({"field": "Region", "count": 99})), Value::Number(TOTAL)); + // A renamed value field keeps its filter; a removed one takes the filter with it. + s.execute("pivot.top10", json!({"field": "Region", "count": 1, "valueField": "Sum of Units"})).unwrap(); + s.execute("pivot.valueSettings", json!({"field": "Sum of Units", "name": "Units Sold"})).unwrap(); + assert_eq!(pt(&s).filters[0].rule.as_ref().and_then(|r| r.value_field()), Some("Units Sold")); + s.execute("pivot.removeField", json!({"field": "Units Sold", "area": "values"})).unwrap(); + assert!(pt(&s).filters.is_empty()); + assert_eq!(grand_total(&s), Value::Number(TOTAL)); + // Within each parent: the top Product of each Region, with totals of the items shown. + s.execute("pivot.addField", json!({"field": "Product", "area": "rows"})).unwrap(); + s.execute("pivot.top10", json!({"field": "Product", "count": 1})).unwrap(); + assert_eq!(grand_total(&s), Value::Number(195.0 + 90.0 + 170.0)); + let east = find_row(&s, "East").unwrap(); + assert_eq!(val(&s, &CellRef::new(east, 1).a1()), Value::Number(195.0), "East's subtotal counts only Pears"); + assert_eq!(text(&s, &CellRef::new(east + 1, 0).a1()), "Pears"); + // A filter follows its field between rows and columns, and goes with it to the filters area. + s.execute("pivot.moveField", json!({"field": "Product", "from": "rows", "to": "columns"})).unwrap(); + assert!(pt(&s).filters[0].rule.is_some()); + s.execute("pivot.moveField", json!({"field": "Product", "from": "columns", "to": "filters"})).unwrap(); + assert_eq!(pt(&s).filters[0].rule, None); + assert_eq!(pt(&s).filters[0].selected, None); + // On a column field: Apples 420 and Pears 270 pass. + s.execute("pivot.moveField", json!({"field": "Product", "from": "filters", "to": "columns"})).unwrap(); + s.execute("pivot.valueFilter", json!({"field": "Product", "op": "greaterThanOrEqual", "value": 270})).unwrap(); + let r = find_row(&s, "Grand Total").unwrap(); + assert_eq!(text(&s, "D4"), "Grand Total", "two product columns"); + assert_eq!(val(&s, &CellRef::new(r, 3).a1()), Value::Number(690.0)); + // Clear Filters on the whole PivotTable; a report filter stays, showing all. + s.execute("pivot.addField", json!({"field": "Date", "area": "filters"})).unwrap(); + s.execute("pivot.filter", json!({"field": "Date", "selected": ["1/15/2026"]})).unwrap(); + s.execute("pivot.clearFilter", json!({})).unwrap(); + assert_eq!(pt(&s).filters.len(), 1); + assert_eq!(pt(&s).filters[0].source_col, "Date"); + assert_eq!(pt(&s).filters[0].selected, None); + } + + #[test] + fn top10_keeps_items_tied_at_the_cut_off() { + // #498 review: Excel keeps every item tied with the last one a Top N keeps. + let mut s = Session::new(); + s.new_workbook(); + let rows = json!([["Item", "Score"], ["A", 5], ["B", 5], ["C", 5], ["D", 10], ["E", 1]]); + s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap(); + s.execute("insert.pivotTable", json!({"source": "Sheet1!A1:B6", "rows": ["Item"], "values": ["Score"]})).unwrap(); + // Top 2: D, then A, B and C all tie at 5 → 25. + s.execute("pivot.top10", json!({"field": "Item", "count": 2})).unwrap(); + assert_eq!(grand_total(&s), Value::Number(25.0)); + // Top 1 has no tie at its cut-off. + s.execute("pivot.top10", json!({"field": "Item", "count": 1})).unwrap(); + assert_eq!(grand_total(&s), Value::Number(10.0)); + // Bottom 2: E, then the three tied at 5 → 16. + s.execute("pivot.top10", json!({"field": "Item", "count": 2, "top": false})).unwrap(); + assert_eq!(grand_total(&s), Value::Number(16.0)); + } + + #[test] + fn filters_round_trip_and_refusals() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region", "Product"], "columns": [], "values": ["Sales", "Units"]})).unwrap(); + s.execute("pivot.labelFilter", json!({"field": "Region", "op": "doesNotContain", "value": "or"})).unwrap(); + s.execute("pivot.top10", json!({"field": "Product", "top": false, "count": 2, "valueField": "Sum of Units"})).unwrap(); + let want = pt(&s).filters; + let total = grand_total(&s); + // West: Pears 3 and Plums 7 units (45 + 70); East: Plums 4 and Pears 13 units (60 + 195). + assert_eq!(total, Value::Number(370.0)); + let b64 = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap()["base64"].as_str().unwrap().to_string(); + s.execute("file.open", json!({"name": "f.xlsx", "base64": b64})).unwrap(); + s.execute("sheet.activate", json!({"sheet": 0})).unwrap(); + assert_eq!(pt(&s).filters, want); + s.execute("pivot.refresh", json!({})).unwrap(); + assert_eq!(grand_total(&s), total); + // Refusals. + let refused = [ + ("pivot.labelFilter", json!({"field": "Units", "op": "equals", "value": "1"})), + ("pivot.labelFilter", json!({"field": "Region", "op": "between", "value": "A"})), + ("pivot.labelFilter", json!({"field": "Region", "op": "near", "value": "A"})), + ("pivot.labelFilter", json!({"field": "Region", "op": "equals"})), + ("pivot.valueFilter", json!({"field": "Region", "op": "contains", "value": 1})), + ("pivot.valueFilter", json!({"field": "Region", "op": "equals", "value": "x"})), + ("pivot.valueFilter", json!({"field": "Region", "op": "equals", "value": 1, "valueField": "Nope"})), + ("pivot.top10", json!({"field": "Region", "count": 0})), + ("pivot.top10", json!({"field": "Region", "count": 1.5})), + ("pivot.top10", json!({"field": "Region", "count": 120, "by": "percent"})), + ("pivot.top10", json!({"field": "Region", "by": "rows"})), + ("pivot.clearFilter", json!({"field": "Nope"})), + ]; + for (c, p) in refused { + assert!(s.execute(c, p.clone()).is_err(), "{c} {p}"); + } + assert_eq!(pt(&s).filters, want, "refusals change nothing"); + s.execute("pivot.removeField", json!({"field": "Sum of Sales", "area": "values"})).unwrap(); + s.execute("pivot.removeField", json!({"field": "Sum of Units", "area": "values"})).unwrap(); + assert!(s.execute("pivot.top10", json!({"field": "Region"})).is_err(), "no value field"); + } + + #[test] + fn number_range_grouping() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Units"], "values": ["Sales"]})).unwrap(); + // Units in fives: 0-4 (2, 4, 3, 1), 5-9 (5, 8, 7, 9, 6, 5), 10-14 (10, 12). + s.execute("pivot.group", json!({"field": "Units", "by": "range", "start": 0, "end": 14, "interval": 5})).unwrap(); + assert_eq!(text(&s, "A4"), "0-4"); + assert_eq!(val(&s, "B4"), Value::Number(150.0)); + assert_eq!(text(&s, "A5"), "5-9"); + assert_eq!(val(&s, "B5"), Value::Number(465.0)); + assert_eq!(text(&s, "A6"), "10-14"); + assert_eq!(val(&s, "B6"), Value::Number(220.0)); + assert_eq!(val(&s, "B7"), Value::Number(TOTAL)); + let l = s.execute("pivot.list", json!({})).unwrap(); + assert_eq!(l[0]["rows"][0]["group"], "range"); + assert_eq!(l[0]["rows"][0]["range"], json!({"start": 0.0, "end": 14.0, "interval": 5.0})); + // Values outside the bounds fall into "end", sorted first and last. + s.execute("pivot.group", json!({"field": "Units", "by": "range", "start": 3, "end": 9, "interval": 5})).unwrap(); + assert_eq!(text(&s, "A4"), "<3"); + assert_eq!(val(&s, "B4"), Value::Number(45.0)); + assert_eq!(text(&s, "A5"), "3-7"); + assert_eq!(text(&s, "A7"), ">9"); + assert_eq!(val(&s, "B7"), Value::Number(220.0)); + // Fractional widths show both ends; the defaults are the data's bounds, in tens. + s.execute("pivot.group", json!({"field": "Units", "by": "range", "start": 0, "end": 12, "interval": 2.5})).unwrap(); + assert_eq!(text(&s, "A4"), "0-2.5"); + s.execute("pivot.group", json!({"field": "Units", "by": "range"})).unwrap(); + assert_eq!(text(&s, "A4"), "1-10"); + assert_eq!(val(&s, "B4"), Value::Number(TOTAL - 120.0)); + assert_eq!(text(&s, "A5"), "11-20"); + // Through xlsx. + let want = pt(&s).rows; + let b64 = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap()["base64"].as_str().unwrap().to_string(); + s.execute("file.open", json!({"name": "p.xlsx", "base64": b64})).unwrap(); + s.execute("sheet.activate", json!({"sheet": 0})).unwrap(); + assert_eq!(pt(&s).rows, want); + // Unusable bounds, widths and fields are refused; `none` ungroups. + for bad in [ + json!({"field": "Units", "by": "range", "interval": 0}), + json!({"field": "Units", "by": "range", "interval": -5}), + json!({"field": "Units", "by": "range", "start": 10, "end": 1}), + json!({"field": "Units", "by": "range", "interval": 1e-9}), + json!({"field": "Units", "by": "range", "start": "low"}), + json!({"field": "Region", "by": "range"}), + ] { + assert!(s.execute("pivot.group", bad.clone()).is_err(), "{bad}"); + } + s.execute("pivot.group", json!({"field": "Units", "by": "none"})).unwrap(); + assert_eq!(text(&s, "A4"), "1"); + assert_eq!(pt(&s).rows[0].number_group, None); + } + #[test] fn date_grouping_by_quarter_and_month() { let mut s = sales(); @@ -2218,6 +3176,10 @@ mod tests { "pivot.moveField", "pivot.valueSettings", "pivot.filter", + "pivot.labelFilter", + "pivot.valueFilter", + "pivot.top10", + "pivot.clearFilter", "pivot.sort", "pivot.group", "pivot.collapse", @@ -2229,6 +3191,24 @@ mod tests { "pivot.refresh", "data.refreshAll", "pivot.changeSource", + "pivot.groupSelection", + "pivot.ungroup", + "pivot.calculatedItem", + "pivot.solveOrder", + "pivot.listFormulas", + "insert.timeline", + "pivot.insertTimeline", + "timeline.select", + "timeline.clear", + "timeline.level", + "timeline.scroll", + "timeline.caption", + "timeline.reportConnections", + "timeline.style", + "timeline.size", + "timeline.showHeader", + "timeline.periods", + "timeline.delete", "pivot.delete", ]; let hostile = [ @@ -2242,6 +3222,14 @@ mod tests { json!({"field": "Region", "selected": [null, 1, {"x": 1}], "index": 18446744073709551615u64, "showAs": "rank"}), json!({"apply": 99, "layout": "weird", "subtotals": 7, "style": "x".repeat(500)}), json!({"field": "Date", "by": "decades", "items": "nope", "order": "sideways"}), + json!({"field": "Region", "op": "between", "value": 1e308, "value2": "x", "count": -1, "by": "sum", "top": "x", "valueField": 7}), + json!({"field": "Region", "op": "notBetween", "value": ["x"], "count": 1e300, "by": "percent", "valueField": "nope"}), + json!({"field": "Date", "by": ["days", "hours", 7, null], "days": 4294967296u64, "start": "x", "end": -1e308, "items": [1, null]}), + json!({"field": "Date", "by": "days", "days": 32767, "start": 0, "end": 2958465}), + json!({"field": "Region", "items": ["East", "North"], "name": "", "formula": "=East[", "order": [null], "move": "sideways"}), + json!({"field": "Region", "name": "x".repeat(300), "formula": format!("={}", "East+".repeat(5000)), "delete": "yes"}), + json!({"timeline": -1, "start": "1/1/9999", "end": "1/1/1900", "level": "eons", "to": 1e300, "caption": 5, "pivots": ["Nope", 3]}), + json!({"timeline": "Date", "style": "x".repeat(100), "width": f64::NAN, "height": -5, "on": "maybe", "fields": ["Date", "Date", 9]}), ]; for c in cmds { for h in &hostile { @@ -2308,4 +3296,44 @@ mod tests { // 64 Sub items under the collapsed "g" show as one column group: (1 + grand total) x 256 columns fit. s.execute("pivot.addField", json!({"field": "Sub", "area": "columns"})).unwrap(); } + + #[test] + fn generate_getpivotdata_is_a_setting_on_by_default() { + let mut s = Session::new(); + assert!(s.prefs.generate_get_pivot_data); + assert_eq!(s.execute("pivot.generateGetPivotData", json!({})).unwrap()["on"], false); + assert_eq!(s.execute("pivot.generateGetPivotData", json!({"on": true})).unwrap()["on"], true); + assert!(s.journal.is_empty(), "a setting, not an edit of the workbook"); + // A prefs.json saved before the setting existed keeps it on. + let old: crate::Prefs = serde_json::from_str(r#"{"userName": "Ann"}"#).unwrap(); + assert!(old.generate_get_pivot_data); + } + + #[test] + fn clear_all_leaves_an_empty_pivottable_and_clear_filters_shows_every_item() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region", "Product"], "values": ["Sales"], "filters": ["Date"]})).unwrap(); + s.execute("pivot.filter", json!({"field": "Product", "selected": ["Apples"]})).unwrap(); + s.execute("pivot.filter", json!({"field": "Date", "selected": ["1/15/2026"]})).unwrap(); + let p = pt(&s); + assert_eq!(p.filters.len(), 2); + s.execute("pivot.clearFilters", json!({})).unwrap(); + let p = pt(&s); + // The report filter stays, showing (All); the row field's hidden items are gone. + assert_eq!(p.filters.len(), 1); + assert_eq!(p.filters[0].source_col, "Date"); + assert_eq!(p.filters[0].selected, None); + let gt = find_row(&s, "Grand Total").unwrap(); + assert_eq!(val(&s, &CellRef::new(gt, 1).a1()), Value::Number(TOTAL)); + // Clear All keeps the PivotTable, now empty, and is one undo step. + s.execute("pivot.calculatedField", json!({"name": "Tax", "formula": "=Sales*0.1", "values": false})).unwrap(); + s.execute("pivot.clearAll", json!({})).unwrap(); + let p = pt(&s); + assert!(p.rows.is_empty() && p.columns.is_empty() && p.values.is_empty() && p.filters.is_empty()); + assert_eq!(p.calculated_fields.len(), 1); + assert!(text(&s, "A3").starts_with("PivotTable1"), "{}", text(&s, "A3")); + assert_eq!(s.doc().unwrap().wb.active().unwrap().pivots.len(), 1); + s.execute("edit.undo", json!({})).unwrap(); + assert_eq!(pt(&s).rows.len(), 2); + } } diff --git a/crates/engine/src/cmd/pivot_tools.rs b/crates/engine/src/cmd/pivot_tools.rs new file mode 100644 index 000000000..5a01de64c --- /dev/null +++ b/crates/engine/src/cmd/pivot_tools.rs @@ -0,0 +1,909 @@ +//! PivotTable Analyze › Group (Group Selection, Ungroup) and › Calculations › Fields, Items & +//! Sets (Calculated Item, Solve Order, List Formulas). Group Field is `pivot.group` in +//! [`super::pivot`]. +//! +//! Behaviour follows Excel's documented rules, checked black-box against Excel 365: +//! - Group Selection gathers two or more items of a row or column field into a group (`Group1`) +//! of a new field named after the base field (`Region2`), placed just outside it. Items not in +//! a group stay items of their own. Ungroup on a group removes it, and the grouped field goes +//! when its last group does. +//! - A calculated item is a formula over the items of its field. It can't be added while a field +//! of the PivotTable is grouped by dates or number ranges, while a value field uses Average, +//! StdDev or Var, or while a field is in the Values area twice; those changes are refused +//! while the PivotTable has calculated items. + +use gridcraft_core::{CellRef, Value}; +use gridcraft_model::*; +use serde_json::{Value as Json, json}; + +use super::pivot::{find_pivot, in_axes, other, update}; +use super::*; +use crate::pivot as pv; + +pub fn specs() -> Vec { + vec![ + cmd!( + "pivot.groupSelection", + "Group Selection", + ["PivotTable Analyze", "Group"], + None, + "{pivot?, field?: row or column field, items?: [\"East\", \"North\"] (default: the item cells selected in the report), replace?} gathers two or more items into a new group (Group1) of a grouped field (Region2) → {field, group}", + has_doc, + group_selection + ), + cmd!( + "pivot.ungroup", + "Ungroup", + ["PivotTable Analyze", "Group"], + Some("Alt+Shift+Left"), + "{pivot?, field?, items?: [group names] (default: the item at the active cell), replace?} removes groups of a grouped field (the field goes with its last group), or a field's date or number grouping", + has_doc, + ungroup + ), + cmd!( + "pivot.calculatedItem", + "Calculated Item", + ["PivotTable Analyze", "Calculations"], + None, + "{pivot?, field?: row or column field (default: the field at the active cell), name, formula?: \"=East+West\" (items bare, 'quoted' or Field[Item]), delete?: bool, replace?} adds, changes or deletes a calculated item; without `name` opens the dialog → {name, field, formula, calculatedItems}", + has_doc, + calculated_item + ), + cmd!( + "pivot.solveOrder", + "Solve Order", + ["PivotTable Analyze", "Calculations"], + None, + "{pivot?, order?: [calculated item names, first to last], item?: name, move?: up|down, replace?} reorders the calculated items (where two meet, the later formula sets the value); without `order` or `item` opens the dialog → {order}", + has_doc, + solve_order + ), + cmd!( + query "pivot.fieldItems", + "Field Items", + [], + None, + "{pivot?, field} → {field, items: [{label, shown}]}: the field's items (grouped as the field is, with its calculated items) and whether its item selection shows each", + has_doc, + field_items_query + ), + cmd!( + "pivot.listFormulas", + "List Formulas", + ["PivotTable Analyze", "Calculations"], + None, + "{pivot?} inserts a sheet after the PivotTable's listing its calculated fields and calculated items with their solve order and formulas → {sheet}", + has_doc, + list_formulas + ), + ] +} + +/// Excel's refusal when a selection can't be grouped. +pub const CANNOT_GROUP: &str = "Cannot group that selection."; +/// Excel's refusal when a selection can't be ungrouped. +const CANNOT_UNGROUP: &str = "Cannot ungroup that selection."; +/// Calculated items and date or number grouping exclude each other. +pub const GROUPED_FIELD: &str = "This PivotTable report field is grouped. You cannot add a calculated item to a grouped field."; +/// A PivotTable with calculated items can't group by dates or number ranges. +pub const GROUP_WITH_CALC_ITEMS: &str = "You cannot group by dates or number ranges in a PivotTable report that has calculated items."; +/// Calculated items and Average, StdDev and Var exclude each other. +pub const NO_AVERAGES: &str = "Averages, standard deviations, and variances are not supported when a PivotTable report has calculated items."; +/// Calculated items and a field in Values twice exclude each other. +pub const NO_DUPLICATE_VALUES: &str = "Multiple data fields of the same field are not supported when a PivotTable report has calculated items."; +/// A field with calculated items stays in Rows or Columns. +pub const CALC_ITEM_AREA: &str = "A field with calculated items can only be in the Rows or Columns area."; +const ITEM_NOT_FOUND: &str = "An item name was not found. Check that you've typed the name correctly and that the item is in the PivotTable report."; +const NO_REFERENCES: &str = "References, names, and arrays are not supported in PivotTable formulas."; + +/// Whether a function can't be used while the PivotTable has calculated items. +pub fn averages(f: PivotFunc) -> bool { + matches!(f, PivotFunc::Average | PivotFunc::StdDev | PivotFunc::StdDevP | PivotFunc::Var | PivotFunc::VarP) +} + +/// Whether any field of the PivotTable is grouped by dates or number ranges. +pub fn has_range_grouping(pt: &PivotTable) -> bool { + pt.rows.iter().chain(pt.columns.iter()).any(|f| f.date_group != PivotDateGroup::None || f.number_group.is_some()) +} + +/// The item cells of the report that the selection covers: (field, item captions in report order). +fn selected_items(s: &Session, si: usize, pi: usize) -> Result<(String, Vec)> { + let d = s.doc()?; + let cells = pv::report_items(&d.wb, si, pi); + let mut field: Option = None; + let mut items: Vec = Vec::new(); + for (c, it) in &cells { + if !d.selection.ranges.iter().any(|r| r.contains(*c)) && d.selection.active != *c { + continue; + } + match &field { + Some(f) if !f.eq_ignore_ascii_case(&it.field) => return Err(other(CANNOT_GROUP)), + Some(_) => {} + None => field = Some(it.field.clone()), + } + if !items.iter().any(|x| x.eq_ignore_ascii_case(&it.label)) { + items.push(it.label.clone()); + } + } + let field = field + .ok_or_else(|| EngineError::Disabled("pivot.groupSelection".into(), "select items of a row or column field in the PivotTable".into()))?; + Ok((field, items)) +} + +/// The field and item at the active cell of the report, if it is an item cell. +pub fn active_item(s: &Session, si: usize, pi: usize) -> Option<(String, String)> { + let d = s.doc().ok()?; + let at = d.selection.active; + pv::report_items(&d.wb, si, pi).into_iter().find(|(c, _)| *c == at).map(|(_, it)| (it.field, it.label)) +} + +/// A list parameter of strings. +fn names_param(p: &Json, key: &str) -> Option> { + p.get(key).and_then(Json::as_array).map(|a| a.iter().take(100_000).filter_map(|x| x.as_str().map(|s| s.trim().to_string())).collect()) +} + +/// A field name not taken by a source, grouped or calculated field: `Region2`, `Region3`… +fn new_field_name(pt: &PivotTable, src: &pv::Source, base: &str) -> String { + let taken = |n: &str| src.col(n).is_some() || pt.group_field(n).is_some() || pt.calculated_fields.iter().any(|c| c.name.eq_ignore_ascii_case(n)); + (2..100_000).map(|k| format!("{base}{k}")).find(|n| !taken(n)).unwrap_or_else(|| format!("{base}2")) +} + +/// The next free group caption (`Group1`, `Group2`…); numbering runs across the grouped fields. +fn new_group_name(pt: &PivotTable) -> String { + let taken = |n: &str| pt.group_fields.iter().any(|g| g.groups.iter().any(|x| x.name.eq_ignore_ascii_case(n))); + (1..100_000).map(|k| format!("Group{k}")).find(|n| !taken(n)).unwrap_or_else(|| "Group1".into()) +} + +fn group_selection(s: &mut Session, p: &Json) -> Result { + const C: &str = "pivot.groupSelection"; + let (si, pi) = find_pivot(s, p)?; + let (field, items) = match (str_param(p, "field"), names_param(p, "items")) { + (Some(f), Some(items)) => (f.trim().to_string(), items), + (Some(_), None) | (None, Some(_)) => return Err(bad(C, "pass both `field` and `items`, or neither (the selected item cells)")), + (None, None) => { + let (field, items) = selected_items(s, si, pi)?; + // One item of a number or date field: Excel opens the Grouping dialog. + if items.len() < 2 { + let numeric = s.doc().ok().and_then(|d| { + let pt = d.wb.sheet(si)?.pivots.get(pi)?; + let src = pv::read_source(&d.wb, &pt.source, si).ok()?; + let c = src.col(&field)?; + src.numeric.get(c).copied() + }); + if numeric == Some(true) { + return super::pivot::group(s, &json!({"pivot": p.get("pivot").cloned().unwrap_or(Json::Null)})); + } + } + (field, items) + } + }; + if items.len() < 2 { + return Err(other(CANNOT_GROUP)); + } + update(s, p, |pt, src, wb| { + let entry = pt.rows.iter().chain(pt.columns.iter()).find(|f| f.source_col.eq_ignore_ascii_case(&field)).cloned(); + let Some(entry) = entry else { return Err(bad(C, format!("`{field}` isn't in the rows or columns area"))) }; + // Fields made by grouping, and fields grouped by dates or number ranges, can't be. + if pt.group_field(&field).is_some() || entry.date_group != PivotDateGroup::None || entry.number_group.is_some() { + return Err(other(CANNOT_GROUP)); + } + let c = src.col(&field).ok_or_else(|| other(CANNOT_GROUP))?; + let labels = pv::distinct_labels(wb, src, c, pv::Grouping::default(), 100_000); + let mut members: Vec = Vec::new(); + for it in &items { + let label = labels.iter().find(|l| l.eq_ignore_ascii_case(it)).ok_or_else(|| other(CANNOT_GROUP))?; + if !members.contains(label) { + members.push(label.clone()); + } + } + if members.len() < 2 { + return Err(other(CANNOT_GROUP)); + } + let name = new_group_name(pt); + let base = entry.source_col.clone(); + let existing = pt.group_fields.iter().position(|g| g.base.eq_ignore_ascii_case(&base)); + let gf_name = match existing.and_then(|i| pt.group_fields.get_mut(i)) { + Some(gf) => { + // Items move out of the groups they were in; a group left empty goes. + for g in gf.groups.iter_mut() { + g.items.retain(|x| !members.iter().any(|m| m.eq_ignore_ascii_case(x))); + } + gf.groups.retain(|g| !g.items.is_empty()); + gf.groups.push(PivotItemGroup { name: name.clone(), items: members }); + gf.name.clone() + } + None => { + let gf_name = new_field_name(pt, src, &base); + pt.group_fields.push(PivotGroupField { + name: gf_name.clone(), + base: base.clone(), + groups: vec![PivotItemGroup { name: name.clone(), items: members }], + }); + // The grouped field goes just outside its base field, in the same area. + let new = PivotField { source_col: gf_name.clone(), sort: entry.sort, ..PivotField::default() }; + let rows = pt.rows.iter().any(|f| f.source_col.eq_ignore_ascii_case(&base)); + let list = if rows { &pt.rows } else { &pt.columns }; + let at = list.iter().position(|f| f.source_col.eq_ignore_ascii_case(&base)).unwrap_or(0); + super::pivot::insert_axis_field(pt, rows, new, Some(at)); + gf_name + } + }; + Ok(json!({"field": gf_name, "group": name})) + }) +} + +/// Drops a grouped field from the PivotTable: its groups, its place in the areas and its filters. +fn drop_group_field(pt: &mut PivotTable, name: &str) { + pt.group_fields.retain(|g| !g.name.eq_ignore_ascii_case(name)); + super::pivot::take_from_axes(pt, name); + pt.filters.retain(|f| !f.source_col.eq_ignore_ascii_case(name)); +} + +fn ungroup(s: &mut Session, p: &Json) -> Result { + const C: &str = "pivot.ungroup"; + let (si, pi) = find_pivot(s, p)?; + let (field, items) = match str_param(p, "field") { + Some(f) => (f.trim().to_string(), names_param(p, "items")), + None => { + let (f, label) = active_item(s, si, pi).ok_or_else(|| EngineError::Disabled(C.into(), "select an item of a grouped field".into()))?; + (f, Some(vec![label])) + } + }; + update(s, p, |pt, _, _| { + if let Some(gi) = pt.group_fields.iter().position(|g| g.name.eq_ignore_ascii_case(&field)) { + let name = pt.group_fields.get(gi).map(|g| g.name.clone()).unwrap_or_default(); + match &items { + None => drop_group_field(pt, &name), + Some(list) => { + let gf = pt.group_fields.get_mut(gi).ok_or_else(|| other(CANNOT_UNGROUP))?; + let before = gf.groups.len(); + gf.groups.retain(|g| !list.iter().any(|x| x.eq_ignore_ascii_case(&g.name))); + if gf.groups.len() == before { + return Err(other(CANNOT_UNGROUP)); + } + if gf.groups.is_empty() { + drop_group_field(pt, &name); + } + } + } + return Ok(json!({"field": name})); + } + // A field grouped by dates or number ranges: its grouping goes (one plain field is left). + let grouped = pt + .rows + .iter() + .chain(pt.columns.iter()) + .any(|f| f.source_col.eq_ignore_ascii_case(&field) && (f.date_group != PivotDateGroup::None || f.number_group.is_some())); + if !grouped { + return Err(other(CANNOT_UNGROUP)); + } + super::pivot::set_grouping(pt, &field, &[], None, None)?; + Ok(json!({"field": field})) + }) +} + +// ---------------------------------------------------------------- calculated items + +/// Why the PivotTable can't take calculated items, if it can't. +pub fn calc_item_blocker(pt: &PivotTable) -> Option<&'static str> { + if has_range_grouping(pt) { + return Some(GROUPED_FIELD); + } + if pt.values.iter().any(|v| averages(v.func)) { + return Some(NO_AVERAGES); + } + let mut seen: Vec<&str> = Vec::new(); + for v in &pt.values { + if seen.iter().any(|x| x.eq_ignore_ascii_case(&v.source_col)) { + return Some(NO_DUPLICATE_VALUES); + } + seen.push(&v.source_col); + } + None +} + +/// The item captions of a source field (for a calculated item's formula), with its calculated items. +fn field_items(pt: &PivotTable, src: &pv::Source, wb: &Workbook, field: &str) -> Vec { + let mut v = match src.col(field) { + Some(c) => pv::distinct_labels(wb, src, c, pv::Grouping::default(), 100_000), + None => vec![], + }; + v.extend(pt.calculated_items.iter().filter(|c| c.field.eq_ignore_ascii_case(field)).map(|c| c.name.clone())); + v +} + +fn calc_items_json(pt: &PivotTable) -> Json { + json!(pt.calculated_items.iter().map(|c| json!({"field": c.field, "name": c.name, "formula": c.formula})).collect::>()) +} + +fn calculated_item(s: &mut Session, p: &Json) -> Result { + const C: &str = "pivot.calculatedItem"; + let (si, pi) = find_pivot(s, p)?; + let field = match str_param(p, "field") { + Some(f) => Some(f.trim().to_string()), + None => active_item(s, si, pi).map(|(f, _)| f), + }; + let Some(name) = str_param(p, "name").map(|n| n.trim().to_string()) else { + // Menu-style call: the Insert Calculated Item dialog for the active field. + let d = s.doc()?; + let pt = d.wb.sheet(si).and_then(|sh| sh.pivots.get(pi)).ok_or_else(|| other("There's no such PivotTable."))?; + let field = field.or_else(|| pt.rows.first().or(pt.columns.first()).map(|f| f.source_col.clone())); + let Some(field) = field else { + return Err(EngineError::Disabled(C.into(), "select an item of a row or column field in the PivotTable".into())); + }; + if pt.group_field(&field).is_some() { + return Err(other("Calculated items can't be added to a field made by Group Selection.")); + } + if let Some(why) = calc_item_blocker(pt) { + return Err(other(why)); + } + let src = pv::read_source(&d.wb, &pt.source, si).map_err(other)?; + let n = + (1..).map(|k| format!("Formula{k}")).find(|n| !pt.calculated_items.iter().any(|c| c.name.eq_ignore_ascii_case(n))).unwrap_or_default(); + let fields: Vec = pt.rows.iter().chain(pt.columns.iter()).map(|f| f.source_col.clone()).filter(|f| src.col(f).is_some()).collect(); + let items = field_items(pt, &src, &d.wb, &field); + let pivot = pt.name.clone(); + let existing = calc_items_json(pt); + s.ui_requests.push(crate::UiRequest::Dialog( + "pivotCalculatedItem".into(), + json!({"pivot": pivot, "field": field, "name": n, "formula": "= 0", "fields": fields, "items": items, "calculatedItems": existing}), + )); + return ok(); + }; + let field = field.ok_or_else(|| bad(C, "missing `field`"))?; + if name.is_empty() || name.chars().count() > 255 { + return Err(bad(C, "`name` must be 1 to 255 characters")); + } + let delete = bool_param(p, "delete").unwrap_or(false); + let formula = str_param(p, "formula").map(|f| { + let f = f.trim(); + if f.starts_with('=') { f.to_string() } else { format!("={f}") } + }); + if !delete && formula.is_none() { + return Err(bad(C, "missing `formula`")); + } + update(s, p, |pt, src, wb| { + let existing = pt.calculated_items.iter().position(|c| c.field.eq_ignore_ascii_case(&field) && c.name.eq_ignore_ascii_case(&name)); + if delete { + let i = existing.ok_or_else(|| bad(C, format!("no calculated item `{name}` in `{field}`")))?; + let gone = pt.calculated_items.remove(i); + // A selection of the field's items no longer lists it. + for f in pt.filters.iter_mut().filter(|f| f.source_col.eq_ignore_ascii_case(&gone.field)) { + if let Some(sel) = f.selected.as_mut() { + sel.retain(|x| !x.eq_ignore_ascii_case(&gone.name)); + } + } + return Ok(json!({"name": gone.name, "field": gone.field, "deleted": true, "calculatedItems": calc_items_json(pt)})); + } + if pt.group_field(&field).is_some() { + return Err(other("Calculated items can't be added to a field made by Group Selection.")); + } + let canon = + src.col(&field).and_then(|c| src.headers.get(c).cloned()).ok_or_else(|| bad(C, format!("no field `{field}` in the source data")))?; + if !in_axes(pt, &canon) { + return Err(other(CALC_ITEM_AREA)); + } + if let Some(why) = calc_item_blocker(pt) { + return Err(other(why)); + } + let data_items = match src.col(&canon) { + Some(c) => pv::distinct_labels(wb, src, c, pv::Grouping::default(), 100_000), + None => vec![], + }; + if data_items.iter().any(|l| l.eq_ignore_ascii_case(&name)) { + return Err(other(format!("\"{canon}\" already has an item named \"{name}\"."))); + } + let formula = formula.clone().unwrap_or_default(); + let mut known = field_items(pt, src, wb, &canon); + if existing.is_none() { + known.push(name.clone()); + } + let lookup = |n: &str| known.iter().find(|k| k.trim().eq_ignore_ascii_case(n.trim())).cloned(); + if let Err(e) = crate::pivot_items::compile(&formula, &canon, &lookup) { + return Err(other(if e.contains("named") { + ITEM_NOT_FOUND.to_string() + } else if e.contains("refer to cells") { + NO_REFERENCES.to_string() + } else { + e + })); + } + let item_name = match existing.and_then(|i| pt.calculated_items.get_mut(i)) { + Some(c) => { + c.formula = formula.clone(); + c.name.clone() + } + None => { + if pt.calculated_items.len() >= crate::pivot_items::MAX_CALC_ITEMS { + return Err(other(format!("A PivotTable can have at most {} calculated items.", crate::pivot_items::MAX_CALC_ITEMS))); + } + pt.calculated_items.push(PivotCalcItem { field: canon.clone(), name: name.clone(), formula: formula.clone() }); + // A selection of the field's items shows the new item too. + for f in pt.filters.iter_mut().filter(|f| f.source_col.eq_ignore_ascii_case(&canon)) { + if let Some(sel) = f.selected.as_mut() { + sel.push(name.clone()); + } + } + name.clone() + } + }; + Ok(json!({"name": item_name, "field": canon, "formula": formula, "calculatedItems": calc_items_json(pt)})) + }) +} + +fn solve_order(s: &mut Session, p: &Json) -> Result { + const C: &str = "pivot.solveOrder"; + let order = names_param(p, "order"); + let item = str_param(p, "item").map(|x| x.trim().to_string()); + if order.is_none() && item.is_none() { + // Menu-style call: the Calculated Item Solve Order dialog. + let (si, pi) = find_pivot(s, p)?; + let d = s.doc()?; + let pt = d.wb.sheet(si).and_then(|sh| sh.pivots.get(pi)).ok_or_else(|| other("There's no such PivotTable."))?; + if pt.calculated_items.is_empty() { + return Err(EngineError::Disabled(C.into(), "the PivotTable has no calculated items".into())); + } + let pivot = pt.name.clone(); + let list = calc_items_json(pt); + s.ui_requests.push(crate::UiRequest::Dialog("pivotSolveOrder".into(), json!({"pivot": pivot, "calculatedItems": list}))); + return ok(); + } + let up = match str_param(p, "move").map(str::to_ascii_lowercase).as_deref() { + None | Some("up") => true, + Some("down") => false, + Some(m) => return Err(bad(C, format!("unknown move `{m}` (up|down)"))), + }; + update(s, p, |pt, _, _| { + let n = pt.calculated_items.len(); + match (&order, &item) { + (Some(order), _) => { + if order.len() != n { + return Err(bad(C, "`order` must name every calculated item once")); + } + let mut sorted: Vec = Vec::with_capacity(n); + for name in order { + let i = pt + .calculated_items + .iter() + .position(|c| c.name.eq_ignore_ascii_case(name)) + .ok_or_else(|| bad(C, format!("no calculated item `{name}`")))?; + sorted.push(pt.calculated_items.remove(i)); + } + pt.calculated_items = sorted; + } + (None, Some(name)) => { + let i = pt + .calculated_items + .iter() + .position(|c| c.name.eq_ignore_ascii_case(name)) + .ok_or_else(|| bad(C, format!("no calculated item `{name}`")))?; + let j = if up { i.checked_sub(1) } else { Some(i + 1).filter(|j| *j < n) }; + if let Some(j) = j { + pt.calculated_items.swap(i, j); + } + } + (None, None) => {} + } + Ok(json!({"order": pt.calculated_items.iter().map(|c| c.name.clone()).collect::>()})) + }) +} + +fn field_items_query(s: &mut Session, p: &Json) -> Result { + const C: &str = "pivot.fieldItems"; + let field = str_param(p, "field").ok_or_else(|| bad(C, "missing `field`"))?.trim().to_string(); + let (si, pi) = find_pivot(s, p)?; + let d = s.doc()?; + let pt = d.wb.sheet(si).and_then(|sh| sh.pivots.get(pi)).ok_or_else(|| other("There's no such PivotTable."))?; + let src = pv::read_source(&d.wb, &pt.source, si).map_err(other)?; + let (c, g) = pv::resolve_name(&src, pt, &field).ok_or_else(|| bad(C, format!("no field `{field}` in the PivotTable")))?; + let mut labels = pv::distinct_labels(&d.wb, &src, c, g, 100_000); + labels.extend(pt.calculated_items.iter().filter(|x| x.field.eq_ignore_ascii_case(&field)).map(|x| x.name.clone())); + let selected = pt.filters.iter().find(|f| f.source_col.eq_ignore_ascii_case(&field)).and_then(|f| f.selected.as_ref()); + let items: Vec = labels + .into_iter() + .map(|l| { + let shown = selected.is_none_or(|sel| sel.iter().any(|x| x.eq_ignore_ascii_case(&l))); + json!({"label": l, "shown": shown}) + }) + .collect(); + Ok(json!({"field": field, "items": items})) +} + +fn list_formulas(s: &mut Session, p: &Json) -> Result { + let (si, pi) = find_pivot(s, p)?; + if s.doc()?.wb.protected_structure { + return Err(other("The workbook is protected and cannot be changed.")); + } + let pt = s.doc()?.wb.sheet(si).and_then(|sh| sh.pivots.get(pi)).cloned().ok_or_else(|| other("There's no such PivotTable."))?; + // (row, column, text, bold, italic) as Excel lays the sheet out. + let mut cells: Vec<(u32, u32, String, bool, bool)> = Vec::new(); + let mut row = 0u32; + let section = |cells: &mut Vec<(u32, u32, String, bool, bool)>, row: &mut u32, title: &str, what: &str, list: Vec<(String, String)>| { + cells.push((*row, 0, title.into(), true, true)); + *row += 1; + for (c, h) in ["Solve Order", what, "Formula"].iter().enumerate() { + cells.push((*row, c as u32, (*h).into(), true, false)); + } + *row += 1; + for (i, (name, formula)) in list.into_iter().enumerate() { + cells.push((*row, 0, (i + 1).to_string(), false, false)); + cells.push((*row, 1, name, false, false)); + cells.push((*row, 2, formula, false, false)); + *row += 1; + } + *row += 1; + }; + let fields: Vec<(String, String)> = pt.calculated_fields.iter().map(|c| (c.name.clone(), c.formula.clone())).collect(); + let items: Vec<(String, String)> = pt.calculated_items.iter().map(|c| (c.name.clone(), c.formula.clone())).collect(); + section(&mut cells, &mut row, "Calculated Field", "Field", fields); + section(&mut cells, &mut row, "Calculated Item", "Item", items); + row += 1; + cells.push((row, 0, "Note:".into(), true, true)); + cells.push((row, 1, "When more than one formula updates a cell,".into(), false, false)); + cells.push((row + 1, 1, "the formula last in the solve order sets its value.".into(), false, false)); + cells.push((row + 3, 1, "To change the solve order of calculated items or fields,".into(), false, false)); + cells.push((row + 4, 1, "use PivotTable Analyze › Calculations › Fields, Items & Sets › Solve Order.".into(), false, false)); + let name = s.doc()?.wb.next_sheet_name(); + let d = s.doc_mut()?; + let at = edit_list(d, si, &name, &cells)?; + d.sheet_selections.insert(at.min(d.sheet_selections.len()), crate::Selection::default()); + Ok(json!({"sheet": name})) +} + +/// Inserts the List Formulas sheet after sheet `si` and writes `cells` as text; returns its index. +fn edit_list(d: &mut crate::DocState, si: usize, name: &str, cells: &[(u32, u32, String, bool, bool)]) -> Result { + super::commit(d, |cx| { + let at = (si + 1).min(cx.wb.sheets.len()); + cx.wb.sheets.insert(at, std::sync::Arc::new(Sheet::new(name.to_string()))); + for n in cx.wb.names.iter_mut() { + if let Some(sc) = n.scope.as_mut() + && *sc >= at + { + *sc += 1; + } + } + for (r, c, text, bold, italic) in cells { + let mut st = Style::default(); + st.font.bold = *bold; + st.font.italic = *italic; + let id = if *bold || *italic { cx.wb.styles.intern(st) } else { StyleId::default() }; + let sh = cx.sheet_mut(at)?; + // Formulas are listed as text, as Excel lists them. + sh.set_cell(CellRef::new(*r, *c), Cell { value: Value::text(text.as_str()), formula: None, style: id }); + } + if let Some(sh) = cx.wb.sheet_mut(at) { + for (c, w) in [(0u32, 80.0f32), (1, 140.0), (2, 200.0)] { + sh.cols.entry(c).or_default().size = Some(w); + } + } + cx.wb.active_sheet = at; + *cx.sel = crate::Selection::default(); + cx.structural = true; + Ok(at) + }) +} + +#[cfg(test)] +mod tests { + use gridcraft_core::{CellRef, Value}; + use serde_json::{Value as Json, json}; + + use super::*; + use crate::Session; + + /// Region, Product, Date, Units, Sales — 12 records over 2026 (as in `pivot`'s tests). + fn sales() -> Session { + let mut s = Session::new(); + s.new_workbook(); + let rows = json!([ + ["Region", "Product", "Date", "Units", "Sales"], + ["East", "Apples", "1/15/2026", 10, 100], + ["West", "Apples", "2/10/2026", 5, 50], + ["East", "Pears", "3/3/2026", 8, 120], + ["North", "Pears", "4/20/2026", 2, 30], + ["West", "Plums", "5/5/2026", 7, 70], + ["East", "Plums", "6/30/2026", 4, 60], + ["North", "Apples", "7/4/2026", 9, 90], + ["West", "Pears", "8/8/2026", 3, 45], + ["East", "Apples", "9/9/2026", 6, 60], + ["North", "Plums", "10/10/2026", 1, 15], + ["West", "Apples", "11/11/2026", 12, 120], + ["East", "Pears", "12/12/2026", 5, 75] + ]); + s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap(); + s.execute("selection.set", json!({"range": "A1"})).unwrap(); + s + } + + fn pt(s: &Session) -> PivotTable { + s.doc().unwrap().wb.active().unwrap().pivots.last().unwrap().clone() + } + fn val(s: &Session, c: CellRef) -> Value { + s.doc().unwrap().wb.active().unwrap().value(c) + } + /// The report's rows as (first column text, second column value). + fn report(s: &Session) -> Vec<(String, Value)> { + let r = pt(s).last_range.unwrap(); + (r.start.row..=r.end.row).map(|row| (val(s, CellRef::new(row, r.start.col)).display(), val(s, CellRef::new(row, r.start.col + 1)))).collect() + } + fn row_of(s: &Session, label: &str) -> CellRef { + let r = pt(s).last_range.unwrap(); + (r.start.row..=r.end.row).map(|row| CellRef::new(row, r.start.col)).find(|c| val(s, *c).display() == label).unwrap() + } + fn amount(s: &Session, label: &str) -> Value { + report(s).into_iter().find(|(l, _)| l == label).map(|(_, v)| v).unwrap_or(Value::Empty) + } + fn err(r: crate::Result) -> String { + r.unwrap_err().to_string() + } + + #[test] + fn group_selection_makes_a_grouped_field_and_ungroup_removes_it() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap(); + // Select East and North in the report, as Excel's Group Selection takes them. + let (east, north) = (row_of(&s, "East"), row_of(&s, "North")); + s.execute("selection.set", json!({"range": format!("{},{}", east.a1(), north.a1())})).unwrap(); + let r = s.execute("pivot.groupSelection", json!({})).unwrap(); + assert_eq!((r["field"].as_str(), r["group"].as_str()), (Some("Region2"), Some("Group1"))); + let p = pt(&s); + assert_eq!(p.rows.iter().map(|f| f.source_col.as_str()).collect::>(), ["Region2", "Region"]); + assert_eq!(p.group_fields[0].groups[0].items, ["East", "North"]); + // Group1 East North / West West / Grand Total, as in Excel. + assert_eq!(amount(&s, "Group1"), Value::Number(550.0)); + assert_eq!(amount(&s, "East"), Value::Number(415.0)); + assert_eq!(amount(&s, "Grand Total"), Value::Number(835.0)); + let labels: Vec = report(&s).into_iter().map(|(l, _)| l).collect(); + assert_eq!(labels[1..], ["Group1", "East", "North", "West", "West", "Grand Total"]); + // A filter on the grouped field selects groups. + s.execute("pivot.filter", json!({"field": "Region2", "selected": ["West"]})).unwrap(); + assert_eq!(amount(&s, "Grand Total"), Value::Number(285.0)); + s.execute("pivot.filter", json!({"field": "Region2", "selected": null})).unwrap(); + // One item can't be grouped; neither can items of a grouped field or unknown items. + assert!(err(s.execute("pivot.groupSelection", json!({"field": "Region", "items": ["West"]}))).contains(CANNOT_GROUP)); + assert!(err(s.execute("pivot.groupSelection", json!({"field": "Region2", "items": ["Group1", "West"]}))).contains(CANNOT_GROUP)); + assert!(err(s.execute("pivot.groupSelection", json!({"field": "Region", "items": ["West", "Nowhere"]}))).contains(CANNOT_GROUP)); + // Ungroup the group at the active cell: the grouped field goes with its last group. + let g = row_of(&s, "Group1"); + s.execute("selection.set", json!({"range": g.a1()})).unwrap(); + s.execute("pivot.ungroup", json!({})).unwrap(); + let p = pt(&s); + assert!(p.group_fields.is_empty()); + assert_eq!(p.rows.iter().map(|f| f.source_col.as_str()).collect::>(), ["Region"]); + assert_eq!(amount(&s, "Grand Total"), Value::Number(835.0)); + // Undo brings the group back. + s.execute("edit.undo", json!({})).unwrap(); + assert_eq!(pt(&s).group_fields.len(), 1); + } + + #[test] + fn groups_of_a_number_field_number_on_and_survive_partial_ungrouping() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Units"], "values": ["Sales"]})).unwrap(); + s.execute("pivot.groupSelection", json!({"field": "Units", "items": ["1", "2"]})).unwrap(); + let r = s.execute("pivot.groupSelection", json!({"field": "Units", "items": ["3", "4"]})).unwrap(); + assert_eq!(r["group"], "Group2"); + let p = pt(&s); + assert_eq!(p.group_fields.len(), 1, "one grouped field holds both groups"); + assert_eq!(amount(&s, "Group1"), Value::Number(45.0)); + assert_eq!(amount(&s, "Group2"), Value::Number(105.0)); + // Regrouping an item moves it out of its old group. + s.execute("pivot.groupSelection", json!({"field": "Units", "items": ["2", "3"]})).unwrap(); + let p = pt(&s); + assert_eq!(p.group_fields[0].groups.iter().map(|g| g.items.len()).collect::>(), [1, 1, 2]); + s.execute("pivot.ungroup", json!({"field": "Units2", "items": ["Group1"]})).unwrap(); + assert_eq!(pt(&s).group_fields[0].groups.len(), 2, "the field stays while it has groups"); + s.execute("pivot.ungroup", json!({"field": "Units2"})).unwrap(); + assert!(pt(&s).group_fields.is_empty()); + assert!(s.execute("pivot.ungroup", json!({"field": "Units"})).is_err(), "Units isn't grouped"); + } + + #[test] + fn date_and_time_grouping_like_the_grouping_dialog() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Date"], "values": ["Sales"]})).unwrap(); + // Months, quarters and years: one field per level, coarsest first. + s.execute("pivot.group", json!({"field": "Date", "by": ["months", "years", "quarters"]})).unwrap(); + let levels: Vec = pt(&s).rows.iter().map(|f| f.date_group).collect(); + assert_eq!(levels, [PivotDateGroup::Years, PivotDateGroup::Quarters, PivotDateGroup::Months]); + let labels: Vec = report(&s).into_iter().map(|(l, _)| l).collect(); + assert_eq!(labels[1..5], ["2026", "Qtr1", "Jan", "Feb"]); + assert_eq!(amount(&s, "Qtr1"), Value::Number(270.0)); + // Seven-day spans from the first date; the last is cut at the day after the last date. + s.execute("pivot.group", json!({"field": "Date", "by": "days", "days": 7})).unwrap(); + let labels: Vec = report(&s).into_iter().map(|(l, _)| l).collect(); + assert_eq!(labels.get(1).map(String::as_str), Some("1/15/2026 - 1/21/2026")); + assert!(labels.iter().any(|l| l == "12/10/2026 - 12/13/2026"), "{labels:?}"); + // Starting at / Ending at: dates outside go to one item each. + s.execute("pivot.group", json!({"field": "Date", "by": "months", "start": "1/1/2026", "end": "6/30/2026"})).unwrap(); + assert_eq!(amount(&s, ">6/30/2026"), Value::Number(405.0)); + assert_eq!(amount(&s, "Jun"), Value::Number(60.0)); + assert_eq!(pt(&s).rows[0].date_range.and_then(|r| r.end), Some(46203.0)); + // Number of days only with Days alone; bad bounds and levels are refused. + assert!(s.execute("pivot.group", json!({"field": "Date", "by": ["days", "months"], "days": 7})).is_err()); + assert!(s.execute("pivot.group", json!({"field": "Date", "by": "months", "start": "6/1/2026", "end": "1/1/2026"})).is_err()); + assert!(s.execute("pivot.group", json!({"field": "Date", "by": "weeks"})).is_err()); + assert!(s.execute("pivot.group", json!({"field": "Date", "by": "days", "days": 0})).is_err()); + // Ungroup leaves one plain field. + s.execute("pivot.ungroup", json!({"field": "Date"})).unwrap(); + let p = pt(&s); + assert_eq!(p.rows.len(), 1); + assert_eq!((p.rows[0].date_group, p.rows[0].date_range), (PivotDateGroup::None, None)); + // Without parameters, Group Field opens the Grouping dialog for the field at the active cell. + let first = pt(&s).last_range.unwrap().start; + s.execute("selection.set", json!({"range": CellRef::new(first.row + 1, first.col).a1()})).unwrap(); + s.ui_requests.clear(); + s.execute("pivot.group", json!({})).unwrap(); + match s.ui_requests.last() { + Some(crate::UiRequest::Dialog(name, p)) => { + assert_eq!(name, "pivotGrouping"); + assert_eq!((p["kind"].as_str(), p["field"].as_str()), (Some("date"), Some("Date"))); + } + other => panic!("expected the Grouping dialog, got {other:?}"), + } + } + + #[test] + fn time_levels_group_hours_minutes_and_seconds() { + let mut s = Session::new(); + s.new_workbook(); + let rows = json!([["When", "N"], ["1/15/2026 8:15:30", 1], ["1/16/2026 8:45:10", 2], ["1/16/2026 13:05:00", 4], ["1/17/2026 0:30:00", 8]]); + s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap(); + s.execute("insert.pivotTable", json!({"source": "Sheet1!A1:B5", "rows": ["When"], "values": ["N"]})).unwrap(); + s.execute("pivot.group", json!({"field": "When", "by": "hours"})).unwrap(); + assert_eq!(amount(&s, "8 AM"), Value::Number(3.0)); + assert_eq!(amount(&s, "1 PM"), Value::Number(4.0)); + assert_eq!(amount(&s, "12 AM"), Value::Number(8.0)); + s.execute("pivot.group", json!({"field": "When", "by": "minutes"})).unwrap(); + assert_eq!(amount(&s, ":15"), Value::Number(1.0)); + assert_eq!(amount(&s, ":05"), Value::Number(4.0)); + s.execute("pivot.group", json!({"field": "When", "by": "seconds"})).unwrap(); + assert_eq!(amount(&s, ":00"), Value::Number(12.0)); + assert_eq!(amount(&s, ":30"), Value::Number(1.0)); + s.execute("pivot.group", json!({"field": "When", "by": ["days", "hours"]})).unwrap(); + let labels: Vec = report(&s).into_iter().map(|(l, _)| l).collect(); + assert_eq!(labels[1..4], ["15-Jan", "8 AM", "16-Jan"]); + } + + #[test] + fn calculated_items_add_up_like_excel() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "columns": ["Product"], "values": ["Sales"]})).unwrap(); + let r = s.execute("pivot.calculatedItem", json!({"field": "Region", "name": "EastWest", "formula": "=East+West"})).unwrap(); + assert_eq!(r["calculatedItems"].as_array().map(Vec::len), Some(1)); + let ew = row_of(&s, "EastWest"); + let cell = |s: &Session, row: CellRef, k: u32| val(s, CellRef::new(row.row, row.col + k)); + // Apples, Pears, Plums, Grand Total — the totals include the calculated item, as in Excel. + assert_eq!([1, 2, 3, 4].map(|k| cell(&s, ew, k)), [330.0, 240.0, 130.0, 700.0].map(Value::Number)); + let gt = row_of(&s, "Grand Total"); + assert_eq!([1, 2, 3, 4].map(|k| cell(&s, gt, k)), [750.0, 510.0, 275.0, 1535.0].map(Value::Number)); + s.execute("pivot.calculatedItem", json!({"field": "Region", "name": "EastPlus", "formula": "= Region[East]*1.1"})).unwrap(); + let ep = row_of(&s, "EastPlus"); + let got: Vec = [1, 2, 3, 4].iter().map(|k| cell(&s, ep, *k).as_f64().unwrap()).collect(); + for (a, b) in got.iter().zip([176.0, 214.5, 66.0, 456.5]) { + assert!((a - b).abs() < 1e-9, "{got:?}"); + } + // Changing a formula, then deleting the item. + s.execute("pivot.calculatedItem", json!({"field": "Region", "name": "EastPlus", "formula": "=East*2"})).unwrap(); + assert_eq!(cell(&s, row_of(&s, "EastPlus"), 4), Value::Number(830.0)); + s.execute("pivot.calculatedItem", json!({"field": "Region", "name": "EastPlus", "delete": true})).unwrap(); + assert!(report(&s).iter().all(|(l, _)| l != "EastPlus")); + // A selection of the field's items without it hides it. + s.execute("pivot.filter", json!({"field": "Region", "selected": ["East", "West"]})).unwrap(); + assert!(report(&s).iter().all(|(l, _)| l != "EastWest")); + s.execute("pivot.filter", json!({"field": "Region", "selected": null})).unwrap(); + // Under Count, the item counts its items' records. + s.execute("pivot.valueSettings", json!({"field": "Sum of Sales", "func": "count"})).unwrap(); + assert_eq!(cell(&s, row_of(&s, "EastWest"), 4), Value::Number(9.0)); + } + + #[test] + fn calculated_item_restrictions() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "columns": ["Date"], "values": ["Sales"]})).unwrap(); + let add = |s: &mut Session, name: &str, formula: &str| { + s.execute("pivot.calculatedItem", json!({"field": "Region", "name": name, "formula": formula})) + }; + // A field grouped by dates anywhere in the PivotTable. + s.execute("pivot.group", json!({"field": "Date", "by": "months"})).unwrap(); + assert!(err(add(&mut s, "X", "=East")).contains("grouped")); + s.execute("pivot.group", json!({"field": "Date", "by": "none"})).unwrap(); + s.execute("pivot.removeField", json!({"field": "Date"})).unwrap(); + // Averages. + s.execute("pivot.valueSettings", json!({"field": "Sum of Sales", "func": "average"})).unwrap(); + assert!(err(add(&mut s, "X", "=East")).contains("Averages")); + s.execute("pivot.valueSettings", json!({"field": "Average of Sales", "func": "sum"})).unwrap(); + // Formula problems and names. + assert!(err(add(&mut s, "X", "=Foo+1")).contains("item name was not found")); + assert!(err(add(&mut s, "X", "=A1*2")).contains("not supported in PivotTable formulas")); + assert!(err(add(&mut s, "X", "=Product[Apples]")).contains("only use items")); + assert!(err(add(&mut s, "east", "=West")).contains("already has an item")); + assert!(s.execute("pivot.calculatedItem", json!({"field": "Units", "name": "X", "formula": "=1"})).is_err(), "Units isn't in the layout"); + add(&mut s, "EW", "=East+West").unwrap(); + // While it has calculated items: no averages, no field twice in Values, no grouping, and + // the field stays in Rows or Columns. + assert!(err(s.execute("pivot.valueSettings", json!({"field": "Sum of Sales", "func": "stdDev"}))).contains("Averages")); + assert!(err(s.execute("pivot.addField", json!({"field": "Sales", "area": "values"}))).contains("Multiple data fields")); + assert!(err(s.execute("pivot.addField", json!({"field": "Units", "area": "values", "func": "average"}))).contains("Averages")); + s.execute("pivot.addField", json!({"field": "Units", "area": "values"})).unwrap(); + s.execute("pivot.addField", json!({"field": "Date", "area": "columns"})).unwrap(); + assert!(err(s.execute("pivot.group", json!({"field": "Date", "by": "years"}))).contains("calculated items")); + assert!(err(s.execute("pivot.moveField", json!({"field": "Region", "from": "rows", "to": "filters"}))).contains("Rows or Columns")); + // Group Selection still works on a field with calculated items, as in Excel. + s.execute("pivot.groupSelection", json!({"field": "Region", "items": ["East", "North"]})).unwrap(); + // Without a name: the Insert Calculated Item dialog for the field at the active cell. + s.ui_requests.clear(); + let c = row_of(&s, "East"); + s.execute("selection.set", json!({"range": c.a1()})).unwrap(); + s.execute("pivot.calculatedItem", json!({})).unwrap(); + match s.ui_requests.last() { + Some(crate::UiRequest::Dialog(name, p)) => { + assert_eq!((name.as_str(), p["name"].as_str()), ("pivotCalculatedItem", Some("Formula1"))); + assert!(p["items"].as_array().is_some_and(|a| a.iter().any(|x| x == "EW")), "{p}"); + } + other => panic!("expected the dialog, got {other:?}"), + } + } + + #[test] + fn groups_calculated_items_and_timelines_survive_an_xlsx_round_trip() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "columns": ["Product"], "values": ["Sales"]})).unwrap(); + s.execute("pivot.groupSelection", json!({"field": "Region", "items": ["East", "North"]})).unwrap(); + s.execute("pivot.calculatedItem", json!({"field": "Product", "name": "AP", "formula": "=Apples+Pears"})).unwrap(); + s.execute("insert.timeline", json!({"field": "Date"})).unwrap(); + s.execute("timeline.select", json!({"start": "3/1/2026", "end": "8/31/2026"})).unwrap(); + let before = report(&s); + let p = pt(&s); + let b64 = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap()["base64"].as_str().unwrap().to_string(); + s.execute("file.open", json!({"name": "depth.xlsx", "base64": b64})).unwrap(); + s.execute("sheet.activate", json!({"sheet": 0})).unwrap(); + let q = pt(&s); + assert_eq!((&q.group_fields, &q.calculated_items, &q.rows), (&p.group_fields, &p.calculated_items, &p.rows)); + let tl = s.doc().unwrap().wb.sheets.iter().flat_map(|sh| sh.timelines.iter()).next().cloned().unwrap(); + assert_eq!(tl.pivots, vec![q.id]); + assert_eq!(tl.selection, Some((46082.0, 46265.0))); + // Refreshed from the file, the report is the same. + s.execute("pivot.refresh", json!({})).unwrap(); + assert_eq!(report(&s), before); + } + + #[test] + fn solve_order_decides_where_calculated_items_meet() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "columns": ["Product"], "values": ["Sales"]})).unwrap(); + s.execute("pivot.calculatedItem", json!({"field": "Region", "name": "EW", "formula": "=East+West"})).unwrap(); + s.execute("pivot.calculatedItem", json!({"field": "Product", "name": "AP", "formula": "=Apples+1"})).unwrap(); + let meet = |s: &Session| { + let row = row_of(s, "EW"); + let r = pt(s).last_range.unwrap(); + let col = (r.start.col..=r.end.col).find(|c| val(s, CellRef::new(r.start.row + 1, *c)).display() == "AP").unwrap(); + val(s, CellRef::new(row.row, col)) + }; + // AP last: Apples of EW (330) + 1. + assert_eq!(meet(&s), Value::Number(331.0)); + s.execute("pivot.solveOrder", json!({"order": ["AP", "EW"]})).unwrap(); + // EW last: AP of East (161) + AP of West (171). + assert_eq!(meet(&s), Value::Number(332.0)); + s.execute("pivot.solveOrder", json!({"item": "AP", "move": "down"})).unwrap(); + assert_eq!(pt(&s).calculated_items.iter().map(|c| c.name.as_str()).collect::>(), ["EW", "AP"]); + assert!(s.execute("pivot.solveOrder", json!({"order": ["AP"]})).is_err()); + // List Formulas: a sheet after the PivotTable's with both sections. + s.execute("pivot.calculatedField", json!({"name": "Bonus", "formula": "=Sales*0.1", "values": false})).unwrap(); + let pivot_sheet = s.doc().unwrap().wb.active_sheet; + let r = s.execute("pivot.listFormulas", json!({})).unwrap(); + let d = s.doc().unwrap(); + assert_eq!(d.wb.active_sheet, pivot_sheet + 1); + let sh = d.wb.active().unwrap(); + assert_eq!(sh.name, r["sheet"].as_str().unwrap()); + let t = |a: &str| sh.value(CellRef::parse(a).unwrap()).display(); + assert_eq!([t("A1"), t("A2"), t("B2"), t("C2")], ["Calculated Field", "Solve Order", "Field", "Formula"]); + assert_eq!([t("A3"), t("B3"), t("C3")], ["1", "Bonus", "=Sales*0.1"]); + assert_eq!( + [t("A5"), t("B6"), t("A7"), t("B7"), t("C7"), t("B8"), t("C8")], + ["Calculated Item", "Item", "1", "EW", "=East+West", "AP", "=Apples+1"] + ); + assert_eq!(t("A11"), "Note:"); + assert!(d.wb.styles.get(sh.style_id(CellRef::parse("A1").unwrap())).font.italic); + } +} diff --git a/crates/engine/src/cmd/timeline.rs b/crates/engine/src/cmd/timeline.rs new file mode 100644 index 000000000..83fe91e47 --- /dev/null +++ b/crates/engine/src/cmd/timeline.rs @@ -0,0 +1,760 @@ +//! Timelines (Insert › Filters › Timeline, PivotTable Analyze › Filter › Insert Timeline and the +//! Timeline contextual tab): floating filters of a PivotTable's date field by one contiguous +//! period at a time level (years, quarters, months or days). +//! +//! A timeline keeps its selection as a first and last day; the PivotTables it is connected to +//! (Report Connections) keep only the records whose date falls in it (see `pivot::compute`). As +//! in Excel, a selection is made of whole periods of the level it was made at, a new timeline +//! shows months and selects all periods, and each PivotTable has one timeline per date field. + +use gridcraft_core::date::{datetime_from_serial, serial_from_ymd}; +use gridcraft_core::{CellRef, DateSystem, Value}; +use gridcraft_model::*; +use serde_json::{Value as Json, json}; + +use super::pivot::{find_pivot, other}; +use super::*; +use crate::pivot as pv; + +pub fn specs() -> Vec { + vec![ + cmd!( + "insert.timeline", + "Timeline", + ["Insert", "Filters"], + None, + "{pivot?, field?: date field | fields?: [..], at?: \"H2\", width?, height?, replace?} adds a timeline filtering the PivotTable by the date field; without `field` opens Insert Timelines (the PivotTable's date fields) → {timelines: [{id, name}]}", + has_doc, + insert_timeline + ), + cmd!( + "pivot.insertTimeline", + "Insert Timeline", + ["PivotTable Analyze", "Filter"], + None, + "{pivot?, field?: date field | fields?: [..], at?, width?, height?, replace?} (same as insert.timeline)", + has_doc, + insert_timeline + ), + cmd!( + "timeline.select", + "Select Timeline Period", + [], + None, + "{timeline?: id or name (default: the last on the active sheet), start: date, end?: date (default start), level?: years|quarters|months|days (default the timeline's), replace?} selects whole periods from start's to end's and filters the connected PivotTables → {selection, label}", + has_doc, + select + ), + cmd!("timeline.clear", "Clear Filter", [], Some("Alt+C"), "{timeline?, replace?} selects all periods (no filter)", has_doc, clear), + cmd!( + "timeline.level", + "Time Level", + [], + None, + "{timeline?, level: years|quarters|months|days} sets the level the timeline shows", + has_doc, + level + ), + cmd!("timeline.scroll", "Scroll Timeline", [], None, "{timeline?, to: date | null} sets the first date the timeline shows", has_doc, scroll), + cmd!("timeline.caption", "Timeline Caption", ["Timeline", "Timeline"], None, "{timeline?, caption}", has_doc, caption), + cmd!( + "timeline.reportConnections", + "Report Connections", + ["Timeline", "Timeline"], + None, + "{timeline?, pivots?: [PivotTable names on the same source], replace?}; without `pivots` opens the Report Connections dialog", + has_doc, + report_connections + ), + cmd!( + "timeline.style", + "Timeline Styles", + ["Timeline", "Timeline Styles"], + None, + "{timeline?, style: TimeSlicerStyleLight1…6 | TimeSlicerStyleDark1…6}", + has_doc, + style + ), + cmd!("timeline.size", "Size", ["Timeline", "Size"], None, "{timeline?, width?, height?} in points", has_doc, size), + cmd!("timeline.showHeader", "Header", ["Timeline", "Show"], None, "{timeline?, on?: bool (default: toggle)}", has_doc, |s, p| show( + s, + p, + Show::Header + )), + cmd!( + "timeline.showSelectionLabel", + "Selection Label", + ["Timeline", "Show"], + None, + "{timeline?, on?: bool (default: toggle)}", + has_doc, + |s, p| show(s, p, Show::SelectionLabel) + ), + cmd!("timeline.showScrollbar", "Scrollbar", ["Timeline", "Show"], None, "{timeline?, on?: bool (default: toggle)}", has_doc, |s, p| show( + s, + p, + Show::Scrollbar + )), + cmd!("timeline.showTimeLevel", "Time Level", ["Timeline", "Show"], None, "{timeline?, on?: bool (default: toggle)}", has_doc, |s, p| show( + s, + p, + Show::TimeLevel + )), + cmd!("timeline.delete", "Delete Timeline", [], None, "{timeline?, replace?} removes the timeline and its filter", has_doc, delete), + cmd!( + query "timeline.periods", + "Timeline Periods", + [], + None, + "{timeline?} → {id, level, selection, label, periods: [{start, end, label, group, selected}]}", + has_doc, + periods_query + ), + ] +} + +/// The timeline styles Excel offers. +pub const STYLES: [&str; 12] = [ + "TimeSlicerStyleLight1", + "TimeSlicerStyleLight2", + "TimeSlicerStyleLight3", + "TimeSlicerStyleLight4", + "TimeSlicerStyleLight5", + "TimeSlicerStyleLight6", + "TimeSlicerStyleDark1", + "TimeSlicerStyleDark2", + "TimeSlicerStyleDark3", + "TimeSlicerStyleDark4", + "TimeSlicerStyleDark5", + "TimeSlicerStyleDark6", +]; + +/// Most periods a timeline lists (days over decades are capped). +const MAX_PERIODS: usize = 20_000; + +const MONTHS: [&str; 12] = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]; + +pub fn level_name(l: TimelineLevel) -> &'static str { + match l { + TimelineLevel::Years => "years", + TimelineLevel::Quarters => "quarters", + TimelineLevel::Months => "months", + TimelineLevel::Days => "days", + } +} + +fn level_param(cmd: &str, v: &str) -> Result { + Ok(match v.trim().to_ascii_lowercase().as_str() { + "years" | "year" => TimelineLevel::Years, + "quarters" | "quarter" => TimelineLevel::Quarters, + "months" | "month" => TimelineLevel::Months, + "days" | "day" => TimelineLevel::Days, + other => return Err(bad(cmd, format!("unknown level `{other}` (years|quarters|months|days)"))), + }) +} + +// ---------------------------------------------------------------- periods + +/// A day serial as (year, month, day). +fn ymd(sys: DateSystem, day: f64) -> Option<(i32, u32, u32)> { + let d = datetime_from_serial(sys, day)?; + Some((d.year, d.month, d.day)) +} + +fn serial(sys: DateSystem, y: i32, m: i64, d: i64) -> Option { + serial_from_ymd(sys, i64::from(y), m, d) +} + +/// The first and last day of the period at `level` holding `day`. +pub fn period_of(sys: DateSystem, level: TimelineLevel, day: f64) -> Option<(f64, f64)> { + let day = day.floor(); + let (y, m, _) = ymd(sys, day)?; + let (first, months) = match level { + TimelineLevel::Days => return Some((day, day)), + TimelineLevel::Years => (1, 12), + TimelineLevel::Quarters => ((m - 1) / 3 * 3 + 1, 3), + TimelineLevel::Months => (m, 1), + }; + let a = serial(sys, y, i64::from(first), 1)?; + let b = serial(sys, y, i64::from(first) + months, 0)?; + Some((a, b)) +} + +/// The dates of the timeline's field in its first connected PivotTable's source: (first, last day). +pub fn data_bounds(wb: &Workbook, tl: &Timeline) -> Option<(f64, f64)> { + let (si, pi) = connected(wb, tl).into_iter().next()?; + let pt = wb.sheet(si)?.pivots.get(pi)?; + let src = pv::read_source(wb, &pt.source, si).ok()?; + let c = src.col(&tl.field)?; + let (lo, hi) = src + .rows + .iter() + .filter_map(|r| match r.get(c) { + Some(Value::Number(n)) => Some(n.floor()), + _ => None, + }) + .fold((f64::INFINITY, f64::NEG_INFINITY), |(lo, hi), n| (lo.min(n), hi.max(n))); + (lo.is_finite() && hi.is_finite()).then_some((lo, hi)) +} + +/// One period a timeline shows. +#[derive(Clone, Debug, PartialEq)] +pub struct Period { + pub start: f64, + pub end: f64, + /// The tile caption (`2026`, `Q1`, `JAN`, `15`). + pub label: String, + /// The caption of the row above (`2026`, `JAN 2026`; empty for years). + pub group: String, +} + +/// The periods of a timeline at its level, over whole years around its data. +pub fn periods(wb: &Workbook, tl: &Timeline) -> Vec { + let sys = wb.date_system; + let Some((lo, hi)) = data_bounds(wb, tl) else { return vec![] }; + let (Some((y0, _, _)), Some((y1, _, _))) = (ymd(sys, lo), ymd(sys, hi)) else { return vec![] }; + let mut out = Vec::new(); + let Some(mut day) = serial(sys, y0, 1, 1) else { return vec![] }; + let Some(last) = serial(sys, y1, 12, 31) else { return vec![] }; + while day <= last && out.len() < MAX_PERIODS { + let Some((a, b)) = period_of(sys, tl.level, day) else { break }; + let Some((y, m, d)) = ymd(sys, a) else { break }; + let mon = MONTHS.get(m.saturating_sub(1) as usize).copied().unwrap_or("Jan").to_uppercase(); + let (label, group) = match tl.level { + TimelineLevel::Years => (y.to_string(), String::new()), + TimelineLevel::Quarters => (format!("Q{}", (m - 1) / 3 + 1), y.to_string()), + TimelineLevel::Months => (mon, y.to_string()), + TimelineLevel::Days => (d.to_string(), format!("{mon} {y}")), + }; + out.push(Period { start: a, end: b, label, group }); + day = b + 1.0; + } + out +} + +/// The selection as the timeline's label shows it: `All Periods`, `2026`, `Q1 - Q2 2026`, +/// `Mar - May 2026`, `Nov 2025 - Feb 2026`, `3/1/2026 - 3/7/2026`. +pub fn selection_label(wb: &Workbook, tl: &Timeline) -> String { + let sys = wb.date_system; + let Some((a, b)) = tl.selection else { return "All Periods".into() }; + let (Some((ya, ma, da)), Some((yb, mb, db))) = (ymd(sys, a), ymd(sys, b)) else { return String::new() }; + let mon = |m: u32| MONTHS.get(m.saturating_sub(1) as usize).copied().unwrap_or("Jan"); + let q = |m: u32| (m - 1) / 3 + 1; + let (one, from, to) = match tl.selection_level { + TimelineLevel::Years => return if ya == yb { ya.to_string() } else { format!("{ya} - {yb}") }, + TimelineLevel::Quarters => (q(ma) == q(mb) && ya == yb, format!("Q{}", q(ma)), format!("Q{}", q(mb))), + TimelineLevel::Months => (ma == mb && ya == yb, mon(ma).to_string(), mon(mb).to_string()), + TimelineLevel::Days => { + return if a == b { format!("{ma}/{da}/{ya}") } else { format!("{ma}/{da}/{ya} - {mb}/{db}/{yb}") }; + } + }; + if one { + format!("{from} {ya}") + } else if ya == yb { + format!("{from} - {to} {ya}") + } else { + format!("{from} {ya} - {to} {yb}") + } +} + +// ---------------------------------------------------------------- lookup + +/// (sheet, index) of each PivotTable a timeline is connected to. +pub fn connected(wb: &Workbook, tl: &Timeline) -> Vec<(usize, usize)> { + let mut out = Vec::new(); + for (si, sh) in wb.sheets.iter().enumerate() { + for (pi, pt) in sh.pivots.iter().enumerate() { + if tl.pivots.contains(&pt.id) { + out.push((si, pi)); + } + } + } + out +} + +/// (sheet, index) of the timeline a call names (`timeline`: id or name), else the last one on +/// the active sheet. +fn find_timeline(s: &Session, p: &Json) -> Result<(usize, usize)> { + let d = s.doc()?; + match p.get("timeline").filter(|v| !v.is_null()) { + Some(v) => { + for (si, sh) in d.wb.sheets.iter().enumerate() { + let hit = sh.timelines.iter().position(|t| match v { + Json::Number(n) => n.as_u64() == Some(u64::from(t.id)), + Json::String(n) => t.name.eq_ignore_ascii_case(n.trim()), + _ => false, + }); + if let Some(i) = hit { + return Ok((si, i)); + } + } + Err(bad("timeline", format!("no timeline `{}`", v.as_str().map(str::to_string).unwrap_or_else(|| v.to_string())))) + } + None => { + let si = d.wb.active_sheet; + let n = d.wb.active().map_or(0, |sh| sh.timelines.len()); + n.checked_sub(1).map(|i| (si, i)).ok_or_else(|| EngineError::Disabled("timeline".into(), "there's no timeline on this sheet".into())) + } + } +} + +/// Applies `f` to a timeline and refreshes the PivotTables it filters (before and after), as one +/// undo step. +fn update_timeline(s: &mut Session, p: &Json, refresh: bool, f: impl FnOnce(&mut Timeline, &Workbook) -> Result) -> Result { + let (si, ti) = find_timeline(s, p)?; + let replace = bool_param(p, "replace").unwrap_or(false); + edit(s, |cx| { + let mut tl = cx.wb.sheet(si).and_then(|sh| sh.timelines.get(ti)).cloned().ok_or_else(|| other("There's no such timeline."))?; + let before = connected(&cx.wb, &tl); + let out = f(&mut tl, &cx.wb)?; + if let Some(slot) = cx.sheet_mut(si)?.timelines.get_mut(ti) { + *slot = tl.clone(); + } + if refresh { + let mut targets = before; + for t in connected(&cx.wb, &tl) { + if !targets.contains(&t) { + targets.push(t); + } + } + for (psi, ppi) in targets { + pv::refresh(&mut cx.wb, psi, ppi, replace).map_err(other)?; + } + cx.structural = true; + } + Ok(out) + }) +} + +// ---------------------------------------------------------------- commands + +/// Date fields of a PivotTable's source, in source order. +fn date_fields(src: &pv::Source) -> Vec { + src.headers.iter().enumerate().filter(|(c, _)| src.dates.get(*c).copied().unwrap_or(false)).map(|(_, h)| h.clone()).collect() +} + +fn insert_timeline(s: &mut Session, p: &Json) -> Result { + const C: &str = "insert.timeline"; + let (si, pi) = find_pivot(s, p)?; + let (pt, src) = { + let d = s.doc()?; + let pt = d.wb.sheet(si).and_then(|sh| sh.pivots.get(pi)).cloned().ok_or_else(|| other("There's no such PivotTable."))?; + let src = pv::read_source(&d.wb, &pt.source, si).map_err(other)?; + (pt, src) + }; + let mut fields: Vec = match (str_param(p, "field"), p.get("fields").and_then(Json::as_array)) { + (Some(f), _) => vec![f.trim().to_string()], + (None, Some(a)) => a.iter().take(64).filter_map(|x| x.as_str().map(|t| t.trim().to_string())).collect(), + (None, None) => { + // Menu-style call: Insert Timelines lists the PivotTable's date fields. + let dates = date_fields(&src); + if dates.is_empty() { + return Err(other("The PivotTable has no fields with dates. A timeline needs a field formatted as dates.")); + } + s.ui_requests.push(crate::UiRequest::Dialog("insertTimeline".into(), json!({"pivot": pt.name, "fields": dates}))); + return ok(); + } + }; + fields.dedup(); + if fields.is_empty() { + return Err(bad(C, "missing `field`")); + } + let sheet = s.doc()?.wb.active_sheet; + let at = cell_param(s, p, "at")?; + let width = f64_param(p, "width").unwrap_or(262.5).clamp(60.0, 10_000.0) as f32; + let height = f64_param(p, "height").unwrap_or(108.0).clamp(40.0, 10_000.0) as f32; + edit(s, |cx| { + let mut made = Vec::new(); + for (k, field) in fields.iter().enumerate() { + let c = src.col(field).ok_or_else(|| bad(C, format!("no field `{field}` in the source data")))?; + if !src.dates.get(c).copied().unwrap_or(false) { + return Err(other(format!("\"{field}\" doesn't contain dates. A timeline can only filter a date field."))); + } + let field = src.headers.get(c).cloned().unwrap_or_default(); + let taken = + cx.wb.sheets.iter().flat_map(|sh| sh.timelines.iter()).any(|t| t.field.eq_ignore_ascii_case(&field) && t.pivots.contains(&pt.id)); + if taken { + return Err(other(format!("A timeline already filters this PivotTable by \"{field}\"."))); + } + let names: Vec = cx.wb.sheets.iter().flat_map(|sh| sh.timelines.iter().map(|t| t.name.to_lowercase())).collect(); + let name = std::iter::once(field.clone()) + .chain((1..10_000).map(|n| format!("{field} {n}"))) + .find(|n| !names.contains(&n.to_lowercase())) + .unwrap_or_else(|| field.clone()); + // Beside the report, like Excel: right of it, one below another. + let end = pt.last_range.map_or(pt.anchor, |r| r.end); + let start_row = pt.last_range.map_or(pt.anchor.row, |r| r.start.row); + let cell = at.unwrap_or_else(|| { + CellRef::new( + start_row.saturating_add(8 * k as u32).min(gridcraft_core::MAX_ROWS - 1), + end.col.saturating_add(2).min(gridcraft_core::MAX_COLS - 1), + ) + }); + let id = cx.wb.next_object_id(); + let tl = Timeline { + id, + name: name.clone(), + caption: field.clone(), + field, + pivots: vec![pt.id], + anchor: Anchor { cell, dx: 4.0, dy: 4.0, width, height, mode: AnchorMode::MoveOnly }, + ..Timeline::default() + }; + cx.sheet_mut(sheet)?.timelines.push(tl); + made.push(json!({"id": id, "name": name})); + } + Ok(json!({"timelines": made})) + }) +} + +/// A date parameter: a serial or date text. +fn date_param(cmd: &str, p: &Json, key: &str, sys: DateSystem) -> Result> { + match p.get(key) { + None | Some(Json::Null) => Ok(None), + Some(Json::String(t)) => match gridcraft_core::parse::parse_input(t.trim(), sys).value { + Value::Number(n) if n.is_finite() && datetime_from_serial(sys, n).is_some() => Ok(Some(n)), + _ => Err(bad(cmd, format!("`{key}` must be a date"))), + }, + Some(v) => { + v.as_f64().filter(|n| datetime_from_serial(sys, *n).is_some()).map(Some).ok_or_else(|| bad(cmd, format!("`{key}` must be a date"))) + } + } +} + +fn select(s: &mut Session, p: &Json) -> Result { + const C: &str = "timeline.select"; + let sys = s.doc()?.wb.date_system; + let start = date_param(C, p, "start", sys)?.ok_or_else(|| bad(C, "missing `start`"))?; + let end = date_param(C, p, "end", sys)?.unwrap_or(start); + let lvl = match str_param(p, "level") { + Some(l) => Some(level_param(C, l)?), + None => None, + }; + update_timeline(s, p, true, |tl, wb| { + let level = lvl.unwrap_or(tl.level); + let (a, b) = if start <= end { (start, end) } else { (end, start) }; + let first = period_of(sys, level, a).ok_or_else(|| bad(C, "`start` must be a date"))?.0; + let last = period_of(sys, level, b).ok_or_else(|| bad(C, "`end` must be a date"))?.1; + tl.selection = Some((first, last)); + tl.selection_level = level; + if let Some(l) = lvl { + tl.level = l; + } + Ok(json!({"selection": [first, last], "label": selection_label(wb, tl)})) + }) +} + +fn clear(s: &mut Session, p: &Json) -> Result { + update_timeline(s, p, true, |tl, _| { + tl.selection = None; + Ok(json!({})) + }) +} + +fn level(s: &mut Session, p: &Json) -> Result { + const C: &str = "timeline.level"; + let l = level_param(C, str_param(p, "level").ok_or_else(|| bad(C, "missing `level`"))?)?; + update_timeline(s, p, false, |tl, _| { + tl.level = l; + Ok(json!({"level": level_name(l)})) + }) +} + +fn scroll(s: &mut Session, p: &Json) -> Result { + const C: &str = "timeline.scroll"; + let sys = s.doc()?.wb.date_system; + let to = date_param(C, p, "to", sys)?; + update_timeline(s, p, false, |tl, _| { + tl.scroll = to.map(f64::floor); + Ok(json!({})) + }) +} + +fn caption(s: &mut Session, p: &Json) -> Result { + const C: &str = "timeline.caption"; + let text = str_param(p, "caption").ok_or_else(|| bad(C, "missing `caption`"))?.to_string(); + if text.chars().count() > 255 { + return Err(bad(C, "`caption` is at most 255 characters")); + } + update_timeline(s, p, false, |tl, _| { + tl.caption = text; + Ok(json!({})) + }) +} + +fn style(s: &mut Session, p: &Json) -> Result { + const C: &str = "timeline.style"; + let raw = str_param(p, "style").ok_or_else(|| bad(C, "missing `style`"))?; + let name = STYLES.iter().find(|n| n.eq_ignore_ascii_case(raw.trim())).ok_or_else(|| bad(C, format!("unknown timeline style `{raw}`")))?; + update_timeline(s, p, false, |tl, _| { + tl.style = (*name).to_string(); + Ok(json!({})) + }) +} + +fn size(s: &mut Session, p: &Json) -> Result { + let (w, h) = (f64_param(p, "width"), f64_param(p, "height")); + update_timeline(s, p, false, |tl, _| { + if let Some(w) = w.filter(|w| w.is_finite()) { + tl.anchor.width = w.clamp(10.0, 10_000.0) as f32; + } + if let Some(h) = h.filter(|h| h.is_finite()) { + tl.anchor.height = h.clamp(10.0, 10_000.0) as f32; + } + Ok(json!({"width": tl.anchor.width, "height": tl.anchor.height})) + }) +} + +#[derive(Clone, Copy)] +enum Show { + Header, + SelectionLabel, + Scrollbar, + TimeLevel, +} + +fn show(s: &mut Session, p: &Json, what: Show) -> Result { + let on = bool_param(p, "on"); + update_timeline(s, p, false, |tl, _| { + let flag = match what { + Show::Header => &mut tl.show_header, + Show::SelectionLabel => &mut tl.show_selection_label, + Show::Scrollbar => &mut tl.show_scrollbar, + Show::TimeLevel => &mut tl.show_time_level, + }; + *flag = on.unwrap_or(!*flag); + Ok(json!({"on": *flag})) + }) +} + +fn report_connections(s: &mut Session, p: &Json) -> Result { + const C: &str = "timeline.reportConnections"; + let (si, ti) = find_timeline(s, p)?; + // PivotTables on the timeline's source that have its field: (id, name, sheet name). + let candidates: Vec<(u32, String, String)> = { + let d = s.doc()?; + let tl = d.wb.sheet(si).and_then(|sh| sh.timelines.get(ti)).ok_or_else(|| other("There's no such timeline."))?; + let home = connected(&d.wb, tl).first().and_then(|(psi, ppi)| { + let pt = d.wb.sheet(*psi)?.pivots.get(*ppi)?; + pv::resolve_source(&d.wb, &pt.source, *psi).ok() + }); + let mut v = Vec::new(); + for (psi, sh) in d.wb.sheets.iter().enumerate() { + for pt in &sh.pivots { + let Ok(src) = pv::read_source(&d.wb, &pt.source, psi) else { continue }; + let same = home.is_none_or(|h| pv::resolve_source(&d.wb, &pt.source, psi).ok() == Some(h)); + if same && src.col(&tl.field).is_some() { + v.push((pt.id, pt.name.clone(), sh.name.clone())); + } + } + } + v + }; + let Some(list) = p.get("pivots").and_then(Json::as_array) else { + let d = s.doc()?; + let tl = d.wb.sheet(si).and_then(|sh| sh.timelines.get(ti)).ok_or_else(|| other("There's no such timeline."))?; + let rows: Vec = candidates.iter().map(|(id, n, sh)| json!({"name": n, "sheet": sh, "connected": tl.pivots.contains(id)})).collect(); + let name = tl.name.clone(); + s.ui_requests.push(crate::UiRequest::Dialog("timelineConnections".into(), json!({"timeline": name, "pivots": rows}))); + return ok(); + }; + let mut ids = Vec::new(); + for v in list.iter().take(1000) { + let n = v.as_str().ok_or_else(|| bad(C, "`pivots` is a list of PivotTable names"))?; + let (id, ..) = candidates + .iter() + .find(|(_, name, _)| name.eq_ignore_ascii_case(n.trim())) + .ok_or_else(|| bad(C, format!("`{n}` isn't a PivotTable on the same data with the field the timeline filters")))?; + if !ids.contains(id) { + ids.push(*id); + } + } + update_timeline(s, p, true, |tl, _| { + tl.pivots = ids; + Ok(json!({"pivots": tl.pivots.len()})) + }) +} + +fn delete(s: &mut Session, p: &Json) -> Result { + let (si, ti) = find_timeline(s, p)?; + let replace = bool_param(p, "replace").unwrap_or(false); + edit(s, |cx| { + let tl = cx.wb.sheet(si).and_then(|sh| sh.timelines.get(ti)).cloned().ok_or_else(|| other("There's no such timeline."))?; + let targets = connected(&cx.wb, &tl); + let sh = cx.sheet_mut(si)?; + if ti < sh.timelines.len() { + sh.timelines.remove(ti); + } + if tl.selection.is_some() { + for (psi, ppi) in targets { + pv::refresh(&mut cx.wb, psi, ppi, replace).map_err(other)?; + } + } + cx.structural = true; + Ok(json!({})) + }) +} + +fn periods_query(s: &mut Session, p: &Json) -> Result { + let (si, ti) = find_timeline(s, p)?; + let d = s.doc()?; + let tl = d.wb.sheet(si).and_then(|sh| sh.timelines.get(ti)).ok_or_else(|| other("There's no such timeline."))?; + let list: Vec = periods(&d.wb, tl) + .into_iter() + .map(|x| { + let on = tl.selection.is_some_and(|(a, b)| x.start <= b && x.end >= a); + json!({"start": x.start, "end": x.end, "label": x.label, "group": x.group, "selected": on}) + }) + .collect(); + Ok(json!({ + "id": tl.id, + "name": tl.name, + "field": tl.field, + "level": level_name(tl.level), + "selection": tl.selection.map(|(a, b)| json!([a, b])), + "label": selection_label(&d.wb, tl), + "periods": list, + })) +} + +#[cfg(test)] +mod tests { + use gridcraft_core::{CellRef, Value}; + use serde_json::json; + + use super::*; + use crate::Session; + + fn sales() -> Session { + let mut s = Session::new(); + s.new_workbook(); + let rows = json!([ + ["Region", "Product", "Date", "Units", "Sales"], + ["East", "Apples", "1/15/2026", 10, 100], + ["West", "Apples", "2/10/2026", 5, 50], + ["East", "Pears", "3/3/2026", 8, 120], + ["North", "Pears", "4/20/2026", 2, 30], + ["West", "Plums", "5/5/2026", 7, 70], + ["East", "Plums", "6/30/2026", 4, 60], + ["North", "Apples", "7/4/2026", 9, 90], + ["West", "Pears", "8/8/2026", 3, 45], + ["East", "Apples", "9/9/2026", 6, 60], + ["North", "Plums", "10/10/2026", 1, 15], + ["West", "Apples", "11/11/2026", 12, 120], + ["East", "Pears", "12/12/2026", 5, 75] + ]); + s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap(); + s.execute("selection.set", json!({"range": "A1"})).unwrap(); + s + } + + /// The Grand Total of the PivotTable named `name`. + fn total(s: &Session, name: &str) -> Value { + let wb = &s.doc().unwrap().wb; + for sh in &wb.sheets { + if let Some(pt) = sh.pivots.iter().find(|p| p.name == name) { + let r = pt.last_range.unwrap(); + let row = (r.start.row..=r.end.row).find(|row| sh.value(CellRef::new(*row, r.start.col)).display() == "Grand Total").unwrap(); + return sh.value(CellRef::new(row, r.start.col + 1)); + } + } + Value::Empty + } + fn timeline(s: &Session) -> Timeline { + s.doc().unwrap().wb.sheets.iter().flat_map(|sh| sh.timelines.iter()).last().unwrap().clone() + } + + #[test] + fn a_timeline_filters_its_pivot_tables_by_whole_periods() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap(); + // Without a field: the Insert Timelines dialog lists the date fields. + s.execute("insert.timeline", json!({})).unwrap(); + match s.ui_requests.last() { + Some(crate::UiRequest::Dialog(name, p)) => assert_eq!((name.as_str(), p["fields"].clone()), ("insertTimeline", json!(["Date"]))), + other => panic!("expected Insert Timelines, got {other:?}"), + } + let r = s.execute("pivot.insertTimeline", json!({"field": "Date"})).unwrap(); + assert_eq!(r["timelines"][0]["name"], "Date"); + let tl = timeline(&s); + assert_eq!((tl.caption.as_str(), tl.level, tl.selection, tl.style.as_str()), ("Date", TimelineLevel::Months, None, "TimeSlicerStyleLight1")); + assert_eq!((tl.anchor.width, tl.anchor.height), (262.5, 108.0)); + // March to May, from dates inside the months: East 120, North 30, West 70. + let r = s.execute("timeline.select", json!({"start": "3/15/2026", "end": "5/2/2026"})).unwrap(); + assert_eq!(r["label"], "Mar - May 2026"); + assert_eq!(timeline(&s).selection, Some((46082.0, 46173.0))); + assert_eq!(total(&s, "PivotTable1"), Value::Number(220.0)); + // Quarters: Q1 is January to March. + s.execute("timeline.select", json!({"start": "2/2/2026", "level": "quarters"})).unwrap(); + assert_eq!(total(&s, "PivotTable1"), Value::Number(270.0)); + assert_eq!(selection_label(&s.doc().unwrap().wb, &timeline(&s)), "Q1 2026"); + assert_eq!(timeline(&s).level, TimelineLevel::Quarters); + // The level the timeline shows can change without changing the selection. + s.execute("timeline.level", json!({"level": "days"})).unwrap(); + assert_eq!(total(&s, "PivotTable1"), Value::Number(270.0)); + let q = s.execute("timeline.periods", json!({})).unwrap(); + assert_eq!(q["periods"].as_array().map(Vec::len), Some(365)); + assert_eq!(q["periods"].as_array().map(|p| p.iter().filter(|x| x["selected"] == true).count()), Some(90)); + s.execute("timeline.level", json!({"level": "months"})).unwrap(); + let q = s.execute("timeline.periods", json!({})).unwrap(); + assert_eq!(q["periods"][0]["label"], "JAN"); + assert_eq!(q["periods"][0]["group"], "2026"); + // Clear Filter: all periods again; undo brings the selection back. + s.execute("timeline.clear", json!({})).unwrap(); + assert_eq!(total(&s, "PivotTable1"), Value::Number(835.0)); + s.execute("edit.undo", json!({})).unwrap(); + assert_eq!(total(&s, "PivotTable1"), Value::Number(270.0)); + // Refusals: a field without dates, a second timeline on the same field, bad levels. + assert!(s.execute("insert.timeline", json!({"field": "Region"})).unwrap_err().to_string().contains("dates")); + assert!(s.execute("insert.timeline", json!({"field": "Date"})).unwrap_err().to_string().contains("already")); + assert!(s.execute("timeline.level", json!({"level": "weeks"})).is_err()); + assert!(s.execute("timeline.select", json!({"start": "soon"})).is_err()); + assert!(s.execute("timeline.style", json!({"style": "SlicerStyleLight1"})).is_err()); + } + + #[test] + fn report_connections_and_timeline_settings() { + let mut s = sales(); + s.execute("insert.pivotTable", json!({"rows": ["Region"], "values": ["Sales"]})).unwrap(); + s.execute("insert.pivotTable", json!({"source": "Sheet1!$A$1:$E$13", "rows": ["Product"], "values": ["Units"]})).unwrap(); + s.execute("sheet.activate", json!({"sheet": "Sheet3"})).unwrap(); + s.execute("insert.timeline", json!({"field": "Date", "pivot": "PivotTable1"})).unwrap(); + s.execute("timeline.select", json!({"start": "1/1/2026", "end": "1/31/2026"})).unwrap(); + assert_eq!(total(&s, "PivotTable1"), Value::Number(100.0)); + assert_eq!(total(&s, "PivotTable2"), Value::Number(TOTAL_UNITS)); + // Without `pivots`: the Report Connections dialog lists both. + s.execute("timeline.reportConnections", json!({})).unwrap(); + match s.ui_requests.last() { + Some(crate::UiRequest::Dialog(name, p)) => { + assert_eq!(name, "timelineConnections"); + assert_eq!(p["pivots"].as_array().map(Vec::len), Some(2)); + } + other => panic!("expected Report Connections, got {other:?}"), + } + s.execute("timeline.reportConnections", json!({"pivots": ["PivotTable1", "PivotTable2"]})).unwrap(); + assert_eq!(total(&s, "PivotTable2"), Value::Number(10.0)); + assert!(s.execute("timeline.reportConnections", json!({"pivots": ["Nope"]})).is_err()); + // Disconnecting refreshes the PivotTable it no longer filters. + s.execute("timeline.reportConnections", json!({"pivots": ["PivotTable2"]})).unwrap(); + assert_eq!(total(&s, "PivotTable1"), Value::Number(835.0)); + // Caption, style, size and the Show checkboxes. + s.execute("timeline.caption", json!({"caption": "Order date"})).unwrap(); + s.execute("timeline.style", json!({"style": "timeslicerstyledark2"})).unwrap(); + s.execute("timeline.size", json!({"width": 300, "height": 1e9})).unwrap(); + s.execute("timeline.showHeader", json!({"on": false})).unwrap(); + s.execute("timeline.showScrollbar", json!({})).unwrap(); + let tl = timeline(&s); + assert_eq!((tl.caption.as_str(), tl.style.as_str()), ("Order date", "TimeSlicerStyleDark2")); + assert_eq!((tl.anchor.width, tl.anchor.height), (300.0, 10_000.0)); + assert!(!tl.show_header && !tl.show_scrollbar && tl.show_selection_label && tl.show_time_level); + // Deleting a PivotTable disconnects it; deleting the timeline clears its filter. + s.execute("timeline.reportConnections", json!({"pivots": ["PivotTable1", "PivotTable2"]})).unwrap(); + s.execute("pivot.delete", json!({"pivot": "PivotTable2"})).unwrap(); + assert_eq!(timeline(&s).pivots.len(), 1); + s.execute("timeline.delete", json!({})).unwrap(); + assert_eq!(total(&s, "PivotTable1"), Value::Number(835.0)); + assert!(s.doc().unwrap().wb.sheets.iter().all(|sh| sh.timelines.is_empty())); + } + + const TOTAL_UNITS: f64 = 72.0; +} diff --git a/crates/engine/src/lib.rs b/crates/engine/src/lib.rs index 258a1cb72..438e3d2a3 100644 --- a/crates/engine/src/lib.rs +++ b/crates/engine/src/lib.rs @@ -18,6 +18,7 @@ pub mod outline; pub mod paper; pub mod pivot; pub mod pivot_calc; +pub mod pivot_items; pub mod sample; pub mod selection; pub mod tables; @@ -119,6 +120,8 @@ const UNDO_LIMIT: usize = 100; impl DocState { pub fn new(mut wb: Workbook, path: Option, title: String) -> DocState { + // GETPIVOTDATA reads the reports of PivotTables that came with the file. + pivot::index_reports(&mut wb); let mut calc = Calc::new(); calc.recalc_all(&mut wb); let wb = Arc::new(wb); @@ -199,6 +202,10 @@ pub struct Prefs { pub user_dictionary: Vec, /// View › Show › Formula Bar (an application option in Excel, not saved in the file). pub formula_bar: bool, + /// PivotTable Analyze › Options › Generate GetPivotData (File › Options › Formulas › Use + /// GetPivotData functions for PivotTable references): pointing at a value of a PivotTable + /// while typing a formula inserts GETPIVOTDATA instead of the cell's reference. + pub generate_get_pivot_data: bool, } impl Default for Prefs { @@ -213,6 +220,7 @@ impl Default for Prefs { r1c1: false, user_dictionary: vec![], formula_bar: true, + generate_get_pivot_data: true, } } } diff --git a/crates/engine/src/pivot.rs b/crates/engine/src/pivot.rs index 6146bd4f9..49cb058c9 100644 --- a/crates/engine/src/pivot.rs +++ b/crates/engine/src/pivot.rs @@ -7,9 +7,11 @@ use std::cmp::Ordering; use std::collections::{HashMap, HashSet}; +use std::sync::Arc; use gridcraft_core::date::datetime_from_serial; use gridcraft_core::{CellError, CellRef, MAX_COLS, MAX_ROWS, RangeRef, Value}; +use gridcraft_formula::FormulaLocale; use gridcraft_model::*; use crate::pivot_calc::CalcSet; @@ -211,8 +213,89 @@ fn cmp_items(a: &Item, b: &Item) -> Ordering { a.class.cmp(&b.class).then(a.num.total_cmp(&b.num)).then_with(|| a.text.cmp(&b.text)) } -/// (identity key, item) for a source value under a field's date grouping. -fn make_item(wb: &Workbook, v: &Value, group: PivotDateGroup, fmt: &str, first: usize) -> (String, Item) { +/// How a field's values become items: as they are, by a date or time part (with the Grouping +/// dialog's bounds and day span), by number range, or into manual groups (Group Selection). +#[derive(Clone, Copy, Debug, Default, PartialEq)] +pub struct Grouping<'a> { + pub date: PivotDateGroup, + pub number: Option, + /// Bounds and day span of a date grouping. + pub range: Option, + /// First day of the first item when grouping by several days from automatic bounds (the + /// first date in the source). + pub first_day: f64, + /// Last day the last N-day item reaches (Ending at, or the automatic end after the last date). + pub last_day: f64, + /// The Group Selection groups the field's items fall into. + pub manual: Option<&'a PivotGroupField>, +} + +impl<'a> Grouping<'a> { + /// The date or number grouping of an axis field entry (no manual groups, automatic first day 0). + pub fn of(f: &PivotField) -> Grouping<'static> { + Grouping { date: f.date_group, number: f.number_group, range: f.date_range, first_day: 0.0, last_day: f64::MAX, manual: None } + } + + /// Fills in what depends on the source: the first day of N-day items with automatic bounds. + fn on(mut self, src: &Source, col: usize) -> Self { + let Some(range) = self.range.filter(|r| self.date == PivotDateGroup::Days && r.days > 1) else { return self }; + let (lo, hi) = src + .rows + .iter() + .filter_map(|r| match r.get(col) { + Some(Value::Number(n)) => Some(*n), + _ => None, + }) + .fold((f64::INFINITY, f64::NEG_INFINITY), |(lo, hi), n| (lo.min(n), hi.max(n))); + self.first_day = range.start.unwrap_or(lo).floor(); + // Excel's automatic end is the day after a last whole date (the last span is cut there). + let auto_end = if hi.fract() == 0.0 { hi + 1.0 } else { hi }; + self.last_day = range.end.unwrap_or(auto_end).floor(); + if !self.first_day.is_finite() || !self.last_day.is_finite() { + self.first_day = 0.0; + self.last_day = f64::MAX; + } + self + } +} + +/// The source column and grouping of an axis field: a source field, or a Group Selection field +/// (`Region2`) over one. +pub fn resolve_field<'a>(src: &Source, pt: &'a PivotTable, f: &PivotField) -> Option<(usize, Grouping<'a>)> { + if let Some(c) = src.col(&f.source_col) { + return Some((c, Grouping::of(f).on(src, c))); + } + let gf = pt.group_field(&f.source_col)?; + let c = src.col(&gf.base)?; + Some((c, Grouping { manual: Some(gf), ..Grouping::default() })) +} + +/// The source column and grouping of a field named `name` wherever it is: its first Rows or +/// Columns entry, a Group Selection field, or a plain source field (report filters). +pub fn resolve_name<'a>(src: &Source, pt: &'a PivotTable, name: &str) -> Option<(usize, Grouping<'a>)> { + match pt.rows.iter().chain(pt.columns.iter()).find(|f| f.source_col.eq_ignore_ascii_case(name.trim())) { + Some(f) => resolve_field(src, pt, f), + None => resolve_field(src, pt, &PivotField { source_col: name.trim().to_string(), ..PivotField::default() }), + } +} + +/// `3/5/2026` for a date serial (the captions of day ranges and of the `<` and `>` items). +fn mdy(wb: &Workbook, serial: f64) -> String { + match datetime_from_serial(wb.date_system, serial) { + Some(d) => format!("{}/{}/{}", d.month, d.day, d.year), + None => gridcraft_core::number_to_text(serial), + } +} + +/// An hour as Excel's Hours grouping shows it: `12 AM`, `1 AM` … `11 PM`. +pub fn hour_caption(h: u32) -> String { + let h = h % 24; + let twelve = if h.is_multiple_of(12) { 12 } else { h % 12 }; + format!("{twelve} {}", if h < 12 { "AM" } else { "PM" }) +} + +/// (identity key, item) for a source value under a field's grouping. +fn make_item(wb: &Workbook, v: &Value, group: Grouping, fmt: &str, first: usize) -> (String, Item) { let it = |value: Value, label: String, class: u8, num: f64, formatted: bool| Item { text: label.to_lowercase(), value, @@ -222,23 +305,63 @@ fn make_item(wb: &Workbook, v: &Value, group: PivotDateGroup, fmt: &str, first: first, formatted, }; + if let Some(gf) = group.manual { + // Group Selection: base items in a group become that group's item. + let base = make_item(wb, v, Grouping::default(), fmt, first); + return match gf.group_of(&base.1.label) { + Some(g) => (format!("G{}", g.name.to_lowercase()), it(Value::text(g.name.as_str()), g.name.clone(), 1, 0.0, false)), + None => base, + }; + } match v { Value::Empty => ("b".into(), it(Value::text("(blank)"), "(blank)".into(), 4, 0.0, false)), Value::Number(n) => { - if group != PivotDateGroup::None + if let Some(ng) = &group.number { + let text = |label: String, num: f64, key: String| (key, it(Value::text(label.as_str()), label, 0, num, false)); + if *n < ng.start { + return text(ng.below_caption(), f64::MIN, "lt".into()); + } + if *n > ng.end { + return text(ng.above_caption(), f64::MAX, "gt".into()); + } + let k = ng.range_index(*n); + return text(ng.caption(k), ng.range_start(k), format!("r{k}")); + } + if group.date != PivotDateGroup::None && let Some(dt) = datetime_from_serial(wb.date_system, *n) { + let text = |label: String, num: f64, key: String| (key, it(Value::text(label.as_str()), label, 0, num, false)); + let range = group.range.unwrap_or_default(); + // Before Starting at / after Ending at: one item each. + if let Some(s) = range.start.filter(|s| n.floor() < s.floor()) { + return text(format!("<{}", mdy(wb, s.floor())), f64::MIN, "lt".into()); + } + if let Some(e) = range.end.filter(|e| n.floor() > e.floor()) { + return text(format!(">{}", mdy(wb, e.floor())), f64::MAX, "gt".into()); + } let mon = MONTHS.get(dt.month.saturating_sub(1) as usize).copied().unwrap_or("Jan"); - let (label, key) = match group { + let (label, key) = match group.date { PivotDateGroup::Years => (dt.year.to_string(), dt.year as f64), PivotDateGroup::Quarters => { let q = (dt.month.saturating_sub(1)) / 3 + 1; (format!("Qtr{q}"), q as f64) } PivotDateGroup::Months => (mon.to_string(), dt.month as f64), + PivotDateGroup::Days if range.days > 1 => { + // Spans of `days` days from Starting at (or the first date). + let start = range.start.unwrap_or(group.first_day).floor(); + let days = f64::from(range.days.min(PivotDateRange::MAX_DAYS)); + let k = ((n.floor() - start) / days).floor(); + let lo = start + k * days; + let hi = (lo + days - 1.0).min(group.last_day).max(lo); + (format!("{} - {}", mdy(wb, lo), mdy(wb, hi)), lo) + } + PivotDateGroup::Hours => (hour_caption(dt.hour), dt.hour as f64), + PivotDateGroup::Minutes => (format!(":{:02}", dt.minute), dt.minute as f64), + PivotDateGroup::Seconds => (format!(":{:02}", dt.second), dt.second as f64), _ => (format!("{}-{mon}", dt.day), (dt.month * 100 + dt.day) as f64), }; - return (format!("g{key}"), it(Value::text(label.as_str()), label, 0, key, false)); + return text(label, key, format!("g{key}")); } let label = crate::display::format(v, fmt, wb).text; (format!("n{}", n.to_bits()), it(v.clone(), label, 0, *n, true)) @@ -254,7 +377,7 @@ fn make_item(wb: &Workbook, v: &Value, group: PivotDateGroup, fmt: &str, first: } impl FieldItems { - fn intern(&mut self, wb: &Workbook, v: &Value, group: PivotDateGroup, fmt: &str, first: usize) -> u32 { + fn intern(&mut self, wb: &Workbook, v: &Value, group: Grouping, fmt: &str, first: usize) -> u32 { let (key, item) = make_item(wb, v, group, fmt, first); if let Some(i) = self.index.get(&key) { return *i; @@ -264,6 +387,29 @@ impl FieldItems { self.index.insert(key, i); i } + /// The item of calculated item `name` (a text item after the data items in source order). + fn intern_calc(&mut self, name: &str) -> u32 { + let key = format!("c{}", name.to_lowercase()); + if let Some(i) = self.index.get(&key) { + return *i; + } + let i = self.items.len() as u32; + self.items.push(Item { + value: Value::text(name), + label: name.to_string(), + class: 1, + num: 0.0, + text: name.to_lowercase(), + first: usize::MAX, + formatted: false, + }); + self.index.insert(key, i); + i + } + /// The item whose label is `label` (case-insensitive). + fn find(&self, label: &str) -> Option { + self.items.iter().position(|it| it.label.eq_ignore_ascii_case(label.trim())).map(|i| i as u32) + } fn get(&self, i: u32) -> Option<&Item> { self.items.get(i as usize) } @@ -272,13 +418,13 @@ impl FieldItems { } } -/// The label of a source value under a date grouping (for filters and field lists). -pub fn item_label(wb: &Workbook, v: &Value, group: PivotDateGroup, fmt: &str) -> String { +/// The label of a source value under a grouping (for filters and field lists). +pub fn item_label(wb: &Workbook, v: &Value, group: Grouping, fmt: &str) -> String { make_item(wb, v, group, fmt, 0).1.label } /// Distinct item labels of a source column, sorted like the field list shows them. -pub fn distinct_labels(wb: &Workbook, src: &Source, col: usize, group: PivotDateGroup, limit: usize) -> Vec { +pub fn distinct_labels(wb: &Workbook, src: &Source, col: usize, group: Grouping, limit: usize) -> Vec { let mut fi = FieldItems::default(); let fmt = src.formats.get(col).map(String::as_str).unwrap_or("General"); for (i, r) in src.rows.iter().enumerate() { @@ -307,26 +453,35 @@ struct Acc { mean: f64, m2: f64, err: Option, + /// Calculated item results added here: how many, and their total (a calculated item's result + /// is already a summary, so under Count it adds its value, not one). + calc_n: u64, + calc_count: f64, } impl Acc { + /// Folds a number into the sum, product, minimum and maximum. + fn fold(&mut self, n: f64) { + let first = self.nums + self.calc_n == 0; + self.sum += n; + if first { + self.prod = n; + self.min = n; + self.max = n; + } else { + self.prod *= n; + self.min = self.min.min(n); + self.max = self.max.max(n); + } + } fn add(&mut self, v: &Value) { match v { Value::Empty => {} Value::Number(n) => { let n = *n; + self.fold(n); self.count += 1; self.nums += 1; - self.sum += n; - if self.nums == 1 { - self.prod = n; - self.min = n; - self.max = n; - } else { - self.prod *= n; - self.min = self.min.min(n); - self.max = self.max.max(n); - } let d = n - self.mean; self.mean += d / self.nums as f64; self.m2 += d * (n - self.mean); @@ -340,11 +495,23 @@ impl Acc { _ => self.count += 1, } } + /// Adds a calculated item's result for one combination of the other fields' items. + fn add_calc(&mut self, v: &Value) { + match v { + Value::Number(n) => { + self.fold(*n); + self.calc_n += 1; + self.calc_count += *n; + } + Value::Error(e) if self.err.is_none() => self.err = Some(*e), + _ => {} + } + } fn result(&self, f: PivotFunc) -> Value { let n = self.nums as f64; match f { - PivotFunc::Count => return Value::number(self.count as f64), - PivotFunc::CountNumbers => return Value::number(n), + PivotFunc::Count => return Value::number(self.count as f64 + self.calc_count), + PivotFunc::CountNumbers => return Value::number(n + self.calc_count), _ => {} } if let Some(e) = self.err { @@ -360,9 +527,9 @@ impl Acc { Value::number(self.sum / n) } } - PivotFunc::Max => Value::number(if self.nums == 0 { 0.0 } else { self.max }), - PivotFunc::Min => Value::number(if self.nums == 0 { 0.0 } else { self.min }), - PivotFunc::Product => Value::number(if self.nums == 0 { 0.0 } else { self.prod }), + PivotFunc::Max => Value::number(if self.nums + self.calc_n == 0 { 0.0 } else { self.max }), + PivotFunc::Min => Value::number(if self.nums + self.calc_n == 0 { 0.0 } else { self.min }), + PivotFunc::Product => Value::number(if self.nums + self.calc_n == 0 { 0.0 } else { self.prod }), PivotFunc::StdDev | PivotFunc::Var => { if self.nums < 2 { div0 @@ -511,6 +678,22 @@ pub struct Output { pub chart_cols: Vec<(u32, String)>, /// Grid column holding the innermost row labels (`None` without row fields). pub label_col: Option, + /// What each line and data column stands for (GETPIVOTDATA reads it). + pub report: PivotReport, + /// The item label cells: which field and item each shows. + pub items: Vec, +} + +/// A report cell that shows an item of a row or column field. +#[derive(Clone, Debug, PartialEq)] +pub struct ItemCell { + /// Grid row and column (relative to the report's top-left). + pub row: u32, + pub col: u32, + /// The field (as named in Rows or Columns) and the item's caption. + pub field: String, + pub label: String, + pub on_rows: bool, } impl Output { @@ -813,6 +996,103 @@ enum ValSrc { Calc(usize), } +// ---------------------------------------------------------------- label, value and Top 10 filters + +/// Orders an item caption against a filter value: as numbers when both are numbers, otherwise +/// as text without regard to case. +fn caption_cmp(a: &str, b: &str) -> Ordering { + match (a.trim().parse::(), b.trim().parse::()) { + (Ok(x), Ok(y)) => x.total_cmp(&y), + _ => a.to_lowercase().cmp(&b.to_lowercase()), + } +} + +/// Whether an item caption passes a label filter. The text comparisons take `*` and `?` +/// wildcards (and `~` to escape them); none depends on case. +fn label_matches(op: PivotCompare, label: &str, a: &str, b: &str) -> bool { + let wild = |pat: String| crate::cmd::edit::wildcard(&label.to_lowercase(), &pat.to_lowercase()); + let (lo, hi) = if caption_cmp(a, b) == Ordering::Greater { (b, a) } else { (a, b) }; + let between = || caption_cmp(label, lo) != Ordering::Less && caption_cmp(label, hi) != Ordering::Greater; + match op { + PivotCompare::Equals => wild(a.to_string()), + PivotCompare::NotEquals => !wild(a.to_string()), + PivotCompare::BeginsWith => wild(format!("{a}*")), + PivotCompare::NotBeginsWith => !wild(format!("{a}*")), + PivotCompare::EndsWith => wild(format!("*{a}")), + PivotCompare::NotEndsWith => !wild(format!("*{a}")), + PivotCompare::Contains => wild(format!("*{a}*")), + PivotCompare::NotContains => !wild(format!("*{a}*")), + PivotCompare::Greater => caption_cmp(label, a) == Ordering::Greater, + PivotCompare::GreaterOrEqual => caption_cmp(label, a) != Ordering::Less, + PivotCompare::Less => caption_cmp(label, a) == Ordering::Less, + PivotCompare::LessOrEqual => caption_cmp(label, a) != Ordering::Greater, + PivotCompare::Between => between(), + PivotCompare::NotBetween => !between(), + } +} + +/// Whether an item's total passes a value filter. +fn value_matches(op: PivotCompare, v: f64, a: f64, b: f64) -> bool { + let (lo, hi) = if a <= b { (a, b) } else { (b, a) }; + match op { + PivotCompare::Equals => v == a, + PivotCompare::NotEquals => v != a, + PivotCompare::Greater => v > a, + PivotCompare::GreaterOrEqual => v >= a, + PivotCompare::Less => v < a, + PivotCompare::LessOrEqual => v <= a, + PivotCompare::Between => v >= lo && v <= hi, + PivotCompare::NotBetween => v < lo || v > hi, + // Text comparisons are refused for value filters; one read from a file keeps every item. + _ => true, + } +} + +/// The items a value or Top 10 filter drops among siblings, given each item's total (`None` when +/// it is not a number). +fn rule_drops(rule: &PivotRule, scored: &[(u32, Option)]) -> Vec { + match rule { + PivotRule::Label { .. } => vec![], + PivotRule::Value { op, value, value2, .. } => { + scored.iter().filter(|(_, v)| !v.is_some_and(|x| value_matches(*op, x, *value, *value2))).map(|(i, _)| *i).collect() + } + PivotRule::Top { top, count, by, .. } => { + // Items with a number, best first (ties keep their order), then those without. + let mut order: Vec<(u32, f64)> = scored.iter().filter_map(|(i, v)| v.map(|x| (*i, x))).collect(); + order.sort_by(|a, b| if *top { b.1.total_cmp(&a.1) } else { a.1.total_cmp(&b.1) }); + let keep = match by { + PivotTopBy::Items => { + let n = count.max(0.0).floor(); + let n = if n >= order.len() as f64 { order.len() } else { n as usize }; + // As in Excel, items tied with the last one kept are kept too (a Top 15 with + // items 14 to 16 all equal shows 16 items). + match n.checked_sub(1).and_then(|i| order.get(i)) { + Some((_, cut)) => n + order.iter().skip(n).take_while(|(_, v)| v == cut).count(), + None => n, + } + } + PivotTopBy::Percent | PivotTopBy::Sum => { + // Items are kept until their running sum reaches the goal. + let total: f64 = order.iter().map(|x| x.1.max(0.0)).sum(); + let goal = if *by == PivotTopBy::Percent { total * count / 100.0 } else { *count }; + let mut acc = 0.0; + let mut n = 0; + for (_, v) in &order { + if acc >= goal { + break; + } + acc += v.max(0.0); + n += 1; + } + n + } + }; + let kept: HashSet = order.iter().take(keep).map(|x| x.0).collect(); + scored.iter().map(|x| x.0).filter(|i| !kept.contains(i)).collect() + } + } +} + /// True when `col_groups` distinct column leaves (plus the grand-total column) times `values` value fields /// (at least one data column per group) already exceed the sheet width. This is a lower bound on the final column count. fn data_cols_exceed_sheet(col_groups: usize, grand_total: bool, values: usize) -> bool { @@ -823,8 +1103,15 @@ fn data_cols_exceed_sheet(col_groups: usize, grand_total: bool, values: usize) - pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result { let src = read_source(wb, &pt.source, pivot_sheet)?; let fmt_of = |c: usize| src.formats.get(c).cloned().unwrap_or_else(|| "General".into()); - let rows: Vec<(usize, &PivotField)> = pt.rows.iter().filter_map(|f| src.col(&f.source_col).map(|c| (c, f))).collect(); - let cols: Vec<(usize, &PivotField)> = pt.columns.iter().filter_map(|f| src.col(&f.source_col).map(|c| (c, f))).collect(); + // Axis fields found in the source (source fields, or Group Selection fields over one), with + // the grouping that makes their items. + let row_axis: Vec<(usize, &PivotField, Grouping)> = pt.rows.iter().filter_map(|f| resolve_field(&src, pt, f).map(|(c, g)| (c, f, g))).collect(); + let col_axis: Vec<(usize, &PivotField, Grouping)> = + pt.columns.iter().filter_map(|f| resolve_field(&src, pt, f).map(|(c, g)| (c, f, g))).collect(); + let rows: Vec<(usize, &PivotField)> = row_axis.iter().map(|(c, f, _)| (*c, *f)).collect(); + let cols: Vec<(usize, &PivotField)> = col_axis.iter().map(|(c, f, _)| (*c, *f)).collect(); + let row_groups: Vec = row_axis.iter().map(|x| x.2).collect(); + let col_groups: Vec = col_axis.iter().map(|x| x.2).collect(); // Value fields: a source column, or a calculated field evaluated on sums (see `pivot_calc`). let calc = CalcSet::new(pt, &src.headers); let vals: Vec<(ValSrc, &PivotValue)> = pt @@ -854,20 +1141,31 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result MAX_VALUE_FIELDS { return Err(format!("A PivotTable can have at most {MAX_VALUE_FIELDS} value fields.")); } - let axis_group = |name: &str| -> Option { - pt.rows.iter().chain(pt.columns.iter()).find(|f| f.source_col.eq_ignore_ascii_case(name)).map(|f| f.date_group) - }; // (column, filter, grouping used for labels, shown as a report filter) - let filters: Vec<(usize, &PivotFilter, PivotDateGroup, bool)> = pt + let filters: Vec<(usize, &PivotFilter, Grouping, bool)> = pt .filters .iter() .filter_map(|f| { - let c = src.col(&f.source_col)?; - let g = axis_group(&f.source_col); - Some((c, f, g.unwrap_or_default(), g.is_none())) + let (c, g) = resolve_name(&src, pt, &f.source_col)?; + let on_axis = pt.rows.iter().chain(pt.columns.iter()).any(|a| a.source_col.eq_ignore_ascii_case(&f.source_col)); + if !on_axis && f.rule.is_some() { + return None; + } + Some((c, f, g, !on_axis)) }) .collect(); - let shown_filters: Vec<&(usize, &PivotFilter, PivotDateGroup, bool)> = filters.iter().filter(|f| f.3).collect(); + // Timelines connected to this PivotTable: (date column, first day, last day) of each selection. + let periods: Vec<(usize, f64, f64)> = wb + .sheets + .iter() + .flat_map(|s| s.timelines.iter()) + .filter(|t| t.pivots.contains(&pt.id)) + .filter_map(|t| { + let (a, b) = t.selection?; + Some((src.col(&t.field)?, a.floor(), b.floor())) + }) + .collect(); + let shown_filters: Vec<&(usize, &PivotFilter, Grouping, bool)> = filters.iter().filter(|f| f.3).collect(); let k = vals.len(); let empty = rows.is_empty() && cols.is_empty() && vals.is_empty(); @@ -875,7 +1173,7 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result = (pt.values_on_rows && k > 1).then(|| { let p = pt.values_position.unwrap_or(pt.rows.len()); - pt.rows.iter().take(p).filter(|f| src.col(&f.source_col).is_some()).count() + pt.rows.iter().take(p).filter(|f| resolve_field(&src, pt, f).is_some()).count() }); let on_rows = vpos.is_some(); let row_levels = rows.len() + usize::from(on_rows); @@ -890,26 +1188,19 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result = filters.iter().map(|f| fmt_of(f.0)).collect(); let row_fmts: Vec = rows.iter().map(|(c, _)| fmt_of(*c)).collect(); let col_fmts: Vec = cols.iter().map(|(c, _)| fmt_of(*c)).collect(); - 'rec: for (ri, rec) in src.rows.iter().enumerate() { - for ((c, f, g, _), fmt) in filters.iter().zip(filter_fmts.iter()) { - if let Some(sel) = &f.selected { - let label = item_label(wb, rec.get(*c).unwrap_or(&Value::Empty), *g, fmt); - if !sel.iter().any(|s| s.eq_ignore_ascii_case(&label)) { - continue 'rec; - } - } - } - if empty { - continue; - } - let mut rp = Vec::with_capacity(rows.len()); - for (((c, f), fi), fmt) in rows.iter().zip(row_items.iter_mut()).zip(row_fmts.iter()) { - rp.push(fi.intern(wb, rec.get(*c).unwrap_or(&Value::Empty), f.date_group, fmt, ri)); - } - let mut cp = Vec::with_capacity(cols.len()); - for (((c, f), fi), fmt) in cols.iter().zip(col_items.iter_mut()).zip(col_fmts.iter()) { - cp.push(fi.intern(wb, rec.get(*c).unwrap_or(&Value::Empty), f.date_group, fmt, ri)); - } + // Value and Top 10 filters, by axis level; they need every record's paths kept for a second pass. + let value_rule = |f: &PivotField| { + pt.filters + .iter() + .find(|x| x.source_col.eq_ignore_ascii_case(&f.source_col)) + .and_then(|x| x.rule.as_ref()) + .filter(|r| r.value_field().is_some()) + }; + let row_rules: Vec> = rows.iter().map(|(_, f)| value_rule(f)).collect(); + let col_rules: Vec> = cols.iter().map(|(_, f)| value_rule(f)).collect(); + let by_value = k > 0 && row_rules.iter().chain(col_rules.iter()).any(Option::is_some); + let mut recs: Vec<(usize, Vec, Vec)> = Vec::new(); + let add = |aggs: &mut AggMap, rec: &[Value], rp: &[u32], cp: &[u32]| -> Result<(), String> { for a in 0..=rp.len() { for b in 0..=cp.len() { let e = aggs.entry((rp[..a].to_vec(), cp[..b].to_vec())).or_insert_with(|| vec![Acc::default(); k + extra.len()]); @@ -923,6 +1214,48 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result 5_000_000 { + return Err("The PivotTable report is too large.".into()); + } + Ok(()) + }; + 'rec: for (ri, rec) in src.rows.iter().enumerate() { + for ((c, f, g, report), fmt) in filters.iter().zip(filter_fmts.iter()) { + let label_rule = f.rule.as_ref().filter(|r| !*report && r.value_field().is_none()); + if f.selected.is_none() && label_rule.is_none() { + continue; + } + let label = item_label(wb, rec.get(*c).unwrap_or(&Value::Empty), *g, fmt); + if let Some(sel) = &f.selected + && !sel.iter().any(|s| s.eq_ignore_ascii_case(&label)) + { + continue 'rec; + } + if let Some(PivotRule::Label { op, value, value2 }) = label_rule + && !label_matches(*op, &label, value, value2) + { + continue 'rec; + } + } + // A timeline keeps the records whose date falls in its selection. + for (c, a, b) in &periods { + match rec.get(*c) { + Some(Value::Number(n)) if n.floor() >= *a && n.floor() <= *b => {} + _ => continue 'rec, + } + } + if empty { + continue; + } + let mut rp = Vec::with_capacity(rows.len()); + for ((((c, _), fi), fmt), g) in rows.iter().zip(row_items.iter_mut()).zip(row_fmts.iter()).zip(row_groups.iter()) { + rp.push(fi.intern(wb, rec.get(*c).unwrap_or(&Value::Empty), *g, fmt, ri)); + } + let mut cp = Vec::with_capacity(cols.len()); + for ((((c, _), fi), fmt), g) in cols.iter().zip(col_items.iter_mut()).zip(col_fmts.iter()).zip(col_groups.iter()) { + cp.push(fi.intern(wb, rec.get(*c).unwrap_or(&Value::Empty), *g, fmt, ri)); + } + add(&mut aggs, rec, &rp, &cp)?; // The column leaf this record lands in: its path cut after the first collapsed item (as in `flatten_cols`). let shown = cols .iter() @@ -933,14 +1266,106 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result 5_000_000 { - return Err("The PivotTable report is too large.".into()); + if by_value { + recs.push((ri, rp.clone(), cp.clone())); + } + row_set.insert(rp); + col_set.insert(cp); + } + + // A value field's result for one report cell: a source column's summary, or a calculated + // field evaluated on the sums of the fields it names. + let value_at = |aggs: &AggMap, rp: &[u32], cp: &[u32], vi: usize| -> Option { + let (vs, v) = vals.get(vi)?; + let accs = aggs.get(&(rp.to_vec(), cp.to_vec()))?; + match vs { + ValSrc::Col(_) => accs.get(vi).map(|a| a.result(v.func)), + ValSrc::Calc(i) => { + let sum = |name: &str| -> Value { + src.col(name) + .and_then(|c| extra.iter().position(|x| *x == c)) + .and_then(|j| accs.get(k + j)) + .map(|a| a.result(PivotFunc::Sum)) + .unwrap_or(Value::Number(0.0)) + }; + Some(calc.value(wb, *i, &sum)) + } + } + }; + + // ---- calculated items: for each combination of the other fields' items, a formula runs on + // the summaries of the items it names; the results then count in subtotals and grand totals + // like any item's, as in Excel. + let slot_funcs: Vec> = vals + .iter() + .map(|(vs, v)| match vs { + ValSrc::Col(_) => Some(v.func), + ValSrc::Calc(_) => None, + }) + .chain(extra.iter().map(|_| Some(PivotFunc::Sum))) + .collect(); + let calc_items = prepare_calc_items(pt, &rows, &cols, &mut row_items, &mut col_items); + let levels = (rows.len(), cols.len()); + apply_calc_items(wb, &calc_items, &slot_funcs, &mut aggs, (&mut row_set, &mut col_set), levels, &|_, _| false)?; + + // ---- value and Top 10 filters: each drops items among their siblings (by each item's total), + // then the report is aggregated again without them, so subtotals and grand totals count only + // the items shown, as in Excel. + if by_value { + let value_index = |name: &str| vals.iter().position(|(_, v)| v.name.eq_ignore_ascii_case(name)); + let mut excl_rows: HashSet> = HashSet::new(); + let mut excl_cols: HashSet> = HashSet::new(); + for (on_rows, rules, set, excl) in [(true, &row_rules, &row_set, &mut excl_rows), (false, &col_rules, &col_set, &mut excl_cols)] { + let mut siblings: HashMap, Vec> = HashMap::new(); + for path in set { + for (d, item) in path.iter().enumerate() { + if rules.get(d).is_some_and(Option::is_some) { + let e = siblings.entry(path[..d].to_vec()).or_default(); + if !e.contains(item) { + e.push(*item); + } + } + } + } + for (parent, items) in &siblings { + let Some(Some(rule)) = rules.get(parent.len()) else { continue }; + let Some(vi) = rule.value_field().and_then(value_index) else { continue }; + let scored: Vec<(u32, Option)> = items + .iter() + .map(|it| { + let mut path = parent.clone(); + path.push(*it); + let v = if on_rows { value_at(&aggs, &path, &[], vi) } else { value_at(&aggs, &[], &path, vi) }; + (*it, v.as_ref().and_then(Value::as_f64)) + }) + .collect(); + for it in rule_drops(rule, &scored) { + let mut path = parent.clone(); + path.push(it); + excl.insert(path); + } + } + } + if !excl_rows.is_empty() || !excl_cols.is_empty() { + let hit = |excl: &HashSet>, p: &[u32]| (1..=p.len()).any(|n| excl.contains(&p[..n])); + aggs.clear(); + row_set.clear(); + col_set.clear(); + for (ri, rp, cp) in recs { + if hit(&excl_rows, &rp) || hit(&excl_cols, &cp) { + continue; + } + let Some(rec) = src.rows.get(ri) else { continue }; + add(&mut aggs, rec, &rp, &cp)?; + row_set.insert(rp); + col_set.insert(cp); + } + let dropped = |rp: &[u32], cp: &[u32]| hit(&excl_rows, rp) || hit(&excl_cols, cp); + apply_calc_items(wb, &calc_items, &slot_funcs, &mut aggs, (&mut row_set, &mut col_set), levels, &dropped)?; } } @@ -953,7 +1378,6 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result, Vec> = HashMap::new(); collect_siblings(&row_tree, &mut Vec::new(), &mut siblings); - let level0: Vec = row_tree.iter().map(|n| n.item).collect(); let val_names: Vec = vals.iter().map(|(_, v)| v.name.clone()).collect(); let ctx = Ctx { pt, rows: row_fields.clone(), row_items, row_fmts: row_fmts.clone(), vpos, val_names: val_names.clone() }; @@ -1057,6 +1481,8 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result Result = Vec::new(); // ---- header rows + // The report row of each column field's item labels. + let mut col_label_rows: Vec = Vec::new(); if cols.is_empty() { let r = g.push_row(RowKind::Header)?; row_header(&mut g, r); @@ -1103,6 +1532,7 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result 1 && !on_rows); let first_level = g.rows.len(); + col_label_rows = (0..cols.len()).map(|lv| (first_level + lv) as u32).collect(); for _ in 0..levels { g.push_row(RowKind::Header)?; } @@ -1118,6 +1548,15 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result 1 @@ -1150,23 +1589,7 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result Option { - let (vs, v) = vals.get(vi)?; - let accs = aggs.get(&(rp.to_vec(), cp.to_vec()))?; - match vs { - ValSrc::Col(_) => accs.get(vi).map(|a| a.result(v.func)), - ValSrc::Calc(i) => { - let sum = |name: &str| -> Value { - src.col(name) - .and_then(|c| extra.iter().position(|x| *x == c)) - .and_then(|j| accs.get(k + j)) - .map(|a| a.result(PivotFunc::Sum)) - .unwrap_or(Value::Number(0.0)) - }; - Some(calc.value(wb, *i, &sum)) - } - } - }; + let get = |rp: &[u32], cp: &[u32], vi: usize| value_at(&aggs, rp, cp, vi); let num = |v: &Option| v.as_ref().and_then(Value::as_f64); let ratio = |a: Value, b: Option| -> Value { match (a, b) { @@ -1185,10 +1608,8 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result> = vals .iter() .map(|(vs, v)| match v.show_as { - PivotShowAs::PercentOfGrandTotal | PivotShowAs::PercentOfColumnTotal | PivotShowAs::PercentOfRowTotal => { - Some(v.number_format.clone().unwrap_or_else(|| "0.00%".into())) - } - PivotShowAs::Rank => v.number_format.clone(), + sa if sa.is_percent() => Some(v.number_format.clone().unwrap_or_else(|| "0.00%".into())), + PivotShowAs::Rank | PivotShowAs::RankAscending | PivotShowAs::Index => v.number_format.clone(), _ => v.number_format.clone().or_else(|| match v.func { PivotFunc::Count | PivotFunc::CountNumbers => None, _ => { @@ -1230,6 +1651,144 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result, Vec> = HashMap::new(); + collect_siblings(&col_tree, &mut Vec::new(), &mut col_siblings); + // Per value field: the base field (axis, depth) and base item. + let bases: Vec> = vals + .iter() + .map(|(_, v)| { + if !v.show_as.needs_base_field() { + return None; + } + let (is_row, depth) = match v.base_field.as_deref() { + Some(b) => match row_fields.iter().position(|f| f.source_col.eq_ignore_ascii_case(b)) { + Some(d) => (true, d), + None => (false, col_fields.iter().position(|f| f.source_col.eq_ignore_ascii_case(b))?), + }, + None if !row_fields.is_empty() => (true, 0), + None if !col_fields.is_empty() => (false, 0), + None => return None, + }; + let items = if is_row { ctx.row_items.get(depth) } else { col_items.get(depth) }; + let item = match v.base_item.as_deref() { + Some(BASE_PREVIOUS) => BaseItem::Previous, + Some(BASE_NEXT) => BaseItem::Next, + Some(label) => items + .and_then(|fi| fi.items.iter().position(|it| it.label.eq_ignore_ascii_case(label))) + .map_or(BaseItem::Missing, |i| BaseItem::At(i as u32)), + None => BaseItem::Missing, + }; + Some((is_row, depth, item)) + }) + .collect(); + let show = |raw: Value, rp: &[u32], cp: &[u32], vi: usize| -> Value { + let Some((_, vdef)) = vals.get(vi) else { return raw }; + match vdef.show_as { + PivotShowAs::Normal => raw, + PivotShowAs::PercentOfGrandTotal => ratio(raw, get(&[], &[], vi)), + PivotShowAs::PercentOfColumnTotal => ratio(raw, get(&[], cp, vi)), + PivotShowAs::PercentOfRowTotal => ratio(raw, get(rp, &[], vi)), + PivotShowAs::PercentOfParentRowTotal => match rp.split_last() { + Some((_, parent)) => ratio(raw, get(parent, cp, vi)), + None => ratio(raw.clone(), Some(raw)), + }, + PivotShowAs::PercentOfParentColumnTotal => match cp.split_last() { + Some((_, parent)) => ratio(raw, get(rp, parent, vi)), + None => ratio(raw.clone(), Some(raw)), + }, + PivotShowAs::Index => match (raw.as_f64(), num(&get(&[], &[], vi)), num(&get(rp, &[], vi)), num(&get(&[], cp, vi))) { + (Some(v), Some(g), Some(r), Some(c)) => { + if r == 0.0 || c == 0.0 { + Value::Error(CellError::Div0) + } else { + Value::number(v * g / (r * c)) + } + } + _ => raw, + }, + _ => { + let Some(Some((is_row, d, item))) = bases.get(vi) else { return raw }; + let (is_row, d) = (*is_row, *d); + let path = if is_row { rp } else { cp }; + // Rows or columns above the base field's level (subtotals, grand totals). + let (Some(&mine), Some(parent)) = (path.get(d), path.get(..d)) else { + return match vdef.show_as { + PivotShowAs::RunningTotal => raw, + _ => Value::Empty, + }; + }; + let sibs: Vec = (if is_row { &siblings } else { &col_siblings }).get(parent).cloned().unwrap_or_default(); + let at = |it: u32| -> Option { + let mut p = path.to_vec(); + if let Some(x) = p.get_mut(d) { + *x = it; + } + if is_row { get(&p, cp, vi) } else { get(rp, &p, vi) } + }; + let pos = sibs.iter().position(|x| *x == mine); + let base = match item { + BaseItem::At(i) => Some(*i), + BaseItem::Previous => pos.and_then(|p| p.checked_sub(1)).and_then(|p| sibs.get(p)).copied(), + BaseItem::Next => pos.and_then(|p| sibs.get(p + 1)).copied(), + BaseItem::Missing => None, + }; + let running = || -> f64 { + let upto = pos.map_or(0, |p| p + 1); + sibs.iter().take(upto).map(|s| num(&at(*s)).unwrap_or(0.0)).sum() + }; + match vdef.show_as { + PivotShowAs::PercentOf => match base { + Some(b) if b == mine => ratio(raw.clone(), Some(raw)), + Some(b) => match at(b) { + Some(bv) => ratio(raw, Some(bv)), + None => Value::Error(CellError::NA), + }, + None => Value::Error(CellError::NA), + }, + PivotShowAs::DifferenceFrom | PivotShowAs::PercentDifferenceFrom => match base { + Some(b) if b == mine => Value::Empty, + Some(b) => match (raw.as_f64(), at(b)) { + (Some(v), Some(bv)) => match (bv.as_f64(), vdef.show_as) { + (Some(x), PivotShowAs::DifferenceFrom) => Value::number(v - x), + (Some(0.0), _) => Value::Error(CellError::Div0), + (Some(x), _) => Value::number((v - x) / x), + (None, _) => bv, + }, + (_, None) => Value::Error(CellError::NA), + (None, _) => raw, + }, + None => Value::Empty, + }, + PivotShowAs::RunningTotal => match raw { + Value::Number(_) => Value::number(running()), + other => other, + }, + PivotShowAs::PercentRunningTotal => match raw { + Value::Number(_) => { + let total: f64 = sibs.iter().map(|s| num(&at(*s)).unwrap_or(0.0)).sum(); + if total == 0.0 { Value::Error(CellError::Div0) } else { Value::number(running() / total) } + } + other => other, + }, + PivotShowAs::Rank | PivotShowAs::RankAscending => match raw.as_f64() { + Some(me) => { + let desc = vdef.show_as == PivotShowAs::Rank; + let ahead = sibs.iter().filter(|s| num(&at(**s)).is_some_and(|x| if desc { x > me } else { x < me })).count(); + Value::number((ahead + 1) as f64) + } + None => raw, + }, + PivotShowAs::PercentOfParentTotal => { + let top = path.get(..=d).unwrap_or(path); + let denom = if is_row { get(top, cp, vi) } else { get(rp, top, vi) }; + ratio(raw, denom) + } + _ => raw, + } + } + } + }; + let mut report_rows: Vec = Vec::new(); for line in &lines { let rk = match line.kind { LineKind::Item(b) => RowKind::Item(b), @@ -1247,74 +1806,232 @@ pub fn compute(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Result continue, + Some(p) if level > p => level - 1, + _ => level, + }; + if let (LineKind::Item(_), Some(&it), Some(f)) = (line.kind, line.path.get(depth), row_fields.get(depth)) { + let label = ctx.row_items.get(depth).map(|fi| fi.label(it)).unwrap_or_default(); + item_cells.push(ItemCell { row: r as u32, col: *c as u32, field: f.source_col.clone(), label, on_rows: true }); + } } if !line.values { continue; } + report_rows.push(ReportKey { offset: r as u32, data: line.vi.map(|v| v as u32), items: key_items(&line.path, rows.len()) }); let rp = line.path.as_slice(); for (j, (si, col_vi)) in data_cols.iter().enumerate() { let line_vi = col_vi.or(line.vi); let (Some(spec), Some(vi)) = (specs.get(*si), &line_vi) else { continue }; - let Some((_, vdef)) = vals.get(*vi) else { continue }; let cp = spec.path.as_slice(); let Some(raw) = get(rp, cp, *vi) else { continue }; - let shown = match vdef.show_as { - PivotShowAs::Normal => raw, - PivotShowAs::PercentOfGrandTotal => ratio(raw, get(&[], &[], *vi)), - PivotShowAs::PercentOfColumnTotal => ratio(raw, get(&[], cp, *vi)), - PivotShowAs::PercentOfRowTotal => ratio(raw, get(rp, &[], *vi)), - PivotShowAs::RunningTotal => match (rp.first(), &raw) { - (Some(first), Value::Number(_)) => { - let mut acc = 0.0; - for it in &level0 { - let mut p = rp.to_vec(); - if let Some(x) = p.first_mut() { - *x = *it; - } - acc += num(&get(&p, cp, *vi)).unwrap_or(0.0); - if it == first { - break; - } - } - Value::number(acc) + let shown = show(raw, rp, cp, *vi); + g.set(r, l + j, shown, ck.1, 0, data_fmt.get(*vi).cloned().flatten()); + } + } + let report_cols: Vec = data_cols + .iter() + .enumerate() + .filter_map(|(j, (si, vi))| { + let spec = specs.get(*si)?; + Some(ReportKey { offset: (l + j) as u32, data: vi.map(|v| v as u32), items: key_items(&spec.path, cols.len()) }) + }) + .collect(); + let field = |(c, f): &(usize, &PivotField), fi: &FieldItems| ReportField { + // A Group Selection field (`Region2`) goes by its own name, not its base field's. + name: src.headers.get(*c).filter(|_| src.col(&f.source_col).is_some()).cloned().unwrap_or_else(|| f.source_col.clone()), + dates: f.date_group == PivotDateGroup::None && src.dates.get(*c).copied().unwrap_or(false), + items: fi.items.iter().map(|it| ReportItem { value: it.value.clone(), label: it.label.clone() }).collect(), + }; + let report = PivotReport { + data_fields: vals.iter().map(|(_, v)| ReportDataField { name: v.name.clone(), source: v.source_col.clone() }).collect(), + row_fields: rows.iter().zip(ctx.row_items.iter()).map(|(f, fi)| field(f, fi)).collect(), + col_fields: cols.iter().zip(col_items.iter()).map(|(f, fi)| field(f, fi)).collect(), + page_fields: shown_filters + .iter() + .map(|&&(c, f, _, _)| { + let one = match &f.selected { + Some(v) if v.len() == 1 => v.first().cloned(), + _ => None, + }; + (src.headers.get(c).cloned().unwrap_or_default(), one) + }) + .collect(), + rows: report_rows, + cols: report_cols, + col_label_rows, + label_cols: l as u32, + compact, + }; + Ok(Output { width: g.width as u32, grid: g.rows, row_kinds: g.kinds, body_offset, chart_rows, chart_cols, label_col, items: item_cells, report }) +} + +/// A calculated item placed on an axis of the report. +struct CalcAt { + on_rows: bool, + /// Depth of its field on that axis. + depth: usize, + /// Its item id in the field's items. + item: u32, + /// Not hidden by its field's own filter. + shown: bool, + compiled: Result, + /// Item ids of the formula's references, in `compiled.refs` order (`None`: not an item now). + refs: Vec>, +} + +/// Most calculated item results one report computes (combinations × value fields). +const MAX_CALC_RESULTS: usize = 2_000_000; + +/// The calculated items whose field is in Rows or Columns, as items of that field. +fn prepare_calc_items( + pt: &PivotTable, + rows: &[(usize, &PivotField)], + cols: &[(usize, &PivotField)], + row_items: &mut [FieldItems], + col_items: &mut [FieldItems], +) -> Vec { + let place = |field: &str| { + let pos = |list: &[(usize, &PivotField)]| list.iter().position(|(_, f)| f.source_col.eq_ignore_ascii_case(field)); + match (pos(rows), pos(cols)) { + (Some(d), _) => Some((true, d)), + (None, Some(d)) => Some((false, d)), + _ => None, + } + }; + // Every calculated item is an item before any formula is read, so formulas may name them. + let mut placed: Vec<(&PivotCalcItem, bool, usize, u32)> = Vec::new(); + for ci in pt.calculated_items.iter().take(crate::pivot_items::MAX_CALC_ITEMS) { + let Some((on_rows, depth)) = place(&ci.field) else { continue }; + let items = if on_rows { row_items.get_mut(depth) } else { col_items.get_mut(depth) }; + let Some(items) = items else { continue }; + placed.push((ci, on_rows, depth, items.intern_calc(&ci.name))); + } + placed + .into_iter() + .map(|(ci, on_rows, depth, item)| { + let items: Option<&FieldItems> = if on_rows { row_items.get(depth) } else { col_items.get(depth) }; + let lookup = |n: &str| items.and_then(|fi| fi.find(n).map(|i| fi.label(i))); + let compiled = crate::pivot_items::compile(&ci.formula, &ci.field, &lookup); + let refs = match &compiled { + Ok(c) => c.refs.iter().map(|r| items.and_then(|fi| fi.find(r))).collect(), + Err(_) => vec![], + }; + let filter = pt.filters.iter().find(|f| f.source_col.eq_ignore_ascii_case(&ci.field)); + let shown = filter.is_none_or(|f| { + f.selected.as_ref().is_none_or(|sel| sel.iter().any(|x| x.eq_ignore_ascii_case(&ci.name))) + && match &f.rule { + Some(PivotRule::Label { op, value, value2 }) => label_matches(*op, &ci.name, value, value2), + _ => true, } - _ => raw, - }, - PivotShowAs::Rank => match (rp.split_last(), raw.as_f64()) { - (Some((_, parent)), Some(me)) => { - let sibs = siblings.get(parent).cloned().unwrap_or_default(); - let mut rank = 1u64; - for sib in sibs { - let mut p = parent.to_vec(); - p.push(sib); - if num(&get(&p, cp, *vi)).is_some_and(|x| x > me) { - rank += 1; - } + }); + CalcAt { on_rows, depth, item, shown, compiled, refs } + }) + .collect() +} + +/// Adds each calculated item's results to the aggregates, in solve order: for every combination +/// of the other fields' items (`levels` = row and column depths of a full path) its formula runs +/// on the summaries of the items it names, and the result counts at every level above, so +/// subtotals and grand totals include it. `skip` leaves out combinations a value filter dropped. +fn apply_calc_items( + wb: &Workbook, + calc: &[CalcAt], + funcs: &[Option], + aggs: &mut AggMap, + sets: (&mut HashSet>, &mut HashSet>), + levels: (usize, usize), + skip: &dyn Fn(&[u32], &[u32]) -> bool, +) -> Result<(), String> { + let (row_set, col_set) = sets; + for ci in calc.iter().filter(|c| c.shown) { + let place = |rp: &[u32], cp: &[u32], item: u32| -> Option<(Vec, Vec)> { + let (mut r, mut c) = (rp.to_vec(), cp.to_vec()); + *(if ci.on_rows { r.get_mut(ci.depth) } else { c.get_mut(ci.depth) })? = item; + Some((r, c)) + }; + let contexts: HashSet<(Vec, Vec)> = aggs + .keys() + .filter(|(rp, cp)| rp.len() == levels.0 && cp.len() == levels.1) + .filter_map(|(rp, cp)| place(rp, cp, ci.item)) + .filter(|(rp, cp)| !skip(rp, cp)) + .collect(); + if contexts.len().saturating_mul(funcs.len().max(1)) > MAX_CALC_RESULTS { + return Err("The PivotTable has too many calculated item values.".into()); + } + let mut results: Vec<(Vec, Vec, Vec>)> = Vec::with_capacity(contexts.len()); + for (rp, cp) in contexts { + let leaf = |item: Option| item.and_then(|it| place(&rp, &cp, it)).and_then(|key| aggs.get(&key)); + let vals: Vec> = funcs + .iter() + .enumerate() + .map(|(j, f)| { + let f = (*f)?; + Some(match &ci.compiled { + Ok(c) => { + let args: Vec = + ci.refs.iter().map(|r| leaf(*r).and_then(|a| a.get(j)).map_or(Value::Number(0.0), |a| a.result(f))).collect(); + crate::pivot_calc::eval(wb, c, &args) + } + Err(_) => Value::Error(CellError::Name), + }) + }) + .collect(); + results.push((rp, cp, vals)); + } + for (rp, cp, vals) in results { + for a in 0..=rp.len() { + for b in 0..=cp.len() { + let e = aggs.entry((rp[..a].to_vec(), cp[..b].to_vec())).or_insert_with(|| vec![Acc::default(); funcs.len()]); + for (acc, v) in e.iter_mut().zip(vals.iter()) { + if let Some(v) = v { + acc.add_calc(v); } - Value::number(rank as f64) } - (None, _) => Value::Empty, - _ => raw, - }, - }; - g.set(r, l + j, shown, ck.1, 0, data_fmt.get(*vi).cloned().flatten()); + } + } + row_set.insert(rp); + col_set.insert(cp); + } + if aggs.len() > 5_000_000 { + return Err("The PivotTable report is too large.".into()); } } - Ok(Output { width: g.width as u32, grid: g.rows, row_kinds: g.kinds, body_offset, chart_rows, chart_cols, label_col }) + Ok(()) +} + +/// A Show Values As base item, resolved against the base field's items. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum BaseItem { + At(u32), + Previous, + Next, + /// None given, or not among the field's items. + Missing, +} + +/// A line's or column's item path as one entry per field of its axis (`None`: a total). +fn key_items(path: &[u32], fields: usize) -> Vec> { + (0..fields).map(|i| path.get(i).copied()).collect() } // ---------------------------------------------------------------- styling and writing /// Colours for a PivotTable style name (our own palette, keyed by the familiar -/// `PivotStyleLight1…28 / Medium1…28 / Dark1…28` names). -struct Look { - header_fill: Color, - header_font: Color, - line: Color, - filter_fill: Color, +/// `PivotStyleLight1…28 / Medium1…28 / Dark1…28` names). The PivotTable Styles gallery draws +/// its swatches from it. +pub struct Look { + pub header_fill: Color, + pub header_font: Color, + pub line: Color, + pub filter_fill: Color, } -fn look(style: &str) -> Look { +pub fn look(style: &str) -> Look { let rest = style.strip_prefix("PivotStyle").unwrap_or(style); let (fam, n) = ["Light", "Medium", "Dark"] .iter() @@ -1489,6 +2206,7 @@ pub fn refresh(wb: &mut Workbook, sheet: usize, idx: usize, replace: bool) -> Re } if let Some(p) = shm.pivots.get_mut(idx) { p.last_range = Some(area); + p.report = PivotReportIndex(Some(Arc::new(out.report.clone()))); } let sheet_name = shm.name.clone(); rebind_charts(wb, &sheet_name, &pt.name, origin, &out); @@ -1590,6 +2308,16 @@ fn rebind_charts(wb: &mut Workbook, sheet: &str, pivot: &str, origin: CellRef, o } } +/// The item label cells of a PivotTable's report, at their sheet cells (an empty list when the +/// report can't be computed). +pub fn report_items(wb: &Workbook, sheet: usize, idx: usize) -> Vec<(CellRef, ItemCell)> { + let Some(pt) = wb.sheet(sheet).and_then(|s| s.pivots.get(idx)) else { return vec![] }; + let Ok(out) = compute(wb, pt, sheet) else { return vec![] }; + let k = out.body_offset; + let origin_row = if pt.anchor.row >= k { pt.anchor.row - k } else { pt.anchor.row }; + out.items.into_iter().filter_map(|it| Some((CellRef::new(origin_row.checked_add(it.row)?, pt.anchor.col.checked_add(it.col)?), it))).collect() +} + /// Clears a pivot's last output area. pub fn clear_output(wb: &mut Workbook, sheet: usize, idx: usize) { let Some(shm) = wb.sheet_mut(sheet) else { return }; @@ -1597,7 +2325,86 @@ pub fn clear_output(wb: &mut Workbook, sheet: usize, idx: usize) { shm.take_cells(r); if let Some(p) = shm.pivots.get_mut(idx) { p.last_range = None; + p.report = PivotReportIndex::default(); + } +} + +/// Indexes the reports of PivotTables read from a file: their cells are on the sheet, but no +/// report was written in this session. A PivotTable is indexed only when its report, computed +/// again, has its values exactly where the sheet has them; until a refresh, GETPIVOTDATA gives +/// `#REF!` for one whose report doesn't match (a stale or differently laid out file). +pub fn index_reports(wb: &mut Workbook) { + let mut found: Vec<(usize, usize, PivotReport)> = Vec::new(); + for (si, sh) in wb.sheets.iter().enumerate() { + for (pi, pt) in sh.pivots.iter().enumerate() { + let Some(area) = pt.last_range else { continue }; + if pt.report.get().is_some() { + continue; + } + let Ok(out) = compute(wb, pt, si) else { continue }; + if out.height().max(1) != area.height() || out.width.max(1) != area.width() || !report_matches(sh, area.start, &out) { + continue; + } + found.push((si, pi, out.report)); + } + } + for (si, pi, report) in found { + if let Some(p) = wb.sheet_mut(si).and_then(|sh| sh.pivots.get_mut(pi)) { + p.report = PivotReportIndex(Some(Arc::new(report))); + } + } +} + +/// Whether every value cell of a computed report holds, on the sheet, the value it computes. +fn report_matches(sh: &Sheet, origin: CellRef, out: &Output) -> bool { + let same = |a: &Value, b: &Value| match (a, b) { + (Value::Number(x), Value::Number(y)) => (x - y).abs() <= 1e-9 * x.abs().max(y.abs()).max(1.0), + _ => a == b, + }; + out.report.rows.iter().all(|r| { + out.report.cols.iter().all(|c| { + let (Some(row), Some(col)) = (origin.row.checked_add(r.offset), origin.col.checked_add(c.offset)) else { return false }; + same(&out.value(r.offset as usize, c.offset as usize), &sh.value(CellRef::new(row, col))) + }) + }) +} + +// ---------------------------------------------------------------- GETPIVOTDATA + +/// The formula (without `=`) Excel's Generate GetPivotData writes for a value cell of a +/// PivotTable: `GETPIVOTDATA("Sum of Sales",$A$3,"Region","East")`, naming the value field by its +/// caption, the PivotTable by its top-left cell and each row and column item of the cell. Dates +/// are written with DATE. `None` when `cell` on `sheet` isn't a value cell of a report. The +/// formula is for a cell on `from_sheet`, in `locale`'s syntax. +pub fn getpivotdata_formula(wb: &Workbook, from_sheet: usize, sheet: usize, cell: CellRef, locale: FormulaLocale) -> Option { + let sh = wb.sheet(sheet)?; + let pt = sh.pivots.iter().find(|p| p.last_range.is_some_and(|r| r.contains(cell)))?; + let origin = pt.last_range?.start; + let report = pt.report.get()?; + let (data, pairs) = report.value_cell(cell.row.checked_sub(origin.row)?, cell.col.checked_sub(origin.col)?)?; + let sep = locale.list_separator(); + let text = |t: &str| format!("\"{}\"", t.replace('"', "\"\"")); + let number = |n: f64| { + let t = gridcraft_core::number_to_text(n); + if locale.decimal_separator() == '.' { t } else { t.replace('.', &locale.decimal_separator().to_string()) } + }; + let anchor = format!("${}${}", gridcraft_core::col_to_letters(pt.anchor.col), u64::from(pt.anchor.row) + 1); + let target = if sheet == from_sheet { anchor } else { format!("{}!{anchor}", quote_sheet(&sh.name)) }; + let mut out = format!("{}({}{sep}{target}", locale.function_name("GETPIVOTDATA"), text(&report.data_fields.get(data as usize)?.name)); + for (field, item) in pairs { + let literal = match &item.value { + Value::Number(n) if field.dates && n.fract() == 0.0 => match datetime_from_serial(wb.date_system, *n) { + Some(d) => format!("{}({}{sep}{}{sep}{})", locale.function_name("DATE"), d.year, d.month, d.day), + None => number(*n), + }, + Value::Number(n) => number(*n), + Value::Bool(b) => locale.function_name(if *b { "TRUE" } else { "FALSE" }).to_string(), + _ => text(&item.label), + }; + out.push_str(&format!("{sep}{}{sep}{literal}", text(&field.name))); } + out.push(')'); + Some(out) } #[cfg(test)] diff --git a/crates/engine/src/pivot_calc.rs b/crates/engine/src/pivot_calc.rs index 014ad2db8..cac93ab7c 100644 --- a/crates/engine/src/pivot_calc.rs +++ b/crates/engine/src/pivot_calc.rs @@ -35,6 +35,12 @@ pub struct Compiled { /// Parses a calculated-field formula. `field` maps a written name to the canonical name of a /// source or calculated field. pub fn compile(formula: &str, field: &dyn Fn(&str) -> Option) -> Result { + compile_named(formula, field, "field") +} + +/// [`compile`] for names of `noun`s (`field`, or `item` for calculated items), which the error +/// messages name. +pub fn compile_named(formula: &str, field: &dyn Fn(&str) -> Option, noun: &str) -> Result { let text = formula.trim(); let text = text.strip_prefix('=').unwrap_or(text).trim(); if text.is_empty() { @@ -99,7 +105,7 @@ pub fn compile(formula: &str, field: &dyn Fn(&str) -> Option) -> Result< } } } - let canon = field(&name).ok_or_else(|| format!("There's no field named '{name}'."))?; + let canon = field(&name).ok_or_else(|| format!("There's no {noun} named '{name}'."))?; out.push_str(&slot(canon, &mut refs)); } c if c.is_ascii_digit() || (c == '.' && at(i + 1).is_some_and(|d| d.is_ascii_digit())) => { @@ -151,8 +157,8 @@ pub fn compile(formula: &str, field: &dyn Fn(&str) -> Option) -> Result< } match e { Expr::Name(n) if n.starts_with(PREFIX) => {} - Expr::Name(n) => problem = Some(format!("There's no field named '{n}'.")), - Expr::Ref(_) | Expr::Struct(_) => problem = Some("A calculated field can't refer to cells or tables; use field names.".into()), + Expr::Name(n) => problem = Some(format!("There's no {noun} named '{n}'.")), + Expr::Ref(_) | Expr::Struct(_) => problem = Some(format!("A calculated {noun} can't refer to cells or tables; use {noun} names.")), _ => {} } }); diff --git a/crates/engine/src/pivot_items.rs b/crates/engine/src/pivot_items.rs new file mode 100644 index 000000000..85f14f301 --- /dev/null +++ b/crates/engine/src/pivot_items.rs @@ -0,0 +1,186 @@ +//! Calculated PivotTable items: a formula over the items of one field (`=East+West`, +//! `=East*1.1`, `='North East'/2`, `=Region[East]-Region['North East']`). +//! +//! As Microsoft documents for Excel, the formula runs on each item's summary for every +//! combination of the other fields' items, and its results then take part in subtotals and grand +//! totals like any item's. Items are written bare, in single quotes, or qualified by their field +//! (`Field[Item]`); only the calculated item's own field may be named. Cell references and +//! defined names are not allowed. The formula machinery is the calculated fields' (see +//! [`crate::pivot_calc`]). + +use crate::pivot_calc::{self, Compiled}; + +/// Most calculated items per PivotTable. +pub const MAX_CALC_ITEMS: usize = 256; + +/// Rewrites `Field[Item]` and `Field['Item']` references to the calculated item's own `field` +/// as quoted item names (`'Item'`). Any other bracket reference is an error. +fn unqualify(formula: &str, field: &str) -> Result { + let chars: Vec = formula.chars().collect(); + let at = |i: usize| chars.get(i).copied(); + let mut out = String::with_capacity(formula.len()); + let mut i = 0; + // A quoted run (`'…'` with `''` for a quote), returned with the index after it. + let quoted = |mut i: usize| -> Result<(String, usize), String> { + let mut name = String::new(); + i += 1; + loop { + match at(i) { + None => return Err("The formula has an unterminated item name.".into()), + Some('\'') if at(i + 1) == Some('\'') => { + name.push('\''); + i += 2; + } + Some('\'') => return Ok((name, i + 1)), + Some(c) => { + name.push(c); + i += 1; + } + } + } + }; + let quote = |name: &str| format!("'{}'", name.replace('\'', "''")); + while let Some(c) = at(i) { + match c { + '"' => { + // Text constants pass through untouched. + out.push('"'); + i += 1; + while let Some(x) = at(i) { + out.push(x); + i += 1; + if x == '"' { + if at(i) == Some('"') { + out.push('"'); + i += 1; + } else { + break; + } + } + } + } + '\'' => { + let (name, next) = quoted(i)?; + if at(next) == Some('[') { + if !name.eq_ignore_ascii_case(field) { + return Err(format!("A calculated item in \"{field}\" can only use items of that field; '{name}' is another field.")); + } + bracket(&chars, next, &mut out, "e)?; + i = skip_bracket(&chars, next); + } else { + out.push_str("e(&name)); + i = next; + } + } + '[' => return Err("The formula contains an error.".into()), + c if c.is_alphabetic() || c == '_' => { + let start = i; + while at(i).is_some_and(|d| d.is_alphanumeric() || d == '_' || d == '.') { + i += 1; + } + let word: String = chars.get(start..i).map(|w| w.iter().collect()).unwrap_or_default(); + if at(i) == Some('[') { + if !word.eq_ignore_ascii_case(field) { + return Err(format!("A calculated item in \"{field}\" can only use items of that field; '{word}' is another field.")); + } + bracket(&chars, i, &mut out, "e)?; + i = skip_bracket(&chars, i); + } else { + out.push_str(&word); + } + } + c => { + out.push(c); + i += 1; + } + } + } + Ok(out) +} + +/// Writes the item of the bracket starting at `open` (`[East]`, `['North East']`) as a quoted +/// name. Positions (`[1]`, `[-1]`) are refused. +fn bracket(chars: &[char], open: usize, out: &mut String, quote: &dyn Fn(&str) -> String) -> Result { + let close = skip_bracket(chars, open); + let inner: String = chars.get(open + 1..close.saturating_sub(1)).map(|w| w.iter().collect()).unwrap_or_default(); + if chars.get(close.saturating_sub(1)) != Some(&']') || close <= open + 1 { + return Err("The formula has an unterminated item reference.".into()); + } + let inner = inner.trim(); + let name = match inner.strip_prefix('\'').and_then(|x| x.strip_suffix('\'')) { + Some(q) => q.replace("''", "'"), + None => inner.to_string(), + }; + if name.is_empty() { + return Err("The formula contains an error.".into()); + } + if name.trim_start_matches(['+', '-']).chars().all(|c| c.is_ascii_digit()) { + return Err("Refer to items by name; item positions are not supported.".into()); + } + out.push_str("e(&name)); + Ok(close) +} + +/// The index just past the `]` closing the bracket opened at `open` (quotes inside are skipped); +/// the end of the text when it isn't closed. +fn skip_bracket(chars: &[char], open: usize) -> usize { + let mut i = open + 1; + let mut in_quote = false; + while let Some(c) = chars.get(i) { + match c { + '\'' => in_quote = !in_quote, + ']' if !in_quote => return i + 1, + _ => {} + } + i += 1; + } + chars.len() +} + +/// Parses a calculated item's formula. `item` maps a written name to the canonical caption of an +/// item of `field` (data items and calculated items). +pub fn compile(formula: &str, field: &str, item: &dyn Fn(&str) -> Option) -> Result { + let text = formula.trim(); + let text = text.strip_prefix('=').unwrap_or(text); + if text.len() > pivot_calc::MAX_FORMULA_LEN { + return Err("The formula is too long.".into()); + } + let plain = unqualify(text, field)?; + pivot_calc::compile_named(&plain, item, "item") +} + +#[cfg(test)] +mod tests { + use super::*; + use gridcraft_core::Value; + use gridcraft_model::Workbook; + + fn items(n: &str) -> Option { + ["East", "West", "North East", "Dave's"].iter().find(|f| f.eq_ignore_ascii_case(n.trim())).map(|f| f.to_string()) + } + + #[test] + fn compiles_item_names() { + let c = compile("=East+west", "Region", &items).unwrap(); + assert_eq!(c.refs, vec!["East", "West"]); + let c = compile("= Region[East]*2 + 'North East' - Region['Dave''s']", "Region", &items).unwrap(); + assert_eq!(c.refs, vec!["East", "North East", "Dave's"]); + let wb = Workbook::new(); + assert_eq!(pivot_calc::eval(&wb, &c, &[Value::Number(10.0), Value::Number(1.0), Value::Number(4.0)]), Value::Number(17.0)); + let c = compile("=SUM(East,West)/2", "Region", &items).unwrap(); + assert_eq!(pivot_calc::eval(&wb, &c, &[Value::Number(10.0), Value::Number(4.0)]), Value::Number(7.0)); + // Refusals: another field, positions, unknown items, cells, broken syntax. + assert!(compile("=Product[Apples]", "Region", &items).unwrap_err().contains("Product")); + assert!(compile("=Region[1]", "Region", &items).is_err()); + assert!(compile("=Region[-1]", "Region", &items).is_err()); + assert!(compile("=Foo+1", "Region", &items).unwrap_err().contains("item named 'Foo'")); + assert!(compile("=A1*2", "Region", &items).is_err()); + assert!(compile("=East*(", "Region", &items).is_err()); + assert!(compile("=Region[East", "Region", &items).is_err()); + assert!(compile("='East", "Region", &items).is_err()); + assert!(compile("=[East]", "Region", &items).is_err()); + assert!(compile("=", "Region", &items).is_err()); + // Text constants keep their brackets and quotes. + assert!(compile("=IF(East>0,\"a[b]'c\",West)", "Region", &items).is_ok()); + } +} diff --git a/crates/engine/tests/getpivotdata.rs b/crates/engine/tests/getpivotdata.rs new file mode 100644 index 000000000..794b3fa4a --- /dev/null +++ b/crates/engine/tests/getpivotdata.rs @@ -0,0 +1,321 @@ +//! GETPIVOTDATA end to end: PivotTables built with the pivot commands, formulas entered with +//! `cell.set`, results checked against Excel's documented and observed behaviour. + +use gridcraft_engine::Session; +use gridcraft_engine::core::{CellError, CellRef, Value}; +use gridcraft_engine::formula::FormulaLocale; +use serde_json::{Value as Json, json}; + +const REF: Value = Value::Error(CellError::Ref); + +/// Region, Product, Date, Units, Sales. East: Apples 120, Pears 60; West: Apples 140, Plums 50; +/// North: Pears 80. 450 in all, 22 units. +fn sales() -> Session { + let mut s = Session::new(); + s.new_workbook(); + let rows = json!([ + ["Region", "Product", "Date", "Units", "Sales"], + ["East", "Apples", "1/15/2026", 5, 100], + ["East", "Pears", "2/10/2026", 3, 60], + ["West", "Apples", "1/20/2026", 7, 140], + ["West", "Plums", "3/15/2026", 2, 50], + ["North", "Pears", "2/25/2026", 4, 80], + ["East", "Apples", "3/5/2026", 1, 20] + ]); + s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap(); + s +} + +/// A PivotTable on a new sheet (Sheet2, report at A3), which becomes the active sheet. +fn pivot(p: Json) -> Session { + let mut s = sales(); + let mut p = p; + p["source"] = json!("Sheet1!$A$1:$E$7"); + s.execute("insert.pivotTable", p).unwrap(); + s +} + +fn at(a: &str) -> CellRef { + CellRef::parse(a).unwrap() +} + +/// Enters `formula` in Z1 of the active sheet and returns its value. +fn eval(s: &mut Session, formula: &str) -> Value { + s.execute("cell.set", json!({"cell": "Z1", "input": formula})).unwrap(); + s.doc().unwrap().wb.active().unwrap().value(at("Z1")) +} + +fn n(v: f64) -> Value { + Value::Number(v) +} + +#[test] +fn grand_total_items_and_subtotals() { + let mut s = pivot(json!({"rows": ["Region", "Product"], "values": ["Sales"]})); + // The value field by its caption or its source field, case ignored. + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sum of Sales",$A$3)"#), n(450.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("sales",$A$3)"#), n(450.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("SUM OF SALES",B7)"#), n(450.0), "any cell of the report"); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",A2:A3)"#), n(450.0), "a range that reaches into the report"); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Units",$A$3)"#), REF, "not a value field"); + // Items: row items, an outer item's subtotal, pairs in any order, items by reference. + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","East")"#), n(180.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"region","EAST","Product","apples")"#), n(120.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Product","Pears","Region","North")"#), n(80.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region",A4)"#), n(180.0), "A4 holds East"); + // An inner item alone: found when it belongs to one outer item only (Plums: West). + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Product","Plums")"#), n(50.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Product","Apples")"#), REF, "East and West sell Apples"); + // Unknown items and fields, items not in the report, odd pairs, no PivotTable. + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","South")"#), REF); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","East","Product","Plums")"#), REF); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Color","East")"#), REF); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region")"#), REF); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$H$30)"#), REF); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales","A3")"#), REF); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",Sheet1!A1)"#), REF, "the source isn't the report"); + // Errors in the arguments come through. + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA(NA(),$A$3)"#), Value::Error(CellError::NA)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region",1/0)"#), Value::Error(CellError::Div0)); + // From another sheet. + s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap(); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",Sheet2!$A$3,"Region","West")"#), n(190.0)); +} + +#[test] +fn rows_and_columns_and_empty_cells() { + let mut s = pivot(json!({"rows": ["Region"], "columns": ["Product"], "values": ["Sales"]})); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","West","Product","Apples")"#), n(140.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Product","Apples")"#), n(260.0), "column total"); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","North")"#), n(80.0), "row total"); + // East sells no Plums: the report shows an empty cell, which counts as 0. + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","East","Product","Plums")"#), n(0.0)); + // Without the row totals, North's total isn't there; Plums' column total still is. + s.execute("pivot.grandTotals", json!({"mode": "columns"})).unwrap(); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","North")"#), REF); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Product","Plums")"#), n(50.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3)"#), REF); +} + +#[test] +fn totals_turned_off_collapsed_and_filtered_items_are_ref() { + let mut s = pivot(json!({"rows": ["Region", "Product"], "values": ["Sales"]})); + let east = r#"=GETPIVOTDATA("Sales",$A$3,"Region","East")"#; + s.execute("pivot.subtotals", json!({"mode": "none"})).unwrap(); + assert_eq!(eval(&mut s, east), REF, "no subtotal shown"); + s.execute("pivot.subtotals", json!({"mode": "bottom"})).unwrap(); + assert_eq!(eval(&mut s, east), n(180.0), "subtotal at the bottom of the group"); + s.execute("pivot.grandTotals", json!({"mode": "off"})).unwrap(); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3)"#), REF); + s.execute("pivot.grandTotals", json!({"mode": "on"})).unwrap(); + // Collapsed: the region's total shows, its products don't. + s.execute("pivot.collapse", json!({"field": "Region", "items": ["East"]})).unwrap(); + assert_eq!(eval(&mut s, east), n(180.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","East","Product","Apples")"#), REF); + s.execute("pivot.collapse", json!({"field": "Region", "collapse": false})).unwrap(); + // Hidden items aren't in the report (North sells only Pears, so it goes too), and the + // totals count what is shown. + s.execute("pivot.filter", json!({"field": "Product", "selected": ["Apples", "Plums"]})).unwrap(); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Product","Pears","Region","East")"#), REF); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","North")"#), REF); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3)"#), n(310.0)); +} + +#[test] +fn report_filters() { + let mut s = pivot(json!({"rows": ["Product"], "values": ["Sales"], "filters": ["Region"]})); + // (All): the whole data, but no single region to name. + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3)"#), n(450.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","East")"#), REF); + s.execute("pivot.filter", json!({"field": "Region", "selected": ["East"]})).unwrap(); + // The report body now starts lower; any cell of the report still names it. + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$1)"#), n(180.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","east")"#), n(180.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","West")"#), REF); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Product","Apples","Region","East")"#), n(120.0)); +} + +#[test] +fn dates_numbers_and_grouped_items() { + let mut s = pivot(json!({"rows": ["Date"], "values": ["Sales"]})); + // Dates by serial or DATE; 46037 is 15 Jan 2026. + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Date",DATE(2026,1,15))"#), n(100.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Date",46037)"#), n(100.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Date",46037.5)"#), REF); + s.execute("pivot.group", json!({"field": "Date", "by": "months"})).unwrap(); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Date","Jan")"#), n(240.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Date","Mar")"#), n(70.0)); + s.execute("pivot.removeField", json!({"field": "Date", "area": "rows"})).unwrap(); + s.execute("pivot.addField", json!({"field": "Units", "area": "rows"})).unwrap(); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Units",5)"#), n(100.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Units","5")"#), n(100.0), "text names the item's label"); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Units",6)"#), REF); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Units",TRUE)"#), REF); +} + +#[test] +fn value_fields_captions_and_show_values_as() { + let mut s = pivot(json!({"rows": ["Region"], "values": ["Sales", "Units"]})); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Units",$A$3)"#), n(22.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sum of Units",$A$3,"Region","West")"#), n(9.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3,"Region","West")"#), n(190.0)); + // Σ Values in rows. + s.execute("pivot.moveField", json!({"field": "Σ Values", "from": "columns", "to": "rows"})).unwrap(); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Units",$A$3,"Region","West")"#), n(9.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3)"#), n(450.0)); + // A custom caption replaces the automatic one; the source field still names it. + s.execute("pivot.valueSettings", json!({"field": "Sum of Sales", "name": "Revenue"})).unwrap(); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Revenue",$A$3)"#), n(450.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sales",$A$3)"#), n(450.0)); + assert_eq!(eval(&mut s, r#"=GETPIVOTDATA("Sum of Sales",$A$3)"#), REF); + // Show Values As: the value as shown. + s.execute("pivot.valueSettings", json!({"field": "Revenue", "showAs": "percentOfGrandTotal"})).unwrap(); + let share = eval(&mut s, r#"=GETPIVOTDATA("Revenue",$A$3,"Region","East")"#).as_f64().unwrap(); + assert!((share - 0.4).abs() < 1e-12, "{share}"); +} + +#[test] +fn follows_refresh_and_source_changes() { + let mut s = pivot(json!({"rows": ["Region"], "values": ["Sales"]})); + let east = r#"=GETPIVOTDATA("Sales",$A$3,"Region","East")"#; + assert_eq!(eval(&mut s, east), n(180.0)); + // Editing the source changes nothing until the PivotTable is refreshed, as in Excel. + s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap(); + s.execute("cell.set", json!({"cell": "E2", "input": "1000"})).unwrap(); + s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap(); + let z1 = |s: &Session| s.doc().unwrap().wb.active().unwrap().value(at("Z1")); + assert_eq!(z1(&s), n(180.0)); + s.execute("pivot.refresh", json!({})).unwrap(); + assert_eq!(z1(&s), n(1080.0), "recalculated by the refresh"); + // A new source with a new region. + s.execute("sheet.activate", json!({"sheet": "Sheet1"})).unwrap(); + s.execute("range.setValues", json!({"range": "A8", "values": [["South", "Plums", "4/1/2026", 3, 33]]})).unwrap(); + s.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap(); + s.execute("cell.set", json!({"cell": "Z2", "input": r#"=GETPIVOTDATA("Sales",$A$3,"Region","South")"#})).unwrap(); + let z2 = |s: &Session| s.doc().unwrap().wb.active().unwrap().value(at("Z2")); + assert_eq!(z2(&s), REF); + s.execute("pivot.changeSource", json!({"source": "Sheet1!$A$1:$E$8"})).unwrap(); + assert_eq!(z2(&s), n(33.0)); + // Moving the field to the columns keeps the formulas right; undo brings the old report back. + s.execute("pivot.moveField", json!({"field": "Region", "from": "rows", "to": "columns"})).unwrap(); + assert_eq!(z1(&s), n(1080.0)); + s.execute("edit.undo", json!({})).unwrap(); + s.execute("edit.undo", json!({})).unwrap(); + assert_eq!(z2(&s), REF); + // Deleting the PivotTable leaves nothing to read. + s.execute("pivot.delete", json!({})).unwrap(); + assert_eq!(z1(&s), REF); +} + +/// Every value cell of a report: the formula Generate GetPivotData writes for it gives the +/// cell's value back. +fn check_generated_formulas(s: &mut Session) -> usize { + let (wb, sheet) = { + let d = s.doc().unwrap(); + (d.wb.clone(), d.wb.active_sheet) + }; + let pt = wb.active().unwrap().pivots[0].clone(); + let area = pt.last_range.unwrap(); + let mut n = 0; + for c in area.iter() { + let Some(f) = gridcraft_engine::pivot::getpivotdata_formula(&wb, sheet, sheet, c, FormulaLocale::En) else { continue }; + let want = match wb.active().unwrap().value(c) { + Value::Empty => Value::Number(0.0), + v => v, + }; + assert_eq!(eval(s, &format!("={f}")), want, "{} {f}", c.a1()); + n += 1; + } + n +} + +#[test] +fn generated_formulas_read_back_their_cell_in_every_layout() { + for (p, layouts) in [ + (json!({"rows": ["Region", "Product"], "values": ["Sales"]}), &["compact", "outline", "tabular"][..]), + (json!({"rows": ["Region"], "columns": ["Product"], "values": ["Sales", "Units"]}), &["compact", "tabular"][..]), + (json!({"rows": ["Region", "Product"], "columns": ["Date"], "values": ["Sales"]}), &["outline"][..]), + (json!({"rows": ["Product"], "values": ["Sales"], "filters": ["Region"]}), &["compact"][..]), + ] { + let mut s = pivot(p.clone()); + for layout in layouts { + s.execute("pivot.reportLayout", json!({"layout": layout})).unwrap(); + assert!(check_generated_formulas(&mut s) > 0, "{p} {layout}"); + } + s.execute("pivot.subtotals", json!({"mode": "bottom"})).unwrap(); + check_generated_formulas(&mut s); + } + // Two value fields in rows, a date field grouped by quarters in the columns. + let mut s = pivot(json!({"rows": ["Region"], "columns": ["Date"], "values": ["Sales", "Units"]})); + s.execute("pivot.moveField", json!({"field": "Σ Values", "from": "columns", "to": "rows", "position": 0})).unwrap(); + s.execute("pivot.group", json!({"field": "Date", "by": "quarters"})).unwrap(); + assert!(check_generated_formulas(&mut s) > 0); +} + +#[test] +fn generated_formula_text_is_excels() { + let mut s = pivot(json!({"rows": ["Region", "Date"], "values": ["Sales"]})); + let d = s.doc().unwrap(); + let (wb, sh) = (d.wb.clone(), d.wb.active_sheet); + let f = |c: &str, from: usize, locale| gridcraft_engine::pivot::getpivotdata_formula(&wb, from, sh, at(c), locale); + // A4 East (subtotal on top), A5 its first date. + assert_eq!(f("B4", sh, FormulaLocale::En).as_deref(), Some(r#"GETPIVOTDATA("Sum of Sales",$A$3,"Region","East")"#)); + assert_eq!(f("B5", sh, FormulaLocale::En).as_deref(), Some(r#"GETPIVOTDATA("Sum of Sales",$A$3,"Region","East","Date",DATE(2026,1,15))"#)); + assert_eq!(f("B5", 1, FormulaLocale::En).as_deref(), Some(r#"GETPIVOTDATA("Sum of Sales",Sheet2!$A$3,"Region","East","Date",DATE(2026,1,15))"#)); + assert_eq!(f("B5", sh, FormulaLocale::Es).as_deref(), Some(r#"GETPIVOTDATA("Sum of Sales";$A$3;"Region";"East";"Date";DATE(2026;1;15))"#)); + let last = wb.active().unwrap().pivots[0].last_range.unwrap().end; + assert_eq!(f(&last.a1(), sh, FormulaLocale::En).as_deref(), Some(r#"GETPIVOTDATA("Sum of Sales",$A$3)"#)); + assert_eq!(f("A4", sh, FormulaLocale::En), None, "a label isn't a value"); + assert_eq!(f("B3", sh, FormulaLocale::En), None, "nor a header"); + assert_eq!(f("C4", sh, FormulaLocale::En), None, "outside the report"); + let _ = &mut s; +} + +#[test] +fn xlsx_round_trip_keeps_and_evaluates_getpivotdata() { + let mut s = pivot(json!({"rows": ["Region", "Product"], "values": ["Sales"]})); + s.execute("cell.set", json!({"cell": "Z1", "input": r#"=GETPIVOTDATA("Sales",$A$3,"Region","West")"#})).unwrap(); + s.execute("cell.set", json!({"cell": "Z2", "input": r#"=GETPIVOTDATA("Sum of Sales",$A$3,"Product","Plums")"#})).unwrap(); + let b64 = s.execute("file.saveBytes", json!({"format": "xlsx"})).unwrap()["base64"].as_str().unwrap().to_string(); + let mut t = Session::new(); + t.execute("file.open", json!({"name": "pivot.xlsx", "base64": b64})).unwrap(); + let d = t.doc().unwrap(); + let si = d.wb.sheet_index("Sheet2").unwrap(); + let sh = d.wb.sheet(si).unwrap(); + assert_eq!(sh.cell(at("Z1")).unwrap().formula.as_ref().unwrap().text, r#"GETPIVOTDATA("Sales",$A$3,"Region","West")"#); + assert_eq!(sh.value(at("Z1")), n(190.0), "evaluated on open"); + assert_eq!(sh.value(at("Z2")), n(50.0)); + // Still right after a refresh in the reopened workbook. + t.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap(); + t.execute("selection.set", json!({"range": "A3"})).unwrap(); + t.execute("pivot.refresh", json!({})).unwrap(); + assert_eq!(t.doc().unwrap().wb.active().unwrap().value(at("Z1")), n(190.0)); +} + +#[test] +fn a_report_that_no_longer_matches_its_file_isnt_read() { + // A workbook whose report cells don't hold what the PivotTable computes (a stale file): + // GETPIVOTDATA gives #REF! until the PivotTable is refreshed. + let mut s = pivot(json!({"rows": ["Region"], "values": ["Sales"]})); + let mut wb = (*s.doc().unwrap().wb).clone(); + let si = wb.active_sheet; + let sh = wb.sheet_mut(si).unwrap(); + sh.pivots[0].report = Default::default(); + sh.cells.set(at("B4"), gridcraft_engine::model::Cell::value(n(1.0))); + sh.cells.set( + at("Z1"), + gridcraft_engine::model::Cell { + formula: Some(std::sync::Arc::new(gridcraft_engine::model::Formula::new(r#"GETPIVOTDATA("Sales",$A$3)"#))), + ..Default::default() + }, + ); + let b64 = gridcraft_engine::io::base64_encode(&gridcraft_engine::io::save_bytes(&wb, "stale.xlsx").unwrap()); + let mut t = Session::new(); + t.execute("file.open", json!({"name": "stale.xlsx", "base64": b64})).unwrap(); + t.execute("sheet.activate", json!({"sheet": "Sheet2"})).unwrap(); + assert_eq!(t.doc().unwrap().wb.active().unwrap().value(at("Z1")), REF); + t.execute("selection.set", json!({"range": "A3"})).unwrap(); + t.execute("pivot.refresh", json!({})).unwrap(); + assert_eq!(t.doc().unwrap().wb.active().unwrap().value(at("Z1")), n(450.0)); + let _ = &mut s; +} diff --git a/crates/model/src/features.rs b/crates/model/src/features.rs index b51f7bc98..87ef0cb42 100644 --- a/crates/model/src/features.rs +++ b/crates/model/src/features.rs @@ -661,6 +661,70 @@ pub struct Slicer { pub columns: u32, } +/// The time level a timeline shows its periods at. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] +pub enum TimelineLevel { + Years, + Quarters, + #[default] + Months, + Days, +} + +/// A timeline: an on-sheet filter of the date field of one or more PivotTables, by one +/// contiguous period at a time level (Insert › Timeline). The PivotTables it is connected to show +/// only the records whose date falls in the selection. As in Excel, the selection is kept in +/// whole days and snapped to whole periods of the level it was made at. +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +#[serde(default)] +pub struct Timeline { + pub id: u32, + /// The timeline's name (unique in the workbook, `Date`, `Date 1`…). + pub name: String, + /// Caption in the header. + pub caption: String, + /// The source field (a date column) it filters. + pub field: String, + /// Ids of the PivotTables it filters (Report Connections). + pub pivots: Vec, + pub anchor: Anchor, + pub level: TimelineLevel, + /// First and last day of the selection, as date serials (inclusive); `None` = all periods. + pub selection: Option<(f64, f64)>, + /// The level the selection was made at (the selection label follows it). + pub selection_level: TimelineLevel, + /// Leftmost date shown (the scroll position); `None` = the start of the data. + pub scroll: Option, + /// Timeline style name (`TimeSlicerStyleLight1`). + pub style: String, + pub show_header: bool, + pub show_selection_label: bool, + pub show_time_level: bool, + pub show_scrollbar: bool, +} + +impl Default for Timeline { + fn default() -> Self { + Timeline { + id: 0, + name: String::new(), + caption: String::new(), + field: String::new(), + pivots: vec![], + anchor: Anchor::default(), + level: TimelineLevel::Months, + selection: None, + selection_level: TimelineLevel::Months, + scroll: None, + style: "TimeSlicerStyleLight1".into(), + show_header: true, + show_selection_label: true, + show_time_level: true, + show_scrollbar: true, + } + } +} + #[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Serialize, Deserialize)] pub enum Orientation { #[default] @@ -908,6 +972,119 @@ pub enum PivotDateGroup { Quarters, Months, Days, + Hours, + Minutes, + Seconds, +} + +impl PivotDateGroup { + /// The date and time levels from finest to coarsest, as the Grouping dialog lists them. + pub const LEVELS: [PivotDateGroup; 7] = [ + PivotDateGroup::Seconds, + PivotDateGroup::Minutes, + PivotDateGroup::Hours, + PivotDateGroup::Days, + PivotDateGroup::Months, + PivotDateGroup::Quarters, + PivotDateGroup::Years, + ]; + + /// Position in [`Self::LEVELS`] (0 = finest); `None` for no grouping. + pub fn fineness(self) -> Option { + Self::LEVELS.iter().position(|g| *g == self) + } +} + +/// The Grouping dialog's bounds and day span for a date or time grouping: values before +/// `start` or after `end` fall into one `end` item, and with Days, `days` days +/// make one item (`1/1/2026 - 1/7/2026`). `None` bounds are automatic (the first and last date). +#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize)] +pub struct PivotDateRange { + pub start: Option, + pub end: Option, + /// Number of days per item when grouping by days (1 = one item per calendar day). + pub days: u32, +} + +impl PivotDateRange { + /// Most days one item may span (Excel's Number of days goes up to 32,767). + pub const MAX_DAYS: u32 = 32_767; +} + +impl Default for PivotDateRange { + fn default() -> Self { + PivotDateRange { start: None, end: None, days: 1 } + } +} + +/// Grouping of a number field into ranges of equal width (Group › Starting at, Ending at, By). +/// Values below `start` fall into one `end` item. +#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize)] +pub struct PivotNumberGroup { + pub start: f64, + pub end: f64, + /// Width of each range; positive. + pub by: f64, +} + +impl PivotNumberGroup { + /// Most ranges a grouping may make (Excel's own limit on group items is lower). + pub const MAX_RANGES: f64 = 100_000.0; + + /// The grouping, if its bounds and width are usable: finite, `start <= end`, a positive + /// width and at most [`Self::MAX_RANGES`] ranges. + pub fn checked(start: f64, end: f64, by: f64) -> Option { + let ok = start.is_finite() && end.is_finite() && by.is_finite() && by > 0.0 && start <= end && (end - start) / by < Self::MAX_RANGES; + ok.then_some(PivotNumberGroup { start, end, by }) + } + + /// The position of the range that holds `n` (0 for the range starting at `start`). + pub fn range_index(&self, n: f64) -> i64 { + let by = if self.by > 0.0 && self.by.is_finite() { self.by } else { 1.0 }; + let k = ((n - self.start) / by).floor(); + if k.is_finite() { k.clamp(-1e15, 1e15) as i64 } else { 0 } + } + + /// The lower bound of range `k`. + pub fn range_start(&self, k: i64) -> f64 { + self.start + k as f64 * self.by + } + + /// The caption of range `k`: `0-9` when the start and width are whole numbers (the range + /// covers whole numbers up to one below the next start), otherwise `0-0.5`. + pub fn caption(&self, k: i64) -> String { + let lo = self.range_start(k); + let whole = self.start.fract() == 0.0 && self.by.fract() == 0.0 && self.by >= 1.0; + let hi = if whole { lo + self.by - 1.0 } else { lo + self.by }; + format!("{}-{}", tidy_number(lo), tidy_number(hi)) + } + + /// The caption of the item for values below `start`. + pub fn below_caption(&self) -> String { + format!("<{}", tidy_number(self.start)) + } + + /// The caption of the item for values above `end`. + pub fn above_caption(&self) -> String { + format!(">{}", tidy_number(self.end)) + } + + /// Captions of every item in order: `end`. + pub fn captions(&self) -> Vec { + let last = self.range_index(self.end).clamp(0, Self::MAX_RANGES as i64); + let mut v = Vec::with_capacity(last as usize + 3); + v.push(self.below_caption()); + v.extend((0..=last).map(|k| self.caption(k))); + v.push(self.above_caption()); + v + } +} + +/// A number as text without the binary noise of repeated additions (`0.30000000000000004`). +fn tidy_number(x: f64) -> String { + let x = if x.abs() < 1e12 { (x * 1e9).round() / 1e9 } else { x }; + // `+ 0.0` turns a rounded -0 into 0. + gridcraft_core::number_to_text(x + 0.0) } /// A field placed in the Rows or Columns area. @@ -918,8 +1095,55 @@ pub struct PivotField { pub source_col: String, pub sort: PivotSort, pub date_group: PivotDateGroup, + /// Number range grouping; when set, `date_group` is `None`. + pub number_group: Option, /// Item labels whose detail is hidden (collapsed). pub collapsed_items: Vec, + /// Bounds and day span of a date grouping (`None`: automatic bounds, one day per item). A + /// field grouped at several levels (Years, Quarters, Months) is one entry per level, each + /// carrying the same range. + pub date_range: Option, +} + +/// Group Selection: a field (`Region2`) whose items gather items of a `base` field into named +/// groups (`Group1`); base items in no group stay items of their own. Excel keeps it as an extra +/// cache field with `discretePr`. Its name is used like a source field's in the areas. +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default)] +pub struct PivotGroupField { + pub name: String, + /// The source field whose items are grouped. + pub base: String, + pub groups: Vec, +} + +/// One manual group: its caption and the base items in it. +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default)] +pub struct PivotItemGroup { + pub name: String, + pub items: Vec, +} + +impl PivotGroupField { + /// The group holding base item `label`, if any. + pub fn group_of(&self, label: &str) -> Option<&PivotItemGroup> { + self.groups.iter().find(|g| g.items.iter().any(|i| i.eq_ignore_ascii_case(label))) + } +} + +/// A calculated item: an item `name` added to `field`, whose values come from a formula over the +/// field's other items (`=East+West`, `=East*1.1`, `=Region['North East']/2`). Its place in +/// [`PivotTable::calculated_items`] is its solve order: where items of two fields meet, the later +/// formula sets the value. +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default)] +pub struct PivotCalcItem { + /// The field the item belongs to (a source field). + pub field: String, + pub name: String, + /// Formula text, starting with `=`. + pub formula: String, } /// Summarize-values-by function. @@ -947,12 +1171,70 @@ pub enum PivotShowAs { PercentOfGrandTotal, PercentOfColumnTotal, PercentOfRowTotal, - /// Running total in the first row field. + /// Running total in the base field (the first row field when none is given). RunningTotal, - /// Rank largest to smallest among sibling row items. + /// Rank largest to smallest among the base field's items. Rank, + /// As a percentage of the base field's base item. + PercentOf, + /// As a percentage of the parent row (the row one level up). + PercentOfParentRowTotal, + /// As a percentage of the parent column. + PercentOfParentColumnTotal, + /// As a percentage of the total of the base field's item that contains it. + PercentOfParentTotal, + /// Minus the base field's base item. + DifferenceFrom, + /// (value − base item) / base item. + PercentDifferenceFrom, + /// Running total in the base field, as a percentage of the field's total. + PercentRunningTotal, + /// Rank smallest to largest among the base field's items. + RankAscending, + /// (value × grand total) / (row total × column total). + Index, } +impl PivotShowAs { + /// Whether the transform needs a base field. + pub fn needs_base_field(self) -> bool { + matches!( + self, + PivotShowAs::RunningTotal + | PivotShowAs::PercentRunningTotal + | PivotShowAs::Rank + | PivotShowAs::RankAscending + | PivotShowAs::PercentOf + | PivotShowAs::PercentOfParentTotal + | PivotShowAs::DifferenceFrom + | PivotShowAs::PercentDifferenceFrom + ) + } + /// Whether it also needs a base item. + pub fn needs_base_item(self) -> bool { + matches!(self, PivotShowAs::PercentOf | PivotShowAs::DifferenceFrom | PivotShowAs::PercentDifferenceFrom) + } + /// Whether the result is a percentage. + pub fn is_percent(self) -> bool { + matches!( + self, + PivotShowAs::PercentOfGrandTotal + | PivotShowAs::PercentOfColumnTotal + | PivotShowAs::PercentOfRowTotal + | PivotShowAs::PercentOf + | PivotShowAs::PercentOfParentRowTotal + | PivotShowAs::PercentOfParentColumnTotal + | PivotShowAs::PercentOfParentTotal + | PivotShowAs::PercentDifferenceFrom + | PivotShowAs::PercentRunningTotal + ) + } +} + +/// A base item that means the previous / next item of the base field. +pub const BASE_PREVIOUS: &str = "(previous)"; +pub const BASE_NEXT: &str = "(next)"; + /// A field in the Values area. #[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] #[serde(default)] @@ -963,15 +1245,112 @@ pub struct PivotValue { pub name: String, pub show_as: PivotShowAs, pub number_format: Option, + /// Show Values As base field (source column name). + pub base_field: Option, + /// Show Values As base item: an item label, [`BASE_PREVIOUS`] or [`BASE_NEXT`]. + pub base_item: Option, +} + +/// A comparison in a label filter or a value filter. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] +pub enum PivotCompare { + Equals, + NotEquals, + BeginsWith, + NotBeginsWith, + EndsWith, + NotEndsWith, + Contains, + NotContains, + Greater, + GreaterOrEqual, + Less, + LessOrEqual, + Between, + NotBetween, } -/// A report filter (or, for a field also in Rows/Columns, its hidden-items filter). +impl PivotCompare { + /// Every comparison, in the order of Excel's Label Filters menu. + pub const ALL: [PivotCompare; 14] = [ + PivotCompare::Equals, + PivotCompare::NotEquals, + PivotCompare::BeginsWith, + PivotCompare::NotBeginsWith, + PivotCompare::EndsWith, + PivotCompare::NotEndsWith, + PivotCompare::Contains, + PivotCompare::NotContains, + PivotCompare::Greater, + PivotCompare::GreaterOrEqual, + PivotCompare::Less, + PivotCompare::LessOrEqual, + PivotCompare::Between, + PivotCompare::NotBetween, + ]; + + /// Whether the comparison takes a second value. + pub fn is_range(self) -> bool { + matches!(self, PivotCompare::Between | PivotCompare::NotBetween) + } + + /// Whether the comparison applies to text only (it has no meaning in a value filter). + pub fn is_text_only(self) -> bool { + matches!( + self, + PivotCompare::BeginsWith + | PivotCompare::NotBeginsWith + | PivotCompare::EndsWith + | PivotCompare::NotEndsWith + | PivotCompare::Contains + | PivotCompare::NotContains + ) + } +} + +/// What a Top 10 filter counts: items, a percent of the total, or a sum. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] +pub enum PivotTopBy { + Items, + Percent, + Sum, +} + +/// A filter rule on a row or column field. +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +pub enum PivotRule { + /// Label Filters: compares each item's caption with `value` (and `value2` for a range). + /// `*` and `?` are wildcards. + Label { op: PivotCompare, value: String, value2: String }, + /// Value Filters: compares each item's total of the value field named `field` (its caption). + Value { op: PivotCompare, field: String, value: f64, value2: f64 }, + /// Top 10: keeps the top (or bottom) items by the value field named `field`. + Top { top: bool, count: f64, by: PivotTopBy, field: String }, +} + +impl PivotRule { + /// The caption of the value field the rule uses, if any. + pub fn value_field(&self) -> Option<&str> { + match self { + PivotRule::Label { .. } => None, + PivotRule::Value { field, .. } | PivotRule::Top { field, .. } => Some(field), + } + } +} + +/// A report filter, an axis field's item selection or a filter rule. #[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] #[serde(default)] pub struct PivotFilter { pub source_col: String, /// Item labels shown; `None` = all. pub selected: Option>, + /// A label, value or Top 10 filter, applied in Rows/Columns. A rule on a field outside + /// those areas is dormant and preserved for when the field returns, not a report filter. + /// As in Excel without + /// "Allow multiple filters per field", a field has one kind of filter at a time: setting a + /// rule clears `selected`, and selecting items clears the rule. + pub rule: Option, } /// A PivotTable: its definition plus where its output was last written. The output itself is @@ -1009,6 +1388,21 @@ pub struct PivotTable { /// Position of Σ Values among the row fields (`None` = after the last). Only used with /// `values_on_rows`: in Columns Σ Values always comes after the last column field. pub values_position: Option, + /// Fields made by Group Selection (`Region2`), usable in the areas by name. + pub group_fields: Vec, + /// Calculated items, in solve order. + pub calculated_items: Vec, + /// What each line and data column of the written report stands for (GETPIVOTDATA reads + /// it). Written with the report; not saved, rebuilt when a workbook is opened. + #[serde(skip)] + pub report: crate::PivotReportIndex, +} + +impl PivotTable { + /// The Group Selection field named `name`, if any. + pub fn group_field(&self, name: &str) -> Option<&PivotGroupField> { + self.group_fields.iter().find(|g| g.name.eq_ignore_ascii_case(name.trim())) + } } /// A calculated field: a formula over other fields (`=Sales*0.1`, `='Unit Price'*Units`). As in @@ -1043,6 +1437,9 @@ impl Default for PivotTable { calculated_fields: vec![], values_on_rows: false, values_position: None, + group_fields: vec![], + calculated_items: vec![], + report: Default::default(), } } } diff --git a/crates/model/src/lib.rs b/crates/model/src/lib.rs index a76f30fc3..75bfc0998 100644 --- a/crates/model/src/lib.rs +++ b/crates/model/src/lib.rs @@ -9,6 +9,7 @@ pub mod cell; mod cellmap; pub mod features; pub mod password; +pub mod pivot_report; pub mod preserved; pub mod sheet; pub mod store; @@ -18,6 +19,7 @@ pub mod workbook; pub use cell::{Cell, CellPicture, Formula}; pub use features::*; pub use password::PasswordHash; +pub use pivot_report::{PivotReport, PivotReportIndex, ReportDataField, ReportField, ReportItem, ReportKey, ReportPart}; pub use preserved::{Preserved, PreservedPart, PreservedRel, PreservedSheet, PreservedXml}; pub use sheet::{DEFAULT_COL_WIDTH, DEFAULT_ROW_HEIGHT, LineIndex, Sheet, SheetViewMode, Visibility, points_to_chars}; pub use store::CellStore; diff --git a/crates/model/src/pivot_report.rs b/crates/model/src/pivot_report.rs new file mode 100644 index 000000000..367a1f41d --- /dev/null +++ b/crates/model/src/pivot_report.rs @@ -0,0 +1,376 @@ +//! What a PivotTable report shows, line by line: which value field and which items each line +//! and each data column of the written report stands for. The engine builds it with the report; +//! GETPIVOTDATA reads it ([`PivotReport::lookup`]), and the PivotTable tabs use it to name the +//! active cell's field and to write GETPIVOTDATA formulas. It is runtime only: not saved, and +//! rebuilt when a workbook is opened. + +use std::sync::Arc; + +use gridcraft_core::{CellError, Value}; + +/// A PivotTable's report index, shared cheaply between workbook snapshots. It caches what the +/// written report shows, so comparing two PivotTables ignores it. +#[derive(Clone, Debug, Default)] +pub struct PivotReportIndex(pub Option>); + +impl PartialEq for PivotReportIndex { + fn eq(&self, _: &Self) -> bool { + true + } +} + +impl PivotReportIndex { + pub fn get(&self) -> Option<&PivotReport> { + self.0.as_deref() + } +} + +/// The layout of a written PivotTable report. Offsets count from the report's top-left cell +/// (`PivotTable::last_range`'s start, report filters included). +#[derive(Clone, Debug, Default, PartialEq)] +pub struct PivotReport { + /// Value fields, in report order. + pub data_fields: Vec, + /// Row fields, outer to inner (those found in the source). + pub row_fields: Vec, + /// Column fields, outer to inner. + pub col_fields: Vec, + /// Report filters: the field and the single item it shows (`None`: (All) or several items). + pub page_fields: Vec<(String, Option)>, + /// The lines (report rows) that hold values: subtotals and grand totals included. + pub rows: Vec, + /// The data columns. + pub cols: Vec, + /// The report row of each column field's item labels. + pub col_label_rows: Vec, + /// How many columns the row labels take (one in Compact form). + pub label_cols: u32, + pub compact: bool, +} + +/// A value field: its caption ("Sum of Sales") and the source field it summarizes. +#[derive(Clone, Debug, Default, PartialEq)] +pub struct ReportDataField { + pub name: String, + pub source: String, +} + +/// A row or column field and the items the report shows for it (hidden items aren't there). +#[derive(Clone, Debug, Default, PartialEq)] +pub struct ReportField { + pub name: String, + /// Its numeric items are dates. + pub dates: bool, + pub items: Vec, +} + +/// An item: its value (a number, text, logical or error; text for grouped dates) and the label +/// the report shows. +#[derive(Clone, Debug, Default, PartialEq)] +pub struct ReportItem { + pub value: Value, + pub label: String, +} + +/// What a line or a data column stands for. +#[derive(Clone, Debug, Default, PartialEq)] +pub struct ReportKey { + /// Report row of a line, report column of a data column. + pub offset: u32, + /// The value field it shows, when that is decided here (Σ Values on this axis). + pub data: Option, + /// One entry per field of its axis: the item, or `None` for all items (a total). + pub items: Vec>, +} + +impl ReportKey { + fn specific(&self) -> usize { + self.items.iter().filter(|i| i.is_some()).count() + } +} + +/// What a report cell belongs to, for the Active Field box. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum ReportPart { + /// A value cell of this value field. + Data(u32), + /// An item label of this row field. + RowField(u32), + /// An item label of this column field. + ColField(u32), +} + +fn same_text(a: &str, b: &str) -> bool { + a == b || a.to_lowercase() == b.to_lowercase() +} + +impl ReportField { + /// The item GETPIVOTDATA's `item` argument names: a number matches a numeric item (a date's + /// serial) or an item labelled with that number; text matches an item's label, ignoring + /// case; a logical matches a logical item. + pub fn find(&self, item: &Value) -> Option { + let i = match item { + Value::Number(n) => self.items.iter().position(|it| match &it.value { + Value::Number(m) => m == n, + Value::Text(_) => it.label == gridcraft_core::number_to_text(*n), + _ => false, + }), + Value::Text(t) => self.items.iter().position(|it| same_text(&it.label, t)), + Value::Bool(b) => self.items.iter().position(|it| it.value == Value::Bool(*b)), + Value::Array(_) => return self.find(&item.scalar()), + Value::Empty | Value::Error(_) => None, + }?; + u32::try_from(i).ok() + } +} + +/// The least specific of the keys that match `want` (a `None` entry matches anything), when it +/// is the only one that specific. +fn least_specific<'a>(keys: &'a [ReportKey], want: &[Option], data: u32) -> Option<&'a ReportKey> { + let mut best: Option<(&ReportKey, usize)> = None; + let mut tie = false; + for k in keys { + if k.data.is_some_and(|d| d != data) || k.items.len() != want.len() { + continue; + } + if !want.iter().zip(&k.items).all(|(w, i)| w.is_none() || w == i) { + continue; + } + let n = k.specific(); + match best { + Some((_, b)) if n > b => {} + Some((_, b)) if n == b => tie = true, + _ => { + best = Some((k, n)); + tie = false; + } + } + } + if tie { None } else { best.map(|(k, _)| k) } +} + +impl PivotReport { + /// The value field GETPIVOTDATA's `data_field` names: a caption, else a source field, case + /// ignored. + pub fn data_field(&self, name: &str) -> Option { + let i = self + .data_fields + .iter() + .position(|d| same_text(&d.name, name)) + .or_else(|| self.data_fields.iter().position(|d| same_text(&d.source, name)))?; + u32::try_from(i).ok() + } + + /// The report cell (row and column offsets) GETPIVOTDATA reads, or `#REF!`. As in Excel, + /// each pair names a row, column or report filter field and one of its shown items; fields + /// left out stand for their total, or, where the report shows no such total, for the one item + /// that the named items leave. The cell must be in the report: hidden, filtered-out and + /// collapsed items, and totals turned off, give `#REF!`. + pub fn lookup(&self, data_field: &str, pairs: &[(&str, &Value)]) -> Result<(u32, u32), CellError> { + let data = self.data_field(data_field).ok_or(CellError::Ref)?; + let mut rows: Vec> = vec![None; self.row_fields.len()]; + let mut cols: Vec> = vec![None; self.col_fields.len()]; + for (field, item) in pairs { + let axis = [(&self.row_fields, &mut rows), (&self.col_fields, &mut cols)] + .into_iter() + .find_map(|(fields, want)| fields.iter().position(|f| same_text(&f.name, field)).map(|i| (fields, want, i))); + match axis { + Some((fields, want, i)) => { + let it = fields.get(i).and_then(|f| f.find(item)).ok_or(CellError::Ref)?; + let slot = want.get_mut(i).ok_or(CellError::Ref)?; + if slot.is_some_and(|x| x != it) { + return Err(CellError::Ref); + } + *slot = Some(it); + } + None => { + // A report filter counts only when it shows exactly that one item. + let (_, shown) = self.page_fields.iter().find(|(n, _)| same_text(n, field)).ok_or(CellError::Ref)?; + let shown = shown.as_deref().ok_or(CellError::Ref)?; + let named = match item { + Value::Text(t) => same_text(t, shown), + Value::Number(n) => gridcraft_core::number_to_text(*n) == shown, + _ => false, + }; + if !named { + return Err(CellError::Ref); + } + } + } + } + let row = least_specific(&self.rows, &rows, data).ok_or(CellError::Ref)?; + let col = least_specific(&self.cols, &cols, data).ok_or(CellError::Ref)?; + if col.data.or(row.data) != Some(data) { + return Err(CellError::Ref); + } + Ok((row.offset, col.offset)) + } + + fn row_at(&self, row: u32) -> Option<&ReportKey> { + self.rows.iter().find(|k| k.offset == row) + } + + fn col_at(&self, col: u32) -> Option<&ReportKey> { + self.cols.iter().find(|k| k.offset == col) + } + + /// The value cell at (row, column) offsets: its value field and the items of its line and + /// column, row fields first (`None` when the cell isn't a value cell). + #[allow(clippy::type_complexity)] + pub fn value_cell(&self, row: u32, col: u32) -> Option<(u32, Vec<(&ReportField, &ReportItem)>)> { + let (r, c) = (self.row_at(row)?, self.col_at(col)?); + let data = c.data.or(r.data)?; + let mut pairs = Vec::new(); + for (fields, key) in [(&self.row_fields, r), (&self.col_fields, c)] { + for (f, it) in fields.iter().zip(&key.items) { + if let Some(it) = it.and_then(|i| f.items.get(i as usize)) { + pairs.push((f, it)); + } + } + } + Some((data, pairs)) + } + + /// The field a report cell belongs to: a value cell's value field, a row label's row field + /// or a column label's column field. + pub fn part_at(&self, row: u32, col: u32) -> Option { + if let (Some(r), Some(c)) = (self.row_at(row), self.col_at(col)) { + return c.data.or(r.data).map(ReportPart::Data); + } + if col < self.label_cols + && let Some(r) = self.row_at(row) + { + let deepest = r.items.iter().rposition(Option::is_some)?; + // Outline and Tabular form give each row field its own column. + let f = if !self.compact && r.items.get(col as usize).is_some_and(Option::is_some) { col as usize } else { deepest }; + return u32::try_from(f).ok().map(ReportPart::RowField); + } + let level = self.col_label_rows.iter().position(|r| *r == row)?; + let c = self.col_at(col)?; + c.items.get(level).copied().flatten()?; + u32::try_from(level).ok().map(ReportPart::ColField) + } + + /// The value cells' area: (first row, first column, last row, last column) offsets. + pub fn data_area(&self) -> Option<(u32, u32, u32, u32)> { + let r0 = self.rows.iter().map(|k| k.offset).min()?; + let r1 = self.rows.iter().map(|k| k.offset).max()?; + let c0 = self.cols.iter().map(|k| k.offset).min()?; + let c1 = self.cols.iter().map(|k| k.offset).max()?; + Some((r0, c0, r1, c1)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn item(label: &str) -> ReportItem { + ReportItem { value: Value::text(label), label: label.into() } + } + + fn key(offset: u32, data: Option, items: &[Option]) -> ReportKey { + ReportKey { offset, data, items: items.to_vec() } + } + + /// Region (East, West) › Product (Apples, Pears, Plums) in rows, Sum of Sales; East sells + /// Apples and Pears, West Apples and Plums; subtotals on top, grand total. + fn report() -> PivotReport { + PivotReport { + data_fields: vec![ReportDataField { name: "Sum of Sales".into(), source: "Sales".into() }], + row_fields: vec![ + ReportField { name: "Region".into(), dates: false, items: vec![item("East"), item("West")] }, + ReportField { name: "Product".into(), dates: false, items: vec![item("Apples"), item("Pears"), item("Plums")] }, + ], + col_fields: vec![], + page_fields: vec![("Channel".into(), Some("Online".into())), ("Year".into(), None)], + rows: vec![ + key(1, None, &[Some(0), None]), + key(2, None, &[Some(0), Some(0)]), + key(3, None, &[Some(0), Some(1)]), + key(4, None, &[Some(1), None]), + key(5, None, &[Some(1), Some(0)]), + key(6, None, &[Some(1), Some(2)]), + key(7, None, &[None, None]), + ], + cols: vec![key(1, Some(0), &[])], + col_label_rows: vec![], + label_cols: 1, + compact: true, + } + } + + fn t(s: &str) -> Value { + Value::text(s) + } + + #[test] + fn named_items_pick_the_cell_and_fields_left_out_are_totals() { + let r = report(); + assert_eq!(r.lookup("Sum of Sales", &[]), Ok((7, 1)), "grand total"); + assert_eq!(r.lookup("sales", &[]), Ok((7, 1)), "by source field, case ignored"); + assert_eq!(r.lookup("Units", &[]), Err(CellError::Ref)); + assert_eq!(r.lookup("Sales", &[("region", &t("EAST"))]), Ok((1, 1)), "subtotal"); + assert_eq!(r.lookup("Sales", &[("Product", &t("Pears")), ("Region", &t("East"))]), Ok((3, 1)), "pairs in any order"); + // Plums only sells in the West: the region follows. Apples sells in both: ambiguous. + assert_eq!(r.lookup("Sales", &[("Product", &t("Plums"))]), Ok((6, 1))); + assert_eq!(r.lookup("Sales", &[("Product", &t("Apples"))]), Err(CellError::Ref)); + assert_eq!(r.lookup("Sales", &[("Region", &t("East")), ("Product", &t("Plums"))]), Err(CellError::Ref)); + assert_eq!(r.lookup("Sales", &[("Region", &t("South"))]), Err(CellError::Ref)); + assert_eq!(r.lookup("Sales", &[("Nope", &t("East"))]), Err(CellError::Ref)); + // The same item twice is fine; two items of one field are not. + assert_eq!(r.lookup("Sales", &[("Region", &t("East")), ("Region", &t("East"))]), Ok((1, 1))); + assert_eq!(r.lookup("Sales", &[("Region", &t("East")), ("Region", &t("West"))]), Err(CellError::Ref)); + // A report filter showing one item may be named with that item only. + assert_eq!(r.lookup("Sales", &[("Channel", &t("online"))]), Ok((7, 1))); + assert_eq!(r.lookup("Sales", &[("Channel", &t("Store"))]), Err(CellError::Ref)); + assert_eq!(r.lookup("Sales", &[("Year", &t("2026"))]), Err(CellError::Ref), "(All) shows no single item"); + } + + #[test] + fn totals_that_are_turned_off_are_not_there() { + let mut r = report(); + r.rows.retain(|k| k.items != [None, None]); + assert_eq!(r.lookup("Sales", &[]), Err(CellError::Ref), "two regions, no grand total"); + r.rows.retain(|k| k.items.iter().all(Option::is_some)); + assert_eq!(r.lookup("Sales", &[("Region", &t("East"))]), Err(CellError::Ref), "no subtotal for East's two products"); + assert_eq!(r.lookup("Sales", &[("Region", &t("West")), ("Product", &t("Plums"))]), Ok((6, 1))); + } + + #[test] + fn items_match_by_value_or_label() { + let f = ReportField { + name: "Units".into(), + dates: false, + items: vec![ + ReportItem { value: Value::Number(5.0), label: "5".into() }, + ReportItem { value: Value::text("2026"), label: "2026".into() }, + ReportItem { value: Value::Bool(true), label: "TRUE".into() }, + ], + }; + assert_eq!(f.find(&Value::Number(5.0)), Some(0)); + assert_eq!(f.find(&t("5")), Some(0), "text matches the label"); + assert_eq!(f.find(&Value::Number(5.000_000_1)), None); + assert_eq!(f.find(&Value::Number(2026.0)), Some(1), "a number names a text item labelled with it"); + assert_eq!(f.find(&Value::Bool(true)), Some(2)); + assert_eq!(f.find(&Value::Empty), None); + } + + #[test] + fn cells_know_their_field_and_items() { + let r = report(); + assert_eq!(r.part_at(3, 1), Some(ReportPart::Data(0))); + assert_eq!(r.part_at(3, 0), Some(ReportPart::RowField(1))); + assert_eq!(r.part_at(4, 0), Some(ReportPart::RowField(0))); + assert_eq!(r.part_at(7, 0), None, "the grand total line has no item"); + assert_eq!(r.part_at(0, 0), None); + let (data, pairs) = r.value_cell(5, 1).unwrap(); + assert_eq!(data, 0); + let names: Vec<(&str, &str)> = pairs.iter().map(|(f, i)| (f.name.as_str(), i.label.as_str())).collect(); + assert_eq!(names, [("Region", "West"), ("Product", "Apples")]); + assert!(r.value_cell(5, 0).is_none()); + assert_eq!(r.data_area(), Some((1, 1, 7, 1))); + // The index is a cache: PivotTables compare equal with or without it. + assert_eq!(PivotReportIndex(Some(Arc::new(r))), PivotReportIndex::default()); + } +} diff --git a/crates/model/src/sheet.rs b/crates/model/src/sheet.rs index b404b891c..aa22cfc79 100644 --- a/crates/model/src/sheet.rs +++ b/crates/model/src/sheet.rs @@ -93,6 +93,9 @@ pub struct Sheet { /// On-sheet slicer widgets (filter a table column by clicking value tiles). #[serde(default)] pub slicers: Vec, + /// Timelines (filter PivotTables by a period of a date field). + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub timelines: Vec, /// PivotTables whose output lives on this sheet. #[serde(default)] pub pivots: Vec, @@ -156,6 +159,7 @@ impl Sheet { shapes: vec![], sparklines: vec![], slicers: vec![], + timelines: vec![], pivots: vec![], protection: None, edit_ranges: vec![], diff --git a/crates/model/src/workbook.rs b/crates/model/src/workbook.rs index 828ca692e..7bb63ab8b 100644 --- a/crates/model/src/workbook.rs +++ b/crates/model/src/workbook.rs @@ -273,6 +273,9 @@ impl Workbook { for c in &s.slicers { m = m.max(c.id); } + for c in &s.timelines { + m = m.max(c.id); + } } m + 1 } diff --git a/crates/ui-egui/src/backstage.rs b/crates/ui-egui/src/backstage.rs index 922baabc5..340d74bda 100644 --- a/crates/ui-egui/src/backstage.rs +++ b/crates/ui-egui/src/backstage.rs @@ -994,6 +994,12 @@ fn options(app: &mut SheetApp, ui: &mut Ui) { } ui.add_space(6.0); dim(ui, lang.tr("Calculation settings are saved with the workbook.")); + ui.add_space(12.0); + ui.label(lang.tr("Working with formulas")); + let mut getpivotdata = app.session.prefs.generate_get_pivot_data; + if ui.checkbox(&mut getpivotdata, lang.tr("Use GetPivotData functions for PivotTable references")).changed() { + app.run_or_alert("pivot.generateGetPivotData", json!({"on": getpivotdata})); + } } #[cfg(test)] diff --git a/crates/ui-egui/src/chartview.rs b/crates/ui-egui/src/chartview.rs index 180f15c21..7b0a68431 100644 --- a/crates/ui-egui/src/chartview.rs +++ b/crates/ui-egui/src/chartview.rs @@ -387,6 +387,11 @@ pub(crate) fn paint_objects(app: &SheetApp, p: &Painter, geo: &Geo, wb: &Workboo } } paint_slicers(app, p, geo, wb, si, sh); + if !sh.timelines.is_empty() + && let Some(d) = app.session.active() + { + crate::timeline_view::paint(app, p, geo, &d.wb, sh); + } painted } @@ -507,6 +512,12 @@ fn selection_frame(p: &Painter, r: Rect) { /// The topmost object under `p`: (kind, id, rect). pub fn hit(app: &SheetApp, geo: &Geo, sh: &Sheet, p: Pos2) -> Option<(&'static str, u32, Rect)> { let _ = app; + for tl in sh.timelines.iter().rev() { + let r = anchor_rect(geo, sh, &tl.anchor); + if r.contains(p) && geo.cells.contains(p) { + return Some(("timeline", tl.id, r)); + } + } for sl in sh.slicers.iter().rev() { let r = anchor_rect(geo, sh, &sl.anchor); if r.contains(p) && geo.cells.contains(p) { diff --git a/crates/ui-egui/src/control.rs b/crates/ui-egui/src/control.rs index 2ea5cfae0..a7191dddb 100644 --- a/crates/ui-egui/src/control.rs +++ b/crates/ui-egui/src/control.rs @@ -182,6 +182,7 @@ pub fn handle(app: &mut SheetApp, ctx: &egui::Context, method: &str, p: &Json) - ("view.theme", "Display Theme", ""), ("view.animations", "Animations", "{on?: bool} (toggles without it) → {on}"), ("options.autoComplete", "Enable AutoComplete for cell values", "{on?: bool} (toggles without it) → {on}"), + ("pivot.showFieldList", "Field List", "{on?: bool} shows or hides the PivotTable Fields pane (toggles without it) → {on}"), ("view.interfaceScale", "Interface Size", "{percent: 75–200 | \"auto\"} (100 = auto; clamped)"), ("ui.dialog", "Open Dialog", ""), ("ui.backstage", "File (Backstage view)", "{page?: home|new|open|info|saveAs|print|export|options|close}"), diff --git a/crates/ui-egui/src/dialogs.rs b/crates/ui-egui/src/dialogs.rs index fb899c702..77cd2cf18 100644 --- a/crates/ui-egui/src/dialogs.rs +++ b/crates/ui-egui/src/dialogs.rs @@ -644,6 +644,62 @@ impl Dialog { vec![Text { key: "name", label: "Name:" }, Text { key: "formula", label: "Formula:" }], json!({"pivot": p("pivot"), "name": p("name"), "formula": p("formula")}), ), + "pivotChangeSource" => Dialog::form( + "pivotChangeSource", + "Change PivotTable Data Source", + "pivot.changeSource", + vec![Text { key: "source", label: "Table/Range:" }], + json!({"pivot": p("pivot"), "source": p("source")}), + ), + "pivotValueFieldSettings" => Dialog::form( + "pivotValueFieldSettings", + "Value Field Settings", + "pivot.valueSettings", + vec![ + Text { key: "name", label: "Custom Name:" }, + Choice { + key: "func", + label: "Summarize value field by", + options: vec![ + ("sum", "Sum"), + ("count", "Count"), + ("average", "Average"), + ("max", "Max"), + ("min", "Min"), + ("product", "Product"), + ("countNumbers", "Count Numbers"), + ("stdDev", "StdDev"), + ("stdDevP", "StdDevp"), + ("var", "Var"), + ("varP", "Varp"), + ], + }, + Choice { + key: "showAs", + label: "Show values as", + options: vec![ + ("normal", "No Calculation"), + ("percentOfGrandTotal", "% of Grand Total"), + ("percentOfColumnTotal", "% of Column Total"), + ("percentOfRowTotal", "% of Row Total"), + ("percentOf", "% Of"), + ("percentOfParentRowTotal", "% of Parent Row Total"), + ("percentOfParentColumnTotal", "% of Parent Column Total"), + ("percentOfParentTotal", "% of Parent Total"), + ("differenceFrom", "Difference From"), + ("percentDifferenceFrom", "% Difference From"), + ("runningTotal", "Running Total In"), + ("percentRunningTotal", "% Running Total In"), + ("rankAscending", "Rank Smallest to Largest"), + ("rankDescending", "Rank Largest to Smallest"), + ("index", "Index"), + ], + }, + Text { key: "baseField", label: "Base field:" }, + Text { key: "baseItem", label: "Base item:" }, + ], + json!({"pivot": p("pivot"), "field": p("field"), "name": p("name"), "func": p("func"), "showAs": p("showAs"), "baseField": p("baseField"), "baseItem": p("baseItem")}), + ), "goToSpecial" => Dialog::form( "goToSpecial", "Go To Special", @@ -782,6 +838,10 @@ impl Dialog { app.run_or_alert("review.showComments", json!({"on": true})); return None; } + n if crate::pivot_ui::DIALOGS.contains(&n) => { + let defaults = crate::pivot_ui::defaults(app, n, ¶ms); + Dialog::custom(n, crate::pivot_ui::title(n), defaults) + } "watch" | "selectionPane" | "formatChart" => { app.grid.pane = Some( match name { @@ -940,6 +1000,12 @@ fn data_validation_form(ui: &mut egui::Ui, d: &mut Dialog, confirm: &mut bool, o /// Builds the command params from form values for dialogs that need translation. fn build_params(d: &Dialog) -> Json { + // Value Field Settings: an empty base field or base item keeps the PivotTable's default. + if d.name == "pivotValueFieldSettings" { + let mut v = d.values.clone(); + v.retain(|k, x| !(matches!(k.as_str(), "baseField" | "baseItem") && x.as_str().is_none_or(|t| t.trim().is_empty()))); + return Json::Object(v); + } let mut v = Json::Object(d.values.clone()); match d.name.as_str() { "dataValidation" => { @@ -1033,6 +1099,7 @@ pub fn show(app: &mut SheetApp, ctx: &egui::Context) { "recommendedCharts" => 600.0, "analysisToolPak" => 420.0, "solver" => 460.0, + n if crate::pivot_ui::DIALOGS.contains(&n) => 440.0, _ => 380.0, }; let window = egui::Window::new(app.ui.language.tr(&d.title)) @@ -1047,6 +1114,7 @@ pub fn show(app: &mut SheetApp, ctx: &egui::Context) { ui.set_min_width(width - 20.0); match d.name.as_str() { "formatCells" => format_cells(app, ui, &mut d, &mut confirm), + n if crate::pivot_ui::DIALOGS.contains(&n) => crate::pivot_ui::dialog(app, ui, &mut d, &mut confirm, &mut open), "dataValidation" => data_validation_form(ui, &mut d, &mut confirm, &mut open), "insertFunction" => insert_function(app, ui, &mut d, &mut confirm), "find" => find(app, ui, &mut d), @@ -1365,7 +1433,7 @@ fn paste_special_shortcuts(d: &mut Dialog, ctx: &egui::Context) { }); } -fn ok_cancel(ui: &mut egui::Ui, confirm: &mut bool, open: &mut bool) { +pub(crate) fn ok_cancel(ui: &mut egui::Ui, confirm: &mut bool, open: &mut bool) { ui.add_space(8.0); ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| { let lang = crate::i18n::current(ui.ctx()); @@ -1435,9 +1503,9 @@ fn form_fields(ui: &mut egui::Ui, d: &mut Dialog) { ui.vertical(|ui| { ui.label(egui::RichText::new(label).strong()); let cur = d.values.get(key).and_then(Json::as_str).unwrap_or("").to_string(); - if d.name == "dataValidation" { + if matches!(d.name.as_str(), "dataValidation" | "pivotValueFieldSettings") { let selected = options.iter().find(|(val, _)| *val == cur).map(|(_, text)| *text).unwrap_or(""); - egui::ComboBox::from_id_salt(("dataValidation", key)).selected_text(selected).width(300.0).show_ui(ui, |ui| { + egui::ComboBox::from_id_salt((d.name.clone(), key)).selected_text(selected).width(300.0).show_ui(ui, |ui| { for (val, text) in options { if ui.selectable_label(cur == val, text).clicked() { d.set_value(key, json!(val)); diff --git a/crates/ui-egui/src/grid.rs b/crates/ui-egui/src/grid.rs index 97b787ef3..491034583 100644 --- a/crates/ui-egui/src/grid.rs +++ b/crates/ui-egui/src/grid.rs @@ -149,6 +149,10 @@ pub struct GridState { pub hscroll: Option, /// Ctrl+wheel zoom in progress (see [`WheelZoom`]). pub wheel_zoom: Option, + /// The open menu of a PivotTable report's field button. + pub pivot_menu: Option, + /// The open time level menu of a timeline: (timeline id, where). + pub timeline_menu: Option<(u32, Pos2)>, } /// Ctrl+wheel zoom for one (doc uid, sheet). egui eases a notch in over several frames and @@ -750,6 +754,7 @@ pub fn show(app: &mut SheetApp, ui: &mut egui::Ui) { } // Interaction. let previous_popups = (app.grid.filter_menu, app.grid.list_picker, app.grid.header_menu); + let previous_pivot_menu = app.grid.pivot_menu.clone(); crate::paper_view::interact(app, ui, &resp, &geo, sh, &wb); interact(app, ui, &resp, &geo, sh, &wb); crate::text_box::show(app, ui, &geo, sh); @@ -757,6 +762,8 @@ pub fn show(app: &mut SheetApp, ui: &mut egui::Ui) { in_cell_editor(app, ui, &geo, sh, &wb); } filter_menu(app, ui, &geo, previous_popups.0 == app.grid.filter_menu); + crate::pivot_ui::field_menu(app, ui, previous_pivot_menu == app.grid.pivot_menu); + crate::timeline_view::level_menu(app, ui); list_picker(app, ui, &geo, previous_popups.1 == app.grid.list_picker); header_menu(app, ui, previous_popups.2 == app.grid.header_menu); context_menu(app, ui); @@ -1392,6 +1399,21 @@ pub(crate) fn paint_quadrant( p.rect_stroke(r, 2.0, Stroke::new(1.0, Color32::from_gray(170)), StrokeKind::Inside); crate::icons::paint(p, r.shrink(1.5), if active { crate::icons::Icon::Filter } else { crate::icons::Icon::Chevron }, Color32::from_gray(70)); } + // PivotTable field buttons (Row Labels, Column Labels, field headers, report filters). + for b in crate::pivot_ui::header_buttons(sh) { + if !range.contains(b.cell) { + continue; + } + let r = crate::pivot_ui::button_rect(geo, sh, b.cell); + p.rect_filled(r, 2.0, Color32::from_gray(250)); + p.rect_stroke(r, 2.0, Stroke::new(1.0, Color32::from_gray(170)), StrokeKind::Inside); + crate::icons::paint( + p, + r.shrink(1.5), + if b.filtered { crate::icons::Icon::Filter } else { crate::icons::Icon::Chevron }, + Color32::from_gray(70), + ); + } } pub fn filter_button_rect(geo: &Geo, sh: &Sheet, c: CellRef) -> Rect { @@ -1909,6 +1931,16 @@ fn interact(app: &mut SheetApp, ui: &mut egui::Ui, resp: &egui::Response, geo: & } return; } + // A click on a timeline's period, clear button, time level or scroll bar. + if primary_clicked + && !resp.drag_started() + && let Some(p) = pos + && let Some((id, Some(act))) = crate::timeline_view::click(geo, sh, &app_wb(app), ui.ctx(), p) + { + app.selected_chart = Some(id); + crate::timeline_view::on_click(app, id, act, mods.shift); + return; + } // Objects (charts, pictures, shapes, slicers) take clicks first. if resp.drag_started() || primary_clicked { // A drag is recognised only after the pointer has moved, so grab the object where the press began. @@ -2058,20 +2090,22 @@ fn interact(app: &mut SheetApp, ui: &mut egui::Ui, resp: &egui::Response, geo: & app.grid.list_picker = Some(sel.active); return; } + // A PivotTable report's field button opens its field menu. + if let Some(button) = crate::pivot_ui::button_at(sh, geo, p) { + app.grid.pivot_menu = Some(crate::pivot_ui::PivotMenu { button, at: geo.mirror_pos(p) }); + return; + } // Filter dropdown buttons. if is_filter_button(sh, geo, p) { app.grid.filter_menu = Some((c.col, c.row, geo.mirror_pos(p))); return; } // Point mode while editing a formula: clicking inserts a reference. + let getpivotdata = app.session.prefs.generate_get_pivot_data; if let Some(ed) = app.editor.as_mut() && ed.can_point() { - let sheet_name = (ed.sheet != wb.active_sheet).then(|| sh.name.clone()); - let r = match sheet_name { - Some(n) => format!("{}!{}", gridcraft_engine::formula::quote_sheet(&n), c.a1()), - None => c.a1(), - }; + let r = point_text(getpivotdata, wb, sh, ed, c); ed.insert_ref(&r); ed.request_focus = true; ed.point_cell = Some(c); @@ -2340,18 +2374,35 @@ fn point_header(app: &mut SheetApp, wb: &Workbook, sh: &Sheet, range: RangeRef) true } +/// What point mode writes for cell `c` of the sheet shown (`sh`): with Generate GetPivotData on, +/// a value of a PivotTable becomes its GETPIVOTDATA formula, as in Excel; anything else is the +/// cell's reference, with its sheet when the formula is on another one. +fn point_text(getpivotdata: bool, wb: &Workbook, sh: &Sheet, ed: &EditState, c: CellRef) -> String { + if getpivotdata && let Some(f) = gridcraft_engine::pivot::getpivotdata_formula(wb, ed.sheet, wb.active_sheet, c, ed.locale) { + return f; + } + if ed.sheet != wb.active_sheet { format!("{}!{}", gridcraft_engine::formula::quote_sheet(&sh.name), c.a1()) } else { c.a1() } +} + fn select_to_pointer(app: &mut SheetApp, geo: &Geo, sh: &Sheet, wb: &Workbook, p: Pos2) { let cell = CellRef::new( geo.row_at(sh, p.y.clamp(geo.cells.top() + 1.0, geo.cells.bottom() - 1.0)), geo.col_at(sh, p.x.clamp(geo.cells.left() + 1.0, geo.cells.right() - 1.0)), ); + let getpivotdata = app.session.prefs.generate_get_pivot_data; if let Some(ed) = app.editor.as_mut() && ed.point.is_some() { if let Some(start) = ed.point_cell { let range = RangeRef::new(start, cell); - let text = if range.is_single() { range.start.a1() } else { range.a1() }; - let text = if ed.sheet != wb.active_sheet { format!("{}!{text}", gridcraft_engine::formula::quote_sheet(&sh.name)) } else { text }; + // Dragging over several cells writes their range; back on one cell, what a click writes. + let text = if range.is_single() { + point_text(getpivotdata, wb, sh, ed, start) + } else if ed.sheet != wb.active_sheet { + format!("{}!{}", gridcraft_engine::formula::quote_sheet(&sh.name), range.a1()) + } else { + range.a1() + }; ed.insert_ref(&text); ed.point_cell = Some(start); } @@ -2504,10 +2555,16 @@ fn object_anchor(sh: &Sheet, kind: &str, id: u32) -> Option sh.charts.iter().find(|c| c.id == id).map(|c| c.anchor), "image" => sh.images.iter().find(|c| c.id == id).map(|c| c.anchor), "slicer" => sh.slicers.iter().find(|c| c.id == id).map(|c| c.anchor), + "timeline" => sh.timelines.iter().find(|c| c.id == id).map(|c| c.anchor), _ => sh.shapes.iter().find(|c| c.id == id).map(|c| c.anchor), } } +/// The active workbook (an empty one without a document). +fn app_wb(app: &SheetApp) -> std::sync::Arc { + app.session.active().map(|d| d.wb.clone()).unwrap_or_default() +} + fn delete_selected_object(app: &mut SheetApp, sh: &Sheet) -> bool { let Some(id) = app.selected_chart else { return false }; let kind = if sh.charts.iter().any(|object| object.id == id) { @@ -2516,6 +2573,10 @@ fn delete_selected_object(app: &mut SheetApp, sh: &Sheet) -> bool { "image" } else if sh.shapes.iter().any(|object| object.id == id) { "shape" + } else if sh.timelines.iter().any(|object| object.id == id) { + app.run_or_alert("timeline.delete", json!({"timeline": id})); + app.selected_chart = None; + return true; } else { return false; }; @@ -2942,10 +3003,14 @@ pub fn point_move(app: &mut SheetApp, ed: &mut EditState, dr: i64, dc: i64, exte }) .unwrap_or(from); RangeRef::new(anchor, to).a1() + } else if app.session.prefs.generate_get_pivot_data + && let Some(f) = gridcraft_engine::pivot::getpivotdata_formula(&d.wb, ed.sheet, d.wb.active_sheet, to, ed.locale) + { + // A value of a PivotTable, as a click on it writes. + f } else { to.a1() }; - let _ = d; ed.insert_ref(&text); ed.point_cell = Some(to); app.grid.ensure_visible = true; diff --git a/crates/ui-egui/src/i18n/de.rs b/crates/ui-egui/src/i18n/de.rs index c20c7d1b3..3c410459e 100644 --- a/crates/ui-egui/src/i18n/de.rs +++ b/crates/ui-egui/src/i18n/de.rs @@ -411,6 +411,238 @@ pub(super) fn translate(text: &str) -> Option<&'static str> { "Dark" => "Dunkel", "Sheet Options|View" => "Anzeige", "Print" => "Drucken", + "Timeline" => "Zeitachse", + "Insert a timeline to filter a PivotTable by date" => "Zeitachse einfügen, um eine PivotTable nach Datum zu filtern", + "Timeline Caption:" => "Zeitachsenbeschriftung:", + "Report Connections" => "Berichtsverbindungen", + "Bring Forward" => "Ebene nach vorne", + "Send Backward" => "Ebene nach hinten", + "Selection Pane" => "Auswahlbereich", + "Align" => "Ausrichten", + "Rotate" => "Drehen", + "Not available for timelines yet" => "Für Zeitachsen noch nicht verfügbar", + "Height:" => "Höhe:", + "Width:" => "Breite:", + "Selection Label" => "Auswahlbeschriftung", + "Scrollbar" => "Bildlaufleiste", + "Time Level" => "Zeitebene", + "All Periods" => "Alle Zeiträume", + "YEARS" => "JAHRE", + "QUARTERS" => "QUARTALE", + "MONTHS" => "MONATE", + "DAYS" => "TAGE", + "Fields, Items & Sets" => "Felder, Elemente und Gruppen", + // PivotTable Analyze and PivotTable Design, in German Excel's words. + "PivotTable Analyze" => "PivotTable-Analyse", + "PivotTable Design" => "Entwurf", + "PivotTable Name:" => "PivotTable-Name:", + "Renaming a PivotTable isn't available in GridCraft yet." => "PivotTables lassen sich in GridCraft noch nicht umbenennen.", + "PivotTable Options" => "PivotTable-Optionen", + "Options…" => "Optionen…", + "Show Report Filter Pages…" => "Berichtsfilterseiten anzeigen…", + "Generate GetPivotData" => "GetPivotData generieren", + "GridCraft can't do this yet." => "Das kann GridCraft noch nicht.", + "Only for PivotTables based on OLAP or the Data Model, which GridCraft doesn't have." => { + "Nur für PivotTables auf Basis von OLAP oder dem Datenmodell, die GridCraft nicht unterstützt." + } + "Select an item or a value of the PivotTable first." => "Wählen Sie zuerst ein Element oder einen Wert der PivotTable aus.", + "Select an item of a grouped field first." => "Wählen Sie zuerst ein Element eines gruppierten Felds aus.", + "Active Field:" => "Aktives Feld:", + "Field Settings" => "Feldeinstellungen", + "Drill\nDown" => "Drilldown", + "Drill Down" => "Drilldown", + "Drill\nUp" => "Drillup", + "Drill Up" => "Drillup", + "Expand Field" => "Feld erweitern", + "Collapse Field" => "Feld reduzieren", + "Group Selection" => "Auswahl gruppieren", + "Group Field" => "Feld gruppieren", + "Insert Slicer" => "Datenschnitt einfügen", + "Insert Timeline" => "Zeitachse einfügen", + "Filter Connections" => "Filterverbindungen", + "Refresh" => "Aktualisieren", + "Refresh (Alt+F5)" => "Aktualisieren (Alt+F5)", + "Refresh All" => "Alle aktualisieren", + "Refresh Status" => "Status aktualisieren", + "Cancel Refresh" => "Aktualisierung abbrechen", + "Connection Properties…" => "Verbindungseigenschaften…", + "Change Data\nSource" => "Datenquelle\nändern", + "Change Data Source" => "Datenquelle ändern", + "Change Data Source…" => "Datenquelle ändern…", + "PivotTable|Clear All" => "Alle löschen", + "Clear Filters" => "Filter entfernen", + "Select" => "Auswählen", + "Labels and Values" => "Beschriftungen und Werte", + "Values" => "Werte", + "Labels" => "Beschriftungen", + "Entire PivotTable" => "Gesamte PivotTable", + "Enable Selection" => "Auswahl aktivieren", + "Move PivotTable" => "PivotTable verschieben", + "Fields, Items,\n& Sets" => "Felder, Elemente\nund Gruppen", + "Fields, Items, & Sets" => "Felder, Elemente und Gruppen", + "Calculated Field…" => "Berechnetes Feld…", + "Calculated Item…" => "Berechnetes Element…", + "Solve Order…" => "Lösungsreihenfolge…", + "List Formulas" => "Formeln auflisten", + "Insert\nTimeline" => "Zeitachse\neinfügen", + "Select field:" => "Feld auswählen:", + "More Sort Options…" => "Weitere Sortieroptionen…", + "Clear Filter From" => "Filter löschen aus", + "Label Filters" => "Beschriftungsfilter", + "Value Filters" => "Wertefilter", + "Clear Filter" => "Filter löschen", + "Equals…" => "Ist gleich…", + "Does Not Equal…" => "Ist nicht gleich…", + "Begins With…" => "Beginnt mit…", + "Does Not Begin With…" => "Beginnt nicht mit…", + "Ends With…" => "Endet mit…", + "Does Not End With…" => "Endet nicht mit…", + "Contains…" => "Enthält…", + "Does Not Contain…" => "Enthält nicht…", + "Greater Than…" => "Größer als…", + "Greater Than Or Equal To…" => "Größer als oder gleich…", + "Less Than…" => "Kleiner als…", + "Less Than Or Equal To…" => "Kleiner als oder gleich…", + "Between…" => "Zwischen…", + "Not Between…" => "Nicht zwischen…", + "Top 10…" => "Top 10…", + "Search" => "Suchen", + "(Select All)" => "(Alle auswählen)", + "Label Filter" => "Beschriftungsfilter", + "Value Filter" => "Wertefilter", + "Top 10 Filter" => "Top 10-Filter", + "Show items for which the label" => "Elemente anzeigen, deren Beschriftung", + "Show items for which" => "Elemente anzeigen, für die", + "equals" => "gleich", + "does not equal" => "ungleich", + "begins with" => "beginnt mit", + "does not begin with" => "beginnt nicht mit", + "ends with" => "endet mit", + "does not end with" => "endet nicht mit", + "contains" => "enthält", + "does not contain" => "enthält nicht", + "is greater than" => "größer als", + "is greater than or equal to" => "größer als oder gleich", + "is less than" => "kleiner als", + "is less than or equal to" => "kleiner als oder gleich", + "is between" => "zwischen", + "is not between" => "nicht zwischen", + "and" => "und", + "Use ? to represent any single character" => "? steht für ein beliebiges einzelnes Zeichen", + "Use * to represent any series of characters" => "* steht für eine beliebige Zeichenfolge", + "Show" => "Anzeigen", + "Top" => "Obersten", + "Bottom" => "Untersten", + "Items" => "Elemente", + "Percent" => "Prozent", + "Sum" => "Summe", + "by" => "nach", + "Grouping" => "Gruppierung", + "Starting at:" => "Starten:", + "Ending at:" => "Beenden:", + "By" => "Nach", + "By:" => "Nach:", + "Seconds" => "Sekunden", + "Minutes" => "Minuten", + "Hours" => "Stunden", + "Days" => "Tage", + "Months" => "Monate", + "Quarters" => "Quartale", + "Years" => "Jahre", + "Number of days:" => "Anzahl der Tage:", + "Insert Calculated Item" => "Berechnetes Element einfügen", + "Calculated item in" => "Berechnetes Element in", + "Name:" => "Name:", + "Formula:" => "Formel:", + "Fields:" => "Felder:", + "Items:" => "Elemente:", + "Insert Item" => "Element einfügen", + "Add" => "Hinzufügen", + "Modify" => "Ändern", + "Calculated Item Solve Order" => "Lösungsreihenfolge für berechnete Elemente", + "Solve order:" => "Lösungsreihenfolge:", + "Move Up" => "Nach oben", + "Move Down" => "Nach unten", + "Insert Timelines" => "Zeitachsen einfügen", + "PivotTable Fields" => "PivotTable-Felder", + "Choose fields to add to report:" => "Felder auswählen, die dem Bericht hinzugefügt werden sollen:", + "Use a field's menu to move it between areas:" => "Felder über ihr Menü zwischen den Bereichen verschieben:", + "Filters" => "Filter", + "Columns" => "Spalten", + "Rows" => "Zeilen", + "Move to Row Labels" => "In Zeilenbeschriftungen verschieben", + "Move to Column Labels" => "In Spaltenbeschriftungen verschieben", + "Move to Report Filter" => "In Berichtsfilter verschieben", + "Move to Values" => "In Werte verschieben", + "Summarize by Sum" => "Werte zusammenfassen nach Summe", + "Summarize by Count" => "Werte zusammenfassen nach Anzahl", + "Summarize by Average" => "Werte zusammenfassen nach Mittelwert", + "Summarize by Max" => "Werte zusammenfassen nach Max.", + "Summarize by Min" => "Werte zusammenfassen nach Min.", + "Show as No Calculation" => "Werte anzeigen als Ohne Berechnung", + "Show as % of Grand Total" => "Werte anzeigen als % des Gesamtergebnisses", + "Show as % of Column Total" => "Werte anzeigen als % des Spaltenergebnisses", + "Show as % of Row Total" => "Werte anzeigen als % des Zeilenergebnisses", + "Show as % of Parent Row Total" => "Werte anzeigen als % des Vorgängerzeilen-Gesamtergebnisses", + "Show as % of Parent Column Total" => "Werte anzeigen als % des Vorgängerspalten-Gesamtergebnisses", + "Show as Running Total" => "Werte anzeigen als Ergebnis in", + "Show as % Running Total" => "Werte anzeigen als % Ergebnis in", + "Show as Rank Smallest to Largest" => "Werte anzeigen als Rangfolge nach Größe (aufsteigend)", + "Show as Rank Largest to Smallest" => "Werte anzeigen als Rangfolge nach Größe (absteigend)", + "Show as Index" => "Werte anzeigen als Index", + "Group by Years" => "Nach Jahren gruppieren", + "Group by Quarters" => "Nach Quartalen gruppieren", + "Group by Months" => "Nach Monaten gruppieren", + "Group by Ranges of 10" => "In Zehnerbereichen gruppieren", + "Remove Field" => "Feld entfernen", + "Compact" => "Kompakt", + "Outline" => "Gliederung", + "Tabular" => "Tabellarisch", + "Create Set Based on Row Items…" => "Gruppe basierend auf Zeilenelementen erstellen…", + "Create Set Based on Column Items…" => "Gruppe basierend auf Spaltenelementen erstellen…", + "Manage Sets…" => "Gruppen verwalten…", + "OLAP Tools" => "OLAP-Tools", + "Relationships" => "Beziehungen", + "Recommended\nPivotTables" => "Empfohlene\nPivotTables", + "Recommended PivotTables" => "Empfohlene PivotTables", + "No recommendations for this data" => "Keine Empfehlungen für diese Daten", + "Field List" => "Feldliste", + "+/- Buttons" => "Schaltflächen +/-", + "Field Headers" => "Feldkopfzeilen", + "Subtotals" => "Teilergebnisse", + "Do Not Show Subtotals" => "Teilergebnisse nicht anzeigen", + "Show all Subtotals at Bottom of Group" => "Alle Teilergebnisse unten in der Gruppe anzeigen", + "Show all Subtotals at Top of Group" => "Alle Teilergebnisse oben in der Gruppe anzeigen", + "Not available in Tabular Form." => "Im Tabellenformat nicht verfügbar.", + "Include Filtered Items in Totals" => "Gefilterte Elemente in Summen einbeziehen", + "Grand\nTotals" => "Gesamt-\nergebnisse", + "Grand Totals" => "Gesamtergebnisse", + "Off for Rows and Columns" => "Für Zeilen und Spalten deaktiviert", + "On for Rows and Columns" => "Für Zeilen und Spalten aktiviert", + "On for Rows Only" => "Nur für Zeilen aktiviert", + "On for Columns Only" => "Nur für Spalten aktiviert", + "Report\nLayout" => "Berichts-\nlayout", + "Report Layout" => "Berichtslayout", + "Show in Compact Form" => "Im Kurzformat anzeigen", + "Show in Outline Form" => "Im Gliederungsformat anzeigen", + "Show in Tabular Form" => "Im Tabellenformat anzeigen", + "Repeat All Item Labels" => "Alle Elementbeschriftungen wiederholen", + "Do Not Repeat Item Labels" => "Elementbeschriftungen nicht wiederholen", + "Blank\nRows" => "Leere\nZeilen", + "Blank Rows" => "Leere Zeilen", + "Insert Blank Line after Each Item" => "Leerzeile nach jedem Element einfügen", + "Remove Blank Line after Each Item" => "Leerzeile nach jedem Element entfernen", + "Row Headers" => "Zeilenkopfzeilen", + "Column Headers" => "Spaltenkopfzeilen", + "Banded Rows" => "Gebänderte Zeilen", + "Banded Columns" => "Gebänderte Spalten", + "PivotTable\nStyles" => "PivotTable-\nFormate", + "PivotTable Styles" => "PivotTable-Formate", + "Medium" => "Mittel", + "Change PivotTable Data Source" => "PivotTable-Datenquelle ändern", + "Value Field Settings" => "Wertfeldeinstellungen", + "Working with formulas" => "Arbeiten mit Formeln", + "Use GetPivotData functions for PivotTable references" => "GetPivotData-Funktionen für PivotTable-Bezüge verwenden", _ => return None, }) } diff --git a/crates/ui-egui/src/keytips/keytip_table/generated.rs b/crates/ui-egui/src/keytips/keytip_table/generated.rs index 79920b5b8..5407ea01b 100644 --- a/crates/ui-egui/src/keytips/keytip_table/generated.rs +++ b/crates/ui-egui/src/keytips/keytip_table/generated.rs @@ -183,44 +183,44 @@ pub const SEQUENCES: &[(&str, Entry)] = &[ ("NSF", Entry::Finish(Action::Command("insert.slicer"))), // Excel: Slicer; insert.slicer missing (gap #8); reserved until the command lands ("NSH", Entry::Finish(Action::Command("insert.shape"))), // Excel: Shapes ("NSP", Entry::Finish(Action::Command("insert.recommendedPivotTables"))), // Excel: Recommended PivotTables - ("NST", Entry::Finish(Action::Pending("insert.timeline"))), // pending; Excel: Timeline; insert.timeline missing (gap #8); reserved until the command lands + ("NST", Entry::Finish(Action::Command("insert.timeline"))), // Excel: Timeline ("NSZ", Entry::Finish(Action::Command("insert.pivotChart"))), // Excel: PivotChart; PivotChart; reserved until the command lands - ("NT", Entry::Finish(Action::Command("insert.table"))), // Excel: Table - ("NU", Entry::Finish(Action::Command("insert.symbol"))), // Excel: Symbols; containment(Symbol) - ("NZS", Entry::Finish(Action::Command("insert.symbol"))), // Excel: Excel: Symbol (modern) + ("NT", Entry::Finish(Action::Command("insert.table"))), // Excel: Table + ("NU", Entry::Finish(Action::Command("insert.symbol"))), // Excel: Symbols; containment(Symbol) + ("NZS", Entry::Finish(Action::Command("insert.symbol"))), // Excel: Excel: Symbol (modern) ("NV", Entry::Finish(Action::Command("insert.pivotTable"))), // Excel: PivotTable - ("NW", Entry::Finish(Action::Pending("insert.wordArt"))), // pending; Excel: WordArt; WordArt; reserved until the command lands - ("NX", Entry::Finish(Action::Command("insert.textBox"))), // Excel: Text Box - ("NZK", Entry::Continue), // Excel: Sparklines Menu; group level: - ("NZKSL", Entry::Finish(Action::Dialog("sparkline"))), // Excel: Line Sparkline - ("NZKSO", Entry::Finish(Action::Dialog("sparkline"))), // Excel: Column Sparkline - ("NZKSW", Entry::Finish(Action::Dialog("sparkline"))), // Excel: Win/Loss Sparkline + ("NW", Entry::Finish(Action::Pending("insert.wordArt"))), // pending; Excel: WordArt; WordArt; reserved until the command lands + ("NX", Entry::Finish(Action::Command("insert.textBox"))), // Excel: Text Box + ("NZK", Entry::Continue), // Excel: Sparklines Menu; group level: + ("NZKSL", Entry::Finish(Action::Dialog("sparkline"))), // Excel: Line Sparkline + ("NZKSO", Entry::Finish(Action::Dialog("sparkline"))), // Excel: Column Sparkline + ("NZKSW", Entry::Finish(Action::Dialog("sparkline"))), // Excel: Win/Loss Sparkline ("NZTX", Entry::Finish(Action::Command("insert.textBox"))), // Excel: Text Box - ("NZT", Entry::Finish(Action::Command("insert.textBox"))), // Excel: Excel: Text Box (modern) + ("NZT", Entry::Finish(Action::Command("insert.textBox"))), // Excel: Excel: Text Box (modern) ("PAE", Entry::Finish(Action::Command("arrange.sendBackward"))), // Excel: Send Backward ("PAF", Entry::Finish(Action::Command("arrange.bringForward"))), // Excel: Bring Forward - ("PB", Entry::Continue), // Excel: Breaks; group level: + ("PB", Entry::Continue), // Excel: Breaks; group level: ("PBI", Entry::Finish(Action::Command("pageLayout.breaks"))), // Excel: Insert Page Break; Insert Page Break ("PBR", Entry::Finish(Action::Command("pageLayout.breaks"))), // Excel: Remove Page Break; Remove Page Break ("PGB", Entry::Finish(Action::Command("pageLayout.background"))), // Excel: Background; Excel badge GB (Background in the Page Setup group) ("PI", Entry::Finish(Action::Command("pageLayout.printTitles"))), // Excel: Print Titles - ("PM", Entry::Continue), // Excel: Margins; group level: + ("PM", Entry::Continue), // Excel: Margins; group level: ("PMA", Entry::Finish(Action::Command("pageLayout.margins"))), // Excel: Custom Margins; containment(Margins) ("PO", Entry::Finish(Action::Command("pageLayout.orientation"))), // Excel: Orientation ("PPG", Entry::Finish(Action::Command("pageLayout.printGridlines"))), // Excel: Print Gridlines ("PPH", Entry::Finish(Action::Command("pageLayout.printHeadings"))), // Excel: Print Headings - ("PR", Entry::Continue), // Excel: Print Area; group level: + ("PR", Entry::Continue), // Excel: Print Area; group level: ("PRS", Entry::Finish(Action::Command("pageLayout.printArea"))), // Excel: Set Print Area - ("PSP", Entry::Finish(Action::Dialog("pageSetup"))), // Excel: Page Setup + ("PSP", Entry::Finish(Action::Dialog("pageSetup"))), // Excel: Page Setup ("PSZ", Entry::Finish(Action::Command("pageLayout.size"))), // Excel: Size ("PTC", Entry::Finish(Action::Command("pageLayout.themeColors"))), // Excel: Theme Colors; containment(Colors) ("PTE", Entry::Finish(Action::Command("pageLayout.themeEffects"))), // Excel: Theme Effects; containment(Effects) ("PTF", Entry::Finish(Action::Command("pageLayout.themeFonts"))), // Excel: Theme Fonts; containment(Fonts) - ("PTH", Entry::Continue), // Excel: Themes; group level: + ("PTH", Entry::Continue), // Excel: Themes; group level: ("PTHB", Entry::Finish(Action::Command("pageLayout.theme"))), // Excel: Browse Themes; containment(Themes) - ("PVG", Entry::Finish(Action::Command("view.gridlines"))), // Excel: View Gridlines - ("PVH", Entry::Finish(Action::Command("view.headings"))), // Excel: View Headings - ("MA", Entry::Continue), // Excel: Remove Arrows Menu; group level: containment(Remove Arrows) + ("PVG", Entry::Finish(Action::Command("view.gridlines"))), // Excel: View Gridlines + ("PVH", Entry::Finish(Action::Command("view.headings"))), // Excel: View Headings + ("MA", Entry::Continue), // Excel: Remove Arrows Menu; group level: containment(Remove Arrows) ("MAA", Entry::Finish(Action::Command("formulas.removeArrows"))), // Excel: Remove All Arrows; landed in the catalog; live since T7 probe ("MAP", Entry::Finish(Action::Pending("formulas.removePrecedentArrows"))), // pending; Excel: Remove Precedent Arrows; Remove Precedent Arrows; reserved until the command lands ("MB", Entry::Finish(Action::Command("formulas.calculateNow"))), // Excel: Calculate Now @@ -359,7 +359,9 @@ pub const SEQUENCES: &[(&str, Entry)] = &[ ("JAM", Entry::Finish(Action::Command("chart.formatSelection"))), // Excel: Chart format: Format Selection pane; containment(Format Selection) ("JCA", Entry::Finish(Action::Command("chart.addElement"))), // Excel: Chart: Add Chart Element; containment(Add Chart Element) ("JCC", Entry::Finish(Action::Command("chart.changeType"))), // Excel: Chart: Change Chart Type; containment(Change Chart Type) - ("JTS", Entry::Finish(Action::Command("table.style"))), // Excel: Table: table style + ("JT", Entry::Tab("PivotTable Analyze")), // Excel: PivotTable Analyze tab; Contextual: only while the active cell is in a PivotTable (Table Design's JT letters follow otherwise). + ("JY", Entry::Tab("PivotTable Design")), // Excel: PivotTable Design tab; Contextual: only while the active cell is in a PivotTable. + ("JTS", Entry::Finish(Action::Command("table.style"))), // Excel: Table: table style ("JTV", Entry::Finish(Action::Command("table.summarizePivot"))), // Excel: Table: Summarize with PivotTable; containment(Summarize with PivotTable) ("FA", Entry::Finish(Action::Command("file.saveAs"))), // Excel: Save / Save As page; fires immediately in Excel; deeper aliases dropped as unreachable ("FC", Entry::Finish(Action::Command("file.close"))), // Excel: Close workbook diff --git a/crates/ui-egui/src/keytips/keytips.json b/crates/ui-egui/src/keytips/keytips.json index 59aeadcfb..ea0a95137 100644 --- a/crates/ui-egui/src/keytips/keytips.json +++ b/crates/ui-egui/src/keytips/keytips.json @@ -1210,9 +1210,8 @@ "seq": "NST", "kind": "command", "target": "insert.timeline", - "pending": true, "excel": "Timeline", - "note": "insert.timeline missing (gap #8); reserved until the command lands" + "note": "" }, { "seq": "NSZ", @@ -2445,6 +2444,20 @@ "excel": "Chart: Change Chart Type", "note": "containment(Change Chart Type)" }, + { + "seq": "JT", + "kind": "tab", + "target": "PivotTable Analyze", + "excel": "PivotTable Analyze tab", + "note": "Contextual: only while the active cell is in a PivotTable (Table Design's JT letters follow otherwise)." + }, + { + "seq": "JY", + "kind": "tab", + "target": "PivotTable Design", + "excel": "PivotTable Design tab", + "note": "Contextual: only while the active cell is in a PivotTable." + }, { "seq": "JTS", "kind": "command", diff --git a/crates/ui-egui/src/lib.rs b/crates/ui-egui/src/lib.rs index f96cdac3c..366843886 100644 --- a/crates/ui-egui/src/lib.rs +++ b/crates/ui-egui/src/lib.rs @@ -26,12 +26,14 @@ pub mod outline; pub mod panes; pub mod paper_view; pub mod pivot_pane; +pub mod pivot_ui; pub mod printing; pub mod ribbon; pub mod selection_glide; pub mod tabs; pub mod text_box; pub mod theme; +pub mod timeline_view; pub mod widgets; use std::collections::HashMap; @@ -60,6 +62,9 @@ pub struct UiState { /// Enable AutoComplete for cell values (File ▸ Options ▸ Editing): typed text is completed /// from the one matching entry in the same column. pub autocomplete: bool, + /// PivotTable Analyze › Show › Field List: the PivotTable Fields pane shows beside a + /// PivotTable's report. + pub pivot_field_list: bool, pub recent: Vec, /// One-shot keytip request for a ribbon control: set when a keytip action resolves to /// [`keytips::Action::Control`], consumed (and cleared) by the control's render site. @@ -103,6 +108,7 @@ impl Default for UiState { status_bar: true, animations: true, autocomplete: true, + pivot_field_list: true, recent: vec![], keytip_invoke: None, language: i18n::Language::En, @@ -608,6 +614,10 @@ impl SheetApp { self.ui.autocomplete = p.get("on").and_then(Json::as_bool).unwrap_or(!self.ui.autocomplete); Ok(json!({"on": self.ui.autocomplete})) } + "pivot.showFieldList" => { + self.ui.pivot_field_list = p.get("on").and_then(Json::as_bool).unwrap_or(!self.ui.pivot_field_list); + Ok(json!({"on": self.ui.pivot_field_list})) + } "view.interfaceScale" => { // `{"percent": 125}` or `{"percent": "auto"}`; out-of-range values are clamped. let percent = match p.get("percent") { @@ -1187,8 +1197,13 @@ impl SheetApp { for action in actions { match action { keytips::Action::Tab(tab) => { - self.run_or_alert("ui.ribbonTab", json!({"tab": tab})); - self.ui.ribbon_collapsed = false; + // A contextual tab's sequence works only while the tab shows: PivotTable + // Analyze's JT also leads to Table Design's commands. + let contextual = [ribbon::pivot_tabs::ANALYZE, ribbon::pivot_tabs::DESIGN].contains(&tab); + if !contextual || ribbon::contextual_tabs(self).contains(&tab) { + self.run_or_alert("ui.ribbonTab", json!({"tab": tab})); + self.ui.ribbon_collapsed = false; + } } keytips::Action::Command(id @ ("edit.copy" | "edit.cut")) => self.copy_to_clipboard(ctx, id), keytips::Action::Command("edit.paste") => self.paste_from_clipboard(), diff --git a/crates/ui-egui/src/pivot_pane.rs b/crates/ui-egui/src/pivot_pane.rs index 55917ef17..91cd2244d 100644 --- a/crates/ui-egui/src/pivot_pane.rs +++ b/crates/ui-egui/src/pivot_pane.rs @@ -3,21 +3,32 @@ //! with move/remove menus. Everything runs `pivot.*` commands. use egui::{Color32, Ui, vec2}; +use gridcraft_engine::model::PivotTable; use serde_json::{Value as Json, json}; use crate::SheetApp; use crate::i18n; use crate::theme::{self, Tokens}; -/// Whether the active cell is inside a pivot report (so the pane should show). -pub fn active_pivot(app: &SheetApp) -> Option { +/// The PivotTable whose report holds the active cell (or whose anchor it is): the pane and the +/// PivotTable Analyze and Design tabs show while there is one. +pub fn at_active_cell(app: &SheetApp) -> Option<&PivotTable> { let d = app.session.active()?; let sh = d.wb.active()?; let a = d.selection.active; - sh.pivots.iter().find(|p| p.last_range.is_some_and(|r| r.contains(a)) || p.anchor == a).map(|p| p.name.clone()) + sh.pivots.iter().find(|p| p.last_range.is_some_and(|r| r.contains(a)) || p.anchor == a) +} + +/// Whether the active cell is inside a pivot report (so the pane should show). +pub fn active_pivot(app: &SheetApp) -> Option { + at_active_cell(app).map(|p| p.name.clone()) } pub fn show(app: &mut SheetApp, ui: &mut Ui) { + // PivotTable Analyze › Show › Field List hides the pane. + if !app.ui.pivot_field_list { + return; + } let Some(name) = active_pivot(app) else { return }; let t = Tokens::get(ui.ctx()); let lang = i18n::current(ui.ctx()); @@ -61,6 +72,8 @@ pub fn show(app: &mut SheetApp, ui: &mut Ui) { .or(info[key].as_array()) .map(|a| { a.iter() + // A row or column field's own filter is not a report filter. + .filter(|x| x["report"].as_bool() != Some(false)) .filter_map(|x| x["name"].as_str().or(x["field"].as_str()).or(x["sourceCol"].as_str()).or(x.as_str()).map(str::to_string)) .collect() }) @@ -121,8 +134,15 @@ pub fn show(app: &mut SheetApp, ui: &mut Ui) { for (sa, l) in [ ("normal", "Show as No Calculation"), ("percentOfGrandTotal", "Show as % of Grand Total"), + ("percentOfColumnTotal", "Show as % of Column Total"), + ("percentOfRowTotal", "Show as % of Row Total"), + ("percentOfParentRowTotal", "Show as % of Parent Row Total"), + ("percentOfParentColumnTotal", "Show as % of Parent Column Total"), ("runningTotal", "Show as Running Total"), - ("rank", "Show as Rank"), + ("percentRunningTotal", "Show as % Running Total"), + ("rankAscending", "Show as Rank Smallest to Largest"), + ("rankDescending", "Show as Rank Largest to Smallest"), + ("index", "Show as Index"), ] { if ui.button(lang.tr(l)).clicked() { app.run_or_alert("pivot.valueSettings", json!({"pivot": name, "field": f, "showAs": sa})); @@ -139,12 +159,39 @@ pub fn show(app: &mut SheetApp, ui: &mut Ui) { ("years", "Group by Years"), ("quarters", "Group by Quarters"), ("months", "Group by Months"), + ("range", "Group by Ranges of 10"), ("none", "Ungroup"), ] { if ui.button(lang.tr(l)).clicked() { app.run_or_alert("pivot.group", json!({"pivot": name, "field": f, "by": g})); } } + ui.separator(); + if ui.button(format!("{} \"{f}\"", lang.tr("Clear Filter From"))).clicked() { + app.run_or_alert("pivot.clearFilter", json!({"pivot": name, "field": f})); + } + if *key != "filters" { + // Label Filters ▸ and Value Filters ▸, as on the report's field button. + ui.menu_button(lang.tr("Label Filters"), |ui| { + for (op, l, _) in crate::pivot_ui::LABEL_OPS { + if ui.button(lang.tr(l)).clicked() { + app.open_dialog("pivotLabelFilter", json!({"pivot": name, "field": f, "op": op})); + } + } + }); + ui.menu_button(lang.tr("Value Filters"), |ui| { + for op in crate::pivot_ui::VALUE_OPS { + let label = crate::pivot_ui::LABEL_OPS.iter().find(|(o, ..)| *o == op).map_or(op, |(_, l, _)| l); + if ui.button(lang.tr(label)).clicked() { + app.open_dialog("pivotValueFilter", json!({"pivot": name, "field": f, "op": op})); + } + } + ui.separator(); + if ui.button(lang.tr("Top 10…")).clicked() { + app.open_dialog("pivotTop10", json!({"pivot": name, "field": f})); + } + }); + } } ui.separator(); if ui.button(lang.tr("Remove Field")).clicked() { diff --git a/crates/ui-egui/src/pivot_ui.rs b/crates/ui-egui/src/pivot_ui.rs new file mode 100644 index 000000000..bd05391e4 --- /dev/null +++ b/crates/ui-egui/src/pivot_ui.rs @@ -0,0 +1,858 @@ +//! PivotTable dialogs and menus beyond the Fields pane: the field buttons on a report's +//! Row Labels / Column Labels / field header cells with their menu (sort, Label Filters ▸, +//! Value Filters ▸, the item checklist), the Label Filter, Value Filter and Top 10 dialogs, the +//! Grouping dialog, Insert Calculated Item, Calculated Item Solve Order, Insert Timelines and a +//! timeline's Report Connections. Every action runs an engine command. + +use egui::{Pos2, Rect, Ui, pos2, vec2}; +use gridcraft_engine::core::CellRef; +use gridcraft_engine::model::{PivotLayout, PivotTable, Sheet}; +use serde_json::{Value as Json, json}; + +use crate::SheetApp; +use crate::dialogs::Dialog; +use crate::grid::Geo; + +/// The dialogs this module draws. +pub const DIALOGS: [&str; 8] = [ + "pivotGrouping", + "pivotCalculatedItem", + "pivotSolveOrder", + "insertTimeline", + "timelineConnections", + "pivotLabelFilter", + "pivotValueFilter", + "pivotTop10", +]; + +/// Excel's title of one of [`DIALOGS`]. +pub fn title(name: &str) -> &'static str { + match name { + "pivotGrouping" => "Grouping", + "pivotCalculatedItem" => "Insert Calculated Item", + "pivotSolveOrder" => "Calculated Item Solve Order", + "insertTimeline" => "Insert Timelines", + "timelineConnections" => "Report Connections", + "pivotLabelFilter" => "Label Filter", + "pivotValueFilter" => "Value Filter", + _ => "Top 10 Filter", + } +} + +/// The label comparisons, in the order of Excel's Label Filters menu: (command name, menu item, dialog choice). +pub const LABEL_OPS: [(&str, &str, &str); 14] = [ + ("equals", "Equals…", "equals"), + ("doesNotEqual", "Does Not Equal…", "does not equal"), + ("beginsWith", "Begins With…", "begins with"), + ("doesNotBeginWith", "Does Not Begin With…", "does not begin with"), + ("endsWith", "Ends With…", "ends with"), + ("doesNotEndWith", "Does Not End With…", "does not end with"), + ("contains", "Contains…", "contains"), + ("doesNotContain", "Does Not Contain…", "does not contain"), + ("greaterThan", "Greater Than…", "is greater than"), + ("greaterThanOrEqual", "Greater Than Or Equal To…", "is greater than or equal to"), + ("lessThan", "Less Than…", "is less than"), + ("lessThanOrEqual", "Less Than Or Equal To…", "is less than or equal to"), + ("between", "Between…", "is between"), + ("notBetween", "Not Between…", "is not between"), +]; + +/// The value comparisons of Excel's Value Filters menu (Top 10… follows them). +pub const VALUE_OPS: [&str; 8] = + ["equals", "doesNotEqual", "greaterThan", "greaterThanOrEqual", "lessThan", "lessThanOrEqual", "between", "notBetween"]; + +fn op_entry(op: &str) -> (&'static str, &'static str, &'static str) { + LABEL_OPS.iter().copied().find(|(o, ..)| *o == op).unwrap_or(LABEL_OPS[0]) +} + +// ---------------------------------------------------------------- report field buttons + +/// A field button on a report's header cell. +#[derive(Clone, Debug, PartialEq)] +pub struct HeaderButton { + pub cell: CellRef, + pub pivot: String, + /// The fields it filters and sorts (Compact form's Row Labels covers every row field). + pub fields: Vec, + /// A report filter's button (its menu is the item list only). + pub page: bool, + /// One of its fields is filtered. + pub filtered: bool, +} + +/// The field buttons of the PivotTables on a sheet, where Excel draws them: Row Labels and +/// Column Labels in Compact form, each field's header otherwise, and each report filter. +pub fn header_buttons(sh: &Sheet) -> Vec { + let mut out = Vec::new(); + for pt in &sh.pivots { + if pt.last_range.is_none() { + continue; + } + let filtered = |f: &str| pt.filters.iter().any(|x| x.source_col.eq_ignore_ascii_case(f) && (x.selected.is_some() || x.rule.is_some())); + let on_axis = |f: &str| pt.rows.iter().chain(pt.columns.iter()).any(|a| a.source_col.eq_ignore_ascii_case(f)); + let a = pt.anchor; + // Report filters sit above the body, one row each, then a blank row. + let pages: Vec<&str> = pt.filters.iter().filter(|f| f.rule.is_none() && !on_axis(&f.source_col)).map(|f| f.source_col.as_str()).collect(); + let top = a.row.saturating_sub(pages.len() as u32 + u32::from(!pages.is_empty())); + for (i, f) in pages.iter().enumerate() { + out.push(HeaderButton { + cell: CellRef::new(top + i as u32, a.col.saturating_add(1)), + pivot: pt.name.clone(), + fields: vec![f.to_string()], + page: true, + filtered: filtered(f), + }); + } + if !pt.show_headers { + continue; + } + let k = pt.values.len(); + let on_rows = pt.values_on_rows && k > 1; + let rows: Vec = dedup(pt.rows.iter().map(|f| f.source_col.clone())); + let cols: Vec = dedup(pt.columns.iter().map(|f| f.source_col.clone())); + let row_levels = pt.rows.len() + usize::from(on_rows); + let compact = pt.layout == PivotLayout::Compact; + let l = if row_levels == 0 && cols.is_empty() { + 0 + } else if compact { + 1 + } else { + row_levels.max(1) + }; + let header_row = if cols.is_empty() { a.row } else { a.row + (pt.columns.len() + usize::from(k > 1 && !on_rows)) as u32 }; + if !cols.is_empty() { + let fields = if compact { cols.clone() } else { cols.iter().take(1).cloned().collect() }; + let any = fields.iter().any(|f| filtered(f)); + out.push(HeaderButton { + cell: CellRef::new(a.row, a.col.saturating_add(l as u32)), + pivot: pt.name.clone(), + fields, + page: false, + filtered: any, + }); + } + if rows.is_empty() { + continue; + } + if compact { + let any = rows.iter().any(|f| filtered(f)); + out.push(HeaderButton { cell: CellRef::new(header_row, a.col), pivot: pt.name.clone(), fields: rows, page: false, filtered: any }); + } else { + let vpos = on_rows.then(|| pt.values_position.unwrap_or(pt.rows.len())); + let mut col = a.col; + for (i, f) in pt.rows.iter().enumerate() { + if vpos == Some(i) { + col = col.saturating_add(1); + } + out.push(HeaderButton { + cell: CellRef::new(header_row, col), + pivot: pt.name.clone(), + fields: vec![f.source_col.clone()], + page: false, + filtered: filtered(&f.source_col), + }); + col = col.saturating_add(1); + } + } + } + out +} + +fn dedup(it: impl Iterator) -> Vec { + let mut v: Vec = Vec::new(); + for s in it { + if !v.iter().any(|x| x.eq_ignore_ascii_case(&s)) { + v.push(s); + } + } + v +} + +/// The button rect of a header cell (the filter button's place). +pub fn button_rect(geo: &Geo, sh: &Sheet, c: CellRef) -> Rect { + let rect = geo.cell_rect(sh, c); + let s = (rect.height() - 4.0).clamp(10.0, 18.0); + let pad = 2.0 * geo.z; + Rect::from_min_size(pos2(rect.right() - s - pad, rect.bottom() - s - pad), vec2(s, s)) +} + +/// The field button under `p`, if any. +pub fn button_at(sh: &Sheet, geo: &Geo, p: Pos2) -> Option { + header_buttons(sh).into_iter().find(|b| button_rect(geo, sh, b.cell).contains(p)) +} + +/// The open field menu. +#[derive(Clone, Debug, PartialEq)] +pub struct PivotMenu { + pub button: HeaderButton, + pub at: Pos2, +} + +/// A PivotTable by name on the active workbook. +fn pivot_by_name(app: &SheetApp, name: &str) -> Option { + app.session.active()?.wb.sheets.iter().flat_map(|s| s.pivots.iter()).find(|p| p.name.eq_ignore_ascii_case(name)).cloned() +} + +/// The menu of a report's field button: Select field (Compact form), Sort A to Z / Z to A, +/// Clear Filter From, Label Filters ▸, Value Filters ▸, then the searchable item checklist. +pub fn field_menu(app: &mut SheetApp, ui: &mut Ui, was_open: bool) { + let Some(menu) = app.grid.pivot_menu.clone() else { return }; + let lang = crate::i18n::current(ui.ctx()); + let pivot = menu.button.pivot.clone(); + let field_key = egui::Id::new(("pivot_menu_field", &pivot, menu.button.cell.row, menu.button.cell.col)); + let mut field: String = ui.ctx().data(|d| d.get_temp(field_key)).unwrap_or_else(|| menu.button.fields.first().cloned().unwrap_or_default()); + if !menu.button.fields.iter().any(|f| f.eq_ignore_ascii_case(&field)) { + field = menu.button.fields.first().cloned().unwrap_or_default(); + } + let items_key = egui::Id::new(("pivot_menu_items", &pivot, &field)); + let search_key = egui::Id::new(("pivot_menu_search", &pivot, &field)); + let mut checks: Vec<(String, bool)> = match ui.ctx().data(|d| d.get_temp::>(items_key)) { + Some(v) => v, + None => app + .session + .run("pivot.fieldItems", json!({"pivot": pivot, "field": field})) + .ok() + .and_then(|r| r["items"].as_array().cloned()) + .unwrap_or_default() + .iter() + .map(|i| (i["label"].as_str().unwrap_or("").to_string(), i["shown"].as_bool().unwrap_or(true))) + .collect(), + }; + let mut close = false; + let area = egui::Area::new(egui::Id::new("pivot_field_menu")).fixed_pos(menu.at + vec2(-200.0, 10.0)).order(egui::Order::Foreground); + let response = area.show(ui.ctx(), |ui| { + egui::Frame::popup(ui.style()).show(ui, |ui| { + ui.set_width(250.0); + if menu.button.fields.len() > 1 { + ui.horizontal(|ui| { + ui.label(lang.tr("Select field:")); + egui::ComboBox::from_id_salt("pivot_menu_select").selected_text(field.as_str()).show_ui(ui, |ui| { + for f in &menu.button.fields { + if ui.selectable_label(f.eq_ignore_ascii_case(&field), f.as_str()).clicked() { + field = f.clone(); + } + } + }); + }); + } + if !menu.button.page { + for (order, label) in [("asc", "Sort A to Z"), ("desc", "Sort Z to A")] { + if ui.button(lang.tr(label)).clicked() { + app.run_or_alert("pivot.sort", json!({"pivot": pivot, "field": field, "order": order})); + close = true; + } + } + ui.add_enabled(false, egui::Button::new(lang.tr("More Sort Options…"))); + ui.separator(); + } + let clear_label = format!("{} \"{field}\"", lang.tr("Clear Filter From")); + if ui.button(clear_label).clicked() { + app.run_or_alert("pivot.clearFilter", json!({"pivot": pivot, "field": field})); + close = true; + } + if !menu.button.page { + ui.menu_button(lang.tr("Label Filters"), |ui| { + if ui.button(lang.tr("Clear Filter")).clicked() { + app.run_or_alert("pivot.clearFilter", json!({"pivot": pivot, "field": field})); + close = true; + } + ui.separator(); + for (op, label, _) in LABEL_OPS { + if ui.button(lang.tr(label)).clicked() { + app.open_dialog("pivotLabelFilter", json!({"pivot": pivot, "field": field, "op": op})); + close = true; + } + } + }); + ui.menu_button(lang.tr("Value Filters"), |ui| { + if ui.button(lang.tr("Clear Filter")).clicked() { + app.run_or_alert("pivot.clearFilter", json!({"pivot": pivot, "field": field})); + close = true; + } + ui.separator(); + for op in VALUE_OPS { + if ui.button(lang.tr(op_entry(op).1)).clicked() { + app.open_dialog("pivotValueFilter", json!({"pivot": pivot, "field": field, "op": op})); + close = true; + } + } + ui.separator(); + if ui.button(lang.tr("Top 10…")).clicked() { + app.open_dialog("pivotTop10", json!({"pivot": pivot, "field": field})); + close = true; + } + }); + } + ui.separator(); + let mut search: String = ui.ctx().data(|d| d.get_temp(search_key)).unwrap_or_default(); + let edit = ui.add(egui::TextEdit::singleline(&mut search).hint_text(lang.tr("Search")).desired_width(f32::INFINITY)); + if !was_open { + edit.request_focus(); + } + let needle = search.trim().to_lowercase(); + let matches = |v: &str| needle.is_empty() || v.to_lowercase().contains(&needle); + let all = checks.iter().filter(|(v, _)| matches(v)).all(|(_, b)| *b); + let mut all_new = all; + if ui.checkbox(&mut all_new, lang.tr("(Select All)")).changed() { + for c in checks.iter_mut().filter(|(v, _)| matches(v)) { + c.1 = all_new; + } + } + egui::ScrollArea::vertical().max_height(200.0).auto_shrink([false, true]).show(ui, |ui| { + for (v, on) in checks.iter_mut().filter(|(v, _)| matches(v)) { + ui.checkbox(on, v.as_str()); + } + }); + ui.ctx().data_mut(|d| d.insert_temp(search_key, search.clone())); + ui.separator(); + ui.horizontal(|ui| { + if ui.button(lang.tr("OK")).clicked() { + let shown: Vec = checks.iter().filter(|(v, on)| *on && matches(v)).map(|(v, _)| v.clone()).collect(); + let every = checks.iter().all(|(v, on)| *on && matches(v)); + let selected = if every { Json::Null } else { json!(shown) }; + app.run_or_alert("pivot.filter", json!({"pivot": pivot, "field": field, "selected": selected})); + close = true; + } + if ui.button(lang.tr("Cancel")).clicked() { + close = true; + } + }); + }); + }); + ui.ctx().data_mut(|d| { + d.insert_temp(field_key, field.clone()); + d.insert_temp(items_key, checks.clone()); + }); + let dismissed = response.response.ctx.input(|i| i.key_pressed(egui::Key::Escape)) || (was_open && response.response.clicked_elsewhere()); + if close || dismissed { + ui.ctx().data_mut(|d| { + d.remove::>(items_key); + d.remove::(search_key); + }); + app.grid.pivot_menu = None; + } +} + +// ---------------------------------------------------------------- dialogs + +/// Initial values of one of [`DIALOGS`], from the parameters it was opened with. +pub fn defaults(app: &SheetApp, name: &str, params: &Json) -> Json { + let mut v = params.clone(); + if !v.is_object() { + v = json!({}); + } + let pivot = params["pivot"].as_str().unwrap_or("").to_string(); + let pt = pivot_by_name(app, &pivot); + let value_fields: Vec = pt.as_ref().map(|p| p.values.iter().map(|x| x.name.clone()).collect()).unwrap_or_default(); + if let Some(m) = v.as_object_mut() { + match name { + "pivotLabelFilter" => { + m.entry("value").or_insert(json!("")); + m.entry("value2").or_insert(json!("")); + } + "pivotValueFilter" => { + m.entry("valueField").or_insert(json!(value_fields.first().cloned().unwrap_or_default())); + m.insert("valueFields".into(), json!(value_fields)); + m.entry("value").or_insert(json!("")); + m.entry("value2").or_insert(json!("")); + } + "pivotTop10" => { + m.entry("top").or_insert(json!(true)); + m.entry("count").or_insert(json!(10)); + m.entry("by").or_insert(json!("items")); + m.entry("valueField").or_insert(json!(value_fields.first().cloned().unwrap_or_default())); + m.insert("valueFields".into(), json!(value_fields)); + } + "pivotGrouping" => { + let sys = app.session.active().map(|d| d.wb.date_system).unwrap_or_default(); + for key in ["start", "end"] { + if params["kind"] == "date" + && let Some(n) = params[key].as_f64() + && let Some(d) = gridcraft_engine::core::date::datetime_from_serial(sys, n) + { + m.insert(format!("{key}Text"), json!(format!("{}/{}/{}", d.month, d.day, d.year))); + } else if let Some(n) = params[key].as_f64() { + m.insert(format!("{key}Text"), json!(gridcraft_engine::core::number_to_text(n))); + } + } + } + "insertTimeline" => { + m.insert("checked".into(), json!([])); + } + "timelineConnections" => { + let on: Vec = params["pivots"] + .as_array() + .cloned() + .unwrap_or_default() + .iter() + .filter(|p| p["connected"] == true) + .map(|p| p["name"].clone()) + .collect(); + m.insert("checked".into(), json!(on)); + } + _ => {} + } + } + v +} + +fn text_of(d: &Dialog, key: &str) -> String { + match d.values.get(key) { + Some(Json::String(s)) => s.clone(), + Some(Json::Number(n)) => n.to_string(), + _ => String::new(), + } +} + +/// A one-line text field bound to `key`. +fn text_field(ui: &mut Ui, d: &mut Dialog, key: &str, width: f32) { + let mut s = text_of(d, key); + if ui.add(egui::TextEdit::singleline(&mut s).desired_width(width)).changed() { + d.values.insert(key.into(), json!(s)); + } +} + +/// A drop-down bound to `key` over (value, caption) pairs. +fn combo(ui: &mut Ui, d: &mut Dialog, key: &str, options: &[(String, String)], width: f32) { + let lang = crate::i18n::current(ui.ctx()); + let cur = d.values.get(key).and_then(Json::as_str).unwrap_or("").to_string(); + let shown = options.iter().find(|(v, _)| *v == cur).map(|(_, c)| lang.tr(c).to_string()).unwrap_or(cur.clone()); + egui::ComboBox::from_id_salt(("pivot_dialog", key)).selected_text(shown).width(width).show_ui(ui, |ui| { + for (v, c) in options { + if ui.selectable_label(*v == cur, lang.tr(c)).clicked() { + d.values.insert(key.into(), json!(v)); + } + } + }); +} + +/// A checklist bound to the string list `key`. +fn checklist(ui: &mut Ui, d: &mut Dialog, key: &str, items: &[(String, String)]) { + let mut on: Vec = + d.values.get(key).and_then(Json::as_array).map(|a| a.iter().filter_map(|x| x.as_str().map(str::to_string)).collect()).unwrap_or_default(); + egui::ScrollArea::vertical().max_height(220.0).show(ui, |ui| { + for (value, caption) in items { + let mut b = on.iter().any(|x| x == value); + if ui.checkbox(&mut b, caption.as_str()).changed() { + if b { + on.push(value.clone()); + } else { + on.retain(|x| x != value); + } + } + } + }); + d.values.insert(key.into(), json!(on)); +} + +/// Runs a dialog's command; an error keeps the dialog open with the message. +fn run(app: &mut SheetApp, d: &Dialog, cmd: &str, params: Json, confirm: &mut bool) -> bool { + match app.run(cmd, params) { + Ok(_) => true, + Err(e) => { + app.message = Some((d.title.clone(), crate::clean_error(&e))); + *confirm = false; + false + } + } +} + +/// Draws one of [`DIALOGS`]. +pub fn dialog(app: &mut SheetApp, ui: &mut Ui, d: &mut Dialog, confirm: &mut bool, open: &mut bool) { + let lang = crate::i18n::current(ui.ctx()); + let s = |k: &str, d: &Dialog| d.values.get(k).and_then(Json::as_str).unwrap_or("").to_string(); + match d.name() { + "pivotLabelFilter" => { + ui.label(format!("{} ({})", lang.tr("Show items for which the label"), s("field", d))); + ui.horizontal(|ui| { + let ops: Vec<(String, String)> = LABEL_OPS.iter().map(|(o, _, c)| (o.to_string(), c.to_string())).collect(); + combo(ui, d, "op", &ops, 170.0); + text_field(ui, d, "value", 120.0); + }); + let range = matches!(s("op", d).as_str(), "between" | "notBetween"); + if range { + ui.horizontal(|ui| { + ui.label(lang.tr("and")); + text_field(ui, d, "value2", 120.0); + }); + } + ui.label(egui::RichText::new(lang.tr("Use ? to represent any single character")).small()); + ui.label(egui::RichText::new(lang.tr("Use * to represent any series of characters")).small()); + crate::dialogs::ok_cancel(ui, confirm, open); + if *confirm { + let mut p = json!({"pivot": s("pivot", d), "field": s("field", d), "op": s("op", d), "value": text_of(d, "value")}); + if range { + p["value2"] = json!(text_of(d, "value2")); + } + run(app, d, "pivot.labelFilter", p, confirm); + } + } + "pivotValueFilter" => { + ui.label(format!("{} ({})", lang.tr("Show items for which"), s("field", d))); + let fields: Vec<(String, String)> = d + .values + .get("valueFields") + .and_then(Json::as_array) + .cloned() + .unwrap_or_default() + .iter() + .filter_map(|x| x.as_str()) + .map(|x| (x.to_string(), x.to_string())) + .collect(); + ui.horizontal(|ui| { + combo(ui, d, "valueField", &fields, 130.0); + let ops: Vec<(String, String)> = VALUE_OPS.iter().map(|o| (o.to_string(), op_entry(o).2.to_string())).collect(); + combo(ui, d, "op", &ops, 150.0); + text_field(ui, d, "value", 80.0); + }); + let range = matches!(s("op", d).as_str(), "between" | "notBetween"); + if range { + ui.horizontal(|ui| { + ui.label(lang.tr("and")); + text_field(ui, d, "value2", 80.0); + }); + } + crate::dialogs::ok_cancel(ui, confirm, open); + if *confirm { + let num = |k: &str| text_of(d, k).trim().replace(',', ".").parse::().ok(); + let Some(value) = num("value") else { + app.message = Some((d.title.clone(), "Enter a number.".into())); + *confirm = false; + return; + }; + let mut p = + json!({"pivot": s("pivot", d), "field": s("field", d), "op": s("op", d), "value": value, "valueField": s("valueField", d)}); + if range { + p["value2"] = json!(num("value2").unwrap_or(0.0)); + } + run(app, d, "pivot.valueFilter", p, confirm); + } + } + "pivotTop10" => { + ui.label(format!("{} ({})", lang.tr("Show"), s("field", d))); + let fields: Vec<(String, String)> = d + .values + .get("valueFields") + .and_then(Json::as_array) + .cloned() + .unwrap_or_default() + .iter() + .filter_map(|x| x.as_str()) + .map(|x| (x.to_string(), x.to_string())) + .collect(); + ui.horizontal(|ui| { + let mut top = d.values.get("top").and_then(Json::as_bool).unwrap_or(true); + egui::ComboBox::from_id_salt("top10_side").selected_text(lang.tr(if top { "Top" } else { "Bottom" })).width(70.0).show_ui(ui, |ui| { + ui.selectable_value(&mut top, true, lang.tr("Top")); + ui.selectable_value(&mut top, false, lang.tr("Bottom")); + }); + d.values.insert("top".into(), json!(top)); + let mut n = d.values.get("count").and_then(Json::as_f64).unwrap_or(10.0); + if ui.add(egui::DragValue::new(&mut n).speed(1.0).range(0.0..=2_147_483_647.0)).changed() { + d.values.insert("count".into(), json!(n)); + } + let by = [("items", "Items"), ("percent", "Percent"), ("sum", "Sum")].map(|(a, b)| (a.to_string(), b.to_string())); + combo(ui, d, "by", &by, 80.0); + ui.label(lang.tr("by")); + combo(ui, d, "valueField", &fields, 130.0); + }); + crate::dialogs::ok_cancel(ui, confirm, open); + if *confirm { + let p = json!({ + "pivot": s("pivot", d), "field": s("field", d), "top": d.values.get("top").cloned().unwrap_or(json!(true)), + "count": d.values.get("count").cloned().unwrap_or(json!(10)), "by": s("by", d), "valueField": s("valueField", d), + }); + run(app, d, "pivot.top10", p, confirm); + } + } + "pivotGrouping" => grouping(app, ui, d, confirm, open), + "pivotCalculatedItem" => calculated_item(app, ui, d, confirm, open), + "pivotSolveOrder" => { + ui.label(lang.tr("Solve order:")); + let list: Vec = d.values.get("calculatedItems").and_then(Json::as_array).cloned().unwrap_or_default(); + let mut sel = d.values.get("selected").and_then(Json::as_u64).unwrap_or(0) as usize; + egui::ScrollArea::vertical().max_height(160.0).show(ui, |ui| { + for (i, c) in list.iter().enumerate() { + let text = format!("'{}' = {}", c["name"].as_str().unwrap_or(""), c["formula"].as_str().unwrap_or("").trim_start_matches('=')); + if ui.selectable_label(i == sel, text).clicked() { + sel = i; + } + } + }); + d.values.insert("selected".into(), json!(sel)); + let name = list.get(sel).and_then(|c| c["name"].as_str()).unwrap_or("").to_string(); + let pivot = s("pivot", d); + ui.horizontal(|ui| { + for (label, step) in [("Move Up", "up"), ("Move Down", "down")] { + if ui.button(lang.tr(label)).clicked() + && !name.is_empty() + && let Ok(r) = app.run("pivot.solveOrder", json!({"pivot": pivot, "item": name, "move": step})) + { + let order: Vec = + r["order"].as_array().cloned().unwrap_or_default().iter().filter_map(|x| x.as_str().map(str::to_string)).collect(); + let mut sorted: Vec = + order.iter().filter_map(|n| list.iter().find(|c| c["name"].as_str() == Some(n)).cloned()).collect(); + if sorted.len() != list.len() { + sorted = list.clone(); + } + let at = order.iter().position(|n| *n == name).unwrap_or(sel); + d.values.insert("calculatedItems".into(), json!(sorted)); + d.values.insert("selected".into(), json!(at)); + } + } + if ui.button(lang.tr("Delete")).clicked() && !name.is_empty() { + let field = list.get(sel).and_then(|c| c["field"].as_str()).unwrap_or("").to_string(); + if app.run("pivot.calculatedItem", json!({"pivot": pivot, "field": field, "name": name, "delete": true})).is_ok() { + let rest: Vec = list.iter().filter(|c| c["name"].as_str() != Some(name.as_str())).cloned().collect(); + d.values.insert("calculatedItems".into(), json!(rest)); + d.values.insert("selected".into(), json!(0)); + } + } + if ui.button(lang.tr("Close")).clicked() { + *open = false; + } + }); + } + "insertTimeline" => { + let fields: Vec<(String, String)> = d + .values + .get("fields") + .and_then(Json::as_array) + .cloned() + .unwrap_or_default() + .iter() + .filter_map(|x| x.as_str()) + .map(|x| (x.to_string(), x.to_string())) + .collect(); + checklist(ui, d, "checked", &fields); + crate::dialogs::ok_cancel(ui, confirm, open); + if *confirm { + let chosen = d.values.get("checked").cloned().unwrap_or(json!([])); + if chosen.as_array().is_none_or(Vec::is_empty) { + return; + } + run(app, d, "insert.timeline", json!({"pivot": s("pivot", d), "fields": chosen}), confirm); + } + } + "timelineConnections" => { + let pivots: Vec<(String, String)> = d + .values + .get("pivots") + .and_then(Json::as_array) + .cloned() + .unwrap_or_default() + .iter() + .map(|p| { + let n = p["name"].as_str().unwrap_or("").to_string(); + (n.clone(), format!("{n} ({})", p["sheet"].as_str().unwrap_or(""))) + }) + .collect(); + checklist(ui, d, "checked", &pivots); + crate::dialogs::ok_cancel(ui, confirm, open); + if *confirm { + let chosen = d.values.get("checked").cloned().unwrap_or(json!([])); + run(app, d, "timeline.reportConnections", json!({"timeline": s("timeline", d), "pivots": chosen}), confirm); + } + } + _ => crate::dialogs::ok_cancel(ui, confirm, open), + } +} + +/// The Grouping dialog: Starting at, Ending at and By for numbers; for dates the same bounds, +/// the levels to group by and Number of days. +fn grouping(app: &mut SheetApp, ui: &mut Ui, d: &mut Dialog, confirm: &mut bool, open: &mut bool) { + let lang = crate::i18n::current(ui.ctx()); + let date = d.values.get("kind").and_then(Json::as_str) == Some("date"); + ui.label(egui::RichText::new(lang.tr("Auto")).strong()); + for (auto, key, label) in [("autoStart", "startText", "Starting at:"), ("autoEnd", "endText", "Ending at:")] { + ui.horizontal(|ui| { + let mut a = d.values.get(auto).and_then(Json::as_bool).unwrap_or(true); + if ui.checkbox(&mut a, lang.tr(label)).changed() { + d.values.insert(auto.into(), json!(a)); + } + ui.add_enabled_ui(!a, |ui| text_field(ui, d, key, 120.0)); + }); + } + let levels_all = [ + ("seconds", "Seconds"), + ("minutes", "Minutes"), + ("hours", "Hours"), + ("days", "Days"), + ("months", "Months"), + ("quarters", "Quarters"), + ("years", "Years"), + ]; + if date { + ui.label(egui::RichText::new(lang.tr("By")).strong()); + let mut levels: Vec = d + .values + .get("levels") + .and_then(Json::as_array) + .map(|a| a.iter().filter_map(|x| x.as_str().map(str::to_string)).collect()) + .unwrap_or_default(); + egui::Frame::group(ui.style()).show(ui, |ui| { + for (v, label) in levels_all { + let on = levels.iter().any(|x| x == v); + if ui.selectable_label(on, lang.tr(label)).clicked() { + if on { + levels.retain(|x| x != v); + } else { + levels.push(v.to_string()); + } + } + } + }); + d.values.insert("levels".into(), json!(levels)); + let only_days = levels.len() == 1 && levels.first().is_some_and(|l| l == "days"); + ui.add_enabled_ui(only_days, |ui| { + ui.horizontal(|ui| { + ui.label(lang.tr("Number of days:")); + let mut n = d.values.get("days").and_then(Json::as_f64).unwrap_or(1.0); + if ui.add(egui::DragValue::new(&mut n).speed(1.0).range(1.0..=32767.0)).changed() { + d.values.insert("days".into(), json!(n.round())); + } + }); + }); + } else { + ui.horizontal(|ui| { + ui.label(lang.tr("By:")); + let mut n = d.values.get("interval").and_then(Json::as_f64).unwrap_or(10.0); + if ui.add(egui::DragValue::new(&mut n).speed(0.5)).changed() { + d.values.insert("interval".into(), json!(n)); + } + }); + } + crate::dialogs::ok_cancel(ui, confirm, open); + if !*confirm { + return; + } + let s = |k: &str| d.values.get(k).and_then(Json::as_str).unwrap_or("").to_string(); + let auto = |k: &str| d.values.get(k).and_then(Json::as_bool).unwrap_or(true); + let mut p = json!({"pivot": s("pivot"), "field": s("field")}); + if !auto("autoStart") { + p["start"] = json!(text_of(d, "startText")); + } + if !auto("autoEnd") { + p["end"] = json!(text_of(d, "endText")); + } + if date { + let levels = d.values.get("levels").cloned().unwrap_or(json!([])); + if levels.as_array().is_none_or(Vec::is_empty) { + app.message = Some((d.title.clone(), "Select one or more time periods to group by.".into())); + *confirm = false; + return; + } + let days = d.values.get("days").and_then(Json::as_f64).unwrap_or(1.0); + p["by"] = levels; + if days > 1.0 { + p["days"] = json!(days as u64); + } + } else { + p["by"] = json!("range"); + p["interval"] = d.values.get("interval").cloned().unwrap_or(json!(10)); + } + run(app, d, "pivot.group", p, confirm); +} + +/// Insert Calculated Item: Name, Formula, the Fields and Items lists with Insert Item, and Add / +/// Delete for the field's calculated items. +fn calculated_item(app: &mut SheetApp, ui: &mut Ui, d: &mut Dialog, confirm: &mut bool, open: &mut bool) { + let lang = crate::i18n::current(ui.ctx()); + let s = |k: &str, d: &Dialog| d.values.get(k).and_then(Json::as_str).unwrap_or("").to_string(); + let (pivot, field) = (s("pivot", d), s("field", d)); + ui.label(format!("{} \"{field}\"", lang.tr("Calculated item in"))); + let existing: Vec = d.values.get("calculatedItems").and_then(Json::as_array).cloned().unwrap_or_default(); + let mine: Vec<(String, String)> = existing + .iter() + .filter(|c| c["field"].as_str().is_some_and(|f| f.eq_ignore_ascii_case(&field))) + .map(|c| (c["name"].as_str().unwrap_or("").to_string(), c["formula"].as_str().unwrap_or("").to_string())) + .collect(); + ui.horizontal(|ui| { + ui.label(lang.tr("Name:")); + let mut name = s("name", d); + egui::ComboBox::from_id_salt("calc_item_name").selected_text(name.clone()).width(160.0).show_ui(ui, |ui| { + for (n, f) in &mine { + if ui.selectable_label(*n == name, n.as_str()).clicked() { + name = n.clone(); + d.values.insert("formula".into(), json!(f)); + } + } + }); + if ui.add(egui::TextEdit::singleline(&mut name).desired_width(140.0)).changed() || mine.iter().any(|(n, _)| *n == name) { + d.values.insert("name".into(), json!(name)); + } + }); + ui.horizontal(|ui| { + ui.label(lang.tr("Formula:")); + text_field(ui, d, "formula", 280.0); + }); + let fields: Vec = + d.values.get("fields").and_then(Json::as_array).cloned().unwrap_or_default().iter().filter_map(|x| x.as_str().map(str::to_string)).collect(); + let items: Vec = + d.values.get("items").and_then(Json::as_array).cloned().unwrap_or_default().iter().filter_map(|x| x.as_str().map(str::to_string)).collect(); + let mut item_sel = d.values.get("itemSelected").and_then(Json::as_u64).unwrap_or(0) as usize; + ui.columns(2, |cols| { + if let Some(ui) = cols.get_mut(0) { + ui.label(lang.tr("Fields:")); + egui::ScrollArea::vertical().id_salt("calc_fields").max_height(140.0).show(ui, |ui| { + for f in &fields { + if ui.selectable_label(f.eq_ignore_ascii_case(&field), f.as_str()).clicked() && !f.eq_ignore_ascii_case(&field) { + // Another field: the dialog for it (its items, its calculated items). + let _ = app.run("pivot.calculatedItem", json!({"pivot": pivot, "field": f})); + *open = false; + } + } + }); + } + if let Some(ui) = cols.get_mut(1) { + ui.label(lang.tr("Items:")); + egui::ScrollArea::vertical().id_salt("calc_items").max_height(140.0).show(ui, |ui| { + for (i, it) in items.iter().enumerate() { + if ui.selectable_label(i == item_sel, it.as_str()).clicked() { + item_sel = i; + } + } + }); + } + }); + d.values.insert("itemSelected".into(), json!(item_sel)); + ui.horizontal(|ui| { + if ui.button(lang.tr("Insert Item")).clicked() + && let Some(it) = items.get(item_sel) + { + let plain = it.chars().all(|c| c.is_alphanumeric() || c == '_') && !it.chars().next().is_some_and(|c| c.is_ascii_digit()); + let name = if plain { it.clone() } else { format!("'{}'", it.replace('\'', "''")) }; + let mut f = text_of(d, "formula"); + if f.trim() == "= 0" || f.trim().is_empty() { + f = "= ".into(); + } + f.push_str(&name); + d.values.insert("formula".into(), json!(f)); + } + let name = s("name", d); + let modify = mine.iter().any(|(n, _)| n.eq_ignore_ascii_case(&name)); + if ui.button(lang.tr(if modify { "Modify" } else { "Add" })).clicked() { + let p = json!({"pivot": pivot, "field": field, "name": name, "formula": text_of(d, "formula")}); + match app.run("pivot.calculatedItem", p) { + Ok(r) => { + d.values.insert("calculatedItems".into(), r["calculatedItems"].clone()); + } + Err(e) => app.message = Some((d.title.clone(), crate::clean_error(&e))), + } + } + if ui.add_enabled(modify, egui::Button::new(lang.tr("Delete"))).clicked() + && let Ok(r) = app.run("pivot.calculatedItem", json!({"pivot": pivot, "field": field, "name": name, "delete": true})) + { + d.values.insert("calculatedItems".into(), r["calculatedItems"].clone()); + } + }); + crate::dialogs::ok_cancel(ui, confirm, open); + if *confirm { + let name = s("name", d); + let formula = text_of(d, "formula"); + let unchanged = mine.iter().any(|(n, f)| n.eq_ignore_ascii_case(&name) && f.trim() == formula.trim()); + if !name.trim().is_empty() && !unchanged && formula.trim() != "= 0" { + run(app, d, "pivot.calculatedItem", json!({"pivot": pivot, "field": field, "name": name, "formula": formula}), confirm); + } + } +} diff --git a/crates/ui-egui/src/ribbon.rs b/crates/ui-egui/src/ribbon.rs index 9c6d89eeb..2476fc8a0 100644 --- a/crates/ui-egui/src/ribbon.rs +++ b/crates/ui-egui/src/ribbon.rs @@ -10,6 +10,8 @@ use crate::keytips::keytip_table; use crate::theme::{self, Tokens}; use crate::widgets::{big_button, color_palette, icon_button, small_button, split_button, toggle_button}; +pub mod pivot_tabs; + pub const TABS: &[&str] = &["Home", "Insert", "Draw", "Page Layout", "Formulas", "Data", "Review", "View", "Automate"]; /// macOS draws the window into the title bar (`with_fullsize_content_view`) and AppKit paints the @@ -179,6 +181,12 @@ pub fn show(app: &mut SheetApp, ui: &mut Ui) { file_menu(app, ui); let mut tabs: Vec<&str> = TABS.to_vec(); let ctx_tabs = contextual_tabs(app); + // Leaving the PivotTable takes its tabs away: the ribbon goes back to Home. + if [pivot_tabs::ANALYZE, pivot_tabs::DESIGN].contains(&app.ui.ribbon_tab.as_str()) + && !ctx_tabs.contains(&app.ui.ribbon_tab.as_str()) + { + app.ui.ribbon_tab = "Home".into(); + } tabs.extend(ctx_tabs.iter()); for tab in tabs { let active = app.ui.ribbon_tab == tab; @@ -245,6 +253,9 @@ pub fn show(app: &mut SheetApp, ui: &mut Ui) { "Automate" => automate(app, ui), "Table Design" => table_design(app, ui), "Chart Design" => chart_design(app, ui), + "Timeline" => crate::timeline_view::ribbon_tab(app, ui), + pivot_tabs::ANALYZE => pivot_tabs::analyze(app, ui), + pivot_tabs::DESIGN => pivot_tabs::design(app, ui), crate::paper_view::TAB => crate::paper_view::ribbon_tab(app, ui), _ => home(app, ui), } @@ -277,7 +288,8 @@ fn theme_choices(app: &mut SheetApp, ui: &mut Ui) { } } -fn contextual_tabs(app: &SheetApp) -> Vec<&'static str> { +/// The contextual tabs to show after the fixed ones: they come and go with what's selected. +pub(crate) fn contextual_tabs(app: &SheetApp) -> Vec<&'static str> { let mut v = Vec::new(); if let Some(d) = app.session.active() && let Some(sh) = d.wb.active() @@ -285,9 +297,16 @@ fn contextual_tabs(app: &SheetApp) -> Vec<&'static str> { if sh.table_at(d.selection.active).is_some() { v.push("Table Design"); } + if crate::pivot_pane::at_active_cell(app).is_some() { + v.push(pivot_tabs::ANALYZE); + v.push(pivot_tabs::DESIGN); + } if app.selected_chart.is_some_and(|id| sh.charts.iter().any(|c| c.id == id)) { v.push("Chart Design"); } + if app.selected_chart.is_some_and(|id| sh.timelines.iter().any(|t| t.id == id)) { + v.push("Timeline"); + } if app.grid.hf_edit.is_some() { v.push(crate::paper_view::TAB); } @@ -295,7 +314,7 @@ fn contextual_tabs(app: &SheetApp) -> Vec<&'static str> { v } -fn sep(ui: &mut Ui) { +pub(crate) fn sep(ui: &mut Ui) { let t = Tokens::get(ui.ctx()); let (r, _) = ui.allocate_exact_size(vec2(13.0, 72.0), Sense::hover()); ui.painter().line_segment([pos2(r.center().x, r.top() + 6.0), pos2(r.center().x, r.bottom() - 6.0)], Stroke::new(1.0, t.separator)); @@ -1524,6 +1543,12 @@ fn insert(app: &mut SheetApp, ui: &mut Ui) { if big_button(ui, Icon::Filter, "Slicer", "Insert a slicer to filter a table", false).clicked() { act(app, "insert.slicer", json!({})); } + let __k = big_button(ui, Icon::Calendar, "Timeline", "Insert a timeline to filter a PivotTable by date", false); + __k.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, ui.is_enabled(), app.ui.language.tr("Timeline"))); + control_keytip(app, ui, &__k, "insert.timeline"); + if __k.clicked() { + act(app, "insert.timeline", json!({})); + } sep(ui); let sp = big_button(ui, Icon::Sparkline, "Sparklines", "Sparklines", true); anchor_keytip(app, ui, &sp, &["dialog:sparkline"]); diff --git a/crates/ui-egui/src/ribbon/pivot_tabs.rs b/crates/ui-egui/src/ribbon/pivot_tabs.rs new file mode 100644 index 000000000..20ced648b --- /dev/null +++ b/crates/ui-egui/src/ribbon/pivot_tabs.rs @@ -0,0 +1,598 @@ +//! The PivotTable contextual tabs, PivotTable Analyze and PivotTable Design, shown while the +//! active cell is in a PivotTable report. Groups, controls and menu items follow Excel 365 in +//! order and control type. A control whose feature GridCraft doesn't have yet keeps its place, +//! disabled, with the reason on hover: wiring the feature in later only touches that control's +//! group function. + +use egui::{Color32, Rect, RichText, Sense, Stroke, StrokeKind, Ui, pos2, vec2}; +use gridcraft_engine::model::{PivotDateGroup, PivotFunc, PivotLayout, PivotShowAs, PivotTable, ReportPart, Workbook}; +use serde_json::{Value as Json, json}; + +use super::{act, menu_items, sep}; +use crate::SheetApp; +use crate::icons::Icon; +use crate::theme::{self, Tokens}; +use crate::widgets::{big_button, small_button}; + +/// Excel's names of the two tabs (German Excel: "PivotTable-Analyse" and "Entwurf"). +pub const ANALYZE: &str = "PivotTable Analyze"; +pub const DESIGN: &str = "PivotTable Design"; + +/// Why a control is disabled (shown on hover). +const NOT_YET: &str = "GridCraft can't do this yet."; +const OLAP: &str = "Only for PivotTables based on OLAP or the Data Model, which GridCraft doesn't have."; +const NO_FIELD: &str = "Select an item or a value of the PivotTable first."; + +/// The PivotTable at the active cell, copied so the tab can run commands while drawing. +fn active(app: &SheetApp) -> Option { + crate::pivot_pane::at_active_cell(app).cloned() +} + +fn pivot_param(pt: &PivotTable) -> Json { + json!({"pivot": pt.name}) +} + +/// A large ribbon button named by its (translated) tooltip, so tests and screen readers find it. +fn big(ui: &mut Ui, icon: Icon, label: &str, tip: &str, dropdown: bool) -> egui::Response { + let r = big_button(ui, icon, label, tip, dropdown); + named(ui, r, tip) +} + +/// A small ribbon button named by its tooltip (see [`big`]). +fn small(ui: &mut Ui, icon: Icon, label: &str, tip: &str, dropdown: bool) -> egui::Response { + let r = small_button(ui, icon, label, tip, dropdown); + named(ui, r, tip) +} + +fn named(ui: &Ui, r: egui::Response, tip: &str) -> egui::Response { + let name = crate::i18n::current(ui.ctx()).tr(tip).to_string(); + let enabled = ui.is_enabled(); + r.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, enabled, &name)); + r +} + +/// A control Excel has here that GridCraft can't run yet: greyed out, with `why` on hover. +fn unavailable(ui: &mut Ui, why: &str, add: impl FnOnce(&mut Ui) -> egui::Response) { + let why = crate::i18n::current(ui.ctx()).tr(why); + ui.add_enabled_ui(false, add).inner.on_disabled_hover_text(why); +} + +/// A menu item GridCraft can't run yet: greyed out, with `why` on hover. +fn unavailable_item(ui: &mut Ui, label: &str, why: &str) { + let lang = crate::i18n::current(ui.ctx()); + ui.add_enabled(false, egui::Button::new(lang.tr(label)).frame(false).min_size(vec2(220.0, 22.0))).on_disabled_hover_text(lang.tr(why)); +} + +/// Menu items of which one describes the current state (ticked, like Excel's highlighted +/// choice): `(label, command, params, current)`. +fn choice_items(app: &mut SheetApp, ui: &mut Ui, items: &[(&str, &str, Json, bool)]) { + let lang = app.ui.language; + for (label, id, params, current) in items { + let text = if *current { format!("✓ {}", lang.tr(label)) } else { format!(" {}", lang.tr(label)) }; + if ui.add(egui::Button::new(text).frame(false).min_size(vec2(260.0, 22.0))).clicked() { + act(app, id, params.clone()); + ui.close(); + } + } +} + +/// Marks a toggle button that is on. +fn toggled(ui: &Ui, r: &egui::Response, on: bool) { + if on { + ui.painter().rect_stroke(r.rect, 4.0, Stroke::new(1.0, Tokens::get(ui.ctx()).accent), StrokeKind::Inside); + } +} + +// ---------------------------------------------------------------- PivotTable Analyze + +/// PivotTable Analyze: PivotTable, Active Field, Group, Filter, Data, Actions, Calculations, +/// Tools and Show. +pub fn analyze(app: &mut SheetApp, ui: &mut Ui) { + let Some(pt) = active(app) else { return }; + pivottable_group(app, ui, &pt); + sep(ui); + active_field_group(app, ui, &pt); + sep(ui); + group_group(app, ui, &pt); + sep(ui); + filter_group(app, ui, &pt); + sep(ui); + data_group(app, ui, &pt); + sep(ui); + actions_group(app, ui, &pt); + sep(ui); + calculations_group(app, ui, &pt); + sep(ui); + tools_group(app, ui, &pt); + sep(ui); + show_group(app, ui, &pt); +} + +/// PivotTable: the PivotTable's name and the Options menu. +fn pivottable_group(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + let lang = app.ui.language; + ui.vertical(|ui| { + ui.label(RichText::new(lang.tr("PivotTable Name:")).small()); + let mut name = pt.name.clone(); + ui.add_enabled(false, egui::TextEdit::singleline(&mut name).desired_width(96.0)) + .on_disabled_hover_text(lang.tr("Renaming a PivotTable isn't available in GridCraft yet.")); + let options = small(ui, Icon::Settings, "Options", "PivotTable Options", true); + egui::Popup::menu(&options).show(|ui| { + unavailable_item(ui, "Options…", NOT_YET); + unavailable_item(ui, "Show Report Filter Pages…", NOT_YET); + // A setting of GridCraft, as in Excel: clicking a value of a PivotTable while + // typing a formula inserts GETPIVOTDATA instead of the cell's reference. + let on = app.session.prefs.generate_get_pivot_data; + let text = if on { format!("✓ {}", lang.tr("Generate GetPivotData")) } else { format!(" {}", lang.tr("Generate GetPivotData")) }; + if ui.add(egui::Button::new(text).frame(false).min_size(vec2(220.0, 22.0))).clicked() { + act(app, "pivot.generateGetPivotData", json!({"on": !on})); + ui.close(); + } + }); + }); +} + +/// What the active cell of a PivotTable stands for, for the Active Field group. +#[derive(Clone, Debug, PartialEq)] +enum ActiveField { + /// A value field: its caption ("Sum of Sales"). + Value(String), + /// A row or column field: its name, and whether it's grouped (by dates or number ranges, or a + /// Group Selection field). + Axis(String, bool), +} + +/// The field of the active cell: a value cell's value field, or the row or column field of an +/// item label (read off the report's index). +fn active_field_of(app: &SheetApp, pt: &PivotTable) -> Option { + let report = pt.report.get()?; + let origin = pt.last_range?.start; + let a = app.session.active()?.selection.active; + let part = report.part_at(a.row.checked_sub(origin.row)?, a.col.checked_sub(origin.col)?)?; + let axis = |fields: &[gridcraft_engine::model::ReportField], defs: &[gridcraft_engine::model::PivotField], f: u32| { + let name = fields.get(f as usize)?.name.clone(); + let grouped = pt.group_field(&name).is_some() + || defs + .iter() + .find(|d| d.source_col.eq_ignore_ascii_case(&name)) + .is_some_and(|d| d.date_group != PivotDateGroup::None || d.number_group.is_some()); + Some(ActiveField::Axis(name, grouped)) + }; + match part { + ReportPart::Data(d) => Some(ActiveField::Value(report.data_fields.get(d as usize)?.name.clone())), + ReportPart::RowField(f) => axis(&report.row_fields, &pt.rows, f), + ReportPart::ColField(f) => axis(&report.col_fields, &pt.columns, f), + } +} + +/// `pivot.valueSettings`'s name for a summary function. +fn func_key(f: PivotFunc) -> &'static str { + match f { + PivotFunc::Sum => "sum", + PivotFunc::Count => "count", + PivotFunc::Average => "average", + PivotFunc::Max => "max", + PivotFunc::Min => "min", + PivotFunc::Product => "product", + PivotFunc::CountNumbers => "countNumbers", + PivotFunc::StdDev => "stdDev", + PivotFunc::StdDevP => "stdDevP", + PivotFunc::Var => "var", + PivotFunc::VarP => "varP", + } +} + +/// `pivot.valueSettings`'s name for a Show Values As choice. +fn show_as_key(v: PivotShowAs) -> &'static str { + match v { + PivotShowAs::Normal => "normal", + PivotShowAs::PercentOfGrandTotal => "percentOfGrandTotal", + PivotShowAs::PercentOfColumnTotal => "percentOfColumnTotal", + PivotShowAs::PercentOfRowTotal => "percentOfRowTotal", + PivotShowAs::RunningTotal => "runningTotal", + PivotShowAs::Rank => "rankDescending", + PivotShowAs::PercentOf => "percentOf", + PivotShowAs::PercentOfParentRowTotal => "percentOfParentRowTotal", + PivotShowAs::PercentOfParentColumnTotal => "percentOfParentColumnTotal", + PivotShowAs::PercentOfParentTotal => "percentOfParentTotal", + PivotShowAs::DifferenceFrom => "differenceFrom", + PivotShowAs::PercentDifferenceFrom => "percentDifferenceFrom", + PivotShowAs::PercentRunningTotal => "percentRunningTotal", + PivotShowAs::RankAscending => "rankAscending", + PivotShowAs::Index => "index", + } +} + +/// Active Field: the field's name and settings, drilling (OLAP only), expanding and collapsing. +fn active_field_group(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + let lang = app.ui.language; + let field = active_field_of(app, pt); + ui.vertical(|ui| { + ui.label(RichText::new(lang.tr("Active Field:")).small()); + let mut text = match &field { + Some(ActiveField::Value(n) | ActiveField::Axis(n, _)) => n.clone(), + None => String::new(), + }; + ui.add_enabled(false, egui::TextEdit::singleline(&mut text).desired_width(96.0)); + match &field { + Some(ActiveField::Value(caption)) => { + if small(ui, Icon::Settings, "Field Settings", "Field Settings", false).clicked() { + let v = pt.values.iter().find(|v| v.name == *caption); + let mut params = json!({"pivot": pt.name, "field": caption, "name": caption}); + if let (Some(v), Some(m)) = (v, params.as_object_mut()) { + m.insert("func".into(), json!(func_key(v.func))); + m.insert("showAs".into(), json!(show_as_key(v.show_as))); + m.insert("baseField".into(), json!(v.base_field.clone().unwrap_or_default())); + m.insert("baseItem".into(), json!(v.base_item.clone().unwrap_or_default())); + } + app.open_dialog("pivotValueFieldSettings", params); + } + } + Some(ActiveField::Axis(..)) => unavailable(ui, NOT_YET, |ui| small(ui, Icon::Settings, "Field Settings", "Field Settings", false)), + None => unavailable(ui, NO_FIELD, |ui| small(ui, Icon::Settings, "Field Settings", "Field Settings", false)), + } + }); + unavailable(ui, OLAP, |ui| big(ui, Icon::Zoom, "Drill\nDown", "Drill Down", false)); + unavailable(ui, OLAP, |ui| big(ui, Icon::Zoom, "Drill\nUp", "Drill Up", false)); + ui.vertical(|ui| { + let axis = match &field { + Some(ActiveField::Axis(name, _)) => Some(name.clone()), + _ => None, + }; + for (icon, label, collapse) in [(Icon::Plus, "Expand Field", false), (Icon::Minus, "Collapse Field", true)] { + match &axis { + Some(name) => { + if small(ui, icon, label, label, false).clicked() { + act(app, "pivot.collapse", json!({"pivot": pt.name, "field": name, "collapse": collapse})); + } + } + None => unavailable(ui, NO_FIELD, |ui| small(ui, icon, label, label, false)), + } + } + }); +} + +/// Group: Group Selection, Ungroup and Group Field. +fn group_group(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + let field = active_field_of(app, pt); + ui.vertical(|ui| { + // Group Selection takes the selected items of a row or column field (one item of a + // number or date field opens the Grouping dialog, as in Excel). + match &field { + Some(ActiveField::Axis(..)) => { + if small(ui, Icon::Group, "Group Selection", "Group Selection", false).clicked() { + act(app, "pivot.groupSelection", pivot_param(pt)); + } + } + _ => unavailable(ui, NO_FIELD, |ui| small(ui, Icon::Group, "Group Selection", "Group Selection", false)), + } + // Ungroup: the group at the active cell, or the field's date or number grouping. + match &field { + Some(ActiveField::Axis(_, true)) => { + if small(ui, Icon::Ungroup, "Ungroup", "Ungroup", false).clicked() { + act(app, "pivot.ungroup", pivot_param(pt)); + } + } + _ => unavailable(ui, "Select an item of a grouped field first.", |ui| small(ui, Icon::Ungroup, "Ungroup", "Ungroup", false)), + } + // Group Field: the Grouping dialog for the field at the active cell. + match &field { + Some(ActiveField::Axis(..)) => { + if small(ui, Icon::Group, "Group Field", "Group Field", false).clicked() { + act(app, "pivot.group", pivot_param(pt)); + } + } + _ => unavailable(ui, NO_FIELD, |ui| small(ui, Icon::Group, "Group Field", "Group Field", false)), + } + }); +} + +/// Filter: slicers, timelines and their connections. +fn filter_group(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + ui.vertical(|ui| { + unavailable(ui, NOT_YET, |ui| small(ui, Icon::Filter, "Insert Slicer", "Insert Slicer", false)); + if small(ui, Icon::Calendar, "Insert Timeline", "Insert Timeline", false).clicked() { + act(app, "pivot.insertTimeline", pivot_param(pt)); + } + unavailable(ui, NOT_YET, |ui| small(ui, Icon::Link, "Filter Connections", "Filter Connections", false)); + }); +} + +/// Data: Refresh and Change Data Source. +fn data_group(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + let refresh = big(ui, Icon::Redo, "Refresh", "Refresh (Alt+F5)", true); + egui::Popup::menu(&refresh).show(|ui| { + menu_items(app, ui, &[("Refresh", "pivot.refresh", pivot_param(pt)), ("Refresh All", "data.refreshAll", json!({}))]); + unavailable_item(ui, "Refresh Status", NOT_YET); + unavailable_item(ui, "Cancel Refresh", NOT_YET); + unavailable_item(ui, "Connection Properties…", NOT_YET); + }); + let source = big(ui, Icon::Table, "Change Data\nSource", "Change Data Source", true); + egui::Popup::menu(&source).show(|ui| { + let lang = app.ui.language; + if ui.add(egui::Button::new(lang.tr("Change Data Source…")).frame(false).min_size(vec2(220.0, 22.0))).clicked() { + app.open_dialog("pivotChangeSource", json!({"pivot": pt.name, "source": pt.source})); + ui.close(); + } + unavailable_item(ui, "Connection Properties…", NOT_YET); + }); +} + +/// Actions: Clear, Select and Move PivotTable. +fn actions_group(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + ui.vertical(|ui| { + let clear = small(ui, Icon::Eraser, "Clear", "Clear", true); + egui::Popup::menu(&clear).show(|ui| { + menu_items( + app, + ui, + &[("PivotTable|Clear All", "pivot.clearAll", pivot_param(pt)), ("Clear Filters", "pivot.clearFilters", pivot_param(pt))], + ); + }); + let select = small(ui, Icon::Checkbox, "Select", "Select", true); + egui::Popup::menu(&select).show(|ui| { + // The body (without report filters) and the whole report. + let whole = pt.last_range; + let body = whole.map(|r| gridcraft_engine::core::RangeRef::new(pt.anchor, r.end)); + let pick = |app: &mut SheetApp, ui: &mut Ui, label: &str, range: Option| { + let lang = app.ui.language; + let resp = ui.add_enabled(range.is_some(), egui::Button::new(lang.tr(label)).frame(false).min_size(vec2(220.0, 22.0))); + if resp.clicked() + && let Some(r) = range + { + act(app, "selection.set", json!({"range": r.a1()})); + ui.close(); + } + }; + pick(app, ui, "Labels and Values", body); + // The value cells, totals included. + let values = pt.report.get().and_then(|r| r.data_area()).zip(whole).and_then(|((r0, c0, r1, c1), w)| { + let at = |r: u32, c: u32| Some(gridcraft_engine::core::CellRef::new(w.start.row.checked_add(r)?, w.start.col.checked_add(c)?)); + Some(gridcraft_engine::core::RangeRef::new(at(r0, c0)?, at(r1, c1)?)) + }); + pick(app, ui, "Values", values); + unavailable_item(ui, "Labels", NOT_YET); + ui.separator(); + pick(app, ui, "Entire PivotTable", whole); + ui.separator(); + unavailable_item(ui, "Enable Selection", NOT_YET); + }); + unavailable(ui, NOT_YET, |ui| small(ui, Icon::Right, "Move PivotTable", "Move PivotTable", false)); + }); +} + +/// Calculations: Fields, Items, & Sets; OLAP Tools; Relationships. +fn calculations_group(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + let fis = big(ui, Icon::Function, "Fields, Items,\n& Sets", "Fields, Items, & Sets", true); + egui::Popup::menu(&fis).show(|ui| { + menu_items( + app, + ui, + &[ + ("Calculated Field…", "pivot.calculatedField", pivot_param(pt)), + ("Calculated Item…", "pivot.calculatedItem", pivot_param(pt)), + ("Solve Order…", "pivot.solveOrder", pivot_param(pt)), + ("List Formulas", "pivot.listFormulas", pivot_param(pt)), + ], + ); + ui.separator(); + unavailable_item(ui, "Create Set Based on Row Items…", OLAP); + unavailable_item(ui, "Create Set Based on Column Items…", OLAP); + ui.separator(); + unavailable_item(ui, "Manage Sets…", OLAP); + }); + // Excel stacks these two once the window is narrower than the whole tab. + ui.vertical(|ui| { + unavailable(ui, OLAP, |ui| small(ui, Icon::Calc, "OLAP Tools", "OLAP Tools", true)); + unavailable(ui, OLAP, |ui| small(ui, Icon::Link, "Relationships", "Relationships", false)); + }); +} + +/// Tools: PivotChart and Recommended PivotTables. +fn tools_group(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + if big(ui, Icon::ChartBar, "PivotChart", "PivotChart", false).clicked() { + act(app, "insert.pivotChart", pivot_param(pt)); + } + // Excel opens a dialog of suggestions for the PivotTable's source; here they are a menu. + let rec = big(ui, Icon::PivotTable, "Recommended\nPivotTables", "Recommended PivotTables", true); + let key = rec.id.with("suggestions"); + if rec.clicked() { + let list = app.session.run("insert.recommendedPivotTables", json!({"source": pt.source})).ok().and_then(|r| r.get("suggestions").cloned()); + ui.data_mut(|d| d.insert_temp(key, list.unwrap_or_default())); + } + egui::Popup::menu(&rec).show(|ui| { + let list: Json = ui.data(|d| d.get_temp(key)).unwrap_or_default(); + let items = list.as_array().cloned().unwrap_or_default(); + if items.is_empty() { + unavailable_item(ui, "No recommendations for this data", NOT_YET); + } + for (i, s) in items.iter().enumerate() { + let title = s.get("title").and_then(Json::as_str).unwrap_or_default(); + if ui.add(egui::Button::new(title).frame(false).min_size(vec2(240.0, 22.0))).clicked() { + act(app, "insert.recommendedPivotTables", json!({"source": pt.source, "apply": i})); + ui.close(); + } + } + }); +} + +/// Show: Field List, +/- Buttons and Field Headers, toggles stacked as in Excel's narrower +/// ribbon. +fn show_group(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + ui.vertical(|ui| { + let on = app.ui.pivot_field_list; + let list = small(ui, Icon::SheetList, "Field List", "Field List", false); + toggled(ui, &list, on); + if list.clicked() { + act(app, "pivot.showFieldList", json!({"on": !on})); + } + unavailable(ui, NOT_YET, |ui| small(ui, Icon::Plus, "+/- Buttons", "+/- Buttons", false)); + let headers = small(ui, Icon::Header, "Field Headers", "Field Headers", false); + toggled(ui, &headers, pt.show_headers); + if headers.clicked() { + act(app, "pivot.layout", json!({"pivot": pt.name, "showHeaders": !pt.show_headers})); + } + }); +} + +// ---------------------------------------------------------------- PivotTable Design + +/// PivotTable Design: Layout, PivotTable Style Options and PivotTable Styles. +pub fn design(app: &mut SheetApp, ui: &mut Ui) { + let Some(pt) = active(app) else { return }; + layout_group(app, ui, &pt); + sep(ui); + style_options_group(app, ui); + sep(ui); + styles_group(app, ui, &pt); +} + +/// Layout: Subtotals, Grand Totals, Report Layout and Blank Rows. +fn layout_group(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + let p = |extra: Json| { + let mut v = pivot_param(pt); + if let (Some(m), Some(e)) = (v.as_object_mut(), extra.as_object()) { + m.extend(e.clone()); + } + v + }; + let tabular = pt.layout == PivotLayout::Tabular; + let subtotals = big(ui, Icon::Subtotal, "Subtotals", "Subtotals", true); + egui::Popup::menu(&subtotals).show(|ui| { + choice_items( + app, + ui, + &[ + ("Do Not Show Subtotals", "pivot.subtotals", p(json!({"mode": "none"})), !pt.subtotals), + ("Show all Subtotals at Bottom of Group", "pivot.subtotals", p(json!({"mode": "bottom"})), pt.subtotals && !pt.subtotals_top), + ], + ); + // Tabular form has no room for a subtotal above its group. + if tabular { + unavailable_item(ui, "Show all Subtotals at Top of Group", "Not available in Tabular Form."); + } else { + choice_items( + app, + ui, + &[("Show all Subtotals at Top of Group", "pivot.subtotals", p(json!({"mode": "top"})), pt.subtotals && pt.subtotals_top)], + ); + } + ui.separator(); + unavailable_item(ui, "Include Filtered Items in Totals", NOT_YET); + }); + let grand = big(ui, Icon::Sum, "Grand\nTotals", "Grand Totals", true); + egui::Popup::menu(&grand).show(|ui| { + let (r, c) = (pt.grand_totals_rows, pt.grand_totals_cols); + choice_items( + app, + ui, + &[ + ("Off for Rows and Columns", "pivot.grandTotals", p(json!({"mode": "off"})), !r && !c), + ("On for Rows and Columns", "pivot.grandTotals", p(json!({"mode": "on"})), r && c), + ("On for Rows Only", "pivot.grandTotals", p(json!({"mode": "rows"})), r && !c), + ("On for Columns Only", "pivot.grandTotals", p(json!({"mode": "columns"})), !r && c), + ], + ); + }); + let report = big(ui, Icon::Table, "Report\nLayout", "Report Layout", true); + egui::Popup::menu(&report).show(|ui| { + choice_items( + app, + ui, + &[ + ("Show in Compact Form", "pivot.reportLayout", p(json!({"layout": "compact"})), pt.layout == PivotLayout::Compact), + ("Show in Outline Form", "pivot.reportLayout", p(json!({"layout": "outline"})), pt.layout == PivotLayout::Outline), + ("Show in Tabular Form", "pivot.reportLayout", p(json!({"layout": "tabular"})), tabular), + ], + ); + ui.separator(); + unavailable_item(ui, "Repeat All Item Labels", NOT_YET); + unavailable_item(ui, "Do Not Repeat Item Labels", NOT_YET); + }); + let blank = big(ui, Icon::Insert, "Blank\nRows", "Blank Rows", true); + egui::Popup::menu(&blank).show(|ui| { + unavailable_item(ui, "Insert Blank Line after Each Item", NOT_YET); + unavailable_item(ui, "Remove Blank Line after Each Item", NOT_YET); + }); +} + +/// PivotTable Style Options: shown with Excel's defaults; GridCraft's styles don't vary by them. +fn style_options_group(app: &mut SheetApp, ui: &mut Ui) { + let lang = app.ui.language; + for (i, column) in + [[("Row Headers", true), ("Column Headers", true)], [("Banded Rows", false), ("Banded Columns", false)]].into_iter().enumerate() + { + if i > 0 { + ui.add_space(8.0); + } + ui.vertical(|ui| { + for (label, on) in column { + let mut on = on; + ui.add_enabled(false, egui::Checkbox::new(&mut on, lang.tr(label))).on_disabled_hover_text(lang.tr(NOT_YET)); + } + }); + } +} + +/// PivotTable Styles: the gallery of `PivotStyleLight1…28`, `Medium1…28` and `Dark1…28`. +fn styles_group(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + let st = big(ui, Icon::FormatTable, "PivotTable\nStyles", "PivotTable Styles", true); + egui::Popup::menu(&st).show(|ui| style_gallery(app, ui, pt)); +} + +fn style_gallery(app: &mut SheetApp, ui: &mut Ui, pt: &PivotTable) { + ui.set_width(430.0); + let Some(wb) = app.session.active().map(|d| d.wb.clone()) else { return }; + let lang = app.ui.language; + egui::ScrollArea::vertical().max_height(420.0).show(ui, |ui| { + for fam in ["Light", "Medium", "Dark"] { + ui.label(RichText::new(lang.tr(fam)).strong()); + egui::Grid::new(("pivot_styles", fam)).spacing(vec2(4.0, 4.0)).show(ui, |ui| { + for i in 1..=28u32 { + let name = format!("PivotStyle{fam}{i}"); + let (r, resp) = ui.allocate_exact_size(vec2(54.0, 38.0), Sense::click()); + resp.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, &name)); + paint_style_swatch(ui, r, &wb, &name); + if name.eq_ignore_ascii_case(&pt.style) || resp.hovered() { + ui.painter().rect_stroke(r.expand(1.5), 2.0, Stroke::new(2.0, Color32::from_rgb(0xE8, 0x7D, 0x2B)), StrokeKind::Outside); + } + if resp.on_hover_text(&name).clicked() { + act(app, "pivot.style", json!({"pivot": pt.name, "style": name})); + ui.close(); + } + if i % 7 == 0 { + ui.end_row(); + } + } + }); + } + }); +} + +/// A small picture of a report in a PivotTable style: header band, item rows, total row. +fn paint_style_swatch(ui: &Ui, r: Rect, wb: &Workbook, style: &str) { + let lk = gridcraft_engine::pivot::look(style); + let res = |c: gridcraft_engine::model::Color| c.resolve(&wb.theme).map(theme::color32); + let p = ui.painter(); + p.rect_filled(r, 0.0, Color32::WHITE); + let rows = 5; + let rh = r.height() / rows as f32; + for i in 0..rows { + let rr = Rect::from_min_size(pos2(r.left(), r.top() + i as f32 * rh), vec2(r.width(), rh)); + if i == 0 + && let Some(f) = res(lk.header_fill) + { + p.rect_filled(rr, 0.0, f); + } + let line = res(lk.line).unwrap_or(Color32::from_gray(160)); + if i == 0 || i == rows - 2 { + p.line_segment([rr.left_bottom(), rr.right_bottom()], Stroke::new(1.0, line)); + } + for k in 0..2 { + let x = rr.left() + 4.0 + k as f32 * (r.width() - 8.0) / 2.0; + let col = if i == 0 { res(lk.header_font).unwrap_or(Color32::BLACK) } else { Color32::from_gray(120) }; + p.line_segment([pos2(x, rr.center().y), pos2(x + 14.0, rr.center().y)], Stroke::new(1.0, col)); + } + } + p.rect_stroke(r, 0.0, Stroke::new(1.0, Color32::from_gray(210)), StrokeKind::Inside); +} diff --git a/crates/ui-egui/src/timeline_view.rs b/crates/ui-egui/src/timeline_view.rs new file mode 100644 index 000000000..d3b2356a1 --- /dev/null +++ b/crates/ui-egui/src/timeline_view.rs @@ -0,0 +1,380 @@ +//! Timelines on the sheet: painting, clicks (a period, Shift for a range, the clear button, the +//! time level menu, the scroll bar) and the Timeline contextual tab. The timeline itself and +//! its filtering live in the engine (`timeline.*` commands); this module draws it and acts +//! through those commands. + +use egui::{Align2, Color32, Painter, Pos2, Rect, Stroke, StrokeKind, Ui, pos2, vec2}; +use gridcraft_engine::cmd::timeline::{self as tlc, Period}; +use gridcraft_engine::model::{Sheet, Timeline, TimelineLevel, Workbook}; +use serde_json::json; + +use crate::SheetApp; +use crate::chartview::anchor_rect; +use crate::grid::Geo; +use crate::theme; + +/// The parts of a timeline on screen. +pub struct Layout { + pub header: Option, + pub clear: Option, + pub label: Option, + pub level: Option, + /// The year (or month) captions above the tiles. + pub groups: Vec<(Rect, String)>, + /// One tile per visible period: (rect, period index). + pub tiles: Vec<(Rect, usize)>, + pub scrollbar: Option, + pub thumb: Option, +} + +/// Width of one period tile, in points, at each level. +fn tile_width(level: TimelineLevel) -> f32 { + match level { + TimelineLevel::Years => 56.0, + TimelineLevel::Quarters => 40.0, + TimelineLevel::Months => 34.0, + TimelineLevel::Days => 24.0, + } +} + +/// The caption of a time level in the level menu. +pub fn level_caption(l: TimelineLevel) -> &'static str { + match l { + TimelineLevel::Years => "YEARS", + TimelineLevel::Quarters => "QUARTERS", + TimelineLevel::Months => "MONTHS", + TimelineLevel::Days => "DAYS", + } +} + +pub fn layout(rect: Rect, z: f32, tl: &Timeline, periods: &[Period]) -> Layout { + let pad = 6.0 * z; + let mut y = rect.top(); + let (mut header, mut clear, mut label, mut level) = (None, None, None, None); + if tl.show_header { + let h = Rect::from_min_size(rect.min, vec2(rect.width(), 26.0 * z)); + let cb = 16.0 * z; + clear = Some(Rect::from_min_size(pos2(h.right() - pad - cb, h.center().y - cb / 2.0), vec2(cb, cb))); + header = Some(h); + y = h.bottom(); + } + if tl.show_selection_label || tl.show_time_level { + let row = Rect::from_min_size(pos2(rect.left() + pad, y), vec2(rect.width() - 2.0 * pad, 20.0 * z)); + if tl.show_selection_label { + label = Some(Rect::from_min_max(row.min, pos2(row.right() - 90.0 * z, row.bottom()))); + } + if tl.show_time_level { + level = Some(Rect::from_min_max(pos2(row.right() - 86.0 * z, row.top()), row.max)); + } + y = row.bottom(); + } + let bar_h = if tl.show_scrollbar { 10.0 * z } else { 0.0 }; + let strip = Rect::from_min_max(pos2(rect.left() + pad, y + 2.0 * z), pos2(rect.right() - pad, rect.bottom() - pad - bar_h)); + let tw = tile_width(tl.level) * z; + let first = tl.scroll.and_then(|d| periods.iter().position(|p| p.end >= d)).unwrap_or(0); + let fit = ((strip.width() / tw).floor() as usize).max(1); + let first = first.min(periods.len().saturating_sub(fit)); + let group_h = 16.0 * z; + let mut tiles = Vec::new(); + let mut groups: Vec<(Rect, String)> = Vec::new(); + for (k, (i, p)) in periods.iter().enumerate().skip(first).take(fit).enumerate() { + let x = strip.left() + k as f32 * tw; + let tile = Rect::from_min_size(pos2(x, strip.top() + group_h), vec2(tw - 2.0 * z, (strip.height() - group_h).max(8.0 * z))); + tiles.push((tile, i)); + let start_group = groups.last().is_none_or(|(_, g)| *g != p.group); + if !p.group.is_empty() && start_group { + groups.push((Rect::from_min_size(pos2(x, strip.top()), vec2(tw, group_h)), p.group.clone())); + } else if let Some((r, _)) = groups.last_mut().filter(|(_, g)| *g == p.group) { + r.max.x = x + tw; + } + } + let (mut scrollbar, mut thumb) = (None, None); + if tl.show_scrollbar && !periods.is_empty() { + let bar = Rect::from_min_size(pos2(strip.left(), rect.bottom() - pad - bar_h + 2.0 * z), vec2(strip.width(), bar_h - 4.0 * z)); + let n = periods.len() as f32; + let a = bar.left() + bar.width() * first as f32 / n; + let b = bar.left() + bar.width() * ((first + fit).min(periods.len()) as f32 / n); + thumb = Some(Rect::from_min_max(pos2(a, bar.top()), pos2(b.max(a + 6.0 * z), bar.bottom()))); + scrollbar = Some(bar); + } + Layout { header, clear, label, level, groups, tiles, scrollbar, thumb } +} + +/// The periods of a timeline, computed once per workbook state. +pub fn periods(ctx: &egui::Context, wb: &std::sync::Arc, tl: &Timeline) -> Vec { + let key = egui::Id::new(("timeline_periods", tl.id, std::sync::Arc::as_ptr(wb) as usize, tl.level as u8)); + if let Some(v) = ctx.data(|d| d.get_temp::>>(key)) { + return (*v).clone(); + } + let v = tlc::periods(wb, tl); + ctx.data_mut(|d| d.insert_temp(key, std::sync::Arc::new(v.clone()))); + v +} + +/// Colours of a timeline style: (selected tile, unselected tile, card, text). +fn style_colors(style: &str, t: &theme::Tokens) -> (Color32, Color32, Color32, Color32) { + let n: usize = style.trim_start_matches(|c: char| !c.is_ascii_digit()).parse().unwrap_or(1); + // Our own palette, one hue per style number. + let hue = [ + Color32::from_rgb(0x3A, 0x7B, 0xD5), + Color32::from_rgb(0xD9, 0x7B, 0x2B), + Color32::from_rgb(0x7A, 0x7F, 0x87), + Color32::from_rgb(0xC9, 0xA2, 0x27), + Color32::from_rgb(0x4B, 0x9C, 0xC9), + Color32::from_rgb(0x5A, 0x9E, 0x52), + ]; + let accent = hue.get(n.saturating_sub(1) % hue.len()).copied().unwrap_or(t.accent); + if style.contains("Dark") { + (accent, Color32::from_gray(0x5A), Color32::from_gray(0x33), Color32::from_gray(0xF0)) + } else { + (accent, Color32::from_gray(0xE4), t.window, t.text) + } +} + +pub fn paint(app: &SheetApp, p: &Painter, geo: &Geo, wb: &std::sync::Arc, sh: &Sheet) { + let tk = theme::Tokens::get(p.ctx()); + for tl in &sh.timelines { + let r = anchor_rect(geo, sh, &tl.anchor); + if !r.intersects(p.clip_rect()) { + continue; + } + let list = periods(p.ctx(), wb, tl); + let lay = layout(r, geo.z, tl, &list); + let (on, off, card, text) = style_colors(&tl.style, &tk); + let radius = 4.0 * geo.z; + p.rect_filled(r.translate(vec2(0.0, 1.5)), radius, Color32::from_black_alpha(18)); + p.rect_filled(r, radius, card); + p.rect_stroke(r, radius, Stroke::new(1.0, tk.separator), StrokeKind::Inside); + let clip = p.with_clip_rect(r.intersect(p.clip_rect())); + if let Some(h) = lay.header { + let cap = if tl.caption.is_empty() { tl.field.as_str() } else { tl.caption.as_str() }; + clip.text(h.left_center() + vec2(8.0 * geo.z, 0.0), Align2::LEFT_CENTER, cap, theme::ui_bold(12.5 * geo.z), text); + } + if let (Some(c), Some(icon)) = (lay.clear, crate::icons::from_name("filter")) { + crate::icons::paint(&clip, c, icon, if tl.selection.is_some() { on } else { tk.text_dim }); + } + if let Some(l) = lay.label { + let label = tlc::selection_label(wb, tl); + // "All Periods" is a fixed caption; a chosen period (`Mar - Aug 2026`) stays as the engine writes it. + let label = app.ui.language.tr(&label); + clip.text(l.left_center(), Align2::LEFT_CENTER, label, theme::ui_font(11.5 * geo.z), text); + } + if let Some(l) = lay.level { + let level = app.ui.language.tr(level_caption(tl.level)); + clip.text(l.right_center(), Align2::RIGHT_CENTER, format!("{level} ▾"), theme::ui_font(10.5 * geo.z), on); + } + for (g, caption) in &lay.groups { + clip.text(g.left_center() + vec2(2.0 * geo.z, 0.0), Align2::LEFT_CENTER, caption, theme::ui_font(10.5 * geo.z), text); + } + for (tile, i) in &lay.tiles { + let Some(period) = list.get(*i) else { continue }; + let selected = tl.selection.is_some_and(|(a, b)| period.start <= b && period.end >= a); + let fill = if selected || tl.selection.is_none() { on } else { off }; + let bar = Rect::from_min_max(pos2(tile.left(), tile.bottom() - 10.0 * geo.z), tile.max); + clip.rect_filled(bar, 1.5 * geo.z, fill); + clip.text(pos2(tile.center().x, bar.top() - 2.0 * geo.z), Align2::CENTER_BOTTOM, &period.label, theme::ui_font(10.0 * geo.z), text); + } + if let (Some(bar), Some(th)) = (lay.scrollbar, lay.thumb) { + clip.rect_filled(bar, 2.0 * geo.z, tk.hover); + clip.rect_filled(th, 2.0 * geo.z, tk.text_dim); + } + if app.selected_chart == Some(tl.id) { + p.rect_stroke(r, 0.0, Stroke::new(1.0, Color32::from_gray(120)), StrokeKind::Outside); + } + } +} + +/// What a click inside a timeline hit. +pub enum Action { + /// A period: its first and last day. + Period(f64, f64), + Clear, + /// The time level caption (opens the level menu). + Level(Pos2), + /// The scroll bar: scroll to this date. + Scroll(f64), +} + +/// The timeline under `p` and what the click hit there (`None`: the timeline's background). +pub fn click(geo: &Geo, sh: &Sheet, wb: &std::sync::Arc, ctx: &egui::Context, p: Pos2) -> Option<(u32, Option)> { + if !geo.cells.contains(p) { + return None; + } + for tl in sh.timelines.iter().rev() { + let r = anchor_rect(geo, sh, &tl.anchor); + if !r.contains(p) { + continue; + } + let list = periods(ctx, wb, tl); + let lay = layout(r, geo.z, tl, &list); + if lay.clear.is_some_and(|c| c.contains(p)) { + return Some((tl.id, Some(Action::Clear))); + } + if let Some(l) = lay.level.filter(|l| l.contains(p)) { + return Some((tl.id, Some(Action::Level(l.left_bottom())))); + } + for (tile, i) in &lay.tiles { + if tile.expand2(vec2(1.0, 0.0)).contains(p) + && let Some(x) = list.get(*i) + { + return Some((tl.id, Some(Action::Period(x.start, x.end)))); + } + } + if let Some(bar) = lay.scrollbar.filter(|b| b.expand(3.0).contains(p)) { + let f = ((p.x - bar.left()) / bar.width().max(1.0)).clamp(0.0, 1.0); + let i = ((list.len() as f32) * f) as usize; + if let Some(x) = list.get(i.min(list.len().saturating_sub(1))) { + return Some((tl.id, Some(Action::Scroll(x.start)))); + } + } + return Some((tl.id, None)); + } + None +} + +/// Runs a click on a timeline: a period (with Shift: from the selection's start to it), the +/// clear button, the level menu or the scroll bar. +pub fn on_click(app: &mut SheetApp, id: u32, action: Action, shift: bool) { + match action { + Action::Period(a, b) => { + let start = if shift { + app.session + .active() + .and_then(|d| d.wb.sheets.iter().flat_map(|s| s.timelines.iter()).find(|t| t.id == id).and_then(|t| t.selection)) + .map(|s| s.0) + } else { + None + }; + let (lo, hi) = match start { + Some(s) if s < a => (s, b), + Some(s) => (a, s.max(b)), + None => (a, b), + }; + app.run_or_alert("timeline.select", json!({"timeline": id, "start": lo, "end": hi})); + } + Action::Clear => app.run_or_alert("timeline.clear", json!({"timeline": id})), + Action::Level(at) => app.grid.timeline_menu = Some((id, at)), + Action::Scroll(to) => app.run_or_alert("timeline.scroll", json!({"timeline": id, "to": to})), + } +} + +/// The time level menu of a timeline, opened from its level caption. +pub fn level_menu(app: &mut SheetApp, ui: &mut Ui) { + let Some((id, at)) = app.grid.timeline_menu else { return }; + let lang = crate::i18n::current(ui.ctx()); + let mut close = false; + let resp = egui::Area::new(egui::Id::new("timeline_level")).fixed_pos(at).order(egui::Order::Foreground).show(ui.ctx(), |ui| { + egui::Frame::popup(ui.style()).show(ui, |ui| { + for (level, name) in [ + (TimelineLevel::Years, "years"), + (TimelineLevel::Quarters, "quarters"), + (TimelineLevel::Months, "months"), + (TimelineLevel::Days, "days"), + ] { + if ui.button(lang.tr(level_caption(level))).clicked() { + app.run_or_alert("timeline.level", json!({"timeline": id, "level": name})); + close = true; + } + } + }); + }); + if close || resp.response.clicked_elsewhere() { + app.grid.timeline_menu = None; + } +} + +/// The selected timeline on the active sheet, if any. +pub fn selected(app: &SheetApp) -> Option { + let id = app.selected_chart?; + app.session.active()?.wb.active()?.timelines.iter().find(|t| t.id == id).cloned() +} + +/// The Timeline contextual tab: Timeline (caption, Report Connections), Timeline Styles, Arrange, +/// Size and Show, in Excel's order. +pub fn ribbon_tab(app: &mut SheetApp, ui: &mut Ui) { + use crate::icons::Icon; + use crate::widgets::small_button; + let Some(tl) = selected(app) else { return }; + let id = tl.id; + let lang = app.ui.language; + ui.vertical(|ui| { + ui.label(egui::RichText::new(lang.tr("Timeline Caption:")).small()); + let field = egui::Id::new(("gridcraft.timeline_caption", id)); + if let Some(caption) = crate::widgets::committed_text(ui, field, &tl.caption, Some(130.0)) + && caption != tl.caption + { + app.run_or_alert("timeline.caption", json!({"timeline": id, "caption": caption})); + } + if small_button(ui, Icon::PivotTable, "Report Connections", "Report Connections", false).clicked() { + app.run_or_alert("timeline.reportConnections", json!({"timeline": id})); + } + }); + crate::ribbon::sep(ui); + // Timeline Styles: the gallery of Light and Dark styles. + egui::Grid::new("timeline_styles").spacing(vec2(3.0, 3.0)).show(ui, |ui| { + for (i, name) in tlc::STYLES.iter().enumerate() { + let tk = theme::Tokens::get(ui.ctx()); + let (on, off, card, _) = style_colors(name, &tk); + let (r, resp) = ui.allocate_exact_size(vec2(30.0, 22.0), egui::Sense::click()); + ui.painter().rect_filled(r, 2.0, card); + ui.painter().rect_filled(Rect::from_min_size(r.min + vec2(3.0, 12.0), vec2(12.0, 6.0)), 1.0, on); + ui.painter().rect_filled(Rect::from_min_size(r.min + vec2(16.0, 12.0), vec2(11.0, 6.0)), 1.0, off); + let border = if tl.style == *name { tk.accent } else { tk.separator }; + ui.painter().rect_stroke(r, 2.0, Stroke::new(1.0, border), StrokeKind::Inside); + if resp.on_hover_text(*name).clicked() { + app.run_or_alert("timeline.style", json!({"timeline": id, "style": name})); + } + if i % 6 == 5 { + ui.end_row(); + } + } + }); + crate::ribbon::sep(ui); + // Arrange: Excel's object arrangement, not available for timelines yet. + ui.add_enabled_ui(false, |ui| { + ui.vertical(|ui| { + for label in ["Bring Forward", "Send Backward", "Selection Pane"] { + small_button(ui, Icon::Shapes, label, "Not available for timelines yet", false); + } + }); + ui.vertical(|ui| { + for label in ["Align", "Group", "Rotate"] { + small_button(ui, Icon::Shapes, label, "Not available for timelines yet", true); + } + }); + }); + crate::ribbon::sep(ui); + ui.vertical(|ui| { + for (label, key, v) in [("Height:", "height", tl.anchor.height), ("Width:", "width", tl.anchor.width)] { + ui.horizontal(|ui| { + ui.label(lang.tr(label)); + let mut cm = f64::from(v) / 72.0 * 2.54; + let resp = ui.add(egui::DragValue::new(&mut cm).speed(0.05).range(0.5..=100.0).suffix(" cm").max_decimals(2)); + if resp.drag_stopped() || (resp.changed() && !resp.dragged()) { + app.run_or_alert("timeline.size", json!({"timeline": id, key: cm / 2.54 * 72.0})); + } + }); + } + }); + crate::ribbon::sep(ui); + ui.vertical(|ui| { + for (label, cmd, v) in + [("Header", "timeline.showHeader", tl.show_header), ("Selection Label", "timeline.showSelectionLabel", tl.show_selection_label)] + { + let mut on = v; + if ui.checkbox(&mut on, lang.tr(label)).changed() { + app.run_or_alert(cmd, json!({"timeline": id, "on": on})); + } + } + }); + ui.vertical(|ui| { + for (label, cmd, v) in + [("Scrollbar", "timeline.showScrollbar", tl.show_scrollbar), ("Time Level", "timeline.showTimeLevel", tl.show_time_level)] + { + let mut on = v; + if ui.checkbox(&mut on, lang.tr(label)).changed() { + app.run_or_alert(cmd, json!({"timeline": id, "on": on})); + } + } + }); +} diff --git a/crates/ui-egui/tests/getpivotdata_point.rs b/crates/ui-egui/tests/getpivotdata_point.rs new file mode 100644 index 000000000..8ebff2788 --- /dev/null +++ b/crates/ui-egui/tests/getpivotdata_point.rs @@ -0,0 +1,144 @@ +//! Generate GetPivotData: pointing at a value of a PivotTable while typing a formula (a click, +//! a drag that ends on one cell, or the arrow keys) inserts GETPIVOTDATA, as Excel does; labels +//! and other cells, or the setting turned off, insert the plain reference. + +use egui::{Event, Key, Modifiers, PointerButton, Pos2, pos2, vec2}; +use gridcraft_engine::Session; +use gridcraft_engine::core::{CellRef, Value}; +use gridcraft_ui_egui::{SheetApp, grid::Geo}; +use serde_json::json; + +type Harness = egui_kittest::Harness<'static, SheetApp>; + +/// Sales by Region and Product on Sheet2 (report A3:B12): East A4/B4, its Apples A5/B5 (120). +/// The active cell is H2, outside the report. +fn harness() -> Harness { + let mut s = Session::new(); + s.new_workbook(); + let rows = json!([ + ["Region", "Product", "Units", "Sales"], + ["East", "Apples", 5, 100], + ["East", "Pears", 3, 60], + ["West", "Apples", 7, 140], + ["West", "Plums", 2, 50], + ["North", "Pears", 4, 80], + ["East", "Apples", 1, 20] + ]); + s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap(); + s.execute("insert.pivotTable", json!({"source": "A1:D7", "rows": ["Region", "Product"], "values": ["Sales"]})).unwrap(); + s.execute("selection.set", json!({"range": "H2"})).unwrap(); + let mut h = egui_kittest::Harness::builder().with_step_dt(1.0 / 60.0).with_size(vec2(1200.0, 800.0)).build_ui_state( + |ui, app: &mut SheetApp| { + app.logic(ui.ctx()); + app.ui(ui); + }, + SheetApp::new(s, Default::default()), + ); + h.run_steps(4); + h +} + +fn center(h: &Harness, a: &str) -> Pos2 { + let c = CellRef::parse(a).unwrap(); + let app = h.state(); + let sheet = app.session.active().unwrap().wb.active().unwrap(); + let geo = Geo::new(sheet, app.grid.rect.unwrap(), app.view().scroll); + pos2(geo.x(sheet, c.col) + sheet.col_width(c.col) * geo.z * 0.5, geo.y(sheet, c.row) + sheet.row_height(c.row) * geo.z * 0.5) +} + +fn click(h: &mut Harness, pos: Pos2) { + for pressed in [true, false] { + h.input_mut() + .events + .extend([Event::PointerMoved(pos), Event::PointerButton { pos, button: PointerButton::Primary, pressed, modifiers: Modifiers::NONE }]); + h.step(); + } + h.run_steps(2); +} + +fn key(h: &mut Harness, key: Key) { + let ev = |pressed| Event::Key { key, physical_key: None, pressed, repeat: false, modifiers: Modifiers::NONE }; + h.input_mut().events.extend([ev(true), ev(false)]); + h.run_steps(2); +} + +fn text(h: &Harness) -> Option { + h.state().editor.as_ref().map(|e| e.text.clone()) +} + +fn start(h: &mut Harness, typed: &str) { + h.state_mut().begin_edit(Some(typed.into()), false); + h.run_steps(2); +} + +const APPLES: &str = r#"GETPIVOTDATA("Sum of Sales",$A$3,"Region","East","Product","Apples")"#; + +#[test] +fn clicking_a_value_inserts_getpivotdata() { + let mut h = harness(); + start(&mut h, "="); + let at = center(&h, "B5"); + click(&mut h, at); + assert_eq!(text(&h).as_deref(), Some(format!("={APPLES}").as_str())); + // A second click replaces it, like any pointed reference; a label is a plain reference. + let at = center(&h, "A5"); + click(&mut h, at); + assert_eq!(text(&h).as_deref(), Some("=A5")); + let at = center(&h, "B4"); + click(&mut h, at); + assert_eq!(text(&h).as_deref(), Some(r#"=GETPIVOTDATA("Sum of Sales",$A$3,"Region","East")"#)); + // The formula reads back the value. + key(&mut h, Key::Enter); + assert!(h.state().editor.is_none()); + let sheet = h.state().session.active().unwrap().wb.active().unwrap().clone(); + assert_eq!(sheet.value(CellRef::parse("H2").unwrap()), Value::Number(180.0)); +} + +#[test] +fn arrow_keys_point_with_getpivotdata_too() { + let mut h = harness(); + start(&mut h, "=10+"); + // From H2 to B5: six columns left, three rows down. + for _ in 0..6 { + key(&mut h, Key::ArrowLeft); + } + assert_eq!(text(&h).as_deref(), Some("=10+B2")); + for _ in 0..3 { + key(&mut h, Key::ArrowDown); + } + assert_eq!(text(&h).as_deref(), Some(format!("=10+{APPLES}").as_str())); + key(&mut h, Key::ArrowDown); + assert_eq!(text(&h).as_deref(), Some(r#"=10+GETPIVOTDATA("Sum of Sales",$A$3,"Region","East","Product","Pears")"#)); + key(&mut h, Key::ArrowRight); + assert_eq!(text(&h).as_deref(), Some("=10+C6"), "outside the report"); +} + +#[test] +fn turned_off_it_inserts_the_reference() { + let mut h = harness(); + assert_eq!(h.state_mut().run("pivot.generateGetPivotData", json!({})), Ok(json!({"on": false}))); + start(&mut h, "="); + let at = center(&h, "B5"); + click(&mut h, at); + assert_eq!(text(&h).as_deref(), Some("=B5")); + h.state_mut().editor = None; + h.state_mut().run("pivot.generateGetPivotData", json!({"on": true})).unwrap(); + start(&mut h, "="); + let at = center(&h, "B5"); + click(&mut h, at); + assert_eq!(text(&h).as_deref(), Some(format!("={APPLES}").as_str())); +} + +#[test] +fn from_another_sheet_the_pivottable_is_named_with_its_sheet() { + let mut h = harness(); + h.state_mut().run("sheet.activate", json!({"sheet": "Sheet1"})).unwrap(); + h.state_mut().run("selection.set", json!({"range": "H2"})).unwrap(); + h.run_steps(2); + start(&mut h, "="); + h.state_mut().run("sheet.activate", json!({"sheet": "Sheet2"})).unwrap(); + h.run_steps(2); + let at = center(&h, "B5"); + click(&mut h, at); + assert_eq!(text(&h).as_deref(), Some(r#"=GETPIVOTDATA("Sum of Sales",Sheet2!$A$3,"Region","East","Product","Apples")"#)); +} diff --git a/crates/ui-egui/tests/keytips_generated.rs b/crates/ui-egui/tests/keytips_generated.rs index 653554857..017d40ba6 100644 --- a/crates/ui-egui/tests/keytips_generated.rs +++ b/crates/ui-egui/tests/keytips_generated.rs @@ -1332,10 +1332,7 @@ fn keytip_nsp_() { fn keytip_nst_() { let mut h = harness(blank()); sequence(&mut h, "NST", false); - assert!(h.state().dialog.is_none(), "NST must not open a dialog"); - assert!(h.state().message.is_none(), "NST must not open a message box"); - assert!(h.state().toast.is_none(), "NST must not raise a toast"); - assert!(h.state().session.journal.is_empty(), "NST must not journal"); + assert!(h.state().message.is_some() || h.state().toast.is_some(), "NST must invoke insert.timeline and surface its missing-params error"); } #[test] @@ -2659,6 +2656,20 @@ fn keytip_jcc_() { assert!(h.state().message.is_some() || h.state().toast.is_some(), "JCC must invoke chart.changeType and surface its missing-params error"); } +#[test] +fn keytip_jt_() { + let mut h = harness(blank()); + sequence(&mut h, "JT", false); + assert_eq!(h.state().ui.ribbon_tab, "Home", "JT must not switch to the hidden PivotTable Analyze tab"); +} + +#[test] +fn keytip_jy_() { + let mut h = harness(blank()); + sequence(&mut h, "JY", false); + assert_eq!(h.state().ui.ribbon_tab, "Home", "JY must not switch to the hidden PivotTable Design tab"); +} + #[test] fn keytip_jts_() { let mut h = harness(blank()); @@ -2741,7 +2752,7 @@ fn keytip_q_() { #[test] fn generated_test_count_matches_the_table() { - const EXPECTED: usize = 364; + const EXPECTED: usize = 366; assert_eq!(gridcraft_ui_egui::keytips::keytip_table::SEQUENCES.len(), EXPECTED); } diff --git a/crates/ui-egui/tests/pivot_tabs.rs b/crates/ui-egui/tests/pivot_tabs.rs new file mode 100644 index 000000000..a262fd800 --- /dev/null +++ b/crates/ui-egui/tests/pivot_tabs.rs @@ -0,0 +1,195 @@ +//! The PivotTable Analyze and PivotTable Design tabs: they come with the active cell in a +//! PivotTable, their key tips (JT, JY) switch to them, and their controls run the pivot +//! commands, driven through real egui input. + +use egui::os::OperatingSystem; +use egui::{Event, Key, Modifiers}; +use egui_kittest::kittest::Queryable as _; +use gridcraft_engine::Session; +use gridcraft_engine::model::PivotTable; +use gridcraft_ui_egui::SheetApp; +use gridcraft_ui_egui::ribbon::pivot_tabs::{ANALYZE, DESIGN}; +use serde_json::json; + +type Harness = egui_kittest::Harness<'static, SheetApp>; + +fn harness(session: Session) -> Harness { + let mut h = egui_kittest::Harness::builder().with_os(OperatingSystem::Nix).with_size(egui::vec2(1800.0, 800.0)).build_ui_state( + |ui, app: &mut SheetApp| { + app.logic(ui.ctx()); + app.ui(ui); + }, + SheetApp::new(session, Default::default()), + ); + h.run_steps(4); + h +} + +/// A PivotTable of Sales by Region and Product on a new sheet; the active cell is its anchor. +fn with_pivot() -> Session { + let mut s = Session::new(); + s.new_workbook(); + let rows = json!([ + ["Region", "Product", "Units", "Sales"], + ["East", "Apples", 5, 100], + ["East", "Pears", 3, 60], + ["West", "Apples", 7, 140], + ["West", "Plums", 2, 50], + ["North", "Pears", 4, 80] + ]); + s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap(); + s.execute("insert.pivotTable", json!({"source": "A1:D6", "rows": ["Region", "Product"], "values": ["Sales"]})).unwrap(); + s +} + +fn pivot(h: &Harness) -> PivotTable { + h.state().session.active().unwrap().wb.active().unwrap().pivots[0].clone() +} + +/// Presses and releases `key`; a letter also types its text, as a real keyboard does. +fn key(h: &mut Harness, key: Key) { + let ev = |pressed| Event::Key { key, physical_key: None, pressed, repeat: false, modifiers: Modifiers::NONE }; + h.input_mut().events.push(ev(true)); + if key.name().len() == 1 { + h.input_mut().events.push(Event::Text(key.name().to_ascii_lowercase())); + } + h.input_mut().events.push(ev(false)); + h.run_steps(2); +} + +fn keytips(h: &mut Harness, seq: &str) { + key(h, Key::F10); + for ch in seq.chars() { + key(h, Key::from_name(&ch.to_string()).unwrap()); + } +} + +fn click(h: &mut Harness, label: &str) { + h.get_by_label(label).click(); + h.run_steps(2); +} + +#[test] +fn key_tips_switch_to_the_tabs_while_a_pivottable_is_selected() { + let mut h = harness(with_pivot()); + keytips(&mut h, "JT"); + assert_eq!(h.state().ui.ribbon_tab, ANALYZE); + // JT stays armed (Table Design's letters continue it): Escape pops J, T, then leaves. + for _ in 0..3 { + key(&mut h, Key::Escape); + } + assert_eq!(h.state().keytips.prefix(), None); + keytips(&mut h, "JY"); + assert_eq!(h.state().ui.ribbon_tab, DESIGN); + key(&mut h, Key::Escape); + // Outside the PivotTable the tabs are gone: JT leads to Table Design's letters only. + h.state_mut().run("selection.set", json!({"range": "K30"})).unwrap(); + h.run_steps(2); + assert_eq!(h.state().ui.ribbon_tab, "Home", "leaving the PivotTable goes back to Home"); + keytips(&mut h, "JT"); + assert_eq!(h.state().ui.ribbon_tab, "Home"); +} + +#[test] +fn design_tab_menus_change_the_layout() { + let mut h = harness(with_pivot()); + h.state_mut().run("ui.ribbonTab", json!({"tab": DESIGN})).unwrap(); + h.run_steps(2); + click(&mut h, "Grand Totals"); + click(&mut h, " Off for Rows and Columns"); + let p = pivot(&h); + assert!(!p.grand_totals_rows && !p.grand_totals_cols); + click(&mut h, "Subtotals"); + click(&mut h, " Show all Subtotals at Bottom of Group"); + let p = pivot(&h); + assert!(p.subtotals && !p.subtotals_top); + click(&mut h, "Report Layout"); + click(&mut h, " Show in Tabular Form"); + assert_eq!(pivot(&h).layout, gridcraft_engine::model::PivotLayout::Tabular); + click(&mut h, "PivotTable Styles"); + click(&mut h, "PivotStyleMedium9"); + assert_eq!(pivot(&h).style, "PivotStyleMedium9"); + // One undo step each. + h.state_mut().run("edit.undo", json!({})).unwrap(); + assert_eq!(pivot(&h).style, "PivotStyleLight16"); +} + +#[test] +fn analyze_tab_runs_the_pivot_commands() { + let mut h = harness(with_pivot()); + h.state_mut().run("ui.ribbonTab", json!({"tab": ANALYZE})).unwrap(); + h.run_steps(2); + // Field List hides and shows the PivotTable Fields pane. + assert!(h.query_by_label("PivotTable Fields").is_some()); + click(&mut h, "Field List"); + assert!(h.query_by_label("PivotTable Fields").is_none()); + click(&mut h, "Field List"); + assert!(h.query_by_label("PivotTable Fields").is_some()); + // Field Headers toggles the headers. + click(&mut h, "Field Headers"); + assert!(!pivot(&h).show_headers); + // Select › Entire PivotTable. + click(&mut h, "Select"); + click(&mut h, "Entire PivotTable"); + let range = pivot(&h).last_range.unwrap(); + assert_eq!(h.state().session.active().unwrap().selection.current(), range); + // Clear › Clear All empties the PivotTable but keeps it. + h.state_mut().run("selection.set", json!({"range": "A3"})).unwrap(); + h.run_steps(2); + click(&mut h, "Clear"); + click(&mut h, "Clear All"); + let p = pivot(&h); + assert!(p.rows.is_empty() && p.values.is_empty()); + assert!(h.state().message.is_none(), "{:?}", h.state().message); +} + +#[test] +fn the_tabs_and_their_controls_speak_german() { + let mut h = harness(with_pivot()); + h.state_mut().run("app.language.set", json!({"language": "de"})).unwrap(); + h.state_mut().run("ui.ribbonTab", json!({"tab": ANALYZE})).unwrap(); + h.run_steps(2); + for label in ["Datenquelle ändern", "Felder, Elemente und Gruppen", "Empfohlene PivotTables", "Feldliste", "Datenschnitt einfügen"] { + assert!(h.query_by_label(label).is_some(), "Analyze shows `{label}`"); + } + click(&mut h, "Aktualisieren (Alt+F5)"); + assert!(h.query_by_label("Alle aktualisieren").is_some()); + h.state_mut().run("ui.ribbonTab", json!({"tab": DESIGN})).unwrap(); + h.run_steps(2); + for label in ["Teilergebnisse", "Gesamtergebnisse", "Berichtslayout", "Leere Zeilen", "PivotTable-Formate"] { + assert!(h.query_by_label(label).is_some(), "Design shows `{label}`"); + } +} + +#[test] +fn active_field_group_works_on_the_active_cells_field() { + let mut h = harness(with_pivot()); + h.state_mut().run("ui.ribbonTab", json!({"tab": ANALYZE})).unwrap(); + // B5 is a value of Sum of Sales: Field Settings opens Value Field Settings for it. + h.state_mut().run("selection.set", json!({"range": "B5"})).unwrap(); + h.run_steps(2); + click(&mut h, "Field Settings"); + let dialog = h.state().dialog.as_ref().expect("Value Field Settings opens"); + assert_eq!(dialog.name(), "pivotValueFieldSettings"); + assert_eq!(dialog.values["field"], "Sum of Sales"); + assert_eq!(dialog.values["func"], "sum"); + h.state_mut().dialog = None; + // A4 is the East item of Region: Collapse Field and Expand Field act on Region. + h.state_mut().run("selection.set", json!({"range": "A4"})).unwrap(); + h.run_steps(2); + click(&mut h, "Collapse Field"); + assert_eq!(pivot(&h).rows[0].collapsed_items.len(), 3, "East, North and West"); + click(&mut h, "Expand Field"); + assert!(pivot(&h).rows[0].collapsed_items.is_empty()); + // Select › Values selects the value cells. + click(&mut h, "Select"); + // The Fields pane has a "Values" area too: the menu item is the button. + h.get_by_role_and_label(egui::accesskit::Role::Button, "Values").click(); + h.run_steps(2); + assert_eq!(h.state().session.active().unwrap().selection.current().a1(), "B4:B12"); + // Options › Generate GetPivotData switches the setting. + assert!(h.state().session.prefs.generate_get_pivot_data); + click(&mut h, "PivotTable Options"); + click(&mut h, "✓ Generate GetPivotData"); + assert!(!h.state().session.prefs.generate_get_pivot_data); +} diff --git a/crates/ui-egui/tests/pivot_timeline.rs b/crates/ui-egui/tests/pivot_timeline.rs new file mode 100644 index 000000000..b055277fe --- /dev/null +++ b/crates/ui-egui/tests/pivot_timeline.rs @@ -0,0 +1,248 @@ +//! PivotTable field buttons, filter dialogs, Grouping / Calculated Item dialogs and timelines, +//! driven through real egui input. + +use egui::{Event, Key, Modifiers, PointerButton}; +use egui_kittest::kittest::Queryable as _; +use gridcraft_engine::Session; +use gridcraft_engine::core::CellRef; +use gridcraft_ui_egui::{SheetApp, grid::Geo}; +use serde_json::json; + +fn harness(session: Session) -> egui_kittest::Harness<'static, SheetApp> { + let mut h = egui_kittest::Harness::builder().with_size(egui::vec2(1400.0, 900.0)).build_ui_state( + |ui, app: &mut SheetApp| { + let ctx = ui.ctx().clone(); + app.logic(&ctx); + app.ui(ui); + }, + SheetApp::new(session, Default::default()), + ); + h.run_steps(4); + h +} + +/// Sales by region on Sheet1 and a PivotTable of it at Sheet1!H3 (rows Region, values Sales). +fn sales_pivot() -> Session { + let mut s = Session::new(); + s.new_workbook(); + let rows = json!([ + ["Region", "Date", "Sales"], + ["East", "1/15/2026", 100], + ["West", "2/10/2026", 50], + ["East", "3/3/2026", 120], + ["North", "4/20/2026", 30], + ["West", "5/5/2026", 70] + ]); + s.execute("range.setValues", json!({"range": "A1", "values": rows})).unwrap(); + s.execute("insert.pivotTable", json!({"source": "Sheet1!$A$1:$C$6", "destination": "Sheet1!E3", "rows": ["Region"], "values": ["Sales"]})) + .unwrap(); + s.execute("selection.set", json!({"range": "E4"})).unwrap(); + s +} + +fn click_at(h: &mut egui_kittest::Harness<'static, SheetApp>, pos: egui::Pos2, modifiers: Modifiers) { + h.input_mut().events.push(Event::ModifiersChanged(modifiers)); + h.input_mut().events.extend([ + Event::PointerMoved(pos), + Event::PointerButton { pos, button: PointerButton::Primary, pressed: true, modifiers }, + Event::PointerButton { pos, button: PointerButton::Primary, pressed: false, modifiers }, + ]); + h.run_steps(3); + h.input_mut().events.push(Event::ModifiersChanged(Modifiers::NONE)); + h.run_steps(1); +} + +fn geo(h: &egui_kittest::Harness<'static, SheetApp>) -> Geo { + let app = h.state(); + let sheet = app.session.active().unwrap().wb.active().unwrap(); + Geo::new(sheet, app.grid.rect.unwrap(), app.view().scroll) +} + +fn pivot(h: &egui_kittest::Harness<'static, SheetApp>) -> gridcraft_engine::model::PivotTable { + h.state().session.active().unwrap().wb.active().unwrap().pivots[0].clone() +} + +#[test] +fn dormant_filter_rule_has_no_report_filter_button() { + use gridcraft_engine::model::{PivotCompare, PivotFilter, PivotRule}; + let s = sales_pivot(); + let mut sheet = s.active().unwrap().wb.active().unwrap().clone(); + let before = gridcraft_ui_egui::pivot_ui::header_buttons(&sheet); + sheet.pivots[0].filters.push(PivotFilter { + source_col: "Date".into(), + selected: None, + rule: Some(PivotRule::Label { op: PivotCompare::Equals, value: "1".into(), value2: String::new() }), + }); + let after = gridcraft_ui_egui::pivot_ui::header_buttons(&sheet); + assert_eq!(after.len(), before.len()); + assert_eq!(after[0].cell, before[0].cell); + assert_eq!(after[0].fields, before[0].fields); + assert!(!after[0].page); +} + +#[test] +fn row_labels_button_opens_label_and_value_filters() { + let mut h = harness(sales_pivot()); + let sheet = h.state().session.active().unwrap().wb.active().unwrap().clone(); + let buttons = gridcraft_ui_egui::pivot_ui::header_buttons(&sheet); + // Compact form: one Row Labels button, on the header cell above the items. + assert_eq!(buttons.len(), 1, "{buttons:?}"); + assert_eq!((buttons[0].cell, buttons[0].fields.clone()), (CellRef::parse("E3").unwrap(), vec!["Region".to_string()])); + let g = geo(&h); + let at = gridcraft_ui_egui::pivot_ui::button_rect(&g, &sheet, buttons[0].cell).center(); + click_at(&mut h, at, Modifiers::NONE); + assert!(h.state().grid.pivot_menu.is_some(), "the field menu opens"); + h.get_by_label("Label Filters").click(); + h.run_steps(3); + h.get_by_label("Begins With…").click(); + h.run_steps(3); + assert_eq!(h.state().dialog.as_ref().map(|d| d.name().to_string()).as_deref(), Some("pivotLabelFilter")); + h.state_mut().dialog.as_mut().unwrap().values.insert("value".into(), json!("E")); + h.run_steps(2); + h.get_by_label(" OK ").click(); + h.run_steps(3); + assert!(h.state().dialog.is_none()); + let rule = pivot(&h).filters.first().and_then(|f| f.rule.clone()); + assert!(matches!(rule, Some(gridcraft_engine::model::PivotRule::Label { .. })), "{rule:?}"); + // The button now shows the field filtered. + let sheet = h.state().session.active().unwrap().wb.active().unwrap().clone(); + assert!(gridcraft_ui_egui::pivot_ui::header_buttons(&sheet)[0].filtered); + // Value Filters › Top 10… opens the Top 10 dialog. + let at = gridcraft_ui_egui::pivot_ui::button_rect(&geo(&h), &sheet, CellRef::parse("E3").unwrap()).center(); + click_at(&mut h, at, Modifiers::NONE); + h.get_by_label("Value Filters").click(); + h.run_steps(3); + h.get_by_label("Top 10…").click(); + h.run_steps(3); + assert_eq!(h.state().dialog.as_ref().map(|d| d.name().to_string()).as_deref(), Some("pivotTop10")); + h.get_by_label(" OK ").click(); + h.run_steps(3); + let rule = pivot(&h).filters.first().and_then(|f| f.rule.clone()); + assert!(matches!(rule, Some(gridcraft_engine::model::PivotRule::Top { .. })), "{rule:?}"); +} + +#[test] +fn insert_timeline_from_the_ribbon_then_pick_a_period() { + let mut h = harness(sales_pivot()); + h.state_mut().ui.ribbon_tab = "Insert".into(); + h.run_steps(2); + h.get_by_label("Timeline").click(); + h.run_steps(3); + assert_eq!(h.state().dialog.as_ref().map(|d| d.name().to_string()).as_deref(), Some("insertTimeline")); + // Tick Date (the Fields pane lists a Date check box too). + h.state_mut().dialog.as_mut().unwrap().values.insert("checked".into(), json!(["Date"])); + h.run_steps(2); + h.get_by_label(" OK ").click(); + h.run_steps(3); + let tl = h.state().session.active().unwrap().wb.active().unwrap().timelines.first().cloned().expect("a timeline"); + // Selecting it shows the Timeline tab. + h.state_mut().selected_chart = Some(tl.id); + h.state_mut().ui.ribbon_tab = "Timeline".into(); + h.run_steps(3); + assert!(h.query_by_label("Timeline Caption:").is_some(), "the Timeline tab shows"); + // A click on the March tile selects March: East 120. + let wb = h.state().session.active().unwrap().wb.clone(); + let sheet = wb.active().unwrap().clone(); + let g = geo(&h); + let a = tl.anchor; + let r = egui::Rect::from_min_size( + egui::pos2(g.x(&sheet, a.cell.col) + a.dx * g.z, g.y(&sheet, a.cell.row) + a.dy * g.z), + egui::vec2(a.width * g.z, a.height.max(1.0) * g.z), + ); + let periods = gridcraft_engine::cmd::timeline::periods(&wb, &tl); + let lay = gridcraft_ui_egui::timeline_view::layout(r, g.z, &tl, &periods); + let (tile, _) = lay.tiles.iter().find(|(_, i)| periods[*i].label == "MAR").copied().expect("a March tile"); + click_at(&mut h, tile.center(), Modifiers::NONE); + let tl = h.state().session.active().unwrap().wb.active().unwrap().timelines[0].clone(); + assert_eq!(tl.selection, Some((46082.0, 46112.0))); + // Shift+click on May extends the selection: March to May. + let (tile, _) = lay.tiles.iter().find(|(_, i)| periods[*i].label == "MAY").copied().unwrap(); + click_at(&mut h, tile.center(), Modifiers::SHIFT); + let tl = h.state().session.active().unwrap().wb.active().unwrap().timelines[0].clone(); + assert_eq!(tl.selection, Some((46082.0, 46173.0))); + // The clear button selects all periods again. + let clear = lay.clear.unwrap(); + click_at(&mut h, clear.center(), Modifiers::NONE); + assert_eq!(h.state().session.active().unwrap().wb.active().unwrap().timelines[0].selection, None); +} + +#[test] +fn pivot_dialogs_open_and_close_in_german() { + let mut s = sales_pivot(); + s.execute("insert.timeline", json!({"field": "Date"})).unwrap(); + let mut h = harness(s); + h.state_mut().run("app.language.set", json!({"language": "de"})).unwrap(); + h.run_steps(2); + for (name, params) in [ + ("pivotGrouping", json!({"pivot": "PivotTable1", "field": "Date", "kind": "date", "levels": ["months"], "start": 46037, "end": 46148})), + ("pivotGrouping", json!({"pivot": "PivotTable1", "field": "Sales", "kind": "number", "start": 30, "end": 120, "interval": 10})), + ( + "pivotCalculatedItem", + json!({"pivot": "PivotTable1", "field": "Region", "name": "Formula1", "formula": "= 0", "fields": ["Region"], "items": ["East", "West"]}), + ), + ("pivotSolveOrder", json!({"pivot": "PivotTable1", "calculatedItems": [{"field": "Region", "name": "EW", "formula": "=East+West"}]})), + ("insertTimeline", json!({"pivot": "PivotTable1", "fields": ["Date"]})), + ("timelineConnections", json!({"timeline": "Date", "pivots": [{"name": "PivotTable1", "sheet": "Sheet1", "connected": true}]})), + ("pivotLabelFilter", json!({"pivot": "PivotTable1", "field": "Region", "op": "between"})), + ("pivotValueFilter", json!({"pivot": "PivotTable1", "field": "Region", "op": "greaterThan"})), + ("pivotTop10", json!({"pivot": "PivotTable1", "field": "Region"})), + ] { + h.state_mut().open_dialog(name, params); + h.run_steps(2); + assert!(h.state().dialog.is_some(), "{name} opens"); + h.input_mut().events.push(Event::Key { key: Key::Escape, physical_key: None, pressed: true, repeat: false, modifiers: Modifiers::NONE }); + h.run_steps(2); + h.state_mut().dialog = None; + h.state_mut().message = None; + } + // German labels: the Insert tab's Zeitachse and the Grouping dialog's title. + h.state_mut().ui.ribbon_tab = "Insert".into(); + h.run_steps(2); + assert!(h.query_by_label("Zeitachse").is_some()); + h.state_mut().open_dialog("pivotGrouping", json!({"pivot": "PivotTable1", "field": "Date", "kind": "date", "levels": ["months"]})); + h.run_steps(2); + assert!(h.query_by_label("Anzahl der Tage:").is_some()); +} + +#[test] +fn fields_pane_menu_has_label_and_value_filters() { + let mut h = harness(sales_pivot()); + h.get_by_label("Region ▾").click(); + h.run_steps(2); + assert!(h.query_by_label_contains("Label Filters").is_some()); + assert!(h.query_by_label_contains("Value Filters").is_some()); +} + +#[test] +fn pivottable_analyze_runs_grouping_calculated_items_and_timelines() { + let mut s = sales_pivot(); + // The active cell on East, an item of Region. + s.execute("selection.set", json!({"range": "E4"})).unwrap(); + let mut h = harness(s); + h.state_mut().ui.ribbon_tab = gridcraft_ui_egui::ribbon::pivot_tabs::ANALYZE.into(); + h.run_steps(3); + // Group Field opens the Grouping dialog only for number and date fields: Region is text. + h.get_by_label("Group Field").click(); + h.run_steps(3); + assert!(h.state().dialog.is_none()); + h.state_mut().message = None; + // Fields, Items, & Sets › Calculated Item… opens Insert Calculated Item for Region. + h.get_by_label("Fields, Items, & Sets").click(); + h.run_steps(3); + h.get_by_label("Calculated Item…").click(); + h.run_steps(3); + let d = h.state().dialog.as_ref().map(|d| (d.name().to_string(), d.values.get("field").cloned())); + assert_eq!(d, Some(("pivotCalculatedItem".to_string(), Some(json!("Region"))))); + h.state_mut().dialog = None; + // Insert Timeline lists the date fields. + h.get_by_label("Insert Timeline").click(); + h.run_steps(3); + assert_eq!(h.state().dialog.as_ref().map(|d| d.name().to_string()).as_deref(), Some("insertTimeline")); + h.state_mut().dialog = None; + // Group Selection on East and North. + h.state_mut().run("selection.set", json!({"range": "E4,E5"})).unwrap(); + h.run_steps(2); + h.get_by_label("Group Selection").click(); + h.run_steps(3); + assert_eq!(pivot(&h).group_fields.first().map(|g| g.name.as_str()), Some("Region2")); +} diff --git a/crates/xlsx/src/drawing.rs b/crates/xlsx/src/drawing.rs index 6dbd17e7e..6a79a411e 100644 --- a/crates/xlsx/src/drawing.rs +++ b/crates/xlsx/src/drawing.rs @@ -175,6 +175,11 @@ pub fn read_drawing(cx: &mut Ctx<'_>, part: &str, sheet: &mut Sheet) -> Result<( None => cx.warn(format!("chart {target} has no supported plot and was skipped")), } } + } else if uri == crate::timeline::GRAPHIC_URI { + // A timeline: its frame gives the anchor of the timeline of that name. + if let Some(name) = gd.child("timeslicer").and_then(|t| t.attr("name")) { + cx.timeline_frames.insert(name.to_lowercase(), anchor); + } } else if uri.contains("chartex") || uri.contains("chartEx") { cx.warn("a histogram, waterfall, treemap… chart without its chart part was skipped"); } else { @@ -311,6 +316,7 @@ pub enum Obj<'a> { Image(&'a Image, String), Chart(&'a gridcraft_model::Chart, String), Shape(&'a Shape), + Timeline(&'a gridcraft_model::Timeline), } /// The drawing part for a sheet's objects (relationship ids already assigned). @@ -359,6 +365,12 @@ pub fn write_drawing(sheet: &Sheet, theme: &Theme, objs: &[Obj<'_>]) -> String { xfrm(&ch.anchor, "xdr:xfrm") ); } + Obj::Timeline(t) => { + let (open, close) = anchor_open(sheet, &t.anchor); + close_tag = close; + s.push_str(&open); + s.push_str(&crate::timeline::frame_xml(t, id, &xfrm(&t.anchor, "xdr:xfrm"), &xfrm(&t.anchor, "a:xfrm"))); + } Obj::Shape(sh) => { let (open, close) = anchor_open(sheet, &sh.anchor); close_tag = close; diff --git a/crates/xlsx/src/fmla.rs b/crates/xlsx/src/fmla.rs index ef263873e..00b206945 100644 --- a/crates/xlsx/src/fmla.rs +++ b/crates/xlsx/src/fmla.rs @@ -134,6 +134,14 @@ mod tests { assert_eq!(to_file(&deleted), "#REF!"); } + #[test] + fn getpivotdata_has_no_prefix() { + // GETPIVOTDATA dates from Excel 2000: files name it as it is, both ways. + let f = r#"GETPIVOTDATA("Sum of Sales",Sheet2!$A$3,"Date",DATE(2026,1,15))"#; + assert_eq!(to_file(&Formula::new(f)), f); + assert_eq!(from_file(f), f); + } + #[test] fn implicit_intersection() { assert_eq!(to_file(&Formula::new("SUM(@A1:A3*B1:B3)")), "SUM(_xlfn.SINGLE(A1:A3)*B1:B3)"); diff --git a/crates/xlsx/src/lib.rs b/crates/xlsx/src/lib.rs index 3300f97fe..51a2e38ea 100644 --- a/crates/xlsx/src/lib.rs +++ b/crates/xlsx/src/lib.rs @@ -33,6 +33,7 @@ mod tables; #[cfg(any(test, feature = "test-support"))] pub mod testing; mod theme; +mod timeline; mod write; mod xls; mod xlsb; diff --git a/crates/xlsx/src/pivot.rs b/crates/xlsx/src/pivot.rs index 609ae530e..7398757df 100644 --- a/crates/xlsx/src/pivot.rs +++ b/crates/xlsx/src/pivot.rs @@ -3,13 +3,13 @@ //! //! Writing: one cache per PivotTable, with a records snapshot of the source list (omitted, with //! `saveData="0"`, for very large sources) and `refreshOnLoad="1"` so a reader recomputes the -//! report from the source on open. Fields used on an axis get shared items (or, for date -//! grouping, `fieldGroup` group items) so the table's `` can refer to them. The report +//! report from the source on open. Fields used on an axis get shared items (or, for date and +//! number range grouping, `fieldGroup` group items) so the table's `` can refer to them. The report //! cells the engine wrote stay on the sheet as plain cells. //! //! Reading: the cache's source and field names plus the table's areas, functions, show-values-as, //! layout flags, grand totals and style become a model [`PivotTable`]. Anything we don't model -//! (OLAP sources, calculated fields, numeric grouping…) becomes a warning. +//! (OLAP sources, manual groups, time grouping…) becomes a warning. use std::collections::HashMap; use std::fmt::Write as _; @@ -18,7 +18,8 @@ use gridcraft_core::date::datetime_from_serial; use gridcraft_core::{CellRef, DateSystem, RangeRef, Value, number_to_text}; use gridcraft_formula::quote_sheet; use gridcraft_model::{ - PivotCalcField, PivotDateGroup, PivotField, PivotFilter, PivotFunc, PivotLayout, PivotShowAs, PivotSort, PivotTable, PivotValue, Workbook, + BASE_NEXT, BASE_PREVIOUS, PivotCalcField, PivotCalcItem, PivotCompare, PivotDateGroup, PivotDateRange, PivotField, PivotFilter, PivotFunc, + PivotGroupField, PivotItemGroup, PivotLayout, PivotNumberGroup, PivotRule, PivotShowAs, PivotSort, PivotTable, PivotTopBy, PivotValue, Workbook, }; use crate::IoError; @@ -36,11 +37,15 @@ const MAX_SOURCE_CELLS: u64 = 10_000_000; const MAX_RECORD_CELLS: u64 = 2_000_000; /// Most shared items kept per field (Excel's limit is 1,048,576 unique items). const MAX_ITEMS: usize = 1_048_576; +/// Most day-span group items written (a span of a few days over centuries is capped). +const MAX_SPANS: usize = 100_000; /// Most cache fields read. const MAX_FIELDS: usize = 16_384; /// The `x14:dataField` extension (Excel 2010 show-values-as additions such as rank). const EXT_X14_DATA_FIELD: &str = "{E15A36E0-9728-4e99-A89B-3F7291B0FE68}"; const NS_X14: &str = "http://schemas.microsoft.com/office/spreadsheetml/2009/9/main"; +/// The `x14:pivotCacheDefinition` extension (the cache id slicers and timelines refer to). +const EXT_X14_CACHE: &str = "{725AE2AE-9491-48be-B2B4-4EB974FC3084}"; const MONTHS: [&str; 12] = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]; @@ -117,6 +122,9 @@ fn group_by_name(g: PivotDateGroup) -> &'static str { PivotDateGroup::Quarters => "quarters", PivotDateGroup::Months => "months", PivotDateGroup::Days => "days", + PivotDateGroup::Hours => "hours", + PivotDateGroup::Minutes => "minutes", + PivotDateGroup::Seconds => "seconds", PivotDateGroup::None => "range", } } @@ -127,10 +135,20 @@ fn group_caption(g: PivotDateGroup) -> &'static str { PivotDateGroup::Quarters => "Quarters", PivotDateGroup::Months => "Months", PivotDateGroup::Days => "Days", + PivotDateGroup::Hours => "Hours", + PivotDateGroup::Minutes => "Minutes", + PivotDateGroup::Seconds => "Seconds", PivotDateGroup::None => "Group", } } +/// An hour as Excel's Hours grouping shows it: `12 AM`, `1 AM` … `11 PM`. +fn hour_caption(h: u32) -> String { + let h = h % 24; + let twelve = if h.is_multiple_of(12) { 12 } else { h % 12 }; + format!("{twelve} {}", if h < 12 { "AM" } else { "PM" }) +} + /// `2024-03-05T00:00:00` for a date serial. fn iso(sys: DateSystem, serial: f64) -> Option { let d = datetime_from_serial(sys, serial)?; @@ -213,19 +231,30 @@ impl Stats { } } -/// Date grouping of a cache field. +/// Date, number range or manual (Group Selection) grouping of a cache field. +#[derive(Default)] struct Group { + /// The date part; `None` for a number range or manual grouping. by: PivotDateGroup, + /// Number range grouping (`groupBy="range"` with `startNum`, `endNum`, `groupInterval`). + number: Option, /// The database field the grouping applies to. base: usize, /// The next (coarser) grouping field of the same base. par: Option, start: f64, end: f64, + /// Starting at / Ending at set by hand (`autoStart="0"`, `autoEnd="0"`). + fixed: (bool, bool), + /// Days per item when grouping by days (`groupInterval`). + days: u32, + /// Group Selection: the group item of each of the base field's shared items (`discretePr`). + discrete: Option>, /// Group item captions, including the `end` items. items: Vec, } +#[derive(Default)] struct CField { name: String, /// Column offset in the source (database fields only). @@ -238,9 +267,17 @@ struct CField { group: Option, /// A calculated field's formula (without `=`); such fields aren't database fields. formula: Option, + /// The Group Selection field made from this one (`` without a grouping). + par: Option, + /// Written as dates although not grouped (the date field of a timeline). + dates: bool, } impl CField { + /// Grouped by a date part, or a timeline's date field (its values are written as dates). + fn date_grouped(&self) -> bool { + self.dates || self.group.as_ref().is_some_and(|g| g.number.is_none() && g.discrete.is_none()) + } /// Item captions the PivotTable's `` indices refer to. fn item_labels(&self) -> Vec { match &self.group { @@ -257,6 +294,10 @@ struct Cache { axis_map: Vec>, source_xml: String, records: Option<(u64, String)>, + /// Calculated items: (cache field, its shared item, formula without `=`). + calc_items: Vec<(usize, u32, String)>, + /// The cache's `x14:pivotCacheDefinition pivotCacheId` (written when a timeline refers to it). + ext_id: Option, } /// PivotTables being written; the tables themselves are finished after the styles part (their @@ -283,8 +324,12 @@ impl PivotWriter { pub fn add_sheet(&mut self, wb: &Workbook, si: usize, out: &mut Out, rels: &mut Rels) { let Some(sheet) = wb.sheet(si) else { return }; for (idx, pt) in sheet.pivots.iter().enumerate() { - let Some(cache) = build_cache(wb, pt, si) else { continue }; + let Some(mut cache) = build_cache(wb, pt, si) else { continue }; let n = self.caches.len() as u32 + 1; + // A timeline cache refers to the pivot cache of the PivotTable it lists by this id. + if wb.sheets.iter().flat_map(|s| s.timelines.iter()).any(|t| t.pivots.first() == Some(&pt.id)) { + cache.ext_id = Some(n); + } let cache_part = format!("xl/pivotCache/pivotCacheDefinition{n}.xml"); let rec_rid = if let Some((count, xml)) = &cache.records { let mut r = Rels::default(); @@ -340,11 +385,32 @@ impl PivotWriter { v } + /// The pivot cache id of the PivotTable at (sheet, index), if it was written. + pub fn cache_id(&self, sheet: usize, idx: usize) -> Option { + self.pending.iter().find(|p| p.sheet == sheet && p.idx == idx).map(|p| p.cache_id) + } + + /// The first and last number of source field `field` of the PivotTable at (sheet, index). + pub fn bounds(&self, sheet: usize, idx: usize, field: &str) -> Option<(f64, f64)> { + let p = self.pending.iter().find(|p| p.sheet == sheet && p.idx == idx)?; + let f = p.cache.fields.iter().find(|f| f.col.is_some() && same(&f.name, field))?; + f.stats.number.then_some((f.stats.min, f.stats.max)) + } + /// Writes the PivotTable parts. `fmt_id` maps a number format code to its styles id. pub fn finish(self, wb: &Workbook, out: &mut Out, fmt_id: &dyn Fn(&str) -> Option) { for p in self.pending { let Some(pt) = wb.sheet(p.sheet).and_then(|s| s.pivots.get(p.idx)) else { continue }; - let xml = table_xml(pt, p.idx, p.cache_id, &p.cache, fmt_id); + // The selections of the timelines filtering this PivotTable. + let timeline_filters: Vec = crate::timeline::filters_of(wb, pt.id) + .into_iter() + .zip(crate::timeline::FILTER_ID_BASE..) + .filter_map(|(t, id)| { + let fld = p.cache.fields.iter().position(|f| f.col.is_some() && same(&f.name, &t.field))?; + Some(crate::timeline::filter_xml(fld, id, t)) + }) + .collect(); + let xml = table_xml(pt, p.idx, p.cache_id, &p.cache, fmt_id, &timeline_filters); out.part(&p.part, Some(CT_PIVOT_TABLE), xml.into_bytes()); } } @@ -435,16 +501,7 @@ fn build_cache(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Option Option)> = Vec::new(); + // (base column field, groupings fine → coarse, the Grouping dialog's bounds and day span) + let mut groupings: Vec<(usize, Vec, Option)> = Vec::new(); for pf in &axis { let Some(b) = find(&fields, &pf.source_col) else { continue }; if pf.date_group == PivotDateGroup::None || !fields.get(b).is_some_and(groupable) { continue; } - match groupings.iter_mut().find(|(x, _)| *x == b) { - Some((_, v)) => { + match groupings.iter_mut().find(|(x, ..)| *x == b) { + Some((_, v, range)) => { if !v.contains(&pf.date_group) { v.push(pf.date_group); } + if range.is_none() { + *range = pf.date_range; + } } - None => groupings.push((b, vec![pf.date_group])), + None => groupings.push((b, vec![pf.date_group], pf.date_range)), } } - let fineness = |g: &PivotDateGroup| match g { - PivotDateGroup::Days => 0, - PivotDateGroup::Months => 1, - PivotDateGroup::Quarters => 2, - PivotDateGroup::Years => 3, - PivotDateGroup::None => 4, - }; - for (b, list) in groupings.iter_mut() { + let fineness = |g: &PivotDateGroup| g.fineness().unwrap_or(PivotDateGroup::LEVELS.len()); + for (b, list, range) in groupings.iter_mut() { list.sort_by_key(fineness); let (min, max) = fields.get(*b).map(|f| (f.stats.min, f.stats.max)).unwrap_or((0.0, 0.0)); - let (start, end) = (min.floor(), max.floor() + 1.0); - let make = |by: PivotDateGroup| Group { by, base: *b, par: None, start, end, items: group_items(wb.date_system, by, start, end) }; + let range = range.unwrap_or_default(); + // Excel's automatic end is the day after a last whole date. + let auto_end = if max.fract() == 0.0 { max + 1.0 } else { max }.floor(); + let (start, end) = (range.start.unwrap_or(min).floor(), range.end.map_or(auto_end, f64::floor)); + let fixed = (range.start.is_some(), range.end.is_some()); + let days = range.days.clamp(1, PivotDateRange::MAX_DAYS); + let make = |by: PivotDateGroup| { + let days = if by == PivotDateGroup::Days { days } else { 1 }; + Group { by, base: *b, start, end, fixed, days, items: group_items(wb.date_system, by, start, end, days), ..Group::default() } + }; let mut prev = *b; for (k, by) in list.iter().enumerate() { if k == 0 { @@ -525,23 +587,50 @@ fn build_cache(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Option = Vec::new(); + for gf in &pt.group_fields { + let used = axis.iter().any(|a| same(&a.source_col, &gf.name)) || pt.filters.iter().any(|f| same(&f.source_col, &gf.name)); + let Some(b) = find(&fields, &gf.base) else { continue }; + if !used || fields.get(b).is_some_and(|f| f.group.is_some() || f.par.is_some()) || fields.iter().any(|f| same(&f.name, &gf.name)) { + continue; + } + let new = fields.len(); + if let Some(f) = fields.get_mut(b) { + f.itemized = true; + f.par = Some(new); + } + fields.push(CField { + name: gf.name.clone(), + group: Some(Group { base: b, days: 1, discrete: Some(vec![]), ..Group::default() }), + ..CField::default() + }); + manual.push((new, gf, b)); + } + let find_manual = |fields: &[CField], name: &str| { + fields.iter().position(|f| f.col.is_none() && f.group.as_ref().is_some_and(|g| g.discrete.is_some()) && same(&f.name, name)) + }; let mut axis_map: Vec> = Vec::with_capacity(axis.len()); for pf in &axis { let Some(b) = find(&fields, &pf.source_col) else { - axis_map.push(None); + axis_map.push(find_manual(&fields, &pf.source_col)); continue; }; let grouped = fields.iter().position(|f| f.group.as_ref().is_some_and(|g| g.base == b && g.by == pf.date_group)); @@ -567,10 +656,26 @@ fn build_cache(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Option Option = fields.get(*b).map(|f| f.items.iter().map(label).collect()).unwrap_or_default(); + let mut items: Vec = Vec::new(); + let mut map: Vec = Vec::with_capacity(base_labels.len()); + for l in &base_labels { + let caption = gf.group_of(l).map(|g| g.name.clone()).unwrap_or_else(|| l.clone()); + let k = match items.iter().position(|x| same(x, &caption)) { + Some(k) => k, + None => { + items.push(caption); + items.len() - 1 + } + }; + map.push(k as u32); + } + if let Some(g) = fields.get_mut(*gi).and_then(|f| f.group.as_mut()) { + g.items = items; + g.discrete = Some(map); + } + } + // Calculated items are shared items of their field, after the data items; no record uses them. + let mut calc_items: Vec<(usize, u32, String)> = Vec::new(); + for ci in &pt.calculated_items { + let Some(i) = find(&fields, &ci.field) else { continue }; + let Some(f) = fields.get_mut(i) else { continue }; + if !f.itemized || f.group.is_some() || f.items.len() >= MAX_ITEMS || f.items.iter().any(|v| same(&label(v), &ci.name)) { + continue; + } + let formula = ci.formula.trim(); + calc_items.push((i, f.items.len() as u32, formula.strip_prefix('=').unwrap_or(formula).trim().to_string())); + f.items.push(Value::text(ci.name.as_str())); + } // Records snapshot. let db_fields = fields.iter().filter(|f| f.col.is_some()).count() as u64; let records = if (rows.len() as u64).saturating_mul(db_fields) <= MAX_RECORD_CELLS { @@ -603,7 +741,7 @@ fn build_cache(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Option s.push_str(&item_xml(&v, wb.date_system, false)), } } else { - s.push_str(&item_xml(&v, wb.date_system, f.group.is_some())); + s.push_str(&item_xml(&v, wb.date_system, f.date_grouped())); } } s.push_str(""); @@ -619,23 +757,24 @@ fn build_cache(wb: &Workbook, pt: &PivotTable, pivot_sheet: usize) -> Option Vec { +fn group_items(sys: DateSystem, by: PivotDateGroup, start: f64, end: f64, days: u32) -> Vec { let mut v = vec![format!("<{}", mdy(sys, start))]; match by { + PivotDateGroup::Days if days > 1 => { + // Spans of `days` days from the start, the last one cut at the end. + let step = f64::from(days); + let mut lo = start; + while lo <= end && v.len() < MAX_SPANS { + v.push(format!("{} - {}", mdy(sys, lo), mdy(sys, (lo + step - 1.0).min(end)))); + lo += step; + } + } PivotDateGroup::Years => { let y0 = datetime_from_serial(sys, start).map(|d| d.year).unwrap_or(1900); let y1 = datetime_from_serial(sys, end - 1.0).map(|d| d.year).unwrap_or(y0).max(y0); @@ -645,6 +784,8 @@ fn group_items(sys: DateSystem, by: PivotDateGroup, start: f64, end: f64) -> Vec } PivotDateGroup::Quarters => v.extend((1..=4).map(|q| format!("Qtr{q}"))), PivotDateGroup::Months => v.extend(MONTHS.iter().map(|m| m.to_string())), + PivotDateGroup::Hours => v.extend((0..24).map(hour_caption)), + PivotDateGroup::Minutes | PivotDateGroup::Seconds => v.extend((0..60).map(|m| format!(":{m:02}"))), PivotDateGroup::Days | PivotDateGroup::None => { const DAYS: [u32; 12] = [31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31]; for (m, n) in MONTHS.iter().zip(DAYS) { @@ -687,7 +828,7 @@ fn cache_xml(c: &Cache, records_rid: Option<&str>, sys: DateSystem) -> String { let _ = write!(s, "{}", c.source_xml); let _ = write!(s, "", c.fields.len()); for f in &c.fields { - let fmt = if f.group.is_some() { 14 } else { 0 }; + let fmt = if f.date_grouped() { 14 } else { 0 }; let _ = write!(s, ", sys: DateSystem) -> String { let st = &f.stats; if f.col.is_some() { s.push_str(", sys: DateSystem) -> String { if let (Some(a), Some(b)) = (iso(sys, st.min), iso(sys, st.max)) { let _ = write!(s, " minDate=\"{a}\" maxDate=\"{b}\""); } - let _ = g; } else { let semi = st.text || st.blank || st.boolean || st.error; if !semi { @@ -743,15 +883,53 @@ fn cache_xml(c: &Cache, records_rid: Option<&str>, sys: DateSystem) -> String { s.push_str("/>"); } } + if let (None, Some(p)) = (&f.group, f.par) { + let _ = write!(s, ""); + } if let Some(g) = &f.group { s.push_str("", g.base); - let _ = write!(s, "", map.len()); + for k in map { + let _ = write!(s, ""); + } + let _ = write!(s, "", g.items.len()); + for it in &g.items { + let _ = write!(s, "", esc_attr(it)); + } + s.push_str(""); + continue; + } + match g.number { + Some(ng) => { + let _ = write!( + s, + " { + s.push_str(" 1 { + let _ = write!(s, " groupInterval=\"{}\"", g.days); + } + } } let _ = write!(s, "/>", g.items.len()); for it in &g.items { @@ -761,16 +939,35 @@ fn cache_xml(c: &Cache, records_rid: Option<&str>, sys: DateSystem) -> String { } s.push_str(""); } - s.push_str(""); + s.push_str(""); + if !c.calc_items.is_empty() { + let _ = write!(s, "", c.calc_items.len()); + for (fi, x, formula) in &c.calc_items { + let _ = write!( + s, + "", + esc_attr(formula) + ); + } + s.push_str(""); + } + if let Some(id) = c.ext_id { + let _ = + write!(s, ""); + } + s.push_str(""); s } -fn table_xml(pt: &PivotTable, idx: usize, cache_id: u32, c: &Cache, fmt_id: &dyn Fn(&str) -> Option) -> String { +fn table_xml(pt: &PivotTable, idx: usize, cache_id: u32, c: &Cache, fmt_id: &dyn Fn(&str) -> Option, extra_filters: &[String]) -> String { let nf = c.fields.len(); let find_db = |name: &str| c.fields.iter().position(|f| f.col.is_some() && same(&f.name, name)); + let find_page = |name: &str| { + find_db(name).or_else(|| c.fields.iter().position(|f| f.group.as_ref().is_some_and(|g| g.discrete.is_some()) && same(&f.name, name))) + }; let row_idx: Vec = c.axis_map.iter().take(pt.rows.len()).filter_map(|x| *x).collect(); let col_idx: Vec = c.axis_map.iter().skip(pt.rows.len()).filter_map(|x| *x).collect(); - // Filters: on an axis field → hidden items there; otherwise a page field. + // Rules off the axes are dormant, not page fields. Item selections off an axis are pages. let mut hidden: HashMap = HashMap::new(); let mut pages: Vec<(usize, &PivotFilter)> = Vec::new(); for f in &pt.filters { @@ -781,14 +978,15 @@ fn table_xml(pt: &PivotTable, idx: usize, cache_id: u32, c: &Cache, fmt_id: &dyn hidden.insert(*i, f); } } - None => { - if let Some(i) = find_db(&f.source_col) + None if f.rule.is_none() => { + if let Some(i) = find_page(&f.source_col) && !pages.iter().any(|(j, _)| *j == i) { pages.push((i, f)); hidden.insert(i, f); } } + None => {} } } let find_calc = |name: &str| c.fields.iter().position(|f| f.formula.is_some() && same(&f.name, name)); @@ -877,7 +1075,7 @@ fn table_xml(pt: &PivotTable, idx: usize, cache_id: u32, c: &Cache, fmt_id: &dyn PivotLayout::Outline => s.push_str(" compact=\"0\""), PivotLayout::Tabular => s.push_str(" compact=\"0\" outline=\"0\""), } - if f.group.is_some() { + if f.date_grouped() { s.push_str(" numFmtId=\"14\""); } let filter = hidden.get(&i).copied(); @@ -897,6 +1095,9 @@ fn table_xml(pt: &PivotTable, idx: usize, cache_id: u32, c: &Cache, fmt_id: &dyn s.push_str(" showAll=\"0\""); if let Some(af) = axis_field(i) { match af.sort { + // Excel sorts the captions of date groups as text under `sortType`; their own + // order is already chronological, so ascending date groups go without it. + PivotSort::Asc if f.date_grouped() => {} PivotSort::Asc => s.push_str(" sortType=\"ascending\""), PivotSort::Desc => s.push_str(" sortType=\"descending\""), PivotSort::None => {} @@ -969,29 +1170,50 @@ fn table_xml(pt: &PivotTable, idx: usize, cache_id: u32, c: &Cache, fmt_id: &dyn if v.func != PivotFunc::Sum { let _ = write!(s, " subtotal=\"{}\"", func_name(v.func)); } - let base_field = row_idx.first().or(col_idx.first()).copied().unwrap_or(0); - match v.show_as { - PivotShowAs::Normal => s.push_str(" baseField=\"0\" baseItem=\"0\""), - PivotShowAs::PercentOfGrandTotal => s.push_str(" showDataAs=\"percentOfTotal\" baseField=\"0\" baseItem=\"0\""), - PivotShowAs::PercentOfColumnTotal => s.push_str(" showDataAs=\"percentOfCol\" baseField=\"0\" baseItem=\"0\""), - PivotShowAs::PercentOfRowTotal => s.push_str(" showDataAs=\"percentOfRow\" baseField=\"0\" baseItem=\"0\""), - PivotShowAs::RunningTotal => { - let _ = write!(s, " showDataAs=\"runTotal\" baseField=\"{base_field}\" baseItem=\"0\""); - } - PivotShowAs::Rank => { - let _ = write!(s, " baseField=\"{base_field}\" baseItem=\"0\""); - } + // The base field: the named axis field, else the first row (or column) field. + let named = v.base_field.as_deref().and_then(|b| { + let p = pt.rows.iter().chain(pt.columns.iter()).position(|a| same(&a.source_col, b))?; + c.axis_map.get(p).copied().flatten() + }); + let base_field = named.or(row_idx.first().or(col_idx.first()).copied()).unwrap_or(0); + let base_item = match v.base_item.as_deref() { + Some(BASE_PREVIOUS) => 1_048_828, + Some(BASE_NEXT) => 1_048_829, + Some(label) => c.fields.get(base_field).and_then(|f| f.item_labels().iter().position(|l| same(l, label))).unwrap_or(0), + None => 0, + }; + let (data_as, x14): (Option<&str>, Option<&str>) = match v.show_as { + PivotShowAs::Normal => (None, None), + PivotShowAs::PercentOfGrandTotal => (Some("percentOfTotal"), None), + PivotShowAs::PercentOfColumnTotal => (Some("percentOfCol"), None), + PivotShowAs::PercentOfRowTotal => (Some("percentOfRow"), None), + PivotShowAs::PercentOf => (Some("percent"), None), + PivotShowAs::DifferenceFrom => (Some("difference"), None), + PivotShowAs::PercentDifferenceFrom => (Some("percentDiff"), None), + PivotShowAs::RunningTotal => (Some("runTotal"), None), + PivotShowAs::Index => (Some("index"), None), + PivotShowAs::PercentOfParentRowTotal => (None, Some("percentOfParentRow")), + PivotShowAs::PercentOfParentColumnTotal => (None, Some("percentOfParentCol")), + PivotShowAs::PercentOfParentTotal => (None, Some("percentOfParent")), + PivotShowAs::PercentRunningTotal => (None, Some("percentOfRunningTotal")), + PivotShowAs::RankAscending => (None, Some("rankAscending")), + PivotShowAs::Rank => (None, Some("rankDescending")), + }; + if let Some(a) = data_as { + let _ = write!(s, " showDataAs=\"{a}\""); } + let _ = write!(s, " baseField=\"{base_field}\" baseItem=\"{base_item}\""); if let Some(id) = v.number_format.as_deref().and_then(fmt_id) { let _ = write!(s, " numFmtId=\"{id}\""); } - if v.show_as == PivotShowAs::Rank { - let _ = write!( - s, - ">" - ); - } else { - s.push_str("/>"); + match x14 { + Some(x) => { + let _ = write!( + s, + ">" + ); + } + None => s.push_str("/>"), } } s.push_str(""); @@ -1001,10 +1223,147 @@ fn table_xml(pt: &PivotTable, idx: usize, cache_id: u32, c: &Cache, fmt_id: &dyn let _ = write!(s, " name=\"{}\"", esc_attr(&pt.style)); } s.push_str(" showRowHeaders=\"1\" showColHeaders=\"1\" showRowStripes=\"0\" showColStripes=\"0\" showLastColumn=\"1\"/>"); + let filter_field = |name: &str| match pt.rows.iter().chain(pt.columns.iter()).position(|a| same(&a.source_col, name)) { + Some(p) => c.axis_map.get(p).copied().flatten(), + None => find_page(name), + }; + filters_xml(&mut s, pt, &filter_field, &vals, extra_filters); s.push_str(""); s } +// ---------------------------------------------------------------- label, value and Top 10 filters + +/// The `ST_PivotFilterType` suffix of a comparison (`captionBeginsWith`, `valueBetween`). +fn compare_suffix(op: PivotCompare) -> &'static str { + match op { + PivotCompare::Equals => "Equal", + PivotCompare::NotEquals => "NotEqual", + PivotCompare::BeginsWith => "BeginsWith", + PivotCompare::NotBeginsWith => "NotBeginsWith", + PivotCompare::EndsWith => "EndsWith", + PivotCompare::NotEndsWith => "NotEndsWith", + PivotCompare::Contains => "Contains", + PivotCompare::NotContains => "NotContains", + PivotCompare::Greater => "GreaterThan", + PivotCompare::GreaterOrEqual => "GreaterThanOrEqual", + PivotCompare::Less => "LessThan", + PivotCompare::LessOrEqual => "LessThanOrEqual", + PivotCompare::Between => "Between", + PivotCompare::NotBetween => "NotBetween", + } +} + +/// The AutoFilter `customFilters` that express a comparison. +fn custom_filters(op: PivotCompare, a: &str, b: &str) -> String { + let one = |oper: Option<&str>, v: &str| match oper { + Some(o) => format!("", esc_attr(v)), + None => format!("", esc_attr(v)), + }; + let (and, parts) = match op { + PivotCompare::Equals => (false, vec![one(None, a)]), + PivotCompare::NotEquals => (false, vec![one(Some("notEqual"), a)]), + PivotCompare::BeginsWith => (false, vec![one(None, &format!("{a}*"))]), + PivotCompare::NotBeginsWith => (false, vec![one(Some("notEqual"), &format!("{a}*"))]), + PivotCompare::EndsWith => (false, vec![one(None, &format!("*{a}"))]), + PivotCompare::NotEndsWith => (false, vec![one(Some("notEqual"), &format!("*{a}"))]), + PivotCompare::Contains => (false, vec![one(None, &format!("*{a}*"))]), + PivotCompare::NotContains => (false, vec![one(Some("notEqual"), &format!("*{a}*"))]), + PivotCompare::Greater => (false, vec![one(Some("greaterThan"), a)]), + PivotCompare::GreaterOrEqual => (false, vec![one(Some("greaterThanOrEqual"), a)]), + PivotCompare::Less => (false, vec![one(Some("lessThan"), a)]), + PivotCompare::LessOrEqual => (false, vec![one(Some("lessThanOrEqual"), a)]), + PivotCompare::Between => (true, vec![one(Some("greaterThanOrEqual"), a), one(Some("lessThanOrEqual"), b)]), + PivotCompare::NotBetween => (false, vec![one(Some("lessThan"), a), one(Some("greaterThan"), b)]), + }; + format!("{}", if and { " and=\"1\"" } else { "" }, parts.concat()) +} + +/// ``: active and dormant label, value and Top 10 filters. `fld` is the +/// cache field and `iMeasureFld` the position of the value field in ``. +fn filters_xml(s: &mut String, pt: &PivotTable, field: &dyn Fn(&str) -> Option, vals: &[(usize, &PivotValue)], extra: &[String]) { + let measure = |name: &str| vals.iter().position(|(_, v)| same(&v.name, name)); + let mut out = String::new(); + let mut n = 0; + for f in &pt.filters { + let Some(rule) = &f.rule else { continue }; + let Some(fld) = field(&f.source_col) else { continue }; + let (ty, attrs, inner) = match rule { + PivotRule::Label { op, value, value2 } => { + let mut attrs = format!(" stringValue1=\"{}\"", esc_attr(value)); + if op.is_range() { + let _ = write!(attrs, " stringValue2=\"{}\"", esc_attr(value2)); + } + (format!("caption{}", compare_suffix(*op)), attrs, custom_filters(*op, value, value2)) + } + PivotRule::Value { op, field, value, value2 } => { + let Some(m) = measure(field) else { continue }; + let (a, b) = (crate::xml::num(*value), crate::xml::num(*value2)); + (format!("value{}", compare_suffix(*op)), format!(" iMeasureFld=\"{m}\""), custom_filters(*op, &a, &b)) + } + PivotRule::Top { top, count, by, field } => { + let Some(m) = measure(field) else { continue }; + let ty = match by { + PivotTopBy::Items => "count", + PivotTopBy::Percent => "percent", + PivotTopBy::Sum => "sum", + }; + let inner = format!( + "", + u8::from(*top), + u8::from(*by == PivotTopBy::Percent), + crate::xml::num(*count) + ); + (ty.to_string(), format!(" iMeasureFld=\"{m}\""), inner) + } + }; + n += 1; + let _ = write!( + out, + "{inner}" + ); + } + // Timelines' date filters come after the label, value and Top 10 filters. + for x in extra.iter().filter(|x| !x.is_empty()) { + n += 1; + out.push_str(x); + } + if n > 0 { + let _ = write!(s, "{out}"); + } +} + +/// Reads a `` into a rule; `measure` gives a value field's caption by its position in +/// ``. `None` for the filter types the model does not have (date filters). +fn read_filter(e: &El, measure: &dyn Fn(u32) -> Option) -> Option { + let ty = e.attr("type")?; + let col = e.child("autoFilter").and_then(|a| a.child("filterColumn")); + let compare = |suffix: &str| PivotCompare::ALL.into_iter().find(|c| compare_suffix(*c) == suffix); + if let Some(op) = ty.strip_prefix("caption").and_then(compare) { + let text = |k: &str| e.attr(k).unwrap_or("").to_string(); + return Some(PivotRule::Label { op, value: text("stringValue1"), value2: text("stringValue2") }); + } + let field = e.attr_u32("iMeasureFld").and_then(measure)?; + if let Some(op) = ty.strip_prefix("value").and_then(compare) { + let nums: Vec = col + .and_then(|c| c.child("customFilters")) + .map(|c| c.kids("customFilter").filter_map(|x| x.attr("val").and_then(|v| v.parse::().ok())).take(2).collect()) + .unwrap_or_default(); + let n = |i: usize, k: &str| nums.get(i).copied().or_else(|| e.attr(k).and_then(|v| v.parse::().ok())).filter(|x| x.is_finite()); + return Some(PivotRule::Value { op, field, value: n(0, "stringValue1")?, value2: n(1, "stringValue2").unwrap_or(0.0) }); + } + let by = match ty { + "count" => PivotTopBy::Items, + "percent" => PivotTopBy::Percent, + "sum" => PivotTopBy::Sum, + _ => return None, + }; + let t = col.and_then(|c| c.child("top10")); + let count = t.and_then(|t| t.attr("val")).and_then(|v| v.parse::().ok()).filter(|x| x.is_finite()).unwrap_or(10.0); + let top = t.is_none_or(|t| t.flag("top", true)); + Some(PivotRule::Top { top, count, by, field }) +} + /// ``/``: the axis's cache field indices, with Σ Values (`-2`) at `values_at`. fn axis_fields_xml(s: &mut String, tag: &str, idx: &[usize], values_at: Option) { let n = idx.len() + usize::from(values_at.is_some()); @@ -1033,12 +1392,24 @@ struct FieldIn { database: bool, /// Shared item captions. items: Vec, - /// (base field, grouping, group item captions); grouping `None` = unsupported grouping. - group: Option<(Option, Option, Vec)>, + /// (base field, grouping, group item captions). + group: Option<(Option, GroupBy, Vec)>, /// A calculated field's formula. formula: Option, } +/// What a cache field's `fieldGroup` groups by. +#[derive(Clone, Debug)] +enum GroupBy { + /// A date or time part, with the Grouping dialog's bounds and day span when set. + Date(PivotDateGroup, Option), + Range(PivotNumberGroup), + /// Group Selection: the group item of each of the base field's shared items. + Discrete(Vec), + /// A grouping the model doesn't have (an unusable range). + Unsupported, +} + impl FieldIn { fn labels(&self) -> &[String] { match &self.group { @@ -1052,6 +1423,8 @@ impl FieldIn { struct CacheIn { source: String, fields: Vec, + /// Calculated items, in solve order. + calc_items: Vec, } /// Workbook-level pivot cache lookup, filled from ``; caches are parsed once. @@ -1060,6 +1433,9 @@ pub struct CachesIn { by_id: HashMap, parsed: HashMap>, next_id: u32, + /// Filters a timeline keeps in a PivotTable: (PivotTable name, filter id). They are the + /// timeline's selection, read with the timeline. + pub timeline_filters: Vec<(String, u32)>, } impl CachesIn { @@ -1074,7 +1450,7 @@ impl CachesIn { } } } - CachesIn { by_id, parsed: HashMap::new(), next_id: 1 } + CachesIn { by_id, parsed: HashMap::new(), next_id: 1, timeline_filters: vec![] } } } @@ -1126,14 +1502,44 @@ fn read_cache(cx: &mut Ctx<'_>, part: &str, sheet_name: &str) -> Option if let Some(cf) = x.child("cacheFields") { for f in cf.kids("cacheField").take(MAX_FIELDS) { let items = f.child("sharedItems").map(|si| si.children.iter().map(|e| item_label_el(e, sys)).collect()).unwrap_or_default(); - let group = f.child("fieldGroup").map(|g| { + // A `fieldGroup` with neither `rangePr` nor `discretePr` only names the field grouped + // from this one (`par`): this field itself isn't grouped. + let group = f.child("fieldGroup").filter(|g| g.child("rangePr").is_some() || g.child("discretePr").is_some()).map(|g| { let base = g.attr_u32("base").map(|b| b as usize); - let by = match g.child("rangePr").map(|r| r.attr("groupBy").unwrap_or("range")) { - Some("years") => Some(PivotDateGroup::Years), - Some("quarters") => Some(PivotDateGroup::Quarters), - Some("months") => Some(PivotDateGroup::Months), - Some("days") => Some(PivotDateGroup::Days), - _ => None, + let rp = g.child("rangePr"); + let num = |k: &str| rp.and_then(|r| r.attr(k)).and_then(|v| v.trim().parse::().ok()); + // Starting at / Ending at set by hand, and Number of days. + let date = |k: &str| rp.and_then(|r| r.attr(k)).and_then(|v| crate::sheet_read::parse_iso_datetime(v, sys)); + let start = rp.filter(|r| !r.flag("autoStart", true)).and_then(|_| date("startDate")); + let end = rp.filter(|r| !r.flag("autoEnd", true)).and_then(|_| date("endDate")); + let range = |level: PivotDateGroup| { + let days = if level == PivotDateGroup::Days { + num("groupInterval").filter(|d| d.is_finite() && *d >= 1.0).map_or(1, |d| d.min(f64::from(PivotDateRange::MAX_DAYS)) as u32) + } else { + 1 + }; + let r = PivotDateRange { start, end, days }; + GroupBy::Date(level, (start.is_some() || end.is_some() || days > 1).then_some(r)) + }; + let by = if let Some(d) = g.child("discretePr") { + GroupBy::Discrete(d.kids("x").take(MAX_ITEMS).map(|x| x.attr_u32("v").unwrap_or(u32::MAX)).collect()) + } else { + match rp.map(|r| r.attr("groupBy").unwrap_or("range")) { + Some("years") => range(PivotDateGroup::Years), + Some("quarters") => range(PivotDateGroup::Quarters), + Some("months") => range(PivotDateGroup::Months), + Some("days") => range(PivotDateGroup::Days), + Some("hours") => range(PivotDateGroup::Hours), + Some("minutes") => range(PivotDateGroup::Minutes), + Some("seconds") => range(PivotDateGroup::Seconds), + Some("range") => match (num("startNum"), num("endNum")) { + (Some(a), Some(b)) => { + PivotNumberGroup::checked(a, b, num("groupInterval").unwrap_or(1.0)).map_or(GroupBy::Unsupported, GroupBy::Range) + } + _ => GroupBy::Unsupported, + }, + _ => GroupBy::Unsupported, + } }; let gi = g.child("groupItems").map(|gi| gi.children.iter().map(|e| item_label_el(e, sys)).collect()).unwrap_or_default(); (base, by, gi) @@ -1151,7 +1557,22 @@ fn read_cache(cx: &mut Ctx<'_>, part: &str, sheet_name: &str) -> Option cx.warn("a PivotTable's cache has no fields; kept as plain cells"); return None; } - Some(CacheIn { source, fields }) + // Calculated items: a formula and the shared item of its field it fills. + let mut calc_items = Vec::new(); + for c in x.child("calculatedItems").into_iter().flat_map(|c| c.kids("calculatedItem")).take(MAX_FIELDS) { + let Some(formula) = c.attr("formula").map(str::trim).filter(|f| !f.is_empty()) else { continue }; + let reference = c.path(&["pivotArea", "references", "reference"]); + let fi = c.attr_u32("field").or_else(|| reference.and_then(|r| r.attr_u32("field"))).map(|f| f as usize); + let item = reference.and_then(|r| r.child("x")).and_then(|x| x.attr_u32("v")).map(|v| v as usize); + let (Some(f), Some(item)) = (fi.and_then(|i| fields.get(i)), item) else { + cx.warn("a PivotTable calculated item without its field or item was skipped"); + continue; + }; + let Some(name) = f.items.get(item).filter(|n| !n.trim().is_empty()) else { continue }; + let formula = if formula.starts_with('=') { formula.to_string() } else { format!("={formula}") }; + calc_items.push(PivotCalcItem { field: f.name.clone(), name: name.clone(), formula }); + } + Some(CacheIn { source, fields, calc_items }) } /// Reads the PivotTables related to a worksheet part. @@ -1184,7 +1605,7 @@ pub fn read_pivots(cx: &mut Ctx<'_>, caches: &mut CachesIn, sheet_part: &str, sh } }; let Some(cache) = cache else { continue }; - if let Some(pt) = read_table(cx, &x, &cache, caches.next_id) { + if let Some(pt) = read_table(cx, &x, &cache, caches.next_id, &caches.timeline_filters) { caches.next_id = caches.next_id.saturating_add(1); out.push(pt); } @@ -1197,34 +1618,73 @@ fn items_of(e: &El) -> Vec<&El> { e.child("items").map(|it| it.kids("item").filter(|i| i.attr("t").is_none_or(|t| t == "data")).collect()).unwrap_or_default() } -/// (source column name, date grouping) of a cache field used on an axis. -fn field_ref(cx: &mut Ctx<'_>, cache: &CacheIn, i: usize) -> Option<(String, PivotDateGroup)> { +/// A cache field as the model names it in an area. +struct FieldRef { + /// The source field, or the Group Selection field's own name. + name: String, + date: PivotDateGroup, + number: Option, + range: Option, + /// The Group Selection field this cache field is. + manual: Option, +} + +/// The groups of a Group Selection cache field: each group item that gathers base items other +/// than a single item of its own caption. +fn manual_groups(cache: &CacheIn, f: &FieldIn, base: usize, map: &[u32], items: &[String]) -> Option { + let base_f = cache.fields.get(base)?; + let base_labels = base_f.labels(); + let mut groups: Vec = Vec::new(); + for (k, caption) in items.iter().enumerate() { + let members: Vec = base_labels.iter().zip(map).filter(|(_, g)| **g as usize == k).map(|(l, _)| l.clone()).collect(); + let own = members.len() == 1 && members.first().is_some_and(|m| same(m, caption)); + if !members.is_empty() && !own { + groups.push(PivotItemGroup { name: caption.clone(), items: members }); + } + } + Some(PivotGroupField { name: f.name.clone(), base: base_f.name.clone(), groups }) +} + +/// How the model names cache field `i` used in an area. +fn field_ref(cx: &mut Ctx<'_>, cache: &CacheIn, i: usize) -> Option { let f = cache.fields.get(i)?; if f.formula.is_some() { cx.warn("calculated PivotTable fields are not supported"); return None; } + let plain = |name: String| FieldRef { name, date: PivotDateGroup::None, number: None, range: None, manual: None }; match &f.group { + Some((Some(base), GroupBy::Discrete(map), items)) if !f.database => { + let Some(manual) = manual_groups(cache, f, *base, map, items) else { + cx.warn(format!("PivotTable field \"{}\" (grouped items) has no base field and was skipped", f.name)); + return None; + }; + Some(FieldRef { manual: Some(manual), ..plain(f.name.clone()) }) + } + Some((_, GroupBy::Discrete(_), _)) => { + cx.warn(format!("PivotTable field \"{}\" (grouped items) is not supported", f.name)); + None + } Some((base, by, _)) => { - let by = match by { - Some(b) => *b, - None => { + let base_name = if f.database { f.name.clone() } else { base.and_then(|b| cache.fields.get(b))?.name.clone() }; + Some(match by { + GroupBy::Date(d, range) => FieldRef { date: *d, range: *range, ..plain(base_name) }, + GroupBy::Range(g) => FieldRef { number: Some(*g), ..plain(base_name) }, + _ => { cx.warn(format!("PivotTable grouping of \"{}\" is not supported; the field is ungrouped", f.name)); - PivotDateGroup::None + plain(base_name) } - }; - let base_name = if f.database { f.name.clone() } else { base.and_then(|b| cache.fields.get(b))?.name.clone() }; - Some((base_name, by)) + }) } None if !f.database => { cx.warn(format!("PivotTable field \"{}\" (grouped items) is not supported", f.name)); None } - None => Some((f.name.clone(), PivotDateGroup::None)), + None => Some(plain(f.name.clone())), } } -fn read_table(cx: &mut Ctx<'_>, x: &El, cache: &CacheIn, id: u32) -> Option { +fn read_table(cx: &mut Ctx<'_>, x: &El, cache: &CacheIn, id: u32, timeline_filters: &[(String, u32)]) -> Option { let mut pt = PivotTable { id, ..PivotTable::default() }; if let Some(n) = x.attr("name").filter(|n| !n.trim().is_empty()) { pt.name = n.to_string(); @@ -1266,52 +1726,72 @@ fn read_table(cx: &mut Ctx<'_>, x: &El, cache: &CacheIn, id: u32) -> Option = Vec::new(); + // Group Selection fields met in the areas. + let mut manual: Vec = Vec::new(); + let keep_manual = |manual: &mut Vec, r: &FieldRef| { + if let Some(m) = &r.manual + && !manual.iter().any(|x| same(&x.name, &m.name)) + { + manual.push(m.clone()); + } + }; // (fields, position of Σ Values (`-2`) among them) - let axis = |cx: &mut Ctx<'_>, list: &[i64], filters: &mut Vec| -> (Vec, Option) { - let mut v = Vec::new(); - let mut values_at = None; - for &i in list { - if i == -2 { - values_at = Some(v.len()); - continue; - } - if i < 0 { - continue; - } - let i = i as usize; - let Some((name, group)) = field_ref(cx, cache, i) else { continue }; - let labels = cache.fields.get(i).map(|f| f.labels().to_vec()).unwrap_or_default(); - let mut f = PivotField { source_col: name.clone(), date_group: group, sort: PivotSort::None, collapsed_items: vec![] }; - if let Some(e) = pf(i) { - f.sort = match e.attr("sortType") { - Some("ascending") => PivotSort::Asc, - Some("descending") => PivotSort::Desc, - _ => PivotSort::None, + let axis = + |cx: &mut Ctx<'_>, list: &[i64], filters: &mut Vec, manual: &mut Vec| -> (Vec, Option) { + let mut v = Vec::new(); + let mut values_at = None; + for &i in list { + if i == -2 { + values_at = Some(v.len()); + continue; + } + if i < 0 { + continue; + } + let i = i as usize; + let Some(r) = field_ref(cx, cache, i) else { continue }; + keep_manual(manual, &r); + let name = r.name.clone(); + let labels = cache.fields.get(i).map(|f| f.labels().to_vec()).unwrap_or_default(); + let mut f = PivotField { + source_col: name.clone(), + date_group: r.date, + number_group: r.number, + date_range: r.range, + sort: PivotSort::None, + ..PivotField::default() }; - let items = items_of(e); - let lab = |it: &El| it.attr_u32("x").and_then(|k| labels.get(k as usize)).cloned(); - f.collapsed_items = items.iter().filter(|it| !it.flag("sd", true)).filter_map(|it| lab(it)).collect(); - if items.iter().any(|it| it.flag("h", false)) { - let shown: Vec = items.iter().filter(|it| !it.flag("h", false)).filter_map(|it| lab(it)).collect(); - if !filters.iter().any(|ff| same(&ff.source_col, &name)) { - filters.push(PivotFilter { source_col: name.clone(), selected: Some(shown) }); + if let Some(e) = pf(i) { + f.sort = match e.attr("sortType") { + Some("ascending") => PivotSort::Asc, + Some("descending") => PivotSort::Desc, + // Date groups in their own (chronological) order are ascending. + _ if f.date_group != PivotDateGroup::None => PivotSort::Asc, + _ => PivotSort::None, + }; + let items = items_of(e); + let lab = |it: &El| it.attr_u32("x").and_then(|k| labels.get(k as usize)).cloned(); + f.collapsed_items = items.iter().filter(|it| !it.flag("sd", true)).filter_map(|it| lab(it)).collect(); + if items.iter().any(|it| it.flag("h", false)) { + let shown: Vec = items.iter().filter(|it| !it.flag("h", false)).filter_map(|it| lab(it)).collect(); + if !filters.iter().any(|ff| same(&ff.source_col, &name)) { + filters.push(PivotFilter { source_col: name.clone(), selected: Some(shown), rule: None }); + } } } + v.push(f); } - v.push(f); - } - (v, values_at) - }; + (v, values_at) + }; let mut page_filters = Vec::new(); for p in &pages { let Some(i) = p.attr_i64("fld").filter(|i| *i >= 0).map(|i| i as usize) else { continue }; - let Some((name, _)) = field_ref(cx, cache, i) else { continue }; + let Some(r) = field_ref(cx, cache, i) else { continue }; + keep_manual(&mut manual, &r); + let name = r.name; let labels = cache.fields.get(i).map(|f| f.labels().to_vec()).unwrap_or_default(); let selected = if let Some(k) = p.attr_u32("item") { labels.get(k as usize).map(|l| vec![l.clone()]) @@ -1327,11 +1807,11 @@ fn read_table(cx: &mut Ctx<'_>, x: &El, cache: &CacheIn, id: u32) -> Option, x: &El, cache: &CacheIn, id: u32) -> Option PivotShowAs::Rank, + (Some("rankAscending"), _) => PivotShowAs::RankAscending, + (Some("percentOfParentRow"), _) => PivotShowAs::PercentOfParentRowTotal, + (Some("percentOfParentCol"), _) => PivotShowAs::PercentOfParentColumnTotal, + (Some("percentOfParent"), _) => PivotShowAs::PercentOfParentTotal, + (Some("percentOfRunningTotal"), _) => PivotShowAs::PercentRunningTotal, (Some(other), _) => { cx.warn(format!("PivotTable show-values-as {other} is not supported")); PivotShowAs::Normal @@ -1371,18 +1856,38 @@ fn read_table(cx: &mut Ctx<'_>, x: &El, cache: &CacheIn, id: u32) -> Option PivotShowAs::PercentOfGrandTotal, (None, Some("percentOfCol")) => PivotShowAs::PercentOfColumnTotal, (None, Some("percentOfRow")) => PivotShowAs::PercentOfRowTotal, + (None, Some("percent")) => PivotShowAs::PercentOf, + (None, Some("difference")) => PivotShowAs::DifferenceFrom, + (None, Some("percentDiff")) => PivotShowAs::PercentDifferenceFrom, (None, Some("runTotal")) => PivotShowAs::RunningTotal, + (None, Some("index")) => PivotShowAs::Index, (None, Some(other)) => { cx.warn(format!("PivotTable show-values-as {other} is not supported")); PivotShowAs::Normal } }; + // Base field and item, where the transform uses them. + let (mut base_field, mut base_item) = (None, None); + if show_as.needs_base_field() + && let Some(bf) = d.attr_u32("baseField").map(|b| b as usize) + && let Some(r) = field_ref(cx, cache, bf) + { + base_field = Some(r.name); + if show_as.needs_base_item() { + base_item = match d.attr_u32("baseItem") { + Some(1_048_828) => Some(BASE_PREVIOUS.to_string()), + Some(1_048_829) => Some(BASE_NEXT.to_string()), + Some(k) => cache.fields.get(bf).and_then(|f| f.labels().get(k as usize).cloned()), + None => None, + }; + } + } let number_format = d .attr_u32("numFmtId") .filter(|id| *id != 0) .and_then(|id| crate::tables::builtin_format(id).map(str::to_string).or_else(|| cx.num_fmts.get(&id).cloned())); let name = d.attr("name").map(str::to_string).unwrap_or_else(|| format!("{} of {}", func_caption(func), f.name)); - pt.values.push(PivotValue { source_col: f.name.clone(), func, name, show_as, number_format }); + pt.values.push(PivotValue { source_col: f.name.clone(), func, name, show_as, number_format, base_field, base_item }); } } @@ -1398,6 +1903,31 @@ fn read_table(cx: &mut Ctx<'_>, x: &El, cache: &CacheIn, id: u32) -> Option = pt.values.iter().map(|v| v.name.clone()).collect(); + let measure = |i: u32| names.get(i as usize).cloned(); + for e in fs.kids("filter").take(MAX_FIELDS) { + // A timeline's date filter is its selection, read with the timeline. + if e.attr_u32("id").is_some_and(|id| timeline_filters.iter().any(|(p, f)| *f == id && same(p, &pt.name))) { + continue; + } + let Some(r) = e.attr_u32("fld").and_then(|i| field_ref(cx, cache, i as usize)) else { continue }; + let Some(rule) = read_filter(e, &measure) else { + cx.warn(format!("PivotTable \"{}\": filter type {} is not supported", pt.name, e.attr("type").unwrap_or(""))); + continue; + }; + keep_manual(&mut manual, &r); + let name = r.name; + // One filter per field: the rule replaces any hidden items. + pt.filters.retain(|f| !same(&f.source_col, &name)); + pt.filters.push(PivotFilter { source_col: name, selected: None, rule: Some(rule) }); + } + } + + pt.group_fields = manual; + pt.calculated_items = cache.calc_items.clone(); pt.anchor = loc.start; let above = if pages.is_empty() { 0 } else { pages.len() as u32 + 1 }; pt.last_range = Some(RangeRef::new(CellRef::new(loc.start.row.saturating_sub(above), loc.start.col), loc.end)); diff --git a/crates/xlsx/src/read.rs b/crates/xlsx/src/read.rs index 1a479081c..428619be8 100644 --- a/crates/xlsx/src/read.rs +++ b/crates/xlsx/src/read.rs @@ -30,6 +30,8 @@ pub struct Ctx<'a> { pub cell_pictures: Vec>>, /// x14 conditional-format ids of the current sheet's data bars, with the index of the rule in `cond_formats`. pub cf_ext_ids: Vec<(String, usize)>, + /// Anchors of the current sheet's timeline frames, by timeline name (lower case). + pub timeline_frames: std::collections::HashMap, } impl Ctx<'_> { @@ -82,6 +84,7 @@ pub fn read_xlsx(bytes: &[u8]) -> Result<(Workbook, ReadReport), IoError> { num_fmts: Default::default(), cell_pictures: vec![], cf_ext_ids: vec![], + timeline_frames: Default::default(), }; let root_rels = cx.pkg.rels("")?; let wb_part = find_rel(&root_rels, "officeDocument").map(|r| r.target.clone()).unwrap_or_else(|| "xl/workbook.xml".into()); @@ -196,9 +199,20 @@ pub fn read_xlsx(bytes: &[u8]) -> Result<(Workbook, ReadReport), IoError> { } let active_tab = wb_xml.path(&["bookViews", "workbookView"]).and_then(|v| v.attr_u32("activeTab")).unwrap_or(0) as usize; + // Timeline caches come first: their PivotTables' date filters are their selections. + let timeline_caches = crate::timeline::read_caches(&mut cx, &wb_rels)?; + // Sheets. `file_to_model[i]` maps the i-th to a model sheet index. let mut file_to_model: Vec> = vec![]; let mut pivot_caches = crate::pivot::CachesIn::new(&wb_xml, &wb_rels); + for c in &timeline_caches { + if let Some(id) = c.filter_id { + pivot_caches.timeline_filters.extend(c.pivots.iter().map(|(_, p)| (p.clone(), id))); + } + } + // Timelines read per model sheet, with their drawing frames; `sheetId` → model sheet. + let mut timelines: Vec<(Vec, std::collections::HashMap)> = vec![]; + let mut tab_ids: std::collections::HashMap = Default::default(); let sheet_els: Vec = wb_xml.child("sheets").map(|s| s.kids("sheet").cloned().collect()).unwrap_or_default(); for (i, se) in sheet_els.iter().enumerate() { let name = se.attr("name").unwrap_or("").to_string(); @@ -219,6 +233,7 @@ pub fn read_xlsx(bytes: &[u8]) -> Result<(Workbook, ReadReport), IoError> { continue; } let name = unique_name(&wb, &name, i); + cx.timeline_frames.clear(); let mut sheet = match crate::sheet_read::read_sheet(&mut cx, &rel.target, &name) { Ok(s) => s, Err(IoError::TooLarge(m)) => return Err(IoError::TooLarge(m)), @@ -238,9 +253,22 @@ pub fn read_xlsx(bytes: &[u8]) -> Result<(Workbook, ReadReport), IoError> { Some("veryHidden") => Visibility::VeryHidden, _ => Visibility::Visible, }; + let list = match crate::timeline::read_sheet(&mut cx, &rel.target) { + Ok(l) => l, + Err(IoError::TooLarge(m)) => return Err(IoError::TooLarge(m)), + Err(e) => { + cx.warn(format!("timelines on sheet \"{name}\" could not be read: {e}")); + vec![] + } + }; + timelines.push((list, std::mem::take(&mut cx.timeline_frames))); + if let Some(id) = se.attr_u32("sheetId") { + tab_ids.insert(id, wb.sheets.len()); + } file_to_model.push(Some(wb.sheets.len())); wb.sheets.push(Arc::new(sheet)); } + crate::timeline::attach(&mut cx, &mut wb, &timeline_caches, timelines, &tab_ids); if wb.sheets.is_empty() { cx.warn("the workbook has no worksheets; added an empty sheet"); wb.sheets.push(Arc::new(Sheet::new("Sheet1"))); @@ -264,6 +292,10 @@ pub fn read_xlsx(bytes: &[u8]) -> Result<(Workbook, ReadReport), IoError> { if lname == "_xlnm._filterdatabase" { continue; } + // A timeline cache's name: written again with the cache. + if timeline_caches.iter().any(|c| c.name.eq_ignore_ascii_case(name)) { + continue; + } if lname.starts_with("_xlchart.") { chart_names.push((lname, crate::fmla::from_file(text))); continue; diff --git a/crates/xlsx/src/sheet_write.rs b/crates/xlsx/src/sheet_write.rs index 78e1afaae..d5dc5817f 100644 --- a/crates/xlsx/src/sheet_write.rs +++ b/crates/xlsx/src/sheet_write.rs @@ -488,8 +488,8 @@ pub fn write_sheet(wb: &Workbook, si: usize, selected: bool, out: &mut Out) -> ( } } - // Drawing: pictures, charts, shapes. - if !sheet.images.is_empty() || !sheet.charts.is_empty() || !sheet.shapes.is_empty() { + // Drawing: pictures, charts, shapes, timelines. + if !sheet.images.is_empty() || !sheet.charts.is_empty() || !sheet.shapes.is_empty() || !sheet.timelines.is_empty() { out.drawings += 1; let dn = out.drawings; let mut drels = Rels::default(); @@ -530,6 +530,9 @@ pub fn write_sheet(wb: &Workbook, si: usize, selected: bool, out: &mut Out) -> ( for sh in &sheet.shapes { objs.push(Obj::Shape(sh)); } + for t in &sheet.timelines { + objs.push(Obj::Timeline(t)); + } out.part(&format!("xl/drawings/drawing{dn}.xml"), Some(CT_DRAWING), crate::drawing::write_drawing(sheet, &wb.theme, &objs).into_bytes()); if !drels.is_empty() { out.part(&format!("xl/drawings/_rels/drawing{dn}.xml.rels"), None, drels.xml().into_bytes()); @@ -584,7 +587,25 @@ pub fn write_sheet(wb: &Workbook, si: usize, selected: bool, out: &mut Out) -> ( .filter(|(_, cf)| !cf.ranges.is_empty() && matches!(cf.rule, CfRule::DataBar { gradient: false, .. })) .map(|(i, cf)| (cf, prio.get(i).copied().unwrap_or(1))) .collect(); - if !solid_bars.is_empty() || !sheet.sparklines.is_empty() { + // Timelines: the sheet's timeline part, listed in its own extension. + let mut timeline_ext = String::new(); + if !sheet.timelines.is_empty() { + out.timelines += 1; + let n = out.timelines; + let names = crate::timeline::cache_names(wb).into_iter().nth(si).unwrap_or_default(); + out.part( + &format!("xl/timelines/timeline{n}.xml"), + Some(crate::timeline::CT_TIMELINE), + crate::timeline::timelines_xml(sheet, &names, wb.date_system).into_bytes(), + ); + let rid = rels.add_uri(crate::timeline::REL_TIMELINE, &format!("../timelines/timeline{n}.xml")); + timeline_ext = format!( + "", + crate::timeline::EXT_SHEET, + crate::timeline::NS_X15 + ); + } + if !solid_bars.is_empty() || !sheet.sparklines.is_empty() || !timeline_ext.is_empty() { s.push_str(""); } if !solid_bars.is_empty() { @@ -622,7 +643,8 @@ pub fn write_sheet(wb: &Workbook, si: usize, selected: bool, out: &mut Out) -> ( } s.push_str(""); } - if !solid_bars.is_empty() || !sheet.sparklines.is_empty() { + s.push_str(&timeline_ext); + if !solid_bars.is_empty() || !sheet.sparklines.is_empty() || !timeline_ext.is_empty() { s.push_str(""); } s.push_str(""); diff --git a/crates/xlsx/src/tests/pivot_tests.rs b/crates/xlsx/src/tests/pivot_tests.rs index 99a15d6e7..b3b985357 100644 --- a/crates/xlsx/src/tests/pivot_tests.rs +++ b/crates/xlsx/src/tests/pivot_tests.rs @@ -104,7 +104,7 @@ fn workbook() -> Workbook { ..PivotValue::default() }, ], - filters: vec![PivotFilter { source_col: "Channel".into(), selected: Some(vec!["Online".into()]) }], + filters: vec![PivotFilter { source_col: "Channel".into(), selected: Some(vec!["Online".into()]), rule: None }], layout: PivotLayout::Tabular, grand_totals_rows: false, grand_totals_cols: true, @@ -116,6 +116,9 @@ fn workbook() -> Workbook { calculated_fields: vec![], values_on_rows: false, values_position: None, + group_fields: vec![], + calculated_items: vec![], + report: Default::default(), }; // Pivot 2: table source, date grouping on two levels, show-values-as, multi-select page // filter and a hidden-items filter on a column field. @@ -125,8 +128,15 @@ fn workbook() -> Workbook { source: "SalesTbl".into(), anchor: at("K4"), rows: vec![ - PivotField { source_col: "Date".into(), date_group: PivotDateGroup::Years, sort: PivotSort::None, collapsed_items: vec!["2024".into()] }, - PivotField { source_col: "Date".into(), date_group: PivotDateGroup::Months, sort: PivotSort::Asc, collapsed_items: vec![] }, + PivotField { + source_col: "Date".into(), + date_group: PivotDateGroup::Years, + // Date groups keep their chronological order: ascending (no `sortType`). + sort: PivotSort::Asc, + collapsed_items: vec!["2024".into()], + ..PivotField::default() + }, + PivotField { source_col: "Date".into(), date_group: PivotDateGroup::Months, sort: PivotSort::Asc, ..PivotField::default() }, ], columns: vec![PivotField { sort: PivotSort::Desc, ..field("Product") }], values: vec![ @@ -135,34 +145,42 @@ fn workbook() -> Workbook { func: PivotFunc::Sum, name: "Share".into(), show_as: PivotShowAs::PercentOfGrandTotal, - number_format: None, + ..PivotValue::default() + }, + PivotValue { + source_col: "Sales".into(), + func: PivotFunc::Max, + name: "Rank".into(), + show_as: PivotShowAs::Rank, + base_field: Some("Date".into()), + ..PivotValue::default() }, - PivotValue { source_col: "Sales".into(), func: PivotFunc::Max, name: "Rank".into(), show_as: PivotShowAs::Rank, number_format: None }, PivotValue { source_col: "Sales".into(), func: PivotFunc::Average, name: "Running".into(), show_as: PivotShowAs::RunningTotal, - number_format: None, + base_field: Some("Date".into()), + ..PivotValue::default() }, PivotValue { source_col: "Units".into(), func: PivotFunc::StdDevP, name: "Spread".into(), show_as: PivotShowAs::PercentOfColumnTotal, - number_format: None, + ..PivotValue::default() }, PivotValue { source_col: "Units".into(), func: PivotFunc::VarP, name: "RowPct".into(), show_as: PivotShowAs::PercentOfRowTotal, - number_format: None, + ..PivotValue::default() }, ], filters: vec![ - PivotFilter { source_col: "Region".into(), selected: Some(vec!["East".into(), "North".into()]) }, - PivotFilter { source_col: "Product".into(), selected: Some(vec!["A".into(), "B".into()]) }, + PivotFilter { source_col: "Region".into(), selected: Some(vec!["East".into(), "North".into()]), rule: None }, + PivotFilter { source_col: "Product".into(), selected: Some(vec!["A".into(), "B".into()]), rule: None }, ], layout: PivotLayout::Outline, grand_totals_rows: true, @@ -175,6 +193,9 @@ fn workbook() -> Workbook { calculated_fields: vec![], values_on_rows: false, values_position: None, + group_fields: vec![], + calculated_items: vec![], + report: Default::default(), }; // Pivot 3: defaults, defined-name source, compact. let p3 = PivotTable { @@ -184,7 +205,7 @@ fn workbook() -> Workbook { anchor: at("AB1"), rows: vec![field("Product")], values: vec![PivotValue { source_col: "Units".into(), func: PivotFunc::Product, name: "Product of Units".into(), ..PivotValue::default() }], - filters: vec![PivotFilter { source_col: "Channel".into(), selected: None }], + filters: vec![PivotFilter { source_col: "Channel".into(), selected: None, rule: None }], last_range: Some(rr("AB1:AC6")), ..PivotTable::default() }; @@ -417,7 +438,7 @@ fn with_pivot(table: &str, cache: Option<&str>) -> Vec { minimal("", &extra, "", "") } -const GOOD_CACHE: &str = r#""#; +const GOOD_CACHE: &str = r#""#; #[test] fn hostile_pivot_parts_warn_not_panic() { @@ -456,8 +477,12 @@ fn hostile_pivot_parts_warn_not_panic() { assert_eq!(pt.rows[0].sort, PivotSort::Desc); assert!(pt.values.iter().any(|v| v.func == PivotFunc::Sum && v.source_col == "V")); assert!(report.warnings.iter().any(|w| w.contains("bogus")), "{:?}", report.warnings); + // A number range with no width is not a grouping the model can hold. assert!(report.warnings.iter().any(|w| w.contains("grouping")), "{:?}", report.warnings); - assert!(report.warnings.iter().any(|w| w.contains("index")), "{:?}", report.warnings); + assert!(pt.columns.iter().all(|f| f.number_group.is_none()), "{:?}", pt.columns); + // Index and % of Parent Total are read now. + assert!(pt.values.iter().any(|v| v.show_as == PivotShowAs::Index), "{:?}", pt.values); + assert!(pt.values.iter().any(|v| v.show_as == PivotShowAs::PercentOfParentTotal), "{:?}", pt.values); } #[test] @@ -518,6 +543,210 @@ fn calculated_fields_round_trip() { } } +#[test] +fn number_range_grouping_round_trips() { + let mut wb = workbook(); + let ng = PivotNumberGroup { start: 0.0, end: 80.0, by: 25.0 }; + { + let p = wb.sheet_mut(1).unwrap().pivots.get_mut(0).unwrap(); + p.rows.push(PivotField { number_group: Some(ng), ..field("Sales") }); + } + let bytes = write_xlsx(&wb).unwrap(); + let parts = unzip(&bytes); + let c1 = String::from_utf8(parts.get("xl/pivotCache/pivotCacheDefinition1.xml").unwrap().clone()).unwrap(); + assert!(c1.contains(""), "{c1}"); + assert!( + c1.contains( + "" + ), + "{c1}" + ); + // The grouped field's records stay numbers, not dates. + assert!(c1.contains(""), "{c1}"); + let (back, report) = read_xlsx(&bytes).unwrap(); + assert!(report.warnings.iter().all(|w| !w.to_lowercase().contains("pivot")), "{:?}", report.warnings); + let orig = &wb.sheet(1).unwrap().pivots; + let read = &back.sheet(1).unwrap().pivots; + for (a, b) in orig.iter().zip(read.iter()) { + assert_eq!(a, b, "pivot {} differs", a.name); + } +} + +#[test] +fn label_value_and_top10_filters_round_trip() { + let mut wb = workbook(); + { + let sh = wb.sheet_mut(1).unwrap(); + // Pivot 1: a label filter on Region (rows) and a value filter on Product (columns). + let p1 = sh.pivots.get_mut(0).unwrap(); + p1.filters.push(PivotFilter { + source_col: "Region".into(), + selected: None, + rule: Some(PivotRule::Label { op: PivotCompare::NotBetween, value: "A & B".into(), value2: "F".into() }), + }); + p1.filters.push(PivotFilter { + source_col: "Product".into(), + selected: None, + rule: Some(PivotRule::Value { op: PivotCompare::Between, field: "Count of Units".into(), value: 1.5, value2: 30.0 }), + }); + // Pivot 3: the bottom 2 items by percent of Product of Units. + let p3 = sh.pivots.get_mut(2).unwrap(); + p3.filters.push(PivotFilter { + source_col: "Product".into(), + selected: None, + rule: Some(PivotRule::Top { top: false, count: 2.0, by: PivotTopBy::Percent, field: "Product of Units".into() }), + }); + } + let bytes = write_xlsx(&wb).unwrap(); + let parts = unzip(&bytes); + let t1 = String::from_utf8(parts.get("xl/pivotTables/pivotTable1.xml").unwrap().clone()).unwrap(); + assert!( + t1.contains( + "" + ), + "{t1}" + ); + assert!(t1.contains("type=\"valueBetween\" evalOrder=\"-1\" id=\"2\" iMeasureFld=\"1\""), "{t1}"); + assert!(t1.contains(""), "{t1}"); + // `` follows the style info, as the schema orders them. + assert!(t1.find(""), "{t3}"); + let (back, report) = read_xlsx(&bytes).unwrap(); + assert!(report.warnings.iter().all(|w| !w.to_lowercase().contains("pivot")), "{:?}", report.warnings); + let orig = &wb.sheet(1).unwrap().pivots; + let read = &back.sheet(1).unwrap().pivots; + for (a, b) in orig.iter().zip(read.iter()) { + assert_eq!(a, b, "pivot {} differs", a.name); + } +} + +/// Synthetic equivalent of PR #1207's field 14 in pivotTable2.xml and pivotTable3.xml. +fn dormant_filter_workbook() -> Workbook { + let mut wb = Workbook::new(); + let sh = wb.sheet_mut(0).unwrap(); + sh.name = "Data".into(); + for c in 0..15 { + let name = match c { + 0 => "Region".into(), + 1 => "Sales".into(), + 14 => "Dormant".into(), + _ => format!("Unused{c}"), + }; + sh.cells.set(CellRef::new(0, c), Cell::value(Value::text(name))); + } + for r in 1..=4 { + sh.cells.set(CellRef::new(r, 0), Cell::value(Value::text(if r <= 2 { "East" } else { "West" }))); + sh.cells.set(CellRef::new(r, 1), Cell::value(Value::number(f64::from(r * 10)))); + sh.cells.set(CellRef::new(r, 14), Cell::value(Value::number(f64::from(r % 2)))); + } + for id in 1..=3 { + sh.pivots.push(PivotTable { + id, + name: format!("PivotTable{id}"), + source: "Data!$A$1:$O$5".into(), + anchor: CellRef::new(2, 16 + id * 3), + rows: vec![field("Region")], + values: vec![PivotValue { source_col: "Sales".into(), name: "Total".into(), ..PivotValue::default() }], + ..PivotTable::default() + }); + } + wb +} + +fn dormant_label_filter() -> PivotFilter { + PivotFilter { + source_col: "Dormant".into(), + selected: None, + rule: Some(PivotRule::Label { op: PivotCompare::Equals, value: "1".into(), value2: String::new() }), + } +} + +#[test] +fn reads_dormant_field14_filters_in_multiple_pivots() { + let mut parts = unzip(&write_xlsx(&dormant_filter_workbook()).unwrap()); + // Inject Excel's reported XML independently of our filter writer. + for id in [2, 3] { + let path = format!("xl/pivotTables/pivotTable{id}.xml"); + let table = String::from_utf8(parts.remove(&path).unwrap()).unwrap(); + let table = table.replace( + "", + r#""#, + ); + parts.insert(path, table.into_bytes()); + } + let list: Vec<_> = parts.iter().map(|(n, d)| (n.as_str(), d.as_slice())).collect(); + let (wb, report) = read_xlsx(&super::make_zip(&list)).unwrap(); + assert!(report.warnings.is_empty(), "{:?}", report.warnings); + for pt in wb.sheet(0).unwrap().pivots.iter().skip(1) { + assert_eq!(pt.filters, vec![dormant_label_filter()], "{} lost its dormant filter", pt.name); + assert_eq!(pt.rows, vec![field("Region")]); + } + let saved = write_xlsx(&wb).unwrap(); + let parts = unzip(&saved); + for id in [2, 3] { + let table = xml::parse(parts.get(&format!("xl/pivotTables/pivotTable{id}.xml")).unwrap()).unwrap(); + assert_eq!(table.child("filters").unwrap().child("filter").unwrap().attr("fld"), Some("14")); + assert!(table.child("pivotFields").unwrap().kids("pivotField").nth(14).unwrap().attr("axis").is_none()); + assert!(table.child("pageFields").is_none()); + } + let (back, report) = read_xlsx(&saved).unwrap(); + assert!(report.warnings.is_empty(), "{:?}", report.warnings); + assert_eq!(wb.sheet(0).unwrap().pivots, back.sheet(0).unwrap().pivots); +} + +#[test] +fn writes_dormant_label_value_and_top10_filters_without_page_fields() { + let rules = [ + dormant_label_filter().rule.unwrap(), + PivotRule::Value { op: PivotCompare::Between, field: "Total".into(), value: 5.0, value2: 50.0 }, + PivotRule::Top { top: false, count: 2.0, by: PivotTopBy::Items, field: "Total".into() }, + ]; + for rule in rules { + let mut wb = dormant_filter_workbook(); + let filter = PivotFilter { source_col: "Dormant".into(), selected: None, rule: Some(rule) }; + wb.sheet_mut(0).unwrap().pivots[0].filters.push(filter.clone()); + for _ in 0..2 { + let bytes = write_xlsx(&wb).unwrap(); + let parts = unzip(&bytes); + let table = xml::parse(parts.get("xl/pivotTables/pivotTable1.xml").unwrap()).unwrap(); + assert!(table.child("pageFields").is_none(), "dormant rule became a report filter: {filter:?}"); + assert_eq!(table.child("filters").unwrap().child("filter").unwrap().attr("fld"), Some("14")); + assert!(table.child("pivotFields").unwrap().kids("pivotField").nth(14).unwrap().attr("axis").is_none()); + let (back, report) = read_xlsx(&bytes).unwrap(); + assert!(report.warnings.is_empty(), "{:?}", report.warnings); + assert_eq!(back.sheet(0).unwrap().pivots[0].filters, vec![filter.clone()]); + wb = back; + } + } +} + +#[test] +fn unsupported_and_hostile_pivot_filters() { + // Unsupported dates, missing value fields/numbers and invalid cache fields are skipped. + // A rule on a field outside the axes is preserved; a Top 10 without its `top10` element + // takes Excel's defaults. + let table = r#""#; + let (wb, report) = read_xlsx(&with_pivot(table, Some(GOOD_CACHE))).unwrap(); + let pt = &wb.sheet(0).unwrap().pivots[0]; + assert!(report.warnings.iter().any(|w| w.contains("thisMonth")), "{:?}", report.warnings); + assert_eq!( + pt.filters, + vec![ + PivotFilter { + source_col: "V".into(), + selected: None, + rule: Some(PivotRule::Label { op: PivotCompare::Equals, value: "x".into(), value2: String::new() }), + }, + PivotFilter { + source_col: "K".into(), + selected: None, + rule: Some(PivotRule::Top { top: true, count: 10.0, by: PivotTopBy::Items, field: "Total".into() }), + }, + ] + ); +} + /// The `x` attributes of an axis list (`rowFields`/`colFields`) in a pivotTable part. fn axis_xs(table: &str, tag: &str) -> Vec { let Some(start) = table.find(&format!("<{tag} ")) else { return vec![] }; @@ -590,3 +819,226 @@ fn data_on_rows_with_one_value_field_reads_without_warning() { let (back, _) = read_xlsx(&write_xlsx(&wb).unwrap()).unwrap(); assert!(back.sheet(0).unwrap().pivots[0].values_on_rows); } + +#[test] +fn group_selection_fields_round_trip_as_discrete_groups() { + let mut wb = workbook(); + { + let p = wb.sheet_mut(1).unwrap().pivots.get_mut(0).unwrap(); + p.group_fields.push(PivotGroupField { + name: "Region2".into(), + base: "Region".into(), + groups: vec![PivotItemGroup { name: "Group1".into(), items: vec!["East".into(), "North".into()] }], + }); + p.rows.insert(0, field("Region2")); + p.filters.push(PivotFilter { source_col: "Region2".into(), selected: Some(vec!["Group1".into()]), rule: None }); + } + let bytes = write_xlsx(&wb).unwrap(); + let parts = unzip(&bytes); + let c1 = String::from_utf8(parts.get("xl/pivotCache/pivotCacheDefinition1.xml").unwrap().clone()).unwrap(); + // The base field names its grouped field; the grouped field maps each base item to a group item. + assert!(c1.contains(""), "{c1}"); + assert!( + c1.contains( + "" + ), + "{c1}" + ); + let t1 = String::from_utf8(parts.get("xl/pivotTables/pivotTable1.xml").unwrap().clone()).unwrap(); + assert!(t1.contains(""), "{t1}"); + let (back, report) = read_xlsx(&bytes).unwrap(); + assert!(report.warnings.iter().all(|w| !w.to_lowercase().contains("pivot")), "{:?}", report.warnings); + assert_eq!(back.sheet(1).unwrap().pivots[0], wb.sheet(1).unwrap().pivots[0]); +} + +#[test] +fn time_levels_bounds_and_day_spans_round_trip() { + let mut wb = workbook(); + { + let p = wb.sheet_mut(1).unwrap().pivots.get_mut(0).unwrap(); + let range = Some(PivotDateRange { start: Some(date(2024, 1, 1)), end: Some(date(2024, 12, 31)), days: 1 }); + p.columns = vec![ + PivotField { date_group: PivotDateGroup::Years, date_range: range, ..field("Date") }, + PivotField { date_group: PivotDateGroup::Months, date_range: range, ..field("Date") }, + ]; + let p3 = wb.sheet_mut(1).unwrap().pivots.get_mut(2).unwrap(); + p3.rows = vec![PivotField { + date_group: PivotDateGroup::Days, + date_range: Some(PivotDateRange { start: None, end: None, days: 30 }), + ..field("Date") + }]; + } + let bytes = write_xlsx(&wb).unwrap(); + let parts = unzip(&bytes); + let c1 = String::from_utf8(parts.get("xl/pivotCache/pivotCacheDefinition1.xml").unwrap().clone()).unwrap(); + assert!( + c1.contains(""), + "{c1}" + ); + assert!(c1.contains(""), "{c1}"); + // Excel sorts date group captions as text under `sortType`: ascending date groups go without it. + let t1 = String::from_utf8(parts.get("xl/pivotTables/pivotTable1.xml").unwrap().clone()).unwrap(); + assert!(t1.contains("numFmtId=\"14\" showAll=\"0\"") && !t1.contains("numFmtId=\"14\" showAll=\"0\" sortType"), "{t1}"); + let c3 = String::from_utf8(parts.get("xl/pivotCache/pivotCacheDefinition3.xml").unwrap().clone()).unwrap(); + assert!(c3.contains("groupBy=\"days\" startDate=\"2024-01-05T00:00:00\" endDate=\"2026-01-01T00:00:00\" groupInterval=\"30\"/>"), "{c3}"); + assert!(c3.contains(""), "{c3}"); + let (back, report) = read_xlsx(&bytes).unwrap(); + assert!(report.warnings.iter().all(|w| !w.to_lowercase().contains("pivot")), "{:?}", report.warnings); + for (a, b) in wb.sheet(1).unwrap().pivots.iter().zip(back.sheet(1).unwrap().pivots.iter()) { + assert_eq!(a, b, "pivot {} differs", a.name); + } + // Hours, minutes and seconds read and write their own `groupBy`. + for (level, name) in [(PivotDateGroup::Hours, "hours"), (PivotDateGroup::Minutes, "minutes"), (PivotDateGroup::Seconds, "seconds")] { + wb.sheet_mut(1).unwrap().pivots[2].rows = vec![PivotField { date_group: level, ..field("Date") }]; + let bytes = write_xlsx(&wb).unwrap(); + let c3 = String::from_utf8(unzip(&bytes).get("xl/pivotCache/pivotCacheDefinition3.xml").unwrap().clone()).unwrap(); + assert!(c3.contains(&format!("groupBy=\"{name}\"")), "{c3}"); + let (back, _) = read_xlsx(&bytes).unwrap(); + assert_eq!(back.sheet(1).unwrap().pivots[2].rows[0].date_group, level); + } +} + +#[test] +fn calculated_items_load_and_round_trip() { + let mut wb = workbook(); + { + let p = wb.sheet_mut(1).unwrap().pivots.get_mut(0).unwrap(); + p.calculated_items = vec![ + PivotCalcItem { field: "Region".into(), name: "East+West".into(), formula: "=East+'West'".into() }, + PivotCalcItem { field: "Product".into(), name: "AB".into(), formula: "=Product[A]*2".into() }, + ]; + } + let bytes = write_xlsx(&wb).unwrap(); + let parts = unzip(&bytes); + let c1 = String::from_utf8(parts.get("xl/pivotCache/pivotCacheDefinition1.xml").unwrap().clone()).unwrap(); + // The items are shared items of their fields, after the data items; the formulas live in the cache. + assert!(c1.contains(""), "{c1}"); + assert!( + c1.contains( + "" + ) || c1.contains("formula=\"East+'West'\""), + "{c1}" + ); + assert!(c1.find("").unwrap() < c1.find(""#; + let table = r#""#; + let (wb, report) = read_xlsx(&with_pivot(table, Some(cache))).unwrap(); + let pt = &wb.sheet(0).unwrap().pivots[0]; + assert_eq!(pt.calculated_items, vec![PivotCalcItem { field: "K".into(), name: "xy".into(), formula: "=x+y".into() }]); + assert_eq!(pt.group_fields.len(), 1); + assert_eq!(pt.group_fields[0].groups, vec![PivotItemGroup { name: "Group1".into(), items: vec!["x".into(), "y".into()] }]); + assert_eq!(pt.rows.iter().map(|f| f.source_col.as_str()).collect::>(), ["K2", "K"]); + assert!(report.warnings.iter().any(|w| w.contains("K3")), "{:?}", report.warnings); + assert!(write_xlsx(&wb).is_ok()); +} + +#[test] +fn timelines_round_trip_with_their_caches_and_pivot_filter() { + let mut wb = workbook(); + let tl = Timeline { + id: 50, + name: "Date".into(), + caption: "Order date".into(), + field: "Date".into(), + pivots: vec![1], + anchor: Anchor { cell: at("H2"), dx: 0.0, dy: 0.0, width: 262.5, height: 108.0, mode: AnchorMode::MoveOnly }, + level: TimelineLevel::Quarters, + selection: Some((date(2024, 1, 1), date(2024, 6, 30))), + selection_level: TimelineLevel::Months, + scroll: Some(date(2024, 1, 1)), + style: "TimeSlicerStyleDark2".into(), + show_scrollbar: false, + ..Timeline::default() + }; + wb.sheet_mut(1).unwrap().timelines.push(tl.clone()); + let bytes = write_xlsx(&wb).unwrap(); + let parts = unzip(&bytes); + let text = |p: &str| String::from_utf8(parts.get(p).unwrap_or_else(|| panic!("missing {p}")).clone()).unwrap(); + let t = text("xl/timelines/timeline1.xml"); + assert!( + t.contains(""), + "{t}" + ); + let c = text("xl/timelineCaches/timelineCache1.xml"); + assert!( + c.contains("name=\"NativeTimeline_Date\" sourceName=\"Date\">"), + "{c}" + ); + assert!( + c.contains("pivotCacheId=\"1\" filterType=\"dateBetween\" filterId=\"1000\" filterPivotName=\"Sales by Region\" filterTabId=\"2\">"), + "{c}" + ); + assert!(c.contains(""), "{c}"); + let w = text("xl/workbook.xml"); + assert!(w.contains("#N/A"), "{w}"); + assert!(w.contains("")); + let ct = text("[Content_Types].xml"); + assert!(ct.contains("application/vnd.ms-excel.timeline+xml") && ct.contains("application/vnd.ms-excel.timelineCache+xml"), "{ct}"); + let pt = text("xl/pivotTables/pivotTable1.xml"); + assert!(pt.contains(""), "{pt}"); + let pc = text("xl/pivotCache/pivotCacheDefinition1.xml"); + assert!(pc.contains(""), "{pc}"); + // The timeline's field is a date field in the cache. + assert!( + pc.contains("#N/A"#; + let wb_rels = r#""#; + let sheet_rels = r#""#; + // A cache naming a missing PivotTable with a reversed selection, a cache without a field, a + // timeline naming a missing cache, one without a name, one with an odd level. + let cache1 = r#""#; + let cache2 = r#""#; + let timeline = r#""#; + let parts = [ + ("xl/worksheets/_rels/sheet1.xml.rels", sheet_rels), + ("xl/timelineCaches/timelineCache1.xml", cache1), + ("xl/timelineCaches/timelineCache2.xml", cache2), + ("xl/timelines/timeline1.xml", timeline), + ]; + let bytes = minimal("", &parts, wb_rels, wb_extra); + let (wb, report) = read_xlsx(&bytes).unwrap(); + let sh = wb.sheet(0).unwrap(); + assert_eq!(sh.timelines.len(), 1); + let t = &sh.timelines[0]; + assert_eq!((t.field.as_str(), t.level, t.selection, t.pivots.len(), t.scroll), ("X", TimelineLevel::Months, None, 0, None)); + assert!(report.warnings.iter().any(|w| w.contains("\"Y\"")), "{:?}", report.warnings); + assert!(report.warnings.iter().any(|w| w.contains("without its name")), "{:?}", report.warnings); + assert!(wb.names.is_empty(), "the cache's name isn't a defined name"); + assert!(write_xlsx(&wb).is_ok()); +} diff --git a/crates/xlsx/src/timeline.rs b/crates/xlsx/src/timeline.rs new file mode 100644 index 000000000..439f6f5d8 --- /dev/null +++ b/crates/xlsx/src/timeline.rs @@ -0,0 +1,348 @@ +//! Timelines ([MS-XLSX] 2.6.x, Excel 2013 and later): a timeline part per sheet +//! (`xl/timelines/timelineN.xml`, `x15:timeline` elements), a timeline cache per timeline +//! (`xl/timelineCaches/timelineCacheN.xml`: the date field, the PivotTables it filters, the +//! selected period and the data bounds), the sheet's `x15:timelineRefs` and the workbook's +//! `x15:timelineCacheRefs` extensions, a drawing frame (`tsle:timeslicer`, with a fallback shape +//! for older readers) and a `NativeTimeline_…` defined name per cache. The filtered PivotTable +//! carries the selection as a `dateBetween` filter that the cache's state names. +//! +//! GridCraft keeps one cache per timeline. A cache can only list PivotTables that share one +//! pivot cache, and GridCraft writes one pivot cache per PivotTable, so a timeline connected to +//! several PivotTables is saved connected to the first of them. + +use std::collections::HashMap; +use std::fmt::Write as _; + +use gridcraft_core::date::{datetime_from_serial, serial_from_ymd}; +use gridcraft_core::{CellRef, DateSystem}; +use gridcraft_model::{Anchor, AnchorMode, Sheet, Timeline, TimelineLevel, Workbook}; + +use crate::read::Ctx; +use crate::xml::{El, esc_attr}; + +pub const NS_X15: &str = "http://schemas.microsoft.com/office/spreadsheetml/2010/11/main"; +pub const REL_TIMELINE: &str = "http://schemas.microsoft.com/office/2011/relationships/timeline"; +pub const REL_TIMELINE_CACHE: &str = "http://schemas.microsoft.com/office/2011/relationships/timelineCache"; +pub const CT_TIMELINE: &str = "application/vnd.ms-excel.timeline+xml"; +pub const CT_TIMELINE_CACHE: &str = "application/vnd.ms-excel.timelineCache+xml"; +/// The sheet extension listing its timeline parts. +pub const EXT_SHEET: &str = "{7E03D99C-DC04-49d9-9315-930204A7B6E9}"; +/// The workbook extension listing the timeline caches. +pub const EXT_WORKBOOK: &str = "{D0CA8CA8-9F24-4464-BF8E-62219DCF47F9}"; +/// `graphicData` URI of a timeline's drawing frame. +pub const GRAPHIC_URI: &str = "http://schemas.microsoft.com/office/drawing/2012/timeslicer"; +/// The `x15:pivotFilter` extension of a timeline's PivotTable filter. +const EXT_PIVOT_FILTER: &str = "{0605FD5F-26C8-4aeb-8148-2DB25E43C511}"; +/// Filter ids of timeline filters in a PivotTable start here (label, value and Top 10 filters +/// number from 1). +pub const FILTER_ID_BASE: u32 = 1000; +/// Most timelines read from one part or one workbook. +const MAX_TIMELINES: usize = 1_000; + +/// The timeline caches' names, per sheet and timeline: `NativeTimeline_Date`, `NativeTimeline_Date1`… +pub fn cache_names(wb: &Workbook) -> Vec> { + let mut taken: Vec = wb.names.iter().map(|n| n.name.to_lowercase()).collect(); + wb.sheets + .iter() + .map(|sh| { + sh.timelines + .iter() + .map(|t| { + let field: String = t.field.chars().map(|c| if c.is_alphanumeric() || c == '_' { c } else { '_' }).collect(); + let base = format!("NativeTimeline_{field}"); + let name = std::iter::once(base.clone()) + .chain((1..10_000).map(|k| format!("{base}{k}"))) + .find(|n| !taken.contains(&n.to_lowercase())) + .unwrap_or(base); + taken.push(name.to_lowercase()); + name + }) + .collect() + }) + .collect() +} + +/// The timelines with a selection that filter PivotTable `pivot`, in workbook order: their +/// filters in that PivotTable get ids from [`FILTER_ID_BASE`] in this order. +pub fn filters_of(wb: &Workbook, pivot: u32) -> Vec<&Timeline> { + wb.sheets.iter().flat_map(|s| s.timelines.iter()).filter(|t| t.selection.is_some() && t.pivots.first() == Some(&pivot)).collect() +} + +fn level_num(l: TimelineLevel) -> u32 { + match l { + TimelineLevel::Years => 0, + TimelineLevel::Quarters => 1, + TimelineLevel::Months => 2, + TimelineLevel::Days => 3, + } +} + +fn level_from(n: Option) -> TimelineLevel { + match n { + Some(0) => TimelineLevel::Years, + Some(1) => TimelineLevel::Quarters, + Some(3) => TimelineLevel::Days, + _ => TimelineLevel::Months, + } +} + +/// `2026-03-01T00:00:00` for a date serial. +fn iso(sys: DateSystem, serial: f64) -> Option { + let d = datetime_from_serial(sys, serial)?; + Some(format!("{:04}-{:02}-{:02}T{:02}:{:02}:{:02}", d.year, d.month, d.day, d.hour, d.minute, d.second)) +} + +// ---------------------------------------------------------------- writing + +/// A sheet's timeline part. +pub fn timelines_xml(sheet: &Sheet, names: &[String], sys: DateSystem) -> String { + let mut s = format!("{}", crate::write::XML_DECL); + for (t, cache) in sheet.timelines.iter().zip(names) { + let _ = write!( + s, + ""); + } + s.push_str(""); + s +} + +/// The PivotTable a timeline cache is written for: its sheet's `sheetId`, name, pivot cache id +/// and the id of the timeline's filter in it. +pub struct CachePivot { + pub tab_id: usize, + pub name: String, + pub cache_id: u32, + pub filter_id: u32, +} + +/// A timeline cache part. `bounds` are the first and last date of the data. +pub fn cache_xml(t: &Timeline, name: &str, pivot: Option<&CachePivot>, bounds: Option<(f64, f64)>, sys: DateSystem) -> String { + let mut s = format!( + "{}", + crate::write::XML_DECL, + esc_attr(name), + esc_attr(&t.field) + ); + if let Some(p) = pivot { + let _ = write!(s, "", p.tab_id, esc_attr(&p.name)); + } + s.push_str(" { + let _ = write!( + s, + " filterType=\"dateBetween\" filterId=\"{}\" filterPivotName=\"{}\" filterTabId=\"{}\"", + p.filter_id, + esc_attr(&p.name), + p.tab_id + ); + } + _ => s.push_str(" filterType=\"unknown\""), + } + s.push('>'); + if let Some((a, b)) = &selection { + let _ = write!(s, ""); + } + // The bounds run over whole years around the data. + let year = |d: f64| datetime_from_serial(sys, d).map(|x| i64::from(x.year)); + if let Some((lo, hi)) = bounds + && let (Some(y0), Some(y1)) = (year(lo), year(hi)) + && let (Some(a), Some(b)) = + (serial_from_ymd(sys, y0, 1, 1).and_then(|d| iso(sys, d)), serial_from_ymd(sys, y1 + 1, 1, 1).and_then(|d| iso(sys, d))) + { + let _ = write!(s, ""); + } + s.push_str(""); + s +} + +/// A PivotTable's filter for a timeline's selection (`dateBetween` on cache field `fld`). +pub fn filter_xml(fld: usize, id: u32, t: &Timeline) -> String { + let Some((a, b)) = t.selection else { return String::new() }; + format!( + "", + esc_attr(&t.field), + crate::xml::num(a.floor()), + crate::xml::num(b.floor()) + ) +} + +/// A timeline's drawing frame, with a fallback shape for readers without timelines. +pub fn frame_xml(t: &Timeline, id: usize, xfrm: &str, a_xfrm: &str) -> String { + format!( + "{xfrm}{a_xfrm}A timeline filters a PivotTable by date. It needs Excel 2013 or later, or GridCraft.", + name = esc_attr(&t.name) + ) +} + +// ---------------------------------------------------------------- reading + +/// A timeline element as read from a sheet's timeline part. +pub struct TimelineIn { + pub name: String, + pub cache: String, + pub caption: String, + pub level: TimelineLevel, + pub selection_level: TimelineLevel, + pub scroll: Option, + pub style: String, + pub show: [bool; 4], +} + +/// A timeline cache as read. +pub struct CacheIn { + pub name: String, + pub field: String, + /// (sheetId, PivotTable name) of each PivotTable it filters. + pub pivots: Vec<(u32, String)>, + pub selection: Option<(f64, f64)>, + /// The id of its filter in each of its PivotTables. + pub filter_id: Option, +} + +/// Reads the workbook's timeline caches (`timelineCache` relationships). +pub fn read_caches(cx: &mut Ctx<'_>, wb_rels: &[crate::package::Rel]) -> Result, crate::IoError> { + let sys = cx.date_system; + let mut out = Vec::new(); + for rel in wb_rels.iter().filter(|r| r.kind == "timelineCache" && !r.external).take(MAX_TIMELINES) { + let Some(x) = cx.optional_xml(&rel.target)? else { continue }; + if x.name != "timelineCacheDefinition" { + cx.warn(format!("{} is not a timeline cache; its timeline was skipped", rel.target)); + continue; + } + let (Some(name), Some(field)) = (x.attr("name"), x.attr("sourceName")) else { + cx.warn("a timeline cache without its name or field was skipped"); + continue; + }; + let pivots = x + .child("pivotTables") + .map(|p| p.kids("pivotTable").take(MAX_TIMELINES).filter_map(|t| Some((t.attr_u32("tabId")?, t.attr("name")?.to_string()))).collect()) + .unwrap_or_default(); + let state = x.child("state"); + let date = |e: Option<&El>, k: &str| e.and_then(|e| e.attr(k)).and_then(|v| crate::sheet_read::parse_iso_datetime(v, sys)); + let sel = state.and_then(|s| s.child("selection")); + let selection = match (date(sel, "startDate"), date(sel, "endDate")) { + (Some(a), Some(b)) if a <= b => Some((a.floor(), b.floor())), + (Some(a), None) => Some((a.floor(), a.floor())), + _ => None, + }; + let filter_id = state.and_then(|s| s.attr_u32("filterId")); + out.push(CacheIn { name: name.to_string(), field: field.to_string(), pivots, selection, filter_id }); + } + Ok(out) +} + +/// Reads a sheet's timeline parts (`timeline` relationships). +pub fn read_sheet(cx: &mut Ctx<'_>, sheet_part: &str) -> Result, crate::IoError> { + let sys = cx.date_system; + let rels = cx.pkg.rels(sheet_part)?; + let mut out = Vec::new(); + for rel in rels.iter().filter(|r| r.kind == "timeline" && !r.external) { + let Some(x) = cx.optional_xml(&rel.target)? else { continue }; + for t in x.kids("timeline").take(MAX_TIMELINES) { + let (Some(name), Some(cache)) = (t.attr("name"), t.attr("cache")) else { continue }; + out.push(TimelineIn { + name: name.to_string(), + cache: cache.to_string(), + caption: t.attr("caption").unwrap_or(name).to_string(), + level: level_from(t.attr_u32("level")), + selection_level: level_from(t.attr_u32("selectionLevel")), + scroll: t.attr("scrollPosition").and_then(|v| crate::sheet_read::parse_iso_datetime(v, sys)).map(f64::floor), + style: t.attr("style").filter(|s| !s.is_empty()).unwrap_or("TimeSlicerStyleLight1").to_string(), + show: [ + t.flag("showHeader", true), + t.flag("showSelectionLabel", true), + t.flag("showTimeLevel", true), + t.flag("showHorizontalScrollbar", true), + ], + }); + } + } + Ok(out) +} + +/// Builds the model's timelines once every sheet and PivotTable is read. `sheets[i]` holds the +/// timelines read from the i-th model sheet with their drawing frames (timeline name → anchor); +/// `tab_ids` maps a `sheetId` to its model sheet. +pub fn attach( + cx: &mut Ctx<'_>, + wb: &mut Workbook, + caches: &[CacheIn], + sheets: Vec<(Vec, HashMap)>, + tab_ids: &HashMap, +) { + for (si, (list, frames)) in sheets.into_iter().enumerate() { + for t in list { + let Some(cache) = caches.iter().find(|c| c.name.eq_ignore_ascii_case(&t.cache)) else { + cx.warn(format!("timeline \"{}\" has no cache and was skipped", t.name)); + continue; + }; + let pivots: Vec = cache + .pivots + .iter() + .filter_map(|(tab, name)| { + let sheet = wb.sheet(*tab_ids.get(tab)?)?; + sheet.pivots.iter().find(|p| p.name.eq_ignore_ascii_case(name)).map(|p| p.id) + }) + .collect(); + if pivots.is_empty() && !cache.pivots.is_empty() { + cx.warn(format!("timeline \"{}\" filters a PivotTable that wasn't read; it filters nothing", t.name)); + } + let anchor = frames.get(&t.name.to_lowercase()).copied().unwrap_or(Anchor { + cell: CellRef::new(0, 0), + dx: 0.0, + dy: 0.0, + width: 262.5, + height: 108.0, + mode: AnchorMode::MoveOnly, + }); + let id = cx.next_object_id(); + let [show_header, show_selection_label, show_time_level, show_scrollbar] = t.show; + let tl = Timeline { + id, + name: t.name, + caption: t.caption, + field: cache.field.clone(), + pivots, + anchor, + level: t.level, + selection: cache.selection, + selection_level: t.selection_level, + scroll: t.scroll, + style: t.style, + show_header, + show_selection_label, + show_time_level, + show_scrollbar, + }; + if let Some(sheet) = wb.sheet_mut(si) { + sheet.timelines.push(tl); + } + } + } +} diff --git a/crates/xlsx/src/write.rs b/crates/xlsx/src/write.rs index 06761f01d..d9dd49d22 100644 --- a/crates/xlsx/src/write.rs +++ b/crates/xlsx/src/write.rs @@ -116,6 +116,10 @@ pub struct Out { /// References of chartex parts, defined as hidden `_xlchart.v1.N` names. pub chart_names: Vec, pub drawings: u32, + /// Timeline parts written so far (one per sheet with timelines). + pub timelines: u32, + /// Timeline cache parts written so far. + pub timeline_caches: u32, pub tables: u32, pub comments: u32, /// Authors of threaded comments, in first-use order; written to `xl/persons/person.xml`. @@ -203,6 +207,8 @@ pub fn write_xlsx(wb: &Workbook, kind: XlsxKind) -> Result, IoError> { charts: 0, chart_names: vec![], drawings: 0, + timelines: 0, + timeline_caches: 0, tables: 0, comments: 0, persons: vec![], @@ -229,10 +235,13 @@ pub fn write_xlsx(wb: &Workbook, kind: XlsxKind) -> Result, IoError> { let mut pivots = crate::pivot::PivotWriter::default(); let mut sheets_xml = String::new(); let mut sheet_id = 1usize; + // The `sheetId` of each sheet (timeline caches name their PivotTables' sheets by it). + let mut tab_ids: Vec = Vec::with_capacity(wb.sheets.len()); for (i, sheet) in wb.sheets.iter().enumerate() { let (before, next) = kept.sheet_entries(i, false, sheet_id); sheets_xml.push_str(&before); sheet_id = next; + tab_ids.push(sheet_id); let (xml, mut rels) = crate::sheet_write::write_sheet(wb, i, i == active, &mut out); pivots.add_sheet(wb, i, &mut out, &mut rels); let name = format!("xl/worksheets/sheet{}.xml", i + 1); @@ -253,8 +262,42 @@ pub fn write_xlsx(wb: &Workbook, kind: XlsxKind) -> Result, IoError> { } sheets_xml.push_str(&kept.sheet_entries(wb.sheets.len(), true, sheet_id).0); - // Defined names, then print areas and titles. + // Timeline caches, each with its `NativeTimeline_…` name and a workbook relationship. + let mut timeline_refs = String::new(); let mut names = String::new(); + let cache_names = crate::timeline::cache_names(wb); + for (sheet, list) in wb.sheets.iter().zip(cache_names.iter()) { + for (t, name) in sheet.timelines.iter().zip(list) { + // The first PivotTable it filters, with its sheet's id, pivot cache and filter id. + let first = t + .pivots + .first() + .and_then(|id| wb.sheets.iter().enumerate().find_map(|(psi, sh)| sh.pivots.iter().position(|p| p.id == *id).map(|pi| (psi, pi)))); + let pivot = first.and_then(|(psi, pi)| { + let pt = wb.sheet(psi)?.pivots.get(pi)?; + let k = crate::timeline::filters_of(wb, pt.id).iter().position(|x| x.id == t.id).unwrap_or(0) as u32; + Some(crate::timeline::CachePivot { + tab_id: *tab_ids.get(psi)?, + name: pt.name.clone(), + cache_id: pivots.cache_id(psi, pi)?, + filter_id: crate::timeline::FILTER_ID_BASE + k, + }) + }); + let bounds = first.and_then(|(psi, pi)| pivots.bounds(psi, pi, &t.field)); + out.timeline_caches += 1; + let n = out.timeline_caches; + out.part( + &format!("xl/timelineCaches/timelineCache{n}.xml"), + Some(crate::timeline::CT_TIMELINE_CACHE), + crate::timeline::cache_xml(t, name, pivot.as_ref(), bounds, wb.date_system).into_bytes(), + ); + let rid = wb_rels.add_uri(crate::timeline::REL_TIMELINE_CACHE, &format!("timelineCaches/timelineCache{n}.xml")); + let _ = write!(timeline_refs, ""); + let _ = write!(names, "#N/A", esc_attr(name)); + } + } + + // Defined names, then print areas and titles. let mut seen = std::collections::HashSet::new(); for n in &wb.names { if n.name.is_empty() || !seen.insert((n.name.to_ascii_lowercase(), n.scope)) { @@ -351,7 +394,15 @@ pub fn write_xlsx(wb: &Workbook, kind: XlsxKind) -> Result, IoError> { } w.push_str("/>"); w.push_str(&pivots.workbook_xml(&mut wb_rels)); - let exts = crate::passthrough::workbook_elements(wb, "ext"); + let mut exts = crate::passthrough::workbook_elements(wb, "ext"); + if !timeline_refs.is_empty() { + let _ = write!( + exts, + "{timeline_refs}", + crate::timeline::EXT_WORKBOOK, + crate::timeline::NS_X15 + ); + } if !exts.is_empty() { let _ = write!(w, "{exts}"); } diff --git a/docs/file-format-parity.md b/docs/file-format-parity.md index 988472414..5867486b7 100644 --- a/docs/file-format-parity.md +++ b/docs/file-format-parity.md @@ -1,6 +1,6 @@ # File-format parity -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-12 · **Change:** trivial (XLSX chart sheets are kept on save (#727); previously minor: CSV save keeps the separator it was opened with) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** minor (PivotTable time grouping, Group Selection fields, calculated items and timelines round-trip) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Every format Excel opens or saves, with GridCraft's read and write support, fidelity and tests. Part of [target-app-parity.md](target-app-parity.md). **File formats: ~50% ready, 80–120 h** @@ -22,9 +22,9 @@ and repair-free opening remain unverified. | Format | Excel | GridCraft read | GridCraft write | Fidelity / notes | Tests | |---|---|---|---|---|---| -| `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, calc settings. **Kept unchanged on save (pass-through, `passthrough.rs`):** custom document properties, ribbon customisation and add-in task panes, custom XML, links to external workbooks (never followed or refreshed), data connections, sheet query tables, printer settings (#324), form controls, ActiveX controls, OLE objects and sheet backgrounds (with their VML shapes), and chart sheets (#727: the sheet, drawing, chart, style and colour parts are written back unchanged in their place, and left out when a sheet they plot was deleted or renamed or a worksheet took their name; they can't be edited or created); dropped when their sheet is deleted or their parts point at content GridCraft rewrites. **Not preserved:** slicers and slicer caches, timelines, data model, query tables attached to tables, sheet and workbook `extLst` entries we don't model, chart details; Strict Open XML files keep nothing. Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | -| `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, notes (with their shown/hidden state) and threaded comments with replies, resolved state and timestamps (`threadedComments` and `persons` parts, #300), hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, calc settings. **Not preserved:** unmodelled parts and extensions (form controls, slicer caches, timelines, external links, OLE objects, custom XML, data model, query tables, chart details). Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | -| `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables), protection, Allow Edit Ranges (`protectedRanges` with their password hashes, #307), calc settings. **Not preserved:** unmodelled parts and extensions (form controls, slicer caches, timelines, external links, OLE objects, custom XML, data model, query tables, chart details). Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | +| `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables, with date, time, number and Group Selection grouping, calculated fields and items, label/value/Top 10 filters), timelines (timeline and timeline cache parts, frames, the PivotTable's date filter), protection, calc settings. **Kept unchanged on save (pass-through, `passthrough.rs`):** custom document properties, ribbon customisation and add-in task panes, custom XML, links to external workbooks (never followed or refreshed), data connections, sheet query tables, printer settings (#324), form controls, ActiveX controls, OLE objects and sheet backgrounds (with their VML shapes), and chart sheets (#727: the sheet, drawing, chart, style and colour parts are written back unchanged in their place, and left out when a sheet they plot was deleted or renamed or a worksheet took their name; they can't be edited or created); dropped when their sheet is deleted or their parts point at content GridCraft rewrites. **Not preserved:** slicers and slicer caches, timelines filtering several PivotTables (saved filtering the first), data model, query tables attached to tables, sheet and workbook `extLst` entries we don't model, chart details; Strict Open XML files keep nothing. Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | +| `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, notes (with their shown/hidden state) and threaded comments with replies, resolved state and timestamps (`threadedComments` and `persons` parts, #300), hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables, with date, time, number and Group Selection grouping, calculated fields and items, label/value/Top 10 filters), timelines (timeline and timeline cache parts, frames, the PivotTable's date filter), protection, calc settings. **Not preserved:** unmodelled parts and extensions (form controls, slicer caches, external links, OLE objects, custom XML, data model, query tables, chart details). Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | +| `.xlsx` Excel Workbook (transitional) | R/W, native | yes | yes | Cells, shared/array/dynamic formulas, styles, themes, merges, CF, validation, tables, comments and notes, hyperlinks, names, freeze panes, outline, print settings, pictures, shapes, charts (simplified, see [chart-parity.md](chart-parity.md)), sparklines, PivotTables (caches and tables, with date, time, number and Group Selection grouping, calculated fields and items, label/value/Top 10 filters), timelines (timeline and timeline cache parts, frames, the PivotTable's date filter), protection, Allow Edit Ranges (`protectedRanges` with their password hashes, #307), calc settings. **Not preserved:** unmodelled parts and extensions (form controls, slicer caches, external links, OLE objects, custom XML, data model, query tables, chart details). Never checked in Excel across a corpus: repair prompts unknown. 100 MB files fail (#175) | `roundtrip.rs`, `fixtures.rs`, `malformed.rs`, `pivot_tests.rs` | | `.xlsx` Strict Open XML | R/W | yes | no (writes transitional) | Excel also saves transitional by default | fixtures | | `.xlsm` Macro-Enabled Workbook | R/W | yes | yes, macros kept, not run | The VBA project is kept byte-for-byte and written back only into `.xlsm`/`.xltm` (#6); macros never run. Saved with the macro-enabled workbook content type | `roundtrip.rs` (`vba_project_roundtrip`, `package_kind_sets_workbook_content_type`), engine `io.rs` (`macros_are_saved_only_into_xlsm`) | | `.xltx` / `.xltm` templates | R/W | yes | partial | Saved with the template (and macro-enabled template) content type, chosen by extension; `.xltm` keeps a VBA project, `.xltx` drops it; "new from template" behaviour missing | `roundtrip.rs` (`package_kind_sets_workbook_content_type`), engine `io.rs` | @@ -73,7 +73,7 @@ refresh. Counted under features (Get & Transform, ~10% ready), not here. | Work | Hours | |---|---| | Real-world XLSX corpus run (craftrules test-corpora standard), and confirm Excel opens our output with no repair prompt (owner runs Excel) | 15–25 | -| Preserve the rest of the unmodelled XLSX content (slicers, timelines, data model, table query tables, `extLst` entries); pass-through landed for the rest (#330) | 6–10 | +| Preserve the rest of the unmodelled XLSX content (slicers, data model, table query tables, `extLst` entries); pass-through landed for the rest (#330) | 6–10 | | Streaming XLSX reader for very large files (#175), part-size limits that degrade instead of failing | 10–15 | | Legacy `.xls` (BIFF8): styles (fonts, fills, borders, alignment), comments, CF, validation, pictures and charts on import; corpus run; then write | 10–15 | | Encrypted workbooks: save with password (#11) | 4–6 | @@ -85,6 +85,7 @@ refresh. Counted under features (Get & Transform, ~10% ready), not here. | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | PivotTables: hours/minutes/seconds grouping, Grouping dialog bounds (`autoStart`/`autoEnd`) and day spans (`groupInterval`), Group Selection fields (`discretePr`), calculated items (`calculatedItems`) and timelines (`timeline`/`timelineCache` parts, `x15` ext lists, `tsle:timeslicer` frames, `dateBetween` filter) are read and written. Excel 365 opens these files and keeps the grouping, but ignores the calculated items and does not apply the timeline's saved selection | | 2026-10-12 | trivial | XLSX chart sheets are kept on save (#727) | | 2026-10-12 | trivial | CSV and TSV export is capped at 512 MiB and refuses a larger used range with an error (#528) | | 2026-10-12 | minor | `.xls` import reads the calculation settings (CalcMode, CalcIter, CalcCount, CalcDelta, CalcPrecision), so circular models calculate instead of showing `#CIRC!` (#724) | diff --git a/docs/function-parity.md b/docs/function-parity.md index 760ffd6ac..5352c8c66 100644 --- a/docs/function-parity.md +++ b/docs/function-parity.md @@ -1,6 +1,6 @@ # Function parity: worksheet functions and formulas -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-12 · **Change:** minor (USDOLLAR added as a compatibility function; MIRR, COUNTIF and INDIRECT edge cases match Excel 365) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** minor (GETPIVOTDATA and Generate GetPivotData) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Excel's worksheet functions, category by category, against GridCraft's function library. Part of [target-app-parity.md](target-app-parity.md); gaps feed [gaps.md](gaps.md). @@ -12,13 +12,9 @@ Excel's worksheet functions, category by category, against GridCraft's function them), plus 14 text functions the page lists under combined entries (FIND/FINDB, LEFT/LEFTB, LEN/LENB, MID/MIDB, REPLACE/REPLACEB, RIGHT/RIGHTB, SEARCH/SEARCHB), BETA.INV, and COPILOT and PY (Microsoft 365 functions not on that page when fetched). -- **Ours (505):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the -- **Ours (503):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the -- **Ours (502):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the -- **Ours (502):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the -- **Ours (505):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the +- **Ours (506):** every function spec in `crates/functions/src/*.rs` (`f!("NAME", …)`) plus the evaluator's special forms in `crates/calc/src/eval.rs` (`SPECIAL_FUNCTIONS`: IF, IFS, CHOOSE, - OFFSET, INDIRECT, INDEX, SUBTOTAL, AGGREGATE, TEXT, LET, LAMBDA, MAP, REDUCE, SCAN, BYROW, BYCOL, + OFFSET, INDIRECT, INDEX, GETPIVOTDATA, SUBTOTAL, AGGREGATE, TEXT, LET, LAMBDA, MAP, REDUCE, SCAN, BYROW, BYCOL, MAKEARRAY…), excluding the test-only `ADD2`. Every one is an Excel name. - **Not on Microsoft's list:** USDOLLAR, a legacy function Excel still evaluates (like DOLLAR), is in our library as a Compatibility function for files that use it. It is not counted above. @@ -29,7 +25,7 @@ Excel's worksheet functions, category by category, against GridCraft's function ## Coverage -**Measured: 513 / 523 by name (98.1%); 504 / 523 working (96.4%).** +**Measured: 514 / 523 by name (98.3%); 505 / 523 working (96.6%).** | Category | Excel | Ours | Working | % working | Missing | |---|---|---|---|---|---| @@ -38,7 +34,7 @@ Excel's worksheet functions, category by category, against GridCraft's function | Financial | 55 | 55 | 55 | 100% | — | | Engineering | 54 | 54 | 54 | 100% | — | | Text | 50 | 48 | 48 | 96% | DETECTLANGUAGE, TRANSLATE | -| Lookup and reference | 40 | 36 | 36 | 90% | GETPIVOTDATA, IMAGE, RTD, TRIMRANGE | +| Lookup and reference | 40 | 37 | 37 | 93% | IMAGE, RTD, TRIMRANGE | | Compatibility | 40 | 40 | 40 | 100% | — | | Date and time | 25 | 25 | 25 | 100% | — | | Information | 22 | 21 | 21 | 95% | STOCKHISTORY | @@ -48,9 +44,9 @@ Excel's worksheet functions, category by category, against GridCraft's function | Web | 3 | 3 | 1 | 33% | (WEBSERVICE, FILTERXML are stubs; ENCODEURL works) | | Add-in and Automation | 3 | 1 | 1 | 33% | CALL, REGISTER.ID | | AI and Python | 2 | 0 | 0 | 0% | COPILOT, PY | -| **Total** | **523** | **505** | **496** | **94.8%** | 18 missing, 9 stubs | +| **Total** | **523** | **506** | **497** | **95.0%** | 17 missing, 9 stubs | -By the Excel version that introduced them (working / total): before 2010 346 / 360, 2010 +By the Excel version that introduced them (working / total): before 2010 347 / 360, 2010 56 / 56, 2013 49 / 51, 2016 1 / 5, 2019 6 / 6, 2021 11 / 11, 2024 22 / 23, Microsoft 365 5 / 11. Dynamic-array and LAMBDA-era functions (FILTER, SORT, UNIQUE, XLOOKUP, VSTACK, TAKE, GROUPBY, PIVOTBY, REGEXTEST…) are all present. @@ -61,7 +57,6 @@ PIVOTBY, REGEXTEST…) are all present. |---|---|---|---| | FORECAST.ETS family (4) | yes | 4–6 h | AAA exponential smoothing with seasonality detection; needed for Forecast Sheet too | | ODDFPRICE, ODDFYIELD, ODDLPRICE, ODDLYIELD | yes | 2–4 h | Odd first/last period bond maths | -| GETPIVOTDATA | yes | 2–3 h | Needs the pivot engine's result cache; also the "Generate GetPivotData" option | | IMAGE | yes | 3–5 h | Pictures in cells (see PR #105); fetching URLs needs a privacy decision | | CUBE* (7), WEBSERVICE, FILTERXML | partly | 4–8 h | FILTERXML can work offline; CUBE needs an OLAP/data-model source | | TRANSLATE, DETECTLANGUAGE, STOCKHISTORY, COPILOT, PY | no (cloud) | — | Microsoft cloud services; open equivalents only by owner decision | @@ -83,6 +78,7 @@ PIVOTBY, REGEXTEST…) are all present. | 15-significant-digit comparison semantics | **missing** (#57: `=0.1+0.2=0.3` is FALSE) | | External workbook references (`[Book.xlsx]Sheet1!A1`) | parsed; links dropped on open, no update | | Localized function names and `;` separators | **missing** (#45); en-US only | +| GETPIVOTDATA and Generate GetPivotData | done: value field by caption or source field, any cell or range of the report, field/item pairs for row, column and single-item report filter fields; fields left out are totals, or the one item the named items leave (Excel's inference, observed); hidden, filtered, collapsed items and totals turned off give `#REF!`, an empty value cell 0; numbers and DATE/serials name numeric and date items, text names an item's label; recalculates with every refresh; reports of PivotTables read from files are indexed on open when they match. Clicking or arrowing onto a PivotTable value while typing a formula inserts the GETPIVOTDATA formula (setting on by default). Difference: a `data_field` given as a cell reference uses the cell's text (Excel gives `#REF!` unless the text is computed, e.g. `T(H2)`) | | Error values | the seven classic errors plus `#SPILL!`, `#CALC!`, `#GETTING_DATA` done; `#FIELD!`, `#BLOCKED!`, `#CONNECT!`, `#BUSY!`, `#UNKNOWN!`, `#PYTHON!` missing (minor: they come from data types, cloud and Python features) | ## Depth (estimated) @@ -108,6 +104,7 @@ workbooks; results checked in as data, not Excel files) and fixing what it finds | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | GETPIVOTDATA (Lookup and reference) and the Generate GetPivotData option; semantics checked against Excel 365 on synthetic PivotTables (inference of outer items, empty cells as 0, report filters, totals turned off) | | 2026-10-12 | minor | USDOLLAR added as a Compatibility function (not on Microsoft's list, so the counts don't change); MIRR with no positive flow is -1, COUNTIF's bare `"<="` / `">="` skip empty strings, INDIRECT with `a1` = FALSE gives `#REF!` for A1-style text (#738) | | 2026-10-11 | minor | TEXT / number formats fixed against Excel-cached results: fraction literals and zero fractions, `0` denominators zero padded, `?` pads the thousands separator, booleans use the text section (226 mismatches → 0 in 352 TEXT cells) | | 2026-10-11 | minor | Iterative calculation: circular references iterate up to Maximum Iterations / Maximum Change when enabled | diff --git a/docs/gaps.md b/docs/gaps.md index 03e8672c5..12e8a7553 100644 --- a/docs/gaps.md +++ b/docs/gaps.md @@ -1,6 +1,6 @@ # Gaps -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-12 · **Change:** minor (rows auto-fit oversized fonts on open and whole-row/column formatting) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-14 · **Last updated:** 2026-10-14 · **Change:** minor (§8: preserve dormant PivotTable filter rules on XLSX round-trip) · **Target:** Microsoft Excel (Microsoft 365) Every known shortfall against Excel, one entry each, ranked by user impact (what blocks someone using GridCraft for real work first). This is the work list: pick from the top, check the open @@ -77,12 +77,26 @@ No alpha blockers: every core workflow passes the [alpha gate](roadmap.md#alpha- routine in business. - **Estimate:** 14–21 h (`.xls` styles and objects 10–15, encryption 4–6). [file-format-parity.md](file-format-parity.md) -### 8. PivotTables: no slicers, calculated items, PivotCharts; unverified in Excel (F) -- **Evidence:** `parity-checklist.md` missing `insert.slicer`, `insert.timeline`, - `insert.pivotChart`, `table.insertSlicer`; calculated items and GETPIVOTDATA missing (calculated fields - and values on rows done). +### 8. PivotTables: no slicers on PivotTables or date filters; Excel ignores saved calculated items and timeline selections (F) +- **Evidence:** slicers (`insert.slicer`, `table.insertSlicer`) filter tables only; PivotTable + slicers are #719. Done: calculated fields and calculated items (with solve order and List + Formulas), Group Selection, date and time grouping with the Grouping dialog's bounds and Number + of days, number ranges, timelines (sheet object, Timeline tab, XLSX timeline and timeline cache + parts), label, value and Top 10 filters from the report's field buttons, values on rows, + PivotCharts, GETPIVOTDATA and the PivotTable Analyze and Design tabs. Label, value and Top 10 + rules on source fields outside Rows/Columns survive XLSX saves without filtering or adding + report-filter rows; adding the field to an axis reactivates its rule (synthetic regression tests). + Missing: date filters + (relative periods), "allow multiple filters per field", OLAP and data-model pivots, Field + Settings for row and column fields, a pivot cache shared by several PivotTables (a timeline + filtering several PivotTables is saved to XLSX filtering the first). Checked in Excel 365 over + COM on synthetic files: date levels, Starting/Ending at bounds, day spans and Group Selection + groups open and refresh to the same items and totals (time levels not checked). Files with + calculated items or a timeline open without repair, but Excel drops the calculated items (marking + them as formula items makes Excel refuse the file), and it shows the timeline with its period + without applying it to the PivotTable. - **Impact:** pivots are the analyst's main tool. -- **Estimate:** 20–30 h for beta (25–40 h full). +- **Estimate:** 10–16 h for beta (15–25 h full). ### 9. Keyboard-first users: key tips, shortcuts, context menus (U) - **Evidence:** #33, #49 (key tips: initial Home-tab support landed in #43 on 10-10, the rest of the ribbon not yet), #110, #28 (selection), @@ -120,10 +134,7 @@ No alpha blockers: every core workflow passes the [alpha gate](roadmap.md#alpha- Margins are dragged on the rulers (#792). Remaining: pictures in headers/footers, Different First Page / Odd & Even pages. Desktop now submits directly through craft-print with printer choice, copies, duplex and grayscale; browser printing stays available. Print ranges and driver tray/options UI remain. **Estimate:** 5–10 h. ### 15. Remaining functions and an oracle suite (F) -- 18 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 16–26 h. -- 20 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 17–29 h. -- 21 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 18–30 h. -- 18 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 18–30 h. +- 17 missing, 9 stubs; no Excel-computed oracle corpus; #137, #138. **Estimate:** 14–23 h. [function-parity.md](function-parity.md) ### 16. Collaboration and review (F) @@ -168,6 +179,9 @@ No alpha blockers: every core workflow passes the [alpha gate](roadmap.md#alpha- | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | §8: dormant label/value/Top 10 rules survive XLSX round-trips and reactivate when their field returns to Rows/Columns (#1207) | +| 2026-10-14 | minor | §8: Group Selection, time grouping and the Grouping dialog, calculated items, timelines and the label/value filter menus landed; slicers on PivotTables and date filters remain | +| 2026-10-14 | minor | §8: GETPIVOTDATA, Generate GetPivotData and the PivotTable Analyze and Design tabs; §15: 17 functions missing | | 2026-10-12 | minor | rows auto-fit oversized fonts on open and whole-row/column formatting (#833) | | 2026-10-12 | minor | Page Layout view: dragging margins on the rulers (#792) removed from the §14 remainder | | 2026-10-12 | minor | §13: Forecast Sheet Forecast Start (hindcast) landed (#785) | diff --git a/docs/parity-checklist.md b/docs/parity-checklist.md index d218c8169..eb86d4ae0 100644 --- a/docs/parity-checklist.md +++ b/docs/parity-checklist.md @@ -4,13 +4,13 @@ Each entry is an Excel ribbon or menu command (feature names only). It counts as implemented when GridCraft registers a command with the same id. This measures breadth only; the overall assessment (depth, fidelity, effort) is in [target-app-parity.md](target-app-parity.md). -**Overall: 274 / 290 commands implemented (94.5%).** +**Overall: 292 / 307 commands implemented (95.1%).** | Tab | Implemented | Total | Parity | |---|---|---|---| | File | 11 | 11 | 100% | | Home | 94 | 94 | 100% | -| Insert | 17 | 25 | 68% | +| Insert | 18 | 25 | 72% | | Draw | 4 | 4 | 100% | | Page Layout | 23 | 23 | 100% | | Formulas | 23 | 23 | 100% | @@ -21,8 +21,9 @@ Each entry is an Excel ribbon or menu command (feature names only). It counts as | Table Design | 13 | 13 | 100% | | Chart Design | 8 | 8 | 100% | | Format | 3 | 3 | 100% | -| PivotTable Analyze | 13 | 13 | 100% | +| PivotTable Analyze | 22 | 22 | 100% | | PivotTable Design | 4 | 4 | 100% | +| Timeline | 8 | 8 | 100% | ## File (11 / 11, 100%) @@ -139,7 +140,7 @@ Implemented: - `edit.goToSpecial` Go To Special (Editing) - `home.addIns` Add-ins (Add-ins) -## Insert (17 / 25, 68%) +## Insert (18 / 25, 72%) Implemented: @@ -154,6 +155,7 @@ Implemented: - `insert.pivotChart` PivotChart (Charts) - `insert.sparkline` Sparklines (Sparklines) - `insert.slicer` Slicer (Filters) +- `insert.timeline` Timeline (Filters) - `insert.link` Link (Links) - `review.newComment` Comment (Comments) - `insert.textBox` Text Box (Text) @@ -166,7 +168,6 @@ Missing: - `insert.smartArt` SmartArt (Illustrations) - `insert.screenshot` Screenshot (Illustrations) - `insert.map` Maps (Charts) -- `insert.timeline` Timeline (Filters) - `insert.wordArt` WordArt (Text) - `insert.signatureLine` Signature Line (Text) - `insert.object` Object (Text) @@ -371,10 +372,11 @@ Implemented: - `chart.shapeFill` Shape Fill (Shape Styles) - `chart.shapeOutline` Shape Outline (Shape Styles) -## PivotTable Analyze (13 / 13, 100%) +## PivotTable Analyze (22 / 22, 100%) Implemented: +- `pivot.generateGetPivotData` Generate GetPivotData (PivotTable) - `pivot.fieldList` Field List (Show) - `pivot.refresh` Refresh (Data) - `pivot.changeSource` Change Data Source (Data) @@ -386,8 +388,16 @@ Implemented: - `pivot.filter` Filter (Filter) - `pivot.sort` Sort (Sort) - `pivot.group` Group Field (Group) +- `pivot.groupSelection` Group Selection (Group) +- `pivot.ungroup` Ungroup (Group) +- `pivot.calculatedItem` Calculated Item (Calculations) +- `pivot.solveOrder` Solve Order (Calculations) +- `pivot.listFormulas` List Formulas (Calculations) +- `pivot.insertTimeline` Insert Timeline (Filter) - `pivot.collapse` Collapse Field (Active Field) -- `pivot.delete` Clear All (Actions) +- `pivot.clearAll` Clear All (Actions) +- `pivot.clearFilters` Clear Filters (Actions) +- `pivot.delete` Delete PivotTable (Actions) ## PivotTable Design (4 / 4, 100%) @@ -397,3 +407,16 @@ Implemented: - `pivot.subtotals` Subtotals (Layout) - `pivot.reportLayout` Report Layout (Layout) - `pivot.style` PivotTable Styles (PivotTable Styles) + +## Timeline (8 / 8, 100%) + +Implemented: + +- `timeline.caption` Timeline Caption (Timeline) +- `timeline.reportConnections` Report Connections (Timeline) +- `timeline.style` Timeline Styles (Timeline Styles) +- `timeline.size` Size (Size) +- `timeline.showHeader` Header (Show) +- `timeline.showSelectionLabel` Selection Label (Show) +- `timeline.showScrollbar` Scrollbar (Show) +- `timeline.showTimeLevel` Time Level (Show) diff --git a/docs/roadmap.md b/docs/roadmap.md index 062dfc708..4cf6e86d5 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,6 +1,6 @@ # Roadmap detail -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-11 · **Change:** minor (PivotTable calculated fields) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** minor (PivotTables: grouping, calculated items, timelines; gate #8 and the beta total) · **Target:** Microsoft Excel (Microsoft 365) Forward-looking plan. The one-page summary is [`ROADMAP.md`](../ROADMAP.md); the ranked work list is [`gaps.md`](gaps.md); numbers and method are in @@ -27,7 +27,7 @@ workflow, would make GridCraft pre-alpha. | Build a model: enter data and formulas, recalc, save XLSX, reopen | yes | 492 working functions, dynamic arrays, LET/LAMBDA, names, tables; XLSX round-trip tests (`crates/xlsx/src/tests/roundtrip.rs`). Edge-case bugs (#57 15-digit equality, stale spills #133/#139) give wrong results in specific cases, not in the workflow | 0 (fixes are beta work, gap #6) | | Open a colleague's XLSX, edit, save, hand it back | partial, not blocking | Typical workbooks open and save with cells, styles, CF, validation, tables, names and pivots; unmodelled parts are dropped with a warning (gap #3); Excel-side repair prompts unverified (gap #1); ~100 MB files fail (#175) | 0 for alpha (gaps #1–#3 are beta gates) | | Format and present: number formats, styles, conditional formatting, print or PDF | yes | Full number-format language, 47 cell styles, all CF kinds, Page Break Preview, PDF export and printing. PDF can't carry non-Latin text (gap #5) | 0 | -| Analyze: sort, filter, tables, PivotTable | yes | Multi-level sort, AutoFilter, tables with totals, PivotTables with field list, layouts, grouping, refresh and XLSX round-trip. Calculated fields done; slicers missing (gap #8) | 0 | +| Analyze: sort, filter, tables, PivotTable | yes | Multi-level sort, AutoFilter, tables with totals, PivotTables with field list, layouts, grouping, refresh and XLSX round-trip. Calculated fields and items, Group Selection, time grouping and timelines done; slicers on PivotTables missing (gap #8) | 0 | | Chart the data | partial, not blocking | Column, bar, line, pie, area, scatter and combo charts are created, edited and survive save and reopen; histogram, box, waterfall, funnel, treemap, sunburst and stock lose their kind in XLSX (fix in PRs #158, #161); no axis options or trendlines (gap #4) | 0 for alpha (2–4 h to merge the kind fixes) | | Import and export CSV | yes | Encoding and delimiter detection on read, CSV/TSV write; exports raw rather than displayed values (gap #19) | 0 | @@ -50,10 +50,10 @@ and edge features. Each gate links its gap. | Text in every script renders and exports | #5 | 12–20 | | Calculation correctness bugs closed; iterative calculation | #6 | 10–15 | | `.xls` import, password-protected workbooks | #7 | 28–42 | -| PivotTables: slicers, calculated fields, PivotCharts, verified in Excel | #8 | 20–30 | +| PivotTables: slicers on PivotTables, date filters, Excel reading saved calculated items and timeline selections | #8 | 10–16 | | Keyboard: key tips, shortcut audit, context menus | #9 | 12–20 | | Field crashes and freezes closed; autosave/recovery soak | #10 | 8–12 | -| **Total** | | **~165–255 (≈160–240 after overlap with the PR queue)** | +| **Total** | | **~155–241 (≈150–230 after overlap with the PR queue)** | With 4–5 agents and an integrator, about 40–60 wall-clock hours. Needs the owner: running Excel to check our files, and Windows machines with the reported GPU drivers. @@ -112,6 +112,7 @@ to check our files, and Windows machines with the reported GPU drivers. | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | PivotTables: Group Selection, time grouping, calculated items and timelines landed; gate #8 20–30 → 10–16 h, beta total ≈160–240 → ≈150–230 h | | 2026-10-11 | minor | PivotTable calculated fields landed | | 2026-10-10 | minor | Current focus and M15 updated for the 10-10 merges | | 2026-10-10 | minor | Added the alpha gate table (six core workflows; passes, stays alpha) | diff --git a/docs/target-app-parity.md b/docs/target-app-parity.md index b623124d9..bd7bd7562 100644 --- a/docs/target-app-parity.md +++ b/docs/target-app-parity.md @@ -1,6 +1,6 @@ # Target-app parity: GridCraft vs Microsoft Excel -> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-12 · **Change:** minor (rows auto-fit oversized fonts on open and row formatting) · **Target:** Microsoft Excel (Microsoft 365) +> **Last reviewed:** 2026-10-11 · **Last updated:** 2026-10-14 · **Change:** minor (PivotTables: Group Selection, time grouping, calculated items, timelines, label and value filter menus; GETPIVOTDATA and the PivotTable tabs) · **Target:** Microsoft Excel (Microsoft 365) This is the authoritative parity assessment. [`ROADMAP.md`](../ROADMAP.md) summarizes it, [`gaps.md`](gaps.md) lists every shortfall, and the area checklists hold the detail: @@ -13,14 +13,14 @@ generated ribbon checklist is [`parity-checklist.md`](parity-checklist.md). | Number | Value | Kind | |---|---|---| | Feature breadth (weighted by use) | **~75%** | estimated from the area table below; anchored on two measurements | -| Ribbon and menu catalog | **274 / 290 (94.5%)** | measured: `cargo xtask parity` (`crates/engine/src/catalog.rs` ∩ registered command ids) | -| Worksheet functions, by name | **513 / 523 (98.1%)** | measured: registered functions vs Microsoft's published function list (method below) | -| Worksheet functions that do real work | **504 / 523 (96.4%)** | measured: excludes 9 stubs (7 CUBE functions, WEBSERVICE, FILTERXML) | -| **Ready for real work** | **~50%** | estimated; written weights below (51.6 computed) | -| **Mainstream practitioner** | **~50%** | estimated; [method](#mainstream-practitioner) (69.5 depth × 0.90 × 0.92 × 0.88 = 50.6) | +| Ribbon and menu catalog | **292 / 307 (95.1%)** | measured: `cargo xtask parity` (`crates/engine/src/catalog.rs` ∩ registered command ids) | +| Worksheet functions, by name | **514 / 523 (98.3%)** | measured: registered functions vs Microsoft's published function list (method below) | +| Worksheet functions that do real work | **505 / 523 (96.6%)** | measured: excludes 9 stubs (7 CUBE functions, WEBSERVICE, FILTERXML) | +| **Ready for real work** | **~50%** | estimated; written weights below (52.0 computed) | +| **Mainstream practitioner** | **~50%** | estimated; [method](#mainstream-practitioner) (69.9 depth × 0.90 × 0.92 × 0.88 = 50.9) | | **Essentials user** | **~63%** | estimated; [method](#essentials-user) (80.2 depth × 0.92 × 0.95 × 0.90 = 63.1) | | Stage | **alpha** | see [Stage](#stage) | -| Remaining to beta | **~160–240 Opus 5.5 agent-hours** | estimated; [roadmap.md](roadmap.md#beta-gates) | +| Remaining to beta | **~150–230 Opus 5.5 agent-hours** | estimated; [roadmap.md](roadmap.md#beta-gates) | | Remaining to full parity | **~700–1,100 Opus 5.5 agent-hours** | estimated; sum of the rows below | ### Readiness by audience @@ -80,11 +80,11 @@ performance 8, stability 8, localization 4, hardware 2, platforms 2, ecosystem 2 | Area | Weight | Breadth | Ready | Hours | Evidence | |---|---|---|---|---|---| | Formula language and calculation | 18 | ~92% | ~72% | 20–30 | A1/R1C1 parser, reference adjustment, dependency-graph recalc, dynamic arrays and `#SPILL!`, `A1#`, LET/LAMBDA and helpers, structured references, names, 3-D references. Open correctness bugs: 15-digit equality (#57), stale spills after structural edits and cleared blockers (#133, #139), 3-D span endpoint deletion (#140), sheet rename and qualified names (#134). Single-threaded | -| Worksheet functions | 12 | 98.1% (measured) | ~82% | 16–26 | 513 / 523 names, 504 working. Missing: GETPIVOTDATA, IMAGE plus cloud functions. No conformance suite against Excel-computed oracles; edge-case bugs keep surfacing (#137, #138). [function-parity.md](function-parity.md) | +| Worksheet functions | 12 | 98.3% (measured) | ~82% | 16–26 | 514 / 523 names, 505 working. Missing: IMAGE plus cloud functions. No conformance suite against Excel-computed oracles; edge-case bugs keep surfacing (#137, #138). [function-parity.md](function-parity.md) | | Editing, navigation, clipboard, fill | 12 | ~90% | ~70% | 15–25 | Point mode, F4, AutoComplete, Paste Special, AutoFill, Flash Fill, Find/Replace, Go To Special. Bugs: rectangular drag-select (#28), monthly fill (#132), cut across sheets (#128), Insert Copied Cells missing (#21), key tips only on part of the Home tab (#43; #33, #49). [ui-parity.md](ui-parity.md) | | Cell formatting, styles, conditional formatting | 10 | ~95% | ~80% | 8–14 | Full number-format language, borders, 47 cell styles, 60 table styles, all CF rule kinds including data bars, colour scales and icon sets. Gaps: font weights beyond regular/bold (#168), system fonts list (#4), themes | | Sort, filter, tables, validation, outline | 8 | ~82% | ~70% | 10–16 | Multi-level sort, AutoFilter, Advanced Filter, tables with totals, validation, outline, subtotals, consolidate. Bugs: subtotal header (#131), validation edits (#129), table style panic caught (#130); no slicers on tables | -| PivotTables | 7 | ~65% | ~45% | 25–40 | Insert/Recommended, field list, three layouts, values on rows or columns, date grouping, show-values-as, refresh, calculated fields, XLSX round-trip. Missing: slicers, timelines, calculated items, PivotCharts, value/label filters, number grouping, GETPIVOTDATA, data-model pivots. Round-trip never checked in Excel | +| PivotTables | 7 | ~78% | ~58% | 10–16 | Insert/Recommended, field list, three layouts, values on rows or columns, Group Selection, date and time grouping (Grouping dialog with bounds and Number of days), number ranges, Show Values As (all 15 calculations, with base field and base item), label, value and Top 10 filters (report field buttons and the field list), calculated fields and calculated items (solve order, List Formulas), timelines (Timeline tab, XLSX parts), refresh, PivotCharts, GETPIVOTDATA, PivotTable Analyze and Design tabs, XLSX round-trip. Missing: slicers on PivotTables, date filters, several filters per field, Field Settings for row and column fields, a shared pivot cache, data-model pivots. In Excel 365 grouped fields round-trip; files with calculated items or timelines open, but Excel drops the calculated items and does not apply the timeline's selection | | Charts and sparklines | 8 | ~50% | ~30% | 40–60 | 25 chart kinds, all 2-D. The chart model holds title, legend position, a data-labels flag, gridlines flag and axis titles; no axis scaling, trendlines, error bars, per-point formatting, chart sheets, 3-D, surface, map, Pareto. Several kinds don't survive XLSX round trip (open PRs #158, #161). [chart-parity.md](chart-parity.md) | | Page layout, print, PDF | 5 | ~80% | ~55% | 12–20 | Page setup, print titles, breaks, Page Break Preview, headers/footers, PDF export and printing. No true Page Layout view; the PDF writer uses only the 14 standard fonts without embedding (no CJK or other non-Latin text in PDF); web print only exports (#141, #143) | | What-if and analysis | 4 | ~50% | ~40% | 22–35 | Goal Seek, Scenario Manager, data tables, Analysis ToolPak (13 of 19 tools, no charts), Solver (Simplex LP, a local nonlinear method and Evolutionary; Answer and Limits reports; no Sensitivity report or options dialog). Missing: Forecast Sheet, Analyze Data | @@ -92,13 +92,13 @@ performance 8, stability 8, localization 4, hardware 2, platforms 2, ecosystem 2 | Insert objects | 3 | ~55% | ~45% | 25–40 | Pictures, 8 shape kinds (Excel has ~170), text boxes, own icon set, ink with Ink to Shape, hyperlinks, checkboxes. Missing: SmartArt, WordArt, equations, signature line, objects, 3-D models, screenshot, pictures in cells (PR #105) | | Review, comments, protection, collaboration | 4 | ~55% | ~40% | 35–55 | Threaded comments, notes, spelling (system dictionary), thesaurus, sheet and workbook protection, Allow Edit Ranges with Excel-compatible password hashes (#307). Missing: comment navigation, Show Changes, notes-to-comments conversion, accessibility checker, co-authoring, opening password-encrypted files (refused with a message since #17) | | Automation | 4 | ~30% | ~20% | 12–20 | Recorded, editable GridCraft scripts (Automate tab). VBA is out of scope by owner decision and is **dropped** when an `.xlsm` is opened (warning shown). No Office Scripts compatibility | -| **Weighted** | 100 | **~75%** | **~59%** | **300–485** | | +| **Weighted** | 100 | **~75%** | **~60%** | **285–461** | | ## Dimensions | Dimension | Weight | Ready | Hours | Evidence | |---|---|---|---|---| -| Features | 45 | ~59% | 300–485 | Table above | +| Features | 45 | ~60% | 285–461 | Table above | | File formats | 15 | ~50% | 80–120 | XLSX read/write is broad but unproven on a real corpus and in Excel; XLSB and ODS are values-only import; no XLS, XML Spreadsheet 2003, SYLK, DIF, PRN; VBA dropped; 100 MB XLSX fails (#175). [file-format-parity.md](file-format-parity.md) | | UI/UX fidelity | 12 | ~55% | 50–80 | Excel-style ribbon, galleries, dialogs, formula bar, status bar; 59 commands with shortcuts vs Excel's 200+; key tips on part of the Home tab only; short context menu; single window. [ui-parity.md](ui-parity.md) | | Performance | 8 | ~35% | 35–55 | [Performance](#performance) | @@ -108,13 +108,14 @@ performance 8, stability 8, localization 4, hardware 2, platforms 2, ecosystem 2 | Platforms | 2 | ~80% | 30–60 | [Platforms](#platforms) | | Ecosystem / add-ins | 2 | ~10% | 25–40 | [Ecosystem and AI](#ecosystem-and-ai) | | AI features | 2 | ~15% | 30–50 | [Ecosystem and AI](#ecosystem-and-ai) | -| **Ready for real work** | 100 | **~50%** (51.6 computed: 0.45 × 59.0 features + the other rows × their weights) | **~700–1,100** | | +| **Ready for real work** | 100 | **~50%** (52.0 computed: 0.45 × 59.9 features + the other rows × their weights) | **~700–1,100** | | How the full number is built: features ready is the weighted mean of the area table (59.0 with charts at 30%); ready for real work is the weighted mean of this table with the weights written under [Weights](#weights). Re-checked after the 10-10 merges: charts were corrected from 38% to 30% (round-trip loss, #190 chart resize) and localization rose from 5% to 10% (#31, #104, #82, -#38). The two moves cancel: 51.6 before and after, so **~50%** stands. +#38). The two moves cancel: 51.6 before and after, so **~50%** stands. The 10-14 PivotTable work +(ready 45 → 58%) lifts features ready from 59.0 to 59.9 and the full number to 52.0; **~50%** stands. ## Mainstream practitioner @@ -131,12 +132,12 @@ functions, co-authoring and admin, specialist hardware, and languages beyond the | Editing, navigation, clipboard, fill | 15 | 70 | Feature-area table | | Cell formatting, styles, conditional formatting | 13 | 80 | Feature-area table | | Sort, filter, tables, validation | 10 | 70 | Feature-area table | -| PivotTables (core: fields, layouts, grouping, refresh) | 8 | 60 | Slicers and calculated fields excluded as less frequent | +| PivotTables (core: fields, layouts, grouping, refresh) | 8 | 65 | Group Selection and date/time grouping done; slicers, calculated fields and items excluded as less frequent | | Charts (column, bar, line, pie, scatter, combo) | 9 | 45 | Common kinds work and survive save; no axis options or trendlines; resize broken (#190) | | Page layout, print, PDF | 5 | 60 | Latin text exports | | Comments, notes, protection | 3 | 55 | Feature-area table | | Pictures and shapes | 2 | 45 | Feature-area table | -| **Average depth** | 100 | **69.5** | | +| **Average depth** | 100 | **69.9** | | Discounts for what still stops real work: @@ -146,7 +147,7 @@ Discounts for what still stops real work: | Stability on real machines | × 0.92 | Never-crash lints, fuzzed commands and the `Session::execute` guard; but field reports of a startup crash on Intel UHD (#120), an error on close (#117), a freeze opening from Finder (#166) and screen flashing (#76) | | File exchange with Excel users | × 0.88 | Excel opening our output is unverified across features; unmodelled parts dropped on save; advanced chart kinds lose their type; 100 MB files fail (#175); `.xls` and encrypted files are refused with a clear message (#17) rather than opened. Harsher than a drawing app's discount because spreadsheets are exchanged constantly | -**Mainstream practitioner: 69.5 × 0.90 × 0.92 × 0.88 = 50.6 → ~50%** (estimated). Unlike most +**Mainstream practitioner: 69.9 × 0.90 × 0.92 × 0.88 = 50.9 → ~50%** (estimated). Unlike most apps it is not higher than the full number: GridCraft's breadth is wide (functions, formatting, ribbon), so excluding the long tail removes little, while the discounts land on exactly what mainstream users do every week (exchange files with Excel users). @@ -242,7 +243,7 @@ Linux, BSD and RISC-V, where Excel doesn't run. (ready ~50%, inside the 40–75% alpha band). Not beta: the main format, XLSX, has not been proven on real-world files or confirmed to open in Excel without repair, large files fail, `.xls` is missing and text in several scripts doesn't render. Distance to beta: ~25 points of readiness and -~160–240 h, itemized in [roadmap.md](roadmap.md#beta-gates). +~150–230 h, itemized in [roadmap.md](roadmap.md#beta-gates). ## Effort and calibration @@ -301,7 +302,7 @@ The inventory carried over from the 2026-10-07 ROADMAP.md, updated for what land average, colour), tables with totals rows, remove duplicates, text to columns, data validation with error alerts, grouping/outline, subtotals. - PivotTables: Insert/Recommended PivotTables, field list pane, compact/outline/tabular layouts, - values on rows or columns, date grouping, sort, filters, show-values-as, refresh, and XLSX round-trip. + values on rows or columns, Group Selection, date, time and number range grouping, sort, filters (items, labels, values, Top 10) from the report's field buttons, Show Values As (all 15 calculations), calculated fields and items, timelines, refresh, and XLSX round-trip. - Page layout & print: Page Break Preview, print settings, PDF export and printing. - What-if: Goal Seek, scenarios, data tables. Spelling (system word list) and thesaurus. - Draw: pen, highlighter, eraser, ink strokes and Ink to Shape. Task panes: Comments (threaded @@ -332,6 +333,8 @@ The inventory carried over from the 2026-10-07 ROADMAP.md, updated for what land | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | PivotTables: Group Selection and Ungroup, date and time grouping with the Grouping dialog, calculated items (solve order, List Formulas), timelines (Insert › Timeline, Timeline tab, XLSX timeline parts), report field buttons with Label Filters and Value Filters; Top N keeps ties at the cut-off (PivotTables ~65 → ~78% breadth, ~45 → ~58% ready; features ready 59.0 → 59.9, full number 51.6 → 52.0, mainstream PivotTables depth 60 → 65, 50.6 → 50.9; headline percentages unchanged); beta ~160–240 → ~150–230 h (gate #8 20–30 → 10–16 h) | +| 2026-10-14 | minor | Worksheet functions and PivotTables: GETPIVOTDATA, Generate GetPivotData, PivotTable Analyze and Design tabs | | 2026-10-12 | minor | Essentials AutoFit evidence: rows auto-fit oversized fonts on open and row formatting (#833) | | 2026-10-12 | minor | Sheet tabs: reveal the selected tab on navigation and resize while preserving manual scrolling (#681); no score change | | 2026-10-12 | minor | PivotTables: values on rows (Σ Values in Rows at any position, all three layouts; `pivot.moveField` and the field pane move it; `dataOnRows` round-trips) (no score change) | @@ -343,6 +346,9 @@ The inventory carried over from the 2026-10-07 ROADMAP.md, updated for what land | 2026-10-11 | minor | Worksheet functions 502 / 523 (96.0%; 493 working): TRIMRANGE | | 2026-10-11 | minor | View › Formula Bar and Zoom 100% are engine commands (CLI/MCP); catalog 258 → 260 / 290 (#289) | | 2026-10-11 | minor | Review › Allow Edit Ranges; catalog 258 → 259 / 290 (#307) | +| 2026-10-12 | minor | PivotTables: label, value and Top 10 filters (`pivot.labelFilter`, `pivot.valueFilter`, `pivot.top10`, `pivot.clearFilter`) | +| 2026-10-11 | minor | PivotTables: number fields group into ranges (`pivot.group` with `by: range`) | +| 2026-10-11 | minor | PivotTables: Show Values As covers all 15 calculations, with base field and base item (`pivot.valueSettings`) | | 2026-10-11 | minor | PivotTables: calculated fields (`pivot.calculatedField`) | | 2026-10-11 | minor | PivotTables: PivotCharts (`insert.pivotChart`) | | 2026-10-10 | minor | Readiness-by-audience table: hours to ~95% for each of the three numbers (full 650–1,050, mainstream 200–310, essentials 60–100). Full number confirmed as the additive weighted sum over the written dimension weights (no change) | diff --git a/docs/ui-parity.md b/docs/ui-parity.md index 22e221e69..5138853c0 100644 --- a/docs/ui-parity.md +++ b/docs/ui-parity.md @@ -1,10 +1,10 @@ # UI parity -> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-13 · **Change:** minor (horizontal scrollbar in the sheet tab bar, #840) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 +> **Last reviewed:** 2026-10-10 · **Last updated:** 2026-10-14 · **Change:** minor (Timeline tab, PivotTable field buttons with Label and Value Filters, Grouping and Calculated Item dialogs) · **Target:** Microsoft Excel (Microsoft 365), Excel for Mac 16.113.4 Interaction, controls and feel, against Excel. Part of [target-app-parity.md](target-app-parity.md). **UI/UX fidelity: ~55% ready, 50–80 h** (estimated). Ribbon command coverage is measured -separately in [parity-checklist.md](parity-checklist.md) (274 / 290). +separately in [parity-checklist.md](parity-checklist.md) (277 / 293). Sources: `crates/ui-egui/src` (ribbon, grid, editor, formula bar, dialogs, panes), `crates/engine/src/cmd` (shortcuts in command specs), the owner's black-box notes @@ -15,7 +15,7 @@ Sources: `crates/ui-egui/src` (ribbon, grid, editor, formula bar, dialogs, panes | Element | Excel | GridCraft | Status | |---|---|---|---| | Title bar, Quick Access Toolbar, AutoSave toggle | yes | yes | done; macOS window controls misaligned (#170), Dock icon size (#169) | -| Ribbon: tabs, groups, split buttons, galleries, contextual tabs | yes | yes (Table Design, Chart Design, Format, PivotTable, Header & Footer tabs) | done; missing contextual tabs for pictures, shapes, sparklines, slicers | +| Ribbon: tabs, groups, split buttons, galleries, contextual tabs | yes | yes (Table Design, Chart Design, Format, PivotTable Analyze and PivotTable Design (key tips JT, JY; controls GridCraft lacks stay disabled with the reason), Timeline, Header & Footer tabs) | done; missing contextual tabs for pictures, shapes, sparklines, slicers | | Ribbon collapse, simplified ribbon, customize ribbon/QAT | yes | partial | customization missing (File ▸ Options is the one missing File command) | | Ribbon: tabs, groups, split buttons, galleries, contextual tabs | yes | yes (Table Design, Chart Design, Format, PivotTable tabs) | done; missing contextual tabs for pictures, shapes, sparklines, header/footer, slicers | | Ribbon collapse, simplified ribbon, customize ribbon/QAT | yes | partial | customization missing | @@ -161,6 +161,8 @@ scrollbar bugs 2–4 · pixel/feel pass against observed Excel 7–12. | Date | Change | Summary | |---|---|---| +| 2026-10-14 | minor | Timeline contextual tab; Insert › Filters › Timeline; PivotTable report field buttons (Row Labels, Column Labels, field headers, report filters) with Sort, Label Filters ▸, Value Filters ▸ and the item checklist; Label Filter, Value Filter, Top 10, Grouping, Insert Calculated Item, Solve Order, Insert Timelines and Report Connections dialogs | +| 2026-10-13 | minor | PivotTable Analyze and PivotTable Design contextual tabs with Excel 365's groups and German labels; Clear All and Clear Filters commands | | 2026-10-12 | minor | The File view's recent list tells when each workbook was last changed, read off the UI thread | | 2026-10-13 | minor | Horizontal scrollbar in the sheet tab bar: drag, track click pages, frozen columns excluded from the page (#840) | | 2026-10-12 | minor | Ctrl+wheel zoom steps along a ladder of percentages about 20% apart that never skips 100%, gliding to each step with animations on, so a notch out undoes a notch in (it drifted: 100 → 122 → 101); Excel steps 15 points, see decisions.md | diff --git a/xtask/src/keytips.rs b/xtask/src/keytips.rs index 36847222f..0aef4ed3f 100644 --- a/xtask/src/keytips.rs +++ b/xtask/src/keytips.rs @@ -12,6 +12,10 @@ use std::path::Path; const TABS: &[&str] = &["Home", "Insert", "Draw", "Page Layout", "Formulas", "Data", "Review", "View", "Automate"]; +/// Contextual tabs: they show only while their object is selected, so their sequence switches +/// to them only then (a blank workbook stays on Home). +const CONTEXTUAL_TABS: &[&str] = &["PivotTable Analyze", "PivotTable Design"]; + /// Commands whose empty-params behaviour depends on the host machine (e.g. the /// system dictionary). These are never probed; their tests assert "invoked /// something observable" so the checked-in file stays byte-identical everywhere. @@ -134,8 +138,8 @@ fn parse_and_probe(source: &str) -> Result<(Vec, Probe), String> { } match kind.as_str() { "tab" => { - if target.is_empty() || !TABS.contains(&target.as_str()) { - return Err(format!("{at}: tab target {target:?} is not one of {TABS:?}")); + if target.is_empty() || !(TABS.contains(&target.as_str()) || CONTEXTUAL_TABS.contains(&target.as_str())) { + return Err(format!("{at}: tab target {target:?} is not one of {TABS:?} or {CONTEXTUAL_TABS:?}")); } } "command" | "dialog" | "control" => { @@ -260,6 +264,11 @@ fn render_tests(entries: &[Entry], probe: &Probe) -> String { let mut out = String::from(HEADER); for e in entries { let body = match e.kind.as_str() { + "tab" if CONTEXTUAL_TABS.contains(&e.target.as_str()) => format!( + "sequence(&mut h, {seq:?}, false);\n assert_eq!(h.state().ui.ribbon_tab, \"Home\", \"{seq} must not switch to the hidden {tab} tab\");", + seq = e.seq, + tab = e.target, + ), "tab" => format!( "sequence(&mut h, {seq:?}, false);\n assert_eq!(h.state().ui.ribbon_tab, {tab:?}, \"{seq} must switch to the {tab} tab\");", seq = e.seq,