Add comprehensive lookups documentation#1164
Open
asinghvi17 wants to merge 64 commits into
Open
Conversation
… for `AbstractDimStack` (#903) * add combine method * test groupby and similar * docs entry
* add preservedims keyword to DimTable * add tests * Apply suggestions from code review Co-authored-by: Anshul Singhvi <anshulsinghvi@gmail.com> * tests, and fix DimSlices * better table docs * cleanup * test * indexing overhaul * fix similar and broadcast for basicdimarray * bugfix rebuildsliced * more indexing cleanup * cleanup similar and gubfix indexing * bugfixes * uncomment * fix doctests * just dont doctest unreproducable failures, for now * combine new Tables integrations * bugfix and cleanup show * bugfix and more tests for preservedims and mergedims --------- Co-authored-by: Anshul Singhvi <anshulsinghvi@gmail.com>
* iterate values where no layer is missing * add tests * add skipmissing to reference
…991) - Add comprehensive tests for hasmultipledimensions trait in merged.jl - Test that regular lookups return false for hasmultipledimensions - Test that MergedLookup returns true for hasmultipledimensions - Test extent passthrough for merged dimensions - Test mixed regular and merged dimensions - Test that operations preserve the hasmultipledimensions trait - Add fallback methods to handle edge cases: - hasmultipledimensions(::Any) = false for non-Lookup types - bounds(x::AbstractArray) for raw arrays - Import Extents in merged.jl test file - Fix bounds ambiguity with explicit module qualification
* Table Materializer Methods * Made col Optional for DimArray * Apply suggestions from code review Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Handle coordinates with different loci * replaced At() with Contains() in _coords_to_ords * Added optional selectors and public methods for table materializer * Updated table constructors for DimArray and DimStack * Updated DimArray and DimStack docs to include table materializer methods * Table materializer test cases * export table materializer methods * Added Random to tables.jl test cases * Update src/array/array.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Removed exports * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Replaced selector type with instance. * Table materializer can now infer dimensions from the coordinates. * Update src/stack/stack.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/array/array.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/table_ops.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Added support for guessing the dimension ordering and span for Dates and DateTimes * Replaced LinRange with StepRangeLen in _build_dim * Added Tables.istable check to DimArray constructor * Update src/array/array.jl * merge materialize2 * fix scuffed merge * filter instead of indexing in test for clarity * fix DimSlices doc * fix ambiguities * bugfixes * do checks and call Tables.columns before constructing stack from table * test dimensions are automatically detected when constructing dimstack * comments not docstrings for internals * check for columnaccess if dims are passed * add type argument to dimarray_from_table * allow passing name to DimStack * add a section to the documentation * use Tables.columnnames instead of keys * make DimArray work with all tables that are abstractarrays * do not treat dimvectors as tables * simplify get_column --------- Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> Co-authored-by: Tiem van der Deure <tiemvanderdeure@gmail.com>
* add _similar dispatch for abstractdimarray * update tests * Update src/array/array.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update src/array/array.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> --------- Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com>
* standardise interface methods and remove index * update Changelog * cleanup * move const * cleanup * remove index from test * dont export index * last index * tweaks * more tweaks * fix tests --------- Co-authored-by: Raf Schouten <schoutenr@ull-pf39vwmc.mobility.unimelb.net.au>
* Forward name keyword in groupby * Add test for setting groupby name explicitly * Update src/groupby.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Update test/groupby.jl Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * Add Changelog entry * Mention name keyword in docstring --------- Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com>
* Remove rtol from At selector * Remove explicit rtol from test * Remove unused type parameter * fix At constructors --------- Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com>
* Complete unfinished docstring example * Support refdims in DimTable * Test refdims in DimTable * By default add no refdims to Tables To make feature non-breaking * Update tables tests * Test getcolumn for DimStack/DimArray * Remove unused variable * Update constructor calls * Correctly compute dimnums * Use all dims to compute colnames * By default include refdims * Make sure data-array is duplicated if necessary * Add preservedims/refdims test * Fix some bugs * Add more joint compatibility tests with refdims * Support AoG selection of refdims * Only extended array with dims if needed * Test AoG with refdims * refdims doc line --------- Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com>
* add DimStackArray * export DimStackArray * add tests * add a docstring * drop inner constructor Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> * fix missing } * fix DimStackArray type definition * add tests for broadcast over stack * add to the docs --------- Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com>
Co-authored-by: Felix Cremer <felix.cremer@dlr.de>
* Swap dims test and value test in == * Add isequal test * Add broken test for dimarray isequal with different axes * Make the same swap in == also for DimStack * Add isequal for AbstractDimArray and AbstractDimstack * Fix stack test * Update test/stack.jl Co-authored-by: Tiem van der Deure <tiemvanderdeure@gmail.com> * Add Changelog entry --------- Co-authored-by: Rafael Schouten <rafaelschouten@gmail.com> Co-authored-by: Tiem van der Deure <tiemvanderdeure@gmail.com>
…#1162) * Fix NearestNeighbor extension Description of the fix from Claude: The issue was in ext/DimensionalDataNearestNeighborsExt.jl at line 35: Problem: The distance vector was being created with the wrong element type: distvec = Vector{NN.get_T(eltype(points))}(undef, 1) NN.get_T(eltype(points)) was returning SVector{2, Float64} (the point type) instead of Float64 (the scalar distance type). This caused knn! to fail with: ArgumentError: dists must have eltype Float64, got StaticArraysCore.SVector{2, Float64} Fix: Changed to use eltype(eltype(points)) which correctly extracts the scalar type: distvec = Vector{eltype(eltype(points))}(undef, 1) * Disable broken inference tests Claude blames the implementation of `_sortdims()`, which is called by `_dims()` and is apparently not type-stable. * Disable CondaPkg verbosity This otherwise spams the terminal. * Revert "fix `sum(da; dims = :notadim)` (#1116)" This reverts commit 2245221. * Improve error messages for reduction methods with missing dims This uses the new `_missingdims()` helper function to select all the missing dimensions, and we now don't call `basetypeof` in `_extradimsmsg()` since `extradims` may not all be `Dim`s. * Fix Makie tests
- Document all lookup types (Sampled, Categorical, Cyclic, NoLookup, Transformed) - Explain trait system (Order, Sampling, Span, Locus) - Cover common operations (bounds, intervalbounds, predicates, set) - Add guide for implementing custom lookups - Include example: LogLookup with custom selector behavior - Include example: MatrixLookup with internal dimensions (X, Y) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Collaborator
Claude sucks! 😄, I'm starting to hate these long summaries. BTW, what is the motivation for this? I feel that this might be too repetitive in some points, looking at https://rafaqz.github.io/DimensionalData.jl/stable/api/lookuparrays#Lookups |
Collaborator
Author
|
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Base methods: parent, size, length, axes, firstindex, lastindex, getindex - Add Lookup interface: order, metadata, span, sampling - Add rebuild with standard data= kwarg plus coords= extension - Add hasinternaldimensions trait - Add dims methods for integration with Dimensions module - Export coords, coord_dim, grid_position accessors Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add getindex/view for AbstractVector to slice both data and coords - Add slice_coords to slice coords along other dimensions (e.g., face) - Export slice_coords Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Returns coordinate extent (min, max) from the coords matrix. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Returns NoLookup when dimension is reduced. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Validates axis length matches lookup data length. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests for construction, slicing, bounds, rebuild, dims, reducelookup, and DimArray integration. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds FacedGridLookup section to the Lookup Types tabs in lookups.md. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
docs/src/lookups.md)🤖 Generated with Claude Code