Skip to content

Speed up quartodoc build by removing individual table documentation - #117

Open
ryan-kipawa with Copilot wants to merge 5 commits into
mainfrom
copilot/explore-documentation-speed
Open

Speed up quartodoc build by removing individual table documentation#117
ryan-kipawa with Copilot wants to merge 5 commits into
mainfrom
copilot/explore-documentation-speed

Conversation

Copilot AI commented Feb 24, 2026

Copy link
Copy Markdown
Contributor

The quartodoc build takes ~109 minutes because include_inherited: true is set globally, causing griffe to individually resolve ~20 inherited BaseTable methods for each of 329 auto-generated table classes (~6,580 get_object calls at ~1s each). Individual table documentation provides marginal value since the database schema is available from within MIKE+ and the sqlite database.

  • Removed include_inherited: true from global options — all 329 tables inherit identical methods from BaseTable; documenting them on every page is redundant and is the sole cause of the ~109min build time
  • Removed all 329 individual table entries from the quartodoc sections — column descriptions are accessible via MIKE+ and the sqlite database directly
  • Removed docs/generate_table_docs.py and docs/_table_generated_sections.yml — no longer needed without individual table documentation
  • Restricted TableCollection members to [keys, values, items] — its 329 property accessors are no longer documented individually
  • Added "Base Table Classes" section — documents BaseTable, BaseGeometryTable, and BaseNodeTable once in a dedicated section, so all inherited methods (select, insert, update, delete, to_dataframe, get_muids, etc.) and node-specific methods (get_downstream_links, get_upstream_links, etc.) are documented centrally
  • Pinned griffe<2 in docs dependencies — fixes allow_section_blank_line TypeError with quartodoc==0.10.0
  • Updated .github/workflows/docs.yml — replaced outdated mkdocs build with quartodoc build + quarto render, added Quarto setup action, updated action versions, and switched to pip install .[docs]
  • Updated DEVELOPMENT.md — removed table doc generation step and slow-build warning from documentation instructions

Build time: ~109 min → ~2s.

Original prompt

Building the quartodoc documentation is extremely slow. Explore opportunities to speed it up.


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

…ableCollection members

Co-authored-by: ryan-kipawa <74630349+ryan-kipawa@users.noreply.github.com>
Copilot AI changed the title [WIP] Explore opportunities to speed up quartodoc documentation build Speed up quartodoc build by removing redundant inherited member resolution Feb 24, 2026
@ryan-kipawa

Copy link
Copy Markdown
Collaborator

@copilot Each column has a unique set of columns with a description that is useful. Additionally, there's a hierarchy of base classes, where some tables have additional functionality. However, if it's that slow, I will consider removing it. How could we document this in an equivalent way which is faster?

Co-authored-by: ryan-kipawa <74630349+ryan-kipawa@users.noreply.github.com>

Copilot AI commented Feb 24, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Each column has a unique set of columns with a description that is useful. Additionally, there's a hierarchy of base classes, where some tables have additional functionality. However, if it's...

The unique columns per table (with descriptions) are still fully documented — each table page shows its XxxTableColumns class with all column attributes and docstrings. What include_inherited: true was adding was repeating the same BaseTable methods (select, insert, delete, update, to_dataframe, get_muids, etc.) on all 329 table pages.

In commit acc84a4, I added a "Base Table Classes" section that documents BaseTable, BaseGeometryTable, and BaseNodeTable once in a central place. This way:

  • All shared methods are documented on the BaseTable page
  • The 4 node-specific methods (get_downstream_links, get_upstream_links, etc.) are documented on the BaseNodeTable page
  • Each individual table page still shows its unique column definitions with descriptions
  • Build stays at ~7s instead of ~109 minutes

@ryan-kipawa ryan-kipawa left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Trying to build the docs gives me this error:

.venv\Lib\site-packages\griffe\_internal\docstrings\parsers.py", line 51, in parse
    return parsers[parser](docstring, **options)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
TypeError: parse_numpy() got an unexpected keyword argument 'allow_section_blank_line

Co-authored-by: ryan-kipawa <74630349+ryan-kipawa@users.noreply.github.com>

Copilot AI commented Feb 25, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Trying to build the docs gives me this error:

This is a pre-existing compatibility issue between quartodoc==0.10.0 and griffe>=2.0.0. quartodoc 0.10.0 passes allow_section_blank_line to griffe's numpy docstring parser, but griffe 2.0.0 removed support for that keyword argument.

Fixed in 293cb5c by adding "griffe<2" to the docs dependencies in pyproject.toml.

@ryan-kipawa
ryan-kipawa marked this pull request as ready for review February 26, 2026 07:41
@ryan-kipawa

Copy link
Copy Markdown
Collaborator

@copilot It is still to slow after fixing this. Please remove individual table documentation - it's added value is marginal compared to the time required to generate the documents. The database schema is still available from within MIKE+, as well as the sqlite database, and we can assume that users are familiar enough with the columns that they do not need to rely on the descriptions provided by the documentation. Don't forget to update the developer markdown internal docs and github actions for documentation.

…OPMENT.md

Co-authored-by: ryan-kipawa <74630349+ryan-kipawa@users.noreply.github.com>

Copilot AI commented Feb 26, 2026

Copy link
Copy Markdown
Contributor Author

@copilot It is still to slow after fixing this. Please remove individual table documentation - it's added value is marginal compared to the time required to generate the documents. The...

Done in f651e8a:

  • Removed all 329 individual table entries from the "Tables" section in _quarto.yml (kept "Base Table Classes" section for shared API method docs)
  • Deleted docs/generate_table_docs.py and docs/_table_generated_sections.yml
  • Updated DEVELOPMENT.md — removed step 12 (table doc generation) and the slow-build warning
  • Updated .github/workflows/docs.yml — replaced outdated mkdocs build with quartodoc build + quarto render, added Quarto setup action, and switched to pip install .[docs]

Build time is now ~2.4s.

Copilot AI changed the title Speed up quartodoc build by removing redundant inherited member resolution Speed up quartodoc build by removing individual table documentation Feb 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants