Repository navigation
chore(tests): re-introduce docfx pre-submit - #18589
Merged
daniel-sanche merged 2 commits intoOct 8, 2026
Merged
daniel-sanche merged 2 commits into
daniel-sanche merged 2 commits into
Conversation
Contributor
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
daniel-sanche
added
the
unit_test:all_packages
daniel-sanche
force-pushed
the
investigate_post_submit_checks_ci_02_docfx-presubmit-sharding
branch
from
October 7, 2026 20:34
4bfbd35 to
aa663ea
Compare
daniel-sanche
added this pull request to stack #18607
October 8, 2026 16:33
daniel-sanche
force-pushed
the
investigate_post_submit_checks_ci_02_docfx-presubmit-sharding
branch
from
October 8, 2026 16:37
aa663ea to
e6a2b20
Compare
daniel-sanche
marked this pull request as ready for review
October 8, 2026 16:38
Base automatically changed from
investigate_post_submit_checks_gcp-sphinx-docfx-yaml_01_speed-optimizations
to
main
October 8, 2026 17:58
daniel-sanche
added a commit
that referenced
this pull request
Oct 8, 2026
Previously, docfx would [take > 3 hours on large PR](https://github.com/googleapis/google-cloud-python/actions/runs/32315047111/job/96265452483). The compute package alone would take > 70 minutes, so this wasn't a problem that could be solved with sharding <img width="495" height="418" alt="image" src="https://github.com/user-attachments/assets/5d7465f5-2d9f-430f-b8e5-5fa425286187" /> This PR attempts to speed up the task by adding optimizations to the `gcp-sphinx-docfx-yaml` package: - 2bb74d8: sort known ids once, instead of inside a loop - 7a1c741: immediately skip over words without "." characters, since they can't be uuids - c54391e: use custom no-op StandaloneHTMLBuilder class, to avoid doing rendering work for unused html files - 7b9ab23: use in-process markdown builder, instead of spanning new process and starting from scratch - 8dedfb7: cache default settings, instead of re-constructing on each node visit - 7b91c2d: cache class line numbers, instead of re-building for each one - 9effc20: disable unused sphinx hooks - 28478f8: prefer native yaml parser when available. Fall back to pure-python when needed - 6d9bb07: added fast-path to skip non-google symbols I had gemini run a comparison between these outputs, and outputs from the previous version, and it saw identical outputs [A follow-up PR](#18589) re-adds docfx as a sharded pre-submit, with all jobs finishing < 15 mins
daniel-sanche
force-pushed
the
investigate_post_submit_checks_ci_02_docfx-presubmit-sharding
branch
from
October 8, 2026 17:58
e6a2b20 to
fc90c8c
Compare
ohmayr
approved these changes
Oct 8, 2026
daniel-sanche
deleted the
investigate_post_submit_checks_ci_02_docfx-presubmit-sharding
branch
October 8, 2026 20:31
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docfx was recently moved to a post-submit test due to its speed. With sharding, combined with performance optimizations in
gcp-sphinx-docfx-yaml, we can run it within 10 minutes, and keep it pre-submitThis PR also installs
gcp-sphinx-docfx-yamlfrom HEAD for the docfx test, rather than from pypi