From 24632dd403b65960100e3ade904cb7f4bd63d83f Mon Sep 17 00:00:00 2001 From: Keith Smiley Date: Tue, 14 Jul 2026 12:57:39 -0700 Subject: [PATCH] Support documenting globals Depends on bazel versions including https://github.com/bazelbuild/bazel/commit/815b9632e58452a7e28c66926c1ffe04c1e17f86 Fixes https://github.com/bazelbuild/stardoc/issues/202 --- docs/advanced_stardoc_usage.md | 15 ++++++++ docs/stardoc_rule.md | 9 +++-- .../build/stardoc/renderer/RendererMain.java | 12 +++++- .../stardoc/renderer/RendererOptions.java | 6 +++ .../stardoc/rendering/MarkdownRenderer.java | 26 ++++++++++++- stardoc/html_tables_stardoc.bzl | 1 + stardoc/private/stardoc.bzl | 9 ++++- stardoc/stardoc.bzl | 11 ++++-- .../html_tables/starlark_other_symbol.vm | 9 +++++ .../markdown_tables/starlark_other_symbol.vm | 9 +++++ .../markdown_tables/table_of_contents.vm | 8 ++++ test/BUILD | 21 ++++++++++ .../starlark_other_symbols_test/golden.md | 38 +++++++++++++++++++ .../html_tables_golden.md | 38 +++++++++++++++++++ .../starlark_other_symbols_test/input.bzl | 17 +++++++++ .../testdata/table_of_contents_test/golden.md | 15 ++++++++ .../testdata/table_of_contents_test/input.bzl | 3 ++ 17 files changed, 235 insertions(+), 12 deletions(-) create mode 100644 stardoc/templates/html_tables/starlark_other_symbol.vm create mode 100644 stardoc/templates/markdown_tables/starlark_other_symbol.vm create mode 100644 test/testdata/starlark_other_symbols_test/golden.md create mode 100644 test/testdata/starlark_other_symbols_test/html_tables_golden.md create mode 100644 test/testdata/starlark_other_symbols_test/input.bzl diff --git a/docs/advanced_stardoc_usage.md b/docs/advanced_stardoc_usage.md index bad41677..8526a941 100644 --- a/docs/advanced_stardoc_usage.md +++ b/docs/advanced_stardoc_usage.md @@ -34,6 +34,21 @@ def my_function(foo, bar): Markdown formatting constructs are handled appropriately by Stardoc's default output format ("markdown_tables"), even as part of a table. +### Documenting global symbols + +With Bazel 9 or newer, global values without a built-in documentation mechanism, +such as strings, lists, dictionaries, and structs, can be documented with `#:` +comments: + +```starlark +#: The default timeout used by this module. +DEFAULT_TIMEOUT = 60 +``` + +The extracted documentation includes the symbol's name, documentation, and +Starlark type. Bazel does not currently include the symbol's value in the +Stardoc output proto. + ## Custom Output diff --git a/docs/stardoc_rule.md b/docs/stardoc_rule.md index 5a4ac42d..2c23a919 100644 --- a/docs/stardoc_rule.md +++ b/docs/stardoc_rule.md @@ -11,11 +11,11 @@ load("@stardoc//stardoc:stardoc.bzl", "stardoc") stardoc(*, name, input, out, deps, format, symbol_names, renderer, aspect_template, func_template, macro_template, header_template, table_of_contents_template, provider_template, rule_template, - repository_rule_template, module_extension_template, footer_template, render_main_repo_name, - stamp, **kwargs) + repository_rule_template, module_extension_template, starlark_other_symbol_template, + footer_template, render_main_repo_name, stamp, **kwargs) -Generates documentation for exported starlark rule definitions in a target starlark file. +Generates documentation for exported symbols in a target Starlark file. **PARAMETERS** @@ -27,7 +27,7 @@ Generates documentation for exported starlark rule definitions in a target starl | out | The file to which documentation will be output (mandatory). | none | | deps | A list of bzl_library dependencies which the input depends on. | `[]` | | format | The format of the output file. Valid values: 'markdown' or 'proto'. | `"markdown"` | -| symbol_names | A list of symbol names to generate documentation for. These should correspond to the names of rule definitions in the input file. If this list is empty, then documentation for all exported rule definitions will be generated. | `[]` | +| symbol_names | A list of exported symbol names to generate documentation for. If this list is empty, documentation for all exported symbols will be generated. | `[]` | | renderer | The location of the renderer tool. | `Label("@stardoc//stardoc:renderer")` | | aspect_template | The input file template for generating documentation of aspects | `Label("@stardoc//stardoc:templates/markdown_tables/aspect.vm")` | | func_template | The input file template for generating documentation of functions, including legacy macros. | `Label("@stardoc//stardoc:templates/markdown_tables/func.vm")` | @@ -38,6 +38,7 @@ Generates documentation for exported starlark rule definitions in a target starl | rule_template | The input file template for generating documentation of rules. | `Label("@stardoc//stardoc:templates/markdown_tables/rule.vm")` | | repository_rule_template | The input file template for generating documentation of repository rules. | `Label("@stardoc//stardoc:templates/markdown_tables/repository_rule.vm")` | | module_extension_template | The input file template for generating documentation of module extensions. | `Label("@stardoc//stardoc:templates/markdown_tables/module_extension.vm")` | +| starlark_other_symbol_template | The input file template for generating documentation of Starlark global symbols, such as constants, which use `#:` doc comments. | `Label("@stardoc//stardoc:templates/markdown_tables/starlark_other_symbol.vm")` | | footer_template | The input file template for generating the footer of the output documentation. Optional. | `None` | | render_main_repo_name | Render labels in the main repository with a repo component (either the module name or workspace name). | `True` | | stamp | Whether to provide stamping information to templates, where it can be accessed via `$util.formatBuildTimestamp()` and`$stamping`. Example: 
Built on `$util.formatBuildTimestamp($stamping.volatile.BUILD_TIMESTAMP, "UTC", "yyyy-MM-dd HH:mm")`
Possible values: Stamped targets are not rebuilt unless their dependencies change. | `-1` | diff --git a/src/main/java/com/google/devtools/build/stardoc/renderer/RendererMain.java b/src/main/java/com/google/devtools/build/stardoc/renderer/RendererMain.java index 9981c807..dd8aea61 100644 --- a/src/main/java/com/google/devtools/build/stardoc/renderer/RendererMain.java +++ b/src/main/java/com/google/devtools/build/stardoc/renderer/RendererMain.java @@ -33,6 +33,7 @@ import com.google.devtools.build.lib.starlarkdocextract.StardocOutputProtos.RepositoryRuleInfo; import com.google.devtools.build.lib.starlarkdocextract.StardocOutputProtos.RuleInfo; import com.google.devtools.build.lib.starlarkdocextract.StardocOutputProtos.StarlarkFunctionInfo; +import com.google.devtools.build.lib.starlarkdocextract.StardocOutputProtos.StarlarkOtherSymbolInfo; import com.google.devtools.build.stardoc.rendering.MarkdownRenderer; import com.google.devtools.build.stardoc.rendering.MarkdownRenderer.Renderer; import com.google.devtools.build.stardoc.rendering.Stamping; @@ -98,6 +99,7 @@ public void println() { rendererOptions.aspectTemplateFilePath, rendererOptions.repositoryRuleTemplateFilePath, rendererOptions.moduleExtensionTemplateFilePath, + rendererOptions.starlarkOtherSymbolTemplateFilePath, !moduleInfo.getFile().isEmpty() ? Optional.of(moduleInfo.getFile()) : Optional.empty(), @@ -154,6 +156,12 @@ public void println() { .sorted(comparing(ModuleExtensionInfo::getExtensionName)) .collect(toImmutableList()); + // Other documented Starlark symbols are printed sorted by their qualified name. + ImmutableList sortedStarlarkOtherSymbolInfos = + ImmutableList.sortedCopyOf( + comparing(StarlarkOtherSymbolInfo::getName), + moduleInfo.getStarlarkOtherSymbolInfoList()); + printWriter.println(renderer.renderMarkdownHeader(moduleInfo)); if (rendererOptions.tableOfContentsTemplateFilePath != null) { printWriter.println( @@ -164,7 +172,8 @@ public void println() { sortedStarlarkFunctions, sortedAspectInfos, sortedRepositoryRuleInfos, - sortedModuleExtensionInfos)); + sortedModuleExtensionInfos, + sortedStarlarkOtherSymbolInfos)); } print(printWriter, renderer::render, sortedRuleInfos); print(printWriter, renderer::render, sortedProviderInfos); @@ -173,6 +182,7 @@ public void println() { print(printWriter, renderer::render, sortedAspectInfos); print(printWriter, renderer::render, sortedRepositoryRuleInfos); print(printWriter, renderer::render, sortedModuleExtensionInfos); + print(printWriter, renderer::render, sortedStarlarkOtherSymbolInfos); if (rendererOptions.footerTemplateFilePath != null) { printWriter.println(renderer.renderMarkdownFooter(moduleInfo)); } diff --git a/src/main/java/com/google/devtools/build/stardoc/renderer/RendererOptions.java b/src/main/java/com/google/devtools/build/stardoc/renderer/RendererOptions.java index ad778b07..55029f41 100644 --- a/src/main/java/com/google/devtools/build/stardoc/renderer/RendererOptions.java +++ b/src/main/java/com/google/devtools/build/stardoc/renderer/RendererOptions.java @@ -86,6 +86,12 @@ class RendererOptions { description = "The template for the documentation of a module extension") String moduleExtensionTemplateFilePath; + @Parameter( + names = "--starlark_other_symbol_template", + required = true, + description = "The template for the documentation of a Starlark global symbol") + String starlarkOtherSymbolTemplateFilePath; + @Parameter(names = "--footer_template", description = "The template for the footer string") String footerTemplateFilePath; diff --git a/src/main/java/com/google/devtools/build/stardoc/rendering/MarkdownRenderer.java b/src/main/java/com/google/devtools/build/stardoc/rendering/MarkdownRenderer.java index 703e6a0d..ee37bfff 100644 --- a/src/main/java/com/google/devtools/build/stardoc/rendering/MarkdownRenderer.java +++ b/src/main/java/com/google/devtools/build/stardoc/rendering/MarkdownRenderer.java @@ -30,6 +30,7 @@ import com.google.devtools.build.lib.starlarkdocextract.StardocOutputProtos.RepositoryRuleInfo; import com.google.devtools.build.lib.starlarkdocextract.StardocOutputProtos.RuleInfo; import com.google.devtools.build.lib.starlarkdocextract.StardocOutputProtos.StarlarkFunctionInfo; +import com.google.devtools.build.lib.starlarkdocextract.StardocOutputProtos.StarlarkOtherSymbolInfo; import com.google.escapevelocity.EvaluationException; import com.google.escapevelocity.ParseException; import com.google.escapevelocity.Template; @@ -61,6 +62,7 @@ public interface Renderer { private final String aspectTemplateFilename; private final String repositoryRuleTemplateFilename; private final String moduleExtensionTemplateFilename; + private final String starlarkOtherSymbolTemplateFilename; private final Optional entrypointBzlFile; private final String footerTemplateFilename; private final Stamping stamping; @@ -75,6 +77,7 @@ public MarkdownRenderer( String aspectTemplate, String repositoryRuleTemplate, String moduleExtensionTemplate, + String starlarkOtherSymbolTemplate, Optional entrypointBzlFile, String footerTemplate, Stamping stamping) { @@ -87,6 +90,7 @@ public MarkdownRenderer( this.aspectTemplateFilename = aspectTemplate; this.repositoryRuleTemplateFilename = repositoryRuleTemplate; this.moduleExtensionTemplateFilename = moduleExtensionTemplate; + this.starlarkOtherSymbolTemplateFilename = starlarkOtherSymbolTemplate; this.entrypointBzlFile = entrypointBzlFile; this.footerTemplateFilename = footerTemplate; this.stamping = stamping; @@ -124,7 +128,8 @@ public String renderTableOfContents( List starlarkFunctions, List aspectInfos, List repositoryRuleInfos, - List moduleExtensionInfos) + List moduleExtensionInfos, + List starlarkOtherSymbolInfos) throws IOException { ImmutableMap vars = @@ -136,7 +141,8 @@ public String renderTableOfContents( "functionInfos", starlarkFunctions, "aspectInfos", aspectInfos, "repositoryRuleInfos", repositoryRuleInfos, - "moduleExtensionInfos", moduleExtensionInfos); + "moduleExtensionInfos", moduleExtensionInfos, + "starlarkOtherSymbolInfos", starlarkOtherSymbolInfos); Reader reader = readerFromPath(tableOfContentsTemplateFilename); try { return Template.parseFrom(reader).evaluate(vars); @@ -348,6 +354,22 @@ public String render(ModuleExtensionInfo moduleExtensionInfo) throws IOException } } + /** Returns a markdown rendering of a documented Starlark global symbol. */ + public String render(StarlarkOtherSymbolInfo starlarkOtherSymbolInfo) throws IOException { + ImmutableMap vars = + ImmutableMap.of( + "util", + new MarkdownUtil(entrypointBzlFile), + "symbolInfo", + starlarkOtherSymbolInfo); + Reader reader = readerFromPath(starlarkOtherSymbolTemplateFilename); + try { + return Template.parseFrom(reader).evaluate(vars); + } catch (ParseException | EvaluationException e) { + throw new IOException(e); + } + } + /** Returns a markdown header string that should appear at the end of Stardoc's output. */ public String renderMarkdownFooter(ModuleInfo moduleInfo) throws IOException { ImmutableMap vars = diff --git a/stardoc/html_tables_stardoc.bzl b/stardoc/html_tables_stardoc.bzl index e2487a5c..7bedfdb8 100644 --- a/stardoc/html_tables_stardoc.bzl +++ b/stardoc/html_tables_stardoc.bzl @@ -35,5 +35,6 @@ def html_tables_stardoc(name, **kwargs): rule_template = Label("//stardoc:templates/html_tables/rule.vm"), repository_rule_template = Label("//stardoc:templates/html_tables/repository_rule.vm"), module_extension_template = Label("//stardoc:templates/html_tables/module_extension.vm"), + starlark_other_symbol_template = Label("//stardoc:templates/html_tables/starlark_other_symbol.vm"), **kwargs ) diff --git a/stardoc/private/stardoc.bzl b/stardoc/private/stardoc.bzl index 0c80b0f0..3bc5a9eb 100644 --- a/stardoc/private/stardoc.bzl +++ b/stardoc/private/stardoc.bzl @@ -42,6 +42,7 @@ def _renderer_action_run(ctx, out_file, proto_file): renderer_args.add("--rule_template=" + str(ctx.file.rule_template.path)) renderer_args.add("--repository_rule_template=" + str(ctx.file.repository_rule_template.path)) renderer_args.add("--module_extension_template=" + str(ctx.file.module_extension_template.path)) + renderer_args.add("--starlark_other_symbol_template=" + str(ctx.file.starlark_other_symbol_template.path)) if ctx.file.footer_template: renderer_args.add("--footer_template=" + str(ctx.file.footer_template.path)) if stamp_enabled: @@ -58,6 +59,7 @@ def _renderer_action_run(ctx, out_file, proto_file): ctx.file.rule_template, ctx.file.repository_rule_template, ctx.file.module_extension_template, + ctx.file.starlark_other_symbol_template, ] if ctx.attr.table_of_contents_template: inputs.append(ctx.file.table_of_contents_template) @@ -138,6 +140,11 @@ _common_renderer_attrs = { allow_single_file = [".vm"], mandatory = True, ), + "starlark_other_symbol_template": attr.label( + doc = "The input file template for generating documentation of Starlark global symbols.", + allow_single_file = [".vm"], + mandatory = True, + ), "footer_template": attr.label( doc = "The input file template for generating the footer of the output documentation. Optional.", allow_single_file = [".vm"], @@ -190,7 +197,7 @@ _stardoc_markdown_renderer_attrs = { stardoc_markdown_renderer = rule( _stardoc_markdown_renderer_impl, doc = """ -Generates markdown documentation for starlark rule definitions from the corresponding binary proto. +Generates Markdown documentation for Starlark symbols from the corresponding binary proto. """, attrs = _stardoc_markdown_renderer_attrs, ) diff --git a/stardoc/stardoc.bzl b/stardoc/stardoc.bzl index 67dbb6da..c0ce6e7b 100644 --- a/stardoc/stardoc.bzl +++ b/stardoc/stardoc.bzl @@ -35,11 +35,12 @@ def stardoc( rule_template = Label("//stardoc:templates/markdown_tables/rule.vm"), repository_rule_template = Label("//stardoc:templates/markdown_tables/repository_rule.vm"), module_extension_template = Label("//stardoc:templates/markdown_tables/module_extension.vm"), + starlark_other_symbol_template = Label("//stardoc:templates/markdown_tables/starlark_other_symbol.vm"), footer_template = None, render_main_repo_name = True, stamp = -1, **kwargs): - """Generates documentation for exported starlark rule definitions in a target starlark file. + """Generates documentation for exported symbols in a target Starlark file. Args: name: The name of the stardoc target. @@ -47,9 +48,8 @@ def stardoc( out: The file to which documentation will be output (mandatory). deps: A list of bzl_library dependencies which the input depends on. format: The format of the output file. Valid values: 'markdown' or 'proto'. - symbol_names: A list of symbol names to generate documentation for. These should correspond to the names of rule - definitions in the input file. If this list is empty, then documentation for all exported rule definitions will - be generated. + symbol_names: A list of exported symbol names to generate documentation for. If this list is + empty, documentation for all exported symbols will be generated. renderer: The location of the renderer tool. aspect_template: The input file template for generating documentation of aspects header_template: The input file template for the header of the output documentation. @@ -62,6 +62,8 @@ def stardoc( rule_template: The input file template for generating documentation of rules. repository_rule_template: The input file template for generating documentation of repository rules. module_extension_template: The input file template for generating documentation of module extensions. + starlark_other_symbol_template: The input file template for generating documentation of + Starlark global symbols, such as constants, which use `#:` doc comments. footer_template: The input file template for generating the footer of the output documentation. Optional. render_main_repo_name: Render labels in the main repository with a repo component (either the module name or workspace name). @@ -132,6 +134,7 @@ def stardoc( rule_template = rule_template, repository_rule_template = repository_rule_template, module_extension_template = module_extension_template, + starlark_other_symbol_template = starlark_other_symbol_template, macro_template = macro_template, footer_template = footer_template, stamp = stamp, diff --git a/stardoc/templates/html_tables/starlark_other_symbol.vm b/stardoc/templates/html_tables/starlark_other_symbol.vm new file mode 100644 index 00000000..64df0b81 --- /dev/null +++ b/stardoc/templates/html_tables/starlark_other_symbol.vm @@ -0,0 +1,9 @@ + + +#[[##]]# ${symbolInfo.name} (`${symbolInfo.typeName}`) + +
+${util.loadStatement($symbolInfo.name)}
+
+ +${util.htmlEscape($symbolInfo.doc)} diff --git a/stardoc/templates/markdown_tables/starlark_other_symbol.vm b/stardoc/templates/markdown_tables/starlark_other_symbol.vm new file mode 100644 index 00000000..e6f9a4ec --- /dev/null +++ b/stardoc/templates/markdown_tables/starlark_other_symbol.vm @@ -0,0 +1,9 @@ + + +#[[##]]# ${symbolInfo.name} (`${symbolInfo.typeName}`) + +
+${util.loadStatement($symbolInfo.name)}
+
+ +${symbolInfo.doc} diff --git a/stardoc/templates/markdown_tables/table_of_contents.vm b/stardoc/templates/markdown_tables/table_of_contents.vm index c5dfd8b4..43f85a3f 100644 --- a/stardoc/templates/markdown_tables/table_of_contents.vm +++ b/stardoc/templates/markdown_tables/table_of_contents.vm @@ -55,3 +55,11 @@ #end #end +#if (!$starlarkOtherSymbolInfos.isEmpty()) +#[[##]]# Globals + +#foreach ($symbolInfo in $starlarkOtherSymbolInfos) +- [$symbolInfo.name](#$symbolInfo.name) +#end + +#end diff --git a/test/BUILD b/test/BUILD index 847d8105..b4e796d3 100644 --- a/test/BUILD +++ b/test/BUILD @@ -54,6 +54,27 @@ stardoc_test( symbol_names = ["my_rule"], ) +stardoc_test( + name = "starlark_other_symbols_test", + golden_file = "testdata/starlark_other_symbols_test/golden.md", + input_file = "testdata/starlark_other_symbols_test/input.bzl", + tags = [ + "bazel_9", + "manual", + ], +) + +stardoc_test( + name = "starlark_other_symbols_html_tables_test", + golden_file = "testdata/starlark_other_symbols_test/html_tables_golden.md", + input_file = "testdata/starlark_other_symbols_test/input.bzl", + tags = [ + "bazel_9", + "manual", + ], + test = "html_tables", +) + stardoc_test( name = "repo_rules_test", golden_file = "testdata/repo_rules_test/golden.md", diff --git a/test/testdata/starlark_other_symbols_test/golden.md b/test/testdata/starlark_other_symbols_test/golden.md new file mode 100644 index 00000000..a7faa1a2 --- /dev/null +++ b/test/testdata/starlark_other_symbols_test/golden.md @@ -0,0 +1,38 @@ + + +A golden test for documented Starlark global symbols. + + + +## A_STRING (`string`) + +
+load("@stardoc//test:testdata/starlark_other_symbols_test/input.bzl", "A_STRING")
+
+ +A documented string constant. + + + + +## STRUCT_CONSTANT (`struct`) + +
+load("@stardoc//test:testdata/starlark_other_symbols_test/input.bzl", "STRUCT_CONSTANT")
+
+ +A documented struct constant. + + + + +## Z_LIST (`list`) + +
+load("@stardoc//test:testdata/starlark_other_symbols_test/input.bzl", "Z_LIST")
+
+ +The first line of documentation for a list. +The second line of documentation for a list. + + diff --git a/test/testdata/starlark_other_symbols_test/html_tables_golden.md b/test/testdata/starlark_other_symbols_test/html_tables_golden.md new file mode 100644 index 00000000..a7faa1a2 --- /dev/null +++ b/test/testdata/starlark_other_symbols_test/html_tables_golden.md @@ -0,0 +1,38 @@ + + +A golden test for documented Starlark global symbols. + + + +## A_STRING (`string`) + +
+load("@stardoc//test:testdata/starlark_other_symbols_test/input.bzl", "A_STRING")
+
+ +A documented string constant. + + + + +## STRUCT_CONSTANT (`struct`) + +
+load("@stardoc//test:testdata/starlark_other_symbols_test/input.bzl", "STRUCT_CONSTANT")
+
+ +A documented struct constant. + + + + +## Z_LIST (`list`) + +
+load("@stardoc//test:testdata/starlark_other_symbols_test/input.bzl", "Z_LIST")
+
+ +The first line of documentation for a list. +The second line of documentation for a list. + + diff --git a/test/testdata/starlark_other_symbols_test/input.bzl b/test/testdata/starlark_other_symbols_test/input.bzl new file mode 100644 index 00000000..528cc166 --- /dev/null +++ b/test/testdata/starlark_other_symbols_test/input.bzl @@ -0,0 +1,17 @@ +"""A golden test for documented Starlark global symbols.""" + +#: A documented string constant. +A_STRING = "a string" + +#: A documented struct constant. +STRUCT_CONSTANT = struct(foo = 1) + +#: The first line of documentation for a list. +#: The second line of documentation for a list. +Z_LIST = ["one", "two"] + +#: Private symbols are not included in generated documentation. +_PRIVATE_CONSTANT = 42 + +# Undocumented constant +UNDOCUMENTED_CONSTANT = _PRIVATE_CONSTANT == 42 diff --git a/test/testdata/table_of_contents_test/golden.md b/test/testdata/table_of_contents_test/golden.md index a999b392..d4d48ecf 100644 --- a/test/testdata/table_of_contents_test/golden.md +++ b/test/testdata/table_of_contents_test/golden.md @@ -34,6 +34,10 @@ Test rules / providers / etc for the table of contents generation test. - [my_ext](#my_ext) +## Globals + +- [MY_GLOBAL](#MY_GLOBAL) + @@ -305,3 +309,14 @@ Artifact tag | group | Group name | String | optional | `"my_group"` | + + +## MY_GLOBAL (`string`) + +
+load("@stardoc//test:testdata/table_of_contents_test/input.bzl", "MY_GLOBAL")
+
+ +A documented global string. + + diff --git a/test/testdata/table_of_contents_test/input.bzl b/test/testdata/table_of_contents_test/input.bzl index f88f55c3..25b421a9 100644 --- a/test/testdata/table_of_contents_test/input.bzl +++ b/test/testdata/table_of_contents_test/input.bzl @@ -27,3 +27,6 @@ my_repo = _my_repo my_ext = _my_ext basic_macro = _basic_macro + +#: A documented global string. +MY_GLOBAL = "my global"