Why does navigation.pages not return full page objects?

[Andre601]

I find it strange that {{ navigation.pages }} only returns a reduced/stripped down version of the Page object, only containing the title (Which most of the time seems to be blank) and the URL, but lacking crucial parts such as the Page meta or even the page content which MkDocs is including in the Page object.

Why is that so? Can this be changed to be more in line with MkDocs' nav jinja2 variable, which contains all the Page's content?

This issue makes it impossible for me to make a macro through a file that would list pages inside a specific folder, which I'm much more used to than making it in Python.

[Andre601]

I also would like to point out that the {{ macros_info() }} macro mentions this code snippet to list titles of all pages:

{% for page in navigation.pages %}
- {{ page.title }}
{% endfor %}

However, only index pages (Those named index.md) seem to have a valid title set, with every other page having [blank] as title.
This could be a (unintentional) side effect of Material for MkDocs' index page feature, or maybe related to the Name set in the nav configuration setting of MkDocs...
Either way, only index pages of a section contain a title with any other not having any defined...

Here is an example of above code snippet in my documentation:

All pages have a H1 header and no title metadata defined. Strangely enough "Twitch Commands" is not an index page (File is named twitch.md), yet has a title, even tho no title metadata is present here...

This is a rather strange behaviour that makes no logical sense for me right now.

[fralau]

The structure of the navigation object is a little beyond my knowledge. What I know could be discovered by exploring the content or doc. If that doesn't answer the question, the best would be to forward that question to the MkDocs team.

[Andre601]

Perhaps the issue is that the plugin either pulls the wrong nav object (One only containing shallow page objects needed for the navigation itself) or maybe at the wrong time (By the looks of it are you setting it during on_nav. Maybe there it only has shallow Page objects?)

I'll make a discussion on the mkdocs repo to get some context, but I wouldn't be surprised if you use the wrong navigation object here.

[fralau]

I see that it's indeed called by on_nav:
https://github.com/fralau/mkdocs_macros_plugin/blob/f07146bbc3d317db38b903c4c282315682ccfd31/mkdocs_macros/plugin.py#L663

I am not sure how how the nav object could be wrong, since it is an argument of the hook function. But perhaps it gets updated after that?

[Andre601]

I don't know... I quickly tried this here to get a populated Navigation instance and it still only gives a collection of Page objects with title and url, like the navigation variable does already.

from mkdocs.structure.files import get_files
from mkdocs.structure.nav import get_navigation

def define_env(env):

    @env.macro
    def get_nav():
        files = get_files(env.conf)
        nav = get_navigation(files, env.conf)
        
        return nav

So my guess remains that the Navigation object either doesn't have the full Page objects at the time it is called by the plugin (And by me during on_env which iirc is during/after on_config, which probs is too early to retrieve the full Page objects list.) or that MkDocs defines a different Navigation object for their jinja2 templating that contains pages with full context.

Not sure if I could move this to one of the other methods offered by the plugin... I would need it before the file macros (or any macro for that matter) are rendered.

[fralau]

It could be that the content is initialized or not according to whether that title was explicitly defined or not in the config file or the page's header; if it wasn't (typically, it's the heading of level 1) then it might be empty.

[Andre601]

The only pattern I can see is related to the visible navigation on the site itself...

Like the only pages with a title are:

• An index.md page
• Having a title meta defined
• Are the only non-index page in the nav section

Not sure what logic is happening here...

[fralau]

I believe that's the logic? 💡

Strictly speaking, other pages do not have a title. Whatever MkDocs decides to consider as title by default is a matter of debate. One thing that could be done (unless we find a variable where that final info is actually stored) would be to emulate the decision process and fill blanks?

Once again, I believe those who have the answer are the MkDocs teams.

[alanpt]

Hi, I am having the same issue too.

(All the None work as links just not title properly and it isn't showing all the tags. In fact it looks like they only use the tag once, and don't show them for following pages. i'e multiple pages are tagged communication, but only one gets tagged.

Using this code:

{% for page in navigation.pages %}
- [{{ page.title }}](../../{{ page.url }}){% if page.meta.tags %}: Tags - 
  {% for tag in page.meta.tags %}{{ tag }}{% if not loop.last %}, {% endif %}{% endfor %}
{% else %}: No Tags
{% endif %}
{% endfor %}

Moving the plugin up and down the plugin list does change how many get shown, so it seems to me a process gets cut short before it finishes.

I'm aiming to pull a list of pages with a specific tag, but for now just getting a list of all the pages with tags is causing me grief.

[fralau]

Note that, as a workaround, you should have at least the page filename (which is a title), and from which you could easily extract the first header (if all else fails).

[angeld-jr2]

The same problem. I made Bad macro response. title=[blank] only in the root folder #268 including a complete test case.

[fralau]

@lovelydinosaur Perhaps (from your expertise of MkDocs) you would have an advice to give in this case?

• In MkDocs-Macros, the environment is copied in the on_config event, at a moment where the pages are not yet rendered.
• The nav information is re-copied in the on_nav event, but this does not answer the need.
• Is there something that could be done, e.g. at the level of the macro (Python code) itself? Note that the macros (Python functions) are actually executed in the on_page_markdown event.

[reidpr]

I ran into this too. I think the underlying issue is what @fralau said in #156, i.e., (Mk|Proper)Docs doesn’t even know the full navigation tree until all the pages are rendered (see diagram), and then MkDocs-Macros is finished and gone. (Navigation menus and the like have the full nav tree because they’re computed after all pages are rendered.)

That is, given the current (Mk|ProperDocs) rendering flow, no plugin operating in the populate_page stage, as MkDocs-Macros does, has reliable access to full page objects without reaching outside the that rendering flow.

This is the workaround I used, which does a duplicate parse (in some sense a “second pass”) to populate the title:

def define_env(env):

   @env.macro
   def entitle(nav):
      """MkDocs-Macros provides a handy `navigation` variable in its Jinja
         context, which is the (Mk|Proper)Docs Navigation object [1], but many
         of its fields are incomplete when the macros are being processed,
         e.g. `title` [2,3]. This is because (Mk|Proper)Docs does not fully
         compute page titles until they are rendered, i.e., the navigation
         tree is simply not fully known until all the pages are rendered and
         MkDocs-Macros is finished and gone.

         Further, when the define_env() hook is called, the navigation
         structure does not contain Page objects but rather just filenames.

         To work around these timing problems, this macro parses page Markdown
         sufficiently to populate titles and sets appropriate fields in the
         Page objects [5]. One could consider the approach a fairly obnoxious
         monkey-patch, but it works for us and does not require modifying the
         render flow. Call it before you need to use `navigation`. This macro
         produces no useful output, only side effects.

         [1]: https://properdocs.org/dev-guide/themes/#navigation-objects
         [2]: https://github.com/fralau/mkdocs-macros-plugin/discussions/198
         [3]: https://github.com/fralau/mkdocs-macros-plugin/issues/156
         [4]: https://properdocs.org/dev-guide/plugins/#events
         [5]: https://github.com/ProperDocs/properdocs/blob/v1.6.7properdocs/structure/pages.py#L227"""
      for i in nav:
         # Populate title with quick-and-dirty parse. Don’t save the results
         # into the Page object because I didn’t want to figure out how to do
         # it right, and it’s the wrong time in the render pipeline anyway.
         if (hasattr(i, "file") and not i._title_from_render):
            LOG.info("fake parse: %s/%s" % (i.file.src_dir, i.file.src_uri))
            md = markdown.Markdown()
            title_prepr = mkdocs.structure.pages._ExtractTitleTreeprocessor()
            title_prepr._register(md)
            md.convert(i.file.content_string)  # discard HTML return
            i._title_from_render = title_prepr.title
            # Page.title() will only use Page._title_from_render if this is
            # non-None, so fool it. This will be overwritten when the proper
            # value is actually read.
            i.markdown = True
         # Recurse if any children.
         if (i.children):
            entitle(i.children)
      return "<!-- entitle() fake-parsing was called here -->"

Once you call it with {{entitle(navigation))}}, titles from any of the four sources are available within the page being rendered.

[fralau]

Thank for that solution!

@Andre601, @alanpt, @angeld-jr2: Does that help solve your issue?