Skip to content

Failing to return some nested data within content types #16

Closed

Description

@IanBoyte

We have a somewhat complex component library with fairly heavy levels of nesting. We're in active development and have recently seen fields not being returned by queries using populate=all. Components that previously returned their full nested dataset are not

Reviewing the plugin readme, you state it builds a standard Strapi populate query via middleware. Is it possible that a sufficiently complex component structure might be hitting the url limits?

Activity

  1. faessler commented on Nov 3, 2025

    @faessler
    Owner

    Hello @IanBoyte 👋 Thx for your bug report.

    Url limits should not apply here because it creates a populate query inside the node env and doesn't pass them through any url.

    There was another bug which is fixed now (#17) that might be related to yours. 🤔 Could you try if v1.2.5 fixes your issue?

    If not can you provide more details? What kind of field (relations, components, dz, string, etc.)? How does your component library schema look like? Or could you provide some kind of reproduction steps? 🙏

  2. self-assigned this
    on Nov 3, 2025
  3. IanBoyte commented on Nov 13, 2025

    @IanBoyte
    Author

    Thanks, I've swapped the plugin back and am having my team test.

  4. IanBoyte commented on Nov 13, 2025

    @IanBoyte
    Author

    @faessler Just realized part of the issue I was having was from copying the example usage:

    module.exports = {
      "populate-all": {
        enabled: true,
        config: {
          cache: true,
          realtions: true,
        },
      },
    };
    

    relations is misspelled.

  5. faessler commented on Nov 13, 2025

    @faessler
    Owner

    Oh my 🤦🏼‍♂️ Thanks for the report. Gonna fix the docs today.

    Also just a fyi the cache: boolean config option hasn‘t been released yet. I need to write some tests.

  6. added a commit that references this issue on Nov 13, 2025
  7. github-actions commented on Nov 13, 2025

    @github-actions
  8. IanBoyte commented on Nov 13, 2025

    @IanBoyte
    Author

    Unfortunately, during testing, it looks like some of our more deeply nested components still aren't being returned in the data response. All we're getting back is that component id. I did try both with and without the caching config set (in case it's implemented in some way).

  9. IanBoyte commented on Nov 13, 2025

    @IanBoyte
    Author

    Some more detail on our schemas. We build with a lot of reuse with an atomic structure. The example I'm seeing the issue on presently is similar to the following:

    Our relevant content structures:
    Content type: page
    Module: contentBlock
    Molecule: commonModuleHeader
    Molecule: buttonGroup
    Atom: button
    Atom: internalLink

    Our content schema would be similar to the following:

    page
    > dynamicZone
    ...
      > contentBlock
        > commonModuleHeader
          > buttonGroup
            > button
              > internalLink
                > relationship to another page
    ...
    
    

    What we would see returned, for example, is

    { page
     ...
     contentBlock {
        commonModuleHeader {
          buttonGroup {
            id: 123
          }
        }
      }
    ...
    }
    
  10. faessler commented on Nov 14, 2025

    @faessler
    Owner

    Thanks for the schema! I rebuilt it in the sandbox. Unlike you I got all the data I'd expect. 🤔

    plugin schema permissions collection view document view result
    Image Image Image Image Image Image

    Followup questions:

    • Do you have the correct permissions set?
    • Is the page you linked in draft and thus cannot be loaded? (add &status=draft)
    • Do other components work? Do other pages work?
    • Which version of Strapi are you using?
  11. IanBoyte commented on Nov 14, 2025

    @IanBoyte
    Author

    "@strapi/strapi": "^5.29.0"

    All pages being tested are published.

    We're using a full access token and I've checked all permissions.

    I suspect part of the issue might be due to the size/complexity of our full schema. We didn't start noticing issues until fairly late in development.

    Other components and content types have similar issues. We've seen nested assets not having the entire asset object being returned, we've tested the buttons where they're used in standalone areas, and the button data might return, but not the nested page_link.

    Ex. we use this masthead component on all pages, and on some content types, the image isn't being returned.

    "masthead": {
          "id": 148,
          "masthead_type": "basic",
          "subhead": null,
          "description": null
        },
    

    vs

    "masthead": {
          "id": 209,
          "masthead_type": "basic",
          "subhead": null,
          "description": null,
          "media": {
            "id": 1031,
            "type": "image",
            "image": {
              ... // all asset data being returned here
            },
            "video": null
          },
          "logo": null,
          "title": {
            "id": 1625,
            "Text": "Module Test 3",
            "Tag": "h2"
          },
          "buttons": null
        },
    
  12. IanBoyte commented on Nov 14, 2025

    @IanBoyte
    Author

    If it's helpful, let me know and I can provide a link to a zip of the full schema.

  13. zhawzid-ottj commented on Dec 9, 2025

    @zhawzid-ottj

    The problem for some cases (sorts of nested containers scenarios)
    is the recently added "prevent infinite loop" check.

    You can also implement a max depth guard like this:

    export const getPopulateQuery = (
      modelUid: UID.Schema,
      parentsModelUids: UID.Schema[] = [],
      depth: number = 0
    ): { populate: object | true } | undefined => {
    ...
    

    and add depth+1 to every recursive call like this (example):

    query.populate[fieldName] = getPopulateQuery(
      attribute.component,
      parentsModelUids,
      depth + 1
    );
    

    then tweak this:

    // prevent infinite loop
    //    if (parentsModelUids.includes(modelUid)) {
    if (depth > 5) {
      strapi.log.debug(
        `[populate-all] max depth detected skipping population: ${modelUid}`
      );
      return { populate: {} };
    } else {
      parentsModelUids.push(modelUid);
    }
    

    This isn't too efficient but it enables some nice scenarios which aren't possible with the current implementation where every entiy can only occur once, otherwise nested entities won't be populated.

    I would let this switch in the config by two separates settings, where you have to enable either one or the other way.

    Cheers

  14. added
    bugSomething isn't working
    and removed on Apr 16, 2026
  15. faessler commented on Apr 16, 2026

    @faessler
    Owner
  16. zhawzid-ottj commented on Apr 18, 2026

    @zhawzid-ottj

    Although the fix is appreciated (as it fixes issues), this still doesn't work for some cases.

    A recursion can't be detected properly by looking at the entity type only.
    A recursion occurs when the same entity instance (same documentID, id) repeats somewhere as a child.
    Then the endless loop happens.

     // prevent infinite loop
        if (parentsModelUids.includes(modelUid)) {
          strapi.log.debug(
            `[populate-all] loop detected skipping population: ${modelUid}`
          );
          return { populate: {} };
        }
    

    So the depth limiter from above is still a thing, at least for my uses cases.
    I appreciate the quick compatibility fix for recent strapi versions, thank you!

  17. zhawzid-ottj commented on Apr 20, 2026

    @zhawzid-ottj

    This is what Claude comes up with:

    Your algorithm is actually correct — parentsModelUids tracks the exact DFS call stack, which is the standard way to detect directed cycles. The same type appearing in different branches (not ancestor→descendant) won't be blocked.

    There are two real improvements worth making:

    1. Replace binary include-check with occurrence count — this lets self-referential schemas (category trees, nested menus) populate N levels deep instead of hard-stopping at 1:
    // Instead of:
    if (parentsModelUids.includes(modelUid)) {
      return { populate: {} };
    }
    
    // Use:
    const maxRecursion = strapi.plugin("populate-all").config<number>("maxRecursion") ?? 1;
    const occurrences = parentsModelUids.filter(uid => uid === modelUid).length;
    if (occurrences >= maxRecursion) {
      strapi.log.debug(`[populate-all] max recursion (${maxRecursion}) reached: ${modelUid}`);
      return { populate: {} };
    }
    
    1. Sort the cache key ancestors — the result only depends on the set of ancestors (not their order), so two calls with the same ancestors in different order produce identical queries but currently miss the cache:
    const buildCacheKey = (modelUid: UID.Schema, parentsModelUids: UID.Schema[]): string => {
      if (parentsModelUids.length === 0) return modelUid;
      return `${[...parentsModelUids].sort().join("|")}|${modelUid}`;
    };
    

    can you explain part 2 using an example?

    Sure. You need a schema with mutual references to hit this case — say two components that can contain each other:

    Page → CompA, CompB
    CompA → CompB (nested)
    CompB → CompA (nested, mutual)
    Both → Media
    

    When building Page's populate, Media gets called twice — once via each path:

    Path parentsModelUids passed to getPopulateQuery('Media', ...)

    Page → CompA → CompB → Media	['Page', 'CompA', 'CompB']
    Page → CompB → CompA → Media	['Page', 'CompB', 'CompA']
    

    Current cache keys (joined in call order):

    'Page|CompA|CompB|Media'   ← first call, computed and cached
    'Page|CompB|CompA|Media'   ← second call, cache MISS — recomputed
    

    With sorted key:

    'CompA|CompB|Media|Page'   ← first call, computed and cached
    'CompA|CompB|Media|Page'   ← second call, cache HIT ✓
    

    Both calls produce identical queries because cycle detection only cares about which ancestors are blocked, not the order you encountered them. The result for Media under {Page, CompA, CompB} is the same regardless of traversal order.

    The impact is minor in small schemas but grows with mutual references and dynamic zones that include overlapping component sets.

  18. faessler commented on Jun 17, 2026

    @faessler
    Owner

    Defining reoccurrence level per model is now possible with new version 1.6.0 .

    You can extend config like this:

    module.exports = {
      "populate-all": {
        enabled: true,
        config: {
          recursion: {
            "api::article.article": 2,
          },
        },
      },
    };
    
    

    See #44 for more details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingenhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions