allow setting `maxExpansionDepth` for XMLParser to avoid `Entity expansion limit exceeded`

[eleith]

after updating to 5.9.0 (and even before, because xml-fast-parser released a breaking change to a minor patch) i'm hitting Entity expansion limit exceeded quite often while using getDirectoryContents with a few hundred objects (not using deep)

this was not the case beforehand because xml-fast-parser defaulted maxTotalExpansions to Infinity. however, now, it is set to 1000.

this conversation upstream is evidence that this error is surprising many others.

webdav users could use a way to configure maxTotalExpansions so they can avoid getting blocked on directory listings.

perhaps an option on WebDAVClientOptions that has a sensible default would go far enough for most people?

[amitguptagwl]

I've set it to infinity as default now.

[Choppel]

@perry-mitchell Setting the default to Infinity by amitguptagwl doesn't help as for this to take effect users of webdav and webdav-fs would still need to control the limits of processEntities. Is there an ETA of this? As for now my app does not work at all.

[amitguptagwl]

@perry-mitchell As your users need to set the limits to Infinity and there is no way to pass the configuration, you can update following files to increase the limit if you want.

• source/tools/xml.ts
• source/tools/dav.ts

configuration the you would have to add is

processEntities: {
          maxTotalExpansions: Infinity,
          maxEntityCount: Infinity
}

Please take decision wisely as Infinity is not recommended option for safety reasons.

[perry-mitchell]

Wow, thanks for dropping by @amitguptagwl! Quite an honour having you here 😄

I'm considering the following:

• Exposing the configuration of processEntities, so consumers of webdav can configure this how they like for their environment.
• Setting the defaults to Infinity for now. I'd consider decreasing this for a future major release so as to steer the library towards a more security-focused configuration, but keeping it as Infinity for now would probably break less dependent applications.

What do you think?

[perry-mitchell]

Merged #408 today to get fast-xml-parser to the latest version in terms of testing/parsing. It had some breaking changes that have now been handled.

I'll make another PR tomorrow for the processEntities handling.

[amitguptagwl]

Thanks for your kind response @perry-mitchell . It would be better if you can wait for a day. I'm working on a feature of this and fixiing on some performance issues. So you will not need changes 2 times.

[perry-mitchell]

Thanks @amitguptagwl - I can hold off until tomorrow. I think that due to the vulnerabilities users will be attempting to patch the package and will encounter errors in doing so; I think I'll need to put out a patch latest this weekend.

[amitguptagwl]

I've just release v5.7.0.
In this release, a new property has been exposed entityDecoder. This allows user to control limits for expansion (not doctype reading, only entity expansion), NCR entity based on XML version, controlling HTMLentities etc.

Based on your requirement, you can either expose this or create a wrapper on it to ensure that user is setting only what you app allows.

Here, you can find the documentaion of EntityDecoder which can be set as an option. When it is set externally, it improve the performance too.

[perry-mitchell]

Thanks @amitguptagwl! I'll do my best to get an updated version out this weekend.