Layer name with special characters causes problems in search

[WilliamQue]

I created a new custom layer and it's feature , then renamed the layer name and feature name to Chinese, according to document, the operation is allowed.

After annotation, I clicked the calculate button to get statistics, system reported:
Error: Unable to parse query [<意向.意向特征=""/>]. This can be caused by an invalid feature name.

Please complete the following information:

• Version and build ID: 22.1
• OS: Windows
• Browser: Chrome

[jcklie]

@Falko1 Does it look like a feature name that is supported by our search?

[reckart]

I am not surprised that the search does not support non-ASCII characters. But yes, the index currently uses the "display name" when indexing with the idea that it would be more intuitive and more easily accessible to users. But it might be a better idea to switch to using the "internal name" instead - that is the initial name of the feature which has a restricted set of acceptable characters.

[WilliamQue]

I think it's not necessary to start a new issue for asking a quick question, when exporting a project and choosing inline xml as second format, what's encoding of the xml file in annotation folder in the zip file? Garbled characters appear, replacing non-ASCII item name of tagset.

[Falko1]

@reckart already pointed out the main problem. Under the hood, to compute statistics we employ searches like
<layer_name.feature_name=""/>
This search fails if feature_name or layer_name are not valid (as you can check yourself by entering <意向.意向特征=""/> into the search sidebar). This is also pointed out in
https://inception-project.github.io/releases/22.2/docs/user-guide.html#_high_level_statistics_sidebar
where it states that "Please make sure that the name of the layer/feature combinations are valid (e.g. they don’t contain incorrect bracketing)." The validity of custom layer_name and feature_name is checked at their creation. Of course, you can rename your custom layer/feature as you want, but if the initial creation fails then the statistics and layer-specific search won't work.

To fix this, we have to make profound changes in the code. Either, as @reckart pointed out, by using the internal name for the index or by using MTAS facets. The latter approach will only affect the statistics section, whereas the first approach will also apply to search.