Add open_file_skipping_BOM function to RC::File

[narknon]

No description provided.

[UE4SS]

Please run clang format.

[UE4SS]

I'm not going to be able to properly review this because I don't know enough about the operations involved in what this PR is trying to accomplish.

[narknon]

The build script was using utf8-sig, which adds the BOM signature to the start of text files. We need to be able to ignore that when parsing mods.txt

[UE4SS]

The build script was using utf8-sig, which adds the BOM signature to the start of text files. We need to be able to ignore that when parsing mods.txt

Yes I know why this is needed, but what I meant was, I'm not qualified to say whether your code is correct and good.

[narknon]

I'm open to any code that resolves this issue. I imagine there are likely better ways. Theoretically this method is temporarily needed for a release cycle

[Yangff]

wfile.imbue should be set for utf8 case instead of utf16..? because handling utf8 with wstring can cause problem .. it will just pack two utf8 chars as if it's utf16...

but if I understand correctly, all UE4SS files should be utf8, so I really don't know why RC::File is using wstring.. (and that's why I'm trying to make all of them utf8 when porting it..)

[bitonality]

Have we considered taking a 3rd party library dep for this use case?

I think it's awesome that we're aiming to stay compatible with the utf8 BOM files we've released, but we also have an opportunity to make our file parsing more robust. Since end users are able to modify config, maybe we should extend to support both utf8 and utf16?

[narknon]

The third party dep for mods.txt is glz. This is merely a required functionality to maintain backwards compatibility, and this PR shouldn't expand in scope. If someone wants to make a bigger PR with other file parsing functionality they can

For some reason, its ignoring the fact that all my questions were resolved.

[igromanru]

What is the state here? It's quite annoying that the first line is still ignored.
I wanted to test the changes, but open_file_skip_BOM is not called anywhere.

[UE4SS]

What is the state here? It's quite annoying that the first line is still ignored. I wanted to test the changes, but open_file_skip_BOM is not called anywhere.

It's used in #540

[igromanru]

The other PR is huge, can't we get this change tested and merged first?
Who knows when #540 will be done.

[UE4SS]

What is the state here? It's quite annoying that the first line is still ignored. I wanted to test the changes, but open_file_skip_BOM is not called anywhere.

It's used in #540

The other PR is huge, can't we get this change tested and merged first? Who knows when #540 will be done.

Yes, and in fact we have to because #540 is dependent on this PR.

[narknon]

This method won't be used. 540 needs to be rebased to use Yang's method, so this can be closed.