Improving RTON format and hopefully some other issues

[nineteendo]

Why would I propose this?

I recently finished 1Zulu's RTONParser and a tool for the reverse written in Python and it's working very well: https://github.com/Nineteendo/PVZ2tools/tree/master/RTONConverter

Format

I suggest a more logical format (but I'm not sure if -uRTON_int32_t is correct):

Bytecode	Type	Limitations
0x0	false / RTID()	False / must be used after 0x83
0x1	true	True
0x2	RTID(d.d.hhhhhhhh@x)	must be used after 0x83
0x3	RTID(x@x)	must be used after 0x83
0x8	int8_t	-128 to 127
0x9	int8_t_zero	0
0xa	uint8_t	0 - 255
0xb	uint8_t_zero	0
0x10	int16_t	-32768 to 32767
0x11	int16_t_zero	0
0x12	uint16_t	0 to 65535
0x13	uint16_t_zero	0
0x20	int32_t	-2147483648 to 2147483647
0x21	int32_t_zero	0
0x22	float	Single-precision floating-point
0x23	float_zero	0.0
0x24	uvarint32_t	0 to 2147483647
0x25	varint32_t	-2147483648 to 2147483647
0x26	uint32_t	0 to 4294967295
0x27	uint32_t_zero	0
0x28	uvaruint32_t	0 to 4294967295
0x40	int64_t	-9223372036854775808 to -9223372036854775807
0x41	int64_t_zero	0
0x42	double	Double-precision floating-point
0x43	double_zero	0.0
0x44	uvarint64_t	0 to 9223372036854775807
0x45	varint64_t	-9223372036854775808 to 9223372036854775807
0x46	uint64_t	0 to 18446744073709551615
0x47	uint64_t_zero	0
0x48	uvaruint64_t	0 to 18446744073709551615
0x81	latin_string	uRTON_t folowed by only Roman
0x82	utf8_string	uRTON_t folowed by only Unicode
0x83	rtid	Subtypes 0x0, 0x2and 0x3
0x84	rtid_empty?	RTID()?
0x85	object	Ends with 0xff
0x86	array	Subtype 0xfd followed by uRTON_t ends with 0xfe
0x90	cached_latin_string	uRTON_t folowed by only Roman
0x91	cached_latin_string_recall	followed by uRTON_t
0x92	cached_utf8_string	uRTON_t folowed by only Unicode
0x93	cached_utf8_string_recall	followed by uRTON_t
0xfd	array_start	must be once followed by 0xfe
0xfe	array_end	has to used after array_start
0xff	object_end	has to used after object

Potential Issues

In my research I have stumbled across multiple issues that might extend to your tool as well:

Duplicate Keys

A file can contain duplicate keys, in most coding languages this leads to the first value of the first key being replaced by the value of the last key.
I'm afraid this will require much work to solve. I used arrays of tuples and a FakeDict class to make the json library interpret them as a dictionary with duplicate keys, no idea how you could approach this.

Latin strings

Non-unicode strings are probably latin-1 and not ASCII, this also complicates converting json back to rton.

Repairing files

catch errors related to end of file:

• not enough data for a number
• no data left to specify the next data type
• unicode with an unexpected character length (just mention it during the conversion)
• array with an unexpected length (use array end to end the array not the length)

Extension

[objects][objclass] defines the extension when the json doesn't include it pp.rton instead of pp.dat.rton:

• ["DraperSaveData", "GlobalSaveData", "PlayerInfoLocalSaveData", "LootSaveData", "SaveGameHeader"] result in .bin
• "PlayerInfo" results in .dat
• others result in rton

[h3x4n1um]

I'm truly thank you for your suggestion, but I'm afraid I'm kinda busy lately (a lot of university deadline are chasing me plz help). I'll take a look on your idea after I have more free time to spend on RETON project :(

[nineteendo]

Where did you find 0x29 & 0x29, as they might not even exist?