GeoPackage data types

[Didymograptus]

GeoPackage data types need to be set to appropriate values. For example legend codes are assigned as real data types, which causes problems with software such as Openlog.

[Ojaybee]

Currently if values are all numeric the field is set as REAL everything else is set to TEXT.

` # Detect column types for each group independently
column_types = {}
for group_name, records in data.items():
if group_name.endswith("_units"):
continue

		if records:
			group_headers = list(records[0].keys())
			column_types[group_name] = {}
			for header in group_headers:
				all_numeric = all(is_numeric(record.get(header)) for record in records if record.get(header))
				column_types[group_name][header] = "REAL" if all_numeric else "TEXT"

	return data, column_types`

Is it feasible to hardcode exceptions such as GEOL_LEG?

[Didymograptus]

I think that the safest way is to use the AGS TYPE value for the field. Basically everything becomes text unless it it is nDP (2 decimal places etc). or nSF (significant figures). I wrote some code for this a while ago. I'll dig it out.

[Ojaybee]

Can we rely on the AGS TYPE value being present? If it is omitted we have the opposite problem which breaks a lot of the QGIS behaviour and SQL querying. This was the problem we had previously with sqlite not respecting datatypes and forcing everything to TEXT.

[Didymograptus]

Yes, if it isn't present the file is not a valid AGS file.
Code from the AGS4 python library that uses dataframes: (iloc[1] is the TYPE heading row)

    # Convert appropriate columns to numeric
    numeric_df = df.loc[:, df.iloc[1].str.contains('DP|MC|SF|SCI', na=False)].apply(to_numeric, errors='coerce')

[Didymograptus]

FWIW I didn't actually write that code. I'm not sure that 'SCI' should be included, unless you also do a conversion of the data. e.g. from '2E3' to '2000'. I would omit from the list. It's rarely used anyway.