"Relative" scale for metrics … comparing layouts to "min" achievable and "max" (not-optimized) costs for each metric

[fredsilon]

Hey, I just want to share a comparison I made for each metric.

I wanted to have some sort of reference points that could relate existing layouts to some "minimum" and "maximum" for each metric.

1. I calculated a "minimum achievable cost" for each metric, by letting the optimizer minimize just for each metric alone. (In case of secondary bigrams and irregularity, the corresponding bigram metrics remained activated, however the trigram metrics were weighted several orders higher)
2. I used optimize_sa with 250 random permutations of "abcdefghijklmnopqrstuvwxyzäöüß,." … an approach that I observed as quite effective to find a min cost solution. optimize_sa [...] -s $permutation_000 … -s $permutation_249
3. As a maximum I evaluated random layouts that weren't optimized at all, but were just evaluated using the evaluate tool. I used 1e5 random layouts for this. (Default ngrams, default evaluation parameters, standard keyboard). The "maximum" reference in my case is defined as the median of the 1e5 rnd layouts … one median for each metric (their unweighted cost).

I then plotted the result showing the unweighted cost for each metric:

• Blue dot: Minimum result that has been achieved for this metric (optimizing just for this metric alone)
• Green x: Noted layout
• Red x: KOY layout
• Yellow x: QWERTZ layout
• Box-Plot: 1e5 random layouts with 0.25 percentile, median and 0.75 percentile

I also scaled the plot as a second version:

1. Blue dot: defined as 0 (except for rolling metrics)
2. Median of random layouts defined as 100

Version 1: Unweighted cost

Version 2: Scaled to 0 (=min) and 100 (=median of random layouts)

I'm not sure, if this is really helpful to assess given layouts. Especially it's unclear, if the defined "min" and "max" reference points could be justified … especially as the random layouts spread a lot. But in the end it was fun to look at the metrics like this.

(and this brings me to a next mini-project to check which correlations are there between each metric … e. g. optimizing just for one metric … and then looking into all other metrics how they changed as well)