create timing comparison with kbmag

[kalmarek]

@RearrangedLetters

• incorporate build.jl into deps directory;
• run timings script, e.g.

kbprog = joinpath(@__DIR__, "usr", "bin", "kbprog")
@assert isfile(kbprog)

function clean_kbprog(file)
    for fn in ("$file.kbprog", "$file.kbprog.ec", "$file.reduce")
        isfile(fn) && rm(fn)
    end
end

kb_data = joinpath(@__DIR__, "..", "kb_data")
@assert isdir(kb_data)
using Test
for fn in readdir(kb_data)
    if endswith(fn, "kbprog") || endswith(fn, "ec") || endswith(fn, "reduce")
        continue
    end
    @info fn
    if fn in ("degen4c", "f27monoid", "heinnilp", "verifynilp")
        @info "skipping"
        continue
    end

    file = joinpath(kb_data, fn)
    clean_kbprog(file)
    @time run(`$kbprog $file`)
    clean_kbprog(file)
end

and generate report/table comparison

• why are the four examples above failing?
• investigate the defaults for kbmag (stack-size= tidyint) etc.
• rerun KnuthBendix with those

[kalmarek]

@RearrangedLetters I rewrote the evaluation using DataFrames idioms (?); here's the updated plot:

[RearrangedLetters]

Here is the relevant part of the flame graph for f27_2gen. The flame graphs for the other examples above look very similar.

[RearrangedLetters]

Using the default parameters of kbmag drastically increases the time needed for the examples above:

One caveat with this comparison lies in the fact that kbmag uses the settings defined in the input files. Not all of them define all settings though.

Here is the flame graph for f27_2gen with kbmag settings:

[RearrangedLetters]

Regarding the examples that initially failed:

• degen4c: maximum number of equations have been reached (32541)
• f27monoid: Equations are too long
• heinnilp: Doesn't fail for me when I run it in the terminal
• verifynilp: Doesn't fail for me when I run it in the terminal

[kalmarek]

most of the changes have been incorporated into #67;
I'll keep this open as a reminder for possible perf improvements