Why is cached blocking tree slower to create?

[sonalgoyal]

I was running some tests when I saw

'************ time taken to create cached blocking tree ************, 5512
************ time taken to create default blocking tree ************, 897
-------- max depth of trees -------- 3
-------- total nodes in a trees ---- 34
************ time taken to create cached blocking tree ************, 489
************ time taken to create default blocking tree ************, 345
-------- max depth of trees -------- 3
-------- total nodes in a trees ---- 21'

this does not look right to me. cached has to be faster. Can you please investigate?

[sophymarine]

That time taken to create cached blocking tree line at ~5.5s next to ~0.9s for the default tree usually points to the cache being filled during the measurement, not just read. First run has extra work. The code still builds the blocking structure, then writes it into whatever cache layer sits behind it.

One quick check: run the exact same job again in the same JVM. If the second run drops close to the 0.9s range, the first timing was paying the cache write cost.

Also worth looking at where the timer starts and ends. If it wraps both build() and cache.put(), that write path will dominate. Disk-backed or Spark-backed caches can add a few seconds on partition materialisation. Profiling that branch once should show where those extra seconds land.

[sonalgoyal]

Yes @sophymarine you are right in the part that the cache is filled while the tree is being created. But this is an in memory and in process optimisation, and there is no state carried over from one run to another in constructing the blocking tree.