Optimize vacuum/analyze - #81
Conversation
Non-shape catalog updates can be applied without boundary
Avoids 20ms stuttering while processing DDL
8f4fec1 to
5b0451e
Compare
Default is to have everything involved on EC2 to avoid local deviation Remove probing based on row count, this was causing probe spam to obscure throughput. Instead probes for latency are at a consistent period & throughput uses an alternative mechanism
…h_timeout period resolution
89439cf to
b46253d
Compare
harshil-goel
left a comment
harshil-goel
left a comment
There was a problem hiding this comment.
At some places i saw comments about laptop-rtt issues. Can you take a quick look and delete comments about old implementations?
| pub const PG_CLASS_OID: u32 = 1259; | ||
| /// Catalogs that store statistics, including indexes and toast heaps | ||
| /// Keep `pg_statistic_ext` outside this list because DDL writes its definition | ||
| const OPAQUE_CATALOG_OIDS: &[u32] = &[ |
There was a problem hiding this comment.
Could it change with postgres versions? Could there be more oids that we should capture? Is it possible to directly just see which tables get changed when ddl happens and track only them?
There was a problem hiding this comment.
Not really. Postgres is pretty good about keeping OIDs constant <16384. This design is done as optimization so that if we miss things (or new PG version adds new stat table) we just get degraded perf rather than missing shape update
Been trying to keep this stuff simple, but maybe we'll need to have more fine grained tracking in future
There was a problem hiding this comment.
count_all function in this file (i can't comment on the exact location here). is counting direct rows. running the benchmark again without clearing clickhouse would lead the benchmark overcounting. we would need distinct(id) or final.
There was a problem hiding this comment.
we should truncate beforehand, benchmarking on differently sized tables wouldn't be clean
(checked: we truncate)
2 participants
Non-shape catalog updates can be applied without boundary