You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#267 asks for OS page-cache stats (Buffers/Cached) in /status?full=true. Investigation while working on #231 showed that cannot be done from /status, for two independent reasons:
Solr's metrics API cannot report it. The JVM OperatingSystemMXBean exposes only freePhysicalMemorySize (= Linux MemFree, which excludes page cache) and totalPhysicalMemorySize. There is no Buffers/Cached anywhere in /admin/metrics, in any group.
Reading /proc/meminfo from the NameRes API pod reports the wrong machine. That is the API node, which equals the Solr node only when the two happen to be co-located — and a number labelled as Solr's page cache that silently isn't is worse than no number. (This was prototyped in Add query-latency and Solr health metrics to /status, with a performance-diagnostics guide #231 as api_node_memory and then removed for exactly that reason.)
Page cache matters here: Solr mmaps its read-only ~127Gi index, so RAM beyond the heap is what keeps queries fast (see #265).
Proposed instead: run a node-level exporter (node_exporter / Prometheus) on the Solr host and read page-cache stats from Grafana, alongside the GC logs which now also go to stdout. /status?full=true already reports the sizing inputs it legitimately can — solr_metrics.host.total_physical_mem_mb, available_processors, load/CPU, and jvm.heap_* / gc_*.
Suggest closing #267 in favour of this once agreed.
#267 asks for OS page-cache stats (Buffers/Cached) in
/status?full=true. Investigation while working on #231 showed that cannot be done from/status, for two independent reasons:OperatingSystemMXBeanexposes onlyfreePhysicalMemorySize(= LinuxMemFree, which excludes page cache) andtotalPhysicalMemorySize. There is no Buffers/Cached anywhere in/admin/metrics, in any group./proc/meminfofrom the NameRes API pod reports the wrong machine. That is the API node, which equals the Solr node only when the two happen to be co-located — and a number labelled as Solr's page cache that silently isn't is worse than no number. (This was prototyped in Add query-latency and Solr health metrics to /status, with a performance-diagnostics guide #231 asapi_node_memoryand then removed for exactly that reason.)Page cache matters here: Solr mmaps its read-only ~127Gi index, so RAM beyond the heap is what keeps queries fast (see #265).
Proposed instead: run a node-level exporter (node_exporter / Prometheus) on the Solr host and read page-cache stats from Grafana, alongside the GC logs which now also go to stdout.
/status?full=truealready reports the sizing inputs it legitimately can —solr_metrics.host.total_physical_mem_mb,available_processors, load/CPU, andjvm.heap_*/gc_*.Suggest closing #267 in favour of this once agreed.