Nameres log analysis#107
Draft
gaurav wants to merge 8 commits into
Draft
Conversation
Start a marimo notebook (analyze_nameres_logs.py) that parses NameRes Solr lookup logs into a QueryLogEntry dataclass and pandas DataFrame, mirroring the NodeNorm log analysis notebook. This first commit covers loading only: the dataclass, a regex parser for both INFO and "SLOW QUERY" WARNING log lines, and DataFrame construction with derived columns (filter flags, app overhead, mode). - Add pandas/numpy as project dependencies (needed by the notebook). - gitignore the raw log JSON exports and marimo's __marimo__ session cache. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Headline metrics (request counts, time span, filter usage) plus a latency percentile table (mean/p50/p90/p95/p99/max) for total time, Solr wait, and app overhead, broken down by query mode. Surfaces that autocomplete queries are far slower than exact lookups and that Solr wait dominates total latency. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
marimo-native Altair charts characterizing query performance: - Reactive latency histogram (metric/scale/range controls) - Total latency by query mode (box plot, log scale) - Solr wait by result limit and mode - Median Solr wait vs query length - Query length distribution - Temporal coverage of the export (with sampling caveat) - Slow-query rate by limit and mode, plus a table of the slowest lookups The charts confirm autocomplete + high-limit + very short queries are the pathological cases (single-character autocomplete lookups take minutes on Solr). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Deduplicate the log to one case per unique (query, params) combination and attach each case's observed Solr baseline latency (n, p50/p95 Solr wait, p50 took, ever_slow, first/last seen). Write the set to benchmark/ as JSON and offer an in-notebook download button. Replaying these cases against an ElasticSearch backend and comparing to baseline_solr yields an apples-to-apples latency comparison on real production queries. The generated benchmark/ artifact is gitignored (regenerated from the uncommitted logs), consistent with keeping raw logs out of the repo. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a "Next steps" section spelling out the follow-up work to turn this Solr-log characterization into a Solr-vs-ElasticSearch comparison: a replay harness over the exported benchmark, latency comparison against baseline_solr, result-quality parity (not just speed), pulling a more representative log sample, extra breakdowns (filters, pod/image), and a CSV export variant. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document lessons from pair-programming the NameRes notebook: use uv (not .venv/pip), the running kernel is source of truth (drive via marimo._code_mode), the cm.get_context() rollback gotcha (never reference a just-created cell inside the same block), verifying cells without Playwright screenshots, marimo graph rules, Altair row-limit/interactivity notes, and git hygiene for logs and generated artifacts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new log-analysis/ workflow for analyzing NameRes production query logs and exporting a replayable benchmark dataset, aligning with the existing NodeNorm log-analysis efforts.
Changes:
- Add a marimo-based NameRes log analysis notebook that parses CloudWatch exports, computes summary stats, and exports benchmark cases.
- Add log-analysis documentation and
.gitignorerules to keep raw logs and generated artifacts out of git. - Update Python dependencies to support the notebook tooling (marimo/Altair + pandas/numpy).
Reviewed changes
Copilot reviewed 5 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
pyproject.toml |
Adds data/analysis dependencies and a dependency-groups.dev section for marimo tooling. |
log-analysis/nodenorm/logs/README.md |
Documents how to export NodeNorm logs from CloudWatch Logs Insights. |
log-analysis/nameres/logs/.gitignore |
Ignores raw NameRes log exports (*.json). |
log-analysis/nameres/analyze_nameres_logs.py |
New marimo notebook for parsing NameRes lookup logs, visualizing latency, and exporting a benchmark JSON. |
log-analysis/nameres/.gitignore |
Ignores marimo cache and generated benchmark artifacts. |
log-analysis/CLAUDE.md |
Adds contributor guidance for working on the log-analysis notebooks and dependency management via uv. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
14
to
18
| "pytest>=8.4.2", | ||
| "pytest-timeout>=2.4.0", | ||
| "pandas>=3.0.3", | ||
| "numpy>=2.4.6", | ||
| ] |
Comment on lines
+39
to
+41
| dev = [ | ||
| "marimo[recommended]>=0.23.13", | ||
| ] |
Comment on lines
+16
to
+17
| "pandas>=3.0.3", | ||
| "numpy>=2.4.6", |
Comment on lines
+123
to
+126
| message = record.get("@message") | ||
| line = message["log"] if isinstance(message, dict) else message | ||
| if not line or "Lookup query to Solr" not in line: | ||
| return None |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
WIP