Skip to content

Feature/add mlp family estimators #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yoshifuminakamura merged 2 commits into from
Jun 20, 2026
Merged

Feature/add mlp family estimators #53

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
91cbda2
Add PerfTools MLP family estimator packages
yoshifuminakamura Jun 19, 2026
5a9499b
Show kernel package comparisons in estimates
yoshifuminakamura Jun 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/guides/add-estimation-package.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,9 @@
- `overlap_max_basic.sh`
- `gpu_kernel_lightgbm_v10.sh`
- `gpu_kernel_mlp_v15.sh`
- `gpu_kernel_mlp_v21.sh`
- `gpu_kernel_mlp_v40.sh`
- `gpu_kernel_mlp_v41.sh`

## 3. top-level package の責務

Expand Down Expand Up @@ -75,6 +78,18 @@ GPU kernel 単位の外部推定ツールは、通常は section package とし
- `gpu_kernel_mlp_v15`
- PerfTools `MLP_NN/v1.5`
- 主な依存: numpy/pandas/torch
- `gpu_kernel_mlp_v21`
- PerfTools `MLP_NN/v2.1`
- v1.5 NN と analytical anchor を組み合わせた hybrid/reference 系
- 主な依存: numpy/pandas/torch
- `gpu_kernel_mlp_v40`
- PerfTools `MLP_NN/v4.0`
- no-ET pure NN 系
- 主な依存: numpy/pandas/torch
- `gpu_kernel_mlp_v41`
- PerfTools `MLP_NN/v4.1`
- v4.0 に single-axis trend 対応を加えた NN 系
- 主な依存: numpy/pandas/torch
- `gpu_kernel_lightgbm_v10`
- PerfTools `LightGBM_model/1.0`
- 主な依存: numpy/pandas/lightgbm/pyyaml と `libgomp`
Expand Down
8 changes: 7 additions & 1 deletion programs/genesis/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,13 +124,19 @@ Single-package selection:
```bash
BK_GENESIS_GPU_SECTION_PACKAGE=gpu_kernel_mlp_v15
# or
BK_GENESIS_GPU_SECTION_PACKAGE=gpu_kernel_mlp_v21
# or
BK_GENESIS_GPU_SECTION_PACKAGE=gpu_kernel_mlp_v40
# or
BK_GENESIS_GPU_SECTION_PACKAGE=gpu_kernel_mlp_v41
# or
BK_GENESIS_GPU_SECTION_PACKAGE=gpu_kernel_lightgbm_v10
```

Multiple-package comparison:

```bash
BK_GENESIS_GPU_SECTION_PACKAGES=gpu_kernel_lightgbm_v10,gpu_kernel_mlp_v15
BK_GENESIS_GPU_SECTION_PACKAGES=gpu_kernel_lightgbm_v10,gpu_kernel_mlp_v15,gpu_kernel_mlp_v21,gpu_kernel_mlp_v40,gpu_kernel_mlp_v41
```

When multiple packages are selected, the app wrapper asks for
Expand Down
3 changes: 1 addition & 2 deletions programs/genesis/estimate.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ genesis_gpu_section_packages() {
elif [[ -n "${BK_GENESIS_GPU_SECTION_PACKAGE:-}" ]]; then
raw="$BK_GENESIS_GPU_SECTION_PACKAGE"
else
raw="gpu_kernel_lightgbm_v10,gpu_kernel_mlp_v15"
raw="gpu_kernel_lightgbm_v10,gpu_kernel_mlp_v15,gpu_kernel_mlp_v21,gpu_kernel_mlp_v40,gpu_kernel_mlp_v41"
fi

printf '%s\n' "$raw" |
Expand Down Expand Up @@ -340,7 +340,6 @@ genesis_run_single_estimate() {
"${BK_ESTIMATION_CURRENT_TARGET_NODES:-1}" \
"${BK_ESTIMATION_CURRENT_PACKAGE:-weakscaling}"
est_current_fom="${est_current_bench_fom:-$est_current_fom}"
est_current_fom_breakdown=""

if [[ "$synthetic_breakdown" -eq 1 ]]; then
genesis_mark_gpu_section_time_missing
Expand Down
91 changes: 89 additions & 2 deletions result_server/templates/_estimated_breakdown_card.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,92 @@
{% endif %}
{%- endmacro %}

{% macro render_kernel_package_comparisons(item) -%}
{% set kernel_summaries = item.get('metrics', {}).get('kernel_summaries', []) %}
{% if kernel_summaries %}
<div class="kernel-comparisons">
<div class="kernel-comparisons-title">Kernel package comparison; metrics are shown as reported by each package.</div>
{% for kernel in kernel_summaries %}
<div class="kernel-comparison">
<div class="kernel-name">{{ kernel.get('name', 'N/A') }}</div>
<table class="kernel-package-table">
<thead>
<tr>
<th>Package</th>
<th>Samples</th>
<th>Source Mean (ns)</th>
<th>Predicted Mean (ns)</th>
<th>Ratio</th>
<th>Source GPU</th>
<th>Target GPU</th>
</tr>
</thead>
<tbody>
{% for package in kernel.get('package_summaries', []) %}
<tr>
<td>{{ package.get('estimation_package', 'N/A') }}</td>
<td>{{ package.get('sample_count', 'N/A') }}</td>
<td>{{ package.get('source_time_ns_mean', 'N/A') }}</td>
<td>{{ package.get('predicted_time_ns_mean', 'N/A') }}</td>
<td>{{ package.get('mean_time_ratio_predicted_over_source', 'N/A') }}</td>
<td>{{ package.get('source_gpus', []) | join(', ') }}</td>
<td>{{ package.get('target_gpus', []) | join(', ') }}</td>
</tr>
{% if package.get('metric_comparisons') %}
<tr>
<td colspan="7">
<div class="kernel-metrics">
<div class="kernel-metrics-summary">{{ package.get('estimation_package', 'N/A') }} metrics</div>
<table class="kernel-metrics-table">
<thead>
<tr>
<th>Metric</th>
<th>Samples</th>
<th>Source Mean</th>
<th>Predicted Mean</th>
<th>Ratio</th>
</tr>
</thead>
<tbody>
{% for metric in package.get('metric_comparisons', []) %}
<tr>
<td>{{ metric.get('name', 'N/A') }}</td>
<td>{{ metric.get('sample_count', 'N/A') }}</td>
<td>{{ metric.get('source_value_mean', 'N/A') }}</td>
<td>{{ metric.get('predicted_value_mean', 'N/A') }}</td>
<td>{{ metric.get('ratio_predicted_over_source_mean', 'N/A') }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</td>
</tr>
{% endif %}
{% endfor %}
</tbody>
</table>
</div>
{% endfor %}
</div>
{% endif %}
{%- endmacro %}

{% macro render_breakdown_table(heading, items, first_column_label, first_column_key, join_list_values=False) -%}
{% if items %}
<h3>{{ heading }}</h3>
<div class="table-wrap">
<table class="breakdown-table">
<thead>
<tr><th>{{ first_column_label }}</th><th>Time</th><th>Package</th><th>Scaling</th><th>Fallback</th><th>Applicability</th></tr>
<tr>
<th>{{ first_column_label }}</th>
<th>Bench Time</th>
<th>Estimated Time</th>
<th>Package</th>
<th>Scaling</th>
<th>Fallback</th>
<th>Applicability</th>
</tr>
</thead>
<tbody>
{% for item in items %}
Expand All @@ -26,6 +105,7 @@ <h3>{{ heading }}</h3>
{% endif %}
<tr>
<td>{{ first_value }}</td>
<td>{{ item.get('bench_time', item.get('time', 'N/A')) }}</td>
<td>{{ item.get('time', 'N/A') }}</td>
<td>{{ item.get('estimation_package', 'N/A') }}</td>
<td>{{ item.get('scaling_method', 'N/A') }}</td>
Expand All @@ -34,7 +114,7 @@ <h3>{{ heading }}</h3>
</tr>
{% if item.get('candidate_estimates') %}
<tr>
<td colspan="6">
<td colspan="7">
<div class="candidate-estimates">
<div class="candidate-estimates-title">Candidate estimates; mean time is used for FOM composition.</div>
<table class="candidate-estimates-table">
Expand All @@ -57,6 +137,13 @@ <h3>{{ heading }}</h3>
</td>
</tr>
{% endif %}
{% if item.get('metrics', {}).get('kernel_summaries') %}
<tr>
<td colspan="7">
{{ render_kernel_package_comparisons(item) }}
</td>
</tr>
{% endif %}
{% endfor %}
</tbody>
</table>
Expand Down
105 changes: 102 additions & 3 deletions result_server/templates/estimated_detail.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,28 @@
.detail-table { margin-bottom: 20px; }
.detail-table th { text-align: left; min-width: 200px; background-color: #eef6f8; }
.detail-table td { min-width: 260px; white-space: normal; }
.system-comparison-table {
width: 100%;
table-layout: fixed;
margin-bottom: 20px;
}
.system-comparison-table th {
text-align: left;
background-color: #eef6f8;
}
.system-comparison-table th,
.system-comparison-table td {
padding: 8px 10px;
border: 1px solid #d8e3e8;
white-space: normal;
overflow-wrap: anywhere;
}
.system-comparison-table th:first-child,
.system-comparison-table td:first-child {
width: 26%;
font-weight: 700;
color: #102a43;
}
.applicability-summary {
margin-bottom: 20px;
padding: 16px 18px;
Expand Down Expand Up @@ -93,6 +115,63 @@
padding: 5px 7px;
border: 1px solid #e2e8f0;
}
.kernel-comparisons {
margin-top: 10px;
padding: 10px 12px;
border-radius: 10px;
background: #f8fafc;
border: 1px solid #e2e8f0;
}
.kernel-comparisons-title {
margin-bottom: 8px;
color: #475569;
font-size: 12px;
font-weight: 600;
}
.kernel-comparison {
margin-top: 10px;
padding-top: 10px;
border-top: 1px solid #e2e8f0;
}
.kernel-comparison:first-of-type {
margin-top: 0;
padding-top: 0;
border-top: 0;
}
.kernel-name {
margin-bottom: 6px;
font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
font-size: 12px;
color: #334155;
overflow-wrap: anywhere;
}
.kernel-package-table,
.kernel-metrics-table {
width: 100%;
border-collapse: collapse;
font-size: 12px;
}
.kernel-package-table th,
.kernel-package-table td,
.kernel-metrics-table th,
.kernel-metrics-table td {
padding: 5px 7px;
border: 1px solid #e2e8f0;
vertical-align: top;
}
.kernel-package-table th,
.kernel-metrics-table th {
background: #eef6f8;
}
.kernel-metrics {
margin-top: 8px;
}
.kernel-metrics-summary {
margin-bottom: 5px;
color: #475569;
font-size: 12px;
font-weight: 600;
}
.empty-note { color: #6b7280; font-size: 13px; }
</style>

Expand Down Expand Up @@ -121,9 +200,29 @@ <h2>Applicability Summary</h2>
{{ render_json_block("Confidence", confidence_json) }}
</div>

{{ render_titled_key_value_table("Current System", current_rows, "detail-table", "detail-table-wrap", "detail-card") }}

{{ render_titled_key_value_table("Future System", future_rows, "detail-table", "detail-table-wrap", "detail-card") }}
<div class="detail-card">
<h2>System Comparison</h2>
<div class="detail-table-wrap">
<table class="system-comparison-table">
<thead>
<tr>
<th>Item</th>
<th>Current System</th>
<th>Future System</th>
</tr>
</thead>
<tbody>
{% for row in system_comparison_rows %}
<tr>
<td>{{ row.label }}</td>
<td{% if row.current_class %} class="{{ row.current_class }}"{% endif %}>{{ row.current }}</td>
<td{% if row.future_class %} class="{{ row.future_class }}"{% endif %}>{{ row.future }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
</div>

<div class="detail-card">
Expand Down
Loading
Loading