backport(0.34.x): the KEDA scaling suite (#7620 #7899 #8008 #8424 #6908 #7709) for pinned jacBuilder deploys - #8940
Conversation
…ght diagnostics (jaseci-labs#7620) ## Summary Closes jaseci-labs#7404. `KEDAAutoscaler.preflight()` only ever verified that the core `keda.sh/v1alpha1 ScaledObject` API existed on a cluster. That check is not sufficient for HTTP activated workloads, which also depend on the KEDA HTTP Add-on's `InterceptorRoute` API, two Kubernetes Services (the external scaler and the interceptor proxy), and a compatible Add-on version. Without these checks, `apply_http_activation` could create a valid `ScaledObject` that never actually activates from HTTP traffic, with no clear diagnostic explaining why. This PR introduces structured capability discovery for the KEDA HTTP Add-on and wires it into `apply_http_activation`, so a broken or partially installed cluster fails with an actionable error instead of silently deploying a nonfunctional workload. ## Changes | File | Changes | |---|---| | `jac/jaclang/scale/deploy/autoscale/keda_capabilities.jac` (new file) | Adds the `KEDACapabilities` data object, matching the shape specified in the issue: `core_available`, `http_addon_available`, `interceptor_route_api_version`, `external_scaler_address`, `interceptor_service_address`, `warnings`, and `errors`. Also adds `SUPPORTED_HTTP_INTERCEPTOR_API_VERSIONS`. | | `jac/jaclang/scale/deploy/autoscale/keda_autoscaler.jac` | Imports `KEDACapabilities`.<br>Adds a new `interceptor_service_address` field, defaulted the same way `http_scaler_address` already is.<br>Replaces the old bool only `_http_preflight_cache` with `_capabilities_cache`, which holds a full `KEDACapabilities` object per cluster.<br>Declares `discover_capabilities`, `invalidate_capabilities`, `_check_service_address`, `format_capabilities`, and `_capabilities_cache_key`. | | `jac/jaclang/scale/deploy/autoscale/keda_autoscaler.impl.jac` | `discover_capabilities` checks the core `ScaledObject` API and the HTTP Add-on's `InterceptorRoute` API independently, so a missing core install is distinguished from a missing Add-on. A 403 on either check is reported as a distinct permission error rather than being folded into "not installed."<br>If the `v1beta1 InterceptorRoute` API is absent (not RBAC denied), it additionally probes the legacy `v1alpha1 HTTPScaledObject` API. If that is present, the result is a distinct "upgrade" warning with the exact `helm upgrade` command, rather than being reported as "Add-on not installed." A 403 on this legacy probe is reported as its own permission error rather than being folded into "not installed."<br>`_check_service_address` verifies the external scaler and interceptor proxy Services actually resolve, and reports which one is missing.<br>Results are cached per cluster identity via a new `_capabilities_cache_key` helper, which combines `_get_cluster_key()` with this instance's configured `http_scaler_address` and `interceptor_service_address`. This keeps two instances that target the same cluster but are configured with different service addresses from reusing an incompatible cached result. `discover_capabilities(refresh=True)` and `invalidate_capabilities()` remain the explicit refresh paths.<br>`format_capabilities` produces a single concise, human readable line for CLI output.<br>`preflight()` now also catches a 403 and returns a distinct permission error instead of letting the `ApiException` propagate unhandled, which is what happened previously.<br>`apply_http_activation` now calls `discover_capabilities()` instead of its own inline, duplicate check. If discovery reports any errors (missing service, RBAC denied, or similar), it raises `ValueError` with the details. If the core KEDA API or the HTTP Add-on is simply absent (no errors, just not installed), it keeps the existing behavior of logging a warning and returning `False` rather than raising. A partial install, where the Add-on and its services are healthy but core KEDA itself is missing, now also falls into this gate instead of proceeding to create resources that would fail against a nonexistent API. | | `jac/jaclang/scale/tests/deploy/test_keda_http_activation.jac` | Adds a new capability discovery test section covering: core only clusters, full installs, missing external scaler service, missing interceptor service, legacy Add-on detection, RBAC denied on the core check, the HTTP Add-on check, and the legacy probe, cache and refresh behavior, cache isolation between instances with different service addresses, explicit invalidation, the raise on broken capabilities and skip on missing core behavior in `apply_http_activation`, `format_capabilities` output, and the `preflight()` 403 fix. Test setup uses small composable mock builders (`_custom_api_for`, `_core_api_for`) rather than one mock per scenario, and one table driven test covers the distinct diagnostic cases to avoid duplicating near identical assertions. | ## Out of scope As noted in the issue: no auto install or upgrade of KEDA, no ingress creation, no general health checking of every scaler type. This PR also does not verify that a workload's traffic is actually routed through the interceptor at runtime, only that the interceptor's own API and Service objects exist and resolve. That distinction is called out explicitly in the implementation plan for this issue. ## Test plan - [x] Core only clusters report core available and HTTP unavailable. - [x] Fully installed clusters report HTTP ready with no errors. - [x] Missing external scaler service, missing interceptor service, legacy Add-on version, and RBAC denied cases (core check, HTTP Add-on check, and legacy probe) each produce a distinct diagnostic. - [x] Results are cached per cluster identity, including the configured service addresses, and refresh via `discover_capabilities(refresh=True)` and `invalidate_capabilities()`. - [x] `apply_http_activation` raises when capabilities are broken, returns `False` gracefully when the Add-on or core KEDA is simply absent, and does not proceed to create resources on a partial install. - [x] Documentation update showing the supported Helm install and upgrade flow. - [x] Full deploy test suite (169 tests) passes with no regressions. --------- Co-authored-by: Kugesan Sivasothynathan <kugesan.sivasothynathan@jaseci.org> (cherry picked from commit 5234c81)
… policy (jaseci-labs#7899) Closes jaseci-labs#7764. `jac-scale`'s generated HPA `behavior` block (added in jaseci-labs#7445) has always shipped one hardcoded scale-down policy — `{"type": "Percent", "value": 50, "periodSeconds": 60}`, no `selectPolicy` — with no way to override it per service. Because the deploy path re-applies the full generated manifest on every `jac start --scale`, any out-of-band `kubectl patch` on the HPA gets wiped on the next deploy, forcing a re-patch-after-every-deploy loop in CI. This PR adds a `hpa.behavior` fragment to the per-service config table, deep-merged over the generated default. It's the first slice of jaseci-labs#7764 — connection-heavy services (WebSockets, SSE) can now ask for a gentle scale-down (e.g. at most 2 pods every 5 minutes) without imposing that rate on every other service through the app-global `autoscaler_cooldown`, and the override survives every subsequent deploy. ```toml [scale.microservices.services.chat_sv.hpa.behavior.scaleDown] stabilizationWindowSeconds = 600 selectPolicy = "Min" policies = [{ type = "Pods", value = 2, periodSeconds = 300 }] ``` - **`AutoscalerSpec`** (`autoscaler.jac`) gains a `behavior_overlay: dict[str, any] = {}` field. - **`Autoscaler._build_behavior`** deep-merges `behavior_overlay` over the hardcoded default via the existing `merge_manifest_overlay` (same semantics already used for `deployment_overlay`: maps merge recursively; the `policies` list carries no `name` key, so it's replaced wholesale rather than merged entry-by-entry, matching how you'd actually want to specify a policy set). Both the HPA and KEDA engines pick this up for free — they already route through the same `_build_behavior` call, so this is a single-point change. - **`ManifestBuilder._get_autoscaler_config`** (`manifest_builder.jac`) resolves the fragment from the per-service `hpa.behavior` table. - **`KubernetesTarget`**'s deploy loop (`target.jac`) threads it into `AutoscalerSpec(...)` as `behavior_overlay=cfg.get("behavior", {})`. - **Config schema** (`plugin_config.jac`) and the **K8s reference table** (`runtime/docs.md`) document the new key. `_build_behavior` resolves `merge_manifest_overlay` via a function-scoped import rather than a top-level one. `manifest_builder.jac` already imports constants from `autoscaler.jac` at module level, so a top-level import in the other direction would be a circular import; a deferred import inside the function (evaluated well after both modules are fully loaded) is the same pattern already used elsewhere in this codebase (e.g. `manifest_builder.jac`'s own `_pod_entrypoint`) for exactly this situation. - `service_overlay` / first-class `Service` `annotations` + gateway `sessionAffinity` knobs - PDB `min_available` and percentage (`"50%"`) support Written test-first: 11 new tests across the files that already exercise the touched code paths (`test_deployment_overlay.jac`, `test_factories.jac`, `test_keda_autoscaler.jac`, `test_memory_trigger_guard.jac`), confirmed red against the pre-implementation tree, then green after: - `_build_behavior` merge semantics in isolation: no-overlay default, empty-overlay no-op, `scaleDown` override, `scaleUp` override, wholesale (not per-item) `policies` replacement. - Both engines apply the overlay: HPA's `_build_manifests` (`spec.behavior`) and KEDA's `_build_manifests` (`advanced.horizontalPodAutoscalerConfig.behavior`). - Config plumbing: `_get_autoscaler_config` surfaces `hpa.behavior`, defaults to `{}` when unset, doesn't leak across services, and `hpa.enabled = false` still short-circuits before behavior is considered. Also verified against a real cluster, not just unit tests. Deployed the `jac-shop` 3-service microservice fixture (`jaclang/scale/tests/fixtures/k8s_e2e`) to a local `kind` cluster with a `hpa.behavior.scaleDown` override configured on `products_app`, via `jac start main.jac --scale`: - **First deploy**: all 4 services (gateway + 3) reached Ready, and the live `products-app-hpa` object's `spec.behavior.scaleDown` matched the configured override exactly (`stabilizationWindowSeconds: 600`, `selectPolicy: Min`, `policies: [{type: Pods, value: 2, periodSeconds: 300}]`), with `scaleUp` untouched at the generated default. - **Redeploy**: re-ran `jac start main.jac --scale` against the same cluster and diffed `products-app-hpa`'s `spec.behavior.scaleDown` before and after — byte-identical. Confirms the actual pain point from the issue: the override survives repeated applies instead of being reset to the old hardcoded `{Percent, 50, 60s}` shape. - [x] A per-service `hpa.behavior` fragment in `jac.toml` survives repeated `jac start --scale` applies with no external patching. - [x] Global `autoscaler_*` keys remain the defaults; per-service fragment wins on merge. - [x] Works for both `autoscaler_engine = "hpa"` and `"keda"`. - [x] Config schema (`plugin_config.jac`) and docs updated; overlay-merge unit tests added alongside the existing `test_deployment_overlay.jac` patterns. --------- Co-authored-by: Kugesan Sivasothynathan <153247429+kugesan1105@users.noreply.github.com> Co-authored-by: Kugesan Sivasothynathan <kugesan.sivasothynathan@jaseci.org> (cherry picked from commit 91c7adc)
…ewed YAML matches what a deploy applies (jaseci-labs#8008) `--show-yaml` printed pvcs, deployments, services, pdbs and ingress but dropped autoscalers, while the totals line counted them and the service view rendered HPA details from them. The reviewed YAML was not the applied artifact, for exactly the resources most likely to cause a production surprise. The bundle's `autoscalers` key holds CONFIGS, not manifests; the real HPA/ScaledObject is synthesized only inside the apply loop. Adding the key to the flatten list would therefore have dumped config dicts. Instead: - `deploy/target/kubernetes/target.jac`: the AutoscalerSpec construction is extracted out of the apply loop into `_autoscaler_spec` (single source), and a new `render_autoscaler_manifests(bundle)` builds the same objects the apply path would, via each engine's `_build_manifests`. Pure - no Kubernetes API calls. - `runtime/cli/plan.jac`: `Plan.from_target` stashes the rendered manifests into the bundle; `_flatten_manifests` emits them into --show-yaml. - New test_show_yaml_autoscalers (3): real KubernetesTarget with the HPA and KEDA engines; asserts kind, scaleTargetRef, min/max mapping, and that KEDA's ScaledObject body (not its name/body wrapper) is rendered. - test_plan extended (+2): autoscaler manifests reach the YAML dump, plus a sentinel test asserting every manifest-bearing bundle key reaches --show-yaml (the issue's drift-proofing suggestion). - jac-ec2 rig, linked-source dev binary on this branch: all passing - test_show_yaml_autoscalers (3, new), test_plan (19, includes the 2 new), test_factories (28), test_keda_autoscaler (34), test_memory_trigger_guard (8), test_dry_run_purity (12). - NOT run: a live cluster apply of HPA/KEDA objects (the render path is pure; the apply path is unchanged except for the spec extraction). Full repo suite not run locally. - KEDA TriggerAuthentication objects are applied but still not rendered into --show-yaml (built inline in `_apply_trigger_auth`; needs the same build/apply split). - Normalize the `_build_manifests` contract: KEDA returns a {name, body} wrapper, HPA returns the bare manifest. Closes jaseci-labs#7915. Part of jaseci-labs#7900. --------- Co-authored-by: Thamirawaran Sathiyalogeswaran <107134124+Thamirawaran@users.noreply.github.com> (cherry picked from commit 9d2cace)
…aml (jaseci-labs#8424) Closes jaseci-labs#8409. `jac start --scale --show-yaml` (via `render_autoscaler_manifests`) prints the autoscaler objects a deploy would apply, but for KEDA it only ever printed the `ScaledObject`. A trigger with `auth.secret_refs` also causes `apply()` to create a `TriggerAuthentication`, and that half of the footprint was invisible to the dry-run preview. This PR unifies the `_build_manifests` contract across both autoscaler engines so the render path renders the same objects `apply()` creates, with nothing built as a side effect of a live API call. `_build_manifests` for KEDA only ever constructed the `ScaledObject` body. The `TriggerAuthentication` for an authenticated trigger was built and written to the cluster exclusively inside `KEDAAutoscaler.apply` via `_apply_trigger_auth`, a function that conflated *building* the manifest with *reconciling* it against the cluster (get → patch-or-create) in one step. Because the manifest never existed as a standalone, returnable value, `render_autoscaler_manifests` (the `--show-yaml` path) had no way to see it. A secondary wart compounded this: `_build_manifests` returned a different shape per engine (HPA a bare manifest dict, KEDA a `{"name", "body"}` wrapper), so the render path added defensive shape-sniffing (`built["body"] if "body" in built else built`) instead of a real contract. That sniff is what let the KEDA gap go unnoticed - it happily returned "a manifest" without anyone checking whether it was the *only* manifest KEDA needed to emit. Net effect: a KEDA autoscaler config with an auth-bearing trigger dry-ran clean (N ScaledObjects, 0 TriggerAuthentications) and then `apply()` silently created N ScaledObjects **and** M TriggerAuthentications the preview never showed - exactly the class of surprise `--show-yaml` exists to prevent, and the same shape of gap jaseci-labs#8008 already fixed once for autoscalers generally. `_build_manifests` now returns one `list[dict]` contract for both engines: - `HPAAutoscaler._build_manifests` returns `[hpa]`. - `KEDAAutoscaler._build_manifests` returns `[scaled_object, *trigger_authentications]`. To make that possible without duplicating logic between the render and apply paths: - The `TriggerAuthentication` body construction is extracted out of the old apply-only `_apply_trigger_auth` into a pure builder, `_build_trigger_auth_manifest`, which returns the manifest (or `None` when a trigger has no auth) and makes no API calls. - A shared `_apply_custom_object` reconciler (get → patch-or-create) replaces the two nearly-identical get/patch/create blocks that previously existed separately for `TriggerAuthentication` and `ScaledObject` writes in `KEDAAutoscaler.apply`. - `render_autoscaler_manifests` (`target.jac`) now does `out.extend(autoscaler._build_manifests(...))` instead of shape-sniffing a single object out of the return value, so it renders every manifest an engine produces, not just the first one. Apply-time ordering is unchanged: `TriggerAuthentication` objects are still written before the `ScaledObject` that references them via `authenticationRef`, since KEDA's admission webhook needs the referenced auth resource to exist first. | File | Change | |---|---| | `jac/jaclang/scale/deploy/autoscale/keda_autoscaler.jac` | Interface updates: `_build_manifests` returns `list[dict[str, any]]`; new `_build_trigger_auth_manifest` and `_apply_custom_object` declarations replace `_apply_trigger_auth`. | | `jac/jaclang/scale/deploy/autoscale/keda_autoscaler.impl.jac` | `_build_manifests` assembles `[scaled_object, *trigger_auths]`; trigger-auth body construction moved into pure `_build_trigger_auth_manifest`; new `_apply_custom_object` reconciler; `apply()` applies trigger-auth manifests before the ScaledObject via the shared reconciler. | | `jac/jaclang/scale/deploy/autoscale/hpa_autoscaler.jac` | `_build_manifests` interface updated to `list[dict[str, any]]`. | | `jac/jaclang/scale/deploy/autoscale/hpa_autoscaler.impl.jac` | `_build_manifests` wraps its single manifest in a list; `apply()` unwraps `[0]`. | | `jac/jaclang/scale/deploy/target/kubernetes/target.jac` | `render_autoscaler_manifests` flattens the manifest list from each engine instead of shape-sniffing a single object. | | `jac/jaclang/scale/tests/deploy/test_factories.jac` | HPA `_build_manifests` assertions updated to the list contract. | | `jac/jaclang/scale/tests/deploy/test_keda_autoscaler.jac` | KEDA `_build_manifests` assertions updated to the list contract (ScaledObject at index 0, no more `name`/`body` wrapper). | | `jac/jaclang/scale/tests/deploy/test_show_yaml_autoscalers.jac` | New regression test: an authenticated KEDA trigger renders both the `ScaledObject` and its matching `TriggerAuthentication` through `render_autoscaler_manifests`. | Ran the full `deploy` test group locally with `jac test` (one process per file, matching how CI's `test-scale (deploy)` matrix leg runs it): all 27 files, 293 tests pass, both before-vs-after on the three files this PR touches and as a full-group run afterward to catch cross-file fallout. - `test_show_yaml_autoscalers` (4, 1 new) - the regression test for this issue. Builds a bundle with one KEDA-engine service carrying an authenticated `redis` trigger (`auth.secret_refs`) and asserts `render_autoscaler_manifests` returns exactly two manifests: the `ScaledObject` at index 0 and a `TriggerAuthentication` at index 1 whose `metadata.name` matches the `authenticationRef.name` KEDA embedded in the ScaledObject's trigger, with the `secretTargetRef` mapped correctly from config. Confirmed this test fails against the pre-fix code (dropping the `_build_trigger_auth_manifest` loop reproduces the original bug - `len(manifests) == 1`) and passes with the fix. The other two existing tests in this file (HPA render, empty-bundle render) are unchanged and still pass, guarding against a regression in the ordinary case. - `test_keda_autoscaler` (34) - full suite for the engine this PR changes most: `_build_manifests` shape and field mapping (9 tests, now indexing `[0]` into the list), trigger-authentication naming/collision rules, `preflight`, and - the part most likely to break silently on a build/apply split - `apply()`'s behavior end to end against a mocked `CustomObjectsApi`: that a `TriggerAuthentication` is still created before its `ScaledObject`, that the name written to the cluster still matches the `authenticationRef` the ScaledObject carries, and that a KEDA-CRDs-not-installed cluster still short-circuits before either write. All pass unmodified in assertion logic (only the `_build_manifests` indexing changed), which is the signal that `apply()`'s observable behavior toward the cluster is unchanged by the refactor. - `test_factories` (32) - HPA `_build_manifests` output (exact-dict equality against the expected manifest, and the `behavior_overlay` merge) re-verified against the new `[hpa]` list contract. Also validated end to end against a real cluster, not just mocks: - **Cluster**: local `kind` (v0.32.0) with KEDA core installed via the standard Helm chart (`kedacore/keda`, namespace `keda`); `scaledobjects.keda.sh` and `triggerauthentications.keda.sh` CRDs confirmed present. A static `hostPath` PV under a `no-provisioner` StorageClass (`jac-rwx`) stands in for the bundle PVC's `ReadWriteMany` requirement, the same technique `jac/jaclang/scale/scripts/e2e_lib.sh`'s `provision_kind_rwx_storage` uses for the kind-based CI e2e leg (kind's `local-path` StorageClass is RWO-only; a static hostPath PV is safe on a single-node cluster). - **Project**: a minimal scratch app (`main.jac`, one walker, no client) with `[dev] jaclang_source` pointing at this checkout, so both the CLI and the deployed pods run the fixed source, not a published release binary. `[scale.kubernetes] autoscaler_engine = "keda"` plus one `[[scale.microservices.services.<name>.triggers]]` entry with `type = "redis"` and `auth.secret_refs` - the exact shape jaseci-labs#8409 described. - **Dry-run** (`jac start --scale --dry-run --show-yaml`, no cluster contact): the rendered YAML stream includes both the `ScaledObject` and a `TriggerAuthentication` object, with the `ScaledObject`'s `authenticationRef.name` matching the `TriggerAuthentication`'s `metadata.name` exactly (`kedaval-7dcd81a6-ta`) - this is the bug fixed, observed through the real CLI rather than only through the test suite. - **Live apply** (`jac start --scale`): the same `TriggerAuthentication` (`kedaval-7dcd81a6-ta`, `SECRET: redis-secret`) is present on the cluster via `kubectl get triggerauthentications`, and the `ScaledObject`'s `AUTHENTICATIONS` column names it - dry-run and `apply()` now agree, which is the whole point of the fix. `kedaval-scaledobject` reports `READY=False`/`ACTIVE=Unknown` only because no real Redis exists at the configured address; that's expected and irrelevant to what's being validated here (the auth object's existence and wiring, not live scaling). - **App health**: both `gateway-deployment` and `kedaval-deployment` reached `1/1 Running`, the fleet rollout reported `2/2 deployments serving`, and `/health` on the gateway reported both services `"status": "healthy"` - confirming the fix doesn't regress the ordinary deploy path. Configure a KEDA autoscaler with an auth-bearing trigger and run `jac start --scale --show-yaml`: both the `ScaledObject` and its `TriggerAuthentication` now render, matching what `apply()` creates. --------- Co-authored-by: Kugesan Sivasothynathan <kugesan.sivasothynathan@jaseci.org> Co-authored-by: Kugesan Sivasothynathan <153247429+kugesan1105@users.noreply.github.com> (cherry picked from commit b908596)
…bel (part 2 - final - of issue jaseci-labs#6535) (jaseci-labs#6908) ## Problem `KEDAAutoscaler` orphaned `TriggerAuthentication` resources in two distinct ways: 1. **Redeploy with a changed trigger set.** TA names are derived from the trigger identity (`app + trigger-key`, hashed). When a redeploy renames, removes, or reorders triggers, `apply()` creates new TAs under new names and re-points the ScaledObject at them, but never deletes the TAs from the previous config. `apply()` had no prune step, so every such redeploy left a stale TA behind, unreferenced by the ScaledObject. 2. **destroy().** `destroy()` deleted only the ScaledObject, leaving every TA its `apply()` created behind. The earlier approach tried to fix (2) by reading `authenticationRef.name` off the live ScaledObject. That could not fix (1) at all (the stale TA is no longer referenced by the ScaledObject, so it is never found), and it left a hole in (2): if the ScaledObject was already gone, the GET returned 404 and `destroy()` returned early, stranding any TAs permanently. ## Root cause There was no ownership-based reconciliation. TAs are children whose lifecycle should be bound to the autoscaler, but the code bound them only by name derivation plus an imperative "read the ScaledObject, delete what it currently points at". Nothing ever reconciled the actual set of managed TAs in the namespace against the desired set. Compounding this, TAs were stamped only with `managed=jac-scale` and no owner scoping, so they could not even be listed per-autoscaler. ## Fix Reconcile TAs by an ownership label instead of by parsing the ScaledObject. - `_build_trigger_auth_manifest()` stamps every managed TA with `jac-scale/owner=<ScaledObject name>` (plus `app` and `managed`). The owner value is the ScaledObject name, hashed if it exceeds the 63-char label limit, via a shared `_owner_label_value()` helper so the apply and destroy selectors always agree. Because the labels live in the manifest builder, `--show-yaml` renders exactly what `apply()` writes. - `apply()` collects the desired TA names, then `_prune_orphan_trigger_auths()` lists this owner's managed TAs by label and deletes any not in the freshly-applied set. This closes orphan pathway (1). - `destroy()` lists TAs by `jac-scale/owner=<name>,managed=jac-scale` and deletes them (404/422-safe), then deletes the ScaledObject. It no longer reads the ScaledObject, so cleanup works even when the ScaledObject was already deleted out of band. This closes orphan pathway (2) with no early-return hole. - `destroy()` (base `Autoscaler`, `HPAAutoscaler`, `KEDAAutoscaler`) now takes `app_name` instead of a raw resource name and resolves the concrete name internally via `resource_name_for()`, matching how the owner label selector is computed. A single `_scaled_object_name(spec, app_name)` helper is the one definition of the ScaledObject name, used by `apply()` and `_build_manifests()`; `destroy()`'s formula is exactly its fallback branch, so the two cannot disagree. ``` apply(spec): build manifests (validation runs inside the builder) for each manifest in _APPLY_ORDER: create/patch (TAs labelled owner=<SO name>, then the ScaledObject) prune: list TA by owner label -> delete any name not in desired set destroy(app_name, namespace): scaled_object_name = resource_name_for(app_name) list TA by owner label=scaled_object_name -> delete each (404-safe) delete ScaledObject named scaled_object_name (404-safe) ``` ## Migration for TAs that predate the owner label There is deliberately no legacy name-pattern sweep (an earlier revision had one; review showed its truncated-prefix match could cross-delete another app's TAs in a shared namespace, so it was removed rather than narrowed). Instead: - A pre-label TA whose trigger still exists is adopted implicitly: `apply()` patches it by name and the merge patch adds the owner label. From then on it is fully reconciled. - A pre-label TA whose trigger was already removed cannot be named from `spec.triggers`, so nothing touches it; the namespace-wide `destroy_collection()` sweep (`managed=jac-scale`) remains the catch-all for those. ## Tests `test_keda_autoscaler.jac`, all mocked, 42 tests including: - TriggerAuthentication manifest carries app and owner labels for reconciliation - apply prunes a TriggerAuthentication the redeployed trigger set no longer references, and keeps one still referenced - destroy resolves the ScaledObject name from app_name and deletes it - destroy sweeps TriggerAuthentications owned by this ScaledObject before deleting it (works with the ScaledObject already gone, the case the earlier patch could not handle) - destroy tolerates an already-deleted TA (404) and an absent auth CRD (404 on list) - duplicate trigger identities are rejected by the manifest builder, so `--show-yaml` refuses what `apply()` refuses `test_show_yaml_autoscalers.jac` covers the rendered TA manifests carrying the same labels `apply()` writes. `test_factories.jac` green. ## Live cluster validation Verified against a real cluster with real KEDA CRDs, driving this branch's `KEDAAutoscaler.apply()`/`destroy()` directly: | Step | Action | Result | | --- | --- | --- | | apply | trigger `cache` + auth | TA created with `jac-scale/owner`, `app`, `managed` labels | | apply prune | rename trigger `cache` to `queue` | old TA pruned, only the new TA remains | | destroy | destroy with ScaledObject present | 0 TAs, 0 ScaledObjects | | destroy (SO gone) | delete ScaledObject out of band, then destroy | TA still swept by label | | migration | pre-label TA, trigger still live | adopted by apply's patch, swept by destroy | Both orphan pathways are provably closed on a real cluster. ## Notes Rebased on the merged jaseci-labs#8424: trigger validation lives in `_build_manifests`, so the render and apply paths accept and refuse identical specs, and the manifest list is the single desired-state description that both paths share. This PR supersedes the earlier ScaledObject-parsing approach and resolves the partial-apply / already-gone-ScaledObject limitation tracked in jaseci-labs#6909 directly: cleanup is now independent of the ScaledObject. Closes jaseci-labs#6535. Closes jaseci-labs#6909. --------- Co-authored-by: Kugesan Sivasothynathan <kugesan.sivasothynathan@jaseci.org> Co-authored-by: Kugesan Sivasothynathan <153247429+kugesan1105@users.noreply.github.com> (cherry picked from commit e092aaa)
…jac.toml config (jaseci-labs#7709) Closes jaseci-labs#7475 PR jaseci-labs#7421 added a programmatic API for KEDA HTTP Add-on scale-to-zero activation (`HTTPActivationSpec`, `KEDAAutoscaler.apply_http_activation` / `destroy_http_activation`), but left it reachable only by writing a Jac driver script. Every other autoscaler knob in jac-scale is configured declaratively through `jac.toml`, so this PR adds that same entry point for HTTP activation, for both the monolith deploy path and per-service microservice deploys. The programmatic API from jaseci-labs#7421 is unchanged and remains the right tool for callers with a dynamic, create/destroy-on-demand lifecycle (for example, an IDE preview orchestrator). The two entry points are additive, not a replacement of one by the other: `jac.toml` covers a known, standing service; the programmatic API covers a workload created and torn down at runtime. **One mutually-exclusive scaling decision per service, resolved at build time.** `manifest_builder` resolves each service (the gateway included) to exactly one scaling mode - `none`, `metric` (HPA/KEDA), or `http_activation` - computed once and carried through the bundle as a single `scaling` map, replacing two independent resolvers (autoscaler config and http_activation config) that could both fire on the same Deployment. Three consequences, each of which was a real apply-side defect before: - **Correct scale target.** The `ScaledObject`'s `scaleTargetRef`, and the `InterceptorRoute`/`ScaledObject` resource names derived from it, come from the built Deployment manifest (`<name>-deployment`), never re-derived from the bare service name. A bare-name target made `apply_http_activation`'s existence check 404 on a real cluster, killing the deploy. - **No dual scalers.** A service gets a metric autoscaler or an HTTP `ScaledObject`, never both; the modes are mutually exclusive by construction, so KEDA's admission webhook can no longer reject a second `ScaledObject` on an HPA-managed target. - **The gateway never scales to zero.** It is the ingress entry point and must stay warm, so it is structurally excluded from HTTP activation and never inherits a shared `enabled = true`; it falls through to a normal HPA. **Teardown sweeps KEDA resources regardless of the base engine.** `destroy()`'s KEDA sweep previously went through `AutoscalerFactory.create(self.k8s_config.autoscaler_engine, ...)`, so when `autoscaler_engine = "hpa"` it resolved to `HPAAutoscaler.destroy_collection`, which does not know about `InterceptorRoute` or HTTP-activation `ScaledObject` resources, orphaning them on teardown. `destroy()` now also runs a `KEDAAutoscaler(...).destroy_collection(...)` sweep whenever `autoscaler_engine != "keda"`, since HTTP activation needs the KEDA HTTP Add-on regardless of the base engine. **Teardown tolerates the HTTP Add-on's separate RBAC surface, without masking other failures.** The HTTP Add-on's CRD group (`http.keda.sh`, covering `InterceptorRoute`) installs and is permissioned separately from core KEDA's `keda.sh` group (`ScaledObject`, `TriggerAuthentication`), which the base autoscaler already needs. A cluster's RBAC can easily cover one without the other. Both `destroy_http_activation` and `destroy_collection` were changed so a 403 on the `http.keda.sh` side never aborts cleanup of the `keda.sh` side (or vice versa for `destroy_collection`, which still requires 404/422 there), both resources are always attempted regardless of the other's outcome, and if one side hits a genuinely unexpected status (500, for example) that is what propagates, not a tolerable 403 that happened to be checked second. This closes a class of gaps found in review: a single unhandled 403 previously could abort the rest of monolith or microservice teardown (Deployment, Service, PVC, ingress, monitoring, database cleanup) before it ran. **Per-service config inherits from the shared default, except the switch itself and the gateway.** A per-service `[scale.microservices.services.NAME.http_activation]` block merges over `[scale.kubernetes.http_activation]` as a base, so a service that only sets `target_port` still picks up a shared `concurrency_target` from the top level, and a service with no local block at all still inherits a top-level `enabled = true`. An explicit per-service `enabled = false` still overrides that default, so a shared "on" switch can be selectively opted out of per service. The gateway is the one service that never inherits the shared switch (see the scaling-decision point above). **`jac plan` surfaces HTTP-activated services.** The dry-run plan reads the same `scaling` map, so an http_activation service renders its `HTTP activation: 0 -> N` line and appears in the Totals, instead of being invisible in the plan. **A redeploy no longer resets a scaled-to-zero target back to its baseline.** The Deployment manifest's `spec.replicas` is always the configured static default, and every apply PATCHes it onto the live Deployment - with no exclusion for HTTP-activation-mode services, that undid KEDA's own scale-to-zero on every redeploy. `_apply_or_replace` now takes an optional `patch_body` distinct from the manifest used for a fresh create, and the deployment-apply loop strips `replicas` from the PATCH body (not the CREATE body) for `HTTP_ACTIVATION`-mode services, so KEDA's current value is left alone. **No PodDisruptionBudget for an HTTP-activated service, and `jac plan` stops flagging it.** `_build_pdb_manifest` used to compute a replica floor from `hpa.min` regardless of mode, so an HTTP-activated service could still get a PDB with `min_available` set - a budget that's permanently unsatisfiable once the target is actually at zero replicas. It now takes the resolved mode and skips PDB generation entirely for `HTTP_ACTIVATION` services (warning if the user had `pdb` settings configured). `PlanValidator._check_hpa`/`_check_pdb` had the same blind spot for `jac plan`'s diagnostics and now skip both checks for `HTTP_ACTIVATION` services too. **Two services can't silently collide on the same HTTP-activation traffic.** `build_http_activation_spec` now rejects an empty `rules` list at build time - the docs already said an empty list means no traffic ever matches, so a service left at the default was silently deploying unreachable. Separately, `generate_manifests` now runs `_validate_http_activation_rules`, which rejects two services that resolve to identical `InterceptorRoute` match criteria (for example, two services that both inherit the shared top-level rules unchanged) - the KEDA HTTP Add-on interceptor has no way to tell which target a matching request belongs to, so this is now a build-time error naming the colliding services, not a runtime routing ambiguity. **Mode transitions reap the old resource before applying the new one.** `apply_manifests` used to apply the new mode's scaling resource (metric autoscaler, or the HTTP `ScaledObject`) before calling `reap_stale_scaling_resources_for_bundle`, which tears down the previous mode's leftover resource. A service switching modes on a given redeploy collided with its own stale resource - KEDA's admission webhook rejects a second `ScaledObject` on an already-managed target - and only got cleaned up on the next deploy. The reap call now runs before both applies. **A deploy bundle that's entirely HTTP-activation-scaled no longer crashes.** `_wait_for_fleet_rollout` called `self._load_kube_config()`, a method that doesn't exist anywhere in this codebase - the real method is `_load_cluster_config()`. This crashed with an `AttributeError` in exactly the case the surrounding code path exists to handle: a fleet with no metric-scaled services to wait on. **Traffic routing to the interceptor is not wired yet (documented limitation).** jac-scale creates the `InterceptorRoute` and `ScaledObject`, but does not yet rewire the gateway or Ingress backend to the KEDA HTTP interceptor proxy. With `min_replicas = 0`, inbound traffic that reaches the app Service directly (instead of the interceptor) will not wake the pod. The reference doc carries a warning to route through the interceptor yourself for now; the routing-plane rewire is tracked separately as jaseci-labs#7959. **Namespace-wide sweeps still lack application-identity scoping.** Every resource-kind sweep in `destroy()` (Deployments, Services, PDBs, Ingresses, ConfigMaps, and the KEDA resources above) uses a namespace-wide `managed=jac-scale` label selector with no application-identity component, so two jac-scale applications sharing one namespace would delete each other's resources on teardown. This is a pre-existing characteristic of the whole method, not something this PR introduces; fixing it properly requires an app-identity label added at manifest-build time across several files. Tracked separately as jaseci-labs#7710 rather than patched partially here. - `test_http_activation_config.jac` (12 tests): config-to-spec translation (disabled/absent, minimal config defaults, one full-config test covering every optional field, and the four jac.toml-relative validation errors - `target_port`/`target_port_name` mutual exclusivity, a missing `concurrency_target`/`request_rate_target`, empty `rules`, and a rule header missing `name`), the injectable apply/destroy wiring helpers, and that only a 403 from `destroy_http_activation` is caught (a 500 still propagates). Uses a shared `_assert_config_error(cfg, *substrings)` helper instead of hand-rolled raise/catch/assert blocks per test. - `test_http_activation_microservices.jac` (17 tests): the per-service config pass-through and top-level inheritance (including `enabled` itself, and an explicit per-service opt-out); the single scaling decision in `_resolve_scaling` - that an http_activation service resolves to `HTTP_ACTIVATION` mode carrying the `<name>-deployment` scale target and emitting no metric autoscaler, that the gateway is never HTTP-activated even under a shared default (exercised with a real logger so a wrong log-method name is caught without a live deploy), and that a plain service resolves to `METRIC`; the per-service apply loop (only HTTP_ACTIVATION-mode services are applied, using the carried scale-target and service names); the mode-transition reap tests (a METRIC service has its stale HTTP activation resources reaped and its autoscaler left alone, and vice versa, a NONE service has both reaped, and an empty bundle is a no-op); and the cross-service rule-collision validation (identical rules raise, distinct/empty/lone-peer rules don't, and `generate_manifests` raises when two services inherit the shared catch-all rule unchanged). - `test_keda_http_activation.jac` (46 tests): imports the shared fixtures instead of defining them locally, plus the `destroy_http_activation`/`destroy_collection` error-handling coverage - that `destroy_http_activation` still attempts the `ScaledObject` delete after a 403 on the `InterceptorRoute` delete; that a 500 on one delete is not masked by a 403 on the other; that `destroy_collection` tolerates a 403 listing `InterceptorRoute`s without aborting the `ScaledObject`/`TriggerAuthentication` sweep; and that a 403 listing `ScaledObject`s still raises. - `test_plan.jac` (23 tests): includes a case asserting an http_activation service renders its scale-to-zero line and counts in Totals, with no HPA line, and that the metric-autoscaler smoke test reads the new `scaling` bundle key. - `test_pdb_budget.jac` (11 tests): the existing PDB floor/emission cases, plus that an HTTP-activated service gets no PDB even with `pdb.min_available` set, and that the plan validator skips both the HPA and PDB checks for an `HTTP_ACTIVATION` service. - `test_show_yaml_autoscalers.jac` (5 tests): `jac plan --show-yaml` renders the HPA/`ScaledObject` body the apply path would build, that the scale target comes from the resolved plan rather than being re-derived from the service name, that an http_activation-scaled service renders no autoscaler manifest, and that a bundle with no scaling renders nothing. - `http_activation_test_support.jac`: shared fixtures (`default_http_activation_spec`, `default_http_activation_config`, `mocked_keda_autoscaler`) extracted from `test_keda_http_activation.jac` so both files use one source of truth instead of duplicating mock setup. The HTTP activation, KEDA autoscaler, plan, memory-trigger-guard, deployment-overlay, dry-run-purity, ms-config-isolation, and superseded-deployment-reap suites all pass under `jac test`. The microservices `destroy()` engine-gap wiring itself has no dedicated test: like the rest of that method, it requires a live cluster (`_load_cluster_config`, real client construction) with no existing unit-test precedent in this codebase. The error-handling logic it depends on (`destroy_collection`'s group-aware tolerance) is unit-tested directly, as listed above. It was, however, exercised end to end on a real cluster (see Notes to Reviewer). | File | What Changed | |---|---| | `jac/jaclang/scale/deploy/autoscale/http_activation_config.jac` | New. `build_http_activation_spec` translates a `[*.http_activation]` config dict into an `HTTPActivationSpec`, with jac.toml-relative validation errors, including an empty-`rules` check and a friendly error (instead of a bare `KeyError`) for a rule header missing its required `name` key. `apply_http_activation_for_target` / `destroy_http_activation_for_target` wrap the `KEDAAutoscaler` calls behind an injectable interface, shared by both deploy paths below. `destroy_http_activation_for_target` is unconditional (no config gate, so a resource created while enabled stays cleanable even if disabled later) and swallows only a 403 from `destroy_http_activation` (via null-safe `e?.status`, since the checker can't prove `ApiException` isn't the `_optdeps` fallback `Exception`), re-raising anything else. | | `jac/jaclang/scale/deploy/autoscale/keda_autoscaler.impl.jac` | `destroy_http_activation` now always attempts both the `InterceptorRoute` and `ScaledObject` deletes regardless of the other's outcome, and prefers surfacing a non-403 error over a 403 if both fail differently. `destroy_collection` now tolerates 403 specifically for the `http.keda.sh` group, leaving `keda.sh` groups at the existing strict 404/422 tolerance. | | `jac/jaclang/scale/deploy/autoscale/keda_autoscaler.jac` | `destroy_collection`'s interface gains a `best_effort: bool = False` parameter for the group-aware 403 tolerance above. | | `jac/jaclang/scale/deploy/autoscale/http_activation.jac` | Renames `HTTPRequestRateMetric.window` to `rate_window` to match the config schema and generated YAML field name. | | `jac/jaclang/scale/config/plugin_config.jac` | New `[scale.kubernetes.http_activation]` schema block (master switch, replica bounds, target port, concurrency/request-rate metrics, routing rules, cold start, timeouts, custom scale target). Documents the per-service `[scale.microservices.services.NAME.http_activation]` override, which falls back to the top-level block. The ~20-field nested schema is now built once by `_http_activation_nested_schema()` and shared by both call sites instead of being duplicated verbatim (the two copies had already drifted: the `rules` description claimed "empty matches all traffic," the opposite of the now-enforced behavior). | | `jac/jaclang/scale/deploy/target/kubernetes/kubernetes_config.jac` | New `http_activation: dict[str, any]` field on `KubernetesConfig`, wired into `to_dict()` / `from_dict()`. | | `jac/jaclang/scale/deploy/target/kubernetes/kubernetes_target.jac` | Monolith teardown (`_destroy_application`) calls `destroy_http_activation_for_target` next to the base-autoscaler destroy, with the engine-gap sweep described above. Also removes `KubernetesTargetBase.destroy`, `_destroy_component`, `_destroy_application`, `_destroy_databases`/`_destroy_database`, and `_wait_for_deletion` (about 360 lines): all were shadowed by `KubernetesTarget`'s own overrides for every real deploy and unreachable in practice. Note for future readers: `_destroy_component` was the reference implementation jaseci-labs#8403 and jaseci-labs#7968 point at for full-teardown component dispatch. After this merges, addressing either issue means writing that dispatch logic fresh in `target.jac` rather than reusing this block. | | `jac/jaclang/scale/deploy/target/kubernetes/manifest_builder.jac` | New `ScalingMode` enum and `_resolve_scaling(svc_name, deployment, service)`: the single scaling decision, reading the scale-target name off the built Deployment manifest, enforcing the gateway exclusion and metric-vs-activation exclusivity. `_get_http_activation_config` merges `[scale.kubernetes.http_activation]` as a base with the per-service block. The build loop emits one `bundle["scaling"]` map (replacing the separate `autoscalers`/`http_activations` keys). `generate_manifests` now also calls `_validate_http_activation_rules` to reject cross-service rule collisions, and `_build_pdb_manifest` takes the resolved mode and skips PDB generation for `HTTP_ACTIVATION` services. | | `jac/jaclang/scale/deploy/target/kubernetes/target.jac` | `apply_http_activations_for_bundle`, the autoscaler apply loop, and the post-deploy reachability-skip set all read `bundle["scaling"]` by mode, using the carried scale-target/service names with no name re-derivation. `destroy()` engine-gap fix (see Notable design points). `apply_manifests` now reaps stale scaling resources before applying the new mode's resource, `_apply_or_replace` takes an optional `patch_body` so a redeploy's PATCH strips `replicas` for `HTTP_ACTIVATION`-mode services instead of resetting KEDA's scale-to-zero, and `_wait_for_fleet_rollout`'s crash on an all-activated fleet is fixed (`_load_cluster_config`, not the nonexistent `_load_kube_config`). | | `jac/jaclang/scale/runtime/cli/plan.jac` | Reads the `scaling` bundle key; new `HTTPActivationView` rendered beside `HPAView`, and Totals counts autoscalers and HTTP activations by mode. `PlanValidator._check_hpa`/`_check_pdb` now read the resolved mode and skip both checks for `HTTP_ACTIVATION` services. | | `jac/jaclang/scale/tests/deploy/http_activation_test_support.jac` | New. Shared fixtures (`default_http_activation_spec`, `default_http_activation_config`, `mocked_keda_autoscaler`) extracted from `test_keda_http_activation.jac`. | | `jac/jaclang/scale/tests/deploy/test_http_activation_config.jac` | Config-to-spec translation tests plus the injectable apply/destroy wiring-helper tests (12 tests), using a shared `_assert_config_error` helper. | | `jac/jaclang/scale/tests/deploy/test_http_activation_microservices.jac` | New. Per-service config inheritance, the `_resolve_scaling` decision (scale-target name, gateway exclusion, metric-vs-activation exclusivity), the mode-transition reap tests, the apply-loop tests, and the cross-service rule-collision validation (17 tests). | | `jac/jaclang/scale/tests/deploy/test_keda_http_activation.jac` | Imports shared fixtures instead of defining them locally, plus the `destroy_http_activation`/`destroy_collection` error-handling tests (46 tests). | | `jac/jaclang/scale/tests/microservices/test_plan.jac` | Reads the new `scaling` bundle key; adds an http_activation render/Totals test (23 tests). | | `jac/jaclang/scale/tests/deploy/test_pdb_budget.jac` | Adds coverage that an `HTTP_ACTIVATION` service gets no PDB even with `pdb.min_available` set, and that the plan validator skips both the HPA and PDB checks for it (11 tests). | | `jac/jaclang/scale/tests/deploy/test_show_yaml_autoscalers.jac` | Adds coverage that `jac plan --show-yaml`'s scale target comes from the resolved plan (not re-derived from the service name) and that an http_activation-scaled service renders no autoscaler manifest (5 tests). | | `jac/jaclang/scale/runtime/cli/diagnostics.jac` | `_check_hpa`/`_check_pdb` read the resolved scaling mode and skip both checks for `HTTP_ACTIVATION` services. | | `jac/jaclang/cli/docs/reference/plugins/jac-scale-kubernetes.md` | Restores the "HTTP Add-on Activation" reference section with the new config keys, both deploy-path examples, a note on the still-available programmatic API, and a warning that inbound traffic must be routed through the interceptor for now (jaseci-labs#7959). | | `jac/jaclang/cli/docs/tutorials/production/kubernetes.md` | New "Scale to zero on an HTTP request" subsection alongside the existing autoscaling tutorial content, with a minimal `jac.toml` example and a link to the full reference section. | | `jac/jaclang/scale/tests/fixtures/keda_http_activation_e2e/` and `jac/jaclang/scale/tests/deploy/keda_http_activation_real_e2e.sh` | Real-cluster e2e fixture and driver, deployed through the `jac.toml` wiring (replaces the old `fixture.yaml`-driven setup). The driver resolves resource names from the unified deploy path's `<app>-deployment` scale target. `keda_http_activation_verify.jac`, the old standalone verify script, is removed - superseded by the `jac.toml`-driven driver. | The full flow was validated end to end on a local `kind` cluster (not just unit tests): - **Runtime channel.** The fixture `jac.toml` carried a `[dev]` stanza (dev binary channel) with `[dev] jaclang_source` pointing at this checkout, so `jac start --scale` ran this branch's deploy code host-side (all manifest generation, the `kubectl apply` calls, and the `InterceptorRoute`/`ScaledObject` reconcile happen in that process). The confirming `jac dev mode - using compiler source at ...` banner appeared on every deploy. - **Cluster.** A single-node `kind` cluster with KEDA core and the KEDA HTTP Add-on installed via helm. Leader election was disabled on the control-plane components for the single-node dev cluster so the pod's first-boot compile could not starve them. - **Result.** `keda_http_activation_real_e2e.sh` PASSED: deploy, idempotent redeploy (get-then-patch, no duplication), both `InterceptorRoute` and `ScaledObject` reconciled to Ready, a request through the interceptor cold-started the target from `0 -> 1` and returned HTTP 200 with the walker's response, then the target scaled back to `0` after the cooldown. - **The three design fixes confirmed on the live cluster:** the `ScaledObject`'s `scaleTargetRef` was `echo-deployment` (not the bare name), the activated service had no metric HPA, and the gateway had a normal HPA rather than a `ScaledObject`. (cherry picked from commit 6ded655)
…at publish time, not from this directory
…a local stdlib copy during backport testing)
MusabMahmoodh
left a comment
There was a problem hiding this comment.
Read the six cherry-picks against their main originals rather than as one diff. The backport discipline is good: -x on every pick so each names its original, the exclusions are stated with reasons instead of being silent, and #7709's test restructuring is mirrored rather than half-applied. #8008 is mine, and test_show_yaml_autoscalers.jac came across intact, so the --show-yaml rendering keeps its coverage on this line. That was the thing I would have checked first.
Three things, and the first is the one I would hold the merge on.
1. Nothing tested this.
ci.yml triggers on pull_request: branches: [main] and push: branches: [main]. This PR targets release/jaclang-0.34.18, so none of it ran. The two checks on the PR are pre-commit.ci (green) and [code]smith (skipped). So 34 files and roughly 3,500 lines of scale changes are going onto the branch jacBuilder pins its deploys to, and the only thing that has looked at them is a formatter.
That is a different risk profile from the same code on main, because on main the next PR pays for a mistake and here a fleet does. The 2026-08-06 template regression is the shape I have in mind: config-level change, correct-looking diff, found in production.
What I would want before this merges is one of: a workflow_dispatch run of CI against this ref (the trigger is already there, it just needs pointing at the branch), or the scale suites run on a rig against a binary built from this head with the result pasted here. Either is cheap next to re-deriving it after a pinned deploy misbehaves.
2. The largest change in the diff is a 470-line deletion that the body does not mention.
8eb3b11b2 (the #7709 pick) takes KubernetesTargetBase from 1,318 to 848 lines, removing destroy, _destroy_application, _destroy_component, _destroy_database, _destroy_databases, _destroy_cache, _destroy_dashboard and _wait_for_deletion. kubernetes_target.jac still exists on main at full size, so this is not something #7709 did upstream.
I chased it and it is fine: KubernetesTarget.destroy in target.jac overrides the base and references none of the seven removed helpers, so the removed family was already unreachable on this branch and nothing calls into it. Good removal, in other words. But the body says every conflict was resolved by applying only the named PR's own delta and rejecting unrelated main drift, and this is unrelated main drift that was taken. On a release branch that is the sentence a release reviewer most needs, and right now they have to derive it the way I just did. One line in "what deliberately did NOT come along" saying the dead destroy family went too, and why it is dead, would close it.
3. A PDB behavior change ships here with its test left behind.
_build_pdb_manifest's HTTP_ACTIVATION early return came across, and test_pdb_budget.jac did not, because the replica-floor machinery it also covers is separate main work. That reasoning is right for the machinery and wrong for the early return: the branch now changes what a PDB looks like for an HTTP_ACTIVATION service with nothing asserting it, on a branch where point 1 means nothing asserts anything. If the budget half of that file is what does not port, the early-return half is small enough to bring over on its own.
|
Filed the general case as #8947, since it is not yours to fix in a backport PR: no workflow in the repo triggers on a Point 1 of my review still stands for this PR specifically. A |
…t test_pdb_budget left behind The early-return in _build_pdb_manifest and the plan validator's HTTP_ACTIVATION skip shipped in the jaseci-labs#7709 pick, but their tests lived in test_pdb_budget.jac alongside main's replica-floor machinery that this line does not carry. These two are the machinery-free half, rehomed in test_http_activation_microservices.jac using its existing fixtures.
|
@MusabMahmoodh all three addressed, in your order. 1. Test evidence, pasted, plus stronger system-level evidence than a CI dispatch would give. A
Beyond the suites, this head has also been under a real fleet: 2. The 470-line deletion is now in the body. Added to "what deliberately did NOT come along", stating the whole legacy destroy family that left with the #7709 pick, that it is unrelated main drift which was taken, and why it is safe: 3. The machinery-free half of the PDB coverage is ported.
|
MusabMahmoodh
left a comment
There was a problem hiding this comment.
Re-reviewed the new commit (5de5390). Rehoming the two machinery-free tests into test_http_activation_microservices.jac and reusing its existing fixtures is the right call over carrying test_pdb_budget.jac and the replica-floor machinery this line does not have. Both tests drive a real ManifestBuilder over a real temp project rather than stubbing the bundle, which is what I would have asked for.
Two things.
The PDB assertion passes for the wrong reason too. assert bundle.get("pdbs", {}).get("billing_ops") is None; holds when the PDB is correctly suppressed, and equally when pdbs is absent, when the service key never made it into the bundle, or when generate_manifests returned early for an unrelated reason. It cannot tell "suppressed because HTTP_ACTIVATION" from "PDB generation did not run".
A second service in the same services dict, same pdb.min_available = 1, without http_activation, fixes it: assert that one is in bundle["pdbs"] and billing_ops is not. Same builder call, one more dict entry, and then the test can only pass for the reason it claims.
The validator test asserts on message prose. "HPA" in m and "PodDisruptionBudget" in m or "pdb." in m go quiet the day someone rewords the diagnostic, and quiet in the direction of passing. Does the diagnostic object carry a code or a rule identifier alongside .service and .message? If it does, asserting on that is the version that survives a rewording.
Point 1 from my last review still stands, and is sharper now. The only check on this head is [code]smith, skipped. A test-only commit landed on a backport branch and nothing executed it. #8947 covers the general case, but for this PR a workflow_dispatch run of CI against the ref would settle it today rather than merging four test commits nothing has run.
Why
jacBuilder runs the pinned 0.34.x release line, and its KEDA scale-to-zero work (jacBuilder#1514, the #8695 wake-latency measurements) needs the declarative KEDA configuration that only exists on main. This PR brings the complete KEDA scaling suite to
release/jaclang-0.34.18so a pinned deploy can drive HTTP scale-to-zero, per-service scale-rate policy, and TriggerAuthentication reconciliation straight fromjac.toml.What is backported (in main's merge order)
hpa.behavioroverlay for HPA/KEDA scale-rate policy--show-yaml(prerequisite: #8424 extends exactly this rendering path, it cannot apply without it)--show-yamljac.toml, one mutually-exclusive scaling decision per serviceAll six are
cherry-pick -x, so each commit names its main-side original. #7709's test restructuring is mirrored faithfully:test_keda_autoscaler.jac,test_keda_http_activation.jacandkeda_http_activation_verify.jacare deleted here exactly as on main, superseded bytest_http_activation_config.jac,test_http_activation_microservices.jacand the sharedhttp_activation_test_support.jac.Conflict resolutions, and what deliberately did NOT come along
The 0.34.x line diverged from main at 0.34.6 (July 25), so every conflict was resolved by applying only the named PR's own delta and rejecting unrelated main drift:
plugin_config.jac/runtime/docs.md: only thehpa.behaviorandhttp_activationschema additions were grafted onto the 0.34 text; main's unrelated schema drift (nested_eachservice schemas,filesubkey, changedcpu_targetdefault) was not taken.manifest_builder.jac: theScalingModeenum,_resolve_scaling, the scaling-map bundle shape, and_build_pdb_manifest's HTTP_ACTIVATION early return came in; main's replica-floor/pdb-budget machinery (_replica_floor,pdb_spec,min_available) is separate main work and was not taken, sotest_pdb_budget.jac(which tests that machinery) is also not ported.target.jac: fix(scale): render autoscaler manifests into --show-yaml, so the reviewed YAML matches what a deploy applies #8008's_autoscaler_spec/render_autoscaler_manifestsextraction and feat(jac-scale): support KEDA HTTP Add-on activation via declarative jac.toml config #7709's scaling-map apply loop were taken; the extracted_resolve_service_triggerskeeps 0.34 semantics (itshas_memory_requestread defaults to true, which reproduces this branch's existing behavior exactly).kubernetes_target.jac: feat(jac-scale): support KEDA HTTP Add-on activation via declarative jac.toml config #7709's deletion of the legacy imperative destroy/wait path applied cleanly once resolved; no orphaned references remain. One deliberate exception to the no-main-drift rule rode along here:KubernetesTargetBasealso loses the whole legacy destroy family (destroy,_destroy_application,_destroy_component,_destroy_database,_destroy_databases,_destroy_cache,_destroy_dashboard,_wait_for_deletion, ~470 lines) that main removed separately. It is dead code on this line too:KubernetesTarget.destroyintarget.jacoverrides the base and references none of the removed helpers, so nothing could reach them before or after.test_pdb_budget.jacitself is not ported (it tests main's replica-floor machinery), but the machinery-free half of its coverage, the_build_pdb_manifestHTTP_ACTIVATION early return and the plan validator's HTTP_ACTIVATION skip, is rehomed as two tests intest_http_activation_microservices.jac(now 32 passing).test_plan.jac: the new HTTP-activation render test was adapted to this branch's_run_plan(ms_cfg_pair=...)fixture (main'sms_tomlparameter is later drift). One assert intest_memory_trigger_guard.jacusesstr(...)around a MagicMock because this line's checker is stricter than main's; it landed one commit later than intended (in the fix(scale): render autoscaler manifests into --show-yaml, so the reviewed YAML matches what a deploy applies #8008 pick) during resolution..github/workflows/ci.yml: feat(jac-scale): support KEDA HTTP Add-on activation via declarative jac.toml config #7709'stest-scale-k8se2e lane was NOT ported. It depends on main-only CI infrastructure (jac-kitaction,build-kitjob, blacksmith runners) that this branch's CI does not have. The real-cluster e2e script itself (keda_http_activation_real_e2e.sh) ships and can be run manually.release_notes/unreleased/).Validation
Everything below ran with the published
jac 0.34.18linux-x86_64 release binary against this branch's source:jac/jaclang/scale/tests/deploy/: 212 passed, 4 skipped after the final pick (232 passed, 4 skipped includingtest_plan.jacafter the fmt pass).test_http_activation_config12 passed,test_http_activation_microservices30 passed, plustest_show_yaml_autoscalers,test_memory_trigger_guard,test_factories,test_deployment_overlay,test_planall green.jac/jaclang/scale/tests/microservices/: 217 passed; the 12 failures intest_drain,test_fleet_pod_optoutandtest_rate_limitfail identically on the untouched branch head (verified side by side at 529fb29), so they are pre-existing on this line and unrelated to the backport.jac fmtwith the 0.34.18 binary over every touched file, committed separately per this branch's convention.