Skip to content

feat: config-driven PGR notifications + role audience + Configurator management - #58

Open
ChakshuGautam wants to merge 105 commits into
developfrom
feat/pgr-notifications-configure
Open

feat: config-driven PGR notifications + role audience + Configurator management#58
ChakshuGautam wants to merge 105 commits into
developfrom
feat/pgr-notifications-configure

Conversation

@ChakshuGautam

Copy link
Copy Markdown
Owner

Config-driven PGR notifications + role audience + Configurator management

Replaces PGR's hardcoded notification decision points with two MDMS masters —
RAINMAKER-PGR.NotificationRouting (who/which channel) + NotificationTemplate
(the message) — authored on the workflow BusinessService and editable in the
Configurator. Feature-flagged (pgr.notification.config.driven), legacy path retained.

Backend

  • pgr-services: NotificationRouter + TemplateRenderer + NotificationService
    config-driven path. audience is a workflow ROLECITIZEN = the filer;
    any role = its holder pool (egov-user roleCodes search); EMPLOYEE kept as a
    legacy alias→assignee. Publishes one pre-rendered event per (recipient × channel)
    to complaints.domain.events.
  • novu-bridge: thin pass-through — identify the Novu subscriber + trigger the
    per-channel workflow (SMS→Twilio, Email→SMTP); WhatsApp → baileys-send-service.
    Adds read-only proxy GET /novu-adapter/v1/logs (from nb_dispatch_log) and
    /integrations (Novu providers, credentials redacted).
  • default-data-handler: PgrWorkflowConfig.json → workflow-v2 + MDMS splitter.
  • baileys-send-service: self-hosted WhatsApp send (QR pairing).

Configurator — Notifications section

  • Configure tab: per-transition notification setup anchored on the BusinessService
    (resolves nextState UUID → applicationStatus), inline add/edit/remove writing the
    two MDMS masters, a static checker, and a business-service JSON toggle.
  • Notification Routing / Templates (MDMS resources), Logs + Providers
    (novu-bridge proxy screens).

Verified

  • All modules compile (configurator tsc; pgr-services / novu-bridge / default-data-handler mvn compile).
  • Live e2e on Bomet: a workflow transition fans out to the correct role pool across
    SMS + Email (Novu 201) — e.g. ASSIGN → PENDINGATLME notifies all PGR_LME holders,
    APPLY notifies the GRO pool + the citizen. WhatsApp via Baileys (pairing-gated).

Notes

🤖 Generated with Claude Code

subhashini-egov and others added 6 commits May 29, 2026 19:27
Expanded the feature request template with additional sections for problem statement, desired outcome, roadmap alignment, scope, capabilities, success criteria, dependencies, risks, child issues, and documentation.
…-patch-1

Enhance feature request template with detailed sections
- Add .github/CONTRIBUTING.md with full guide on labels, titles,
  feature/area classification, Source field, and priorities
- Update bug_report.md: [Bug] prefix, classification section with
  feature/area/priority/source prompts, deployment-aware environment
- Update feature_request.md: add Release 2.11 to roadmap alignment
- Add enhancement.md template with [Enhancement] prefix
- Update user-story-template.md: [Story] prefix, classification section
- Add config.yml: disable blank issues, link to contributing guide
- Delete epic-template.md (stale, referenced near-wallet-roadmap)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ng-guide

docs: Issue filing guide and updated templates
The minio_egov_rainmaker_alias feature flag and its /egov-rainmaker/
location block reproduced Bomet's original (2026-05-18) filestore
presigned-URL fix (Approach B: filestore signs against the public
domain, served via a host-preserving passthrough + TLS hairpin).

That approach has been superseded by the canonical edge-rewrite path
(Approach A): filestore keeps MINIO_URL=http://minio:9000 and nginx
rewrites the host in the JSON body via sub_filter, with /file-store/
re-setting Host: minio:9000 so the AWS4 signature still validates.
Approach A is hostname-portable (uses $host since egovernments#856), needs no
per-tenant MINIO_URL, and drops the gateway-IP/cert-dependent hairpin.

Both live tenants that ever used Approach B are now on A — verified:
egov-filestore running with MINIO_URL=http://minio:9000 and empty
ExtraHosts, and no /egov-rainmaker/ block in either live vhost:
  - bometfeedbackhub.digit.org
  - naipepea
The flag is enabled by no committed host_var and the block is dead
code on every box, so this removal is a no-op at deploy time.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…obileNumberValidation

Completes the mobile number schema rename started in PR egovernments#877:
- digit-mcp tenant_bootstrap: read/write common-masters.MobileNumberValidation
  (countryCode + mobileNumberRegex) instead of common-masters.UserValidation
  (zone + rules.pattern). Source-inherit both mobile_regex and mobile_prefix
  from source tenant's MobileNumberValidation record. user_validation input
  schema updated to the new { countryCode, mobileNumberRegex, default } shape.
- Test utils (local-setup + integration-tests mdms-mobile.ts): query
  common-masters.MobileNumberValidation; derive min/max lengths from regex
  via deriveMobileLengths(); extract allowedStartingDigits from regex pattern.
- Description strings and comments in specs updated throughout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e6b6b82f-b998-45c4-b6d1-daa2a4d1cf9f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/pgr-notifications-configure

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…nmaker-template-block

chore(deploy): drop dead /egov-rainmaker nginx passthrough + flag
@ChakshuGautam

ChakshuGautam commented Jul 2, 2026

Copy link
Copy Markdown
Owner Author

📐 Design doc + PR review + automated test plan: #59

Covers the full architecture (authoring on the BusinessService → DDH splitter → MDMS masters → PGR role-pool resolution → novu-bridge → Novu), the review findings for this PR (severity table — incl. the unknown-channel SMS fallback, the unauthenticated proxy route, and Configure-tab partial-write semantics), and the prioritized automated test plan building on the existing API-driven e2e.

Note: the design demotes Baileys to test-scaffolding — it is not a legitimate WhatsApp provider and is slated for removal (scope enumerated in the doc §2). WHATSAPP stays in the config schema; delivery activates only when a production provider (Meta Cloud API / Twilio WhatsApp via Novu) is onboarded, and until then must terminate as an explicit SKIPPED/NB_NO_PROVIDER — never a fallback.

vinothrallapalli-eGov and others added 2 commits July 2, 2026 14:08
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ChakshuGautam pushed a commit that referenced this pull request Jul 2, 2026
…ness, proxy auth, cache TTL, configurator write-path)

Executes the findings-closure plan (docs/plans/2026-07-02-notification-findings-closure-plan.md),
resolving the actionable review findings on PR #58 across five workstreams.

W1 — Baileys decommission + channel-enable gate
- Delete utilities/baileys-send-service module + BaileysSendClient/BaileysProviderStrategy(+test).
- getNovuWorkflowId() THROWS NB_UNSUPPORTED_CHANNEL for null/unknown channels; never
  defaults to the SMS workflow.
- DispatchPipelineService.process() gates before delivery: unknown channel -> SKIPPED/
  NB_UNSUPPORTED_CHANNEL; disabled channel -> SKIPPED/NB_NO_PROVIDER. channelsEnabled
  defaults to SMS,EMAIL, so WHATSAPP persists an explicit SKIPPED row instead of routing
  to an unprovisioned workflow. WhatsAppBusinessApiProviderStrategy reclaims the bare
  "whatsapp" alias. Remove dead config knobs; correct stale "bypass the log" Javadocs.

W2 — bridge robustness + PGR emitter fixes
- Wrap identifyThenTrigger in try/catch: persist FAILED (ce.getCode() / NB_DELIVERY_ERROR /
  NB_NOVU_TRIGGER_FAILED) then rethrow. RestTemplate connect/read timeouts.
- Per-channel contact gate at both layers (EMAIL needs email; else phone).
- resolveUsersByRole paginates egov-user (pageSize/pageNumber) with uuid dedupe + cap WARN;
  recipients memoized per (audience|assigneeOnly) on success only. Dedupe key consumed only
  after successful publish. Epoch heuristics 10 -> 1e12. Splitter skips+WARNs null fields.

W3 — proxy auth + PII masking + Kong route
- ProxyAuthFilter validates EMPLOYEE Bearer token + allowlisted role, 60s hashed cache.
- Server-side PiiMask on /logs recipient/transaction; IntegrationController allowlist
  projection; mask PII across 6 log statements; DEBUG->INFO. kong.yml novu-bridge-proxy
  GET-only /logs + /integrations routes.

W4 — MDMS notification cache TTL
- 60s TTL (pgr.notification.mdms.cache.ttl.ms); never cache empties; serve last non-empty
  stale on MDMS blip. Corrected false "fall back to legacy" messaging.

W5 — Configurator write-path (the blocker)
- Extracted notificationWritePath.ts: template-first upsert, MDMS phantom-200 ->
  MDMS_DUPLICATE -> update+reactivate, old-pair deactivation on key-change; every mutation
  carries returnPromise. Schema descriptor help text fixed; dead filter + generated
  artifacts removed.

Build: pgr-services / novu-bridge / default-data-handler compile; data-provider +
configurator tsc clean. New/updated unit tests green (channel gate, FAILED-persist,
contact-missing, role-pool pagination, cache TTL, write-path upsert).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ChakshuGautam pushed a commit that referenced this pull request Jul 2, 2026
Findings W1–W5 -> a819c12; test plan Phase 1+2 -> e9ffbbb. Both on fork PR #58.
Phase 3 live E2E deferred to the pilot server + owner-supplied LME/GRO contacts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pradeepkumarcm-egov and others added 5 commits July 2, 2026 16:35
…-validation-schema-update

fix(877): mobile number validation schema update + DB dump cleanup
The Kong `auth-enrichment` pre-function decodes every tokened POST body with
lua-cjson to inject RequestInfo.userInfo, then re-encodes it. Plain lua-cjson
cannot distinguish an empty array from an empty object (both are empty Lua
tables) and re-encodes `[]` as `{}`, silently corrupting every empty JSON array
in the request body before it reaches the upstream service. Non-empty arrays
and genuinely-empty objects are unaffected.

Concretely this turns an MDMS schema's `"x-ref-schema": []` into `{}`, which is
stored verbatim by mdms-v2 and later makes mdms-v2's data `_create` throw
`ClassCastException: org.json.JSONObject cannot be cast to org.json.JSONArray`
in MdmsDataValidator.validateReference — blocking new state-root tenant
onboarding (first hit on ACCESSCONTROL-ROLES.roles). Tenants seeded by a direct
DB dump keep `[]` because they never traverse Kong.

Fix: use a scoped `require("cjson").new()` instance with
`decode_array_with_array_mt(true)` in the auth-enrichment block, so decoded
arrays (including empty ones) carry the array metatable and re-encode as `[]`,
while genuinely-empty objects stay `{}`. `.new()` scopes the setting so it does
not leak to the shared cjson used by other pre-functions. Only the
auth-enrichment block is touched; the boundary-adapter pre-function forwards the
raw body verbatim and is unchanged.

Verified generically on a clean Kong 3.6 + echo upstream: empty arrays
(top-level, nested, deeply nested, arrays-of-empty-arrays) are preserved and
empty objects are not affected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ty-array

fix(kong): preserve empty JSON arrays in auth-enrichment pre-function
…ost-bootstrap

- Add user_only flag to bootstrap endpoint to skip MDMS copy and only re-create ADMIN user
- Call bootstrap twice post-bootstrap: once for state_root tenant, once for city tenant
- Ensures ADMIN user is encrypted with correct STATE_LEVEL_TENANT_ID after egov-user restart
- Fixes "Invalid login credentials" error caused by encryption key mismatch between bootstrap (STATE_LEVEL=pg) and post-restart (STATE_LEVEL=state_root)
- Update Kenya mobile validation to set default=false for proper routing
- Add detailed comments explaining the encryption lifecycle issue and the two-phase bootstrap solution
…key-rekey-post-bootstrap

fix(bootstrap): re-provision ADMIN user with correct encryption key p…
ChakshuGautam pushed a commit that referenced this pull request Jul 6, 2026
Single-file consolidation of all notification-consolidation work: commits on PR #58,
what's live on Bomet, delivery channel state, Twilio templates, test recipients, locale/
preference design, gotchas, and the open work list (preference seeding, configurator
preferences, fresh-box Ansible test, per-recipient locale, WhatsApp emitter wiring).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lokendra-egov and others added 10 commits July 6, 2026 13:53
…nts#1009)

Employee Name was incorrectly required when assigning a complaint,
blocking submission unless an assignee was selected. Flip isMandatory
to false to match the already-correct frontend/micro-ui copy and the
REOPEN/REASSIGN actions in the same config.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…nee-optional

fix(pgr): make assignee optional in Assign Complaint modal (egovernments#1009)
…Exception (egovernments#813)

egov-hrms/employees/_update NPEs on Employee.getReActivateEmployee()
.booleanValue() whenever the field is null. The plain edit form never
forwarded it and the deactivate/reactivate action never set it either,
so every configurator edit failed once an employee lacked this field.

- EditForm: carry over the value from the _search response, falling
  back to false only when it's null.
- EmployeeAction: set it explicitly to match the action being
  performed (false on deactivate, true on reactivate), same as the
  adjacent isActive flag.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ling

Adds per-stack review guidance (Spring Boot, React, Helm/K8s, devops
scripts, docs), path filters for generated artifacts, and enables
gitleaks, hadolint, actionlint, yamllint, and markdownlint. Also
covers release/* base branches and adds a skip-review label escape
hatch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…it-config

ci: add CodeRabbit config to auto-review PRs targeting develop
…eactivate-employee-null

fix(hrms): always set reActivateEmployee to avoid _update NullPointer…
…nts#916)

pgr-services' _count wraps the same LIMIT/OFFSET-bound query used for
_search in count(*), so it never reports more than the requested
limit — passing the UI's page size (10) as the count's limit meant
the reported total was always capped at one page, keeping Next
disabled/pagination broken for any inbox with more than one page.

Request the count with a limit far above any realistic per-tenant
filtered inbox instead of the page size, so it reflects the true
total. This is a ceiling, not a fix for the underlying backend query
builder — verified against the live deployment: limit=10 capped the
count at 10 for a tenant with 396 matching complaints; limit=10000
correctly returned 396.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…eiling (egovernments#916)

Per review on PR egovernments#1058 (vinothrallapalli-eGov): pgr-services' _count is
meant to be unpaginated. Verified against the live deployment that
omitting limit/offset entirely returns the true total (396), since
LIMIT NULL is treated as no limit — no arbitrary ceiling needed.
Replaces the limit:10000 workaround with simply excluding limit/offset
from the count request's params.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…teps

enable_otp_services flipped to true by default in a prior commit, which
also dropped the Kong request-termination mock on /user-otp. Without the
mock, /user-otp/* requests have nothing to answer them unless the real
egov-otp/user-otp/egov-notification-sms stack is running with an actual
SMS gateway wired up (egov-notification-sms defaults to SMS_PROVIDER_CLASS:
Console, which only logs OTP SMS instead of sending it) — so only the
fixed 123456 OTP kept working, breaking first-time real OTP logins.

Restores the mock as the default and documents the exact steps to switch
to real OTP for production directly in kong.yml and _example.yml.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Chakshu and others added 14 commits July 8, 2026 23:12
…) + robust NOTIF_TENANT

count_rows hit the path-variant _search that returns 0; use the body-schemaCode
form (same quirk the configurator hit). NOTIF_TENANT falls back to the root of
state_tenant_id when state_root is unset. Add-on run on Bomet verified: ok=4
failed=0, created the 18 missing hi_IN template rows (idempotent otherwise).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…PII scrub)

Removes docs/plans/2026-07-06-notification-work-log-resume.md (internal resume with
a real Novu key + test-recipient PII + private IPs) and the two 06-29 plan docs
(superseded by the 2026-07-02 design; carried 10.0.0.x registry refs). Kept the
clean product-design docs. NOTE: the Novu key that was in the work-log should be
rotated — it existed in earlier commit history.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
10.0.0.4:5000 -> <vpc-registry>:5000 in the .example host_vars (the group_vars
docker_registry default stays — it's the documented convention). Secret/PII scrub.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The SMTP/nodemailer provider form was missing a port field, and Novu validates
credentials.port as a STRING (numeric -> 422). Added as a text input so it
serializes as "587". Closes an env finding from the TASK-031 e2e run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…quired (add audience)

Strict MDMS v2 rejects the ProviderTemplate schema with UNIQUE_ATTRIBUTE_LIST_ERR
because x-unique included 'audience' but 'required' did not. Add 'audience' to
required. Surfaced by the fresh-install test on a clean box (Bomet's older MDMS
was lenient). Schema now creates on a fresh install.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tency

Seed no longer aborts the whole install on a single failing row: it logs + skips,
seeds the core Routing+Template masters regardless, and exits non-zero only if a
CORE master fails (ProviderTemplate = WhatsApp ContentSid layer → WARNING). Also
add 'audience' to the SQL-seed script's required list (mirror the schema fix).
Surfaced by the fresh-install test: ProviderTemplate DATA create hits a real MDMS
ClassCastException on strict MDMS (masked on Bomet where those rows were SQL-seeded).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ling (fixes ClassCastException)

Root cause of the ProviderTemplate data-create ClassCastException: MDMS's
schema-create API stores an empty x-ref-schema:[] as {} (an empty JSONObject),
then MdmsDataValidator.validateReference does (JSONArray) get(x-ref-schema) on
that {} and throws for EVERY data create. The validator only runs when the key is
present, so the seed now drops an empty x-ref-schema before create → MDMS stores
none → validator skips it. Verified on a fresh mh-iterations schema: has_xref=false,
all 14 ProviderTemplate rows create. (Upstream MDMS bug: empty-array->object on
schema store — reported separately; this is the deploy-side workaround.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…2E suite catalog

- docs/notification-onboarding/install-fresh.md — enable config-driven
  notifications on a clean box (single two-pass deploy; seed runs with the
  full deploy).
- docs/notification-onboarding/install-upgrade.md — add notifications to an
  existing deployment (bring up Novu beside live data, --tags notifications
  seed add-on, per-tenant gated cutover + rollback).
- local-setup/tests/e2e/notifications/README.md — the A-F case catalog
  (40 cases: 27 PASS / 0 FAIL / 13 SKIP on Bomet), each case linked to its
  test anchor and the production code it exercises, plus the related
  unit/component tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Self-hosted Novu's 'first signup at /novu/' is just a REST call. novu-mint-key.sh
does it headlessly: POST /v1/auth/register (fresh Novu, empty Mongo) creates the
first user + org + Development/Production envs → JWT → read the Development env
key. Idempotent (re-run logs in). Verified against novu self-hosted api:2.3.0.

Wired into playbook-deploy.yml novu-bootstrap: when novu_api_key is unset and
enable_novu is on, wait for novu-api health → mint → resolve effective_novu_key →
wire into compose .env + recreate novu-bridge + (if twilio_* set) bootstrap the
Twilio integration. Collapses the old two-deploy 'sign up + paste key' flow into
a single deploy. Deployments that pin novu_api_key (Bomet/Nairobi) skip minting.

install-fresh.md / install-upgrade.md updated to the single-deploy auto-mint flow
+ 'add a provider via the Configurator or twilio_* host_vars'.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ER seed

Fresh Kenya install aborted at the HRMS INTERNAL_USER seed: the fast-path base
dump ships India's mobile UserValidation (^[6-9][0-9]{9}$), egov-user caches it
in Redis (validationRules, survives restarts). The MCP tenant bootstrap reseeds
the tenant's rule into MDMS (Kenya ^0?[17][0-9]{8}$), but the stale India rule
stays cached -> the INTERNAL_USER mobile (derived from the tenant rule) is
rejected -> deploy aborts -> no ADMIN -> HRMS/user-seed/notif-seed all fail.

Add a task after the tenant bootstrap + egov-user health-wait, before the
INTERNAL_USER seed, that DELs the validationRules key on digit-redis so egov-user
re-reads the freshly-seeded tenant rule from MDMS. Verified on mh: after DEL,
India mobile is rejected and the Kenya rule is enforced. failed_when:false, gated
state_root != 'pg'.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ion is live

Correcting the previous fix (fb99fd0): the root cause is NOT a stale Redis cache
but ASYNC persister lag — the MCP tenant bootstrap reseeds the tenant's mobile
rule (Kenya ^0?[17][0-9]{8}$) into MDMS via Kafka/persister, so for a short
window after bootstrap egov-user still validates against the India base-dump
default (^[6-9][0-9]{9}$). A one-shot INTERNAL_USER create in that window is
rejected → deploy aborts → no ADMIN.

Replace the one-shot uri create with a shell retry loop: up to 30×10s, each round
DELs the validationRules cache and re-POSTs, exiting on 200 or DuplicateUserName.
Verified on mh: once the tenant rule goes live (a few min post-bootstrap), the
derived mobile 199999999 seeds INTERNAL_USER (200).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Next domino after the INTERNAL_USER fix: the MCP tenant bootstrap creates the
state_root ADMIN eg_user, but when it runs during the async UserValidation
window its ADMIN create is rejected on the mobile rule; the task is
ignore_errors, so the deploy continues with NO ADMIN on state_root and every
later ADMIN@state_root token mint (configurator-i18n, dashboards) 400s.

Add an idempotent ADMIN-seed task right after the INTERNAL_USER seed (tenant
rule guaranteed live by then), creating ADMIN with the expected password +
SUPERUSER/LOC_ADMIN/MDMS_ADMIN/EMPLOYEE and a distinct valid mobile.
DuplicateUserName is a no-op. Verified manually on mh: create -> 200, then
ADMIN@ke token -> 200.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…n-fatal

Fresh mh deploy reached the novu-bootstrap and PROGRAMMATICALLY MINTED the Novu
key in-playbook (validates the auto-mint end-to-end), then aborted at the Twilio
integration bootstrap: 'Required command not found: jq' (jq absent on the fresh
box; mh's twilio_* are placeholders anyway).

- Add jq to the Docker-prereq apt install (bootstrap-novu-whatsapp.sh needs it).
- The provider bootstrap is OPTIONAL (key already minted+wired; a provider can be
  added later via the Configurator), so failed_when:false + a visible warning —
  a missing/placeholder cred or transient Novu error no longer aborts the deploy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ring, WhatsApp row identity, email test-send, sidebar alignment

Review findings on the configurator↔novu-bridge provider integration:

1. "SMS templates can't be read from Twilio" — by design (Twilio has no SMS
   template registry; Content templates are WhatsApp-only), but the endpoint
   ignored its channel param and the dialog claimed per-provider discovery.
   /providers/templates now filters by the workflow's Novu step types
   (stepTypeOverviews: SMS/WHATSAPP→sms, EMAIL→email), surfaces `channels`
   per row, and the dialog is retitled "Novu Workflows" with an honest hint.

2. Provider/channel disappearing after page actions — two cooperating causes:
   - WHATSAPP is stored as a Novu `sms` integration, so the created row
     re-listed as a second "sms · twilio" entry. The create path now defaults
     a blank identifier to a deterministic `whatsapp-<sha>` marker and
     rowChannel() derives WHATSAPP back from identifier/name for display.
   - The synthesized react-admin fallback id collapsed both Twilio rows onto
     "twilio:sms" (duplicate ids → datagrid drops/mispaints a row on
     re-render). ensureId now includes identifier/name and customFetchList
     de-dupes ids within a batch.

3. "Provider Templates (WhatsApp)" sidebar label misaligned — the only nav
   label long enough to wrap, and the <span> inherited the button's default
   text-align:center; also `w-4.5 h-4.5` is not a Tailwind v3 utility so the
   icon rendered at intrinsic 24px. Now w-4 h-4 + flex-1 min-w-0 text-left.

Also fixed while verifying:
- EMAIL test-send never delivered: to.email was parsed then dropped (phone-only
  trigger overload); the synthetic nb-test-* subscriber has no stored email, so
  Novu acknowledged and delivered nothing. Now uses the email-capable overload.
- The 7-arg NovuClient.trigger logged the full request body (raw phone +
  message text) AND headers (Novu ApiKey) at INFO. Now masked like the
  pass-through overload.

Tests: novu-bridge 86/86 (adds EMAIL pass-through, WhatsApp identifier marker,
channel-filter coverage); configurator vitest 44/44; vite build clean. e2e A8
updated for the `channels` field + new A8b channel-filter case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@KDwevedi
KDwevedi force-pushed the feat/pgr-notifications-configure branch from 8e0aec5 to f0beb1d Compare July 8, 2026 17:43
KDwevedi and others added 15 commits July 9, 2026 00:29
…plates dialog, dead Credentials column, login tenant prefill

- i18nProvider: update en bundle templates_title/hint/empty to the corrected
  'Novu Workflows' wording; the stale bundle values were overriding the
  component's inline defaults and reverting the channel-filter relabel.
- NotificationProviderList: PullTemplatesDialog is opened via the
  parent-controlled open prop, so Radix onOpenChange never fires on open and
  load() never ran (dialog always empty, zero /providers/templates calls).
  Fetch via useEffect on [open, channel, providerId]; onOpenChange path now
  handles close-cleanup only, so there is no double-fetch.
- Remove the dead Credentials column: the backend allowlist projection never
  emits credentials, so it rendered '--' on every row. Also drop the unused
  Integration.credentials field and the app.providers.col_credentials key.
- LoginPage: the tenant 'prefill' was placeholder-only, so submitting with it
  untouched sent an empty tenantId and no /user/oauth/token request fired.
  Seed formData.tenantCode from a DEFAULT_TENANT_CODE constant (same 'ke'
  default App.tsx already uses) so submit works without re-typing.

Investigated the Logs 'Template' column showing '--': binding is correct
(record.templateKey matches the API model); DispatchPipelineService.persist()
simply never sets templateKey on real dispatch rows, so the data is genuinely
null. Column left in place.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dead-code removals (each re-verified with repo-wide grep, zero production
callers):
- NovuClient: triggerWithProviderConfig + deprecated
  triggerWithProviderCredentials (only caller was ProviderAgnosticTest,
  deleted), plus the now-unused NovuProviderStrategyFactory field/ctor param
- NovuProviderStrategyFactory: getStrategyByName, getAvailableProviders
  (getStrategy stays — live via ProviderController)
- service/UserServiceClient: whole class orphaned (no injection sites);
  userHost/userDetailsPath/user-search config kept for ProxyAuthFilter
- PreferenceServiceClient.getUserPreferredLocale (PGR owns locale now)
- DispatchResult.resolvedTemplate/resolvedProvider/missingRequiredVars +
  the ResolvedTemplateResponse/ResolvedProviderResponse models they alone
  referenced (updated DispatchPipelinePassThroughTest assertions)
- novu.bridge.max.retries property + maxRetries field (zero readers);
  retry Kafka topic wiring kept, commented as reserved/not-yet-published;
  dropped the matching inert NOVU_BRIDGE_MAX_RETRIES compose env
- compose: removed inert NOVU_BRIDGE_CONFIG_HOST/_RESOLVE_PATH/_SEARCH_PATH
  (no novu-bridge code reads a config-service host)
- deleted README-Provider-Agnostic-Setup.md (documented the deleted
  ConfigServiceClient/triggerWithProviderCredentials architecture)

Non-deletion fixes:
- Renamed error code INVALID_MOBILE_NUMBER -> NB_INVALID_MOBILE_NUMBER
  (DispatchPipelineService) to match the NB_ convention
- kong.yml: corrected the novu-bridge proxy comment to the 7 actual routes
  (3 GETs + 4 provider-management), /dispatch/* trio noted as intentionally
  unrouted
- Rewrote README.md to describe the actual pass-through contract: PGR
  pre-renders/localizes one event per (recipient x channel); novu-bridge
  validates the envelope, applies channel + preference gates, and triggers
  the per-channel Novu workflow — no config-service _resolve/_search calls

Provider strategy classes (Vonage/ValueFirst/WhatsAppBusinessApi) left in
place for future extensibility.

mvn test (JDK17 container): Tests run: 84, Failures: 0, Errors: 0 — BUILD
SUCCESS.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The notification-preference custom resource had customTenantScoped:false,
so customFetchList sent no tenantId to
GET /novu-bridge/novu-adapter/v1/preferences. The backend treats a missing
tenantId as unscoped and returns CROSS-TENANT rows (capped at 100), so the
screen leaked other tenants' user preferences and could hide the session
tenant's own rows behind the cap.

Flip the flag to true, matching notification-log's scoped pattern. The
backend's tenantId query param is optional and verified live on bomet:
?tenantId=ke returns 200 with only ke rows, while the unscoped call
returned rows from both ke and ke.bomet.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ws on fresh install

The Ansible bootstrap only created the legacy complaints-whatsapp-v1
workflow, but novu-bridge triggers the fixed per-channel workflows
complaints-sms / complaints-email (novu.bridge.workflow.id.*). On a fresh
box those workflows never existed, so every SMS/EMAIL notification logged
FAILED/NB_NOVU_TRIGGER_FAILED — while the install docs already claimed the
bootstrap created them (they were created out-of-band on the reference
install).

- Extract the inline workflow-create block into ensure_channel_workflow()
  (same GET /v2/workflows existence check, same POST /v2/workflows payload
  wrapper — whatsapp-v1 create payload verified byte-identical).
- Idempotently create complaints-sms (one sms step, body
  "{{ payload.body }}") and complaints-email (one email step, subject
  "{{ payload.subject }}", html body "{{ payload.body }}",
  disableOutputSanitization) — pure pass-throughs of the bridge's
  pre-rendered trigger payload, matching the live definitions on the
  reference install verbatim (steps dumped via GET /v2/workflows/<id>).
- NOVU_SMS_WORKFLOW_ID / NOVU_EMAIL_WORKFLOW_ID env overrides mirror the
  bridge's NOVU_BRIDGE_WORKFLOW_ID_SMS / _EMAIL.
- Docs: install-fresh.md / install-upgrade.md now list all three
  bootstrap-created workflows.

Skip-path verified against the reference install's live workflow list
(zero creates); fresh-list path verified to create exactly the two
missing workflows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…eKey on dispatch rows

Fix 1 — PII leak in /logs: the nested providerResponse map on each dispatch
log row echoed the RAW Novu transactionId (and can carry raw phone numbers
for phone-fallback subscriber keys) to any EMPLOYEE via the Logs screen,
while the top-level recipientValue/transactionId were already masked. Adds
PiiMask.maskDeep(...) (recursive over maps/lists; strings go through
maskEmbedded, scalars pass through exactly) and applies it at READ time in
DispatchLogController's /logs projection only — stored rows are untouched
and non-PII fields (novuStatus, test, status codes) keep exact values, so
nothing downstream breaks.

Fix 2 — real dispatch rows persisted template_key = NULL (only the
ProviderController test-send wrote "TEST"): the pre-rendered event from
pgr-services carries no template identifier on the wire, so persist(...) now
records the best available value — an explicit event templateKey when
present (forward-compatible field added to ComplaintsDomainEvent; pgr's
NotificationService.publishRenderedEvent must start emitting it), else the
routing key audience.action.toState.channel.locale reconstructed from
segments the event already carries, else eventName for legacy envelopes.

Tests: providerResponse deep-mask projection (raw phone/txn never crosses
the wire, scalars intact, no mutation of stored row), maskDeep unit tests,
SENT-row templateKey derivation + explicit-wire-value precedence.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…live bomet verification

Author authoritative FE-architecture + live-state docs for the PGR supervisor
dashboard now embedded in digit-ui-esbuild (PR egovernments#1062, live on bomet), and
correct stale references across the existing config docs.

New:
- dashboard-configuration/70-esbuild-embedding.md — module registry, App.js
  enabledModules, the always-on route fallback vs citymodule-gated card,
  roles.js DASHBOARD_ROLES, embedded mode, analyticsService base + MDMS
  context-path resolution (a7b8a6d), the catalog->tile render pipeline.
- dashboard-configuration/80-live-bomet-state.md — reproducible bomet probe
  (2026-07-09): 37 published defs / 10 PUBLIC tiles, two-pack first-match
  (executive-default vs supervisor-default, the PGR_VIEWER surprise), anon
  inline lock, the catalog-divergence trap (repo seed vs mdms-v2 store vs
  served catalog; egovernments#1026 stale-record no-op), dead-sidebar platform bug,
  empty-tile triage.
- dashboard-rbac-design/62-shipped-and-verified.md — maps each design layer
  (A-F) to the shipped file:line + live proof; carries forward the 2 open items.

Corrected: FE paths frontend/micro-ui -> digit-ui-esbuild/products/dashboard;
stale def count (28 -> live-verified 37/10 PUBLIC); resolved the
DASHBOARD_CARD_HEADER TODO; PR provenance (in-flight egovernments#1074/75/76/79 -> merged
egovernments#1026/28/30/31, embedding egovernments#1062); card-vs-route-vs-sidebar three-gate model.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds docs/dashboard-demo/ — a click-by-click demo runbook for the PGR
supervisor dashboard on bomet, plus a VERIFIED login matrix (who sees the
dashboard vs who doesn't) and a reproducible verify script.

All login/role/pack claims verified live on bomet (2026-07-09) via
/user/oauth/token, /user/_search and pgr-services/v2/analytics/packs.
Gate source: digit-ui-esbuild/products/dashboard/roles.js (DASHBOARD_ROLES).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…otifications

Add docs/notifications-guide/ — a code-referenced, operator-grade guide to the
config-driven PGR notification pipeline (CCRS egovernments#1059, live on bomet):

- README.md: end-to-end architecture, the pgr-services (brain) / novu-bridge
  (pass-through hands) boundary, 12-step flow, a text sequence diagram, feature
  flags/topics, and what is intentionally not notified (escalation, WhatsApp).
- 01-mdms-masters.md: the 3 MDMS masters (Routing/Template/ProviderTemplate) —
  shape, uid schemes, runtime matching (fromState ignored), caching + stale-serve,
  and ke -> ke.bomet state-root inheritance.
- 02-novu-bridge.md: the pass-through pipeline gate table + SENT/SKIPPED/FAILED
  and NB_* code reference, the 7 Kong-routed proxy endpoints + the unrouted
  /dispatch/* trio, ProxyAuthFilter, PII masking, idempotency/DLQ, templateKey.
- 03-channels-providers.md: channel gate, provider self-service
  (add/verify/pull-templates/test-send), Twilio + Gmail SMTP, WhatsApp-via-Twilio,
  preferences/consent (gated off at runtime).
- 04-localization.md: locale as a template dimension vs the single-locale runtime.
- 05-operations.md: runbooks (add routing+template, onboard provider, read logs),
  failure-code table, feature on/off, a read-only health-check probe.

Claims cite file:line; live bomet (ke) probes verify channel/provider/log/consent
state. Supersedes docs/Novu_Adapter and docs/notification-onboarding for the
shipped behavior.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Click-by-click demo script for the 7 Notifications pages in the
configurator (Configure, Notification Routing, Notification Templates,
Provider Templates, Notification Logs, Notification Providers, User
Preferences): what each page is, its MDMS/novu-bridge backing, exact
demo clicks, and talk track. Adds an author->trigger->observe->deliver
e2e sequence, a pre-demo checklist, a 5-minute highlight reel, and the
do-not-demo list (WhatsApp w/o QR, escalation notifications).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ate (CCRS#1106)

The sidebar-outage and egovernments#1026 sections were written mid-investigation with a
since-disproven root cause (mdms-v2 JDK21 image / empty IN()). Correct them to
the real cause (ACCESSCONTROL actions seeded under -TEST vs the standard module
egov-accesscontrol reads; ACTIONS-bridge schema race) and current state (bridged
live on bomet -> sidebar renders; egovernments#1026 table returns 35 rows). Durable bootstrap
fix tracked as fix/mcp-actions-bridge-schema-wait.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The header search input renders its text under the magnifier icon. Root cause:
a global digit-ui `input[type=...]:not(.a):not(.b)…` rule (declared !important,
with a long :not(.class) chain that inflates its specificity well above a plain
class) matches this type=search input and overrides its left padding down to
~12px — so the placeholder/text starts at ~x+12, under the icon (~x+10..26).

Fix: a padding-left override scoped with a no-op `:not(#…)` that lifts the
selector into the ID-specificity column so it beats the global rule's
class/attr chain, plus !important. Applied to both the bundled dashboard.css
(what esbuild actually bundles) and the input.css source. Verified live: the
computed padding-left becomes 2rem and the icon clears the text.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…icates

The generic MDMS create/edit pages (Notification Templates, Routing,
Provider Templates, and every other resource) surfaced a raw
DUPLICATE_RECORD / 'Duplicate record' banner when an operator re-added a
row whose unique key already exists — e.g. a CITIZEN·APPLY·SMS·en_IN
notification template that the seed already provides. Map that case to a
clear message plus a hint naming the fields that make the record unique,
so operators know to edit the existing row or vary a key field.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Backend (novu-bridge):
- DispatchPipelineService: persist a FAILED nb_dispatch_log row on the
  missing-subscriberId path (was the only terminal branch skipping the log);
  mask subscriberId + transactionId in logs (both can embed a raw phone via
  the tenantId:mobile fallback).
- IntegrationController: return 502 instead of 200-with-empty-list when the
  upstream Novu integrations call is non-2xx.
- PreferenceController: clamp /preferences limit (1..500) so a huge page size
  can't force an unbounded result set.

Backend (pgr-services):
- NotificationService: log (DEBUG) instead of silently swallowing the
  assignee/HRMS placeholder-resolution exceptions, so a real HRMS/MDMS
  regression is visible while the expected no-assignee case stays quiet.

utilities (default-data-handler):
- DataHandlerService: re-throw on IOException reading the bundled PGR
  workflow/notification config (unreadable config is a genuine setup failure,
  not something to swallow); log with full stack trace.

configurator:
- validateNotifications: stop R1 (audience-role-exists) flagging the backend
  EMPLOYEE assignee pseudo-audience as an unknown role code.

Ops / seed / docs:
- playbook-deploy.yml: add no_log:true to the INTERNAL_USER and ADMIN seed
  tasks (they interpolate passwords into curl bodies).
- novu-mint-key.sh: build register/login JSON via python json.dumps so values
  with quotes/backslashes don't break the payload.
- seed-notifications.py: drop dead unreachable line referencing undefined 'ok'.
- provider-onboarding-runbook.md: fold verified appendix corrections into body
  (locale property name, credential allowlist projection, NB_TEMPLATE_NOT_APPROVED).
- notification-findings-closure-plan.md: replace real citizen phone/email with
  placeholders (no PII in source control).
- e2e/notifications/README: repeat the E2E_EMP_* env prefix on the subset example.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nfig

- PgrWorkflowConfig.json: ASSIGN->PGR_LME 'lme_assigned' rows (SMS/WHATSAPP/
  EMAIL) set assigneeOnly:true. The template says 'assigned to you', but
  assigneeOnly:false fanned it out to the entire PGR_LME role pool instead of
  just the assignee. CITIZEN row stays fan-out.
- playbook-deploy.yml: the INTERNAL_USER/ADMIN mobile-number fallback guarded
  on 'core_mobile_configs is defined', but group_vars sets it to {} (defined
  but empty) — so the sub-key access broke on the default inventory. Guard the
  actual sub-keys (mobileNumberAllowedStartingCharacters + mobileNumberLength).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
seed-notifications.py prints 'DONE:' even when it exits non-zero on core-master
failures, so the DONE-only failed_when let a broken seed pass and the deploy
continue with incomplete notification config. Also require rc == 0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants