feat: Operations Passwords tab and Service Health (#1635, layer 1 of 3)

CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- ✨ **Administration > Operations** now shows **Service Health**: whether the Worker and the Scheduler are alive, what each is doing and since when, where it runs and which version, from a heartbeat each service writes to the database every 5 seconds. Each service is **Healthy**, **Degraded** or **Unhealthy**, with the condition behind it in plain words: a service that has stopped or never started is Unhealthy (no heartbeat, or never started), one whose heartbeat is late or whose work has not moved for 10 minutes is Degraded (heartbeat overdue, or stalled). An Unhealthy or stalled service raises an administrator-only banner across the portal with links to Operations and the logs; a service running a different version from the portal is called out too. Until now the only liveness signal was the container health check, which nobody at the portal could see, so a stopped Worker looked exactly like an idle one until a Schedule failed to run. Also available as `GET /api/v1/system/health` and `Get-JIMServiceHealth`, whose `-Summary` gives a monitoring script one `Overall` status to alert on. See [Operations](https://docs.junctional.io/configuration/operations/). (#1635)
 - ✨ Run Profiles gain Safeguards against turning one run into a mass change. Export Run Profiles can cap how many creates, updates and deletes a single run may attempt; Full Import Run Profiles can cap how many Connected System Objects a single run's deletion detection may newly mark as deleted, as a count and/or a share of the Connector Space. A run that would exceed a limit attempts or marks none of that kind of change, leaves it exactly as it was, and warns rather than failing, so a broken filter, base DN or a mistaken rule change cannot turn one run into a mass write or a mass deprovisioning; a refused Full Import deletion detection also does not count as successful for the post-clear reconciliation gate. Available in the portal, REST API and PowerShell (`New-`/`Set-JIMRunProfile -MaxCreates/-MaxUpdates/-MaxDeletes/-MaxDetectedDeletions/-MaxDetectedDeletionsPercent`). See [Run Profiles > Safeguards](https://docs.junctional.io/configuration/run-profiles/#safeguards). (#1618)
 
 - ✨ The cancellation of a scheduled Metaverse Object deletion is now recorded on the rejoining record's Lineage, naming the Connected System that rejoined and the Deletion Rule that permitted the cancellation, so an administrator can see why an object survived a deletion the Lineage shows was scheduled. See [Grace period](https://docs.junctional.io/configuration/metaverse/#grace-period). (#1620)
@@ -47,9 +48,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - ✨ A Connected System can now be told to refuse to send a password over a connection JIM cannot confirm is encrypted. **Only send passwords over an encrypted connection**, on the Connected System's Settings tab under Passwords, governs every password JIM sends to that system: the first password on an account it provisions, one an administrator sets by hand, and a synchronised password change. JIM previously warned and sent anyway in all three cases, because a signed and sealed bind is a legitimate encrypted alternative it cannot detect from a system's settings, so refusing by default would refuse a valid configuration; this is how an administrator who knows their deployment closes that gap. Nothing is discarded when JIM refuses: queued password changes wait, accounts stay owed their first password, and an administrator setting one by hand is told at the time. Also on the Connected System REST resource and `Set-JIMConnectedSystem -RequireSecureTransport`. (#1119)
 - ✨ A password change can now be synchronised to every system a person has an account in, from the portal, the REST API and PowerShell. The Metaverse Object's Actions tab gains **Synchronise Password** beside the existing Set Password, `POST /api/v1/metaverse/objects/{id}/password` accepts one, and `Sync-JIMMetaverseObjectPassword` scripts it. It is a different operation from setting a password on chosen accounts, not a replacement: that one sets a password you choose on the accounts you tick, straight away, and tells you whether each target accepted it; this one records that the person's password changed and delivers it to every Connected System enabled for Password Synchronisation, retrying if one is unavailable. It defaults to leaving expiry to each system's own policy, because a password the person chose should not demand they choose another one at next sign-in. (#1119)
 - 🔒 REST endpoints that accept a password now refuse the request unless JIM can confirm the transport is encrypted, rather than relying on the deployment having enabled HTTPS. An operator who had not sent the one value JIM can never rotate quietly across the network in the clear, with nothing anywhere saying so. The refusal names its remedy, including the likeliest legitimate cause: TLS terminating at a reverse proxy JIM has not been told to trust, which is fixed by setting `JIM_TRUSTED_PROXIES`. Local development over plain HTTP is unaffected. A build-time guard fails if a new password-accepting endpoint is added without the check. (#1119)
-- ✨ Password Synchronisation now has a queue page. **Administration > Password Synchronisation** lists every password change on its way to a Connected System, one row per person per system, with the target's own words on anything that failed, counts of what is waiting, parked, expired and cancelled, and filters by system, state and failure reason. Delivery already worked; what it lacked was somewhere to look, and a change that parked because a directory refused it was visible only by hunting through Activities. No page, API response or log line shows the queued password, and the type the surfaces bind to has nowhere to put one. (#1119)
+- ✨ Password Synchronisation now has a queue view. The **Passwords** tab of **Administration > Operations** lists every password change on its way to a Connected System, one row per person per system, with the target's own words on anything that failed, counts of what is waiting, parked, expired and cancelled, and filters by system, state and failure reason; the tab is badged with how many changes are waiting on a person. Delivery already worked; what it lacked was somewhere to look, and a change that parked because a directory refused it was visible only by hunting through Activities. No page, API response or log line shows the queued password, and the type the surfaces bind to has nowhere to put one. (#1119, #1635)
 - ✨ Queued password changes can now be retried or cancelled, from the portal, the REST API and PowerShell. Retry makes matching changes due immediately and raises a delivery pass, which is what an administrator runs once the reason a directory was refusing passwords has been dealt with; cancel stops JIM delivering them. Both act on a single row or on everything the current filters show, over `POST /api/v1/password-synchronisation/queue/retry` and `/cancel`, and through `Resume-` and `Stop-JIMPendingPasswordChange`. Whatever it covers, each is recorded as one Activity rather than one per change: a retry over a directory that has just come back is a single decision, and it is recorded even when nothing matched, so a retry that changed nothing can be told from one that never ran. (#1119)
-- ✨ Cancelling a queued password change now records a **Cancelled** outcome rather than deleting the change, with who cancelled it and when. Deleting it would leave that person's password divergent on that system with nothing anywhere saying so, which is the same reasoning that already keeps expired changes. Cancelled changes are counted on the queue page, trimmed by retention alongside any other finished change, and can be put back on the queue by retrying them. (#1119)
+- ✨ Cancelling a queued password change now records a **Cancelled** outcome rather than deleting the change, with who cancelled it and when. Deleting it would leave that person's password divergent on that system with nothing anywhere saying so, which is the same reasoning that already keeps expired changes. Cancelled changes are counted on the Passwords tab of Operations, trimmed by retention alongside any other finished change, and can be put back on the queue by retrying them. (#1119)
 - ✨ The Connected Systems list now shows each system's Password Synchronisation state, sortable and filterable, with its parked and expired counts beside it. The four states are distinguished deliberately: a Connector that cannot set passwords, a system nobody has configured, one that is configured and switched off, and one that is delivering. Switched off is not the same as inactive, because changes keep accumulating for it, and a **Needs attention** filter cuts across the states to find the systems with work stuck behind them. (#1119)
 - ✨ A person's page now has an administrator-only Password Synchronisation tab: what is still owed to which of their Connected Systems, and what their recent password changes actually did on each one. It reads their history from Activities rather than from the queue, because a delivered change leaves the queue, and a view built on the queue alone would show a person's failures and none of their successes. (#1119)
 - ✨ Queued password changes can now be listed and summarised from the REST API and PowerShell: `GET /api/v1/password-synchronisation/queue` and `/queue/summary`, and `Get-JIMPendingPasswordChange`. A recovery across a directory that has just come back is not a job for a browser. (#1119)

docs/administration/deployment.md

@@ -388,8 +388,9 @@ Alternatively, add the `ports` mapping directly to `docker-compose.production.ym
 |---------------------------|-------------------------------------------------|
 | `/api/v1/health`          | Basic liveness check                            |
 | `/api/v1/health/ready`    | Readiness check (includes database connectivity)|
+| `/api/v1/system/health`   | Worker and Scheduler health from their database heartbeats (requires the Administrator role) |
 
-The `jim.web` container includes a Docker healthcheck using the readiness endpoint.
+The `jim.web` container includes a Docker healthcheck using the readiness endpoint. The two unauthenticated endpoints answer for the web tier only; `system/health` is how the background services are observed from outside the portal, and the same report is shown on **Administration > Operations** and returned by `Get-JIMServiceHealth`. See [Operations > Service Health](../configuration/operations.md#service-health).
 
 The `jim.worker` and `jim.scheduler` containers use file-based healthcheck monitoring. Each service writes a heartbeat file periodically during normal operation, and the Docker healthcheck verifies the file is recent. This means `docker compose ps` and orchestrators like Docker Swarm or Kubernetes can detect when a worker or scheduler has stalled, even if the process itself has not exited.
 

docs/api/index.md

@@ -55,3 +55,5 @@ These behaviours are common across the API. The interactive API reference is aut
 ## System endpoints
 
 A small set of system-level endpoints (health, readiness, liveness, version, auth config, user info) are useful for orchestrators, load balancers, and client bootstraps rather than identity management workflows. They are documented in the interactive API reference alongside everything else.
+
+The unauthenticated health endpoints answer for the web tier only. `GET /api/v1/system/health` (Administrator role) reports whether the Worker and the Scheduler are alive and what each is doing, from the heartbeats they write to the database; it is the endpoint for monitoring that needs to know whether JIM's work is actually being done. See [Operations > Service Health](../configuration/operations.md#service-health) for what each status and condition means.

docs/concepts/passwords.md

@@ -217,7 +217,7 @@ Delivery is a Password Delivery task in the Operations queue, so a pass is visib
 
 ### 🔎 Watching the queue
 
-Delivery works on its own, which is exactly why you need somewhere to look when it does not. The **Password Synchronisation** page under Administration lists every change on its way to a Connected System, one row per person per system, with what the target said about it.
+Delivery works on its own, which is exactly why you need somewhere to look when it does not. The **Passwords** tab of **Administration > Operations** lists every change on its way to a Connected System, one row per person per system, with what the target said about it. It sits beside the Queue, History and Schedules tabs because it answers the same question they do: what JIM is doing, and what it has stopped doing. The tab is badged with how many changes are waiting on a person (parked plus expired), so a backlog is visible from anywhere on the Operations page.
 
 It never shows a password, and cannot: the queued value is encrypted in the database and has no representation on any page, in any API response, or in any log line.
 
@@ -242,7 +242,7 @@ You are also told where the work is without going looking for it. The **Connecte
 
 That last view reads from the Activities rather than from the queue, deliberately. A delivered change leaves the queue, so a view built on the queue alone would show a person's failures and none of their successes.
 
-Everything on the page is scriptable, because a recovery across a directory that has just come back is not a job for a browser:
+Everything on the tab is scriptable, because a recovery across a directory that has just come back is not a job for a browser:
 
 ```powershell
 # What needs a person right now

docs/configuration/index.md

@@ -25,8 +25,9 @@ The central identity store and the searches that surface it.
 
 ## Automation and Operations
 
-Scheduled execution and the audit trail of what JIM did.
+Scheduled execution, the audit trail of what JIM did, and whether the services doing it are alive.
 
+- [Operations](operations.md) -- the queue, history, Schedules and Passwords tabs in one place, with the Service Health of the Worker and Scheduler
 - [Schedules](schedules.md) -- automated, ordered sequences of operations
 - [Activities](activities.md) -- the audit trail of every operation, with status and execution detail