Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 1 addition & 11 deletions .horde.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ license:
uri: http://www.horde.org/licenses/gpl
dependencies:
required:
php: ^7.4 || ^8
php: ^8.1
composer:
horde/compress: ^3
horde/date: ^3
Expand Down Expand Up @@ -67,16 +67,6 @@ keywords:
- eas
- exchange
ci-platform:
'8.0':
- php: '>=8.1'
- composer-plugin-api: ^2.2.0
- ext-bcmath
- ext-dom
- ext-hash
- ext-json
- ext-pcre
- ext-simplexml
- ext-spl
'8.1':
- php: ^8.1
- composer-plugin-api: ^2.2.0
Expand Down
5 changes: 4 additions & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
"repositories": [],
"require": {
"horde/horde-installer-plugin": "dev-FRAMEWORK_6_0 || ^3 || ^2",
"php": "^7.4 || ^8",
"php": "^8.1",
"horde/compress": "^3 || dev-FRAMEWORK_6_0",
"horde/date": "^3 || dev-FRAMEWORK_6_0",
"horde/exception": "^3 || dev-FRAMEWORK_6_0",
Expand Down Expand Up @@ -51,6 +51,9 @@
"autoload": {
"psr-0": {
"Horde_ActiveSync": "lib/"
},
"psr-4": {
"Horde\\ActiveSync\\": "src/"
}
},
"autoload-dev": {
Expand Down
170 changes: 170 additions & 0 deletions doc/ops-monitor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# ActiveSync operations monitor

See the [operations monitor epic](https://github.com/horde/ActiveSync/issues/99).

## Purpose and scope

Version 1 provides a shared, read-only health model for answering:

1. Which devices are active right now?
2. Which sync keys or collections look stuck?
3. Where are loop-guard or heartbeat problems concentrating?
4. How can an operator jump from a bad device to its per-device log and
existing admin actions?

The monitor does not provide historical metrics, alerting, or webhooks. It
does not add HTTP 503 responses or `X-MS-Throttle`, replace
`logging.type = perdevice`, mutate ActiveSync protocol behaviour, or expose
wipe and block operations through the CLI.

## Architecture

| Piece | Package and location |
|-------|----------------------|
| Health heuristics and DTOs | `horde/activesync`, `src/Ops` |
| Snapshot service | `horde/core` |
| CLI `horde-activesync summary\|show\|top` | `horde/horde` (the `horde/base` repository) |
| Admin health badges | `horde/horde` (the `horde/base` repository) |

The evaluator consumes plain facts and has no storage dependency.
`DeviceHealthFactory` is the compatibility boundary that converts legacy
ActiveSync device, cache, and state rows into those facts. The snapshot
service loads fleet state, and both operator interfaces consume its shared
results.

A device's collections are a `CollectionFactList`: an immutable map of
`CollectionFacts`, keyed by collection id and readable with array access.
Each fact copies `class`, `serverid`, `lastsynckey`, `backlog`,
`backlogpings`, and `pingable` from the SyncCache row. Sync options such as
window size, filter type, and body preferences stay on the cache row.

## Health model

Device and collection status is one of `ok`, `warn`, or `critical`. The worst
signal determines collection status, and the worst device or collection
status determines device status.

A device is active when a heartbeat is inferred to be in flight, or when its
latest SyncCache or last-sync timestamp is no older than `activeWithin`.
A device is stuck when it has `hb_stuck`, `hb_missing_end`, `fsr_critical`, or
`backlog_stuck` on either the device or one of its collections.

| Tuning option | Default | Meaning |
|---------------|---------|---------|
| `now` | current time | Evaluation timestamp; fixed values make tests and snapshots deterministic |
| `activeWithin` | 300 seconds | Maximum state age counted as active |
| `hbStuckAfter` | derived | Explicit heartbeat-stuck threshold, when set |
| `hbDefaultInterval` | 900 seconds | Heartbeat interval used when the cache has none |
| `hbSlack` | 60 seconds | Allowance added to the heartbeat interval |
| `hbAbandonedAfter` | 2 × heartbeat-stuck threshold | Age after which an unfinished heartbeat is treated as abandoned rather than stuck |
| `fsrWarnAt` | 3 | FolderSync-required warning threshold |
| `fsrCriticalAt` | 5 | FolderSync-required critical threshold |
| `backlogGrace` | 60 seconds | Minimum backlog age before a pending warning |
| `backlogTriggerMax` | 3 | Backlog recovery count considered stuck |
| `backlogAbandonedAfter` | 3600 seconds | Age after which a capped backlog is treated as abandoned rather than stuck |

When `hbStuckAfter` is unset, the threshold is the cached heartbeat interval
(or `hbDefaultInterval`) plus `hbSlack`.

## Reason codes

| Code | Condition | Severity | What to do |
|------|-----------|----------|------------|
| `hb_in_flight` | Heartbeat start is newer than its end, or no end is recorded, and its age is at most the stuck threshold | `ok` | No action; this is activity context |
| `hb_stuck` | Heartbeat start is newer than a recorded end and its age is between the stuck threshold and the abandoned threshold | `warn` | Check whether the client recently disconnected, suspended radio activity, rebooted, or updated; inspect the device log if the signal persists |
| `hb_missing_end` | A heartbeat start has no recorded end and its age is between the stuck threshold and the abandoned threshold | `warn` | Check for a transient client disconnect; inspect the device log if the signal persists |
| `hb_abandoned` | An unfinished heartbeat is older than the abandoned threshold | `ok` | Nothing — common after client disconnects; if it repeats for an otherwise active device, check the device log |
| `fsr_warn` | FolderSync-required count is at least 3 and below 5 by default | `warn` | Check whether the client completes a FolderSync cycle |
| `fsr_critical` | FolderSync-required count is at least 5 by default | `critical` | Inspect hierarchy state and the device log. The counter clears on FolderSync, or on a later successful PING/SYNC that does not request FolderSync (after ghost-collection heal) |
| `blocked` | Device properties mark the device blocked | `warn` | Confirm the block is intentional in the admin device page |
| `wipe_pending` | Full or account-only remote wipe is pending | `warn` | Confirm provisioning reaches the device |
| `wipe_complete` | Full or account-only remote wipe is recorded complete | `warn` | Confirm the completed state is expected |
| `backlog_pending` | Backlog age reached the grace period and recovery count remains below the trigger maximum | `warn` | Check whether the client drains subsequent Sync windows |
| `backlog_stuck` | A backlog exists, recovery count reached the trigger maximum, and the MOREAVAILABLE is still newer than `backlogAbandonedAfter` | `critical` | Inspect collection sync keys and the per-device log |

The numeric conditions use the configured thresholds rather than fixed
defaults when options are overridden.

## Honest limits

Version 1 is derived from currently persisted device and SyncCache state; it
cannot prove that a PHP request is still running. An unfinished heartbeat is
therefore a warning rather than critical: a radio gap, client update, reboot,
or suspended app can leave the same persisted timestamps as a server-side
problem. An old unfinished heartbeat is treated as abandoned, not stuck,
because a PHP request cannot plausibly remain active beyond that window.
The initial fleet snapshot costs one
SyncCache load per device; it is not a constant-cost aggregate query.
Per-device protocol logs remain the deep-dive source.

## Cookbook (CLI + admin)

### Triage active devices

Start with active devices ordered by health and keep the display refreshed:

```sh
horde-activesync top --watch --active-within=300 --sort=health
```

Narrow a large fleet with `--user`, `--device`, `--health`, or
`--stuck-only`, and cap output with `--limit`. Use `--format` when another
tool consumes the result. The snapshot performs one SyncCache read per device
considered after filtering; it is not a constant-cost fleet aggregate.

### Run a cron or Nagios check

```sh
horde-activesync summary --format=json
```

The command exits with `0` when no critical device is present, `1` when at
least one critical device is present, and `2` when monitoring data is
unavailable. Cron and Nagios wrappers should preserve that exit status and
retain the JSON output for diagnosis.

### Inspect one device

Use the exact user and device identifiers shown by `top` or `summary`:

```sh
horde-activesync show alice@example.com DEVICE1
```

When `logging.type = perdevice`, the protocol log is:

```text
{configured logging path}/{UPPERCASE DEVICE ID}.txt
```

For example, device `DEVICE1` maps to `DEVICE1.txt` beneath the configured
logging path. The path comes from the deployment configuration; do not embed
credentials or deployment-specific paths in scripts or reports.

### Confirm and act in the admin page

Open **Administration → ActiveSync Devices**, find the same user and device,
and verify the health signal against the device details and per-device log.
Only then use the existing wipe, block, remove, or reset actions, with care:
they change device state and can disrupt or erase client data. The
`horde-activesync` CLI has no mutation commands.

### Interpret common signals

- `hb_stuck` and `hb_missing_end` are warnings for an unfinished heartbeat
still inside the plausible stuck window. Transient client disconnects,
radio gaps, reboots, and updates commonly produce this state; inspect the
request lifecycle and device log only when it persists or repeats.
`hb_abandoned` is older than that window and is intentionally `ok`
because the original request can no longer plausibly be running. Repeated
abandoned heartbeats on an otherwise active device still merit log review.
- `backlog_stuck` means a collection has repeatedly failed to drain pending
changes *and* that MOREAVAILABLE is still recent. After
`backlogAbandonedAfter` (or after PING/looping-SYNC recovery hits its
trigger cap and clears the flag), an undrained folder is no longer
fleet-critical: leftover `sync_pending` waits for the next real SYNC, which
some clients never send for folders they only PING (for example iOS Mail
and Trash). Inspect sync keys and the log only while the signal is present.
`fsr_warn` / `fsr_critical` instead mean repeated FolderSync-required
recovery (FSR). After ghost collections are healed, a later successful
PING/SYNC clears that counter even if the client never FolderSyncs again.
31 changes: 31 additions & 0 deletions doc/todo.md
Original file line number Diff line number Diff line change
Expand Up @@ -216,6 +216,37 @@ when work proceeds on FRAMEWORK_6_1.
- `Horde_ActiveSync_Sync_Options` (and similar) for collection options /
body preferences instead of raw arrays.
- Collection object replacing the associative collection array in `Sync.php`.
The array carries ~34 keys across ~440 read sites (`Request/Sync.php` 181,
`SyncCache.php` 138, `Collections.php` 63) and mixes four concerns:
identity (`id`, `class`, `serverid`, `type`), persisted sync state
(`synckey`, `newsynckey`, `lastsynckey`, `getchanges`, `pingable`,
`backlog`, `backlogpings`), client-negotiated options (`windowsize`,
`filtertype`, `truncation`, `mimesupport`, `bodyprefs`, `conflict`,
`deletesasmoves`, `conversationmode`, …), and per-request scratch
(`clientids`, `fetchids`, `removes`, `modifiedids`, `importedchanges`,
`importfailures`, `atchash`, `instanceid_removes`, …). Target shape:
- `SyncCollection` for identity plus persisted sync state, with
`Horde_ActiveSync_SyncKey` as a first-class type (see *State, storage,
and identity* above); `Sync_Options` for the negotiated options; the
Changes object below for per-request scratch. Do not fold all four into
one interface.
- Storage stays an array. `SyncCache` hydrates rows into objects on load
and dehydrates on save; never `serialize()` the objects into
`cache_data`, which would change the blob format.
- `ArrayAccess` on the concrete `SyncCollection` class is a migration
shim so `$collection['class']` keeps working in `Sync.php`, ideally
logging deprecated access. Remove it once `Sync.php` is converted. Keep
it off the interface so the array API does not become permanent.
- Order: `SyncCache::addCollection()` first (it defines the persisted
fields), then `Collections.php`, then `Sync.php` (hot protocol path,
largest, riskiest).
- `Horde\ActiveSync\Ops\CollectionFacts` (health monitor, #100) is a
seven-field read-only projection built in
`DeviceHealthFactory::collectionFacts()`, the only place that converts
cache rows. Once `SyncCollection` exists, build the projection from it
or let `HealthEvaluator` consume `SyncCollection` directly; then drop
`CollectionFacts`. It is not the domain object and must not grow into
one.
- Changes object (array, `SplFixedArray`, or temp stream) to cap memory on
large initial mailbox syncs and to unify the `add` / `modify` / `delete`
shapes between email and PIM collections. Streaming v1 ([#83](https://github.com/horde/ActiveSync/issues/83))
Expand Down
2 changes: 1 addition & 1 deletion lib/Horde/ActiveSync.php
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
<?php

/**
Expand Down Expand Up @@ -789,7 +789,7 @@

// Autodiscovery handles authentication on it's own.
if ($cmd == 'Autodiscover') {
$request = new Horde_ActiveSync_Request_Autodiscover($this, new Horde_ActiveSync_Device($this->_state));
$request = new Horde_ActiveSync_Request_Autodiscover($this);

if (!empty(self::$_logger)) {
$request->setLogger(self::$_logger);
Expand Down
4 changes: 2 additions & 2 deletions lib/Horde/ActiveSync/Request/Base.php
Original file line number Diff line number Diff line change
Expand Up @@ -106,8 +106,8 @@ abstract class Horde_ActiveSync_Request_Base
/**
* Const'r
*
* @param Horde_ActiveSync $as The ActiveSync server.
* @param Horde_ActiveSync_Device $device The device descriptor.
* @param Horde_ActiveSync $as The ActiveSync server. The device
* descriptor is read from the server.
*
* @return Horde_ActiveSync_Request_Base
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ class HordeActiveSyncRemoveSyncCounters extends Horde_Db_Migration_Base
{
public function up()
{
$this->announce('Removing SyncKeyCounter data from cache.', 'cli.message');
$this->announce('Removing SyncKeyCounter data from cache.');
$sql = 'SELECT * FROM horde_activesync_cache';
$rows = $this->_connection->select($sql);
$insert_sql = 'UPDATE horde_activesync_cache SET cache_data = ? WHERE cache_devid = ? AND cache_user = ?';
Expand Down
1 change: 1 addition & 0 deletions phpunit.xml.dist
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
<source>
<include>
<directory suffix=".php">lib</directory>
<directory suffix=".php">src</directory>
</include>
</source>
<testsuites>
Expand Down
107 changes: 107 additions & 0 deletions src/Ops/CollectionFactList.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
<?php

declare(strict_types=1);

/**
* Copyright 2026 The Horde Project (http://www.horde.org/)
*
* See the enclosed file LICENSE for license information (GPL). If you
* did not receive this file, see http://www.horde.org/licenses/gpl.
*
* @author Torben Dannhauer <torben@dannhauer.de>
* @category Horde
* @copyright 2026 The Horde Project
* @license http://www.horde.org/licenses/gpl GPLv2
* @package ActiveSync
*/

namespace Horde\ActiveSync\Ops;

use ArrayAccess;
use ArrayIterator;
use BadMethodCallException;
use Countable;
use InvalidArgumentException;
use IteratorAggregate;
use OutOfBoundsException;
use Traversable;

/**
* Immutable collection-id map of SyncCache health snapshots.
*
* @implements ArrayAccess<string, CollectionFacts>
* @implements IteratorAggregate<int, CollectionFacts>
*/
final class CollectionFactList implements ArrayAccess, Countable, IteratorAggregate
{
/**
* @var array<string, CollectionFacts>
*/
private array $byId;

/**
* @param iterable<mixed, CollectionFacts> $collections
*/
public function __construct(iterable $collections = [])
{
$byId = [];
foreach ($collections as $collection) {
if (!$collection instanceof CollectionFacts) {
throw new InvalidArgumentException(
'Collection fact lists accept CollectionFacts entries only.'
);
}
$byId[$collection->id] = $collection;
}
$this->byId = $byId;
}

public function get(string $id): ?CollectionFacts
{
return $this->byId[$id] ?? null;
}

/**
* @return list<CollectionFacts>
*/
public function all(): array
{
return array_values($this->byId);
}

public function getIterator(): Traversable
{
return new ArrayIterator($this->all());
}

public function count(): int
{
return count($this->byId);
}

public function offsetExists(mixed $offset): bool
{
return is_string($offset) && isset($this->byId[$offset]);
}

public function offsetGet(mixed $offset): CollectionFacts
{
if (!is_string($offset) || !isset($this->byId[$offset])) {
throw new OutOfBoundsException(
'Collection fact list has no entry for this id.'
);
}

return $this->byId[$offset];
}

public function offsetSet(mixed $offset, mixed $value): void
{
throw new BadMethodCallException('Collection fact lists are immutable.');
}

public function offsetUnset(mixed $offset): void
{
throw new BadMethodCallException('Collection fact lists are immutable.');
}
}
Loading
Loading