Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 9 additions & 15 deletions RELEASE.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,21 @@
> v0.6.67 ~ "Telematics reliability"
> v0.6.68 ~ "Telematics sync status and SASCO fuel"

---
## What's New
- **Safee / DSCO syncs use a few batched requests.** Each sweep fetches every vehicle's current position, speed, heading and odometer in batches of up to 1,000 vehicles, with the vehicle list cached between sweeps. The old sync made three extra requests per vehicle; a 93-vehicle fleet now uses one request per minute instead of about 281. Temperature, door and history lookups stay out of routine polling.
- **Safee sign-in is reused and refreshed.** Access tokens are cached encrypted and refreshed before they expire, requests stay within Safee's 50-per-second account limit, and rate-limit responses are honoured.
- **Telematics can run on dedicated queue workers.** `TELEMATICS_POLL_QUEUE`, `TELEMATICS_INGESTION_QUEUE` and the new `TELEMATICS_BROADCAST_QUEUE` move polling, position processing and live-map broadcasts off the `default` queue for instances that need it. Nothing changes when they are unset. See `docs/TELEMATICS_QUEUES.md`.
- **Live telemetry status is easier to read.** Connection details show telemetry inside the page layout with a compact grid and collapsible setup and failure sections.
- **SASCO is a native fuel provider.** Connect a SASCO B2B account alongside PetroApp to import fuel transactions, with sandbox and production environments, amounts in SAR, driver name and phone, and receipt images. Transactions are matched to vehicles by plate.
- **"Sync Devices" works while telematics polling is running.** A manual sync requested while a scheduled sweep is queued or running is recorded and completed by the next sweep, instead of failing with "already queued or running".
- **Telematics connection status stays current.** Each completed scheduled sweep updates the connection status and last sync time, so a connection that has recovered no longer shows "Needs attention" or an old last sync date.

---
## Fixes
- Telematics syncs no longer fail with "SyncTelematicDevicesJob has been attempted too many times" for providers on bounded polling. Poll attempts finish within the queue's 90-second reservation, and older queued sync jobs hand off to it instead of running for up to an hour.
- A transient provider or TLS timeout no longer pauses scheduled polling for several minutes, and connections left in an error state keep being polled.
- Vehicles that have never reported a position no longer mark every sweep as partial, quarantine deliveries, or show the connection as degraded.
- Older or delayed samples cannot move a device or attached vehicle position backwards, and partial messages keep existing device identity and metadata.
- AFAQY recovers from unreadable cached tokens and shares one request deadline across sign-in and unit retrieval.
- Pausing Safee polling no longer queues failing sync jobs every minute.
- Device event lookups use a new index instead of full-table scans.
- The Radar dashboard widget links correctly when rendered outside the Fleet-Ops engine.
- Manual telematics sync requests can no longer stay queued or failed indefinitely after their job is lost; the next scheduled sweep completes them.
- Fuel provider environment labels and sync run details no longer refer to PetroApp for other providers.

---
## Testing
- Backend tests cover Safee batching, token refresh, rate limits, ordered ingestion, tenant isolation, delivery checkpoints, paused providers, retry backoff and queue routing.
- A local run on a single default queue worker polled a 93-vehicle Safee fleet: 5 sweeps used 7 API requests and each completed with 91 positions applied and 2 vehicles without a fix.
- Backend tests cover manual syncs during scheduled sweeps, stale failure recovery, partial sweeps, and adoption of abandoned requests.
- SASCO provider tests cover login, token caching, re-login after a 401, connection tests, pagination limits, error handling and transaction normalization.
- `docs/TELEMATICS_QUEUES.md` now documents that dedicated telematics workers must share the queue worker's image and `APP_KEY`, with verification commands and troubleshooting.

---
## Need help?
Expand Down
4 changes: 2 additions & 2 deletions addon/components/fuel-integration/form.hbs
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,8 @@
<div class="grid grid-cols-1 gap-2 lg:grid-cols-2 no-input-group-padding text-xs">
<InputGroup @name="Environment">
<select class="form-select w-full" data-test-fuel-environment {{on "change" this.setEnvironment}}>
<option value="sandbox" selected={{eq this.environment "sandbox"}}>Sandbox (PetroApp staging)</option>
<option value="production" selected={{eq this.environment "production"}}>Production</option>
<option value="sandbox" selected={{eq this.environment "sandbox"}}>{{this.sandboxLabel}}</option>
<option value="production" selected={{eq this.environment "production"}}>{{this.productionLabel}}</option>
</select>
</InputGroup>
{{#each this.credentialFields as |field|}}
Expand Down
8 changes: 8 additions & 0 deletions addon/components/fuel-integration/form.js
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,14 @@ export default class FuelIntegrationFormComponent extends Component {
return this.args.resource?.environment ?? 'production';
}

get sandboxLabel() {
return this.selectedProvider?.metadata?.environment_labels?.sandbox ?? 'Sandbox';
}

get productionLabel() {
return this.selectedProvider?.metadata?.environment_labels?.production ?? 'Production';
}

get baseUrl() {
return this.credentials.base_url?.trim() || this.selectedProvider?.metadata?.base_urls?.[this.environment];
}
Expand Down
2 changes: 1 addition & 1 deletion addon/components/fuel-integration/hub.js
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ export default class FuelIntegrationHubComponent extends Component {
get formatSpend() {
const currencies = new Map();
for (const connection of this.connections) {
const currency = connection.last_sync_state?.summary?.currency || (connection.provider === 'petroapp' ? 'SAR' : null);
const currency = connection.last_sync_state?.summary?.currency || (['petroapp', 'sasco'].includes(connection.provider) ? 'SAR' : null);
if (currency) currencies.set(currency, (currencies.get(currency) || 0) + Number(connection.last_sync_state?.summary?.amount || 0));
}
return [...currencies].map(([currency, amount]) => fuelMoney(amount, currency)).join(' · ') || 'No spend recorded';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
<p class="mt-2 text-xs text-gray-500 dark:text-gray-400">Requested dates: {{this.details.latestRun.window}}</p>
{{#unless this.details.latestRun.received}}
<p class="mt-3 text-xs leading-5 text-gray-500 dark:text-gray-400">If you expected purchases, choose their dates on the Sync tab and confirm that they belong to this
PetroApp account.</p>
provider account.</p>
{{/unless}}
{{else}}
<p class="text-sm text-gray-900 dark:text-gray-100">No syncs yet</p>
Expand Down
2 changes: 2 additions & 0 deletions addon/utils/fuel-transaction.js
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,8 @@ export function purchaseGroups(transaction) {
['Account branch', value('branch_name')],
['Representative', value('delegate_name')],
['Representative phone', petroapp ? value('delegate_mobile') : null],
['Driver', value('driver_name')],
['Driver phone', value('driver_phone')],
]),
group('Vehicle & trip references', [
['Plate number', transaction.plate_number],
Expand Down
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "fleetbase/fleetops-api",
"version": "0.6.67",
"version": "0.6.68",
"description": "Fleet & Transport Management Extension for Fleetbase",
"keywords": [
"fleetbase-extension",
Expand Down
5 changes: 5 additions & 0 deletions docs/SAFEE.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,11 @@ cached configuration as part of the application's normal deployment process.
Do not retry an old failed monolithic job to recover a connection; start a new
manual sync or let scheduled polling recover it.

A completed scheduled sweep updates the connection's status and last sync time, so
a recovered connection no longer shows an old failure. Clicking "Sync Devices" while
a sweep is queued or running records the request for the next sweep instead of
returning an error. A partial sweep leaves the last reported state unchanged.

Inspect the shared telemetry diagnostics for fetched/applied/failed counts,
incomplete sweeps, pending deliveries, queue delay, and source age. Manual sync
completion is tied to its ingestion run; queueing or fetching alone is not
Expand Down
67 changes: 59 additions & 8 deletions docs/TELEMATICS_QUEUES.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,28 +51,56 @@ no worker wait indefinitely.

### 1. Add telematics workers

Add worker services with the same image, environment, and volumes as the existing
`queue` service. Change only the command and healthcheck. Docker Compose example:
Each telematics worker must be an exact copy of the existing `queue` worker: the
same image and tag, the same `env_file`, `environment`, volumes and `depends_on`,
including every Compose override file that changes `queue`. Change only the command.
Workers running a different build, or missing `APP_KEY`, fail every telematics job.

Docker Compose example, assuming `queue` is defined in `docker-compose.yml` and any
image or environment overrides for it live in `docker-compose.override.yml`:

```yaml
# docker-compose.yml: copy the full `queue` service definition, then change the command.
services:
telematics-queue:
image: fleetbase/fleetbase-api:latest
# image, env_file, environment, volumes, depends_on: identical to `queue`
command: ["php", "artisan", "queue:work", "--queue=telematics", "--sleep=1", "--max-time=3600"]
restart: unless-stopped
deploy:
replicas: 2
# environment, volumes and depends_on: copy from the existing `queue` service

telematics-broadcast-queue:
image: fleetbase/fleetbase-api:latest
# image, env_file, environment, volumes, depends_on: identical to `queue`
command: ["php", "artisan", "queue:work", "--queue=telematics-broadcasts", "--sleep=1", "--max-time=3600"]
restart: unless-stopped
# environment, volumes and depends_on: copy from the existing `queue` service
```

```yaml
# docker-compose.override.yml: repeat every override that applies to `queue`.
services:
telematics-queue:
image: "fleetbase-api:latest" # the same image as `queue`
telematics-broadcast-queue:
image: "fleetbase-api:latest"
```

Notes:

- **Use the same image as `queue`, not a separate `fleetbase/fleetbase-api:latest`.**
Docker does not download a newer `latest` when a copy already exists locally, so a
new service can start from an old cached image. If you do use a registry image,
pin a version tag and pull it for every API container together.
- **Every API container needs the same `APP_KEY`**: application, scheduler, the
default worker and the telematics workers. Provider credentials, token caches and
telematics deliveries are encrypted with it. If only the application service
receives it (for example through a mounted `.env` or an override), put it in a
shared `env_file` instead. Never generate a new key for a worker.
- **Check the merged configuration, including when using `extends`.** Run
`docker compose config` and compare the resolved `image`, `env_file` and
`environment` of `queue` and each telematics service before deploying.
- The `--queue` names must match the variables in step 2 exactly. A worker listening
on `telematics-broadcast` while broadcasts go to `telematics-broadcasts` receives
nothing.
- Leave the existing `queue` service unchanged. `queue:work` without `--queue`
processes the connection's default queue (`REDIS_QUEUE`, normally `default`).
- Leave `--timeout` at its default (60 seconds) or set any value below the Redis
Expand All @@ -83,8 +111,6 @@ Notes:
memory. `restart: unless-stopped` starts them again.
- On Kubernetes, ECS, or another platform, create equivalent deployments with the
same image, environment, and commands.
- Every worker container needs the same `APP_KEY`, database, Redis, and broadcasting
settings as the application. Deliveries are encrypted with `APP_KEY`.

### 2. Set the queue variables for this instance only

Expand Down Expand Up @@ -154,6 +180,19 @@ of 50 requests per second per account across all workers.

## Verification

First confirm every API container runs the same build and application key. Each line
must show the same key fingerprint (a hash, never the key) and Fleet-Ops version:

```bash
for c in $(docker ps --format '{{.Names}}' | grep -E 'application|scheduler|queue'); do echo "== $c $(docker inspect --format '{{.Config.Image}}' $c)"; docker exec $c php artisan tinker --execute="echo config('app.key') ? substr(hash('sha256', config('app.key')), 0, 12) : 'MISSING', ' ', json_decode(file_get_contents('vendor/fleetbase/fleetops-api/composer.json'))->version, PHP_EOL;"; done
```

Confirm Laravel loaded the queue names in the application and a telematics worker:

```bash
php artisan config:show telematics.telemetry
```

Queue depth should stay close to zero between minute ticks:

```bash
Expand All @@ -168,6 +207,18 @@ Also verify:
on the telematics workers, and broadcast jobs only on the broadcast worker.
- Other jobs on `default`, such as notifications and order events, start promptly
again.
- The connection's status and "Last sync" update after each completed scheduled
sweep. Clicking "Sync Devices" while a sweep is queued or running records the
request, and the next sweep completes it.

## Troubleshooting

| Symptom | Cause |
| --- | --- |
| Telematics worker logs show `MissingAppKeyException` | The worker has no `APP_KEY`. Give every API container the same key and recreate the workers. |
| A worker reports a different Fleet-Ops version | The worker uses a different or stale cached image. Use the `queue` service's image. |
| Broadcast jobs are never processed | The broadcast worker's `--queue` name does not match `TELEMATICS_BROADCAST_QUEUE`. |
| Scheduled sync commands report `Queued 0 telematics sync job(s)` for an enabled connection | A poll job failed on a worker that could not load it, leaving its one-hour uniqueness lock. It expires on its own; after fixing the workers you can release it with `Cache::lock('laravel_unique_job:Fleetbase\\FleetOps\\Jobs\\PollTelematicTelemetry' . $telematicUuid)->forceRelease()`. |

## Rollback

Expand Down
2 changes: 1 addition & 1 deletion extension.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "Fleet-Ops",
"version": "0.6.67",
"version": "0.6.68",
"description": "Fleet & Transport Management Extension for Fleetbase",
"repository": "https://github.com/fleetbase/fleetops",
"license": "AGPL-3.0-or-later",
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@fleetbase/fleetops-engine",
"version": "0.6.67",
"version": "0.6.68",
"description": "Fleet & Transport Management Extension for Fleetbase",
"fleetbase": {
"route": "fleet-ops"
Expand Down
53 changes: 53 additions & 0 deletions server/config/fuel-providers.php
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,62 @@
'auth_type' => 'ws_sk_header',
'supported_auth_types' => ['ws_sk_header', 'bearer_token'],
'base_urls' => \Fleetbase\FleetOps\Support\FuelProviders\Providers\PetroAppFuelProvider::BASE_URLS,
'environment_labels' => ['sandbox' => 'Sandbox (PetroApp staging)', 'production' => 'Production'],
'pagination' => 'page',
'default_currency' => 'SAR',
],
],
[
'key' => 'sasco',
'label' => 'SASCO',
'type' => 'native',
'category' => 'Fuel card integration',
'icon' => 'gas-pump',
'driver_class' => \Fleetbase\FleetOps\Support\FuelProviders\Providers\SascoFuelProvider::class,
'description' => 'SASCO B2B fleet fuel transactions and vehicles.',
'required_fields' => [
[
'name' => 'username',
'label' => 'B2B admin username',
'type' => 'text',
'help_text' => 'The SASCO B2B company admin login, usually an email address.',
'required' => true,
],
[
'name' => 'password',
'label' => 'B2B admin password',
'type' => 'password',
'help_text' => 'Used to request a short-lived access token from SASCO. Fleetbase re-authenticates automatically when it expires.',
'required' => true,
],
[
'name' => 'base_url',
'label' => 'Base URL override (optional)',
'type' => 'url',
'help_text' => 'Leave blank to use the selected environment. A custom URL overrides the environment selection.',
'required' => false,
],
],
'capabilities' => ['vehicles', 'transactions'],
'sync_defaults' => [
'window_days' => 7,
'matching_order' => ['plate_number', 'internal_id', 'vin', 'serial_number', 'call_sign', 'fuel_card_number', 'trip_number'],
'auto_create_fuel_reports' => true,
],
'setup_instructions' => [
'Request B2B API access from SASCO and obtain the company admin username and password for QC/UAT and Production.',
'Select Sandbox for the SASCO QC environment, enter the credentials, and run Test Connection.',
'SASCO transactions carry the vehicle plate only, so make sure Fleetbase vehicle plate numbers use the letters-then-digits format (for example NXD 1240).',
'Sync a small date window first to verify imported fuel transactions and vehicle matches.',
],
'metadata' => [
'auth_type' => 'username_password',
'supported_auth_types' => ['username_password'],
'base_urls' => \Fleetbase\FleetOps\Support\FuelProviders\Providers\SascoFuelProvider::BASE_URLS,
'environment_labels' => ['sandbox' => 'Sandbox (SASCO QC)', 'production' => 'Production'],
'pagination' => 'current_page',
'default_currency' => 'SAR',
],
],
],
];
Loading
Loading