From 545c37a54bda8d5d9aadd7d0a9c2d3eee5c42725 Mon Sep 17 00:00:00 2001 From: "Ronald A. Richardson" Date: Thu, 17 Sep 2026 18:49:59 +0800 Subject: [PATCH 1/3] Keep telematics sync status current and accept manual syncs during sweeps - A completed scheduled sweep updates the connection's status and last sync fields, replacing a stale failure. It never overrides a manual request still in flight, and a partial sweep leaves the last reported state unchanged. The update bypasses model events so minute sweeps do not add activity log entries. - "Sync Devices" no longer fails with "already queued or running" while a scheduled sweep holds the connection's poll lease. The request is recorded and the next sweep adopts and completes it. - Scheduled sweeps also adopt manual requests abandoned by a lost or interrupted job, so a request cannot stay queued, running or retrying indefinitely. - docs/TELEMATICS_QUEUES.md: telematics workers must match the queue worker's image and environment including APP_KEY, queue names must match exactly, and a troubleshooting table covers the failures seen in production. --- docs/SAFEE.md | 5 + docs/TELEMATICS_QUEUES.md | 67 ++++++++++++-- server/src/Jobs/PollTelematicTelemetry.php | 36 ++++++++ server/src/Jobs/SyncTelematicDevicesJob.php | 4 +- .../Support/Telematics/TelematicService.php | 10 +- .../Support/Telematics/Telemetry/Inbox.php | 36 ++++++-- .../Http/ManualTelemetryPollingTest.php | 92 ++++++++++++++++++- 7 files changed, 223 insertions(+), 27 deletions(-) diff --git a/docs/SAFEE.md b/docs/SAFEE.md index 2fc56ca61..1588e856f 100644 --- a/docs/SAFEE.md +++ b/docs/SAFEE.md @@ -86,6 +86,11 @@ cached configuration as part of the application's normal deployment process. Do not retry an old failed monolithic job to recover a connection; start a new manual sync or let scheduled polling recover it. +A completed scheduled sweep updates the connection's status and last sync time, so +a recovered connection no longer shows an old failure. Clicking "Sync Devices" while +a sweep is queued or running records the request for the next sweep instead of +returning an error. A partial sweep leaves the last reported state unchanged. + Inspect the shared telemetry diagnostics for fetched/applied/failed counts, incomplete sweeps, pending deliveries, queue delay, and source age. Manual sync completion is tied to its ingestion run; queueing or fetching alone is not diff --git a/docs/TELEMATICS_QUEUES.md b/docs/TELEMATICS_QUEUES.md index e5db9213a..0f9829c1f 100644 --- a/docs/TELEMATICS_QUEUES.md +++ b/docs/TELEMATICS_QUEUES.md @@ -51,28 +51,56 @@ no worker wait indefinitely. ### 1. Add telematics workers -Add worker services with the same image, environment, and volumes as the existing -`queue` service. Change only the command and healthcheck. Docker Compose example: +Each telematics worker must be an exact copy of the existing `queue` worker: the +same image and tag, the same `env_file`, `environment`, volumes and `depends_on`, +including every Compose override file that changes `queue`. Change only the command. +Workers running a different build, or missing `APP_KEY`, fail every telematics job. + +Docker Compose example, assuming `queue` is defined in `docker-compose.yml` and any +image or environment overrides for it live in `docker-compose.override.yml`: ```yaml +# docker-compose.yml: copy the full `queue` service definition, then change the command. services: telematics-queue: - image: fleetbase/fleetbase-api:latest + # image, env_file, environment, volumes, depends_on: identical to `queue` command: ["php", "artisan", "queue:work", "--queue=telematics", "--sleep=1", "--max-time=3600"] restart: unless-stopped deploy: replicas: 2 - # environment, volumes and depends_on: copy from the existing `queue` service telematics-broadcast-queue: - image: fleetbase/fleetbase-api:latest + # image, env_file, environment, volumes, depends_on: identical to `queue` command: ["php", "artisan", "queue:work", "--queue=telematics-broadcasts", "--sleep=1", "--max-time=3600"] restart: unless-stopped - # environment, volumes and depends_on: copy from the existing `queue` service +``` + +```yaml +# docker-compose.override.yml: repeat every override that applies to `queue`. +services: + telematics-queue: + image: "fleetbase-api:latest" # the same image as `queue` + telematics-broadcast-queue: + image: "fleetbase-api:latest" ``` Notes: +- **Use the same image as `queue`, not a separate `fleetbase/fleetbase-api:latest`.** + Docker does not download a newer `latest` when a copy already exists locally, so a + new service can start from an old cached image. If you do use a registry image, + pin a version tag and pull it for every API container together. +- **Every API container needs the same `APP_KEY`**: application, scheduler, the + default worker and the telematics workers. Provider credentials, token caches and + telematics deliveries are encrypted with it. If only the application service + receives it (for example through a mounted `.env` or an override), put it in a + shared `env_file` instead. Never generate a new key for a worker. +- **Check the merged configuration, including when using `extends`.** Run + `docker compose config` and compare the resolved `image`, `env_file` and + `environment` of `queue` and each telematics service before deploying. +- The `--queue` names must match the variables in step 2 exactly. A worker listening + on `telematics-broadcast` while broadcasts go to `telematics-broadcasts` receives + nothing. - Leave the existing `queue` service unchanged. `queue:work` without `--queue` processes the connection's default queue (`REDIS_QUEUE`, normally `default`). - Leave `--timeout` at its default (60 seconds) or set any value below the Redis @@ -83,8 +111,6 @@ Notes: memory. `restart: unless-stopped` starts them again. - On Kubernetes, ECS, or another platform, create equivalent deployments with the same image, environment, and commands. -- Every worker container needs the same `APP_KEY`, database, Redis, and broadcasting - settings as the application. Deliveries are encrypted with `APP_KEY`. ### 2. Set the queue variables for this instance only @@ -154,6 +180,19 @@ of 50 requests per second per account across all workers. ## Verification +First confirm every API container runs the same build and application key. Each line +must show the same key fingerprint (a hash, never the key) and Fleet-Ops version: + +```bash +for c in $(docker ps --format '{{.Names}}' | grep -E 'application|scheduler|queue'); do echo "== $c $(docker inspect --format '{{.Config.Image}}' $c)"; docker exec $c php artisan tinker --execute="echo config('app.key') ? substr(hash('sha256', config('app.key')), 0, 12) : 'MISSING', ' ', json_decode(file_get_contents('vendor/fleetbase/fleetops-api/composer.json'))->version, PHP_EOL;"; done +``` + +Confirm Laravel loaded the queue names in the application and a telematics worker: + +```bash +php artisan config:show telematics.telemetry +``` + Queue depth should stay close to zero between minute ticks: ```bash @@ -168,6 +207,18 @@ Also verify: on the telematics workers, and broadcast jobs only on the broadcast worker. - Other jobs on `default`, such as notifications and order events, start promptly again. +- The connection's status and "Last sync" update after each completed scheduled + sweep. Clicking "Sync Devices" while a sweep is queued or running records the + request, and the next sweep completes it. + +## Troubleshooting + +| Symptom | Cause | +| --- | --- | +| Telematics worker logs show `MissingAppKeyException` | The worker has no `APP_KEY`. Give every API container the same key and recreate the workers. | +| A worker reports a different Fleet-Ops version | The worker uses a different or stale cached image. Use the `queue` service's image. | +| Broadcast jobs are never processed | The broadcast worker's `--queue` name does not match `TELEMATICS_BROADCAST_QUEUE`. | +| Scheduled sync commands report `Queued 0 telematics sync job(s)` for an enabled connection | A poll job failed on a worker that could not load it, leaving its one-hour uniqueness lock. It expires on its own; after fixing the workers you can release it with `Cache::lock('laravel_unique_job:Fleetbase\\FleetOps\\Jobs\\PollTelematicTelemetry' . $telematicUuid)->forceRelease()`. | ## Rollback diff --git a/server/src/Jobs/PollTelematicTelemetry.php b/server/src/Jobs/PollTelematicTelemetry.php index 633403d78..260776cea 100644 --- a/server/src/Jobs/PollTelematicTelemetry.php +++ b/server/src/Jobs/PollTelematicTelemetry.php @@ -109,6 +109,7 @@ public function handle(TelematicProviderRegistry $registry, Inbox $inbox): void try { $started = microtime(true); DB::table('telematic_sync_runs')->insert(['uuid' => $run, 'telematic_uuid' => $telematic->uuid, 'status' => 'fetching', 'created_at' => now(), 'updated_at' => now()]); + $this->adoptPendingRequest($run); $this->manualProgress([ 'last_sync_run_uuid' => $run, 'last_sync_run_job_id' => $this->manualJobId, 'last_sync_result' => 'running', 'last_sync_phase' => 'fetching_inventory', @@ -186,6 +187,41 @@ private function remainingBudget(float $started): int return $remaining; } + /** + * Bind a manual request that has no active run to this scheduled sweep. + * + * Every poll job for a connection shares one unique lease, so while a scheduled sweep + * runs, the request's own job is either not queued (the request was recorded while + * the lease was held) or no longer exists (lost before or after an interrupted run). + */ + private function adoptPendingRequest(string $run): void + { + if ($this->manualJobId) { + return; + } + DB::transaction(function () use ($run) { + $connection = Telematic::withoutGlobalScopes()->where('uuid', $this->telematicUuid)->lockForUpdate()->first(); + $meta = $connection?->meta ?? []; + $request = data_get($meta, 'last_sync_job_id'); + if (!$request || !in_array(data_get($meta, 'last_sync_result'), Inbox::PENDING_REQUEST_RESULTS, true)) { + return; + } + $bound = data_get($meta, 'last_sync_run_uuid'); + if ($bound && DB::table('telematic_sync_runs')->where('uuid', $bound)->whereIn('status', ['fetching', 'ingesting'])->exists()) { + return; + } + $connection->status = 'synchronizing'; + $connection->meta = array_merge($meta, [ + 'last_sync_run_uuid' => $run, 'last_sync_run_job_id' => $request, + 'last_sync_result' => 'running', 'last_sync_phase' => 'fetching_inventory', + 'last_sync_fetched_total' => 0, 'last_sync_page_count' => 0, + 'last_sync_linked_total' => 0, 'last_sync_failed_total' => 0, 'last_sync_error' => null, + 'last_sync_progress_at' => now()->toDateTimeString(), + ]); + $connection->save(); + }); + } + private function manualProgress(array $attributes): void { if (!$this->manualJobId) { diff --git a/server/src/Jobs/SyncTelematicDevicesJob.php b/server/src/Jobs/SyncTelematicDevicesJob.php index aa4f78ba3..d32093fb3 100644 --- a/server/src/Jobs/SyncTelematicDevicesJob.php +++ b/server/src/Jobs/SyncTelematicDevicesJob.php @@ -58,8 +58,8 @@ public function handle(TelematicProviderRegistry $registry, TelematicService $se try { $service->queueTelemetrySync($this->telematic, $this->options, $this->jobId); } catch (\Illuminate\Validation\ValidationException) { - // Paused or already queued/running. Failing here would mark the connection - // as errored and detach the active request's progress tracking. + // Polling was paused after this job was queued. Failing here would mark the + // connection as errored and detach any active request's progress tracking. Log::info('Skipped legacy telematics sync redirect.', ['telematic_uuid' => $this->telematic->uuid]); } } diff --git a/server/src/Support/Telematics/TelematicService.php b/server/src/Support/Telematics/TelematicService.php index 853bdbbfe..42015bf79 100644 --- a/server/src/Support/Telematics/TelematicService.php +++ b/server/src/Support/Telematics/TelematicService.php @@ -188,18 +188,20 @@ public function queueTelemetrySync(Telematic $telematic, array $options = [], ?s } $job = (new \Fleetbase\FleetOps\Jobs\PollTelematicTelemetry($connection->uuid, $jobId, $options)) ->onQueue($settings['poll_queue'] ?? 'default'); - if (!Telemetry\Queue::dispatch($job)) { + $queued = Telemetry\Queue::dispatch($job); + if (!$queued) { $existing = data_get($connection->meta, 'last_sync_job_id'); - if ($existing && in_array(data_get($connection->meta, 'last_sync_result'), ['queued', 'running', 'retrying'], true)) { + if ($existing && in_array(data_get($connection->meta, 'last_sync_result'), Telemetry\Inbox::PENDING_REQUEST_RESULTS, true)) { return $existing; } - throw ValidationException::withMessages(['telematic' => ['Telemetry synchronization is already queued or running.']]); + // A scheduled sweep holds this connection's poll lease. Record the request so + // the next sweep adopts and completes it, rather than rejecting the user. } $connection->status = 'synchronizing'; $connection->meta = array_merge($connection->meta ?? [], [ 'last_sync_job_id' => $jobId, 'last_sync_run_uuid' => null, 'last_sync_run_job_id' => null, 'last_sync_started_at' => now()->toDateTimeString(), 'last_sync_result' => 'queued', - 'last_sync_phase' => 'queued', 'last_sync_error' => null, 'last_sync_failed_reason' => null, + 'last_sync_phase' => $queued ? 'queued' : 'waiting_for_sweep', 'last_sync_error' => null, 'last_sync_failed_reason' => null, ]); $connection->save(); diff --git a/server/src/Support/Telematics/Telemetry/Inbox.php b/server/src/Support/Telematics/Telemetry/Inbox.php index 21749a1e5..b1e998409 100644 --- a/server/src/Support/Telematics/Telemetry/Inbox.php +++ b/server/src/Support/Telematics/Telemetry/Inbox.php @@ -10,6 +10,9 @@ class Inbox { + /** Manual request states that a sweep may still finalize. */ + public const PENDING_REQUEST_RESULTS = ['queued', 'running', 'retrying']; + public function accept(Telematic $telematic, array $payload, string $source, ?string $run = null, ?string $receivedAt = null): string { $json = json_encode($payload, JSON_THROW_ON_ERROR); @@ -64,17 +67,12 @@ public static function finishRun(?string $id): void if ($status === 'incomplete') { return; } - // A previous sweep may finish after another manual request was queued. - // Only the run explicitly associated with that request may finalize it. $connection = Telematic::withoutGlobalScopes()->where('uuid', $run->telematic_uuid)->lockForUpdate()->first(); - if (!$connection || !self::enabled($connection) - || data_get($connection->meta, 'last_sync_run_uuid') !== $id - || !data_get($connection->meta, 'last_sync_run_job_id') - || data_get($connection->meta, 'last_sync_run_job_id') !== data_get($connection->meta, 'last_sync_job_id')) { + if (!$connection || !self::enabled($connection)) { return; } - $connection->status = $status === 'completed' ? 'active' : 'error'; - $connection->meta = array_merge($connection->meta ?? [], [ + $meta = $connection->meta ?? []; + $result = [ 'last_sync_result' => $status === 'completed' ? 'success' : $status, 'last_sync_phase' => $status, 'last_sync_progress_at' => now()->toDateTimeString(), 'last_sync_fetched_total' => (int) $run->units, 'last_sync_page_count' => (int) $run->pages, @@ -82,8 +80,28 @@ public static function finishRun(?string $id): void 'last_sync_failed_total' => (int) $failed, 'last_sync_error' => $status === 'completed' ? null : ($run->error ?? 'Some units could not be applied; inspect telemetry diagnostics.'), $status === 'completed' ? 'last_sync_completed_at' : 'last_sync_failed_at' => now()->toDateTimeString(), + ]; + // A previous sweep may finish after another manual request was queued. + // Only the run explicitly associated with that request may finalize it. + if (data_get($meta, 'last_sync_run_uuid') === $id && data_get($meta, 'last_sync_run_job_id') + && data_get($meta, 'last_sync_run_job_id') === data_get($meta, 'last_sync_job_id')) { + $connection->status = $status === 'completed' ? 'active' : 'error'; + $connection->meta = array_merge($meta, $result); + $connection->save(); + + return; + } + // A completed scheduled sweep proves the connection is healthy, so it replaces a + // stale failure. It never finalizes or overrides a manual request still in flight, + // and a partial sweep leaves the last reported state alone. + if ($status !== 'completed' || in_array(data_get($meta, 'last_sync_result'), self::PENDING_REQUEST_RESULTS, true)) { + return; + } + // Written without model events: sweeps run every minute and the model logs every save. + DB::table('telematics')->where('uuid', $connection->uuid)->update([ + 'status' => in_array($connection->status, ['error', 'synchronizing'], true) ? 'active' : $connection->status, + 'meta' => json_encode(array_merge($meta, $result), JSON_THROW_ON_ERROR), ]); - $connection->save(); }); } } diff --git a/server/tests/Feature/Http/ManualTelemetryPollingTest.php b/server/tests/Feature/Http/ManualTelemetryPollingTest.php index 72901a1f3..016dad09b 100644 --- a/server/tests/Feature/Http/ManualTelemetryPollingTest.php +++ b/server/tests/Feature/Http/ManualTelemetryPollingTest.php @@ -104,10 +104,11 @@ class ManualTelemetryProvider extends ExampleTelemetryProvider public array $requests = []; public bool $fail = false; public ?int $failOnCall = null; + public bool $polling = true; public function telemetryOptions(): array { - return array_merge(parent::telemetryOptions(), ['manual_batch_sync' => true, 'batch_size' => 1, 'webhooks_enabled' => false]); + return array_merge(parent::telemetryOptions(), ['manual_batch_sync' => true, 'batch_size' => 1, 'webhooks_enabled' => false, 'polling_enabled' => $this->polling]); } public function fetchDevices(array $options = []): array @@ -213,13 +214,37 @@ function manualTelemetryProcessPending(TelematicService $service): void } }); -test('old queued discovery jobs finish quietly when a scheduled poll already holds the request', function () { +test('manual requests made while a scheduled sweep holds the poll lease are completed by the next sweep', function () { [$connection, $provider, $registry, $service] = manualTelemetrySetup(); - expect(Fleetbase\FleetOps\Support\Telematics\Telemetry\Queue::dispatch(new PollTelematicTelemetry($connection->uuid)))->toBeTrue(); + $scheduled = new PollTelematicTelemetry($connection->uuid); + expect(Fleetbase\FleetOps\Support\Telematics\Telemetry\Queue::dispatch($scheduled))->toBeTrue(); + $id = $service->discoverDevices($connection); + // Old queued discovery jobs join the same pending request instead of failing. + (new SyncTelematicDevicesJob($connection->fresh(), [], 'old-job'))->handle($registry, $service); + $waiting = $connection->fresh(); + expect(count($GLOBALS['manual_telemetry_jobs']))->toBe(1) + ->and($waiting->status)->toBe('synchronizing') + ->and(data_get($waiting->meta, 'last_sync_job_id'))->toBe($id) + ->and(data_get($waiting->meta, 'last_sync_phase'))->toBe('waiting_for_sweep'); + ExampleTelemetryProvider::$pages = [['devices' => [manualTelemetrySample()], 'has_more' => false, 'next_cursor' => null]]; + $scheduled->handle($registry, new Inbox()); + expect(data_get($connection->fresh()->meta, 'last_sync_result'))->toBe('running'); + manualTelemetryProcessPending($service); + $fresh = $connection->fresh(); + expect($fresh->status)->toBe('active') + ->and(data_get($fresh->meta, 'last_sync_job_id'))->toBe($id) + ->and(data_get($fresh->meta, 'last_sync_result'))->toBe('success') + ->and(data_get($fresh->meta, 'last_sync_completed_at'))->toBe('2026-09-15 12:00:00') + ->and(data_get($fresh->meta, 'last_sync_total'))->toBe(1); +}); + +test('old queued discovery jobs finish quietly when polling was paused after they were queued', function () { + [$connection, $provider, $registry, $service] = manualTelemetrySetup(); + $provider->polling = false; (new SyncTelematicDevicesJob($connection, [], 'old-job'))->handle($registry, $service); $fresh = $connection->fresh(); // A thrown ValidationException would fail the tries=1 job and mark this connection as errored. - expect(count($GLOBALS['manual_telemetry_jobs']))->toBe(1) + expect(count($GLOBALS['manual_telemetry_jobs']))->toBe(0) ->and($fresh->status)->toBe('active') ->and(data_get($fresh->meta, 'last_sync_job_id'))->toBeNull(); }); @@ -459,3 +484,62 @@ function manualPollingQueuePayload(PollTelematicTelemetry $job): ManualPollingQu expect(count($GLOBALS['manual_telemetry_jobs']))->toBe(0) ->and(DB::table('telematics')->value('status'))->toBe('disabled'); }); + + +function manualTelemetryConnectionState(Telematic $connection, string $status, array $meta): void +{ + DB::table('telematics')->where('uuid', $connection->uuid)->update(['status' => $status, 'meta' => json_encode($meta)]); +} + +test('completed scheduled sweeps replace a stale sync failure without saving through the model', function () { + [$connection, $provider, $registry, $service] = manualTelemetrySetup(); + manualTelemetryConnectionState($connection, 'error', [ + 'last_sync_job_id' => 'lost-request', 'last_sync_result' => 'failed', 'last_sync_error' => 'Polling retries exhausted. MaxAttemptsExceededException', + 'last_sync_completed_at' => '2026-06-24 07:32:18', 'preserved' => 'yes', + ]); + ExampleTelemetryProvider::$pages = [['devices' => [manualTelemetrySample()], 'has_more' => false, 'next_cursor' => null]]; + (new PollTelematicTelemetry($connection->uuid))->handle($registry, new Inbox()); + manualTelemetryProcessPending($service); + $fresh = $connection->fresh(); + expect($fresh->status)->toBe('active') + ->and(data_get($fresh->meta, 'last_sync_result'))->toBe('success') + ->and(data_get($fresh->meta, 'last_sync_error'))->toBeNull() + ->and(data_get($fresh->meta, 'last_sync_completed_at'))->toBe('2026-09-15 12:00:00') + ->and(data_get($fresh->meta, 'last_sync_fetched_total'))->toBe(1) + ->and(data_get($fresh->meta, 'last_sync_job_id'))->toBe('lost-request') + ->and(data_get($fresh->meta, 'preserved'))->toBe('yes') + ->and($fresh->updated_at)->toBeNull(); +}); + +test('partial scheduled sweeps leave the last reported sync state unchanged', function () { + [$connection, $provider, $registry, $service] = manualTelemetrySetup(); + manualTelemetryConnectionState($connection, 'error', ['last_sync_result' => 'failed', 'last_sync_error' => 'Earlier failure']); + ExampleTelemetryProvider::$pages = [['devices' => [manualTelemetrySample()], 'has_more' => false, 'next_cursor' => null]]; + (new PollTelematicTelemetry($connection->uuid))->handle($registry, new Inbox()); + $run = DB::table('telematic_sync_runs')->value('uuid'); + DB::table('telematic_deliveries')->update(['status' => 'quarantined', 'failed' => 1]); + Inbox::finishRun($run); + $fresh = $connection->fresh(); + expect(DB::table('telematic_sync_runs')->value('status'))->toBe('partial') + ->and($fresh->status)->toBe('error') + ->and(data_get($fresh->meta, 'last_sync_error'))->toBe('Earlier failure'); +}); + +test('scheduled sweeps adopt abandoned requests but never one bound to a sweep in progress', function (string $boundStatus, bool $adopted) { + [$connection, $provider, $registry, $service] = manualTelemetrySetup(); + DB::table('telematic_sync_runs')->insert(['uuid' => 'bound-run', 'telematic_uuid' => $connection->uuid, 'status' => $boundStatus, 'created_at' => now(), 'updated_at' => now()]); + manualTelemetryConnectionState($connection, 'synchronizing', [ + 'last_sync_job_id' => 'abandoned-request', 'last_sync_result' => 'retrying', + 'last_sync_run_uuid' => 'bound-run', 'last_sync_run_job_id' => 'abandoned-request', + ]); + ExampleTelemetryProvider::$pages = [['devices' => [manualTelemetrySample()], 'has_more' => false, 'next_cursor' => null]]; + (new PollTelematicTelemetry($connection->uuid))->handle($registry, new Inbox()); + manualTelemetryProcessPending($service); + $fresh = $connection->fresh(); + expect($fresh->status)->toBe($adopted ? 'active' : 'synchronizing') + ->and(data_get($fresh->meta, 'last_sync_result'))->toBe($adopted ? 'success' : 'retrying') + ->and(data_get($fresh->meta, 'last_sync_run_uuid'))->toBe($adopted ? DB::table('telematic_sync_runs')->where('uuid', '!=', 'bound-run')->value('uuid') : 'bound-run'); +})->with([ + 'interrupted run' => ['incomplete', true], + 'run still ingesting' => ['ingesting', false], +]); From 112604df96d9c7deeaea14cb6ca5a51d4c0e2e70 Mon Sep 17 00:00:00 2001 From: "Ronald A. Richardson" Date: Thu, 17 Sep 2026 18:56:40 +0800 Subject: [PATCH 2/3] Add SASCO B2B as a native fuel provider - Add SascoFuelProvider: username/password login against the B2B admin endpoint, JWT cached until one minute before expiry, a single re-login when a request returns 401, and connection testing via the vehicles list. - Import driver transactions with Current_page pagination bounded by totalItems and the existing max_pages guard; normalize plate letters and digits into "NXD 1240", amounts into SAR minor units, and keep fuel type, driver and receipt image details in the normalized payload. - Register the sasco provider config with QC and production base URLs. - Make the connection form environment labels provider-specific and drop PetroApp-only wording from the sync run details; show SASCO spend in SAR and surface driver name and phone on transaction details. --- addon/components/fuel-integration/form.hbs | 4 +- addon/components/fuel-integration/form.js | 8 + addon/components/fuel-integration/hub.js | 2 +- .../fuel-providers/details/index.hbs | 2 +- addon/utils/fuel-transaction.js | 2 + server/config/fuel-providers.php | 53 +++ .../Providers/SascoFuelProvider.php | 272 ++++++++++++++ .../Unit/Support/SascoFuelProviderTest.php | 338 ++++++++++++++++++ .../unit/fuel-integration/transaction-test.js | 6 + 9 files changed, 683 insertions(+), 4 deletions(-) create mode 100644 server/src/Support/FuelProviders/Providers/SascoFuelProvider.php create mode 100644 server/tests/Unit/Support/SascoFuelProviderTest.php diff --git a/addon/components/fuel-integration/form.hbs b/addon/components/fuel-integration/form.hbs index 35151f443..87f764a45 100644 --- a/addon/components/fuel-integration/form.hbs +++ b/addon/components/fuel-integration/form.hbs @@ -99,8 +99,8 @@
{{#each this.credentialFields as |field|}} diff --git a/addon/components/fuel-integration/form.js b/addon/components/fuel-integration/form.js index fa2575ab4..09390a903 100644 --- a/addon/components/fuel-integration/form.js +++ b/addon/components/fuel-integration/form.js @@ -69,6 +69,14 @@ export default class FuelIntegrationFormComponent extends Component { return this.args.resource?.environment ?? 'production'; } + get sandboxLabel() { + return this.selectedProvider?.metadata?.environment_labels?.sandbox ?? 'Sandbox'; + } + + get productionLabel() { + return this.selectedProvider?.metadata?.environment_labels?.production ?? 'Production'; + } + get baseUrl() { return this.credentials.base_url?.trim() || this.selectedProvider?.metadata?.base_urls?.[this.environment]; } diff --git a/addon/components/fuel-integration/hub.js b/addon/components/fuel-integration/hub.js index 14686674c..0a839a7aa 100644 --- a/addon/components/fuel-integration/hub.js +++ b/addon/components/fuel-integration/hub.js @@ -116,7 +116,7 @@ export default class FuelIntegrationHubComponent extends Component { get formatSpend() { const currencies = new Map(); for (const connection of this.connections) { - const currency = connection.last_sync_state?.summary?.currency || (connection.provider === 'petroapp' ? 'SAR' : null); + const currency = connection.last_sync_state?.summary?.currency || (['petroapp', 'sasco'].includes(connection.provider) ? 'SAR' : null); if (currency) currencies.set(currency, (currencies.get(currency) || 0) + Number(connection.last_sync_state?.summary?.amount || 0)); } return [...currencies].map(([currency, amount]) => fuelMoney(amount, currency)).join(' ยท ') || 'No spend recorded'; diff --git a/addon/templates/connectivity/fuel-providers/details/index.hbs b/addon/templates/connectivity/fuel-providers/details/index.hbs index 02c774d03..06dc1eb2b 100644 --- a/addon/templates/connectivity/fuel-providers/details/index.hbs +++ b/addon/templates/connectivity/fuel-providers/details/index.hbs @@ -30,7 +30,7 @@

Requested dates: {{this.details.latestRun.window}}

{{#unless this.details.latestRun.received}}

If you expected purchases, choose their dates on the Sync tab and confirm that they belong to this - PetroApp account.

+ provider account.

{{/unless}} {{else}}

No syncs yet

diff --git a/addon/utils/fuel-transaction.js b/addon/utils/fuel-transaction.js index 5ec67bba2..bcd905beb 100644 --- a/addon/utils/fuel-transaction.js +++ b/addon/utils/fuel-transaction.js @@ -99,6 +99,8 @@ export function purchaseGroups(transaction) { ['Account branch', value('branch_name')], ['Representative', value('delegate_name')], ['Representative phone', petroapp ? value('delegate_mobile') : null], + ['Driver', value('driver_name')], + ['Driver phone', value('driver_phone')], ]), group('Vehicle & trip references', [ ['Plate number', transaction.plate_number], diff --git a/server/config/fuel-providers.php b/server/config/fuel-providers.php index 24b4fdc36..21a786b04 100644 --- a/server/config/fuel-providers.php +++ b/server/config/fuel-providers.php @@ -55,9 +55,62 @@ 'auth_type' => 'ws_sk_header', 'supported_auth_types' => ['ws_sk_header', 'bearer_token'], 'base_urls' => \Fleetbase\FleetOps\Support\FuelProviders\Providers\PetroAppFuelProvider::BASE_URLS, + 'environment_labels' => ['sandbox' => 'Sandbox (PetroApp staging)', 'production' => 'Production'], 'pagination' => 'page', 'default_currency' => 'SAR', ], ], + [ + 'key' => 'sasco', + 'label' => 'SASCO', + 'type' => 'native', + 'category' => 'Fuel card integration', + 'icon' => 'gas-pump', + 'driver_class' => \Fleetbase\FleetOps\Support\FuelProviders\Providers\SascoFuelProvider::class, + 'description' => 'SASCO B2B fleet fuel transactions and vehicles.', + 'required_fields' => [ + [ + 'name' => 'username', + 'label' => 'B2B admin username', + 'type' => 'text', + 'help_text' => 'The SASCO B2B company admin login, usually an email address.', + 'required' => true, + ], + [ + 'name' => 'password', + 'label' => 'B2B admin password', + 'type' => 'password', + 'help_text' => 'Used to request a short-lived access token from SASCO. Fleetbase re-authenticates automatically when it expires.', + 'required' => true, + ], + [ + 'name' => 'base_url', + 'label' => 'Base URL override (optional)', + 'type' => 'url', + 'help_text' => 'Leave blank to use the selected environment. A custom URL overrides the environment selection.', + 'required' => false, + ], + ], + 'capabilities' => ['vehicles', 'transactions'], + 'sync_defaults' => [ + 'window_days' => 7, + 'matching_order' => ['plate_number', 'internal_id', 'vin', 'serial_number', 'call_sign', 'fuel_card_number', 'trip_number'], + 'auto_create_fuel_reports' => true, + ], + 'setup_instructions' => [ + 'Request B2B API access from SASCO and obtain the company admin username and password for QC/UAT and Production.', + 'Select Sandbox for the SASCO QC environment, enter the credentials, and run Test Connection.', + 'SASCO transactions carry the vehicle plate only, so make sure Fleetbase vehicle plate numbers use the letters-then-digits format (for example NXD 1240).', + 'Sync a small date window first to verify imported fuel transactions and vehicle matches.', + ], + 'metadata' => [ + 'auth_type' => 'username_password', + 'supported_auth_types' => ['username_password'], + 'base_urls' => \Fleetbase\FleetOps\Support\FuelProviders\Providers\SascoFuelProvider::BASE_URLS, + 'environment_labels' => ['sandbox' => 'Sandbox (SASCO QC)', 'production' => 'Production'], + 'pagination' => 'current_page', + 'default_currency' => 'SAR', + ], + ], ], ]; diff --git a/server/src/Support/FuelProviders/Providers/SascoFuelProvider.php b/server/src/Support/FuelProviders/Providers/SascoFuelProvider.php new file mode 100644 index 000000000..232bbb479 --- /dev/null +++ b/server/src/Support/FuelProviders/Providers/SascoFuelProvider.php @@ -0,0 +1,272 @@ + 'https://loyalty-api-qc.sasco.sa', + 'production' => 'https://loyalty-api.sasco.com.sa', + ]; + + public const LOGIN_ENDPOINT = 'apigateway/api/identity/user/b2badmin-login'; + public const VEHICLES_ENDPOINT = 'apigateway/api/loyaltycore/b2bexternaladmin/b2bvehicles'; + public const TRANSACTIONS_ENDPOINT = 'apigateway/api/analytics/v1/b2bexternal/drivers-transactions'; + + /** Fallback token lifetime when the JWT carries no readable exp claim. */ + protected const DEFAULT_TOKEN_TTL = 3600; + + public function key(): string + { + return 'sasco'; + } + + public function name(): string + { + return 'SASCO'; + } + + public function authenticate(FuelProviderConnection $connection): array + { + $credentials = (array) $connection->credentials; + $response = Http::baseUrl($this->baseUrl($connection)) + ->timeout((int) data_get($credentials, 'timeout', 30)) + ->acceptJson() + ->asJson() + ->post(self::LOGIN_ENDPOINT, [ + 'username' => trim((string) data_get($credentials, 'username')), + 'password' => (string) data_get($credentials, 'password'), + ]); + + $token = $response->json('accessToken'); + if (!$response->successful() || !is_string($token) || $token === '') { + return [ + 'success' => false, + 'message' => $this->errorMessage($response, 'SASCO login failed. Check the B2B admin username and password.'), + 'status' => $response->status(), + ]; + } + + Cache::put($this->tokenCacheKey($connection), $token, $this->tokenTtl($token)); + + return ['success' => true, 'message' => 'SASCO login successful.', 'token' => $token]; + } + + public function testConnection(FuelProviderConnection $connection): array + { + $metadata = [ + 'environment' => $connection->environment ?? 'production', + 'host' => parse_url($this->baseUrl($connection), PHP_URL_HOST), + ]; + + try { + $response = $this->get($connection, self::VEHICLES_ENDPOINT, ['Page_size' => 1, 'Current_page' => 1]); + $metadata['status'] = $response->status(); + + if (!$response->successful() || !is_array($response->json('vehicles'))) { + return [ + 'success' => false, + 'message' => $this->errorMessage($response, 'Unable to connect to SASCO.'), + 'metadata' => $metadata, + ]; + } + + return [ + 'success' => true, + 'message' => 'SASCO connection successful.', + 'metadata' => array_merge($metadata, ['total_vehicles' => $response->json('totalItems')]), + ]; + } catch (\Throwable $e) { + return ['success' => false, 'message' => $e->getMessage(), 'metadata' => $metadata]; + } + } + + public function listVehicles(FuelProviderConnection $connection, array $options = []): Collection + { + return $this->paginatedGet($connection, self::VEHICLES_ENDPOINT, 'vehicles', $options); + } + + public function listTransactions(FuelProviderConnection $connection, Carbon $from, Carbon $to, array $options = []): Collection + { + $params = [ + 'FromDate' => $from->toDateString(), + 'ToDate' => $to->toDateString(), + ]; + + if (array_key_exists('is_trip', $options)) { + $params['IsTrip'] = $options['is_trip'] ? 'true' : 'false'; + } + + return $this->paginatedGet($connection, self::TRANSACTIONS_ENDPOINT, 'driverTransactions', array_merge($options, $params)) + ->map(fn ($payload) => $this->normalizeTransaction($payload)); + } + + protected function baseUrl(FuelProviderConnection $connection): string + { + $credentials = (array) $connection->credentials; + $override = trim((string) data_get($credentials, 'base_url', '')); + + return rtrim($override ?: self::BASE_URLS[$connection->environment ?? 'production'], '/'); + } + + protected function headers(FuelProviderConnection $connection): array + { + return ['Authorization' => 'Bearer ' . $this->token($connection)]; + } + + /** + * Returns a cached access token, logging in when none is cached. + */ + protected function token(FuelProviderConnection $connection): string + { + $token = Cache::get($this->tokenCacheKey($connection)); + if (is_string($token) && $token !== '') { + return $token; + } + + $result = $this->authenticate($connection); + if (!$result['success']) { + throw new \RuntimeException($result['message']); + } + + return $result['token']; + } + + /** + * Keyed by host and username rather than connection uuid so unsaved + * connections built for credential tests share the same cache entry. + */ + protected function tokenCacheKey(FuelProviderConnection $connection): string + { + $credentials = (array) $connection->credentials; + + return 'fuel-provider:sasco:token:' . hash('sha256', $this->baseUrl($connection) . '|' . trim((string) data_get($credentials, 'username'))); + } + + protected function tokenTtl(string $token): int + { + $segments = explode('.', $token); + $payload = isset($segments[1]) ? json_decode((string) base64_decode(strtr($segments[1], '-_', '+/')), true) : null; + $expires = (int) data_get($payload, 'exp', 0); + + if ($expires <= 0) { + return self::DEFAULT_TOKEN_TTL; + } + + // Refresh a minute early so a request never goes out with a token about to lapse. + return max(1, $expires - now()->timestamp - 60); + } + + /** + * Sends an authenticated GET, re-authenticating once when SASCO rejects a cached token. + */ + protected function get(FuelProviderConnection $connection, string $endpoint, array $params = []): Response + { + $response = $this->client($connection)->get($endpoint, $params); + + if ($response->status() === 401) { + Cache::forget($this->tokenCacheKey($connection)); + $response = $this->client($connection)->get($endpoint, $params); + } + + return $response; + } + + protected function paginatedGet(FuelProviderConnection $connection, string $endpoint, string $dataKey, array $params = []): Collection + { + $page = max(1, (int) data_get($params, 'page', 1)); + $pageSize = max(1, (int) data_get($params, 'page_size', 100)); + $maxPages = (int) data_get($params, 'max_pages', 100); + $query = collect($params)->except(['page', 'page_size', 'max_pages', 'is_trip'])->all(); + + $items = collect(); + + do { + $response = $this->get($connection, $endpoint, array_merge($query, ['Page_size' => $pageSize, 'Current_page' => $page])); + + if (!$response->successful()) { + throw new \RuntimeException($this->errorMessage($response, "SASCO {$endpoint} request failed.")); + } + + $data = $response->json($dataKey); + if (!is_array($data)) { + throw new \RuntimeException("SASCO {$endpoint} returned an invalid response."); + } + $items = $items->merge($data); + + $total = (int) $response->json('totalItems', 0); + $hasMore = count($data) > 0 && $items->count() < $total; + $page++; + } while ($hasMore && $page <= $maxPages); + + if ($hasMore) { + throw new \RuntimeException('SASCO returned more pages than the sync limit. Choose a shorter date range and retry; the import was not marked complete.'); + } + + return $items; + } + + protected function errorMessage(Response $response, string $fallback): string + { + $message = $response->json('message') ?? $response->json('title') ?? $response->json('error'); + + return is_string($message) && $message !== '' ? $message : $fallback; + } + + /** + * Saudi plates are printed as three letters and up to four digits, e.g. "NXD 1240". + */ + protected function plateNumber(array $plate): ?string + { + $number = $this->compactIdentifier($plate['plateNumber'] ?? null); + $letters = collect([$plate['firstPlateLetter'] ?? null, $plate['secondPlateLetter'] ?? null, $plate['thirdPlateLetter'] ?? null]) + ->map(fn ($letter) => $this->compactIdentifier($letter)) + ->filter() + ->implode(''); + + return $this->compactIdentifier(trim($letters . ' ' . $number)); + } + + protected function normalizeTransaction(array $transaction): array + { + $plate = (array) ($transaction['vehiclePlate'] ?? []); + $providerId = $this->compactIdentifier($transaction['transactionNumber'] ?? null); + $fallbackId = $this->transactionHash([ + $transaction['createdAt'] ?? null, + $plate, + $transaction['stationName'] ?? null, + $transaction['noOfLitters'] ?? null, + $transaction['amount'] ?? null, + $transaction['driverPhone'] ?? null, + ]); + + return [ + 'provider' => $this->key(), + 'provider_transaction_id' => (string) ($providerId ?: $fallbackId), + 'plate_number' => $this->plateNumber($plate), + 'trip_number' => $this->compactIdentifier($transaction['tripId'] ?? null), + 'station_name' => $this->compactIdentifier($transaction['stationName'] ?? null), + 'transaction_at' => $this->dateFrom($transaction['createdAt'] ?? null), + 'volume' => $transaction['noOfLitters'] ?? null, + 'metric_unit' => 'l', + 'amount' => $this->minorCurrencyUnit($transaction['amount'] ?? null), + 'currency' => 'SAR', + 'normalized_payload' => [ + 'fuel_type' => $transaction['fuelType'] ?? null, + 'driver_name' => $transaction['driverName'] ?? null, + 'driver_phone' => $transaction['driverPhone'] ?? null, + 'type' => $transaction['type'] ?? null, + 'period' => $transaction['period'] ?? null, + 'fuel_image_url' => $transaction['fuelImageUrl'] ?? null, + ], + 'raw_payload' => $transaction, + ]; + } +} diff --git a/server/tests/Unit/Support/SascoFuelProviderTest.php b/server/tests/Unit/Support/SascoFuelProviderTest.php new file mode 100644 index 000000000..04651bb17 --- /dev/null +++ b/server/tests/Unit/Support/SascoFuelProviderTest.php @@ -0,0 +1,338 @@ +forgetInstance(Illuminate\Http\Client\Factory::class); + Cache::clear(); +} + +function fleetopsSascoConnection(array $credentials = [], ?string $environment = null): FuelProviderConnection +{ + $connection = new FuelProviderConnection(); + $connection->setRawAttributes([ + 'uuid' => 'conn-sasco', + 'company_uuid' => 'company-1', + 'provider' => 'sasco', + 'environment' => $environment, + 'credentials' => json_encode(array_merge([ + 'base_url' => SASCO_TEST_BASE, + 'username' => 'fleet@example.test', + 'password' => 'secret', + ], $credentials)), + ], true); + + return $connection; +} + +function fleetopsSascoJwt(?int $exp): string +{ + $segment = fn (array $data) => rtrim(strtr(base64_encode(json_encode($data)), '+/', '-_'), '='); + + return $segment(['alg' => 'RS256']) . '.' . $segment($exp ? ['exp' => $exp] : ['sub' => 'user']) . '.signature'; +} + +function fleetopsSascoInvoke(SascoFuelProvider $provider, string $method, ...$arguments) +{ + $reflection = new ReflectionMethod($provider, $method); + $reflection->setAccessible(true); + + return $reflection->invoke($provider, ...$arguments); +} + +function fleetopsSascoTransaction(array $overrides = []): array +{ + return array_merge([ + 'transactionNumber' => '27307485', + 'stationName' => ' Station North ', + 'noOfLitters' => 42.735, + 'fuelType' => '95', + 'driverName' => 'Annabella Francesconi', + 'driverPhone' => '518596945', + 'createdAt' => '2025-11-03T17:19:49.595558+03:00', + 'amount' => 100.5, + 'fuelImageUrl' => 'https://storage.example.test/receipt.jpg', + 'vehiclePlate' => [ + 'plateNumber' => '1240', + 'firstPlateLetter' => 'N', + 'secondPlateLetter' => 'X', + 'thirdPlateLetter' => 'D', + ], + 'type' => 'Trip', + 'tripId' => null, + 'period' => 4, + ], $overrides); +} + +test('sasco provider identifies itself and resolves environment base urls', function () { + $provider = new SascoFuelProvider(); + + expect($provider->key())->toBe('sasco') + ->and($provider->name())->toBe('SASCO') + ->and(fleetopsSascoInvoke($provider, 'baseUrl', fleetopsSascoConnection(['base_url' => ''], 'sandbox')))->toBe('https://loyalty-api-qc.sasco.sa') + ->and(fleetopsSascoInvoke($provider, 'baseUrl', fleetopsSascoConnection(['base_url' => ''])))->toBe('https://loyalty-api.sasco.com.sa') + ->and(fleetopsSascoInvoke($provider, 'baseUrl', fleetopsSascoConnection(['base_url' => ' https://custom.test/root/ '])))->toBe('https://custom.test/root'); +}); + +test('sasco authenticate caches the access token until shortly before it expires', function () { + fleetopsSascoFresh(); + Carbon::setTestNow(Carbon::createFromTimestamp(1_000_000)); + $token = fleetopsSascoJwt(1_000_000 + 7200); + Http::fake([SASCO_TEST_BASE . '/apigateway/api/identity/user/b2badmin-login' => Http::response(['accessToken' => $token, 'loginSucces' => true], 200)]); + + $provider = new SascoFuelProvider(); + $connection = fleetopsSascoConnection(['username' => ' fleet@example.test ']); + $result = $provider->authenticate($connection); + + expect($result)->toBe(['success' => true, 'message' => 'SASCO login successful.', 'token' => $token]) + ->and(Cache::get(fleetopsSascoInvoke($provider, 'tokenCacheKey', $connection)))->toBe($token) + ->and(fleetopsSascoInvoke($provider, 'tokenTtl', $token))->toBe(7140) + ->and(fleetopsSascoInvoke($provider, 'tokenTtl', fleetopsSascoJwt(null)))->toBe(3600) + ->and(fleetopsSascoInvoke($provider, 'tokenTtl', 'not-a-jwt'))->toBe(3600) + ->and(fleetopsSascoInvoke($provider, 'tokenTtl', fleetopsSascoJwt(1_000_000 + 10)))->toBe(1); + + Http::assertSent(fn (Request $request) => $request->method() === 'POST' + && $request->data() === ['username' => 'fleet@example.test', 'password' => 'secret']); + + Carbon::setTestNow(); +}); + +test('sasco authenticate reports rejected logins with the provider message or a fallback', function () { + fleetopsSascoFresh(); + Http::fake([SASCO_TEST_BASE . '/*' => Http::sequence() + ->push(['message' => 'Invalid user credentials'], 400) + ->push(['loginSucces' => false], 200) + ->push(['message' => ['not', 'a', 'string']], 500), + ]); + + $provider = new SascoFuelProvider(); + $invalid = $provider->authenticate(fleetopsSascoConnection()); + $noToken = $provider->authenticate(fleetopsSascoConnection()); + $badBody = $provider->authenticate(fleetopsSascoConnection()); + + expect($invalid)->toBe(['success' => false, 'message' => 'Invalid user credentials', 'status' => 400]) + ->and($noToken['success'])->toBeFalse() + ->and($noToken['message'])->toBe('SASCO login failed. Check the B2B admin username and password.') + ->and($badBody['message'])->toBe('SASCO login failed. Check the B2B admin username and password.'); +}); + +test('sasco test connection reports success failure and login errors', function () { + fleetopsSascoFresh(); + Http::fake([ + SASCO_TEST_BASE . '/apigateway/api/identity/user/b2badmin-login' => Http::response(['accessToken' => fleetopsSascoJwt(null)], 200), + SASCO_TEST_BASE . '/apigateway/api/loyaltycore/b2bexternaladmin/b2bvehicles*' => Http::sequence() + ->push(['vehicles' => [['id' => 'vehicle-1']], 'totalItems' => 12], 200) + ->push(['title' => 'Forbidden'], 403) + ->push(['vehicles' => null], 200), + ]); + + $provider = new SascoFuelProvider(); + $success = $provider->testConnection(fleetopsSascoConnection([], 'sandbox')); + $failure = $provider->testConnection(fleetopsSascoConnection()); + $invalid = $provider->testConnection(fleetopsSascoConnection()); + + expect($success)->toBe([ + 'success' => true, + 'message' => 'SASCO connection successful.', + 'metadata' => ['environment' => 'sandbox', 'host' => 'sasco.test', 'status' => 200, 'total_vehicles' => 12], + ]) + ->and($failure['success'])->toBeFalse() + ->and($failure['message'])->toBe('Forbidden') + ->and($failure['metadata'])->toBe(['environment' => 'production', 'host' => 'sasco.test', 'status' => 403]) + ->and($invalid['message'])->toBe('Unable to connect to SASCO.'); + + Http::assertSentCount(4); + + fleetopsSascoFresh(); + Http::fake([SASCO_TEST_BASE . '/*' => Http::response(['error' => 'Account locked'], 401)]); + $locked = $provider->testConnection(fleetopsSascoConnection()); + + expect($locked['success'])->toBeFalse() + ->and($locked['message'])->toBe('Account locked') + ->and($locked['metadata'])->toBe(['environment' => 'production', 'host' => 'sasco.test']); +}); + +test('sasco requests reuse the cached token and log in again once after a 401', function () { + fleetopsSascoFresh(); + Http::fake([ + SASCO_TEST_BASE . '/apigateway/api/identity/user/b2badmin-login' => Http::sequence() + ->push(['accessToken' => 'token-1'], 200) + ->push(['accessToken' => 'token-2'], 200), + SASCO_TEST_BASE . '/apigateway/api/loyaltycore/b2bexternaladmin/b2bvehicles*' => Http::sequence() + ->push(['vehicles' => [['id' => 'a']], 'totalItems' => 1], 200) + ->push(['message' => 'expired'], 401) + ->push(['vehicles' => [['id' => 'b']], 'totalItems' => 1], 200), + ]); + + $provider = new SascoFuelProvider(); + $first = $provider->listVehicles(fleetopsSascoConnection()); + $second = $provider->listVehicles(fleetopsSascoConnection()); + + $authorizations = collect(Http::recorded()) + ->map(fn ($pair) => $pair[0]) + ->filter(fn (Request $request) => $request->method() === 'GET') + ->map(fn (Request $request) => $request->header('Authorization')[0]) + ->values() + ->all(); + + expect($first->pluck('id')->all())->toBe(['a']) + ->and($second->pluck('id')->all())->toBe(['b']) + ->and($authorizations)->toBe(['Bearer token-1', 'Bearer token-1', 'Bearer token-2']); +}); + +test('sasco requests raise the login error when no token can be obtained', function () { + fleetopsSascoFresh(); + Http::fake([SASCO_TEST_BASE . '/*' => Http::response(['message' => 'Invalid user credentials'], 400)]); + + expect(fn () => (new SascoFuelProvider())->listVehicles(fleetopsSascoConnection())) + ->toThrow(RuntimeException::class, 'Invalid user credentials'); +}); + +test('sasco pagination walks current pages until totalItems and guards the page limit', function () { + fleetopsSascoFresh(); + Http::fake([ + SASCO_TEST_BASE . '/apigateway/api/identity/user/b2badmin-login' => Http::response(['accessToken' => 'token'], 200), + SASCO_TEST_BASE . '/apigateway/api/loyaltycore/b2bexternaladmin/b2bvehicles*' => Http::sequence() + ->push(['vehicles' => [['id' => 1], ['id' => 2]], 'totalItems' => 3], 200) + ->push(['vehicles' => [['id' => 3]], 'totalItems' => 3], 200) + ->push(['vehicles' => [], 'totalItems' => 9], 200) + ->push(['vehicles' => [['id' => 4]], 'totalItems' => 5], 200), + ]); + + $provider = new SascoFuelProvider(); + $vehicles = $provider->listVehicles(fleetopsSascoConnection(), ['page_size' => 2, 'SearchKey' => 'NXD']); + $empty = $provider->listVehicles(fleetopsSascoConnection()); + + expect($vehicles->pluck('id')->all())->toBe([1, 2, 3]) + ->and($empty->all())->toBe([]); + + $queries = collect(Http::recorded()) + ->map(fn ($pair) => $pair[0]) + ->filter(fn (Request $request) => $request->method() === 'GET') + ->map(fn (Request $request) => parse_url($request->url(), PHP_URL_QUERY)) + ->values() + ->all(); + + expect($queries[0])->toBe('SearchKey=NXD&Page_size=2&Current_page=1') + ->and($queries[1])->toBe('SearchKey=NXD&Page_size=2&Current_page=2') + ->and($queries[2])->toBe('Page_size=100&Current_page=1'); + + expect(fn () => $provider->listVehicles(fleetopsSascoConnection(), ['page' => 3, 'max_pages' => 3])) + ->toThrow(RuntimeException::class, 'SASCO returned more pages than the sync limit.'); +}); + +test('sasco pagination raises runtime errors for failed and malformed responses', function () { + fleetopsSascoFresh(); + Http::fake([ + SASCO_TEST_BASE . '/apigateway/api/identity/user/b2badmin-login' => Http::response(['accessToken' => 'token'], 200), + SASCO_TEST_BASE . '/apigateway/api/loyaltycore/b2bexternaladmin/b2bvehicles*' => Http::sequence() + ->push('gateway timeout', 504) + ->push(['data' => []], 200), + ]); + + $provider = new SascoFuelProvider(); + + expect(fn () => $provider->listVehicles(fleetopsSascoConnection())) + ->toThrow(RuntimeException::class, 'SASCO apigateway/api/loyaltycore/b2bexternaladmin/b2bvehicles request failed.') + ->and(fn () => $provider->listVehicles(fleetopsSascoConnection())) + ->toThrow(RuntimeException::class, 'SASCO apigateway/api/loyaltycore/b2bexternaladmin/b2bvehicles returned an invalid response.'); +}); + +test('sasco list transactions filters by date and normalizes driver transactions', function () { + fleetopsSascoFresh(); + Http::fake([ + SASCO_TEST_BASE . '/apigateway/api/identity/user/b2badmin-login' => Http::response(['accessToken' => 'token'], 200), + SASCO_TEST_BASE . '/apigateway/api/analytics/v1/b2bexternal/drivers-transactions*' => Http::sequence() + ->push([ + 'driverTransactions' => [ + fleetopsSascoTransaction(), + fleetopsSascoTransaction(['transactionNumber' => ' ', 'tripId' => ' TRIP-7 ', 'vehiclePlate' => null, 'amount' => null]), + ], + 'totalItems' => 2, + ], 200) + ->push(['driverTransactions' => [], 'totalItems' => 0], 200) + ->push(['driverTransactions' => [], 'totalItems' => 0], 200), + ]); + + $provider = new SascoFuelProvider(); + $from = Carbon::parse('2025-10-30 08:00:00'); + $to = Carbon::parse('2025-11-03 20:00:00'); + + $transactions = $provider->listTransactions(fleetopsSascoConnection(), $from, $to); + $provider->listTransactions(fleetopsSascoConnection(), $from, $to, ['is_trip' => true]); + $provider->listTransactions(fleetopsSascoConnection(), $from, $to, ['is_trip' => false]); + + [$first, $second] = $transactions->all(); + + expect($first)->toMatchArray([ + 'provider' => 'sasco', + 'provider_transaction_id' => '27307485', + 'plate_number' => 'NXD 1240', + 'trip_number' => null, + 'station_name' => 'Station North', + 'volume' => 42.735, + 'metric_unit' => 'l', + 'amount' => 10050, + 'currency' => 'SAR', + 'normalized_payload' => [ + 'fuel_type' => '95', + 'driver_name' => 'Annabella Francesconi', + 'driver_phone' => '518596945', + 'type' => 'Trip', + 'period' => 4, + 'fuel_image_url' => 'https://storage.example.test/receipt.jpg', + ], + 'raw_payload' => fleetopsSascoTransaction(), + ]) + ->and($first['transaction_at']->toIso8601String())->toBe('2025-11-03T17:19:49+03:00') + ->and($second['provider_transaction_id'])->toHaveLength(64) + ->and($second['plate_number'])->toBeNull() + ->and($second['trip_number'])->toBe('TRIP-7') + ->and($second['amount'])->toBeNull(); + + $queries = collect(Http::recorded()) + ->map(fn ($pair) => $pair[0]) + ->filter(fn (Request $request) => $request->method() === 'GET') + ->map(fn (Request $request) => parse_url($request->url(), PHP_URL_QUERY)) + ->values() + ->all(); + + expect($queries)->toBe([ + 'FromDate=2025-10-30&ToDate=2025-11-03&Page_size=100&Current_page=1', + 'FromDate=2025-10-30&ToDate=2025-11-03&IsTrip=true&Page_size=100&Current_page=1', + 'FromDate=2025-10-30&ToDate=2025-11-03&IsTrip=false&Page_size=100&Current_page=1', + ]); +}); + +test('sasco plate numbers tolerate partial plates', function () { + $provider = new SascoFuelProvider(); + + expect(fleetopsSascoInvoke($provider, 'plateNumber', ['plateNumber' => '435']))->toBe('435') + ->and(fleetopsSascoInvoke($provider, 'plateNumber', ['firstPlateLetter' => 'R', 'secondPlateLetter' => ' ', 'thirdPlateLetter' => 'R']))->toBe('RR') + ->and(fleetopsSascoInvoke($provider, 'plateNumber', []))->toBeNull(); +}); + +test('sasco is registered as a native fuel provider', function () { + $providers = require __DIR__ . '/../../../config/fuel-providers.php'; + $sasco = collect($providers['providers'])->firstWhere('key', 'sasco'); + + expect($sasco['driver_class'])->toBe(SascoFuelProvider::class) + ->and(collect($sasco['required_fields'])->pluck('name')->all())->toBe(['username', 'password', 'base_url']) + ->and($sasco['metadata']['base_urls'])->toBe(SascoFuelProvider::BASE_URLS) + ->and($sasco['capabilities'])->toBe(['vehicles', 'transactions']); +}); diff --git a/tests/unit/fuel-integration/transaction-test.js b/tests/unit/fuel-integration/transaction-test.js index cd2c18f7c..ad5330a57 100644 --- a/tests/unit/fuel-integration/transaction-test.js +++ b/tests/unit/fuel-integration/transaction-test.js @@ -32,6 +32,12 @@ module('Unit | fuel integration | transaction presentation', function () { assert.false(JSON.stringify(groups).includes('nested')); const other = purchaseGroups({ provider: 'other', raw_payload: { fuel_type: 'Petrol', vat_percent: 15, token: 'secret' } }); assert.deepEqual(other[0].rows, [{ label: 'Fuel type', value: 'Petrol' }], 'unknown provider units are not guessed'); + const sasco = purchaseGroups({ provider: 'sasco', station_name: 'Station', normalized_payload: { driver_name: 'Missy Champerlen', driver_phone: '517444245' } }); + assert.deepEqual(sasco[0].rows, [ + { label: 'Station', value: 'Station' }, + { label: 'Driver', value: 'Missy Champerlen' }, + { label: 'Driver phone', value: '517444245' }, + ]); assert.deepEqual(purchaseGroups({}), []); }); test('all supported states have a semantic badge style', function (assert) { From 803f99db5ae74d9cbd2ff23d143e498dbdf689ea Mon Sep 17 00:00:00 2001 From: "Ronald A. Richardson" Date: Thu, 17 Sep 2026 19:37:12 +0800 Subject: [PATCH 3/3] chore: open release v0.6.68 --- RELEASE.md | 24 +++++++++--------------- composer.json | 2 +- extension.json | 2 +- package.json | 2 +- 4 files changed, 12 insertions(+), 18 deletions(-) diff --git a/RELEASE.md b/RELEASE.md index 42cc61c14..c39d64c9f 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,27 +1,21 @@ -> v0.6.67 ~ "Telematics reliability" +> v0.6.68 ~ "Telematics sync status and SASCO fuel" --- ## What's New -- **Safee / DSCO syncs use a few batched requests.** Each sweep fetches every vehicle's current position, speed, heading and odometer in batches of up to 1,000 vehicles, with the vehicle list cached between sweeps. The old sync made three extra requests per vehicle; a 93-vehicle fleet now uses one request per minute instead of about 281. Temperature, door and history lookups stay out of routine polling. -- **Safee sign-in is reused and refreshed.** Access tokens are cached encrypted and refreshed before they expire, requests stay within Safee's 50-per-second account limit, and rate-limit responses are honoured. -- **Telematics can run on dedicated queue workers.** `TELEMATICS_POLL_QUEUE`, `TELEMATICS_INGESTION_QUEUE` and the new `TELEMATICS_BROADCAST_QUEUE` move polling, position processing and live-map broadcasts off the `default` queue for instances that need it. Nothing changes when they are unset. See `docs/TELEMATICS_QUEUES.md`. -- **Live telemetry status is easier to read.** Connection details show telemetry inside the page layout with a compact grid and collapsible setup and failure sections. +- **SASCO is a native fuel provider.** Connect a SASCO B2B account alongside PetroApp to import fuel transactions, with sandbox and production environments, amounts in SAR, driver name and phone, and receipt images. Transactions are matched to vehicles by plate. +- **"Sync Devices" works while telematics polling is running.** A manual sync requested while a scheduled sweep is queued or running is recorded and completed by the next sweep, instead of failing with "already queued or running". +- **Telematics connection status stays current.** Each completed scheduled sweep updates the connection status and last sync time, so a connection that has recovered no longer shows "Needs attention" or an old last sync date. --- ## Fixes -- Telematics syncs no longer fail with "SyncTelematicDevicesJob has been attempted too many times" for providers on bounded polling. Poll attempts finish within the queue's 90-second reservation, and older queued sync jobs hand off to it instead of running for up to an hour. -- A transient provider or TLS timeout no longer pauses scheduled polling for several minutes, and connections left in an error state keep being polled. -- Vehicles that have never reported a position no longer mark every sweep as partial, quarantine deliveries, or show the connection as degraded. -- Older or delayed samples cannot move a device or attached vehicle position backwards, and partial messages keep existing device identity and metadata. -- AFAQY recovers from unreadable cached tokens and shares one request deadline across sign-in and unit retrieval. -- Pausing Safee polling no longer queues failing sync jobs every minute. -- Device event lookups use a new index instead of full-table scans. -- The Radar dashboard widget links correctly when rendered outside the Fleet-Ops engine. +- Manual telematics sync requests can no longer stay queued or failed indefinitely after their job is lost; the next scheduled sweep completes them. +- Fuel provider environment labels and sync run details no longer refer to PetroApp for other providers. --- ## Testing -- Backend tests cover Safee batching, token refresh, rate limits, ordered ingestion, tenant isolation, delivery checkpoints, paused providers, retry backoff and queue routing. -- A local run on a single default queue worker polled a 93-vehicle Safee fleet: 5 sweeps used 7 API requests and each completed with 91 positions applied and 2 vehicles without a fix. +- Backend tests cover manual syncs during scheduled sweeps, stale failure recovery, partial sweeps, and adoption of abandoned requests. +- SASCO provider tests cover login, token caching, re-login after a 401, connection tests, pagination limits, error handling and transaction normalization. +- `docs/TELEMATICS_QUEUES.md` now documents that dedicated telematics workers must share the queue worker's image and `APP_KEY`, with verification commands and troubleshooting. --- ## Need help? diff --git a/composer.json b/composer.json index fc1412ed1..f586305df 100644 --- a/composer.json +++ b/composer.json @@ -1,6 +1,6 @@ { "name": "fleetbase/fleetops-api", - "version": "0.6.67", + "version": "0.6.68", "description": "Fleet & Transport Management Extension for Fleetbase", "keywords": [ "fleetbase-extension", diff --git a/extension.json b/extension.json index 7bf2b2f40..9d1f35126 100644 --- a/extension.json +++ b/extension.json @@ -1,6 +1,6 @@ { "name": "Fleet-Ops", - "version": "0.6.67", + "version": "0.6.68", "description": "Fleet & Transport Management Extension for Fleetbase", "repository": "https://github.com/fleetbase/fleetops", "license": "AGPL-3.0-or-later", diff --git a/package.json b/package.json index cac7a7342..87b5ef586 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@fleetbase/fleetops-engine", - "version": "0.6.67", + "version": "0.6.68", "description": "Fleet & Transport Management Extension for Fleetbase", "fleetbase": { "route": "fleet-ops"