feat!: introduce storage abstraction layer

CHANGELOG.md

@@ -8,6 +8,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- Storage abstraction: `ActivityLogStorageInterface`, `CurrentDataTrackerStorageInterface` and `StorageInitializerInterface` in the new `Locastic\Loggastic\Storage` namespace; implement and alias them to store activity logs in a custom backend
+- Elasticsearch implementations of the storage interfaces in `Locastic\Loggastic\Bridge\Elasticsearch\Storage` (used by default, no configuration changes needed)
 - Support for Symfony 8 and DoctrineBundle 3 (`symfony/*` now `^6.4 || ^7.0 || ^8.0`, `doctrine/doctrine-bundle` now `^2.8 || ^3.0`), with Symfony 8 CI legs
 - Basic auth (`elastic_user`, `elastic_password`) and `elastic_ssl_verification` config options for secured Elasticsearch clusters (originally proposed in #26 by @jakubdusek)
 - Tests matrix now also runs against Elasticsearch 9
@@ -18,6 +20,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Breaking:** migrated to the modern bundle directory layout: service configuration moved from `src/Resources/config/` to `config/`, and the bundle now extends `AbstractBundle`; the `LocasticLoggasticExtension` and `Configuration` classes were removed (see UPGRADE-2.0.md)
 - **Breaking:** upgraded to `elasticsearch/elasticsearch` `^8.0 || ^9.0`; Elasticsearch 7 servers are no longer supported and a PSR-18 HTTP client implementation is required (see UPGRADE-2.0.md)
 - **Breaking:** `ElasticsearchClientInterface::getClient()` now returns `Elastic\Elasticsearch\Client` instead of `Elasticsearch\Client`
+- **Breaking:** core services (`ActivityLogProcessor`, `ActivityLogProvider`, `CurrentDataTrackerProvider`, `PopulateCurrentDataTrackersHandler` and the console commands) now depend on the storage interfaces instead of `ElasticsearchServiceInterface`/`ElasticsearchContextFactoryInterface`/`ElasticsearchIndexFactoryInterface`; their constructor signatures changed (see UPGRADE-2.0.md)
+- **Breaking:** `ActivityLogProviderInterface::getActivityLogsByIndexAndId()` was removed; use `ElasticsearchActivityLogStorage::findByIndexAndObjectId()` for index-based reads
+- **Breaking:** `ElasticNormalizationContextTrait` moved to `Locastic\Loggastic\Serializer\Traits\NormalizationContextTrait` (it is not Elasticsearch-specific)
+
+### Removed
+- **Breaking:** `ActivityLogProvider::getCurrentDataTrackerByClassAndId()`, which declared an `?array` return type but returned an object and threw a `TypeError` on every hit; use `CurrentDataTrackerProviderInterface` instead
+- **Breaking:** the unused `IndexNotFoundException` class
 
 ### Fixed
 - `ElasticsearchService::getItemById()` always returned null because it compared the hit total against an integer while Elasticsearch returns an object

README.md

@@ -144,9 +144,9 @@ Here are the examples for displaying activity logs in twig or as Api endpoints:
     public function getActivityLogsByClass(string $className, array $sort = []): array;
 
     public function getActivityLogsByClassAndId(string $className, $objectId, array $sort = []): array;
-
-    public function getActivityLogsByIndexAndId(string $index, $objectId, array $sort = []): array;
 ```
+If you need to read logs directly from a specific Elasticsearch index, use
+`Locastic\Loggastic\Bridge\Elasticsearch\Storage\ElasticsearchActivityLogStorage::findByIndexAndObjectId()`.
 Use them to fetch data from the Elasticsearch and display it in your views. Example for displaying results in Twig:
 ```twig
 Activity logs for Blog Posts:
@@ -199,6 +199,27 @@ Each time some change happens in the database for loggable entities, the activit
 ## Customization guide
 Now that you have the basic setup, you can add some additional options and customize the library to your needs.
 
+### Custom storage backends
+Activity logs are stored in Elasticsearch by default, but the core services only talk to three storage interfaces:
+
+```php
+Locastic\Loggastic\Storage\ActivityLogStorageInterface        # writes and reads activity logs
+Locastic\Loggastic\Storage\CurrentDataTrackerStorageInterface # tracks the latest state of each loggable object
+Locastic\Loggastic\Storage\StorageInitializerInterface        # creates the underlying storage (indexes, tables, ...)
+```
+
+To store logs somewhere else, implement the three interfaces and alias them to your services:
+
+```yaml
+# config/services.yaml
+services:
+    Locastic\Loggastic\Storage\ActivityLogStorageInterface: '@App\Loggastic\MyActivityLogStorage'
+    Locastic\Loggastic\Storage\CurrentDataTrackerStorageInterface: '@App\Loggastic\MyCurrentDataTrackerStorage'
+    Locastic\Loggastic\Storage\StorageInitializerInterface: '@App\Loggastic\MyStorageInitializer'
+```
+
+The loggers, message handlers, data providers and console commands will use your implementations without any further changes.
+
 ## Configuration reference
 Default configuration:
 ```yaml

UPGRADE-2.0.md

@@ -53,3 +53,35 @@ the deprecation shipped in a 1.x release so you can migrate before upgrading.
   (handled by `AttributeLoggableContextCollectionFactory`) or with XML/YAML
   extractors. The `doctrine/annotations` package is no longer a dependency
   since 1.2.
+
+## Storage abstraction
+
+Core services no longer talk to Elasticsearch directly. Three new interfaces
+in `Locastic\Loggastic\Storage` sit between the core and the backend:
+`ActivityLogStorageInterface`, `CurrentDataTrackerStorageInterface` and
+`StorageInitializerInterface`. The default Elasticsearch implementations live
+in `Locastic\Loggastic\Bridge\Elasticsearch\Storage` and are aliased to the
+interfaces, so nothing changes unless you extended the affected classes:
+
+- The constructor signatures of `ActivityLogProcessor`, `ActivityLogProvider`,
+  `CurrentDataTrackerProvider`, `PopulateCurrentDataTrackersHandler`,
+  `CreateLoggableIndexesCommand` and `PopulateCurrentDataTrackersCommand`
+  changed: they now receive the storage interfaces instead of
+  `ElasticsearchServiceInterface`, `ElasticsearchContextFactoryInterface` and
+  `ElasticsearchIndexFactoryInterface`. Update any decorators or service
+  definitions overriding their arguments.
+- `ActivityLogProviderInterface::getActivityLogsByIndexAndId()` was removed
+  from the interface because it leaks the Elasticsearch index name into the
+  storage-agnostic API. Inject
+  `Locastic\Loggastic\Bridge\Elasticsearch\Storage\ElasticsearchActivityLogStorage`
+  and call `findByIndexAndObjectId()` instead.
+- `ActivityLogProvider::getCurrentDataTrackerByClassAndId()` was removed. It
+  declared an `?array` return type while the denormalizer returns an object,
+  so every call that found a tracker threw a `TypeError`. Use
+  `CurrentDataTrackerProviderInterface::getCurrentDataTrackerByClassAndId()`.
+- `Locastic\Loggastic\Bridge\Elasticsearch\Context\Traits\ElasticNormalizationContextTrait`
+  moved to `Locastic\Loggastic\Serializer\Traits\NormalizationContextTrait`;
+  it only forces the `\DateTime::ATOM` date format and is not
+  Elasticsearch-specific. Update the import if you used it.
+- The unused `Locastic\Loggastic\Exception\IndexNotFoundException` class was
+  removed.

config/elastic.yaml

@@ -42,4 +42,23 @@ services:
     Locastic\Loggastic\Bridge\Elasticsearch\Index\ElasticsearchIndexFactory:
         autowire: true
 
+#   Storage
+    Locastic\Loggastic\Storage\ActivityLogStorageInterface:
+        alias: 'Locastic\Loggastic\Bridge\Elasticsearch\Storage\ElasticsearchActivityLogStorage'
+
+    Locastic\Loggastic\Bridge\Elasticsearch\Storage\ElasticsearchActivityLogStorage:
+        autowire: true
+
+    Locastic\Loggastic\Storage\CurrentDataTrackerStorageInterface:
+        alias: 'Locastic\Loggastic\Bridge\Elasticsearch\Storage\ElasticsearchCurrentDataTrackerStorage'
+
+    Locastic\Loggastic\Bridge\Elasticsearch\Storage\ElasticsearchCurrentDataTrackerStorage:
+        autowire: true
+
+    Locastic\Loggastic\Storage\StorageInitializerInterface:
+        alias: 'Locastic\Loggastic\Bridge\Elasticsearch\Storage\ElasticsearchStorageInitializer'
+
+    Locastic\Loggastic\Bridge\Elasticsearch\Storage\ElasticsearchStorageInitializer:
+        autowire: true
+
 

config/logger.yaml

@@ -24,9 +24,9 @@ services:
 
     Locastic\Loggastic\DataProcessor\ActivityLogProcessor:
         arguments:
-            - '@Locastic\Loggastic\Bridge\Elasticsearch\Context\ElasticsearchContextFactoryInterface'
             - '@serializer.normalizer.object'
-            - '@Locastic\Loggastic\Bridge\Elasticsearch\ElasticsearchServiceInterface'
+            - '@Locastic\Loggastic\Storage\ActivityLogStorageInterface'
+            - '@Locastic\Loggastic\Storage\CurrentDataTrackerStorageInterface'
             - '@Locastic\Loggastic\Factory\ActivityLogInputFactoryInterface'
             - '@Locastic\Loggastic\Factory\CurrentDataTrackerInputFactoryInterface'
             - '@Locastic\Loggastic\Metadata\LoggableContext\Factory\LoggableContextFactoryInterface'

config/message_handlers.yaml

@@ -19,7 +19,6 @@ services:
             - '@Doctrine\Persistence\ManagerRegistry'
             - '@Locastic\Loggastic\Factory\CurrentDataTrackerInputFactoryInterface'
             - '@serializer.normalizer.object'
-            - '@Locastic\Loggastic\Bridge\Elasticsearch\ElasticsearchServiceInterface'
-            - '@Locastic\Loggastic\Bridge\Elasticsearch\Context\ElasticsearchContextFactoryInterface'
+            - '@Locastic\Loggastic\Storage\CurrentDataTrackerStorageInterface'
         tags:
             - { name: messenger.message_handler }

phpstan-baseline.neon

@@ -282,36 +282,6 @@ parameters:
 			count: 1
 			path: src/DataProvider/ActivityLogProvider.php
 
-		-
-			message: '#^Method Locastic\\Loggastic\\DataProvider\\ActivityLogProvider\:\:getActivityLogsByIndexAndId\(\) has parameter \$objectId with no type specified\.$#'
-			identifier: missingType.parameter
-			count: 1
-			path: src/DataProvider/ActivityLogProvider.php
-
-		-
-			message: '#^Method Locastic\\Loggastic\\DataProvider\\ActivityLogProvider\:\:getActivityLogsByIndexAndId\(\) has parameter \$sort with no value type specified in iterable type array\.$#'
-			identifier: missingType.iterableValue
-			count: 1
-			path: src/DataProvider/ActivityLogProvider.php
-
-		-
-			message: '#^Method Locastic\\Loggastic\\DataProvider\\ActivityLogProvider\:\:getActivityLogsByIndexAndId\(\) return type has no value type specified in iterable type array\.$#'
-			identifier: missingType.iterableValue
-			count: 1
-			path: src/DataProvider/ActivityLogProvider.php
-
-		-
-			message: '#^Method Locastic\\Loggastic\\DataProvider\\ActivityLogProvider\:\:getCurrentDataTrackerByClassAndId\(\) has parameter \$objectId with no type specified\.$#'
-			identifier: missingType.parameter
-			count: 1
-			path: src/DataProvider/ActivityLogProvider.php
-
-		-
-			message: '#^Method Locastic\\Loggastic\\DataProvider\\ActivityLogProvider\:\:getCurrentDataTrackerByClassAndId\(\) return type has no value type specified in iterable type array\.$#'
-			identifier: missingType.iterableValue
-			count: 1
-			path: src/DataProvider/ActivityLogProvider.php
-
 		-
 			message: '#^Method Locastic\\Loggastic\\DataProvider\\ActivityLogProviderInterface\:\:getActivityLogsByClass\(\) has parameter \$sort with no value type specified in iterable type array\.$#'
 			identifier: missingType.iterableValue
@@ -342,24 +312,6 @@ parameters:
 			count: 1
 			path: src/DataProvider/ActivityLogProviderInterface.php
 
-		-
-			message: '#^Method Locastic\\Loggastic\\DataProvider\\ActivityLogProviderInterface\:\:getActivityLogsByIndexAndId\(\) has parameter \$objectId with no type specified\.$#'
-			identifier: missingType.parameter
-			count: 1
-			path: src/DataProvider/ActivityLogProviderInterface.php
-
-		-
-			message: '#^Method Locastic\\Loggastic\\DataProvider\\ActivityLogProviderInterface\:\:getActivityLogsByIndexAndId\(\) has parameter \$sort with no value type specified in iterable type array\.$#'
-			identifier: missingType.iterableValue
-			count: 1
-			path: src/DataProvider/ActivityLogProviderInterface.php
-
-		-
-			message: '#^Method Locastic\\Loggastic\\DataProvider\\ActivityLogProviderInterface\:\:getActivityLogsByIndexAndId\(\) return type has no value type specified in iterable type array\.$#'
-			identifier: missingType.iterableValue
-			count: 1
-			path: src/DataProvider/ActivityLogProviderInterface.php
-
 		-
 			message: '#^Method Locastic\\Loggastic\\DataProvider\\CurrentDataTrackerProvider\:\:getCurrentDataTrackerByClassAndId\(\) has parameter \$objectId with no type specified\.$#'
 			identifier: missingType.parameter

src/Bridge/Elasticsearch/ElasticsearchService.php

@@ -2,14 +2,14 @@
 
 namespace Locastic\Loggastic\Bridge\Elasticsearch;
 
-use Locastic\Loggastic\Bridge\Elasticsearch\Context\Traits\ElasticNormalizationContextTrait;
+use Locastic\Loggastic\Serializer\Traits\NormalizationContextTrait;
 use Symfony\Component\Serializer\Normalizer\AbstractNormalizer;
 use Symfony\Component\Serializer\Normalizer\DenormalizerInterface;
 use Symfony\Component\Serializer\Normalizer\NormalizerInterface;
 
 final class ElasticsearchService implements ElasticsearchServiceInterface
 {
-    use ElasticNormalizationContextTrait;
+    use NormalizationContextTrait;
 
     public function __construct(
         private readonly ElasticsearchClientInterface $elasticsearchClient,

src/Bridge/Elasticsearch/Storage/ElasticsearchActivityLogStorage.php

@@ -0,0 +1,88 @@
+<?php
+
+namespace Locastic\Loggastic\Bridge\Elasticsearch\Storage;
+
+use Locastic\Loggastic\Bridge\Elasticsearch\Context\ElasticsearchContextFactoryInterface;
+use Locastic\Loggastic\Bridge\Elasticsearch\ElasticsearchServiceInterface;
+use Locastic\Loggastic\Model\Input\ActivityLogInputInterface;
+use Locastic\Loggastic\Model\Output\ActivityLog;
+use Locastic\Loggastic\Model\Output\ActivityLogInterface;
+use Locastic\Loggastic\Storage\ActivityLogStorageInterface;
+
+final class ElasticsearchActivityLogStorage implements ActivityLogStorageInterface
+{
+    public function __construct(
+        private readonly ElasticsearchServiceInterface $elasticsearchService,
+        private readonly ElasticsearchContextFactoryInterface $elasticsearchContextFactory,
+    ) {
+    }
+
+    public function save(ActivityLogInputInterface $activityLog, string $className): void
+    {
+        $elasticContext = $this->elasticsearchContextFactory->create($className);
+
+        $this->elasticsearchService->createItem($activityLog, $elasticContext->getActivityLogIndex(), ['activity_log']);
+    }
+
+    /**
+     * @param array<string, string> $sort
+     *
+     * @return array<int, ActivityLogInterface>
+     */
+    public function findByClass(string $className, array $sort = [], int $limit = 20, int $offset = 0): array
+    {
+        $elasticContext = $this->elasticsearchContextFactory->create($className);
+
+        $body = [
+            'sort' => $sort,
+        ];
+
+        // todo move class to config
+        return $this->elasticsearchService->getCollection(
+            $elasticContext->getActivityLogIndex(),
+            ActivityLog::class,
+            $body,
+            $limit,
+            $offset
+        );
+    }
+
+    /**
+     * @param array<string, string> $sort
+     *
+     * @return array<int, ActivityLogInterface>
+     */
+    public function findByClassAndObjectId(string $className, mixed $objectId, array $sort = [], int $limit = 20, int $offset = 0): array
+    {
+        $elasticContext = $this->elasticsearchContextFactory->create($className);
+
+        return $this->findByIndexAndObjectId($elasticContext->getActivityLogIndex(), $objectId, $sort, $limit, $offset);
+    }
+
+    /**
+     * Elasticsearch-specific: reads activity logs directly from an index, bypassing
+     * the class name to index resolution.
+     *
+     * @param array<string, string> $sort
+     *
+     * @return array<int, ActivityLogInterface>
+     */
+    public function findByIndexAndObjectId(string $index, mixed $objectId, array $sort = [], int $limit = 20, int $offset = 0): array
+    {
+        $body = [
+            'sort' => $sort,
+            'query' => [
+                'term' => ['objectId' => $objectId],
+            ],
+        ];
+
+        // todo move class to config
+        return $this->elasticsearchService->getCollection(
+            $index,
+            ActivityLog::class,
+            $body,
+            $limit,
+            $offset
+        );
+    }
+}

src/Bridge/Elasticsearch/Storage/ElasticsearchCurrentDataTrackerStorage.php

@@ -0,0 +1,59 @@
+<?php
+
+namespace Locastic\Loggastic\Bridge\Elasticsearch\Storage;
+
+use Locastic\Loggastic\Bridge\Elasticsearch\Context\ElasticsearchContextFactoryInterface;
+use Locastic\Loggastic\Bridge\Elasticsearch\ElasticsearchServiceInterface;
+use Locastic\Loggastic\Model\Input\CurrentDataTrackerInputInterface;
+use Locastic\Loggastic\Model\Output\CurrentDataTracker;
+use Locastic\Loggastic\Model\Output\CurrentDataTrackerInterface;
+use Locastic\Loggastic\Storage\CurrentDataTrackerStorageInterface;
+
+final class ElasticsearchCurrentDataTrackerStorage implements CurrentDataTrackerStorageInterface
+{
+    public function __construct(
+        private readonly ElasticsearchServiceInterface $elasticsearchService,
+        private readonly ElasticsearchContextFactoryInterface $elasticsearchContextFactory,
+    ) {
+    }
+
+    public function save(CurrentDataTrackerInputInterface $currentDataTracker, string $className): void
+    {
+        $elasticContext = $this->elasticsearchContextFactory->create($className);
+
+        $this->elasticsearchService->createItem($currentDataTracker, $elasticContext->getCurrentDataTrackerIndex(), ['current_data_tracker']);
+    }
+
+    public function update(mixed $id, CurrentDataTrackerInputInterface $currentDataTracker, string $className): void
+    {
+        $elasticContext = $this->elasticsearchContextFactory->create($className);
+
+        $this->elasticsearchService->updateItem($id, $currentDataTracker, $elasticContext->getCurrentDataTrackerIndex(), ['current_data_tracker']);
+    }
+
+    /**
+     * @param array<int, CurrentDataTrackerInputInterface> $currentDataTrackers
+     */
+    public function bulkSave(array $currentDataTrackers, string $className): void
+    {
+        $elasticContext = $this->elasticsearchContextFactory->create($className);
+
+        $this->elasticsearchService->bulkCreate($currentDataTrackers, $elasticContext->getCurrentDataTrackerIndex(), ['current_data_tracker']);
+    }
+
+    public function findByClassAndObjectId(string $className, mixed $objectId): ?CurrentDataTrackerInterface
+    {
+        $elasticContext = $this->elasticsearchContextFactory->create($className);
+
+        $body = [
+            'query' => ['term' => ['objectId' => $objectId]],
+        ];
+
+        // todo move class to config
+        return $this->elasticsearchService->getItemByQuery(
+            $elasticContext->getCurrentDataTrackerIndex(),
+            CurrentDataTracker::class,
+            $body
+        );
+    }
+}