Skip to content

Commit 01c3181

Browse files
committed
fix(schema): stop ImportHandler grafting an INT $ref onto array properties, and drop any $ref Opis cannot use
`ObjectService::saveObject()` answered 500 with Opis\JsonSchema\Exceptions\InvalidKeywordException: $ref must be a non-empty string on a clean install, from a payload the caller had no way to fix. Reproduced, and the cause is ours in two places. **1. The import writes the resolved id to the wrong key.** `importSchema()` maps `$ref` slugs to schema ids. The `items.$ref` block's `schemasMap` fallback assigned to `$property['$ref']` where it meant `$property['items']['$ref']`, so it did two wrong things at once: it left `items.$ref` an unmapped slug, and it grafted a schema id — an INT — onto the ARRAY property as a top-level `$ref`. Only the `schemasMap` fallback is affected, i.e. a reference that resolves against schemas already present rather than against this import's own slug->id map. That is the FIRST install of such a configuration, which is exactly why it never appeared on a long-lived instance and appeared on every clean one. **2. Validation had no floor under it.** `ValidateObject` strips `$ref` before handing a schema to Opis, but only for the shapes its branches recognise: string-typed properties, `items`, self-references, and object properties carrying an `objectConfiguration.handling`. An int `$ref` on an array property matches none of them and survives. Opis then parses that property's subschema — LAZILY, only when the property is present in the written data — and throws, naming neither the property nor the schema. `dropUnusableRef()` removes any `$ref` that is not a non-empty string, which a JSON Schema `$ref` must be. OpenRegister only ever uses `$ref` as a relation marker, never for validation-time resolution, so nothing is lost — and this is what heals the databases already carrying the bad value, which fixing the import alone would not. **Evidence.** Negative control: with `dropUnusableRef()` reverted, all four new tests error with the production exception and the production stack — `PropertiesKeyword:79 -> SchemaLoader:99 -> LazySchema:55 -> RefKeywordParser:58` — the same frames hermiq's clean-install e2e logged on `PUT /api/agents/{id}/tool-grants` (run 30865280923). With the fix, four pass. One of the four is itself a negative control: a wrong-TYPED value must still fail validation, so "no throw" cannot be reached by switching the property off. The pre-existing `testImportSchemaResolvesItemsRefFromSchemasMap` asserted only `instanceof Schema`, which is true whatever that branch writes — it exercised the defect and reported green. It now asserts on the payload handed to the mapper: `items.$ref` resolved, and no top-level `$ref` invented. 15961 unit tests green. phpcs lib zero errors, phpmd clean.
1 parent ca11506 commit 01c3181

4 files changed

Lines changed: 335 additions & 5 deletions

File tree

lib/Service/Configuration/ImportHandler.php

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1490,11 +1490,34 @@ public function importSchema(
14901490
}
14911491
}
14921492

1493+
// 🔴 Both branches write to `items.$ref`. The second one used to
1494+
// write to `$property['$ref']` — the ARRAY's own ref — which did
1495+
// two wrong things at once: it left `items.$ref` as an unmapped
1496+
// slug, and it grafted a schema ID (an INT) onto the array
1497+
// property as a top-level `$ref`.
1498+
//
1499+
// That second effect is not cosmetic. `ValidateObject` strips a
1500+
// top-level `$ref` from string-typed properties and from
1501+
// `items`, but never from an array-typed property, so the int
1502+
// survived into the schema handed to Opis. Opis parses a
1503+
// property's subschema lazily — only when that property is
1504+
// PRESENT in the written data — and then throws
1505+
// `$ref must be a non-empty string` because an int is not a
1506+
// string. The message names neither the property nor the
1507+
// schema, so it reads like a broken register.
1508+
//
1509+
// Only reachable on the `schemasMap` fallback, i.e. when the
1510+
// referenced slug was not part of this import's own
1511+
// slug->id map. That is the FIRST install of a configuration
1512+
// whose cross-references resolve against already-present
1513+
// schemas — which is why it never showed on a long-lived
1514+
// instance and surfaced on every clean one (hermiq's `agent`
1515+
// schema: skillInstalls / contextRefs / delegationAllowlist).
14931516
if (($property['items']['$ref'] ?? null) !== null) {
14941517
if (($slugsAndIdsMap[$property['items']['$ref']] ?? null) !== null) {
14951518
$property['items']['$ref'] = $slugsAndIdsMap[$property['items']['$ref']];
14961519
} else if (($this->schemasMap[$property['items']['$ref']] ?? null) !== null) {
1497-
$property['$ref'] = $this->schemasMap[$property['items']['$ref']]->getId();
1520+
$property['items']['$ref'] = $this->schemasMap[$property['items']['$ref']]->getId();
14981521
}
14991522
}
15001523

lib/Service/Object/ValidateObject.php

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -468,6 +468,27 @@ private function transformOpenRegisterObjectConfigurations(object $schemaObject)
468468
*/
469469
private function transformPropertyForOpenRegister(object $propertySchema): void
470470
{
471+
// 🔴 First, drop any `$ref` that CANNOT be a JSON Schema reference.
472+
//
473+
// OpenRegister stores a `$ref` as a relation marker, and every branch
474+
// below already removes it before the schema reaches Opis — but only for
475+
// the shapes those branches recognise. A `$ref` that is an int, a null,
476+
// an array or an empty string is not one of them, survives, and makes
477+
// Opis throw `$ref must be a non-empty string` from
478+
// RefKeywordParser::parse().
479+
//
480+
// That exception fires LAZILY, when the offending property is present in
481+
// the written data, so it names neither the property nor the schema and
482+
// reads like a broken register rather than a stored schema defect. It is
483+
// also unrecoverable for the caller: no payload shape fixes a schema.
484+
//
485+
// Registers imported before openregister#2321 already carry exactly this
486+
// (an int `$ref` grafted onto an array property by ImportHandler), so
487+
// this normalisation is what heals them without a migration. A VALID
488+
// reference — a non-empty string — is untouched here and handled by the
489+
// branches below exactly as before.
490+
$this->dropUnusableRef(schema: $propertySchema);
491+
471492
// UUID pattern for related object references.
472493
$uuidPat = '^([a-z]+-)?([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}|[0-9a-f]{32}|[0-9]+)$';
473494

@@ -539,6 +560,9 @@ private function transformPropertyForOpenRegister(object $propertySchema): void
539560
if ($isArrayType === true && $hasItems === true && is_object($propertySchema->items) === true) {
540561
$itemsSchema = $propertySchema->items;
541562

563+
// Same normalisation as the property itself — see dropUnusableRef().
564+
$this->dropUnusableRef(schema: $itemsSchema);
565+
542566
// Handle inversedBy relationships for array items.
543567
// TODO: Move writeBack, removeAfterWriteBack, and inversedBy from items to config.
544568
if (($itemsSchema->inversedBy ?? null) !== null) {
@@ -598,6 +622,37 @@ private function transformPropertyForOpenRegister(object $propertySchema): void
598622
}
599623
}//end transformPropertyForOpenRegister()
600624

625+
/**
626+
* Remove a `$ref` that cannot be a JSON Schema reference.
627+
*
628+
* A JSON Schema `$ref` MUST be a non-empty string; Opis enforces that in
629+
* `RefKeywordParser::parse()` and throws `$ref must be a non-empty string`
630+
* for anything else. OpenRegister only ever uses `$ref` as a relation
631+
* marker, never for validation-time resolution, so removing an unusable one
632+
* loses nothing and turns an opaque 500 back into a normal validation pass.
633+
*
634+
* Valid refs (non-empty strings) are deliberately left alone — the
635+
* surrounding transform branches own those.
636+
*
637+
* @param object $schema The property or items schema to normalise in place.
638+
*
639+
* @return void
640+
*/
641+
private function dropUnusableRef(object $schema): void
642+
{
643+
if (property_exists($schema, '$ref') === false) {
644+
return;
645+
}
646+
647+
$ref = $schema->{'$ref'};
648+
if (is_string($ref) === true && $ref !== '') {
649+
return;
650+
}
651+
652+
unset($schema->{'$ref'});
653+
654+
}//end dropUnusableRef()
655+
601656
/**
602657
* Transforms object properties based on OpenRegister object configuration.
603658
*

tests/Unit/Service/Configuration/ImportHandlerCoverageTest.php

Lines changed: 38 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -367,9 +367,22 @@ public function testGetDuplicateRegisterInfoShowsUnknownWhenNoCreatedDate(): voi
367367
// =========================================================================
368368

369369
/**
370-
* importSchema resolves items.$ref from schemasMap.
370+
* importSchema resolves items.$ref from schemasMap — INTO `items`, and
371+
* without inventing a top-level `$ref` on the array property.
372+
*
373+
* 🔴 This used to assert only `instanceof Schema`, which is true whatever
374+
* the branch writes, so it exercised the defect and reported green. The
375+
* branch wrote the resolved id to `$property['$ref']` instead of
376+
* `$property['items']['$ref']`, which left the items ref an unmapped slug
377+
* AND grafted an INT `$ref` onto an array property. Opis rejects a
378+
* non-string `$ref` with `$ref must be a non-empty string` the moment such
379+
* a property appears in a written object — the 500 hermiq's clean-install
380+
* e2e hit on `PUT /api/agents/{id}/tool-grants`.
381+
*
382+
* The assertions are on the payload handed to the mapper, because that is
383+
* what gets persisted.
371384
*/
372-
public function testImportSchemaResolvesItemsRefFromSchemasMap(): void
385+
public function testImportSchemaResolvesItemsRefIntoItemsAndNotOntoTheProperty(): void
373386
{
374387
$refSchema = $this->makeSchema(42, 'related-schema');
375388
$this->setProperty($this->handler, 'schemasMap', ['related-schema' => $refSchema]);
@@ -388,19 +401,40 @@ public function testImportSchemaResolvesItemsRefFromSchemasMap(): void
388401
];
389402

390403
$createdSchema = $this->makeSchema(100, 'items-ref-schema');
404+
$persisted = null;
391405

392406
$this->schemaMapper->method('find')
393407
->willThrowException(new \OCP\AppFramework\Db\DoesNotExistException(''));
394408
$this->schemaMapper->method('createFromArray')
395-
->willReturn($createdSchema);
409+
->willReturnCallback(
410+
function (array $payload) use (&$persisted, $createdSchema): Schema {
411+
$persisted = $payload;
412+
return $createdSchema;
413+
}
414+
);
396415
$this->schemaMapper->method('update')
397416
->willReturnArgument(0);
398417

399418
$result = $this->handler->importSchema($data, [], null, null, '1.0.0');
400419

401420
$this->assertInstanceOf(Schema::class, $result);
421+
$this->assertIsArray($persisted, 'the schema payload must reach the mapper');
422+
423+
$property = $persisted['properties']['items_prop'];
424+
425+
$this->assertSame(
426+
42,
427+
$property['items']['$ref'],
428+
'the items $ref must be resolved to the referenced schema id'
429+
);
430+
$this->assertArrayNotHasKey(
431+
'$ref',
432+
$property,
433+
'an ARRAY property must never gain a top-level $ref from its items — '
434+
.'a non-string $ref makes Opis throw "$ref must be a non-empty string"'
435+
);
402436

403-
}//end testImportSchemaResolvesItemsRefFromSchemasMap()
437+
}//end testImportSchemaResolvesItemsRefIntoItemsAndNotOntoTheProperty()
404438

405439

406440
// =========================================================================
Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
<?php
2+
3+
declare(strict_types=1);
4+
5+
/**
6+
* ValidateObject unusable-`$ref` Unit Tests.
7+
*
8+
* A JSON Schema `$ref` MUST be a non-empty string. Opis enforces that in
9+
* `RefKeywordParser::parse()` and throws
10+
* `Opis\JsonSchema\Exceptions\InvalidKeywordException: $ref must be a non-empty string`
11+
* for anything else.
12+
*
13+
* OpenRegister stores `$ref` as a RELATION marker, never as something Opis
14+
* should resolve, and `ValidateObject` strips it before validation — but only
15+
* for the shapes its transform branches recognise: string-typed properties,
16+
* `items`, self-references and object properties carrying an
17+
* `objectConfiguration.handling`. A `$ref` that is an INT, null, an array or an
18+
* empty string matches none of them, survives into the schema handed to Opis,
19+
* and blows up.
20+
*
21+
* That is not hypothetical. `ImportHandler::importSchema()` wrote a resolved
22+
* schema ID (an int) to `$property['$ref']` when it meant
23+
* `$property['items']['$ref']`, so any array-of-relations property imported
24+
* through the `schemasMap` fallback carried an int `$ref`. Measured on hermiq's
25+
* clean-install e2e (run 30865280923): `PUT /api/agents/{id}/tool-grants`
26+
* answered 500 with `$ref must be a non-empty string` from
27+
* `ObjectService::saveObject()`.
28+
*
29+
* The import side is fixed, but every database imported before that fix still
30+
* carries the bad value, so this normalisation is what heals them — and these
31+
* tests are what stop it regressing.
32+
*
33+
* @category Tests
34+
* @package OCA\OpenRegister\Tests\Unit\Service\Object
35+
*
36+
* @author Conduction Development Team <info@conduction.nl>
37+
* @copyright 2026 Conduction B.V.
38+
* @license EUPL-1.2 https://joinup.ec.europa.eu/collection/eupl/eupl-text-eupl-12
39+
*
40+
* @link https://OpenRegister.app
41+
*/
42+
43+
namespace OCA\OpenRegister\Tests\Unit\Service\Object;
44+
45+
use OCA\OpenRegister\Db\MagicMapper;
46+
use OCA\OpenRegister\Db\Schema;
47+
use OCA\OpenRegister\Db\SchemaMapper;
48+
use OCA\OpenRegister\Service\Object\ValidateObject;
49+
use OCP\IAppConfig;
50+
use OCP\IURLGenerator;
51+
use PHPUnit\Framework\TestCase;
52+
use Psr\Log\LoggerInterface;
53+
54+
/**
55+
* Locks the behaviour of an unusable `$ref` on a stored schema.
56+
*/
57+
class ValidateObjectUnusableRefTest extends TestCase
58+
{
59+
60+
/**
61+
* The subject under test.
62+
*
63+
* @var ValidateObject
64+
*/
65+
private ValidateObject $handler;
66+
67+
68+
/**
69+
* Build a ValidateObject with mocked collaborators.
70+
*
71+
* @return void
72+
*/
73+
protected function setUp(): void
74+
{
75+
parent::setUp();
76+
77+
$urlGenerator = $this->createMock(IURLGenerator::class);
78+
$urlGenerator->method('getBaseUrl')->willReturn('http://localhost:8080');
79+
80+
$this->handler = new ValidateObject(
81+
$this->createMock(IAppConfig::class),
82+
$this->createMock(MagicMapper::class),
83+
$this->createMock(SchemaMapper::class),
84+
$urlGenerator,
85+
$this->createMock(LoggerInterface::class)
86+
);
87+
88+
}//end setUp()
89+
90+
91+
/**
92+
* A Schema entity with the given slug.
93+
*
94+
* @param string $slug Schema slug.
95+
*
96+
* @return Schema
97+
*/
98+
private function schema(string $slug='agent'): Schema
99+
{
100+
$schema = new Schema();
101+
$schema->setSlug($slug);
102+
$schema->setTitle('Agent');
103+
return $schema;
104+
105+
}//end schema()
106+
107+
108+
/**
109+
* A schema object with one array property carrying the given top-level
110+
* `$ref` — the exact shape a pre-fix import left behind.
111+
*
112+
* @param mixed $ref The stored `$ref` value.
113+
*
114+
* @return object
115+
*/
116+
private function schemaWithPropertyRef(mixed $ref): object
117+
{
118+
return json_decode(
119+
json_encode(
120+
[
121+
'type' => 'object',
122+
'properties' => [
123+
'delegationAllowlist' => [
124+
'type' => 'array',
125+
'$ref' => $ref,
126+
'items' => [
127+
'type' => 'string',
128+
'format' => 'uuid',
129+
],
130+
],
131+
],
132+
]
133+
)
134+
);
135+
136+
}//end schemaWithPropertyRef()
137+
138+
139+
/**
140+
* An INT `$ref` — the ImportHandler shape — validates instead of throwing.
141+
*
142+
* The property must be PRESENT in the object: Opis parses a property's
143+
* subschema lazily, which is why this endpoint looked healthy for months
144+
* on instances whose optional relation arrays were never written.
145+
*
146+
* @return void
147+
*/
148+
public function testIntegerRefOnAnArrayPropertyDoesNotThrow(): void
149+
{
150+
$object = ['delegationAllowlist' => ['550e8400-e29b-41d4-a716-446655440000']];
151+
152+
$result = $this->handler->validateObject($object, $this->schema(), $this->schemaWithPropertyRef(4365));
153+
154+
$this->assertTrue(
155+
$result->isValid(),
156+
'an int $ref must be dropped, not handed to Opis as a JSON Schema reference'
157+
);
158+
159+
}//end testIntegerRefOnAnArrayPropertyDoesNotThrow()
160+
161+
162+
/**
163+
* An EMPTY-STRING `$ref` is equally unusable and equally dropped.
164+
*
165+
* @return void
166+
*/
167+
public function testEmptyStringRefDoesNotThrow(): void
168+
{
169+
$object = ['delegationAllowlist' => ['11111111-1111-1111-1111-111111111111']];
170+
171+
$result = $this->handler->validateObject($object, $this->schema(), $this->schemaWithPropertyRef(''));
172+
173+
$this->assertTrue($result->isValid(), 'an empty-string $ref must be dropped');
174+
175+
}//end testEmptyStringRefDoesNotThrow()
176+
177+
178+
/**
179+
* A NULL `$ref` is dropped too — and a null property value, which survives
180+
* OpenRegister's own empty-value filter, is what actually reaches Opis.
181+
*
182+
* @return void
183+
*/
184+
public function testNullRefWithANullValueDoesNotThrow(): void
185+
{
186+
$object = ['delegationAllowlist' => null];
187+
188+
$result = $this->handler->validateObject($object, $this->schema(), $this->schemaWithPropertyRef(null));
189+
190+
$this->assertTrue($result->isValid(), 'a null $ref must be dropped');
191+
192+
}//end testNullRefWithANullValueDoesNotThrow()
193+
194+
195+
/**
196+
* 🔑 NEGATIVE CONTROL. Dropping the unusable `$ref` must not switch
197+
* validation off for the property — a value of the wrong TYPE still fails.
198+
*
199+
* Without this, all three tests above would pass just as well if the fix
200+
* had removed the property from validation altogether.
201+
*
202+
* @return void
203+
*/
204+
public function testDroppingTheRefStillLeavesTheTypeEnforced(): void
205+
{
206+
$object = ['delegationAllowlist' => 'not an array at all'];
207+
208+
$result = $this->handler->validateObject($object, $this->schema(), $this->schemaWithPropertyRef(4365));
209+
210+
$this->assertFalse(
211+
$result->isValid(),
212+
'a string where the schema declares an array must still fail validation'
213+
);
214+
215+
}//end testDroppingTheRefStillLeavesTheTypeEnforced()
216+
217+
218+
}//end class

0 commit comments

Comments
 (0)