Skip to content

Pin migration notes to release 1.14.0 #1493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sven1103 merged 1 commit into from
Jul 30, 2026
Merged

Pin migration notes to release 1.14.0 #1493

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
150 changes: 2 additions & 148 deletions docs/migrations/NEXT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Migration: Next release

**Current pom version:** 1.12.10
**Current pom version:** TBD
**Target release version:** TBD *(renamed to `released/v<version>.md` at release cut)*
**Status:** under development

Expand All @@ -21,152 +21,6 @@ For the migration documentation structure, see [`README.md`](README.md).

| # | Script | Description | Risk |
|---|---|---|---|
| 1 | [`sql/migrations/create-associated-dataset.sql`](../sql/migrations/create-associated-dataset.sql) | Create `associated_dataset` table for connecting InvenioRDM datasets to projects (FEAT-DATSET-01) | low — new empty table, no data loss on rollback |

Each row links to its incremental script. The sections below expand each entry
with apply / verify / rollback detail.

---

## Migration #1: `associated_dataset` table

| Field | Value |
|---|---|
| **Story** | [#1467 — FEAT-DATSET-01: Connecting open, published datasets](https://github.com/qbicsoftware/data-manager-app/issues/1467) |
| **Feature** | [#1466 — FEAT-DATASET-CONNECTION](https://github.com/qbicsoftware/data-manager-app/issues/1466) |
| **ADRs** | [0001](../adr/0001-associated-datasets-domain-model.md), [0002](../adr/0002-invenio-rdm-api-client-credentials.md), [0003](../adr/0003-connection-lifecycle-stewardship.md) |
| **Scope** | Database schema only (new table) |
| **Script** | [`sql/migrations/create-associated-dataset.sql`](../sql/migrations/create-associated-dataset.sql) |
| **Target datasource** | `data_management` |

### What it does

Creates an empty `associated_dataset` table. Per [ADR-0001](../adr/0001-associated-datasets-domain-model.md):

- Lives in the `project-management` bounded context (table name starts with `associated_dataset`, not under the existing `dataset_*` namespace, to keep the InvenioRDM connection concept distinct from the legacy OpenBIS raw-data tables).
- Has four **universal columns** (`title`, `pid`, `version`, `publication_date`) duplicated from the JSON blob for SQL sort/filter.
- Stores **source-specific metadata** (InvenioRDM creators, community, access details) in a MariaDB `JSON` column (`resource_metadata`). At the expected scale of <100 rows/project, this is efficient; the escape hatch for future scale is a generated virtual column + index — no entity change required.
- **Soft-delete for removal** (ADR-0001, ADR-0003): deleting a connection sets `connection_state = 'REMOVED'`, leaving the row as an audit tombstone. Active query paths filter `REMOVED` out. There is no `DELETE` statement in v1.
- **No SQL FK constraint on `experiment_id`** — the application enforces referential integrity in memory so that project-ACL changes and experiment deletions don't cascade-delete dataset connections.

### What is NOT in this migration

A second table, `user_invenio_rdm_credential` (for per-user Personal Access
Tokens to access restricted datasets), is planned for Stories 14/15
(FEAT-DATSET-14, FEAT-DATSET-15). It is intentionally **not** created by this
migration because FEAT-DATSET-01 covers only public, open datasets. A follow-up
entry in this file — or in the next release's `NEXT.md` — will cover the
credentials table when those stories land.

### Pre-flight

Run these checks against the target database **before** applying:

```sql
-- 1. Confirm the table does NOT already exist
SELECT COUNT(*) AS table_exists
FROM information_schema.tables
WHERE table_schema = DATABASE()
AND table_name = 'associated_dataset';
-- Expected: 0

-- 2. Confirm target datasource charset/collation matches the rest of the schema
SELECT @@character_set_database AS charset,
@@collation_database AS collation;
-- Expected: utf8mb4, utf8mb4_unicode_ci

-- 3. Confirm you are connected to the data-management datasource
SELECT DATABASE();
-- Expected: data_management (or whatever your DM schema is named)
```

If the table already exists (return value = 1), this migration was already
applied — no further action required (the DDL is idempotent).

### Apply

```bash
mysql -u <user> -h <host> -P <port> data_management \
< sql/migrations/create-associated-dataset.sql
```

Or inline:

```sql
-- The full DDL is inlined in the script for review; see
-- sql/migrations/create-associated-dataset.sql
```

### Verify

```sql
-- 1. Confirm 15 columns
SELECT COUNT(*) AS column_count
FROM information_schema.columns
WHERE table_schema = DATABASE()
AND table_name = 'associated_dataset';
-- Expected: 15

-- 2. Confirm indexes are in place (4 keys + PK = 5 rows)
SELECT index_name, GROUP_CONCAT(column_name ORDER BY seq_in_index) AS columns
FROM information_schema.statistics
WHERE table_schema = DATABASE()
AND table_name = 'associated_dataset'
GROUP BY index_name
ORDER BY index_name;
-- Expected rows:
-- PRIMARY id
-- idx_assoc_ds_project project_id
-- idx_assoc_ds_project_state project_id,connection_state
-- idx_assoc_ds_source_type source_type
-- idx_assoc_ds_state connection_state

-- 3. Confirm the application can query the table
SELECT COUNT(*) AS active_connected_count
FROM associated_dataset
WHERE connection_state <> 'REMOVED';
-- Expected: 0 (empty table immediately after migration)

-- 4. Confirm no unintended table was created on the finance datasource
SELECT table_schema, table_name
FROM information_schema.tables
WHERE table_name = 'associated_dataset';
-- Expected: exactly one row, for the data-management schema
```

### Rollback

Rollback is destructive — any rows inserted by the application after the
migration will be lost. Because FEAT-DATSET-01 is the first release introducing
this table, rollback is only meaningful before any datasets have been connected
by users.

```sql
-- DANGER: drops all connected-dataset connections.
-- Safe only when the table is empty (see rollback plan below).
DROP TABLE IF EXISTS `associated_dataset`;
```

If the table contains connected-dataset rows, rollback requires one of:

1. **Reverse the feature deployment** to a version that does not reference the
table, then drop the table.
2. **Truncate the table**, then drop it. Both cases lose all connection data.

In either case, notify users that previously connected datasets will no longer
appear in their projects.

### Operator notes

- **No data is moved.** This migration only creates an empty table.
- **No downtime required.** `CREATE TABLE IF NOT EXISTS` on MariaDB takes
milliseconds and does not lock other tables.
- **No foreign key constraint is created** between
`associated_dataset.experiment_id` and `experiments_datamanager.id`. The
application enforces referential integrity in memory.
- **Future migration (same feature):** Stories 14/15 will add the
`user_invenio_rdm_credential` table. Expect a follow-up entry in this file
(or in a subsequent release's `NEXT.md`).

---

Expand Down Expand Up @@ -230,4 +84,4 @@ When this release is ready to ship:
5. **Reset `NEXT.md`:** create a fresh empty copy from the template above
for the *subsequent* release.

See [`README.md`](README.md) for the full migration documentation structure.
See [`README.md`](README.md) for the full migration documentation structure.
6 changes: 3 additions & 3 deletions docs/migrations/released/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ guide.

## Current state

**Empty.** The three-tier migration documentation structure was established in
2026-07 for FEAT-DATSET-01. Until the first release that uses the new
structure is cut, no files will appear here.
| Release | Guide |
|---|---|
| **1.14.0** | [`v1.14.0.md`](v1.14.0.md) |

For migrations targeting the upcoming (unreleased) version, see
[`../NEXT.md`](../NEXT.md).
Expand Down
Loading
Loading