You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/constraints.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ description: Anything that constrains teams in design and implementation decisio
11
11
12
12
The architectural constraints outlined in this section shape the design decisions for the 1+MG Network. They are non-negotiable limitations imposed by regulation, politics, or organization.
@@ -23,14 +23,14 @@ The architectural constraints outlined in this section shape the design decision
23
23
|**L06**|**NIS2 Directive (Cybersecurity)**| As a "Critical Entity" (Health Sector), the infrastructure must implement state-of-the-art cyber hygiene, mandatory incident reporting (24h warning/72h full report), and supply chain security auditing.[^5]|
24
24
|**L07**|**International Transfers (GDPR Ch. V)**| Connections to **Global Resources** (e.g., All of Us, H3Africa) situated in non-EEA countries without an adequacy decision must rely on **Standard Contractual Clauses (SCCs)** or specific derogations (Art. 49).[^27]|
|**P01**|**Local Sovereignty (The "Red Line")**| The architecture must ensure that **Local Data Authorities (DACs)** retain absolute vetting power over every access request. No "Central Super-Admin" can exist who can view data without local approval.[^6]|
31
31
|**P02**|**Equity & Inclusivity**| The system must support smaller Member States with limited infrastructure. It cannot mandate expensive hardware (e.g., proprietary HPC) that would exclude less-resourced nations (The "Widening" principle).[^7]|
@@ -50,7 +50,7 @@ The architectural constraints outlined in this section shape the design decision
50
50
|**O14**|**Project Funding Cycles**| Sustainability must be planned beyond typical 3-5 year EU project funding cycles. |
51
51
|**O15**|**Reprocessing Cost Sensitivity**| The architecture must account for significant cost disparities in data reprocessing (e.g., ~€5 for short-read vs ~€530 for long-read per genome), necessitating careful decisions on when and where to standardize data [^10]. |
Copy file name to clipboardExpand all lines: docs/crosscutting-concepts.md
+12-12Lines changed: 12 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,9 +9,9 @@ description: Overall, principal regulations and solution approaches relevant in
9
9
10
10
# Cross-cutting Concepts
11
11
12
-
## 1. Data Strategy
12
+
## Data Strategy
13
13
14
-
### 1.1. Metadata Model (HealthDCAT-AP)
14
+
### Metadata Model (HealthDCAT-AP)
15
15
16
16
To ensure semantic interoperability and alignment with the **European Health Data Space (EHDS)**, 1+MG adopts the **HealthDCAT-AP 5.0** standard (a health-specific profile of DCAT-AP)[^1][^2][^3].
17
17
@@ -24,7 +24,7 @@ To ensure semantic interoperability and alignment with the **European Health Dat
24
24
-**Distribution:** The physical form (`dcat:Distribution`).
25
25
-_Example:_ VCF File, Beacon API Endpoint, WES Endpoint.
26
26
27
-
### 1.2. Identifier Schema
27
+
### Identifier Schema
28
28
29
29
All resources within the 1+MG network MUST adhere to the following **Persistent Identifier (PID)** pattern to guarantee global uniqueness across the federation[^1]:
30
30
@@ -42,11 +42,11 @@ All resources within the 1+MG network MUST adhere to the following **Persistent
42
42
-`GDI-FI-COHORT-001` (First cohort from Finland GDI Node)
43
43
-`GOE-ES-SAMPLE-9942` (Genome of Europe sample from Spain)
44
44
45
-
## 2. Security & Compliance
45
+
## Security & Compliance
46
46
47
-
## 2. Security & Compliance
47
+
## Security & Compliance
48
48
49
-
### 2.1. Data Protection by Design (Five Safes)
49
+
### Data Protection by Design (Five Safes)
50
50
51
51
We adhere to the **Five Safes** framework to demonstrate compliance with GDPR Data Protection by Design and Default (DPbDD)[^11]:
52
52
@@ -56,33 +56,33 @@ We adhere to the **Five Safes** framework to demonstrate compliance with GDPR Da
56
56
4.**Safe Settings:** Does the access facility limit unauthorized use? (Secure Processing Environment).
57
57
5.**Safe Outputs:** Is confidentiality maintained for outputs? (Airlock/Egress Control).
58
58
59
-
### 2.2. Controlled Vocabularies
59
+
### Controlled Vocabularies
60
60
61
61
We rely on the **Data Privacy Vocabulary (DPV)** to express legal bases and consent status in a machine-readable way[^5].
62
62
63
63
-`dpv:Consent`: Processing based on explicit data subject consent.
64
64
-`dpv:LegitimateInterest`: Processing based on legitimate interest (secondary use).
65
65
66
-
### 2.3. Encryption
66
+
### Encryption
67
67
68
68
-**Data at Rest:** All genomic files (VCF/BAM) are encrypted using **Crypt4GH** (standard container format for encrypted genomic data)[^6].
69
69
-**Data in Transit:** TLS 1.3 is mandatory for all APIs.
70
70
71
-
### 2.4. Authentication & Authorization
71
+
### Authentication & Authorization
72
72
73
73
-**Authentication:** Federated via **LS AAI**. Users authenticate at their Home Organisation (IdP).
74
74
-**Authorization:** Claims are transported via **GA4GH Passports**[^7].
75
75
-**Visas:** Signed JWTs asserting permissions (e.g., `ControlledAccessGrants`).
76
76
-**Verification:** The National Node PEP MUST cryptographically verify the Visa signature against the DAC's public key[^8].
77
77
78
-
## 3. Observability & Reliability
78
+
## Observability & Reliability
79
79
80
-
### 3.1. Logging & Auditing
80
+
### Logging & Auditing
81
81
82
82
-**Audit Trails:** Every access to genomic data (successful or denied) MUST be logged.
83
83
-**Privacy:** Logs must NOT contain PII (e.g., query parameters with variant details) but MUST record User ID, Resource ID, Timestamp, and Decision[^5].
84
84
85
-
### 3.2. Error Handling
85
+
### Error Handling
86
86
87
87
-**Standard:** APIs must return standard HTTP status codes (401 vs 403 distinction is critical).
88
88
-**Format:** Error bodies MUST follow **RFC 7807** (Problem Details for HTTP APIs)[^10].
Copy file name to clipboardExpand all lines: docs/oneplusmg-infrastructure/building-block-view.md
+8-8Lines changed: 8 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ description: Static decomposition of the system, abstractions of source-code, sh
9
9
10
10
# Building Block View
11
11
12
-
## 1. Whitebox Overall System (Level 1)
12
+
## Whitebox Overall System (Level 1)
13
13
14
14
The 1+MG Infrastructure is decomposed into three primary logical zones:
15
15
@@ -19,7 +19,7 @@ The 1+MG Infrastructure is decomposed into three primary logical zones:
19
19
|**Trust & Identity Broker (Central)**|**Functional Description:** Aggregates Claims from DACs and identities from LS AAI into GA4GH Passports.<br/>**Interfaces:** OIDC, GA4GH Passport API.<br/>**Quality:** High Availability, High Security (Signatures).<br/>**Location:**`src/broker`. |
20
20
|**National Node (Federated)**|**Functional Description:** Stores encrypted genomic data and executes local compute workflows.<br/>**Interfaces:** WES (Compute), DRS (Data), Beacon (Discovery).<br/>**Quality:** High Security (Encryption), Data Sovereignty.<br/>**Location:**`src/node`, Hosted by Member State. |
21
21
22
-
## 2. Level 2: 1+MG User Portal
22
+
## Level 2: 1+MG User Portal
23
23
24
24
The User Portal is the central entry point for researchers, integrating several distinct services[^4].
25
25
@@ -29,17 +29,17 @@ The User Portal is the central entry point for researchers, integrating several
29
29
|**Dataset Catalogue**|**Functional Description:** Searchable registry of available datasets (CKAN/FAIR).<br/>**Interfaces:** HTTPS.<br/>**Location:**`https://catalogue.portal.gdi.lu`|
30
30
|**DAAMS**|**Functional Description:** Manages the **Single Access Principle** workflow. Routes requests to Central DAC for review and then to National Nodes for Veto/Validation.<br/>**Interfaces:** HTTPS.<br/>**Location:**`https://daam.portal.gdi.lu`|
31
31
32
-
## 3. Level 2: National Node (GDI Node)
32
+
## Level 2: National Node (GDI Node)
33
33
34
34
The National Node is the most complex component. It must be deployed by each Member State.
35
35
36
-
### 3.1. Discovery Layer (Beacon)
36
+
### Discovery Layer (Beacon)
37
37
38
38
-**Beacon v2 API:** Receives queries (e.g., "Do you have variant X?").
39
39
-**Metadata DB:** Stores the HealthDCAT-AP catalogue (Cohorts, Datasets).
40
40
-**Response Strategy:** Checks local policies before returning `TRUE`/`FALSE` or counts[^2].
41
41
42
-
### 3.2. Secure Processing Environment (SPE) / TRE
42
+
### Secure Processing Environment (SPE) / TRE
43
43
44
44
This component follows the **TRE-FX** reference architecture for secure computation[^3].
45
45
@@ -50,20 +50,20 @@ This component follows the **TRE-FX** reference architecture for secure computat
50
50
-**Isolation:** The Compute Nodes have **NO direct internet access**. All dependencies (containers, reference genomes) must be available locally or proxied via the Airlock[^4].
51
51
-**Data Staging:** Pulls encrypted data from the Archive for the duration of the job.
52
52
53
-
### 3.3. Data Archive
53
+
### Data Archive
54
54
55
55
-**Encryption Layer:** Manages keys for data-at-rest encryption (Crypt4GH)[^5].
56
56
-**Data Integrity Service:** Ensures file validity using checksums and periodic scrubbing[^4].
57
57
-**Object Storage:** S3-compatible storage for VCF, BAM, and CRAM files.
58
58
-**Data Repository Service (DRS):** Resolves logical IDs (`drs://gdi...`) to physical signed URLs for internal compute use.
59
59
60
-
### 3.4. Data Onboarding & Curation
60
+
### Data Onboarding & Curation
61
61
62
62
-**Ingest Pipeline:** Validates quality of incoming data from Data Providers (integrity, format).
63
63
-**Pseudonymisation Service:** Replaces direct identifiers with 1+MG specific pseudonyms before long-term storage (Separation of Concerns).
64
64
-**Metadata Mapper:** Transforms local metadata into the common GDI model (HealthDCAT-AP).
65
65
66
-
### 3.5. Local Access Control
66
+
### Local Access Control
67
67
68
68
-**Policy Enforcement Point (PEP):** Intercepts every API call. Validates the User's GA4GH Passport and the specific Visas against the dataset's Access Control List (ACL).
The deployment architecture respects the **Data Residency** constraints (Constraint L02) by physically separating the Central Services from the National Storage.
13
13
14
-
## 1. Infrastructure Overview
14
+
## Infrastructure Overview
15
15
16
16
| Infrastructure Element | Environment | Geography | Responsibility |
Copy file name to clipboardExpand all lines: docs/oneplusmg-infrastructure/runtime-view.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@ description: Behavior of building blocks as scenarios, covering important use ca
9
9
10
10
# Runtime View
11
11
12
-
## 1. Data Access Workflow (The "Single Access Principle")
12
+
## Data Access Workflow (The "Single Access Principle")
13
13
14
14
This scenario describes the primary flow for a researcher to discover, request, and access genomic data across borders, illustrating the interplay between central governance and local sovereignty[^1][^2].
15
15
@@ -58,7 +58,7 @@ sequenceDiagram
58
58
5.**Provisioning:** If approved by all (or a subset of) nodes, the corresponding permissions are minted as **GA4GH Visas** by the **Trust Broker**.
59
59
6.**Secure Processing:** The researcher receives a **GA4GH Passport** containing these Visas. They submit a compute request (WES) to the National Node. The Node's **Policy Enforcement Point (PEP)** validates the Passport against the local access control list before executing the workflow in the isolated **SPE**.
60
60
61
-
## 2. Secure Processing Workflow (TRE-FX)
61
+
## Secure Processing Workflow (TRE-FX)
62
62
63
63
This scenario details the execution phase (`ujGdi08`), focusing on how the **Secure Processing Environment (SPE)** ensures isolation while allowing external analysis[^2][^3].
Copy file name to clipboardExpand all lines: docs/oneplusmg-infrastructure/solution-strategy.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,22 +9,22 @@ description: Summary of the fundamental decisions and solution strategies that s
9
9
10
10
# Solution Strategy
11
11
12
-
## 1. Governance & Federation
12
+
## Governance & Federation
13
13
14
14
The fundamental strategic decision for 1+MG is **Federation**.
15
15
16
16
-**No Central Data Lake:** Genomic data is too sensitive and voluminous to move. It remains legally and physically within the Member State (National Node)[^1].
17
17
-**Bring Compute to Data:** Analysis algorithms are containerized (e.g., Nextflow pipelines) and dispatched to the data location, returning only aggregated results.
18
18
-**Sovereignty First:** The architecture prioritizes the **Local Data Authority (DAC)**. No access is granted without a cryptographically verifiable token from the local DAC.
19
19
20
-
## 2. Trust Framework
20
+
## Trust Framework
21
21
22
22
We employ a **Zero Trust** approach where identity and authorization are decoupled[^2].
23
23
24
24
-**Identity (Who are you?):** Verified by **LS AAI** (Life Science Authentication and Authorization Infrastructure). Users log in via their home university or research institute.
25
25
-**authorization (What can you do?):** Managed by **GA4GH Passports**. DACs issue "Visas" (signed claims) that travel with the user's request. The National Node verifies these Visas before executing any query or workload.
26
26
27
-
## 3. Technology Stack & Standards
27
+
## Technology Stack & Standards
28
28
29
29
To ensure interoperability across 27+ countries, we rely strictly on open standards[^3].
30
30
@@ -37,7 +37,7 @@ To ensure interoperability across 27+ countries, we rely strictly on open standa
|**Orchestration**|**Kubernetes**| Managing the lifecycle of microservices and compute jobs. |
39
39
40
-
## 4. Organizational Strategy
40
+
## Organizational Strategy
41
41
42
42
-**GDI Starter Kit:** To support smaller Member States, we provide a "Reference Implementation" (Starter Kit) that can be deployed out-of-the-box to spin up a compliant National Node[^4]. It includes **Galaxy Europe** integration, **Beacon v2**, and **WfExS**.
43
43
-**Separation of Concerns:** The **Genome EDIC** handles the catalog and governance logic, while **Member States** handle the storage and compute infrastructure.
0 commit comments