Feature/graphql sources 20260906

[filiperochalopes]

Solves: OpenConceptLab/ocl_issues#2757

• TBv3 Add References form: Show concept display names via GraphQL on ID entry ocl_issues#2577
• Defer GraphQL permission decorators until group authorization service is complete ocl_issues#2356
• perf(graphql): align concept search with REST and add Elasticsearch safe-path #816
• Create GraphQL service permission archtecture #838
• Bump strawberry-graphql from 0.285.0 to 0.315.7 #877

Why? CIEL Lab needs improvements when loading worklists to load fast using ES-only retrieve for minimal payloads
Also: OpenConceptLab/ocl_issues#2577

GraphQL: permission-aware source and concept projections

Adds a GraphQL source query and extends concepts search so that simple payloads
(name, description, canonicalUrl, uri on sources; id, conceptId, externalId,
display, description, conceptClass, datatype.name on concepts) are served directly
from Elasticsearch instead of hydrating ORM objects, while still enforcing the existing
REST visibility rules. Selections that need relationships, mappings, extras, audit metadata,
or datatype details still hydrate from the database as before.

⚠️ Breaking changes / required action before deploy

Existing Elasticsearch indexes must be repopulated before this version is put into service.
This is a data requirement, not a code migration — there are no new database migrations or
environment variables — but skipping it produces incorrect GraphQL results with no error.

• ConceptDocument gains four fields with no equivalent in the current index:
  is_active, is_head, preferred_description, parent_public_can_view.
• SourceDocument gains three fields: description, uri, is_active.
• These are additive to the Elasticsearch mapping (existing REST search fields and behavior
  are unchanged), but documents indexed before this change do not have these fields
  populated, and Elasticsearch cannot backfill them on its own — they require reading the
  source data again and reindexing.

Concrete impact if you deploy without reindexing first:

• concepts query: the new index projection filters on is_active (and, for global/HEAD
  scope, is_head). Old documents don't have these fields set, so the filter matches zero
  documents. This is not surfaced as an error — the query returns a normal, valid-looking
  response: {"totalCount": 0, "hasNextPage": false, "results": []}. This is indistinguishable
  from a legitimate "no matches" search result. There is no partial-result flag, warning, or
  degraded-mode indicator in the response.
• source query: degrades gracefully. If is_active is missing on the source document, the
  index projection returns None and the resolver automatically falls back to the existing
  ORM path. Slower, but correct — no silent data loss here.
• Net effect: after deploying this change, concepts searches can silently go to zero
  results across the board until the reindex finishes, with no visible indication to API
  consumers that anything is wrong.

Required rollout step — reindex both models before/immediately after deploy:

docker exec <api-container> python manage.py search_index --populate --models sources.Source concepts.Concept -f --parallel

Use your deployment's own rebuild procedure if it recreates indexes from scratch instead
(e.g. blue/green index + alias swap). Either way, do not route GraphQL traffic to this
version against an index that hasn't been repopulated. REST search availability should be
accounted for during the reindex regardless of which procedure you use.

Recommendation: treat the reindex as a pre-deploy gate, not a post-deploy cleanup step,
given that failures are silent. Consider adding an operational check (e.g. a canary GraphQL
query with a known concept, or an index field-presence check) before flipping traffic, since
totalCount: 0 alone cannot be trusted to mean "index not ready" versus "no matches."

New GraphQL API surface

query Dictionary($org: String!, $source: String!, $version: String) {
  source(org: $org, source: $source, version: $version) {
    name
    description
    canonicalUrl
    uri
    classes
    datatypes
    mapTypes
    externalSources { name url }
    summary { activeConcepts mappings }
  }
}

query FindConcepts($org: String, $source: String, $query: String!, $page: Int, $limit: Int) {
  concepts(org: $org, source: $source, query: $query, page: $page, limit: $limit) {
    totalCount
    hasNextPage
    versionResolved
    results { conceptId display description conceptClass datatype { name } }
  }
}

• conceptIds performs exact, case-sensitive mnemonic matching, deduplicates input, preserves
  order, and takes precedence over query.
• Omit both org and source for a global concept search.
• page/limit must be supplied together; the supported result window is 10,000. Without
  pagination, index responses are capped at 10,000, but totalCount still reflects the true
  total match count.
• Omitted version resolves to HEAD, falling back to the latest released version only when
  HEAD itself is absent. An explicitly requested version that doesn't exist does not fall back.
• Authentication: existing OCL token, OIDC bearer token, or session auth. Invalid credentials
  are rejected before resolvers run. Anonymous callers see public data only; authenticated
  callers still require the existing graphql_api group.

Permission model

• Repository visibility reuses the shared REST visibility rule directly (no fabricated
  requests), scoped by both owner mnemonic and owner type.
• Global concept search additionally enforces parent-repository visibility via
  parent_public_can_view.
• Mapping hydration for externalSources independently checks target-repository visibility
  and excludes the current source and any linked private targets the caller cannot view.
• HEAD scope uses the same versioned-object identity as Source.get_concepts_queryset()
  (id == versioned_object_id); released versions use their version membership lists.

Compatibility notes

• No database migrations. No new environment variables.
• Existing REST search relevance and excluded-word semantics are unchanged.
• The older, fully-hydrated text-search path (used for selections needing names, mappings,
  extras, or datatype details) keeps its existing empty-index database fallback — only the
  new lean index projection path lacks a fallback for the zero-hit case described above.

Testing

• 75 tests passing, including 6 against a real Elasticsearch instance (index preparation,
  zero-SQL projections, owner isolation, HEAD/release selection, inactive/retired filtering,
  private-parent visibility).
• 98% statement coverage on core.graphql (739/753), excluding test files.
• Pylint clean on the changed modules.

[filiperochalopes]

Still draft, but updated, please consider check @snyaggarwal

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 091bfb0377

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Reindex locale-derived fields after source locale changes

When a source's default_locale or supported_locales changes, these projections continue serving the previously indexed display_name and preferred_description, even though both values are derived from the parent source's locale configuration. Source persistence only reindexes concepts for release or match-algorithm changes, and the new signal propagation handles only activity and visibility, so a lean query requesting display or description remains stale indefinitely while the ORM path returns the new locale selection. Reindex the source's concepts when the relevant locale settings change, or avoid directly projecting these derived fields.

Useful? React with 👍 / 👎.