Feat/provider platform rebased

[gtonye]

Pre-Submission PR Checklist

• No debug/console/fmt.Println statements
• Unnecessary development comments removed
• All acceptance criteria verified
• Functions according to ticket specifications
• Tested manually where applicable
• Branch rebased with latest main
• No business logic exists within the database layer

Description of the change

This PR introduces full Canvas provider platform integration — surfacing Canvas courses as programs, merging Canvas calendar events into the admin schedule view, and providing facility admins with tooling to link (map) residents to their Canvas accounts.

Canvas Programs via live API lookup

When the Programs overview table is fetched, the backend performs a live lookup against every enabled Canvas provider platform and synthesizes a ProgramsOverviewTable entry per provider. These entries appear alongside native UnlockEd programs but carry source: "canvas" and a stable synthetic ID (CanvasProgramIDOffset + providerID). The facility admin program view shows a read-only banner for Canvas programs and suppresses edit controls accordingly.

sequenceDiagram
    participant Client
    participant API as GET /api/programs
    participant NATS as NATS KV (canvas_programs)
    participant Canvas as Canvas API

    Client->>API: GET /api/programs
    API->>API: query DB for native programs
    loop each enabled Canvas provider
        API->>NATS: get cache key canvas_program_{id}
        alt cache hit (< 5 min old)
            NATS-->>API: cached ProgramsOverviewTable
        else cache miss / stale
            API->>Canvas: GET /accounts/{id}/courses?per_page=100
            Canvas-->>API: course list
            API->>NATS: put canvas_program_{id} (TTL 5 min)
            NATS-->>API: ok
        end
    end
    API-->>Client: native programs + Canvas programs merged

Loading

Results are persisted in a dedicated canvas_programs NATS KV bucket with a 5-minute TTL. On a cache miss the handler calls Canvas directly, then writes the result back. Individual program routes (/programs/:id, archive-check, history) are guarded by the same CanvasProgramIDOffset sentinel and return safe no-op responses for Canvas IDs, so the frontend does not need special-casing beyond the read-only banner.

Canvas User Matching

Facility admins must link residents before Canvas enrollment data appears on resident profiles. Canvas users and UnlockEd residents are matched automatically using a composite name-similarity score (Jaro-Winkler 60% + Smith-Waterman-Gotoh 40%):

• Auto-confirmed (score ≥ 0.90): mappings are applied without admin review.
• Ambiguous (0.50–0.89): surfaced in the Provider User Management UI for admin confirmation.
• Unmatched (< 0.50): the admin can manually select an existing resident or create a new one.

Until mappings exist, Canvas enrollment counts reflect 0 mapped users.

Canvas calendar events on admin schedule

Canvas calendar events are fetched (paginated via the Link header) and merged into the admin calendar. Canvas events render in blue (blue-700) and show a "Canvas" badge on the event tile. Clicking a Canvas event opens a CanvasEventSheet (separate from the existing SessionDetailSheet used for native sessions).

Other changes

• enrollment_types field added to ProviderPlatform (new migration, []string serialized as JSON).
• OIDC client recreation endpoint added for Canvas OIDC re-registration flows.
• Access key is now whitespace-trimmed after decryption (AfterFind hook) to avoid auth failures from keys pasted with trailing newlines.
• New ProviderPlatformManagement and ProviderPlatformDetail admin pages wired into sidebar navigation under a ProviderAccess feature flag.

Screenshot(s)

Screenshots of the different screens are available in this document.

Additional context

Local setup

To test locally, you will need a running Canvas instance.

You can use the Docker-based one:

1. Clone the repository: https://github.com/instructure/canvas-lms
2. Install Docker and Docker Compose if not already installed.
3. Install Ruby if not already installed.
4. Build the application and start it with Docker Compose. Note: Because UnlockEd also uses a Postgres instance, configure the Canvas Docker Postgres to bind to a dedicated port to avoid conflicts.

Technical notes

• Canvas program IDs use an offset (100_000_000) to avoid collisions with native program IDs. All routes that accept a program ID check this sentinel to route to Canvas-specific handlers. Reviewers should verify this boundary is consistent across the frontend (isCanvasProgram = id >= 100_000_000) and backend (CanvasProgramIDOffset).
• Facility admins must link residents before Canvas data is visible — this is not automated on first sync. The matching UI is the primary workflow for this.
• The greedy bipartite 1:1 matching algorithm ensures a Canvas user and an UnlockEd resident can only appear in one confirmed match pair, preventing duplicate mappings.
• Known gap: Canvas course pagination is currently capped at 100 per page. Providers with more than 100 courses will have all pages fetched, but the first live call may be slow before the cache is warm.

Considerations for Future Maintenance

High-Level Architecture and Philosophy

The Provider system uses an abstraction layer that separates provider logic from the main UnlockEd codebase.

This comes with tradeoffs to consider:

1. It assumes that future providers share a common set of behaviors.
2. New providers must inherit from the base provider class.

This may require drilling into provider APIs and data structures to map them against UnlockEd data models. We have done that mapping work here: https://docs.google.com/spreadsheets/d/15BZiP_TyY-Cg2LqoM22GkLnEK3lU_2UobL64EzukgyM/edit?usp=sharing

Program Concept

Canvas is course-based, so building a program dynamically will always involve latency, as the Canvas API is designed to be consumed per course.

We have implemented parallel calls combined with caching to mitigate this, but a Canvas instance with a large number of courses will still present challenges.

Persisting the data would help, but any persistence strategy needs to account for keeping data in sync with Canvas. The provider code includes logic for background jobs to support this.

Timezone

We have noticed some discrepancies in time handling. While storing times in UTC is generally good practice, we have seen in other projects that storing the timezone alongside the UTC timestamp can help with display reconciliation on the frontend.

We have also observed that frontend code sometimes uses UTC without applying the user's timezone (see Program Overview and the class header).

For Canvas, we propagate the timezone from the course all the way to the frontend to ensure consistent display between Canvas and UnlockEd.

One suggestion would be to display the timezone to the user to avoid confusion — this is especially worthwhile if users spanning multiple timezones is a realistic scenario.

[carddev81]

@gtonye expected date of enrollment to be a real date versus 0001-01-01. See pic below:

[gtonye]

It is ready for a re-review.

Below is the status of all the items:

1. Inconsistent time issues still exist between pages. ✅
2. Class roster shows 4 residents when only 1 is enrolled. (dept admin) ✅
3. Canvas username shows an em dash instead of the name once linked (should be Sam Student / Sara Student). ✅
4. Canvas program ('Missouri College Canvas Program') doesn't appear in the program filter dropdown — open question on whether it should. ⚠️
5. Create in the Unmatched section skips the expected resident-info modal and can't be undone — differs from what we discussed on May 29. Created residents also remain selectable. ✅
6. Enrollment date shows 0001-01-01 instead of a real date. ✅

On

Canvas program ('Missouri College Canvas Program') doesn't appear in the program filter dropdown — open question on whether it should.

I was unable to repro — I do see my instance in the dropdown (I was signed in as a department admin for my test). Can you try again with the latest changes?

Change Log

• Updated user mapping page to allow single user linking or creation for all residents
• Added dynamic Canvas user information retrieval when reviewing linked users
• Fixed issue with Canvas retrieved classes not using the class timezone on display
• Added support for dynamic facility / course pairs

[carddev81]

@gtonye a part of listed item 5 is a reoccurring issue

Create in the Unmatched section skips the expected resident-info modal and can't be undone — differs from what we discussed on May 29. Created residents also remain selectable. ✅

1. I am having issues with the Linked residents section of the Learning Platform Users screen. Note that in the pic below is a resident (StudentSteve) listed within the Linked residents section. From the Needs review section when I click Link existing next to one of the residents here, StudentSteve is available to choose from in this list. Is this intended behavior?

2. Unable to cancel or clear out the 'Confirm' selections I made from the Needs review section. I only have the option of Apply # match(es). See video below.

2026-06-16.14-42-57.mp4

3. Noticed that all available canvas users (teachers, TAs, students, ect.) are being displayed rather than just students. Knowing this I revisited the SQL migration for this PR. Makes me wonder if the column named enrollment_types within the table public.provider_platforms is used or needed? In a previous PR this value was needed, but now it seems like this column value is not being utilized or accessed and the value is currently hardcoded as 'student'. The enrollment_type when querying Canvas API is returning all types of users even though we request just students. See pic below:

From CANVAS

If you feel that this column will not be needed in the future then we should be able to remove the following migration script along with any related code.

00071_add_enrollment_types_to_provider_platforms.sql

4. While running tests against item Class roster shows 4 residents when only 1 is enrolled. (dept admin) ✅ I stumbled across data inconsistencies related to Total Enrollments across programs and Classes. I created a video to display the inconsistencies. This occurs when I Unlink and then Link existing residents. The old records will be used in the counting of enrollments and causes false positives. I will post the video shortly. In the video you should be able to see that the enrollment count is off, it should have stayed the same. The deleted (Unlinked) users are being counted along with the new unlinked users.

2026-06-16.15-53-07.mp4

@gtonye The number within the Enrollment column on the Programs listing page does not match the number of enrollments within the metrics card on the Statewide Program View. Also, to be consistent the classes column value on the Programs listing page does not match the metrics card on the Statewide Program View. These should match exactly? see pics below for example, but you should be able to see the issues I was running into in the video above.

Also, for facility admins, numbers tend to be off for number of enrollments, please see facility admin video below:

2026-06-16.16-17-15.mp4

5. Note: the classes page is loading really slow when I have more than 10 facilities in the system? This can be seen in the 1st video in number 4 above.

[gtonye]

Thank you @carddev81 for taking the time to review in detail.

The recent changes should address all the elements in the feedback.

1. A resident already in Linked residents (StudentSteve) is still selectable under 'Link existing' from Needs review — intended?

Residents awaiting confirmation of their match are now filtered out of the available residents for mapping.

2. Can't cancel/clear 'Confirm' selections in Needs review — only option is 'Apply # match(es)'.

Added a new cancel option for confirmed matches.

3. All Canvas user types are showing instead of just students. Looks like the enrollment_types column on provider_platforms is now hardcoded to 'student' and unused — if it's no longer needed, we could drop migration 00071_add_enrollment_types_to_provider_platforms.sql and related code? What is your recommendation?

It was indeed no longer used, so I removed it.

4. Enrollment counts are inconsistent: Unlink + Link existing keeps counting deleted records (false positives), the Programs listing Enrollment/Classes columns don't match the Statewide Program View metric cards, and facility admin numbers are off (fac admin numbers off could be because of false positives).

The program overview page was displaying the number of residents enrolled in the program. This now represents the total number of enrollments across all classes. There was also a missing facility ID scope, which explains the additional discrepancy for facility admins.

5. New Classes page loads slowly with more than 10 facilities (see first video in #4).

This one is harder to fix. Canvas has no concept of a "Facility" — that grouping exists only in UnlockEd. To compute per-facility enrollment counts, the code must fetch every course from Canvas, fetch every active enrollment per course, look up which of those Canvas users are mapped to local residents, and then group those residents by facility. With N courses, this means N Canvas API calls on every page load, with no shortcut available — Canvas provides no bulk enrollment endpoint and no way to query enrollments by resident or group.

flowchart TD
    A([Page load request]) --> B

    B["<b>GET</b> /api/v1/accounts/{id}/courses\nOne call — returns all courses as a list"]

    B --> C

    subgraph loop ["Per-course loop — repeats N times (once per course)"]
        C["<b>GET</b> /courses/{id}/enrollments\nReturns Canvas user IDs of active students"]
        C --> D["<b>DB:</b> provider_user_mappings\nMatch Canvas IDs → local residents"]
    end

    D --> E["Group residents by facility\n(facility concept exists only in UnlockEd)"]
    E --> F([Per-facility enrollment counts\nOne row per facility × course])

    style B fill:#CECBF6,stroke:#534AB7,color:#26215C
    style C fill:#F5C4B3,stroke:#993C1D,color:#4A1B0C
    style D fill:#9FE1CB,stroke:#0F6E56,color:#04342C
    style E fill:#9FE1CB,stroke:#0F6E56,color:#04342C
    style loop fill:#F1EFE8,stroke:#888780,color:#2C2C2A

Loading

The underlying limitation is that the Canvas API does not provide a single endpoint to retrieve enrollments across all courses, nor a way to query which courses a given resident is enrolled in. As a result, the page performance scales linearly with the number of courses rather than the number of facilities.

To mitigate this, we've added a cache layer so the computation only runs on the first load and is served from cache on subsequent requests. The tradeoff is that enrollment data may be slightly delayed when changes are made in Canvas — the cache will reflect the previous state until it expires.

[carddev81]

@gtonye

1. A few issues still exist as part of listed item 4 from last review...counts are still not reflective of the actual number of enrollments. See pics below, the total number of enrollments should match the number of currently enrolled. The total enrollments includes historical enrollments if any. Also notice the negative number on the dept admin view. See video, you should be able to spot that the deleted records are being counted as enrollments when signed in as dept. admin. I also screenshot the result set of the provider_user_mappings table to display the deleted_at column.

2026-06-17.10-20-07.mp4

2. Recieving error on initial load of classes page for facility/dept. admins. After mapping residents I immediately went to the Programs > Classes page. See video.

2026-06-17.10-13-47.mp4

3. Didn't catch this on the last reviews, but Import All Users is giving a false positive, when clicked it displays the message 'Users imported successfully. Please check for accounts not created.' No users were actually imported and the underlying API call fails.

4. Teachers and TAs are still being pulled from the canvas API, is there no way to filter these out?. I noticed in the code the call to attempt to pull only students but this does not seem to work.

[gtonye]

@carddev81

This should be ready for re-review. The recent changes include fixes for:

• Inconsistencies in enrollment calculation: The Programs list page, program details page, and class page are now consistent.
• Enrollment calculation excludes deleted provider user mappings: The calculation no longer includes deleted provider user mappings.
• Error on page load for facility and department admins: After linking a user, the program list page would often throw an error — this has now been resolved.
• "Import All Users" has been removed: This feature had become redundant and has been removed.

Note: As discussed offline, there is an inconsistency across Canvas instances where certain API versions appear to handle the enrollment_type parameter differently. This would need further investigation, potentially by looking at open issues on the Canvas LMS repository to see if it has been flagged.

[carddev81]

@gtonye I was able to duplicate error page consistently. The error occurs the very first time the cache is loaded from the programs page. I captured the logging output for you as well as the devtools. Check out video below:

Here are the steps to reproduce on your machine

1. Unlink all users
2. Log out of unlocked
3. Stop all docker containers or processes
4. Start server back up
5. Link users
6. Go to programs page and click a program
7. See error.

2026-06-18.10-03-16.mp4

INFO[0189] /app/backend/src/database/provider_user_mappings.go:20 record not found
[0.249ms] [rows:0] SELECT * FROM "provider_user_mappings" WHERE (user_id = 208 AND provider_platform_id = 8) AND "provider_user_mappings"."deleted_at" IS NULL ORDER BY "provider_user_mappings"."id" LIMIT 1 
INFO[0189] /app/backend/src/database/provider_user_mappings.go:20 record not found
[0.239ms] [rows:0] SELECT * FROM "provider_user_mappings" WHERE (user_id = 209 AND provider_platform_id = 8) AND "provider_user_mappings"."deleted_at" IS NULL ORDER BY "provider_user_mappings"."id" LIMIT 1 
INFO[0189] /app/backend/src/database/provider_user_mappings.go:20 record not found
[0.154ms] [rows:0] SELECT * FROM "provider_user_mappings" WHERE (user_id = 210 AND provider_platform_id = 8) AND "provider_user_mappings"."deleted_at" IS NULL ORDER BY "provider_user_mappings"."id" LIMIT 1 
INFO[0189] /app/backend/src/database/provider_user_mappings.go:20 record not found
[0.224ms] [rows:0] SELECT * FROM "provider_user_mappings" WHERE (user_id = 207 AND provider_platform_id = 8) AND "provider_user_mappings"."deleted_at" IS NULL ORDER BY "provider_user_mappings"."id" LIMIT 1 
{"admin_id":33,"audit":"admin_action","facility_id":1,"facility_name":"Palmer Correctional Center","handler":"handleApplyMatches","ip_address":"172.18.0.14:35446","level":"info","method":"POST","msg":"","path":"/api/actions/provider-platforms/8/apply-matches","role":"department_admin","session_id":"0a660bd4-60a6-4a04-82c4-ad2a064c2d1a","time":"2026-06-18T15:03:31Z","username":"rich.salas"}
{"level":"info","msg":"Init request for provider service","time":"2026-06-18T15:03:31Z"}
{"level":"info","msg":"request: \u0026{GET http://provider-service:8081/api/users/all?id=8 HTTP/1.1 1 1 map[] \u003cnil\u003e \u003cnil\u003e 0 [] false provider-service:8081 map[] map[] \u003cnil\u003e map[]   \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e  {{}} \u003cnil\u003e [] map[]}","time":"2026-06-18T15:03:31Z"}
{"level":"info","msg":"url: http://provider-service:8081/api/users/all?id=8 \n","time":"2026-06-18T15:03:31Z"}
{"level":"info","msg":"Init request for provider service","time":"2026-06-18T15:03:31Z"}
{"level":"info","msg":"url: http://provider-service:8081/api/users?id=8 \n","time":"2026-06-18T15:03:31Z"}
{"level":"info","msg":"request: \u0026{GET http://provider-service:8081/api/users?id=8 HTTP/1.1 1 1 map[] \u003cnil\u003e \u003cnil\u003e 0 [] false provider-service:8081 map[] map[] \u003cnil\u003e map[]   \u003cnil\u003e \u003cnil\u003e \u003cnil\u003e  {{}} \u003cnil\u003e [] map[]}","time":"2026-06-18T15:03:31Z"}
{"level":"debug","msg":"users received from middleware: [{StudentSidney Sidney Student student5@example.com 9 student5@example.com} {StudentSimon Simon Student student6@example.com 10 student6@example.com} {TATina Tina TA ta@example.com 4 ta@example.com} {TeacherTerry Terry Teacher teacher@example.com 2 teacher@example.com} {TeacherTom Tom Teacher teacher2@example.com 3 teacher2@example.com}]","time":"2026-06-18T15:03:42Z"}

[gtonye]

@carddev81

I'm trying another approach. I suspect the Canvas API is slow, especially on that initial call.

Can you test again with the most recent change and let me know if it's still causing the error?

Technical Notes

We've moved all program computation inside a goroutine triggered by a cache miss. This is due to Canvas being slow and program computation requiring iteration over all courses to calculate enrollment counts.
We've added a visual signal on the frontend to indicate to the user that the particular program is loading asynchronously.

canvas-integration-async-demo.mov

[carddev81]

Looks and works great.