Re-enable Native Auth SSPR tests with Mail.tm, Fixes AB#3729032 by djanardhan98 · Pull Request #2553 · AzureAD/microsoft-authentication-library-for-android

Disha Janardhan (djanardhan98) · 2026-08-12T23:15:07Z

Summary

Re-enable the Native Auth SSPR end-to-end tests using authenticated Mail.tm inboxes.
Update Native Auth end-to-end tests to create registered inboxes when OTP retrieval is required and record checkpoints before requesting or resending an OTP.
Run the complete Native Auth end-to-end test package in PR validation with the secured Mail.tm password.
Update the Common submodule for the companion Mail.tm integration.

Motivation

Native Auth end-to-end tests need both dynamically created signup inboxes and an authenticated persistent inbox for SSPR. The Mail.tm integration in Common supports both workflows and allows the previously disabled SSPR coverage to run in CI.

Validation

All MSAL tests passed after applying these changes against Common commit ddb306a3884f9621e53b6eb3256fc14ea1497c7c.

Pipeline configuration

PR validation passes the secured EMAIL_PROVIDER_PASSWORD variable to the Native Auth end-to-end test task.
The dedicated task runs the complete Native Auth end-to-end test package, including SSPR.

Dependency and merge sequencing

Merge the Common PR first, then update this PR's Common submodule reference to the merged commit before merging.
Fixes AB#3729032

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add a dedicated PR-branch test invocation with the Native Auth and Mail.tm inputs required by SSPRTest. Use a deterministic too-short password for the invalid-format case and update Common for Mail.tm HTTP/1.1 compatibility. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Update Common to the latest mail-tm-native-auth-e2e commit and broaden the dedicated PR test filter to the complete Native Auth E2E package, including SSPR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions · 2026-08-14T05:00:10Z

✅ Work item link check complete. Description contains link AB#3720943 to an Azure Boards work item.

Co-authored-by: djanardhan98 <307255345+djanardhan98@users.noreply.github.com>

Point the Common submodule at the latest commit from AzureAD/microsoft-authentication-library-common-for-android#3219. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83947e30-5bc9-4ec7-a8ed-c03d4415da92

Copilot

Pull request overview

Migrates Native Auth E2E tests to Mail.tm and re-enables SSPR coverage. The current Common submodule API is incompatible with the shared test helper, causing compilation failure.

Changes:

Adds registered inboxes and OTP checkpoints.
Runs Native Auth E2E tests in PR validation.
Updates Common and AB ID extraction.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
`.github/workflows/validate-pr-ab-id.yml`	Extracts explicit AB IDs.
`azure-pipelines/pull-request-validation/pr-msal.yml`	Adds Native Auth E2E validation.
`common`	Updates the Common submodule.
`NativeAuthPublicClientApplicationAbstractTest.kt`	Centralizes email service and retries.
`SSPRTest.kt`	Re-enables Mail.tm-backed SSPR tests.
`SignInEmailOTPTest.kt`	Adds OTP checkpoints.
`SignInJITTest.kt`	Creates authenticated inboxes.
`SignInMFATest.kt`	Adds MFA OTP checkpoints.
`SignUpEmailPasswordTest.kt`	Migrates signup inbox handling.
`SignUpEmailPasswordAttributesTest.kt`	Migrates attribute-flow inboxes.
`SignUpEmailOTPTest.kt`	Migrates OTP signup inboxes.
`SignUpEmailOTPAttributesTest.kt`	Migrates attributed OTP signup inboxes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83947e30-5bc9-4ec7-a8ed-c03d4415da92

Restore validate-pr-ab-id.yml to the dev branch version. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f268d2a-b8c4-4a22-826e-de2d9b9f59a6

…3219) ## Summary - Replace the Native Auth 1secmail inbox integration with authenticated Mail.tm mailboxes. - Pass the Mail.tm password through the existing Gradle `BuildConfig`/`BuildValues` pattern without committing credentials. - Add checkpoint-aware OTP polling, resend handling, MIME-aware OTP extraction, and bounded retries. - Force HTTP/1.1 for compatibility with the existing OkHttp client and Mail.tm. - Remove obsolete 1secmail response models and add regression coverage for the Mail.tm transport and inbox behavior. ## Motivation Native Auth end-to-end tests need both dynamically created signup inboxes and an authenticated persistent inbox for SSPR. Mail.tm supports both workflows, while the previous unauthenticated 1secmail integration cannot support the persistent SSPR mailbox used by the re-enabled tests. ## Validation - `.\gradlew.bat :testutils:clean :testutils:assembleDebug :testutils:testDebugUnitTest --no-daemon`: passed locally. - All MSAL tests passed after applying the corresponding MSAL changes against Common commit `ddb306a3884f9621e53b6eb3256fc14ea1497c7c`. - The corresponding MSAL changes will be submitted in a separate PR. ## Pipeline configuration - The MSAL consumer-validation task passes the secured Azure DevOps variable `EMAIL_PROVIDER_PASSWORD` from the pipeline's linked variable groups to Common as `-PemailProviderPassword=$(EMAIL_PROVIDER_PASSWORD)`; Common exposes it through generated `BuildConfig` and `BuildValues` without committing the secret. ## Related work - Fixes [AB#3720943](https://identitydivision.visualstudio.com/fac9d424-53d2-45c0-91b5-ef6ba7a6bf26/_workitems/edit/3720943) - AzureAD/microsoft-authentication-library-for-android#2553 ## Merge sequencing This PR is the expand step of a cross-repository migration: it temporarily retains `generateRandomEmailAddressLocally()` so both `dev` branches remain buildable. After AzureAD/microsoft-authentication-library-for-android#2553 migrates callers to `createRandomEmailAddress()` or `generateRandomUnregisteredEmailAddress()` and merges, the compatibility method will be removed in a Common follow-up. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Copilot-Session: 83947e30-5bc9-4ec7-a8ed-c03d4415da92

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Resolve the Common submodule pointer in favor of the current Common dev commit containing the merged Mail.tm integration. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Shahzaib (shahzaibj) · 2026-08-24T18:00:02Z

+      - task: Gradle@2
+        displayName: Run Native Auth E2E Tests
+        inputs:
+          tasks: msal:testLocalDebugUnitTest -Plabtest -ProbolectricSdkVersion=${{variables.robolectricSdkVersion}} -PnativeAuthConfigString=$(NATIVE_AUTH_CONFIG_STRING) --tests com.microsoft.identity.client.e2e.tests.network.nativeauth.*


Are we gating every PR on these now? They hit mail.tm + a live tenant, and these are the same tests that were iGnOrE (@ignore)'d for OTP flakiness. I feel like unrelated PRs will start failing validation whenever mail.tm has a hiccup or the tenant throttles OTPs. Could these run on a nightly/scheduled pipeline instead of the per-PR gate?

Shahzaib (shahzaibj) · 2026-08-24T18:00:02Z

+          testResultsFiles: '**/build/test-results/testLocalDebugUnitTest/TEST-*.xml'
+          testRunTitle: Native Auth E2E Tests (PR Branch)
+        env:
+          EMAIL_PROVIDER_PASSWORD: $(EMAIL_PROVIDER_PASSWORD)


This drops EMAIL_PROVIDER_PASSWORD into the env of a task that runs the PR branch's own code (checkout: self), and these tests already make outbound http calls. So anyone who can push to a PR branch could read it with a getenv and ship it out to themselves. It's a throwaway mail.tm password so low stakes, but on a public repo are we sure fork/external PRs don't get secrets here?

Shahzaib (shahzaibj) · 2026-08-24T18:00:02Z

-                    retryCount++
+                return
+            } catch (e: AssertionError) {
+                if (!e.message.orEmpty().contains("AADSTS701014")) {


What's AADSTS701014 here? We only retry AssertionErrors whose message contains it and rethrow everything else. Is it a transient eSTS thing that's actually worth riding out with the backoff? Matching the code as a substring of the assert message also feels a bit brittle since that text isn't really a stable contract.

Shahzaib (shahzaibj) · 2026-08-24T18:00:02Z

+        }
+
+        // Avoid repeatedly requesting OTPs while the Native Auth test tenant is throttling them.
+        Thread.sleep(5_000L * (1L shl retryCount))


maxRetries=5 with this backoff means a persistently throttled test sleeps 5+10+20+40+80 = ~155s before it finally fails, and every retry re-runs the whole flow (new inbox + signup + OTP request). Across the suite on a throttling day that adds up fast. Do we really need 5 retries here?

Re-enable Native Auth SSPR tests with Mail.tm

a4b718a

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions Bot added the msal label Aug 12, 2026

Disha Janardhan (djanardhan98) and others added 4 commits August 12, 2026 16:20

Update Common after dev rebase

4ad73aa

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Document Native Auth E2E CI design

e9d6059

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Correct Native Auth E2E task design

dd6b5ce

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions Bot added the docs label Aug 13, 2026

Disha Janardhan (djanardhan98) mentioned this pull request Aug 13, 2026

Migrate Native Auth test email provider to Mail.tm, Fixes AB#3720943 AzureAD/microsoft-authentication-library-common-for-android#3215

Closed

Disha Janardhan (djanardhan98) and others added 2 commits August 13, 2026 21:24

Delete docs/superpowers/specs/2026-08-12-native-auth-e2e-ci-design.md

a6aba48

Run all Native Auth E2E tests in PR validation

a90fc17

Update Common to the latest mail-tm-native-auth-e2e commit and broaden the dedicated PR test filter to the complete Native Auth E2E package, including SSPR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

github-actions Bot changed the title ~~Re-enable Native Auth SSPR tests with Mail.tm~~ Re-enable Native Auth SSPR tests with Mail.tm, Fixes AB#3720943 Aug 14, 2026

Update pr-msal.yml

f56e844

Disha Janardhan (djanardhan98) mentioned this pull request Aug 14, 2026

Migrate Native Auth test email provider to Mail.tm, Fixes AB#3720943 AzureAD/microsoft-authentication-library-common-for-android#3219

Merged

Azure Active Directory (AzureAD) deleted a comment from github-actions Bot Aug 14, 2026

Disha Janardhan (djanardhan98) changed the title ~~Re-enable Native Auth SSPR tests with Mail.tm, Fixes AB#3720943~~ Re-enable Native Auth SSPR tests with Mail.tm, Fixes AB#3729032 Aug 14, 2026

Copilot started work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 15:58 View session

Copilot AI and others added 3 commits August 14, 2026 15:59

Fix AB ID extraction in PR workflow

e94d321

Co-authored-by: djanardhan98 <307255345+djanardhan98@users.noreply.github.com>

Restore retries for SSPR E2E tests

3c4a8e3

Co-authored-by: djanardhan98 <307255345+djanardhan98@users.noreply.github.com>

Pass email password through pipeline env

e1124b5

Co-authored-by: djanardhan98 <307255345+djanardhan98@users.noreply.github.com>

Copilot finished work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 16:05

Copilot started work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 16:06 View session

Copilot finished work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 16:07

Copilot started work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 16:31 View session

Restore email password Gradle property

97dcc4b

Co-authored-by: djanardhan98 <307255345+djanardhan98@users.noreply.github.com>

Copilot finished work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 16:33

Copilot started work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 17:00 View session

Skip rate-limited SSPR OTP E2E scenarios

f259295

Co-authored-by: djanardhan98 <307255345+djanardhan98@users.noreply.github.com>

Copilot finished work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 17:04

Copilot started work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 17:04 View session

Copilot finished work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 17:05

Copilot started work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 17:06 View session

Copilot AI and others added 2 commits August 14, 2026 17:07

Retry rate-limited Native Auth E2E tests

97f1df6

Co-authored-by: djanardhan98 <307255345+djanardhan98@users.noreply.github.com>

Preserve Native Auth retry failure cause

41b12b8

Co-authored-by: djanardhan98 <307255345+djanardhan98@users.noreply.github.com>

Copilot finished work on behalf of Disha Janardhan (djanardhan98) August 14, 2026 17:10

Disha Janardhan (djanardhan98) and others added 2 commits August 14, 2026 13:51

Update Common Mail.tm integration

9686b50

Point the Common submodule at the latest commit from AzureAD/microsoft-authentication-library-common-for-android#3219. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Update Common runtime email credential integration

e356e84

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83947e30-5bc9-4ec7-a8ed-c03d4415da92

Disha Janardhan (djanardhan98) requested a balanced review from Copilot August 17, 2026 20:03

Copilot started reviewing on behalf of Disha Janardhan (djanardhan98) August 17, 2026 20:04 View session

Copilot AI reviewed Aug 17, 2026

View reviewed changes

Comment thread ...dentity/client/e2e/tests/network/nativeauth/NativeAuthPublicClientApplicationAbstractTest.kt Outdated

Fix Native Auth email service initialization

a599345

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83947e30-5bc9-4ec7-a8ed-c03d4415da92

Disha Janardhan (djanardhan98) marked this pull request as ready for review August 18, 2026 05:44

Disha Janardhan (djanardhan98) requested review from a team as code owners August 18, 2026 05:44

Revert PR AB ID workflow change

4bd47e5

Restore validate-pr-ab-id.yml to the dev branch version. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4f268d2a-b8c4-4a22-826e-de2d9b9f59a6

Silviu Petrescu (spetrescu84) approved these changes Aug 21, 2026

View reviewed changes

Disha Janardhan (djanardhan98) and others added 2 commits August 24, 2026 09:40

Update Common to latest dev

62183f7

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Merge dev into Native Auth SSPR changes

6bbd3b1

Resolve the Common submodule pointer in favor of the current Common dev commit containing the merged Mail.tm integration. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Disha Janardhan (djanardhan98) requested review from Sergei Demchenko (antrix1989) and fadidurah August 24, 2026 16:58

Shahzaib (shahzaibj) reviewed Aug 25, 2026

View reviewed changes

Navigation Menu

Uh oh!

Re-enable Native Auth SSPR tests with Mail.tm, Fixes AB#3729032 - #2553

Re-enable Native Auth SSPR tests with Mail.tm, Fixes AB#3729032#2553
Disha Janardhan (djanardhan98) wants to merge 21 commits into
devfrom
djanardhan/mail-tm-native-auth-e2e

Disha Janardhan (djanardhan98) commented Aug 12, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 14, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Shahzaib (shahzaibj) Aug 24, 2026

Uh oh!

Shahzaib (shahzaibj) Aug 24, 2026

Uh oh!

Shahzaib (shahzaibj) Aug 24, 2026

Uh oh!

Shahzaib (shahzaibj) Aug 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

Disha Janardhan (djanardhan98) commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Validation

Pipeline configuration

Dependency and merge sequencing

Uh oh!

github-actions Bot commented Aug 14, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Shahzaib (shahzaibj) Aug 24, 2026

Choose a reason for hiding this comment

Uh oh!

Shahzaib (shahzaibj) Aug 24, 2026

Choose a reason for hiding this comment

Uh oh!

Shahzaib (shahzaibj) Aug 24, 2026

Choose a reason for hiding this comment

Uh oh!

Shahzaib (shahzaibj) Aug 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Disha Janardhan (djanardhan98) commented Aug 12, 2026 •

edited

Loading