Update .env.ci and .env.testing APP_ENV

[allella]

Per conversation in #726 we had support for forcing PHPUnit to use testing.

The Github action that runs our tests is copying .env.ci and that's set to 'local', which appears to preempt the phpunit.xml value.

[allella]

@bogdankharchenko here's the original concern. I've copied the comment below since the link jumping seems to be doing goofy things.

I'm pretty sure the only reason I messed with it to begin with is I noticed it was failing on the GH action saying the 'local' environment didn't match its expectation of 'testing'.

The action runs cp .env.ci .env. I'm guessing that's preempting what's set for the phpunit.xml APP_ENV ?

Should we set phpunit.xml to force the environment to testing?

Our .env.ci does declare the 'local' environment and the workflow file is running cp .env.ci .env. That seems to be causing Github to run tests as 'local' instead of 'testing'

If I recall, the test_robots_txt_disallows_all_in_non_production_test was failing on Github saying that 'local' didn't match 'testing'.

Before we do anything else, it's probably worth verifying if that's the case by emitting the environment variable as part of the testing workflow to find out if Github is running tests in 'local' (via to the .env.ci copying) or 'testing' (via phpunit.xml).

Note that I changed test_robots_txt_disallows_all_in_non_production_test in #726 so it checks for 'not production' instead of specifically for 'testing'. That change allowed for running artistan test to pass locally and it makes sense because robots.txt should be Disallow in all environments besides production.

Here's test_robots_txt_disallows_all_in_non_production_test as it is now. This test would pass regardless of if Github is running in 'testing' or 'local', but that's by design as we expect to see Disallow in any env that's not production.

[oliviasculley]

I'm not sure about this change, to me needing to add something that says force="true" seems hacky but I am not as up to date with the PHP ecosystem

[allella]

@oliviasculley @bogdankharchenko the main question is if we are intentionally importing .env.ci into the workflow.

It declares local, which seems odd to me. I would have assumed 'testing'

If that's the fix, then I agree that we don't need this PR, but rather a PR to change .env.ci to testing.

[oliviasculley]

I just read a little more and I think we should definitely be modifying the .env.ci and/or the workflow to accomplish this just for the CI environment rather than the phpunit.xml which would affect all environments

[allella]

I'm pulling @irby back in as the .env.ci was created by Matt with 'local' in the original file.

I feel like he thought the phpunit.xml change was okay, but it could have been that I should have suggested a better solution, like could we "fix" the .env.ci, or should we change the Github workflows to use the .env.testing.

The .env.ci and .env.testing both use
APP_ENV=local

I'm not a testing guru, so I'm not even sure if/why we need both and if we could simply change .env.ci or .env.testing to APP_ENV=testing.

If that's the easy fix, then we'd also want to compare .env.testing and .env.ci because the former is probably out of date.

Side note, since I looked it up.

There is a lot of overriding that can happen (summary below), so I agree the PHPUnit thing feels sort of ick if we can fix it at another level.

1. GitHub Actions Environment Context: Variables defined directly in your .github/workflows/*.yml file (via the env: key at the step, job, or workflow level) take ultimate priority.
2. PHP CLI Execution: Variables passed as inline arguments right before the execution command in GitHub Actions (e.g., APP_ENV=testing php artisan test).
3. The Block in phpunit.xml: Any variables explicitly set within or tags in your phpunit.xml file.
4. Specific Environment Files: Laravel loads the corresponding environment file based on the context (e.g., .env.testing if it exists).
5. The Base .env File: The fallback for any variables not defined by the above methods.

[bogdankharchenko]

lets update .env.ci to be testing instead.

[allella]

@bogdankharchenko I'll do that. What about .env.testing? It's also local. Do we want / need both files?

[allella]

Does anybody know when we're using .env.testing?

[oliviasculley]

I assume in the Github CI which is still passing
Edit: Yep not correct lol

[allella]

@oliviasculley the confusion I'm having is that we are importing .env.ci in the main workflow.

The .env.testing has diverged a bit from .env.ci and it's still unclear if Github is using .env.testing or if that's an artifact. It looks like @zach2825 created .env.testing in Feb 2021 and this file appears to be a Laravel convention.

@irby created .env.ci in July 2023, presumably as a Github naming convention.

If both have a purpose, then we could document that in CONTRIBUTING.md, which currently just says

Maybe .env.ci for Github and .env.testing for Laravel php artistan test --env=testing runs? It's clear when the .env.ci is being used, but not where in the current Labs flow where .env.testing could be used.

[irby]

Hey @allella, so there's actually a bug in our current GH Actions setup that's worth understanding before we make any changes.

The .env.ci file is intended to have our code run against the MySQL container instance running in the GH Actions job. .env.ci is copied over to .env before running any command. When php artisan migrate --seed is run, it will run migrations against the MySQL instance as intended. However, when php artisan test runs, Laravel internally forces APP_ENV=testing and because .env.testing exists, it loads that file instead of .env. What .env.testing does is it tells Artisan "hey, run our tests against an in-memory SQLite instance". So, currently our unit tests are not actually running against MySQL.

If we set APP_ENV=testing in the .env.ci file, this will cause our Artisan commands to use the .env.testing file when running the php artisan migrate command, making our MySQL instance entirely dead weight in our GH Actions job.

So we have two options here:

Option 1: Run tests against MySQL
Keep the MySQL service container and run php artisan test --env=ci. This forces tests to use .env.ci (and therefore MySQL) instead of .env.testing. The migrate step already works correctly as-is.

• Pro: Better assurance our business logic and migrations work against the real database engine we use in production.
• Con: Slower test execution than SQLite in-memory.

Option 2: Drop MySQL from CI entirely
Remove the MySQL service container from the GitHub Actions workflow and run php artisan migrate --seed --env=testing so the migration step also targets SQLite. Tests continue as they do today, but now intentionally.

• Pro: Faster, simpler CI with no external service dependency.
• Con: SQLite and MySQL differ in meaningful ways — nullable handling, strict mode, JSON columns, certain index constraints. Migrations or queries that pass on SQLite can fail on MySQL in production. This is a non-trivial risk depending on schema complexity.

My recommendation is Option 1. We already pay the cost of running MySQL in CI; we should get the benefit of actually testing against it. @bogdankharchenko what are your thoughts?

[oliviasculley]

I'm also in favor of running MySQL in ci, we want to test with MySQL quirks since we're using that in production

[oliviasculley]

I decided to double check this with Opus 4.8 as well

Nice writeup @irby — this matches what I'd found digging into it independently, and I verified the moving parts locally so we can decide with confidence. A couple of confirmations and one correction:

Confirmed: php artisan test --env=ci does run tests against MySQL. I probed config('database.default') under that command and it resolves to mysql — phpunit.xml's hard-coded sqlite does not win once .env.ci is loaded via --env. So Option 1's command works as you describe.

One correction on the migrate step: setting APP_ENV=testing in .env.ci does not make php artisan migrate --seed fall back to .env.testing/SQLite. I tested it (cp .env.ci .env with APP_ENV=testing, then a plain artisan command): it resolves to APP_ENV=testing, DB=mysql, and .env.testing is never loaded. The asymmetry is that .env.testing only gets pulled in when APP_ENV is already set in the environment before boot — which the php artisan test subprocess does (it exports APP_ENV=testing), but a plain php artisan migrate does not. So MySQL doesn't become dead weight; the migrate step keeps hitting MySQL as intended.

I'm with you on Option 1. We're already paying for the MySQL container, and SQLite hides exactly the things that bite in production (strict mode, JSON columns, FK/constraint behavior, DDL differences). Testing against the engine we actually deploy is worth the few extra seconds.

Concrete changes for Option 1:

1. Set APP_ENV=testing in .env.ci (so CI tests run as testing, not local — verified this yields testing + MySQL together).
2. Change the workflow test step to php artisan test --env=ci.
3. Keep .env.testing as-is. It's what makes a local php artisan test use SQLite :memory: — fast for TDD, and it stops a dev from running RefreshDatabase against their real local database. So the end state is deliberate: CI tests on MySQL, local tests on SQLite. Worth a line in CONTRIBUTING.md so nobody re-litigates this later.

The one tradeoff to call out: a dev could get green tests locally (SQLite) that fail in CI (MySQL). That's acceptable — CI is the gate and the failure is the signal — but it's the reason to keep .env.ci and .env.testing otherwise in sync, so the DB engine is the only intended difference between them.

Also I'd like to see all of this documented in the readme/docs somewhere

[allella]

@oliviasculley @irby

The .env files and docs have been tweaked based on the above suggestions.

Thanks, and please give this PR a fresh review when you have a moment.

[oliviasculley]

Looks great!

[allella]

Welp, the latest tests from the homepage rework started failing as soon as this was merged.

I opened #750 with more details.