Tests to staging - #71
Merged
Merged
Conversation
Add support for an environment override variable 'EDI_TEST_ENV' to dynamically redirect production API and portal queries to staging during CI runs.
Motivation:
When running integration tests with live requests (e.g. in GitHub Actions workflows), we have a mix of tests targeting staging (write/mutation operations) and production (default read-only operations). However, the configured 'EDI_API_KEY' secret is environment-specific. When using a staging key, any default production-bound tests failed authentication; conversely, using a production key caused staging-bound tests to fail. Redirecting all production-bound queries to staging under CI ensures that all live requests authenticate successfully with a single staging 'EDI_API_KEY'.
Changes:
* R/utilities.R: Update 'base_url()' and 'base_url_portal()' to check for 'EDI_TEST_ENV' and rewrite 'production' requests to the specified override environment, provided a VCR cassette is not actively replaying ('!is_vcr_replaying()').
* tests/testthat/test_utilities.R: Isolate 'base_url()' and 'base_url_portal()' raw URL mapping unit tests from the override by temporarily unsetting 'EDI_TEST_ENV' during execution.
* .github/workflows/check-standard.yaml:
* .github/workflows/check-standard-real-requests.yaml: Add 'EDI_TEST_ENV: staging' to the job env blocks.
Migrate all unit tests and mock VCR cassette fixtures to target the PASTA staging environment (https://pasta-s.lternet.edu). Previously, tests hit a mix of production and staging endpoints, causing CI workflow failures when evaluating authenticated endpoints with a staging-specific API key. Key changes: - Update all testthat test files to explicitly target env = "staging". - Synchronize all VCR cassettes in tests/fixtures to match staging URLs and staging package identifiers. - Remove deprecated EDI_TEST_ENV environment variable logic from base_url() and base_url_portal(). - Update get_test_package() to dynamically query the latest revision of edi.1923 during live testing while resolving to edi.1923.1 during VCR replay. - Add env parameter to read_data_package_from_doi() and update documentation. - Comment out test_read_data_package_from_doi due to DOI resolution targeting production endpoints incompatible with staging credentials. - Remove obsolete legacy cassettes in tests/testthat/_vcr.
…iting
Running the test suite against the live PASTA staging environment
(with VCR_TURN_OFF=true or during live CI workflows) sent rapid bursts
of consecutive requests, triggering HTTP 429 ("Too Many Requests")
rate limiting responses from the PASTA server.
To address this without modifying any core package functions in R/,
a test-level response throttling hook is registered in
tests/testthat/setup-EDIutils.R using httr::set_callback("response", ...).
This introduces a 1-second delay between real HTTP requests by default,
configurable via the EDI_TEST_THROTTLE_DELAY environment variable.
Because httr's response callback only executes when an actual network
request completes, hermetic VCR cassette playback bypasses this hook
completely, preserving instantaneous mock test performance.
In addition, vignettes/tests_requiring_authentication.Rmd is updated
to document live testing workflows, VCR bypass settings, and throttle
delay configuration for developers and maintainers.
- Dynamically resolve entity IDs in data entity tests: Update test_read_data_entity, test_read_data_entity_checksum, test_read_data_entity_name, test_read_data_entity_resource_metadata, test_read_data_entity_size, and test_read_metadata_entity to query available entity IDs via read_data_entity_names() instead of hardcoding entity IDs. - Add get_test_derived_package() helper in R/utilities.R: Query descendants of the test package to dynamically resolve derived package identifiers for test_list_data_sources. - Add VCR cassette support for test_read_data_package_archive: Wrap test_read_data_package_archive tests in VCR cassettes to prevent live HTTP requests during standard test execution while retaining skip_on_cran() and handling cached binary fixture payload extraction during playback. - Update is_vcr_replaying() in R/utilities.R to use cass() for accurate state detection between recording and replay. - Refresh affected VCR fixture cassettes in tests/fixtures/.
…lows - Add EDI_TEST_THROTTLE_DELAY=2 to R-CMD-check and test-coverage jobs in check-standard-real-requests.yaml to prevent HTTP 429 rate limit errors when running live test requests against staging. - Add workflow_dispatch triggers to check-standard.yaml and check-standard-real-requests.yaml to allow manual workflow execution from the GitHub Actions UI.
- Bump package version to 3.0.1 in DESCRIPTION, NEWS.md, codemeta.json, and cran-comments.md. - Document 3.0.1 release notes for staging migration, dynamic entity resolution, request throttling, and workflow updates in NEWS.md. - Extract test-only helper functions out of R/utilities.R into tests/testthat/helper-test-package.R to keep package runtime namespace clean and modular.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.