fix(ci): scope pytest discovery to open_ksa/tests - #13
Open
github-actions[bot] wants to merge 72 commits into
Open
fix(ci): scope pytest discovery to open_ksa/tests#13github-actions[bot] wants to merge 72 commits into
github-actions[bot] wants to merge 72 commits into
Conversation
How to use the functions to be able to download data into the same folder as your workbook
Using the example in the showcase which is ultimately going to live as a vignette of how to work with the library Will likely create more workbooks where something like this could apply
Making the names a little more relevant to show what is happening during the runtime examples
Removing the requirements file since the dependencies live in the environment of the pyproject.toml
Giving the user the option to only download a single resource if desired All the same values available as the get_dataset_resources.py function
Enabling single resource download
The ability to download files and the default to skip blank files It gets annoying to use the downloadUrl to only have it fail
relying on the modular funciton to be able to reuse code within the package effectively Additionally, no need to duplicate code unnecessarily
New run of file with new orgs to validate and battle test the data dictionary (still in testing)
Using the test cases as examples Added them for the key functions and how to use them in a runtime for a script
Moving out the main function from the script to be able to run the code successfully
testing out the functions for downloading and collecting high-level information about the resources and the corresponding organizations
Adding a dev container for the build environment
Enabling JSON to be able to show the files for the devcontainer configuration
Adjusting the container to include dependencies
Updating the SSL Adapter for formatting
Using the versioning of the SCM package to effectively maintain the state of the package
Including a fallback for the SCM versions
Making an update to accommodate more flexible headers due to limited downloads Might suggest adding an override for the headers Additionally there is now a component to enable an additional child directory for the extensions based recording
Enabling the ability to download in parallel based on the number passed If the value doesn't exist, then a regular for loop is run Max workers currently has no limitation Consider forcing a stop in the runtime if max_workers exceed max available
Working on replicating issues in different environments
Additionally compatibility fixes with python to add a condition for versions lower than 3.12 to avoid a legacy connection (#3)
Eliminating empty dataframes which throw an error with the runtime to build data dictionaries
Fixing a lazy string from 3.12.7 compatible text Had to make sure that the quotations were distinct between single quotes and double quotes. The string was exiting Fixes #3
Updating the container and runtime to ensure that it's reduced in size and fewer items are pulled into the environment Removed as much as I could with dockerignore, but too much irrelevant subject matter was coming into the container
Adding the requirements environment for the devcontainer runtime
Pulling refs/heads/dev into production
Not sure if there's a versioning update that happened to a previous release but seems to defeat the purpose of a release
Fixing the semantic versioning scheme for the test deployment
Pulling refs/heads/dev into production
Adjusting the headers of the API to handle the new header addition
This is a big change to enable models, contracts and forced typing for loading data in the environment
including data contract, integration and unit tests
basic build up of garbage to avoid bloating the repo
001 create an extensible
Pulling refs/heads/dev into production
Replace the previous dataset-summary downloader flow with a canonical fetch_and_load orchestrator that normalizes SelectionRequest inputs, discovers resources at organization, dataset, and resource scope, and records one manifest entry per resource. Add the shared data-model vocabulary for selection, manifest entries, completeness summaries, and resource health while preserving the legacy compatibility properties still referenced by the current tests and public surface. Update manifest serialization and CLI integration so the canonical DownloadManifest object is written directly, and add focused orchestration and contract coverage for rerun skipping, resource filtering, and deterministic non-network notebook/CLI flows.
Add a GitHub Actions CI workflow that installs the package, runs the full test suite, and builds distributions across CPython 3.10, CPython 3.12, and PyPy 3.10 so source reliability changes are validated before release. Modernize the package metadata for publishing by switching to an SPDX license string, declaring PyPy support, adding build to the dev extra, and excluding test packages from wheel contents so generated artifacts stay reviewable for dev and PyPI workflows.
…nly) - Auto-fixed 39 style issues via ruff --fix (unused imports, empty f-strings, multi-statement lines) across examples/ and tests/ - __init__.py: removed genuinely-dead urlparse/quote imports; converted the 7 intentional public-API re-exports to explicit `X as X` form so ruff (and type checkers) recognize them as intentional, not accidental - Removed 3 dead-code assignments (organization_id computed but never used in 2 example scripts; resource_one test fixture never referenced) - Fixed a real bug: downloader.py's sample_and_load() type hint used Union[str, "file"] -- "file" isn't a type in Python 3 (Py2 leftover), ruff correctly flagged it as an undefined name. Changed to typing.IO per the docstring's "file-like object" intent. - Manually split 2 `if x: print(...)` one-liners in _translate_dir.py Verified: 30/30 tests pass in an isolated venv, package's public API (open_ksa.download_file, .get_org_resources, etc.) still imports and resolves correctly after the re-export change. 3 E701 (multi-statement) errors remain, all inside .ipynb notebook cells -- left untouched to avoid risking notebook JSON corruption for a pure style fix.
Bare `pytest` picks up any test_*.py in the whole repo, including examples/workbooks/test_api.py -- a demo script (not a test) that imports openai, which isn't installed in CI. Added testpaths so discovery only walks the real test suite.
…it planning docs - .gitignore: add CLAUDE.md, AGENTS.md, GEMINI.md, .wolf/, and AI-IDE scaffolding patterns (.cursorrules, .clinerules, .windsurfrules, .roo*, memory-bank/, .github/copilot-instructions.md) - remove specs/ (001-create-an-extensible, 002-consistency-source-reliability): internal spec-kit planning docs, not user-facing, noise on a public repo
- publish-to-testpypi job ran on every push (no tag guard, unlike its publish-to-pypi sibling), re-uploading the same dev-version filename and failing with PyPI's file-already-exists 400 on every non-tag push. Add the matching if: startsWith(github.ref, 'refs/tags/') guard. - release.yml: add set -euo pipefail so a failed git tag/gh release call fails the job instead of silently continuing.
Dead code left over from development; no behavior change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-opened by the
Dev to Trunk Auto-PRaction on push todev(base:master). Esturban requested as reviewer.