Upd/fix big ctx#3
Merged
Merged
Conversation
feat(pagination): add cursor-based streaming across providers - Add getJSONAny to Bitbucket to follow absolute `next` cursor URLs while keeping relative paths resolved against the repo API root - Rework TagsStream to honor context cancellation and use a safer default page size (50) that stays within the 8 MiB GetJSON cap - Skip release assets with unparseable download URLs instead of dereferencing a nil URL on GitHub - Add stream.go and pagination tests
build(deps): replace shell version scripts with gometagen - Migrate build tooling from _run/scripts/*.sh to the gometagen CLI for manifest and version management - Read package name and version from _run/values.yml via gometagen manifest/version commands in CI actions - Update release workflow to bump minor version through gometagen and stage values.yml - Rewrite commit-hook.sh and firststart.sh to use gometagen for version, branch, and git hook installation - Add _run/values.yml as the single source of truth, removing values/name.txt and values/ver.txt - Delete obsolete git.sh, go_creator_const.sh, and sys.sh - Harden hook scripts with set -Eeuo pipefail and safe temp-file rewrites
fix tests
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Streaming hardening, Bitbucket cursor fix, and _run migration to gometagen
Streaming / pagination
StreamPageshelper (stream.go): drives offset-based pagination for all providers, adaptively halving the page size when a response exceeds theGetJSON8 MiB body cap (ErrResponseTooLarge). Offset-based remap guarantees no duplicated or skipped items after a shrink.Sendhelper: every channel send in every stream now goes throughselect { send / ctx.Done() }. Previously a canceled context could not unblock a stream whose consumer stopped reading — the goroutine leaked forever.StreamPagesinput validation:perPage <= 0or nilfetch/emitreturns an error instead of panicking (division by zero). Nil context is normalized tocontext.Background()everywhere.Bug fixes
TagsStreambroke on page 2+: it appended the absolutenextcursor URL to the API root, producing.../repositories/owner/repo/https://api.bitbucket.org/.... Now usesgetJSONAny(moved tobitbucket/func.go) like the release code already did. Only visible on repos with >50 tags.url.Parseerrors were ignored and the result dereferenced ingithub.buildReleaseObjandbitbucket.listDownloads. Broken assets are now skipped.GetJSONreports an explicitErrResponseTooLargesentinel instead of a misleading truncated-JSON decode error._runmigration (bash scripts → gometagen)_run/values/{name,ver}.txt+ ~400 lines of bash (sys.sh,git.sh,go_creator_const.sh) replaced by_run/values.ymlandgometagen(version bump, branch name, hook install, meta generation).firststart.sh/push-hook.sh(both called a non-existentgo_tidy_all.sh); all scripts now run withset -Eeuo pipefail.gen.gogeneratestarget/meta_gen.goviagometagen generate(content hash instead of git hash).generate-build-envand the release version-bump step now read/writevalues.ymlviagometagen.Tests
tests/pagination_test.go, httptest + client swap, no real network): adaptive page shrink with exact request-sequence assertions, body-cap edge cases, cancel-unblocks-send, Bitbucketnext-cursor following, parameter validation.func_test.go,TestGlobal) are skipped under-short; the commit hook runsgo test -short ./...so commits don't fail on provider rate limits.README
RoundTripperonlightweigit.HttpClient(GitHub Bearer, GitLabPRIVATE-TOKEN, Bitbucket), host-scoped for multi-provider use.target/package,_run/firststart.shbootstrap (Linux/macOS), manual steps for Windows.targetpackage constants renamed:GlobalName/GlobalVersion/GlobalHash/GlobalVersionMajor/GlobalVersionMinor/GlobalVersionPatch→Name/Version/Hash/VersionMajor/VersionMinor/VersionPatch(GlobalDateUpdate→DateUpdate). Affects consumers importingtargetfrom release tags. Covered by the upcoming minor bump (v0.x).