Skip to content

bugfix: Keep the user of ssh URLs and restore it on mirrors stripped by v1.0.31 to v1.0.34 - #163

Merged
seeraven merged 5 commits into
seeraven:masterfrom
Bilb:fix/keep-ssh-user-in-mirror-url
Sep 28, 2026
Merged

seeraven merged 5 commits into
seeraven:masterfrom
Bilb:fix/keep-ssh-user-in-mirror-url

Conversation

@Bilb

@Bilb Bilb commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #159.

Problem

strip_credentials() and normalize_url() treat the user of ssh:// and scp-style URLs as a credential and drop it. The mirror remote is stored as github.com:org/repo and only works while the user is restored from the URL of the current command. Two situations have nothing to restore it from:

Both stay invisible on machines where ~/.ssh/config sets User git for the host.

Changes

  1. Cherry-pick of waipeng/gitcache@50b2aac by @waipeng, the approach preferred in SSH mirrors fail on gitcache -u because URL normalization drops the username #159: keep the user of ssh:// and scp-style URLs when normalizing URLs and stripping credentials. The commit is kept with its original author. Tests and the changelog entry are added here.
  2. Set the ssh user of the mirror remote to the user of the command URL. Keeping the user has a side effect on upgrade: the remote of a mirror created by v1.0.31 to v1.0.34 is no longer rewritten from the command URL, so the ssh commands that still worked before start failing as well. Before an update, if the mirror remote is an ssh URL of the same repository whose user is missing or differs from the one of the command URL, the remote is set to the command URL and the database URL is refreshed. This repairs those mirrors on their next use with an ssh URL, and keeps the pre-v1.0.31 behaviour of the mirror following the ssh user of the last command, for example on a cache shared between ssh accounts. Mirrors only ever updated through gitcache -u keep the user-less database URL and still need a fresh mirror or a manual git remote set-url.
  3. The ssh functional tests derived the database URL by removing git@. They now use the remote URL without the .git suffix.

Verification

  • Unit tests: 107 passed, with new tests for strip_credentials(), get_user(), is_ssh_url(), Database.set_url() and the remote repair, including a remote with a different user, a remote of another repository and a failing git remote set-url.
  • Functional tests gated on Workhorse/hermes, run locally against fa7b35e with platform.node() patched: 53 passed. The variants without a user (github.com:..., ssh://github.com/...) need User git in the ssh configuration, emulated with GIT_SSH_COMMAND="ssh -o User=git". The follow-up commit only changes the repair condition, the database refresh and the credits, and is covered by the unit tests.
  • Manual replay against an isolated GITCACHE_DIR, comparing v1.0.34, v1.0.34 plus change 1 only, and fa7b35e:
Sequence v1.0.34 change 1 only this branch
HTTPS clone, ssh command, HTTPS command refused ok ok
ssh clone, gitcache -u refused ok ok
ssh clone, HTTPS command refused ok ok
mirror stripped by v1.0.34, HTTPS command or -u refused refused refused
mirror stripped by v1.0.34, ssh command ok refused ok, remote repaired
same mirror, HTTPS command afterwards refused refused ok

waipeng and others added 3 commits September 25, 2026 16:37
…pping credentials.

The username of an SSH URL is part of the connection specification and
required for authentication, not a secret. Previously normalize_url()
dropped the 'user@' part before storing the upstream URL in the
database, so mirrors created from SSH URLs failed to authenticate on
later updates (e.g. 'gitcache -u') once the URL was read back from the
database and strip_credentials() found nothing to restore.

Now normalize_url() and strip_credentials() keep the username for
ssh:// and scp-style URLs and only strip (or mask) a password for
ssh:// URLs. SCP-style URLs (user@host:path) have no syntax for a
password, so their username is kept as-is without any stripping
logic. Behaviour for http(s)/ftp(s)/git URLs is unchanged, and the
mirror path still ignores the username so cache sharing is unaffected.

Fixes seeraven#159

(cherry picked from commit 50b2aac)
The functional tests derived the database URL by removing `git@` from
the remote URL. With the user kept, the database URL is the remote URL
without the `.git` suffix. Also cover strip_credentials() with unit
tests and add the changelog entry for seeraven#159.
…0.34

Those versions removed the user of ssh URLs from the mirror remote and
restored it from the URL of the current command before each update.
Keeping the user means the remote is no longer rewritten, so a mirror
created by those versions would stay without user and fail to
authenticate for every command, including the ssh commands that worked
before.

Before an update, compare the mirror remote with the URL of the command:
if the remote is an ssh URL of the same repository without a user while
the command URL has one, set the remote to the command URL. Mirrors only
ever updated through `gitcache -u` keep the user-less URL stored in the
database and still need the remote fixed by hand or a fresh mirror.

@seeraven seeraven left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! I've noticed that you haven't added yourself as an author in the CHANGELOG.md nor in the newly added files. Otherwise it's good to merge.

Comment thread test/unittests/test_git_cache_git_mirror_add_credentials_to_remote.py Outdated
Comment thread CHANGELOG.md Outdated
The repair of mirrors stripped by v1.0.31 to v1.0.34 only acted on a
remote without a user. A remote with a different ssh user was left
alone, so on a cache shared between ssh accounts every update ran as
whoever created the mirror. Compare the users instead and set the
remote to the command URL whenever they differ.

Refresh the database URL at the same time, so that `gitcache -s` shows
the repaired URL and `gitcache -d` accepts it.

Also add myself as author of the new test files and the changelog
entry, as requested in review.
_remote_user_differs() relied on _add_credentials_to_remote() returning
early for every non-ssh URL with a user. Check the mirror URL itself so
the function holds on its own.

Also add unit tests for normalize_url(): the ssh functional tests are
gated on specific hosts, so CI never exercised the ssh user being kept.
@Bilb

Bilb commented Sep 26, 2026

Copy link
Copy Markdown
Contributor Author

Hey @seeraven I have updated the PR with the credit and pushed some more tests. Let me know what you think

@seeraven

Copy link
Copy Markdown
Owner

Hi @Bilb , looks good to me! If it's ok with you I'll merge it.

@Bilb

Bilb commented Sep 27, 2026

Copy link
Copy Markdown
Contributor Author

@seeraven yeah this is ready to be merged!

@waipeng

waipeng commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

@Bilb thanks for working on this. Sorry was away last week.

@seeraven
seeraven merged commit 536558f into seeraven:master Sep 28, 2026
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SSH mirrors fail on gitcache -u because URL normalization drops the username

3 participants