[SYNPY-1882] preserve file name casing when uploading/downloading data - #1431
Conversation
os.path.normcase() in normalize_path() lowercases the entire path on Windows (a no-op on POSIX), which cascaded into guess_file_name() and corrupted the derived entity name (e.g. File_Name.tsv -> file_name.tsv) while the FileHandle fileName kept its case. It also lowercased downloaded files on disk. Remove normcase() from normalize_path() so casing is preserved on every platform, and move case-insensitive cache matching into a dedicated _match_cache_map_key() helper in cache.py. The helper prefers an exact (case-sensitive) match -- correct for case-sensitive NTFS directories -- and falls back to an os.path.normcase() comparison so cache entries written by older clients with lowercased keys still resolve. This avoids forcing Windows users to re-download already-cached files. Cache.add() now migrates a legacy lowercased key to the case-preserving key instead of accumulating duplicates. Adds regression tests for case-preserving normalize_path/guess_file_name and for the case-tolerant cache matching, add-migration, and legacy lowercased key lookup.
…g-v1bujx-wbf2ka [SYNR-1534] Preserve file name casing on Windows uploads/downloads
There was a problem hiding this comment.
Pull request overview
This PR updates the Synapse Python Client’s path normalization to preserve original path casing (notably on Windows) so derived entity names and downloaded filenames are not unintentionally lowercased, while updating cache lookup behavior to remain compatible with legacy (lowercased) cache keys.
Changes:
- Updated
utils.normalize_path()to stop usingos.path.normcase()so path casing is preserved. - Added cache-key matching logic in
core/cache.pythat prefers exact matches and falls back toos.path.normcasematching to support legacy Windows cache maps. - Added unit tests to validate case preservation and legacy cache-key compatibility.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
synapseclient/core/utils.py |
Preserves path casing by removing os.path.normcase() from normalize_path(). |
synapseclient/core/cache.py |
Adds _match_cache_map_key() and updates cache operations to tolerate legacy lowercased keys. |
tests/unit/synapseclient/core/unit_test_utils.py |
Adds coverage ensuring normalize_path() and guess_file_name() preserve case. |
tests/unit/synapseclient/core/unit_test_Cache.py |
Adds tests for legacy lowercased cache keys and key migration behavior. |
synapseclient/core/CLAUDE.md |
Updates internal documentation to reflect the new normalize_path()/cache behavior contract. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
merge changes in develop
thomasyu888
left a comment
There was a problem hiding this comment.
🔥 Thanks for taking a look at this, and what an interesting bug fix. I'll defer to @BryanFauble for final review of this!
BryanFauble
left a comment
There was a problem hiding this comment.
I would remove all integration tests on this change and replace it with 100% branch code coverage via unit tests instead.
Approving ahead of that because the core logic looks good to me.
merge develop changes
Problem
utils.normalize_path()calledos.path.normcase()on every path, which lowercases the entire string on Windows but not on other operating systems. This corrupted derived entity/file names (SYNR-1534) — e.g., an uploaded file like File_Name.TSV would get its name mangled to lowercase when the path round-tripped through normalize_path, and downloaded files could be renamed on disk to the lowercased form.Solution
os.path.normcase()withos.path.abspathfromnormalize_pathso path casing is preserved on all platforms. Since Cache previously relied on that implicit lowercasing for case-insensitive matching, added a new_match_cache_map_key()helper incache.pythat tries an exact match first and only falls back to a normcase-based case-insensitive comparison.Cache.contains(),Cache.get(),Cache.add()(replacing a legacy lowercased key instead of duplicating it), andCache.remove(). On POSIX, os.path.normcase is a no-op, so the fallback is inert there and behavior is unchanged.Note: Cache.get() (download path) still only matches by path — a rename/move with unchanged content still forces a real re-download, since there's no MD5 fallback on the download side (unlike upload, which self-heals via _needs_upload's MD5 comparison against Synapse's record).
Testing
unit_test_utils.pyconfirm normalize_path() and guess_file_name() preserve casing.unit_test_Cache.pycover_match_cache_map_key()directly andCache.contains()/get()/add()/remove()against legacy lowercased keys and casing-only renames, simulating Windows via a mockedos.path.normcase.test_upload.pyconfirmFile.store_asyncavoids redundant re-uploads after local renames, moves, and (on POSIX) casing changes, since MD5 comparison against Synapse's record catches these even when the cache misses by path.test_download.pyexercise the download-side cache-hit/miss behavior for renamed, moved, and re-cased files under both simulated case-sensitive and case-insensitive filesystems.