Fix: Omit threading.get_ident() from cache token for async implementations in sync mode

[yuxin00j]

The Problem:
Currently, the AbstractFileSystem instance cache incorporates threading.get_ident() into its cache token. For workloads using multi-threading (like Hugging Face datasets fetching metadata), this completely bypasses the cache, forcing fsspec to create a brand-new filesystem instance and aiohttp.ClientSession for every single thread. This results in redundant authentication and connection overhead, defeating the purpose of the cache.

The Solution:
As discussed in #2020, because fsspec currently uses a single global fsspecIO event loop for all synchronous calls (asynchronous=False), all sync() coroutine executions are inherently funneled into that single background thread. Therefore, multiple calling Python threads can perfectly and safely share the exact same AsyncFileSystem instance.

This PR modifies fsspec/spec.py to conditionally remove threading.get_ident() from the caching token if:

1. The class uses an async implementation (async_impl = True).
2. The user requests a synchronous interface (asynchronous=False).

For purely synchronous filesystems (e.g., FTP) or when asynchronous=True, the thread ID remains in the cache token to ensure thread safety and avoid cross-loop boundary violations.

Changes:

• Conditionally generated the cache token in _Cached.__call__.
• Added tests in fsspec/tests/test_cache_threads.py to explicitly verify the caching behavior across multiple threads for both AsyncFileSystem and AbstractFileSystem under different modes.

Closes #2020

[yuxin00j]

Hi @martindurant, can we retry the CI for this PR? Thanks.

[martindurant]

It's your specific new tests that are failing, but only for older python versions.

[yuxin00j]

Hi @martindurant, I have updated the test. Could you help run the CI again?

[martindurant]

We seem to be missing some test dependencies in the windows CI env. Maybe they should all be in the [test] env define in pyproject. I added pytest-asyncio directly to the env, but there are more.

[martindurant]

@yuxin00j , let's leave investigating what's going on with the install process for windows another time.