Skip to content

Make ShardedHashMap::with_capacity split capacity between shards - #161127

Open
Zoxc wants to merge 1 commit into
rust-lang:mainfrom
Zoxc:shard-cap
Open

Make ShardedHashMap::with_capacity split capacity between shards#161127
Zoxc wants to merge 1 commit into
rust-lang:mainfrom
Zoxc:shard-cap

Conversation

@Zoxc

@Zoxc Zoxc commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

This makes ShardedHashMap::with_capacity split capacity between shards. The callers in CtxtInterners::new did not divide these by the shard count, so it reserved 32 times greater capacity, significantly contributing to startup costs.

Spotted by Claude Opus while investigating startup stack usage.

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 15, 2026
@rustbot

rustbot commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator

r? @nnethercote

rustbot has assigned @nnethercote.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: compiler, incremental, query-system
  • compiler, incremental, query-system expanded to 75 candidates
  • Random selection from 19 candidates

@Zoxc

Zoxc commented Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

With -Z threads=sync:

BenchmarkBeforeAfterBeforeAfterBeforeAfter
TimeTime%Physical MemoryPhysical Memory%Committed MemoryCommitted Memory%
🟣 helloworld:check0.0612s0.0530s💚 -13.44%55.64 MiB41.71 MiB💚 -25.04%165.45 MiB100.23 MiB💚 -39.42%
Total0.0612s0.0530s💚 -13.44%55.64 MiB41.71 MiB💚 -25.04%165.45 MiB100.23 MiB💚 -39.42%
Summary1.0000s0.8656s💚 -13.44%1 byte0.75 bytes💚 -25.04%1 byte0.61 bytes💚 -39.42%

@rust-log-analyzer

Copy link
Copy Markdown
Collaborator

The job x86_64-gnu-miri failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)
tests/pass-dep/libc/libc-socketpair.rs (revision `run`) ... ok
tests/pass-dep/libc/libc-time.rs (revision `run`) ... ok

FAILED TEST: tests/pass-dep/libc/libc-epoll-blocking.rs (revision `edge_triggered.run`)
command: MIRI_BE_RUSTC="host" MIRI_ENV_VAR_TEST="0" MIRI_TEMP="/tmp/miri-uitest-ePNOAC" RUST_BACKTRACE="1" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/tmp/miri_ui/0/tests/pass-dep/libc/libc-epoll-blockingedge_triggered"

error: test got exit status: 101, but expected 0
##[error]   --> tests/pass-dep/libc/libc-epoll-blocking.rs:316:58
    |
316 |                 let data = read_exact_array::<4>(fds[0]).unwrap();
    |                                                          ^^^^^^^^^ called `Result::unwrap()` on an `Err` value: Os { code: 9, kind: Uncategorized, message: "Bad file descriptor" }
    |

full stderr:

---
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: libc_epoll_blocking::waiting_threads_unblocked_after_socketpair_close::{closure#0}
   4: std::thread::scoped::scope::<libc_epoll_blocking::waiting_threads_unblocked_after_socketpair_close::{closure#0}, ()>::{closure#0}
   5: <core::panic::unwind_safe::AssertUnwindSafe<std::thread::scoped::scope<libc_epoll_blocking::waiting_threads_unblocked_after_socketpair_close::{closure#0}, ()>::{closure#0}> as core::ops::function::FnOnce<()>>::call_once
   6: std::panicking::catch_unwind::do_call::<core::panic::unwind_safe::AssertUnwindSafe<std::thread::scoped::scope<libc_epoll_blocking::waiting_threads_unblocked_after_socketpair_close::{closure#0}, ()>::{closure#0}>, ()>
   7: __rust_try
   8: std::panicking::catch_unwind::<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::scoped::scope<libc_epoll_blocking::waiting_threads_unblocked_after_socketpair_close::{closure#0}, ()>::{closure#0}>>
   9: std::thread::scoped::scope::<libc_epoll_blocking::waiting_threads_unblocked_after_socketpair_close::{closure#0}, ()>
  10: libc_epoll_blocking::waiting_threads_unblocked_after_socketpair_close
  11: libc_epoll_blocking::main
  12: <fn() as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

full stdout:
---

Location:
   /cargo/registry/src/index.crates.io-1949cf8c6b5b557f/ui_test-0.30.7/src/lib.rs:365

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
error: test failed, to rerun pass `--test ui`

Caused by:
  process didn't exit successfully: `/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/build/miri/855274d73082e308/out/ui-855274d73082e308 tests/pass tests/panic` (exit status: 1)
Command `/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo test --target x86_64-unknown-linux-gnu -Zbinary-dep-depinfo -j 4 -Zroot-dir=/checkout --locked --color=always --profile=release --manifest-path /checkout/src/tools/miri/Cargo.toml -- tests/pass tests/panic [workdir=/checkout]` failed with exit code 1
Created at: src/bootstrap/src/core/build_steps/tool.rs:196:21
Executed at: src/bootstrap/src/core/build_steps/test.rs:771:19

Command has failed. Rerun with -v to see more details.
Bootstrap failed while executing `test --stage 2 miri -- tests/pass tests/panic`
Currently active steps:
test::Miri { target: x86_64-unknown-linux-gnu } at src/bootstrap/src/core/build_steps/test.rs:697
Build completed unsuccessfully in 0:01:55
  local time: Sat Aug 15 09:41:50 UTC 2026

@nnethercote

Copy link
Copy Markdown
Contributor

Oh dear. Let's see if this affects any of the parallel benchmarks on CI.

@bors try @rust-timer queue

@rust-timer

Copy link
Copy Markdown
Collaborator

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

@rust-bors

rust-bors Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

⌛ Trying commit 76c314f with merge 346a635

To cancel the try build, run the command @bors try cancel.

Workflow: https://github.com/rust-lang/rust/actions/runs/31881292297

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Aug 15, 2026
rust-bors Bot pushed a commit that referenced this pull request Aug 15, 2026
Make ShardedHashMap::with_capacity split capacity between shards
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-perf Status: Waiting on a perf run to be completed. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants