Rust platform size

I went to this page:

https://forge.rust-lang.org/other-installation-methods

and I discovered that the download is 203 MB. This was surprising to me, so I
looked at other languages:

Language	Size	Link
Go	135 MB	https://golang.org/dl
Perl	102 MB	http://strawberryperl.com/releases.html
Julia	50 MB	https://julialang.org/downloads
Python	25 MB	https://python.org/downloads/release/python-373
PHP	25 MB	https://windows.php.net/download
D	23 MB	https://dlang.org/download.html
Nim	19 MB	https://nim-lang.org/install_windows.html
Ruby	11 MB	https://rubyinstaller.org/downloads

So Rust is 70% larger than Go. Or to put another way, Rust is larger than Julia,
Python, PHP, D, Nim and Ruby combined. Can anything be done about this or is
the large size unavoidable?

[jonas-schievink]

IIUC #59800 will reduce the size, potentially by a lot

[jonas-schievink]

Also the comparison might want to include Haskell, whose installer is 268 MB (https://www.haskell.org/platform/windows.html).

[Centril]

These comparisons are probably missing various aspects, e.g. Haskell includes the full haskell platform with a list of packages as well as a runtime. Rust might include other things... We should compare with our past selves first and foremost.

[est31]

@cup where are you getting that 205 MB number from? When I try to download https://static.rust-lang.org/dist/rust-1.35.0-x86_64-unknown-linux-gnu.tar.gz it tells me 252 MB so the problem is even worse than you say it is... Maybe linux vs windows? Some points for where the potential bloat could come from:

• licenses are stored multiple times for each package. Costs a few KB but I guess it can't be avoided. It would probably be useful to have a compression format that can do content based deduplication, but I guess there is none out there, tar.gz/zip is what we need to use.

• save analysis of all std crates is being stored. Using json, the save analysis format is extremely verbose. It contains repetitive stuff like references to the same source file over and over. It could probably be improved a lot by a) interning stuff like source file names and b) a binary format. As IIRC serde is used already for json output and input, adopting bincode would be a first easy step then followed by interning (interning can be done transparently from bincode). The save-analysis format is unstable and probably always will be, given that replacement is on its way with rust-analyzer. If there is ever a desire to stabilize, one can switch back to json. As for the point that we are using deflate compression, sure it helps but a native compression would help even more.

• rust's copy of LLVM is 76 MiB while julia's copy of LLVM is 48 MB (both uncompressed). Why is that the case? Rust has LLVM 8.0 and julia 6.0 but that can't be the reason, has LLVM grown this much in size between two versions?

• there is a documentation directory that contains rustdoc/mdbook of various components. Each component has its very own copy of fonts (each almost half of a MB large). It's repeated multiple times. I'm sure much can be saved by using a shared copy of those fonts.

[mati865]

rust's copy of LLVM is 76 MiB while julia's copy of LLVM is 48 MB (both uncompressed). Why is that the case?

At least for Linux a lot of dependencies are linked statically (even libstdc++) to make it work on old Linux releases.

[est31]

@cup no need to wait for nightlies, the built artifacts are already being uploaded to public servers. This gives us the following size impact of #59800 :

file name	dbeed58 (prior to #59800 merge)	dd2e804 (after #59800 merge)
rustc-nightly-x86_64-unknown-linux-gnu.tar.gz	122 073 207	100 939 501
rust-std-nightly-x86_64-unknown-linux-gnu.tar.gz	82 951 040	211 427 679
cargo-nightly-x86_64-unknown-linux-gnu.tar.gz	6 751 312	6 752 518
rustc-nightly-x86_64-pc-windows-msvc.tar.gz	82 391 063	59 805 292
rust-std-nightly-x86_64-pc-windows-msvc.tar.gz	74 663 690	206 604 497
cargo-nightly-x86_64-pc-windows-msvc.tar.gz	4 450 362	4 443 326

Numbers collected via commands like export COMMIT=dd2e8040a35883574ae0c4cc7a4e887ecb66469c; export TOOL=rustc-nightly-x86_64-unknown-linux-gnu.tar.gz; curl -I https://s3-us-west-1.amazonaws.com/rust-lang-ci2/rustc-builds/${COMMIT}/${TOOL} | rg Content-Length.

So for rustc there is a nice sweet reduction but std has a massive size increase. @Zoxc why is this the case? Maybe some cache not being emptied?

[Zoxc]

@est31 rust-std also includes the compiler crates, and with #59800 there's duplication for these as the same code is included in the rlibs and rustc_driver's dylib.

[lilydjwg]

Hi, I have a question. What needs those rlib files in rust-std? Can I remove those rlib files from rust-std and have most crates build as normal? (I'm repackaging rust-nightly for Arch Linux.)

It's 109MiB larger now and it'll take me four more minutes to download (what's worse, I can't watch online videos meanwhile)....