Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion alpha/engagements/2026/Rust Foundation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,4 @@ In addition to the monthly reports here in GitHub, the Rust Foundation provides
* Tobias Bieniek - Software Engineer, Rust Foundation
* Marco Ieni - Infrastructure Engineer, Rust Foundation
* Ubiratan Soares - Infrastructure Engineer, Rust Foundation
* Jon Bauman - Interop Software Engineer Lead, Rust Foundation
* Joel Marcey - Director of Technology, Rust Foundation
129 changes: 129 additions & 0 deletions alpha/engagements/2026/Rust Foundation/update-2026-07.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
## AI Security Engineer in Residence (SEIR)

Jacob Finkelman continued his role as Security Engineer in Residence (SEIR) for the Rust Foundation, using Scrutineer as a harness for various AI models to find and report bugs across the Rust ecosystem. Jacob has been working closely with Walter and Adam.

### Bug Reports Found and Reported

Using Scrutineer with multiple AI models:

- [memchr](https://github.com/BurntSushi/memchr/issues/225)
- [thread_local-rs](https://github.com/Amanieu/thread_local-rs/issues/93)
- [rand](https://github.com/rust-random/rand/pull/1790)
- [rend](https://github.com/rkyv/rend/issues/18)
- [rend](https://github.com/rkyv/rend/issues/19)
- [zstd-rs](https://github.com/KillingSpark/zstd-rs/pull/111#discussion_r3521243842)
- [tinyvec](https://github.com/Lokathor/tinyvec/pull/222)
- [RustCrypto/traits](https://github.com/RustCrypto/traits/pull/2471)
- [RustCrypto/formats](https://github.com/RustCrypto/formats/pull/2379)
- Independently rediscovered [zlib-ng](https://github.com/zlib-ng/zlib-ng/issues/2244)
- [bumpalo](https://github.com/fitzgen/bumpalo/issues/330).... caught before the code was published to crates.io
- Independently rediscovered [humantime](https://github.com/chronotope/humantime/issues/66)
- [crates.io](https://github.com/rust-lang/crates.io/pull/14230)

Using Scrutineer with Fable:

- [wasm-tools](https://github.com/bytecodealliance/wasm-tools/pull/2566)
- [wasm-tools](https://github.com/bytecodealliance/wasm-tools/pull/2569)
- [wasm-tools](https://github.com/bytecodealliance/wasm-tools/pull/2565)
- [wasm-tools](https://github.com/bytecodealliance/wasm-tools/pull/2570)
- [wasm-tools](https://github.com/bytecodealliance/wasm-tools/pull/2567)
- [wasm-tools remaining issues](https://github.com/bytecodealliance/wasm-tools/issues/2568)
- [image-webp](https://github.com/image-rs/image-webp/pull/186)
- [anstyle](https://github.com/rust-cli/anstyle/pull/344)
- [sysinfo](https://github.com/GuillaumeGomez/sysinfo/issues/1704) and [libpnet](https://github.com/libpnet/libpnet/issues/772)
- [minifier-rs](https://github.com/GuillaumeGomez/minifier-rs/pull/129)
- [minifier-rs](https://github.com/GuillaumeGomez/minifier-rs/pull/130)
- [clru-rs](https://github.com/marmeladema/clru-rs/issues/67)
- [simd](https://github.com/Nugine/simd/issues/78)
- [heapless](https://github.com/rust-embedded/heapless/issues/674)
- [time](https://github.com/time-rs/time/commit/2595e1cfb418f51b03dded39fb364c4fbb5cd0d0)
- [time](https://github.com/time-rs/time/commit/583bf0e8942b488b575059b585687e093a848511)
- [time](https://github.com/time-rs/time/commit/037f91b32ead986819e72e430c5cb9811fb7ce0d)
- [stylo (Mozilla)](https://bugzilla.mozilla.org/show_bug.cgi?id=2056395)... CSS files causing stack overflow in stylo; browsers confirmed not security-related due to other mitigations

There are approximately 300 additional findings not yet discussed. Some are too small to warrant attention, most would be worth fixing if there were a more scalable way to communicate with maintainers, and a handful are pending response or remediation process discussion. The team is working on more scalable solutions for disclosure communication.

### Analysis: Why Rust Has Fewer Critical Bugs

Jacob shared analysis on why AI-driven scanning of Rust projects is finding fewer (and less critical) bugs compared to other ecosystems. Contributing factors include:

- Rust's strong tooling (borrow checker, Clippy, rust-fuzz, Miri) prevents many bug classes that require AI tooling in other ecosystems
- Rust is a newer ecosystem with less accumulated technical debt
- Miri's type-signature checking allows going straight from finding a bug to working on a fix, without needing a full end-to-end RCE to get attention
- Possible need for more scanning volume and broader crate coverage in case not looking at the right crates

### Process and Approach

All bugs involve manual human-in-the-loop triage: Jacob reads reports, verifies and simplifies reproducers, reaches out to maintainers through back channels, and hand-writes issues. In some cases, maintainers were willing to triage raw AI reports directly (only done with consent). The team is working to multiplex accounts and improve tooling to use multiple AI providers (including underutilized OpenAI capacity) to increase scan throughput. Jacob is focusing on his strengths in deep knowledge of the Rust community and its people, rather than competing on prompt engineering for AI bug-finding.

## Signing and Mirroring

In mid-July, Arlo Siemsen began work as our contracted distribution engineer, helping in the effort to bring production-level [signing and mirroring](https://rust-lang.github.io/rust-project-goals/2026/mirroring.html) to crates.io. Thank you to Alpha-Omega for their funding support in this important role. We are excited to have Arlo on board.

In his first couple of weeks, Arlo filed a [github-feedback issue](https://github.com/rust-lang/github-feedback/issues/29) about running the crates.io mirror on GitHub infrastructure, and connected with GitHub staff to begin discussions. On the signing front, Arlo began adding P-256 ECDSA support to `rust-tuf` to address the fact that neither YubiKeys (pre-5.7 firmware) nor GCP HSM support ED25519, and uncovered JSON parsing issues in tuf-on-ci related to how public keys with newlines are stored. Arlo also set up [tuf-on-ci in the rf-signing-experiment GitHub project](https://github.com/rf-signing-experiment/tuf-on-ci) and offered to conduct signing ceremonies with interested parties.

Arlo investigated the TUF spec, `rust-tuf` crate, and `tuf-on-ci` repository and documented several compatibility issues:
- `rust-tuf` only supports Ed25519 keys (which lack YubiKey and GCP HSM support)
- `tuf-on-ci` generates `keyid`s differently from `rust-tuf` (causing keys to be discarded on load)
- the "canonical JSON" normalization used for signing produces invalid JSON that `serde_json` cannot parse which is problematic for PEM-encoded public keys. The alternative is to re-parse the pre-canonicalized json, but this increases the attack surface area.

Arlo raised the question of whether the Rust Project should abandon strict TUF spec compliance, noting that crates.io's frequent content changes and the teams' desire for automated releases require online keys for `timestamp`, `snapshots`, and `targets` roles, which eliminates many of TUF's security properties. Arlo suggested the project could instead adopt select portions of TUF (particularly root metadata management and key rotation via `tuf-on-ci`) and build a simpler framework tailored to Rust's specific threat model. He also proposed a possible compromise for the toolchain by using a delegated target with human-in-the-loop signing for stable releases while keeping nightly/beta online in a separate TUF target.

Walter set up the [rf-signing-experiment GitHub organization](https://github.com/rf-signing-experiment/experiments) with several repositories. He created issues to track work across [tuf-on-ci](https://github.com/rf-signing-experiment/tuf-on-ci/issues), [rustlang_tuf](https://github.com/rf-signing-experiment/rustlang_tuf/issues), [readme](https://github.com/rf-signing-experiment/readme/issues), and [rustup](https://github.com/rf-signing-experiment/rustup/issues). He was also tasked with providing code, a todo list, and writing a short doc for others working on this effort.

## Security Engineering

Walter made improvements to Scrutineer...

- Added [database support](https://github.com/alpha-omega-security/scrutineer/pull/658) to Scrutineer for persistent storage of findings
- Added [Rust ecosystem support](https://github.com/alpha-omega-security/scrutineer/pull/351) to Scrutineer
- Built a [sharing platform](https://github.com/walterhpearce/scrutineer/tree/ui-sharing) leveraging GitHub authentication to share findings directly with maintainers

Walter implemented automated deployment to the scanning cluster and upgraded the cluster to high-memory instances for better performance.

Walter deployed a Grafana + InfluxDB + Postgres stack for sustainability and ecosystem insights on crates.io and internal repositories.

Adam improved tooling to keep reference tables up to date in Datadog to aid in security response.

## DDOS Attack

On July 11, 2026, crates.io experienced a coordinated DoS attack against its API endpoints. Members of the Rust Foundation crates.io oncall team deployed countermeasures and servers returned to normal within approximately 80 minutes. Fastly's CDN services and fast response time are credited in helping resolve the incident. Crate downloads and dependency resolution through the sparse/git index were not affected.

## crates.io

Several security-related crates.io items were addressed this month:

- rejecting `..` in tarball entry path components and rejecting additional insecure tarball entry types
- fixing internal server error issues caused by null-bytes in URLs
- adding a configurable limit for the number of entries in uploaded tarballs
- investigating and addressing URL invalidation failures on Fastly,
- responding to a DDoS attack by enabling DDoS protection in Fastly and adjusting the NGWAF configuration
- implementing Datadog service checks to eventually replace PagerDuty for monitoring.

You can see a 6-month crates.io development update in a recent [blog post](https://blog.rust-lang.org/2026/07/13/crates-io-development-update/) written by Tobias.

## More Secure Infra

RISCV users who want to use musl now don't have to compile Rust tools like rustc and cargo. More details about what “tier 2 host tools” means in the [docs](https://doc.rust-lang.org/nightly/rustc/platform-support.html#tier-2-with-host-tools). [Github PR](https://github.com/rust-lang/rust/pull/158766).

Documentation was [added](https://github.com/rust-lang/rust-forge/pull/1074) covering the setup for Yubikeys for the usage in the Rust Project, detailing common use cases and best practices.

[Audit Logs](https://docs.github.com/en/enterprise-cloud@latest/admin/monitoring-activity-in-your-enterprise/reviewing-audit-logs-for-your-enterprise/streaming-the-audit-log-for-your-enterprise#setting-up-streaming-to-datadog) were enabled for the [Rust GitHub Enterprise](https://github.com/enterprises/rust-lang), so that the Rust Foundation security team can analyze them in DataDog using [Cloud SIEM](https://www.datadoghq.com/product/cloud-siem/).

Renovate was enabled on the [rust-lang/rust](https://github.com/rust-lang/rust) repository. It automatically opens PRs for GitHub Actions update and security updates. You can see the PRs opened by renovate [here](https://github.com/rust-lang/rust/pulls?q=is%3Apr+author%3Arenovate-bot+is%3Aclosed).
From the [dependency dashboard](https://github.com/rust-lang/rust/issues/134129), maintainers can open PRs for individual dependencies when they want to update them. [Configuration file](https://github.com/rust-lang/rust/blob/main/.github/renovate.json5).

Fastly WAF was [enabled](https://github.com/rust-lang/simpleinfra/issues/1007) to enforce crates-io rate limits and block potential security threats to 100% of crates-io traffic. This was already useful to mitigate a DDoS attack on July 11, 2026.

The GitHub setting "Dependabot alerts" in the `rust-lang` GitHub organization was enabled, so that (1) Maintainers receive alerts for vulnerabilities affecting their repositories (2) Maintainers can see the vulnerabilities affecting their repositories at `https://github.com/rust-lang/<repo>/security/dependabot` (3) Renovate (if configured) can send PRs to fix vulnerabilities. See the [Renovate Docs](https://docs.renovatebot.com/configuration-options/#vulnerabilityalerts) to learn more. Note: This setting doesn't activate automatic Dependabot PRs. This is another setting called "Dependabot security updates" that we have no intention of enabling, because we prefer using Renovate.

[Custom Properties](https://docs.github.com/en/organizations/managing-organization-settings/managing-custom-properties-for-repositories-in-your-organization) are now [configurable](https://github.com/rust-lang/team/pull/2512) in the [`team`](https://github.com/rust-lang/team) repo. This allows repositories to opt-in into automations developed by the Infrastructure team.

GitHub Pages are now [configured](https://github.com/rust-lang/team/issues/2518) in the [`team`](https://github.com/rust-lang/team) repo. See [docs](https://github.com/rust-lang/team/blob/main/docs/toml-schema.md#github-pages) and [example](https://github.com/rust-lang/team/blob/9a1b2b6b0aa87d0ad1a7e712de5dea1cd4b3ef10/repos/rust-lang/rust-forge.toml#L7). To setup github pages for your repository, Rust Project members can just raise a PR now. This will increase the developer experience of contributors and help the Infrastructure Team detect drifts more easily. Thanks [cuba0001](https://github.com/cuba0001) for working on this.

A [policy](https://github.com/rust-lang/rust-forge/pull/1051) was established mandating the usage of hardware security keys as the default method for multi-factor authentication (MFA) for those accessing critical Rust infrastructure. We added [documentation](https://github.com/rust-lang/rust-forge/pull/1074) explaining the capabilities of hardware security keys and how they can be used within the Rust Project. In the future, we want to enable more use cases on top of these devices, including using them as part of TUF-in-CI.

The team [documented](https://github.com/rust-lang/infra-team/pull/285) how Rust Project members can have their repository scanned via [DataDog Code Security](https://docs.datadoghq.com/security/code_security/). We started with [crates-io](https://github.com/rust-lang/crates.io) and linked the findings in the DataDog UI to the crates-io team.

Marco and Ubi wrote about the Q3 plans for Rust infra in a recent [blog post](https://blog.rust-lang.org/inside-rust/2026/07/15/infrastructure-team-q2-recap-and-q3-plan/).

Loading