Skip to content

Add OmniGCP - #385

Merged
AlCutter merged 8 commits into
transparency-dev:mainfrom
AlCutter:omnigcp
Aug 20, 2025
Merged

AlCutter merged 8 commits into
transparency-dev:mainfrom
AlCutter:omnigcp

Conversation

@AlCutter

@AlCutter AlCutter commented Aug 20, 2025

Copy link
Copy Markdown
Contributor

This PR adds a GCP-specific binary for running omniwitness, along with a Spanner-based persistence implementation, and a Secret Manager based signer.

$ go run ./cmd/gcp/omniwitness/ --signer_private_key_secret_name=projects/1234/secrets/privy/versions/1 --spanner=projects/myproject/instances/spanner/databases/db
I0820 13:48:27.709821  677822 main.go:83] Prometheus configured to listen on ":8081"
I0820 13:48:27.827359  677822 omniwitness.go:113] Added log "go.sum database tree": a32d071739c062f4973f1db8cc1069f517428d77105962b285bbf918c4062591
I0820 13:48:27.827394  677822 omniwitness.go:113] Added log "Armory Drive Prod 2": 48d31bf4bc3c95c7daddf7f8d33bb9ef1bff7500a40566eb56c97ff30eb6d44b
I0820 13:48:27.827418  677822 omniwitness.go:113] Added log "rekor.sigstore.dev - 3904496407287907110": c43a5887c927e0594e0a5feb87c2311ec86c3367613001b40dec552473aaa5dc
I0820 13:48:27.827437  677822 omniwitness.go:113] Added log "rekor.sigstore.dev - 2605736670972794746": 254283943e39c2a88e4d9185d4d7aa9f21afe369749872358204e8c25a00a80a
I0820 13:48:27.827451  677822 omniwitness.go:113] Added log "rekor.sigstore.dev - 1193050959916656506": 83650d970a8ad7c072dabb8d0d26d5a358b682fd43b432d39ccfbb7a200a6dba
I0820 13:48:27.827468  677822 omniwitness.go:113] Added log "developers.google.com/android/binary_transparency/0": 2371c7aa76ca0d588c7f21317b789071bb63593bd2c3d7b02ca5842f03680fda
I0820 13:48:27.827484  677822 omniwitness.go:113] Added log "developers.google.com/android/binary_transparency/google1p/0": f149a2505f9807fa39641c568f6161b2d496a4e43833350399330791f97501b9
I0820 13:48:27.827497  677822 omniwitness.go:113] Added log "lvfs": fabf738c02d466c9aef96e226f02bcc193f7093fd41995db036349ff38715e54
I0820 13:48:27.827511  677822 omniwitness.go:113] Added log "transparency.dev/armored-witness/firmware_transparency/prod/1": d4be095420ee6b18b699b2958a609f754f4784c81eda1e6d8f726b97a86e076d
I0820 13:48:27.827524  677822 omniwitness.go:113] Added log "transparency.dev/armored-witness/firmware_transparency/ci/2": 6df87b390820f503021ae2bef14dc582f23af18f158713982f243b6e414252d6
I0820 13:48:27.827537  677822 omniwitness.go:113] Added log "transparency.dev/armored-witness/firmware_transparency/ci/3": 0b963db2162e516836b4afbf8ff75aca120af717d3b93a332464cd5af4146e99
I0820 13:48:27.827551  677822 omniwitness.go:113] Added log "sigsum.org/v1/tree/44ad38f8226ff9bd27629a41e55df727308d0a1cd8a2c31d3170048ac1dd22a1": e5a19e2a99bc3fe4968b64bedaf64e43a9888e485320746cc46ade52ea33a328
I0820 13:48:27.827564  677822 omniwitness.go:113] Added log "tlog.andxor.it": a827ae6361d25a3349faef47a9a5ed64a3886b68e5b14284d3ee7ee79c0cc26e
I0820 13:48:30.994505  677822 omniwitness.go:198] HTTP server-shutdown goroutine started
I0820 13:48:30.994499  677822 omniwitness.go:155] Feeder "rekor.sigstore.dev - 2605736670972794746" goroutine will start after spread delay of 37.562068809s
I0820 13:48:30.994538  677822 omniwitness.go:155] Feeder "Armory Drive Prod 2" goroutine will start after spread delay of 55.700126271s
I0820 13:48:30.994529  677822 omniwitness.go:155] Feeder "go.sum database tree" goroutine will start after spread delay of 2.413955743s
I0820 13:48:30.994555  677822 omniwitness.go:155] Feeder "developers.google.com/android/binary_transparency/0" goroutine will start after spread delay of 33.304139458s
I0820 13:48:30.994549  677822 omniwitness.go:155] Feeder "lvfs" goroutine will start after spread delay of 24.085008625s
I0820 13:48:30.994572  677822 omniwitness.go:155] Feeder "transparency.dev/armored-witness/firmware_transparency/ci/3" goroutine will start after spread delay of 52.01526897s
I0820 13:48:30.994585  677822 omniwitness.go:155] Feeder "tlog.andxor.it" goroutine will start after spread delay of 47.534911705s
I0820 13:48:30.994590  677822 omniwitness.go:155] Feeder "developers.google.com/android/binary_transparency/google1p/0" goroutine will start after spread delay of 33.084219325s
I0820 13:48:30.994571  677822 omniwitness.go:155] Feeder "transparency.dev/armored-witness/firmware_transparency/prod/1" goroutine will start after spread delay of 4.12085867s
I0820 13:48:30.994609  677822 omniwitness.go:192] HTTP server goroutine started
I0820 13:48:30.994614  677822 omniwitness.go:155] Feeder "rekor.sigstore.dev - 3904496407287907110" goroutine will start after spread delay of 20.955473027s
I0820 13:48:30.994594  677822 omniwitness.go:155] Feeder "rekor.sigstore.dev - 1193050959916656506" goroutine will start after spread delay of 56.116787319s
I0820 13:48:30.994643  677822 omniwitness.go:155] Feeder "transparency.dev/armored-witness/firmware_transparency/ci/2" goroutine will start after spread delay of 22.113599597s
I0820 13:48:33.409563  677822 omniwitness.go:159] Feeder "go.sum database tree" running
I0820 13:48:35.115521  677822 omniwitness.go:159] Feeder "transparency.dev/armored-witness/firmware_transparency/prod/1" running
I0820 13:48:51.950149  677822 omniwitness.go:159] Feeder "rekor.sigstore.dev - 3904496407287907110" running
I0820 13:48:53.109303  677822 omniwitness.go:159] Feeder "transparency.dev/armored-witness/firmware_transparency/ci/2" running
I0820 13:48:55.079638  677822 omniwitness.go:159] Feeder "lvfs" running
I0820 13:49:04.079074  677822 omniwitness.go:159] Feeder "developers.google.com/android/binary_transparency/google1p/0" running
I0820 13:49:04.299170  677822 omniwitness.go:159] Feeder "developers.google.com/android/binary_transparency/0" running

Towards #386

@AlCutter
AlCutter force-pushed the omnigcp branch 2 times, most recently from 4410c13 to af3e3be Compare August 20, 2025 13:52
@codecov-commenter

codecov-commenter commented Aug 20, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 27.21893% with 123 lines in your changes missing coverage. Please review.
✅ Project coverage is 19.57%. Comparing base (3c58af4) to head (d914c94).
⚠️ Report is 199 commits behind head on main.

Files with missing lines Patch % Lines
cmd/gcp/omniwitness/main.go 0.00% 56 Missing ⚠️
cmd/gcp/omniwitness/persistence.go 55.42% 25 Missing and 12 partials ⚠️
cmd/gcp/omniwitness/secret_manager.go 0.00% 30 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (3c58af4) and HEAD (d914c94). Click for more details.

HEAD has 2 uploads less than BASE
Flag BASE (3c58af4) HEAD (d914c94)
4 2
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #385       +/-   ##
===========================================
- Coverage   51.05%   19.57%   -31.48%     
===========================================
  Files          11       28       +17     
  Lines         903     1977     +1074     
===========================================
- Hits          461      387       -74     
- Misses        374     1526     +1152     
+ Partials       68       64        -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@AlCutter
AlCutter marked this pull request as ready for review August 20, 2025 13:58
@AlCutter
AlCutter requested a review from mhutchinson August 20, 2025 13:58
@AlCutter AlCutter added the enhancement New feature or request label Aug 20, 2025
Comment thread cmd/gcp/omniwitness/main.go Outdated
Comment thread cmd/gcp/omniwitness/README.md Outdated
@AlCutter
AlCutter force-pushed the omnigcp branch 2 times, most recently from 6fe6333 to 5032c6e Compare August 20, 2025 14:20
@AlCutter
AlCutter force-pushed the omnigcp branch 2 times, most recently from 3312f0a to f5b53ab Compare August 20, 2025 14:46
@AlCutter
AlCutter merged commit e7bf327 into transparency-dev:main Aug 20, 2025
6 checks passed
@AlCutter
AlCutter deleted the omnigcp branch August 20, 2025 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants