fix(deps): update module github.com/twmb/franz-go/plugin/kprom to v1.5.0 (main)

[renovate-sh-app]

This PR contains the following updates:

Package	Change	Age	Confidence
github.com/twmb/franz-go/plugin/kprom	v1.2.1 → v1.5.0

---

Warning

Some dependencies could not be looked up. Check the Dependency Dashboard for more information.

---

Release Notes

twmb/franz-go (github.com/twmb/franz-go/plugin/kprom)

v1.5.0

Compare Source

===

This release adds a few new APIs, has a few small behavior changes, and has one
"breaking" change.

Breaking changes

The kerberos package is now a dedicated separate module. Rather than
requiring a major version bump, since this fix is entirely at the module level
for an almost entirely unused package, I figured it is okayish to technically
break compatibility for the few usages of this package, when the fix can be
done entirely when go geting.

The gokrb5 library, basically the only
library in the Go ecosystem that implements Kerberos, has a slightly broken
license. Organizations that
are sensitive to this were required to not use franz-go even if they did not
use Kerberos because franz-go pulls in a dependency on gokrb5.

Now, with kerberos being a distinct and separate module, depending on
franz-go only will not cause an indirect dependency on gokrb5.

If your upgrade is broken by this change, run:

go get github.com/twmb/franz-go/pkg/sasl/kerberos@v1.0.0
go get github.com/twmb/franz-go@v1.5.0

Behavior changes

• UnknownTopicRetries now allows -1 to signal disabling the option (meaning
  unlimited retries, rather than no retries). This follows the convention of
  other options where -1 disables limits.

Improvements

• Waiting for unknown topics while producing now takes into account both the
  produce context and aborting. Previously, the record context was only taken
  into account after a topic was loaded. The same is true for aborting buffered
  records: previously, abort would hang until a topic was loaded.

• New APIs are added to kmsg to deprecate the previous Into functions. The
  Into functions still exist and will not be removed until kadm is stabilized
  (see #​141).

Features

• ConsumeResetOffset is now clearer, you can now use NoResetOffset with
  start or end or exact offsets, and there is now the very useful
  Offset.AfterMilli function. Previously, NoResetOffset only allowed starting
  consuming at the start and it was not obvious why. We keep the previous
  default-to-start behavior, but we now allow modifying it. As well, AfterMilli
  can be used to largely replace AtEnd. Odds are, you want to consume all
  records after your program starts even if new partitions are added to a
  topic. Previously, if you added a partition to a topic, AtEnd would miss
  records that were produced until the client refreshed metadata and discovered
  the partition. Because of this, you were safer using AtStart, but this
  unnecessarily forced you to consume everything on program start.

• Custom group balancers can now return errors, you can now intercept commits
  to attach metadata, and you can now intercept offset fetches to read
  metadata. Previously, none of this was possible. I considered metadata a bit
  of a niche feature, but accessing it (as well as returning errors when
  balancing) is required if you want to implement streams. New APIs now exist to
  support the more advanced behavior: PreCommitFnContext, OnOffsetsFetched,
  and GroupMemberBalancerOrError. As well, BalancePlan.AsMemberIDMap now
  exists to provide access to a plan's underlying plan map. This did not exist
  previously because I wanted to keep the type opaque for potential future
  changes, but the odds of this are low and we can attempt forward compatibility
  when the time arises.

• RecordReader now supports regular expressions for text values.

Relevant commits

• a2cbbf8 go.{mod,sum}: go get -u ./...; go mod tidy
• ce7a84f kerberos: split into dedicated module, p1
• e8e5c82 and 744a60e kgo: improve ConsumeResetOffset, NoResetOffset, add Offset.AfterMilli
• 78fff0f and e8e5117 and b457742: add GroupMemberBalancerOrError
• b5256c7 kadm: fix long standing poor API (Into fns)
• 8148c55 BalancePlan: add AsMemberIDMap
• 113a2c0 add OnOffsetsFetched function to allow inspecting commit metadata
• 0a4f2ec and cba9e26 kgo: add PreCommitFnContext, enabling pre-commit interceptors for metadata
• 42e5b57 producer: allow a canceled context & aborting to quit unknown wait
• 96d647a UnknownTopicRetries: allow -1 to disable the option
• 001c6d3 RecordReader: support regular expressions for text values

v1.4.0

Compare Source

===

This release adds a lot of new features and changes a few internal behaviors.
The new features have been tested, but it is possible that a bug slipped by—if
you see one, please open an issue and the bug can be fixed promptly.

Behavior changes

• Promises are now serialized. Previously, promises were called at the end
  of handling produce requests. As well, errors that caused records to fail
  independent of producing could fail whenever. Now, all promises are called in
  one loop. Benchmarking showed that concurrent promises did not really help,
  even in cases where the promises could be concurrent. As well, my guess is that
  most people serialize promises, resulting in more complicated logic punted to
  the users. Now with serializing promises, user code can be simpler.

• The default MetadataMinAge has been lowered from 5s to 2.5s. Metadata
  refreshes internally on retryable errors, 2.5s helps fail records for
  non-existing topics quicker. Related, for sharded requests, we now cache topic
  & partition metadata for the MetadataMinAge. This mostly benefits
  ListOffsets, where usually a person may list both the start and end back to
  back. We cannot cache indefinitely because a user may add partitions outside
  this client, but 2.5s is still helpful especially for how infrequently sharded
  requests are issued.

• Group leaders now track topics that the leader is not interested in
  consuming. Previously, if leader A consumed only topic foo and member B only
  bar, then leader A would not notice if partitions were added to bar. Now, the
  leader tracks bar. This behavior change only affects groups where the members
  consume non-overlapping topics.

• Group joins & leaves now include a reason, as per KIP-800. This will be
  useful when Kafka 3.2 is released.

• Transactions no longer log CONCURRENT_TRANSACTIONS errors at the info
  level. This was a noisy log that meant nothing and was non-actionable. We
  still track this at the debug level.

Features

A few new APIs and options have been added. These will be described shortly
here, and the commits are linked below.

• ConcurrentTransactionsBackoff: a new option that allows configuring the
  backoff when starting a transaction runs into the CONCURRENT_TRANSACTIONS
  error. Changing the backoff can decrease latency if Kafka is fast, but can
  increase load on the cluster.

• MaxProduceRequestsInflightPerBroker: a new option that allows changing the
  max inflight produce requests per broker if you disable idempotency.
  Idempotency has an upper bound of 5 requests; by default, disabling idempotency
  sets the max inflight to 1.

• UnknownTopicRetries: a new option that sets how many times a metadata load
  for a topic can return UNKNOWN_TOPIC_OR_PARTITION before all records
  buffered for the topic are failed. As well, we now use this option more widely:
  if a topic is loaded successfully and then later repeatedly experiences these
  errors, records will be failed. Previously, this limit was internal and was
  only applied before the topic was loaded successfully once.

• NoResetOffset: a new special offset that can be used with
  ConsumeResetOffset to trigger the client to enter a fatal state if
  OffsetOutOfRange is encountered.

• Client.PurgeTopicsFromClient: a new API that allows for completely removing
  a topic from the client. This can help if you regex consume and delete a topic,
  or if you produce to random topics and then stop producing to some of them.

• Client.AddConsumeTopics: a new API that enables you to consume from topics
  that you did not initially configure. This enables you to add more topics to
  consume from without restarting the client; this works both both direct
  consumers and group consumers.

• Client.TryProduce: a new API that is a truly non-blocking produce. If the
  client has the maximum amount of records buffered, this function will
  immediately fail a new promise with ErrMaxBuffered.

• Client.ForceMetadataRefresh: a new API that allows you to manually trigger
  a metadata refresh. This can be useful if you added partitions to a topic and
  want to trigger a metadata refresh to load those partitions sooner than the
  default MetadataMaxAge refresh interval.

• Client.EndAndBeginTransaction: a new API that can be used to have higher
  throughput when producing transactionally. This API requires care; if you use
  it, read the documentation for what it provides and any downsides.

• BlockRebalancesOnPoll and Client.AllowRebalance: a new option and
  corresponding required API that allows for easier reasoning about when
  rebalances can happen. This option can be greatly beneficial to users for
  simplifying code, but has a risk around taking so long that your group member
  is booted from the group. Two examples were added using these options.

• kversion.V3_1_0: the kversion package now officially detects v3.1 and has
  an API for it.

Relevant commits

• PR #​137 and c3fc8e0: add two more goroutine per consumer examples (thanks @​JacobSMoller)
• cffbee7 consumer: add BlockRebalancesOnPoll option, AllowRebalance (commit accidentally pluralized)
• 39af436 docs: add metrics-and-logging.md
• 83dfa9d client: add EndAndBeginTransaction
• d11066f committing: internally retry on some errors when cooperative
• 31f3f5f producer: serialize promises
• e3ef142 txn: move concurrent transactions log to debug level
• 10ee8dd group consuming: add reasons to JoinGroup, LeaveGroup per KIP-800
• 0bfaf64 consumer group: track topics that the leader is not interested in
• e8495bb client: add ForceMetadataRefresh
• c763c9b consuming: add NoResetOffset
• 4e0e1d7 config: add UnknownTopicRetries option, use more widely
• 7f58a97 config: lower default MetadataMinAge to 2.5s
• e7bd28f Client,GroupTransactSession: add TryProduce
• 2a2cf66 consumer: add AddConsumeTopics
• d178e26 client: add PurgeTopicsFromClient
• 336d2c9 kgo: add ConcurrentTransactionsBackoff, MaxProduceRequestsInflightPerBroker
• fb04711 kversion: cut v3.1

v1.3.0

Compare Source

===

This release contains three new features, a few behavior changes, and one minor
bugfix.

For features, you can now adjust fetched offsets before they are used (thanks
@​michaelwilner!), you can now "ping" your cluster to see if the client can
connect at all, and you can now use SetOffsets when consuming partitions
manually. As a somewhat of a feature-ish, producing no longer requires a
context, instead if a context is nil, context.Background is used (this was
added to allow more laziness when writing small unimportant files).

The transactional behavior change is important: the documentation changes are
worth reading, and it is worth using a 2.5+ cluster along with the
RequireStableFetchOffsets option if possible. The metadata leader epoch
rewinding behavior change allows the client to continue in the event of odd
cluster issues.

In kadm, we now return individual per-partition errors if partitions are not
included in OffsetCommit responses. The generated code now has a few more enums
(thanks @​weeco!)

Lastly, as a small bugfix, client.Close() did not properly stop seed brokers.
A previous commit split seed brokers and non-seed brokers internally into two
fields but did not add broker shutdown on the now-split seed broker field.

• e0b520c behavior change kadm: set per-partition errors on missing offsets in CommitOffsets
• 32425df feature client: add Ping method
• a059901 behavior change txns: sleep 200ms on commit, preventing rebalance / new commit
• 12eaa1e behavior change metadata: allow leader epoch rewinds after 5 tries
• 029e655 feature-ish Produce{,Sync}: default to context.Background if no ctx is provided
• eb2cec3 bugfix client: stop seed brokers on client.Close
• 2eae20d feature consumer: allow SetOffsets for direct partition consuming
• pr #​120 feature Add groupopt to swizzle offset assignments before consumption (thanks @​michaelwilner!)

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Enabled.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

Need help?

You can ask for more help in the following Slack channel: #proj-renovate-self-hosted. In that channel you can also find ADR and FAQ docs in the Resources section.

[tempo-ci-app]

🤖 fix-renovate-prs: opened a CI fix → #7491

[tempo-ci-app]

🤖 fix-renovate-prs failed_to_fix: kprom v1.4.0 adds a new buffered_produce_bytes GaugeFunc (vendor/github.com/twmb/franz-go/plugin/kprom/kprom.go:366) that collides with Tempo's own tempo_distributor_buffered_produce_bytes Summary registered in pkg/ingest/writer_client.go:209 — same fully-qualified name, different type/help, so promauto panics on distributor startup and every affected test crashes. A human needs to pick a path (rename Tempo's Summary and update operations/tempo-mixin/runbook.md:493 which references it, pin kprom back to 1.3.x, or add a filtering registerer that drops the kprom metric) — I don't want to rename a user-visible metric or drop the dep bump on my own.

[github-actions]

⚠️ The workflow jobs listed below don't declare a permissions: block and may break when the organization's default GITHUB_TOKEN permissions are restricted to read-only.

Expand for findings

warning[excessive-permissions]: overly broad permissions
  --> ./.github/workflows/deploy-pr-preview.yml:13:3
   |
13 | /   deploy-pr-preview:
14 | |     if: "!github.event.pull_request.head.repo.fork"
15 | |     uses: grafana/writers-toolkit/.github/workflows/deploy-preview.yml@main # zizmor: ignore[unpinned-uses]
16 | |     with:
...  |
31 | |       title: ${{ github.event.pull_request.title }}
   | |                                                    ^
   | |                                                    |
   | |____________________________________________________this job
   |                                                      default permissions used due to no permissions: block
   |
   = note: audit confidence → Medium
   = help: audit documentation → https://docs.zizmor.sh/audits/#excessive-permissions

[renovate-sh-app]

Rebase requested. Renovate is processing this repository now.

[renovate-sh-app]

Rebase failed. Please check the logs or retry.

[tempo-ci-app]

🤖 fix-renovate-prs failed_to_fix: kprom v1.5.0 renamed its client gauge from buffered_produce_bytes_total to buffered_produce_bytes, which now collides with Tempo's own tempo_distributor_buffered_produce_bytes summary in pkg/ingest/writer_client.go:230, causing every distributor startup to panic. A human needs to decide whether to rename Tempo's summary and its _limit gauge (updating operations/tempo-mixin/runbook.md:493 and downstream dashboards) or namespace kprom under a new prefix — both are public-metric breaks that shouldn't be picked by automation.

[tempo-ci-app]

🤖 fix-renovate-prs failed_to_fix: kprom v1.5.0 adds a new buffered_produce_bytes gauge that collides with tempo's Summary of the same fqName tempo_distributor_buffered_produce_bytes (defined in pkg/ingest/writer_client.go:239 and registered against the same tempo_distributor_-prefixed registerer as the kprom metrics in modules/distributor/distributor.go:353,357), causing Prometheus to panic during distributor startup. Next step for a human: rename or drop tempo's Summary and update the operations/tempo-mixin/runbook.md:501 reference — kprom v1.5.0 provides no option to disable the new metric, so the fix is user-visible and requires maintainer judgment.