google-cloud-java: create a workflow to run `librarian generate -all` and create issue on failure

[JoeWang1127]

Create a workflow in google-cloud-java to run librarian generate -all and commit code change in eligible pull request, similar to https://github.com/googleapis/google-cloud-java/blob/main/.github/workflows/hermetic_library_generation.yaml

The tools used in the workflow:

• librarian, the version from librarian.yaml
• sources (googleapis), commit SHA from librarian.yaml
• protoc, grpc plugin, the version from librarian.yaml
• gapic-generator-java, built from the HEAD of google-cloud-java, we need librarian install to support this, tracked via java: add support for librarian install #5154.

Initially, we will create this workflow as a presubmit postsubmit check, it will run librarian generate -all and fail if any code change happens in the branch. This can verify the generation result is the same among hermetic build and librarian.

After running the check for a week, we should add to workflow to create issue on failure. we will remove hermetic build workflow and update the presubmit check to commit code change. (tracked in #6219)

For reference, in google-cloud-java, this job builds an hermetic build image when a commit is pushed to main and a workflow runs the latest image for library generation in eligible pull requests.

[JoeWang1127]

@suztomo we need librarian install to support installing an unreleased version of gapic-generator-java.

/cc @blakeli0 @zhumin8

[suztomo]

librarian install

When thinking about future Dockerfile (go/sdk-librarian-tool-command?resourcekey=0-HrVBqWVEDzMCG7847cEcVQ&tab=t.0#heading=h.7uqd55auupmn), librarian install installing gapic-generator-java would make the gapic-generator-java version in Docker image older than the one in the latest google-cloud-java.

[JoeWang1127]

Maybe we can treat google-cloud-java as a dependency and include the SHA in Dockerfile.

[zhumin8]

Step one: "run librarian generate -all and fail if any code change" is done with googleapis/google-cloud-java#13284

Step two: "update the presubmit check to add a commit for code change from generate -all" remaining.

Thoughts: this check takes about an hour, it's probably not a good idea for presubmit check.
FYI - last hermetic build run take about 1h 40m. Problem is not librarian making it slower.

We have a few options moving forward:

1. Leave this as a postsubmit diff check. This is how Go, Python and Rust are configured.
2. Keep as original proposed: presubmit check that adds a generation commit. (This makes each PR to wait for ~1hour)
3. Develop "change detection" maybe similar to hermetic build.
4. Explore make java generate parallel

Today, Java hermetic build uses this py script to identify what libraries changed in the config.

• It will re-generate all if: global params like generator version, Java parent POM changes.
• Otherwise, it identifies libraries that has config changes (includes new config additions for new libraries) for regeneration
• Also, it analyze googleapis changes, compares between commits, and identifies individual libraries that has proto changes for regeneration.

The above logic is pretty intense, 1) I am not sure how much of these detection we want 2) if we want, I think librarian should own that logic, maybe as a separate command (maybe worth a proposal if we go done this path).

We may want a broader discussion, before that, WDTY? @JoeWang1127

[zhumin8]

From off-line discussion: for short term (migration), let's slit to 2 CIs:

1. Convert this check to postsubmit and create an issue on failure. (This runs on all changes, and help catch anything that deviates from generate result)
2. Add a separate check: run generate -all and then add commit of changes if any. Skip this check if no change to librarian.yaml. This should avoid running this check on most PRs in google-cloud-java.

Let's track 1st in this issue and #6219 tracking for 2nd.