Skip to content

Script to automatically update all library versions #65

Open
Open
Script to automatically update all library versions#65
Labels
P3enhancementNew feature or request

Description

@cpovirk
cpovirk
opened on Apr 22, 2019

Similar to Maven commands like:

And just to spam more keywords, here are some Maven commands that display updates but don't make the changes:

  • mvn versions:display-property-updates
  • mvn versions:display-dependency-updates

I poked around the web and found some a project for keeping Bazel deps up to date, but it has its own system that might not play well with bazel-common:
https://github.com/johnynek/bazel-deps

I notice that it's mentioned in:
https://github.com/bazelbuild/BUILD_file_generator#third-party-maven-dependencies

[edit: Years later, I had Gemini whip up a script in a local bumpdeps branch. I used it to create #291.]

Activity

  1. ronshapiro commented on Apr 22, 2019

    @ronshapiro
    ronshapiro
    on Apr 22, 2019
    Contributor

    We could decide to do something with this: https://blog.bazel.build/2019/03/31/rules-jvm-external-maven.html. I'd like to figure out what it does with dependency conflicts first to make sure.

    Then we can install that with a private repository name and then wire up processors + exports manually

  2. jin commented on May 23, 2019

    @jin
    jin
    on May 23, 2019
    Member

    @ronshapiro it has the resolution logic of Coursier. I can imagine instantiating a custom maven_install for bazel-common, like

    maven_install(
    name = "bazel_common_maven",
    artifacts = # bazel common artifacts,
    repositories = # repos,
)

    And then downstream libraries will depend on @bazel_common_maven//:com_google_guava_guava and @bazel_common_maven//:junit_junit, for example. They can also declare additional dependencies using their own maven_install declarations, which will be namespaced by the repository name.

    Let me know if you have other questions or suggestions for rules_jvm_external.

  3. cpovirk
    added
    enhancementNew feature or request
    P3
    on Jul 25, 2019

  4. cpovirk commented on Jan 8, 2020

    @cpovirk
    cpovirk
    on Jan 8, 2020
    MemberAuthor

    Another thing for us to consider is using the Google Cloud Platform Libraries BOM (Maven central, GitHub). It provides a set of libraries that have been statically checked for linkage errors. This could help us avoid errors like this one (fix) (and maybe there was another error that required this?).

    If I did my testing right, then the BOM doesn't yet cover everything that we'd like it to. But maybe we can help add some libraries, or at least we can use the BOM as our own starting point.

    [edit: The commands below are my attempt to generate a list of libraries that bazel-common includes but the BOM does not specify a version for -- neither explicitly (by listing them) nor implicitly (by listing a library that depends on them).]

    $ ( tmp && mvn help:effective-pom -f $HOME/.m2/repository/com/google/cloud/libraries-bom/3.3.0/libraries-bom-3.3.0.pom -Doutput=/tmp/wholeep && ( echo "plugins {"; echo "    id 'java-library'"; echo "}"; echo "repositories {"; echo "    jcenter()"; echo "}"; echo "dependencies {"; xmlstarlet sel -t -m '/_:project/_:dependencyManagement/_:dependencies/_:dependency' -v _:groupId -o : -v _:artifactId -o : -v _:version -n /tmp/wholeep | sort | prepend "    implementation '" | append "'" ; echo '}' ) > build.gradle && $HOME/gradle-6.0.1/bin/gradle :dependencies | grep -v '[-]>' | grep -o '[^: ]*:[^: ]*:[^ ]*' | sort -u | grep -Fv '[' > /tmp/bomversions )

$ ( cd $HOME/clients/bazel-common-gray/bazel-common/ && bazel query --output=xml 'attr(tags, maven_coordinates.*, kind(java_import, deps(//...)))'  | grep -oP 'maven_coordinates=\K[^:]*:[^:]*:[^:"]*' | sort > /tmp/bazelcommonversions )

$ comm -23 <(grep -o '^[^: ]*:[^: ]*' /tmp/bazelcommonversions) <(grep -o '^[^: ]*:[^: ]*' /tmp/bomversions)
com.google.auto:auto-common
com.google.auto.factory:auto-factory
com.google.auto.service:auto-service
com.googlecode.java-diff-utils:diffutils
com.google.common.inject:inject-common
com.google.errorprone:error_prone_annotation
com.google.errorprone:error_prone_check_api
com.google.errorprone:javac-shaded
com.google.googlejavaformat:google-java-format
com.google.guava:guava-beta-checker
com.google.testing.compile:compile-testing
com.google.truth.extensions:truth-java8-extension
com.google.truth:truth
com.squareup:javapoet
io.grpc:grpc-all
javax.annotation:jsr250-api
javax.enterprise:cdi-api
javax.inject:javax.inject-tck
log4j:log4j
net.bytebuddy:byte-buddy
net.bytebuddy:byte-buddy-agent
net.ltgt.gradle.incap:incap
net.ltgt.gradle.incap:incap-processor
org.apache.ant:ant
org.apache.ant:ant-launcher
org.apache.bcel:bcel
org.apache.logging.log4j:log4j-api
org.apache.logging.log4j:log4j-core
org.apache.maven:maven-artifact
org.apache.maven:maven-model
org.apache.maven:maven-plugin-api
org.checkerframework:checker-qual
org.checkerframework:dataflow
org.checkerframework:javacutil
org.codehaus.plexus:plexus-classworlds
org.codehaus.plexus:plexus-component-annotations
org.codehaus.plexus:plexus-utils
org.eclipse.sisu:org.eclipse.sisu.inject
org.eclipse.sisu:org.eclipse.sisu.plexus
org.mockito:mockito-core
org.objenesis:objenesis
org.ow2.asm:asm
org.ow2.asm:asm-commons
org.ow2.asm:asm-tree
org.pantsbuild:jarjar

  5. cpovirk commented on Jan 17, 2020

    @cpovirk
    cpovirk
    on Jan 17, 2020
    MemberAuthor

    I just saw 842dd97, which fixes a linkage error, fly by.

    (One thing worth noting from that is that some of the artifacts in our list are build tools, not "traditional dependencies" in the narrower sense. If we're lucky, we can easily find compatible versions of all those artifacts, build tools and "traditional dependencies" both. But if not, it would be sufficient to find a set of compatible versions for each individual tool (in addition, of course, to the set for all the "traditional dependencies").)

  6. cpovirk commented on Oct 8, 2020

    @cpovirk
    cpovirk
    on Oct 8, 2020
    MemberAuthor

    Another linkage error fixed in 6db2b87.

  7. cpovirk commented on Jun 1, 2021

    @cpovirk
    cpovirk
    on Jun 1, 2021
    MemberAuthor

    The even easier thing would be automatic updates, like we get from Dependabot for most of our repos.

    As of this writing, Dependabot doesn't support Bazel, but RenovateBot does. Note that RenovateBot does not yet support maven_install, but we are not using that yet.

    I see a smallish number of renovate.json files in the third-party section of our internal repository. But I see some evidence that it's used from almost all repos under https://github.com/googleapis, and I see that Google's GitHub admins are willing to grant the needed permissions to the RenovateBot app. So this seems doable.

  8. cpovirk commented on Sep 25, 2024

    @cpovirk
    cpovirk
    on Sep 25, 2024
    MemberAuthor

    I suspect that this will essentially happen automatically if we switch to rules_jvm_external or whatever the most standard thing is nowadays.

  9. cpovirk commented on Dec 8, 2025

    @cpovirk
    cpovirk
    on Dec 8, 2025
    MemberAuthor

    (We recently set up Dependabot, but that supports only bumping things like rules_java, not our actual Java dependencies.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions