BigQuery TAP support

[dspeck1]

Add tap support for BigQuery. Includes the geometric conversions from TAP to BigQuery geometric functions. The TAP server example for running this will be provided in a different repository.

[pdowler]

Can you also add this to the .github/workflows/gradle.yml?

Below cadc-tap-server-oracle (which I just fixed so you will need to pull main) and in that same style.

We haven't got to it yet in tap.git (technical debt), but in many repos we also run the checkstyleMain target in CI so if checkstyle passes for your code you can add that and the PR looks even better. Something like this:

    - name: build and test cadc-tap-server-bigquery
      run: cd cadc-tap-server-bigquery && ../gradlew --info clean build javadoc checkstyleMain install

[pdowler]

A few comments on the build

[dspeck1]

Hi @pdowler - is there anything else needed on this PR? I believe the build steps were addressed.

[pdowler]

I started reviewing the details but got side tracked. The ball is in my court now.

…

-- Patrick Dowler Canadian Astronomy Data Centre Victoria, BC, Canada

On Thu, 6 Feb 2025 at 06:09, dspeck1 ***@***.***> wrote: Hi @pdowler <https://github.com/pdowler> - is there anything else needed on this PR? I believe the build steps were addressed. — Reply to this email directly, view it on GitHub <#172 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADIEM7U4DPJHGOVY72DRWED2ONULHAVCNFSM6AAAAABNXAPIMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMZZHEZTIMJWGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[pdowler]

Could you add some details here to help a developer that wants to build a service using the library and a BigQuery backend?

What JDBC driver do they need to install and configure? (preferrably: class name and maven dependency info to add as a runtime dependency)

Do they put the tap_schema tables and content into BigQuery or a side db? (cadc-tap-server does not currently support them being in separate DBs but we are working on that)

Is it expected to support the optional TAP UPLOAD feature? without a custom ca.nrc.cadc.tap.db.DatabaseDataType it seems like that won't work (at least for all data types)

It looks like this is the ADQL -> {big query dialect} conversion. Even for that, I would provide a single class that encapsulates all of that.

As an example, the youcat module in this repo is a fully functioning TAP service for PostgreSQL. In the src/main/resources you will see the config file for various plugins needed to make that work and some of those are in youcat, some are in cadc-tap-server-pg (or other libs), and some are the defaults from cadc-tap-server. Admittedly, those might not have the optimal organization as some things in youcat could possibly be moved up to the pg specific lib, but it shows what's needed.

Here, it would be good to document the provided implementation classes (like the implementation of TapQuery interface that encapsulates the ADQL query processing) and which DatabaseDataType implementation must/should be used, and any others as outlined in that example PluginFactory.properties

[pdowler]

Without a Format object for these column data types, any query that selects a point, polygon, etc will fail to write output. I think this is incomplete and was intended to be added but not?

[pdowler]

We have updated builds (../opencadc.gradle) to work with gradle 6-8 so

• merge master in if you haven't done that recently
• plugins: java-library, maven-publish, checkstyle
• do not use the old maven plugin

The included ../opencadc.gradle configures the maven-publish plugin to do the old install task so usage is the same.

Also, below you can update sourceCompatibility to 11... probably need to by now.