Skip to content

chore(hive): vendor Hive 3.1 metastore + fb303 Thrift IDL #694

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MisterRaindrop wants to merge 1 commit into
base: main
Choose a base branch
from
+2,393 −0
Open

chore(hive): vendor Hive 3.1 metastore + fb303 Thrift IDL #694

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/.licenserc.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,5 +31,6 @@ header:
- 'requirements.txt'
- 'src/iceberg/util/murmurhash3_internal.*'
- 'src/iceberg/test/resources/**'
- 'third_party/**'
Copy link
Copy Markdown
Member

@wgtmac wgtmac Jun 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we rename third_party to thirdparty which is more widely used?


comment: on-failure
5 changes: 5 additions & 0 deletions NOTICE
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,8 @@ Copyright 2024-2026 The Apache Software Foundation

This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).

Third-party Thrift IDLs vendored under third_party/hive_metastore/ are
Copy link
Copy Markdown
Member

@wgtmac wgtmac Jun 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, ASF projects are exempted here. cc expert @jbonofre

copies of upstream Apache projects (Apache Hive, Apache Thrift) and are
distributed under the Apache License, Version 2.0. See
third_party/hive_metastore/NOTICE for attribution details.
26 changes: 26 additions & 0 deletions third_party/hive_metastore/NOTICE
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Apache Iceberg C++ — Vendored Hive Metastore Thrift IDL
Copyright 2026 The Apache Software Foundation

This directory contains vendored Thrift interface definitions used to
generate the C++ Hive Metastore (HMS) client consumed by the
`iceberg_hive` library. The interface files are sourced verbatim from
upstream Apache projects (only trailing whitespace and final newlines
were normalized by the repository's pre-commit hooks) and remain under
the Apache License, Version 2.0 (see LICENSE in the repository root).

Vendored sources
================

* hive_metastore.thrift
Apache Hive 3.1 standalone-metastore.
Copy link
Copy Markdown
Member

@wgtmac wgtmac Jun 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this specific version? How do we want to upgrade or maintain multiple versions in the future?

Copy link
Copy Markdown
Contributor Author

@MisterRaindrop MisterRaindrop Jun 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I selected Hive 3.1.3 because it is a mature and widely deployed HMS
version. I also considered the precedent from iceberg-rust, which maintains a single
client generated from a Hive 2.3 IDL and integration-tests it against Hive 3.1.3.

For future maintenance, I propose keeping a single vendored IDL pinned to an
immutable Hive release tag or commit, rather than maintaining separate generated
clients for each Hive version. The implementation should use RPCs shared across the
supported versions whenever possible.

If future Hive releases introduce incompatible RPC changes, we can add narrowly
scoped runtime adapters or fallback logic, allowing one build of iceberg-cpp to
support multiple Hive versions. We should validate and document the supported
versions through a CI compatibility matrix before claiming compatibility.

Source: https://github.com/apache/hive/blob/branch-3.1/standalone-metastore/src/main/thrift/hive_metastore.thrift
Copyright (c) The Apache Software Foundation.

* share/fb303/if/fb303.thrift
Apache Thrift `contrib/fb303` (the Facebook fb303 management
service IDL, transitively included by hive_metastore.thrift).
Source: https://github.com/apache/thrift/blob/master/contrib/fb303/if/fb303.thrift
Originally developed at Facebook, Inc. and contributed to the
Apache Software Foundation under the Apache License, Version 2.0.

Both files retain their original Apache 2.0 license headers in-place.
Loading
Loading