HADOOP-19962. Support IPv6 CIDR rules in MachineList

[smengcl]

Generated-by: Claude Code (Opus 5.5)

Description of PR

MachineList parses CIDR entries with commons-net SubnetUtils, which supports IPv4 only. Two things fail as a result:

• An IPv6 CIDR entry such as 2001:db8::/32 makes the MachineList constructor throw IllegalArgumentException. A hadoop.proxyuser.*.hosts value or a service level authorization host list with an IPv6 range cannot be loaded.
• An IPv6 client checked against an existing IPv4 CIDR entry makes MachineList.includes(InetAddress) throw IllegalArgumentException instead of returning false. This affects a dual stack cluster with an unchanged IPv4 configuration.

This PR adds prefix matching for IPv6 CIDR entries and makes the range lookup aware of the address family.

• IPv4 CIDR entries still use SubnetUtils, so their results are unchanged.
• An address never matches a CIDR entry of the other address family. The lookup returns false and does not throw.
• IPv4 mapped addresses (::ffff:a.b.c.d): the JDK returns a mapped client address as Inet4Address, so it matches IPv4 entries only. A mapped CIDR entry is rejected at construction because it could never match. It should be written as an IPv4 CIDR.
• An invalid IPv6 CIDR entry is rejected at construction with IllegalArgumentException, as an invalid IPv4 CIDR entry is today. The prefix length must be a number from 0 to 128.
• A CIDR entry with a scope id (fe80::%eth0/64) is rejected. The scope id of a client address is ignored, as it already is for exact IPv6 entries.
• Exact IPv6 entries (not CIDR) already work and are unchanged.

Behavior change to note: on trunk, an IPv6 client checked against a list with an IPv4 CIDR entry failed with an exception, which dropped the RPC connection. Such a client is now evaluated normally. With a blocked host list (*.hosts.blocked) that has only IPv4 ranges, IPv6 clients are no longer rejected as a side effect. Add IPv6 ranges to the blocked list if they should be blocked.

Sub-task of HADOOP-11890.

How was this patch tested?

Added 10 tests to TestMachineList:

• IPv6 CIDR includes and excludes, including an entry with host bits set.
• Boundary addresses of /33 and /61 prefixes, which do not end on a byte boundary.
• /128 and ::/0.
• IPv4 and IPv6 entries in one list.
• Lookups against CIDR entries of the other address family return false and do not throw.
• IPv4 mapped client addresses, and a client address with a scope id.
• Invalid IPv6 CIDR entries are rejected (prefix length out of range, negative or not a number, scope id, IPv4 mapped address, brackets, characters that are not ASCII).
• IPv4 CIDR edge cases (/32, /31, /0) and exact IPv6 entries, to show that existing behavior is kept.

On trunk, 8 of the 10 new tests fail with IllegalArgumentException. The other 2 cover behavior that already works (exact IPv6 entries, rejection of invalid entries).

Added TestServiceAuthorization#testBlockedMachineListWithIPv6Client, which covers the behavior change above through ServiceAuthorizationManager.authorize: an IPv6 client is authorized when the blocked host list has only IPv4 ranges, and is rejected with AuthorizationException once a matching IPv6 CIDR is added to the blocked host list.

mvn test -Dtest='TestMachineList,TestProxyUsers,TestServiceAuthorization,TestFileBasedIPList' in hadoop-common with JDK 17: 58 tests run, 0 failures, 0 errors, 0 skipped.

For code changes:

• Does the title of this PR start with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: Have the integration tests been executed and the endpoint
  declared according to the connector-specific documentation? Note: Automated CI
  testing doesn't cover all cases so manual testing with cloud storage is still
  required.
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

AI Tooling

Contains content generated by Claude Code.

• The PR includes the phrase "Contains content generated by "
  where is the name of the AI tool used.
• My use of AI contributions follows the ASF legal policy
  https://www.apache.org/legal/generative-tooling.html

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

IPv4-mapped IPv6 CIDRs are not rejected as intended, causing the corresponding test to fail.

1 open finding

• Reject IPv4-mapped IPv6 addresses before adding the subnet · New

What changed in this PR

Adds IPv6 CIDR parsing and family-aware matching to MachineList, with expanded IPv4/IPv6 test coverage.

Changes:

• Implements IPv6 prefix matching and validation.
• Prevents cross-family CIDR lookup errors.
• Adds boundary, compatibility, and invalid-input tests.

File	Summary
hadoop-common-project/​hadoop-common/​src/​main/​java/​org/​apache/​hadoop/​util/​MachineList.java	Adds IPv6 CIDR parsing and matching.
hadoop-common-project/​hadoop-common/​src/​test/​java/​org/​apache/​hadoop/​util/​TestMachineList.java	Adds IPv4/IPv6 CIDR and regression coverage.

🧠 Review effort: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[slfan1989]

The implementation looks good to me. +1

I independently ran the four relevant test classes: all 57 tests passed, including the IPv4-mapped CIDR rejection case.

One suggestion: could you add a service-authorization test covering IPv6 clients with IPv4-only and matching IPv6 blocked CIDRs? This would explicitly cover the behavior change documented in the PR.

Let's also wait for the remaining CI checks before merging.

[smengcl]

One suggestion: could you add a service-authorization test covering IPv6 clients with IPv4-only and matching IPv6 blocked CIDRs? This would explicitly cover the behavior change documented in the PR.

Thanks @slfan1989 for taking a look. Added testBlockedMachineListWithIPv6Client

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	50m 24s		trunk passed
+1 💚	compile	18m 2s		trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	compile	18m 40s		trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚	checkstyle	1m 32s		trunk passed
+1 💚	mvnsite	2m 2s		trunk passed
+1 💚	javadoc	1m 27s		trunk passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	javadoc	1m 24s		trunk passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚	spotbugs	3m 18s		trunk passed
+1 💚	shadedclient	36m 50s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 17s		the patch passed
+1 💚	compile	16m 58s		the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	javac	16m 58s		the patch passed
+1 💚	compile	18m 29s		the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚	javac	18m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 28s		the patch passed
+1 💚	mvnsite	2m 2s		the patch passed
+1 💚	javadoc	1m 23s		the patch passed with JDK Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu
+1 💚	javadoc	1m 24s		the patch passed with JDK Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
+1 💚	spotbugs	3m 28s		the patch passed
+1 💚	shadedclient	36m 35s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	23m 40s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 15s		The patch does not generate ASF License warnings.
		244m 31s

Subsystem	Report/Notes
Docker	ClientAPI=1.56 ServerAPI=1.56 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8812/3/artifact/out/Dockerfile
GITHUB PR	#8812
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 93466dd970c9 5.15.0-185-generic #195-Ubuntu SMP Fri Jun 19 17:11:50 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 7219b86
Default Java	Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.12.1+1-1-24.04.4-Ubuntu /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.20.1+1-1-24.04-Ubuntu
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8812/3/testReport/
Max. process+thread count	3105 (vs. ulimit of 10000)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8812/3/console
versions	git=2.43.0 maven=3.9.15 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.1 https://yetus.apache.org

This message was automatically generated.

[smengcl]

Thanks @slfan1989 @joseluisll for the reviews.