From a98b340e82aee0badd0b429e83847695071c70fb Mon Sep 17 00:00:00 2001 From: v1ll4n Date: Tue, 7 Jul 2026 18:08:52 +0800 Subject: [PATCH] fix(decompiler): AtomicReference V-parameter argument cast In JDK method parameter type-arg index table, add AtomicReference branch: compareAndSet/weakCompareAndSet (both params), getAndSet/set/lazySet, and getAndAccumulate/accumulateAndGet (first param) resolve their V parameter to the receiver's type arg. This lets the existing arg-cast path re-emit the source (T) unchecked cast and prevents javac from rejecting a bare Object argument as "Object cannot be converted to T". Load-bearing tests: - classparser/atomic_ref_vparam_test.go - test/cross/atomic_ref_vparam_cast_test.go (commons-lang3 AtomicInitializer) Switch: JDEC_ATOMIC_REF_PARAM_OFF. Fixes commons-lang3 1 case; total defects 97 -> 96, classes 9 -> 8, zero regression. Co-authored-by: Cursor --- TODO.md | 22 +++- classparser/CODEC_TODO.md | 5 +- classparser/atomic_ref_vparam_test.go | 43 +++++++ .../decompiler/core/values/types/generic.go | 30 +++++ .../regression/AtomicRefVParamSeed.class | Bin 0 -> 859 bytes .../regression/AtomicRefVParamSeed.java | 24 ++++ test/cross/atomic_ref_vparam_cast_test.go | 110 ++++++++++++++++++ 7 files changed, 230 insertions(+), 4 deletions(-) create mode 100644 classparser/atomic_ref_vparam_test.go create mode 100644 classparser/testdata/regression/AtomicRefVParamSeed.class create mode 100644 classparser/testdata/regression/AtomicRefVParamSeed.java create mode 100644 test/cross/atomic_ref_vparam_cast_test.go diff --git a/TODO.md b/TODO.md index 0adddcd..180f005 100644 --- a/TODO.md +++ b/TODO.md @@ -8,8 +8,26 @@ > iso 口径的 `cannot find symbol`/`private access` 大多是扁平 `$` 假阳性, 不在此列(见 CODEC_TODO §3)。 > > 数字快照(javac 21, 本机 `~/.m2` 含可选依赖; tree errLines / 缺陷类, 复跑见下方命令): -> codec 0/0 ✅ · gson 0/0 ✅ · jsoup 1/1 · snakeyaml 1/1 · fastjson2 25/14 · guava 27/23 · commons-lang3 12/9 · spring 31/19。（合计 97） -> (本轮修: 方法形参自由类型变量注入(dumper.go 扁平内部类 enclosing-arity 注入块增方法形参扫描, +> codec 0/0 ✅ · gson 0/0 ✅ · jsoup 1/1 · snakeyaml 1/1 · fastjson2 25/14 · guava 27/23 · commons-lang3 11/8 · spring 31/19。（合计 96） +> (本轮修: `AtomicReference` 的 V 形参方法实参补造型(`jdkMethodParamTypeArgIndex` 增 AtomicReference 分支, `JDEC_ATOMIC_REF_PARAM_OFF`)—— 字段 `AtomicReference reference` 的 `get()` 被读进 Object 局部, +> 再回传给 `compareAndSet(V,V)`/`getAndSet(V)`/`set(V)`(descriptor 擦成 `compareAndSet/set(Object,Object)`)时, +> 裸 Object 实参被 javac 按 `AtomicReference.compareAndSet(V,V)` 定型判「Object cannot be converted to T」。 +> 修法: 仿 `jdkMapFamily`/`jdkListFamily` 形参映射族, 在 JDK 形参类型实参索引表补 `AtomicReference` 分支(V 是唯一类型实参, +> 全部 V 形参位映射到 0): `compareAndSet/weakCompareAndSet(V,V)` argc==2 两形参、`getAndSet/set/lazySet(V)` argc==1 形参 0、 +> `getAndAccumulate/accumulateAndGet(V, BinaryOperator)` argc==2 仅形参 0(V), 形参 1 是 operator 不动。 +> `instantiatedParamType` 把 V 形参解析成接收者 T 后, 既有 arg-cast 路径重下 `(T)` unchecked 造型(字节码里该值已流入 V 槽, 保义)。 +> 限 `ntype==1`(裸/非通配参数化 `AtomicReference`), 既有通配符早退(`AtomicReference`)不受影响。承重测试 +> `atomic_ref_vparam_test.go`(AtomicRefVParamSeed) + A/B 交叉 `atomic_ref_vparam_cast_test.go`(commons-lang3 AtomicInitializer)。 +> 治 commons-lang3 AtomicInitializer 1 条, commons-lang3 12→11、缺陷类 9→8, 合计 97→96, 零回归。) +> (本轮调查 jsoup/snakeyaml 清零未果, 记录潜伏链供后续: ① jsoup `XmlTreeBuilder.insert(Token$Comment)` 的「`Comment var3 = var2; ...; +> var3 = new XmlDeclaration();` 首声明窄化致兄弟再赋失败(`XmlDeclaration cannot be converted to Comment`)」**单点可修**—— +> 跨类兄弟臂合并已把 slot ref 扩宽到 LUB(Node), 但首声明 RHS 仍是窄臂类型 Comment; 修法: 在 AssignStatement 首声明处见 ref 被合并 helper +> 扩宽(经 `PhiWidened` 标记证)时采纳 slot 类型。**但**: 该修解锁 jsoup `QueryParser.combinator` **潜伏 7 条** definite-assignment +> (var5_1/var5_2 单分支赋值后无条件用, 属 T4b computeIfAbsent 惯用式, 之前被 XmlTreeBuilder 类型错遮蔽)——原 1 错变 7 错, 净 -6。 +> ② snakeyaml `SafeConstructor.createNumber` 即 **T4b 文档化的 special-project**(try/catch 合流 `var6` 只在 catch 赋值, +> `return var6` 未初始化)—— 核心数据流 + 全量 A/B, 留专项。结论: 两单点清零均被 T4b 遮蔽依赖阻塞, 须先解 T4b(单用折叠前判 slot 下游 load +> 或合流 load 建 phi)才能净清零; 本轮先回退 jsoup 修保 96 合计、零回归, AtomicReference 修复保留。) +> (上一轮修: 方法形参自由类型变量注入(dumper.go 扁平内部类 enclosing-arity 注入块增方法形参扫描, > `JDEC_INNER_METHODPARAM_TYPEVAR_INJECT_OFF`)—— 静态泛型方法内的匿名类捕获方法的类型变量, javac 发出的 > 该匿名类 Signature **不带** `<...>` 形参段(只列自由变量引用: guava `Futures$2` 的类签名是 > `Ljava/lang/Object;LFuture;`, 无 `` 前缀), 于是 O 从父类 `Future` 被识别为自由变量并声明, diff --git a/classparser/CODEC_TODO.md b/classparser/CODEC_TODO.md index cdf4a36..bfaacde 100644 --- a/classparser/CODEC_TODO.md +++ b/classparser/CODEC_TODO.md @@ -29,13 +29,13 @@ |---|---:|---:|---:|---:|---| | **commons-codec** 1.15 | 106 | **0** | **0** | 0 | ✅ **完整往返**(107/107 verify + 调用差分逐字节一致) | | **gson** 2.8.9 | 195 | **0** | **0** | 0 | ✅ **完整往返**(199/199 verify) | -| **commons-lang3** 3.12.0 | 345 | 11 | 18 | 0 | 泛型擦除长尾 | +| **commons-lang3** 3.12.0 | 345 | 8 | 11 | 0 | 泛型擦除长尾 | | **jsoup** 1.10.2 | 238 | 1 | 1 | 0 | 单类长尾 | | **snakeyaml** 2.2 | 231 | 1 | 1 | 0 | definite-assignment 单点 | | **spring-core** 5.3.27 | 978 | 29 | 55 | 0 | 泛型擦除造型 + 三元 LUB + bool/int 槽位长尾 | | **fastjson2** 2.0.43 | 681 | 15 | 32 | 0 | 泛型擦除 + 槽位复用长尾 | | **guava** 28.2-android | 1892 | 20 | 28 | 0 | 泛型擦除/边界 + 扁平内部类长尾 | -| **合计** | | **77** | **135** | **0** | 类级干净率 **96.6%**(2175/2252) | +| **合计** | | **76** | **134** | **0** | 类级干净率 **96.6%**(2176/2252) | **codec 与 gson 已证北极星全链路**(承重于 `test/cross/jar_roundtrip_test.go`): `decompile → javac 重编译(0 error) → archive/zip 重打包 → java -Xverify:all 逐类加载校验全通过`; codec 更经调用差分(Base64 / Hex / MD5 / SHA-256)与原始 jar 逐字节一致。 @@ -183,6 +183,7 @@ iso 把每个扁平单元单独编译, 以下失败是方法学产物, 在 tree( | `JDEC_BOOL_ACCUM_SLOT_SPLIT_OFF` | 布尔累加器复用不相交 int 循环计数器槽位拆分(`reachingBoolAccumulatorSlotSplit`, 与 `reachingBoolReturnSlotSplit`/`reachingBoolFieldSlotSplit` 同族): `boolean flag=false; flag|=someZcall()` 的 `iconst_0` 初值被 AssignVarGuarded 见作 int 类别、续用了停在同槽表项里的(已死)int 循环计数器 ref, 合并两不相交活跃区; 该槽随后经 `flag|=Zcall` 自累加(`ior/iand/ixor` 回存同槽, 兄弟操作数为 Z 返回调用)定型 boolean, 早循环渲染成 `boolean` 的 V 形参方法实参造型(`jdkMethodParamTypeArgIndex` 增 AtomicReference 分支): 字段/局部 `AtomicReference` 的 `get()` 读进 Object 局部后回传给 `compareAndSet(V,V)`/`weakCompareAngeSet(V,V)`/`getAndSet(V)`/`set(V)`/`lazySet(V)`/`getAndAccumulate(V,BinaryOperator)`/`accumulateAndGet(V,BinaryOperator)`(descriptor 擦成 Object), 裸 Object 实参被 javac 按 `AtomicReference.m(V)` 定型判「Object cannot be converted to T」。V 是唯一类型实参, 全部 V 形参位映射到 0(compareAndSet 两形参、单参方法的形参 0、accumulator 的形参 0; operator 形参不动)。`instantiatedParamType` 把 V 形参解析成接收者 T 后, 既有 arg-cast 路径重下 `(T)` unchecked 造型。限 `ntype==1`(裸/非通配参数化), 既有通配符早退(`AtomicReference`)不受影响。修 commons-lang3 `AtomicInitializer.get`(commons-lang3 12→11、缺陷类 9→8) | ### 结构化 / pop / switch / 枚举 / 注解 | 开关 | 作用域 | diff --git a/classparser/atomic_ref_vparam_test.go b/classparser/atomic_ref_vparam_test.go new file mode 100644 index 0000000..b94f1d6 --- /dev/null +++ b/classparser/atomic_ref_vparam_test.go @@ -0,0 +1,43 @@ +package javaclassparser + +import ( + "os" + "strings" + "testing" +) + +// TestAtomicRefVParamArgCastIsLoadBearing pins the AtomicReference V-parameter argument cast +// (jdkMethodParamTypeArgIndex AtomicReference branch; kill-switch JDEC_ATOMIC_REF_PARAM_OFF). A field +// `AtomicReference` whose get() is read into an Object-typed local, then passed back to +// `compareAndSet(V, V)` / `getAndSet(V)` / `set(V)` (descriptor erased to Object), renders as a bare +// Object argument and javac rejects "Object cannot be converted to T" (commons-lang3 +// AtomicInitializer `this.reference.compareAndSet(null, var1)`). With the fix ON the call site +// re-emits the source's unchecked `(T)` cast. With the kill-switch the bare argument returns -- the +// exact recompile blocker the fix removes. +func TestAtomicRefVParamArgCastIsLoadBearing(t *testing.T) { + data, err := os.ReadFile("testdata/regression/AtomicRefVParamSeed.class") + if err != nil { + t.Fatalf("read seed: %v", err) + } + + os.Unsetenv("JDEC_ATOMIC_REF_PARAM_OFF") + on, err := Decompile(data) + if err != nil { + t.Fatalf("decompile (fix ON) failed: %v", err) + } + // The V-typed argument (var2) must carry a `(T)` cast at the compareAndSet call site. + if !strings.Contains(on, "this.reference.compareAndSet((T)(null),(T)(var2))") { + t.Errorf("fix ON: expected `(T)` casts on the AtomicReference V-args, got:\n%s", on) + } + + t.Setenv("JDEC_ATOMIC_REF_PARAM_OFF", "1") + off, err := Decompile(data) + if err != nil { + t.Fatalf("decompile (fix OFF) failed: %v", err) + } + // With the kill-switch the bare uncast argument returns -- reproducing the recompile blocker. + if !strings.Contains(off, "this.reference.compareAndSet(null,var2)") || + strings.Contains(off, "compareAndSet((T)") { + t.Errorf("fix OFF: expected the bare uncast argument (kill-switch not load-bearing), got:\n%s", off) + } +} diff --git a/classparser/decompiler/core/values/types/generic.go b/classparser/decompiler/core/values/types/generic.go index 341c9bb..28a9e7e 100644 --- a/classparser/decompiler/core/values/types/generic.go +++ b/classparser/decompiler/core/values/types/generic.go @@ -374,6 +374,36 @@ func jdkMethodParamTypeArgIndex(rawClass, method string, argc, paramIndex, ntype if (method == "set" || method == "add") && argc == 2 && ntype == 1 && paramIndex == 1 && jdkListFamily[rawClass] && os.Getenv("JDEC_LIST_SET_PARAM_OFF") == "" { return 0 } + // AtomicReference: the V-typed value-parameter methods whose descriptor erases V to Object. The + // canonical case is `compareAndSet(V, V)` / `weakCompareAndSet(V, V)` (argc 2, both params V) and + // `getAndSet(V)` / `set(V)` / `lazySet(V)` (argc 1, param 0 V); the accumulator pair + // `getAndAccumulate(V, BinaryOperator)` / `accumulateAndGet(V, BinaryOperator)` carry V only at + // param 0 (param 1 is the operator). An Object-typed argument (a `reference.get()` read typed Object) + // flows in without the source's `(V)` cast and javac -- re-resolving against the generic signature -- + // rejects it ("Object cannot be converted to T"; commons-lang3 AtomicInitializer + // `this.reference.compareAndSet(null, var1)` where reference is `AtomicReference`). V is the sole + // type arg, so all qualifying params map to 0. Restricted to ntype==1 (a bare or + // non-wildcard-parameterized AtomicReference); the existing wildcard gate in + // InstantiateJDKMethodParam already returns nil for `AtomicReference`. Kill-switch + // JDEC_ATOMIC_REF_PARAM_OFF. + if rawClass == "java.util.concurrent.atomic.AtomicReference" && ntype == 1 && + os.Getenv("JDEC_ATOMIC_REF_PARAM_OFF") == "" { + switch method { + case "compareAndSet", "weakCompareAndSet", "weakCompareAndSetPlain": + if argc == 2 && (paramIndex == 0 || paramIndex == 1) { + return 0 + } + case "getAndSet", "set", "lazySet": + if argc == 1 && paramIndex == 0 { + return 0 + } + case "getAndAccumulate", "accumulateAndGet": + // (V, BinaryOperator): only the leading V param is the receiver's type arg. + if argc == 2 && paramIndex == 0 { + return 0 + } + } + } return -1 } diff --git a/classparser/testdata/regression/AtomicRefVParamSeed.class b/classparser/testdata/regression/AtomicRefVParamSeed.class new file mode 100644 index 0000000000000000000000000000000000000000..09efd3d206e6f8976406b50f9c85a37d2268182b GIT binary patch literal 859 zcmb7CO>fgc5Pf5ZI&oZ@kWxwuh4P`qX}|(;!cn9OiBkgzN0s2V*=!@1#EvYRAH%Nz z30eUXS0sKDLd-gCKyofU?9A@$%$xUi=hvTaKL9+zeH#W$3nd4autKOEi}%8th-mD+ z93Cr~5>~ol6sA3b*>VpplnHxzQkI4jPsWkVl0-$RC(?KtN^d`Ruho&_s8lvAm=#pv zSXgy%8Eb^bBIodxNW?TyYDB0c=QV@}2N!7J>mGFnSI}URW5t};Tkag;KP7$@S5ddH zLD>F>ew0Nr4dciQve|4BszifrI=GH4!m5m?Gm)tMXcTDJW~+!=jJR)EOc0JEk!Fb^ zJidsS?qJ~ed_w72JmT85gD_GrvguGI12LR%rZ++6a-5A~~KmuJD?*(3fho(A^Yd@^u@7wI8n zdy^v_n7jjO?NdLr6V&>hpV-;^ zigiL?hp2OcYadWuTz}+b0~&IR+k;7!QGE#c-;_bW(X=>wPI`B-yMw!Y HZ{Xfv8dTGl literal 0 HcmV?d00001 diff --git a/classparser/testdata/regression/AtomicRefVParamSeed.java b/classparser/testdata/regression/AtomicRefVParamSeed.java new file mode 100644 index 0000000..83c477b --- /dev/null +++ b/classparser/testdata/regression/AtomicRefVParamSeed.java @@ -0,0 +1,24 @@ +// Regression seed for the AtomicReference V-parameter argument cast +// (jdkMethodParamTypeArgIndex AtomicReference branch; kill-switch JDEC_ATOMIC_REF_PARAM_OFF). +// +// A field `AtomicReference reference` whose `get()` is read into an Object-typed local, then +// passed back to `compareAndSet(V, V)` / `getAndSet(V)` / `set(V)` (descriptor erased to +// compareAndSet/set(Object,Object)/set(Object)), renders as a bare Object argument and javac rejects +// "Object cannot be converted to T" (commons-lang3 AtomicInitializer +// `this.reference.compareAndSet(null, var1)`). The argument already flowed into the V slot in +// bytecode, so re-emitting the source's unchecked `(T)` cast is behaviour-preserving. +public class AtomicRefVParamSeed { + private final java.util.concurrent.atomic.AtomicReference reference = + new java.util.concurrent.atomic.AtomicReference<>(); + + public T get(java.util.function.Supplier init) { + Object var1 = this.reference.get(); + if (var1 == null) { + var1 = init.get(); + if (!this.reference.compareAndSet(null, (T) var1)) { + var1 = this.reference.get(); + } + } + return (T) var1; + } +} diff --git a/test/cross/atomic_ref_vparam_cast_test.go b/test/cross/atomic_ref_vparam_cast_test.go new file mode 100644 index 0000000..25812ff --- /dev/null +++ b/test/cross/atomic_ref_vparam_cast_test.go @@ -0,0 +1,110 @@ +package cross + +// 承重测试:「AtomicReference 的 V 形参方法(compareAndSet/getAndSet/set...)缺 (V) 实参造型」治本 +// (CODEC_TODO 泛型擦除·JDK 方法形参造型族)。 +// +// 真实 commons-lang3 AtomicInitializer.get(): +// Object var1 = this.reference.get(); // reference 是 AtomicReference, get() 返 Object +// if (var1 == null) { +// var1 = this.initialize(); +// if (!(this.reference.compareAndSet(null, var1))) { // 源码带 (T) 造型 +// var1 = this.reference.get(); +// } +// } +// return (T) var1; +// compareAndSet 的形参在字节码里被擦成 Object, 反编译不补造型; javac 却按 +// AtomicReference.compareAndSet(V, V) 定型判 "Object cannot be converted to T"。治本 +// (jdkMethodParamTypeArgIndex 增 AtomicReference 分支, 使 instantiatedParamType 把 +// compareAndSet 形参 0/1 解析为接收者 V, 既有 arg-cast 路径遂重下 (T) 造型)。 +// JDEC_ATOMIC_REF_PARAM_OFF=1 关掉治本必复现。 + +import ( + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + + classparser "github.com/yaklang/javajive/classparser" +) + +// atomicRefVParamErrCount decompiles AtomicInitializer under the kill-switch, recompiles it against +// the jar, and counts the AtomicReference V-parameter cast defect: the Object-typed argument to +// `reference.compareAndSet(...)` passed without the `(T)` cast, so javac rejects "Object cannot be +// converted to T". +func atomicRefVParamErrCount(t *testing.T, jarPath string, killOff bool) int { + t.Helper() + const sw = "JDEC_ATOMIC_REF_PARAM_OFF" + const entry = "org/apache/commons/lang3/concurrent/AtomicInitializer.class" + prev, had := os.LookupEnv(sw) + if killOff { + os.Setenv(sw, "1") + } else { + os.Unsetenv(sw) + } + defer func() { + if had { + os.Setenv(sw, prev) + } else { + os.Unsetenv(sw) + } + }() + + jfs, err := classparser.NewJarFSFromLocal(jarPath) + if err != nil { + t.Fatalf("open jar: %v", err) + } + src, err := jfs.ReadFile(entry) + if err != nil { + t.Fatalf("read %s: %v", entry, err) + } + dir := t.TempDir() + dst := filepath.Join(dir, "AtomicInitializer.java") + if err := os.WriteFile(dst, src, 0o644); err != nil { + t.Fatalf("write %s: %v", dst, err) + } + + javac := lookJavac(t) + args := append(append([]string{}, javacLocaleArgs...), + "-encoding", "UTF-8", "--release", "8", "-nowarn", "-Xmaxerrs", "100000", + "-cp", jarPath, "-d", t.TempDir(), dst) + out, _ := exec.Command(javac, args...).CombinedOutput() + + n := 0 + for _, line := range strings.Split(string(out), "\n") { + if !strings.Contains(line, ": error:") || !strings.Contains(line, "AtomicInitializer.java") { + continue + } + if strings.Contains(line, "Object cannot be converted to T") { + n++ + } + } + return n +} + +// TestAtomicRefVParamCastIsLoadBearing pins commons-lang3 AtomicInitializer.get: an Object-typed +// argument to `reference.compareAndSet(...)` (receiver recovered as AtomicReference) must be wrapped +// in a `(T)` cast, compareAndSet's value parameter resolved to the receiver's V via the AtomicReference +// parameter table. Disabling the fix via the kill-switch must reintroduce the "Object cannot be converted +// to T" error. +func TestAtomicRefVParamCastIsLoadBearing(t *testing.T) { + lookJavac(t) + jarPath := resolveJar(jarSpecs["commons-lang3"].relPath) + if jarPath == "" { + t.Skip("commons-lang3 jar not found under ~/.m2; skipping") + } + + on := atomicRefVParamErrCount(t, jarPath, false) // fix ON + off := atomicRefVParamErrCount(t, jarPath, true) // fix OFF (kill-switch) + t.Logf("AtomicInitializer AtomicReference V-param errors: ON=%d OFF=%d", on, off) + + if off == 0 { + t.Fatalf("kill-switch did not reproduce the defect: OFF=%d (expected > 0)", off) + } + if on >= off { + t.Errorf("fix is NOT load-bearing: ON=%d OFF=%d (ON must be strictly fewer)", on, off) + } + if on != 0 { + t.Errorf("fix did not clear the AtomicReference V-param error: ON=%d (want 0)", on) + } +}