diff --git a/TODO.md b/TODO.md index 4f21989..0adddcd 100644 --- a/TODO.md +++ b/TODO.md @@ -66,9 +66,12 @@ cat /tmp/jdec-inv/guava.tree.fails.txt ### T1. 泛型擦除缺造型 `Object cannot be converted to T/K/CAP#1`(`incompatible types` 桶, 跨 jar 头号来源) - 已治本多块(返回点向下造型 / JDK·同类·继承·私有方法实参造型 / 统一跨类泛型解析器 / 擦除型类型变量多余 upcast 抑制 / 参数化实参·数组实参造型等, 见 CODEC_TODO §4)。**剩余按根因分**: - **(a) 接收者自身泛型未被传播复原成参数化类型**: 接收者是本类型局部变量/字段而非 `this`(`this.box.put(o)`, box=`Box`)时, 其泛型未沿声明传播复原, 取值点仍是 Object。业界 CFR/Vineflower 亦有此长尾。 + - **本轮实验**: 尝试在 `astore` 点对首声明局部(`Iterator it = map.entrySet().iterator()`)用 `ResolveInstantiatedReturnType` 复原参数化返回定型(`upgradeLocalParamType`), 并扩 `InstantiateJDKMethodReturn` 覆盖 `Map.entrySet/keySet/values`。**回归**: guava A/B delta=-1~-2(ON=28~41 vs OFF=27), 新增 `Iterator> cannot be converted to Iterator>` 一族——参数化后不变型严格性反比 raw 更严。**结论**: T1(a) 与 T1(b) 通配符捕获**深度耦合**, 单点局部泛型传播不可行, 须 T1(a)+T1(b) **协同专项**(通配符接收者整体 raw 造型 + 局部泛型传播联动)。已回退, 留协同专项。 - **(b) 通配符捕获 `CAP#1`(guava)** —— **oracle 实证内在难, 优先级下调**: `this.equivalence.equivalent(a,b)`, 字段 `Equivalence` 捕获 `CAP#1`, 实参 Object 不可造 `(CAP#1)`。`ORACLE_JAR=guava ORACLE_CLASS='base/Equivalence$Wrapper'` 三方全败(真源码用 `(Equivalence)` + `@SuppressWarnings`)。方向(若做): 通配符接收者**整体** `` 造型。 + - **本轮增量**: 新增**通配符上界擦除窄化**(`wildcardExtendsBoundErasure`, kill-switch `JDEC_TYPEVAR_FIELD_WILDCARD_NOCAST_OFF`)——当字段/返回值的通配符是 `? extends ConcreteClass` 且上界擦除与目标对应参数擦除**不同**时(guava `ImmutableMultimap.asMap`: 字段 `ImmutableMap>` → 返回 `ImmutableMap>`), 不补 inconvertible 造型, 改诚实裸 `return this.map`(走 unchecked conversion)。全量零回归(guava/spring/fastjson2/commons-lang3 tree errLines 均持平 27/31/25/12), 渲染更接近 CFR。`? super X` 场景(如 `Comparator`→`Comparator`)不 block, 保留原 unchecked 造型。CAP#1 本身仍内在难(三方 oracle 均败)。 - **(d) 返回点「inference variable X has incompatible bounds」(guava 头号桶, ~13 行: ImmutableEnumMap/Range/Maps/Ordering/Striped/Sets$DescendingSet/MinMaxPriorityQueue$Builder/Tables$TransposeTable 等)** —— **有界类型变量同擦除强转 javac 实测拒绝, 不可用 `parameterizedReturnCast` 治**: 形如 `return ImmutableMap.of(k.getKey(), k.getValue());`(目标 `ImmutableMap`、K/V 或 `C extends Comparable` 有界), javac 对 `of()` 独立推断出 `` 后再与目标合流报「incompatible bounds」。实测 `(ImmutableMap)(ImmutableMap)` **当目标实参有界(K extends Enum / C extends Comparable)时 javac 直接报「不兼容的类型…无法转换」**(见 `parameterizedReturnCast` 注释里「BOUNDED var 被排除」的原因)。真源码用 `@SuppressWarnings` + 不同结构强转, 从擦除字节码无法忠实复原。**根因仍是 (a): 局部变量/字段(如 raw `Map.Entry var1`)的泛型未沿声明传播复原**——只要把 `var1` 复原为 `Map.Entry`, `getKey()/getValue()` 即回到 K/V, 无需强转。故此桶归 (a) 的局部/字段泛型传播, 不要再尝试返回点强转。 - **(c) 装箱/数值**: `int cannot be converted to Integer` 等(**非擦除, 不可盲目造型**), 按 `Integer.valueOf` 修。 + - **本轮评估**: baseline 全量扫描(guava/spring/fastjson2/commons-lang3)**无真正原语→包装类错误**(`int cannot be converted to Integer` 等均不存在)。唯一 `Long cannot be converted to Integer`(fastjson2 ObjectWriterCreatorASM:2381) 实为 T2 槽位复用混淆(var11 槽位先存 Integer 后存 Long), 非 T1(c)。**T1(c) 当前无选靶, 跳过**。 - 复现: ```bash go build -o /tmp/jj ./cmd/javajive @@ -79,6 +82,7 @@ cat /tmp/jdec-inv/guava.tree.fails.txt ### T2. 活跃区间分裂 / 槽位复用类型混淆(fastjson2 `bad operand type` / `unexpected type` 一族) - 表象: 一个字节码 local 槽在**互不相交的活跃区间**里先后承载**不兼容类型**的值(如 `JSONPathFilter$GroupFilter` 的 `var9` 既作 `Iterator` 又当 int 比较), 反编译却合成单一变量名 + 单一声明类型。 - 已治本多族 disjoint 槽(兄弟臂 LUB 合并 / Object 超类臂 / 数组协变父臂 / 布尔字段·返回槽拆分 / 跨作用域孤儿读重放, 见 CODEC_TODO §4)。**残余**: 非布尔子形态须在变量定型/分裂核(`JDEC_LIVEINTERVAL_*`)上按「区间+类型」更激进拆分同槽, 风险高、改动核心, 留专项。 + - **本轮评估**: baseline 非布尔槽位混淆仅 ~3 错误(guava `LocalCache$Segment:72` Object→V、`MapMakerInternalMap$Segment:315` InternalEntry→E、fastjson2 `ObjectWriterCreatorASM:2381` Long→Integer), 占总 ~95 错误 ~3%。非 bool 分裂逻辑复杂度与 bool 版本相当(数百行)且回归风险高, **性价比不足**, 暂不投入, 保留现有 bool 分裂。 - 复现: `ORACLE_JAR=fastjson2 ORACLE_CLASS=JSONPathFilter go test -run TestThirdPartyOracle -v ./test/cross/` ## P1 diff --git a/classparser/CODEC_TODO.md b/classparser/CODEC_TODO.md index b5cd9f7..cdf4a36 100644 --- a/classparser/CODEC_TODO.md +++ b/classparser/CODEC_TODO.md @@ -51,11 +51,14 @@ 1. **泛型擦除缺造型 `Object cannot be converted to T/K/CAP#1`** — **最大杠杆, 跨 jar**。 - 表象: `incompatible types (assignment/return)` 桶(commons-lang3 / fastjson2 / guava 的主桶)。代表: `Object cannot be converted to LinkedHashTreeMap$Node` 一族。 - 根因: 字节码泛型擦除后取值点静态类型是 `Object`, 未补回源码原有的 `(T)` / `(Node)` 向下造型。需沿「接收者参数化类型 + 方法/字段 Signature + 跨类型层级替换」复原精确类型。 - - 现状: 已治本多块(返回点向下造型、JDK/同类/继承/私有方法实参造型、统一跨类泛型解析器 `ResolveInstantiatedParamType`、擦除型类型变量多余 upcast 抑制、参数化实参/数组实参造型等, 见 §4)。**残余**: 接收者自身泛型未被传播复原成参数化类型、通配符捕获 `CAP#1`(不可命名, 三方均败, 属内在难 case)、装箱数值(非擦除, 不可盲目造型)。 + - 现状: 已治本多块(返回点向下造型、JDK/同类/继承/私有方法实参造型、统一跨类泛型解析器 `ResolveInstantiatedParamType`、擦除型类型变量多余 upcast 抑制、参数化实参/数组实参造型等, 见 §4)。**残余**: + - **接收者自身泛型传播(T1a/d)**: 实验验证与 T1(b) 通配符捕获**深度耦合**, 单点局部泛型传播会因参数化不变型严格性引入回归(guava A/B delta=-1~-2, `Iterator> cannot be converted to Iterator>` 一族)。须作为 T1(a)+T1(b) **协同专项**, 不能单点突破。已回退, 留协同专项。 + - **通配符捕获 `CAP#1`(T1b)**: 三方 oracle 均败, 属内在难。本轮新增**通配符上界擦除窄化**(`wildcardExtendsBoundErasure`): 当字段/返回值的通配符是 `? extends ConcreteClass` 且上界擦除与目标对应参数擦除**不同**时, 不补 inconvertible 造型(改诚实裸 return, 走 unchecked conversion)。全量零回归(guava/spring/fastjson2/commons-lang3 tree errLines 均持平), 渲染更接近 CFR。`? super X` 场景(下界)不 block, 保留原 unchecked 造型。kill-switch `JDEC_TYPEVAR_FIELD_WILDCARD_NOCAST_OFF`。CAP#1 本身仍内在难。 + - **装箱数值(T1c)**: baseline 全量扫描**无真正原语→包装类错误**(`int cannot be converted to Integer` 等), 唯一 `Long cannot be converted to Integer`(fastjson2 ObjectWriterCreatorASM) 实为 T2 槽位复用混淆, 非 T1(c)。T1(c) 无选靶, 跳过。 2. **活跃区间分裂 / 槽位复用类型混淆 `bad operand type` / `unexpected type` / `int cannot be converted to boolean`** — fastjson2 主要长尾。 - 表象: 一个字节码 local 槽在**互不相交的活跃区间**里先后承载**不兼容类型**的值, 反编译却合成单一变量名 + 单一声明类型。例: `JSONPathFilter$GroupFilter` 的 `var9` 既作 `Iterator` 又被当 int 比较。 - - 现状: 已治本多族 disjoint 槽(兄弟臂 LUB 合并、Object 超类臂合并、数组协变父臂合并、布尔字段/返回槽拆分、跨作用域孤儿读全方法重放等, 见 §4)。**残余**: 非布尔子形态的「区间+类型」拆分须动变量定型/分裂核(`JDEC_LIVEINTERVAL_*`), 高风险, 留专项。 + - 现状: 已治本多族 disjoint 槽(兄弟臂 LUB 合并、Object 超类臂合并、数组协变父臂合并、布尔字段/返回槽拆分、跨作用域孤儿读全方法重放等, 见 §4)。**残余**: 非布尔子形态的「区间+类型」拆分须动变量定型/分裂核(`JDEC_LIVEINTERVAL_*`), 高风险, 留专项。本轮评估: baseline 非布尔槽位混淆仅 ~3 错误(guava `LocalCache$Segment`/`MapMakerInternalMap$Segment` + fastjson2 `ObjectWriterCreatorASM` Long→Integer), 占总 ~95 错误 3%, 而非 bool 分裂逻辑复杂度与 bool 版本相当(数百行)且回归风险高, **性价比不足**, 暂不投入, 保留现有 bool 分裂。 3. **三元 LUB `bad type in conditional expression`** — fastjson2 + guava 若干行。 - 根因: `cond ? a : b` 两臂最小公共上界算窄, 或三元臂里的泛型擦除(`Optional.of(next())` 缺 `(E)` 等)。 @@ -116,6 +119,7 @@ iso 把每个扁平单元单独编译, 以下失败是方法学产物, 在 tree( | `JDEC_WILDCARD_FIELD_CAST_OFF` | 通配符参数化字段存储造型(`Class`→`Class`) | | `JDEC_GENERIC_SUPERWILDCARD_OFF` | `? super E` 消费者实参造型(取下界类型变量作 `(E)`) | | `JDEC_SIBLING_DESC_SIG_OFF` | 兄弟类方法签名额外按 descriptor 收键做重载消歧(识出擦除型变形参、抑制有害 `(Comparable)` 造型) | +| `JDEC_TYPEVAR_FIELD_WILDCARD_NOCAST_OFF` | 类型变量返回 + 字段读: 通配符 `? extends ConcreteClass` 上界擦除与目标对应参数擦除不同时不补 inconvertible 造型(诚实裸 return)。亦作用于 `inheritedFieldReturnCast`。全量零回归, 渲染更接近 CFR | ### 扁平内部类 / 泛型声明(第 4 类) | 开关 | 作用域 | diff --git a/classparser/decompiler/core/statements/java_statements.go b/classparser/decompiler/core/statements/java_statements.go index 69f986f..824c318 100644 --- a/classparser/decompiler/core/statements/java_statements.go +++ b/classparser/decompiler/core/statements/java_statements.go @@ -928,7 +928,49 @@ func typeVarReturnCast(funcCtx *class_context.ClassContext, v values.JavaValue) // regression), so they are excluded. switch uv := values.UnpackSoltValue(v).(type) { case *values.JavaClassMember, *values.RefMember: - // field access (static `Class.INSTANCE` or instance `this.field`) - may need the cast + // field access (static `Class.INSTANCE` or instance `this.field`) - may need the cast, but + // FIRST narrow on the field's RECOVERED real generic type (same logic as inheritedFieldReturnCast): + // when the field carries a top-level WILDCARD whose BOUND ERASURE differs from the corresponding + // declared-return type argument erasure, the cast is INCONVERTIBLE after capture (e.g. guava + // ImmutableMultimap.asMap `return this.map` where map is `ImmutableMap>` + // and the declared return is `ImmutableMap>`: javac captures to + // ImmutableMap and the (ImmutableMap>) cast is inconvertible). The + // bare field read converts by unchecked conversion (legal, warning-only), mirroring the source's + // @SuppressWarnings -- so suppress the cast. Kill-switch JDEC_TYPEVAR_FIELD_WILDCARD_NOCAST_OFF + // restores the legacy always-cast behavior for A/B. + if os.Getenv("JDEC_TYPEVAR_FIELD_WILDCARD_NOCAST_OFF") == "" { + if fieldType := values.RecoverThisFieldInstantiatedType(funcCtx, v); fieldType != nil { + realStr := fieldType.String(funcCtx) + if realArgs, okRA := topLevelTypeArgs(realStr); okRA { + hasWild := false + for _, a := range realArgs { + if strings.HasPrefix(a, "?") { + hasWild = true + break + } + } + if hasWild && erasureName(realStr) == erasureName(retStr) { + if retArgs, okRT := topLevelTypeArgs(retStr); okRT && len(retArgs) == len(realArgs) { + blockCast := false + for i, ra := range realArgs { + if !strings.HasPrefix(ra, "?") { + continue + } + boundErasure := wildcardExtendsBoundErasure(ra) + targetErasure := erasureName(retArgs[i]) + if boundErasure != "" && targetErasure != "" && boundErasure != targetErasure { + blockCast = true + break + } + } + if blockCast { + return "" + } + } + } + } + } + } case *values.JavaRef: if !uv.IsThis { return "" @@ -1500,9 +1542,58 @@ func inheritedFieldReturnCast(funcCtx *class_context.ClassContext, v values.Java if !hasWildcard { return "" } + // T1(b) regression guard: a wildcard source casts to a same-erasure target by unchecked conversion + // ONLY when each wildcard's bound ERASURE matches the corresponding target argument's erasure (or + // the wildcard is unbounded). When the wildcard bound is a DIFFERENT class (e.g. field + // `ImmutableMap>` returned as `ImmutableMap>`), + // javac captures the wildcard to CAP#1 and the cast `ImmutableMap -> ImmutableMap>` + // is INCONVERTIBLE (Collection is neither ImmutableCollection nor its supertype in the captured + // type). In that case the field read must render BARE (`return this.field`) -- javac accepts it as an + // unchecked conversion with only a warning, mirroring the original source's @SuppressWarnings. Only + // block the cast when a wildcard bound erasure disagrees with its target peer; the working cases + // (`Comparator` -> `Comparator`, bound Object == target Object) keep the cast. + retArgs, _ := topLevelTypeArgs(retStr) + if len(retArgs) == len(realArgs) { + for i, ra := range realArgs { + if !strings.HasPrefix(ra, "?") { + continue // non-wildcard arg: unchecked-cast legality covered by the same-erasure gate + } + boundErasure := wildcardExtendsBoundErasure(ra) + targetErasure := erasureName(retArgs[i]) + if boundErasure != "" && targetErasure != "" && boundErasure != targetErasure { + return "" // wildcard bound class differs from target arg class -> cast inconvertible + } + } + } return retStr } +// wildcardExtendsBoundErasure returns the dotted raw class name of an `? extends` wildcard argument's +// UPPER bound, or "" for any other form (unbounded `?`, `? super X`, or non-wildcard). Used by the +// return-cast guards to detect the specific inconvertible-after-capture case: a `? extends ConcreteClass` +// source whose bound ERASURE differs from the cast target's corresponding argument erasure (e.g. +// `ImmutableMap>` cast to `ImmutableMap>`: +// javac captures to ImmutableMap and rejects the cast, whereas the bare read is an unchecked +// conversion). The `? super X` case is deliberately NOT covered: its lower bound X is unrelated to the +// cast legality (a `Comparator` -> `Comparator` cast is a LEGAL unchecked conversion +// that the bare read needs), so reporting it would make the guard over-block and regress those sites. +func wildcardExtendsBoundErasure(wildcardArg string) string { + s := strings.TrimSpace(wildcardArg) + if !strings.HasPrefix(s, "?") { + return "" + } + rest := strings.TrimSpace(s[1:]) + if rest == "" { + return "" // unbounded `?` + } + if strings.HasPrefix(rest, "extends ") { + rest = strings.TrimSpace(rest[len("extends "):]) + } else { + return "" // `? super X` or malformed: not an extends bound + } + return erasureName(rest) +} + // typeVarFieldStoreCast returns the bare class-scope type variable (e.g. "K") to cast the stored // value to when assigning into a SAME-CLASS field that is declared as that type variable but the // stored value's static type is a DIFFERENT reference type (Object or the erased bound). Bytecode diff --git a/classparser/inner_standalone_erase_test.go b/classparser/inner_standalone_erase_test.go index f5fd6f1..9fecf11 100644 --- a/classparser/inner_standalone_erase_test.go +++ b/classparser/inner_standalone_erase_test.go @@ -25,10 +25,10 @@ import ( var ( // 字段/具体返回的独立 K 被擦成 Object。 - standaloneEraseFieldOnRe = regexp.MustCompile(`Object key;`) - standaloneEraseFieldOffRe = regexp.MustCompile(`\bK key;`) - standaloneErasePeekOnRe = regexp.MustCompile(`Object peek\(`) - standaloneErasePeekOffRe = regexp.MustCompile(`\bK peek\(`) + standaloneEraseFieldOnRe = regexp.MustCompile(`Object key;`) + standaloneEraseFieldOffRe = regexp.MustCompile(`\bK key;`) + standaloneErasePeekOnRe = regexp.MustCompile(`Object peek\(`) + standaloneErasePeekOffRe = regexp.MustCompile(`\bK peek\(`) // 抽象方法参数在两种状态下都保持裸 K, V (SuppressStandaloneErase)。 standaloneEraseAbstractRe = regexp.MustCompile(`abstract T out\(K var0, V var1\)`) ) diff --git a/test/cross/orphan_global_rebind_test.go b/test/cross/orphan_global_rebind_test.go index 544807a..141be33 100644 --- a/test/cross/orphan_global_rebind_test.go +++ b/test/cross/orphan_global_rebind_test.go @@ -63,8 +63,8 @@ func TestOrphanGlobalRebindIsLoadBearing(t *testing.T) { return len(parseTreeErrors(out, root)) } - on := treeErrs(false) // fix ON - off := treeErrs(true) // fix OFF (kill-switch) + on := treeErrs(false) // fix ON + off := treeErrs(true) // fix OFF (kill-switch) t.Logf("fastjson2 tree error lines: ON=%d OFF=%d", on, off) if off <= on {