Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,12 @@ cat /tmp/jdec-inv/guava.tree.fails.txt
### T1. 泛型擦除缺造型 `Object cannot be converted to T/K/CAP#1`(`incompatible types` 桶, 跨 jar 头号来源)
- 已治本多块(返回点向下造型 / JDK·同类·继承·私有方法实参造型 / 统一跨类泛型解析器 / 擦除型类型变量多余 upcast 抑制 / 参数化实参·数组实参造型等, 见 CODEC_TODO §4)。**剩余按根因分**:
- **(a) 接收者自身泛型未被传播复原成参数化类型**: 接收者是本类型局部变量/字段而非 `this`(`this.box.put(o)`, box=`Box<E>`)时, 其泛型未沿声明传播复原, 取值点仍是 Object。业界 CFR/Vineflower 亦有此长尾。
- **本轮实验**: 尝试在 `astore` 点对首声明局部(`Iterator it = map.entrySet().iterator()`)用 `ResolveInstantiatedReturnType` 复原参数化返回定型(`upgradeLocalParamType`), 并扩 `InstantiateJDKMethodReturn` 覆盖 `Map.entrySet/keySet/values`。**回归**: guava A/B delta=-1~-2(ON=28~41 vs OFF=27), 新增 `Iterator<Entry<CAP#1,CAP#2>> cannot be converted to Iterator<Entry<? extends K,? extends V>>` 一族——参数化后不变型严格性反比 raw 更严。**结论**: T1(a) 与 T1(b) 通配符捕获**深度耦合**, 单点局部泛型传播不可行, 须 T1(a)+T1(b) **协同专项**(通配符接收者整体 raw 造型 + 局部泛型传播联动)。已回退, 留协同专项。
- **(b) 通配符捕获 `CAP#1`(guava)** —— **oracle 实证内在难, 优先级下调**: `this.equivalence.equivalent(a,b)`, 字段 `Equivalence<? super T>` 捕获 `CAP#1`, 实参 Object 不可造 `(CAP#1)`。`ORACLE_JAR=guava ORACLE_CLASS='base/Equivalence$Wrapper'` 三方全败(真源码用 `(Equivalence<Object>)` + `@SuppressWarnings`)。方向(若做): 通配符接收者**整体** `<Object>` 造型。
- **本轮增量**: 新增**通配符上界擦除窄化**(`wildcardExtendsBoundErasure`, kill-switch `JDEC_TYPEVAR_FIELD_WILDCARD_NOCAST_OFF`)——当字段/返回值的通配符是 `? extends ConcreteClass` 且上界擦除与目标对应参数擦除**不同**时(guava `ImmutableMultimap.asMap`: 字段 `ImmutableMap<K, ? extends ImmutableCollection<V>>` → 返回 `ImmutableMap<K, Collection<V>>`), 不补 inconvertible 造型, 改诚实裸 `return this.map`(走 unchecked conversion)。全量零回归(guava/spring/fastjson2/commons-lang3 tree errLines 均持平 27/31/25/12), 渲染更接近 CFR。`? super X` 场景(如 `Comparator<? super E>`→`Comparator<Object>`)不 block, 保留原 unchecked 造型。CAP#1 本身仍内在难(三方 oracle 均败)。
- **(d) 返回点「inference variable X has incompatible bounds」(guava 头号桶, ~13 行: ImmutableEnumMap/Range/Maps/Ordering/Striped/Sets$DescendingSet/MinMaxPriorityQueue$Builder/Tables$TransposeTable 等)** —— **有界类型变量同擦除强转 javac 实测拒绝, 不可用 `parameterizedReturnCast` 治**: 形如 `return ImmutableMap.of(k.getKey(), k.getValue());`(目标 `ImmutableMap<K,V>`、K/V 或 `C extends Comparable` 有界), javac 对 `of()` 独立推断出 `<Object,Object>` 后再与目标合流报「incompatible bounds」。实测 `(ImmutableMap<K,V>)(ImmutableMap<Object,Object>)` **当目标实参有界(K extends Enum / C extends Comparable)时 javac 直接报「不兼容的类型…无法转换」**(见 `parameterizedReturnCast` 注释里「BOUNDED var 被排除」的原因)。真源码用 `@SuppressWarnings` + 不同结构强转, 从擦除字节码无法忠实复原。**根因仍是 (a): 局部变量/字段(如 raw `Map.Entry var1`)的泛型未沿声明传播复原**——只要把 `var1` 复原为 `Map.Entry<K,V>`, `getKey()/getValue()` 即回到 K/V, 无需强转。故此桶归 (a) 的局部/字段泛型传播, 不要再尝试返回点强转。
- **(c) 装箱/数值**: `int cannot be converted to Integer` 等(**非擦除, 不可盲目造型**), 按 `Integer.valueOf` 修。
- **本轮评估**: baseline 全量扫描(guava/spring/fastjson2/commons-lang3)**无真正原语→包装类错误**(`int cannot be converted to Integer` 等均不存在)。唯一 `Long cannot be converted to Integer`(fastjson2 ObjectWriterCreatorASM:2381) 实为 T2 槽位复用混淆(var11 槽位先存 Integer 后存 Long), 非 T1(c)。**T1(c) 当前无选靶, 跳过**。
- 复现:
```bash
go build -o /tmp/jj ./cmd/javajive
Expand All @@ -79,6 +82,7 @@ cat /tmp/jdec-inv/guava.tree.fails.txt
### T2. 活跃区间分裂 / 槽位复用类型混淆(fastjson2 `bad operand type` / `unexpected type` 一族)
- 表象: 一个字节码 local 槽在**互不相交的活跃区间**里先后承载**不兼容类型**的值(如 `JSONPathFilter$GroupFilter` 的 `var9` 既作 `Iterator` 又当 int 比较), 反编译却合成单一变量名 + 单一声明类型。
- 已治本多族 disjoint 槽(兄弟臂 LUB 合并 / Object 超类臂 / 数组协变父臂 / 布尔字段·返回槽拆分 / 跨作用域孤儿读重放, 见 CODEC_TODO §4)。**残余**: 非布尔子形态须在变量定型/分裂核(`JDEC_LIVEINTERVAL_*`)上按「区间+类型」更激进拆分同槽, 风险高、改动核心, 留专项。
- **本轮评估**: baseline 非布尔槽位混淆仅 ~3 错误(guava `LocalCache$Segment:72` Object→V、`MapMakerInternalMap$Segment:315` InternalEntry→E、fastjson2 `ObjectWriterCreatorASM:2381` Long→Integer), 占总 ~95 错误 ~3%。非 bool 分裂逻辑复杂度与 bool 版本相当(数百行)且回归风险高, **性价比不足**, 暂不投入, 保留现有 bool 分裂。
- 复现: `ORACLE_JAR=fastjson2 ORACLE_CLASS=JSONPathFilter go test -run TestThirdPartyOracle -v ./test/cross/`

## P1
Expand Down
8 changes: 6 additions & 2 deletions classparser/CODEC_TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,14 @@
1. **泛型擦除缺造型 `Object cannot be converted to T/K/CAP#1`** — **最大杠杆, 跨 jar**。
- 表象: `incompatible types (assignment/return)` 桶(commons-lang3 / fastjson2 / guava 的主桶)。代表: `Object cannot be converted to LinkedHashTreeMap$Node<K,V>` 一族。
- 根因: 字节码泛型擦除后取值点静态类型是 `Object`, 未补回源码原有的 `(T)` / `(Node<K,V>)` 向下造型。需沿「接收者参数化类型 + 方法/字段 Signature + 跨类型层级替换」复原精确类型。
- 现状: 已治本多块(返回点向下造型、JDK/同类/继承/私有方法实参造型、统一跨类泛型解析器 `ResolveInstantiatedParamType`、擦除型类型变量多余 upcast 抑制、参数化实参/数组实参造型等, 见 §4)。**残余**: 接收者自身泛型未被传播复原成参数化类型、通配符捕获 `CAP#1`(不可命名, 三方均败, 属内在难 case)、装箱数值(非擦除, 不可盲目造型)。
- 现状: 已治本多块(返回点向下造型、JDK/同类/继承/私有方法实参造型、统一跨类泛型解析器 `ResolveInstantiatedParamType`、擦除型类型变量多余 upcast 抑制、参数化实参/数组实参造型等, 见 §4)。**残余**:
- **接收者自身泛型传播(T1a/d)**: 实验验证与 T1(b) 通配符捕获**深度耦合**, 单点局部泛型传播会因参数化不变型严格性引入回归(guava A/B delta=-1~-2, `Iterator<Entry<CAP#1,CAP#2>> cannot be converted to Iterator<Entry<? extends K,? extends V>>` 一族)。须作为 T1(a)+T1(b) **协同专项**, 不能单点突破。已回退, 留协同专项。
- **通配符捕获 `CAP#1`(T1b)**: 三方 oracle 均败, 属内在难。本轮新增**通配符上界擦除窄化**(`wildcardExtendsBoundErasure`): 当字段/返回值的通配符是 `? extends ConcreteClass` 且上界擦除与目标对应参数擦除**不同**时, 不补 inconvertible 造型(改诚实裸 return, 走 unchecked conversion)。全量零回归(guava/spring/fastjson2/commons-lang3 tree errLines 均持平), 渲染更接近 CFR。`? super X` 场景(下界)不 block, 保留原 unchecked 造型。kill-switch `JDEC_TYPEVAR_FIELD_WILDCARD_NOCAST_OFF`。CAP#1 本身仍内在难。
- **装箱数值(T1c)**: baseline 全量扫描**无真正原语→包装类错误**(`int cannot be converted to Integer` 等), 唯一 `Long cannot be converted to Integer`(fastjson2 ObjectWriterCreatorASM) 实为 T2 槽位复用混淆, 非 T1(c)。T1(c) 无选靶, 跳过。

2. **活跃区间分裂 / 槽位复用类型混淆 `bad operand type` / `unexpected type` / `int cannot be converted to boolean`** — fastjson2 主要长尾。
- 表象: 一个字节码 local 槽在**互不相交的活跃区间**里先后承载**不兼容类型**的值, 反编译却合成单一变量名 + 单一声明类型。例: `JSONPathFilter$GroupFilter` 的 `var9` 既作 `Iterator` 又被当 int 比较。
- 现状: 已治本多族 disjoint 槽(兄弟臂 LUB 合并、Object 超类臂合并、数组协变父臂合并、布尔字段/返回槽拆分、跨作用域孤儿读全方法重放等, 见 §4)。**残余**: 非布尔子形态的「区间+类型」拆分须动变量定型/分裂核(`JDEC_LIVEINTERVAL_*`), 高风险, 留专项。
- 现状: 已治本多族 disjoint 槽(兄弟臂 LUB 合并、Object 超类臂合并、数组协变父臂合并、布尔字段/返回槽拆分、跨作用域孤儿读全方法重放等, 见 §4)。**残余**: 非布尔子形态的「区间+类型」拆分须动变量定型/分裂核(`JDEC_LIVEINTERVAL_*`), 高风险, 留专项。本轮评估: baseline 非布尔槽位混淆仅 ~3 错误(guava `LocalCache$Segment`/`MapMakerInternalMap$Segment` + fastjson2 `ObjectWriterCreatorASM` Long→Integer), 占总 ~95 错误 3%, 而非 bool 分裂逻辑复杂度与 bool 版本相当(数百行)且回归风险高, **性价比不足**, 暂不投入, 保留现有 bool 分裂。

3. **三元 LUB `bad type in conditional expression`** — fastjson2 + guava 若干行。
- 根因: `cond ? a : b` 两臂最小公共上界算窄, 或三元臂里的泛型擦除(`Optional.of(next())` 缺 `(E)` 等)。
Expand Down Expand Up @@ -116,6 +119,7 @@ iso 把每个扁平单元单独编译, 以下失败是方法学产物, 在 tree(
| `JDEC_WILDCARD_FIELD_CAST_OFF` | 通配符参数化字段存储造型(`Class<?>`→`Class<? super T>`) |
| `JDEC_GENERIC_SUPERWILDCARD_OFF` | `? super E` 消费者实参造型(取下界类型变量作 `(E)`) |
| `JDEC_SIBLING_DESC_SIG_OFF` | 兄弟类方法签名额外按 descriptor 收键做重载消歧(识出擦除型变形参、抑制有害 `(Comparable)` 造型) |
| `JDEC_TYPEVAR_FIELD_WILDCARD_NOCAST_OFF` | 类型变量返回 + 字段读: 通配符 `? extends ConcreteClass` 上界擦除与目标对应参数擦除不同时不补 inconvertible 造型(诚实裸 return)。亦作用于 `inheritedFieldReturnCast`。全量零回归, 渲染更接近 CFR |

### 扁平内部类 / 泛型声明(第 4 类)
| 开关 | 作用域 |
Expand Down
93 changes: 92 additions & 1 deletion classparser/decompiler/core/statements/java_statements.go
Original file line number Diff line number Diff line change
Expand Up @@ -928,7 +928,49 @@ func typeVarReturnCast(funcCtx *class_context.ClassContext, v values.JavaValue)
// regression), so they are excluded.
switch uv := values.UnpackSoltValue(v).(type) {
case *values.JavaClassMember, *values.RefMember:
// field access (static `Class.INSTANCE` or instance `this.field`) - may need the cast
// field access (static `Class.INSTANCE` or instance `this.field`) - may need the cast, but
// FIRST narrow on the field's RECOVERED real generic type (same logic as inheritedFieldReturnCast):
// when the field carries a top-level WILDCARD whose BOUND ERASURE differs from the corresponding
// declared-return type argument erasure, the cast is INCONVERTIBLE after capture (e.g. guava
// ImmutableMultimap.asMap `return this.map` where map is `ImmutableMap<K, ? extends ImmutableCollection<V>>`
// and the declared return is `ImmutableMap<K, Collection<V>>`: javac captures to
// ImmutableMap<K, CAP#1> and the (ImmutableMap<K,Collection<V>>) cast is inconvertible). The
// bare field read converts by unchecked conversion (legal, warning-only), mirroring the source's
// @SuppressWarnings -- so suppress the cast. Kill-switch JDEC_TYPEVAR_FIELD_WILDCARD_NOCAST_OFF
// restores the legacy always-cast behavior for A/B.
if os.Getenv("JDEC_TYPEVAR_FIELD_WILDCARD_NOCAST_OFF") == "" {
if fieldType := values.RecoverThisFieldInstantiatedType(funcCtx, v); fieldType != nil {
realStr := fieldType.String(funcCtx)
if realArgs, okRA := topLevelTypeArgs(realStr); okRA {
hasWild := false
for _, a := range realArgs {
if strings.HasPrefix(a, "?") {
hasWild = true
break
}
}
if hasWild && erasureName(realStr) == erasureName(retStr) {
if retArgs, okRT := topLevelTypeArgs(retStr); okRT && len(retArgs) == len(realArgs) {
blockCast := false
for i, ra := range realArgs {
if !strings.HasPrefix(ra, "?") {
continue
}
boundErasure := wildcardExtendsBoundErasure(ra)
targetErasure := erasureName(retArgs[i])
if boundErasure != "" && targetErasure != "" && boundErasure != targetErasure {
blockCast = true
break
}
}
if blockCast {
return ""
}
}
}
}
}
}
case *values.JavaRef:
if !uv.IsThis {
return ""
Expand Down Expand Up @@ -1500,9 +1542,58 @@ func inheritedFieldReturnCast(funcCtx *class_context.ClassContext, v values.Java
if !hasWildcard {
return ""
}
// T1(b) regression guard: a wildcard source casts to a same-erasure target by unchecked conversion
// ONLY when each wildcard's bound ERASURE matches the corresponding target argument's erasure (or
// the wildcard is unbounded). When the wildcard bound is a DIFFERENT class (e.g. field
// `ImmutableMap<K, ? extends ImmutableCollection<V>>` returned as `ImmutableMap<K, Collection<V>>`),
// javac captures the wildcard to CAP#1 and the cast `ImmutableMap<K,CAP#1> -> ImmutableMap<K,Collection<V>>`
// is INCONVERTIBLE (Collection<V> is neither ImmutableCollection<V> nor its supertype in the captured
// type). In that case the field read must render BARE (`return this.field`) -- javac accepts it as an
// unchecked conversion with only a warning, mirroring the original source's @SuppressWarnings. Only
// block the cast when a wildcard bound erasure disagrees with its target peer; the working cases
// (`Comparator<? super E>` -> `Comparator<Object>`, bound Object == target Object) keep the cast.
retArgs, _ := topLevelTypeArgs(retStr)
if len(retArgs) == len(realArgs) {
for i, ra := range realArgs {
if !strings.HasPrefix(ra, "?") {
continue // non-wildcard arg: unchecked-cast legality covered by the same-erasure gate
}
boundErasure := wildcardExtendsBoundErasure(ra)
targetErasure := erasureName(retArgs[i])
if boundErasure != "" && targetErasure != "" && boundErasure != targetErasure {
return "" // wildcard bound class differs from target arg class -> cast inconvertible
}
}
}
return retStr
}

// wildcardExtendsBoundErasure returns the dotted raw class name of an `? extends` wildcard argument's
// UPPER bound, or "" for any other form (unbounded `?`, `? super X`, or non-wildcard). Used by the
// return-cast guards to detect the specific inconvertible-after-capture case: a `? extends ConcreteClass`
// source whose bound ERASURE differs from the cast target's corresponding argument erasure (e.g.
// `ImmutableMap<K, ? extends ImmutableCollection<V>>` cast to `ImmutableMap<K, Collection<V>>`:
// javac captures to ImmutableMap<K,CAP#1> and rejects the cast, whereas the bare read is an unchecked
// conversion). The `? super X` case is deliberately NOT covered: its lower bound X is unrelated to the
// cast legality (a `Comparator<? super E>` -> `Comparator<Object>` cast is a LEGAL unchecked conversion
// that the bare read needs), so reporting it would make the guard over-block and regress those sites.
func wildcardExtendsBoundErasure(wildcardArg string) string {
s := strings.TrimSpace(wildcardArg)
if !strings.HasPrefix(s, "?") {
return ""
}
rest := strings.TrimSpace(s[1:])
if rest == "" {
return "" // unbounded `?`
}
if strings.HasPrefix(rest, "extends ") {
rest = strings.TrimSpace(rest[len("extends "):])
} else {
return "" // `? super X` or malformed: not an extends bound
}
return erasureName(rest)
}

// typeVarFieldStoreCast returns the bare class-scope type variable (e.g. "K") to cast the stored
// value to when assigning into a SAME-CLASS field that is declared as that type variable but the
// stored value's static type is a DIFFERENT reference type (Object or the erased bound). Bytecode
Expand Down
8 changes: 4 additions & 4 deletions classparser/inner_standalone_erase_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,10 @@ import (

var (
// 字段/具体返回的独立 K 被擦成 Object。
standaloneEraseFieldOnRe = regexp.MustCompile(`Object key;`)
standaloneEraseFieldOffRe = regexp.MustCompile(`\bK key;`)
standaloneErasePeekOnRe = regexp.MustCompile(`Object peek\(`)
standaloneErasePeekOffRe = regexp.MustCompile(`\bK peek\(`)
standaloneEraseFieldOnRe = regexp.MustCompile(`Object key;`)
standaloneEraseFieldOffRe = regexp.MustCompile(`\bK key;`)
standaloneErasePeekOnRe = regexp.MustCompile(`Object peek\(`)
standaloneErasePeekOffRe = regexp.MustCompile(`\bK peek\(`)
// 抽象方法参数在两种状态下都保持裸 K, V (SuppressStandaloneErase)。
standaloneEraseAbstractRe = regexp.MustCompile(`abstract T out\(K var0, V var1\)`)
)
Expand Down
4 changes: 2 additions & 2 deletions test/cross/orphan_global_rebind_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,8 @@ func TestOrphanGlobalRebindIsLoadBearing(t *testing.T) {
return len(parseTreeErrors(out, root))
}

on := treeErrs(false) // fix ON
off := treeErrs(true) // fix OFF (kill-switch)
on := treeErrs(false) // fix ON
off := treeErrs(true) // fix OFF (kill-switch)
t.Logf("fastjson2 tree error lines: ON=%d OFF=%d", on, off)

if off <= on {
Expand Down
Loading