Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/debug-diff/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ uv run diffyscan <config-path> --yes --cache-explorer --cache-github
- `--cache-github` (`-G`) reuses cached GitHub file fetches
- `--support-brownie` enables recursive retrieval for brownie-verified contracts with flattened import paths

To accept known diffs, prefer config `allowed_diffs` rules over CLI flags. The `--allow-source-diff 0xAddr` / `--allow-bytecode-diff 0xAddr` flags still work but are **deprecated** shorthands for `any: true` (a blanket wildcard that hides all future drift). When a diff is uncovered, diffyscan prints a ready-to-paste `allowed_diffs` snippet in the final summary — paste it into the config and replace the placeholder `reason`, tightening `any: true` to a granular facet (`immutables`, `byte_ranges`, `cbor_metadata`, `line_ranges`, `files`) wherever possible. See the "Granular allowlists" section of the README.
To accept known diffs, use config `allowed_diffs` rules. (The former `--allow-source-diff` / `--allow-bytecode-diff` CLI flags have been removed; they were blanket `any: true` shorthands.) When a diff is uncovered, diffyscan prints a ready-to-paste `allowed_diffs` snippet in the final summary — paste it into the config and replace the placeholder `reason`, tightening `any: true` to a granular facet (`immutables`, `byte_ranges`, `cbor_metadata`, `line_ranges`, `files`) wherever possible. See the "Granular allowlists" section of the README.

## Known limitations

Expand Down
1 change: 0 additions & 1 deletion .claude/skills/new-config/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,6 @@ YAML gotcha: addresses and hex strings MUST be quoted (`"0xabc..."`) — unquote
- `constructor_calldata` — raw hex calldata per address: `{"0xAddr": "0xabcd..."}`
- `constructor_args` — typed args per address: `{"0xAddr": ["0xarg1", true, 42]}`
- `libraries` — per source path: `{"contracts/lib/Foo.sol": {"Foo": "0xLibAddr"}}`
- `hardhat_config_name` — (deprecated) name of a hardhat config file
- `fail_on_bytecode_comparison_error` — defaults to `true`; when `false`, a per-contract exception (e.g. failing to fetch/verify a contract from the explorer) is logged and the run continues instead of aborting. Despite the name, an actual error inside `run_bytecode_diff` is always caught and recorded as a mismatch (`match=False`) regardless of this flag.
- `allowed_diffs` — declare expected, known diffs so the run still passes while everything else stays verified. Map of `bytecode` / `source` → `{"0xAddr": [rules]}`; each rule needs a `reason`. Prefer the **most specific** facet — for bytecode: `immutables` (exact on-chain values), `byte_ranges`, `cbor_metadata: true`, `constructor_args`/`constructor_calldata`; for source: `line_ranges` (exact hunks), `files` (whole files). `any: true` is a blanket wildcard that hides *all* future drift — use it only when a diff genuinely cannot be scoped (e.g. bytecode that can't be reproduced), and explain why in the `reason`. When a diff is uncovered, diffyscan prints a ready-to-paste snippet in the final summary; paste it and tighten the placeholder. Example:
```yaml
Expand Down
4 changes: 1 addition & 3 deletions .claude/skills/validate-config/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ The `Config` TypedDict (`diffyscan/utils/custom_types.py`) defines:
- `metadata` — `dict` (free-form project metadata)

The `BinaryConfig` TypedDict has all-optional fields:
- `hardhat_config_name` — `str` (deprecated, ignored at runtime)
- `constructor_calldata` — `dict[str, str]` mapping address to raw hex calldata
- `constructor_args` — `dict[str, list]` mapping address to a list of ABI-encodable arguments
- `libraries` — `dict[str, dict[str, str]]` mapping source path to `{LibraryName: "0xAddress"}`
Expand Down Expand Up @@ -92,7 +91,6 @@ However, the test suite (`tests/test_configs.py:test_contract_addresses_format`)
- `constructor_args` values must be lists (arrays of ABI-encodable values)
- A contract address must NOT appear in both `constructor_calldata` and `constructor_args` -- the runtime raises `CalldataError` if it does (see `get_calldata` in `diffyscan/utils/calldata.py`)
- `libraries` maps Solidity source file paths to `{LibraryName: "0xAddress"}` dicts
- `hardhat_config_name` is deprecated and ignored at runtime (a warning is logged)

### 8. Cross-reference checks

Expand Down Expand Up @@ -127,6 +125,6 @@ Beyond schema validity, **flag every `any: true` rule as a smell**: it suppresse

Report issues in two categories:
- **Errors** (must fix): missing required fields, type mismatches, YAML hex coercion, duplicate entries in both `constructor_calldata` and `constructor_args`
- **Warnings** (should review): missing `explorer_token_env_var`, short commit SHA, addresses not matching 0x/42-char format, missing `network`, deprecated `hardhat_config_name`, unused bytecode_comparison entries
- **Warnings** (should review): missing `explorer_token_env_var`, short commit SHA, addresses not matching 0x/42-char format, missing `network`, unused bytecode_comparison entries

If the config looks good, confirm it passes validation.
5 changes: 2 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Entry point: `diffyscan/diffyscan.py:main` — parses CLI args, loads config (JS
Configs live in `config_samples/` organized by chain (ethereum, optimism, zksync, etc.):
```
contracts: { "0xaddr": "ContractName" }
network: "mainnet" # required in TypedDict, unused at runtime
network: "mainnet" # optional legacy label, unused at runtime
explorer_hostname: "api.etherscan.io"
explorer_token_env_var: "ETHERSCAN_EXPLORER_TOKEN"
github_repo: { url, commit, relative_root }
Expand All @@ -72,10 +72,9 @@ bytecode_comparison: { constructor_calldata, constructor_args, libraries, deploy
allowed_diffs: { bytecode: {...}, source: {...} } # optional; see below
```

`allowed_diffs` declares known/expected diffs per contract so a run still passes while everything else stays verified (validated by `diffyscan/utils/allowed_diffs.py`). Each rule needs a `reason`. **Prefer the tightest facet** — bytecode: `immutables`, `byte_ranges`, `cbor_metadata`, `constructor_args`/`constructor_calldata`; source: `line_ranges`, `files`. `any: true` is a blanket wildcard that hides all future drift — avoid it unless a diff genuinely can't be scoped, and say why in the `reason`. `tests/test_no_wildcard_regression.py` guards against new wildcards.
`bytecode_comparison.extra_sources` (`{ "0xaddr": ["path/to/File.sol"] }`, optional) names additional source files to fetch from the configured GitHub repo/commit and add to the compilation — for contracts whose explorer-verified source set omits a file the pinned GitHub source imports (e.g. a newly-added interface). The files are fetched the same way as all others (honoring `dependencies`), so verification stays honest.

`allowed_diffs` declares known/expected diffs per contract so a run still passes while everything else stays verified (validated by `diffyscan/utils/allowed_diffs.py`). Each rule needs a `reason`. **Prefer the tightest facet** — bytecode: `immutables`, `byte_ranges`, `cbor_metadata`, `constructor_args`/`constructor_calldata`; source: `line_ranges`, `files`. `any: true` is a blanket wildcard that hides all future drift — avoid it unless a diff genuinely can't be scoped, and say why in the `reason`. The deprecated `--allow-source-diff`/`--allow-bytecode-diff` CLI flags are just shorthands for `any: true`. `tests/test_no_wildcard_regression.py` guards against new wildcards.

### Environment variables

Supports loading from `.env` (see `.env.example`), or set directly: `GITHUB_API_TOKEN`, `ETHERSCAN_EXPLORER_TOKEN`, `REMOTE_RPC_URL` (for bytecode comparison).
Expand Down
2 changes: 0 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,8 +225,6 @@ Recommended workflow to tighten a wildcard:
2. Copy the `allowed_diffs` snippet it prints for the uncovered diff.
3. Paste it into the config, replace the placeholder `reason` with the real explanation, and re-run to confirm it now passes.

`--allow-bytecode-diff` and `--allow-source-diff` still work, but they are **deprecated** shorthands for `any: true` (diffyscan warns when they are used). Move those rules into the config file so the summary suggestions can help you tighten them over time.

Start the script

```bash
Expand Down
60 changes: 8 additions & 52 deletions diffyscan/diffyscan.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@
compile_contract_from_explorer,
get_contract_from_explorer,
get_explorer_chain_id,
get_explorer_hostname,
get_solc_sources,
merge_libraries,
parse_compiled_contract,
Expand Down Expand Up @@ -331,12 +330,12 @@ def run_source_diff(
source_files = get_solc_sources(contract_code["solcInput"])
standard_json_format = is_standard_json_contract(source_files)

explorer_hostname = get_explorer_hostname(config)
explorer_hostname = config.get("explorer_hostname")
explorer_chain_id = get_explorer_chain_id(config)
logger.divider()
logger.okay("Contract", contract_address_from_config)
logger.okay("Blockchain explorer Hostname", explorer_hostname)
if explorer_chain_id:
if explorer_chain_id is not None:
logger.okay("Blockchain explorer Chain ID", explorer_chain_id)
else:
logger.warn("Blockchain explorer Chain ID isn't set")
Expand Down Expand Up @@ -589,44 +588,20 @@ def _log_explorer_bytecode_metadata(
)


def _warn_deprecated_hardhat_settings(
config: dict,
hardhat_config_path: str | None,
) -> None:
if hardhat_config_path:
logger.warn("--hardhat-path is deprecated and ignored")

bytecode_comparison = config.get("bytecode_comparison", {})
if isinstance(bytecode_comparison, dict) and bytecode_comparison.get(
"hardhat_config_name"
):
logger.warn(
'Config key "bytecode_comparison.hardhat_config_name" is deprecated and ignored'
)


def process_config(
path: str,
hardhat_config_path: str | None,
recursive_parsing: bool,
enable_binary_comparison: bool,
cache_explorer: bool,
cache_github: bool,
cli_allowed_source_diffs: list[str] | None,
cli_allowed_bytecode_diffs: list[str] | None,
skip_user_input: bool = False,
contract_filter: list[str] | None = None,
):
ExceptionHandler.initialize(True)

logger.info(f"Loading config {path}...")
config: dict = load_config(path) # type: ignore[assignment]
_warn_deprecated_hardhat_settings(config, hardhat_config_path)
effective_allowed_diffs = build_effective_allowed_diffs(
config,
cli_allowed_source_diffs,
cli_allowed_bytecode_diffs,
)
effective_allowed_diffs = build_effective_allowed_diffs(config)

explorer_token = _load_explorer_token(config)
github_api_token = load_env("GITHUB_API_TOKEN", masked=True, required=True)
Expand All @@ -652,8 +627,11 @@ def process_config(
remote_chain_id = get_chain_id(remote_rpc_url)
logger.okay("Remote chain ID", remote_chain_id)

explorer_hostname = get_explorer_hostname(config)
explorer_hostname = config.get("explorer_hostname")
if explorer_hostname is None:
logger.warn('Failed to find "explorer_hostname" in the config')
assert explorer_hostname is not None, "explorer_hostname is required"
explorer_chain_id = get_explorer_chain_id(config)
Comment thread
TheDZhon marked this conversation as resolved.

# Apply contract filter if specified
filter_set = (
Expand All @@ -670,7 +648,7 @@ def process_config(
explorer_hostname,
contract_address,
contract_name,
get_explorer_chain_id(config),
explorer_chain_id,
cache_explorer,
)

Expand Down Expand Up @@ -769,11 +747,6 @@ def parse_arguments() -> argparse.Namespace:
default=None,
help="Path to config or directory with configs",
)
parser.add_argument(
"--hardhat-path",
default=None,
help=argparse.SUPPRESS,
)
parser.add_argument(
"--yes",
"-Y",
Expand Down Expand Up @@ -803,20 +776,6 @@ def parse_arguments() -> argparse.Namespace:
help="Cache files retrieved from GitHub to avoid re-fetching on repeated runs",
action="store_true",
)
parser.add_argument(
"--allow-source-diff",
dest="allow_source_diff",
action="append",
default=[],
help="DEPRECATED shorthand for an allowed_diffs source 'any: true' rule; prefer config allowed_diffs. Allow source diffs for a specific contract address (0x...). Can be passed multiple times.",
)
parser.add_argument(
"--allow-bytecode-diff",
dest="allow_bytecode_diff",
action="append",
default=[],
help="DEPRECATED shorthand for an allowed_diffs bytecode 'any: true' rule; prefer config allowed_diffs. Allow bytecode diffs for a specific contract address (0x...). Can be passed multiple times.",
)
parser.add_argument(
"--log-level",
choices=["info", "okay", "warn", "error"],
Expand Down Expand Up @@ -917,13 +876,10 @@ def main() -> None:
for config_path in config_paths:
result = process_config(
config_path,
args.hardhat_path,
args.support_brownie,
enable_binary_comparison,
args.cache_explorer,
args.cache_github,
args.allow_source_diff,
args.allow_bytecode_diff,
args.yes,
args.contract_filter,
)
Expand Down
31 changes: 1 addition & 30 deletions diffyscan/utils/allowed_diffs.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,11 +77,7 @@ def validate_allowed_diffs_config(config: dict, path: str) -> None:
_validate_source_rule_entry(entry, scope)


def build_effective_allowed_diffs(
config: dict,
cli_source_addrs: list[str] | None = None,
cli_bytecode_addrs: list[str] | None = None,
) -> dict:
def build_effective_allowed_diffs(config: dict) -> dict:
allowed_diffs = config.get("allowed_diffs") or {}
effective: dict[str, dict[str, list[dict]]] = {"source": {}, "bytecode": {}}

Expand All @@ -96,31 +92,6 @@ def build_effective_allowed_diffs(
_normalize_rule_entry(diff_kind, entry) for entry in entries
]

for diff_kind, addresses in (
("source", cli_source_addrs or []),
("bytecode", cli_bytecode_addrs or []),
):
if not addresses:
continue
logger.warn(
f"--allow-{diff_kind}-diff is deprecated",
"move these rules into config.allowed_diffs",
)
for address in addresses:
normalized_address = address.lower()
if normalized_address in effective[diff_kind]:
logger.warn(
f"Ignoring CLI --allow-{diff_kind}-diff for {address}",
"config.allowed_diffs takes precedence",
)
continue
effective[diff_kind][normalized_address] = [
{
"reason": f"CLI allow-{diff_kind}-diff",
"any": True,
}
]

return effective


Expand Down
44 changes: 0 additions & 44 deletions diffyscan/utils/binary_verifier.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
from .logger import logger, bgYellow, bgRed, bgGreen, red, green, to_hex
from .constants import OPCODES, PUSH0, PUSH32
from .custom_exceptions import BinVerifierError


def format_bytecode(bytecode: str) -> str:
Expand Down Expand Up @@ -199,49 +198,6 @@ def log_bytecode_diff_analysis(analysis: dict) -> None:
)


def deep_match_bytecode(
actual_bytecode: str,
expected_bytecode: str,
immutables: dict,
) -> bool:
"""
Backward-compatible wrapper around bytecode analysis.

Metadata-only differences are still ignored here. The CLI now uses the
structured analysis result directly for granular allowlists.
"""
analysis = analyze_bytecode_diff(actual_bytecode, expected_bytecode, immutables)

if (
not analysis["runtime_mismatch_ranges"]
and not analysis["string_literal_mismatch"]
and not analysis["length_mismatch"]
):
logger.okay("Bytecodes match (after trimming metadata and string literals)")
return True

log_bytecode_diff_analysis(analysis)

if analysis["length_mismatch"]:
raise BinVerifierError(
"Bytecodes have different length after trimming metadata and string literals"
)

if analysis["string_literal_mismatch"]:
raise BinVerifierError("Bytecodes have different string literals")

if any(
not range_info["immutable"]
for range_info in analysis["runtime_mismatch_ranges"]
):
raise BinVerifierError(
"Bytecodes have differences not on the immutable reference position"
)

logger.warn("Bytecodes have differences only on the immutable reference position")
return False


def _compute_runtime_mismatch_ranges(
local_runtime: str,
remote_runtime: str,
Expand Down
5 changes: 0 additions & 5 deletions diffyscan/utils/custom_exceptions.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ class BaseCustomException(Exception):
def __init__(self, reason: str):
message = f"{self.prefix}: {reason}" if self.prefix else reason
super().__init__(message)
self.message = message


class CompileError(BaseCustomException):
Expand All @@ -34,10 +33,6 @@ class ExplorerError(BaseCustomException):
prefix = "Failed to communicate with a remote resource"


class BinVerifierError(BaseCustomException):
prefix = "Failed in binary comparison"


class ExceptionHandler:
raise_exception = True

Expand Down
7 changes: 4 additions & 3 deletions diffyscan/utils/custom_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@


class BinaryConfig(TypedDict):
hardhat_config_name: NotRequired[str]
constructor_calldata: NotRequired[dict[str, str]]
constructor_args: NotRequired[dict[str, list]]
deployment_from: NotRequired[dict[str, str]]
Expand Down Expand Up @@ -70,13 +69,15 @@ class GithubRepo(TypedDict):

class Config(TypedDict):
contracts: dict[str, str]
network: str
network: NotRequired[str]
github_repo: GithubRepo
dependencies: NotRequired[dict[str, GithubRepo]]
explorer_hostname: str
explorer_hostname_env_var: NotRequired[str]
explorer_token_env_var: NotRequired[str]
explorer_chain_id: NotRequired[int]
explorer_chain_id: NotRequired[int | str]
rpc_url_env_var: NotRequired[str]
deployment_gas_limit: NotRequired[int]
bytecode_comparison: NotRequired[BinaryConfig]
allowed_diffs: NotRequired[AllowedDiffsConfig]
fail_on_bytecode_comparison_error: NotRequired[bool]
Expand Down
27 changes: 1 addition & 26 deletions diffyscan/utils/explorer.py
Original file line number Diff line number Diff line change
Expand Up @@ -743,31 +743,6 @@ def parse_compiled_contract(
return contract_creation_code_without_calldata, deployed_bytecode, immutables


def get_config_value(config: dict, key: str, warn_if_missing: bool = True):
"""
Get a value from config with optional warning if missing.

Args:
config: Configuration dictionary
key: Key to look up
warn_if_missing: Whether to warn if the key is not found

Returns:
The config value or None if not found
"""
value = config.get(key)
if value is None and warn_if_missing:
logger.warn(f'Failed to find "{key}" in the config')
return value


def get_explorer_hostname(config: dict) -> str | None:
"""Get explorer hostname from config."""
value = get_config_value(config, "explorer_hostname", warn_if_missing=True)
return str(value) if value is not None else None


def get_explorer_chain_id(config: dict) -> int | None:
"""Get explorer chain ID from config."""
value = get_config_value(config, "explorer_chain_id", warn_if_missing=False)
value = config.get("explorer_chain_id")
return int(value) if value is not None else None
Comment thread
TheDZhon marked this conversation as resolved.
Loading