Sanitize Deprecated adapter_config.json Experimental Keys in Loaded Checkpoints

[purvanshjoshi]

Technical Overview

When training is interrupted and subsequently resumed, the pipeline attempts to load intermediate weights from a local checkout directory. If the model checkpoint was originally processed using customized PEFT forks or experimental versions, adapter_config.json contains parameter definitions (e.g., alora_invocation_tokens, corda_config, or arrow_config) that the installed version of LoraConfig cannot parse.

Affected Modules

• File: src/train.py (Line 60-83)
• Target Classes: LoraConfig and PeftModel

Detailed Traceback / Context

TypeError: __init__() got an unexpected keyword argument 'alora_invocation_tokens'

Acceptance Criteria

1. Flexible Loading: Resuming training from existing checkpoints works even when configurations contain non-standard attributes.
2. Precision Cleaning: Only unsupported parameters are deleted from the JSON config, leaving the critical r, lora_alpha, target_modules, and class configurations intact.
3. Safety Verification: Safe configurations are serialized back to disk to prevent identical errors on subsequent save intervals.

Proposed Implementation Approach

Implement a dynamic configuration inspection layer utilizing Python's inspect library before reloading the adapter:

import json
import inspect
from peft import LoraConfig

config_path = os.path.join(last_checkpoint, "adapter_config.json")
if os.path.exists(config_path):
    with open(config_path, "r") as f:
        config_data = json.load(f)
    
    # Extract only parameters that map to the current PEFT signature
    valid_keys = set(inspect.signature(LoraConfig.__init__).parameters.keys())
    new_config = {k: v for k, v in config_data.items() if k in valid_keys}
    
    if len(new_config) < len(config_data):
        with open(config_path, "w") as f:
            json.dump(new_config, f, indent=4)

Severity & Priority

• Severity: High (Prevents checkpoint recovery and resumes)
• Priority: P1

[Saksham-dev-del]

Hi @purvanshjoshi,

I would like to work on this issue. I will add a config-sanitization step in src/train.py (lines 60–83) that:

Uses inspect.signature on LoraConfig.init to determine currently valid parameters.
Filters adapter_config.json to retain only supported keys (preserving r, lora_alpha, target_modules, etc.).
Rewrites the cleaned config back to disk to prevent repeat failures on future checkpoint saves.

I will test this against a checkpoint containing legacy/experimental keys (e.g., alora_invocation_tokens) to confirm resumption succeeds without errors.

Could you please assign this issue to me? I will submit a PR for review once complete.