Skip to content

Notebook 27 — Train SAE on a fine-tuned model (LoRA) #5

Description

@caiovicentino

User has a LoRA-fine-tuned checkpoint, wants to train an SAE on its residuals. Document the PEFT hook path + warn about LoRA merging. Target Gemma-2-2B + a small LoRA.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions