About stage1 and stage2

[Madta907]

Hi @Haruko386. Thank you for your great work! 🎉🎉🎉
I have finished the stage-1 training, and how should I use it in stage-2?
I wish you all the best! Looking forward to your reply.

[Haruko386]

Hi @Madta907, thank you for your Q&A
Taking my local env for example:

# ApDeth/pretrained_checkpoint

pretrained_checkpoint
└── stable-diffusion-2
    ├── .cache/huggingface
    │   ├── download
    │   └── .gitignore
    ├── feature_extractor
    │   └── preprocessor_config.json
    ├── scheduler
    │   └── scheduler_config.json
    ├── text_encoder
    │   ├── config.json
    │   ├── model.fp16.safetensors
    │   ├── model.safetensors
    │   ├── pytorch_model.bin
    │   └── pytorch_model.fp16.bin
    ├── tokenizer
    │   ├── merges.txt
    │   ├── special_tokens_map.json
    │   ├── tokenizer_config.json
    │   └── vocab.json
    ├── unet
    │   ├── config.json
    │   ├── diffusion_pytorch_model.bin
    │   ├── diffusion_pytorch_model.fp16.bin
    │   ├── diffusion_pytorch_model.fp16.safetensors
    │   └── diffusion_pytorch_model.safetensors
    └── vae
        ├── config.json
        ├── diffusion_pytorch_model.bin
        ├── diffusion_pytorch_model.fp16.bin
        ├── diffusion_pytorch_model.fp16.safetensors
        └── diffusion_pytorch_model.safetensors

After finish stage one training, you will get a U-Net Model under ApDepth_Stage1/output/latest/unet.

└── output/stage1
    ├── checkpoint
    │   ├── iter_018000
    │   │   ├── unet
    │   │   └── dinov2_adapter.pth
    │   ├── iter_020000
    │   │   ├── unet
    │   │   └── dinov2_adapter.pth
    │   └── latest
    │       ├── unet
    │       │   ├── config.json
    │       │   └── diffusion_pytorch_model.bin
    │       ├── dinov2_adapter.pth
    │       ├── iter_020000
    │       └── trainer.ckpt
    ├── evaluation
    ├── tensorboard
    │   └── events.out.tfevents.1775463495.de37...
    ├── visualization
    ├── code_snapshot.tar
    ├── config.yaml
    └── logging.log

Next, replace the U-Net Model located at ApDepth/pretrained_checkpoint/stable-diffusion-2/unet/diffusion_pytorch_model.safetensors with the U-Net Model trained in stage 1.

---

Note

It is important to note that we have not yet uploaded the latest version of the Phase 1 training code, so the results obtained may differ from those in the paper; we plan to release the latest code by July. 🤗

[Madta907]

Ok, Thank you for taking time