Hi. Thanks for your good work.
I am struggling to reproduce the FID results mentioned in the paper.
These are my findings:
<style>
</style>
| |
Sunglasses |
Babies |
MetFaces |
| FID |
32.977 |
72.152 |
130.59 |
| Intra-LPIPS |
0.5027 |
0.515 |
0.414 |
While these are the reported values in the paper:
<style>
</style>
| |
Sunglasses |
Babies |
MetFaces |
| FID |
24.62 |
48.52 |
94.86 |
| Intra-LPIPS |
NaN |
0.52 |
0.41 |
Below you could see the configurations I have used. Babies and Sunglasses are the same, except num_gradient=15 and t_end=20.
This is the CONFIG for MetFaces dataset
├── num_samples: 10
├── epochs: 120
├── model_path: checkpoints/ddpm/ffhq.pt
├── csv_file: datasets/metfaces/metfaces.csv
├── t_start: 5
├── t_end: 15
├── num_gradient: 10
├── print_config: True
├── category: metfaces
├── normalization: True
├── random_q_noise: True
├── image_size: 256
├── batch_size: 5
├── penalty_weight: 1
├── learning_rate: 0.05
├── num_workers: 0
├── tqdm: True
├── save_gradients: True
├── clip_denoised: True
├── use_ddim: True
├── classifier_scale: 2.0
├── timestep_respacing: 25
├── num_channels: 256
├── dropout: 0.1
├── num_head_channels: 64
├── num_res_blocks: 2
├── resblock_updown: True
├── attention_resolutions: 32,16,8
├── class_cond: False
├── use_fp16: False
├── use_scale_shift_norm: True
├── num_heads: 4
├── num_heads_upsample: -1
├── channel_mult:
├── use_checkpoint: False
├── use_new_attention_order: False
├── noise_schedule: linear
├── learn_sigma: True
├── diffusion_steps: 1000
├── use_kl: False
├── predict_xstart: False
├── rescale_learned_sigmas: False
├── rescale_timesteps: False
└── experiment_gradient_path: checkpoints
The only code that I add on top of your code is the creation of the NPZ files from the actual datasets (1336 images of MetFaces for example), which I do using the following function which I wrote trying to match logics of other functions in src/fid_score.py:
`
def compute_statistics_of_path(path, batch_size=32, device="cuda", dims=2048, num_workers=4):
if path.endswith(".npz"):
with np.load(path) as f:
return f["mu"][:], f["sigma"][:]
# Prepare transform (no resize!)
transform = transforms.Compose([
transforms.ToTensor(), # [0, 1], float32
])
class ImageFolderDataset(Dataset):
def __init__(self, folder):
self.paths = sorted([
os.path.join(folder, fname)
for fname in os.listdir(folder)
if fname.lower().endswith((".jpg", ".jpeg", ".png", ".bmp"))
])
self.transform = transform
def __len__(self):
return len(self.paths)
def __getitem__(self, idx):
img = Image.open(self.paths[idx]).convert("RGB")
return self.transform(img)
dataset = ImageFolderDataset(path)
dataloader = DataLoader(dataset, batch_size=batch_size, num_workers=num_workers, shuffle=False)
# Stack all samples into a single tensor
all_samples = []
for batch in tqdm(dataloader, desc=f"Loading images from {path}"):
all_samples.append(batch)
samples_tensor = torch.cat(all_samples, dim=0)
# Create model only once
block_idx = InceptionV3.BLOCK_INDEX_BY_DIM[dims]
model = InceptionV3([block_idx], resize_input=True, normalize_input=True).to(device).eval()
# Use the same logic as compute_statistics_of_tensor
mu, sigma = calculate_activation_statistics(samples_tensor, model, batch_size, dims, device, num_workers)
# Save NPZ file
np.savez("datasets/fid_npz/metfaces.npz", mu=mu, sigma=sigma)
return mu, sigma`
I would appreciate it if you let me know whether I am doing something wrong. I am trying to reproduce your results so I could cite your research.
Hi. Thanks for your good work.
I am struggling to reproduce the FID results mentioned in the paper.
These are my findings:
<style> </style>While these are the reported values in the paper:
<style> </style>Below you could see the configurations I have used. Babies and Sunglasses are the same, except num_gradient=15 and t_end=20.
This is the CONFIG for MetFaces dataset
├── num_samples: 10
├── epochs: 120
├── model_path: checkpoints/ddpm/ffhq.pt
├── csv_file: datasets/metfaces/metfaces.csv
├── t_start: 5
├── t_end: 15
├── num_gradient: 10
├── print_config: True
├── category: metfaces
├── normalization: True
├── random_q_noise: True
├── image_size: 256
├── batch_size: 5
├── penalty_weight: 1
├── learning_rate: 0.05
├── num_workers: 0
├── tqdm: True
├── save_gradients: True
├── clip_denoised: True
├── use_ddim: True
├── classifier_scale: 2.0
├── timestep_respacing: 25
├── num_channels: 256
├── dropout: 0.1
├── num_head_channels: 64
├── num_res_blocks: 2
├── resblock_updown: True
├── attention_resolutions: 32,16,8
├── class_cond: False
├── use_fp16: False
├── use_scale_shift_norm: True
├── num_heads: 4
├── num_heads_upsample: -1
├── channel_mult:
├── use_checkpoint: False
├── use_new_attention_order: False
├── noise_schedule: linear
├── learn_sigma: True
├── diffusion_steps: 1000
├── use_kl: False
├── predict_xstart: False
├── rescale_learned_sigmas: False
├── rescale_timesteps: False
└── experiment_gradient_path: checkpoints
The only code that I add on top of your code is the creation of the NPZ files from the actual datasets (1336 images of MetFaces for example), which I do using the following function which I wrote trying to match logics of other functions in src/fid_score.py:
`
def compute_statistics_of_path(path, batch_size=32, device="cuda", dims=2048, num_workers=4):
I would appreciate it if you let me know whether I am doing something wrong. I am trying to reproduce your results so I could cite your research.