From 0fe706304cd2d505cf9345c363edeecc50290c56 Mon Sep 17 00:00:00 2001 From: Haruko386 Date: Wed, 23 Sep 2026 11:20:10 +0800 Subject: [PATCH] docs: refresh README and V2-1 badges --- README.md | 56 ++++++++++++++++++------------------- doc/badges/v21-demo.svg | 4 +-- doc/badges/v21-paper.svg | 4 +-- doc/badges/v21-training.svg | 4 +-- doc/badges/v21-version.svg | 4 +-- doc/badges/v21-website.svg | 4 +-- doc/badges/v21-weights.svg | 4 +-- 7 files changed, 39 insertions(+), 41 deletions(-) diff --git a/README.md b/README.md index db83cf8..deacafa 100755 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@

ApDepth

-

Aiming for Precise Monocular Depth Estimation Based on Diffusion Models

+

Aiming for Precise Monocular Depth Estimation Based on Diffusion Models

@@ -16,20 +16,18 @@ ![ApDepth monocular depth estimation examples](doc/cover.png) -## Overview - -**ApDepth** turns Stable Diffusion 2.1 into a deterministic, single-step +We present **ApDepth**, a deterministic, single-step monocular depth estimator. It combines a frozen Depth Anything V2 prior with a fine-tuned diffusion U-Net to recover accurate scene geometry and crisp object boundaries without iterative denoising. > [!IMPORTANT] > -> ApDepth builds on [**Marigold**](https://marigoldmonodepth.github.io), the -> CVPR 2024 Best Paper [*Repurposing Diffusion-Based Image Generators for -> Monocular Depth Estimation*](https://arxiv.org/abs/2312.02145). +> This repository builds on [**Marigold**](https://marigoldmonodepth.github.io), the +> CVPR 2024 Best Paper [***Repurposing Diffusion-Based Image Generators for +> Monocular Depth Estimation***](https://arxiv.org/abs/2312.02145). -## News +## ๐Ÿ“ข News - **2026-09-22:** Preparing **ApDepth V2-1** with direct SD2.1 fine-tuning, SDWT and optional FFT refinement. @@ -42,23 +40,23 @@ boundaries without iterative denoising. - **2025-08-10:** Trying to make some optimizations in Feature Expression. - **2025-05-08:** Clone `Marigold` to local. -## Quick start +## ๐Ÿš€ Usage Try ApDepth in the [live demo](https://huggingface.co/spaces/developy/ApDepth), browse the [project gallery](https://haruko386.github.io/research), or run it locally using the instructions below. -### Requirements +## ๐Ÿ› ๏ธ Setup The model was trained on: -- Ubuntu 22.04 LTS, Python 3.12.9, CUDA 11.8, `NVIDIA RTX 6000 Ada Generation` +- Ubuntu 22.04 LTS, Python 3.12.9, CUDA 11.8, **`NVIDIA RTX 6000 Ada Generation`** Inference was tested on: -- Ubuntu 22.04 LTS, Python 3.12.9, CUDA 11.8, `NVIDIA GeForce RTX 4090` +- Ubuntu 22.04 LTS, Python 3.12.9, CUDA 11.8, **`NVIDIA GeForce RTX 5090`** -#### Windows +### ๐Ÿชง A Note for Windows users We recommend running the code in WSL2: @@ -66,7 +64,7 @@ We recommend running the code in WSL2: 1. Install CUDA support for WSL following [installation guide](https://docs.nvidia.com/cuda/wsl-user-guide/index.html#cuda-support-for-wsl-2). 1. Find your drives in `/mnt//`; check [WSL FAQ](https://learn.microsoft.com/en-us/windows/wsl/faq#how-do-i-access-my-c--drive-) for more details. Navigate to the working directory of choice. -### Installation +### ๐Ÿ“ฆ Repository Clone the repository (requires git): @@ -90,7 +88,7 @@ pip install -r requirements.txt > Keep the environment activated before running the inference script. > Activate the environment again after restarting the terminal session. -### Docker +### ๐Ÿณ Docker Setup For a streamlined setup, we provide a Docker environment that pre-installs all necessary dependencies, including PyTorch, CUDA, and evaluation tools. @@ -115,7 +113,7 @@ docker run --gpus all -it --rm \ Inside the container, the `apdepth` Conda environment is activated automatically. -## Inference +## ๐Ÿ–ผ๏ธ Inference on images ### Prepare images @@ -138,7 +136,7 @@ python run.py \ You can find all results in `output/example-1`. Enjoy! -### Inference options +### โš™๏ธ Inference options The default settings are optimized for the best result. However, the behavior of the code can be customized: @@ -154,7 +152,7 @@ The default settings are optimized for the best result. However, the behavior of - `--color_map`: [Colormap](https://matplotlib.org/stable/users/explain/colors/colormaps.html) used to colorize the depth prediction. Default: Spectral. Set to `None` to skip colored depth map generation. - `--apple_silicon`: Use Apple Silicon MPS acceleration. -## Evaluation +## ๐Ÿ”ฌ Evaluation Install additional dependencies: @@ -194,7 +192,7 @@ You can get the result under `output/eval` -## Training +## ๐Ÿ‹๏ธ Training Three training methods are retained. The mixed SDWT + FFT method is the recommended recipe; the SDWT-only method is kept for controlled comparison, and the @@ -231,7 +229,7 @@ Prepare [Hypersim](https://github.com/apple/ml-hypersim) and [Virtual KITTI 2](h ------------ -### Method 1: direct SD2 training with SDWT + FFT (recommended) +### ๐Ÿฅ‡ Method 1: direct SD2 training with SDWT + FFT (recommended) This method starts from the original SD2.1 weights. During iterations 0-8,000, it learns the basic depth mapping with latent MSE, pixel L1 and gradient loss. @@ -256,7 +254,7 @@ python train.py \ --resume_run output/train_sd2_sdwt_fft/checkpoint/latest --no_wandb ``` -### Method 2: direct SD2 training with SDWT only +### ๐Ÿฅˆ Method 2: direct SD2 training with SDWT only This method uses the same original SD2.1 initialization, reconstruction-loss schedule and VAE decoder adaptation as Method 1, but leaves latent FFT disabled. @@ -277,7 +275,7 @@ For both direct-SD2 methods, `BASE_CKPT_DIR/stable-diffusion-2-1` must contain the original SD2.1 Diffusers pipeline. Their checkpoints save both the U-Net and the fine-tuned VAE. -### Method 3: legacy Stage 1 + ApDepth/FFT training (ApDepth v2-0) +### ๐Ÿฅ‰ Method 3: legacy Stage 1 + ApDepth/FFT training (ApDepth v2-0) First complete the feature-alignment Stage 1 training. Stage 1 is maintained separately in @@ -313,7 +311,7 @@ Resume from a checkpoint, e.g. python train.py --resume_run output/train_apdepth/checkpoint/latest --no_wandb ``` -### Inference after Method 1 or Method 2 +### ๐Ÿงช Inference after Method 1 or Method 2 Direct-SD2 checkpoints are training-component checkpoints, so inference combines the original SD2.1 pipeline with the saved U-Net and VAE. For Method 1: @@ -330,7 +328,7 @@ path. `infer.py` also accepts `--training_checkpoint` for dataset evaluation. ------------ -## Optional post-training after Stage 2 +## ๐ŸŽŸ๏ธ Optional post-training after Stage 2 **Start this step only after Stage 2 training has finished.** The new VKITTI far-depth supervision feature is enabled by `config/train_sky_finetune.yaml` for @@ -395,7 +393,7 @@ Monitor `train/far_loss`, `train/far_pixel_ratio` and `train/far_latent_ratio`. Compare KITTI and NYU against the completed Stage 2 model after post-training; this change alone does not establish a metric improvement. -## Evaluating trained checkpoints +## ๐Ÿ›๏ธ Evaluating trained checkpoints The new direct-SD2 recipe saves both U-Net and VAE; use `--training_checkpoint` as shown above to load them together. The following instructions apply to legacy @@ -413,11 +411,11 @@ alone is not a complete inference pipeline. Then refer to [evaluation](#evaluati -## Contributing +## โœ๏ธ Contributing Please refer to [this](CONTRIBUTING.md) instruction. -## Troubleshooting +## ๐Ÿค” Troubleshooting | Problem | Solution | | --------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | @@ -430,7 +428,7 @@ Please refer to [this](CONTRIBUTING.md) instruction. -## Citation +## ๐ŸŽ“ Citation Please cite our paper: ```bibtex @@ -442,7 +440,7 @@ Please cite our paper: } ``` -## License +## ๐ŸŽซ License This work is licensed under the Apache License, Version 2.0 (as defined in the [LICENSE](LICENSE.txt)). diff --git a/doc/badges/v21-demo.svg b/doc/badges/v21-demo.svg index d821362..cbf4b0b 100644 --- a/doc/badges/v21-demo.svg +++ b/doc/badges/v21-demo.svg @@ -1,7 +1,7 @@ - + Live demo - + Live Demo diff --git a/doc/badges/v21-paper.svg b/doc/badges/v21-paper.svg index bce8d0a..d515448 100644 --- a/doc/badges/v21-paper.svg +++ b/doc/badges/v21-paper.svg @@ -1,7 +1,7 @@ - + Paper and citation - + Paper diff --git a/doc/badges/v21-training.svg b/doc/badges/v21-training.svg index 524748a..77bd2cb 100644 --- a/doc/badges/v21-training.svg +++ b/doc/badges/v21-training.svg @@ -1,7 +1,7 @@ - + Training recipes - + Training diff --git a/doc/badges/v21-version.svg b/doc/badges/v21-version.svg index c127127..b0c3337 100644 --- a/doc/badges/v21-version.svg +++ b/doc/badges/v21-version.svg @@ -1,7 +1,7 @@ - + ApDepth v2-1 - + v2-1 diff --git a/doc/badges/v21-website.svg b/doc/badges/v21-website.svg index 8ba559a..5c201ac 100644 --- a/doc/badges/v21-website.svg +++ b/doc/badges/v21-website.svg @@ -1,7 +1,7 @@ - + Project website - + Website diff --git a/doc/badges/v21-weights.svg b/doc/badges/v21-weights.svg index 7c7ccc0..c18d571 100644 --- a/doc/badges/v21-weights.svg +++ b/doc/badges/v21-weights.svg @@ -1,7 +1,7 @@ - + Model weights - + Weights