From 4e7cb4ea96eb6226be333093c35e279fe1a0c8e3 Mon Sep 17 00:00:00 2001 From: Songyuan Tang Date: Wed, 21 Jan 2026 14:03:00 -0600 Subject: [PATCH 1/3] fixed minor issues in the arguments related to model training and updated performance aware fusion module into the project information --- README.md | 62 +++++++- xfusion/config.py | 98 +++++++----- xfusion/inference/infer_swin_ddp.py | 8 +- ...with_auto_calibration_main4_continuous2.py | 2 +- xfusion/train/basicsr/metrics/metric_util.py | 18 --- xfusion/train/basicsr/metrics/niqe.py | 4 +- .../basicsr/metrics/niqe_pris_params.npz | Bin 11850 -> 0 bytes xfusion/train/basicsr/metrics/psnr_ssim.py | 15 +- ...ono_prepost_xfull_plus_mult_poiss_u1v3.yml | 141 +++++++++++++++++ ...4_xray_scratch_remove1_u1v3_contextmap.yml | 143 ++++++++++++++++++ ...e_gray4_VSWINIR_xray_multinode_ddp_new4.py | 34 +++-- .../train_simple_gray6_xfull_mult_poiss.py | 91 ++++++++++- 12 files changed, 514 insertions(+), 102 deletions(-) delete mode 100644 xfusion/train/basicsr/metrics/niqe_pris_params.npz create mode 100644 xfusion/train/train_EDVR_L_x4_STF_REDS_mono_prepost_xfull_plus_mult_poiss_u1v3.yml create mode 100644 xfusion/train/train_SwinVR_VSTFx4_xray_scratch_remove1_u1v3_contextmap.yml diff --git a/README.md b/README.md index a0f6de6..5faf56a 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,24 @@ # XFusion Deep learning-based spatiotemporal fusion for high-fidelity ultra-high-speed x-ray radiography A model to reconstruct high quality x-ray images by combining the high spatial resolution of high-speed camera and high temporal resolution of ultra-high-speed camera image sequences. +Models and methods for spatial and temporal calibration are also included. ## Prerequisites -This implementation is based on the [BasicSR toolbox](https://github.com/XPixelGroup/BasicSR). Data for model pre-training are collected from the [REDS dataset](https://seungjunnah.github.io/Datasets/reds). +This implementation is based on the [BasicSR toolbox](https://github.com/XPixelGroup/BasicSR) and the [RAFT model](https://github.com/princeton-vl/RAFT). Data for model pre-training are collected from the [REDS dataset](https://seungjunnah.github.io/Datasets/reds). ## Usage ### Package description -Currently, xfusion supports 2 model familities for high-quality xray image sequence reconstruction- the EDVR and Swin vision transformer, respectively. +Currently, xfusion supports 2 model familities for high-quality xray image sequence reconstruction- the EDVR and Swin vision transformer, respectively. +In addition, xfusion is functional with x-ray image sequences from both the actual (data type: actual) and virtual (data type: virtual) experiments. +* Actual experiments utilize the dual-detector setup for data acquisition and virtual experiments synthetize dual-detector data from single-camera experiments. + +With the actual dual-detector experimental data, xfusion first conducts the spatial and temporal calibration of the two image sequences. +* Temporal calibration is specific to the choice of the Phantom TMX7510 (Vision Research Inc., USA) camera and Shimadzu HPV-X2 (Shimadzu Corp., Japan) camera. +* Spatial calibration is based on the use of an electroformed copper grid (G400, Gilder Grids Ltd., UK). + +Then at inference time, xfusion traverses the dual-camera image sequences in two ways: continuous and double-interval. +* Under the continuous mode, high-quality xray images corresponding to each Shimadzu camera image are reconstructed. +* Under the double-interval mode, independent high-quality xray images corresponding to each Phantom camera image are reconstructed to evaluate performance metrics. ### Package installation Navigate to the project root directory and then run @@ -19,7 +30,7 @@ to install the package to the selected virtual environment. ### Initialization Run 
-xfusion init --model_type [EDVRModel or SwinIRModel]
+xfusion init --model_type [EDVRModel or SwinIRModel] --data_type [virtual or actual]
After initialization, a configuration file "xfusion.conf" will be generated in the home directory. This configuration file will be updated automatically within the workflow of the xfusion package. @@ -38,9 +49,27 @@ xfusion convert --dir-lo-convert [directory/to/low resolution/RGB/training im ### Training -Run +To train the model with data from the virtual experiments, run +
+xfusion train --dir-lo-train [directory/to/low resolution/gray-scale/training image] --dir-hi-train [directory/to/high resolution/gray-scale/training image] --dir-lo-val [directory/to/low resolution/gray-scale/validation image] --dir-hi-val [directory/ti/high resolution/gray-scale/validation image] --opt directory/to/training setting/yaml file --path-train-meta-info-file [directory/to/training image/meta data] --path-val-meta-info-file [directory/to/validation image/meta data] --pretrain_network_g [path/to/model weight/file/for/model initialization]
+
+To train the model with data from the actual experiments, run +
+xfusion train --dir-lo-train [directory/to/low resolution/gray-scale/paired training image] --dir-hi-train [directory/to/high resolution/gray-scale/paired training image] --dir-lo-val [directory/to/low resolution/gray-scale/paired validation image] --dir-hi-val [directory/ti/high resolution/gray-scale/paired validation image] --dataroot_context [directory/to/low resolution/gray-scale/training image full] --opt directory/to/training setting/yaml file --path-train-meta-info-file [directory/to/training image/meta data] --path-val-meta-info-file [directory/to/validation image/meta data] --meta_info_context [directory/to/paired/high/lo/resolution/training image/index map] --pretrain_network_g [path/to/model weight/file/for/model initialization]
+
+ +### Train calibration model +To train the calibration model with data from the virtual experiments (pretrain), run +
+xfusion train_calibration --image_root [root/directory/to/training images] --meta_info_file [directory/to/training image/meta data]
+--evaluator_weights [path/to/spatiotemporal/fusion/model weight/file/for/training loss]
+--flow_weights None --full_weights None
+
+To train the calibration model with data from the actual experiments (fine tune), run 
-xfusion train --dir-lo-train [directory/to/low resolution/gray-scale/training image] --dir-hi-train [directory/to/high resolution/gray-scale/training image] --dir-lo-val [directory/to/low resolution/gray-scale/validation image] --dir-hi-val [directory/ti/high resolution/gray-scale/validation image] --opt directory/to/training setting/yaml file --path-train-meta-info-file [directory/to/training image/meta data] --path-val-meta-info-file [directory/to/validation image/meta data] --pretrain_network_g [directory/to/model weight/file/for/model initialization]
+xfusion train_calibration --image_root [root/directory/to/training images] --meta_info_file [directory/to/training image/meta data]
+--evaluator_weights [path/to/spatiotemporal/fusion/model weight/file/for/training loss]
+--full_weights [path/to/pretrained/calibration/model weight/file/for/model initialization]
### Test data @@ -49,9 +78,26 @@ To download test data, run xfusion download --dir-inf [tomobank/link/address/of/test/dataset] --out-dir-inf [directory/to/testing image] -### Inference +### Spatial calibration Run 
-xfusion inference --opt directory/to/testing dataset/setting/yaml file --arch-opt directory/to/training setting/yaml file --model_file [path/to/model file] --machine tomo or polaris
+xfusion calibrate --cal_model_file [path/to/calibration/model file] --cal_dir [root/directory/to/calibration/experiment/data]
+--cal_id [calibration experiment subdirectory name]
-Currently to work for EDVRModel in the single-process mode and SwinIRModel in the multi-process mode. + +### Inference +For virtual experiment data, run +
+xfusion inference --opt directory/to/testing dataset/setting/yaml file --arch-opt directory/to/model architecture/yaml file --img-class dataset1 or dataset2 --model_file [path/to/model file] --machine tomo or polaris
+
+For actual experiment data, run +
+xfusion inference --arch-opt directory/to/model architecture/yaml file --img-class name of high-speed experiment --model_file [path/to/model file] --traverse_mode double or continuous --tfm_file [path/to/rigid/body/spatial/transformation/file] --meta_file_scale [path/to/nonuniform/scaling/parameters] --input_dir [root/directory/to/high/speed/experiment/data] --case_list experiment subdirectory names
+
+ +### Suggested useage +For virtual experiment data, currently xfusion works for EDVRModel in the single-process mode and SwinIRModel in the multi-process mode. +For actual experiment data, xfusion works for both models in a unified way. +Do an independent initialization every time the model (EDVR-STF or SWIN-XVR) or the type of data (virtual or actual) changes. +When working on the actual experiment data, first initialize the software, then do the spatial calibration, and last do the inference. +Multiple experiments sharing the same spatial calibration can be grouped to run the model inference. diff --git a/xfusion/config.py b/xfusion/config.py index ea144f0..d571050 100644 --- a/xfusion/config.py +++ b/xfusion/config.py @@ -20,11 +20,19 @@ 'parse_known_args', 'write'] def list_of_ints(arg): + if ',' in arg: + return list(map(int,arg.split(','))) + else: + if arg.lower() != 'none': + return [int(arg)] + else: + return None + +def list_of_strs(arg): if ',' in arg: return list(arg.split(',')) else: return [arg] - CONFIG_FILE_NAME = os.path.join(str(pathlib.Path.home()), 'xfusion.conf') XFUSION_HOME = os.path.join(str(pathlib.Path.home()), 'xfusion') XFUSION_TRAIN_HOME = os.path.join(str(XFUSION_HOME), 'train') @@ -188,10 +196,49 @@ def list_of_ints(arg): 'help': 'Relative frame separation in the hr images after and before the augmentation' }, 'poisson_noise_b0_exp': { - 'default': [], + 'default': 'none', 'type': list_of_ints, 'help': 'upper and lower bounds of the blank scan factor in log scale' }, + # newly added ddp configs + 'iter_div_factor': { + 'default': 10, + 'type': int, + 'help': "Divisor of the total number of iterations" + }, + 'batch_size': { + 'default': 1, + 'type': int, + 'help': "Number of samples in a batch" + }, + 'num_workers': { + 'default': 3, + 'type': int, + 'help': "Number of workers for data loader"}, + 'warmup_iter': { + 'default': -1, + 'type': int, + 'help': "Number of iterations for training warm up"}, + 'dataset_enlarge_ratio': { + 'default': 10.0, + 'type': float, + 'help': "Multiplier of the original number of training samples"}, + 'sub_dataset': { + 'default': False, + 'type': lambda x: bool(strtobool(x)), + 'help': "When set the original training data are randomly sub-sampled for training"}, + 'sub_dataset_frac': { + 'default': 1.0, + 'type': float, + 'help': "Fraction of the original training data set size to sub-sample"}, + 'single_epoch_ok': { + 'default': False, + 'type': lambda x: bool(strtobool(x)), + 'help': "If to only train data for one epoch"}, + 'profiler_iter_num': { + 'default': 3, + 'type': int, + 'help': "Number of subsequent iterations to track computation performance for HTA"}, } MODEL_TYPES = {'train':OrderedDict(),'inference':OrderedDict()} @@ -225,50 +272,17 @@ def list_of_ints(arg): 'type': int, 'help': "Size of convolution window if used in the vit residual connection"}, # newly added ddp configs - 'iter_div_factor': { - 'default': 10, - 'type': int, - 'help': "Divisor of the total number of iterations"}, + 'num_frame': { 'default': 3, 'type': int, 'help': "Number of input low resolution frames to the model"}, - 'batch_size': { - 'default': 1, - 'type': int, - 'help': "Number of samples in a batch"}, + 'initial_lr': { - 'default': 2e-4, + 'default': 5e-6, 'type': float, 'help': "Initial learning rate"}, - 'num_workers': { - 'default': 3, - 'type': int, - 'help': "Number of workers for data loader"}, - 'warmup_iter': { - 'default': -1, - 'type': int, - 'help': "Number of iterations for training warm up"}, - 'dataset_enlarge_ratio': { - 'default': 10.0, - 'type': float, - 'help': "Multiplier of the original number of training samples"}, - 'sub_dataset': { - 'default': False, - 'type': lambda x: bool(strtobool(x)), - 'help': "When set the original training data are randomly sub-sampled for training"}, - 'sub_dataset_frac': { - 'default': 1.0, - 'type': float, - 'help': "Fraction of the original training data set size to sub-sample"}, - 'single_epoch_ok': { - 'default': False, - 'type': lambda x: bool(strtobool(x)), - 'help': "If to only train data for one epoch"}, - 'profiler_iter_num': { - 'default': 3, - 'type': int, - 'help': "Number of subsequent iterations to track computation performance for HTA"},} + } MODEL_TYPES['train']['EDVRModel'] = { 'initial_lr': { @@ -326,7 +340,7 @@ def list_of_ints(arg): 'default': 'none', 'type': Path, 'help': "When set the path to the model weights to initialize the model"}, - 'num_frame': { + 'num_frame_infer': { 'default': 3, 'type': int, 'help': "Number of input low resolution frames to the model"}, @@ -510,7 +524,7 @@ def list_of_ints(arg): 'help': 'when set the model has separate encoders for the hr/lr input image tensors' }, 'cal_train_poisson_b0_exponent': { - 'default': [], + 'default': 'none', 'type': list_of_ints, 'help': 'upper and lower bounds of the blank scan factor in log scale' }, @@ -688,7 +702,7 @@ def list_of_ints(arg): 'help': "Full path to the user experiment data" }, 'case_list': {'default': [], - 'type': list_of_ints, + 'type': list_of_strs, 'help': "Names of the individual experiment subdirectories" }, 'device': {'default': 0, diff --git a/xfusion/inference/infer_swin_ddp.py b/xfusion/inference/infer_swin_ddp.py index f9c6858..4f14786 100644 --- a/xfusion/inference/infer_swin_ddp.py +++ b/xfusion/inference/infer_swin_ddp.py @@ -59,10 +59,10 @@ def run_inference(ts, args): opt['network_g']['embed_dim'] = args.embed_dim opt['network_g']['num_feat_ext'] = args.embed_dim - opt['network_g']['num_frame'] = args.num_frame + opt['network_g']['num_frame'] = args.num_frame_infer opt['network_g']['depths'] = [d * args.scale_depth for d in opt['network_g']['depths']] - opt['network_g']['window_size'][0] += (args.num_frame - 3) + opt['network_g']['window_size'][0] += (args.num_frame_infer - 3) opt['network_g']['window_size'][1:] = [args.window_size_spatial] * len(opt['network_g']['window_size'][1:]) opt['network_g']['num_heads'] = [int(np.ceil(h * args.scale_mha)) for h in opt['network_g']['num_heads']] @@ -75,7 +75,7 @@ def run_inference(ts, args): dataroot = config.get_inf_data_dirs(img_class) inf_home_dir = Path(dataroot).parent - out_dir = Path(inf_home_dir / f'{test_set_name}_{mode}_align_{args.align_features_ok}_prepostx_fixed_lr_r_{lo_frame_sep}_hr_d_{hi_frame_sep*2}_b0_{b0}_em_{args.embed_dim}_nf_{args.num_frame}_dep_{args.scale_depth}_nh_{args.scale_mha}_win_{args.window_size_spatial}_model_{args.model_file.parent.stem}_ddp_swinVRHP_rem1_{ts}') + out_dir = Path(inf_home_dir / f'{test_set_name}_{mode}_align_{args.align_features_ok}_prepostx_fixed_lr_r_{lo_frame_sep}_hr_d_{hi_frame_sep*2}_b0_{b0}_em_{args.embed_dim}_nf_{args.num_frame_infer}_dep_{args.scale_depth}_nh_{args.scale_mha}_win_{args.window_size_spatial}_model_{args.model_file.parent.stem}_ddp_swinVRHP_rem1_{ts}') out_dir.mkdir(exist_ok=True,parents=True) log_file = os.path.join(out_dir, f"inference.log") @@ -129,7 +129,7 @@ def run_inference(ts, args): # continue dataset_opt['scale'] = 4 dataset_opt['gt_size'] = gt_size[:2] - dataset_opt['num_frame'] = args.num_frame + dataset_opt['num_frame'] = args.num_frame_infer #test_set = VideoTestDatasetSTF(dataset_opt) #test_set = build_dataset(dataset_opt) test_set = XrayVideoTestDatasetSTF(dataset_opt) diff --git a/xfusion/inference/infer_with_auto_calibration_main4_continuous2.py b/xfusion/inference/infer_with_auto_calibration_main4_continuous2.py index b5f2ec1..fa9ec71 100644 --- a/xfusion/inference/infer_with_auto_calibration_main4_continuous2.py +++ b/xfusion/inference/infer_with_auto_calibration_main4_continuous2.py @@ -79,7 +79,7 @@ def run_inference(args): exp_root = Path(config.get_inf_data_dirs(input_dir.name)) device = args.device - opt_path = args.opt + opt_path = args.arch_opt opt = yaml_load(opt_path) opt['manual_seed'] = 10 torch.manual_seed(opt['manual_seed']) diff --git a/xfusion/train/basicsr/metrics/metric_util.py b/xfusion/train/basicsr/metrics/metric_util.py index 06d81d4..79f4f6d 100644 --- a/xfusion/train/basicsr/metrics/metric_util.py +++ b/xfusion/train/basicsr/metrics/metric_util.py @@ -15,8 +15,6 @@ import numpy as np -from xfusion.train.basicsr.utils import bgr2ycbcr - def reorder_image(img, input_order='HWC'): """Reorder images to 'HWC' order. @@ -42,19 +40,3 @@ def reorder_image(img, input_order='HWC'): if input_order == 'CHW': img = img.transpose(1, 2, 0) return img - - -def to_y_channel(img): - """Change to Y channel of YCbCr. - - Args: - img (ndarray): Images with range [0, 255]. - - Returns: - (ndarray): Images with range [0, 255] (float type) without round. - """ - img = img.astype(np.float32) / 255. - if img.ndim == 3 and img.shape[2] == 3: - img = bgr2ycbcr(img, y_only=True) - img = img[..., None] - return img * 255. diff --git a/xfusion/train/basicsr/metrics/niqe.py b/xfusion/train/basicsr/metrics/niqe.py index 74a6237..6e8d5c3 100644 --- a/xfusion/train/basicsr/metrics/niqe.py +++ b/xfusion/train/basicsr/metrics/niqe.py @@ -20,7 +20,7 @@ from scipy.ndimage import convolve from scipy.special import gamma -from xfusion.train.basicsr.metrics.metric_util import reorder_image, to_y_channel +from xfusion.train.basicsr.metrics.metric_util import reorder_image from xfusion.train.basicsr.utils.matlab_functions import imresize from xfusion.train.basicsr.utils.registry import METRIC_REGISTRY @@ -198,7 +198,7 @@ def calculate_niqe(img, crop_border, input_order='HWC', convert_to='y', **kwargs if input_order != 'HW': img = reorder_image(img, input_order=input_order) if convert_to == 'y': - img = to_y_channel(img) + raise Exception('not implemented yet...') elif convert_to == 'gray': img = cv2.cvtColor(img / 255., cv2.COLOR_BGR2GRAY) * 255. img = np.squeeze(img) diff --git a/xfusion/train/basicsr/metrics/niqe_pris_params.npz b/xfusion/train/basicsr/metrics/niqe_pris_params.npz deleted file mode 100644 index 204ddcee87c4cd39aca04a42b539f0a5bfccecc3..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 11850 zcmbt)cUaH;+jr3=DyzX|m!hnqC9AVZMABZ`oA%y&X;D&HMUqmXgpjuqEi+{lC3KOH zvO@CYcVDlL`*+><^T+f2zK%G)KA+F`=s3UU`8r?Yt*tqhOOWHgULqW0eZL;d<>cV_ z{WFzAki*5t!rjx!%fj8t)5=BC)jfcdW6HlS{(j*1O}}61TKZeGjX61dIfBG(?YwL} z#a9c9ueDbYUn(qa@8;?4Y2|9+=4os9`~TNl?ewx6`F}4*D|fq*_Yy0X3d_hZS}GhO z{QvvnnI`=;G2{-HlBB`x_0d1xQg?O8QO%(qnsX|_U3TGla@yMO{j0i~EZ13=saQ19 zs$Ija!?JJad3blt(Dzf6o4IXBA@>?hy~l6iD$qc}-_4DDJ>QUV{rspT(fzE#WmAQm zXg&R`|LdE$!gbQGerum;`G!JI?(dQr%w|tgvUZDpipczx02yA zzneR2-cxJ-d%+jWdTD-fMhoBc3VJ4AyhiGHEy=i5$!~4ENn?ficteAG$>j5Eo?kM> z7DnTQ4dnDn}rvW|bKT@{wg)=4h9w>YDT)_R!DHJe^f&obA< z&S~zYiU8|hAwA=4Gn!hTB&h@&)bYvErN)GvQ)U*g1Q7XExe z`||h_;>fYBHaT{jE^V&f*>I(q1pUq&HXJ%l3bpR}?TtlL(%QMb;EOAj42_wmE_j5Z zzq$3!OU|crzZRu)brn%H*9C_gnx5>Nq>WIGaUR*m8H6hMy-ACe6_nob+v?2Z{d&fCCCX-~=>-`b8)5*cpIw-|{AK5+ssC)6O zHjDYp-|o(}hvYsxu5Ep{hf3y`n3})LpnXdgrfRSDVG~}QKY9FUIyv9nAbNXQ3eDdT zIVRaJgBt62+Gx2q>)e~!W0jXq^6zz8-qao^qUwx1Tz&JvfW@Muvu5_DrA+ zYE)2}t;}wYskAO|iKCrWaV^1W33Q;O^XJW1@pM%_F6o7lC7U$l;pF)>o|GbLoSyBA zqdQYSM>+?`Qq7c#7RfMI=Fy=zJT$b6Ottn~?b{tin;$<+N{opi<-v||%^}Y0WQC0` zx5X|};(RT-<9O6)zo0JXBUWs_Cz3{eb2sIOggVl?MWxC!<044-*h_UCi6klv5i7D< zn?@mbhP?H19w_q1U;cEyCWe;ry=+Tf5>Mqyp+|)>_fX`=-Q!)KTCrPxX}J=-2^24& z{?`oCXeys5D6X?@4-K{^j&HnR$r}E2-QB~VK)!tH7iN`3(U8*u{qgaIwENYZzhu6r zke8f_cJ)v!#hq5V6w0xZPI;7f&3GL~d0DesjV44<(Ljr4eM2k_E>PU);Ix(ro#^KH zaVwncn)j`Y?v0=)`?jd@B}UWxguH6?Jafj+OYOYJB8d&Nw5*z78WU5pslO?TWIh?M zTyx)q#ogkm;NB5Q8`bv9-Ea>cog?%M>T=gfan)XNciJMYQhO{+iP>0r{ygC9L`Hmj z+a zRO8@wT?r+YV>ZiD-`bLPzPR2uvE5)4Uoc1L7u4l@k}uE2i$V1Eokhd&+fXV!w(8E5 z;&7UoR;OH5CQV5j3vRBi3M9`ftM{dz3Z*p`vRPL@hS8hij1TuSELrmBl*BWf;WR6M zwab`ve;O0k*p{>;jGWJ>UD=~(#Rl$3HBS69oW8t|s+n2uN9&BObogQuX?#hyTgYGp z-HcW9zTo3Ucg#3)=O!AkU~awG)8*bYba|b0!5^N~JI!YHy&vv$@PNk8_0mF2!?p0i zj4BU0cEa&xy{H$}Z%)Z*H*}%bO=%|&>FBa^y4g7<_r2&@+P6tk(ymnPZ6Nh|)=m-= zOLe}Vs>_t-_8q)b=S4S~PKb(LcNxW9;0xvm{err*1(qwmS56>}{*TLZHIhjnFj&Fg zL7&}Q(#dmYNeuZVC?-k7M3a`7v&}q@M2d9LeyEsX%dD%$*>rx2rL2UWxus_!XhW#v z<(JbF$t?Ze_Q}5X?959+9{w$HRQ9qzTj)hNO}B3iwbnjNio+9lhh2Bk%JN5KW)Mz; zwUaHpG}TElS7=`Tl>kzEc6sRT^)PZ-6m2MSJ)CAASSOv|V#1V;@mqV{4JDDqA~O9+ zLG-|&D%yKZ2;I!w`qO`#1>=#oD-ul%BP-pD1FO{h$c`md8b}Az`PViF+qf*)(gtts z@AhHDUEC_8`ptKA9|3oPFPJ0r3+l4ly^M23VLH8OA5W7e=&;ig3l<-}8b@Qce=py< zHJ-RmnV9)mr%}P3v{K$$8x~jA@RaXk0%fjX3!TnHQ=7-td0z9+`S=m zyMni2+Wjaxr7>n1w|zV<ztwS z#*!sQI~K|Q8AUyF-I@gpg6Vyo`Z%$cXsWGK`1Qv-6Q(h3N~!qqNV0a85P$kOcyv!= z9|3oPFPJ0r3+gh?`a55WryFID^>?_t!H)e|74IRXrA|jWX1DY`Q)PquXKejxXh2gc zO+&dZ>#?(ACYx+j7iFn2PlmQ}sWa2unA1(oMr1Yq_A$PELzY;S^q{JXgRZFvL(Cgv--B0q3*xjU8QHJ)$^zKBG7MF|+1<1_|U38n1^(}d6 zK3>$0_UZ;icgwA3)9vE(g$GTj^zf7K-jz!!;dtT4#j94)W(NhX*6wu_KaaoUhVW+k zk$mEZkeL{r*c_s(OpD3aS*rZ#3>6xvJF6QIw1L21u&1$)fV;pK%n|wpbvfMQFme0M zC<^GYdnFeVPAv-S+>|0?NP#n~v_js2ndV#Z7R?{|e$1_F+9!n2pOPz&ipa*$?DB3$ zttrl|M|(zzcWyM9Z%Hy-xiN@*B>U_pf7wgk<_m>4eTbqBj*{Fbl>O+sLQ3bQ32M~y zre@;ubDm_kTKCHGbAGh?+`7Cqd;O`ua`vuC=dIb|{_ch6^aIG|M8(9mr=H|JQBtMx zPaj%dJ9mT0cr(@&-rBD>)t|zR%a%>K=uRg&jMg^0dDGezYnGayH(?)}CO^}@;78*^ zo4faQjO?esc^&u*_B8epa2NQ3IYPgnE(TZPxSpm(QK*2-`-)j{G}P2K5M*q@?hJTu zPRWa++3^B7Z*#(E+@l;{xx=xf!6UUpwA!34b`L6(IS@rrDmrZkuZ2?1nyMVdMf<78 zLvDNZqj;)Uy%Jhb6+#jxpYKZEs>bfiDn&@_3Z(3TF9Sk5f{1vkLZ|BllWz6B%AsI! z_O5QBVWv+2&5$1I-x3o<&OzaVYp(gzd0yq-Z{0?0VK~nlKhHzRv@H+4p>}l*H z;4bh5bA*0DUCPuI<8=lcsl)r4@;EM6cJ9fV!yjYySjorspc9qW%w9u(gr=xbmC~SB z)(Sh8y1m}fsLz5mxw%$enxV#yC_0E)R_n74)6_RgCl#5v%L)2=Re}t1e*M~dMx9Nz zY<;L3tj>Ok9BNobs~E3A^FXKEHcERg*ekYNmra(>+1fO)gweuU!Ak!1Y)Zti`Mad; z^gV{_(5X*GOuNv_cG3g^sm8#Rm_}uaBppFej!y#-6p_ zW-}w5VyWQ9t)FKd!pUNJf1lKzY`VXv-aB!5JoSB!IPzlJ$oyodUG%NzVcMCyBo@i~ zQ_v=Z(gdR*+9zEmBbgjTiY0qD|MA_BO}<J`9)7VGAUEmAm2>pV(jGeyOQ>8A5#w))cQH{}5vj6l{F$XI)!F`rk z`j!Y1nxqx07#KjG45oxjf61hx$dcY+7EOP>OKEog=}&5_jVnLy=4X4#yZ>-f_n|V& z2d{P>_al>+UtfA%_NQi*o=0ZZD$Ff?z0v)pzGP|N;wl#EOL=cwuIwG_OKZ*+9^WLi zjdhhbc6{0GM^^I{(p~@XB730@5!W~$Dt~pkkDJSkiO!h*)%mSIopV%gPY?1K)zi?8 z&_~b#;Je^;;4j$I*hj!!;0xvm{errrW^`Duu}!Bb-$V6FPusDvs@*P{PZLO9d*)e* zgeZ#Zo?dm`qnH$=8@4$dO{NF_UzW-G#gf6<#%YFb`lK}Tp7x9pe0`bG`1taTND|ob zl~+$Bj>dgBG01M95VjN9{rS|&wN^vNV{RiB_yp89a21tOGKcUhqE z>%ITsnuavHaqy~b!QdvE za%ZvUn>1~v(7i>>Jx`A1d)|9HTw!ul7!}@ z<)+$9aIIa;uOAkazdCDE@FP>^p6b4K(~cik;J?6UfS!hKgg$}}0N(|#1AoDu#y$e> z0$(sk=oi$b@x5eceXfs#Y5}QrSFLP(SeI7?*iBcO2_mALAwpzi0 zpQ7y6RgQJ7V*d2RP~Y}ceIN0mD&Ghq#A2O*gy=>(=e~QiXU%anQh;;6Y^e)r#CKaQBs-GkJNyzY*-{3bt zirxNxw~eDJqgS6_+En?FPVj)$Qdk$7BOA9;v|0xo*3(- zVc|g!L&rZ%+u%(L3xC;*S8QRbyjh8t_V`l7jm;Z-R(g*5b?~L&zrbgJo`!COK7tMa z-vzG&f5D!{J_7CnUoc1L7t}?(jYG@$c@ODX_&qUvS4M}z{If)C&ePAT&a>+AzLX%v zAKCW(7-b6GQ`+8jl4LUu)$-52Kr7wf284|0@@9U&>73K@$+@b@!Km*feOk7a*W+0w z4embNzg5JH`J_rOiRdbVR)jYn(ZKBppgMZ-Cgsemo>gxR1cPN(nUfLcp@=l`Eg{Nz- zPPJpvfrp~?Y75hA zorZXlNi;=jvff)w=8*M{Q)I*!FzFT0Xx0uN^}XTm!pDSP2VV;Q3w#FXY3N4iBj^C| zUGO^a7wl>5Bj7IZ1#^UcL0t-R<0RDQxKi-^sVBM~nJ}^Ps`F#Xm?U0&ZO!+xr3OK< zIq5Rn*$c<6(2PxnCrj4Q_rN#PW`@a9g5}w}(|puM^8onX@OR;3!moob1^)#;1N1a> zBlHn;0QfF=9rz3OH1-j27x;oXLcgFc9|{DTJXJ#ILF-IA>%>5s>t;Du-6e!Lnm;S) z_2`gK^6n)mjXre9H?~VcKZNcV^`88EEO^8}xB8y?K$8XhjN>=l7DO|LFY%l?=tn<1 z&PK+L?7wrR8v6DHM(owZX{)>If@x-1lUDXBpV6EIc>sKG_`C2i;n%^Jg8u@a0eTv` z5&8%^0DKp`4*Ugs8v6*i3w*&Gp+{ z0KN-e2mXRRjeP{%1-@X8&@ZSteU>F@VA)BZI-92-BFFqz!B+>@s0k*59ileRJ{beVhX%tMWu*EwDe5#&lV7MGDEBrk3@bz(YMczWyW&Tn^-iTZc`4Msw z3RCX2bBp_q=CH^ck*gs;LQaA_0KPZ;UHF*r>)=ble}T^cJw2)$ z|JFy)0pPpfb>J`9)7VGAUEmAm2>pV(G-XKNk0L-jl#(~YCZ@$s#nPui6jJ-^d5 zPb!QgCuSsWa|@>YO^-J0jgO=iT8+{0KN-e2mXRRjeP{%1-@X8&@ZUVJ6$og zj;1}d;_{o*9VOb##`;*1cyKgDiND}};wISKLr_}=h$;bX$DgD(aD1wI4xG;|~M5p)3fE_faI z3-&bj5pWmyf;mFJpf0-?2TTu=wRj50 z-@LcvwbKC|_ONk&c!S?a9$7JGfrD${=s6FZXFzU`d>T0{@>0l6WgJ-8 zkNzWRS}RA-OW|A)&cEQC2hKAfw?{sW92R*aay8^f$Vrd~!1so~3m+4H9egSHFYpB4G3!8O}@LToBH` z;G74}Ga$D|K8+j}c_VT)%d>I zr?HQKyTBLB5&8vnF}$Ux-Yw_*k293zU$gwikKy3>{Ub1PhSI^x$IHuUz!e_h!Xm^{PL9%!{9nR`LwC2Qt(f7EH=kfWtKW>sn*S?lwvMqV! z=XCch;XSV7e*8Z8eB9svaO&986R(UukL$P}&*Ss(`FI}haUJ*L_r>pz>-c=U$93HQ z&-&o={`vd-^L}kj&M|^x|8J+R|Ep(y{~i9X1J?iJ4E8@C|MwHU|NZe7BOmZT&-ecG k`G4=`|NgxA5|00^x3x9@829_Ou_J%3j{NJr?DxC>1Ij;2^Z)<= diff --git a/xfusion/train/basicsr/metrics/psnr_ssim.py b/xfusion/train/basicsr/metrics/psnr_ssim.py index 25f4f42..423f276 100644 --- a/xfusion/train/basicsr/metrics/psnr_ssim.py +++ b/xfusion/train/basicsr/metrics/psnr_ssim.py @@ -18,8 +18,7 @@ import torch import torch.nn.functional as F -from xfusion.train.basicsr.metrics.metric_util import reorder_image, to_y_channel -from xfusion.train.basicsr.utils.color_util import rgb2ycbcr_pt +from xfusion.train.basicsr.metrics.metric_util import reorder_image from xfusion.train.basicsr.utils.registry import METRIC_REGISTRY @@ -51,8 +50,7 @@ def calculate_psnr(img, img2, crop_border, input_order='HWC', test_y_channel=Fal img2 = img2[crop_border:-crop_border, crop_border:-crop_border, ...] if test_y_channel: - img = to_y_channel(img) - img2 = to_y_channel(img2) + raise Exception('not implemented yet...') img = img.astype(np.float64) img2 = img2.astype(np.float64) @@ -86,8 +84,7 @@ def calculate_psnr_pt(img, img2, crop_border, test_y_channel=False, **kwargs): img2 = img2[:, :, crop_border:-crop_border, crop_border:-crop_border] if test_y_channel: - img = rgb2ycbcr_pt(img, y_only=True) - img2 = rgb2ycbcr_pt(img2, y_only=True) + raise Exception('not implemented yet...') img = img.to(torch.float64) img2 = img2.to(torch.float64) @@ -131,8 +128,7 @@ def calculate_ssim(img, img2, crop_border, input_order='HWC', test_y_channel=Fal img2 = img2[crop_border:-crop_border, crop_border:-crop_border, ...] if test_y_channel: - img = to_y_channel(img) - img2 = to_y_channel(img2) + raise Exception('not implemented yet...') img = img.astype(np.float64) img2 = img2.astype(np.float64) @@ -172,8 +168,7 @@ def calculate_ssim_pt(img, img2, crop_border, test_y_channel=False, **kwargs): img2 = img2[:, :, crop_border:-crop_border, crop_border:-crop_border] if test_y_channel: - img = rgb2ycbcr_pt(img, y_only=True) - img2 = rgb2ycbcr_pt(img2, y_only=True) + raise Exception('not implemented yet...') img = img.to(torch.float64) img2 = img2.to(torch.float64) diff --git a/xfusion/train/train_EDVR_L_x4_STF_REDS_mono_prepost_xfull_plus_mult_poiss_u1v3.yml b/xfusion/train/train_EDVR_L_x4_STF_REDS_mono_prepost_xfull_plus_mult_poiss_u1v3.yml new file mode 100644 index 0000000..830ae5e --- /dev/null +++ b/xfusion/train/train_EDVR_L_x4_STF_REDS_mono_prepost_xfull_plus_mult_poiss_u1v3.yml @@ -0,0 +1,141 @@ +# Copyright 2018-2022 BasicSR Authors + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Modifications Copyright 2025 xfusion authors + +# general settings +name: 104_EDVR_L_x4_STF_REDS_600k_B4G8_valREDS4_103pretrain_wandb_mono_prepost_xfull_plus_mult_poiss +model_type: EDVRModel +scale: 4 +num_gpu: 4 # set num_gpu: 0 for cpu mode +manual_seed: 10 + +# dataset and data loader settings +datasets: + train: + name: XRAY + type: XrayDatasetSTF + flag: grayscale + dataroot_gt: [/train/HR] + dataroot_lq: [/train/LR] + dataroot_flow: ~ + meta_info_file: [meta_info_xray_GT_u1_.txt] + val_partition: official #REDS4 # set to 'official' when use the official validation partition + io_backend: + type: disk + + num_frame: 3 + num_frame_hi: 2 + gt_size: 256 + interval_list: [1] + hi_interval_mult: 1 + random_reverse: false + use_hflip: true + use_rot: true + + # data loader + num_worker_per_gpu: 3 + batch_size_per_gpu: 1 + dataset_enlarge_ratio: 400 + prefetch_mode: ~ + + val: + name: XRAY + type: XrayVideoTestDatasetSTF + flag: grayscale + dataroot_gt: [/val/HR] + dataroot_lq: [/val/LR] + meta_info_file: [meta_info_ADDMA_test_GT.txt] #basicsr/data/meta_info/meta_info_REDS4_test_GT.txt + # change to 'meta_info_REDSofficial4_test_GT' when use the official validation partition + io_backend: + type: disk + + cache_data: false + num_frame: 3 + num_frame_hi: 2 + padding: reflection_circle + +# network structures +network_g: + #ema_decay: 0.999 + type: EDVRSTF + num_in_ch: 1 + num_out_ch: 1 + num_feat: 128 + num_frame: 3 + num_frame_hi: 2 + deformable_groups: 8 + num_extract_block: 5 + num_reconstruct_block: 40 + center_frame_idx: ~ + hr_in: false + with_predeblur: false + with_tsa: true + +# path +path: + pretrain_network_g: net_g_latest.pth + strict_load_g: true + resume_state: ~ + +# training settings +train: + optim_g: + type: Adam + lr: !!float 1e-4 + weight_decay: 0 + betas: [0.9, 0.99] + + scheduler: + type: CosineAnnealingRestartLR + periods: [300000] + restart_weights: [1] + eta_min: !!float 1e-7 + + total_iter: 300000 + warmup_iter: -1 # no warm up + tsa_iter: 0 + dcn_lr_mul: 1 + + # losses + pixel_opt: + type: CharbonnierLoss + loss_weight: 1.0 + reduction: sum + +# validation settings +val: + val_freq: !!float 5e3 + save_img: false + + metrics: + psnr: # metric name, can be arbitrary + type: calculate_psnr + crop_border: 0 + test_y_channel: false + +# logging settings +logger: + print_freq: 100 + save_checkpoint_freq: !!float 5e3 + use_tb_logger: true + wandb: + project: ~ + resume_id: ~ + +# dist training settings +dist_params: + backend: nccl + port: 29500 + +find_unused_parameters: true diff --git a/xfusion/train/train_SwinVR_VSTFx4_xray_scratch_remove1_u1v3_contextmap.yml b/xfusion/train/train_SwinVR_VSTFx4_xray_scratch_remove1_u1v3_contextmap.yml new file mode 100644 index 0000000..e06fd34 --- /dev/null +++ b/xfusion/train/train_SwinVR_VSTFx4_xray_scratch_remove1_u1v3_contextmap.yml @@ -0,0 +1,143 @@ +# Copyright 2018-2022 BasicSR Authors + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# Modifications Copyright 2025 xfusion authors + +# general settings +name: train_SwinVR_STFx4_scratch_P48W8_DIV2K_500k_B4G8_s18_x2_0.4 +model_type: SwinIRModel +scale: 4 +num_gpu: 4 +manual_seed: 0 + +# dataset and data loader settings +datasets: + train: + name: XRAY + type: XrayDatasetSTF + flag: grayscale + dataroot_gt: [/train/HR] + dataroot_lq: [/train/LR] + dataroot_context: [/train/LRC] + dataroot_flow: ~ + meta_info_file: [meta_info_xray_GT_u1.txt] + meta_info_context: [context_map.csv] + use_nearest_context: true + val_partition: official #REDS4 # set to 'official' when use the official validation partition + io_backend: + type: disk + + num_frame: 3 + num_frame_hi: 2 + gt_size: 96 + interval_list: [1] + hi_interval_mult: 1 + random_reverse: false + use_hflip: true + use_rot: true + use_local_renorm: false + poisson_noise_b0_exp: [1,7] + + # data loader + num_worker_per_gpu: 3 + batch_size_per_gpu: 1 + dataset_enlarge_ratio: 400 + prefetch_mode: ~ + + val: + name: XRAY + type: XrayVideoTestDatasetSTF + flag: grayscale + dataroot_gt: [/val/HR] + dataroot_lq: [/val/LR] + meta_info_file: [meta_info_ADDMA_test_GT.txt] #basicsr/data/meta_info/meta_info_REDS4_test_GT.txt + # change to 'meta_info_REDSofficial4_test_GT' when use the official validation partition + io_backend: + type: disk + + cache_data: false + num_frame: 3 + num_frame_hi: 2 + padding: reflection_circle + use_local_renorm: false + +# network structures +network_g: + type: PatchTransformerSTF + upscale: 4 + num_in_ch: 1 + num_frame: 3 + #img_size: 48 + window_size: [5,8,8] + #img_range: 1. + depths: [2,2,18,2] + embed_dim: 192 + num_heads: [3,3,3,3] + mlp_ratio: 2 + upsampler: 'pixelshuffle' + resi_connection: '1conv' + +# path +path: + pretrain_network_g: net_g_latest.pth + strict_load_g: true + resume_state: ~ + +# training settings +train: + ema_decay: 0.999 + optim_g: + type: Adam + lr: !!float 2e-4 + weight_decay: 0 + betas: [0.9, 0.99] + + scheduler: + type: CosineAnnealingRestartLR + periods: [300000] + restart_weights: [1] + eta_min: !!float 1e-7 + + total_iter: 300000 + warmup_iter: -1 # no warm up + + # losses + pixel_opt: + type: L1Loss + loss_weight: 1.0 + reduction: mean + +# validation settings +val: + val_freq: !!float 5e3 + save_img: false + + metrics: + psnr: # metric name, can be arbitrary + type: calculate_psnr + crop_border: 4 + test_y_channel: false + +# logging settings +logger: + print_freq: 100 + save_checkpoint_freq: !!float 5e3 + use_tb_logger: true + wandb: + project: ~ + resume_id: ~ + +# dist training settings +dist_params: + backend: nccl + port: 29500 diff --git a/xfusion/train/train_simple_gray4_VSWINIR_xray_multinode_ddp_new4.py b/xfusion/train/train_simple_gray4_VSWINIR_xray_multinode_ddp_new4.py index e7ee2ca..631013d 100644 --- a/xfusion/train/train_simple_gray4_VSWINIR_xray_multinode_ddp_new4.py +++ b/xfusion/train/train_simple_gray4_VSWINIR_xray_multinode_ddp_new4.py @@ -68,6 +68,8 @@ def parse_options_(root_path,args,opt): opt['datasets']['train']['interval_list'] = args.interval_list opt['datasets']['train']['hi_interval_mult'] = args.hi_interval_mult opt['datasets']['val']['num_frame'] = args.num_frame + if args.poisson_noise_b0_exp is not None: + opt['datasets']['train']['poisson_noise_b0_exp'] = args.poisson_noise_b0_exp opt['network_g']['num_frame'] = args.num_frame opt['network_g']['window_size'][0] += (args.num_frame - 3) opt['network_g']['window_size'][1:] = [args.window_size_spatial] * len(opt['network_g']['window_size'][1:]) @@ -257,91 +259,91 @@ def train_pipeline(args,opt): opt['root_path'] = root_path if type(args.dir_hi_train) is not list: - if str(args.dir_hi_train) != 'none': + if str(args.dir_hi_train) not in ['none','.']: if opt['datasets']['train']['type'] == 'XrayDatasetSTF': opt['datasets']['train']['dataroot_gt'] = [str(args.dir_hi_train)] else: opt['datasets']['train']['dataroot_gt'] = str(args.dir_hi_train) else: assert opt['datasets']['train']['type'] == 'XrayDatasetSTF' - if not any([True if str(dh) == 'none' else False for dh in args.dir_hi_train]): + if not any([True if str(dh) in ['none','.'] else False for dh in args.dir_hi_train]): opt['datasets']['train']['dataroot_gt'] = [str(dh) for dh in args.dir_hi_train] if type(args.dir_lo_train) is not list: - if str(args.dir_lo_train) != 'none': + if str(args.dir_lo_train) not in ['none','.']: if opt['datasets']['train']['type'] == 'XrayDatasetSTF': opt['datasets']['train']['dataroot_lq'] = [str(args.dir_lo_train)] else: opt['datasets']['train']['dataroot_lq'] = str(args.dir_lo_train) else: assert opt['datasets']['train']['type'] == 'XrayDatasetSTF' - if not any([True if str(dl) == 'none' else False for dl in args.dir_lo_train]): + if not any([True if str(dl) in ['none','.'] else False for dl in args.dir_lo_train]): opt['datasets']['train']['dataroot_lq'] = [str(dl) for dl in args.dir_lo_train] if type(args.dir_hi_val) is not list: - if str(args.dir_hi_val) != 'none': + if str(args.dir_hi_val) not in ['none','.']: if opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF': opt['datasets']['val']['dataroot_gt'] = [str(args.dir_hi_val)] else: opt['datasets']['val']['dataroot_gt'] = str(args.dir_hi_val) else: assert opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF' - if not any([True if str(dh) == 'none' else False for dh in args.dir_hi_val]): + if not any([True if str(dh) in ['none','.'] else False for dh in args.dir_hi_val]): opt['datasets']['val']['dataroot_gt'] = [str(dh) for dh in args.dir_hi_val] if type(args.dir_lo_val) is not list: - if str(args.dir_lo_val) != 'none': + if str(args.dir_lo_val) not in ['none','.']: if opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF': opt['datasets']['val']['dataroot_lq'] = [str(args.dir_lo_val)] else: opt['datasets']['val']['dataroot_lq'] = str(args.dir_lo_val) else: assert opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF' - if not any([True if str(dl) == 'none' else False for dl in args.dir_lo_val]): + if not any([True if str(dl) in ['none','.'] else False for dl in args.dir_lo_val]): opt['datasets']['val']['dataroot_lq'] = [str(dl) for dl in args.dir_lo_val] if type(args.path_train_meta_info_file) is not list: - if str(args.path_train_meta_info_file) != 'none': + if str(args.path_train_meta_info_file) not in ['none','.']: if opt['datasets']['train']['type'] == 'XrayDatasetSTF': opt['datasets']['train']['meta_info_file'] = [str(args.path_train_meta_info_file)] else: opt['datasets']['train']['meta_info_file'] = str(args.path_train_meta_info_file) else: assert opt['datasets']['train']['type'] == 'XrayDatasetSTF' - if not any([True if str(pt) == 'none' else False for pt in args.path_train_meta_info_file]): + if not any([True if str(pt) in ['none','.'] else False for pt in args.path_train_meta_info_file]): opt['datasets']['train']['meta_info_file'] = [str(pt) for pt in args.path_train_meta_info_file] if type(args.path_val_meta_info_file) is not list: - if str(args.path_val_meta_info_file) != 'none': + if str(args.path_val_meta_info_file) not in ['none','.']: if opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF': opt['datasets']['val']['meta_info_file'] = [str(args.path_val_meta_info_file)] else: opt['datasets']['val']['meta_info_file'] = str(args.path_val_meta_info_file) else: assert opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF' - if not any([True if str(pv) == 'none' else False for pv in args.path_val_meta_info_file]): + if not any([True if str(pv) in ['none','.'] else False for pv in args.path_val_meta_info_file]): opt['datasets']['val']['meta_info_file'] = [str(pv) for pv in args.path_val_meta_info_file] if type(args.dataroot_context) is not list: - if str(args.dataroot_context) != 'none': + if str(args.dataroot_context) not in ['none','.']: if opt['datasets']['train']['type'] == 'XrayDatasetSTF': opt['datasets']['train']['dataroot_context'] = [str(args.dataroot_context)] else: opt['datasets']['train']['dataroot_context'] = str(args.dataroot_context) else: assert opt['datasets']['train']['type'] == 'XrayDatasetSTF' - if not any([True if str(dc) == 'none' else False for dc in args.dataroot_context]): + if not any([True if str(dc) in ['none','.'] else False for dc in args.dataroot_context]): opt['datasets']['train']['dataroot_context'] = [str(dc) for dc in args.dataroot_context] if type(args.meta_info_context) is not list: - if str(args.meta_info_context) != 'none': + if str(args.meta_info_context) not in ['none','.']: if opt['datasets']['train']['type'] == 'XrayDatasetSTF': opt['datasets']['train']['meta_info_context'] = [str(args.meta_info_context)] else: opt['datasets']['train']['meta_info_context'] = str(args.meta_info_context) else: assert opt['datasets']['train']['type'] == 'XrayDatasetSTF' - if not any([True if str(mic) == 'none' else False for mic in args.meta_info_context]): + if not any([True if str(mic) in ['none','.'] else False for mic in args.meta_info_context]): opt['datasets']['train']['meta_info_context'] = [str(mic) for mic in args.meta_info_context] opt['datasets']['train']['use_nearest_context'] = args.use_nearest_context opt['datasets']['train']['num_worker_per_gpu'] = args.num_workers diff --git a/xfusion/train/train_simple_gray6_xfull_mult_poiss.py b/xfusion/train/train_simple_gray6_xfull_mult_poiss.py index d229ee8..899dccc 100644 --- a/xfusion/train/train_simple_gray6_xfull_mult_poiss.py +++ b/xfusion/train/train_simple_gray6_xfull_mult_poiss.py @@ -36,7 +36,8 @@ def parse_options_(root_path,args,opt): import random from xfusion.utils import _postprocess_yml_value - + if args.force_yml == 'none': + args.force_yml = None # distributed settings if args.launcher == 'none': print(f"debugging...{opt}") @@ -213,6 +214,94 @@ def train_pipeline(args,opt): opt['train']['total_iter'] /= args.iter_div_factor opt['train']['optim_g']['lr'] = args.initial_lr opt['val']['val_freq'] = args.val_freq + + if type(args.dir_hi_train) is not list: + if str(args.dir_hi_train) not in ['none','.']: + if opt['datasets']['train']['type'] == 'XrayDatasetSTF': + opt['datasets']['train']['dataroot_gt'] = [str(args.dir_hi_train)] + else: + opt['datasets']['train']['dataroot_gt'] = str(args.dir_hi_train) + else: + assert opt['datasets']['train']['type'] == 'XrayDatasetSTF' + if not any([True if str(dh) in ['none','.'] else False for dh in args.dir_hi_train]): + opt['datasets']['train']['dataroot_gt'] = [str(dh) for dh in args.dir_hi_train] + + if type(args.dir_lo_train) is not list: + if str(args.dir_lo_train) not in ['none','.']: + if opt['datasets']['train']['type'] == 'XrayDatasetSTF': + opt['datasets']['train']['dataroot_lq'] = [str(args.dir_lo_train)] + else: + opt['datasets']['train']['dataroot_lq'] = str(args.dir_lo_train) + else: + assert opt['datasets']['train']['type'] == 'XrayDatasetSTF' + if not any([True if str(dl) in ['none','.'] else False for dl in args.dir_lo_train]): + opt['datasets']['train']['dataroot_lq'] = [str(dl) for dl in args.dir_lo_train] + + if type(args.dir_hi_val) is not list: + if str(args.dir_hi_val) not in ['none','.']: + if opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF': + opt['datasets']['val']['dataroot_gt'] = [str(args.dir_hi_val)] + else: + opt['datasets']['val']['dataroot_gt'] = str(args.dir_hi_val) + else: + assert opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF' + if not any([True if str(dh) in ['none','.'] else False for dh in args.dir_hi_val]): + opt['datasets']['val']['dataroot_gt'] = [str(dh) for dh in args.dir_hi_val] + + if type(args.dir_lo_val) is not list: + if str(args.dir_lo_val) not in ['none','.']: + if opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF': + opt['datasets']['val']['dataroot_lq'] = [str(args.dir_lo_val)] + else: + opt['datasets']['val']['dataroot_lq'] = str(args.dir_lo_val) + else: + assert opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF' + if not any([True if str(dl) in ['none','.'] else False for dl in args.dir_lo_val]): + opt['datasets']['val']['dataroot_lq'] = [str(dl) for dl in args.dir_lo_val] + + if type(args.path_train_meta_info_file) is not list: + if str(args.path_train_meta_info_file) not in ['none','.']: + if opt['datasets']['train']['type'] == 'XrayDatasetSTF': + opt['datasets']['train']['meta_info_file'] = [str(args.path_train_meta_info_file)] + else: + opt['datasets']['train']['meta_info_file'] = str(args.path_train_meta_info_file) + else: + assert opt['datasets']['train']['type'] == 'XrayDatasetSTF' + if not any([True if str(pt) in ['none','.'] else False for pt in args.path_train_meta_info_file]): + opt['datasets']['train']['meta_info_file'] = [str(pt) for pt in args.path_train_meta_info_file] + + if type(args.path_val_meta_info_file) is not list: + if str(args.path_val_meta_info_file) not in ['none','.']: + if opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF': + opt['datasets']['val']['meta_info_file'] = [str(args.path_val_meta_info_file)] + else: + opt['datasets']['val']['meta_info_file'] = str(args.path_val_meta_info_file) + else: + assert opt['datasets']['val']['type'] == 'XrayVideoTestDatasetSTF' + if not any([True if str(pv) in ['none','.'] else False for pv in args.path_val_meta_info_file]): + opt['datasets']['val']['meta_info_file'] = [str(pv) for pv in args.path_val_meta_info_file] + + if type(args.dataroot_context) is not list: + if str(args.dataroot_context) not in ['none','.']: + if opt['datasets']['train']['type'] == 'XrayDatasetSTF': + opt['datasets']['train']['dataroot_context'] = [str(args.dataroot_context)] + else: + opt['datasets']['train']['dataroot_context'] = str(args.dataroot_context) + else: + assert opt['datasets']['train']['type'] == 'XrayDatasetSTF' + if not any([True if str(dc) in ['none','.'] else False for dc in args.dataroot_context]): + opt['datasets']['train']['dataroot_context'] = [str(dc) for dc in args.dataroot_context] + + if type(args.meta_info_context) is not list: + if str(args.meta_info_context) not in ['none','.']: + if opt['datasets']['train']['type'] == 'XrayDatasetSTF': + opt['datasets']['train']['meta_info_context'] = [str(args.meta_info_context)] + else: + opt['datasets']['train']['meta_info_context'] = str(args.meta_info_context) + else: + assert opt['datasets']['train']['type'] == 'XrayDatasetSTF' + if not any([True if str(mic) in ['none','.'] else False for mic in args.meta_info_context]): + opt['datasets']['train']['meta_info_context'] = [str(mic) for mic in args.meta_info_context] if args.poisson_noise_b0_exp is not None: opt['datasets']['train']['poisson_noise_b0_exp'] = args.poisson_noise_b0_exp torch.backends.cudnn.benchmark = True From fb4827ea5b2361a04a030ec4d4133425ffeeab8a Mon Sep 17 00:00:00 2001 From: Songyuan Tang Date: Wed, 21 Jan 2026 14:10:40 -0600 Subject: [PATCH 2/3] updated the project information in the readme file --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 5faf56a..e19fa99 100644 --- a/README.md +++ b/README.md @@ -96,8 +96,10 @@ xfusion inference --arch-opt directory/to/model architecture/yaml file -- ### Suggested useage -For virtual experiment data, currently xfusion works for EDVRModel in the single-process mode and SwinIRModel in the multi-process mode. -For actual experiment data, xfusion works for both models in a unified way. +* For virtual experiment data, currently xfusion works for EDVRModel in the single-process mode and SwinIRModel in the multi-process mode. +When working on the virtual experiment data, first initialize the software, and then do the inference. + +* For actual experiment data, xfusion works for both models in a unified way. Do an independent initialization every time the model (EDVR-STF or SWIN-XVR) or the type of data (virtual or actual) changes. When working on the actual experiment data, first initialize the software, then do the spatial calibration, and last do the inference. Multiple experiments sharing the same spatial calibration can be grouped to run the model inference. From 0d388f9d72be926929d7e23bfda8752571e80ea7 Mon Sep 17 00:00:00 2001 From: Songyuan Tang Date: Wed, 21 Jan 2026 18:25:08 -0600 Subject: [PATCH 3/3] revised readme file --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index e19fa99..0b392b7 100644 --- a/README.md +++ b/README.md @@ -95,7 +95,7 @@ For actual experiment data, run xfusion inference --arch-opt directory/to/model architecture/yaml file --img-class name of high-speed experiment --model_file [path/to/model file] --traverse_mode double or continuous --tfm_file [path/to/rigid/body/spatial/transformation/file] --meta_file_scale [path/to/nonuniform/scaling/parameters] --input_dir [root/directory/to/high/speed/experiment/data] --case_list experiment subdirectory names -### Suggested useage +### Suggested usage * For virtual experiment data, currently xfusion works for EDVRModel in the single-process mode and SwinIRModel in the multi-process mode. When working on the virtual experiment data, first initialize the software, and then do the inference.