TRIPSI: Triple-source Realigned Integrated Perceptual Semantic Index

Python 3.10+ | PyTorch | Full-Reference + No-Reference IQA

TRIPSI is a comprehensive Image Quality Assessment (IQA) framework that fuses semantic guidance, multitask vision-language knowledge, and human-aligned perceptual similarity embeddings. By integrating three state-of-the-art deep metrics through rank-based fusion, TRIPSI aims to improve alignment with human judgments of image quality, particularly in scenarios involving semantic importance and color naturalness.

---

Key Components

TOPIQ

TOPIQ uses high-level semantics to guide the network toward semantically important regions (e.g., foreground objects), mirroring how humans prioritize visual information. For TRIPSI, the full-reference version of TOPIQ trained on KADID-10k dataset is used.

LIQE

LIQE is trained to simultaneously perform quality assessment, scene classification, and distortion identification. It possesses the ability to identify specific artifacts, including color-related distortions, contrast issues, and over/under-exposure.

DreamSim

DreamSim is tuned to capture attributes that humans use to determine sameness, such as object pose, perspective, and holistic layout. DreamSim contributes a perceptual similarity signal using embedding cosine similarity between each evaluated image and its reference, capturing structural and color consistency aligned with human perceptual judgments.

---

Fusion Methodology

Rank Normalization

Raw scores from TOPIQ, LIQE, and DreamSim are converted to relative ranks to remove scale differences and place all metrics on a common footing.

Equal-Weight Rank Fusion

The final TRIPSI score is computed as an unweighted sum of normalized ranks. This simple aggregation leverages complementary strengths of semantic fidelity, distortion awareness, and perceptual similarity.

---

Implementation

TRIPSI performs:

• TOPIQ score extraction
• LIQE score extraction
• DreamSim cosine similarity computation
• Rank normalization of all metric outputs
• Equal-weight rank fusion
• Spearman correlation evaluation with MOS
• CSV export of raw and fused scores

---

Installation

Requirements

Install dependencies with:

pip install torch torchvision numpy pandas Pillow scipy timm pyiqa dreamsim

or:

pip install -r requirements.txt

Core Dependencies

Pretrained model weights are downloaded automatically on first use.

Hardware

CUDA-enabled GPU inference is optional but strongly recommended.

---

Datasets and External Inputs

This repository does not contain datasets. Users must provide:

Evaluated Images

A directory containing distorted or colorized images to be assessed.

Reference Images

A directory containing corresponding pristine ground-truth images.

MOS File

A CSV file containing image names and subjective Mean Opinion Scores:

Image,MOS
001_xxx.jpg,4.23
002_xxx.jpg,3.87
...

Supported Dataset Configurations

CIN25

Directory structure:

colorized_images/
original_color_images/
MOS_FINAL_subj.csv

Reference naming convention:

<ID>_orig.JPEG

---

HECD

Directory structure:

HECDImages/
GroundTruth_HECD/
MOS_HECD.csv

Reference naming convention:

<ID>_gt.jpg

---

Configure Paths

Edit paths in the settings section of the script:

EVAL_IMAGE_DIR = "path/to/evaluation_images"
REF_IMAGE_DIR  = "path/to/reference_images"
MOS_FILE       = "path/to/mos.csv"
OUT_CSV        = "results.csv"

Switch between CIN25 and HECD by uncommenting the corresponding block.

---

Usage

Run:

python tripsi.py

---

🎨 Design Motivation: Color Naturalness

The fusion of these metrics makes TRIPSI particularly effective at assessing color naturalness:

• LIQE identifies explicit color-related artifacts.

• TOPIQ ensures that color distortions are assessed within the context of semantically important regions.

• DreamSim provides high-fidelity color recovery and a focus on foreground color, preventing global background shifts from masking critical subject-level color degradations.

---

Citation

If you use this repository, please cite our work:

@Article{Zeger_Grgic_TRIPSI,
AUTHOR = {Žeger, Ivana and Grgic, Sonja},
TITLE = {Pattern Recognition in Semantic Feature Spaces for Image Colorization Quality Assessment},
JOURNAL = {Electronics},
VOLUME = {15},
YEAR = {2026},
NUMBER = {9},
ARTICLE-NUMBER = {1969},
URL = {https://www.mdpi.com/2079-9292/15/9/1969},
ISSN = {2079-9292},
DOI = {10.3390/electronics15091969}
}

---

Acknowledgments

This work builds upon the research of: Zhang et al. (LIQE), Chen et al. (TOPIQ), and Fu et al. (DreamSim) and utilizes the following open-source projects:

• IQA-PyTorch, which provides implementations of multiple image quality assessment models, including TOPIQ and LIQE.

• DreamSim, used for extracting semantic similarity features.

We thank the authors for making their code and models publicly available.