[BUG] Representativeness gets hard coded to "local" normalization

[tigmoore]

🗃️ Describe the problem

In the _cluster_ranker function, the norm method can be set to either 'global' or 'local'. However, within the function itself, this value gets hard coded to 'local'. Then, with an elif statement, the local normalization branch ends up setting centerness_ranking = sample_dists. This is the inverse of what should be happening, as distance goes up, we expect that centerness ranking should decrease.

🖥️ Operating System Platform & Distribution

• Linux
• MacOS
• Windows
• Other (please specify)

💻 System Details

---

---OS---
Linux BOPA-Nitro-ANV15-52 7.0.0-29-generic #29~24.04.2-Ubuntu SMP PREEMPT_DYNAMIC Wed Aug 12 17:25:56 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
---HWinfo---
/0/0 memory 31GiB System memory
/0/1 processor 13th Gen Intel(R) Core(TM) i7-13620H
/0/100/2 /dev/fb0 display Raptor Lake-P [UHD Graphics]
/0/100/6/0 display NVIDIA Corporation
/0/100/e storage Volume Management Device NVMe RAID Controller Intel Corporation
/0/100/14.2 memory RAM memory
/0/6.2/0 storage 3500 NVMe SSD
/1 /dev/nvme0 storage Micron_3500_MTFDKBA1T0TGD-1BK15ABYY

🌐 Browser

Not applicable

⤵️ Installation type

---

• Package (pip)
• Source (git)

⌨️ Code to reproduce issue

---

"""
Minimal reproduction: fiftyone.brain.compute_representativeness() (method=
"cluster-center", the default) returns values with inverted semantics.

The field is documented/named as "representativeness": high value = sample is
close to a cluster center = typical/representative. In practice, the value
returned is the sample's *distance* to its cluster center, normalized within
that cluster -- i.e. an outlier score, not a representativeness score. A
sample sitting exactly on its cluster's center gets a value near 0; a sample
at the far edge of its cluster gets a value near 1.

Root cause (traced in fiftyone/brain/internal/core/representativeness.py,
function `_cluster_ranker`, lines ~186-231):

    centerness_ranking = 1 / (1 + sample_dists)   # correct: high = central

    norm_method = "local"          # hardcoded, ignores the `norm_method` arg
    if norm_method == "global":
        centerness_ranking = centerness_ranking / centerness_ranking.max()
    elif norm_method == "local":
        for unique_id in unique_ids:
            cluster_indices = np.where(cluster_ids == unique_id)[0]
            cluster_dists = sample_dists[cluster_indices]
            cluster_dists /= cluster_dists.max()
            sample_dists[cluster_indices] = cluster_dists
        centerness_ranking = sample_dists   # <-- overwrites centerness with
                                             #     normalized raw distance

`norm_method` is hardcoded on the line right before the branch, so the
"global" branch is unreachable regardless of any argument passed through the
public API. In the "local" branch that always runs, the final assignment
discards the correctly-computed `centerness_ranking` from two lines above and
replaces it with the (per-cluster-normalized) raw *distance* -- the inverse
of centerness.

This script builds a synthetic 2D embedding space (1000 points in two
well-separated Gaussian blobs) and shows that the returned
"representativeness" field is essentially a rescaling of each point's
distance to its own KMeans cluster center -- not the inverse of it as the
name/docs imply. It independently re-runs the same clustering
(KMeans, n_clusters=20, random_state=1234 -- matching the internal
algorithm's hardcoded parameters) purely to compute each point's true
distance-to-assigned-center for comparison; this does not depend on any
internal fiftyone.brain function.

No external data/images required -- embeddings are synthetic and a tiny
blank image is generated per sample so FiftyOne has a valid filepath.
"""

import os
import tempfile

import numpy as np
from PIL import Image
from sklearn.cluster import KMeans

import fiftyone as fo
import fiftyone.brain as fob

np.random.seed(0)

# Two well-separated Gaussian blobs, 500 points each -- large enough that
# KMeans(n_clusters=20) (fiftyone-brain's hardcoded cluster count) doesn't
# produce degenerate near-empty clusters.
cluster_a = np.random.normal(loc=[0, 0], scale=1.0, size=(500, 2))
cluster_b = np.random.normal(loc=[30, 30], scale=1.0, size=(500, 2))
embeddings = np.vstack([cluster_a, cluster_b]).astype(np.float32)

tmpdir = tempfile.mkdtemp()
filepaths = []
for i in range(len(embeddings)):
    path = os.path.join(tmpdir, f"{i}.jpg")
    Image.new("RGB", (4, 4)).save(path)
    filepaths.append(path)

dataset = fo.Dataset()
dataset.add_samples([fo.Sample(filepath=p) for p in filepaths])

fob.compute_representativeness(
    dataset,
    embeddings=embeddings,
    representativeness_field="representativeness",
)

returned = np.array(dataset.values("representativeness"))

# Recompute each point's true distance to its own cluster center, using the
# exact same clustering fiftyone-brain uses internally (KMeans, N=20,
# random_state=1234; see _cluster_ranker in representativeness.py).
clusterer = KMeans(n_clusters=20, random_state=1234).fit(embeddings)
dists_to_center = np.linalg.norm(
    embeddings - clusterer.cluster_centers_[clusterer.labels_], axis=1
)

r = np.corrcoef(returned, dists_to_center)[0, 1]
print("Correlation between returned 'representativeness' and each point's")
print("true distance to its own cluster center:")
print(f"  r = {r:.4f}")
print(
    "  (expected ~ -1 if 'representativeness' means centerness/typicality;"
    " observed ~ +1 confirms the field is actually a distance/outlier score)\n"
)

closest_idx = np.argmin(dists_to_center)
farthest_idx = np.argmax(dists_to_center)
print(
    f"Point closest to its cluster center (dist={dists_to_center[closest_idx]:.4f}): "
    f"representativeness = {returned[closest_idx]:.4f}  (expected: high, near 1)"
)
print(
    f"Point farthest from its cluster center (dist={dists_to_center[farthest_idx]:.4f}): "
    f"representativeness = {returned[farthest_idx]:.4f}  (expected: low, near 0)"
)

dataset.delete()

🐍 What version of Python?

---

Python 3.12.3

🔖 What version of FiftyOne?

FiftyOne 1.21.0

🪵 Other info/logs

---

No response

🚀 Willingness to contribute

---

Yes, I can contribute a fix for this bug independently

📸 Upload screenshots

---

No response