Selection, Not Salience: The Shape and Limits of Personalization in Social Highlighting

Public reproducibility bundle for the paper. Self-contained and ready to push to a dedicated public repository.

This is the follow-up to Personal Salience: Highlighting Is Social, but Individuality Lives in Selection (glasp-co/personal-salience), which located where individuality lives. This paper asks about its shape and limits: how large is the selection signal at a coarser altitude (which document, not which span), and can personalization be pushed into the salience layer?

TL;DR

Using Glasp (a social web highlighter with hundreds of thousands of active users and millions of highlighted URLs) and a co-readership identity control (the same document highlighted by many users, which holds document and topic fixed and asks whether a person's own history predicts their marks better than another reader's does), we map personalization across reading altitudes.

• Individuality lives in selection, and it is altitude-invariant. At the document altitude — which documents in a co-reading neighborhood are yours — your own history is a clean, leakage-free predictor: own-vs-other identity gap +0.169 against community negatives and +0.119 against topic-matched hard negatives (both highly significant, every bootstrap positive). This is the same ballpark as the span-level selection gap (+0.14) from Paper I: the selection signal is of comparable magnitude across altitudes (+0.12 to +0.17), and about a third of it is coarse topic.
• It is mostly thematic, and content-grounded (not a title artifact). Representing each candidate by its article-content centroid instead of its title, a reader's content profile still predicts their held-out documents well above chance (own−random +0.154 hard, CI [0.043, 0.283]). The identity gap stays positive but is underpowered at that smaller N.
• Personalization does not help at the salience layer. A two-stage personalized auto-highlight (an impersonal Stage 1 proposes K candidate sentences; a personal Stage 2 re-ranks them) does not beat its impersonal baseline. Two off-the-shelf zero-shot LLMs (including a frontier model) recover highlight locations worse than a trivial lead baseline, and personal re-ranking is beaten by the impersonal salience order.
• The null is not a Stage-1 ceiling artifact. On the highest-recall candidate pool (the crowd union), personal re-ranking still loses to the salience order by −0.118 at K=10 and −0.135 at K=20 (every bootstrap negative). Raising recall enlarges the loss — the opposite of what a ceiling would produce.
• Popularity is altitude-dependent, which is itself the finding. Crowd popularity is the strongest Stage-1 generator at the sentence level, yet near or below chance at the span and document altitudes. Popularity is diagnostic of what is salient but anti-diagnostic of which salient thing is an individual's.
• A reusable methodological caution. A control-in-negatives bias inflated our own document gap to a spurious +0.227 until audited: the identity controls' own documents sat in the candidate negatives, so scoring by a control's profile ranked its own documents to the top and dragged the controls below the random floor. The diagnostic is a control scoring below random; the fix (exclude control documents from the negatives, plus a near-duplicate guard) gives the honest +0.169.

The take-away for products: a generic auto-highlight is the right tool at the sentence level; personalization pays at selection (which documents/sections/topics to surface), and even there the signal is modest and largely thematic. Going beyond the shared salience layer is better approached by aggregating individuals than by personalizing them harder.

Results (mean average precision; cluster bootstrap)

Experiment A — document selection (519 pairs, 489 users, 94 communities):

Scorer	community neg.	hard (topic-ctrl) neg.
own (A's history)	0.529	0.411
near (closest peer)	0.388	0.299
other (control B)	0.360	0.292
random floor	0.310	0.320
crowd (in-pool popularity)	0.228	0.224
leaked profile	0.565	0.448
own − other (identity gap)	+0.169 [0.141, 0.199]	+0.119 [0.094, 0.146]

Experiment B — sentence-level two-stage auto-highlight (~1,011 pairs, 867 users, 60 documents). Stage-1 recall@10: crowd union 0.42, lead 0.29, generic 0.22, LLM 0.22, random 0.19. Within-pool personal re-rank loses to the impersonal salience order on every pool: LLM pool −0.05 [−0.092, −0.008]; crowd-union pool −0.118 (K=10) and −0.135 (K=20), every bootstrap negative.

All effect sizes carry 95% cluster-bootstrap confidence intervals by document and by user.

Contents

• paper.tex — the paper (compile with pdflatex / arXiv).
• paper.pdf — the compiled paper (9 pages, 1 figure, 3 tables).
• figures/ — the altitude figure (PDF + PNG): the own-vs-other identity gap at each reading altitude, showing the salience whisper (+0.017) versus the consistent selection signal (~+0.13). Carries 95% cluster-bootstrap error bars.

Reproducibility

The paper reports every effect size with a 95% cluster-bootstrap confidence interval by document and, separately, by user. The data-extraction and scoring pipeline runs against Glasp's private user data and is not released; per-pair results derive from individual highlighting behavior and are therefore not published. The cluster-bootstrap estimator and aggregate per-pair statistics are available to researchers on reasonable request.

Citation

@misc{nakayashiki2026selection,
  title  = {Selection, Not Salience: The Shape and Limits of
            Personalization in Social Highlighting},
  author = {Nakayashiki, Kazuki and Watanabe, Keisuke},
  year   = {2026},
  eprint = {2606.10398},
  archivePrefix = {arXiv},
  primaryClass  = {cs.IR}
}

License

Paper (paper.tex, paper.pdf) and figures: CC BY 4.0. See LICENSE.