Skip to content

Latest commit

 

History

History
205 lines (141 loc) · 8.42 KB

File metadata and controls

205 lines (141 loc) · 8.42 KB

The For You ranking function, written out in full

Source: github.com/xai-org/x-algorithm @ a389166. Every symbol below maps to a named function or parameter in the repo. Default (non-MPN) path, EnableMpnScoring = false (param.rs:255).


0. Notation

Symbol Meaning
v viewer of the timeline
c candidate post
A the action set Phoenix predicts
Pₐ(c,v) Phoenix-predicted probability of action a
wₐ(c,v) blending weight for action a

1. The complete equation

For a candidate c shown to viewer v:

                 ⎧ γ(c,v) · m(k(c)) · max( φ(Δ(c,v)), τ )   if c = b(v)
   S(c,v)   =    ⎨
                 ⎩ γ(c,v) · m(k(c)) · φ(Δ(c,v))              otherwise

subject to c surviving every filter predicate in §7, where:

   Δ(c,v)  =  Σ  wₐ(c,v) · Pₐ(c,v)                    weighted sum
             a∈A

              ⎧ Δ + ε                     if Δ ≥ 0
   φ(Δ)   =   ⎨                                        offset / normalization
              ⎩ (Δ + N)/T · ε             if Δ < 0

   m(k)   =  (1 − f)·dᵏ + f                            author diversity

   γ(c,v) =  θ  if  ¬net(c,v) ∨ (net(c,v) ∧ (rep(c) ∨ rt(c)));  else 1

Composed, for the non-cold-start case:

   S(c,v) = γ(c,v) · [(1−f)·d^k(c) + f] · φ( Σ wₐ(c,v)·Pₐ(c,v) )

2. Constants (production defaults, runtime-configurable)

   ε = 0.001          NEGATIVE_SCORES_OFFSET          config.rs:40
   d = 0.5            AuthorDiversityDecay            param.rs:228
   f = 0.25           AuthorDiversityFloor            param.rs:234
   θ = 0.75           OonWeightFactor                 param.rs:246
   θ = 0.50           TopicOonWeightFactor (topic requests)   param.rs:266
   N = 367.22         negative_sum   = −Σ(negative weights)
   T = 410.54         total_sum      = positive_sum + negative_sum
                      positive_sum   = 43.32

N and T are computed in ScoringWeights::from_params (ranking_scorer.rs:105-127) and are constants of the weight vector, not of the candidate. Note positive_sum excludes the bidirectional reply boost and the continuous dwell weights, so T does not change with mutual-follow status.


3. The weight vector wₐ

Twenty-six terms, enumerated at ranking_scorer.rs:470-509. Fixed weights:

   favorite            0.5        not_interested     −43.2
   reply               5.0        block_author       −31.2
   retweet             1.0        mute_author        −58.8
   quote               5.0        report            −234.0
   share               2.0        not_dwelled         −0.02
   share_via_dm        5.0
   share_via_copy_link 20.0       cont_dwell_time      0.004
   follow_author       4.0        cont_click_dwell     0.0
   click               0.4        active_secs_5m       0.0
   open_link           0.2
   photo_expand        0.05       profile_click        0.0
   video_open          0.05       dwell                0.0
   vqv                 0.05       quoted_vqv           0.0
   quoted_click        0.05       post_unexplored      0.02

Three weights are conditional, which is where authorship choices enter the math:

   w_reply(c,v) = 5 + 15·1[ ¬rep(c) ∧ ¬rt(c) ∧ mutual(c,v) ]      ranking_scorer.rs:186-193
   w_vqv(c)     = 0.05·1[ video_duration(c) ≥ 10000 ms ]           param.rs:678
   w_pu(c)      = 0.02·1[ net(c,v) ]                               PostUnexploredWeightInNetworkOnly

The reply indicator is the only one an author controls through behavior rather than media choice.


4. φ, the offset map (ranking_scorer.rs:525-533)

   φ: ℝ → ℝ≥0
   φ(Δ) = Δ + ε              for Δ ≥ 0
   φ(Δ) = (Δ + N)/T · ε      for Δ < 0
   φ(Δ) = max(Δ, 0)          if T = 0

Properties, both load-bearing:

  1. φ is monotone increasing, so it never reorders candidates by itself.
  2. φ(Δ) ≥ 0 always. Since Δ ≥ −N in the worst case, Δ < 0 maps into [0, εN/T] = [0, 0.00089], strictly below the ε = 0.001 floor of the non-negative branch.

Point 2 is why the later stages can multiply. m(k) and γ are multiplicative factors, and multiplying a negative score by 0.625 would raise it. φ guarantees non-negativity first, so every downstream multiplier is a genuine penalty.


5. Cold start (author_cold_start.rs)

Eligibility, all required:

   E(v) = { c : ¬rep(c) ∧ ¬rt(c)
              ∧ followers(author(c)) ≤ 1000
              ∧ impressions(c) < 1000
              ∧ age(c) ≤ 86400 s
              ∧ rank(c) < 0.85 · |{c : S₀(c) ≠ 0}| }

Selection and lift:

   b(v) = argmax φ(Δ(c,v))          the single best eligible candidate
          c ∈ E(v)

   τ    = S₀⁽ʲ⁾ ,  j ~ U{15}         the score at slot j of the sorted slate

   S₁(c) = max(S₀(c), τ)   if c = b(v);   else S₀(c)

j is drawn uniformly from [ColdStartSlotMin, ColdStartSlotMax) = [15,16), so j = 15 deterministically at defaults. τ is a rank-relative quantity, not a constant. It has no fixed numeric value because it depends on the other candidates in that request, which is why no numeric "boost size" can be plotted.


6. Order of operations, and a discrepancy worth noting

Execution order in RankingScorer::score (ranking_scorer.rs:822-857):

   S₀ = φ(Δ)                     weighted sum, then offset
   S₁ = cold_start(S₀)           lift one eligible candidate
   k  = rank_within_author(S₁)   contexts computed on POST-lift scores
   S₂ = S₁ · m(k)                author diversity
   S₃ = S₂ · γ                   out-of-network discount

The README prose lists these as "weighted sum, then repeated-author decay, an out-of-network discount, a new-author boost", placing the new-author boost last. The code applies it first, before diversity and before the OON factor. Consequence: a cold-start lift is not final. It is subsequently multiplied by m(k) and γ, so a lifted post that is the author's second in the slate still takes the 0.625, and a lifted reply would take 0.75 (though replies are ineligible anyway).

I am reporting the code order, not the prose order.


7. Filters: the equation is gated, not just scored

Ranking only orders what survives. Display requires:

   shown(c,v) ⟺ Π 1[ ¬fᵢ(c,v) ] = 1
                 i

Pre-scoring predicates (home-mixer/filters/), any one of which zeroes the post: duplicate across sources, hydration failure, age > 48h, viewer's own post, ¬net(c,v) ∧ (rep(c) ∨ rt(c)) (oon_retweet_reply_filter.rs), NSFW SimClusters, repeated reposts, inaccessible subscriber content, previously seen, previously served, muted keyword, blocked or muted author, video excluded, topic mismatch, new-user engagement threshold, inventory holdout.

Post-selection: VFFilter (visibility rules), AncillaryVFFilter, and conversation dedup:

   keep(c) ⟺ c = argmax S₃(c')  over  { c' : conv(c') = conv(c) }

where conv(c) = min(ancestors(c)). This is why a thread contributes exactly one candidate.


8. What cannot be written in closed form

Four components are genuinely outside the equation, and any "full algorithm formula" that claims otherwise is fabricating:

  1. Pₐ(c,v) is a transformer forward pass over the viewer's action-history sequence with candidate isolation. It is the dominant term and has no analytic form. The weights only blend its outputs.
  2. Candidate generation. C(v) = Thunder(v) ∪ PhoenixRetrieval(v) ∪ SimClusters(v), where retrieval is argtop-K ⟨u(v), q(c)⟩ over a two-tower index with residual-quantized semantic IDs, resolved by approximate nearest neighbour search.
  3. VMRanker reorders the selected list afterwards via a separate service (vm-ranker/), whose policy is not reducible to the above.
  4. The ads blender, which reorders posts for ad adjacency in the blending pipeline.

So the honest scope statement: §1 is the complete and exact scoring function for the ranking stage, given Pₐ as input. It is not the whole system.


9. One-line summary

   S = γ · m(k) · φ( Σ wₐ Pₐ )        subject to Π(1 − fᵢ) = 1

A weighted sum of predicted probabilities, mapped to the non-negative reals, then multiplied by two penalties: one for repeating an author within a slate, one for being out of network or being a reply or repost.