Skip to content

perf(unique): verifyUniqueWithinMutation is O(N^2), dominates @unique cost on batched mutations #9814

Description

@matthewmcneely

What

@unique write cost grows quadratically with the number of unique-predicate edges in a single
mutation. At 16k edges in one mutation a write takes 9.7s against 142ms for the same mutation on a
plain @index(exact) predicate.

The cost sits entirely in verifyUniqueWithinMutation, not in the injected uniqueness queries.

Measurements

v25.3.8, one mutation, p: string @index(exact) . against p: string @unique @index(exact) .:

edges/mutation plain @unique ratio growth per doubling
500 7.1 ms 19.4 ms 2.7x
1,000 12.6 ms 60.4 ms 4.8x
2,000 24.0 ms 203.6 ms 8.5x
4,000 39.2 ms 554.5 ms 14.1x 2.91x
8,000 74.1 ms 2,242.7 ms 30.3x 4.04x
16,000 141.6 ms 9,762.4 ms 68.9x 4.35x

Plain grows about 2x per doubling. @unique converges on 4x, which is O(N^2).

Single-edge mutations are unaffected: 1.66 ms/mutation with @unique against 1.77 ms plain over 300
sequential inserts. The cost scales with edges-per-mutation rather than with total write volume or
how hot the predicate is.

Where the time goes

The server latency breakdown separates the two candidate costs, because addQueryIfUnique runs
during parsing and the duplicate scan runs during processing:

N phase plain @unique
8,000 parsing 1.1 ms 1.4 ms
8,000 processing 93.4 ms 2,249.4 ms

Parsing stays flat, so injecting one eq() query per edge is cheap. The per-pair constant confirms
the nested loop as the source: 64M pairs / 2,249 ms at N=8k and 256M pairs / 9,762 ms at N=16k both
work out to 35-38 ns per pair.

Cause

verifyUniqueWithinMutation in edgraph/server.go iterates qc.uniqueVars (a
map[uint64]uniquePredMeta) inside itself. Two things make it worse than the algorithm requires:

  • the inner loop starts at for j := range qc.uniqueVars rather than i+1, so every pair is
    compared twice
  • dql.TypeValFrom(pred2.ObjectValue) is called on every pair before any match is established

Nested map iteration also explains why the per-pair constant is 35 ns rather than a few.

Suggested fix

Hash (predicate, value) into a set and make a single linear pass, keeping the first-seen subject so
the existing error message is preserved. Starting j at i+1 is a free 2x if a smaller change is
wanted first.

Impact

Live loader's default --batch is 1000, which lands at the ~5x point and never reaches the bad part
of the curve. Application code that builds one large mutation from an import is where this bites.

Numbers are single-threaded HTTP against a dgraph/standalone:v25.3.8 container, so treat the
absolute values as indicative. The scaling exponent is the durable part, and it is CPU-bound work
that will contend across concurrent writers rather than disappear.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions