Status: draft
Depends on: gtfs-sqljs (injected instance, not owned)
Given a user-defined grouping of GTFS routes, produce a flat, ordered, deduplicated inventory of every stop in the feed, partitioned across those groups, plus a reason-tagged list of every stop that did not make it in.
The primary use case is data QA: an auditor works through the inventory stop by stop. Consequently:
- Completeness outranks elegance. A stop missing from the inventory never gets audited. Silent drops are the one unacceptable failure mode.
- Every placement must be explainable. "Why is this stop here?" needs an answer.
- The residual list is a first-class output, not an edge case.
The library is headless. It returns data structures; formatting (CSV, UI, JSON) is the caller's concern.
The result is a pure function of (feed, options, grouping). There is no per-record correction layer and no hidden state. Identical inputs always yield byte-identical output; all internal iteration is over sorted keys.
This is a deliberate constraint, and §1.3 lists what it costs.
- No UI, no CSV/spreadsheet writing.
- No geometry-based ordering (
shapes.txtis not consulted). - A route belongs to at most one group. Splitting a route's two directions across two groups is explicitly out of scope (see §6.3).
- No cross-feed-version diffing. The library emits a stable
auditKeyper stop so the caller can implement it. - No GTFS-RT.
Three capabilities are absent by design. Each is cheap to add later; none is half-present.
| Omitted | Consequence |
|---|---|
| Per-stop overrides (force a stop into a group or position) | Placement is fully derived. A grouping that orders badly must be fixed by changing group or route order, not by hand-editing the result. |
| Cluster locks (force a merge or split) | No escape hatch for a clustering mistake. Clustering is tunable only globally, via maxDistanceMeters, linkage and normalizeName. This raises the stakes on the conservative defaults in §6.1 — a false merge can now only be prevented, not corrected. |
| Service-date filtering | All trips are always considered. Stops served only by school-day, seasonal or one-off detour trips are included with no distinction. Completeness is maximal; ordering may absorb some noise from unusual patterns. |
There is no notion of a "rare" pattern. If a trip exists in the feed, its stops are
valid inventory. Frequency is reported (PatternInfo.tripCount) but never used to
filter or downweight.
The unit of inventory is a canonical stop (a "key"), not a stop_id. One physical
location served in two directions is typically 2+ stop_ids; the inventory must list
it once.
Two mechanisms, in strict priority order:
parent_station— authoritative and frozen. All children of a station collapse to the station's key.- Name + proximity fallback — applies only to parentless stops. Exact match on normalized name, clustered by distance.
The fallback never attaches a parentless stop to an existing station cluster. Feed authors' explicit modelling is not overridden by a heuristic.
A group is a named, ordered bucket of canonical stops, produced from an ordered list of routes. Groups themselves are ordered.
The library assigns no semantics to a group. The reference use case is a geographic
corridor, but nothing in the algorithm assumes that: a disconnected route inside a
group yields a warning diagnostic, never an error, and never blocks output.
Each route carries a reference direction — the direction whose stop order defines the reading order of that route's segment.
Both directions are always processed. The reference direction controls ordering only, never inclusion. A wrong reference direction cannot lose stops; it can only reverse a segment's reading order.
A canonical stop is claimed by the first route to reach it, scanning groups in order and routes within groups in order. Once claimed it is never re-added.
interface InventoryInput {
/** Ordered. Group order defines claim precedence. */
groups: GroupSpec[];
/** Routes explicitly ruled out, so the remaining-work count can reach zero. */
excludedRoutes?: ExcludedRoute[];
options?: InventoryOptions;
}
interface GroupSpec {
id: string;
label?: string;
/** Ordered. First route seeds the group; later routes splice in. */
routes: RouteSpec[];
}
interface RouteSpec {
routeId: string;
/**
* Reference direction — ordering only, both directions are always processed.
* `null` / omitted: the library picks the direction with the larger stop union
* and reports the choice via RouteReport.referenceDirectionSource.
*/
referenceDirection?: 0 | 1 | null;
/**
* Alternative selector, resolved to a direction_id. Preferred for
* user-facing config: headsigns are verifiable, 0/1 is not.
*/
referenceHeadsign?: string;
}
interface ExcludedRoute {
routeId: string;
reason?: string;
}interface InventoryOptions {
/** location_type values in scope. Default [0, 1]. */
includeLocationTypes?: number[];
clustering?: ClusteringOptions;
/** Fall back to reciprocal-pattern detection when direction_id is absent. Default true. */
inferDirections?: boolean;
}
interface ClusteringOptions {
/** Default true. False: parentless stops are never merged. */
enabled?: boolean;
/** Default 250. Values above ~400 materially raise false-merge risk (§6.1). */
maxDistanceMeters?: number;
/** Default 'complete'. See §6.1 on chaining. */
linkage?: 'complete' | 'single';
/** Default: trim, collapse whitespace, casefold, strip diacritics. */
normalizeName?: (raw: string) => string;
/** Clusters larger than this get a review diagnostic. Default 4. */
reviewClusterSize?: number;
/** Clusters wider than this get a review diagnostic. Default 150. */
reviewDiameterMeters?: number;
}clustering is the only tuning surface for canonicalization. Since there are no
per-cluster locks, normalizeName is the intended lever for feeds with systematic
naming quirks — it runs before bucketing, so a caller can strip direction suffixes,
punctuation or operator prefixes there.
The library does not import gtfs-sqljs. It depends on a narrow structural
interface, and all calls are awaited. This means the same code works against a
direct instance and against a Comlink-wrapped worker proxy, where every method
becomes a promise.
type Awaitable<T> = T | Promise<T>;
interface GtfsSource {
getStops(filters?: { limit?: number }): Awaitable<Stop[]>;
getRoutes(filters?: { routeId?: string | string[] }): Awaitable<Route[]>;
getTrips(filters?: {
routeId?: string | string[];
directionId?: 0 | 1;
}): Awaitable<Trip[]>;
getStopTimes(filters?: { tripId?: string | string[] }): Awaitable<StopTime[]>;
buildOrderedStopList(tripIds: string[]): Awaitable<string[]>;
}No calendar access is required: all trips are always in scope.
Lifecycle is the caller's. The library never calls close() and never mutates the
database.
interface InventoryResult {
groups: GroupResult[];
/** The difference list. The core QA artifact. */
unassigned: UnassignedStop[];
/** Every canonical stop and how it was formed. */
clusters: ClusterInfo[];
routes: RouteReport[];
diagnostics: Diagnostic[];
coverage: CoverageReport;
}interface GroupResult {
id: string;
label?: string;
entries: InventoryEntry[];
/**
* Count of contiguous runs. >1 means at least one route could not be
* anchored to the rest of the group.
*/
segmentCount: number;
}
interface InventoryEntry {
stopKey: string;
/** 1-based within the group. */
position: number;
/** 0-based segment index; entries sharing a value are contiguous. */
segment: number;
name: string;
/** Centroid of members. Null when no member has usable coordinates. */
lat: number | null;
lon: number | null;
/** Set when the key came from a parent_station. */
stationId: string | null;
/** Constituent raw stops. Always at least one. */
members: StopMember[];
/** All routes serving this key, whether or not they claimed it. */
servedByRouteIds: string[];
/** The route whose traversal placed it here. */
claimedByRouteId: string;
/** Directions this key is served in, on the claiming route. */
servedDirections: 'both' | 'reference' | 'mirror' | 'unknown';
placement: Placement;
tags: EntryTag[];
/** Stable across feed versions; for persisting audit state. See §4.6. */
auditKey: string;
}
interface StopMember {
stopId: string;
stopCode: string | null;
name: string;
lat: number | null;
lon: number | null;
locationType: number;
parentStation: string | null;
}
interface Placement {
kind: 'seed' | 'spliced' | 'appended-segment';
/** Route that introduced the key. */
introducedBy: string;
/** Keys the run was spliced between, when applicable. */
anchorBefore: string | null;
anchorAfter: string | null;
}
type EntryTag =
/** Part of a terminal loop; relative order is approximate. */
| 'terminus-loop'
/** Placed with weak or conflicting anchors. */
| 'order-approximate'
/** Key came from the name+proximity fallback, not parent_station. */
| 'heuristic-cluster'
/** No usable coordinates on any member. */
| 'missing-coordinates';interface ClusterInfo {
stopKey: string;
source: 'parent-station' | 'name-proximity' | 'singleton';
members: StopMember[];
/** Max pairwise distance, metres. Null when coordinates are unusable. */
diameterMeters: number | null;
/** True when it exceeds reviewClusterSize or reviewDiameterMeters. */
needsReview: boolean;
reviewReasons: string[];
}Every raw in-scope stop_id appears in exactly one ClusterInfo. This is what makes
a clustering false-merge visible: an aggregate stop count cannot distinguish a bad
merge from an unassigned route, but the cluster table can. With locks removed, this
table is the only way a false merge surfaces — callers should treat
needsReview as required reading, not an optional report.
interface UnassignedStop {
stopKey: string;
name: string;
lat: number | null;
lon: number | null;
members: StopMember[];
reason: UnassignedReason;
/** Routes serving it, if any — the actionable follow-up. */
servedByRouteIds: string[];
auditKey: string;
}
type UnassignedReason =
/** No stop_times reference any member. A feed defect. */
| 'no-stop-times'
/** Served, but only by routes not yet in a group. Keep working the queue. */
| 'served-only-by-ungrouped-routes'
/** Served only by routes in excludedRoutes. Expected. */
| 'served-only-by-excluded-routes'
/** location_type outside includeLocationTypes. Also excluded from the denominator. */
| 'excluded-location-type';Four reasons, with four different remedies. Grouping by reason matters: a flat count
invites the auditor to grind at a number that will never reach zero.
Note what is not here — no stop is ever unassigned because of a date filter, a trip frequency threshold or a manual exclusion. If a stop is served by a grouped route, it is in a group.
interface RouteReport {
routeId: string;
shortName: string | null;
longName: string | null;
status: 'grouped' | 'excluded' | 'ungrouped';
groupId: string | null;
referenceDirection: 0 | 1 | null;
referenceDirectionSource: 'explicit' | 'headsign' | 'auto' | 'inferred' | 'unavailable';
mirrorApplied: boolean;
mirrorSkippedReason: 'circular' | 'single-direction' | 'no-trips' | null;
patterns: PatternInfo[];
canonicalStopCount: number;
/**
* For ungrouped routes: canonical stops it would contribute that no
* current group holds. Lets the caller order remaining work by leverage
* rather than alphabetically.
*/
wouldAddCount: number | null;
/** For grouped routes: how many of its stops were already claimed. */
alreadyClaimedCount: number | null;
}
interface PatternInfo {
patternId: string;
directionId: 0 | 1 | null;
headsign: string | null;
/** Informational only. Never used to filter. */
tripCount: number;
stopCount: number;
}interface Diagnostic {
code: DiagnosticCode;
severity: 'error' | 'warning' | 'info';
message: string;
groupId?: string;
routeId?: string;
stopKeys?: string[];
}
type DiagnosticCode =
// Input validation
| 'unknown-route-id'
| 'route-in-multiple-groups' // error
| 'route-grouped-and-excluded'
| 'duplicate-group-id' // error
// Ordering
| 'reversed-anchors' // reference direction likely inverted vs group
| 'disconnected-route' // no shared stop with the group so far
| 'anchors-claimed-elsewhere' // §6.2 — actionable via group reordering
| 'order-approximate'
// Clustering
| 'cluster-review-diameter'
| 'cluster-review-size'
| 'missing-coordinates'
| 'low-parent-station-coverage' // info; how much work the fallback is doing
// Feed findings
| 'orphan-stop' // in stops.txt, in no stop_times
| 'station-without-served-children'
| 'single-direction-route'
| 'no-direction-id'
| 'circular-route'
// Integrity
| 'coverage-imbalance'; // error — see belowinterface CoverageReport {
/** Both bases are reported. A single number is not interpretable. */
raw: {
total: number; // all rows in stops.txt
inScope: number; // after includeLocationTypes
inAssignedClusters: number;
inUnassignedClusters: number;
};
canonical: {
total: number;
assigned: number;
unassigned: number;
};
routes: {
total: number;
grouped: number;
excluded: number;
ungrouped: number;
};
parentStationCoverage: number; // 0..1, share of in-scope stops with a parent
heuristicClusterCount: number;
/** canonical.assigned + canonical.unassigned === canonical.total */
balanced: boolean;
}balanced === false emits a coverage-imbalance error. In a QA tool an unbalanced
reconciliation means the output is lying to the auditor, and should be surfaced as
such rather than smoothed over.
Audit state ("this stop has been checked") must survive a feed refresh, and
stop_id does not. Derivation, first available wins:
stop_codeof the primary member, if present and unique in the feedparent_stationid, for station-derived keyssha1(normalizedName + '|' + lat.toFixed(5) + ',' + lon.toFixed(5))
Stability is best-effort, not guaranteed. Callers persisting audit state should
diff auditKey sets across versions and present matched / moved / vanished / new.
/** One-shot. */
export function buildInventory(
gtfs: GtfsSource,
input: InventoryInput
): Promise<InventoryResult>;
/**
* Stateful builder. Caches the canonical stop index and per-route sequences,
* which are the expensive parts and are independent of grouping. Recomputing
* after a grouping change is then group assembly only.
*/
export function createInventoryBuilder(
gtfs: GtfsSource,
options?: InventoryOptions
): Promise<InventoryBuilder>;
interface InventoryBuilder {
/** Cheap: reuses caches. */
build(input: Omit<InventoryInput, 'options'>): Promise<InventoryResult>;
/** Route metadata + leverage counts without any grouping. */
analyzeRoutes(input?: Partial<InventoryInput>): Promise<RouteReport[]>;
/** The canonical stop index alone. */
stopIndex(): StopIndex;
/** Invalidate caches affected by an options change. */
setOptions(options: InventoryOptions): Promise<void>;
}
interface StopIndex {
clusters: ClusterInfo[];
keyByStopId: ReadonlyMap<string, string>;
clusterByKey: ReadonlyMap<string, ClusterInfo>;
}Constraints: ESM-only, Node 18+, matching gtfs-sqljs. gtfs-sqljs is a peer
dependency and is used for types only — the runtime dependency is the GtfsSource
port.
Four phases. Phases 1 and 2 depend only on options and are cached; phases 3 and 4 rerun on every grouping change.
1. Load all stops. Partition by location_type:
in-scope = locationType ∈ options.includeLocationTypes (default 0, 1)
out-of-scope = everything else → UnassignedReason 'excluded-location-type'
2. Station pass:
for each in-scope stop with parentStation P:
key(stop) = 'station:' + P
for each in-scope stop with locationType 1:
key(stop) = 'station:' + stopId
These clusters are frozen. source = 'parent-station'.
3. Fallback pass (parentless locationType 0 stops only):
if clustering.enabled === false:
each stop its own key; source = 'singleton'
else:
bucket stops by normalizeName(name)
for each bucket with >1 member:
cluster by distance, using clustering.linkage
each resulting cluster → key 'cluster:' + sha1(sorted stopIds)
source = 'name-proximity'
singletons → 'singleton'
4. Flag for review:
diameter > clustering.reviewDiameterMeters → 'cluster-review-diameter'
size > clustering.reviewClusterSize → 'cluster-review-size'
no member has usable coordinates → 'missing-coordinates',
and the stop is left as a singleton (it cannot be clustered)
Report parentStationCoverage and heuristicClusterCount as `info`.
Linkage. Default complete: every pair in a cluster must be within
maxDistanceMeters. single linkage chains — A–B at 200m and B–C at 200m merges
A and C at 400m, and along a street with a repeated name this can propagate for
kilometres. single is offered but callers using it should also cap
reviewDiameterMeters and expect review work.
Threshold. Names like Mairie, Gare, Église, Centre, Hôpital collide at range, especially in feeds spanning several adjacent municipalities. In an inventory a false merge is worse than a false split: a false split shows the stop twice and is obvious, a false merge removes it from the checklist. Hence the conservative 250m default — and since there are no locks, a bad merge cannot be corrected after the fact, only prevented.
Run once per route in the input, independent of grouping.
buildRouteSequence(routeId, referenceDirection):
# 2a. Resolve reference direction
if referenceHeadsign given: resolve to its direction_id
if referenceDirection == null: pick the direction with the larger canonical
stop union; record referenceDirectionSource = 'auto'
if the feed has no direction_id on this route:
if options.inferDirections:
detect reciprocal pattern families: two families whose shared-stop
order is largely inverted → treat as opposite directions
referenceDirectionSource = 'inferred'
else:
referenceDirectionSource = 'unavailable'; single family only
emit 'no-direction-id'
# 2b. Reference spine — union of ALL trips in the direction
trips = getTrips({ routeId, directionId: ref })
group trips into patterns by identical stop_id sequence # reporting only
spine = await buildOrderedStopList(all trip ids)
spine = collapseConsecutiveDuplicates(map(spine, canonicalKey))
# 2c. Circular check
if spine.first == spine.last or the sequence otherwise indicates a loop:
emit 'circular-route'; mirrorSkippedReason = 'circular'; return spine
# 2d. Mirror pass
mirrorTrips = getTrips({ routeId, directionId: 1 - ref })
if mirrorTrips empty:
emit 'single-direction-route'; mirrorSkippedReason = 'single-direction'
return spine
mirror = collapseConsecutiveDuplicates(map(
await buildOrderedStopList(mirrorTrips), canonicalKey))
incoming = reverse(mirror)
# 2e. Merge — same splice routine as Phase 3, LOOSE policy
sequence = splice(spine, incoming, policy = LOOSE)
# 2f. Tag
for each key: servedDirections =
in spine and in mirror → 'both'
spine only → 'reference'
mirror only → 'mirror'
keys within a detected terminal loop → tag 'terminus-loop', 'order-approximate'
Every trip in the direction goes into buildOrderedStopList. Nothing is filtered by
calendar or frequency, so a stop served by one school-day trip a year is present.
Pattern grouping happens only to populate PatternInfo for reporting.
Canonicalizing before the diff is essential. Diffing raw stop_ids makes every
opposite-side platform look new and doubles the sequence. Done on canonical keys, the
mirror pass surfaces few stops — and the ones it does surface are findings.
Reverse direction is not a mirror in practice: one-way couplets on parallel streets, terminal loops, peak-only deviations. Local order disagreement between the two directions of one route is normal, which is why this merge uses the LOOSE policy and never raises a reversed-direction diagnostic. Between two different routes the same disagreement is a signal, and the policy is STRICT.
claimed = {} # stopKey -> groupId, global across groups
for group in input.groups: # in order
list = []
for route in group.routes: # in order
seq = phase2(route)
# Keys claimed by an EARLIER group are dropped, but their absence is
# tracked so anchor loss can be reported (see §6.2).
available = [k for k in seq if claimed[k] is unset]
if list is empty:
list = available
placement(k) = { kind: 'seed', introducedBy: route }
else:
list = splice(list, available, policy = STRICT, context = {group, route})
for k in available: claimed[k] = group.id
assign positions and segment indices
Groups are processed in order, so group order is claim precedence. Trunk corridors should come first; the consequence of not doing so is described in §6.2.
1. For every canonical stop not in any group, classify by UnassignedReason,
checking in this order:
excluded-location-type
→ no-stop-times (also emits 'orphan-stop')
→ served-only-by-excluded-routes
→ served-only-by-ungrouped-routes
2. Compute wouldAddCount for each ungrouped route:
|canonicalKeys(route) \ claimed|
Pure set arithmetic against the Phase 2 cache.
3. Stations with no served children → 'station-without-served-children'.
4. Build CoverageReport on both raw and canonical bases; verify the balance
invariant; emit 'coverage-imbalance' on failure.
One routine, two conflict policies. This is the only place ordering decisions are made.
splice(base, incoming, policy, context):
# 1. Order-agreement check
shared = [k for k in incoming if k in base]
if |shared| >= 2:
agreement = concordant pairs / total pairs, comparing index-in-base
against index-in-incoming over `shared`
if agreement < 0.5:
policy == STRICT → emit 'reversed-anchors' (warning) on context
policy == LOOSE → no diagnostic
# base order stays authoritative either way; incoming is never
# re-reversed here. Phase 2 has already applied its reversal, and
# the reference direction is the user's explicit choice.
# 2. Cut incoming into runs of consecutive new keys
runs = maximal spans of keys not in base, each carrying:
anchorBefore = last shared key before the span, or null
anchorAfter = first shared key after the span, or null
# 3. Place each run, in incoming order, recomputing indices after each
# insertion (earlier insertions shift later ones)
for run in runs:
b = index of anchorBefore in base (or -1)
a = index of anchorAfter in base (or -1)
case both present and b < a:
insert run at a # the ordinary case
case both present and b >= a:
insert run at b + 1
tag run 'order-approximate'
case only anchorAfter:
insert run at a # run precedes the shared part
case only anchorBefore:
insert run at b + 1 # run follows the shared part
case neither:
append run as a new segment
if any neighbour key of the run is claimed by another group:
emit 'anchors-claimed-elsewhere'
else:
emit 'disconnected-route'
tag run 'order-approximate'
placement(k in run) = { kind, introducedBy, anchorBefore, anchorAfter }
return base
Plain appending is what this replaces. If route B shares route A's middle but has unique stops before the shared segment, appending puts those stops after all of A's — geographically backwards. Anchoring fixes that.
A stop claimed by an earlier group is absent from a later group's list, so it cannot serve as an anchor there. A route whose shared stops were all claimed elsewhere has no anchors left and lands as a disconnected segment, even though it overlaps the network heavily.
This is inherent to one-slot-per-stop and is not solved. It is reported: the
anchors-claimed-elsewhere diagnostic distinguishes it from a genuinely isolated
route, which makes it actionable — usually by reordering groups so trunk corridors
claim first. With overrides removed, group and route ordering are the only
controls the caller has over placement.
Because both directions are processed together, a route's whole stop set lands in one group. Where the two directions genuinely belong to different corridors — long one-way couplets, large loops covering different neighbourhoods — part of the resulting ordering will be poor.
Everything is still counted exactly once, so the inventory stays complete. Accepted deliberately: supporting split-direction grouping reintroduces the per-(route, direction) work unit and the whole class of "is one direction enough?" ambiguity.
| Phase | Cost |
|---|---|
| 1, station pass | O(S) |
| 1, fallback clustering | O(Σ nᵢ²) over name buckets; buckets are small |
| 2 | O(T) stop_times per route, dominated by buildOrderedStopList |
| 3 | O(G · R · n·m) on canonical keys, small after Phase 2 |
| 4 | O(S + R) set operations |
For a metropolitan feed the dominant cost is Phase 2, which is cached. Regrouping is cheap, which is what makes interactive iteration viable.
Considering all trips rather than a single service date raises Phase 2's constant
factor — a large feed may have several times more trips than any one day's — but
buildOrderedStopList merges identical patterns, so the marginal cost is roughly
proportional to distinct patterns, not trips.
- Is a
strictCoverageoption worth having — throw oncoverage-imbalancerather than reporting it? - Should the name+proximity fallback offer a fuzzy mode (token-set ratio) for feeds
that encode direction in the stop name, e.g.
Mairie (vers Centre)? Worth measuring on real target feeds before adding: if most parentless names are already unique, exact matching buys little and fuzzy matching adds false-merge risk. With locks removed there is no per-cluster correction, sonormalizeNameis currently the only answer to this class of feed.