Skip to content

feat(gtx7): resolve odd comma landings in fabric behind RX_ODD_ALIGN_MODE_G - #1475

Open
ruck314 wants to merge 3 commits into
pre-releasefrom
gtx7-rx-odd-align-mode
Open

feat(gtx7): resolve odd comma landings in fabric behind RX_ODD_ALIGN_MODE_G#1475
ruck314 wants to merge 3 commits into
pre-releasefrom
gtx7-rx-odd-align-mode

Conversation

@ruck314

@ruck314 ruck314 commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Description

Gtx7RxFixedLatPhaseAligner answered an odd comma landing by forcing Gtx7RxRst back to INIT, unbounded and unobservable, so a link whose CDR kept landing odd never finished its RX reset.

New RX_ODD_ALIGN_MODE_G on Gtx7Core and Gtx7RxFixedLatPhaseAligner:

  • "RESET" (default) - today's behavior, bit for bit. Callers that leave the generic unset are unaffected.
  • "BITSLIP" - resolves an odd landing in fabric: slide to a residue of 1 using only even rxSlide counts, then present a constant one-bit slice of the aligner's word history. The RX is never reset, so bring-up is bounded.
"RESET" "BITSLIP"
Fabric latency added 0 exactly 1 rxUsrClk: 5.385 ns @ 3.714 Gbps, 8.403 ns @ 2.380 Gbps
Odd landing asserts the aligner's rxReset, re-locks the CDR resolved in fabric, RX never reset
Bring-up unbounded relock loop bounded

"BITSLIP" is fixed-latency for every landing, odd and even. Three quantities are in play and they are easy to conflate:

Quantity Granularity Landing-dependent?
Fabric stages to rxDataOut whole rxUsrClk No. One stage, all twenty landings.
Absolute latency vs "RESET" whole rxUsrClk No. Constant +1 rxUsrClk.
PMA recovered-clock phase sub-UI, bounded by 1 UI Open, see below.

The first row is what a user application sees on rxData, and it holds because rxDataAlignedSel asserts in both of the aligner's terminal states, not only the slipped one:

rxDataAlignedSelInt <= '1' when (BITSLIP_MODE_C and ((r.state = ALIGNED_S) or (r.state = ALIGNED_SLIP_S))) else '0';

An even landing therefore takes the registered path too. Letting it fall through to rxDataInt is the natural-looking design and is the trap: even landings would be combinational off RXDATA while odd landings sat one stage behind, so the two parities would differ by a full parallel-clock period, 5.385 ns @ 3.714 Gbps. Both terminal states instead source the word one rxUsrClk after the GT presented it, ALIGNED_S from the previous GT word unshifted and ALIGNED_SLIP_S from that word shifted up one bit with its missing MSB taken live. One stage is the floor rather than a convenience: at a final offset of 1 the aligned word's last bit only arrives with the next GT word, so it cannot be presented combinationally.

Getting there also depends on slideCount := i-2 in the odd branch, which issues i-1 pulses, an even count, so every odd landing settles at a residue of exactly 1 and the fabric slice is a constant bit range rather than an offset-dependent one.

That constant is EXPECTED_TRAIL in the regression, asserted per mode rather than argued, and each of the twenty landings is checked to present the same frame on the same cycle as every other one in its mode. Choosing "BITSLIP" costs one re-calibration of the absolute number, not run-to-run jitter.

Opting into "BITSLIP"

Needs RX_ALIGN_MODE_G = "FIXED_LAT" and RX_BUF_EN_G = false, both enforced by an elaboration assert, plus rxDataValidIn driven from a decoder rather than left at its '1' default. "BITSLIP" never asserts the aligner's rxReset, so recovery from a lost alignment rests entirely on Gtx7RxRst's DATA_VALID supervision. Several in-tree callers hardwire rxDataValidIn => '1' and would need that port wired first.

The one open question: sub-UI, silicon and not RTL

Even landings settle at GT offset 0, odd landings at offset 1. Whether a final offset of 1 costs a sub-UI recovered-clock phase step relative to offset 0 is a property of the GTX PMA, not of this RTL, and no fabric simulation can settle it. It is bounded by 1 UI, one twentieth of the word clock: 269.2 ps @ 3.714 Gbps, 420.2 ps @ 2.380 Gbps. This is separate from the fabric latency contract above, which is constant across landings either way.

Removing the term at all would mean refusing odd landings, which is exactly what "RESET" does and what "BITSLIP" declines to do. Settling it is a hardware measurement (power-cycle N times, bin a latency measurement on landing parity) that has not been made, so a link with a sub-UI phase budget should stay on "RESET" until it is.

For that measurement, landedValid is the bin discriminator, not landedOffset: the observer latches only the first odd offset seen since reset, so landedValid = '1' marks the odd class with its actual offset in landedOffset, and landedValid = '0' marks the even class. There is no positive even-landing marker.

Also on this branch, independent of the above

  • cPllRefClkLostOut added to Gtx7Core, exposing CPLLREFCLKLOST. This is an entity interface change: a new out port with no default, so a caller using positional association must be updated. Named association, the SLAC convention, needs no change.
  • WAIT_TIME_CDRLOCK_G added, decoupling the CDR-lock wait count from SIM_RESET_SPEEDUP. Its -1 default selects the previous derivation, so it is behavior-preserving.

Verification

  • Cocotb (tests/xilinx/gtx7/test_Gtx7RxFixedLatPhaseAligner.py): both modes across all twenty comma landings, driven from a bit-accurate serial stream. Each landing must align, present a correctly comma-aligned word, and present the same frame on the same cycle as every other landing in its mode; per-mode absolute latency is pinned against EXPECTED_TRAIL and mutation-tested. Scope limit: the aligner is elaborated, but Gtx7Core's output mux is reproduced by the harness, since Gtx7Core needs GTXE2_CHANNEL and does not build under GHDL.
  • "RESET" equivalence: pre-change and post-change aligners co-simulated from identical stimulus are cycle-identical on rxReset, rxSlide and rxPhaseAlignmentDone over 12000 cycles spanning five aligner-driven RX resets.
  • VCS co-simulation with real GTX7 primitives: under "BITSLIP" the affected lane reaches RX reset done from an odd landing and holds it for 2.07 ms with no further cycling. With both mode generics at their defaults, the same bench reproduces the pre-change failure identically.
  • Three Vivado 2024.1 runs on XC7K160TFBG484-2: timing closes with no new critical-warning message classes, and the untouched lane's elaborated GTXE2_CHANNEL parameters match the pre-change baseline apart from an enumerated set of corrected constants.
  • Hardware: the affected lane holds RX reset done through a full software startup sequence, and the previously-working lane still links up. Hardware covered link-up and regression, not latency; the latency contract rests on simulation.

landedOffset and oddLandingCount are declared inside the "BITSLIP" generate, so the legacy path gains no logic, but they are reachable only by hierarchical netlist name under dont_touch. Promoting them to real output ports is left to a follow-up.

Two independent Gtx7Core additions, both backward compatible.

cPllRefClkLostOut exposes CPLLREFCLKLOST from the GTXE2_CHANNEL, which
the core already consumed internally but never brought out to a caller.

WAIT_TIME_CDRLOCK_G decouples the CDR-lock wait from the primitive's
simulation speedup. SIM_GTRESET_SPEEDUP_G previously fed both the
WAIT_TIME_CDRLOCK_C fabric constant and the GTXE2_CHANNEL
SIM_RESET_SPEEDUP attribute, so a simulation could not select the
CDR-lock wait independently of the primitive's own reset speedup. The
new integer generic defaults to -1, which reproduces the previous
derivation exactly and changes no elaborated behavior for any instance
that leaves it at the default. Any value of 0 or greater is used
directly as the stable-clock cycle count for WAIT_TIME_CDRLOCK_C.
SIM_RESET_SPEEDUP continues to consume SIM_GTRESET_SPEEDUP_G unmodified.
…MODE_G

Gtx7RxFixedLatPhaseAligner unilaterally forced Gtx7RxRst back to INIT on
every odd comma landing, unbounded and unobservable. Add
RX_ODD_ALIGN_MODE_G ("RESET" or "BITSLIP") to gate a fabric escape from
that relock loop.

Under BITSLIP, an odd landed offset i issues i-2 rxSlide pulses, which
SLIDE_WAIT_S turns into i-1 actual pulses (always even, so the recovered
clock phase is never disturbed), reducing the residue to 1. The aligned
word is then presented as a 1-bit-shifted slice of the aligner's own
r.last history through two new output ports, rxDataAligned and
rxDataAlignedSel, muxed onto Gtx7Core's rxDataOut only when
RX_ODD_ALIGN_MODE_G = "BITSLIP".

Both terminal states are sourced one register stage deep so that the
resulting latency does not depend on where the comma landed: ALIGNED_S
from the previous GT word, ALIGNED_SLIP_S from that word shifted up one
bit with its missing MSB taken live. One stage is the floor, since at
offset 1 the aligned word's last bit only arrives with the next GT word.
The select therefore asserts in both terminal states, and BITSLIP always
costs exactly one rxUsrClk of fiber-to-rxDataOut latency relative to
RESET, whose RX_DATA_OUT_RESET_GEN drives rxDataOut combinationally off
RXDATA and adds no fabric stage. That is 5.385 ns on an LCLS-II link at
3.714 Gbps and 8.403 ns on an LCLS-I link at 2.380 Gbps. It is constant
across bring-ups, so it costs a caller one re-calibration rather than
run-to-run jitter, but it is a real change to the absolute number.

rxDataAligned and rxDataAlignedSel are driven exclusively inside
RX_FIX_LAT_ALIGN_GEN, so BITSLIP asserts that generate's full condition,
RX_BUF_EN_G = false and RX_ALIGN_MODE_G = "FIXED_LAT". RX_BUF_EN_G
defaults to true, which makes the unguarded combination the likeliest
caller mistake: it would elaborate RX_NO_ALIGN_GEN instead of the
aligner, tie rxPhaseAlignmentDone high, leave rxDataAlignedSel at its
declared '0', and fall through to the raw rxDataInt path, giving no
comma alignment at all while reporting alignment done. No caller in surf
selects BITSLIP, and the one external caller that does (lcls-timing-core
TimingGtCoreWrapper) already sets RX_BUF_EN_G => false, so nothing that
elaborates today starts failing.

BITSLIP never asserts the aligner rxReset, which removes one of the two
paths that previously restarted alignment. The remaining path is
Gtx7RxRst MONITOR_DATA_VALID/FSM_DONE, fed by Gtx7Core rxDataValidIn,
which defaults to '1'. Left at that default the supervision loop is
permanently satisfied and a lost alignment is never re-attempted;
DO_PHASE_ALIGNMENT time_out_wait_bypass only covers an alignment that
never completes, not one that completes and is later lost. Several
in-tree callers hardwire rxDataValidIn => '1' while already being
RX_BUF_EN_G => false with FIXED_LAT, so this is a live constraint. Both
properties are documented at the generic and at the mux.

A dont_touch'd signal inside a BITSLIP_MODE_C-gated generate exposes the
first resolved odd offset and an odd-landing count for hardware
observability. Only the first odd offset is latched, since every odd
landing above 1 re-scans at 1 and would otherwise overwrite the offset
the CDR actually landed on. Scoping the state to the generate keeps it
out of the netlist entirely under RESET; the earlier null-range record
form did not, and Vivado tied four degenerate bits low on lanes that use
neither field.

RX_ODD_ALIGN_MODE_G defaults to "RESET", reproducing today's behavior
exactly. Every change is behind BITSLIP_MODE_C or inside the gated
generate, and RX_DATA_OUT_RESET_GEN is untouched. Verified by
co-simulating the pre-change and post-change aligners from identical
stimulus: cycle-identical on rxReset, rxSlide and rxPhaseAlignmentDone
over 12000 cycles across 5 aligner-driven RX resets.

Not addressed here: even landings settle at GT offset 0 and odd landings
at offset 1. If an odd final offset costs a sub-UI recovered-clock phase
step, a <=1 UI difference between the two classes remains that no fabric
logic can remove, since crossing parity classes requires a fresh CDR
lock. That is a property of the silicon and is being measured on
hardware, binned on landing parity.
Add the aligner to the non-Vivado GHDL source list and a pytest/cocotb
suite sweeping all 20 comma landings in both RX_ODD_ALIGN_MODE_G modes.

Per-landing invariants. Every landing settles with an even slide count,
so the recovered clock phase is never disturbed; BITSLIP never asserts
rxReset; and every landing presents the same frame on the same cycle as
every other landing in its mode.

Absolute latency, not just landing-invariance. EXPECTED_TRAIL carries
one entry per mode, in whole rxUsrClk of fiber-to-rxDataOut latency, so
what switching a link from RESET to BITSLIP costs is stated rather than
recoverable only by reading Gtx7Core's two output-mux legs against the
aligner's pipeline depth. RESET adds no fabric stage; BITSLIP always
adds the aligner's one stage. The delta is one rxUsrClk.

Mutation-tested per property. Restoring the odd-branch off-by-one fails
the 9 odd BITSLIP landings; restoring a combinational even path fails
the 10 even ones. Moving the RESET trail entry alone fails exactly the
ten even RESET landings and nothing else; moving the BITSLIP entry alone
fails all twenty BITSLIP landings and nothing else. Deepening both
aligner terminal states by one stage fails all twenty BITSLIP landings,
confirming the trail entry tracks the RTL rather than a constant.

Scope is recorded in the file. The aligner's end of the contract is
elaborated, but Gtx7Core's output mux is reproduced by
Harness.data_out() rather than built, because Gtx7Core needs
GTXE2_CHANNEL and does not compile under GHDL. An edit to the mux
expressions themselves is out of reach of this suite.
@ruck314
ruck314 force-pushed the gtx7-rx-odd-align-mode branch from 9101d63 to 134e40c Compare August 23, 2026 23:32
@ruck314
ruck314 marked this pull request as ready for review August 23, 2026 23:46
This was referenced Aug 23, 2026
@ruck314
ruck314 requested a review from thatweaver August 24, 2026 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant