Dated entries as problems actually happened: symptom, root cause, options, chosen fix and why, and how it was verified. This is the raw material for the debug report. First person, no polish.
Symptom: after installing the OSS CAD Suite nightly, sta was missing while
yosys, iverilog, verilator, and sby were all present. The only binary matching
sta was icebox_stat.
Root cause: the bundle simply does not ship OpenSTA.
Options: (a) use OpenROAD's bundled STA, (b) build OpenSTA from source. The spec already anticipates this and asks for a source build, so I took (b).
Fix: cloned The-OpenROAD-Project/OpenSTA (git 97d84f0ed7, reports version 3.1.0) and built with cmake. Recorded the commit. Verified the binary prints its banner and accepts commands.
Symptom: three cmake configure or link failures in a row. First
Could NOT find GTest. After installing gtest, CUDD_LIB ... set to NOTFOUND
and FLEX_INCLUDE_DIR ... NOTFOUND.
Root cause: this OpenSTA revision links CUDD unconditionally (the option() is
advisory only; the target uses ${CUDD_LIB} directly), needs GTest for its unit
test targets, and needs FlexLexer.h from libfl-dev. I had installed flex but
suppressed its recommended libfl-dev with --no-install-recommends.
Fix: installed libgtest-dev, libgmock-dev, libfl-dev, then built CUDD and
pointed cmake at it with -DCUDD_DIR.
Symptom: make in the CUDD tree died with
aclocal-1.14: command not found ... Error 127.
Root cause: the OpenROAD CUDD fork ships autotools output stamped for automake
1.14, but this box has automake 1.18. A timestamp skew made make try to
regenerate aclocal.m4 with the old aclocal version.
Options: install an old automake, or regenerate the build system with the local tools. I regenerated.
Fix: make distclean then autoreconf -fi before ./configure. CUDD 3.0.0
then built and installed cleanly, and OpenSTA linked against it.
Symptom: sourcing my scripts/env.sh worked when run as bash script.sh but
wiped every variable and exited when run as bash -c '. scripts/env.sh; ...',
which is exactly how make runs recipe commands.
Root cause: env.sh was sourcing the bundle's environment file, which opens
with a guard if [ "${BASH_SOURCE-}" = "$0" ]; then exit 1; fi meant to catch
someone executing it instead of sourcing it. At the top level of a sh -c
string that comparison misfires and the exit tears down the whole recipe
shell.
Options: force every consumer to be an interactive-like bash, or stop sourcing the guarded file. The second is cleaner and shell agnostic.
Fix: env.sh now sets the three variables that file actually provides (prepends
the bundle bin and py3bin to PATH, sets VERILATOR_ROOT and GHDL_PREFIX)
directly, and never sources it. Verified that a plain sh -c recipe now sees
yosys, sby, and sta on PATH.
Symptom: after the fix above, REPO_ROOT resolved to the parent directory when
env.sh was sourced under dash, because dash has no BASH_SOURCE and sets $0
to the calling script, not to env.sh.
Fix: resolve the repo root by walking up from $PWD until a directory contains
both scripts/env.sh and Makefile. That is shell agnostic and works from any
subdirectory. The Makefile also exports REPO_ROOT from $(CURDIR) so make
recipes never depend on the walk.
Symptom: the first make check-no-dashes reported two violations, both inside
check_no_dashes.py, on the lines that defined the dash characters to search
for.
Root cause: obvious in hindsight. A scanner that hard codes the glyphs it hunts for contains those glyphs.
Fix: build the two characters with chr(0x2013) and chr(0x2014) so the source
file is pure ASCII. The scanner then reported zero hits across the tree.
Symptom: the first -Wall lint of fifo_top produced seven warnings.
Root causes and fixes, all legitimate:
- WIDTHEXPAND on the almost full and almost empty comparisons, because a five bit occupancy was compared against a 32 bit int parameter. Fixed with a sized cast of the threshold to the occupancy width.
- UNUSEDSIGNAL on the upper bits of a 64 bit temporary that held the result of the package gray to binary function. Fixed by doing the exclusive or reduction inline at the exact pointer width, which also dropped a needless dependency.
- NOLATCH on the isolation always_latch, because I had tied the isolation enable to a constant zero, so there was no latch to infer. Fixed by driving isolation from the coarse domain enable (isolate when the write domain is idle) and modeling the clamp as a retention register rather than a latch, which is both live logic and cleaner for synthesis.
- SYNCASYNCNET on the reset synchronizer, which is expected: a reset synchronizer exists to drive an asynchronous reset from a clocked value. Waived for that module only, with the reason recorded in the waiver file.
Symptom: every integration run aborted immediately with
vvp: vthread.cc:3793 ... Assertion child->wt_context ... failed inside
of_JOIN_DETACH.
Root cause: I used fork ... join_any followed by disable on the named block
to implement a wait with timeout inside an automatic task. Icarus mishandles the
thread context when a detached child of an automatic task is disabled.
Options: restructure the wait, or move it out of an automatic task. I restructured.
Fix: replaced the fork with a plain bounded polling loop that waits on the read clock until empty deasserts or a cycle budget expires. No behavior change, and the plain fork and join used elsewhere for concurrent drivers are unaffected because they wait for all children rather than detaching one.
Symptom: sby with read -formal -sv died on fifo_pkg.sv:13 with a syntax
error at the function declaration.
Root cause: Yosys's built in Verilog frontend has limited SystemVerilog support and does not parse the package and typed function style used here. This is the same reason the build spec calls for a slang based frontend.
Fix: switched the sby script to read_slang, which is bundled in this OSS CAD
Suite and reads the full SystemVerilog design in one pass.
Symptom: read_slang then failed with unknown system name '$anyseq' on the
lines that drove the free clocks and inputs.
Root cause: the slang frontend does not implement the $anyseq formal function.
Fix: dropped $anyseq entirely and made the clocks and inputs primary inputs of
the formal top instead. A formal engine already treats primary inputs as free
every step, so with multiclock this models arbitrary clock frequency and phase
without any vendor specific function.
Symptom: the write Gray adjacency assertion failed at step 11. The trace showed the design running correctly (the write pointer advancing 0, 1, 3 in Gray) while my history register that was supposed to hold the previous Gray value stayed stuck at zero, so the exclusive or looked like a two bit change.
Root cause: I sampled design signals into registers clocked on the (* gclk *)
global clock. Under the multiclock transform the design updates on its own
clocks and a separate global clock observer sees those signals with a sampling
skew, so the observer never tracked the real value.
Fix: keep only the truly combinational cross domain facts on the global clock (the occupancy bound and the domain disambiguation, which need no history), and move the Gray adjacency history into registers clocked on the design's own write and read clocks, where there is no skew. That made the adjacency checks track correctly.
Symptom: after the adjacency fix, the naive assert(!(full && empty)) failed at
step 23. The trace showed a state where the write domain asserted full and the
read domain asserted empty at the same time.
Root cause, and the interesting part: this is not a bug, it is a real property of an asynchronous FIFO. Full is computed in the write domain from a synchronized, and therefore stale, copy of the read pointer, and empty is computed in the read domain from a stale copy of the write pointer. During a fast drain the write domain can still believe the FIFO is full (it has not yet seen the reads) while the read domain already believes it is empty, and both flags are conservatively correct for their own domain. There is no single observer that ever sees both, so nothing is actually wrong; the naive cross domain invariant simply does not hold for a real CDC FIFO.
Fix: I replaced the naive property with the guarantee the extra wrap bit actually provides: within a single domain the full condition and the empty condition are never both true, because they differ in the top two pointer bits by construction. That, plus the occupancy bound (which alone proves both no overflow and no underflow, since an underflow would wrap the occupancy above DEPTH), is the honest and provable statement of FIFO safety. All four properties then pass bounded model checking to depth 60. An unbounded k induction proof was tried and its base case passes, but the induction step needs a large set of strengthening invariants over the synchronizer chains to rule out unreachable start states, so per the build spec I kept the justified bound rather than burning days on it.
Symptom: upf_lint.py reported zero instances, so every domain element looked
missing.
Root cause: by default read_slang elaborates the whole design into a single
flat module, so the named submodule instances the UPF refers to no longer exist
as cells.
Fix: pass --keep-hierarchy to read_slang. The instances then appear as real
cells of fifo_top and the cross reference against the UPF domain elements works.
The lint now passes with 10 instances placed across the four domains.
Symptom: after synthesis the netlist carried unmapped $_DLATCH_N_ cells, so the
area report flagged an unknown cell and OpenSTA could not have read the netlist.
Root cause: the behavioral clock gate is a latch plus AND, and dfflibmap maps
flip flops, not latches, so the latch was left as a primitive.
First attempt: map the whole clk_gate_cell module onto the real integrated
clock gate cell sky130_fd_sc_hd__dlclkp_1 with a module level techmap and
--keep-hierarchy. That did not fire, because --keep-hierarchy mangles each
instance's module name (clk_gate_cell$fifo_top.u_ram_icg), so a techmap keyed
on the plain module name matched nothing.
Second attempt: map the $_DLATCH_N_ primitive onto the Sky130 dlxtp latch
with the enable inverted using ~E. That left an unmapped $not on the clock
network, because ABC does not map clock network logic.
Fix: map $_DLATCH_N_ onto dlxtp and do the enable inversion with an explicit
library clock inverter cell in the same techmap, so nothing is left for ABC to
miss. The netlist is now fully mapped. Yosys separately turned the gated pointer
and RAM registers into clock enable flip flops, which is a fine realization of
the gating; the honest gap, that a production flow would insert the dlclkp cell,
is documented in the synthesis flow doc.
Symptom: read_power_activities -vcd reported a deprecation and then an argument
count error.
Root cause: this OpenSTA renamed the command to read_vcd, which is exactly what
the build spec names.
Fix: use read_vcd -scope tb_fifo_top/dut $vcd. The measured activity power then
annotated cleanly and came out higher than the default assumption, which is the
interesting delta the report discusses.