Skip to content

BUG-007: halt fires early during chained MUL/DIV/REM ops after UCIe decoupling

Field Value
Severity HIGH
Component RTL / Testbench
File(s) rtl/cpu_chiplet_top.sv, tb/tb_server_dispatch_unit.py
Status Fixed
Date 2026-07-20

Symptom

After splitting system_top into base_die_top / cpu_chiplet_top / npu_chiplet_top connected by UCIe links (see chiplets.md), run_riscv_program started failing on cpu_peak_tests.asm:

t3 [x28]  actual=0x00000000  expected=0x00000001  [FAIL]

t3 is written by remu t3, s1, s2 (205 % 12 = 1), immediately followed in the program by remu t4, s5, s1 and then the jal x0, 0 halt word. t4 passed; only t3 — the first of the two chained REMU ops — came back as 0.

Root Cause

Two independent facts combined:

  1. riscv_core.sv's fetch stage has a pre-existing one-cycle race. if_pc only stalls on the registered mul_busy (= math_busy_r), which lags math_start by one cycle. So the cycle a MUL/DIV/REM is dispatched into EX, if_pc is still free to advance one more word — fetching the next instruction into if_id ahead of schedule. This is normal, harmless prefetch-one-ahead behavior for every other instruction type, but it means if_pc can reach an address before the instruction it just walked past has actually retired.

  2. The chiplet split intentionally decouples network ingestion from CPU stall cycles. In v4, cpu_ready = !core_stall, so the NoC router (and therefore the testbench's instruction-send loop) couldn't push new words into imem while the core was mid-MUL/DIV — including the trailing halt word. That incidentally kept imem's write pointer from ever reaching the halt address until the core had caught up, masking fact (1). In v5, cpu_ready is UCIe channel credit (ld_tx_ready) instead — correct for a real die-to-die link, which shouldn't need to know about the receiver's internal micro-op stalls — so the entire program, halt word included, typically lands in imem almost immediately.

With the masking gone, if_pc could race from remu t3's address straight to the halt word's address while remu t3 (34-cycle iterative) had barely started, and t4's REMU hadn't even been dispatched yet. halt in cpu_chiplet_top.sv was computed purely as instr_valid && (active_instruction == HALT_WORD) — a function of the raw fetch address only, with no notion of in-flight work — so it asserted immediately. HALT_DRAIN_CYCLES = 10 in the testbench was nowhere near enough to cover the ~34 remaining cycles of remu t3 plus all of remu t4 behind it.

Fix

Gate halt on !core_stall (the wire was already routed out of riscv_core into cpu_chiplet_top but left unused):

- assign halt = instr_valid && (active_instruction == HALT_WORD);
+ assign halt = instr_valid && (active_instruction == HALT_WORD) && !core_stall;

This removes the case where halt fires while the currently-executing op is still busy. It does not fully close the race — a freshly-dispatched second chained op can still start on the exact cycle core_stall drops, one cycle before its own math_busy_r registers — so HALT_DRAIN_CYCLES was also raised from 10 to 40 (one full DIV/REM latency of margin) in tb/tb_server_dispatch_unit.py.

Notes

  • riscv_core.sv itself was not modified — the one-cycle math_start-vs-math_busy_r race is a pre-existing property of its hazard detection, just never exercised end-to-end before because v4's network pacing happened to hide it. Worth a closer look if it ever needs the "unmodified" invariant broken for a real fix (stall fetch directly on math_start, not just the registered mul_busy).
  • base_die_top.sv's cpu_ready = ld_tx_ready decoupling is correct and intentional — do not revert it as a workaround for this class of bug.
  • Confirmed unrelated to BUG-006: the MOESI integration suite's 1 PASS / 2 FAIL result is unchanged before and after the chiplet split.