Skip to content

BUG-003: MOESI flush dram_ack timing race in coherency FSM

Field Value
Severity CRITICAL
Component RTL
File(s) rtl/duck_coherency_ctrl.sv — ST_NPU_FLWAIT → ST_NPU_DRAM transition
Status Fixed
Date 2025-06

Symptom

TC-MOESI-1 and TC-MOESI-2 both failed: NPU DMA reads returned 0 instead of the CPU-written values (42 and 77 respectively). TC-MOESI-3 (clean read, no flush) passed, isolating the fault to the dirty-line flush path.

Root Cause

The cocotb DRAM model is reactive with 1-cycle latency:

Cycle N:   RTL asserts dram_write or dram_read (combinational from state)
Cycle N+1: cocotb model samples signals → sets dram_ack=1, dram_rdata=data
Cycle N+2: RTL always_ff sees dram_ack=1 → advances FSM

After flushing all 8 dirty cache words, ST_NPU_FLWAIT transitioned directly to ST_NPU_DRAM:

ST_NPU_FLWAIT (word 7, dram_ack=1) → ST_NPU_DRAM   ← bug: stale ack visible immediately

On the very first cycle of ST_NPU_DRAM: - dram_ack = 1 — stale, set by the DRAM model for the last flush write - dram_rdata = 0 — the value at the flush word address, not the NPU target

The FSM captured npu_rdata_reg = 0 and jumped to ST_NPU_DONE, returning 0.

TC-MOESI-3 bypasses the flush entirely (IDLE → ST_NPU_DRAM), where dram_ack starts at 0, so it was unaffected.

The CPU miss path (ST_FILL → ST_DONE → ST_IDLE) has always been correct because ST_DONE is an idle gap that lets dram_ack de-assert before the CPU re-issues.

Fix

Add a one-cycle gap state ST_NPU_GAP between ST_NPU_FLWAIT and ST_NPU_DRAM. In ST_NPU_GAP neither dram_write nor dram_read is driven, so the DRAM model clears dram_ack = 0. ST_NPU_DRAM then waits for a fresh ack from the actual read.

+ localparam [3:0] ST_NPU_GAP = 4'd9;

  ST_NPU_FLWAIT: if (dram_ack && npu_flush_word_r == 7) begin
      dirty_store[...][npu_flush_way_r] <= 0;
-     state <= ST_NPU_DRAM;
+     state <= ST_NPU_GAP;   // let dram_ack de-assert first
  end

+ ST_NPU_GAP: state <= ST_NPU_DRAM;

Corrected NPU DMA state sequence:

IDLE → ST_NPU_FLUSH → ST_NPU_FLWAIT ⇆ (×8 words)
         └─ (word 7 done) → ST_NPU_GAP → ST_NPU_DRAM → ST_NPU_DONE → IDLE

Notes

Any FSM that asserts a DRAM signal (read or write) and then immediately enters a state that checks dram_ack will see a stale ack on the first cycle. The pattern to avoid:

STATE_A: dram_write = 1  →  STATE_B: if (dram_ack) ...   ← STATE_B fires on leftover ack

Always insert a gap state (with no DRAM activity) whenever transitioning between different DRAM operations within the same FSM.