TidalSim: Fast and Accurate Microarchitectural Simulation via Sampled RTL Simulation

The New-Era of Domain-Specialized Heterogeneous SoCs

• Two trends in SoC design:
  • Heterogeneous cores targeting different power/perf curves + workloads
  • Domain-specific accelerators
• Need a pre-silicon evaluation strategy for rapid, PPA optimal design of these units
  • Limited time per design cycle → limited time per evaluation
  • More evaluations = more opportunities for optimization

The Microarchitectural Iteration Loop (Industry)

Rapid RTL performance validation that can be used in the RTL design cycle is valuable.

The Microarchitectural Iteration Loop (Academia)

Academics rarely write RTL partly due to the difficulty of evaluation, instead opting for uArch simulators.

Academia needs rapid RTL evaluation as a part of an RTL-first research methodology

Limitations of Existing Evaluators

• ISA simulation: no accuracy
• Trace/Cycle uArch simulation: low accuracy
• RTL simulation: low throughput
• FPGA prototyping: high startup latency
• HW emulators: high cost

We will propose a simulation methodology that can deliver on all axes (accuracy, throughput, startup latency, cost).

TidalSim Overview

TidalSim Components

TidalSim is not a new simulator. It is a simulation methodology that combines the strengths of architectural simulators, uArch models, and RTL simulators.

TidalSim Execution

What Could Our Proposal Enable?

Industry

Industry

Academia

Academia

TidalSim enables new design methodologies for industry, academia, and lean chip design teams.

Simulator Metrics

Simulation techniques span the gamut on various axes. Each simulation technique assumes a particular hardware abstraction.

• Throughput
  • How many instructions can be simulated per real second? (MIPS = millions of instructions per second)
• Accuracy
  • Do the output metrics of the simulator match those of the modeled SoC in its real environment?
• Startup latency
  • How long does it take from the moment the simulator's parameters/inputs are modified to when the first instruction is executed?
• Cost
  • What hardware platform does the simulator run on?
  • How much does it cost to run a simulation?

Existing Hardware Simulation Techniques

TidalSim combines the strengths of each technique to produce a meta-simulator that achieves high throughput, low latency, high accuracy, and low cost.

Accuracy of Microarchitectural Simulators

Trends aren't enough^[2]. Note the sensitivity differences - gradients are critical!

uArch simulators are not accurate enough for microarchitectural evaluation.

Sampled Simulation

Instead of running the entire program in uArch simulation, run the entire program in functional simulation and only run samples in uArch simulation

The full workload is represented by a selection of sampling units.

1. How should sampling units be selected?
2. How can we accurately estimate the performance of a sampling unit?
3. How can we estimate errors when extrapolating from sampling units?

Existing Sampling Techniques

SimPoint

• Program execution traces aren’t random
  • They execute the same code again-and-again
  • Workload execution traces can be split into phases that exhibit similar μArch behavior
• SimPoint-style representative sampling
  • Compute an embedding for each program interval (e.g. blocks of 100M instructions)
  • Cluster interval embeddings using k-means
  • Choose representative intervals from each cluster as sampling units

SMARTS

• Rigorous statistical sampling enables computation of confidence bounds
  • Use random sampling on a full execution trace to derive a population sample
  • Central limit theorem provides confidence bounds
• SMARTS-style random sampling
  • Pick a large number of samples to take before program execution
  • If the sample variance is too high after simulation, then collect more sampling units
  • Use CLT to derive a confidence bound for the aggregate performance metric

Functional Warmup

The state from a sampling unit checkpoint is only architectural state. The microarchitectural state of the uArch simulator starts at the reset state!

• We need to seed long-lived uArch state at the beginning of each sampling unit
• This process is called functional warmup

Importance of Functional Warmup

Long-lived microarchitectural state (caches, branch predictors, prefetchers, TLBs) has a substantial impact on the performance of a sampling unit

Why RTL-Level Sampled Simulation?

• Eliminate modeling errors
  • Remaining errors can be handled via statistical techniques
• No need to correlate performance model and RTL
  • Let the RTL serve as the source of truth
• Can produce RTL-level collateral
  • Leverage for applications in verification and power modeling

This RTL-first evaluation flow is enabled by highly parameterized RTL generators and SoC design frameworks (e.g. Chipyard).

Overview of the TidalSim v0.1 Flow

Implementation Details For TidalSim v0.1

• Basic block identification
  • BB identification from spike commit log or from static ELF analysis
• Basic block embedding of intervals
• Clustering and checkpointing
  • k-means, PCA-based n-clusters
  • spike-based checkpoints
• RTL simulation and performance metric extraction
  • Custom force-based RTL state injection, out-of-band IPC measurement
• Extrapolation
  • Estimate IPC of each interval based on its embedding and distances to RTL-simulated intervals

IPC Trace Prediction: huffbench

• Huffman compression from Embench (huffbench)
• N=10000, C=18
• Full RTL sim takes 15 minutes, TidalSim runs in 10 seconds
• Large IPC variance

IPC Trace Prediction: wikisort

• Merge sort benchmark from Embench (wikisort)
• N=10000, C=18
• Full RTL sim takes 15 minutes, TidalSim runs in 10 seconds
• Can capture general trends and time-domain IPC variation

Aggregate IPC Prediction for Embench Suite

Typical IPC error (without functional warmup and with fine time-domain precision of 10k instructions) is < 5%

From Tidalsim v0.1 to v1

• Functional L1-only cache warmup
• Functional branch predictor warmup
• Use robust checkpointing fork of spike
  • Better arch state initialization technique (via program snippet + selective forces depending on bits that can't be set via ISA)
• Characterization on other baremetal workloads
  • dhrystone, coremark, riscv-tests benchmarks, MiBench
• Explore more sophisticated clustering and extrapolation techniques
  • Binary and PC-agnostic interval embeddings

Demonstrate we can hit <1% IPC error

Leveraging HDLs for TidalSim Methodology

• HW DSE with TidalSim requires an RTL injection harness
• Automatic harness generation using high-level HDLs
  • Chisel API to semantically mark arch and uArch state
  • FIRRTL pass to generate a state-injecting test harness

class RegFile(n: Int, w: Int, zero: Boolean = false) {
  val rf = Mem(n, UInt(w.W))
  (0 until n).map { archStateAnnotation(rf(n), Riscv.I.GPR(n)) }
  // ...
}
  

class L1MetadataArray[T <: L1Metadata] extends L1HellaCacheModule()(p) {
  // ...
  val tag_array = SyncReadMem(nSets, Vec(nWays, UInt(metabits.W)))
  (0 until nSets).zip((0 until nWays)).map { case (set, way) =>
    uArchStateAnnotation(tag_array.read(set)(way), Uarch.L1.tag(set, way, cacheType=I))
  }
}
  

TidalSim for Verification

5.b: Past Work on Specification Mining

• Take waveforms from RTL simulation and attempt to mine unfalsified specifications involving 2+ RTL signals^[1]

• Specifications are constructed from LTL templates
  • Until: $ \mathbf{G}\, (a \rightarrow \mathbf{X}\, (a\, \mathbf{U}\, b)) $
  • Next: $ \mathbf{G}\, (a \rightarrow \mathbf{X}\, b) $
  • Eventual: $ \mathbf{G}\, (a \rightarrow \mathbf{X F}\, b) $
  • $a$ and $b$ are atomic propositions constructed from signals in the RTL design

5.b: Specification Mining Used for RTL Bug Localization

Introduce a bug in the riscv-mini cache

-  hit := v(idx_reg) && rmeta.tag === tag_reg
+  hit := v(idx_reg) && rmeta.tag =/= tag_reg
  

• This bug does not affect most ISA tests but a multiply benchmark failed by hanging
• Checking the VCD against the mined properties gives these violations

The violated properties point to an anomaly with the hit signal and localize the bug

5.b: Coverpoint Synthesis as Complement of Spec Mining

• Coverpoint synthesis is an alternative take on spec mining where we synthesize μArch properties that we want to see more of
  • Instead of monitoring properties just for falsification, we also monitor them for completion
  • Properties that are falsified or completed, but not too often, are good candidates for coverpoints
• Evaluation
  • Synthesize coverpoints on Rocket using waveforms from TidalSim and regular RTL sim with the same compute budget
  • Demonstrate we can synthesize more, and more interesting coverpoints using TidalSim data
  • Evaluate off-the-shelf RISC-V instgen on synthesized coverpoints vs structural coverage