Part IV: 1.5hr: Details of MacSim simulator
Download
Report
Transcript Part IV: 1.5hr: Details of MacSim simulator
HPArch Research Group
| Part III: Overview of MacSim
Features of MacSim
Basic MacSim architecture
How to simulate architectures with MacSim
| Part IV: Details of MacSim
For computer architecture researchers
| Part V:
MacSim-SST case studies
Research with MacSim
MacSim Tutorial (In ICPADS 2013)
| Heterogeneous architecture simulator (x86+PTX)
| Developed from Georgia Tech
| Trace driven simulator
X86 traces – using Pin, PTX traces – using GPUOcelot
Internal RISC style micro-op generation module
| Cycle-level simulator
Cores, caches, memory systems are modeled
| Support various simulations - single/multi-threaded
application, multi-program, heterogeneous (CPU+GPU)
MacSim Tutorial (In ICPADS 2013)
| Flexible design to support various platforms
| Integration with a parallel simulator (SST) to support highperformance computing systems
| From mobile to Exascale computing systems
MacSim Tutorial (In ICPADS 2013)
Prof. Yalamanchili
(Georgia Tech)
CUDA code
(.cu)
X86 binaries
NVCC
(Compiler)
PTX code
GPUOcelot
Trace Generator
PIN
Trace Generator
Instruction
Thread information
QEMU based
Trace Generator
ARM binaries
QEMU based
Trace Generator
Ongoing Work
MacSim Tutorial (In ICPADS 2013)
Heterogeneous
Architecture
Timing & Power
Simulator
| Getting MacSim
Stable version (2.01) – google code project
http://macsim.googlecode.com/files/macsim-2.01.tar.gz
Latest code from SVN
repositoryhttp://code.google.com/p/macsim/wiki/Download
| How to build
http://code.google.com/p/macsim/wiki/BuildingMacsim
Chapter 2 of manual provides an instruction to build
README file in the simulator directory
MacSim Tutorial (In ICPADS 2013)
| MacSim package
IRIS (NoC simulator from Prof. Yalamanchili’s group) is included
CPU trace generator
Download PIN separately. Trace generator tool is in the MacSim Package
GPU trace generator
Download Ocelot Separately. Trace generator is in the Ocelot’s package
| MacSim-SST
SST needs to be downloaded separately
| Energy Introspector (From Prof. Yalamanchili’s group)
EI is a power model based on McPAT, HotSpot.
Because of McPAT license issue, currently EI cannot be distributed, but
we will resolve this issue soon
MacSim Tutorial (In ICPADS 2013)
MacSim Tutorial (In ICPADS 2013)
| Once build process is successful, binary will be created in
macsim-top/trunk/bin/macsim
| Screenshot of a simulation
| Now, How to configure simulation models ?
MacSim Tutorial (In ICPADS 2013)
| Knob variables need to set up (3 ways)
Default value in the source code
Params.in
Command line
Core type 1
Core type 1
Core type 1
Core type 1
Core type 1
Core type 2
Core type 2
Core type 2
Core type 2
Core type 2
Memory
MacSim Tutorial (In ICPADS 2013)
Core type 3
Core type 3
Core type 3
Core type 3
Core type 3
| Configuration
4 cores
2-way SMT
.def
param<NUM_SIM_CORES, num_sim_cores, int, 4>
num_sim_cores 4
// 4 cores
num_sim_small_cores 0
num_sim_medium_cores 0
num_sim_large_cores 4
max_threads_per_large_core 2
large_core_type x86
repeat_trace 1
./macsim –num_sim_cores=4
MacSim Tutorial (In ICPADS 2013)
commandline
params.in
| To configure CPU+GPU arch.
Set up number of cores and
type accordingly
num_sim_cores 8
// 4 CPUs + 4 GPUs
num_sim_small_cores 4 // 4 GPU
num_sim_medium_cores 0
num_sim_large_cores 4 // 4 CPUs
core_type ptx
// specify small cores
large_core_type x86
cpu_frequency 3
gpu_frequency 1.5
repeat_trace 1
MacSim Tutorial (In ICPADS 2013)
| Usually, we use small
core for GPU and large
for CPU
| GPU has internally
multiple processing
elements (N-wide SIMD)
| Multiple Applications
Set up from trace_file_list
Blackscholes
4 <-- number of applications
/sample/mcf/trace.txt <- appl 1
/sample/gcc/trace.txt <- appl 2
/sample/mm/trace.txt <- appl 3
/sample/blackscholes/trace.txt <- appl 4
MacSim Tutorial (In ICPADS 2013)
MCF
GCC
MM
MM
thread thread
1
2
| Execution time for each application is different.
| Provide an option to enable repeat short traces until the
longest trace ends
Program 1
Program 2
Program 3
mcf
gcc
bfs
gcc
bfs
gcc
bfs
| Whether it’s the right way to simulate?
MacSim Tutorial (In ICPADS 2013)
gcc
bfs
bfs
File name
Contents
params_8800gt
GeForce 8800 GT (G80)
params_gtx280
GeForce GTX 280 (GT200)
params_gtx465 NVIDIA GeForce GTX
465 (Fermi)
params_gtx465
GeForce GTX 465 (Fermi)
params_x86
Intel’s Sandy Bridge (CPU part only)
params_hetero_4c_4g
Intel’s Sandy Bridge (CPU + GPU)
| Sample configuration files in
macsim-top/trunk/params
MacSim Tutorial (In ICPADS 2013)
| Thread spawn is modeled.
| Lock is not modeled.
Host thread
Main thread
Threads spawn
GPU Kernel invocation
Barrier
core
MacSim Tutorial (In ICPADS 2013)
core
core
core
| It will be covered in Part-IV
| Trace generator will generate thread execution information
automatically
| Users do not need to worry about this
MacSim Tutorial (In ICPADS 2013)
| MacSim has 5 different clock domains
CPU
GPU
Last-level cache
Interconnection network
DRAM
# Clock
clock_cpu
clock_gpu
clock_l3
clock_noc
clock_mc
MacSim Tutorial (In ICPADS 2013)
3
1.5
1
1
1.6
| X86 instructions are mapped to uops
| PTX instructions are mapped to uops (almost 1-1 mapping)
MacSim
Macro instructions
with decoded
information from
Pin’s XED
Pin
XED
Trace
decoder
uops
Timing/
power
simulator
| Pipeline stages
Front-end
Decode
Rename
Schedule
Execution
Memory
MacSim Tutorial (In ICPADS 2013)
Retire
| Front-end, DEC/Rename: Just a simple FIFO queue
fetch_latency 5 // front-end depth
alloc_latency 5 // decode/allocation depth
width // pipeline width (same width for all the pipeline)
bp_dir_mech gshare
bp_hist_length 14 // branch history length
| Rename: create RAW dependency (map structure)
rob_size 96 // ROB size
| Scheduler // in-order scheduler, ooo scheduler
schedule io, ooo // instruction scheduling policy
MacSim Tutorial (In ICPADS 2013)
| Execution latency
Fixed uop latency (macsim-top/def/uop_latency_[x86,ptx].def)
Variable latency: Cache/Memory latency
| Instruction scheduling rates
isched_rate 4 // # of integer inst. that can be executed per cycle
msched_rate 2 // # of memory inst. that can be executed per cycle
fsched_rate 2 // # of FP inst. That can be executed per cycle
MacSim Tutorial (In ICPADS 2013)
| Cache configuration
# of sets, # of associativity, line size, # of banks, etc. (See manual)
| Cache size = # of sets x assoc x line_size x # of tiles
L3 only
| DRAM configuration
Frequency, bus width, column/activate/precharge latency
# of Memory controllers, # banks, # channels, row buffer size, DRAM
scheduling policy
Simple, but fast DRAM model that models key features
| MacSim can be easily connected with other DRAM models
Users can use DRAM-SIM2 for a detailed DRAM timing simulation
MacSim Tutorial (In ICPADS 2013)
| Statistics
Simulation outputs: *.stat.out
macsim/trunk/def file has stat definition (more details in Part-IV)
| Important Stats
IPC = INST_COUNT_TOT/CYC_COUNT_TOT
CPI = CYC_COUNT_TOT/INST_COUNT_TOT
| Per Core stats
IPC for core 0 INST_COUNT_CORE_0/CYC_COUNT_CORE_0
| Multiple applications stats
*.stat.out.<application_id> e.g.) memory.stat.out.0, bp.stat.out.1
Each stat file contains stats only for the first running (repeated
simulations are ignored)
MacSim Tutorial (In ICPADS 2013)
| Memory Systems
L[1-3]_HIT_CPU/L[1-3]_HIT_GPU
L[1-3]_MISS_CPU/L[1-3]_MISS_GPU
| Front-end
BP_ON_PATH_[CORRECT/MISPREDICT/MISFETCH ]
| Instruction profiling
Based on instruction category. inst.stat.out
| More details regarding statistics are in the documentation
| We will provide simple script file to fetch stat data
MacSim Tutorial (In ICPADS 2013)
MacSim Tutorial (In ICPADS 2013)
| Multi-threading support is already there
| Different ISAs: using micro-ops
| Warp?
One warp is treated as one thread. Each thread generates its own
trace file. Active bit information is included
Trace format will be explained in Part-IV
| Thread and block scheduling
Block-level barrier, block-level scheduling/retirement
More details will be explained in Part-IV
| Different memory structures
Memory systems (coalescing)
MacSim Tutorial (In ICPADS 2013)
Addr 0 Addr 1 Addr 2 Addr 3 Addr 4 Addr 5 Addr 6 Addr 7
Coalesced
Mem inst with 128B size
SIMD load instruction
Uncoalesced
64B Request
32B Req.
Trace file
Trace file
TraceInst
TraceInst_begin
TraceMem1
TraceMem2
TraceMem3
TraceInst_end
32B Req.
start of memory
instruction marker
end of memory
instruction marker
| Include the memory access by each thread of a warp as a
separate instruction in the trace
| In trace, mark these accesses as coming from the same warp
MacSim Tutorial (In ICPADS 2013)
Trace file
TraceInst_begin
TraceMem1
TraceMem2
TraceMem3
…
TraceMemN
TraceInst_end
MacSim
start of memory
instruction marker
Parent uop
uop
Mem_type: ld
#children: 8
Children uops
end of memory
instruction marker
addr0
addr1
addr2
addr3
addr4
addr5
…
addrN
| During simulation, form a “parent” uop that holds all the
individual memory accesses as its child uops
| Parent uop flows through the pipeline, only in the memory
stage, the individual children uops are issued to the memory
Parent uop is ready for retirement when all children have completed
MacSim Tutorial (In ICPADS 2013)