Transcript slides

PERFORMANCE AND POWER
BENCHMARKS
Zhang Yu
TIRED? LET’S BEGIN WITH A STORY
P4 VS K8

A few years ago (like centuries in electronic
world) frequency was considered the key
element of a cpu`s performance, because of
Intel`s dominance in the market place and
their continuous bragging about frequency.
A TIME BOMB WAS BURIED…

In November 2000 Intel's heavily advertised
advances in clock speed reached an extreme
with the release of the Pentium 4 which
sacrificed per-cycle performance and used a
deep instruction pipeline to gain higher clock
speeds, ignoring problems that this introduced
of heat production and power consumption.
ARGUMENT AROUSED
Intel: “I am the fastest!”
 AMD, Apple etc. : “No you are not!”

NO ARGUE BUT BENCHMARK!
They decided to settle this…. Let`s race!
BENCHMARK START
GAME OVER

Bunny is fast indeed but unfortunately prone to
make mistakes….
MISTAKES
Deep instruction pipeline
(e.g. if a prediction is wrong performance
penalty will be greater because of deeper
pipeline.)
 higher clock speeds introduced heat production
and power consumption

HOW TO DETERMINE PERFORMANCE?
As computer architecture advanced, it became
more difficult to compare the performance of
various computer systems simply by looking at
their specifications. Therefore, tests were
developed that allowed comparison of different
architectures
 Benchmarks are particularly important in CPU
design, giving processor architects the ability to
measure and make tradeoffs in
microarchitectural decisions

HOW DOES IT WORK

Benchmarks are designed to mimic a particular
type of workload on a component or system
COMMON BENCHMARKS
Industry Standard (audited and verifiable)
 Business Applications Performance Corporation
(BAPCo)
 Embedded Microprocessor Benchmark
Consortium (EEMBC)
 Standard Performance Evaluation Corporation
(SPEC)
 Transaction Processing Performance Council
(TPC)
MICROSOFT WINDOWS BENCHMARKS
BAPCo: MobileMark, SYSmark, WebMark
 Futuremark:3DMark, PCMark
 Whetstone
 PiFast
 Super PI
 WinSAT, exclusively for Windows Vista, providing
an index for consumers to rate their systems
easily

CHALLENGES
Benchmarking is not easy and often involves
several iterative rounds in order to arrive at
predictable, useful conclusions.
 Vendors tend to tune their products specifically
for industry-standard benchmarks

LIMITATIONS AND PITFALLS OF
BENCHMARKS
Benchmarks do not address questions which you
did not ask… only see what you measure
 Specific application benchmarks will not tell you
about the performance of other applications
without proper analysis => extrapolation is not
straightforward
 You must understand the benchmark itself to
understand what it tells you
 General benchmarks will not tell you about the
details of your specific application

POWER ISSUES

When more power is used, a portable system
will have a shorter battery life and require
recharging more often. This is often the
antithesis of performance as most
semiconductors require more power to switch
faster
NVIDIA’S JOKE
nVidia`s Geforce graphic card:
 The cooling fan on this card is so powerful that
it can be used to clean the door way.

SAY GOODBYE TO P4

In 2004 problems of overheating led Intel to
abandon further development of its Netburst
microarchitecture
TREND: POWER! POWER! POWER!
The newest version of SPEC
(SPECpower_ssj2008) counts power
consumption as one of the key factor in
performance.
 overall ssj_ops/Watt
 Nowadays we often use Gflops/Watt as an
indicator of performance

REFERENCE
http://en.wikipedia.org/wiki/Computer_bench
mark
 http://www.spec.org/power_ssj2008/
 http://en.wikipedia.org/wiki/Megahertz_myth
 http://en.wikipedia.org/wiki/Performance_per_
watt
