Transcript slides
PERFORMANCE AND POWER
BENCHMARKS
Zhang Yu
TIRED? LET’S BEGIN WITH A STORY
P4 VS K8
A few years ago (like centuries in electronic
world) frequency was considered the key
element of a cpu`s performance, because of
Intel`s dominance in the market place and
their continuous bragging about frequency.
A TIME BOMB WAS BURIED…
In November 2000 Intel's heavily advertised
advances in clock speed reached an extreme
with the release of the Pentium 4 which
sacrificed per-cycle performance and used a
deep instruction pipeline to gain higher clock
speeds, ignoring problems that this introduced
of heat production and power consumption.
ARGUMENT AROUSED
Intel: “I am the fastest!”
AMD, Apple etc. : “No you are not!”
NO ARGUE BUT BENCHMARK!
They decided to settle this…. Let`s race!
BENCHMARK START
GAME OVER
Bunny is fast indeed but unfortunately prone to
make mistakes….
MISTAKES
Deep instruction pipeline
(e.g. if a prediction is wrong performance
penalty will be greater because of deeper
pipeline.)
higher clock speeds introduced heat production
and power consumption
HOW TO DETERMINE PERFORMANCE?
As computer architecture advanced, it became
more difficult to compare the performance of
various computer systems simply by looking at
their specifications. Therefore, tests were
developed that allowed comparison of different
architectures
Benchmarks are particularly important in CPU
design, giving processor architects the ability to
measure and make tradeoffs in
microarchitectural decisions
HOW DOES IT WORK
Benchmarks are designed to mimic a particular
type of workload on a component or system
COMMON BENCHMARKS
Industry Standard (audited and verifiable)
Business Applications Performance Corporation
(BAPCo)
Embedded Microprocessor Benchmark
Consortium (EEMBC)
Standard Performance Evaluation Corporation
(SPEC)
Transaction Processing Performance Council
(TPC)
MICROSOFT WINDOWS BENCHMARKS
BAPCo: MobileMark, SYSmark, WebMark
Futuremark:3DMark, PCMark
Whetstone
PiFast
Super PI
WinSAT, exclusively for Windows Vista, providing
an index for consumers to rate their systems
easily
CHALLENGES
Benchmarking is not easy and often involves
several iterative rounds in order to arrive at
predictable, useful conclusions.
Vendors tend to tune their products specifically
for industry-standard benchmarks
LIMITATIONS AND PITFALLS OF
BENCHMARKS
Benchmarks do not address questions which you
did not ask… only see what you measure
Specific application benchmarks will not tell you
about the performance of other applications
without proper analysis => extrapolation is not
straightforward
You must understand the benchmark itself to
understand what it tells you
General benchmarks will not tell you about the
details of your specific application
POWER ISSUES
When more power is used, a portable system
will have a shorter battery life and require
recharging more often. This is often the
antithesis of performance as most
semiconductors require more power to switch
faster
NVIDIA’S JOKE
nVidia`s Geforce graphic card:
The cooling fan on this card is so powerful that
it can be used to clean the door way.
SAY GOODBYE TO P4
In 2004 problems of overheating led Intel to
abandon further development of its Netburst
microarchitecture
TREND: POWER! POWER! POWER!
The newest version of SPEC
(SPECpower_ssj2008) counts power
consumption as one of the key factor in
performance.
overall ssj_ops/Watt
Nowadays we often use Gflops/Watt as an
indicator of performance
REFERENCE
http://en.wikipedia.org/wiki/Computer_bench
mark
http://www.spec.org/power_ssj2008/
http://en.wikipedia.org/wiki/Megahertz_myth
http://en.wikipedia.org/wiki/Performance_per_
watt