Transcript slides
PERFORMANCE AND POWER BENCHMARKS Zhang Yu TIRED? LET’S BEGIN WITH A STORY P4 VS K8 A few years ago (like centuries in electronic world) frequency was considered the key element of a cpu`s performance, because of Intel`s dominance in the market place and their continuous bragging about frequency. A TIME BOMB WAS BURIED… In November 2000 Intel's heavily advertised advances in clock speed reached an extreme with the release of the Pentium 4 which sacrificed per-cycle performance and used a deep instruction pipeline to gain higher clock speeds, ignoring problems that this introduced of heat production and power consumption. ARGUMENT AROUSED Intel: “I am the fastest!” AMD, Apple etc. : “No you are not!” NO ARGUE BUT BENCHMARK! They decided to settle this…. Let`s race! BENCHMARK START GAME OVER Bunny is fast indeed but unfortunately prone to make mistakes…. MISTAKES Deep instruction pipeline (e.g. if a prediction is wrong performance penalty will be greater because of deeper pipeline.) higher clock speeds introduced heat production and power consumption HOW TO DETERMINE PERFORMANCE? As computer architecture advanced, it became more difficult to compare the performance of various computer systems simply by looking at their specifications. Therefore, tests were developed that allowed comparison of different architectures Benchmarks are particularly important in CPU design, giving processor architects the ability to measure and make tradeoffs in microarchitectural decisions HOW DOES IT WORK Benchmarks are designed to mimic a particular type of workload on a component or system COMMON BENCHMARKS Industry Standard (audited and verifiable) Business Applications Performance Corporation (BAPCo) Embedded Microprocessor Benchmark Consortium (EEMBC) Standard Performance Evaluation Corporation (SPEC) Transaction Processing Performance Council (TPC) MICROSOFT WINDOWS BENCHMARKS BAPCo: MobileMark, SYSmark, WebMark Futuremark:3DMark, PCMark Whetstone PiFast Super PI WinSAT, exclusively for Windows Vista, providing an index for consumers to rate their systems easily CHALLENGES Benchmarking is not easy and often involves several iterative rounds in order to arrive at predictable, useful conclusions. Vendors tend to tune their products specifically for industry-standard benchmarks LIMITATIONS AND PITFALLS OF BENCHMARKS Benchmarks do not address questions which you did not ask… only see what you measure Specific application benchmarks will not tell you about the performance of other applications without proper analysis => extrapolation is not straightforward You must understand the benchmark itself to understand what it tells you General benchmarks will not tell you about the details of your specific application POWER ISSUES When more power is used, a portable system will have a shorter battery life and require recharging more often. This is often the antithesis of performance as most semiconductors require more power to switch faster NVIDIA’S JOKE nVidia`s Geforce graphic card: The cooling fan on this card is so powerful that it can be used to clean the door way. SAY GOODBYE TO P4 In 2004 problems of overheating led Intel to abandon further development of its Netburst microarchitecture TREND: POWER! POWER! POWER! The newest version of SPEC (SPECpower_ssj2008) counts power consumption as one of the key factor in performance. overall ssj_ops/Watt Nowadays we often use Gflops/Watt as an indicator of performance REFERENCE http://en.wikipedia.org/wiki/Computer_bench mark http://www.spec.org/power_ssj2008/ http://en.wikipedia.org/wiki/Megahertz_myth http://en.wikipedia.org/wiki/Performance_per_ watt