Performance Benchmarks in EMME/2

Matt Carlson INRO Seminar, Arup, London 2004-09-28

Introduction

• • • A look at some issues regarding performance benchmarks in EMME/2 Show a sample of benchmarks achieved by a range of hardware on the same EMME/2 model Hopefully stimulate some debate!

Purpose (1)

• • • • • Inspired by discussions on the INRO Lists regarding the influence of the following features on performance: Operating System (Windows, Unix, Linux) Speed of Processors Number of Processors Amount of memory Disk type (SCSI, RAID Arrays)

Purpose (2)

• • Winnipeg Model: The Winnipeg model runs ‘too quickly’ to get meaningful benchmarks on current hardware Some users have recently wondered why there are not more complex elements in the Winnipeg model, such as more complex turn penalties.

Meanwhile, Some Benchmarks

PLANET South AM Benchmarks 1381 zones, 1855 lines, 38764 segments, 24 MSA Iterations, 3 Trip Purposes 120 100 80 60 40 20 0 100 52 65 40 34 Minutes 26 PIII 733MHz 128Mb 133FSB Win2k Athlon XP 1.33GHz

512Mb 266FSB XP PIV 1.8GHz

512Mb 400FSB Win2k PIV 2.6GHz

512Mb 800FSB Win2k Dual Xeon 3.06GHz

2Gb 533FSB XP SCSI Dual Xeon 3.2GHz HT 2Gb 800FSB XP SCSI RAID0

Age of Hardware

Some Observations (1)

• • • • As expected, run times decrease as spec increases AMD Athlons look promising in terms of performance per clock cycle Front Side Bus (FSB) appears to be a significant indicator of bottleneck Athlon 64s should be investigated due to no FSB bottleneck and performance per clock cycle

Some Observations (2)

• • Generally, ‘more is more’, however: Hyperthreaded or Multiple Processors do not speed up run times, as expected • BUT: In the real world, the potential to do other things at the same time as a model run IS vastly increased by more physical or virtual processors

Some Observations (3)

• • Rule of Thumb (Assignment) - Zvi Leve, INRO: Highway assignment is more influenced by processor speed Transit assignment is more influenced by disk speed

Some Observations (4)

• • Rule of Thumb (Operating Systems) – Mike Florian, INRO: Linux: faster for disk access for matrix calculations Windows: faster for assignment

A Larger Demonstration Databank?

• • A larger model available to all users would provide a large sample of benchmarks There would be an opportunity to further showcase the potential of EMME/2 and ENIF, such as: – – – – More complex turn penalties Crowding on Transit Services Park & Ride Combined Assignment-Distribution-Mode Choice techniques such as the Santiago Model (Michael Florian and Shuguang He, 11 th EEUG, Madrid, 2002)

An Alternative Idea

• • • A ‘fictitious’ large model could be constructed: Network Data – Constructed from publicly available GIS data (freely available in the US) Matrix Data – Purely synthetic data, perhaps constructed using a gravity model

Conclusion

• • • Performance benchmarks could be more easily obtained on a more complex or larger model – Difficult to form conclusions based on a model not publicly available This could be achieved by: – – More complexity for the Winnipeg model A different, larger model – real or fictitious!

This could have the by-product of showcasing more EMME/2 and ENIF features