Transcript Slide 1
Parallel File System Benchmarking Project Tracey Wilson DICE, Program Manager CSC [email protected] Outline DICE Overview What is the DICE TAP? The File System Comparison Issue and Why? DICE Plan for Framework Normalization Summary Nationwide Test Environment for Technology Innovations Testing & Validation Services Independent 3rd Party Evaluations Product & Technology Evaluation & Validations Proof-of-Concept & Performance Validations Technology Research & Development Projects Benchmark & Standards Development & Validation Evaluating HPC, Storage, Networking and Data Management Technologies DICE Architecture DICE utilizes a geographically distributed test bed Real world research networks DICE sites indicative of HPC Data Centers Quite Simply… A Test Bed for Advancing Innovation Independent, Unbiased 3rd Party Evaluations Genesis of a Vision by Government, Industry & Academia to Address Critical HPC Technology Challenges Community Input is Key The DICE Technical Advisory Panel Advises DICE on: Technology directions Project concepts Testing methodologies Utilizes working groups to focus on more specific issues Current members include: Kevin Regimbal, PNNL Chris Jordan, TACC Lee Ward, Sandia Steve Conway, IDC Kevin Wilson, P&G Dan Duffy, NASA Ron Bewtra, NOAA Lloyd Slonaker, DoD Matt Leininger, LLNL What is the File System Issue? Lack of standardized metrics for performance Need exists for file system performance comparison Need exists for defined benchmarks for local and remote file systems Different HPC systems and their file system architectures have no direct correlations Scaling: Scaling of current file systems has unpredictable performance impacts Load: Performance varies with file systems with loads of 50-70% Fragmented data is not accounted for Why do HPC Centers Care? Centers need input for: New procurements User requirements Application performance File systems are a critical piece of an HPC solution How do you as a Center Director make an informed decision? “Can you prove or rather disprove the performance or value of one file system compared to another?” How did the Effort Begin? In 2007, DICE received 3 proposals for evaluation projects: Panasas DoD Microsoft DICE Advisory Panel offered new solution “Develop a standard benchmark framework for file system evaluation” 1. 2. 3. Make the framework expandable to use Make it fully available for the HPC community to use Effort was to include normalization of results Benefits of Standardization Standardized Metric One set of benchmarks for all parallel file systems Results will be normalized for comparison HPC Community needs non-biased benchmark(s) Increased understanding of the impacts on current and proposed upgrades to a center’s file system: The storage subsystem performance Scaling Load performance Current Ideas Parameterized Benchmark Suite Simulate many different workloads Expose more tuning/configuration options in the benchmarks Suite Extensible so users can add new benchmarks Have unique benchmarks for synthetic workloads Transactions Streaming Random I/O Various Read/Write Ratios What is the plan? Currently surveying the HPC benchmark and file system community including: Carnegie Melon University of California/Santa Cruz ORNL Argonne National Labs PNNL TACC LLNL And many more Application I/O Patterns Many different I/O intensive applications Investigate the development of an I/O analyzer Recognize I/O utilization of applications Similar to trace and trace replay Add this functionality to the framework to simulate application I/O patterns Verification of performance Can be applied on different architectures Normalization Normalization is needed between differing architectures Need a formula Performance Cost Additional Criteria will be evaluated Aging and fragmentation of data should be considered Difficult to develop Need to develop on a small scale first Will the normalization hold as you scale systems? Plan is to create the framework first then normalize Summary Lot of collaborative effort from the Team and the Community is required Full project plan is in draft Survey underway to identify current efforts Plan to have initial framework for file system testing by early 2010 Plan to use this on other file system types How To Get Involved Collaboration is key Developing a DICE Forum for this project Will be collecting traces DICE TAP will be forming specific working group for this effort Interested parties should send email to [email protected] DICE Website www.diceprogram.org