Transcript Randy Boone, Natural Resource Ecology Laboratory, CSU

“Big Data” at the
Natural Resource Ecology Laboratory
Randall B. Boone
Research Scientist, Natural Resource Ecology Laboratory and
Associate Professor, Department of Ecosystem Science and Sustainability
ISTeC Big Data Forum
Colorado State University
April 18, 2013
Department of Ecosystem
Science and Sustainability
Apr 18, 2013
ISTeC Big Data Forum, Colorado State University
1
Integrating across scales through
Top-down
and
Bottom-up
approaches
D. Ojima
Apr 18, 2013
ISTeC Big Data Forum, Colorado State University
2
Agent-based Modeling and Big Data
Apr 18, 2013
ISTeC Big Data Forum, Colorado State University
3
G-Range, A Global Rangeland Model
Apr 18, 2013
ISTeC Big Data Forum, Colorado State University
4
T. Hilinski
Apr 18, 2013
ISTeC Big Data Forum, Colorado State University
5
DayCent and Century Simulations
T. Hilinski
Potential Vegetation: Mean Annual NPP (gC/m2) 1961-2006 Mean
Apr 18, 2013
ISTeC Big Data Forum, Colorado State University
6
Managing Big Data at NREL
Our data are big, but well described
Unlike some industrial applications, the system must be
exceedingly flexible
Must be responsive to a variety of users (e.g., more
diverse uses than the ISTeC Cray)
Calculation-intensive uses
Users are unlikely to have the ability to parallelize tools
Apr 18, 2013
ISTeC Big Data Forum, Colorado State University
7
Rubel Cluster – The Backbone of NREL’s
“Mid-Performance Computing”
Distributed memory computer cluster
256 processors
500 Gflops
463 GB memory
2 TB immediate storage
50-60 TB extended storage, plus 120 TB coming online
Private 1 Gbps Ethernet interconnect, with 10 Gbps
access to storage
Apr 18, 2013
ISTeC Big Data Forum, Colorado State University
8
Rubel Cluster
Provides a testing platform for the CRAY for some
Others use alternatives, such as R, which can make
use of multiple processors
Good, but not high performance computing
A mix of desktop and cluster analyses yield “midperformance computing”
But it is effective!
Apr 18, 2013
ISTeC Big Data Forum, Colorado State University
9
Tom Hilinski
Apr 18, 2013
ISTeC Big Data Forum, Colorado State University
10
Rubel – Past, Present, and Future
Apr 18, 2013
ISTeC Big Data Forum, Colorado State University
11