Transcript US View of GDB

Global Scientific Collaborations
and Grids: A View from Fermilab
Victoria A. White
Head, Computing Division Fermilab
[email protected]
f
The 3 Top issues for Particle
Physics today
(1) Collaboration (2) Collaboration (3) Collaboration
› Global within the field - needed for every stage of
an experiment
 to build the detector and the software, operate and
monitor the experiment, produce Monte Carlo data,
process data, analyze data, participate in the physics
› With other sciences – for Funding need
›
recognition and partnerships outside our field.
Also to train the next generation of scientists.
With Educators and Communicators – For public
education and to make a difference in society
 This is needed worldwide – on both sides of the digital
divide
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
2
The Fermilab Experimental
Program Statistics
f
“Active” experiments from the 2003
Research Program Workbook
› Of 213 Institutions involved 114 of
them are non-US
› Of 1916 physicists, 753 are non-US
› Of 699 students, 234 are non-US
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
3
f
›
›
›
›
›
›
›
›
›
›
›
Significant Contributions to the
Fermilab Program from the “other
side” of the digital divide
Brasil
Argentina
Colombia
Mexico
Czech Republic
Taiwan
Russia
China
Korea
India
…. and others
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
4
f
Highlighting Brasil (1)
History of truly significant contributions to Fermilab
Experiments since 1983
 Series of charm and heavy flavor expts
• E791 was ahead of its time in volume of data
• Analyzed about 10% of E791 data in Rio - one of only 4
institutions - Ohio, Mississipi, Fermilab, CBPF
• Significant contribution to the physics - data led to the 1st
publication for E791.
• Focus
 Selex
CBPF/LAFEX + Fermilab hosted CHEP95 – pushing
networking and videoconferencing technology
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
5
f
Highlighting Brasil (2)
› Most recently big contributions to the D0
experiment
 Designed and built the POTS for the Forward
Proton Detector giving the D0 expt a new
physics reach
 Physics Leadership – far beyond the size of the
effort
› Leon Lederman, Fermilab Director
Emeritus, has throughout provided support
and encouragement for Physics in South
America
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
6
f
IT Technology and Physics
› Network connectivity and IT infrastructure must
be available at some minimal level in order to
participate in a Physics Experiment.
 Being on the wrong side of the digital divide is painful
› We need to help ensure that governments
understand
 Just how important this contribution is to the Science &
 THAT HIGH ENERGY PHYSICS IS IN THE
FOREFRONT OF SHOWING HOW GLOBAL E-SCIENCE
CAN BE CARRIED OUT AND OF DEMONSTRATING
THE USE OF TECHNOLOGIES AND BEHAVIORS THAT
WILL PROFOUNDLY AFFECT EDUCATION AND
COMMERCE AND SOCIETY.
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
7
f
›
›
›
›
›
›
›
›
›
›
Fermilab Ongoing Scientific
Program, 2004
Run II (CDF, D0, Accelerator. Upgrades) ***
MINOS *
MiniBoone
Sloan Digital Sky Survey **
CMS *****
Lattice QCD Theory collaboration **
Pierre Auger and CDMS
Test Beams
BTeV in pre-project stage, preparing for review ****
SNAP, Off-axis neutrinos and a few other efforts in R&D, LOI
stages
*represents Large Global Collaboration/Petabyte Datasets/Massive
distributed computational power needed
And R&D for the Linear Collider – a worldwide effort we assume
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
8
f
Inclusive Worldwide Collaboration
is essential for Particle Physics
What are we doing at Fermilab to help this ?
 Networks and Network research
 Grids -- lots on this – important strategically
 Guest Scientist program
 Education and Outreach program
 Experiment sociology and leadership – changes
 Videoconferencing
 Virtual Control Rooms
 Physics Analysis Center for CMS
 Public Relations
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
9
f
Networks
› Starting 1 year ago Fermilab moved
aggressively to make a plan for ~3 GigE
connectivity in the (fairly near) future
 Dark Fiber to Starlight (contract just about in
place finally)
 Worked on Office of Science Strategic
Networking plan
• Expect Metropolitan Area Network (MAN) in Chicago
area linking ANL, Fermilab and Starlight in a ring
 Public Planning processes and reports are
important – e.g. TAN report, ICFA SCIC
• We try to participate whenever we are invited to
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
10
ESnet in 2003: OC192 and OC48 Links Coming Into Service;
Consider Links to US HENP Labs; Evolution Not Sufficient
f
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
11
f
ESNet proposal for Chicago area
MAN (Metropolitan Area Network)
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
12
f


Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
13
f
Network Research
› Also in the past year Fermilab chose to become a
›
beginning player in the Network research arena
As Harvey Newman says often raw bandwidth is
not enough – we need to learn how to use it,
monitor the network, look for research partners
and opportunities
 Joining with several partners on proposal for UltraNet
(DOE research network) exploitation and with Caltech et.
al. on Ultralight program of work/proposal
 Some international opportunities for broader
collaborations
• UK, CA, Czech, Netherlands, DataTag, ?
› Research agenda pushes provisioning of network
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
14
f
Network Monitoring
› Modest effort to help with Les
Cottrell’s IEPM and Pinger projects
› Modest efforts by Fermilab network
group to help track down network
bottlenecks and problems
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
15
f
Grids for Science
› There is no doubt that computing
resources for Particle Physics are
going to be distributed globally and
shared between “Virtual
Organizations”
Happening already with Run II, Babar,
RHIC and LHC experiments
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
16
LHC: Key Driver for Grids
f
› Complexity: Millions of individual detector channels
› Scale:
PetaOps (CPU), 100s of Petabytes (Data)
› Distribution: Global distribution of people & resources
2000+ Physicists
159 Institutes
36 Countries
Vicky White
CMS Collaboration
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
17
Grid Projects and Coordinating
Bodies
f
Fermilab is involved in numerous Grid projects and
coordination bodies
 PPDG, GriPhyN and iVDGL - the Trillium US Grid
projects
 LCG - GDB, SC2, (PEB until recently), bodies and working
groups on Security, Operations Center, Rollout, etc.
 SRM – storage systems standards
 Joint Technical Board and Interoperability work
 Global Grid Forum – Physics Research Area
 Global Grid Forum – Security Research Area
 Open Science Grid – meta-level collaboration and
coordination
 And probably some I forgot…..
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
18
f
Fermilab Grid Strategy
› Strategy (from a Fermi-centric view)
 Common approaches and technologies across our
entire portfolio of experiments and projects
› Strategy (from a global HEP view)
 Work towards common standards and
interoperability
› At Fermilab we are working aggressively to
put all of our Computational and Storage
Fabric on “the Grid” and to contribute
constructively to interoperability of Grid
middleware
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
19
f
Successes – D0 and CDF
› SAM-GRID – a functional Grid system
developed for D0 and now in use by CDF
for Data Storage with meta-data, Data
Delivery and caching, Job Execution (JIM)
and Monitoring
 SAM “stations” worldwide
 Lots of real operational experience and
solutions for robustness and scalability
 Evolving to use Globus and Condor based
“standards”
• Supports GridFTP + other storage access/file Xfr
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
20
f
SAM-GRID System at D0
Integrated Files Consumed vs Month (DØ)
4.0 M Files Consumed
Summary of Resources (DØ)
Registered Users
600
Number of Stations
56
Registered Nodes
900
Total Disk Cache
40 TB
Number of Files
1.5M
Integrated GB Consumed vs Month (DØ)
1.2 PB Consumed
3/02
Vicky White
3/03
Regional Center
Analysis site
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
21
f
SAM at a Glance – D0 and CDF
› http://d0db.fnal.gov/sam_local/SamAtAGlance/
 Includes Stations in U.S., Czech Republic, India, France,
UK, Netherlands, Germany, Canada
• Monitored
• Drill down to display jobs running and files being delivered
› http://cdfdb.fnal.gov/sam_local/SamAtAGlance/
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
22
f
Vicky White
Successes - US-CMS
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
23
f
Grid2003: An Operational Grid Huge Success !
28 sites (2100-2800 CPUs)
 400-1100 concurrent jobs
 10 applications
 Running since October 2003

Korea
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
http://www.ivdgl.org/grid2003
24
Grid2003: A Collaborative
Effort
f
› Trillium Grid projects
 PPDG + GriPhyN + iVDGL
 (US-ATLAS, US-CMS, BTEV, LIGO, SDSS, Computer
Science)
› US-ATLAS and US-CMS projects
 Fermilab, LBL, Argonne
 U. New Mexico, U. Texas Arlington
› Korean site
 Kyungpook National University (CMS)
› New sites
 University of Buffalo (CCR)
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
25
f
Vicky White
Grid2003: 3 Months Usage
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
26
f
Open Science Grid
› Goals (http://www.opensciencegrid.org/)
 Support US-LHC research program, other scientific
efforts
 Federate with LCG and peer with EGEE in Europe
 Involve laboratories (DOE) and universities (NSF)
› Getting there: Grid2004 (OSG-1), Grid2005
(OSG-2), …
 Series of releases  increasing functionality & scale
 Persistent Production quality Grid + Grid Laboratory
› Jan. 12 meeting in Chicago
 >60 participants from labs and universities, Computer
Science and Physics and Biology, and Grid Projects
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
27
f
Federating Grids
As the US representative to the LHC Computing Grid
Grid Deployment Board (LCG GDB) I have
repeatedly pushed the case for the LHC
Computing Grid to be a “Grid of Grids” – not a
distributed Computing center
 Resources must be able to belong to >1 Grid
• E.g. NorduGrid, UK Grid, Open Science Grid, TeraGrid,
Region XYZ Grid, University ABC Grid and Fermilab Grid!
 Grids must learn to interoperate
 Governance, policies, security, resource usage and service
level agreements must be considered at many levels
including at the level of a Federation of Grids
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
28
f
Inclusive Worldwide Collaboration
is essential for Particle Physics
What are we doing at Fermilab and in
Fermilab experiments to help this ?
 Networks and Network research
 Grids -- lots on this – important strategically
 Guest Scientist program
 Education and Outreach program
 Experiment sociology and leadership – changes
 Videoconferencing
 Virtual Control Rooms
 Physics Analysis Center for CMS
 Public Relations
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
29
f
Support for Guest Scientists and
Engineers
› Although budgets are tight Fermilab
(and especially the Computing
Division) wants to encourage and
support a guest scientist and guest
engineer/software engineer program
Tremendously beneficial to the expts
Tremendously important for ongoing
collaboration and communication
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
30
f
Education
› Quarknet meets “Grid”
› Traditional K-12 Education Program at
Fermilab is working with Grid
Projects and Computing Division on
the educational opportunities
afforded by the Grid
› Open Science Grid – will have a
significant Educational component.
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
31
f
Sociology and Leadership changes
› Increasing awareness at Fermilab of
the need to include those outside
Fermilab
Meeting times to take account of time
zones
Videoconferencing
Posting materials and documenting
Leaders not resident at Fermilab
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
32
f
Videoconferencing
› We do more and more at Fermilab each
year – 5 new fully equipped conference
rooms last year
 Total of 19 rooms - all Polycom based
› Fermilab is instituting better “official”
support for VRVS and H323-based
conferencing.
 New server v. soon
 Expect to see growth in desktop video in next 2
years
 User Requirements meeting soon – to plan next
steps
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
33
f
Virtual Control Room
› MINOS has one at Fermilab for a detector
at the Sudan Mine in Minnesota
 Control Room Logbook
› US CMS is going to build one at Fermilab
 Now we in the US are the ones who will be far
from the apparatus and the action!
 We hope our work in the next few years will be
broad reaching and benefit all who are “far
from the experiment”.
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
34
f
CMS Physics Analysis Center(s) in
the U.S.
› Serious thought going into this now. What is a
›
PAC?
We can’t all go to CERN, all the time, to do physics
 Must empower physicists to do analysis away from CERN
 Technology is important – networks, videoconferencing,
access to data
• But there is more
 What do we need at Fermilab to have a vibrant center
for physics where people come to work together both
physically and virtually?
› The effort we put into this may have far reaching
effects on the “democratization of science”.
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
35
f
Public Relations
› Judy Jackson, head of Public Affairs at
›
›
Fermilab, works closely with many other
public affairs groups – at CERN, SLAC and
elsewhere
Interactions.org website
Forming a Strategic Grid Communications
group – presented a plan for this at a
recent Open Science Grid meeting
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
36
f
Role of Science in the Information
Society
Fermilab/SLAC exhibit from SC2003
on
Networking/Grids/Storage/Simulatio
ns went to the RSIS exhibition in
Geneva recently
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
37
f
Inclusive Worldwide Collaboration
is essential for Particle Physics
What are we doing at Fermilab to help this ?
 Networks and Network research
 Grids -- lots on this – important strategically
 Guest Scientist program
 Education and Outreach program
 Experiment sociology and leadership – changes
 Videoconferencing
 Virtual Control Rooms
 Physics Analysis Center for CMS
 Public Relations
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
38
f
IT Technology and Physics
› Network connectivity and IT infrastructure must
be available at some minimal level in order to
participate in a Physics Experiment.
 Being on the wrong side of the digital divide is painful
› We need to help ensure that governments
everyone understands
 Just how important this contribution is to the Science &
 THAT HIGH ENERGY PHYSICS IS IN THE
FOREFRONT OF SHOWING HOW GLOBAL E-SCIENCE
CAN BE CARRIED OUT AND OF DEMONSTRATING
THE USE OF TECHNOLOGIES AND BEHAVIORS THAT
WILL PROFOUNDLY AFFECT EDUCATION AND
COMMERCE AND SOCIETY.
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
39
f
Conclusions
› We must keep working on all fronts to
collaborate, educate, make the case
for Science and for the Information
Technology and Networking
improvements that are essential for
success – everywhere
in particular to ensure that the Digital
Divide does not exclude physicists.
Vicky White
Digital Divide Workshop- Rio de Janeiro, Feb 16-20, 2004
40