Transcript Exporting Raw/ESD data from Tier-0  Tier-1s Wrap-up The problem (Bernd) • One copy of the LHC raw data for each of the LHC.

Exporting Raw/ESD data from
Tier-0  Tier-1s
Wrap-up
The problem (Bernd)
• One copy of the LHC raw data for each of the
LHC experiments is shared among the Tier-1’s
• Full copies of the ESD data (1/2 of raw data size)
• Total ~10PB/year exported from CERN
• The full machinery for doing this automatically
should be in place for full-scale tests in 2006
Tier-1 resources
TRIUMF
• 2 machines purchased, 1Gbit(?)
RAL
• Gbit link at present
• Parallel activities from ATLAS and CMS
• Not enough effort to dedicate for the moment
• More hardware in September
FNAL
• Just finished CMS DC – very labor intensive
• Enough resources to sustain 2TB/day
GridKA
• 1Gbit at present, expanding to 10Gbit in October/November
• Storage system is ready (dCache + TSM)
BNL
• SRM service almost ready (in a month)
•One gridftp node
• OC12 connection, not much used
NIKHEF/SARA
• 10Gbit since more than a year
•Running data challenges for experiments but mainly CPU intensive
IN2P3/Lyon
• Not yet ready with interface to MSS
• 1Gbit
Agreed tests
1.
2.
3.
4.
Simple disk-to-disk, peer-to-peer
Simple disk-to-disk, one-to-many
MSS-to-MSS
In parallel?
a) Transfer scheduling
b) Replica catalogue & management
Test 1: Simple test
• disk to disk: gridftp or bbftp (loadbalanced)
• Aggregated peer-to-peer transfers
• Files written, deleted and rewritten (process driven
from CERN)
• ~3 weeks preparation
• Goal: 1 week running “without” intervention
• Tier-1 responsibility: tune gridftp headnodes for
best transfer performance
Test 2: simple but one-to-many
• Additional Tier-0 complexity: export data to
more than one Tier-1 in parallel
Test 3: slide in MSS
• Issue with SRM interoperability
– GGF/GSM WG initiative to define a SRMv1.1
evolution ‘SRM-basic’
• Standardized error reporting
Test 4/5 (parallel)
• Test 4: slot in transfer scheduling: a
scheduling service controls the replication
• Test 5: add Replica catalogue and
management services
Timescales
TRIUMF
RAL
FNAL
June
July
one-to-many?
BNL
Aug
Transfer
scheduling
Sept
Oct
SRM-Basic
ready(?)
Nov
Dec
Jan’05
Next steps
• Statements from the other Tier-1’s
– NIKHEF, GridKA, Lyon
– PIC, CNAF, others?
•
•
•
•
Who is driving/coordinating? site contacts?
Meetings ?
Speed up SRM-basic specification process
...