Transcript Document 7489010

EU DataGrid Data Management
Workpackage : WP2
Status and Plans
Peter Z Kunszt
IT/DB
http://cern.ch/grid-data-management/
EU Datagrid Conference Piliscsaba/Hungary
P.Kunszt
DGC5 4.9.2002
1
Outline
• WP2: Tasks and Constituency
• Task status
– Replication
– Metadata
– Optimization
– Security
• Plans
• Issues
P.Kunszt
DGC5 4.9.2002
2
Data Management Tasks
• Data Transfer
Efficient, secure and reliable transfer of data between sites
• Data Replication
Replicate data consistently across sites
• Data Access Optimization
Optimize data access using replication and remote open
• Data Access Control
Authentication, ownership, access rights on data
• Metadata Storage
Grid-wide persistent metadata store for all kinds of Grid
information
P.Kunszt
DGC5 4.9.2002
3
Current Constituency
• CERN Peter Kunszt, Heinz Stockinger, Leanne Guy,
Diana Bosio, Akos Frohner, Wolfgang Hoschek, Kurt
Stockinger
• INFN Flavia Donno, Andrea Domenici, Livio Salconi,
Giuseppe Andronico, Federico Di Carlo, Marco Serra
• PPARC Gavin McCance, William Bell, David Cameron,
Paul Millar
• Trento Cultural Institute Ruben Carvajal Schiaffino,
Luciano Serafini, Floriano Zini
• U.Helsinki/CSC Mika Silander, Joni Hahkala, Ville
Nenonen, Niklas Karlsson
• KDC Stockholm Olle Mulmo, Gian Luca Volpato
P.Kunszt
DGC5 4.9.2002
4
Additional Resources,
collaborators
•
•
•
•
•
P.Kunszt
LCG:Erwin Laure, Itzhak Ben-Akiva, James Casey
PPDG: Andy Hanushevsky
Condor: Alain Roy
NorduGrid: Aleksandr Konstantinov
Globus/ISI: Ann Chervenak, Bob Schwarzkopf
DGC5 4.9.2002
5
Current Status
• Replication: edg-replica-catalog – deployed
– GSI version available.
– Graphical User Interface available.
• Replication: edg-replica-manager – deployed
– Atomic replication operations.
– This component should be used for data management tests.
• Replication: GDMP – deployed
– More complex subscription-based functionality.
– Has been tested and is proven to work.
– Can be ‘abused’ to provide the same functionality as edgreplica-manager but it is overkill and much harder to do.
• Metadata Storage: SQL Database Service Spitfire – deployed
– Servlets on HTTP(S) with XML (XSQL)
– GSI enabled access + extensions
• Optimization: Replica Selection OptorSim – working
P.Kunszt
DGC5 4.9.2002
6
Current Status (cont.)
•
•
•
•
•
•
•
P.Kunszt
Replication: Replica Location Service – deployable
– Joint Globus – EDG development
– Unit and performance tests successful
– Distributed Replica Catalog service
– Graphical User Interface available.
Security: edg-security – in testing
– Security for Web Services
Metadata Storage: SQL Database Service Spitfire – in testing
– Web Service interface over SOAP
– Security through edg-security and proper local authorization
– Secure data browser through a web interface
Replication: Replica Manager Reptor – in testing and development
Optimization: Replica Selection Optor – in testing and development
Replication: Replica Metadata Catalog Repmec – in testing and
development
Security: edg-voms – in development, joint effort with WP6, WP7
DGC5 4.9.2002
7
WP2 Strategy for 1.2
• Integrate a single-instance RLS into TB1.2 (replacement for edgreplica-catalog)
• Integrate RLS into edg-replica-manager and GDMP
• Focus users away from GDMP and towards edg-replica-manager
for simple data management use cases.
• Increment stability and usability of edg-replica-manager
• Configuration
– Simplify configuration and build system of deployed
components
– Unify build for all components, adopt edg-standards
• Testing
– Large effort on unit testing and functionality tests inside WP2
for all components.
– Close interaction with ITeam and the new testing group
P.Kunszt
DGC5 4.9.2002
8
Towards Testbed 2
• Focus away from GDMP
• Introduction of Reptor (‘refactorization of GDMP’)
– Initially with identical functionality
– Incremental and controlled addition of requested functionality
– Use R-GMA for information gathering and publishing
• edg-replica-manager will stay as a command-line interface to
Reptor, easy migration for users
• Security: work towards new security model with VOMS and local
authorization.
• Documentation:
– Guides
– Examples, How-to
– Tutorials
P.Kunszt
DGC5 4.9.2002
9
Progress in WP2 Workshop
• Application of the conclusions of the Retreat towards
the WP2 Strategy
• Reallocation of resources according to the strategy.
• Re-focus Development
• Detailed workplan for the next incremental releases.
• Better understanding of the technical issues,
interoperation of components.
P.Kunszt
DGC5 4.9.2002
10
Progress in this Conference
• Joint session with WP5
– Agreement to continue with current solution in
GDMP, will be used in edg-replica-manager as well
– WP5 will produce a prototype service by the end of
September. These services will be used by Reptor.
– Close collaboration to make MSS access for Atlas
successful.
• Joint session with WP10
– Control flow, Metadata requirements
– Security requirements
– Close collaboration on deploying a WP10 metadata
catalog.
P.Kunszt
DGC5 4.9.2002
11
Progress in this Conference
• Interaction with WP3
– Agreement on usage of edg-security module for
R-GMA
• Joint session with WP1-4-5
– Agreement on the Zamboni daemon for worker
node storage administration. Joint development of
WP1-2-4
• Joint session with applications WP8-9-10
– Detailed feedback on current status was very
useful.
– We have taken the conclusions to heart in our
current strategy.
P.Kunszt
DGC5 4.9.2002
12
Issues
• Testing on the ‘real’ testbed.
• Dependencies upon components by other WPs need to
be understood so that we will have a good release
plan
• Optimizing Communication
P.Kunszt
DGC5 4.9.2002
13
Thank you for your Attention
P.Kunszt
DGC5 4.9.2002
14