www.eu-egee.org

GRIDPP10 meeting at CERN, 3 June 2004

R-GMA status and plans

Abdeslem DJAOUI / RAL EGEE is a project funded by the European Union under contract IST-2003-508833

Contents

• • • • Who we are Reminder of what is R-GMA What we have done so far What we plan for the first release of the EGEE middleware

GRIDPP10 Meeting at CERN, 3 June 2004 - 2

JRA1-UK in EGEE

GRIDPP10 Meeting at CERN, 3 June 2004 - 3

Role of JRA1-UK in EGEE

• To meet all information and monitoring needs   Expect to base much of our re-engineering work on R-GMA R-GMA used in two modes: • • as standalone Inter-work with other information systems and displays • JRA1-UK are not responsible for the contents of the (Events) schema but will coordinate as necessary

GRIDPP10 Meeting at CERN, 3 June 2004 - 4

Team Organization

• • • • • • Mostly based in PPD at RAL Developers have mostly computer science/Maths/Physics background We have at least one weekly group meeting Using dotProject    Allows individual to record their own progress Allows all members to see the big picture Collects the information for the EGEE time sheets We can be e-mailed at:  [email protected]

Web page at:  http://hepunx.rl.ac.uk/egee/jra1-uk/

GRIDPP10 Meeting at CERN, 3 June 2004 - 5

People and Roles

• Based at RAL  Robin Middleton (0.5) • Cluster Manager  Steve Fisher • Technical Manager, Documentation and Quality  Abdeslem Djaoui • Architect, deputy technical manager and developer   Antony Wilson • Deputy technical manager and developer Linda Cornwall (0.5) • Documentation and Quality •      Rob Byrom • Developer Martin Craig • Developer New (and started) Steve Hicks • Developer John Walk • Developer New (and started) A.N.Other

• Short listing Integrator, sysadmin and developer Based at QMUL  Roney Cordenonsi • Developer

GRIDPP10 Meeting at CERN, 3 June 2004 - 6

Unified approach to information and monitoring

• • • Provides information on both:  The Grid itself (resources and services) • Mainly for the other middleware packages • The user may query it to monitor the status of the Grid  Grid applications • For users Applied to both information and monitoring  All tuples have time stamp Basis for other capabilities in the Grid     Problem determination Resource usage Logging of information Application monitoring

GRIDPP10 Meeting at CERN, 3 June 2004 - 7

R-GMA

Information Producer Producer service Registry service R-GMA services Mediator service Schema service Consumer service • • • A Relational implementation of GMA (Grid Monitoring Architecture from GGF) Powerful data model and query language • All data modelled as tables • SQL can express most queries in one expression Creates impression that you have one RDBMS per VO Information Consumer

GRIDPP10 Meeting at CERN, 3 June 2004 - 8

R-GMA Relational Data Model

• • • • • • Producers Consumers announce: publish: collect: SQL “CREATE TABLE” SQL “INSERT” SQL “SELECT” Some Producers (and the Registry and Schema) make use of RDBMS as appropriate – but what is central is the relational model .

Some producers make use of an in-memory data structure (for performance) Not a general distributed RDBMS system, but a way to use the relational model in a distributed environment where global consistency is not important

GRIDPP10 Meeting at CERN, 3 June 2004 - 9

LCG Installation of R-GMA

• • • Initially R-GMA will be added to LCG-2 to allow job accounting and job monitoring as requested by some experiments, it will *not* be used as an information system Following successful deployment at a number of sites in the UK and elsewhere, we have produced RPMs and instructions for adding R-GMA to LCG-2 Current status in LCG-2  RPM’s delivered 21 May. These include fixes requested by LCG   Testing by Piera Bettini (LCG) resumes 1 June.

R-GMA is expected to be approved as an LCG-2 add-on by 15 June

GRIDPP10 Meeting at CERN, 3 June 2004 - 10

JRA1: Milestones and Deliverables for the first year Month M03 M03 M03 M05 M05 M09 M10 M12 Deliverables & Milestones

MJRA1.1

DJRA1.1

MJRA1.2

MJRA1.3

DJRA1.2

MJRA1.4

MJRA1.5

DJRA1.3

Item Lead Partner

Tools for middleware engineering and integration deployed (Document) Architecture and Planning (Release 1) Software cluster development and testing infrastructure available CERN CERN CERN Integration and testing infrastructure in place including test plans (Release 1) CERN (Document) Design of grid services (Release 1) CERN Software for the Release Candidate 1 Integrated Release Candidate 1 enters testing and validation period (Release 1) CERN CERN (Software) Software and associated documentation (Release 1) CERN

GRIDPP10 Meeting at CERN, 3 June 2004 - 11

The challenge

• We need to re-engineer existing code, and introduce new functionality while at the same time providing continuously improved frequent releases and all without breaking backwards compatibility.

GRIDPP10 Meeting at CERN, 3 June 2004 - 12

Achievements so far

• • • • • New API implemented (partially) as wrapper round the old one for the EGEE MW prototype  We value feedback on the new API Registry replication completed and tested on our (UK) cluster testbed The new API includes authorisation  Not complete yet  No implementation yet Web service prototypes written for all components Extensive UML diagrams for new design    Once design agreed we will refactor code towards it Many potential simplifications have been identified Outline plan agreed for Schema replication

GRIDPP10 Meeting at CERN, 3 June 2004 - 13

Deliverables up to end of August

• • • • May 31   Detailed draft specification of our software On time (John Walk) June 30  code will be moved to the new repository, suitably restructured for the new SCM and build system. July 31   Finalise Java wrapper API Complete design of new services August 31  The C,C++ and Python APIs will finalised

GRIDPP10 Meeting at CERN, 3 June 2004 - 14

Up to the end of December

• • • • Sept 30  A Web services implementation release will be produced Oct 31  Multi VO support will be released Nov 30   The web services themselves will be ported to Windows We will make it very easy to build and distribute the system in source form such that it can be readily built and installed on new platforms.

 Replication of the schema and registry will be complete Dec 20  A complete XP port will be ready  Some more improvements to the mediator and to the SQL parser

GRIDPP10 Meeting at CERN, 3 June 2004 - 15

Over the year

• During the year:  Authorization code will be developed but it is not yet clear how much of it will be ready for RC1.  We will also start to look at interfacing to displays and other information and monitoring systems  by working with other clusters as appropriate understand requirements for the R-GMA role in Accounting, Job provenance and Logging & bookkeeping.

GRIDPP10 Meeting at CERN, 3 June 2004 - 16

And finally

• • Note that after August 31st there will be no external changes to the software so we will not be under pressure to release code before it is ready (improved quality). Delivering a working system is our top priority, so we may release less improvements than planned by Dec 20th.

GRIDPP10 Meeting at CERN, 3 June 2004 - 17