Transcript Slide presentation Template

Experiment requirements for
middleware
F Harris (Oxford/CERN)
(WP8 coordinator )
[email protected]
Outline of the presentation
 Where
are we now – recap on expt recommendations
 Ongoing
work on use cases (HEPCAL +GAG)
 Reminder
large VOs
of some major problems–scalability and management of
Joint recommendations from Atlas/CMS work

There are essential developments (see EDG 2.0) needed in





Data Management
(robustness and functionality)
Information Systems
(robustness and scalability)
Workload Management
file specification)
(scalability for high rates, batch
Mass Storage Support
(gridified support due in EDG 2.0)
submissions,output
We must maintain and strengthen joint Experiment/EDG work in the
evaluation of system components AND the architecture
(both will
need to evolve – GRID developments are R/D)

Once middleware providers have done their ‘unit tests’ the applications must work
with them in the areas of:




Performance evaluation for the user with increasing rates of job submission and data
handling, and an expanding TB configuration
Streamlining procedures for feedback to middleware providers
EDG should provide site validation and monitoring procedures
EDG should provide good user tools for fault detection and diagnosis (what
is job status?, why did it fail?……..)
Ongoing work on use cases and reqts

‘Common Use Cases for A HEP Common Application Layer’ (HEPCAL)
http://lcg.web.cern.ch/LCG/SC2/RTAG4/finalreport.doc




General (authorisation,login,browse resources)
4 use cases
Data Management (metadata and data operations)
19 use cases
Job Management (submission,control,monitoring,errors
16 use cases

,resource estimation, job splitting…….)
VO Management (resource reservation,user rights
4 use cases
,software publishing…)
. EDG 1.4.3 satisfies use cases for a basic system(authorisation/authentication,data handling,job
submission)
. EDG 2.0 will satisfy more advanced data handling e.g. (metadata) and HEP data transformation
. There are other areas for discussion e.g. virtual data, experiment s/w publishing
(and security + VO Management is a crucial area)

This work is continuing within EDG and LCG (Grid Applications Group)

IN ATF there is regular scenario playing for use cases to check existing and future design
Current and future requirement analysis
 WP8
writing ‘ Experiment evaluation of EDG Testbed’ in which
experiment evaluations are being mapped to use cases (answers are
not black and white!…levels of satisfaction)
 LCG/GAG(




Grid Applications Group) starts series of meetings Feb 28
Reps from expts + WP8
Will review requirements and add ‘quality of service’
Will add requirements especially for interactive analysis, and for
management of multi-group/multi-function Vos
This will take into account developing experiment analysis models
Reminder of major problems – SCALABILITY
and management of large VOs




We hope that EDG 2.0 s/w will help with scalability problems in data
management and information system.
Let’s not dive off into complex new thinking before we have done detailed
large scale testing on RLS,R-GMA, job submission etc.
1.4.3 was fragile at the level of 10’s of users , 1000’s of files and 100’s of
jobs
Remind ourselves of experiment target requirements (e.g. CMS)


http://cmsdoc.cern.ch/cgi-doc/Grep_Search_F/documents/allxx.html?STRING=NOTE2001037

2000 users ( a good VOMS is essential!)

Up to 10K jobs/day

Managing millions of files
We may need some architecture changes for our Grid Services?( the jury is
out on that one…)