Transcript The LCG Project common solutions for the LHC

The LHC Computing Project
Common Solutions for the LHC
ACAT 2002
Presented by
Matthias Kasemann
FNAL and CERN
1
Matthias Kasemann, FNAL and CERN, June 25, 2002
Outline
2/39
 The LCG Project: goal and organization
 Common solutions:
Why common solutions
 How to …
 The Run2 common projects

 The LCG Project: status of planning
Results of the LCG workshop in March 02
 Planning in the Applications Area

For the LCG Grid see: Les Robertson (Thursday)
“The LHC Computing Grid Project - Creating a Global Virtual
Computing Center for Particle Physics”
From Raw Data to Physics:
what happens during analysis
Matthias Kasemann, FNAL and CERN, June 25, 2002
250Kb – 1 Mb
100 Kb
25 Kb
5 Kb
e+
2037 2446 1733 1699
4003 3611 952 1328
2132 1870 2093 3271
4732 1102 2491 3216
2421 1211 2319 2133
3451 1942 1121 3429
3742 1288 2343 7142
Raw data
Convert to
physics
quantities
3/39
500 b
f
Z0
eDetector
response
apply
calibration,
alignment,
_
f
Interaction with Fragmentation, Basic physics
detector material Decay
Pattern,
Physics
Results
recognition,
analysis
Particle
identification
Analysis
Reconstruction
Simulation (Monte-Carlo)
HEP analysis chain:
common to LHC experiments
Detector
alignment
Matthias Kasemann, FNAL and CERN, June 25, 2002
Detector
description
Detector
calibration
Reconstruction
parameters
Generate
Events
4/39
Build
Simulation
Geometry
Build
Reconstruction
Geometry
Physics
Reconstuction
geometry Reconstruct
Events
Analyze
Events
Simulation
geometry
ATLAS
Detector
Simulate
Events
Raw
Data
ESD
AOD
Developing Software for LHC experiments
Matthias Kasemann, FNAL and CERN, June 25, 2002
 Challenges in big collaborations
5/39
Long and careful planning process
 More formal procedure required to commit resources
 Long lifetime, need flexible solutions which allow for
change




Any state of experiment longer than typical Ph.D. or
postdoc time
Need for professional IT participation and support
New development, maintenance and support model
required
 Challenges in smaller collaborations
Limited in resources
 Adapt and implement available solutions (“b-b-s”)

Matthias Kasemann, FNAL and CERN, June 25, 2002
CMS - CCS schedule (V33):
the bottom line
6/39
Milestones of ~ next year:
Milestones a few yrs away:
delays of ~9 months
delays of ~15 months
 LHC starts
CMS - CCS Software
Baseline:
DDD
ready for OSCAR,L2
ORCA,milestones
IGUANA
Switch from Geant3 to Geant4:
Matthias Kasemann, FNAL and CERN, June 25, 2002
•Data model defined
•Persistent and transient representations
•Demonstrably as correct as existing CMS description
Software Infrastructure
deployed and working
Framework for processing CMS data
•Working for simulation, reconstruction, analysis
•Supporting persistency and data management
7/39 •Strongly dependent on LCG success
•Date not decided (just my estimate)
•E.g. it needs the new persistency
CCS Baseline
Software for TDR’s
User analysis components
•Framework with coherent user interface
•Event display / interactive visualisation
•Tools for browsing / manipulating data sets
•Data presentation, histograms, numerical,…
Areas
Common Solutins
TheWork
LHC
Computing Grid
Applications Support &
Experiments and Regional
Project
(LCG)
Coordination
Centres agree on
requirements for common
Project Overview Board
Applications Support &
Coordination
Computing Systems
Grid Technology
Grid Deployment
Software and
Computing
Project
Execution Work Plan Definition Committee
(SC2)
Board
Matthias Kasemann, FNAL and CERN, June 25, 2002
Computing Systems
Grid Technology
Grid Deployment
8/39
Work
Areas
projects
WP WP WP WP WP
RTAG
Matthias Kasemann, FNAL and CERN, June 25, 2002
LCG - Fundamental goal:
9/39
The experiments have to get the best, most
reliable and accurate physics results from the
data provided by their detectors
Their computing projects are fundamental to the
achievement of this goal
The LCG project at CERN was set up to help
them all in this task
Corollary
Success of LCG is fundamental to success of
LHC Computing
Matthias Kasemann, FNAL and CERN, June 25, 2002
Fulfilling LCG Project Goals
 Prepare and deploy the LHC Computing Environment
 Applications - provide the common components, tools and
infrastructure for the physics application software
 Computing system – fabric, grid, global analysis system
 Deployment – foster collaboration and coherence
 Not just another grid technology project
 Validate the software by participating in Data Challenges using the
progressively more complex Grid Prototype
 Phase 1 - 50% model production grid in 2004
 Produce a TDR for full system to be built in Phase 2
 Software performance impacts on size and cost of production
facility
 Analysis models impact on exploitation of production grid
 Maintain opportunities for reuse of deliverables outside LHC
experimental programme
10/39
Applications Activity Areas
Matthias Kasemann, FNAL and CERN, June 25, 2002
 Application software infrastructure
11/39

physics software development environment, standard
libraries, development tools
 Common frameworks for simulation and analysis

Development and integration of toolkits & components
 Support for physics applications

Development, support of common software tools &
frameworks
 Adaptation of Physics Applications to Grid environment
 Object persistency and data management tools

Event data, metadata, conditions data, analysis objects,
Goals for Applications Area
Matthias Kasemann, FNAL and CERN, June 25, 2002
 Many Software Production Teams
LHC experiments
 CERN IT groups, ROOT team, ..
 HEP software collaborations – CLHEP, Geant4 , ..
 External Software – python, Qt, XML, …

 Strive to work together to develop and use software in common
 Will involve identifying and packaging existing HEP software for
reuse as well as developing new components
 Each unit has its own approach to design and to supporting the
development

Sharing in the development and deployment of software
will be greatly facilitated if units follow a common
approach
 Recognise that there will be start-up costs associated with
adapting to use new common products and development tools
12/39
Why common and when?
Matthias Kasemann, FNAL and CERN, June 25, 2002
 Why not:
Experiments have independent detectors and analysis
tools verify physics results
 Competition for best physics results
 Coordination of common software development is
significant overhead

 Why common solutions:
Need mature engineered software
 Resources are scarce, in particular manpower
 Effort: Common projects are a good way to become more
efficient ( 1 ,  2 n , n , n 2 ?)
 Lessons need to be learnt from past experience

 For LHC experiments:
Everything non experiment–specific is a potential
candidate for a common project
13/39
Matthias Kasemann, FNAL and CERN, June 25, 2002
FNAL: CDF/D0/CD - Run 2
Joint Project Organization
14/39
D0 Collaboration
Directorate
CDF Collaboration
External Review
Committee
R2JOP
Steering Committee
Task
Coordinators
Run II
Committee
Basic Infrastructure
Run II Computing
Project Office
Mass Storage &
Data Access
Serial Media
Storage
Fermilab
Working
Management
Simulation
Class
Group
Library
Configuration Support
MSS
Data
Access
Management
Hardware
Databases
Reconstruction
Systems
Physics Analysis
Support
Reconstruction Networking
farm hardware hardware
Production Reconstruction
Management input pipeline
15 joint projects defined,
4 years before start of data taking
Physics
analysis
hardware
Visualization
Physics Analysis Software
Perceptions of Common Projects
 Experiments
Whilst may be very enthusiastic about long-term
advantages ….
 …have to deliver on short term milestones
 Devoting resources to both will be difficult
 Already experience an out-flux of effort into common
projects
 Hosting projects in experiments excellent way of
integrating effort
Matthias Kasemann, FNAL and CERN, June 25, 2002


For initial phase and prototyping
 Technology groups
15/39
Great motivation to use expertise to produce useful
solutions
 Need the involvement of the experiments

Matthias Kasemann, FNAL and CERN, June 25, 2002
Common solutions - How to do?
 Requirements are set by experiments in the SC2 +
Requirements Technical Assessment Groups (RTAGs)
 Planning and implementation is done by LCG together
with experiments
 Monitoring of progress and
adherence by the SC2
 Frequent releases and testing
 Guaranteed life-time maintenance and support
Issues:
 ‘How will applications area cooperate with other areas?’
 ‘Not feasible to have a single LCG architect to cover
all areas.’
 Need mechanisms to bring coherence to the project
16/39
Workflow around the
organisation chart
Matthias Kasemann, FNAL and CERN, June 25, 2002
WPn
PEB
PEB develops
workplan
Project plan
PEB manages
LCG resources
Release 1
~4 mths
17/39
PEB tracks
Release 2
progress
SC2
Prioritised
requirements
SC2 Sets RTAGm
the
mandate
requirements
requirements
~2 mths
Updated workplan
Workplan feedback
SC2 approves the
workplan
Status report
Review feedback
time
SC2 reviews the
status
Matthias Kasemann, FNAL and CERN, June 25, 2002
Issues related to partitioning
the work
18/39
 ‘How do you go from present to future without
dismantling existing projects?’
 ‘Have to be careful that we don’t partition into too
small chunks and lose coherence of overall software’
 We are not starting afresh, we have a good knowledge
of what the broad categories are going to be
 Experiment architectures help to ensure coherency.
Matthias Kasemann, FNAL and CERN, June 25, 2002
Coherent Architecture
19/39
 Applications common projects must follow a coherent
overall architecture
 The software needs to be broken down into
manageable pieces i.e. down to the component level
 Component-based, but not a bag of disjoint
components
components designed for interoperability through clean
interfaces
 Does not preclude a common implementation foundation,
such as ROOT, for different components




The ‘contract’ in the architecture is to respect the
interfaces
No hidden communication among components
Starting point is existing products, not a clean slate
Matthias Kasemann, FNAL and CERN, June 25, 2002
Approach to making workplan
20/39
 “Develop a global workplan from which the RTAGs can be derived”
 Considerations for the workplan:
 Experiment need and priority
 Is it suitable for a common project
 Is it a key component of the architecture e.g. object
dictionary
 Timing: when will the conditions be right to initiate a common
project



Availability of resources and allocation of effort


Do established solutions exist in the experiments
Are they open to review or are they entrenched
Is there existing effort which would be better spent doing
something else
Availability, maturity of associated third party software

E.g. grid software
 Pragmatism and seizing opportunity. A workplan derived from a
grand design does not fit the reality of this project
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG: ‘blueprint’ of LCG
application architecture
21/39
 Mandate: define the architectural ‘blueprint’ for LCG applications:
 Define the main architectural domains (‘collaborating
frameworks’) of LHC experiments and identify their principal
components. (For example: Simulation is such an architectural
domain; Detector Description is a component which figures in
several domains.)
 Define the architectural relationships between these
‘frameworks’ and components, including Grid aspects, identify
the main requirements for their inter-communication, and
suggest possible first implementations. (The focus here is on
the architecture of how major ‘domains’ fit together, and not
detailed architecture within a domain.)
 Identify the high-level milestones for each domain and
provide a first estimate of the effort needed. (Here the
architecture within a domain could be considered.)
 Derive a set of requirements for the LCG
 Time-scale: started in June 02, draft report in July, final report
in August 02
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG status
22/39
 Identified and started eight
Requirement Technical Assessments (RTAGs)

in application software area







finished
finished
finished
started
started
started
in compute fabric area


Data persistency
Software support process and tools
Mathematical libraries
Detector Geometry & Materials descriptions
‘blueprint’ architecture of applications
Monte Carlo event generators
mass storage requirements
finished
in Grid technology and deployment area


Grid technology use cases
regional center category and services definition
finished
finished
Matthias Kasemann, FNAL and CERN, June 25, 2002
Software Process RTAG
23/39
 Mandate:
 Define a process for managing LCG software. Specific tasks to
include: Establish a structure for organizing software, for managing
versions and coherent subsets for distribution
 Identify external software packages to be supported
 Identify recommended tools for use within the project – to include
configuration and release management
 Estimate resources (person power) needed to run an LCG support
activity
 Guidance:
 Procedures and tools will be specified
 Will be used within project
 Can be packaged and supported for general use
 Will evolve with time
 The RTAG does not make any recommendations on how experiment
internal software should be developed and managed. However, if an
experiment specific program becomes an LCG product it should
adhere to the development practices proposed by this RTAG
Process RTAG –
Recommendations(1)
Matthias Kasemann, FNAL and CERN, June 25, 2002
 All LCG projects must adopt the same set of tools,
standards and procedures. The tools must be centrally
installed, maintained and supported.
 Adopt commonly used open-source or commercial
software where available. Try to avoid “do it yourself”
solutions in this area where we don’t have core
competency.
 Concerning commercial software, avoid commercial
software that has to be installed on individual’s
machines as this will cause well known problems of
license agreements and management in our widely
distributed environment. Commercial solutions for
web-portals or other centrally managed solutions would
be fine.
24/39
Process RTAG –
Recommendations(2)
Matthias Kasemann, FNAL and CERN, June 25, 2002
 ‘Release early, release often’ implies
25/39
major release 2-3 times per year
 Development release every 2-3 weeks
 Automated nightly builds, regression tests, benchmarks

 Test and quality assurance
 Support of external software

installation and build up of local expertise
 Effort needed for filling support roles
Librarian
 Release manager
 Toolsmith
 Quality assurance
 Technical writer

Matthias Kasemann, FNAL and CERN, June 25, 2002
Persistency RTAG
26/39
 Mandate:
 Write the product specification for the Persistency Framework for
Physics Applications at LHC
 Construct a component breakdown for the management of all types
of LHC data
 Identify the responsibilities of Experiment Frameworks, existing
products (such as ROOT) and as yet to be developed products
 Develop requirements/use cases to specify (at least) the metadata
/navigation component(s)
 Estimate resources (manpower) needed to prototype missing
components
 Guidance:
 The RTAG may decide to address all types of data, or may decide to
postpone some topics for other RTAGS, once the components have
been identified.
 The RTAG should develop a detailed description at least for the
event data management.
 Issues of schema evolution, dictionary construction and storage,
object and data models should be addressed.
Matthias Kasemann, FNAL and CERN, June 25, 2002
Persistency –
Near term recommendations
27/39
 to develop a common object streaming layer and
associated persistence infrastructure.
a common object streaming layer based on ROOT-IO and
several related components to support it,
 including a (currently lightweight) relational database
layer.
 Dictionary services are included in the near-term
project specification.


dictionary services may have additional clients
 This is first step towards a complete data management
environment, one with enormous potential for
commonality among the experiments.
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG: math library review
28/39
 Mandate: Review the current situation with math
libraries and make recommendations
Review the current situation of the usage of the various
math libraries in the experiments (including but not
limited to NagC++, GSL, CLHEP, ROOT)
 Identify and recommend which ones should be adopted,
which ones could be discontinued
 Suggest possible improvements to the existing ones
 Estimate resources needed for this activity

 Guidance – The result of the RTAG should allow to
establish a clear program of work to streamline the
status of math libraries and find the maximum
commonality between experiments, taking into account
cost, maintenance and projected evolution of the
experiment needs
Matthias Kasemann, FNAL and CERN, June 25, 2002
Math Library: Recommendations
29/39
 To design a support group
 to provide advice and information about the use of existing
libraries,
 to assure their continued availability,
 to identify where new functionality is needed, and
 to develop that functionality themselves or by coordinating
with other HEP-specific library developers.
 The goal would be to have close contact with the experiments
and provide expertise on mathematical methods, aiming at
common solutions,
 The experiments should maintain a data base of mathematical
libraries used in their software, and within each library, the
individual modules used.
 A detailed study should be undertaken to determine whether
there is any functionality needed by the experiments and
available in the NAG library which is not covered as well by a
free library such as GSL.
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG:
30/39
Detector Geometry
& Materials Description
 Write the product specification for detector geometry and
materials description services.
 Specify scope: e.g. Services to define, provide transient
access to, and store the geometry and materials descriptions
required by simulation, reconstruction, analysis, online and
event display applications, with the various descriptions using
the same information source
 Identify requirements including end-user needs such as ease
and naturalness of use of the description tools, readability
and robustness against errors e.g. provision for named
constants and derived quantities
 Explore commonality of persistence requirements with
conditions data management



Interaction of the DD with a conditions DB. In that context
versioning and ‘configuration management’ of the detector
description, coherence issues…
Identify where experiments have differing requirements and
examine how to address them within common tools
Address migration from current tools
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG:
Monte Carlo Event Generators
31/39
 Mandate: To best explore the common solutions
needed and how to engage the HEP community external
to the LCG it is proposed to study:
How to maintain a common code repository for the
generator code and related tools such as PDFLIB.
 The development or adaptation of generator-related
tools (e.g.HepMC) for LHC needs.
 How to provide support for the tuning, evaluation and
maintenance of the generators.
 The integration of the Monte Carlo generators into the
experimental software frameworks.
 The structure of possible forums to facilitate
interaction with the distributed external groups who
provide the Monte Carlo generators.

Possible Organisation of
activities
Matthias Kasemann, FNAL and CERN, June 25, 2002
Architect
Overall management, coordination, architecture, integration, support
Activity area
Project
leader
32/39
Activity area
Project
Project
WP WP WP
WP
Project
WP
WP WP
Activity area
Project
WP
WP
Example:
Activity area: Physics data management
Possible projects: Hybrid event store, Conditions DB, …
Work Packages: Component breakdown and work plan lead to Work
Package definitions. ~1-3 FTEs per WP
Global Workplan –
1st priority level
Matthias Kasemann, FNAL and CERN, June 25, 2002
1.
33/39
2.
3.
Establish process and infrastructure
 Nicely covered by software process RTAG
Address core areas essential to building a coherent
architecture
 Object dictionary – essential piece
 Persistency - strategic
 Interactive frameworks - also driven by assigning personnel
optimally
Address priority common project opportunities
 Driven by a combination of experiment need,
appropriateness to common project, and ‘the right moment’
(existing but not entrenched solutions in some
experiments)


Detector description and geometry model
Driven by need and available manpower

Simulation tools
Matthias Kasemann, FNAL and CERN, June 25, 2002
Global Workplan –
2nd priority level
34/39
 Build outward from the core top-priority components
Conditions database
 Statistical analysis
 Framework services, class libraries

 Address common project areas of less immediate
priority
Math libraries
 Physics packages (scope?)

 Extend and elaborate the support infrastructure

Software testing and distribution
Matthias Kasemann, FNAL and CERN, June 25, 2002
Global Workplan –
3rd priority level
35/39
 The core components have been addressed,
architecture and component breakdown laid out, work
begun. Grid products have had another year to develop
and mature. Now explicitly address physics
applications integration into the grid applications layer.
Distributed production systems. End-to-end grid
application/framework for production.
 Distributed analysis interfaces. Grid-aware analysis
environment and grid-enabled tools.

 Some common software components are now available.
Build on them.
Lightweight persistency, based on persistency
framework
 Release LCG benchmarking suite

Matthias Kasemann, FNAL and CERN, June 25, 2002
Global Workplan –
4th priority level
 Longer term items waiting for their moment
36/39

‘Hard’ ones, perhaps made easier by a growing common
software architecture


Address evolution of how we write software


Event processing framework
OO language usage
Longer term needs; capabilities emerging from R&D (more
speculative)

Advanced grid tools, online notebooks, …
Matthias Kasemann, FNAL and CERN, June 25, 2002
Candidate RTAGs (1)
37/39
Simulation tools
Non-physics activity
Detector description,
model
Description tools, geometry
model
Conditions database
If necessary after existing RTAG
Data dictionary
Key need for common service
Interactive frameworks
What do we want, have, need
Statistical analysis
Tools, interfaces, integration
Visualization
Tools, interfaces, integration
Physics packages
Important area but scope unclear
Framework services
If common framework is too
optimistic…
C++ class libraries
Standard foundation libraries
Matthias Kasemann, FNAL and CERN, June 25, 2002
Candidate RTAGs (2)
38/39
Event processing
framework
Hard, long term
Distributed analysis
Application layer over grid
Distributed production
Application layer over grid
Small scale
persistency
Simple persistency tools
Software testing
May be covered by process RTAG
Software distribution
From central ‘Program Library’ to
convenient broad distribution
OO language usage
C++, Java (..?) roles in the future
Benchmarking suite
Comprehensive suite for LCG
software
Online notebooks
Long term; low priority
Matthias Kasemann, FNAL and CERN, June 25, 2002
Common Solutions: Conclusions
39/39
 Common Solutions for LHC software are required for
success
Common solutions are agreed upon by experiments
 The requirements are set by the experiments
 The development is done jointly by the LCG project and
the LHC experiments
 All LCG software is centrally supported and maintained.

 What makes us believe that we succeed?
What is key to success?
The process in the LCG organization
 The collaboration between players
 Common technology
 Central resources, jointly steer-able by experiments and
management
 Participants have prototyping experience !!

Backup & Additional slides
40
Post-RTAG Participation of
Architects – Draft Proposal (1)
Matthias Kasemann, FNAL and CERN, June 25, 2002
 Monthly open meeting (expanded weekly meeting)
41/39
Accumulated issues to be taken up with architects
 Architects in attendance; coordinators invited

 Information has gone out beforehand, so architects
are ‘primed’
 Meeting is informational, and decision-making (for the
easier decisions)

An issue is either


Resolved (the easy ones)
Flagged for addressing in the ‘architects committee’
Matthias Kasemann, FNAL and CERN, June 25, 2002
Post-RTAG Participation of
Architects – Draft Proposal (2)
42/39
 Architects committee:
 Members: experiment architects + applications manager
(chair)
 Invited: computing coordinators, LCG project manager and
CTO
 Others invited at discretion of members

e.g. project leader of project at issue

Such decisions can be accepted or challenged
 Meets shortly after the open meeting (also bi-weekly?)
 Decides the difficult issues
 Most of the time, committee will converge on a decision
 If not, try harder
 If still not, applications manager takes decision
 Challenged decisions go to full PEB, then if necessary to SC2
 PEB role of raising issues to be taken up by SC2
 We all abide happily by an SC2 decision
 Committee meetings also cover general current issues and
exchange of views
 Committee decisions, actions documented in public minutes
Distributed Character of
Components (1)
Matthias Kasemann, FNAL and CERN, June 25, 2002

43/39
Persistency framework
Naming based on logical filenames
 Replica catalog and management
 Cost estimators; policy modules


Conditions database


Interactive frameworks


Grid-aware environment; ‘transparent’ access to gridenabled tools and services
Statistical analysis, visualization


Inherently distributed (but configurable for local use)
Integral parts of distributed analysis environment
Framework services

Grid-aware message and error reporting, error
handling, grid-related framework services
Distributed Character of
Components (2)
Matthias Kasemann, FNAL and CERN, June 25, 2002
 Event processing framework
44/39

Cf. framework services, persistency framework,
interactive frameworks
 Distributed analysis
 Distributed production
 Software distribution

Should use the grid
 OO language usage

Distributed computing considerations
 Online notebook

Grid-aware tool
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Simulation tools
45/39
 Geant4 is establishing a HEP physics requirements
body within the collaboration, accepted by SC2 as a
mechanism for addressing G4 physics performance
issues
 However, there are important simulation needs to
which LCG resources could be applied in the near term.
 By the design of LCG, this requires SC2 delivering
requirements to PEB
 John Apostolakis has recently assembled G4 requests
and requirements from the LHC collaborations
 Proposal: Use these requirements as the groundwork
for a quick 1-month RTAG to guide near term
simulation activity in the project, leaving the
addressing of physics performance requirements to
the separate process within Geant4
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Simulation tools (2)
46/39
 Some possible activity areas in simulation, from the
Geant4 requests/requirements received from the
experiments, which would be input to the RTAG:
Error propagation tool for reconstruction (‘GEANE’)
 Assembly and documentation of standard physics lists
 Python interface
 Documentation, tutorials, communication
 Geant4 CVS server access issues

 The RTAG could also address FLUKA support
Requested by ALICE as an immediate priority
 Strong interest expressed by other experiments as well

Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Detector geometry & materials
description and modeling services
47/39
 Write the product specification for detector geometry and
materials description and modeling services
 Specify scope: eg. Services to define, provide transient
access to, and store the geometry and materials
descriptions required by simulation, reconstruction,
analysis, online and event display applications, with the
various descriptions using the same information source
 Identify requirements including end-user needs such as
ease and naturalness of use of the description tools,
readibility and robustness against errors e.g. provision for
named constants and derived quantities
 Explore commonality of persistence requirements with
conditions data management
 Identify where experiments have differing requirements
and examine how to address them within common tools
 Address migration from current tools
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Conditions database
48/39
 Will depend on persistency RTAG outcome
 Refine the requirements and product specification of a
conditions database serving the needs of the LHC
experiments, using the existing requirements and
products as a reference point. Give due consideration
to effective distributed/remote usage.
 Identify the extent to which the persistency
framework (hybrid store) can be directly used at the
lower levels of a conditions database implementation.
 Identify the component(s) and interfaces atop a
common persistency foundation that complete the
conditions database
RTAG?: Data dictionary
service
Matthias Kasemann, FNAL and CERN, June 25, 2002

49/39



Can the experiments converge on common data definition and
dictionary tools in the near term?
Even if the answer is no, it should be possible to establish a
standard dictionary service (generic API) by which common
tools can interact, while leaving free to the experiments how
their class models are defined and implemented
Develop a product specification for a generic high-level data
dictionary service able to accommodate distinct data definition
and dictionary tools and present a common, generic interface to
the dictionary
Review the current data definition and dictionary approaches
and seek to expand commonality among the experiments. Write
the product specifications for common (even if N<4)
components.
RTAG?: Interactive
frameworks
Matthias Kasemann, FNAL and CERN, June 25, 2002

50/39




Frameworks providing interactivity for various
environments including physics analysis and event
processing control (simulation and reconstruction) are
critical. They serve end users directly and must match
end user requirements extremely well. They can be a
powerful and flexible ‘glue’ in a modular environment,
providing interconnectivity between widely distinct
components and making the ‘whole’ offered by such an
environment much greater than the sum of its parts.
Develop the requirements for an interactive framework
common across the various application environments
Relate the requirements to existing tools and approaches
(e.g. ROOT/CINT, Python-based tools)
Write a product specification, with specific
recommendations on tools and technologies to employ
Address both command line and GUI interactivity
RTAG?: Statistical analysis
interfaces & tools
Matthias Kasemann, FNAL and CERN, June 25, 2002

51/39
Address requirements on analysis tools
What data analysis services and tools are required
 What is and is not provided by existing tools


Address what existing tools should be supported and
what further development is needed


Including long term maintenance issues
Address role of abstract interfaces to statistical
analysis services
Are they to be used?
 If so, what tools should be interfaced to a common
abstract interface to meet LHC needs (and how, when,
etc.)


Address requirements and approaches to persistency
and data interchange
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Detector and event
visualization
52/39
 Examine the range of tools available and identify those
which should be developed as common components
within the LCG Applications architecture
 Address requirements, recommendations and
needed/desired implementations in such areas as
existing and planned standard interfaces and their
applicability
 GUI integration
 Interactivity requirements (picking)
 Interface to visualizing objects (eg. Draw() method)
 Use of standard 3D graphics libraries

 Very dependent on other RTAG outcomes
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Physics packages
53/39
 Needs and requirements in event generators and their
interfaces & persistency, particle property services, …
 Scope of the LCG in this area needs to be made
clearer before a well defined candidate RTAG can be
developed
RTAG?: Framework services
Matthias Kasemann, FNAL and CERN, June 25, 2002

54/39
While converging on a common event processing
framework among the LHC experiments may be
impractical at least on the near term, this does not
preclude adopting common approaches and tools for
Framework services



Examples: message handling and error reporting;
execution monitoring and state management;
exception handling and recovery; job state persistence
and recording of history information; dynamic
component loading; interface definition, versioning,
etc.
Seek to identify framework services and tools which
can be developed in common, possibly starting from
existing products.
Develop requirements on their functionality and
interfaces.
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: C++ class libraries
55/39
 Address needs and requirements in standard C++ class
libraries, with recommendations on specific tools
 Provide recommendations on the application and
evolution of community libraries such as ROOT, CLHEP,
HepUtilities, …
 Survey third party libraries and provide
recommendations on which should be adopted and what
should be used from them
 Merge with Framework Services candidate RTAG?
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Event processing
framework
56/39
 There is no consensus to pursue a common event
processing framework in the near term. There is
perhaps more agreement that this should be pursued in
the long term (but there’s no consensus on a likely
candidate for a common framework in the long term)
 This looks at best to be a long term RTAG
 Two experiments do use a common event processing
framework kernel (Gaudi)
 Many difficult issues in growing N past 2, whether
with Gaudi, AliRoot, COBRA or something else!
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Interfaces to
distributed analysis
57/39
 Develop requirements on end-user interfaces to
distributed analysis, layered over grid middleware
services, and write a product specification
Grid portals, but not only; e.g. PROOF and Jas fall into
this category
 A grid portal for analysis is presumably an evolution of
tools like these

 Focus on analysis interface; address the distinct
requirements of production separately

Production interface should probably be addressed first,
as it is simpler and will probably have components usable
as parts of the analysis interface
RTAG?: Distributed
production systems
Matthias Kasemann, FNAL and CERN, June 25, 2002

58/39



Distributed production systems will have much common ground
at the grid middleware level. How much can be done in common
at the higher level of end-to-end distributed production
applications layered over the grid middleware?
 Recognizing that the grid projects are active at this level
too, and coordination is needed
Survey existing and planned production components and end-toend systems at the application level (AliEn, MOP, etc.) and
identify tools and approaches to develop in common
Write product specifications for common components, and/or
explicitly identify specific tools to be adapted and developed as
common components
Include end user (production operations) interface
 Grid portal for production
RTAG?: Small-scale
persistency & databases
Matthias Kasemann, FNAL and CERN, June 25, 2002

59/39


If not covered by the existing persistency RTAG, and if there
is agreement this is needed…
Write the product specification for a simple, self-contained,
low-overhead object persistency service for small-scale
persistency in C++ applications
 Marshal objects to a byte stream which may be stored on a
file, in an RDBMS record, etc.
 In implementation, very likely a simplified derivative of the
object streamer of the hybrid store
 For small scale persistence applications, e.g. saving state,
saving configuration information
Examine the utility of and requirements on a simple, standard,
easily installed and managed database service complementing
the persistency service for small scale applications
 MySQL, PostgreSQL etc are casually adopted for simple
applications with increasing frequency. Is it possible and
worthwhile to converge on a common database service
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Software testing
tools & services
60/39
 How much commonality can be achieved in the
infrastructure and tools used

Memory checking, unit tests, regression tests, validation
tests, performance tests
 A large part of this has been covered by the process
RTAG
Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Software distribution
61/39
 May or may not be adequately addressed in the
process RTAG
 Requirements for a central distribution point at CERN

A ‘CERN LHC Program Library Office’
 Requirements on software distribution taking into
account all tiers
 Survey and recommend on the various approaches,
their utility, complementarity
Tarballs (DAR)
 RPMs and other standard open software tools
 Role of AFS, asis
 Higher level automated distribution tools (pacman)

Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Evolution of OO
language usage
62/39
 Long-term evolution of C++
 Role for other language(s), e.g. Java?
Near, medium and (to the extent possible) long term
application of other languages among LHC experiments
 Implications for tools and support requirements

 Identify any requirements arising
Applications, services to be developed in common
 Third party tools to be integrated and supported
 Compilers and other infrastructure to be supported
 Libraries required

RTAG?: LCG Benchmarking
suite
Matthias Kasemann, FNAL and CERN, June 25, 2002
 Below threshold for an RTAG?
63/39

Every LCG application should come with a benchmarking
suite, and should be made available and readily usable as
part of a comprehensive benchmarking suite
 Develop requirements for a comprehensive
benchmarking suite of LCG applications for use in
performance evaluation, testing, platform validation
and performance measurement, etc.
Tools which should be represented
 Tests which should be included
 Packaging and distribution requirements

Matthias Kasemann, FNAL and CERN, June 25, 2002
RTAG?: Online notebooks and other
remote control / collaborative tools
64/39
 Identify near term and long term needs and
requirements common across the experiments
 Survey existing, planned tools and approaches
 Develop recommendations for common
development/adaptation and support of tools for LHC