Transcript LCG PEB Applications Area Status Report

Blueprint RTAG Status
Torre Wenaus, BNL/CERN
SC2 Meeting
July 5, 2002
Blueprint RTAG Status


Current RTAG page:
 Linked from the applications page
Activity so far:
 Meetings Wed June 12, Fri June 14


Between these meetings: a long email dialogue on architecture
and design


‘Opening statement’ type talks and discussions
Particularly on the role and architecture of ROOT, following on
from a proposal made by Rene on June 14
All day meeting Wed July 3




Response/reaction to Rene’s proposal and the email dialogue
Some resolved issues and decisions
Domain decomposition
On to developing the architectural blueprint
SC2 meeting, July 5 2002
Slide 2
Torre Wenaus, BNL/CERN
The main software areas
Rene Brun
GRID
DAQ
middleware
Online
ETC...
Event Models
Folders
Event
RDBMS
Generators
Object
Object
persistencyv
persistency
run/file
catalogs
Event Display
Dectector
System
Histograming
Simulation
services
2-d, 3-d
Fitting
graphics
Ntuple
Detector
analysis
Math Libs
Statistics
GUI
Interpreters
SC2 meeting, July 5 2002
Slide 3
Toolkits
Torre Wenaus, BNL/CERN
Geometry
Framework with Object bus
Rene Brun
User
Applications
Higher level
Experiment
framework
User
Applicat
ions
Higher
level
Higher
level
framewor
k
services
framewor
k
services
framework services
Higher
Higher
level
level
framewor
k
services
framewor
k
services
Object bus: Object dictionary
Data Interface (I/O): Functional
Interface
SC2 meeting, July 5 2002
Slide 4
Torre Wenaus, BNL/CERN
Python as a Software Bus
Pere Mato


Python is an object-oriented scripting language commonly used in a
wide variety of domains
But, it could also be seen as framework where you can plug easily
“extension modules” in binary form implemented using other languages.
 Very easy and natural to interface to C++ classes (C++ API)
 Python should only be the “glue” between modules developed in
C++ or other languages
 The interface (API) for Python extension modules is quite simple
and at the same time flexible
SC2 meeting, July 5 2002
Slide 5
Torre Wenaus, BNL/CERN
Python as a Software Bus
Pere Mato
Very rich set
specialized
generic
modules
LHC modules
Gateways to
other frameworks
EDG API
PVSS
XML
Several GUI
toolkits
Database
GUI
GUI
Python
JPE
rootpython
gaudipython
Java
Classes
Root
Classes
Gaudi
Framework
SC2 meeting, July 5 2002
Slide 6
math
math
Very rich set
of Python
standard
modules
Torre Wenaus, BNL/CERN
shell
Architecture: Devil is in the Details
Vincenzo Innocente

Build independent components: Avoid
 Dependencies among components at the same level
 Gratuitous and exaggerated re-use
One hammer does not fit all screws
global states (even cout)
 Exposure of internal relationships (a->b()->c(i)->d(“b”))
 assumptions on higher level behavior (lent pointers)
 Interfaces that force your environment on user code
Balance inheritance (white box) vs composition (black box)
Distinguish Framework API, Client API and User API
These are Architectural issues NOT coding guidelines



I do not mind of “#define int float” in your .cc, I mind if in a .h
SC2 meeting, July 5 2002
Slide 7
Torre Wenaus, BNL/CERN
Toward a Project Praxis
Vincenzo Innocente

Define the global software model
 Granularity, role and nature of “Modules”
Physical vs logical modules
(yesterday at CMS plenary M.Livny concluded asking for staticly linked, checkpointable executables…)
Reuse model of sub-components
 Which “glues” have to be used, where and how
Define THE set of basic components
Agree on Metrics to measure modularity
 Not only Frameworks, also applications based on them



SC2 meeting, July 5 2002
Slide 8
Torre Wenaus, BNL/CERN
Rene’s Proposal
The existing set of ROOT libs is
the starting core of the LCG
software.
Because the system is already
widely distributed and used, it
guarantees the initial acceptance
to a wide community.
We invite architects and key
developers to review the current
organisation of libraries and to
propose an evolution if it proves
necessary, keeping in mind that
this may affect thousands of
existing users.
SC2 meeting, July 5 2002
Slide 9
Torre Wenaus, BNL/CERN
LCG Software Architecture and ROOT




Architectural principles that CMS, LHCb, ATLAS want to follow
were developed in presentations by Pere and Vincenzo and in a long
email dialogue
Exchanges indicated sharply the different views on design between
the ROOT/ALICE team and CMS/LHCb/ATLAS
A clear conclusion, agreed in Wed meeting…
 It is not going to work for ROOT to be taken as the starting
core of LCG software
 The design differences are too great, and the ROOT team is not
going to redesign ROOT
What we know is going to happen is that ROOT will be used heavily
by the LCG software and all four experiments
 And has been taken directly as the core foundation for one
SC2 meeting, July 5 2002
Slide 10
Torre Wenaus, BNL/CERN
How will ROOT be used?


While design discussions should continue, we cannot rely on
resolving them in order to progress…
We accepted that
 We are not going to converge anytime soon on a common
approach to architecture and design
 What we must take up and resolve is working out an
architecturally acceptable way to make use of a big grey (not
black) ROOT box
 With accommodation on both sides




Changes to ROOT library organization?
More substantive changes to improve factorization?
Dealing constructively with ‘linking in the kitchen sink’ issues
Agreed metrics meaningful for usability, maintability, modularity
etc.
SC2 meeting, July 5 2002
Slide 11
Torre Wenaus, BNL/CERN
How will ROOT be used?



LCG software developed as a ROOT user will draw on a great ROOT
strength: users are listened to very carefully!
 Much more carefully than software designers proposing major
design changes!!
 The ROOT team has been very responsive to needs for new and
extended functionality coming from the persistency effort
Drawing on ROOT in a user-provider relationship matches much
better the reality of the ROOT development model of a very small
number of ‘empowered’ developers
 The ROOT development team is small and they like it that way
While ROOT will be used at the core of much LCG software for the
foreseeable future, we agree there needs to be a line with ROOT
proper on one side and ‘LCG software’ on the other.
SC2 meeting, July 5 2002
Slide 12
Torre Wenaus, BNL/CERN
LCG Software Architecture

With the ‘ROOT relationship’ resolved to be not architecture/design
wars to be (endlessly) fought, but rather designing how ROOT will be
used in a distinct LCG architecture, we can (and we began to) move
on to developing the LCG architectural blueprint
SC2 meeting, July 5 2002
Slide 13
Torre Wenaus, BNL/CERN
Views I expressed, to no vocal objection






I think that within the LCG software architecture ‘bare ROOT’ should
be an integral, trivially accessible part of the architecture
 e.g., most obviously, affording the interactive user the ability to
easily move between a Python prompt (see coming slide) and a
ROOT prompt with access to their object model from ROOT;
possible now given the foreign class support in ROOT
As Rene often says: Users vote with their feet
The design and evolution of LCG software should be well attuned to
the user, as is ROOT
LCG software we develop that does not use ‘bare ROOT’, or ROOT
at all, will live or die on its merits
We should build it so that it lives on its merits, not on life support
through architecturally walling off ROOT in some way
Plenty of people have bet against ROOT in the past. They have all
been wrong. LCG software should not make this mistake
SC2 meeting, July 5 2002
Slide 14
Torre Wenaus, BNL/CERN
Interactivity and software buses

Points of agreement
 A common object dictionary will be a key component at the
foundation of the architecture
 A Python-based interactive environment and ‘software bus’ will
be part of the architecture
 ROOT and the ROOTCINT interpreter will be trivially accessible
from the Python environment and vice versa
 The architecture will provide for access to data objects from
programmatic and interactive environments throughout the
system
SC2 meeting, July 5 2002
Slide 15
Torre Wenaus, BNL/CERN
Python in the architecture

Rene requested expression of his views…
 Python should not be the primary interactive interface.


Personally I agree; I think that ROOTCINT and Python should essentially
be on a level playing field
Will lead to confusion and waste of time by users
Actually I think users will insist on easy availability of their preferred
interactive environment, as I expressed on the ‘views’ slide a few slides
ago

Python should be available, to take advantage of services for which a
Python interface is already available
 In the same way, we must encourage cross-language communication
Rene in favour of a public presentation of a complete and coherent framework
 I think the outcome of this RTAG should be presented clearly in a public
form, eg. in an applications area meeting (with the anticipation of larger
than average attendance!)


SC2 meeting, July 5 2002
Slide 16
Torre Wenaus, BNL/CERN
Notes on Domain Decomposition
- NB user and software provider
perspectives on domain
decomposition will be different
- basic layer - software bus / object
dictionary
- persistency/data management
- object persistency
- relational cataloging
- event-specific data management
- conditions-specific data management
- event processing framework
- event models
- event generation **defer
- detector simulation **defer
- geometry and materials
- persistent representation
- transient geometry model
- reconstruction
SC2 meeting, July 5 2002
Slide 17
- reconstruction
- scripting (Python) interpreter tool as
software bus
- GUI **defer
- NB requirements coming from support
for picking
- ability to put any kind of object into a
list of primitives
- 2D & 3D graphics **defer
- analysis tools **defer
- ntuple analysis
- math libraries, statistical analysis
- histogramming, fitting
- grid middleware **defer
- meta-services (Pere will elucidate; cf.
Gaudi)
- utility and foundation libraries
- system services. OS interaction
- packaging and distribution
Torre Wenaus, BNL/CERN
Blueprint RTAG plans






Another series of 3 meetings next week
 Into the specific details of techniques and patterns via proposals
presented as concrete examples
 Refining the domain decomposition; walking through it and identifying
candidate implementations; interdependencies
The following week:
 Begin assembling the report
 Too many absences for meetings
Then more meetings, and on and on…
Will invite some speakers (Tony Johnson,
There is some skepticism within the RTAG that we will make early Sep (we
will not make early Aug), but this is our objective
 We recognize the importance of getting this done in good time
 It has to be done right
I think the RTAG is off to a good start and has made some important,
clarifying decisions.
SC2 meeting, July 5 2002
Slide 18
Torre Wenaus, BNL/CERN