Transcript Hybrid Event Store Integration with Athena/StoreGate ATLAS Software Week

Hybrid Event Store Integration
with Athena/StoreGate
ATLAS Software Week
Event Data Model and Detector
Description Session
David Adams
BNL
March 5, 2002
David Adams
ATLAS
Contents
• Introduction
• Athena/HES components
–HES
–StoreView
–StoreSvc
–Proxy
–DataHandle
–StoreGateSvc
–DataLink
–AHesView
• Tasks and schedule
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
2
Athena/HES components
Figure
• The following page shows selected components
of Athena and HES and their physical
dependencies.
• The Athena/StoreGate components
(DataLink,…) shaded in blue are new or will
have to be modified to integrate HES into
Athena/StoreGate.
– These are discussed in the following sections.
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
3
Athena/HES components (cont)
XAlgorithm
AHesView
DataLink
HesView
StoreGateSvc
DataHandle
Proxy
StoreSvc
ConversionSvc
Algorithm
StoreView
Svc
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
4
HES
Definition
• Define a subsystem called HES which has no
dependencies on StoreGate, Athena or Gaudi.
• HesView is the top of this subsystem.
• This subsystem will be discussed in the DB
meeting on Thursday.
• Links to this talk and that one can be found on
the hybrid web page
– http://www.usatlas.bnl.gov/~dladams/hybrid
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
5
StoreView
Purpose
• StoreView is a new abstract interface which
allows Athena to interact with different
persistent stores in a generic manner.
• Each persistent storage mechanism to be used
in Athena will provide an implementation of
this interface.
– HES
– ZEBRA
– Objectivity?
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
6
StoreView (cont)
Interface
• Select the next event ID.
• Read an input event
– Identify the included type-keys and create proxies in
StoreGate
– Locate the data (on demand from proxy) for any of
these EDO’s
• Write an output event
– Take ownership of selected new EDO’s
> (add EDO ID to proxy)
– Write EDO’s to selected output streams
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
7
StoreView (cont)
Configuring a job
• The choice of store view (or views) and their
configuration is made at run time using Gaudi
JobOptions
– StoreView is a Gaudi service
– No physical dependence on concrete
implementation except that library must be provided
at run time
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
8
StoreSvc
Purpose
• StoreSvc provides the interface between the
store views and the rest of Athena/StoreGate
• StoreSvc holds one or more store views
– Requests are fulfilled by forwarding to a selected
view or looping over all views.
– Each view is assigned an ID called store ID.
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
9
StoreSvc (cont)
Interface for reading
• Assign the next event ID
– Pass request to a designated store
– Or assign in some other manner (e.g. user input)
• Define input event
– Loop over store views
• Fetch data
– Given a proxy for an EDO that has not be located,
> identify the appropriate view and
> ask the view to locate the EDO
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
10
StoreSvc (cont)
Interface for writing
• Stream selection
– Run selection algorithms and notify views which
streams should be written.
• EDO ownership
– Give each view the opportunity to take ownership of
EDO’s in StoreGate.
• Writing data
– Give each view the option to to write EDO’s from
StoreGate (or from that view).
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
11
StoreSvc (cont)
Configuring a job
• Configure in job options
– StoreSvc is a Gaudi service
– No physical dependency on Athena/StoreGate
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
12
Proxy
Purpose
• The transient data store (StoreGateSvc) “holds”
EDO’s by proxy.
• This allows us to
– Defer output conversion until all data is produced
and selections are made
– Defer input conversion until data is accessed
HES environment
• It is sensible to also defer location on input
• Some data may not be accessible
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
13
Proxy (cont)
Proxies will be modified for HES
• From persistent store, there will be four states:
– No data; proxy has type, key, store ID and EDO ID
– Above plus IOpaqueAddress (EDO located)
– Above plus transient address (EDO converted)
– Marked inaccessible (EDO cannot be located)
• For proxies from transient store, it will be
possible to assign ownership (EDO ID).
• Proxy uses StoreSvc to locate EDO when
desired.
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
14
DataHandle
Current usage
• A data handle is returned when a user asks
StoreGate for an EDO.
• Conversion may be deferred.
Changes (implied by change in proxy)
• Location may also be deferred.
• DataHandle may be invalid because the EDO
cannot be located.
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
15
StoreGateSvc
Purpose and current implementation
• StoreGateSvc is the transient data store.
• Single instance holds data (by proxy) for
current event.
– Pileup adds more
• Given a type and key, StoreGate returns a
DataHandle for the corresponding EDO
– Any type-key may not be duplicated in the store
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
16
StoreGateSvc (cont)
Changes
• One instance will hold proxies for visible data
– Directly accessed from algorithms
• Other(s) will hold proxies for data which is
accessible (by reference) but not visible.
• Request for data from DataLink will include
EDO ID in addition to type-key.
– Only EDO with matching ID is returned.
– Someone has to search the other transient stores if
match is not found in the visible store
> Visible store knows the reference stores?
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
17
DataLink
Purpose and current implementation
• DataLink provides a persistent mechanism to
reference objects in other EDO’s.
• Persistent data includes:
– EDO specifier and
– index specifying the location of object in EDO.
• At present, EDO specifier is the type and key
and an event ID implied by context.
• Any StoreGate EDO with matching type and
key can serve as the reference EDO
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
18
DataLink (cont)
Changes:
• In the new model, we explicitly recognize that
persistent store may hold multiple EDO’s with
the same event ID, type and key.
• Persistent data for the DataLink will also
include the ID of the referenced EDO.
• May also need to add (transient) store ID to
resolve EDO ID ambiguities between different
store types.
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
19
AHesView
Implementation of StoreView for HES
• Depends on both Athena/StoreGate and the
HES core software.
– Creates proxies
– Puts proxies in StoreGate
– Calls converters to write data
• See talk at DB session for information on the
HES core.
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
20
Tasks and schedule
Requirements
• Sufficient functionality to use HES as the
primary storage mechanism for DC1-2.
• We would like a first implementation by June 1.
• Estimate the Athena/HES piece will require
7 FTE-weeks for this implementation.
– 2X contingency
• We need to adjust priorities to meet this
schedule.
David Adams
ATLAS
HES in Athena
SW week – EDM session
March 5, 2002
21