Transcript CARF Functional Prototype Software framework, services and persistency

CARF
Functional Prototype
Software framework, services and persistency
in high level trigger, reconstruction and analysis
Vincenzo Innocente
CERN/EP/CMC
CARF Functional Prototype
Vincenzo Innocente, CERN/EP
CMS Experiment-Data Analysis
Quasi-online
Reconstruction
Environmental data
Detector Control
Online Monitoring
store
Request part
of event
Event Filter
Request part
of event
Object Formatter
Store rec-Obj
Request part
of event
store
Persistent Object Store Manager
Object Database Management System
store
Simulation
G3or G4
CARF Functional Prototype
Vincenzo Innocente, CERN/EP
Store rec-Obj
and calibrations
Data Quality
Calibrations
Group Analysis
Request part of event
User Analysis
on demand
2
CARF
CMS Analysis & Reconstruction Framework
Physics modules
Specific
Framework
Event
Filter
Generic
Application
Framework
Reconstruction
Algorithms
Calibration
Objects
Physics
Analysis
Configuration
Objects
Data
Monitoring
Event Objects
CMS adapters and extensions
Utility Toolkit
ODBMS
CARF Functional Prototype
Vincenzo Innocente, CERN/EP
Geant3/4
CLHEP
Paw
Replacement
C++
standard library
Extension toolkit
3
CMS Software R&D
95-96: RD41 --- OO Detector Reconstruction

Detector model, Local hit cache, Pattern recognition
95-97: RD45 --- OO Event Model (persistent)

Event structure, Raw data, Reconstructed objects
95-97: RD45 --- Calibration Database

Time dependent data, Versioning, Experience with Objectivity/DB
96-98: Program Architecture

Implicit invocation, Event dispatching, Reconstruction on demand
97-98: Test-Beam (H2, X5)

OO DAQ, Online filtering, ODB population, Interactive analysis
99-00: Test-Beam & ORCA Production
Event-Collections, Concurrent jobs, Multi-threading, RT dynamic loading
 Production management, MSS interface, Data import-export
 User databases, User event-collections
 Event visualization, Interactive analysis

CARF Functional Prototype
Vincenzo Innocente, CERN/EP
4
CARF Components
Basic Utility ToolKit
Once was CERNLIB
Today a set of classes that extends the standard C and C++ libraries
Basic Mechanisms
Sets of collaborating classes, implementation of the most popular patterns
Base of Object Oriented Programming
Basic Persistency Support
Objectivity Wrappers
 Generic Persistent Classes

User Interface
Links keywords to objects
Inputs from ascii file, database objects, interactive shell
CARF Functional Prototype
Vincenzo Innocente, CERN/EP
5
CARF Components
Persistent Object Model
Event Catalog, Configuration and MetaData
 Simulated Event
 RawEvent (SimHits and Digis)
 Reconstructed Objects

CARF Functional Prototype
Vincenzo Innocente, CERN/EP
6
CARF Components
Framework
Package Initializer
 Event Reader
 “Configuration” management
 DataBase Populator

API (for developers)
Simulated-Event Source
 SimHit Formatter
 SimHit Loader
 Reconstruction Detector
 Reconstruction Unit

API (for users)
Selectors
 Event Observers (Analyzers, Filters, etc)
 RecObj Collection

CARF Functional Prototype
Vincenzo Innocente, CERN/EP
7
Functionalities
Basic Application
Iterates over an input Event Collection
Selects events based on “MetaData”
Today based just on event- and run-number and data availability
 Tomorrow will use user-defined Annotations

Dispatch Events to Observers

Standard and/or user module to
Analyze
and classify the event
produce or update reconstructed event objects
anti-select events based on reconstruction information
Force production of persistent objects if required
Produce an output Event Collection
Any combination of shallow and deep copy of any part of the event is in
principle allowed
 In practice few predefined options

CARF Functional Prototype
Vincenzo Innocente, CERN/EP
8
Functionalities
Simulation Application
Back-end to Detector Simulation to provide persistency support
for simulated tracks and hits
Standard CMSIM : converts from FZ to CARF/ORCA
 new G4 prototype: converts from G4 to CARF/ORCA

CARF Functional Prototype
Vincenzo Innocente, CERN/EP
9
Functionalities
SimReader Application
Reads output of Simulation and perform digitization
Reads pile-up pseudo-randomly from a large event collection
 Can perform reconstruction if required
 Force digitization for all registered “detectors”

CARF Functional Prototype
Vincenzo Innocente, CERN/EP
10
Functionalities
Reconstruction Application
Read Output of a previous Digitization or Reconstruction
will re-digitize and re-reconstruct any detector declared “obsolete”
 will digitize and reconstruct any newly registered detector and
reconstruction-unit

CARF Functional Prototype
Vincenzo Innocente, CERN/EP
11
Quality
Performance
Framework performance not yet critical
 Some problems in opportunistic MetaData update from several
concurrent jobs

Reliability
All components (CARF, Objy servers) seem to be reliable for single-user
applications (some problems from HPSS)
 Still some problems in massive production (above 150 concurrent jobs)

Code quality still at prototype level:

Major coding guide-lines were not correctly identified or overlooked
too
much use of inline code
too many compilation- and run-time dependencies
user API too complex

Requirements and priorities should be redefined
CARF Functional Prototype
Vincenzo Innocente, CERN/EP
12
Impact of Strategic choices
CMS has choose a “non-traditional” approach in several
strategic areas:
Object-Oriented Programming in C++
 Object Database Management System
 Plug-in
 Implicit invocation
 Multi-thread
 Opportunistic Database access

All these technologies have proven to work well and fulfill the
requirements for the framework and in several prototype
We should now evaluate their impact on non-core software and
on users’ code.
CARF Functional Prototype
Vincenzo Innocente, CERN/EP
13
Short Term Plan
Basic ToolKit
Integrate with IGUANA, FAMOS and OSCAR equivalent
 Integrate with ANAPHE?

Basic Persistency
Extend to other projects
 integrate with HEPODBMS?

Persistent Data Model
Extend to other projects
 Prototype realistic RecObjs
 Investigate Generic User-data such as

Annotations
tuples

Answer to the question:
What
“new” kind of objects should be stored during physics analysis?
CARF Functional Prototype
Vincenzo Innocente, CERN/EP
14
Outlook
Most of the functionalities identified in the CTP and in later
more detailed requests (HLT for instance) have been
implemented
Performance and Reliability not yet at production level
Code quality still at prototype level:

Requirements and priorities should be redefined in this area
Non-traditional technologies seem working well for coresoftware

Impact on non-core software should be evaluated.
CARF Functional Prototype
Vincenzo Innocente, CERN/EP
15