Transcript ADA - nw.ru
www.eu-egee.org
INFSO-RI-508833 Enabling Grids for E-sciencE
ATLAS Distributed Analysis
A. Zalite / PNPI
Enabling Grids for E-sciencE
• • • • • • •
Why?
Goal ADA Model First steps Demo example More examples Conclusion
INFSO-RI-508833
Overview
Why?
Enabling Grids for E-sciencE
•
Huge amount of data
– Atlas experiment is expected to record several petabytes of data per year – Atlas offline system will produce similar amount of data (ESD, AOD, …) •
Globally-distributed members of Atlas collaboration
– Over 1000 physicists from all over the world will take part in data analysis •
The data have to be available to all members of the collaboration
INFSO-RI-508833
Goal
Enabling Grids for E-sciencE
• • • •
Provide to globally distributed users
– Access to globally distributed data – Tools to perform globally distributed processing on this data
Easy to use and access from analysis environment
– Flexible to adopt to environment
Enable effective use of all ATLAS computing resources Trace information about processing with any data
– Where did this data (event or analysis) come from?
INFSO-RI-508833
ADA Model
Enabling Grids for E-sciencE
• • •
Components: Data described by a Datasets (collection of data)
– Location of the data (e.g. files) – Content (e.g. list of event ID’s and the type of the data for each event)
Transformation describes an operation that can act on a dataset to produce a new dataset
– Application scripts used to run job to build task or process data – Task carries user parameters or code (E.g. atlas release, job options, and/or algorithm code)
Job is an instance of the transformation acting on a dataset
INFSO-RI-508833
ADA Model
Enabling Grids for E-sciencE
• •
Many ATLAS-specific transformations have been defined
– – Atlasopt: user provides ATLAS release and job options Aodhisto: atlasopt plus code to build in UserAnalysis package – – Atlasdev: atlasopt plus local development directory Atlasdev-src: same as atlasdev except development area is tarred up and will be rebuilt if platform changes
All these transformations run Athena
INFSO-RI-508833
INFSO-RI-508833
ADA Model
Enabling Grids for E-sciencE
Dataset 1 Dataset 2 Result 1 9. fill 7. create Event data, summary data, tuples, ..
ROOT, GANGA, ...
Athena, dialpaw, ROOT, ...
Dataset 4. select 6. split 10. gather Analysis Service User analysis framework 2. select 1. create or locate 5. submit(app,tsk,ds) 3. create or select Application Task Result 2 8. run(app,tsk,ds1) 8. run(app,tsk,ds2) 9. fill Job 1 Job 2 Code Params DIAL components September 20, 2004
Transformation
ADA Model
Enabling Grids for E-sciencE
• • •
This view enables distributed processing: Split input dataset
– Along event, file, or sub-dataset boundaries
Create separate sub-job for each sub-dataset Implies post-processing stage to merge results (output datasets)
• • •
Users carry out processing by Defining a job
– Application, task and dataset
Submitting this definition to a scheduler
– Typically an analysis service
Polling for status
– Job state (and sub-job states) – Result dataset INFSO-RI-508833
ADA Model
Enabling Grids for E-sciencE
• • • •
On receiving a job request, the scheduler Builds the task (or locates an existing build) Split the dataset into sub-datasets Create and submits a sub-job for each sub-dataset Merge the results (output datasets) from each sub-job into overall result
INFSO-RI-508833
ADA Architecture
Enabling Grids for E-sciencE
GUI and command line clients ROOT PYTHON AMI ws AMI DBS DIAL AS sh LSF, CONDOR AJDL ATPROD AS SQL ATPROD AJDL ARDA AS gLite High level services for cataloging and job submission and monitoring Workload management systems gLite WMS INFSO-RI-508833
ADA
Enabling Grids for E-sciencE ADA uses DIAL framework. Release 1.20 of DIAL is the basis for the current ADA system.
• •
To use ADA it is necessary To have Grid certificate
– Certificate from Russian CA is OK
To be member of Atlas VO
– Takes some time INFSO-RI-508833
Enabling Grids for E-sciencE
•
Working node - LXPLUS at CERN
First Steps
•
Setup grid environment:
• • •
. /afs/cern.ch/project/gd/LCG-share/sl3/etc/profile.d/grid_env.sh
Certification proxy initialization: grid-proxy-init
DIAL setup (setup script that defines a few environmental variables and aliases) at CERN: DIALSETDIR=/afs/cern.ch/user/d/dial/apps/dial/setup Verify user certificate and check the status of the unique ID service by issuing the command "uidtest" after setting up dial.
INFSO-RI-508833
INFSO-RI-508833
Enabling Grids for E-sciencE
First Steps
First Steps
Enabling Grids for E-sciencE
• • • •
The best way to start with DIAL is to run the demos inside ROOT These demos define a job
– – – application (papp) task (ptsk) dataset (pdst)
and submit it to the current scheduler (msch) Start: dialroot –i
flag –i means that any missing DIAL configuration, example or demo files will be copied into the local directory (necessary only 1 st time) INFSO-RI-508833
INFSO-RI-508833
Enabling Grids for E-sciencE
First Steps
INFSO-RI-508833
Enabling Grids for E-sciencE
First Steps
Demo Example
Enabling Grids for E-sciencE
• •
Distributed analysis is an iterative process where a physicist defines a job, submits it to a processing system, examines the result and then repeats the sequence.
Demo selects an application, task and dataset which are then submitted to a scheduler to define a job.
root [0] .x demos/demo4.C
This defines papp, ptsk and pdst
root [1] submit()
Submit a job based on papp, ptsk and pdst
root [2] get_results()
Get job status and partial result
root [3] TBrowser br
Check ouput ntuples and histgrams INFSO-RI-508833
Demo Example
Enabling Grids for E-sciencE
• • •
A job is specified by defining a transformation and selecting a dataset to process with this transformation. The transformation is specified by an application and a task. The application carries the scripts that do the processing and the task carries user configuration data.
Demo4 uses aodhisto to create histograms and ntuples from user source code The demo identifies objects by name, extract the corresponding ID from a selection catalog and use this ID to extract the object from a repository.
INFSO-RI-508833
}
Demo Example
void demo4() { Enabling Grids for E-sciencE string aname = "aodhisto"; string tname = "aodhisto_zll_aod"; string dname = "hma.dc2.003007.digit.A1_z_ee.aod-1000.10files"; aid = asc.id(aname); tid = tsc.id(tname); did = dsc.id(dname); papp = ar.extract(aid); ptsk = tr.extract(tid); pdst = dr.extract(did);
INFSO-RI-508833
INFSO-RI-508833
Enabling Grids for E-sciencE
Demo Example
INFSO-RI-508833
Enabling Grids for E-sciencE
Demo Example
INFSO-RI-508833
Enabling Grids for E-sciencE
Demo Example
INFSO-RI-508833
Enabling Grids for E-sciencE
Demo Example
INFSO-RI-508833
Enabling Grids for E-sciencE
Demo Example
INFSO-RI-508833
Enabling Grids for E-sciencE
Demo Example
INFSO-RI-508833
Enabling Grids for E-sciencE
Demo Example
INFSO-RI-508833
Enabling Grids for E-sciencE
Demo Example
INFSO-RI-508833
Enabling Grids for E-sciencE
Demo Example
Demo Example
Enabling Grids for E-sciencE
• •
Objects: papp - pointer to the current application ptsk - pointer to the current task pdst - pointer to the current dataset Can be displayed root [4] pprint(papp)
Display the application
root [5] pprint(ptsk)
Display the task
root [6] pprint(pdst)
Display the dataset INFSO-RI-508833
INFSO-RI-508833
Enabling Grids for E-sciencE
Demo example
INFSO-RI-508833
Enabling Grids for E-sciencE
Demo Example
Enabling Grids for E-sciencE There are more examples:
More Examples
• • • •
Demo5 uses esd2aod to create AOD from ESD using the prodsys transformation Demo6 uses atlasopt to run a job with provided job options Demo7 uses atlasdev to run a job based on a users atlas development area Demo8 uses atlasdev-src to run a job based on a tarball of a user development area
INFSO-RI-508833
More Examples
Enabling Grids for E-sciencE
•
Displaying the status of all catalogs to verify connection and see the size of each: root [4] show_catalogs()
INFSO-RI-508833
More Examples
Enabling Grids for E-sciencE
• • • • •
A list of available datasets may be obtained by querying the DSC (dataset selection catalog, object dsc). The DSC is the primary user interface to datasets and it plays a role of what is often called a metadata catalog. Limit the query to 100 results (received 12). The query resticts the selection to TOP level datasets, i.e. complete samples intended for user access and then uses the name to select Rome samples with v10 reconstruction, SUSY data using all AOD data avaialble at BNL. AOD-bnl replaced with AOD to get samples available at both CERN and BNL. Counting datasets matching a query with the query_count method
INFSO-RI-508833
INFSO-RI-508833
Enabling Grids for E-sciencE
More Examples
More Examples
Enabling Grids for E-sciencE
•
DCS supports list of parameters which can be used in selection of Datasets
INFSO-RI-508833
More Examples
Enabling Grids for E-sciencE
• •
List attributes for given Dataset Record ID and fetch the Dataset from repository
INFSO-RI-508833
INFSO-RI-508833
Enabling Grids for E-sciencE
More Examples
More Examples
Enabling Grids for E-sciencE
•
Select an application in a similar way
INFSO-RI-508833
More Examples
Enabling Grids for E-sciencE
•
Select a task in a similar way
INFSO-RI-508833
More Examples
Enabling Grids for E-sciencE
• •
The application usually is not modified, but necessity of task modification is very likely Extract the files from the task
INFSO-RI-508833
More Examples
Enabling Grids for E-sciencE
•
The list of jobOptions can be found in CVS repository at atlas/PhysicsAnalysis/AnalysisCommon/ AnalysisExamples/share/
INFSO-RI-508833
More Examples
Enabling Grids for E-sciencE
• •
Now it is possibly to build a new task from the modified files:
ptsk = new dial::Task("atlas_release jo.py output_content", "mytask"); The list of files used to construct the task may be replaced with "*" if you want all the files from the directory
Now papp, ptsk and pdst are defined, and job can be submited
INFSO-RI-508833
INFSO-RI-508833
Enabling Grids for E-sciencE
More Examples
INFSO-RI-508833
Enabling Grids for E-sciencE
More Examples
More on More Examples
Enabling Grids for E-sciencE
• • • • • •
It is not necessary to do a lot of typing (as we did before) to perform previous analysis There is simple way to avoid this – job definition script that defines the application, task and dataset (variables papp, ptsk and pdst).
Sample script can be found here:
http://www.usatlas.bnl.gov/~dladams/dial/releases/1.20/jobdef.C
The sample script is copied into the local directory when the dialroot files are installed (dialroot -i).
Edit the top part of this script to specify the application, task and dataset of interest.
Run:
root [0] .x jobdef.C root [1] submit() ... INFSO-RI-508833
More on More Examples
Enabling Grids for E-sciencE
void jobdef() { // Specify names for the application, task and dataset. // Typical job definition is created by changing these values. // Depending on the following code, a name may be intepreted as // one or more of the following. // 1. ID: Object identifier. // 2. name: Object name in the default selection catalog. // 3. directory: Name of a directory holding files to be used // construct the object. // 4. xml: Name of a file holding the XML description of the object. // Application: directory, name, or ID. string aname = "atlasopt"; // Task: directory, xml, name, or ID. string tname = "atlasopt_example_zll-10.0.1"; // Dataset: ID or name. string dname = "hma.dc2.003007.digit.A1_z_ee.aod-1000.10files"; …….
INFSO-RI-508833
•
More on More Examples
Enabling Grids for E-sciencE There is web interface “DIAL CATALOG QUERY PAGE”
http://www.atlasgrid.bnl.gov/dialds/dlShowMain-new.pl
INFSO-RI-508833
More on More Examples
Enabling Grids for E-sciencE
• • •
This interface permit to switch to:
– Dataset Selection Catalog (DSC) – – Task Selection Catalog (TSC) Application Selection Catalog (ASC)
A list of available datasets may be obtained from DSC query page Some useful applications and example tasks are cataloged as well. The application and task catalogs may also be examined using the ASC query page and TSC query page
INFSO-RI-508833
INFSO-RI-508833
More on More Examples
Enabling Grids for E-sciencE
INFSO-RI-508833
More on More Examples
Enabling Grids for E-sciencE
INFSO-RI-508833
More on More Examples
Enabling Grids for E-sciencE
INFSO-RI-508833
More on More Examples
Enabling Grids for E-sciencE
INFSO-RI-508833
More on More Examples
Enabling Grids for E-sciencE
INFSO-RI-508833
More on More Examples
Enabling Grids for E-sciencE
More on Transformations
Enabling Grids for E-sciencE
• • • • •
Transformations are applied to datasets to produce new datasets. A transformation includes:
– an application which carries out the processing – a task used to configure the application
An application provides two entry points: one to build (e.g compile) a task and one to process a dataset A task is a collection of named text files It is not sensible to arbitrarily combine any task with any application.
INFSO-RI-508833
More on Transformations
Enabling Grids for E-sciencE
• • •
There is a task interface that specifies which files must or may be present in a task and how these files are to be used. Tasks are labeled with the interface they provide and applications with the task interface they expect. Task interfaces:
– – – – – – – atlas_release atlas_job_options atlas_simple_analysis atlas_user_analysis atlas_developer_directory atlas_developer atlas_xform INFSO-RI-508833
More on Transformations
Enabling Grids for E-sciencE
• •
The task interface atlas_release specifies an atlas release List of files for atlas_release:
–
atlas_release - ATLAS release version, e.g. 10.0.1.
• •
The task interface atlas_job_options specifies an atlas release, job options and output content List of files for atlas_job_options:
– – –
atlas_release - ATLAS release version, e.g. 10.0.1. jo.py - User job options. output_content - describes the output to be saved (content label and name
- HIST hist.root ) INFSO-RI-508833
More on Transformations
Enabling Grids for E-sciencE
• •
The ADA task interface atlas_user_analysis specifies an atlas release and files to replace those in the UserAnalysis package List of files for atlas_user_analysis:
–
atlas_release - ATLAS release version, e.g. 10.0.1.
– – – – –
*.h - header files. *.cxx - C++ source files requirements - CMT requirements file AnalysisSkeleton_jobOptions.py - job options file output_content - describes the output to be saved
INFSO-RI-508833
Documentation
Enabling Grids for E-sciencE
•
ADA system described on ADA home page
– http://www.usatlas.bnl.gov/ADA •
This page also has a link to DIAL 1.20 release page
INFSO-RI-508833
Conclusion
Enabling Grids for E-sciencE
•
ADA permits now to perform distributed analysis for Atlas experiment
•
Available documentation permits to newcomers to start using of ADA
•
Further development (especially user-oriented) will allows more wider distribution of ADA among physicists
INFSO-RI-508833