Transcript StoRM latest performance test results

StoRM latest performance test
results
Alberto Forti
Otranto, Jun 8 2006
– n° 1
StoRM : Storage Resource Manager
Description
: StoRM is a disk based storage resource manager.
It implements the SRM v.2.1.1 standard interface. StoRM is
designed to support guaranteed space reservation and direct
access (native posix I/O call), as well as other I/O libraries
like RFIO. Security aspects are based on user identity (VOMS
certificate).
StoRM is designed for taking advantage from high performance
distributed file systems like GPFS. Also standard POSIX file
systems are supported, like ext3 and XFS, and new plug-ins
could be easily developed.

– n° 2
StoRM: Functionalities

SRM Interface V.2.1.1

Dynamic management of (disk) storage resources (files and space):



Introduces concepts of lifetime of a file (volatile with a fixed amount of life-time, durable or permanent),
file pinning (to ensure a file is not cancelled during operation), space pre-allocation (to ensure the
requested disk space is available for the whole life of the application since the beginning).
Files are no longer permanent entities on the storage, but dynamical ones that can appear and disappear
according to the user’s specifications (when life-time expires it is available to a garbage collector for
deletion without further notice).
More relevant functionalities (already implemented and tested in StoRM):

Data Transfer:






Space Management: srmReserveSpace(),


srmPrepareToPut()  creates a file and on request allocates disk space.
srmPrepareToGet()  pins a file and forbids deletion by the SRM.
srmCopy()  asynchronously, creates a file on one side, pins it on the other side, and execute the transfer
srm*Status()  check the status of asynchronous operations submitted.
srmPutDone()  tells the SRM that the file has been written and then can be deleted if e.g. lifetime
expires.
srmGetSpaceMetaData().
To allocate big chunks of disk space to be subsequently used by independent files (similar to allocating
space for a single file, but done at once).
Directory functions : srmLs() with recursive option, srmRm(), srmRmDir(), srmMkDir().
– n° 3
StoRM – Grid scenario
SE
CE
GridFTP
I/O
server
Site A
Data
Management
Access
Job Submission
StoRM
CE
Direct
Access
GridFTP
GPFS
WN
WN
WN
WN
Site B
– n° 4
Simple minded use case (I)
 Analysis


job without StoRM (i.e. without SRM v2.1):
The job locates the input data physical path on the SE (i.e. via file catalogues).
The job copies (e.g. via GridFTP if remote SE or RFIO if local SE) the input
datasets from the SE onto the local disk of the WN.

The job processes the dataset and writes output data onto the local disk.

The job copies the locally produced data (e.g. ntuples) back to the SE.
 DRAWBACKS!



if the local disk gets full during the job lifetime (e.g. other jobs running on the
same WN exhaust the available space) the job will fail.
If the SE fills up (or the quota is over) during the job execution data cannot be
copied back, and the job will fail.
No dynamical usage of the SE disk space is allowed, files should be taken
permanently resident on the disk, and just cleaned by some central administrator
or the owner him/herself at some point.
– n° 5
Simple minded use case (II)
 Analysis
with StoRM (i.e. with SRM v2.1)

The job locates the input data physical path on the SE (i.e. via file catalogues).

In case the file is already on the local SE:


In case the file is available from a remote SE:






The job executes a srmCopy v2 call to transfer the data from the remote SE to the local
SE (this assuming that a local disk-based cache storage is always available) assigning to
the local copy a fixed lifetime according to the estimated lifetime of the job itself.
The job opens the input files on the local SE (using UNIX-like, i.e. POSIX system
calls – no need of additional protocols embedded in the application).
The job creates output files for writing to the local SE (SRM prepare to put call),
pre-allocating the estimated size of disk space for the job output, and opens
them.
The job processes the input datasets and writes the output data files from/to the
local SE.
The job unpins the input data files and the output data files (releaseFile and
PutDone SRM v2 calls).
It does not need to rely on the availability of the local disk, and no further copies
are needed.
 NB:

The job pins the file on the SE through a SRM prepare to get call, to ensure it is not
cancelled during job operation.
More advanced scenarios include higher layers as FTS for data transfer
SRM v2 is not a replacement of FTS!
– n° 6
Early StoRM testbed at CNAF T1 (pre-Mumbai)
Worker Nodes
mounting GPFS
Worker Node
StoRM
Ethernet
Storage servers GPFS
and GridFTPd
Cluster GPFS
4 x Sun
Microsystem
SUnFire V20Z
CE
 Framework:
The disk storage was
composed by roughly 40 TB, provided
by 20 logical partitions of one dedicated
StorageTEK FLexLine FLX680 disk array
storage, aggregated by GPFS.
 Write
Qlogic
2340 HBA
8 x 2 Gb/s FC
SAN Fabric
4 x 2 Gb/s FC
20 x 2 TB
(40 TB)
StorageTEK
FLexLine
FLX680
test: srmPrepareToPut() with implicit
reserveSpace of 1GB files. globus-url-copy from local
source to the returned TURL. 80 simultaneous client
processes.
 Read
test: srmPrepareToGet() followed by globus-urlcopy from the returned TURL to a local file (1 GB files).
80 simultaneous client processes.
 Results:


Sustained read and write throughputs measured : 4 Gb/s and 3 Gb/s respectively.
The two tests are meant to validate the functionality and robustness of the
srmPrepareToPut() and srmPrepareToGet() functions provided by StoRM, as well as
to measure the read and write throughputs of the underlying GPFS file system.
– n° 7
Latest functionalities stress test
 Early
testbed not available any more due to storage needs of T1 operations
 Current
(small) testbed: 2 StoRM instances

Server1: Dual-PIII 1GHz, 512MB, Fast Ethernet (100 Mb/s)

Server2: Dual-Xeon 2.4 GHz, 2GB RAM, Fast Ethernet (100 Mb/s)

Each machine mounts a small GPFS volume and runs:

StoRM server(s).

MySQL, GridFTP and other services.
 Description:


stress tests for each functionality provided.
Large number of requests (1000-10000) performed with different submission rate
(10 - 30 requests per second).
For each test was evaluated:

Execution rate.

Number of failures.
 NB:

Performances hereby reported are strictly related to the underlying systems used (CPU, disks,
memory), and would scale up significantly by using more performant hardware.
– n° 8
Functionalities stress test: results
 Synch

Mkdir:





Functions:
2000 requests with submission rate of ~30 Hz.
Executed at ~10 Hz (depends on underlying GPFS, and in this case it was a
toy one on local disks) without failures.
Rmdir:

2000 requests with submission rate of ~30 Hz.

Executed at ~30 Hz without failures.
Rm:

1000 requests for files of 1 MB with submission rate of ~30 Hz.

Executed at ~30 Hz without failures.
SrmLS:

1000 requests on single files submitted at ~60 Hz , executed at ~20 Hz.

1 single LS on directory with 1000 file, 6 s.

3 simultaneous LS on directory with 1000 file, 10 s.

12 simultaneous LS on directory with 1000 file, 30 s. …

No failures.
– n° 9
Functionalities stress test: results
 Asynch


Functions:
PrepareToPut

10000 request submitted at 10 Hz rate (10 request for seconds)

Executed at 10Hz rate by StoRM.
PrepareToGet

1000 request with submission rate 30 Hz, executed at 20 Hz.
 SrmCopy

10000 request of 10 MB files submitted at rate of 10 Hz, 50 gridftp
transfers at once (transfer pool threads in StoRM was set to 50, but can
be tuned by StoRM option files). Poor data throughput (just Fast Ethernet
connectivity) but perfect enqueuing of the copies, all executed slowly at
1 Hz according to the available bandwidth (100 Mb/s).
– n° 10
Test with official srm-client v2.1.x
 For
interoperability issues we performed tests with srm clients
ditributed by the srm working group.
 Results:




Clients are able to communicate with our server.
Most functionalities are executed with success.(e.g. SrmRm, SrmMkdir,
SrmReserveSpace, SrmPrepareToPut, SrmPrepareToGet,
SrmGetRequestSummary, SrmCopy).
We have some problems with interpretation of some specific parameter
(e.g. Date time in SrmLS , recursive flag in SrmRmdir, …).
We believe that with some work and interaction with client developers
StoRM will gain full interoperability.
– n° 11
StoRM : Conclusions
 The
results obtained, in terms of rates and efficiencies, are very
interesting, even if using cheap and old hardware, as we did for our
latest tests.
 The
system can be scaled up to N StoRM servers working in parallel on the
same (or different) GPFS volume(s), with a centralized persistent database
(mysql at the moment, but the support for other vendors, e.g.
Oracle, can be easily put in place in future releases) similarly to the
Castor-SRM or dCache.
 After
a fruitful debugging phase, that will be now extended over the
next 2-3 weeks, StoRM will be ready to be deployed to production
environments. The production release candidate will be ready by the
end of June.
 The
StoRM Project is a collaboration between INFN/CNAF (3
developers) and EGRID/ICTP (3 developers).
– n° 12