Using SAS Parallel-Processing Features To Reduce Program Execution Time

Presented by Berwick Chan Kaiser Permanente Vaccine Study Center NCAL Division of Research October 2009 For BASAS October 2009 meeting 1

MP in SAS

 Starting in version 6, SAS introduced multi-processing feature  Starting in version 9, SAS introduced a new SPD engine that provide multi thread processing power 2

MP-Connect

  Provided by the SAS-Connect module Uses default SAS engine   LIBNAME

libref

'

SAS-data-library

Allows independent tasks / steps within the same SAS program to be executed concurrently Requires additional programming 3

SPDE Engine

 Uses Scalable Performance Database Engine LIBNAME

libref spde

'

SAS-data-library

'  Provides multi-thread process automatically  Requires no additional programming  Requires additional setup process for SAS library / table 4

SAS (Default) Processing Flow

Execution Time SAS starts Task 1 X seconds Task 2 SAS ends y seconds Total: (x+y) seconds 5

SAS MP Processing Flow

SAS starts SAS starts Execution Time Task1 Task 2 SAS starts Task 2 SAS ends x sec y sec Task 1 SAS ends Total exec time: max(x,y) sec SAS ends 6

Sequential / MP Process Comparison  Program executed in sequential process readdb.sas.txt

readdb.log.txt

 Program executed in MP readdb_mp.sas.txt

readdb_mp.log.txt

 Result: MP version takes about 59% execution time of the sequential version (34.78 sec vs. 59.2 sec), a 41% savings.

7

More About MP-Connect

  Provides a new set of SAS commands Enables communications among concurrent sessions / tasks: – Work libraries sharing – Macro variables passing – Session status checking – Pipeline parallelism (using SASESOCK engine) http://support.sas.com/onlinedoc/913/getDoc/en/connref.hlp/a002626625.htm

 Manages computer resource more efficient across systems 8

More About MP-Connect

 Not applicable to all programs, good for running independent tasks within the same program. Ideal applications: – Production jobs with short update windows – ETL applications that access different databases – Online applications that demand quick response  Performance enhancement on individual program may vary and is depending on system resource availability during program execution 9

More About MP-Connect

 Useful links:

http://support.sas.com/onlinedoc/913/getDoc/en/connref.hlp/connrefwhatsnew900.htm

http://support.sas.com/rnd/scalability/tricks/connect.html

http://www2.sas.com/proceedings/sugi28/279-28.pdf

10