Transcript Comparison of Liam2 and Genesis Dynamic

Model
Development Unit
MDU
Comparison of LIAM2 and
Genesis Model Generators
Howard Redway
Model Development Unit
UK Department for Work and Pensions
[email protected]
IMA Conference
May 18 2012
Model
Development Unit
MDU
Presentation Overview
Background:
Why Were Genesis and LIAM2 Developed
How Are They Used
Overview of the Structure of Genesis and LIAM2
Main Differences in Functionality
Structure of a Genesis Model
Structure of a LIAM2 Model
Simple Example of Genesis and LIAM2 Code
More Complex Example of Genesis and LIAM2 Code
Conclusions
Questions Further Information
Model
Development Unit
MDU
Why Genesis was Developed
and How is it Used (1)
Originally Developed for Pensim2 (replacing for Pensim)
•
To model State and Private Pensions policies
•
Desktop tool
•
Parameter driven by users
•
Maintainable within DWP (Not black box like Pensim)
•
Based on DWP standard language – SAS
Model
Development Unit
MDU
Why Genesis was Developed
and How is it Used (1)
Originally Developed for Pensim2 (replacing for Pensim)
•
To model State and Private Pensions policies
•
Desktop tool
•
Parameter driven by users
•
Maintainable within DWP (not black box like Pensim)
•
Based on DWP standard language – SAS
Realised it could generate a wide class of models
•
Became Genesis separate from Pensim2
•
5 other Genesis models in addition to Pensim2
•
Genesis is owned and maintained by DWP
Model
Development Unit
MDU
Why Genesis was Developed
and How is it Used (2)
Pensim2:
•
First release 2004
•
Used by the Pensions Commission
•
Private and State Pension Policy Development
•
Forecasting
•
Up to 20 users at any one time
•
High turnover of users and developers
•
About 6 released a year (2 major)
Model
Development Unit
MDU
Why LIAM2 Developed and How is it Used
LIAM2 was developed:
•
For the development and management of large dynamic
microsimulation models by teams of researchers
•
To be flexible so that modellers can update the models fast to
changes in the circumstances
•
To separate the computer programming from the model
development
•
Microsimulation teams no longer have to reinvent the wheel
•
Different teams can cooperate more easily
Model
Development Unit
MDU
Why LIAM2 Developed and How is it Used
LIAM2 was developed:
•
For the development and management of large dynamic
microsimulation models by teams of researchers
•
To be flexible so that modellers can update the models fast to
changes in the circumstances
•
To separate the computer programming from the model
development
•
Microsimulation teams no longer have to reinvent the wheel
•
Different teams can cooperate more easily
•
As an open source tool running on a standard PC
Model
Development Unit
MDU
Why LIAM2 Developed and How is it Used
LIAM2 was developed:
•
For the development and management of large dynamic
microsimulation models by teams of researchers
•
To be flexible so that modellers can update the models fast to
changes in the circumstances
•
To separate the computer programming from the model
development
•
Microsimulation teams no longer have to reinvent the wheel
•
Different teams can cooperate more easily
•
As an open source tool running on a standard PC
Several models produced or under development
Model
Development Unit
MDU
Why LIAM2 Developed and How is it Used
LIAM2 was developed:
•
For the development and management of large dynamic
microsimulation models by teams of researchers
•
To be flexible so that modellers can update the models fast to
changes in the circumstances
•
To separate the computer programming from the model
development
•
Microsimulation teams no longer have to reinvent the wheel
•
Different teams can cooperate more easily
•
As an open source tool running on a standard PC
Several models produced or under development
LIAM2 is being developed at the Federal Planning Bureau (Belgium), with funding and testing by
CEPS/INSTEAD (Luxembourg) and IGSS (Luxembourg), and EU funding
Model
Development Unit
MDU
Structure of a Model: LIAM2 (1)
LIAM2 is:
•
An interpreter/compiler
•
Written in Python
(model developer and users do not need to know this)
•
The model is specified in text as a LIAM2 program
•
In YAML-markup language (independent of Python)
•
Most LIAM2 statements are interpreted and executed
(some are compiled)
Model
Development Unit
MDU
Structure of a Model: LIAM2 (2)
LIAM2
Model
Development Unit
MDU
Structure of a Model: LIAM2 (2)
Model Specification
(YAML format)
LIAM2
Model
Development Unit
MDU
Structure of a Model: LIAM2 (2)
Model Specification
(YAML format)
Alignment
Matrices
LIAM2
Model
Development Unit
MDU
Structure of a Model: LIAM2 (2)
Model Specification
(YAML format)
Alignment
Matrices
Base Data
LIAM2
Model
Development Unit
MDU
Structure of a Model: LIAM2 (2)
Model Specification
(YAML format)
Alignment
Matrices
Base Data
LIAM2
Output
Model
Development Unit
MDU
Log and PC Output
Structure of a Model: Genesis (1)
Genesis is:
•
A code generator
•
Written in SAS
•
Model specified as parameters in Excel sheets
•
Writes and then runs a SAS program
Model
Development Unit
MDU
Structure of a Model: Genesis (1)
Genesis Model Engine
Model
Development Unit
MDU
Structure of a Model: Genesis (1)
Parameters
Excel Spreadsheets
Genesis Model Engine
Model
Development Unit
MDU
Structure of a Model: Genesis (1)
Parameters
Static Code
Excel Spreadsheets
Developers Own SAS
Genesis Model Engine
Model
Development Unit
MDU
Structure of a Model: Genesis (1)
Parameters
Static Code
Excel Spreadsheets
Developers Own SAS
Genesis Model Engine
SAS Program
Model
Development Unit
MDU
Structure of a Model: Genesis (1)
Parameters
Static Code
Excel Spreadsheets
Developers Own SAS
Genesis Model Engine
SAS Program
Model
Development Unit
MDU
Base Data
Structure of a Model: Genesis (1)
Parameters
Static Code
Excel Spreadsheets
Developers Own SAS
Genesis Model Engine
Base Data
SAS Program
Output
Model
Development Unit
MDU
Log
Structure of a Model: Genesis (1)
Tools
Parameters
Static Code
Developers Own SAS
Excel Spreadsheets
Genesis Model Engine
Base Data
SAS Program
Tools
Model
Development Unit
MDU
Output
Log
Tools
Main Differences in Functionality (1)
Genesis has:
•
Static Code: Provides the flexibility to code in SAS processes that
are not supported by the specific Genesis functionality
•
Missing values: SAS missing value indicator enable missing values
to be trapped (LIAM2 uses -1 and FALSE for Integer and Boolean
missing values).
•
Date processing
•
Unordered Polychotomous Regressions
Model
Development Unit
MDU
Main Differences in Functionality (1)
Genesis has:
•
Static Code: Provides the flexibility to code in SAS processes that
are not supported by the specific Genesis functionality
•
Missing values: SAS missing value indicator enable missing values
to be trapped (LIAM2 uses -1 and FALSE for Integer and Boolean
missing values).
•
Date processing
•
Unordered Polychotomous Regressions
•
Ordered Polychotomous Regressions
•
Probit Regression (specific functionality)
Model
Development Unit
MDU
Main Differences in Functionality (2)
LIAM2 has:
•
Interactive Console and Breakpoints: Powerful debugging tool
(inspection of variables and step through procedures)
•
Temporary Variables: Not saved automatically and may be local to
a procedure
Model
Development Unit
MDU
Main Differences in Functionality (2)
LIAM2 has:
•
Interactive Console and Breakpoints: Powerful debugging tool
(inspection of variables and step through procedures)
•
Temporary Variables: Not saved automatically and may be local to
a procedure
•
Macros (more general than Genesis Selection Criteria)
•
One2Many links and link functions
•
Forced outcomes with alignment (take/leave)
•
Partnership matching
•
Cloning: create a new record copying selected variables
•
Output functions (can be switched off)
Model
Development Unit
MDU
Example 1: Simple LIAM2 Choice Process
How would you assign level of education in LIAM2 for persons aged 25
and over?
Assuming the variables Age and EducationLevel had be defined
Specify the following as a process for the entity person.
Model
Development Unit
MDU
Example 1: Simple LIAM2 Choice Process
How would you assign level of education in LIAM2 for persons aged 25
and over?
Assuming the variables Age and EducationLevel had be defined
Specify the following as a process for the entity person.
EducationLevel: if(
(Age>25) and (EducationLevel != -1) ,
choice([2,3,4], [0.25, 0.39, 0.36]) ,
EducationLevel )
Model
Development Unit
MDU
Example 1: Simple LIAM2 Choice Process
How would you assign level of education in LIAM2 for persons aged 25
and over?
Assuming the variables Age and EducationLevel had be defined
Specify the following as a process for the entity person.
EducationLevel: if(
(Age>25) and (EducationLevel != -1) ,
choice([2,3,4], [0.25, 0.39, 0.36]) ,
EducationLevel )
The within the simulation block of the program:
person: [EducationLevel]
Model
Development Unit
MDU
Example 1: Simple Genesis DPM Action
SelectionCriteriaRule Sheet
SelectionCriteria
Condition1
Condition2
Over25_Education_Not_Assigned
pa_Age>25
MISSING(pa_EducationLevel)
Model
Development Unit
MDU
Example 1: Simple Genesis DPM Action
SelectionCriteriaRule Sheet
SelectionCriteria
Condition1
Condition2
Over25_Education_Not_Assigned
pa_Age>25
MISSING(pa_EducationLevel)
DPM_Assign_Education Sheet
Table Name
pa
Selection Criteria
Data Item
EducationLevel
Outcome
2002
2003
2004
2005
2
0.25
0.25
0.25
0.25
3
0.39
0.39
0.39
0.39
4
Model
Development Unit
MDU
Example 1: Simple Genesis DPM Action
SelectionCriteriaRule Sheet
SelectionCriteria
Condition1
Condition2
Over25_Education_Not_Assigned
pa_Age>25
MISSING(pa_EducationLevel)
DPM_Assign_Education Sheet
Table Name
pa
Data Item
EducationLevel
Selection Criteria
Outcome
2002
2003
2004
2005
2
0.25
0.25
0.25
0.25
3
0.39
0.39
0.39
0.39
4
ActionRule Sheet
Actions
Name
DPM_Assign_Education
Model
Development Unit
MDU
Selection Criteria
Tablename
Variable
Over25_Education_Not_Assigned
pa
EducationLevel
Example 2: More Complex Genesis DPM Action
DPM_Assign_Education Sheet
Table Name
pa
Data Item
EducationLevel
Outcome
2002
2003
2004
2005
Man_UK_Born
2
0.25
0.24
0.23
0.23
Man_UK_Born
3
0.39
0.39
0.38
0.37
Man_UK_Born
4
Man_NonUK_Born
2
0.25
0.25
0.25
0.25
Man_NonUK_Born
3
0.39
0.39
0.39
0.39
Man_NonUK_Born
4
Woman_UK_Born
2
0.23
0.23
0.23
0.23
Woman_UK_Born
3
0.23
0.23
0.23
0.23
Woman_UK_Born
4
Woman_NonUK_Born
2
0.23
0.23
0.23
0.23
Woman_NonUK_Born
3
0.23
0.23
0.23
0.23
Woman_NonUK_Born
4
Selection Criteria
Model
Development Unit
MDU
Example 2: More Complex LIAM2 Choice Process
EducationLevel: if(
(age>25) and (EducationLevel != -1) and (Man_UK_Born) ,
if((period=2002),
choice([2,3,4], [0.25,0.39,0.36]) ,
if((period=2003),
choice([2,3,4], [0.24,0.39,0.37]) ,
if((period=2004),
choice([2,3,4], [0.23,0.38,0.39]) ,
if((period=2005),
choice([2,3,4], [0.23,0.37,0.4]) ,
EducationLevel
)))),
Model
Development Unit
MDU
Example 2: More Complex LIAM2 Choice Process
EducationLevel: if(
(age>25) and (EducationLevel != -1) and (Man_UK_Born) ,
if((period=2002),
choice([2,3,4], [0.25,0.39,0.36]) ,
f((period=2003),
choice([2,3,4], [0.24,0.39,0.37]) ,
if((period=2004),
choice([2,3,4], [0.23,0.38,0.39]) ,
if((period=2005),
choice([2,3,4], [0.23,0.37,0.4]) ,
EducationLevel
)))),
if( (age>25) and (EducationLevel != -1) and (Man_NonUK_Born)
etc
)
Model
Development Unit
MDU
Conclusions (1)
LIAM2 and Genesis are both able to generate discrete dynamic
microsimulation models containing similar processes (demographic,
labour market, pension processes)
Genesis requires SAS (currently version 9.1)
Genesis is owned and maintained by DWP
LIAM2 is Open Source
Model
Development Unit
MDU
Conclusions (1)
LIAM2 and Genesis are both able to generate discrete dynamic
microsimulation models containing similar processes (demographic,
labour market, pension processes)
Genesis requires SAS (currently version 9.1)
Genesis is owned and maintained by DWP
LIAM2 is Open Source
There are a few major differences in functionality
Genesis has Static Code
LIAM2 has Interactive Console and Temporary variables
Model
Development Unit
MDU
Conclusions (1)
LIAM2 and Genesis are both able to generate discrete dynamic
microsimulation models containing similar processes (demographic,
labour market, pension processes)
Genesis requires SAS (currently version 9.1)
Genesis is owned and maintained by DWP
LIAM2 is Open Source
There are a few major differences in functionality
Genesis has Static Code
LIAM2 has Interactive Console and Temporary variables
Many other differences in functionality
Model
Development Unit
MDU
Conclusions (1)
LIAM2 and Genesis are both able to generate discrete dynamic
microsimulation models containing similar processes (demographic,
labour market, pension processes)
Genesis requires SAS (currently version 9.1)
Genesis is owned and maintained by DWP
LIAM2 is Open Source
There are a few major differences in functionality
Genesis has Static Code
LIAM2 has Interactive Console and Temporary variables
Many other differences in functionality
Different approach to how a model is specified
Model
Development Unit
MDU
Conclusions (2)
The slow running of Genesis is a concern
Currently looking for ways to speed this up
LIAM2 appears to be significantly faster
No direct comparison available yet
Model
Development Unit
MDU
Conclusions (2)
The slow running of Genesis is a concern
Currently looking for ways to speed this up
LIAM2 appears to be significantly faster
No direct comparison available yet
If both Genesis and LIAM2 are capable of generating a specific model
with acceptable runtimes then
the most appropriate interface for the intended users
is likely to be a significant factor
Model
Development Unit
MDU
Questions Further Information
Questions?
Genesis Information:
[email protected]
Or any member of the Model Development Unit
Two presentations in the next session
LIAM2 information:
http://liam2.plan.be/
Model
Development Unit
MDU