Tools to support good model building and use

Download Report

Transcript Tools to support good model building and use

Tools to support good model building and model use

Ir. Willy Baltussen MBA Dr. Wietse Dol Ir. Foppe Bouma Dr. Hans Vrolijk

Contents of presentation

• Background • Problem description • Protocol on good modeling practice • Some tools to support the protocol • First experiences • Discussion • Conclusion

Aim of presentation

Present a protocol and some tools to increase

the speed and quality of the model development process

the quality of the models and

the quality of research

Background

• LEI is a research institute with a core business in data, models, and research in the area of agriculture and economics.

• Small and large models are frequently used in the research process • There is a necessity to increase the quality of our research, i.e. data and models • Tools are being developed to support the quality improvement process

Necessity to increase model quality

• More multidisciplinary research • More re-use of (parts of) models • More involvement of customers (ISO) • IT offers more possibilities

Some Quality aspects

• Actual model and the documented conceptual model should be consistent • Control of versions and scenarios of model • Knowledge is accessible for others (documentation, definition of used data, variables: Corporate knowledge) • Model is peer reviewed (technical and scientific) • Separation of model, user interface and input/output data (extending live of model)

Modeling Protocol

Context analysis Conceptualization Data analysis Mathematical model Implementation Validation Model use Policy analysis

Support tools for the phases

• Context analysis: interviews, document analysis • Conceptualization: workshops • Data analysis, Mathematical model and implementation: check dimensions, robustness, sensitivity (questionnaires, workbooks).

• Validation: check extreme conditions, reference mode of behavior.

The modeling protocol in practice

The protocol was applied to the MAM model which was developed for the first time in the 1980’s and extended and changed in the last 20 years A project team of domain experts and model experts had a session of one week to get through the conceptualization phase and a first global sketch of a mathematical model (the concept analysis was executed by the domain expert before this week).

Model protocol in practice II

• MicroWave is a research project LEI, Univ. Gent and Univ. Leuven to build a general reusable micro simulation model (enterprises).

Experiences with the protocol

• Context analysis: future requirements of the model are depicted. An integral evaluation took place + priorities • Conceptual modeling: a more flexible model can be obtained after identifying and describing general structures. New requirements could be fit in.

• Mathematical modeling has proven to be more compact than the old software code.

• The focus was more on the general structure than on the exceptions.

• All participants: (domain experts and model experts) had a positive feeling on the approach

Software tool GSE

The models are implemented in GAMS and GSE. GSE: GAMS Simulation Environment GSE software package • to built a user interface for GAMS models • To store your model versions and scenarios • To exchange models with others (researchers, users and customers).

Advantages of GSE

• Control of model versions and scenarios • A user interface can be developed quick and easy • Data viewer to combine inputs with outputs and to compare different scenarios. • Options for documentation of variables, model version, scenarios (supports model documentation and documentation of model results).

• Options for export model results other software packages • Options for exchange model (results) with other people (researchers or customers).

Discussion

• Quality will always be relative • Tools and protocols are available.

Question is how to select the right tools for your modeling group and how to use them.

• Software can support the quality but is not a guarantee for high quality

Conclusions

• There is need for quality improvement of model building and model use.

• Many more or less sophisticated tools are available • Application of these tools will strongly depend on the modeling group and environment.

• So in the end the quality of modelling still depends on the modelling team and its skills (Scholten et .al. 2000)