How much data are needed to calibrate and test agent-based models?
share
Agent-based models (ABMs) are widely used to gain insights into the dynamics of coupled natural human systems and to assess risk management strategies Choosing a sound model structure and parameters requires careful calibration. However, ABMs are often not calibrated in a formal statistical sense. One key reason for this lack of formal calibration is the potentially large data requirements for ABMs with path-dependence and nonlinear feedbacks. Using a perfect model experiment, we examine the impact of varying data record structures on (i) model calibration and (ii) the ability to distinguish a model with agent interactions from one without. We show how limited data sets may not constrain even a model with just four parameters. This finding raises doubts about many ABM's predictive abilities in the absence of informative priors. We also illustrate how spatially aggregate data can be insufficient to identify the correct model structure. This emphasises the need for carefully fusing independent lines of evidence, for example from judgment and decision making experiments to select sound and informative priors.
READ FULL TEXT