Table 2 Criteria used to evaluate individual modelling studies.

Scientific

Quality of data input

Are the data used to estimate impact (e.g. associations between exposure and risk factors, or risk factor and disease, cost input) based on relevant evidence?

Low: Model inputs are derived from multiple sources with varying quality. Many assumptions based on expert opinions.

Medium: Model inputs are derived from multiple sources with good quality. Only some assumptions are based on expert opinions.

High: Majority of model inputs are derived from small number of studies or a single study with “high-quality” data providing information. e.g. about the associations between exposure factors, outcomes, quality of life, and health care utilization.

Model robustness

Is the model structure consistent with both a coherent theory of health condition being modeled and with available evidence regarding causal linkages between variables?

High robustness: if using one of the recognized models (e.g. RIVM, PRIME or IMPACT).

Medium robustness: if it concerns a model based on a consistent theory of health condition (e.g. clear health consequences hypertension leading to CVD) and when estimated results are “validated” or “compared” with other estimates.

Low robustness: if none of the above

Transparency

Are all details/assumptions of the modelling revealed or is the model a black box?

High transparency: if model equations, table with input parameters (and underlying distributions) and some “flow chart” is provided

Medium transparency: if only a flow chart is provided

Low transparency: if none of the above.

Sensitivity/uncertainty analysis

Are assumptions tested using sensitivity analyses?

Deterministic: Reports deterministic sensitivity analyses, either one-way sensitivity analysis (varying one factor at the time) or multiway (varying several factors simultaneously).

Probabilistic: Reports probabilistic sensitivity analysis based on (second-order) Monte Carlo simulation to evaluate total parameter uncertainty (i.e. imprecision of parameters). Microsimulation (patient-level) is applied to evaluate the role of “random variation” on final outcomes.

Different models: All previous + model structure uncertainty (i.e. different model structures are tested). Different survival models are tested to study sensitivity of applied model on final outcomes.

None/Not applicable: If none of the above.

Applicability

Data needs

What are the data needs in order to be able to run the model?

Little: The model runs on relatively little and readily available epidemiological data.

Modest: The model needs less readily available data on incidence, prevalence, and mortality by risk-factor status (e.g., for smokers and nonsmokers) or on transitions between risk-factor states.

Considerable: The model needs data on longitudinal panel/cohort data in order to be able to perform microsimulation.

Accessibility

Are models accessible for use by others?

Open source model: Established open source model, accessible for use by others.

Reproducible: All model parameters and software used are available for use by others to reproduce the model.

Not accessible: Established or self-built model, not accessible by others.

Other

Source of funding

How has the study been funded?

Public: Funded by public money only.

Private: Funded by private sector.

Both public and private: Funded by public and private money.

Not clear: Funding source not mentioned.

No funding: Not founded by public or private money.

Potential conflict of interest

Is the study subject to potential conflict of interest?

No: Study performed by a research institution/academia. Authors clearly state the funder(s) had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Yes: Involvement and funding by a party that has a clear personal or financial interest in the outcome of the results.

Not clear: Authors do not clearly state that the funding body had no role in the study.