Table 2 Main alternative approaches when combining subjective and objective data.

Searching for correlations and causality

Empirical research focuses on recognizing correlations and causal mechanisms that connect subjective and objective data. Such inquiries often presuppose either subjective or objective conception (as in De Neve and Sachs, 2020; Frijters et al., 2020; Costanza et al., 2008; Inglehart et al., 2008; Delhey and Steckermeier, 2016).

Low substitution assumes a conception that combines both aspects, regardless of correlations and causal connections between them. The two approaches may complement each other by utilizing empirical causal relationships in order to improve the low substitution scores and rankings of societies.

Importantly, with low substitution, we highlight the outlier societies—those that deviate from the average correlation (see Figs. 2, 5).

Dashboards

Acknowledging the necessity of multiple criteria, such solutions present the two types of information separately with different sets of scores and rankings. The assumption is that even though the two conceptions are regarded as necessary, the two sets of data are regarded as incomparable (Jean-Paul and Martine, 2018, p. 40; Ravallion, 2011, pp. 246–247).

Unlike the low substitution approach, this solution does not provide a unified scale. In some contexts, such a scale may be considered a useful and efficient comparative communication tool. The pitfall of a dashboard is that, in many cases, the user cannot come up with a conclusion, a bottom line, regarding the ‘basic-level concept’ (the level of ‘social progress’). The two approaches, nevertheless, share a requirement of not allowing one aspect to compensate for a lack of the other.

Data-driven approaches

These approaches assume that combining subjective and objective indicators should be based on data-driven (statistical) procedures, such as factor analysis, MIMIC multiple indicators and multiple causes, structural equation models (SEM), etc. (Alaimo et al., 2020; Brulé and Maggino, 2017; Krishnakumar and Nagar, 2008).

Low substitution assumes that the particular combination should be resolved by normative reasoning and not by data-driven procedures (for a discussion, see Decancq and Lugo, 2013; Goertz, 2020, pp. 57–66; Goertz and Mahoney, 2012). This is not necessarily to reject the use of data-driven techniques to construct each of the ‘objective’ and ‘subjective’ components (as in Krishnakumar and Ballon, 2008; Busseri et al., 2007; Busseri, 2018).

Preference-based approaches

These approaches advise eliciting individuals’ preferences regarding social progress aspects weights, either by equivalent income or stated preferences and other techniques (Decancq et al., 2015; Fleurbaey and Blanchet, 2013, chapter 4; Benjamin et al., 2014).

Unlike the low substitution approach, preference-based approaches refer to the subjective point of view not necessarily as an essential component per se but as the procedure through which weights are ascribed to components.

High substitution

A vast part of the existing social progress indices recognizes the two aspects and represent them together in one common functional form and score that implicitly assumes a high degree of substitution between them (Munda and Nardo, 2009; Ravallion, 2010; OECD, 2011, p. 26).

The low substitution approach adds to this approach the insight that, in some contexts, each of the two aspects is necessary and cannot be compensated by the other. This is reflected in a different functional form (the CES function would be a useful tool in making this adjustment).