Table 1 Dimensions of validity assessed with hypotheses, evidence sources, and statistical criteria

From: Cumulative psychosocial risk and early child development: validation and use of the Childhood Psychosocial Adversity Scale in global health research

Validation proposition: Use of the CPAS as a research measure assessing early psychosocial adversity as a child developmental risk factor among low-SES, urban Bangladeshi children aged 18–60 months

Validity dimension

A priori hypotheses

Sources of evidence

Statistical criteria

Construct validity

 Scientific soundness of measured construct

(a) Cumulative early psychosocial stress, influenced by child and caregiver experiences, shapes human development

(b) The scientific construct of psychosocial adversity has locally specific and embedded manifestations in Mirpur

Literature review

Not applicable

Expert review

Content validity

 Extent to which content captures construct

(a) Item content captures major themes in the conceptual model without extraneous content

(b) Factor analysis will support a subscale structure corresponding to the conceptual model

Expert review

Significant item loading (e.g., ≥0.4) on primary factors, minimal cross-loading30

Cognitive pretesting

EFAs

Internal consistency

 Content cohesion

Subscales will show good internal consistency after final item selection

Cronbach’s α within subscales

Cronbach’s α ≥ 0.729

Test–retest and inter-rater reliability

 Stability of scores over time/raters

(a) Total scores will have acceptable test–retest reliability

(b) Total scores will have acceptable inter-rater reliability, with lower reliability than for test–retest administrations due to layered variance related to rater and occasion

Retests over 2-week interval with same interviewer (test–retest) or different (inter-rater)

Average ICC ≥ 0.75 for test–retest (excellent), ≥0.60 for inter-rater (good)31

Convergent and discriminant validity

 Agreement with similar & distinctness from dissimilar measures

(a) Subscales scores will correlate with similar instruments, likely only moderately given non-identical constructs

(b) Comparator instrument scores will correlate more strongly with associated CPAS subscale than with CPAS total score

Data from CPAS and comparator instruments

Pearson’s r with p < 0.05

Predictive validity

 Association with outcomes

Full-scale and subscale scores significantly predict future child cognitive performance, both in bivariate analyses and when controlling for other risks.

48-month CPAS and 60-month WPPSI-IV scores

Pearson’s r with p < 0.05

Incremental validity

 Extent of novel value

(a) The CPAS will explore new domains of psychosocial risks while taking less time to administer than related instruments

(b) It will show similar or better internal consistency

Age-matched data from CPAS and comparator measures

Cronbach’s α greater for CPAS greater than comparator measures

  1. CPAS Global Child Adversity Scale, EFA exploratory factor analysis, ICC  intraclass correlation coefficient, SES socioeconomic status, WPPSI-IV Wechsler Preschool and Primary Scale of Intelligence, 4th Ed