Performance of a risk score for predicting preterm pre‐eclampsia

Brunelli et al (BJOG 2020) evaluated a previously validated maternal history -based risk score in a multicentric retrospective cohort from Italy. The risk score discriminated poorly in this study (AUC=0.659 [95% CI 0.579-0.726]) but it discriminated well in the Pregnancy Outcome Prediction (POP) study (AUC=0.846 [95% CI 0.787-0.906]) (Sovio et al, BJOG 2019;126:963-970), although the proportion of women who developed preterm pre-eclampsia was similar in both studies (1.0% and 0.7%, respectively). What could be the possible reasons for these differences and what should be the next steps for evaluating such risk scores?

Brunelli et al. (BJOG 2020;127:1210-15) evaluated a previously validated maternal history-based risk score in a multicentric retrospective cohort from Italy. The risk score discriminated poorly in this study (area under the curve [AUC] = 0.659, 95% CI 0.579-0.726) but it discriminated well in the Pregnancy Outcome Prediction (POP) study (AUC = 0.846, 95% CI 0.787-0.906) (Sovio et al. BJOG 2019;126:963-70), although the proportion of women who developed preterm pre-eclampsia was similar in both studies (1.0 and 0.7%, respectively). What could be the possible reasons for these differences and what should be the next steps for evaluating such risk scores?
The prevalence of the maternal risk factors that constitute the risk score was very low in this study. Chronic hypertension has a major contribution, i.e. it strongly predicts (superimposed) preterm pre-eclampsia. Only 0.5% of all the women in the study and 15% of the cases had chronic hypertension, whereas in the POP study, the prevalences were 5 and 35%, respectively (Sovio et al. Hypertension 2017;69:731-8). An analysis of the SCOPE study (Myers. Pregnancy Hypertension 2019;17:S21) illustrated that the performance of the risk score was similarly poor in a healthy nulliparous population which excluded all women with chronic hypertension (AUC = 0.661, 95% CI 0.596-0.725). This is not surprising.
In this study from Italy, it was not clear how representative the cohort was of the source population. In this regard, comparisons between recruited women and eligible nonrecruited women would help in assessing selection bias, but these are not always possible due to limitations in data or ethical approvals. Misclassification may also contribute to the poor performance of the risk score. In the POP study, pre-eclampsia was ascertained through a careful review of case records and linkage to electronic databases (Sovio et al. Hypertension 2017;69:731-8). In the study from Italy, electronic data were used as a basis of defining preeclampsia, and data quality issues should always be considered and discussed. Moreover, measurement error or misreporting of risk factors or aspirin use could have increased random error or bias in the risk score.
Future evaluations of the risk score should be performed in datasets representative of the underlying population in terms of prior maternal risk. Only the women undergoing interventions which may alter the risk of preterm pre-eclampsia should be excluded. The risk score has already been validated against the model for predicted gestational age at preeclampsia (PGAPE) in nulliparous women ( (BJOG 2020; https://doi.org/10.1111/ 1471-0528.16246) did not attempt to validate the risk score against PGAPE but instead compared it with the full FMF algorithm, the latter performing better. The need for additional measurements to those included in the risk score was expected, given the low prevalence of prior maternal risk factors (Myers. Pregnancy Hypertension 2019;17:S21).
A suggested scoring for maternal risk has been published for parous women (Sovio et al. BJOG 2019;126:963-70). Evaluation of the risk score and validation against PGAPE in parous women is now needed, as this could not be done in either the POP study or the SCOPE study.

Disclosure of interests
Dr Sovio reports grants from NIHR Cambridge Biomedical Research Centre during the conduct of the study. In addition, Dr Sovio has a patent application for a novel predictive test for fetal growth restriction (FGR) pending. A completed disclosure of interest form is available to view online as supporting information. &