Inconsistent Application of the Reliable Change Index in Clinical Psychology Trials Undermines Comparability of Individual Change Estimates.

Abstract

The reliable change index (RCI) is a valuable idiographic tool used by clinicians and researchers to evaluate whether individual clients experienced statistically detectable change on a given measure. However, numerous RCI methods exist, and the popular Jacobson and Truax (1991) method is underspecified, raising the potential for inconsistent operationalization across studies. The present study evaluated how the RCI is operationalized in psychology clinical trials and explored how RCI operationalization can influence numerical thresholds. We conducted a methodological scoping review of four clinical psychology journals published from 2020 to 2023 and coded the operationalization of reliable change in psychology clinical trials. We then used descriptive data from a convenience sample of published psychometric studies to illustrate the impact of operationalization on reliable change thresholds. Among 226 clinical trials, 29 (13%) formally assessed reliable change. We identified at least seven distinct operationalizations of the RCI, including five distinct operationalizations of the same formula from Jacobson and Truax (1991). In illustrative examples, differences in the specific operationalization of this formula produced absolute RCI threshold discrepancies as large as 79%, or 1.02 standard deviation units. Operationalization of reliable change is highly inconsistent in clinical trials, which can harm comparability of results across studies. We recommend improved reporting and that clinical trials compute two RCIs. First, the Jacobson & Truax method-using the baseline standard deviation and internal consistency-supports clinician use and improves between-study comparability, though other sources of non-comparability remain. Second, one of several modern methods provides improved within-study RCI evaluation.