Subgroup identification and membership prediction.

Abstract

In clinical trials, a treatment rarely benefits every patient, underscoring the need to identify subgroups that are more likely to respond. Traditional subgroup analysis approaches, including finite mixture and threshold models, often rely on stringent distributional assumptions and prespecified subgroup structures that may be unrealistic in practice. Moreover, the resulting subgroups can be difficult to interpret and may not generalize well to new patients. To address these challenges, we propose a new least-squares regression framework that accommodates flexible subgroup structure in heterogeneous data. Our model is distribution-free and allows subgroup membership to depend on covariates, while permitting both the number and the organization of coefficient groups to vary across covariates. Building on regularization, we develop a computationally efficient procedure to detect subgroup structure in linear regression coefficients and then use a support vector machine to recover the corresponding partitions, enabling subgroup membership prediction for future individuals. Relative to pairwise fused regularization, our approach substantially reduces computational complexity. We also establish theoretical guarantees for estimation of group-specific parameters and recovery of the underlying partitions. Simulation studies and a real-data application illustrate the practical effectiveness of the proposed method.