Development and Validation of Human-AI Collaborative Workflow in SNOMED CT Mapping of Bilingual Clinical Text.

Abstract

Systematized Nomenclature of Medicine-Clinical Terminology (SNOMED CT) is the principal international standard for semantic interoperability of clinical information, but mapping free-text clinical narratives to SNOMED CT concepts remains labor-intensive. We developed a large language model agent system for mapping bilingual clinical text to SNOMED CT concepts and evaluated its effect on mapping accuracy and efficiency within a human-AI collaborative workflow. We designed a three-module agent system comprising translation, abbreviation expansion, and vector-based retrieval components, integrated with a pre-embedded SNOMED CT vector database. Three health information managers independently mapped bilingual clinical text segments using three approaches: human-only, Agent-only, and Agent-assisted human mapping. Performance was evaluated by using hit rate, precision, recall, and F1 score at k = 1 and 5, and R-precision. Mapping time was compared between human-only and human-AI collaborative approaches. A total of 2,261 de-identified clinical text segments across nine clinical categories were collected at a tertiary academic hospital in South Korea. The human-AI collaborative workflow, which expanded the set of valid SNOMED CT candidates presented at each mapping decision, raised pooled hit rate@1 from 0.837 to 0.868 (difference 0.031, 95% confidence interval [CI] 0.021 to 0.042; p < 0.001), raised R-precision from 0.632 to 0.674 (difference 0.042, 95% CI 0.034 to 0.051; p < 0.001), and reduced total mapping time by 53.9% (from 1.57 to 0.72 min per segment, including agent processing). By expanding the space of valid SNOMED CT candidates available to expert mappers, the human-AI collaborative approach improved SNOMED CT mapping accuracy while reducing time by about half. Its modular architecture, supporting periodic vector database updates without retraining, offers a sustainable and efficient solution for bilingual clinical terminology standardization.