Improving readability of layperson abstracts and summaries in oncology using task-specific large language model powered tool: results from the BRIDGE-AI 7 study.

Abstract

To compare the performance of task-specific generative AI, with general-purpose large language models (LLMs) in generating more readable lay abstracts and summaries (LASs) of Oncology research. Twenty-five randomly selected abstracts from the top 5 journals in Oncology were processed into LASs using a task specific LLM-powered tool (Pub2Post) and 5 general-purpose LLMs (ChatGPT-5, Claude, Gemini, DeepSeek and Grok). Two prompting strategies (Specific and Generic) were applied. Consistency was tested across 3 outputs, producing a total of 825 LASs. Readability-scores and text-metrics were calculated. The "best test" per model was selected based on lowest SMOG Index, which was subsequently used to compare the 6 GAI platforms. Comparisons were performed using Kruskal-Wallis tests, with significance set at <i>P</i>&#x2009;&lt;&#x2009;.05. All the platforms demonstrated consistent intra-model outputs across triplicate generations (all <i>P</i>&#x2009;&gt;&#x2009;.05). However, inter-model comparisons revealed significant differences (all <i>P</i>&#x2009;&lt;&#x2009;.001) with Pub2Post outperforming the LLMs, across the 2 prompting styles, demonstrating superior readability-scores (FRES 82.3; FKGL 5.2; GFS 6.6; SI 4.4; CLI 10.4; ARI 6.2) with longer outputs (27 sentences; 388 words) and fewer complex-words (3.7%). The general-purpose LLMs generated shorter, denser outputs (4-9 sentences; 81-156 words) with higher grade-levels (FRES 38.0-61.6; FKGL 9.6-13.4; GFS 10.6-15.9; SI 7.6-11.2; CLI 13.4-17.0; ARI 11.0-15.2). Task-specific GAI powered tools (Pub2Post) generated consistently more readable LASs compared to 5 commercially available LLMs, regardless of prompting strategy. These findings highlight the value of purpose-built GAI tools for enhancing public understanding and accessibility of oncology research, with implications for improving patient-education.