[MA 2024 21] Patient-friendly terminology with large language models
ChipSoft, Department of Research & Development, Amsterdam
Proposed by: Hugo van Mens, PhD [h.van.mens@chipsoft.com]
Introduction
Patients access their health information, receive patient education, schedule healthcare appointments, answer medical questionnaires and interact with healthcare professionals through patient portals and personal health records. However, medical records are full of jargon: controlled vocabularies, such as SNOMED CT and LOINC codes, and free-text content, such as treatment reports and discharge letters. These might contain abbreviations, acronyms, typo’s and a mix of medical Dutch, English, Latin and Greek terms. A large group of patients experiences difficulty in understanding and using health information to the benefit of their own health. Medical jargon can alienate patients and can sometimes be considered confusing or even offending.
Description of the SRP Project/Problem
Some terminology systems offer patient-friendly or consumer-oriented terms and definitions to clarify medical data. In the Netherlands, patient-friendly terms and definitions for SNOMED CT concepts are developed in a costly and laborious way by a translation agency and an extensive validation process with feedback from terminologists, clinicians and patient groups. Many of these descriptions are repetitive and follow regular patterns. For example, primary malignant neoplasm of pancreas can be clarified with the Dutch term “alvleesklierkanker” and the Dutch definition “kwaadaardige tumor in de alvleesklier”. This follows the pattern [morphologic abnormality] of [finding site]. This can thus be applied to many of the thousands of neoplasms that are registered in Dutch hospitals, but also to inflammations, fractures and bleedings, which are fully defined by their location and morphology. Large language models, possibly in combination with knowledge graphs such as SNOMED CT, hence offer a viable and more maintainable alternative to manual labour in generating quality patient-friendly descriptions. There is some controversy, however, about the quality of the results. It is important to avoid providing incorrect information to patients. Therefore, such clarifications need to be efficiently validated and evaluated by terminologists, clinicians and patients as well in a reliable manner.
Research questions
What are requirements for accessible, easy-to-understand, patient-friendly or consumer-oriented clarifications of medical concepts?
What is the quality of clarifications of medical concepts by state of the art large language models?
How does this quality compare to golden standard, manually developed clarifications by translation agencies?
What are the requirements for the validation and evaluation of such clarifications before using them in clinical practice?
Expected results
Clarifications of medical concepts from one or more medical terminology systems, with two or more state of the art LLM’s
Evaluation of the performance of clarification quality of each LLM and terminology system
A comparison of the quality of clarifications by LLM’s with clarifications by a manual translation process
An in-depth analysis of the issues encountered
Recommendations for improvements of the clarifications, for the use of LLM’s in the translation process and for the implementation of the clarifications in the electronic health record in clinical practice
Time period
Seven months: e.g. December – June or May – January.
Contact
Hugo van Mens, PhD [h.van.mens@chipsoft.com, h.j.vanmens@amsterdamumc.nl]
Software engineer, Department of Research & Development, ChipSoft, Amsterdam.
Post-doctoral researcher, Department of Medical Informatics, Amsterdam UMC, location AMC.
References
van Mens HJT, van Eysden MM, Nienhuis R, van Delden JJM, de Keizer NF, Cornet R. Evaluation of lexical clarification by patients reading their clinical notes: a quasi-experimental interview study. BMC Medical Informatics and Decision Making. 2020 Dec 15;20(Suppl 10):278. DOI: 10.1186/s12911-020-01286-9.
van Mens HJT, Martens SSM, Paiman EHM, Mertens AC, Nienhuis R, de Keizer NF, Cornet R. Diagnosis clarification by generalization to patient-friendly terms and definitions: validation study. Journal of Biomedical Informatics. 2022 May;129:104071. DOI: 10.1016/j.jbi.2022.104071.
van Mens HJT, Hannen GEG, Nienhuis R, Bolt RJ, de Keizer NF, Cornet R. Evaluation of patient-friendly diagnosis clarifications in a hospital patient portal. Applied Clinical Informatics. 2023 Apr 1. DOI: 10.1055/a-2067-5310.