User-Driven Research of Medical Note Generation Software

Tom Knoll, Francesco Moramarco, Alex Papadopoulos Korfiatis, Rachel Young, Claudia Ruffini, Mark Perera, Christian Perstl, Ehud Reiter, Anja Belz, Aleksandar Savkov

Research output: Working paper


A growing body of work uses Natural Language Processing (NLP) methods to automatically generate medical notes from audio recordings of doctor-patient consultations.
However, there are very few studies on how
such systems could be used in clinical practice,
how clinicians would adjust to using them, or
how system design should be influenced by
such considerations. In this paper, we present
three rounds of user studies, carried out in the
context of developing a medical note generation system. We present, analyse and discuss
the participating clinicians’ impressions and
views of how the system ought to be adapted
to be of value to them. Next, we describe a
three-week test run of the system in a live telehealth clinical practice. Major findings include
(i) the emergence of five different note-taking
behaviours; (ii) the importance of the system
generating notes in real time during the consultation; and (iii) the identification of a number
of clinical use cases that could prove challenging for automatic note generation systems.
Original languageEnglish
Number of pages13
Publication statusPublished - 5 May 2022

Bibliographical note

We would like to thank all the clinicians who participated in the UX studies and the live test of the Note Generation system. We are grateful to Oriol Valldeperas for the beautiful diagram of the Wizard of Oz setup. Lastly, a warm ‘thank you’ to all the
patients who have consented to their consultations being observed for the purpose of this study


Dive into the research topics of 'User-Driven Research of Medical Note Generation Software'. Together they form a unique fingerprint.

Cite this