Barriers and enabling factors for error analysis in NLG research

Emiel Van Miltenburg, Miruna Clinciu, Ondřej Dušek, Dimitra Gkatzia, Stephanie Inglis, Leo Leppänen, Saad Mahamood, Stephanie Schoch, Craig Thomson, Luou Wen

Research output: Contribution to journalArticlepeer-review

3 Downloads (Pure)

Abstract

Earlier research has shown that few studies in Natural Language Generation (NLG) evaluate their system outputs using an error analysis, despite known limitations of automatic evaluation metrics and human ratings. This position paper takes the stance that error analyses should be encouraged, and discusses several ways to do so. This paper is not just based on our shared experience as authors, but we also distributed a survey as a means of public consultation. We provide an overview of existing barriers to carry out error analyses, and proposes changes to improve error reporting in the NLG literature.
Original languageEnglish
Number of pages22
JournalNorthern European Journal of Language Technology
Volume9
Issue number1
Early online date21 Feb 2023
DOIs
Publication statusPublished - 21 Feb 2023

Fingerprint

Dive into the research topics of 'Barriers and enabling factors for error analysis in NLG research'. Together they form a unique fingerprint.

Cite this