Task-Based Evaluation of NLG Systems: Control vs Real-World Context

Ehud Baruch Reiter

Task-Based Evaluation of NLG Systems: Control vs Real-World Context

Ehud Baruch Reiter

Computing Science

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

Abstract

Currently there is little agreement about, or even discussion of, methodologies for taskbased evaluation of NLG systems. I discuss one specific issue in this area, namely the importance of control vs the importance of ecological validity (real-world context), and suggest that perhaps we need to put more emphasis on ecological validity in NLG evaluations.

Original language	English
Title of host publication	Proceedings of UCNLG+Eval 2011
Publisher	Association for Computational Linguistics
Pages	28-32
Number of pages	5
Publication status	Published - 2011
Event	UCNLG+Eval: Language Generation and Evaluation Workshop - Edinburgh, United Kingdom Duration: 21 Jul 2011 → 21 Jul 2011

Conference

Conference	UCNLG+Eval: Language Generation and Evaluation Workshop
Country/Territory	United Kingdom
City	Edinburgh
Period	21/07/11 → 21/07/11

Access to Document

http://aclweb.org/anthology-new/W/W11/W11-2704.pdf

Cite this

@inproceedings{016fb154f30f465fa28e707c2556451b,

title = "Task-Based Evaluation of NLG Systems: Control vs Real-World Context",

abstract = "Currently there is little agreement about, or even discussion of, methodologies for taskbased evaluation of NLG systems. I discuss one specific issue in this area, namely the importance of control vs the importance of ecological validity (real-world context), and suggest that perhaps we need to put more emphasis on ecological validity in NLG evaluations.",

author = "Reiter, \{Ehud Baruch\}",

year = "2011",

language = "English",

pages = "28--32",

booktitle = "Proceedings of UCNLG+Eval 2011",

publisher = "Association for Computational Linguistics",

note = "UCNLG+Eval: Language Generation and Evaluation Workshop ; Conference date: 21-07-2011 Through 21-07-2011",

}

TY - GEN

T1 - Task-Based Evaluation of NLG Systems

T2 - UCNLG+Eval: Language Generation and Evaluation Workshop

AU - Reiter, Ehud Baruch

PY - 2011

Y1 - 2011

N2 - Currently there is little agreement about, or even discussion of, methodologies for taskbased evaluation of NLG systems. I discuss one specific issue in this area, namely the importance of control vs the importance of ecological validity (real-world context), and suggest that perhaps we need to put more emphasis on ecological validity in NLG evaluations.

AB - Currently there is little agreement about, or even discussion of, methodologies for taskbased evaluation of NLG systems. I discuss one specific issue in this area, namely the importance of control vs the importance of ecological validity (real-world context), and suggest that perhaps we need to put more emphasis on ecological validity in NLG evaluations.

M3 - Published conference contribution

SP - 28

EP - 32

BT - Proceedings of UCNLG+Eval 2011

PB - Association for Computational Linguistics

Y2 - 21 July 2011 through 21 July 2011

ER -

Task-Based Evaluation of NLG Systems: Control vs Real-World Context

Abstract

Conference

Access to Document

Fingerprint

Cite this