Generating Expressions that Refer to Visible Objects

Margaret Mitchell; Kees Van Deemter; Ehud Reiter

Generating Expressions that Refer to Visible Objects

Margaret Mitchell, Kees Van Deemter, Ehud Reiter

Computing Science

Johns Hopkins University

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

49 Citations (Scopus)

10 Downloads (Pure)

Abstract

We introduce a novel algorithm for generating referring expressions, informed by human and computer vision and designed to refer to visible objects. Our method separates absolute properties like color from relative properties like size to stochastically generate a diverse set of outputs. Expressions generated using this method are often overspecified and may be underspecified, akin to expressions produced by people. We call such expressions identifying descriptions. The algorithm outperforms the well-known Incremental Algorithm (Dale and Reiter, 1995) and the Graph- Based Algorithm (Krahmer et al., 2003; Viethen et al., 2008) across a variety of images in two domains. We additionally motivate an evaluation method for referring expression generation that takes the proposed algorithm's non-determinism into account.

Original language	English
Title of host publication	Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics
Subtitle of host publication	Human Language Technologies
Editors	Lucy Vanderwende, Hal Daumé III, Katrin Kirchhoff
Publisher	Association for Computational Linguistics (ACL)
Pages	1174-1184
Number of pages	11
ISBN (Print)	9781937284473
Publication status	Published - Jun 2013
Event	2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2013 - Atlanta, United States Duration: 9 Jun 2013 → 14 Jun 2013

Conference

Conference	2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2013
Country/Territory	United States
City	Atlanta
Period	9/06/13 → 14/06/13

Bibliographical note

Funding for this research has been provided by SICSA and ORSAS. We thank the anonymous reviewers for useful comments on this paper.

Access to Document

Generating Expressions that Refer to Visible Objects
ACL materials are Copyright © 1963-2018 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 License. https://creativecommons.org/licenses/by-nc-sa/3.0/
Final published version, 888 KBLicence: CC BY-NC-SA

http://www.aclweb.org/anthology/N13-1137Licence: CC BY-NC-SA

Cite this

Mitchell, M., Van Deemter, K., & Reiter, E. (2013). Generating Expressions that Refer to Visible Objects. In L. Vanderwende, H. Daumé III, & K. Kirchhoff (Eds.), Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1174-1184). Article N13-1137 Association for Computational Linguistics (ACL). http://www.aclweb.org/anthology/N13-1137

Generating Expressions that Refer to Visible Objects. / Mitchell, Margaret; Van Deemter, Kees; Reiter, Ehud.
Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ed. / Lucy Vanderwende; Hal Daumé III; Katrin Kirchhoff. Association for Computational Linguistics (ACL), 2013. p. 1174-1184 N13-1137.

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

Mitchell, M, Van Deemter, K & Reiter, E 2013, Generating Expressions that Refer to Visible Objects. in L Vanderwende, H Daumé III & K Kirchhoff (eds), Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies., N13-1137, Association for Computational Linguistics (ACL), pp. 1174-1184, 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2013, Atlanta, United States, 9/06/13. <http://www.aclweb.org/anthology/N13-1137>

Mitchell, Margaret ; Van Deemter, Kees ; Reiter, Ehud. / Generating Expressions that Refer to Visible Objects. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. editor / Lucy Vanderwende ; Hal Daumé III ; Katrin Kirchhoff. Association for Computational Linguistics (ACL), 2013. pp. 1174-1184

@inproceedings{d9e9212b42334aa1a0f7113de3af68f1,

title = "Generating Expressions that Refer to Visible Objects",

abstract = "We introduce a novel algorithm for generating referring expressions, informed by human and computer vision and designed to refer to visible objects. Our method separates absolute properties like color from relative properties like size to stochastically generate a diverse set of outputs. Expressions generated using this method are often overspecified and may be underspecified, akin to expressions produced by people. We call such expressions identifying descriptions. The algorithm outperforms the well-known Incremental Algorithm (Dale and Reiter, 1995) and the Graph- Based Algorithm (Krahmer et al., 2003; Viethen et al., 2008) across a variety of images in two domains. We additionally motivate an evaluation method for referring expression generation that takes the proposed algorithm's non-determinism into account.",

author = "Margaret Mitchell and {Van Deemter}, Kees and Ehud Reiter",

note = "Funding for this research has been provided by SICSA and ORSAS. We thank the anonymous reviewers for useful comments on this paper.; 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2013 ; Conference date: 09-06-2013 Through 14-06-2013",

year = "2013",

month = jun,

language = "English",

isbn = "9781937284473",

pages = "1174--1184",

editor = "Lucy Vanderwende and {Daum{\'e} III}, Hal and Katrin Kirchhoff",

booktitle = "Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

}

TY - GEN

T1 - Generating Expressions that Refer to Visible Objects

AU - Mitchell, Margaret

AU - Van Deemter, Kees

AU - Reiter, Ehud

N1 - Funding for this research has been provided by SICSA and ORSAS. We thank the anonymous reviewers for useful comments on this paper.

PY - 2013/6

Y1 - 2013/6

N2 - We introduce a novel algorithm for generating referring expressions, informed by human and computer vision and designed to refer to visible objects. Our method separates absolute properties like color from relative properties like size to stochastically generate a diverse set of outputs. Expressions generated using this method are often overspecified and may be underspecified, akin to expressions produced by people. We call such expressions identifying descriptions. The algorithm outperforms the well-known Incremental Algorithm (Dale and Reiter, 1995) and the Graph- Based Algorithm (Krahmer et al., 2003; Viethen et al., 2008) across a variety of images in two domains. We additionally motivate an evaluation method for referring expression generation that takes the proposed algorithm's non-determinism into account.

AB - We introduce a novel algorithm for generating referring expressions, informed by human and computer vision and designed to refer to visible objects. Our method separates absolute properties like color from relative properties like size to stochastically generate a diverse set of outputs. Expressions generated using this method are often overspecified and may be underspecified, akin to expressions produced by people. We call such expressions identifying descriptions. The algorithm outperforms the well-known Incremental Algorithm (Dale and Reiter, 1995) and the Graph- Based Algorithm (Krahmer et al., 2003; Viethen et al., 2008) across a variety of images in two domains. We additionally motivate an evaluation method for referring expression generation that takes the proposed algorithm's non-determinism into account.

UR - http://www.scopus.com/inward/record.url?scp=84908171705&partnerID=8YFLogxK

M3 - Published conference contribution

AN - SCOPUS:84908171705

SN - 9781937284473

SP - 1174

EP - 1184

BT - Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics

A2 - Vanderwende, Lucy

A2 - Daumé III, Hal

A2 - Kirchhoff, Katrin

PB - Association for Computational Linguistics (ACL)

T2 - 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2013

Y2 - 9 June 2013 through 14 June 2013

ER -

Generating Expressions that Refer to Visible Objects

Abstract

Conference

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this