TY - GEN
T1 - Unsupervised extraction of keywords from news archives
AU - Palomino, Marco A.
AU - Wuytack, Tom
PY - 2011/3/22
Y1 - 2011/3/22
N2 - We present a comparison of four unsupervised algorithms to automatically acquire the set of keywords that best characterise a particular multimedia archive: the Belga News Archive. Such keywords provide the basis of a controlled vocabulary for indexing the pictures in this archive. Our comparison shows that the most successful algorithm is TextRank, derived from Google’s PageRank, which determines the importance of a word by the number of words with which it co-occurs, and the relative importance of those co-occurring words. Next most successful is information radius, originally used to estimate the overall semantic distance between two corpora, but here adapted to examine the contributions of individual words to that overall distance. Third most successful was the chi-square test, which determined which keywords were more typical of Belga’s Archive than a representative corpus of English language. Finally, the least successful approach was the use of raw frequency, whereby the most frequent words were the most important ones, unless they were present in a stop-word list. All four algorithms are readily portable to other domains and languages, though TextRank has the advantage that it does not require a comparison corpus.
AB - We present a comparison of four unsupervised algorithms to automatically acquire the set of keywords that best characterise a particular multimedia archive: the Belga News Archive. Such keywords provide the basis of a controlled vocabulary for indexing the pictures in this archive. Our comparison shows that the most successful algorithm is TextRank, derived from Google’s PageRank, which determines the importance of a word by the number of words with which it co-occurs, and the relative importance of those co-occurring words. Next most successful is information radius, originally used to estimate the overall semantic distance between two corpora, but here adapted to examine the contributions of individual words to that overall distance. Third most successful was the chi-square test, which determined which keywords were more typical of Belga’s Archive than a representative corpus of English language. Finally, the least successful approach was the use of raw frequency, whereby the most frequent words were the most important ones, unless they were present in a stop-word list. All four algorithms are readily portable to other domains and languages, though TextRank has the advantage that it does not require a comparison corpus.
UR - https://www.scopus.com/pages/publications/79953168037
U2 - 10.1007/978-3-642-20095-3_50
DO - 10.1007/978-3-642-20095-3_50
M3 - Published conference contribution
SN - 978-3-642-20094-6
T3 - Lecture Notes in Computer Science
SP - 544
EP - 555
BT - Human Language Technology. Challenges for Computer Science and Linguistics
A2 - Vetulani, Z.
PB - Springer
CY - Berlin, Heidelberg
ER -