Reporting methodological search filter performance comparisons: a literature review

Jennifer Harbour; Cynthia Fraser; Carol Lefebvre; Julie Glanville; Sophie Beale; Charles Boachie; Steven Duffy; Rachael McCool; Lynne Smith

doi:10.1111/hir.12070

Reporting methodological search filter performance comparisons: a literature review

Jennifer Harbour
, Cynthia Fraser
, Carol Lefebvre
, Julie Glanville
, Sophie Beale
, Charles Boachie
, Steven Duffy
, Rachael McCool
, Lynne Smith

Aberdeen Centre for Evaluation

HEALTHCARE IMPROVEMENT SCOTLAND (NHS SCOTLAND)

Research output: Contribution to journal › Article › peer-review

17 Citations (Scopus)

9 Downloads (Pure)

Abstract

BACKGROUND: Methodological search filters are tools for retrieving database records reporting studies which use a specific research method. Choosing a filter is likely to be based on filter performance data. This review examines which measures are reported, and the way that filter performance is presented, in filter comparisons.

METHODS: Studies were identified from the current content and pending update (2010) of a filter website. Eligible studies compared two or more methodological search filters designed to identify randomised controlled trials, diagnostic test accuracy studies, systematic reviews or economic evaluations.

RESULTS: Eighteen studies met the inclusion criteria. The number of filters compared in a single study ranged from 2 to 38. The most commonly reported measures were sensitivity/recall and precision. All studies displayed results in tables and gave results as percentages or proportions. Two studies supplemented results tables with graphical displays of data: a bar graph of the proportion of retrieved and missed gold standard references per filter; a forest plot of the overall sensitivity and specificity of each filter.

CONCLUSIONS: Sensitivity/recall and precision are the most frequently reported performance measures. This review highlights the potential for presenting results in novel and innovative ways to aid filter selection.

Original language	English
Pages (from-to)	176-194
Number of pages	19
Journal	Health Information & Libraries Journal
Volume	31
Issue number	3
Early online date	1 Aug 2014
DOIs	https://doi.org/10.1111/hir.12070
Publication status	Published - Sept 2014

Bibliographical note

Keywords

bibliographic databases
information storage and retrieval
methodological filters
precision
recall
review
literature

Access to Document

10.1111/hir.12070

Harbour 2014
This is the accepted version of the following article: Harbour, J, Fraser, C, Lefebvre, C, Glanville, J, Beale, S, Boachie, C, Duffy, S, McCool, R & Smith, L 2014, 'Reporting methodological search filter performance comparisons: a literature review' Health Information & Libraries Journal, vol 31, no. 3, pp. 176-194., which has been published in final form at 10.1111/hir.12070,
Accepted author manuscript, 515 KB

Cite this

@article{a7ddfdb2b5dd43e59261367590d3f812,

title = "Reporting methodological search filter performance comparisons: a literature review",

abstract = "BACKGROUND: Methodological search filters are tools for retrieving database records reporting studies which use a specific research method. Choosing a filter is likely to be based on filter performance data. This review examines which measures are reported, and the way that filter performance is presented, in filter comparisons.METHODS: Studies were identified from the current content and pending update (2010) of a filter website. Eligible studies compared two or more methodological search filters designed to identify randomised controlled trials, diagnostic test accuracy studies, systematic reviews or economic evaluations.RESULTS: Eighteen studies met the inclusion criteria. The number of filters compared in a single study ranged from 2 to 38. The most commonly reported measures were sensitivity/recall and precision. All studies displayed results in tables and gave results as percentages or proportions. Two studies supplemented results tables with graphical displays of data: a bar graph of the proportion of retrieved and missed gold standard references per filter; a forest plot of the overall sensitivity and specificity of each filter.CONCLUSIONS: Sensitivity/recall and precision are the most frequently reported performance measures. This review highlights the potential for presenting results in novel and innovative ways to aid filter selection.",

keywords = "bibliographic databases, information storage and retrieval, methodological filters , precision, recall, review, literature",

author = "Jennifer Harbour and Cynthia Fraser and Carol Lefebvre and Julie Glanville and Sophie Beale and Charles Boachie and Steven Duffy and Rachael McCool and Lynne Smith",

note = "{\textcopyright} 2014 The authors. Health Information and Libraries Journal {\textcopyright} 2014 Health Libraries Journal.",

year = "2014",

month = sep,

doi = "10.1111/hir.12070",

language = "English",

volume = "31",

pages = "176--194",

journal = "Health Information \& Libraries Journal",

issn = "1471-1834",

publisher = "John Wiley and Sons Inc.",

number = "3",

}

TY - JOUR

T1 - Reporting methodological search filter performance comparisons

T2 - a literature review

AU - Harbour, Jennifer

AU - Fraser, Cynthia

AU - Lefebvre, Carol

AU - Glanville, Julie

AU - Beale, Sophie

AU - Boachie, Charles

AU - Duffy, Steven

AU - McCool, Rachael

AU - Smith, Lynne

PY - 2014/9

Y1 - 2014/9

N2 - BACKGROUND: Methodological search filters are tools for retrieving database records reporting studies which use a specific research method. Choosing a filter is likely to be based on filter performance data. This review examines which measures are reported, and the way that filter performance is presented, in filter comparisons.METHODS: Studies were identified from the current content and pending update (2010) of a filter website. Eligible studies compared two or more methodological search filters designed to identify randomised controlled trials, diagnostic test accuracy studies, systematic reviews or economic evaluations.RESULTS: Eighteen studies met the inclusion criteria. The number of filters compared in a single study ranged from 2 to 38. The most commonly reported measures were sensitivity/recall and precision. All studies displayed results in tables and gave results as percentages or proportions. Two studies supplemented results tables with graphical displays of data: a bar graph of the proportion of retrieved and missed gold standard references per filter; a forest plot of the overall sensitivity and specificity of each filter.CONCLUSIONS: Sensitivity/recall and precision are the most frequently reported performance measures. This review highlights the potential for presenting results in novel and innovative ways to aid filter selection.

AB - BACKGROUND: Methodological search filters are tools for retrieving database records reporting studies which use a specific research method. Choosing a filter is likely to be based on filter performance data. This review examines which measures are reported, and the way that filter performance is presented, in filter comparisons.METHODS: Studies were identified from the current content and pending update (2010) of a filter website. Eligible studies compared two or more methodological search filters designed to identify randomised controlled trials, diagnostic test accuracy studies, systematic reviews or economic evaluations.RESULTS: Eighteen studies met the inclusion criteria. The number of filters compared in a single study ranged from 2 to 38. The most commonly reported measures were sensitivity/recall and precision. All studies displayed results in tables and gave results as percentages or proportions. Two studies supplemented results tables with graphical displays of data: a bar graph of the proportion of retrieved and missed gold standard references per filter; a forest plot of the overall sensitivity and specificity of each filter.CONCLUSIONS: Sensitivity/recall and precision are the most frequently reported performance measures. This review highlights the potential for presenting results in novel and innovative ways to aid filter selection.

KW - bibliographic databases

KW - information storage and retrieval

KW - methodological filters

KW - precision

KW - recall

KW - review

KW - literature

U2 - 10.1111/hir.12070

DO - 10.1111/hir.12070

M3 - Article

C2 - 25082456

SN - 1471-1834

VL - 31

SP - 176

EP - 194

JO - Health Information & Libraries Journal

JF - Health Information & Libraries Journal

IS - 3

ER -

Reporting methodological search filter performance comparisons: a literature review

Abstract

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this