Tablet versus paper marking in assessment: feedback matters

Alan Denison; Emily Bate; Jessica Thompson

doi:10.1007/s40037-016-0262-8

Tablet versus paper marking in assessment: feedback matters

Alan Denison^* (Corresponding Author), Emily Bate, Jessica Thompson

^*Corresponding author for this work

Medical Education

University of Aberdeen

Research output: Contribution to journal › Article › peer-review

20 Citations (Scopus)

6 Downloads (Pure)

Abstract

Background

The Objective Structured Clinical Examination (OSCE) is a cornerstone in healthcare assessment. As a potential tool for providing learner-centred feedback on a large scale, the use of tablet devices has been proposed for the recording of OSCE marks, moving away from the traditional, paper-based checklist.

Methods

Examiner-recorded comments were collated from successive first year formative and summative OSCE examinations, with paper-based checklists used in 2012 and iPad-based checklists used in 2013. A total of 558 and 498 examiner-candidate interactions took place in the January OSCE examinations, and 1402 and 1344 for the May OSCE examination for 2012 and 2013 respectively. Examiner comments were analyzed for quantity and quality. A tool was developed and validated to assess the quality of the comments left by examiners for use as feedback (Kappa = 0.625).

Results

A direct comparison of paper-based checklists and iPad-recorded examinations showed an increase in the quantity of comments left from 41 to 51 % (+ 10 %). Furthermore, there was an increase in the number of comments left for students deemed ‘borderline’: + 22 %. In terms of the quality of the comments for feedback, there was a significant improvement (p < 0.001) between comments left in written-recorded and iPad-recorded examinations.

Conclusions

iPad-marked examinations resulted in a greater quantity and quality of examiner comment for use as feedback, particularly for students performing less well, enabling tutors to direct further learning for these students.

Original language	English
Pages (from-to)	108-113
Number of pages	6
Journal	Perspectives on medical education
Volume	5
Issue number	2
Early online date	14 Mar 2016
DOIs	https://doi.org/10.1007/s40037-016-0262-8
Publication status	Published - 30 Apr 2016

Bibliographical note

University of Aberdeen Medical Education Summer Teaching Bursary.

Keywords

Assessment
Feedback
OSCE
Technology
Undergraduate medical education

Access to Document

10.1007/s40037-016-0262-8Licence: CC BY

Tablet versus paper marking in assessment: feedback matters
© The Author(s) 2016 This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, duplication, adaptation, distribution, and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Final published version, 248 KBLicence: CC BY

Cite this

@article{adb4928b0f414ee7a9f225c726644b05,

title = "Tablet versus paper marking in assessment: feedback matters",

abstract = "BackgroundThe Objective Structured Clinical Examination (OSCE) is a cornerstone in healthcare assessment. As a potential tool for providing learner-centred feedback on a large scale, the use of tablet devices has been proposed for the recording of OSCE marks, moving away from the traditional, paper-based checklist.MethodsExaminer-recorded comments were collated from successive first year formative and summative OSCE examinations, with paper-based checklists used in 2012 and iPad-based checklists used in 2013. A total of 558 and 498 examiner-candidate interactions took place in the January OSCE examinations, and 1402 and 1344 for the May OSCE examination for 2012 and 2013 respectively. Examiner comments were analyzed for quantity and quality. A tool was developed and validated to assess the quality of the comments left by examiners for use as feedback (Kappa = 0.625).ResultsA direct comparison of paper-based checklists and iPad-recorded examinations showed an increase in the quantity of comments left from 41 to 51 % (+ 10 %). Furthermore, there was an increase in the number of comments left for students deemed {\textquoteleft}borderline{\textquoteright}: + 22 %. In terms of the quality of the comments for feedback, there was a significant improvement (p < 0.001) between comments left in written-recorded and iPad-recorded examinations.ConclusionsiPad-marked examinations resulted in a greater quantity and quality of examiner comment for use as feedback, particularly for students performing less well, enabling tutors to direct further learning for these students.",

keywords = "Assessment, Feedback, OSCE, Technology, Undergraduate medical education",

author = "Alan Denison and Emily Bate and Jessica Thompson",

note = "University of Aberdeen Medical Education Summer Teaching Bursary.",

year = "2016",

month = apr,

day = "30",

doi = "10.1007/s40037-016-0262-8",

language = "English",

volume = "5",

pages = "108--113",

journal = "Perspectives on medical education",

issn = "2212-2761",

publisher = "Bohn Stafleu van Loghum",

number = "2",

}

TY - JOUR

T1 - Tablet versus paper marking in assessment

T2 - feedback matters

AU - Denison, Alan

AU - Bate, Emily

AU - Thompson, Jessica

N1 - University of Aberdeen Medical Education Summer Teaching Bursary.

PY - 2016/4/30

Y1 - 2016/4/30

N2 - BackgroundThe Objective Structured Clinical Examination (OSCE) is a cornerstone in healthcare assessment. As a potential tool for providing learner-centred feedback on a large scale, the use of tablet devices has been proposed for the recording of OSCE marks, moving away from the traditional, paper-based checklist.MethodsExaminer-recorded comments were collated from successive first year formative and summative OSCE examinations, with paper-based checklists used in 2012 and iPad-based checklists used in 2013. A total of 558 and 498 examiner-candidate interactions took place in the January OSCE examinations, and 1402 and 1344 for the May OSCE examination for 2012 and 2013 respectively. Examiner comments were analyzed for quantity and quality. A tool was developed and validated to assess the quality of the comments left by examiners for use as feedback (Kappa = 0.625).ResultsA direct comparison of paper-based checklists and iPad-recorded examinations showed an increase in the quantity of comments left from 41 to 51 % (+ 10 %). Furthermore, there was an increase in the number of comments left for students deemed ‘borderline’: + 22 %. In terms of the quality of the comments for feedback, there was a significant improvement (p < 0.001) between comments left in written-recorded and iPad-recorded examinations.ConclusionsiPad-marked examinations resulted in a greater quantity and quality of examiner comment for use as feedback, particularly for students performing less well, enabling tutors to direct further learning for these students.

AB - BackgroundThe Objective Structured Clinical Examination (OSCE) is a cornerstone in healthcare assessment. As a potential tool for providing learner-centred feedback on a large scale, the use of tablet devices has been proposed for the recording of OSCE marks, moving away from the traditional, paper-based checklist.MethodsExaminer-recorded comments were collated from successive first year formative and summative OSCE examinations, with paper-based checklists used in 2012 and iPad-based checklists used in 2013. A total of 558 and 498 examiner-candidate interactions took place in the January OSCE examinations, and 1402 and 1344 for the May OSCE examination for 2012 and 2013 respectively. Examiner comments were analyzed for quantity and quality. A tool was developed and validated to assess the quality of the comments left by examiners for use as feedback (Kappa = 0.625).ResultsA direct comparison of paper-based checklists and iPad-recorded examinations showed an increase in the quantity of comments left from 41 to 51 % (+ 10 %). Furthermore, there was an increase in the number of comments left for students deemed ‘borderline’: + 22 %. In terms of the quality of the comments for feedback, there was a significant improvement (p < 0.001) between comments left in written-recorded and iPad-recorded examinations.ConclusionsiPad-marked examinations resulted in a greater quantity and quality of examiner comment for use as feedback, particularly for students performing less well, enabling tutors to direct further learning for these students.

KW - Assessment

KW - Feedback

KW - OSCE

KW - Technology

KW - Undergraduate medical education

UR - http://www.scopus.com/inward/record.url?scp=85012006451&partnerID=8YFLogxK

U2 - 10.1007/s40037-016-0262-8

DO - 10.1007/s40037-016-0262-8

M3 - Article

AN - SCOPUS:85012006451

SN - 2212-2761

VL - 5

SP - 108

EP - 113

JO - Perspectives on medical education

JF - Perspectives on medical education

IS - 2

ER -

Tablet versus paper marking in assessment: feedback matters

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this