Image overlay surgery based on augmented reality: a systematic review

Laura Perez-Pachon*, Matthieu Poyade, Terry Lowe, Flora Gröning

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingChapter (peer-reviewed)peer-review

12 Citations (Scopus)
8 Downloads (Pure)


Augmented Reality (AR) applied to surgical guidance is gaining relevance in clinical practice. AR-based image overlay surgery (i.e. the accurate overlay of patient-specific virtual images onto the body surface) helps surgeons to transfer image data produced during the planning of the surgery (e.g. the correct resection margins of tissue flaps) to the operating room, thus increasing accuracy and reducing surgery times. We systematically reviewed 76 studies published between 2004 and August 2018 to explore which existing tracking and registration methods and technologies allow healthcare professionals and researchers to develop and implement these systems in-house. Most studies used non-invasive markers to automatically track a patient’s position, as well as customised algorithms, tracking libraries or software development kits (SDKs) to compute the registration between patient-specific 3D models and the patient’s body surface. Few studies combined the use of holographic headsets, SDKs and user-friendly game engines, and described portable and wearable systems that combine tracking, registration, hands-free navigation and direct visibility of the surgical site. Most accuracy tests included a low number of subjects and/or measurements and did not normally explore how these systems affect surgery times and success rates. We highlight the need for more procedure-specific experiments with a sufficient number of subjects and measurements and including data about surgical outcomes and patients’ recovery. Validation of systems combining the use of holographic headsets, SDKs and game engines is especially interesting as this approach allows to easily develop mobile AR applications, thus facilitating the implementation of AR-based image overlay surgery in clinical practice.
Original languageEnglish
Title of host publicationBiomedical Visualisation
Subtitle of host publicationAdvances in Experimental Medicine and Biology
EditorsPaul M. Rea
Number of pages21
ISBN (Electronic)9783030474836
ISBN (Print)9783030474829
Publication statusPublished - 2020

Publication series

NameAdvances in Experimental Medicine and Biology
ISSN (Print)0065-2598

Bibliographical note

We thank the staff of the Medical Library of the University of Aberdeen for their advice and Prof. Jennifer Cleland and Dr Jenny Gregory for discussion and support. This work was funded by the Roland Sutton Academic Trust (0053/R/17) and an Elphinstone PhD Scholarship from the University of Aberdeen.


  • Augmented Reality
  • Mixed Reality
  • Surgical Guidance
  • Surgical Navigation
  • Holographic Headsets
  • Head-Mounted Displays


Dive into the research topics of 'Image overlay surgery based on augmented reality: a systematic review'. Together they form a unique fingerprint.

Cite this