Towards Automatic Cetacean Photo-Identification: A Framework for Fine-Grain, Few-Shot Learning in Marine Ecology

Cameron Trotter, Nick Wright, A. Stephen McGough, Matt Sharpe, Barbara Cheney, Mònica Arso Civil, Reny Tyson Moore, Jason Allen, Per Berggren

Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution


Photo-identification (photo-id) is one of the main non-invasive capture-recapture methods utilised by marine researchers for monitoring cetacean (dolphin, whale, and porpoise) populations. This method has historically been performed manually resulting in high workload and cost due to the vast number of images collected. Recently automated aids have been developed to help speed-up photo-id, although they are often disjoint in their processing and do not utilise all available identifying information. Work presented in this paper aims to create a fully automatic photo-id aid capable of providing most likely matches based on all available information without the need for data pre-processing such as cropping. This is achieved through a pipeline of computer vision models and post-processing techniques aimed at detecting cetaceans in unedited field imagery before passing them downstream for individual level catalogue matching. The system is capable of handling previously uncatalogued individuals and flagging these for investigation thanks to catalogue similarity comparison. We evaluate the system against multiple real-life photo-id catalogues, achieving mAP@IOU[0.5] = 0.91, 0.96 for the task of dorsal fin detection on catalogues from Tanzania and the UK respectively and 83.1, 97.5% top-10 accuracy for the task of individual classification on catalogues from the UK and USA.
Original languageEnglish
Title of host publicationProceedings - 2022 IEEE International Conference on Big Data, Big Data 2022
PublisherIEEE Explore
Number of pages8
ISBN (Electronic)978-1-6654-8046-8
ISBN (Print)978-1-6654-8045-1
Publication statusPublished - 26 Jan 2023
Event2022 IEEE International Conference on Big Data : IEEE BigData 2022 - Osaka, Japan
Duration: 17 Dec 202220 Dec 2022


Conference2022 IEEE International Conference on Big Data
Internet address


  • Few
  • Shot
  • Fine-Grain
  • Classification
  • Detection


Dive into the research topics of 'Towards Automatic Cetacean Photo-Identification: A Framework for Fine-Grain, Few-Shot Learning in Marine Ecology'. Together they form a unique fingerprint.

Cite this