A Model-Based Approach to Finding Substitute Tools in 3D Vision Data

Paulo Abelha Ferreira, Frank Guerin, Markus Schoeler

Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution

24 Citations (Scopus)
13 Downloads (Pure)


A robot can feasibly be given knowledge of a set of tools for manipulation activities (e.g. hammer, knife, spatula). If the robot then operates outside a closed environment it is likely to face situations where the tool it knows is not available, but alternative unknown tools are present. We tackle the problem of finding the best substitute tool based solely on 3D vision data. Our approach has simple hand-coded models of known tools in terms of superquadrics and relationships among them. Our system attempts to fit these models to point clouds of unknown tools, producing a numeric value for how good a fit is. This value can be used to rate candidate substitutes. We explicitly control how closely each part of a tool must match our model, under direction from parameters of a target task. We allow bottom-up information from segmentation to dictate the sizes that should be considered for various parts of the tool. These ideas allow for a flexible matching so that tools may be superficially quite different, but similar in the way that matters. We evaluate our system's ratings relative to other approaches and relative to human performance in the same task. This is an approach to knowledge transfer, via a suitable representation and reasoning engine, and we discuss how this could be extended to transfer in planning.
Original languageEnglish
Title of host publication2016 IEEE International Conference on Robotics and Automation (ICRA 2016)
PublisherIEEE Press
ISBN (Print)9781467380270
Publication statusPublished - 2016

Bibliographical note

Paulo Abelha is supported by the Brazilian agency CAPES through the program Science without Borders. Thanks to the University of Aberdeen’s ABVenture Zone for equipment use. We got a lot of very helpful advice and assistance from the following people: Severin Fichtl, Dirk Kraft, Norbert Kruger, Karthik Varadarajan, Markus Vincze, Florentin ¨ Worg ¨ otter.


  • three-dimensional (3-D) displays
  • shape
  • data models
  • robots
  • computational modeling
  • solid modeling
  • numerical models


Dive into the research topics of 'A Model-Based Approach to Finding Substitute Tools in 3D Vision Data'. Together they form a unique fingerprint.

Cite this