Bootstrapping Relational Affordances of Object Pairs using Transfer

Severin Fichtl, Dirk Kraft, Norbert Krüger, Frank Guerin

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)
16 Downloads (Pure)


Robots acting in everyday environments need a good knowledge of how a manipulation action can affect pairs of objects in a relationship, such as ‘inside’ or ‘behind’ or ‘ontop’. These relationships afford certain means-end actions such as pulling a container to retrieve the contents, or pulling a tool to retrieve a desired object. We investigate how these relational affordances could be learnt by a robot from its own action experience. A major challenge in this approach is to reduce the number of training samples needed to achieve accuracy, and hence we investigate an approach which can leverage past knowledge to accelerate current learning (which we call bootstrapping). We learn Random Forest based affordance predictors from visual inputs and demonstrate two approaches to knowledge transfer for bootstrapping. In the first approach (direct bootstrapping), the state-space for a new affordance predictor is augmented with the output of previously learnt affordances. In the second approach (category based bootstrapping), we form categories that capture underlying commonalities of a pair of existing affordances and augment the state-space with this category classifier’s output. In addition, we introduce a novel heuristic, which suggests how a
large set of potential affordance categories can be pruned to leave only those categories which are most promising for bootstrapping future affordances. Our results show that both bootstrapping approaches outperform learning without bootstrapping. We also show that there is no significant difference in performance between direct and category based bootstrapping.
Original languageEnglish
Pages (from-to)56-71
Number of pages16
JournalIEEE Transactions on Cognitive and Developmental Systems
Issue number1
Early online date11 Oct 2016
Publication statusPublished - 9 Mar 2018

Bibliographical note

This work was supported in part by the U.K. EPSRC DTG EP/J5000343/1 at Aberdeen, and in part by the EU Cognitive Systems Project XPERIENCE at SDU under Grant FP7-ICT-270273.


  • Robot sensing systems
  • planning
  • visualisation
  • containers
  • acceleration
  • service robots


Dive into the research topics of 'Bootstrapping Relational Affordances of Object Pairs using Transfer'. Together they form a unique fingerprint.

Cite this