Augmented Behavioral Cloning from Observation

Juarez Monteiro, Nathan Gavenski, Roger Granada, Felipe Meneguzzi, Rodrigo Barros

Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution

5 Citations (Scopus)

Abstract

Imitation from observation is a computational technique that teaches an agent on how to mimic the behavior of an expert by observing only the sequence of states from the expert demonstrations. Recent approaches learn the inverse dynamics of the environment and an imitation policy by interleaving epochs of both models while changing the demonstration data. However, such approaches often get stuck into sub-optimal solutions that are distant from the expert, limiting their imitation effectiveness. We address this problem with a novel approach that overcomes the problem of reaching bad local minima by exploring: (i) a self-attention mechanism that better captures global features of the states; and (ii) a sampling strategy that regulates the observations that are used for learning. We show empirically that our approach outperforms the state-of-the-art approaches in four different environments by a large margin.

Original languageEnglish
Title of host publication2020 International Joint Conference on Neural Networks, IJCNN 2020 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728169262
DOIs
Publication statusPublished - Jul 2020
Event2020 International Joint Conference on Neural Networks, IJCNN 2020 - Virtual, Glasgow, United Kingdom
Duration: 19 Jul 202024 Jul 2020

Conference

Conference2020 International Joint Conference on Neural Networks, IJCNN 2020
Country/TerritoryUnited Kingdom
CityVirtual, Glasgow
Period19/07/2024/07/20

Bibliographical note

Funding Information:
This study was financed in part by the Coordenac¸ão de Aperfeic¸oamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001, and CAPES/FAPERGS agreement (DOCFIX 04/2018) process number 18/2551-0000500-2. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the graphics cards used for this research.

Publisher Copyright:
© 2020 IEEE.

Keywords

  • Behavioral Cloning
  • Deep Learning
  • Imitation Learning
  • Learning from Demonstration

Fingerprint

Dive into the research topics of 'Augmented Behavioral Cloning from Observation'. Together they form a unique fingerprint.

Cite this