Regularized evolutionary algorithm for dynamic neural topology search

Cristiano Saltori; Subhankar Roy; Nicu Sebe; Giovanni Iacca

doi:10.48550/arXiv.1905.06252

Regularized evolutionary algorithm for dynamic neural topology search

Cristiano Saltori^*, Subhankar Roy, Nicu Sebe, Giovanni Iacca

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

9 Citations (Scopus)

Abstract

Designing neural networks for object recognition requires considerable architecture engineering. As a remedy, neuro-evolutionary network architecture search, which automatically searches for optimal network architectures using evolutionary algorithms, has recently become very popular. Although very effective, evolutionary algorithms rely heavily on having a large population of individuals (i.e., network architectures) and are therefore memory expensive. In this work, we propose a Regularized Evolutionary Algorithm with low memory footprint to evolve a dynamic image classifier. In details, we introduce novel custom operators that regularize the evolutionary process of a micro-population of 10 individuals. We conduct experiments on three different digits datasets (MNIST, USPS, SVHN) and show that our evolutionary method obtains competitive results with the current state-of-the-art.

Original language	English
Title of host publication	Image Analysis and Processing – ICIAP 2019 - 20th International Conference, Proceedings
Editors	Elisa Ricci, Nicu Sebe, Samuel Rota Bulò, Cees Snoek, Oswald Lanz, Stefano Messelodi
Publisher	Springer Verlag
Pages	219-230
Number of pages	12
ISBN (Print)	9783030306410
DOIs	https://doi.org/10.48550/arXiv.1905.06252 https://doi.org/10.1007/978-3-030-30642-7_20
Publication status	Published - 2019
Externally published	Yes
Event	20th International Conference on Image Analysis and Processing, ICIAP 2019 - Trento, Italy Duration: 9 Sept 2019 → 13 Sept 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11751 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	20th International Conference on Image Analysis and Processing, ICIAP 2019
Country/Territory	Italy
City	Trento
Period	9/09/19 → 13/09/19

Bibliographical note

Funding Information:
We gratefully acknowledge the support of NVIDIA Corporation with the donation of the TITAN Xp GPU used for this research.

Publisher Copyright:
© 2019, Springer Nature Switzerland AG.

Keywords

Deep Learning
Evolutionary algorithms
Neural Architecture Search
Regularized Evolution

Access to Document

Cite this

Saltori, C., Roy, S., Sebe, N., & Iacca, G. (2019). Regularized evolutionary algorithm for dynamic neural topology search. In E. Ricci, N. Sebe, S. Rota Bulò, C. Snoek, O. Lanz, & S. Messelodi (Eds.), Image Analysis and Processing – ICIAP 2019 - 20th International Conference, Proceedings (pp. 219-230). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11751 LNCS). Springer Verlag. https://doi.org/10.48550/arXiv.1905.06252, https://doi.org/10.1007/978-3-030-30642-7_20

Regularized evolutionary algorithm for dynamic neural topology search. / Saltori, Cristiano; Roy, Subhankar; Sebe, Nicu et al.
Image Analysis and Processing – ICIAP 2019 - 20th International Conference, Proceedings. ed. / Elisa Ricci; Nicu Sebe; Samuel Rota Bulò; Cees Snoek; Oswald Lanz; Stefano Messelodi. Springer Verlag, 2019. p. 219-230 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11751 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

Saltori, C, Roy, S, Sebe, N & Iacca, G 2019, Regularized evolutionary algorithm for dynamic neural topology search. in E Ricci, N Sebe, S Rota Bulò, C Snoek, O Lanz & S Messelodi (eds), Image Analysis and Processing – ICIAP 2019 - 20th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11751 LNCS, Springer Verlag, pp. 219-230, 20th International Conference on Image Analysis and Processing, ICIAP 2019, Trento, Italy, 9/09/19. https://doi.org/10.48550/arXiv.1905.06252, https://doi.org/10.1007/978-3-030-30642-7_20

Saltori C, Roy S, Sebe N, Iacca G. Regularized evolutionary algorithm for dynamic neural topology search. In Ricci E, Sebe N, Rota Bulò S, Snoek C, Lanz O, Messelodi S, editors, Image Analysis and Processing – ICIAP 2019 - 20th International Conference, Proceedings. Springer Verlag. 2019. p. 219-230. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.48550/arXiv.1905.06252, 10.1007/978-3-030-30642-7_20

Saltori, Cristiano ; Roy, Subhankar ; Sebe, Nicu et al. / Regularized evolutionary algorithm for dynamic neural topology search. Image Analysis and Processing – ICIAP 2019 - 20th International Conference, Proceedings. editor / Elisa Ricci ; Nicu Sebe ; Samuel Rota Bulò ; Cees Snoek ; Oswald Lanz ; Stefano Messelodi. Springer Verlag, 2019. pp. 219-230 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{be8718705b3a4eb89204ea9257d1369b,

title = "Regularized evolutionary algorithm for dynamic neural topology search",

abstract = "Designing neural networks for object recognition requires considerable architecture engineering. As a remedy, neuro-evolutionary network architecture search, which automatically searches for optimal network architectures using evolutionary algorithms, has recently become very popular. Although very effective, evolutionary algorithms rely heavily on having a large population of individuals (i.e., network architectures) and are therefore memory expensive. In this work, we propose a Regularized Evolutionary Algorithm with low memory footprint to evolve a dynamic image classifier. In details, we introduce novel custom operators that regularize the evolutionary process of a micro-population of 10 individuals. We conduct experiments on three different digits datasets (MNIST, USPS, SVHN) and show that our evolutionary method obtains competitive results with the current state-of-the-art.",

keywords = "Deep Learning, Evolutionary algorithms, Neural Architecture Search, Regularized Evolution",

author = "Cristiano Saltori and Subhankar Roy and Nicu Sebe and Giovanni Iacca",

note = "Funding Information: We gratefully acknowledge the support of NVIDIA Corporation with the donation of the TITAN Xp GPU used for this research. Publisher Copyright: {\textcopyright} 2019, Springer Nature Switzerland AG.; 20th International Conference on Image Analysis and Processing, ICIAP 2019 ; Conference date: 09-09-2019 Through 13-09-2019",

year = "2019",

doi = "10.48550/arXiv.1905.06252",

language = "English",

isbn = "9783030306410",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "219--230",

editor = "Elisa Ricci and Nicu Sebe and {Rota Bul{\`o}}, Samuel and Cees Snoek and Oswald Lanz and Stefano Messelodi",

booktitle = "Image Analysis and Processing – ICIAP 2019 - 20th International Conference, Proceedings",

address = "Germany",

}

TY - GEN

T1 - Regularized evolutionary algorithm for dynamic neural topology search

AU - Saltori, Cristiano

AU - Roy, Subhankar

AU - Sebe, Nicu

AU - Iacca, Giovanni

N1 - Funding Information: We gratefully acknowledge the support of NVIDIA Corporation with the donation of the TITAN Xp GPU used for this research. Publisher Copyright: © 2019, Springer Nature Switzerland AG.

PY - 2019

Y1 - 2019

N2 - Designing neural networks for object recognition requires considerable architecture engineering. As a remedy, neuro-evolutionary network architecture search, which automatically searches for optimal network architectures using evolutionary algorithms, has recently become very popular. Although very effective, evolutionary algorithms rely heavily on having a large population of individuals (i.e., network architectures) and are therefore memory expensive. In this work, we propose a Regularized Evolutionary Algorithm with low memory footprint to evolve a dynamic image classifier. In details, we introduce novel custom operators that regularize the evolutionary process of a micro-population of 10 individuals. We conduct experiments on three different digits datasets (MNIST, USPS, SVHN) and show that our evolutionary method obtains competitive results with the current state-of-the-art.

AB - Designing neural networks for object recognition requires considerable architecture engineering. As a remedy, neuro-evolutionary network architecture search, which automatically searches for optimal network architectures using evolutionary algorithms, has recently become very popular. Although very effective, evolutionary algorithms rely heavily on having a large population of individuals (i.e., network architectures) and are therefore memory expensive. In this work, we propose a Regularized Evolutionary Algorithm with low memory footprint to evolve a dynamic image classifier. In details, we introduce novel custom operators that regularize the evolutionary process of a micro-population of 10 individuals. We conduct experiments on three different digits datasets (MNIST, USPS, SVHN) and show that our evolutionary method obtains competitive results with the current state-of-the-art.

KW - Deep Learning

KW - Evolutionary algorithms

KW - Neural Architecture Search

KW - Regularized Evolution

UR - http://www.scopus.com/inward/record.url?scp=85072986326&partnerID=8YFLogxK

U2 - 10.48550/arXiv.1905.06252

DO - 10.48550/arXiv.1905.06252

M3 - Published conference contribution

AN - SCOPUS:85072986326

SN - 9783030306410

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 219

EP - 230

BT - Image Analysis and Processing – ICIAP 2019 - 20th International Conference, Proceedings

A2 - Ricci, Elisa

A2 - Sebe, Nicu

A2 - Rota Bulò, Samuel

A2 - Snoek, Cees

A2 - Lanz, Oswald

A2 - Messelodi, Stefano

PB - Springer Verlag

T2 - 20th International Conference on Image Analysis and Processing, ICIAP 2019

Y2 - 9 September 2019 through 13 September 2019

ER -

Regularized evolutionary algorithm for dynamic neural topology search

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this