Painting Music: Using artificial intelligence to create music from live painted drawings

Andrew Starkey* (Corresponding Author), Kate Steenhauer* (Corresponding Author), Jack Caven

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)
15 Downloads (Pure)


This article describes the development of artificial intelligence (AI) techniques that monitor a painting or drawing evolving in real time and produce musical notes that relate to the individual elements of art as the artwork develops on the canvas. The article describes the practical approach required to capture the artwork unfolding in real time and then describes the framework used to develop the correlations between visual art and music. The AI technique exploits these areas of similarity within the two distinct artforms in order to respond to the live-painted elements and produce musical notes that reflect the development of
the evolving artwork. A prototype of this system was implemented in a live stage performance at Aberdeen May Festival 2019 whose narrative centred on the question Is AI good or bad? Other outputs of this project are a 20-minute film and a body of (tangible) visual artwork for digital platforms and gallery environments informed and inspired by AI. The integration of these disciplines through AI transforms a static artform into one that is dynamic, interactive, transformational, transient and temporal
Original languageEnglish
Pages (from-to)209-224
Number of pages16
JournalDrawing: Research, Theory, Practice
Issue number2
Publication statusPublished - 1 Dec 2020


  • artificial intelligence
  • live drawing
  • music improvisation
  • drawing performance
  • collaboration
  • figurative drawing


Dive into the research topics of 'Painting Music: Using artificial intelligence to create music from live painted drawings'. Together they form a unique fingerprint.

Cite this