Layered evaluation of interactive adaptive systems: framework and formative methods

Alexandros Paramythis, Stephan Weibelzahl, Judith Masthoff

Research output: Contribution to journalArticlepeer-review

137 Citations (Scopus)
16 Downloads (Pure)


The evaluation of interactive adaptive systems has long been acknowledged
to be a complicated and demanding endeavour. Some promising approaches
in the recent past have attempted tackling the problem of evaluating adaptivity by “decomposing” and evaluating it in a “piece-wise” manner. Separating the evaluation of different aspects can help to identify problems in the adaptation process. This paper presents a framework that can be used to guide the “layered” evaluation of adaptive systems, and a set of formative methods that have been tailored or specially developed for the evaluation of adaptivity. The proposed framework unifies previous approaches in the literature and has already been used, in various guises, in recent research work. The presented methods are related to the layers in the framework and the stages in the development lifecycle of interactive systems. The paper also discusses practical issues surrounding the employment of the above, and provides a brief overview of complementary and alternative approaches in the literature.
Original languageEnglish
Pages (from-to)383-453
Number of pages71
JournalUser Modelling and User-Adapted Interaction
Issue number5
Early online date28 Nov 2010
Publication statusPublished - 1 Dec 2010


  • layered evaluation
  • evaluation framework
  • formative evaluation methods
  • design


Dive into the research topics of 'Layered evaluation of interactive adaptive systems: framework and formative methods'. Together they form a unique fingerprint.

Cite this