Towards Accountability Driven Development for Machine Learning Systems

Chiu Pang Fung, Wei Pang, Iman Naja, Milan Markovic, Pete Edwards

Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution

17 Downloads (Pure)


With rapid deployment of Machine Learning (ML) systems into diverse domains such as healthcare and autonomous driving, important questions regarding accountability in case of incidents resulting from ML errors remain largely unsolved. To improve accountability of ML systems, we introduce a framework called Accountability Driven Development (ADD). Our framework reuses Behaviour Driven Development (BDD) approach to describe testing scenarios and system behaviours in ML Systems’ development using natural language, guides and forces developers and intended users to actively record necessary accountability information in the design and implementation stages. In this paper, we illustrate how to transform accountability requirements to specific scenarios and provide syntax to describe them. The use of natural language allows non technical collaborators such as stakeholders and non ML domain experts deeply engaged in ML system development to provide more comprehensive evidence to support system’s accountability. This framework also attributes the responsibility to the whole project team including the intended users rather than putting all the accountability burden on ML engineers only. Moreover, this framework can be considered as a combination of both system test and acceptance test, thus making the development more efficient. We hope this work can attract more engineers to use our idea, which enables them to create more accountable ML systems
Original languageEnglish
Title of host publicationCEUR Workshop Proceedings
Subtitle of host publicationProceedings of the SICSA eXplainable Artifical Intelligence Workshop 2021
EditorsKyle Martin, Nirmalie Wiratunga , Anjana Wijekoon
Number of pages8
Publication statusPublished - 2 Jul 2021
EventSICSA Workshop on eXplainable Artificial Intelligence (XAI) 2021 -
Duration: 1 Jun 20211 Jun 2021

Publication series

ISSN (Electronic)1613-0073


WorkshopSICSA Workshop on eXplainable Artificial Intelligence (XAI) 2021
Abbreviated titleXAI
Internet address

Bibliographical note

Conference Proceedings: ISSN 16130073
CEUR Workshop Proceedings Volume 2894, Pages 25 - 322021 2021 SICSA eXplainable Artifical Intelligence Workshop, SICSA XAI 2021, Aberdeen, 1 June 2021, 169872


Dive into the research topics of 'Towards Accountability Driven Development for Machine Learning Systems'. Together they form a unique fingerprint.

Cite this