Tutorial AAMAS 2021 – 04 May

Explainability, Trust and Ethics for Robots and Autonomous Systems

Jim Torresen, Department of Informatics
 University of Oslo,  Norway



ABSTRACT – A brief description of the tutorial

Explainability, trust and ethics for robots and autonomous systems are getting increased attention. This is partly as a result of the currently limited deployment of such systems in human support and interaction settings. The tutorial will give an overview of the most commonly expressed ethical challenges and ways being undertaken to reduce their impact using the findings in an earlier undertaken review supplemented with recent work and initiatives. The presentation will exemplify the challenges related to privacy, security and safety through several examples from own and others’ work.


Ethics, Robotics, Autonomous systems, Privacy, Security and Safety

Outline of the tutorial

1.     Introduction 

2.     Ethical challenges and considerations and the importance of explainable and trustworthy systems

3.     How to address ethical considerations in research? – Examples from own research at Univ of Oslo

4.     Future opportunities in ethics related research

Tutorial slides can be found here (draft, to be updated)

A detailed outline of the tutorial

Robots and artificial intelligence demonstrate to effectively contribute to an increasing number of different domains. At the same time, an increasing number of people – in the general public as well as in research – have started to consider a number of potential ethical challenges related to the development and use of such technology. There are also initiatives across countries like the European Commission appointed High-Level Expert Group on Artificial Intelligence (AI HLEG) that has as a general objective to support the implementation of the European Strategy on Artificial Intelligence. This tutorial will give an overview of the most commonly expressed ethical challenges and ways being undertaken to reduce their impact using the findings in an earlier undertaken review supplemented with recent work and initiatives.


Among the most important challenges are those related to privacy, safety and security. Countermeasures can be taken first at design time, second, when a user should decide where and when to apply a system and third when a system is in use in its environment. In the latter case, there will be a need for the system by itself to perform some ethical reasoning if operating in autonomous mode. Specific attention will be needed if multiple, fully autonomous systems are to interact and make decisions together. We are currently undertaking research in various projects where the challenges appear, including in robots for elderly at home and mental health care technology.  The tutorial will introduce some examples from our own and others work and how the challenges can be addressed both from a technical and human side [1]-[5]. Ethical issues should not be seen only as challenges but also as new research opportunities contributing to more useful services and systems.


One example of the latter is to study a robot to be located in the home of an older person living without a partner. That is with the goal of developing multi-sensor mobile robot solutions that can sense, learn and notify caregiver about abnormal events. We have focussed on three crucial aspects of such a robot by addressing the sensing system, the motion control system and considering the design and behaviour of a robot from a user perspective. We see that key challenges for such a robot relate to privacy, security and safety. The technological choices being made with regards to hardware and software both make an impact on how well the challenges are handled. The privacy needs to be compromised between limiting sensor data collection and the ability of a system to correctly notify the caregiver when some abnormal and emergency situation has happened. At the same time, light conditions in a home can vary a lot with sunlight at daytime and darkness at night-time. This puts certain restrictions on what sensors that are most effective and that multiple different sensors may be needed. However, rather than transmitting a lot of personal sensor data to a caregiver facility, the local robot should train a model to represent the regular activities of the person it supports. Then only in the case of abnormal events, the robot would transmit data out of the home. E.g., if the person does some daily exercises, a sudden increase in heart rate would be expected while it is no so for a person not usually doing any physical activity. Only in the latter case, sensor data should be forwarded out of the home but only data that is regarded as essential for the follow-up.  


The tutorial will contain some parts where participants discuss and comment on ethical challenges in small groups and in plenary. Further, opinions within the audience will be collected through using the Kahoot! voting tool (responding using smartphones to answer multiple-choice questions). If virtual, text Q & A, live Q & A and attendee polls will be used.


Length/format: Half day


Target audience

There is increasing attention on the ethical implications of robotics and AI research. Thus, the tutorial will be targeting all attendees of the AAMAS-2021 conference. The tutorial will give the audience insight into the main ethical implication currently being discussed and also how that can open up for new directions in autonomous agents and multiagent systems research. There is no specific prerequisite knowledge required.


A brief resume of the presenter


Picture Jim Torresen


Name: Jim Torresen


Postal address:

PO Box 1080 Blindern,  N-0316 Oslo, Norway


Phone numbers: +4722852454 (office) +4792846669 (mobile)


E-mail address: jimtoer@ifi.uio.no



Background in the tutorial area

Jim Torresen is a professor at University of Oslo where he leads the Robotics and Intelligent Systems research group. He received his M.Sc. and Dr.ing. (Ph.D) degrees in computer architecture and design from the Norwegian University of Science and Technology, Univ. of Trondheim in 1991 and 1996, respectively. He has been employed as a senior hardware designer at NERA Telecom-munications (1996-1998) and at Navia Aviation (1998-1999). Since 1999, he has been a professor at the Department of Informatics at the Univ. of Oslo (associate professor 1999-2005). Jim Torresen has been a visiting researcher at Kyoto University, Japan for one year (1993-1994), four months at Electrotechnical laboratory (now AIST), Tsukuba, Japan (1997 and 2000) and a visiting professor at Cornell University, USA for one year (2010-2011).


His research interests at the moment include artificial intelligence, ethical aspects of AI and robotics, machine learning, robotics, and applying this to complex real-world applications. Several novel methods have been proposed. He has published over 200 scientific papers in international journals, books and conference proceedings. 10 tutorials and a number of invited talks have been given at international conferences and research institutes. He is in the program committee of more than ten different international conferences, associate editor of three international scientific journals as well as a regular reviewer of a number of other international journals. He has also acted as an evaluator for proposals in EU FP7 and Horizon2020 and is currently project manager/principal investigator in four externally funded research projects/centres. He is a member of the Norwegian Academy of Technological Sciences (NTVA) and the National Committee for Research Ethics in Science and Technology (NENT) where he is a member of a working group on research ethics for AI. More information and a list of publications can be found here: http://jimtoer.no/



Information about previous tutorials delivered by the presenters

See an overview of past invited talks and tutorials here.


This work is partially supported by The Research Council of Norway as a part of Multimodal Elderly Care systems (MECS) project, under grant agreement 247697, the INTROducing Mental health through Adaptive Technology (INTROMAT) project under grant agreement 259293 and Vulnerability in the Robot Society (VIROS) under grant agreement 288285, Predictive and Intuitive Robot Companion (PIRC)  under Grant Agreement No. 312333 and through its Centres of Excellence scheme, RITMO with Project No. 262762.


[1] Ryo Saegusa, Hirokazu Ito, Duc Minh Duong, “Human-Care Rounds Robot with Contactless Breathing Measurement", in proc. of 2019 International Conference on Robotics and Automation (ICRA), Montreal, Canada, May 20-24, 2019 https://doi.org/10.1109/ICRA.2019.8794037


[2] Bringsjord, Selmer & Tokhi, M.O. & Isabel, Maria & Aldinhas Ferreira, Maria Isabel & Govindarajulu, Naveen. (2019). Hybrid Worlds: Societal and Ethical Challenges, CLAWAR Association Series on Robot Ethics and Standards, http://kryten.mm.rpi.edu/HybridWorlds.pdf


[3] Torresen, J., “A Review of Future and Ethical Perspectives of Robotics and AI.” Frontiers in Robotics and AI, vol. 4, article 75, 2018. https://doi.org/10.3389/frobt.2017.00075


[4] Winfield, A. F., Blum, C., and Liu, W. (2014). “Towards an ethical robot: internal models, consequences and ethical action selection,” in Advances in Autonomous Robotics Systems, eds M. Mistry, A. Leonardis, M. Witkowski, and C. Melhuish, 2014 Springer, pp. 85–96. https://doi.org/10.1007/978-3-319-10401-0_8


[5] Winfield, A. F., Michael, K., Pitt J. and Evers V., "Machine Ethics: The Design and Governance of Ethical AI and Autonomous Systems [Scanning the Issue]," in Proceedings of the IEEE, vol. 107, no. 3, pp. 509-517, March 2019. https://ieeexplore.ieee.org/document/8662743

Conference Name:ACM Woodstock conference

Conference Short Name:WOODSTOCK’18

Conference Location:El Paso, Texas USA




Copyright Year:2018

Copyright Statement:rightsretained


RRH: F. Surname et al.