« Retour

Capallera Marine

Collaboratrice scientifique HES - Doctorante

Contrat principal

Collaboratrice scientifique HES - Doctorante

Bureau: HEIA_D30.05

Haute école d'ingénierie et d'architecture de Fribourg
Boulevard de Pérolles 80, 1700 Fribourg, CH

Institut
HumanTech - Technology for Human Wellbeing Institute

Intervenante externe

Haute école de travail social Fribourg
Route des Arsenaux 16a, 1700 Fribourg, CH

Aucune donnée à afficher pour cette section

En cours

AdVitam - Adaptive Driver-Vehicle InTerAction to Make future driving safer

Rôle: Collaborateur/trice

Financement: Hasler Stiftung

Description du projet :

AdVitam stands for Adaptive Driver-Vehicle InTerAction to Make future driving safer. It is a 3 years (2018-2021) research project co-funded by Hasler Fundation.

AdVitam explores how human-vehicle interface (HVI) can be designed at different level of attention in shared control driving to keep the driver at the optimum cognitive load and to fully support the potential of semi-automated driving.

The project aims at contributing to the vision of a car as a companion where the driver and the car work collaboratively and interactively as team-members to control the vehicle.

This project involves three PhD students. The first one works on multi-sensory experiences considering the entire car interior in order to support driver supervision tasks and increase situational awareness. The second one investigates how to use different modalities and combine them to improve the intervention of the driver during take-over requests. The third one studies how we can use several psychophysiological signals in order to assess the driver state in real-time.

The work of the three students will outcome in a novel collaborative interaction model between the driver and the vehicle in shared-control driving.

Equipe de recherche au sein de la HES-SO: Mugellini Elena , Abou Khaled Omar , Capallera Marine , Meteier Quentin , De Salis Emmanuel , Carrino Stefano , Angelini Leonardo

Partenaires académiques: Andreas Sonderegger, EPFL+ECAL Lab; Jean-Pierre Bresciani, UniFR

Url du site du projet: http://advitam.humantech.institute/

Statut: En cours

2024

Classification of driver fatigue in conditionally automated driving using physiological signals and machine learning

Article scientifique ArODES

Quentin Meteier, Renée Favre, Sofia Viola, Marine Capallera, Leonardo Angelini, Elena Mugellini, Andreas Sonderegger

Transportation Research Interdisciplinary Perspectives, 26, 101148

Lien vers la publication

Résumé:

Conditionally automated vehicles (Level 3 SAE) are emerging on the roads, but long periods without engaging in a non-driving-related task can reduce drivers’ vigilance. This study aims to determine whether driver fatigue can be accurately predicted using physiological signals and machine learning (ML) techniques in such a context. 63 young drivers completed two separate conditional automated drives of 30 min each, in either a rural or a urban area. Half of them had been mildly sleep-deprived the previous night (slept less than six hours). Electrocardiogram (ECG), electrodermal activity (EDA), and respiration were collected, along with subjective measures of sleepiness and affective state. Using ML, sleep deprivation, driving environment, and sleepiness could be predicted from physiological features with an accuracy of 99%, 85%, and 73% respectively. Signal segmentation increased model accuracy, and EDA features were the most predictive. The differences between the results obtained from statistical analyses of sleepiness measures and the accuracy achieved by ML models are discussed. The results of this empirical study indicate that even mild sleep deprivation affects the physiological state of drivers, which can have serious consequences when combined with long periods of inactivity. Car manufacturers and researchers should take this into account when designing intelligent systems capable of providing drivers with appropriate warnings before a critical situation arises.

4.3 Pro-GIS – Configurer des scénarios d’interactions sociales en réalité virtuelle avec et pour des jeunes ayant une déficience intellectuelle

Article scientifique

Dans Nanchen, B., Bovigny-Sandoz, S., Bussy, G., Fragnière, E., Mizeret, J., Weissbrodt, R. et Rullac, S. (dir.), Innovation Booster Technologie et Handicap Un dispositif, des personnes engagées et des projets pour une innovation sociale par la science.

Piérart Geneviève, Rossier Amélie, Capallera Marine

Association Sociographe, 2024 , pp. 199-209

Lien vers la publication

2023

Adaptive and multimodal human-vehicle interaction model for supervision in conditionally automated driving

Thèse de doctorat ArODES

Marine Capallera

2023, Fribourg, Switzerland : Department of Informatics, University of Fribourg. xv, 177 p.

Ingold, Rolf, Mugellini, Elena

Lien vers la publication

Résumé:

The development of driving assistance aims to increase user safety, reduce accidents, traffic and the environmental impact of driving. Another challenge is to assist drivers in their driving and navigation tasks. Thus, vehicles become more and more autonomous in certain tasks related to driving. The framework of this thesis work is placed in the context of conditionally automated vehicles. That is to say that drivers are not required to monitor the behavior of their vehicle but that they must be able to take over control of it at the request of the latter. Thus, this automation modality tends to change the role of drivers. They will go from being the main actor in the driving to being the supervisor. However, if drivers are engaged in a non-driving related task when takeover is requested, it can be even more dangerous because they are no longer in the control loop. This is why it is important to keep the drivers aware of the situation during the autonomous driving phases in order to be able to perform an efficient and safe takeover if necessary. The main objective of this thesis is to propose an adaptive multimodal system taking into account the whole interior environment of the car in the form of peripheral and subtle interactions in order to support the drivers in their supervision task and to increase/- maintain their situational awareness. For this purpose, a first analysis has been performed in order to highlight the factors limiting the proper functioning of autonomous systems and the information to be transmitted. A second analysis was performed in order to identify the parts of the vehicle used to transmit the peripheral information. This second analysis allowed to propose the association of interfaces using different modalities and parts of the vehicle which will be implemented in the adaptive system. These interfaces are ambient lights around the dashboard, vibrations in the driver’s seat and contextual and visual information displayed on the drivers personal device used to perform a non-driving task. The proposed system is part of a larger architecture allowing to predict the driver’s state and to adapt the interfaces to support the supervision of the drivers but also their intervention in case of takeover request. In order to test and validate the system, several scenarios and interaction prototypes have been realized and evaluated. Finally, the proposed system is governed by a rule-based model. The combination and characteristics of the interfaces depend on the state of the driver and the situation. The results encourage further studies to provide adaptive contextual information depending on the circumstances and state of the driver. It seems beneficial to maintain situational awareness, confidence and user experience of drivers by using a combination of interfaces to provide a complete multisensory and body experience.

A dataset on the physiological state and behavior of drivers in conditionally automated driving

Article scientifique ArODES

Quentin Meteier, Marine Capallera, Emmanuel de Salis, Leonardo Angelini, Stefano Carrino, Marino Widmer, Omar Abou Khaled, Elena Mugellini, Andreas Sonderegger

Data in Brief, 2023, 47, 109027

Lien vers la publication

Résumé:

This dataset contains data of 346 drivers collected during six experiments conducted in a fixed-base driving simulator. Five studies simulated conditionally automated driving (L3-SAE), and the other one simulated manual driving (L0-SAE). The dataset includes physiological data (electrocardiogram (ECG), electrodermal activity (EDA), and respiration (RESP)), driving and behavioral data (reaction time, steering wheel angle, …), performance data of non-driving-related tasks, and questionnaire responses. Among them, measures from standardized questionnaires were collected, either to control the experimental manipulation of the driver's state, or to measure constructs related to human factors and driving safety (drowsiness, mental workload, affective state, situation awareness, situational trust, user experience). In the provided dataset, some raw data have been processed, notably physiological data from which physiological indicators (or features) have been calculated. The latter can be used as input for machine learning models to predict various states (sleep deprivation, high mental workload, ...) that may be critical for driver safety. Subjective self-reported measures can also be used as ground truth to apply regression techniques. Besides that, statistical analyses can be performed using the dataset, in particular to analyze the situational awareness or the takeover quality of drivers, in different states and different driving scenarios. Overall, this dataset contributes to better understanding and consideration of the driver's state and behavior in conditionally automated driving. In addition, this dataset stimulates and inspires research in the fields of physiological/affective computing and human factors in transportation, and allows companies from the automotive industry to better design adapted human-vehicle interfaces for safe use of automated vehicles on the roads.

ID Tech :

Article scientifique ArODES

a virtual reality simulator training for teenagers with intellectual disabilities

Marine Capallera, Geneviève Piérart, Francesco Carrino, Robin Cherix, Amélie Rossier, Elena Mugellini, Abou Khaled Omar

Applied Sciences, 2023, vol. 13, no. 3679, pp. 1-15

Lien vers la publication

Résumé:

People with intellectual disability (ID) should routinely train themselves to carry out a variety of daily challenging tasks while being supervised by one or more supervisors. Virtual reality (VR) technology enables the simulation of certain learning scenarios that would be risky or difficult to set up or repeatedly replicate in the actual world. This paper introduces a VR simulator created for this aim with the assistance of social educators. The purpose is to use VR to conduct learning exercises with teenagers with ID and assess the extent to which the abilities learned in VR can be transferred to the real world. This project focuses mainly on urban mobility with three types of exercise. A study was conducted with 18 students in five institutions for 7 months. Post-tests were also carried out after 1 month with 11 teenagers. In the end, four teenagers are completely autonomous in their travels: two travel on foot and two travel by public transport. Regarding the impact analysis, the results are stable over time regarding self-reported ease, satisfaction, and fatigability. Finally, the system received valuable feedback from the educators.

A contextual multimodal system for increasing situation awareness and takeover quality in conditionally automated driving

Article scientifique ArODES

Marine Capallera, Quentin Meteier, Marino Widmer, Leonardo Angelini, Stefano Carrino, Andreas Sonderegger, Omar Abou Khaled, Elena Mugellini

IEEE Access, 2023, vol. 11, pp. 5746-5771

Lien vers la publication

Résumé:

Advanced driver assistances are becoming increasingly common in commercial cars, not only to assist but also to free drivers from manual driving whenever possible. Soon, drivers should be allowed to engage in non-driving-related tasks. The fact that responsibility for driving is shifting from humans to machines must be considered in the development of these assistances in order to guarantee safety and trust. In this article, we introduce AdVitam (for Advanced Driver-Vehicle Interaction to Make future driving safer), an autonomous system aiming at maintaining driver’s situation awareness and optimizing takeover quality during conditionally automated driving. The information conveyed to drivers is dynamically adapted to achieve these goals, depending on the driving environment and the driver’s physiological state. This system consists of three connected modules. The first module (Driver State) predicts the driver’s state with machine learning and physiological signals as inputs. The second module (Supervision) uses different interfaces (a haptic seat, a personal device, and ambient lights) to maintain the drivers’ situation awareness during the autonomous driving phases. The third module (Intervention) is a machine learning model that chooses the most appropriate combination among haptic, auditory, and visual modalities to request the driver to take over control and thus optimize takeover quality. To evaluate the system and each module independently, a preliminary user study with 35 drivers was conducted in a fixed-base driving simulator. All participants drove in two different environments (rural and urban). In addition, the activation of the Supervision and Intervention modules were manipulated as two between-subject factors. Results show that conveying information on the driving environment status through multimodal interfaces increases drivers’ situation awareness (i.e., better identification of potential problems in the environment) and trust in the automated vehicle. However, the system does not show positive outcomes on takeover quality. Besides, the Driver State module provided consistent predictions with the experimental manipulation. The system proposed in this paper could lead to better acceptance and safety when conditionally automated vehicles will be released by increasing drivers’ trust during phases of automated driving.

2022

Human-Vehicle Interaction to Support Driver's Situation Awareness in Automated Vehicles: A Systematic Review

Article scientifique ArODES

Marine Capallera, Leonardo Angelini, Quentin Meteier, Omar Abou Khaled, Elena Mugellini

IEEE Transactions on Intelligent Vehicles, 2023, Vol. 8, n° 3, pp. 2551 - 2567

Lien vers la publication

Résumé:

Autonomous driving will change the role of the driver. From being the main actor in driving, the driver will now have a supervisory role during the autonomous driving phases. However, if the driver has to take over control of the vehicle, he must be aware of the situation around him. This is why it is important to develop interfaces to keep him in the loop. This article proposes a systematic review of Human Vehicle Interaction (HVI) providing situation awareness in the context of autonomous driving. 37 articles presenting such interactions are analyzed in terms of design of the interaction (modalities, location, conveyed information) but also in term of evaluation and experimental conditions. We present an overview of previous studies in order to highlight the work already done or in progress. Current studies present mainly monomodal interfaces although the evaluation of multimodal interactions present promising results in this field.

Carrying a passenger and relaxation before driving :

Article scientifique ArODES

classification of young drivers’ physiological activation

Quentin Meteier, Marine Capallera, Emmanuel de Salis, Marino Widmer, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Andreas Sonderegger

Physiological Reports, 2022, vol. 10, article no. e15229

Lien vers la publication

Résumé:

Drivers are often held responsible for road crashes. Previous research has shown that stressors such as carrying passengers in the vehicle can be a source of accidents for young drivers. To mitigate this problem, this study investigated whether the presence of a passenger behind the wheel can be predicted using machine learning, based on physiological signals. It also addresses the question whether relaxation before driving can positively influence the driver's state and help controlling the potential negative consequences of stressors. Sixty young participants completed a 10-min driving simulator session, either alone or with a passenger. Before their driving session, participants spent 10 min relaxing or listening to an audiobook. Physiological signals were recorded throughout the experiment. Results show that drivers experience a higher increase in skin conductance when driving with a passenger, which can be predicted with 90%-accuracy by a k-nearest neighbors classifier. This might be a possible explanation for increased risk taking in this age group. Besides, the practice of relaxation can be predicted with 80% accuracy using a neural network. According to the statistical analysis, the potential beneficial effect of relaxation did not carry out on the driver's physiological state while driving, although machine learning techniques revealed that participants who exercised relaxation before driving could be recognized with 70% accuracy. Analysis of physiological characteristics after classification revealed several relevant physiological indicators associated with the presence of a passenger and relaxation.

Relevant physiological indicators for assessing workload in conditionally automated driving, through three-class classification and regression

Article scientifique ArODES

Quentin Meteier, Emmanuel de Salis, Marine Capallera, Marino Widmer, Leonardo Angelini, Omar Abou Khaled, Andreas Sonderegger, Elena Mugellini

Frontiers in Computer Science, 2022, vol. 3, article no. 775282

Lien vers la publication

Résumé:

In future conditionally automated driving, drivers may be asked to take over control of the car while it is driving autonomously. Performing a non-driving-related task could degrade their takeover performance, which could be detected by continuous assessment of drivers' mental load. In this regard, three physiological signals from 80 subjects were collected during 1 h of conditionally automated driving in a simulator. Participants were asked to perform a non-driving cognitive task (N-back) for 90 s, 15 times during driving. The modality and difficulty of the task were experimentally manipulated. The experiment yielded a dataset of drivers' physiological indicators during the task sequences, which was used to predict drivers' workload. This was done by classifying task difficulty (three classes) and regressing participants' reported level of subjective workload after each task (on a 0–20 scale). Classification of task modality was also studied. For each task, the effect of sensor fusion and task performance were studied. The implemented pipeline consisted of a repeated cross validation approach with grid search applied to three machine learning algorithms. The results showed that three different levels of mental load could be classified with a f1-score of 0.713 using the skin conductance and respiration signals as inputs of a random forest classifier. The best regression model predicted the subjective level of workload with a mean absolute error of 3.195 using the three signals. The accuracy of the model increased with participants' task performance. However, classification of task modality (visual or auditory) was not successful. Some physiological indicators such as estimates of respiratory sinus arrhythmia, respiratory amplitude, and temporal indices of heart rate variability were found to be relevant measures of mental workload. Their use should be preferred for ongoing assessment of driver workload in automated driving.

2021

Classification of drivers' workload using physiological signals in conditional automation

Article scientifique ArODES

Quentin Meteier, Marine Capallera, Simon Ruffieux, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Andreas Sonderegger

Frontiers in Psychology, 2021, vol. 12, article no. 596038

Lien vers la publication

Résumé:

The use of automation in cars is increasing. In future vehicles, drivers will no longer be in charge of the main driving task and may be allowed to perform a secondary task. However, they might be requested to regain control of the car if a hazardous situation occurs (i.e., conditionally automated driving). Performing a secondary task might increase drivers' mental workload and consequently decrease the takeover performance if the workload level exceeds a certain threshold. Knowledge about the driver's mental state might hence be useful for increasing safety in conditionally automated vehicles. Measuring drivers' workload continuously is essential to support the driver and hence limit the number of accidents in takeover situations. This goal can be achieved using machine learning techniques to evaluate and classify the drivers' workload in real-time. To evaluate the usefulness of physiological data as an indicator for workload in conditionally automated driving, three physiological signals from 90 subjects were collected during 25 min of automated driving in a fixed-base simulator. Half of the participants performed a verbal cognitive task to induce mental workload while the other half only had to monitor the environment of the car. Three classifiers, sensor fusion and levels of data segmentation were compared. Results show that the best model was able to successfully classify the condition of the driver with an accuracy of 95%. In some cases, the model benefited from sensors' fusion. Increasing the segmentation level (e.g., size of the time window to compute physiological indicators) increased the performance of the model for windows smaller than 4 min, but decreased for windows larger than 4 min. In conclusion, the study showed that a high level of drivers' mental workload can be accurately detected while driving in conditional automation based on 4-min recordings of respiration and skin conductance.

2020

The effect of instructions and context-related information about limitations of conditionally automated vehicles on situation awareness

Article scientifique ArODES

Quentin Meteier, Marine Capallera, Andreas Sonderegger, Leonardo Angelini, Stefano Carrino, Elena Mugellini, Omar Abou Khaled

Proceedings of AutomotiveUI '20: 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 21-22 September 2020, Online Conference, pp. 241-251

Lien vers la publication

Résumé:

In conditionally automated driving, drivers do not have to constantly monitor their vehicle but they must be able to take over control when necessary. In this paper, we assess the impact of instructions about limitations of automation and the presentation of context-related information through a mobile application on the situation awareness and takeover performance of drivers. We conducted an experiment with 80 participants in a fixed-base driving simulator. Participants drove for an hour in conditional automation while performing secondary tasks on a tablet. Besides, they had to react to five different takeover requests. In addition to the assessment of behavioral data (e.g. quality of takeover), participants rated their situation awareness after each takeover situation. Instructions and context-related information on limitations combined showed encouraging results to raise awareness and improve takeover performance.

2019

Owner Manuals Review and Taxonomy of ADAS Limitations in Partially Automated Vehicles

Article scientifique

Capallera Marine, Meteier Quentin, De Salis Emmanuel, Carrino Stefano, Angelini Leonardo, Abou Khaled Omar, Mugellini Elena

AutomotiveUI '19: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 2019

Lien vers la publication

Résumé:

In the context of highly automated driving, the driver has to be aware of driving risks and to take over control of the car in hazardous situations. The goal of this paper is to categorize and analyze the factors that lead to such critical scenarios. To this purpose, we analyzed limitations of Advanced Driver-Assistance Systems (ADAS) extracted from owner manuals of 12 partially automated cars available on the market. A taxonomy with 6 macro-categories and 26 micro-categories is proposed to classify and better understand the limitations of these vehicles. We also investigated if these limitations are conveyed to the driver through Human-Machine Interaction (HMI) in the car. Some suggestions are made to better communicate these limitations to the driver in order to raise his/her situation awareness.

Workshop on explainable AI in automated driving: a user-centered interaction approach

Article scientifique

Meteier Quentin, Capallera Marine, Angelini Leonardo, Mugellini Elena, Abou Khaled Omar, Carrino Stefano, De Salis Emmanuel, Stéphane Galland, Susanne Boll

AutomotiveUI '19: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings, 2019

Lien vers la publication

Résumé:

With the increasing use of automation, users tend to delegate more tasks to the machines. Such complex systems are usually developed with "black box" Artificial Intelligence (AI), which makes these systems difficult to understand for the user. This assumption is particularly true in the field of automated driving since the level of automation is constantly increasing via the use of state-of-the-art AI solutions. We believe it is important to investigate the field of Explainable AI (XAI) in the context of automated driving since interpretability and transparency are key factors for increasing trust and security. In this workshop, we aim at gathering researchers and industry practitioners from different fields to brainstorm about XAI with a special focus on human-vehicle interaction. Questions like "what kind of explanation do we need", "which is the best trade-off between performance and explainability" and "how granular should the explanations be" will be addressed in this workshop.

2024

La réalité virtuelle permet-elle d'apprendre ?

Conférence

Rossier Amélie, Piérart Geneviève, Capallera Marine, Cherix Robin, Carrino Francesco, Abou Khaled Omar

Congrès Transformations des pratiques et nouvelles représentations du handicap, 28.08.2024 - 28.08.2024, Université du Québec à Montréal, Québec, Canada

2023

Projet Pro-GIS Participer pour créer un scénario de réalité virtuelle

Conférence

Piérart Geneviève, Rossier Amélie, Capallera Marine

Semaine de l’accessibilité de la Fondation pour la Recherche en faveur des personnes handicapées (FRH), 23.05.2023 - 25.05.2023, Lausanne

Training nurses in VR :

Conférence ArODES

exploring spatial mapping and free-hand interaction

Marine Capallera, Leonardo Angelini, Alexandre Favre, François Magnin, Omar Abou Khaled, Elena Mugellini

Proceeding sof the 34th international francophone conference on human-computer interaction

Lien vers la conférence

Résumé:

La réalité virtuelle offre un environnement de formation immersif dans l'enseignement des soins infirmiers, en particulier pour expérimenter des situations ne pouvant être perçues dans la réalité. Par un processus de conception centré sur l'humain avec des éducateurs.trices et des étudiant.es d'une école d'infirmières, nous avons développé une application de RV de formation à l'asepsie lors d'un prélèvement sanguin. La simulation permet de visualiser les risques de contamination lors de la procédure. Nous avons réalisé trois tests et amélioré continuellement l'application afin d'augmenter la facilité d'utilisation et le sentiment d'immersion. Nous avons comparé deux modalités d'interaction, deux visualisations des mains et différents degrés d'immersion (téléportation ou marche dans une représentation réelle d'une salle pour toucher les objets physiques). Ces expériences ont permis de tirer une conclusion sur le niveau d'immersion atteint par les nouvelles technologies de RV et sur leur implication possible dans la formation pratique des étudiant.es.

Effect of obstacle type and cognitive task on situation awareness and takeover performance in conditionally automated driving

Conférence ArODES

Quentin Meteier, Marine Capallera, Emmanuel de Salis, Leonardo Angelini, Stefano Carrino, Omar Abou Khaled, Elena Mugellini, Andreas Sonderegger

Proceeding sof the 34th international francophone conference on human-computer interaction

Lien vers la conférence

Résumé:

En conduite automatisée, plusieurs facteurs peuvent affecter la conscience de la situation et la reprise de contrôle. Pour mieux comprendre cela, 88 participants ont passé 20 minutes dans un simulateur de conduite conditionnellement automatisée. Ils devaient réagir à quatre obstacles dont le danger et le mouvement variaient. La moitié des participants devaient s'engager dans une tâche cognitive verbale non liée à la conduite. La conscience de la situation, la performance et les réponses physiologiques ont été mesurées. Les résultats suggèrent que le mouvement de l'obstacle influence la perception du danger, la conscience de la situation et le temps de réaction, ce dernier étant également influencé par le danger de l'obstacle. La tâche cognitive verbale a également un effet sur le temps de réaction. Ainsi, l’état cognitif du conducteur et la situation de conduite (mouvement/danger des entités autour du véhicule) devraient être considérés lors de la transmission d'informations via les interfaces homme-véhicule.

2022

Virtual reality simulator training for teenagers with intellectual disabilities

Conférence ArODES

Marine Capallera, Robin Cherix, Francesco Carrino, Geneviève Périart, Amélie Rossier, Omar Abou Khaled

Proceedings of GSGS'22 : 7th International Conference on Gamification Serious Game, 29 June - 1st July 2022, Geneva, Switzerland

Lien vers la conférence

Résumé:

People with intellectual disabilities (ID) need to train regularly to perform several complex daily tasks while being accompanied by one or more supervisors. The use of virtual reality (VR) allows the simulation of these learning situations that would be difficult to set up or reproduce systematically in the real world. This paper presents an enhancement to a VR simulator designed for this purpose with the help of social educators. The goal is to use virtual reality to perform learning exercises with teenagers with ID. The main exercises are: (a) walking on the sidewalk and crossing crosswalks, (b) taking one or more buses and getting off at the right stops, (c) taking the right train at the station and getting off at the right stop. The simulation also proposes tomanage social interactions thanks to the use of avatars in these scenarios. Each scenario is customized for the students' learning objectives.

ID Tech Project

Conférence

Virtual Reality Simulator Training for Teenagers with Intellectual Disabilities

Piérart Geneviève, Capallera Marine, Cherix Robin, Carrino Francesco, Rossier Amélie, Abou Khaled Omar

Gamification and Serious Games 2022 (GSGS'22), 29.06.2022 - 01.07.2022, Sion

Machine learning agent to recommend the best modality for takeover during conditionally automated driving

Conférence ArODES

Emmanuel de Salis, Quentin Meteier, Marine Capallera, Colin Pelletier, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Stefano Carrino

AHFE International, Human Interaction Emerging Technologies (IHIET-AI 2022): Artificial Intelligence Future Applications

Lien vers la conférence

Résumé:

In this study, we investigate how to make takeover in conditionally automated vehicles safer optimizing the takeover request modalities. Using Machine Learning algorithms, we created a smart agent recommending the best modalities combination to use (haptic-visual, auditory-visual and haptic-auditory-visual) to convey a takeover request. This agent considers the driver state before recommending modalities, as well as the weather condition. The proposed agent is able to predict takeover quality better than the baseline by 56.5% (baseline MSE : 0.0600, our agent MSE : 0.0261) and use different recommendations based on the situation. Modality impact on takeover quality was shown to have a mean score of 4.95% (standard deviation: 2.7%). Evaluation of the agent gain on takeover quality compared to standard takeover request design is currently ongoing.

Comparaison de concepts d’interactions homme-véhicule pour soutenir le conducteur en conduite conditionnellement automatisée

Conférence ArODES

Marine Capallera, Jean Pierre Bresciani, Quentin Meteier, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini

IHM ’22 : Proceedings of the 33rd Conference on l'Interaction Humain-Machine, 5-8 April 2022, Namur, Belgium

Lien vers la conférence

Résumé:

Les actuelles assistances à la conduite modifient progressivement le rôle du conducteur. Pour certains niveaux d’automatisation, son rôle actif devient alors celui de superviseur. Ce changement peut entraîner une réduction de la compréhension de la situation mais également une charge cognitive plus élevée lors de la reprise de contrôle manuelle. C’est pourquoi il est important de concevoir des interactions permettant de soutenir le conducteur dans cette tâche. Cet article propose d’utiliser le siège du conducteur et une tablette pour transmettre des informations au conducteur relatives à son environnement pendant les phases de conduite autonome et ainsi l’accompagner dans sa tâche de supervision. Une première expérience a permis de valider l’utilisation de vibrations dans le siège pour transmettre des informations. Une seconde expérience compare ce concept avec l’envoi d’icônes pour transmettre ces mêmes informations. Les résultats montrent que l’utilisation de ces concepts a un impact sur la conscience de la situation.

Predicting takeover quality in conditionally autonomous vehicles based on takeover request modalities, driver physiological state and the environment

Conférence ArODES

Emmanuel de Salis, Quentin Meteier, Marine Capallera, Quentin Pelletier, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Stefano Carrino

Proceedings of the 5th International Conference on Intelligent Human Systems Integration (IHSI 2022) Integrating People and Intelligent Systems, Venice, Italy, 22-24 February 2022

Lien vers la conférence

2021

Clustering of drivers’ state before takeover situations based on physiological features using unsupervised machine learning

Conférence ArODES

Emmanuel de Salis, Quentin Meteier, Colin Pelletier, Marine Capallera, Leonardo Angelini, Andreas Sonderegger, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Stefano Carrino

International Conference on Human Interaction and Emerging Technologies ; Proceedings of IHIET 2021: Human Interaction, Emerging Technologies and Future Systems V, 28-30 October 2021, Reims, France

Lien vers la conférence

Résumé:

Conditionally automated cars share the driving task with the driver. When the control switches from one to another, accidents can occur, especially when the car emits a takeover request (TOR) to warn the driver that they must take the control back immediately. The driver’s physiological state prior to the TOR may impact takeover performance and as such was extensively studied experimentally. However, little was done about using Machine Learning (ML) to cluster natural states of the driver. In this study, four unsupervised ML algorithms were trained and optimized using a dataset collected in a driving simulator. Their performances for generating clusters of physiological states prior to takeover were compared. Some algorithms provide interesting insights regarding the number of clusters, but most of the results were not statistically significant. As such, we advise researchers to focus on supervised ML using ground truth labels after experimental manipulation of drivers’ states.

Workshop for designing biofeedback of driver’s state and emotion in automated vehicles

Conférence ArODES

Marine Capallera, Quentin Meteier, Kevin Koch, Markus Funk, Mira El Kamali, Karl Daher, Omar Abou Khaled, Elena Mugellini

Proceedings of AutomotiveUI '21 Adjunct: 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 9-14 September 2021, Leeds, United Kingdom

Lien vers la conférence

Résumé:

Different drivers’ states and emotions can affect negatively the driving performance. Recent advances in affective computing now give the opportunity to measure the users’ state or emotions using various sources of data such as physiological signals or voice samples. Conveying biofeedback in the car could help to make roads safer and improve users’ health and mental state during a ride in an autonomous car. This workshop aims at selecting the drivers’ hazardous states and emotions that are crucial to be assessed, as well as how to convey the appropriate biofeedback to the driver, using multimodal interaction in the car.

Empathic interactions in automated vehicles #EmpathicCHI

Conférence ArODES

Karl Daher, Marine Capallera, Chiara Lucifora, Jacky Casas, Quentin Meteier, Mira El Kamali, Abdallah El Ali, Giorgio Mario Grosso, Omar Abou Khaled, Elena Mugellini

Proceedings of CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 8-13 May 2021, Yokohama, Japan

Lien vers la conférence

Résumé:

Automation in driving will change the role of the drivers from actor to passive supervisor. Although the vehicle will be responsible for driving manoeuvres, drivers will need to rely on automation and understand its decisions to establish a trusting relationship between them and the vehicle. Progress has been made in conversational agents and affective machines recently. Moreover, it seems to be promising in this establishment of trust between humans and machines. We believe it is essential to investigate the use of emotional conversational agents in the automotive context to build a solid relationship between the driver and the vehicle. In this workshop, we aim at gathering researchers and industry practitioners from different fields of HCI, ML/AI, NLU and psychology to brainstorm about affective machines, empathy and conversational agent with a particular focus on human-vehicle interaction. Questions like ”What would be the specificities of a multimodal and empathic agent in a car?”, ”How the agent could make the driver aware of the situation?” and ”How to measure the trust between the user and the autonomous vehicle?” will be addressed in this workshop.

Predicting takeover quality in conditionally automated vehicles using machine learning and genetic algorithms

Conférence ArODES

Emmanuel de Salis, Quentin Meteier, Marine Capallera, Leonardo Angelini, Andreas Sonderegger, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Stefano Carrino

Advances in Intelligent Systems and Computing ; Proceedings of International Conference on Intelligent Human Systems Integration IHSI 2021: Intelligent Human Systems Integration 2021, 22-24 February 2021, Palermo, Italy

Lien vers la conférence

Résumé:

Takeover requests in conditionally automated vehicles are a critical point in time that can lead to accidents, and as such should be transmitted with care. Currently, several studies have shown the impact of using different modalities for different psychophysiological states, but no model exists to predict the takeover quality depending on the psychophysiological state of the driver and takeover request modalities. In this paper, we propose a machine learning model able to predict the maximum steering wheel angle and the reaction time of the driver, two takeover quality metrics. Our model is able to achieve a gain of 42.26% on the reaction time and 8.92% on the maximum steering wheel angle compared to our baseline. This was achieved using up to 150 s of psychophysiological data prior to the takeover. Impacts of using such a model to choose takeover modalities instead of using standard takeover requests should be investigated.

2020

Human vehicle interaction model for supervision in conditionally automated driving cars

Conférence ArODES

Marine Capallera, Omar Abou Khaled, Leonardo Angelini, Elena Mugellini

Proceedings of ETIS 2020 : the Fourth European Tangible Interaction Studio, 16-20 November 2020, Siena, Italy

Lien vers la conférence

Résumé:

Developments in the field of semi-autonomous driving will increasingly free the driver from the main driving task. Conditional autonomous driving allows the driver not to constantly monitor her/his environment, but s/he must still be able to regain control of the vehicle at any time if the situation requires it. That’s why it is important to maintain driver’s Situation Awareness while is performing a secondary task. This paper presents a model of an adaptive, full-body and multimodal Human Vehicle Interaction (HVI) for supervision.

Inside the cockpit of semi-autonomous cars of tomorrow :

Conférence ArODES

gamification design of a carpolling-based freight transport

Quentin Meteier, Marine Capallera, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Jean-Pierre Bresciani, Andreas Sonderegger, Stefano Carrino

Proceedings of GSGS'20 : 5th Gamification & Serious Game Symposium, September-November 2020, Switzerland

Lien vers la conférence

Résumé:

Simulations play a crucial role to investigate hazardous situations that are impossible to test in real-life conditions without endangering the user’s safety. This paper presents a simulator of conditionally automated cars aiming at enhancing the driver safety and driving comfort. In addition, thanks to the simulator’s highly repeatability, integrated sensors and controlled conditions we collected valuable scientific data, which is otherwise very difficult to gather.

Designing an AI-companion to support the driver in highly autonomous cars

Conférence ArODES

Marine Capallera, Quentin Meteier, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Stefano Carrino

Proceedings of HCII 2020 : 22nd International Conference on Human-Computer Interaction, 19-24 July 2020, Copenhagen, Denmark ; Lecture Notes in Computer Science

Lien vers la conférence

Résumé:

In this paper, we propose a model for an AI-Companion for conditionally automated cars, able to maintain awareness of the driver regarding the environment but also to able design take-over requests (TOR) on the fly, with the goal of better support the driver in case of a disengagement. Our AI-Companion would interact with the driver in two ways: first, it could provide feedback to the driver in order to raise the driver Situation Awareness (SA), prevent them to get out of the supervision loop and so, improve takeover during critical situations by decreasing their cognitive workload. Second, in the case of TOR with a smart choice of modalities for convey the request to the driver. In particular, the AI-Companion can interact with the driver using many modalities, such as visual messages (warning lights, images, text, etc.), auditory signals (sound, speech, etc.) and haptic technologies (vibrations in different parts of the seat: back, headrest, etc.). The ultimate goal of the proposed approach is to design smart HMIs in semi-autonomous vehicles that are able to understand 1) the user state and fitness to drive, 2) the current external situation (vehicle status and behavior) in order to minimize the automation surprise and maximizing safety and trust, and 3) leverage AI to provide adaptive TOR and useful feedback to the driver.

Réalisations

PEOPLE@HES-SO Annuaire et Répertoire des compétences

Capallera Marine

Collaboratrice scientifique HES - Doctorante

Collaboratrice scientifique HES - Doctorante

Intervenante externe

PEOPLE@HES-SO
Annuaire et Répertoire des compétences