« Retour

Mugellini Elena

Professeure HES ordinaire/Responsable d'institut

Compétences principales

Technology for Behavior Change

Professeure HES ordinaire/Responsable d'institut

Téléphone: +41 26 429 68 70

Bureau: HEIA_D20.17

Haute école d'ingénierie et d'architecture de Fribourg
Boulevard de Pérolles 80, 1700 Fribourg, CH

Institut
HumanTech - Technology for Human Wellbeing Institute

MSc HES-SO en Engineering - HES-SO Master

Advanced Human Computer Interaction
Machine Learning
IT Project Management
Design Thinking

BSc HES-SO en Informatique - Haute école d'ingénierie et d'architecture de Fribourg

Génie logiciel
Machine Learning
Interfaces Hommes-Machine avancées

En cours

For a complete list of RaD projects please visit http://humantech.institute/

Rôle: Requérant(e) principal(e)

Description du projet :

For a complete list of RaD projects please visit http://humantech.institute/

Equipe de recherche au sein de la HES-SO: Mugellini Elena

Durée du projet: 28.09.2020

Statut: En cours

AdVitam - Adaptive Driver-Vehicle InTerAction to Make future driving safer

Rôle: Requérant(e) principal(e)

Financement: Hasler Stiftung

Description du projet :

AdVitam stands for Adaptive Driver-Vehicle InTerAction to Make future driving safer. It is a 3 years (2018-2021) research project co-funded by Hasler Fundation.

AdVitam explores how human-vehicle interface (HVI) can be designed at different level of attention in shared control driving to keep the driver at the optimum cognitive load and to fully support the potential of semi-automated driving.

The project aims at contributing to the vision of a car as a companion where the driver and the car work collaboratively and interactively as team-members to control the vehicle.

This project involves three PhD students. The first one works on multi-sensory experiences considering the entire car interior in order to support driver supervision tasks and increase situational awareness. The second one investigates how to use different modalities and combine them to improve the intervention of the driver during take-over requests. The third one studies how we can use several psychophysiological signals in order to assess the driver state in real-time.

The work of the three students will outcome in a novel collaborative interaction model between the driver and the vehicle in shared-control driving.

Equipe de recherche au sein de la HES-SO: Mugellini Elena , Abou Khaled Omar , Capallera Marine , Meteier Quentin , De Salis Emmanuel , Carrino Stefano , Angelini Leonardo

Partenaires académiques: Andreas Sonderegger, EPFL+ECAL Lab; Jean-Pierre Bresciani, UniFR

Url du site du projet: http://advitam.humantech.institute/

Statut: En cours

2024

Classification of driver fatigue in conditionally automated driving using physiological signals and machine learning

Article scientifique ArODES

Quentin Meteier, Renée Favre, Sofia Viola, Marine Capallera, Leonardo Angelini, Elena Mugellini, Andreas Sonderegger

Transportation Research Interdisciplinary Perspectives, 2024, 26, 101148

Lien vers la publication

Résumé:

Conditionally automated vehicles (Level 3 SAE) are emerging on the roads, but long periods without engaging in a non-driving-related task can reduce drivers’ vigilance. This study aims to determine whether driver fatigue can be accurately predicted using physiological signals and machine learning (ML) techniques in such a context. 63 young drivers completed two separate conditional automated drives of 30 min each, in either a rural or a urban area. Half of them had been mildly sleep-deprived the previous night (slept less than six hours). Electrocardiogram (ECG), electrodermal activity (EDA), and respiration were collected, along with subjective measures of sleepiness and affective state. Using ML, sleep deprivation, driving environment, and sleepiness could be predicted from physiological features with an accuracy of 99%, 85%, and 73% respectively. Signal segmentation increased model accuracy, and EDA features were the most predictive. The differences between the results obtained from statistical analyses of sleepiness measures and the accuracy achieved by ML models are discussed. The results of this empirical study indicate that even mild sleep deprivation affects the physiological state of drivers, which can have serious consequences when combined with long periods of inactivity. Car manufacturers and researchers should take this into account when designing intelligent systems capable of providing drivers with appropriate warnings before a critical situation arises.

The smart home, a true home? How new technologies disrupt the experience of home for older persons

Article scientifique ArODES

Nadine Andrea Felber, Hamed Alavi, Elena Mugellini, Tenzin Wangmo

Universal Access in the Information Society, 2024

Lien vers la publication

Résumé:

Smart home technologies (SHTs) can support older persons to age in place. However, adoption of SHTs remains low among this population. A reason for this is that they are not accustomed to having a home that is technologically enhanced. In this article, we focus on the older persons’ lived experience of “home” and show how SHTs potentially disrupt it. In consulting the currently available literature, both theoretical and empirical, we propose and use the concept of somatic capability assessment (SCA) in the discussion surrounding the design of SHTs for older persons. First, we propose SCA as a concept to grasp how humans take decisions while relying on their physical body, undisturbed by suggestions from technologies. Furthermore, we show that SCA functions best in a familiar and private environment—the home. SHTs have the potential to make the home seem unfamiliar and exposed, precisely through added data and the resulting suggestions, as we show through related empirical studies. Thus, SHTs hold an increased disruptive potential for older persons at home. By introducing SCA into the discussion of SHTs for older persons, and thus paying attention on how SHTs potentially disrupt the experience of home, further advances the ethical discussion on the adequate use and design of technologies in daily life, especially for the group of older persons. Our analysis offers important insights for the design and implementation processes of SHTs for older persons.

Exploring the benefits of physical features in virtual reality on customer experience

Article scientifique

Cherix Robin, Julmy Maud, Capallera Marine, Roland Julien, Mugellini Elena

Proceedings of the IHCI 2024+ GET 2024+ CGVCVIP 2024, 13-15 July 2024, Budapest, Hungary, 2024 , pp. 162-165

Lien vers la publication

Résumé:

Virtual try-out experiences suffer from a lack of physicality. If consumers cannot touch or feel what they see, it is less
likely for them to buy it. Here we present work in progress of an immersive luxury watch try-out experience. We added a
3D printed, weighted mock-up to enrich the experience. This mock-up is worn on the wrist, where the virtual watch appears.
User tests show that the proposed experience is very well received and sparks interest for such experience.

2023

Older adults' perspectives on multimodal interaction with a conversational virtual coach

Article scientifique ArODES

Mira El Kamali, Denis Lalanne, Omar Abou Khaled, Elena Mugellini, Leonardo Angelini

Frontiers in Computer Science, 5, 112895

Lien vers la publication

Résumé:

The use of multiple interfaces may improve the perception of a stronger relationship between a conversational virtual coach and older adults. The purpose of this paper is to show the effect of output combinations [single-interface (chatbot, tangible coach), multi-interface (assignment, redundant-complementary)] of two distinct conversational agent interfaces (chatbot and tangible coach) on the eCoach-user relationship (closeness, commitment, complementarity) and the older adults' feeling of social presence of the eCoach. Our study was conducted with two different study settings: an online web survey and a face to face experiment. Our online study with 59 seniors shows that the output modes in multi-interface redundant-complementary manner significantly improves the eCoach-user relationship and social presence of the eCoach compared to only using single-interfaces outputs. Whereas in our face to face experiment with 15 seniors, significant results were found only in terms of higher social presence of multi-interface redundant complementary manner compared to chatbot only. We also investigated the effect of each study design on our results, using both quantitative and qualitative methods.

A dataset on the physiological state and behavior of drivers in conditionally automated driving

Article scientifique ArODES

Quentin Meteier, Marine Capallera, Emmanuel de Salis, Leonardo Angelini, Stefano Carrino, Marino Widmer, Omar Abou Khaled, Elena Mugellini, Andreas Sonderegger

Data in Brief, 2023, 47, 109027

Lien vers la publication

Résumé:

This dataset contains data of 346 drivers collected during six experiments conducted in a fixed-base driving simulator. Five studies simulated conditionally automated driving (L3-SAE), and the other one simulated manual driving (L0-SAE). The dataset includes physiological data (electrocardiogram (ECG), electrodermal activity (EDA), and respiration (RESP)), driving and behavioral data (reaction time, steering wheel angle, …), performance data of non-driving-related tasks, and questionnaire responses. Among them, measures from standardized questionnaires were collected, either to control the experimental manipulation of the driver's state, or to measure constructs related to human factors and driving safety (drowsiness, mental workload, affective state, situation awareness, situational trust, user experience). In the provided dataset, some raw data have been processed, notably physiological data from which physiological indicators (or features) have been calculated. The latter can be used as input for machine learning models to predict various states (sleep deprivation, high mental workload, ...) that may be critical for driver safety. Subjective self-reported measures can also be used as ground truth to apply regression techniques. Besides that, statistical analyses can be performed using the dataset, in particular to analyze the situational awareness or the takeover quality of drivers, in different states and different driving scenarios. Overall, this dataset contributes to better understanding and consideration of the driver's state and behavior in conditionally automated driving. In addition, this dataset stimulates and inspires research in the fields of physiological/affective computing and human factors in transportation, and allows companies from the automotive industry to better design adapted human-vehicle interfaces for safe use of automated vehicles on the roads.

ID Tech :

Article scientifique ArODES

a virtual reality simulator training for teenagers with intellectual disabilities

Marine Capallera, Geneviève Piérart, Francesco Carrino, Robin Cherix, Amélie Rossier, Elena Mugellini, Abou Khaled Omar

Applied Sciences, 2023, vol. 13, no. 3679, pp. 1-15

Lien vers la publication

Résumé:

People with intellectual disability (ID) should routinely train themselves to carry out a variety of daily challenging tasks while being supervised by one or more supervisors. Virtual reality (VR) technology enables the simulation of certain learning scenarios that would be risky or difficult to set up or repeatedly replicate in the actual world. This paper introduces a VR simulator created for this aim with the assistance of social educators. The purpose is to use VR to conduct learning exercises with teenagers with ID and assess the extent to which the abilities learned in VR can be transferred to the real world. This project focuses mainly on urban mobility with three types of exercise. A study was conducted with 18 students in five institutions for 7 months. Post-tests were also carried out after 1 month with 11 teenagers. In the end, four teenagers are completely autonomous in their travels: two travel on foot and two travel by public transport. Regarding the impact analysis, the results are stable over time regarding self-reported ease, satisfaction, and fatigability. Finally, the system received valuable feedback from the educators.

Co-designing an embodied e-coach with older adults :

Article scientifique ArODES

the tangible coach journey

Mira El Kamali, Leonardo Angelini, Maurizio Caon, Nick Dulake, Paul Chamberlain, Claire Craig, Carlo Emilio Standoli, Giuseppe Andreoni, Omar Abou Khaled, Elena Mugellini

International Journal of Human–Computer Interaction, 40, 10, 2736-2759

Lien vers la publication

Résumé:

This article describes a tangible interface for an e-coach, co-designed in four countries to meet older adults’ needs and expectations. The aim of this device is to coach the user by giving recommendations, personalized tasks and to build empathy through vocal, visual, and physical interaction. Through our co-design process, we collected insights that helped identifying requirements for the physical design, the interaction design and the privacy and data control. In the first phase, we collected users’ needs and expectations through several workshops. Requirements were then trans- formed into three design concepts that were rated and commented by our target users. The final design was implemented and tested in three countries. We discussed the results and the open challenges for the design of physical e-coaches for older adults. To encourage further developments in this field, we released the research outputs of this design process in an open-source repository.

A contextual multimodal system for increasing situation awareness and takeover quality in conditionally automated driving

Article scientifique ArODES

Marine Capallera, Quentin Meteier, Marino Widmer, Leonardo Angelini, Stefano Carrino, Andreas Sonderegger, Omar Abou Khaled, Elena Mugellini

IEEE Access, 2023, vol. 11, pp. 5746-5771

Lien vers la publication

Résumé:

Advanced driver assistances are becoming increasingly common in commercial cars, not only to assist but also to free drivers from manual driving whenever possible. Soon, drivers should be allowed to engage in non-driving-related tasks. The fact that responsibility for driving is shifting from humans to machines must be considered in the development of these assistances in order to guarantee safety and trust. In this article, we introduce AdVitam (for Advanced Driver-Vehicle Interaction to Make future driving safer), an autonomous system aiming at maintaining driver’s situation awareness and optimizing takeover quality during conditionally automated driving. The information conveyed to drivers is dynamically adapted to achieve these goals, depending on the driving environment and the driver’s physiological state. This system consists of three connected modules. The first module (Driver State) predicts the driver’s state with machine learning and physiological signals as inputs. The second module (Supervision) uses different interfaces (a haptic seat, a personal device, and ambient lights) to maintain the drivers’ situation awareness during the autonomous driving phases. The third module (Intervention) is a machine learning model that chooses the most appropriate combination among haptic, auditory, and visual modalities to request the driver to take over control and thus optimize takeover quality. To evaluate the system and each module independently, a preliminary user study with 35 drivers was conducted in a fixed-base driving simulator. All participants drove in two different environments (rural and urban). In addition, the activation of the Supervision and Intervention modules were manipulated as two between-subject factors. Results show that conveying information on the driving environment status through multimodal interfaces increases drivers’ situation awareness (i.e., better identification of potential problems in the environment) and trust in the automated vehicle. However, the system does not show positive outcomes on takeover quality. Besides, the Driver State module provided consistent predictions with the experimental manipulation. The system proposed in this paper could lead to better acceptance and safety when conditionally automated vehicles will be released by increasing drivers’ trust during phases of automated driving.

2022

Human-Vehicle Interaction to Support Driver's Situation Awareness in Automated Vehicles: A Systematic Review

Article scientifique ArODES

Marine Capallera, Leonardo Angelini, Quentin Meteier, Omar Abou Khaled, Elena Mugellini

IEEE Transactions on Intelligent Vehicles, 2023, Vol. 8, n° 3, pp. 2551 - 2567

Lien vers la publication

Résumé:

Autonomous driving will change the role of the driver. From being the main actor in driving, the driver will now have a supervisory role during the autonomous driving phases. However, if the driver has to take over control of the vehicle, he must be aware of the situation around him. This is why it is important to develop interfaces to keep him in the loop. This article proposes a systematic review of Human Vehicle Interaction (HVI) providing situation awareness in the context of autonomous driving. 37 articles presenting such interactions are analyzed in terms of design of the interaction (modalities, location, conveyed information) but also in term of evaluation and experimental conditions. We present an overview of previous studies in order to highlight the work already done or in progress. Current studies present mainly monomodal interfaces although the evaluation of multimodal interactions present promising results in this field.

Carrying a passenger and relaxation before driving :

Article scientifique ArODES

classification of young drivers’ physiological activation

Quentin Meteier, Marine Capallera, Emmanuel de Salis, Marino Widmer, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Andreas Sonderegger

Physiological Reports, 2022, vol. 10, article no. e15229

Lien vers la publication

Résumé:

Drivers are often held responsible for road crashes. Previous research has shown that stressors such as carrying passengers in the vehicle can be a source of accidents for young drivers. To mitigate this problem, this study investigated whether the presence of a passenger behind the wheel can be predicted using machine learning, based on physiological signals. It also addresses the question whether relaxation before driving can positively influence the driver's state and help controlling the potential negative consequences of stressors. Sixty young participants completed a 10-min driving simulator session, either alone or with a passenger. Before their driving session, participants spent 10 min relaxing or listening to an audiobook. Physiological signals were recorded throughout the experiment. Results show that drivers experience a higher increase in skin conductance when driving with a passenger, which can be predicted with 90%-accuracy by a k-nearest neighbors classifier. This might be a possible explanation for increased risk taking in this age group. Besides, the practice of relaxation can be predicted with 80% accuracy using a neural network. According to the statistical analysis, the potential beneficial effect of relaxation did not carry out on the driver's physiological state while driving, although machine learning techniques revealed that participants who exercised relaxation before driving could be recognized with 70% accuracy. Analysis of physiological characteristics after classification revealed several relevant physiological indicators associated with the presence of a passenger and relaxation.

Empathic and empathetic systematic review to standardize the development of reliable and sustainable empathic systems

Article scientifique ArODES

Karl Daher, Dahlia Saad, Elena Mugellini, Denis Lalanne, Omar Abou Khaled

Sensors, 2022, vol. 22, no. 8, article no. 3046

Lien vers la publication

Résumé:

Empathy plays a crucial role in human life, and the evolution of technology is affecting the way humans interact with machines. The area of affective computing is attracting considerable interest within the human–computer interaction community. However, the area of empathic interactions has not been explored in depth. This systematic review explores the latest advances in empathic interactions and behaviour. We provide key insights into the exploration, design, implementation, and evaluation of empathic interactions. Data were collected from the CHI conference between 2011 and 2021 to provide an overview of all studies covering empathic and empathetic interactions. Two authors screened and extracted data from a total of 59 articles relevant to this review. The features extracted cover interaction modalities, context understanding, usage fields, goals, and evaluation. The results reported here can be used as a foundation for the future research and development of empathic systems and interfaces and as a starting point for the gaps found.

The NESTORE e-Coach : designing a multi-domain pathway to well-being in older age

Article scientifique ArODES

Leonardo Angelini, Mira El Kamali, Elena Mugellini, Omar Abou Khaled, Christina Röcke, Simone Porcelli, Alfonso Mastropietro, Giovanni Rizzo, Noemi Boqué, Josep Maria Del Bas, Filippo Palumbo, Michele Girolami, Antonino Crivello, Canan Ziylan, Paula Subías-Beltrán, Silvia Orte, Carlo Emilio Standoli, Laura Fernandez Maldonado, Maurizio Caon, Martin Sykora, Suzanne Elayan, Sabrina Guye, Giuseppe Andreoni

Technologies, 2022, vol. 10, no 2, pp. 50-77

Lien vers la publication

Résumé:

This article describes the coaching strategies of the NESTORE e-coach, a virtual coach for promoting healthier lifestyles in older age. The novelty of the NESTORE project is the definition of a multi-domain personalized pathway where the e-coach accompanies the user throughout different structured and non-structured coaching activities and recommendations. The article also presents the design process of the coaching strategies, carried out including older adults from four European countries and experts from the different health domains, and the results of the tests carried out with 60 older adults in Italy, Spain and The Netherlands.

PEGASO e-Diary: :

Article scientifique ArODES

user engagement and dietary behavior change of a mobile food record for adolescents

Maurizio Caon, Federica Prinelli, Leonardo Angelini, Stefano Carrino, Elena Mugellini, Silvia Orte, José C. E. Serrano, Sarah Atkinson, Anne Martin, Fulvio Adorni

Frontiers in Nutrition, 2022, vol. 9, article 727480

Lien vers la publication

Résumé:

Background: Obesity amongst children and adolescents is becoming a major health problem globally and mobile food records can play a crucial role in promoting healthy dietary habits. Objective: To describe the methodology for the implementation of the e-Diary mobile food record, to assess its capability in promoting healthy eating habits, to evaluate the factors associated with its usage and engagement. Methods: This is a descriptive study that compared the characteristics of participants engaged in the e-Diary, which was part of the PEGASO project in which an app to provide proactive health promotion was given to 365 students at 4 European sites enrolled during October to December 2016: England (UK), Scotland (UK), Lombardy (Italy), and Catalonia (Spain). The e-Diary tracked the users' dietary habits in terms of food groups, dietary indexes, and 6 dietary target behaviors relating to consumption of: fruit; vegetable; breakfast; sugar-sweetened beverages; fast-food; and snacks. The e-Diary provided also personalized suggestions for the next meal and gamification. Results: The e-Diary was used for 6 months by 357 adolescents (53.8% females). The study showed that females used the e-Diary much more than males (aOR 3.8, 95% CI 1.6–8.8). Participants aged 14 years were more engaged in the e-Diary than older age groups (aOR 5.1, 95% CI 1.4–18.8) as were those with a very good/excellent self-perceived health status compared to their peers with fair/poor health perception (aOR 4.2, 95% CI 1.3–13.3). Compared to the intervention sites, those living in Catalonia (aOR 13.2 95% CI 2.5–68.8) were more engaged. In terms of behavior change, a significant positive correlation between fruit (p < 0.0001) and vegetables (p = 0.0087) intake was observed in association with increased engagement in the e-Diary. Similarly, adolescents who used the app for more than 2 weeks had significantly higher odds of not skipping breakfast over the study period (aOR 2.5, 95% CI 1.0–6.3). Conclusions: The users highly engaged with the e-Diary were associated with improved dietary behaviors: increased consumption of fruit and vegetables and reduced skipping of breakfast. Although the overall usage of the e-Diary was high during the first weeks, it declined thereafter. Future applications should foster user engagement, particularly targeting adolescents at high risk.

Non-intrusive contact respiratory sensor for vehicles

Article scientifique ArODES

Quentin Meteier, Michiel Kindt, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini

Sensors, 2022, vol. 22, no. 3, article no. 880

Lien vers la publication

Résumé:

In this work, we propose a low-cost solution capable of collecting the driver’s respiratory signal in a robust and non-intrusive way by contact with the chest and abdomen. It consists of a microcontroller and two piezoelectric sensors with their respective 3D printed plastic housings attached to the seat belt. An iterative process was conducted to find the optimal shape of the sensor housing. The location of the sensors can be easily adapted by sliding them along the seat belt. A few participants took part in three test sessions in a driving simulator. They had to perform various activities: resting, deep breathing, manual driving, and a non-driving-related task during automated driving. The subjects’ breathing rates were calculated from raw data collected with a reference chest belt, each sensor alone, and the fusion of the two. Results indicate that respiratory rate could be assessed from a single sensor located on the chest with an average absolute error of 0.92 min−1 across all periods, dropping to 0.13 min−1 during deep breathing. Sensor fusion did not improve system performance. A 4-pole filter with a cutoff frequency of 1 Hz emerged as the best option to minimize the error during the different periods. The results suggest that such a system could be used to assess the driver’s breathing rate while performing various activities in a vehicle.

Relevant physiological indicators for assessing workload in conditionally automated driving, through three-class classification and regression

Article scientifique ArODES

Quentin Meteier, Emmanuel de Salis, Marine Capallera, Marino Widmer, Leonardo Angelini, Omar Abou Khaled, Andreas Sonderegger, Elena Mugellini

Frontiers in Computer Science, 2022, vol. 3, article no. 775282

Lien vers la publication

Résumé:

In future conditionally automated driving, drivers may be asked to take over control of the car while it is driving autonomously. Performing a non-driving-related task could degrade their takeover performance, which could be detected by continuous assessment of drivers' mental load. In this regard, three physiological signals from 80 subjects were collected during 1 h of conditionally automated driving in a simulator. Participants were asked to perform a non-driving cognitive task (N-back) for 90 s, 15 times during driving. The modality and difficulty of the task were experimentally manipulated. The experiment yielded a dataset of drivers' physiological indicators during the task sequences, which was used to predict drivers' workload. This was done by classifying task difficulty (three classes) and regressing participants' reported level of subjective workload after each task (on a 0–20 scale). Classification of task modality was also studied. For each task, the effect of sensor fusion and task performance were studied. The implemented pipeline consisted of a repeated cross validation approach with grid search applied to three machine learning algorithms. The results showed that three different levels of mental load could be classified with a f1-score of 0.713 using the skin conductance and respiration signals as inputs of a random forest classifier. The best regression model predicted the subjective level of workload with a mean absolute error of 3.195 using the three signals. The accuracy of the model increased with participants' task performance. However, classification of task modality (visual or auditory) was not successful. Some physiological indicators such as estimates of respiratory sinus arrhythmia, respiratory amplitude, and temporal indices of heart rate variability were found to be relevant measures of mental workload. Their use should be preferred for ongoing assessment of driver workload in automated driving.

2021

Conversational agents for health and well-being across the life course :

Article scientifique ArODES

protocol for an evidence map

Mara Pereira Guerreiro, Leonardo Angelini, Helga Rafael Henriques, Mira El Kamali, Cristina Baixinho, João Balsa, Isa Félix Brito, Omar Abou Khaled, Maria Beatriz Carmo, Ana Paula Claudio, Maurizio Caon, Karl Daher, Bruno Alexandre, Mafalda Padinha, Elena Mugellini

JMIR Research Protocols, 2021, vol. 10, no. 9, article no. e26680

Lien vers la publication

Résumé:

Background: Conversational agents, which we defined as computer programs that are designed to simulate two-way human conversation by using language and are potentially supplemented with nonlanguage modalities, offer promising avenues for health interventions for different populations across the life course. There is a lack of open-access and user-friendly resources for identifying research trends and gaps and pinpointing expertise across international centers. Objective: Our aim is to provide an overview of all relevant evidence on conversational agents for health and well-being across the life course. Specifically, our objectives are to identify, categorize, and synthesize—through visual formats and a searchable database—primary studies and reviews in this research field. Methods: An evidence map was selected as the type of literature review to be conducted, as it optimally corresponded to our aim. We systematically searched 8 databases (MEDLINE; CINAHL; Web of Science; Scopus; the Cochrane, ACM, IEEE, and Joanna Briggs Institute databases; and Google Scholar). We will perform backward citation searching on all included studies. The first stage of a double-stage screening procedure, which was based on abstracts and titles only, was conducted by using predetermined eligibility criteria for primary studies and reviews. An operational screening procedure was developed for streamlined and consistent screening across the team. Double data extraction will be performed with previously piloted data collection forms. We will appraise systematic reviews by using A Measurement Tool to Assess Systematic Reviews (AMSTAR) 2. Primary studies and reviews will be assessed separately in the analysis. Data will be synthesized through descriptive statistics, bivariate statistics, and subgroup analysis (if appropriate) and through high-level maps such as scatter and bubble charts. The development of the searchable database will be informed by the research questions and data extraction forms. Results: As of April 2021, the literature search in the eight databases was concluded, yielding a total of 16,351 records. The first stage of screening, which was based on abstracts and titles only, resulted in the selection of 1282 records of primary studies and 151 records of reviews. These will be subjected to second-stage screening. A glossary with operational definitions for supporting the study selection and data extraction stages was drafted. The anticipated completion date is October 2021. Conclusions: Our wider definition of a conversational agent and the broad scope of our evidence map will explicate trends and gaps in this field of research. Additionally, our evidence map and searchable database of studies will help researchers to avoid fragmented research efforts and wasteful redundancies. Finally, as part of the Harnessing the Power of Conversational e-Coaches for Health and Well-being Through Swiss-Portuguese Collaboration project, our work will also inform the development of an international taxonomy on conversational agents for health and well-being, thereby contributing to terminology standardization and categorization.

Nutritional monitoring in older people prevention services

Chapitre de livre ArODES

Bhalaji Nagarajan, Rupali Khatun, Marc Bolaños, Eduardo Aguilar, Leonardo Angelini, Mira El Kamali, Elena Mugellini, Omar Abou Khaled, Noemi Boqué, Lucia Taro, Petia Radeva

Dans Andreoni, Giuseppe, Mambretti, Cinzia, Digital Health Technology for Better Aging : a multidisciplinary approach (pp. 77-102). 2021, Cham : Springer

Lien vers la publication

Résumé:

Nutritional monitoring is an important aspect of providing a healthy lifestyle and smarter food consumption. Food recognition is a key task in nutrition monitoring applications especially of interest for older people suffering from malnutrition and unhealthy eating. In order to ease the problem of food intake monitoring, recent health apps involve Artificial Intelligence algorithms for food image analysis. Food recognition is a highly challenging task as the dishes could be composed of mixed types, it could have a lower inter-class variance and also the number of dishes to be recognized is quite higher. In this chapter, we present an in-depth analysis and insights into the development of a nutritional monitoring system that would act as a Key Performance Indicator for the older people food intake monitoring. This system is aimed at providing a convenient food logging environment that would assist older people and their relatives as well as geriatrists in monitoring and maintaining the healthy daily dietary nutritional needs.

Understanding older adults’ affect states in daily life for promoting self-reflection about mental wellbeing

Chapitre de livre ArODES

Martin Sykora, Suzanne Elayan, Leonardo Angelini, Christina Röcke, Mira El Kamali, Elena Mugellini, Sabrina Guye

Dans Andreoni, Giuseppe, Mambretti, Cinzia, Digital Health Technology for Better Aging : a multidisciplinary approach (pp. 179-193). 2021, Cham : Springer

Lien vers la publication

Résumé:

In recent years, due to the rise in availability of unstructured big data and technological affordances, there has been a proverbial “explosion” of research within the field of sentiment analysis and affective computing, facilitating the study of in-situ communicated emotion and its accurate detection at an unprecedented scale. Increasingly the emotional dimension and its key important role is being recognized in e-coaching, with attempts at its integration within such systems. This chapter reviews the predominant extant models for representing affect, as well as emotion related states relevant to older persons’ experiences, including the state-of-art for their automated detection and integration within e-coaching systems. Given the approaches available, we describe how the NESTORE system addresses emotion recognition, self-tracking and subsequent feedback intended to help older adults make sense of their affective experiences across their daily life, as well as the role it can play in holistic e-coaching systems. Finally, from the NESTORE experience, we draw recommendations about how future e-coaching system could be implemented in the future to include the emotional dimension.

Coaching older adults towards a healthier lifestyle: psychological and technological methods

Chapitre de livre ArODES

Christina Röcke, Leonardo Angelini, Sabrina Guye, Mira El Kamali, Maurizio Caon, Omar Abou Khaled, Elena Mugellini

Dans Andreoni, Giuseppe, Mambretti, Cinzia, Digital health technology for better aging: a multidisciplinary approach (Pp. 161-177). 2021, Cham : Springer

Lien vers la publication

Résumé:

This chapter discusses the theoretical behaviour change framework and its integration and implementation of behaviour change techniques that form the conceptual psychological basis for innovative but efficient coaching approach. We review the current state of e-coaching solutions for older adults that can be found in the literature and identify gaps. Finally, we review and discuss the technological implementation of behaviour change techniques applied in NESTORE to support healthy older adults to adopt a healthy lifestyle in all NESTORE well-being domains as reference case study.

Building trust and companionship in e-coaching through embodiment

Chapitre de livre ArODES

Mira El Kamali, Leonardo Angelini, Maurizio Caon, Guiseppe Andreoni, Nick Dulake, Paul Chamberlain, Omar Abou Khlaed, Elena Mugellini

Dans Andreoni, Giuseppe, Mambretti, Cinzia, Digital health technology for better aging: a multidisciplinary approach (Pp. 195-204). 2021, Cham : Springer

Lien vers la publication

Résumé:

E-coaches are promising tools for health interventions, being scalable and relatively low costs compared to human coaches. Nevertheless, they cannot compare to humans when it comes to trust and social companionship. Hence, many researches were held around the importance of tangible user interfaces in building trust and companionship during human-machine interaction. Co-design, in particular, is the act of creating with stakeholders within the design development process to ensure the results meet their needs and are usable. This chapter describes a design process method and the rationale for building an embodiment for a companion and a trustable e-coach. We also shed lights on the co-design process held in the NESTORE project, which was specifically conducted with older adults to build a vocal assistant that supports user’s behavioral change, as well as to increase the social bonding with the NESTORE e-coach.

Locomotor illusions are generated by perceptual body-environment organization

Article scientifique ArODES

Martin Dobricki, David Weibel, Leonardo Angelini, Elena Mugellini, Fred W. Mast

PLOS ONE, 2021, vol. 16, no. 5, article no. e0251562

Lien vers la publication

Résumé:

While one is walking, the stimulation by one’s body forms a structure with the stimulation by the environment. This locomotor array of stimulation corresponds to the human-environment relation that one’s body forms with the environment it is moving through. Thus, the perceptual experience of walking may arise from such a locomotor array of stimulation. Humans can also experience walking while they are sitting. In this case, there is no stimulation by one’s walking body. Hence, one can experience walking although a basic component of a locomotor array of stimulation is missing. This may be facilitated by perception organizing the sensory input about one’s body and environment into a perceptual structure that corresponds to a locomotor array of stimulation. We examined whether locomotor illusions are generated by this perceptual formation of a locomotor structure. We exposed sixteen seated individuals to environmental stimuli that elicited either the perceptual formation of a locomotor structure or that of a control structure. The study participants experienced distinct locomotor illusions when they were presented with environmental stimuli that elicited the perceptual formation of a locomotor structure. They did not experience distinct locomotor illusions when the stimuli instead elicited the perceptual formation of the control structure. These findings suggest that locomotor illusions are generated by the perceptual organization of sensory input about one’s body and environment into a locomotor structure. This perceptual body-environment organization elucidates why seated human individuals experience the sensation of walking without any proprioceptive or kinaesthetic stimulation.

Classification of drivers' workload using physiological signals in conditional automation

Article scientifique ArODES

Quentin Meteier, Marine Capallera, Simon Ruffieux, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Andreas Sonderegger

Frontiers in Psychology, 2021, vol. 12, article no. 596038

Lien vers la publication

Résumé:

The use of automation in cars is increasing. In future vehicles, drivers will no longer be in charge of the main driving task and may be allowed to perform a secondary task. However, they might be requested to regain control of the car if a hazardous situation occurs (i.e., conditionally automated driving). Performing a secondary task might increase drivers' mental workload and consequently decrease the takeover performance if the workload level exceeds a certain threshold. Knowledge about the driver's mental state might hence be useful for increasing safety in conditionally automated vehicles. Measuring drivers' workload continuously is essential to support the driver and hence limit the number of accidents in takeover situations. This goal can be achieved using machine learning techniques to evaluate and classify the drivers' workload in real-time. To evaluate the usefulness of physiological data as an indicator for workload in conditionally automated driving, three physiological signals from 90 subjects were collected during 25 min of automated driving in a fixed-base simulator. Half of the participants performed a verbal cognitive task to induce mental workload while the other half only had to monitor the environment of the car. Three classifiers, sensor fusion and levels of data segmentation were compared. Results show that the best model was able to successfully classify the condition of the driver with an accuracy of 95%. In some cases, the model benefited from sensors' fusion. Increasing the segmentation level (e.g., size of the time window to compute physiological indicators) increased the performance of the model for windows smaller than 4 min, but decreased for windows larger than 4 min. In conclusion, the study showed that a high level of drivers' mental workload can be accurately detected while driving in conditional automation based on 4-min recordings of respiration and skin conductance.

2020

"Hi This Is NESTORE, your personal assistant" :

Article scientifique ArODES

design of an integrated IoT system for a personalized coach for healthy aging

Filippo Palumbo, Antonino Crivello, Francesco Furfari, Michele Girolami, Alfonso Mastropietro, Giorgio Manfredelli, Christina Röcke, Sabrina Guye, Antoni Salvá Casanovas, Maurizio Caon, Francesco Carrino, Omar Abou Khaled, Elena Mugellini, Enrico Denna, Marco Mauri, David Ward, Paula Subías-Beltrán, Silvia Orte, Ciprian Candea, Gabriele Candea, Giovanna Rizzo

Frontiers in Digital Health, 2020, vol. 2, article 545949, pp. 1-17

Lien vers la publication

Résumé:

In the context of the fourth revolution in healthcare technologies, leveraging monitoring and personalization across different domains becomes a key factor for providing useful services to maintain and promote well-being. This is even more crucial for older people, with aging being a complex multi-dimensional and multi-factorial process which can lead to frailty. The NESTORE project was recently funded by the EU Commission with the aim of supporting healthy older people to sustain their well-being and capacity to live independently. It is based on a multi-dimensional model of the healthy aging process that covers physical activity, nutrition, cognition, and social activity. NESTORE is based on the paradigm of the human-in-the-loop cyber-physical system that, exploiting the availability of Internet of Things technologies combined with analytics in the cloud, provides a virtual coaching system to support healthy aging. This work describes the design of the NESTORE methodology and its IoT architecture. We first model the end-user under several domains, then we present the NESTORE system that, analyzing relevant key-markers, provides coaching activities and personalized feedback to the user. Finally, we describe the validation strategy to assess the effectiveness of NESTORE as a coaching platform for healthy aging.

Design of an ergonomic gestural interface for professional road cycling

Article scientifique ArODES

Maurizio Caon, Rico Süsse, Benoit Grelier, Omar Abou Khaled, Elena Mugellini

Work, 2020, vol. 66, pp. 933-944

Lien vers la publication

Résumé:

BACKGROUND: Connected bike computers can support professional cyclists in achieving better performances but interacting with them requires taking their hands off the handlebar compromising focus and safety. OBJECTIVE: This research aims at exploring the design of an ergonomic interface based on micro-gestures that can allow cyclists to interact with a device while holding the handlebar. METHODS: Three different studies were conducted with seven professional cyclists adopting the gesture-elicitation technique. One study aimed at eliciting free micro-gestures; a second to evaluate gestures recognizable with a smart glove; the last focused on the gestures recognized through an interactive armband. RESULTS: The analysis of the micro-gestures elicited during these studies allowed producing a first set of guidelines to design gestural interfaces for drop-bars (a specific type of handlebar for road bikes). These guidelines suggest which fingers to use and how to design their movement in order to provide an ergonomic interface. It also introduces the principle of symmetry for the attribution of symbols to symmetric referents. Finally, it provides suggestions on the design of the interactive drop-bar. CONCLUSIONS: The guidelines provided in this paper can support the design of gestural interfaces for professional cyclists that can enhance performance and increase safety.

The effect of instructions and context-related information about limitations of conditionally automated vehicles on situation awareness

Article scientifique ArODES

Quentin Meteier, Marine Capallera, Andreas Sonderegger, Leonardo Angelini, Stefano Carrino, Elena Mugellini, Omar Abou Khaled

Proceedings of AutomotiveUI '20: 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 21-22 September 2020, Online Conference, pp. 241-251

Lien vers la publication

Résumé:

In conditionally automated driving, drivers do not have to constantly monitor their vehicle but they must be able to take over control when necessary. In this paper, we assess the impact of instructions about limitations of automation and the presentation of context-related information through a mobile application on the situation awareness and takeover performance of drivers. We conducted an experiment with 80 participants in a fixed-base driving simulator. Participants drove for an hour in conditional automation while performing secondary tasks on a tablet. Besides, they had to react to five different takeover requests. In addition to the assessment of behavioral data (e.g. quality of takeover), participants rated their situation awareness after each takeover situation. Instructions and context-related information on limitations combined showed encouraging results to raise awareness and improve takeover performance.

Virtual coaches for older adults’ wellbeing :

Article scientifique ArODES

a systematic review

Mira El Kamali, Leonardo Angelini, Maurizio Caon, Francesco Carrino, Christina Röcke, Sabrina Guye, Giovanna Rizzo, Alfonso Mastropietro, Martin Sykora, Suzanne Elayan, Isabelle Kniestedt, Canan Ziylan, Emanuele Lettieri, Omar Abou Khaled, Elena Mugellini

IEEE Access, 2020, vol. 8, pp. 101884-101902

Lien vers la publication

Résumé:

Virtual Coaches, also known as e-coaches, are a disruptive technology in healthcare. Indeed, among other usages, they might provide cost-effective solutions for increasing human wellbeing in different domains, such as physical, nutritional, cognitive, social, and emotional. This paper presents a systematic review of virtual coaches specifically aimed at improving or maintaining older adults' health in the aforementioned domains. Such digital systems assume various forms, from classic apps, to more advanced conversational agents or robots. Fifty-six articles describing a virtual coach for older adults and aimed at improving their wellbeing were identified and further analyzed. In particular, we presented how previous studies defined their virtual coaches, which behavioral change models and techniques they adopted and the overall system architecture, in terms of monitoring solutions, processing methods and modalities for intervention delivery. Our results show that few thorough evaluations of e-coaching systems have been conducted, especially regarding multi-domain coaching approaches. Through our analysis, we identified the wellbeing domains that should be addressed in future studies as well as the most promising behavior change models and techniques and coaching interfaces. Previous work illustrates that older adults often appreciate conversational agents and robots. However, the lack of a multidomain intervention approach in the current literature motivates us to seek to define future solutions.

Personage VR – A Virtual Reality Story-Telling Tool to Raise Awareness About Ageism.

Chapitre de livre

Carrino Francesco, Moullet Valentin, Abou Khaled Omar, Mugellini Elena, Maggiori Christian

Dans T. Ahram, R. Taiar, V. Gremeaux-Bader & K. Aminian, Human Interaction, Emerging Technologies and Future Applications II. 2020, Cham, Switzerland : Advances in Intelligent Systems and Computing

Étude exploratoire pour un programme de sensibilisation vis-à-vis de l’âgisme

Rapport

Rapport intermédiaire

Maggiori Christian, Mugellini Elena, Carrino Francesco, F. Angelini, Curty Gaël

2020, Lausanne, Suisse : Fondation Leenaards, 000 p.

Training Pedestrian Safety Skills in Youth with Intellectual Disabilities Using Fully Immersive Virtual Reality.

Article scientifique

A Feasibility Study

Cherix Robin, Piérart Geneviève, Abou Khaled Omar, Mugellini Elena,

HCI International 2020 - Late Breaking Papers., 2020

Lien vers la publication

2019

No evidence that frontal optical flow affects perceived locomotor speed and locomotor biomechanics when running on a treadmill

Article scientifique ArODES

Martina Caramenti, Claudio L. Lafortuna, Elena Mugellini, Omar Abou Khaled, Jean-Pierre Bresciani, Amandine Dubois

Applied Sciences, 2019, vol. 9, no. 21

Lien vers la publication

Résumé:

We investigated how the presentation and the manipulation of an optical flow while running on a treadmill affect perceived locomotor speed (Experiment 1) and gait parameters (Experiment 2). In Experiment 1, 12 healthy participants were instructed to run at an imposed speed and to focus on their sensorimotor sensations to be able to reproduce this running speed later. After a pause, they had to retrieve the reference locomotor speed by manipulating the treadmill speed while being presented with different optical flow conditions, namely no optical flow or a matching/slower/faster optical flow. In Experiment 2, 20 healthy participants ran at a previously self-selected constant speed while being presented with different optical flow conditions (see Experiment 1). The results did not show any effect of the presence and manipulation of the optical flow either on perceived locomotor speed or on the biomechanics of treadmill running. Specifically, the ability to retrieve the reference locomotor speed was similar for all optical flow conditions. Manipulating the speed of the optical flow did not affect the spatiotemporal gait parameters and also failed to affect the treadmill running accommodation process. Nevertheless, the virtual reality conditions affected the heart rate of the participants but without affecting perceived effort.

The NESTORE e-coach :

Article scientifique ArODES

accompanying older adults through a personalized pathway to wellbeing

Leonardo Angelini, Elena Mugellini, Omar Abou Khaled, Christina Röcke, Sabrina Guye, Simone Porcelli, Alfonso Mastropietro, Giovanna Rizzo, Noemi Boqué, Paula Subias, Silvia Orte, Giuseppe Andreoni

ICPS Proceedings, PETRA'19 : Proceedings of the 12th ACM International Conference on Pervasive Technologies Related to Assistive Environments, 5-7 June 2019, Rhodes, Greece, pp. 620-628

Lien vers la publication

Résumé:

This paper describes the NESTORE e-coaching strategy and system architecture and its unique approach to support older adults to achieve a healthier lifestyle. The novelty of the NESTORE project is the definition of a multi-domain personalized pathway where the e-coach accompany the user throughout different structured and non-structured coaching activities and recommendations. The NESTORE e-Coach is the result of a highly multidisciplinary EU project, granted under the call SC1-PM-15-2017.

Codesign of public spaces for intercultural communication, diversity and inclusion

Chapitre de livre ArODES

Alessandra Rinaldi, Leonardo Angelini, Abou Khaled, Elena Mugellini, Maurizio Caon

Dans Di Bucchianico, Giuseppe, Advances in design for inclusion : proceedings of the AHFE 2019 International Conference on design for inclusion and the AHFE 2019 International Conference on human factors for apparel and textile engineering, July 24-28, 2019, Washington D.C., USA (Pp. 186-195). 2019, Cham, Switzerland : Springer

Lien vers la publication

Résumé:

The consistent phenomenon of immigration from countries with different cultures, that characterized the last decade, has changed the use of public spaces in European cities. The need to develop innovative solutions aimed at improving intercultural inclusion and mutual understanding of all citizens emerges, so that urban spaces become inclusive, safe and sustainable. This research project is focused on two main questions: (1) can design and technology enable the creation of a different way of living the public space to catalyze interpersonal interaction and to promote a shared sense of citizenship? (2) is it possible to support intercultural communication and to facilitate social inclusion through the design of innovative and interactive public spaces equipped with smart urban furniture? This paper presents three Design Orienting Scenarios as results of the research conducted by several experts, professionals and people with different cultural backgrounds during a focus groups and two different workshops where they made converge competences from design, business, engineering and social sciences to find innovative solutions for the social inclusion of migrants in urban areas.

Regular physical activity modulates perceived visual speed when running in treadmill-mediated virtual environments

Article scientifique ArODES

Martina Caramenti, Claudio L. Lafortuna, Elena Mugellini, Omar Abou Khaled, Jean-Pierre Bresciani, Amandine Dubois

PLOS ONE,

Lien vers la publication

Résumé:

In virtual reality, visual speed is usually underestimated relative to locomotor speed. Here we investigated how physical activity and fitness affect perceived visual speed when running in a treadmill-mediated virtual environment. Thirty healthy participants (ten sedentary individuals, ten team sport players and ten expert runners) ran on a treadmill at two different speeds (8, 12km/h) in front of a moving virtual scene. Participants were asked to match the speed of the visual scene to their running speed (i.e. treadmill speed), indicating for each trial whether the scene was moving slower or faster than the treadmill. The speed of the visual scene was adjusted according to the participant’s response using a staircase until visual and running speeds were perceived as equivalent. More sedentary participants underestimated visual speed relative to their actual running speed. Specifically, visual speed had to exceed running speed to be perceived as equivalent. The underestimation of visual speed was speed-dependent, and it was significantly larger for sedentary participants than for team sports players and expert runners. The volume of physical activity per week was found to be the best predictor of visual speed perception for both running speeds, while the perceived effort constituted a good predictor only at 8km/h. Physical fitness, on the other hand turned out to be a poor predictor of visual speed perception. Therefore, in order to enhance users’ engagement and their adherence to physical activity programs, the development of “personalized” treadmill-mediated virtual environments should take into account users’ personal characteristics to provide the most natural and engaging feedback possible.

Owner Manuals Review and Taxonomy of ADAS Limitations in Partially Automated Vehicles

Article scientifique

Capallera Marine, Meteier Quentin, De Salis Emmanuel, Carrino Stefano, Angelini Leonardo, Abou Khaled Omar, Mugellini Elena

AutomotiveUI '19: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 2019

Lien vers la publication

Résumé:

In the context of highly automated driving, the driver has to be aware of driving risks and to take over control of the car in hazardous situations. The goal of this paper is to categorize and analyze the factors that lead to such critical scenarios. To this purpose, we analyzed limitations of Advanced Driver-Assistance Systems (ADAS) extracted from owner manuals of 12 partially automated cars available on the market. A taxonomy with 6 macro-categories and 26 micro-categories is proposed to classify and better understand the limitations of these vehicles. We also investigated if these limitations are conveyed to the driver through Human-Machine Interaction (HMI) in the car. Some suggestions are made to better communicate these limitations to the driver in order to raise his/her situation awareness.

Workshop on explainable AI in automated driving: a user-centered interaction approach

Article scientifique

Meteier Quentin, Capallera Marine, Angelini Leonardo, Mugellini Elena, Abou Khaled Omar, Carrino Stefano, De Salis Emmanuel, Stéphane Galland, Susanne Boll

AutomotiveUI '19: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings, 2019

Lien vers la publication

Résumé:

With the increasing use of automation, users tend to delegate more tasks to the machines. Such complex systems are usually developed with "black box" Artificial Intelligence (AI), which makes these systems difficult to understand for the user. This assumption is particularly true in the field of automated driving since the level of automation is constantly increasing via the use of state-of-the-art AI solutions. We believe it is important to investigate the field of Explainable AI (XAI) in the context of automated driving since interpretability and transparency are key factors for increasing trust and security. In this workshop, we aim at gathering researchers and industry practitioners from different fields to brainstorm about XAI with a special focus on human-vehicle interaction. Questions like "what kind of explanation do we need", "which is the best trade-off between performance and explainability" and "how granular should the explanations be" will be addressed in this workshop.

2018

Utilizzare il design partecipativo per l'innovazione sostenibile

Article scientifique ArODES

Maurizio Caon, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini

Rivista italiana di ergonomia, 2018, no. 16, pp. 9-23

Lien vers la publication

Designing urban smart furniture for facilitating migrants' integration :

Chapitre de livre ArODES

the co-design workshop as approach for supporting inclusive design

Alessandra Rinaldi, Maurizio Caon, Omar Abou Khaled, Elena Mugellini

Dans Bagnara, Sebastiano, Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018) : volume VII: ergonomics in design, design for all, activity theories for work analysis and design, affective design (Pp. 461-470). 2018, Cham : Springer

Lien vers la publication

Résumé:

European cities are changing due the immigration of people from different cultures. There are many issues related to the integration and dialogue between cultures. Urban design plays a key role in how migrants participate in their host community and it is an important driver for the inclusion process. Facilitating the participation of residents in designing public spaces and their use helps to create a better sense of belonging. Promoting in public spaces the interaction of different cultures becomes a crucial element to facilitate social cohesion and living together. The research project explores an innovative approach in the use of public spaces, through the design of smart urban furniture. Based on a survey of the user’s needs, and on co-design activities, the project investigates innovative solutions for facilitating migrants’ integration and the dialogue between different cultures, through the design of innovative urban furniture, with smart technologies embodied inside. The project, based on the co-design approach of rapid prototyping, creates different kinds of new interaction among urban space and people, and between users of different social or cultural background. The results presented in this paper, were conducted by the research unit of the University of Florence, in cooperation with the Human Tech Institute and the School of Management of the University of Applied Sciences and Arts Western Switzerland.

Gesturing on the handlebar :

Chapitre de livre ArODES

a user-elicitation study for on-bike gestural interaction

Maurizio Caon, Rico Süsse, Benoit Grelier, Omar Abou Khaled, Elena Mugellini

Lien vers la publication

Résumé:

There are growing numbers of apps and smartphone-mounts for professional cyclists, since they are crucial to track performances during training. However, these solutions require the athlete to take her hand off the handlebar to interact with it. This represents a major safety issue for the cyclists since it requires leaving the brake control, shifting the attention and, possibly, compromising posture. This paper reports the findings of a user elicitation study conducted with seven professional and semi-professional cyclists in order to design gestures that can be performed while maintaining the hands in the correct position on the handlebar. Results report the frequency of fingers used for these gestures, with the index being the favorite. Furthermore, it provides a classification of gestures in three categories: press, extension and swipe. The most convenient gestures were the thumb and index press, followed by the extension of different combinations of fingers.

My pain coach :

Chapitre de livre ArODES

a mobile system with tangible interface for pain assessment

Maurizio Caon, Leonardo Angelini, Katharina Ledermann, Chantal Martin-Sölch, Omar Abou Khaled, Elena Mugellini

Lien vers la publication

Résumé:

Pain assessment is an essential first step in planning for and providing pain management. Access to effective pain management programs is often limited, due to scarcity of services. Other barriers to treatment include physical symptoms that limit mobility, distance from a clinic, transportation requirements and cost constraints. In response to these barriers to service delivery, the demand for online health resources is growing. However, current solutions present a number of shortcomings, in particular when referring to usability and accessibility. This paper presents the vision of the My Pain Coach system and the first prototype generated as the result of the first iteration of the development process. This prototype is based on a smartphone app and presents two different interfaces: a force-sensitive touchscreen, and a tangible interface based on a textile mat. A preliminary usability evaluation has been conducted and the results show that these interfaces are perceived as excellent from a usability point of view. Nevertheless, further development and testing are still required for the tangible interface.

Eudaimonic gamification to engage cancer patients in positive coping strategies

Chapitre de livre ArODES

João Ventura, Sandy Ingram, Maurizio Caon, Maya Zumstein-Shaha, Omar Abou Khaled, Elena Mugellini

Dans Kurosu, Masaaki, Human-computer interaction : interaction in context : 20th International Conference, HCI International 2018, Las Vegas, NV, USA, July 15–20, 2018, Proceedings, Part II (Pp. 206-218). 2018, Cham : Springer

Lien vers la publication

Résumé:

This paper presents a mobile gamified application encouraging positive coping strategies for patients of serious and possibly fatal illnesses. The application concept is based on the analogy between one’s lifetime memories and future bucket lists and a journey in the sea of remembrance, traveling back and forth between past and present positive moments and future wish-lists, whilst being aware that life as water, goes on. This concept was co-designed by a team of UX researchers, engineers, and domain experts applying the principles of Eudaimonic design. The iterative development process brought to the final prototype: a multimedia diary with a gameful interface, which is thoroughly described, along with the relative heuristic and empirical evaluations, in this paper.

Linking user profiles in social networks :

Article scientifique ArODES

a comparative review

Hussein Hazimeh, Elena Mugellini, Omar Abou Khaled, Philippe Cudré-Mauroux

International Journal of Social Network Mining, 2018, vol. 2, no. 4, pp. 333-361

Lien vers la publication

Résumé:

The existence of user profiles belonging to a single user across different social networking sites poses several challenges to the research community. The main technical issue in that context is to detect the same user profiles across several different social networks by leveraging a set of mechanisms that identify the similarity among the user profiles. This problem is commonly referred to as entity matching or identity linkage on social networks. In this review, we describe and compare the 27 most important (to the best of our knowledge) research papers in this area. The main contributions of this article are to provide a systematic and integrated review of papers in this area, to provide comparative points that simplify the understanding of such systems, and finally to discuss future research avenues.

Lexicon-based sentiment analysis approach for ranking event entities

Article scientifique ArODES

Sajida Chamass, Hussein Hazimeh, Jawad Makki, Elena Mugellini, Omar Abou Khaled

International Journal of Services and Standards, 2018, vol. 12, no. 2, pp. 126-139

Lien vers la publication

Résumé:

Social media platforms (SMP) are new resource for data analytics. Multiple aspects can be studied by using its variety of features. Sentiment analysis (SA) is a rising research topic in SMPs. SA approaches on studying and analysing events are still missing several shortcomings. In this paper, we address the problem of ranking event entities and propose a novel approach for this goal. An entity is a person who presents some task in such event, for e.g., a researcher in a conference. To achieve our target, we employ the lexical approach, in addition to associating features from both Facebook and Twitter platforms. We used Facebook reactions also, that not been used in the state-of-the-art approaches. Our results have shown that by associating both features from Facebook and Twitter and by using reactions, we can successfully rank entities participating in a specific event having high precision.

Matching optical flow to motor speed in virtual reality while running on a treadmill

Article scientifique ArODES

Martina Caramenti, Claudio L. Lafortuna, Elena Mugellini, Omar Abou Khaled, Jean-Pierre Bresciani, Amandine Dubois

PLOS ONE,

Lien vers la publication

Résumé:

We investigated how visual and kinaesthetic/efferent information is integrated for speed perception in running. Twelve moderately trained to trained subjects ran on a treadmill at three different speeds (8, 10, 12 km/h) in front of a moving virtual scene. They were asked to match the visual speed of the scene to their running speed–i.e., treadmill’s speed. For each trial, participants indicated whether the scene was moving slower or faster than they were running. Visual speed was adjusted according to their response using a staircase until the Point of Subjective Equality (PSE) was reached, i.e., until visual and running speed were perceived as equivalent. For all three running speeds, participants systematically underestimated the visual speed relative to their actual running speed. Indeed, the speed of the visual scene had to exceed the actual running speed in order to be perceived as equivalent to the treadmill speed. The underestimation of visual speed was speed-dependent, and percentage of underestimation relative to running speed ranged from 15% at 8km/h to 31% at 12km/h. We suggest that this fact should be taken into consideration to improve the design of attractive treadmill-mediated virtual environments enhancing engagement into physical activity for healthier lifestyles and disease prevention and care.

Feet fidgeting detection based on accelerometers using decision tree learning and gradient boosting

Chapitre de livre ArODES

Julien Esseiva, Maurizio Caon, Elena Mugellini, Omar Abou Khaled, Kamiar Aminian

Dans Ortuño, Francesco, Rojas, Ignacio, Bioinformatics and Biomedical Engineering : 6th International Work-Conference, IWBBIO 2018, Granada, Spain, April 25–27, 2018, Proceedings, Part I (Pp. 75-84). 2018, Cham : Springer

Lien vers la publication

Résumé:

Detection of fidgeting activities is a field which has not been much explored as of now. Studies have shown that fidgeting has a beneficial impact on people's healthiness as it burns a significant amount of energy. Being able to detect when someone is fidgeting would allow to study more closely the health impact of fidgeting. The purpose of this work is to propose an algorithm being able to detect feet fidgeting period of subjects while sitting using 3D accelerometers on both shoes. Initial results on data from 5 subjects collected during this work shows an accuracy of 95% for a classification between sitting with fidgeting and sitting without fidgeting.

Internet of tangible things (IoTT) :

Article scientifique ArODES

challenges and opportunities for tangible interaction with IoT

Leonardo Angelini, Elena Mugellini, Omar Abou Khaled, Nadine Couture

Informatics, 2018, vol. 5(1), no. 7

Lien vers la publication

Résumé:

In the Internet of Things era, an increasing number of everyday objects are able to offer innovative services to the user. However, most of these devices provide only smartphone or web user interfaces. As a result, the interaction is disconnected from the physical world, decreasing the user experience and increasing the risk of user alienation from the physical world. We argue that tangible interaction can counteract this trend and this article discusses the potential benefits and the still open challenges of tangible interaction applied to the Internet of Things. After an analysis of open challenges for Human-Computer Interaction in IoT, we summarize current trends in tangible interaction and extrapolate eight tangible interaction properties that could be exploited for designing novel interactions with IoT objects. Through a systematic review of tangible interaction applied to IoT, we show what has been already explored in the systems that pioneered the field and the future explorations that still have to be conducted. In order to guide future work in this field, we propose a design card set for supporting the generation of tangible interfaces for IoT objects. The card set has been evaluated during a workshop with 21 people and the results are discussed.

Towards the nestore e-coach: a tangible and embodied conversational agent for older adults

Article scientifique

El Kamali Mira, Angelini Leonardo, Caon Maurizio, Giuseppe Andreoni, Abou Khaled Omar, Mugellini Elena

Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, 2018

Lien vers la publication

Résumé:

The ability to engage the user in a conversation and the credibility of the system are two fundamental characteristics of virtual coaches. In this paper, we present the architecture of a conversational e-coach for promoting healthy lifestyles in older age, developed in the context of the NESTORE H2020 EU project. The proposed system allows multiple access points to a conversational agent via different interaction modalities: a tangible companion that embodies the virtual coach will leverage voice and other non-verbal cues in the domestic environment, while a mobile app will integrate a text-based chat for a ubiquitous intervention. In both cases, the conversational agent will deliver personalized interventions based on behavior change models and will promote trust by means of emotionally rich conversations.

2017

Human behavior modeling In video surveillance of conference halls

Article scientifique ArODES

Nour Charara, Hussein Charara, Omar Abou Khaled, Hani Abdallah, Elena Mugellini

World Academy of Science, Engineering and Technology ; International Journal of Computer and Information Engineering, 2017, vol. 11, no. 9, pp. 996-1001

Lien vers la publication

Résumé:

In this paper, we present a human behavior modeling approach in videos scenes. This approach is used to model the normal behaviors in the conference halls. We exploited the Probabilistic Latent Semantic Analysis technique (PLSA), using the 'Bag-of-Terms' paradigm, as a tool for exploring video data to learn the model by grouping similar activities. Our term vocabulary consists of 3D spatio-temporal patch groups assigned by the direction of motion. Our video representation ensures the spatial information, the object trajectory, and the motion. The main importance of this approach is that it can be adapted to detect abnormal behaviors in order to ensure and enhance human security.

Bien-être psychologique avec l'âge

Article professionnel ArODES

Leonardo Angelini, Francesco Carrino, Maurizio Caon, Elena Mugellini, Omar Abou Khaled, Nadine Couture

Angewandte GERONTOLOGIE Appliquée, 2017, vol. 2

Lien vers la publication

Résumé:

La perte progressive de mobilité chez les seniors peut avoir de lourdes conséquences, pas seulement physiques, mais aussi psychiques et cognitives due à un isolement social croissant. La fenêtre interactive est un outil permettant la communication à distance. Elle est issue d’un projet de recherche qui vise à réduire l’isolement social des seniors en utilisant de nouvelles technologies de manière innovante.

2016

Senior Living Lab :

Article scientifique ArODES

an ecological approach to foster social innovation in an ageing society

Stefano Angelini, Stefano Carrino, Omar Abou Khaled, Susie Riva-Mossman, Elena Mugellini

Future Internet, 2016, vol. 8, no. 50

Lien vers la publication

Résumé:

The Senior Living Lab (SLL) is a transdisciplinary research platform created by four Universities that aims at promoting ageing well at home through the co-creation of innovative products, services and practices with older adults. While most living labs for ageing well are focused on Information and Communication Technologies (ICTs), this social laboratory adopts a transdisciplinary approach, bringing together designers, economists, engineers and healthcare professionals to develop multiple forms of social innovation using participatory methods. The SLL is based on an ecological approach, connecting professionals and users in a cooperative network and involving all of the stakeholders concerned with ageing well, such as existing associations, business entities and policy-makers. Three main themes for the co-design of products and services were identified at the beginning of the SLL conception, each sustained by a major business partner: healthy nutrition to cope with frailty, improved autonomous mobility to foster independence and social communication to prevent isolation. This article shows the innovative transdisciplinary approach of the SLL and discusses the particular challenges that emerged during the first year of its creation, investigating the role of ICTs when designing products and services for older adults.

iNUIT :

Article scientifique ArODES

Internet of Things for Urban Innovation

Francesco Carrino, Elena Mugellini, Omar Abou Khaled, Nabil Ouerhani, Juergen Ehrensberger

Future Internet, 2016, vol. 8, no. 18

Lien vers la publication

Résumé:

Internet of Things (IoT) seems a viable way to enable the Smart Cities of the future. iNUIT (Internet of Things for Urban Innovation) is a multi-year research program that aims to create an ecosystem that exploits the variety of data coming from multiple sensors and connected objects installed on the scale of a city, in order to meet specific needs in terms of development of new services (physical security, resource management, etc.). Among the multiple research activities within iNUIT, we present two projects: SmartCrowd and OpEc. SmartCrowd aims at monitoring the crowd’s movement during large events. It focuses on real-time tracking using sensors available in smartphones and on the use of a crowd simulator to detect possible dangerous scenarios. A proof-of-concept of the application has been tested at the Paléo Festival (Switzerland) showing the feasibility of the approach. OpEc (Optimisation de l’Eclairage public) aims at using IoT to implement dynamic street light management and control with the goal of reducing street light energy consumption while guaranteeing the same level of security of traditional illumination. The system has been tested during two months in a street in St-Imier (Switzerland) without interruption, validating its stability and resulting in an overall energy saving of about 56%.

Testing the tangible interactive window with older adults

Article scientifique ArODES

Leonardo Angelini, Francesco Carrino, Maurizio Caon, Frédéric Lemaréchal, Nadine Couture, Omar Abou Khaled, Elena Mugellini

GeroPsych, 2016, vol. 29, no. 4, pp. 215-224

Lien vers la publication

Résumé:

Social relationships are crucial for older adults’ well-being. Past research demonstrated that video-communication technology could reduce social isolation. However, most commercial telecommunication systems have interfaces that are too complex to be operated by older adults without assistance. In this paper, we present a system that exploits the tangible affordances of a physical window to provide a natural interface for an always-on video-communication system. In addition, we present a case study focused on the system accessibility and user acceptance with 8 older adults living in a retirement home. We also describe the preliminary findings based on an observation study and on a semistructured interview. Finally, we discuss the possible improvements and impact this system might have on older adults’ life.

iNUIT: Internet of Things for Urban Innovation

Article scientifique

Ouerhani Nabil, Carrino Francesco, Abou Khaled Omar, Mugellini Elena, Ehrensberger Jürgen

Future Internet Journal, 2016

2015

Move, hold and touch :

Article scientifique ArODES

a framework for tangible gesture interactive systems

Leonardo Angelini, Denis Lalanne, Elise van den Hoven, Omar Abou Khaler, Elena Mugellini

Machines, 2015, vol. 3, pp. 173-207

Lien vers la publication

Résumé:

Technology is spreading in our everyday world, and digital interaction beyond the screen, with real objects, allows taking advantage of our natural manipulative and communicative skills. Tangible gesture interaction takes advantage of these skills by bridging two popular domains in Human-Computer Interaction, tangible interaction and gestural interaction. In this paper, we present the Tangible Gesture Interaction Framework (TGIF) for classifying and guiding works in this field. We propose a classification of gestures according to three relationships with objects: move, hold and touch. Following this classification, we analyzed previous work in the literature to obtain guidelines and common practices for designing and building new tangible gesture interactive systems. We describe four interactive systems as application examples of the TGIF guidelines and we discuss the descriptive, evaluative and generative power of TGIF.

Gesture recognition corpora and tools :

Article scientifique ArODES

a scripted ground truthing method

Simon Ruffieux, Denis Lalanne, Elena Mugellini, Omar Abou Khaled

Computer Vision and Image Understanding, 2015, vol. 131, pp. 72-87

Lien vers la publication

Résumé:

This article presents a framework supporting rapid prototyping of multimodal applications, the creation and management of datasets and the quantitative evaluation of classification algorithms for the specific context of gesture recognition. A review of the available corpora for gesture recognition highlights their main features and characteristics. The central part of the article describes a novel method that facilitates the cumbersome task of corpora creation. The developed method supports automatic ground truthing of the data during the acquisition of subjects by enabling automatic labeling and temporal segmentation of gestures through scripted scenarios. The temporal errors generated by the proposed method are quantified and their impact on the performances of recognition algorithm are evaluated and discussed. The proposed solution offers an efficient approach to reduce the time required to ground truth corpora for natural gestures in the context of close human–computer interaction.

2014

PEGASO :

Chapitre de livre ArODES

a personalised and motivational ICT system to empower adolescents towards healthy lifestyle

Stefano Carrino, Maurizio Caon, Leonardo Angelini, Elena Mugellini, Omar Abou Khaled, Silvia Orte, Eloisa Vargiu, Neil Coulson, José C. E. Serrano, Sarah Tabozzi, Neil Lafortuna, Giovanna Rizzo

Dans Graña, Manuel, Howlett, Robert J., Jain, Lakhmi C., Toro, Carlos, Innovation in medicine and helathcare 2014 (pp. 350-359). 2014, Amsterdam, The Netherlands : IOS Press

Lien vers la publication

Résumé:

Unhealthy alimentary behaviours and physical inactivity habits are key risk factors for major non communicable diseases. Several researches demonstrate that juvenile obesity can lead to serious medical conditions, pathologies and have important psycho-social consequences. PEGASO is a multidisciplinary project aimed at promoting healthy lifestyles among teenagers through assistive technology. The core of this project is represented by the ICT system, which allows providing tailored interventions to the users through their smartphones in order to motivate them. The novelty of this approach consists of developing a Virtual Individual Model (VIM) for user characterization, which is based on physical, functional and behavioural parameters opportunely selected by experts. These parameters are digitised and updated thanks to the user monitoring through smartphone; data mining algorithms are applied for the detection of activity and nutrition habits and this information is used to provide personalised feedback. The user interface will be developed using gamified approaches and integrating serious games to effectively promote health literacy and facilitate behaviour change.

Affective interaction in smart environments

Article scientifique ArODES

Maurizio Caon, Leonardo Angelini, Omar Abou Khaled, Denis Lalanne, Yong Yue, Elena Mugellini

Procedia Computer Science, 2014, vol. 32, pp. 1016-1021

Lien vers la publication

Résumé:

We present a concept where the smart environments of the future will be able to provide ubiquitous affective communication. All the surfaces will become interactive and the furniture will display emotions. In particular, we present a first prototype that allows people to share their emotional states in a natural way. The input will be given through facial expressions and the output will be displayed in a context-aware multimodal way. Two novel output modalities are presented: a robotic painting that applies the concept of affective communication to the informative art and an RGB lamp that represents the emotions remaining in the user's peripheral attention. An observation study has been conducted during an interactive event and we report our preliminary findings in this paper.

Enhancement of conference organization using ontology based information correlation

Article scientifique ArODES

Hassan Noureddine, Iman Jarkass, Maria Sokhn, Omar Abou Khaled, Elena Mugellini

Computer Science and Information Technology, 2014, vol. 2, no. 2, pp. 63-71

Lien vers la publication

Résumé:

The world today witnessed an important transmission to the virtual world across the web. After years of interaction and exchanges between people, the web becomes saturated with enormous quantity of data in various fields. Therefore, the (semi-)automatic applications become necessity to find appropriate information in a brief time. In this context, we propose a new approach of ontology based information correlation from various web resources. As validator of this approach, we introduce the conference organizer system that will be useful during the setting up of a conference. Where we benefit from semantic web technologies to extract, correlate, rank and store information, and consequently propose a ranked lists of experts and social events depending on the user requests.

2012

Demonstration of a Monitoring Lamp to Visualize the Energy Consumption in Houses

Article scientifique

Gisler Christophe, Grazia Barchi, Gérôme Bovet, Mugellini Elena, Monnet Stephen

The 10th International Conference on Pervasive Computing (Pervasive 2012), 2012

Lien vers la publication

2009

MEMODULES as tangible shortcuts to multimedia information

Chapitre de livre ArODES

Aleksandar Boder, Elena Mugellini, Denis Lalanne, Bruno Dumas, Florian Evequoz, Sandro Gerardi, Anne Le Calve, Rolf Ingold, Omar Abou Khaled

In : LALANNE, Denis, KOHLAS, Jürg (eds). Human machine interaction : research results of the MMI program. Berlin : Springer, 2009. P. 103-132. (Lecture notes in computer science ; 5440). 2009

Lien vers la publication

6th Sense– Toward a generic framework for end-to-end adaptive wearable augmented reality

Article scientifique ArODES

Damien Perritaz, Christophe Salzmann, Denis Gillet, Olivier Naef, Jacques Bapst, Frédéric Barras, Elena Mugellini, Omar Abou Khaled

Lecture Notes in Computer Science, 2009, vol. 5440, pp. 280-310

Lien vers la publication

Résumé:

Augmented Reality enhances user perception by overlaying real world information with virtual computer-generated information. The aims of the 6th Sense project are to improve real-time interaction between the real environment and the virtual world and to maximize the user experience in mobile Augmented Reality. To achieve these objectives a generic framework constituted of two main layers is proposed. The End-to-End Adaptation Layer adapts in real-time the parameters of the Augmented Reality system to provide the user with the best possible experience despite the varying operating conditions such as the transmission link and user head motion. The Generic Augmented Reality Layer encompasses solutions to the problem of overlaying adequate information in the real scene and manages multimodal interaction with the virtual environment.

2024

Exploring the benefits of physical features in virtual reality on customer experience

Conférence ArODES

Robin Cherix, Maud Julmy, Marine Capallera, Julien Roland, Elena Mugellini

Proceedings of the IHCI 2024 + GET 2024 + CGVCVIP 2024, 13-15 July 2024, Budapest, Hungary

PEOPLE@HES-SO Annuaire et Répertoire des compétences

Mugellini Elena

Professeure HES ordinaire/Responsable d'institut

Compétences principales

Interdisciplinary Innovation

Human Computer Interaction

Technology for Behavior Change

User Research

Human Analytics

Machine Learning

Professeure HES ordinaire/Responsable d'institut

PEOPLE@HES-SO
Annuaire et Répertoire des compétences