Bureau: HEG 411
Téléphone: +41 26 429 67 45
Rôle: Requérant(e) principal(e)
HES-SO, SwissUniversities, Loterie Romande, Canton de Fribourg, Associations et Institutions Fribourgeoises
Description du projet :
Le projet Silver & Home du Gérontopôle Fribourg/Freiburg recherche une meilleure adéquation entre les produits et les services développés par les entreprises et les besoins des seniors cherchant une meilleure qualité de vie à la maison, dans les appartements adaptés ou en EMS.
Silver&Home collabore avec des entreprises qui offrent des gérontechnologies dans leur catalogue de produits ou services, des prestataires dans le domaine du vieillissement, enfin et surtout, des seniors, utilisateurs finaux de ces produits et services. Silver & Home s’intéresse à différents types de gérontechnologies qui peuvent être regroupées autour de 4 catégories principales: la sécurité, les liens sociaux, la prise en soins et la mobilité.
Equipe de recherche au sein de la HES-SO:
, Mugellini Elena
, Houmard Laurent
Partenaires professionnels: Emmanuel Michielan, Gérontopôle Fribourg / AFIPA; Thierry Müller, Associations Fribourgeoise Aide Soins à domicile; Jean Marc Groppo, Pro Senectute Fribourg
Durée du projet:
01.11.2018 - 31.07.2020
Montant global du projet: 200'000 CHF
Archivage des données:
Statut: En cours
Requérant(e)s: Giuseppe Andreoni, Politecnico di Milano
EU SC1-PM-15-2017 H2020
NESTORE (Novel Empowering Solutions and Technologies for Older people to Retain Everyday life activities) is a H2020 EU project funded under the SC1-PM15-2017 programme aimed at developing virtual coaches for older adults.
Equipe de recherche au sein de la HES-SO:
, Mugellini Elena
, Abou Khaled Omar
, El Kamali Mira
, Crausaz Didier
, Carrino Francesco
, Caon Maurizio
Partenaires académiques: Giuseppe Andreoni, Politecnico di Milano
Durée du projet:
01.09.2017 - 01.01.2021
Archivage des données:
Requérant(e)s: FR - EIA - Institut HumanTech, Mugellini Elena, FR - EIA - Institut HumanTech
FR - Ecole d'ingénieurs de Fribourg; Hasler; SPO
Description du projet :
Novel interaction paradigms for home and work environment are still an open-issue.
Since its invention in the 60's, the most popular mode of interaction remains the mouse-
keyboard artifact, principally because of its accuracy, ergonomic and effortless usage.
The aim of this project is to set up a joint PhD research initiative, between University of
Applied Sciences of Western Switzerland-Fribourg and University of Fribourg, to
develop a novel natural and economic gesture-based HCI for everyday use. This
document presents our vision, our motivation and finally outlines the major research
issues that will be handled during the project.
Equipe de recherche au sein de la HES-SO:
, Tscherrig Julien
, Riedo Olivier
, Chassot Laurent
, Capallera Marine
, Mugellini Elena
, Ruffieux Simon
Partenaires académiques: FR - EIA - Institut HumanTech; Mugellini Elena, FR - EIA - Institut HumanTech
Durée du projet:
01.09.2010 - 31.12.2020
Montant global du projet: 614'092 CHF
Mira El Kamali, Denis Lalanne, Omar Abou Khaled, Elena Mugellini, Leonardo Angelini
Frontiers in Computer Science,
Lien vers la publication
The use of multiple interfaces may improve the perception of a stronger relationship between a conversational virtual coach and older adults. The purpose of this paper is to show the effect of output combinations [single-interface (chatbot, tangible coach), multi-interface (assignment, redundant-complementary)] of two distinct conversational agent interfaces (chatbot and tangible coach) on the eCoach-user relationship (closeness, commitment, complementarity) and the older adults' feeling of social presence of the eCoach. Our study was conducted with two different study settings: an online web survey and a face to face experiment. Our online study with 59 seniors shows that the output modes in multi-interface redundant-complementary manner significantly improves the eCoach-user relationship and social presence of the eCoach compared to only using single-interfaces outputs. Whereas in our face to face experiment with 15 seniors, significant results were found only in terms of higher social presence of multi-interface redundant complementary manner compared to chatbot only. We also investigated the effect of each study design on our results, using both quantitative and qualitative methods.
Quentin Meteier, Mira El Kamali, Leonardo Angelini, Omar Abou Khaled
23, 18, 7974
Photovoltaic installations can be environmentally beneficial to a greater or lesser extent, depending on the conditions. If the energy produced is not used, it is redirected to the grid, otherwise a battery with a high ecological footprint is needed to store it. To alleviate this problem, an innovative recommender system is proposed for residents of smart homes equipped with battery-free solar panels to optimise the energy produced. Using artificial intelligence, the system is designed to predict the energy produced and consumed for the day ahead using three data sources: sensor logs from the home automation solution, data collected by the solar inverter, and weather data. Based on these predictions, recommendations are then generated and ranked by relevance. Data collected over 76 days were used to train two variants of the system, considering or without considering energy consumption. Recommendations selected by the system over 14 days were randomly picked to be evaluated for relevance, ranking, and diversity by 11 people. The results show that it is difficult to predict residents’ consumption based solely on sensor logs. On average, respondents reported that 74% of the recommendations were relevant, while the values contained in them (i.e., accuracy of times of day and kW energy) were accurate in 66% (variant 1) and 77% of cases (variant 2). Also, the ranking of the recommendations was considered logical in 91% and 88% of cases. Overall, residents of such solar-powered smart homes might be willing to use such a system to optimise the energy produced. However, further research should be conducted to improve the accuracy of the values contained in the recommendations.
Julie Restout, Iouri Bernache-Assollant, Charles Morizio, Arnaud Boujut, Leonardo Angelini, Achille Tchalla, Anaick Perrochon
Journal of the American Medical Directors Association,
24, 4, 564-572
Objective: The development of negative behavioral and psychosocial factors (depression, anxiety, apathy, etc) is associated with poor well-being, which can contribute to health issues in ageing, especially in the context of COVID-19. Despite its relative novelty, fully immersive virtual reality (VR) interventions through 360 immersive videos are becoming more accessible and flexible and constitute an emerging method to potentially enhance well-being. The aim of this scoping review is to assess the effectiveness of 360 interventions on well-being in older adults with or without cognitive impairment, as well as cybersickness and attitudes toward this technology.
Design: Scoping review.
Setting and Participants: Older adults with or without cognitive impairment.
Methods: The PRISMA-SR guideline was followed. Four databases were used, and we selected articles published until April 2022. We have analyzed the effect of 360° videos on the well-being of older adults with respect to the study design, the population, the contents, the duration of intervention, and the outcomes. Results: A total of 2262 articles were screened, of which 10 articles were finally included in this review. Most of them are pilot studies and used mixed methods including scales and interviews. The material and content of VR are diversified. Many behavioral and psychological outcomes were assessed, including anxiety, apathy, loneliness, depression, social engagement, quality of life, and emotions. The results were positive or mixed, according to the outcomes. We recorded few adverse events, and the interviews show contrasting results concerning the participants’ feelings (ie, degree of immersion, familiarity with technology, and VR content). Conclusions and Implications: The use of VR 360° videos seems feasible in community-dwelling older adults or residential aged care facilities, as they are safe and provide enjoyment. It constitutes an emerging and promising therapeutic tool to manage psychosocial disorders. This review provides key considerations for the design and implementation of interventions using VR 360° video in clinical practice.
Quentin Meteier, Marine Capallera, Emmanuel de Salis, Leonardo Angelini, Stefano Carrino, Marino Widmer, Omar Abou Khaled, Elena Mugellini, Andreas Sonderegger
Data in Brief,
To be published.
This dataset contains data of 346 drivers collected during six experiments conducted in a fixed-base driving simulator. Five studies simulated conditionally automated driving (L3-SAE), and the other one simulated manual driving (L0-SAE). The dataset includes physiological data (electrocardiogram (ECG), electrodermal activity (EDA), and respiration (RESP)), driving and behavioral data (reaction time, steering wheel angle, …), performance data of non-driving-related tasks, and questionnaire responses. Among them, measures from standardized questionnaires were collected, either to control the experimental manipulation of the driver's state, or to measure constructs related to human factors and driving safety (drowsiness, mental workload, affective state, situation awareness, situational trust, user experience).
In the provided dataset, some raw data have been processed, notably physiological data from which physiological indicators (or features) have been calculated. The latter can be used as input for machine learning models to predict various states (sleep deprivation, high mental workload, ...) that may be critical for driver safety. Subjective self-reported measures can also be used as ground truth to apply regression techniques. Besides that, statistical analyses can be performed using the dataset, in particular to analyze the situational awareness or the takeover quality of drivers, in different states and different driving scenarios.
Overall, this dataset contributes to better understanding and consideration of the driver's state and behavior in conditionally automated driving. In addition, this dataset stimulates and inspires research in the fields of physiological/affective computing and human factors in transportation, and allows companies from the automotive industry to better design adapted human-vehicle interfaces for safe use of automated vehicles on the roads.
Mira El Kamali, Leonardo Angelini, Maurizio Caon, Nick Dulake, Paul Chamberlain, Claire Craig, Carlo Emilio Standoli, Giuseppe Andreoni, Omar Abou Khaled, Elena Mugellini
International Journal of Human–Computer Interaction,
This article describes a tangible interface for an e-coach, co-designed in four countries to meet older adults’ needs and expectations. The aim of this device is to coach the user by giving recommendations, personalized tasks and to build empathy through vocal, visual, and physical interaction. Through our co-design process, we collected insights that helped identifying requirements for the physical design, the interaction design and the privacy and data control. In the first phase, we collected users’ needs and expectations through several workshops. Requirements were then trans- formed into three design concepts that were rated and commented by our target users. The final design was implemented and tested in three countries. We discussed the results and the open challenges for the design of physical e-coaches for older adults. To encourage further developments in this field, we released the research outputs of this design process in an open-source repository.
Marine Capallera, Quentin Meteier, Marino Widmer, Leonardo Angelini, Stefano Carrino, Andreas Sonderegger, Omar Abou Khaled, Elena Mugellini
2023, vol. 11, pp. 5746-5771
Advanced driver assistances are becoming increasingly common in commercial cars, not only to assist but also to free drivers from manual driving whenever possible. Soon, drivers should be allowed to engage in non-driving-related tasks. The fact that responsibility for driving is shifting from humans to machines must be considered in the development of these assistances in order to guarantee safety and trust. In this article, we introduce AdVitam (for Advanced Driver-Vehicle Interaction to Make future driving safer), an autonomous system aiming at maintaining driver’s situation awareness and optimizing takeover quality during conditionally automated driving. The information conveyed to drivers is dynamically adapted to achieve these goals, depending on the driving environment and the driver’s physiological state. This system consists of three connected modules. The first module (Driver State) predicts the driver’s state with machine learning and physiological signals as inputs. The second module (Supervision) uses different interfaces (a haptic seat, a personal device, and ambient lights) to maintain the drivers’ situation awareness during the autonomous driving phases. The third module (Intervention) is a machine learning model that chooses the most appropriate combination among haptic, auditory, and visual modalities to request the driver to take over control and thus optimize takeover quality. To evaluate the system and each module independently, a preliminary user study with 35 drivers was conducted in a fixed-base driving simulator. All participants drove in two different environments (rural and urban). In addition, the activation of the Supervision and Intervention modules were manipulated as two between-subject factors. Results show that conveying information on the driving environment status through multimodal interfaces increases drivers’ situation awareness (i.e., better identification of potential problems in the environment) and trust in the automated vehicle. However, the system does not show positive outcomes on takeover quality. Besides, the Driver State module provided consistent predictions with the experimental manipulation. The system proposed in this paper could lead to better acceptance and safety when conditionally automated vehicles will be released by increasing drivers’ trust during phases of automated driving.
Marine Capallera, Leonardo Angelini, Quentin Meteier, Omar Abou Khaled, Elena Mugellini
IEEE Transactions on Intelligent Vehicles,
2023, Vol. 8, n° 3, pp. 2551 - 2567
Autonomous driving will change the role of the driver. From being the main actor in driving, the driver will now have a supervisory role during the autonomous driving phases. However, if the driver has to take over control of the vehicle, he must be aware of the situation around him. This is why it is important to develop interfaces to keep him in the loop. This article proposes a systematic review of Human Vehicle Interaction (HVI) providing situation awareness in the context of autonomous driving. 37 articles presenting such interactions are analyzed in terms of design of the interaction (modalities, location, conveyed information) but also in term of evaluation and experimental conditions. We present an overview of previous studies in order to highlight the work already done or in progress. Current studies present mainly monomodal interfaces although the evaluation of multimodal interactions present promising results in this field.
Quentin Meteier, Marine Capallera, Emmanuel de Salis, Marino Widmer, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Andreas Sonderegger
2022, vol. 10, article no. e15229
Drivers are often held responsible for road crashes. Previous research has shown that stressors such as carrying passengers in the vehicle can be a source of accidents for young drivers. To mitigate this problem, this study investigated whether the presence of a passenger behind the wheel can be predicted using machine learning, based on physiological signals. It also addresses the question whether relaxation before driving can positively influence the driver's state and help controlling the potential negative consequences of stressors. Sixty young participants completed a 10-min driving simulator session, either alone or with a passenger. Before their driving session, participants spent 10 min relaxing or listening to an audiobook. Physiological signals were recorded throughout the experiment. Results show that drivers experience a higher increase in skin conductance when driving with a passenger, which can be predicted with 90%-accuracy by a k-nearest neighbors classifier. This might be a possible explanation for increased risk taking in this age group. Besides, the practice of relaxation can be predicted with 80% accuracy using a neural network. According to the statistical analysis, the potential beneficial effect of relaxation did not carry out on the driver's physiological state while driving, although machine learning techniques revealed that participants who exercised relaxation before driving could be recognized with 70% accuracy. Analysis of physiological characteristics after classification revealed several relevant physiological indicators associated with the presence of a passenger and relaxation.
Leonardo Angelini, Mira El Kamali, Elena Mugellini, Omar Abou Khaled, Christina Röcke, Simone Porcelli, Alfonso Mastropietro, Giovanni Rizzo, Noemi Boqué, Josep Maria Del Bas, Filippo Palumbo, Michele Girolami, Antonino Crivello, Canan Ziylan, Paula Subías-Beltrán, Silvia Orte, Carlo Emilio Standoli, Laura Fernandez Maldonado, Maurizio Caon, Martin Sykora, Suzanne Elayan, Sabrina Guye, Giuseppe Andreoni
2022, vol. 10, no 2, pp. 50-77
This article describes the coaching strategies of the NESTORE e-coach, a virtual coach for promoting healthier lifestyles in older age. The novelty of the NESTORE project is the definition of a multi-domain personalized pathway where the e-coach accompanies the user throughout different structured and non-structured coaching activities and recommendations. The article also presents the design process of the coaching strategies, carried out including older adults from four European countries and experts from the different health domains, and the results of the tests carried out with 60 older adults in Italy, Spain and The Netherlands.
Maurizio Caon, Federica Prinelli, Leonardo Angelini, Stefano Carrino, Elena Mugellini, Silvia Orte, José C. E. Serrano, Sarah Atkinson, Anne Martin, Fulvio Adorni
Frontiers in Nutrition,
2022, vol. 9, article 727480
Background: Obesity amongst children and adolescents is becoming a major health problem globally and mobile food records can play a crucial role in promoting healthy dietary habits.
Objective: To describe the methodology for the implementation of the e-Diary mobile food record, to assess its capability in promoting healthy eating habits, to evaluate the factors associated with its usage and engagement.
Methods: This is a descriptive study that compared the characteristics of participants engaged in the e-Diary, which was part of the PEGASO project in which an app to provide proactive health promotion was given to 365 students at 4 European sites enrolled during October to December 2016: England (UK), Scotland (UK), Lombardy (Italy), and Catalonia (Spain). The e-Diary tracked the users' dietary habits in terms of food groups, dietary indexes, and 6 dietary target behaviors relating to consumption of: fruit; vegetable; breakfast; sugar-sweetened beverages; fast-food; and snacks. The e-Diary provided also personalized suggestions for the next meal and gamification.
Results: The e-Diary was used for 6 months by 357 adolescents (53.8% females). The study showed that females used the e-Diary much more than males (aOR 3.8, 95% CI 1.6–8.8). Participants aged 14 years were more engaged in the e-Diary than older age groups (aOR 5.1, 95% CI 1.4–18.8) as were those with a very good/excellent self-perceived health status compared to their peers with fair/poor health perception (aOR 4.2, 95% CI 1.3–13.3). Compared to the intervention sites, those living in Catalonia (aOR 13.2 95% CI 2.5–68.8) were more engaged. In terms of behavior change, a significant positive correlation between fruit (p < 0.0001) and vegetables (p = 0.0087) intake was observed in association with increased engagement in the e-Diary. Similarly, adolescents who used the app for more than 2 weeks had significantly higher odds of not skipping breakfast over the study period (aOR 2.5, 95% CI 1.0–6.3).
Conclusions: The users highly engaged with the e-Diary were associated with improved dietary behaviors: increased consumption of fruit and vegetables and reduced skipping of breakfast. Although the overall usage of the e-Diary was high during the first weeks, it declined thereafter. Future applications should foster user engagement, particularly targeting adolescents at high risk.
Quentin Meteier, Michiel Kindt, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini
2022, vol. 22, no. 3, article no. 880
In this work, we propose a low-cost solution capable of collecting the driver’s respiratory signal in a robust and non-intrusive way by contact with the chest and abdomen. It consists of a microcontroller and two piezoelectric sensors with their respective 3D printed plastic housings attached to the seat belt. An iterative process was conducted to find the optimal shape of the sensor housing. The location of the sensors can be easily adapted by sliding them along the seat belt. A few participants took part in three test sessions in a driving simulator. They had to perform various activities: resting, deep breathing, manual driving, and a non-driving-related task during automated driving. The subjects’ breathing rates were calculated from raw data collected with a reference chest belt, each sensor alone, and the fusion of the two. Results indicate that respiratory rate could be assessed from a single sensor located on the chest with an average absolute error of 0.92 min−1 across all periods, dropping to 0.13 min−1 during deep breathing. Sensor fusion did not improve system performance. A 4-pole filter with a cutoff frequency of 1 Hz emerged as the best option to minimize the error during the different periods. The results suggest that such a system could be used to assess the driver’s breathing rate while performing various activities in a vehicle.
Quentin Meteier, Emmanuel de Salis, Marine Capallera, Marino Widmer, Leonardo Angelini, Omar Abou Khaled, Andreas Sonderegger, Elena Mugellini
Frontiers in Computer Science,
2022, vol. 3, article no. 775282
In future conditionally automated driving, drivers may be asked to take over control of the car while it is driving autonomously. Performing a non-driving-related task could degrade their takeover performance, which could be detected by continuous assessment of drivers' mental load. In this regard, three physiological signals from 80 subjects were collected during 1 h of conditionally automated driving in a simulator. Participants were asked to perform a non-driving cognitive task (N-back) for 90 s, 15 times during driving. The modality and difficulty of the task were experimentally manipulated. The experiment yielded a dataset of drivers' physiological indicators during the task sequences, which was used to predict drivers' workload. This was done by classifying task difficulty (three classes) and regressing participants' reported level of subjective workload after each task (on a 0–20 scale). Classification of task modality was also studied. For each task, the effect of sensor fusion and task performance were studied. The implemented pipeline consisted of a repeated cross validation approach with grid search applied to three machine learning algorithms. The results showed that three different levels of mental load could be classified with a f1-score of 0.713 using the skin conductance and respiration signals as inputs of a random forest classifier. The best regression model predicted the subjective level of workload with a mean absolute error of 3.195 using the three signals. The accuracy of the model increased with participants' task performance. However, classification of task modality (visual or auditory) was not successful. Some physiological indicators such as estimates of respiratory sinus arrhythmia, respiratory amplitude, and temporal indices of heart rate variability were found to be relevant measures of mental workload. Their use should be preferred for ongoing assessment of driver workload in automated driving.
Mara Pereira Guerreiro, Leonardo Angelini, Helga Rafael Henriques, Mira El Kamali, Cristina Baixinho, João Balsa, Isa Félix Brito, Omar Abou Khaled, Maria Beatriz Carmo, Ana Paula Claudio, Maurizio Caon, Karl Daher, Bruno Alexandre, Mafalda Padinha, Elena Mugellini
JMIR Research Protocols,
2021, vol. 10, no. 9, article no. e26680
Conversational agents, which we defined as computer programs that are designed to simulate two-way human conversation by using language and are potentially supplemented with nonlanguage modalities, offer promising avenues for health interventions for different populations across the life course. There is a lack of open-access and user-friendly resources for identifying research trends and gaps and pinpointing expertise across international centers. Objective: Our aim is to provide an overview of all relevant evidence on conversational agents for health and well-being across the life course. Specifically, our objectives are to identify, categorize, and synthesize—through visual formats and a searchable database—primary studies and reviews in this research field. Methods: An evidence map was selected as the type of literature review to be conducted, as it optimally corresponded to our aim. We systematically searched 8 databases (MEDLINE; CINAHL; Web of Science; Scopus; the Cochrane, ACM, IEEE, and Joanna Briggs Institute databases; and Google Scholar). We will perform backward citation searching on all included studies. The first stage of a double-stage screening procedure, which was based on abstracts and titles only, was conducted by using predetermined eligibility criteria for primary studies and reviews. An operational screening procedure was developed for streamlined and consistent screening across the team. Double data extraction will be performed with previously piloted data collection forms. We will appraise systematic reviews by using A Measurement Tool to Assess Systematic Reviews (AMSTAR) 2. Primary studies and reviews will be assessed separately in the analysis. Data will be synthesized through descriptive statistics, bivariate statistics, and subgroup analysis (if appropriate) and through high-level maps such as scatter and bubble charts. The development of the searchable database will be informed by the research questions and data extraction forms. Results: As of April 2021, the literature search in the eight databases was concluded, yielding a total of 16,351 records. The first stage of screening, which was based on abstracts and titles only, resulted in the selection of 1282 records of primary studies and 151 records of reviews. These will be subjected to second-stage screening. A glossary with operational definitions for supporting the study selection and data extraction stages was drafted. The anticipated completion date is October 2021. Conclusions: Our wider definition of a conversational agent and the broad scope of our evidence map will explicate trends and gaps in this field of research. Additionally, our evidence map and searchable database of studies will help researchers to avoid fragmented research efforts and wasteful redundancies. Finally, as part of the Harnessing the Power of Conversational e-Coaches for Health and Well-being Through Swiss-Portuguese Collaboration project, our work will also inform the development of an international taxonomy on conversational agents for health and well-being, thereby contributing to terminology standardization and categorization.
Christina Röcke, Leonardo Angelini, Sabrina Guye, Mira El Kamali, Maurizio Caon, Omar Abou Khaled, Elena Mugellini
Dans Andreoni, Giuseppe, Mambretti, Cinzia, Digital health technology for better aging: a multidisciplinary approach
(Pp. 161-177). 2021,
Cham : Springer
Lien vers la publication
This chapter discusses the theoretical behaviour change framework and its integration and implementation of behaviour change techniques that form the conceptual psychological basis for innovative but efficient coaching approach. We review the current state of e-coaching solutions for older adults that can be found in the literature and identify gaps. Finally, we review and discuss the technological implementation of behaviour change techniques applied in NESTORE to support healthy older adults to adopt a healthy lifestyle in all NESTORE well-being domains as reference case study.
Mira El Kamali, Leonardo Angelini, Maurizio Caon, Guiseppe Andreoni, Nick Dulake, Paul Chamberlain, Omar Abou Khlaed, Elena Mugellini
Dans Andreoni, Giuseppe, Mambretti, Cinzia, Digital health technology for better aging: a multidisciplinary approach
(Pp. 195-204). 2021,
Cham : Springer
E-coaches are promising tools for health interventions, being scalable and relatively low costs compared to human coaches. Nevertheless, they cannot compare to humans when it comes to trust and social companionship. Hence, many researches were held around the importance of tangible user interfaces in building trust and companionship during human-machine interaction. Co-design, in particular, is the act of creating with stakeholders within the design development process to ensure the results meet their needs and are usable. This chapter describes a design process method and the rationale for building an embodiment for a companion and a trustable e-coach. We also shed lights on the co-design process held in the NESTORE project, which was specifically conducted with older adults to build a vocal assistant that supports user’s behavioral change, as well as to increase the social bonding with the NESTORE e-coach.
Bhalaji Nagarajan, Rupali Khatun, Marc Bolaños, Eduardo Aguilar, Leonardo Angelini, Mira El Kamali, Elena Mugellini, Omar Abou Khaled, Noemi Boqué, Lucia Taro, Petia Radeva
Dans Andreoni, Giuseppe, Mambretti, Cinzia, Digital Health Technology for Better Aging : a multidisciplinary approach
(pp. 77-102). 2021,
Cham : Springer
Nutritional monitoring is an important aspect of providing a healthy lifestyle and smarter food consumption. Food recognition is a key task in nutrition monitoring applications especially of interest for older people suffering from malnutrition and unhealthy eating. In order to ease the problem of food intake monitoring, recent health apps involve Artificial Intelligence algorithms for food image analysis. Food recognition is a highly challenging task as the dishes could be composed of mixed types, it could have a lower inter-class variance and also the number of dishes to be recognized is quite higher. In this chapter, we present an in-depth analysis and insights into the development of a nutritional monitoring system that would act as a Key Performance Indicator for the older people food intake monitoring. This system is aimed at providing a convenient food logging environment that would assist older people and their relatives as well as geriatrists in monitoring and maintaining the healthy daily dietary nutritional needs.
Martin Sykora, Suzanne Elayan, Leonardo Angelini, Christina Röcke, Mira El Kamali, Elena Mugellini, Sabrina Guye
Dans Andreoni, Giuseppe, Mambretti, Cinzia, Digital Health Technology for Better Aging : a multidisciplinary approach
(pp. 179-193). 2021,
Cham : Springer
In recent years, due to the rise in availability of unstructured big data and technological affordances, there has been a proverbial “explosion” of research within the field of sentiment analysis and affective computing, facilitating the study of in-situ communicated emotion and its accurate detection at an unprecedented scale. Increasingly the emotional dimension and its key important role is being recognized in e-coaching, with attempts at its integration within such systems. This chapter reviews the predominant extant models for representing affect, as well as emotion related states relevant to older persons’ experiences, including the state-of-art for their automated detection and integration within e-coaching systems. Given the approaches available, we describe how the NESTORE system addresses emotion recognition, self-tracking and subsequent feedback intended to help older adults make sense of their affective experiences across their daily life, as well as the role it can play in holistic e-coaching systems. Finally, from the NESTORE experience, we draw recommendations about how future e-coaching system could be implemented in the future to include the emotional dimension.
Martin Dobricki, David Weibel, Leonardo Angelini, Elena Mugellini, Fred W. Mast
2021, vol. 16, no. 5, article no. e0251562
While one is walking, the stimulation by one’s body forms a structure with the stimulation by the environment. This locomotor array of stimulation corresponds to the human-environment relation that one’s body forms with the environment it is moving through. Thus, the perceptual experience of walking may arise from such a locomotor array of stimulation. Humans can also experience walking while they are sitting. In this case, there is no stimulation by one’s walking body. Hence, one can experience walking although a basic component of a locomotor array of stimulation is missing. This may be facilitated by perception organizing the sensory input about one’s body and environment into a perceptual structure that corresponds to a locomotor array of stimulation. We examined whether locomotor illusions are generated by this perceptual formation of a locomotor structure. We exposed sixteen seated individuals to environmental stimuli that elicited either the perceptual formation of a locomotor structure or that of a control structure. The study participants experienced distinct locomotor illusions when they were presented with environmental stimuli that elicited the perceptual formation of a locomotor structure. They did not experience distinct locomotor illusions when the stimuli instead elicited the perceptual formation of the control structure. These findings suggest that locomotor illusions are generated by the perceptual organization of sensory input about one’s body and environment into a locomotor structure. This perceptual body-environment organization elucidates why seated human individuals experience the sensation of walking without any proprioceptive or kinaesthetic stimulation.
Quentin Meteier, Marine Capallera, Simon Ruffieux, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Andreas Sonderegger
Frontiers in Psychology,
2021, vol. 12, article no. 596038
The use of automation in cars is increasing. In future vehicles, drivers will no longer be in charge of the main driving task and may be allowed to perform a secondary task. However, they might be requested to regain control of the car if a hazardous situation occurs (i.e., conditionally automated driving). Performing a secondary task might increase drivers' mental workload and consequently decrease the takeover performance if the workload level exceeds a certain threshold. Knowledge about the driver's mental state might hence be useful for increasing safety in conditionally automated vehicles. Measuring drivers' workload continuously is essential to support the driver and hence limit the number of accidents in takeover situations. This goal can be achieved using machine learning techniques to evaluate and classify the drivers' workload in real-time. To evaluate the usefulness of physiological data as an indicator for workload in conditionally automated driving, three physiological signals from 90 subjects were collected during 25 min of automated driving in a fixed-base simulator. Half of the participants performed a verbal cognitive task to induce mental workload while the other half only had to monitor the environment of the car. Three classifiers, sensor fusion and levels of data segmentation were compared. Results show that the best model was able to successfully classify the condition of the driver with an accuracy of 95%. In some cases, the model benefited from sensors' fusion. Increasing the segmentation level (e.g., size of the time window to compute physiological indicators) increased the performance of the model for windows smaller than 4 min, but decreased for windows larger than 4 min. In conclusion, the study showed that a high level of drivers' mental workload can be accurately detected while driving in conditional automation based on 4-min recordings of respiration and skin conductance.
Quentin Meteier, Marine Capallera, Andreas Sonderegger, Leonardo Angelini, Stefano Carrino, Elena Mugellini, Omar Abou Khaled
Proceedings of AutomotiveUI '20: 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 21-22 September 2020, Online Conference,
In conditionally automated driving, drivers do not have to constantly monitor their vehicle but they must be able to take over control when necessary. In this paper, we assess the impact of instructions about limitations of automation and the presentation of context-related information through a mobile application on the situation awareness and takeover performance of drivers. We conducted an experiment with 80 participants in a fixed-base driving simulator. Participants drove for an hour in conditional automation while performing secondary tasks on a tablet. Besides, they had to react to five different takeover requests. In addition to the assessment of behavioral data (e.g. quality of takeover), participants rated their situation awareness after each takeover situation. Instructions and context-related information on limitations combined showed encouraging results to raise awareness and improve takeover performance.
Mira El Kamali, Leonardo Angelini, Maurizio Caon, Francesco Carrino, Christina Röcke, Sabrina Guye, Giovanna Rizzo, Alfonso Mastropietro, Martin Sykora, Suzanne Elayan, Isabelle Kniestedt, Canan Ziylan, Emanuele Lettieri, Omar Abou Khaled, Elena Mugellini
2020, vol. 8, pp. 101884-101902
Virtual Coaches, also known as e-coaches, are a disruptive technology in healthcare. Indeed, among other usages, they might provide cost-effective solutions for increasing human wellbeing in different domains, such as physical, nutritional, cognitive, social, and emotional. This paper presents a systematic review of virtual coaches specifically aimed at improving or maintaining older adults' health in the aforementioned domains. Such digital systems assume various forms, from classic apps, to more advanced conversational agents or robots. Fifty-six articles describing a virtual coach for older adults and aimed at improving their wellbeing were identified and further analyzed. In particular, we presented how previous studies defined their virtual coaches, which behavioral change models and techniques they adopted and the overall system architecture, in terms of monitoring solutions, processing methods and modalities for intervention delivery. Our results show that few thorough evaluations of e-coaching systems have been conducted, especially regarding multi-domain coaching approaches. Through our analysis, we identified the wellbeing domains that should be addressed in future studies as well as the most promising behavior change models and techniques and coaching interfaces. Previous work illustrates that older adults often appreciate conversational agents and robots. However, the lack of a multidomain intervention approach in the current literature motivates us to seek to define future solutions.
Alessandra Rinaldi, Leonardo Angelini, Abou Khaled, Elena Mugellini, Maurizio Caon
Dans Di Bucchianico, Giuseppe, Advances in design for inclusion : proceedings of the AHFE 2019 International Conference on design for inclusion and the AHFE 2019 International Conference on human factors for apparel and textile engineering, July 24-28, 2019, Washington D.C., USA
(Pp. 186-195). 2019,
Cham, Switzerland : Springer
The consistent phenomenon of immigration from countries with different cultures, that characterized the last decade, has changed the use of public spaces in European cities. The need to develop innovative solutions aimed at improving intercultural inclusion and mutual understanding of all citizens emerges, so that urban spaces become inclusive, safe and sustainable. This research project is focused on two main questions: (1) can design and technology enable the creation of a different way of living the public space to catalyze interpersonal interaction and to promote a shared sense of citizenship? (2) is it possible to support intercultural communication and to facilitate social inclusion through the design of innovative and interactive public spaces equipped with smart urban furniture? This paper presents three Design Orienting Scenarios as results of the research conducted by several experts, professionals and people with different cultural backgrounds during a focus groups and two different workshops where they made converge competences from design, business, engineering and social sciences to find innovative solutions for the social inclusion of migrants in urban areas.
Capallera Marine, Meteier Quentin, De Salis Emmanuel, Carrino Stefano, Angelini Leonardo, Abou Khaled Omar, Mugellini Elena
AutomotiveUI '19: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 2019
In the context of highly automated driving, the driver has to be aware of driving risks and to take over control of the car in hazardous situations. The goal of this paper is to categorize and analyze the factors that lead to such critical scenarios. To this purpose, we analyzed limitations of Advanced Driver-Assistance Systems (ADAS) extracted from owner manuals of 12 partially automated cars available on the market. A taxonomy with 6 macro-categories and 26 micro-categories is proposed to classify and better understand the limitations of these vehicles. We also investigated if these limitations are conveyed to the driver through Human-Machine Interaction (HMI) in the car. Some suggestions are made to better communicate these limitations to the driver in order to raise his/her situation awareness.
Meteier Quentin, Capallera Marine, Angelini Leonardo, Mugellini Elena, Abou Khaled Omar, Carrino Stefano, De Salis Emmanuel, Stéphane Galland, Susanne Boll
AutomotiveUI '19: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings, 2019
With the increasing use of automation, users tend to delegate more tasks to the machines. Such complex systems are usually developed with "black box" Artificial Intelligence (AI), which makes these systems difficult to understand for the user. This assumption is particularly true in the field of automated driving since the level of automation is constantly increasing via the use of state-of-the-art AI solutions. We believe it is important to investigate the field of Explainable AI (XAI) in the context of automated driving since interpretability and transparency are key factors for increasing trust and security. In this workshop, we aim at gathering researchers and industry practitioners from different fields to brainstorm about XAI with a special focus on human-vehicle interaction. Questions like "what kind of explanation do we need", "which is the best trade-off between performance and explainability" and "how granular should the explanations be" will be addressed in this workshop.
Maurizio Caon, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini
Rivista italiana di ergonomia,
2018, no. 16, pp. 9-23
Maurizio Caon, Leonardo Angelini, Katharina Ledermann, Chantal Martin-Sölch, Omar Abou Khaled, Elena Mugellini
Dans Bagnara, Sebastiano, Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018) : volume VII: ergonomics in design, design for all, activity theories for work analysis and design, affective design
(Pp. 1372-1381). 2018,
Cham : Springer
Pain assessment is an essential first step in planning for and providing pain management. Access to effective pain management programs is often limited, due to scarcity of services. Other barriers to treatment include physical symptoms that limit mobility, distance from a clinic, transportation requirements and cost constraints. In response to these barriers to service delivery, the demand for online health resources is growing. However, current solutions present a number of shortcomings, in particular when referring to usability and accessibility. This paper presents the vision of the My Pain Coach system and the first prototype generated as the result of the first iteration of the development process. This prototype is based on a smartphone app and presents two different interfaces: a force-sensitive touchscreen, and a tangible interface based on a textile mat. A preliminary usability evaluation has been conducted and the results show that these interfaces are perceived as excellent from a usability point of view. Nevertheless, further development and testing are still required for the tangible interface.
El Kamali Mira, Angelini Leonardo, Caon Maurizio, Giuseppe Andreoni, Abou Khaled Omar, Mugellini Elena
Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, 2018
The ability to engage the user in a conversation and the credibility of the system are two fundamental characteristics of virtual coaches. In this paper, we present the architecture of a conversational e-coach for promoting healthy lifestyles in older age, developed in the context of the NESTORE H2020 EU project. The proposed system allows multiple access points to a conversational agent via different interaction modalities: a tangible companion that embodies the virtual coach will leverage voice and other non-verbal cues in the domestic environment, while a mobile app will integrate a text-based chat for a ubiquitous intervention. In both cases, the conversational agent will deliver personalized interventions based on behavior change models and will promote trust by means of emotionally rich conversations.
Leonardo Angelini, Elena Mugellini, Omar Abou Khaled, Nadine Couture
2018, vol. 5(1), no. 7
In the Internet of Things era, an increasing number of everyday objects are able to offer innovative services to the user. However, most of these devices provide only smartphone or web user interfaces. As a result, the interaction is disconnected from the physical world, decreasing the user experience and increasing the risk of user alienation from the physical world. We argue that tangible interaction can counteract this trend and this article discusses the potential benefits and the still open challenges of tangible interaction applied to the Internet of Things. After an analysis of open challenges for Human-Computer Interaction in IoT, we summarize current trends in tangible interaction and extrapolate eight tangible interaction properties that could be exploited for designing novel interactions with IoT objects. Through a systematic review of tangible interaction applied to IoT, we show what has been already explored in the systems that pioneered the field and the future explorations that still have to be conducted. In order to guide future work in this field, we propose a design card set for supporting the generation of tangible interfaces for IoT objects. The card set has been evaluated during a workshop with 21 people and the results are discussed.
Leonardo Angelini, Francesco Carrino, Maurizio Caon, Elena Mugellini, Omar Abou Khaled, Nadine Couture
Angewandte GERONTOLOGIE Appliquée,
2017, vol. 2
La perte progressive de mobilité chez les seniors peut avoir de lourdes conséquences, pas seulement physiques, mais aussi psychiques et cognitives due à un isolement social croissant. La fenêtre interactive est un outil permettant la communication à distance. Elle est issue d’un projet de recherche qui vise à réduire l’isolement social des seniors en utilisant de nouvelles technologies de manière innovante.
Leonardo Angelini, Francesco Carrino, Maurizio Caon, Frédéric Lemaréchal, Nadine Couture, Omar Abou Khaled, Elena Mugellini
2016, vol. 29, no. 4, pp. 215-224
Social relationships are crucial for older adults’ well-being. Past research demonstrated that video-communication technology could reduce social isolation. However, most commercial telecommunication systems have interfaces that are too complex to be operated by older adults without assistance. In this paper, we present a system that exploits the tangible affordances of a physical window to provide a natural interface for an always-on video-communication system. In addition, we present a case study focused on the system accessibility and user acceptance with 8 older adults living in a retirement home. We also describe the preliminary findings based on an observation study and on a semistructured interview. Finally, we discuss the possible improvements and impact this system might have on older adults’ life.
Leonardo Angelini, Denis Lalanne, Elise van den Hoven, Omar Abou Khaler, Elena Mugellini
2015, vol. 3, pp. 173-207
Technology is spreading in our everyday world, and digital interaction beyond the screen, with real objects, allows taking advantage of our natural manipulative and communicative skills. Tangible gesture interaction takes advantage of these skills by bridging two popular domains in Human-Computer Interaction, tangible interaction and gestural interaction. In this paper, we present the Tangible Gesture Interaction Framework (TGIF) for classifying and guiding works in this field. We propose a classification of gestures according to three relationships with objects: move, hold and touch. Following this classification, we analyzed previous work in the literature to obtain guidelines and common practices for designing and building new tangible gesture interactive systems. We describe four interactive systems as application examples of the TGIF guidelines and we discuss the descriptive, evaluative and generative power of TGIF.
Stefano Carrino, Maurizio Caon, Leonardo Angelini, Elena Mugellini, Omar Abou Khaled, Silvia Orte, Eloisa Vargiu, Neil Coulson, José C. E. Serrano, Sarah Tabozzi, Neil Lafortuna, Giovanna Rizzo
Dans Graña, Manuel, Howlett, Robert J., Jain, Lakhmi C., Toro, Carlos, Innovation in medicine and helathcare 2014
(pp. 350-359). 2014,
Amsterdam, The Netherlands : IOS Press
Unhealthy alimentary behaviours and physical inactivity habits are key risk factors for major non communicable diseases. Several researches demonstrate that juvenile obesity can lead to serious medical conditions, pathologies and have important psycho-social consequences. PEGASO is a multidisciplinary project aimed at promoting healthy lifestyles among teenagers through assistive technology. The core of this project is represented by the ICT system, which allows providing tailored interventions to the users through their smartphones in order to motivate them. The novelty of this approach consists of developing a Virtual Individual Model (VIM) for user characterization, which is based on physical, functional and behavioural parameters opportunely selected by experts. These parameters are digitised and updated thanks to the user monitoring through smartphone; data mining algorithms are applied for the detection of activity and nutrition habits and this information is used to provide personalised feedback. The user interface will be developed using gamified approaches and integrating serious games to effectively promote health literacy and facilitate behaviour change.
Maurizio Caon, Leonardo Angelini, Omar Abou Khaled, Denis Lalanne, Yong Yue, Elena Mugellini
Procedia Computer Science,
2014, vol. 32, pp. 1016-1021
We present a concept where the smart environments of the future will be able to provide ubiquitous affective communication. All the surfaces will become interactive and the furniture will display emotions. In particular, we present a first prototype that allows people to share their emotional states in a natural way. The input will be given through facial expressions and the output will be displayed in a context-aware multimodal way. Two novel output modalities are presented: a robotic painting that applies the concept of affective communication to the informative art and an RGB lamp that represents the emotions remaining in the user's peripheral attention. An observation study has been conducted during an interactive event and we report our preliminary findings in this paper.
Jean-Michel Vasse, Alain Verdon, Catherine Senn-Dubey, Leonardo Angelini, Elena Mugellini, Quentin Meteier
Resuscitation Official Journal of the European Resuscitation Council Abstracts of Resuscitation 2023, 2-4 November 2023, Barcelona, Spain
Lien vers la conférence
Sophie Jaquenod, Daniel Ducraux, Elena Mugellini, Leonardo Angelini
Proceedings of Human Aspects of IT for the Aged Population 9th International Conference, ITAP 2023, Held as Part of the 25th HCI International Conference, HCII 2023, 23-28 July 2023, Copenhagen, Denmark Proceedings, Part I
With the aging population and the increasing number of major neurocognitive disorders in nursing homes, non-pharmacological interventions will play an important role in the future of care. The train therapy is an emerging intervention for coping with behavioral and psychological symptoms of dementia that relies on a railway travel simulator. Seniors, accompanied by healthcare professionals, can experience a train journey in a physical space disguised as a train wagon and displaying prerecorded videos from a train in a television disguised as a train window. This paper presents a novel and accessible railway travel simulator integrating an interactive ticket counter and we discuss the challenges and perspectives for the adoption of the train therapy.
Leonardo Angelini, Corentin Bompard, Omar Abou Khaled, Elena Mugellini
This paper presents the architecture and a preliminary prototype of a system for detecting stress during immersive virtual reality (VR) simulations. The VR4AGE system is designed for professionals that wish to use virtual reality headsets to allow older adults with dementia to experience 360 videos with an increased safety. The system aims at using eye tracking and unobtrusive psychophysiological signal monitoring to detect cognitive activation and distress. Informing as soon as possible the healthcare professionals about the patient’s status would allow them to continue or stop the therapy, depending on the patient’s reactions. This could help assessing whether nursing home residents with dementia that are unable to speak and express their emotions react positively or negatively to the VR simulation.
Quentin Meteier, Elena Mugellini, Leonardo Angelini, Alain Adrien Verdon, Catherine Senn-Dubey, Jean-Michel Vasse
Proceedings of the 2023 Symposium on Eye Tracking Research and Applications (ETRA'23), 30 may-2 June 2023, Tubingen, Germany
Practical simulation is increasingly used to develop reasoning skills during learning. The analysis of the scene and the correct execution of actions require an awareness of the situation and the activities performed by the student. Eye-tracking feedback (i.e., a video recording of the practical simulation with an overlay of the gaze point) can allow students and teachers to enhance the skills of analysis and execution of the practical activities performed. In this article, we present the implementation of an innovative pedagogical process for nursing students in Switzerland. It involves the use of eye-tracking glasses to improve learning through the enhancement of metacognition after a simulation. The results of a first test session done with 15 undergraduate students are reported.
Marine Capallera, Leonardo Angelini, Alexandre Favre, François Magnin, Omar Abou Khaled, Elena Mugellini
Proceeding sof the 34th international francophone conference on human-computer interaction
La réalité virtuelle offre un environnement de formation immersif dans l'enseignement des soins infirmiers, en particulier pour expérimenter des situations ne pouvant être perçues dans la réalité. Par un processus de conception centré sur l'humain avec des éducateurs.trices et des étudiant.es d'une école d'infirmières, nous avons développé une application de RV de formation à l'asepsie lors d'un prélèvement sanguin. La simulation permet de visualiser les risques de contamination lors de la procédure. Nous avons réalisé trois tests et amélioré continuellement l'application afin d'augmenter la facilité d'utilisation et le sentiment d'immersion. Nous avons comparé deux modalités d'interaction, deux visualisations des mains et différents degrés d'immersion (téléportation ou marche dans une représentation réelle d'une salle pour toucher les objets physiques). Ces expériences ont permis de tirer une conclusion sur le niveau d'immersion atteint par les nouvelles technologies de RV et sur leur implication possible dans la formation pratique des étudiant.es.
Quentin Meteier, Marine Capallera, Emmanuel de Salis, Leonardo Angelini, Stefano Carrino, Omar Abou Khaled, Elena Mugellini, Andreas Sonderegger
En conduite automatisée, plusieurs facteurs peuvent affecter la conscience de la situation et la reprise de contrôle. Pour mieux comprendre cela, 88 participants ont passé 20 minutes dans un simulateur de conduite conditionnellement automatisée. Ils devaient réagir à quatre obstacles dont le danger et le mouvement variaient. La moitié des participants devaient s'engager dans une tâche cognitive verbale non liée à la conduite. La conscience de la situation, la performance et les réponses physiologiques ont été mesurées. Les résultats suggèrent que le mouvement de l'obstacle influence la perception du danger, la conscience de la situation et le temps de réaction, ce dernier étant également influencé par le danger de l'obstacle. La tâche cognitive verbale a également un effet sur le temps de réaction. Ainsi, l’état cognitif du conducteur et la situation de conduite (mouvement/danger des entités autour du véhicule) devraient être considérés lors de la transmission d'informations via les interfaces homme-véhicule.
Leonardo Angelini, Nadine Couture, Kamali Mira, Quentin Meteier, Elena Mugellini
Proceedings of the european tangible interaction studio (ETS) 2022
The goal of this workshop is to discuss the potential benefits and the open challenges of giving a physical form to Artificial Intelligence (AI), towards the definition of tangible, or graspable AI. In the workshop we will focus on the use-case of a convolutional Neural Network for image analysis and we will carry out a hands-on paper prototyping activity to imagine tangible interactive AI-powered systems where critical parameters of the Neural Network are physicalized. Such systems have the potential to lower the barrier for AI education and for making AI systems more trustable and explainable.
Emmanuel de Salis, Quentin Meteier, Marine Capallera, Colin Pelletier, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Stefano Carrino
AHFE International, Human Interaction Emerging Technologies (IHIET-AI 2022): Artificial Intelligence Future Applications
In this study, we investigate how to make takeover in conditionally automated vehicles safer optimizing the takeover request modalities. Using Machine Learning algorithms, we created a smart agent recommending the best modalities combination to use (haptic-visual, auditory-visual and haptic-auditory-visual) to convey a takeover request. This agent considers the driver state before recommending modalities, as well as the weather condition. The proposed agent is able to predict takeover quality better than the baseline by 56.5% (baseline MSE : 0.0600, our agent MSE : 0.0261) and use different recommendations based on the situation. Modality impact on takeover quality was shown to have a mean score of 4.95% (standard deviation: 2.7%). Evaluation of the agent gain on takeover quality compared to standard takeover request design is currently ongoing.
Marine Capallera, Jean Pierre Bresciani, Quentin Meteier, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini
IHM ’22 : Proceedings of the 33rd Conference on l'Interaction Humain-Machine, 5-8 April 2022, Namur, Belgium
Les actuelles assistances à la conduite modifient progressivement le rôle du conducteur. Pour certains niveaux d’automatisation, son rôle actif devient alors celui de superviseur. Ce changement peut entraîner une réduction de la compréhension de la situation mais également une charge cognitive plus élevée lors de la reprise de contrôle manuelle. C’est pourquoi il est important de concevoir des interactions permettant de soutenir le conducteur dans cette tâche. Cet article propose d’utiliser le siège du conducteur et une tablette pour transmettre des informations au conducteur relatives à son environnement pendant les phases de conduite autonome et ainsi l’accompagner dans sa tâche de supervision. Une première expérience a permis de valider l’utilisation de vibrations dans le siège pour transmettre des informations. Une seconde expérience compare ce concept avec l’envoi d’icônes pour transmettre ces mêmes informations. Les résultats montrent que l’utilisation de ces concepts a un impact sur la conscience de la situation.
Emmanuel de Salis, Quentin Meteier, Marine Capallera, Quentin Pelletier, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Stefano Carrino
Proceedings of the 5th International Conference on Intelligent Human Systems Integration (IHSI 2022) Integrating People and Intelligent Systems, Venice, Italy, 22-24 February 2022
Emmanuel de Salis, Quentin Meteier, Colin Pelletier, Marine Capallera, Leonardo Angelini, Andreas Sonderegger, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Stefano Carrino
International Conference on Human Interaction and Emerging Technologies ; Proceedings of IHIET 2021: Human Interaction, Emerging Technologies and Future Systems V, 28-30 October 2021, Reims, France
Conditionally automated cars share the driving task with the driver. When the control switches from one to another, accidents can occur, especially when the car emits a takeover request (TOR) to warn the driver that they must take the control back immediately. The driver’s physiological state prior to the TOR may impact takeover performance and as such was extensively studied experimentally. However, little was done about using Machine Learning (ML) to cluster natural states of the driver. In this study, four unsupervised ML algorithms were trained and optimized using a dataset collected in a driving simulator. Their performances for generating clusters of physiological states prior to takeover were compared. Some algorithms provide interesting insights regarding the number of clusters, but most of the results were not statistically significant. As such, we advise researchers to focus on supervised ML using ground truth labels after experimental manipulation of drivers’ states.
Leonardo Angelini, Alexandre Favre, François Magnin, Mariateresa De Vito Woods, Omar Abou Khaled, Elena Mugellini
10e Conférence sur les environnements Informatiques pour l’apprentissage humain (EIAH), 7-10 juin 2021, Fribourg, Suisse
Dans une collaboration interdisciplinaire entre une Haute École d’Ingénierie et une Haute École de Santé, nous avons développé une plateforme de réalité virtuelle pour l’entraînement à l’asepsie dans un contexte de soins. La première application développée réplique soigneusement un scénario de prise de sang. La plateforme permet de visualiser les erreurs d’asepsie et les risques de contamination pendant l’exécution des gestes. Un système de points permet de ludifier l’entraînement et de comparer les scores des étudiants. Nous avons mené des tests préliminaires avec 9 étudiants avec un casque de réalité virtuelle Oculus Quest 2. Lors du test, nous avons comparé différentes modalités d’interaction (manettes ou suivi des mains) et différents degrés d’immersion (téléportation dans un environnement complètement virtuel, ou mappage d’une vraie salle de simulation, avec possibilité de toucher des éléments physiques). Ces expériences nous ont permis de tirer un bilan sur le niveau d’immersion atteint par les nouvelles technologies de réalité virtuelle et sur leur implication possible dans la formation pratique des étudiants.
Leonardo Angelini, Maurizio Caon, Emmanuel Michielan, Omar Abou Khaled, Elena Mugellini
Proceedings of the 21st Congress of the International Ergonomics Association (IEA 2021)
As the European population is getting older, there is an increasing need in maintaining older adults living independently at home. Vocal assistants may offer various services that can be beneficial for senior citizens. In the context of the Silver&Home living lab, we tested the Google Home Smart speaker connected to smart lighting installation with 7 people to understand the strengths, weaknesses and possible usage for improving the quality of life of older adults. The test and the questions asked to participants were framed according to the Unified Theory on Acceptance and Use of Technology (UTAUT2). Participants generally appreciated the interaction with the smart speaker, although they also identified some barriers, such as the “OK Google” wakeword or the assistant speaking too fast for some answers. Finally, they considered it particularly adapted to people living alone.
Mira El Kamali, Leonardo Angelini, Maurizio Caon, Francesco Carrino, Carlo Emilio Standoli, Paolo Perego, Giuseppe Andreoni, Filippo Palumbo, Alfonso Mastropietro, Omar Abou Khaled, Elena Mugellini
The current research proposes a technological system "NESTORE" designed for and with older adults in four different countries in order to improve and sustain their wellbeing. The system personalized activities and architecture, co-designed interfaces, and its multilingual aspect aim to establish an ‘inclusion’ criterion based on the user’s sociocultural profile and health condition.
Fayez Chris Lteif, Karl Daher, Leonardo Angelini, Elena Mugellini, Omar Abou Khaled, Hayssam El Hajj
Advances in Intelligent Systems and Computing ; Proceedings of International Conference on Human Interaction and Emerging Technologies IHIET-AI 2021: Human Interaction, Emerging Technologies and Future Applications IV, 28-30 April 2021, Strasbourg, France
The integration of technology, namely virtual reality, with arts is implemented in this paper. An observational study is performed by replicating an actual museum by a Swiss-Lebanese artist and having user experience it in a virtual immersive environment. Data is then collected to monitor the user experience and its effects on his emotional and psychological state. This information is used to reflect on future developments and enhancements to elevate the user experience and potentially upgrade it with further advanced technologies.
Emmanuel de Salis, Quentin Meteier, Marine Capallera, Leonardo Angelini, Andreas Sonderegger, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Stefano Carrino
Advances in Intelligent Systems and Computing ; Proceedings of International Conference on Intelligent Human Systems Integration IHSI 2021: Intelligent Human Systems Integration 2021, 22-24 February 2021, Palermo, Italy
Takeover requests in conditionally automated vehicles are a critical point in time that can lead to accidents, and as such should be transmitted with care. Currently, several studies have shown the impact of using different modalities for different psychophysiological states, but no model exists to predict the takeover quality depending on the psychophysiological state of the driver and takeover request modalities. In this paper, we propose a machine learning model able to predict the maximum steering wheel angle and the reaction time of the driver, two takeover quality metrics. Our model is able to achieve a gain of 42.26% on the reaction time and 8.92% on the maximum steering wheel angle compared to our baseline. This was achieved using up to 150 s of psychophysiological data prior to the takeover. Impacts of using such a model to choose takeover modalities instead of using standard takeover requests should be investigated.
Chiara Lucifora, Leonardo Angelini, Quentin Meteier, Carmelo M. Vicario, Omar Abou Khaled, Elena Mugellini, Giorgio M. Grasso
Cyber Therapy is a research project based on the relationship between Computer Science and Psychology. We are working on post-traumatic stress disorder (PTSD) due to severe traffic accidents using a virtual reality driving simulator and ECG, EDA and breathing sensors. With the help of virtual reality (VR) our goal is to build one software that can process the user’s biofeedback signals - heart rate, body temperature, state of tension, etc. - in real time, to make the phobic stimulus autonomous. To this purpose, we developed a platform capable of adapting the phobic stimulus based on the user’s biofeedback signals. We believe that this human-computer integrated system could be useful to patients as it would allow them to face fear autonomously, and to the psychotherapist, as it would allow a real time - physiologically based - knowledge of fear symptoms severity able to promote a timely and more appropriate program of intervention.
Quentin Meteier, Marine Capallera, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Jean-Pierre Bresciani, Andreas Sonderegger, Stefano Carrino
Proceedings of GSGS'20 : 5th Gamification & Serious Game Symposium, September-November 2020, Switzerland
Simulations play a crucial role to investigate hazardous situations that are impossible to test in real-life conditions without endangering the user’s safety. This paper presents a simulator of conditionally automated cars aiming at enhancing the driver safety and driving comfort. In addition, thanks to the simulator’s highly repeatability, integrated sensors and controlled conditions we collected valuable scientific data, which is otherwise very difficult to gather.
Marine Capallera, Omar Abou Khaled, Leonardo Angelini, Elena Mugellini
Proceedings of ETIS 2020 : the Fourth European Tangible Interaction Studio, 16-20 November 2020, Siena, Italy
Developments in the field of semi-autonomous driving will increasingly free the driver from the main driving task. Conditional autonomous driving allows the driver not to constantly monitor her/his environment, but s/he must still be able to regain control of the vehicle at any time if the situation requires it. That’s why it is important to maintain driver’s Situation Awareness while is performing a secondary task. This paper presents a model of an adaptive, full-body and multimodal Human Vehicle Interaction (HVI) for supervision.
Mira El Kamali, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini
Proceedings of ETIS 2020 : Fourth European Tangible Interaction Studio 2020, 16-20 November 2020, Siena, Italy
Older adults are increasing in Europe . Technology may play a role in older adults' health. Giving technology the ability to be physically manipulated can result in creating trust . According to Spreicer , co-designing tangible user interface with elderly people can increase the chance of technology acceptance. NESTORE  is a virtual coach that comes in different forms of modalities/interfaces: a chatbot, a mobile application and a tangible coach. This article focuses on presenting one of the interfaces of NESTORE: the “tangible coach”. The tangible coach is an embodied conversational agent that seeks to empower older adults in becoming the co-producers of their wellbeing lifestyle through its tangible and its vocal interaction.
Leonardo Angelini, Mira El Kamali, Elena Mugellini, Omar Abou Khaled, Yordan Dimitrov, Vera Veleva, Zlatka Gospodinova, Nadejda Miteva, Richar Wheeler, Panagiotis Bamidis, Evdokimos Konstantinidis, Despoina Petsani, Andoni Beristain Iraola, Gérard Chollet, Inés Torres, Zoraida Callejas, David Griol, Kawtar Benghazi, Manuel Noguera, Dimitrios I. Fotiadis
Proceedings of ICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction, 25-29 October 2020, The Netherlands
T e-Coaches are promising intelligent systems that aims at supporting human everyday life, dispatching advices through different interfaces, such as apps, conversational interfaces and augmented reality interfaces. This workshop aims at exploring how e-coaches might benefit from spatially and time-multiplexed interfaces and from different communication modalities (e.g., text, visual, audio, etc.) according to the context of the interaction.
Mira El Kamali, Leonardo Angelini, Denis Lalanne, Omar Abou Khaled, Elena Mugellini
Proceedings of ICMI '20 Companion : International Conference on Multimodal Interaction, 25-29 October 2020, Virtual event, Netherlands
Publié dans les actes de la conférence WACAI 2020, Workshop sur les affects, compagnons artificiels et interactions, 2-4 juin 2021, Île d'Oléron
Dans le cadre du projet Européen NESTORE, nous envisageons de créer un coach virtuel pour le bien-être des seniors. Ce dernier est un agent conversationnel qui peut avoir une conversation utilisant le langage naturel avec l'utilisateur afin de coacher, de se lier d'amitié et d'accompagner les seniors tout au long de leur parcours. Cet agent conversationnel se décline sous différentes formes d'interfaces telles qu’une application de messagerie textuelle ou un assistant vocal intégré. Cet agent conversationnel multimodal cherche à construire une relation empathique avec les utilisateurs basée sur son omniprésence, son attitude de coaching, sa fidélité et enfin sa capacité de comprendre l'émotion de l'utilisateur. Dans cet article, nous présentons la conception de ce coach virtuel, son but, ses rôles, l’architecture du système, ses capacités et ses multiinterfaces.
Marine Capallera, Quentin Meteier, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Marino Widmer, Stefano Carrino
Proceedings of HCII 2020 : 22nd International Conference on Human-Computer Interaction, 19-24 July 2020, Copenhagen, Denmark ; Lecture Notes in Computer Science
In this paper, we propose a model for an AI-Companion for conditionally automated cars, able to maintain awareness of the driver regarding the environment but also to able design take-over requests (TOR) on the fly, with the goal of better support the driver in case of a disengagement.
Our AI-Companion would interact with the driver in two ways: first, it could provide feedback to the driver in order to raise the driver Situation Awareness (SA), prevent them to get out of the supervision loop and so, improve takeover during critical situations by decreasing their cognitive workload. Second, in the case of TOR with a smart choice of modalities for convey the request to the driver. In particular, the AI-Companion can interact with the driver using many modalities, such as visual messages (warning lights, images, text, etc.), auditory signals (sound, speech, etc.) and haptic technologies (vibrations in different parts of the seat: back, headrest, etc.).
The ultimate goal of the proposed approach is to design smart HMIs in semi-autonomous vehicles that are able to understand 1) the user state and fitness to drive, 2) the current external situation (vehicle status and behavior) in order to minimize the automation surprise and maximizing safety and trust, and 3) leverage AI to provide adaptive TOR and useful feedback to the driver.
Mira El Kamali, Leonardo Angelini, Maurizio Caon, Denis Lalanne, Omar Abou Khaled, Elena Mugellini
Human-computer Interaction : human values and quality of life : thematic area, HCI 2020, held as part of the 22nd International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, proceedings, part III
The population of people age 65 or over is increasing especially inEurope . Granting to this target population a longer and healthier life isparamount for the European Community. In the context of the H2020 EUfunded project“NESTORE”, an embodied and ubiquitous e-coach is beingdeveloped seeking to change the lifestyle of seniors in different domains ofwellbeing. NESTORE e-coach is known as a personalized embodied andubiquitous e-coach that plays three essential roles in elderly’s wellbeing: acoach, a friend and a companion. As a coach, NESTORE will give trainings andadvice following a wellbeing path that is proposed by experts in wellbeing. As afriend, this e-coach knows and understands the user. As a companion, this e-coach has the ability to detect the user’s emotion and aims at building empathywith the user based by providing support throughout their daily training.The NESTORE e-coach is based on three different intervention medium: amobile application, a chatbot and an embodied vocal assistant. These interfaceshave different forms, different capabilities and different visions. Users cancommunicate with the NESTORE e-coach through different interfaces exclu-sively, sequentially, concurrently and synergistically. The interaction can beinitiated from the user side to different interfaces and/or from the e-coach side.In this paper, we present the NESTORE’s full vision for building the threeessential roles of this e-coach which are: a coach, a companion and a friend forseniors. Furthermore, we explain the NESTORE system design, architecture,capabilities and how the different interfaces of this E-coach contribute to make amulti-modal system. Finally, we conclude our work with the state of this H2020project.
Mira El Kamali, Leonardo Angelini, Maurizio Caon, Omar Abou Khaled, Elena Mugellini, Nick Dulack, Paul Chamberlin, Claire Craig, Giuseppe Andreoni
Proceedings of the 2nd Conference on Conversational User Interfaces (CUI'20), 9-10 July, Bilbao, Spain
Conversational agents are becoming widespread thanks to the development of natural language processing and deep learning techniques. NESTORE is a virtual coach designed to support seniors during their journey to wellbeing. This virtual coach interacts with the user through two different type of conversational agents: a text-based chatbot integrated in a mobile application and a tangible coach, which is a physical object based on vocal interaction and tangible interaction. This paper presents the design of each conversational agent interface developed to support seniors in enhancing their wellbeing. A cloud conversational agent manages the user interaction with the NESTORE virtual coach, which can occur through interfaces with different design, forms and capabilities, i.e., the chatbot and the tangible coach: two different intelligent interfaces with a shared mind.
Filippo Palumbo, Paolo Baronti, Antonino Crivello, Francesco Furfari, Michele Girolami, Fabio Mavilia, Marta Civiello, Enrico Denna, Leonardo Angelini, Mira El Kamali, Omar Abou Khaled, Elena Mugellini, Giancarlo Pace
Proceedings of the 5th Italian Workshop on Artificial Intelligence for Ambient Assisted Living 2019, co-located with 18th International Conference of the Italian Association for Artificial Intelligence, AI*AAL@AI*IA 2019, 22 November 2019, Rende, Italy
Monitoring physiological and behavioural data related to the five domains of well-being (i.e., physical, mental, cognitive, social, and nutritional) is relevant for assessing the profile of people using assistive technologies, in order to provide early detection and adaptive support to his changing individual needs related to ageing. In this paper, we present a system called NESTORE that aims at addressing such a challenge. In particular, we focus on the enabling technology that composes the core set of devices of the so-called environmental monitoring system, namely the NESTORE Bluetooth Low Energy beacons. The presented system performs a range of services including data collection and analysis of short- and long-term trends in social and behavioural parameters. Furthermore, using the same set of devices the system provides insights on the status of the user’s vital space in terms of thermal comfort. We provide an overview of the NESTORE environmental monitoring system and details and evaluation of the software modules built upont the chosen technology: social interaction detection, indoor behavioural index inference, and indoor thermal comfort detection.
Paula Subias-Beltran, Silvia Orte, Eloisa Vargiu, Filippo Palumbo, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Maurizio Caon
Proceedings of the 32nd IEEE CBMS International Symposium on Computer-Based Medical Systems
This paper presents the decision support system that has been defined and developed under the umbrella of the NESTORE project. The main goal of the proposed system is to help users in selecting coaching plans by proposing personalised recommendations based on their behaviours and preferences. Recognising such behaviours and their evolution over time is therefore a crucial element for tailoring the interaction of the system with the user. A three-layer system composed of pathways, coaching activity plans, and coaching events, constitutes the so-called coaching timeline on which the analysis is grounded. Various techniques are used to model and personalise the recommendations and feedback. Firstly, the indicators are extracted from disparate data sources, then these are modelled through a profiling system and, finally, recommendations on the pathways and coaching plans are performed through a scoring and a tagging system.
Leonardo Angelini, Maurizio Caon, Jacky Casas, Frederica Cena, Amon Rapp, Omar Abou Khaled, Elena Mugellini
Proceedings of ACM International Symposium on Wearble Computers (ISWC'18), Ubicomp 2018, 8-12 October 2018, Singapore
Human-computer interaction is progressively shifting towards natural language communication, determining the rise of conversational agents. In the context of ubiquitous computing, the opportunities for interacting with new services and systems in a conversational manner are increasing and, nowadays, it is common to talk to home assistants to interact with a smart environment or to write to chatbots to access an online service. This workshop aims at bringing together researchers from academia and industry in order to establish a multidisciplinary community interested in discovering and exploring the challenges and opportunities coming from the ubiquity of conversational agents.
Mira El Kamali, Leonardo Angelini, Maurizio Caon, Giuseppe Andreoni, Omar Abou Khaled, Elena Mugellini
Maurizio Caon, Stefano Carrion, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Filip Velickovski, Giuseppe Andreoni
Proceedings of the 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC'18)
In contemporary society, non-communicable diseases linked to unhealthy lifestyles, such as obesity, are on the rise with a major impact on global deaths. Prevention is the new frontier, promising to increase life expectancy and quality, while reducing costs related to healthcare. The PEGASO project developed a mobile ecosystem where the digital Companion aims at empowering teenagers in the adoption of healthy lifestyles. The pilot study conducted in three European countries (Spain, UK and Italy) shows a good acceptance of the system and that teenagers are keen to use mobile technology to improve their lifestyle, although wearable devices did not engage the young users
Maurizio Caon, Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Assunta Matassa
Proceedings of Designing Interactive Systems (DIS), 9-13 June 2018, Hong Kong, China
Current digital system interfaces are mainly based on vision and hearing. HCI practitioners and researchers need to introduce unconventional senses in interaction design in order to avoid sensory deprivation in digital life and enhance information accessibility. In this paper, we present a novel technique, called Multisensory Storming, which aims at supporting the multisensory interaction design process. Multisensory Storming is a group method enabling the generation of new ideas and design proposals in physical contexts through exercises that allow exploring all the human senses.
Leonardo Angelini, Zuzanna Lechelt, Eva Hornecker, Paul Marshall, Can Liu, Margot Brereton, Alessandro Soro, Nadine Couture, Omar Abou Khaled, Elena Mugellini
Proceedings of CHI 2018, 21-26 April 2018, Montréal, Canada
The rise of the Internet of Things (IoT) brings abundant new opportunities to create more effective and pleasing tangible user interfaces that capitalize on intuitive interaction in the physical world, whilst utilizing capabilities of sensed data and Internet connectivity. However, with these new opportunities come new challenges; little is still known how to best design tangible IoT interfaces that simultaneously provide engaging user experiences and foster a sense of understanding about the often-complex functionality of IoT systems. How should we map previous taxonomies and design principles for tangible interaction into the new landscape of IoT systems? This workshop will bring together a community of researchers from the fields of IoT and tangible interaction, in order to explore and discuss how parallels between tangible interaction and the properties of IoT systems can best be capitalised on as HCI research moves increasingly toward the Internet of Tangible Things (IoTT). Through ideation and discussion, the workshop will function as a springboard for the community to begin creating new taxonomies and design considerations for the emerging IoTT
Leonardo Angelini, Elena Mugellini, Omar Abou Khaled, Nadine Couture, Elise van der Hoven, Saskia Bakker
Proceedings of 12th International Conference on Tangible, Embedded and Embodied Interactions (TEI'18), 18-21 March 2018, Stockholm, Sweden
There is an increasing interest in the HCI research community to design richer user interactions with the Internet of Things (IoT). This studio will allow exploring the design of tangible interaction with the IoT, what we call Internet of Tangibles. In particular, we aim at investigating the full interaction-attention continuum, with the purpose of designing IoT tangible interfaces that can switch between peripheral interactions that do not disrupt everyday routines, and focused interactions that support user’s reflections. This investigation will be conducted through hands-on activities where participants will prototype tangible IoT objects, starting by a paper prototyping phase, supported by design cards, and followed by an Arduino prototype phase. The purpose of the studio is also establishing a community of researchers and practitioners, from both academy and industry, interested in the field of tangible interaction with the Internet of Things.
Leonardo Angelini, Elena Mugellini, Nadine Couture, Omar Abou Khaled
Proceedings of the 12th International Conference on Tangible, Embedded, and Embodied Interaction (TEI'18), 18-21 March 2018, Stockholm, Sweden
Current interactions for the Internet of Things are often constrained behind a screen. With the Internet of Tangible Things (IoTT) we aim at promoting the design of richer interactions, embodied in physical IoT objects. To this purpose, we propose a card set for the design of tangible interaction with IoT objects, which contains 8 cards for tangible interaction properties and 8 for IoT properties, in order to explore how tangible properties can be exploited for enhancing the interaction with IoT objects. We tested the card set in a dedicated workshop, observing that participants were able to explore most of the tangible and IoT properties. To complement the IoT card set, a hardware prototyping toolkit with examples for each of the 8 tangible properties is currently under development.
Leonardo Angelini, Jürgen Baumgartner, Francesco Carrino, Stefano Carrino, Maurizio Caon, Omar Abou Khaled, Jürgen Sauer, Denis Lalanne, Elena Mugellini, Andreas Sonderegger
Actes de la 28e conférence francophone sur l'Interaction Homme-Machine on - IHM '16, 25-28 octobre 2016, Fribourg, Suisse
This paper compares an emergent interaction modality for the In-Vehicle Infotainment System (IVIS), i.e., gesturing on the steering wheel, with two more popular modalities in modern cars: touch in the central dashboard and speech. We conducted a between-subjects experiment with 20 participants for each modality to assess the interaction performance with the IVIS and the impact on the driving performance. Moreover, we compared the three modalities in terms of usability, subjective workload and emotional response. The results showed no statically significant differences between the three interaction modalities regarding the various indicators for the driving task performance, while significant differences were found in measures of IVIS interaction performance: users performed less interactions to complete the secondary tasks with the speech modality, while, in average, a lower task completion time was registered with the touch modality. The three interfaces were comparable in terms of perceived usability, mental workload and emotional response.
Maurizio Caon, Leonardo Angelini, Elena Mugellini, Assunta Matassa, Nadia Bianchi-Berthouze, Aneesha Singh, Ana Tajadura-Jiménez
Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, UbiComp '16, 12-16 September 2016, Heidelberg, Germany
The workshop on "Full-Body and Multisensory Experience" aims at discussing the rich possibilities that the body offers to experience the external world and the prospects that arise for interaction designers when these often-neglected abilities are taken into account. In particular, the workshop will focus on the rediscovery of the human senses, either alone or in a multimodal combination, and of the perceptual-motor abilities of our body. The one-day workshop is divided in three steps: first phase is for the generation of ideas on multisensory interfaces, in the second phase, participants will have the possibilities to explore and rediscover their sensorimotor abilities through several exercises and games; in the third and last phase, there will be a further creative session in order to evaluate how the full body and multisensory activities have fostered people's creative processes. The aim of the whole experience is twofold: first, inspiring participants in designing novel concepts for multisensory interfaces; second, providing a preliminary study on the effect of these exercises in fostering creativity and supporting the design process of multisensory interfaces.
Leonardo Angelini, Maurizio Caon, Stefania Caparrota, Omar Abou Khaled, Elena Mugellini
Plants are live beings with often underestimated sensory and communicative abilities. In this paper, we sensitize to these natural abilities and we show how they can be extended through technology. To these purpose, we propose a framework for multisensory augmented plants and we present the design of three different augmented plants that are able to communicate with humans through different modalities. The three systems aim at bridging the communication gap between plants and humans, with a particular focus on cognitive or sensory impaired people.
Maurizio Caon, Leonardo Angelini, Omar Abou Khaled, Dzmitry Aliakseyeu, Jon Mason, Elena Mugellini
Interacting with digital lighting systems opens new opportunities and challenges, especially when those systems are connected to the Internet of Things (IoT). This workshop aims at investigating richer interactions with interconnected lights. We strive for interactions that happen in our everyday environments and that go beyond not only swiping and tapping on a screen, but also the mere tangible switch. This workshop will be an opportunity to discuss and practice novel interactions with light, exploring human bodily and cognitive skills to grasp better the ephemerality of light and of the IoT.
Assunta Matassa, Leonardo Angelini, Maurizio Caon, Marianna Obrist, Elena Mugellini
Proceedings of the TEI '16 Tenth International Conference on Tangible, Embedded, and Embodied Interaction, 14-17 February 2016, Eindhoven, The Netherlands
This workshop aims at discussing the rich possibilities that the body offers to experience the external world and the prospects that arise for interaction designers when these often-neglected abilities are taken into account. In particular, the workshop will focus on the rediscovery of the five senses, either alone or in a multimodal combination, and of the perceptual-motor abilities of our body. The one-day workshop is divided in two steps: first, participants will have the possibility to briefly present and discuss with the other attendees their research. Workshop candidates are requested to send a position paper, including a short biography and detailing their research interests and background. Second, participants will have the possibilities to explore and rediscover their sensorimotor abilities through several exercises and games abilities using a critical design approach. Interdisciplinary groups will be challenged to design and develop new interaction experience concepts using our natural 'tools' as prototyping tools.
Leonardo Angelini, Omar Abou Khaled, Elena Mugellini
Actes de la 27ème Conférence francophone sur l’Interaction Homme-Machine, 27-30 Octobre 2015, Toulouse, France
Cet article présente EmotiPlant, un système conçu pour faciliter l’entretien d’une plante d’appartement aux seniors. Nous proposons un concept de plante capable d’exprimer des émotions et d’afficher son état en relation aux conditions physiques de son environnement. Grâce à des interactions de proximité avec la plante et à des expressions anthropomorphisées de son état, la plante devient ainsi un compagnon pour le senior. Cet article présente aussi un premier prototype du système et discute les défis à surmonter pour obtenir un produit utilisable par des seniors dans leur vie quotidienne.
Leonardo Angelini, Maurizio Caon, Nadine Couture, Omar Abou Khaled, Elena Mugellini
Proceedings of UBICOMP/ISWC '15 ADJUNCT, 7-11 September 2015, Osaka, Japan
This paper presents the Multisensory Interactive Window, a system that allows experiencing distant places and communicating with distant people through a tangible installation that exploits the window metaphor and multisensory feedback. Thanks to the feeling of being just in front of a natural landscape or a room inhabited by a beloved person, we strive to enhance the older adult's well-being at home. We illustrate two interaction concepts and a feasibility study for the implementation of the first prototype.
Assunta Matassa, Luca Console, Leonardo Angelini, Maurizio Caon, Omar Abou Khaled
The ubiquitous computing era is bringing to the human the possibility to interact always and everywhere with digital information. However, the interaction means used to access this information exploit only few of the human sensorimotor abilities. Most of these interactions happen through traditional desktop or mobile interfaces, which often involve just vision and hearing senses and require the movement of only one finger. The aim of this workshop is rediscovering the role of human body and senses, focusing on abilities that are often forgotten by the HCI designers, in order to provide new body experiences through the design of novel interactions in smart environments. The focus of this workshop will go beyond the mere design of multimodal interfaces and will exploit theories of embodied cognition to design new full-body experiences to explore ambient space and, more in general, the environment.
Maurizio Caon, Leonardo Angelini, Stefano Carrino, Omar Abou Khaled, Elena Mugellini
Proceedings of the 17th International Conference on Human-Computer Interaction, HCI 2015: Human-Computer Interaction: Design and Evaluation, 2-7 August 2015, Los Angeles, CA, USA
In this paper, we describe a smartphone application that aims at motivating users to use facial expressions. This has a twofold goal: to reintroduce the use of facial expressions as nonverbal means in the computer-mediated communication of emotions and to provide the opportunity for self-reflection about the personal emotional states while fostering smiles in order to improve mental wellbeing. This paper provides a description of the developed prototype and reports the results of a first observation study conducted during an interactive event.
Leonardo Angelini, Omar Abou Khaled, Maurizio Caon, Elena Mugellini, Denis Lalanne
Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '15), 15-19 January 2015, Stanford, CA, USA
This paper presents the concept of a lamp that allows displaying and collecting user’s emotional states. In particular, it displays the emotional information changing colors and facial expressions; in fact, the lamp is characterized by anthropomorphic form and behavior in order to make the interaction more natural and spontaneous. The user can interact with the lamp through tangible gestures typically used in social interactions by humans. Two different scenarios involving the use of the lamp as a companion and for computer-mediated communication are presented.
Leonardo Angelini, Denis Lalanne, Elise van den Hoven, Ali Mazalek, Omar Abou Khaled, Elena Mugellini
More and more objects of our everyday environment are becoming smart and connected, offering us new interaction possibilities. Tangible interaction and gestural interaction are promising communication means with these objects in this post-WIMP interaction era. Although based on different principles, they both exploit our body awareness and our skills to provide a richer and more intuitive interaction. Occasionally, when user gestures involve physical artifacts, tangible interaction and gestural interaction can blend into a new paradigm, i.e., tangible gesture interaction . This workshop fosters the comparison among these different interaction paradigms and offers a unique opportunity to discuss their analogies and differences, as well as the definitions, boundaries, strengths, application domains and perspectives of tangible gesture interaction. Participants from different backgrounds are invited.
Leonardo Angelini, Omar Abou Khaled, Elena Mugellini, Denis Lalanne
Actes de la conférence IHM'14, 28-31 octobre 2014, Villeneuve d'Ascq, France
Dans cet article nous proposons une classification des systèmes tangibles selon les qualités tangibles présentes dans ces systèmes. Nous montrons un exemple d’analyse pour une lampe qui permet d’afficher et récolter des états émotionnels de l’utilisateur au travers d’interactions multimodales, basées sur des gestes tangibles du côté utilisateur, et des couleurs et des expressions faciales du côté lampe. En particulier, la lampe bénéfice d’une forme et d’un comportement anthropomorphiques pour rendre l’interaction plus naturelle et spontanée.
Cet article présente la conception d’une lampe qui permet d’afficher et récolter des états émotionnels de l’utilisateur au travers d’interactions multimodales, basées sur des gestes tangibles du côté utilisateur, et des couleurs et des expressions faciales du côté lampe. En particulier, la lampe bénéfice d’une forme et d’un comportement anthropomorphiques pour rendre l’interaction plus naturelle et spontanée. Deux scénarios d’utilisation sont présentés, ainsi que l’implémentation de l’un d’eux.
Maurizio Caon, Michele Tagliabue, Leonardo Angelini, Paolo Perego, Elena Mugellini, Giuseppe Andreoni
Proceedings of the AutomotiveUI '14: Adjunct Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 17-19 September 2014, Sattle, WA, USA
Wearable technologies are spreading into human's everyday life. Smart bracelets, watches and glasses promise to provide us with ready-at-hand access to several information and interaction possibilities. What happens when these technologies enter the car? Are they a mere danger or can also offer information and interaction opportunities that can improve the vehicle inhabitants' user experience and safety? This workshop aims to discuss all the opportunities and dangers that such technologies not only are already bringing in nowadays vehicles, but could also bring in future semi-autonomous and autonomous vehicles.
Leonardo Angelini, Frnacesco Carrino, Stefano Carrino, Maurizio Caon, Omar Abou Khaled, Jürgen Baumgartner, Andreas Sonderegger, Denis Lalanne, Elena Mugellini
Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI '14, 17-19 September 2014, Seattle, WA, USA
"Eyes on the road, hands on the wheel" is a crucial principle to be taken into account designing interactions for current in-vehicle interfaces. Gesture interaction is a promising modality that can be implemented following this principle in order to reduce driver distraction and increase safety. We present the results of a user elicitation for gestures performed on the surface of the steering wheel. We asked to 40 participants to elicit 6 gestures, for a total of 240 gestures. Based on the results of this experience, we derived a taxonomy of gestures performed on the steering wheel. The analysis of the results offers useful suggestions for the design of in-vehicle gestural interfaces based on this approach.
Proceedings of the 2014 ACM International Symposium on Wearable Computers: Adjunct Program, ISWC '14 Adjunct, 13-17 September 2014, Seattle, WA, USA
Physical contact has an important role in human well-being. In this paper, we present Hugginess, a concept of interactive system that encourages people to hug by augmenting this gesture with digital information exchange. As a proof of concept, we developed two t-shirts that reciprocally send information to the hugged person through the conductive fabric.
Matthias Schwaller, Jan Kühni, Leonardo Angelini, Denis Lalanne
Proceedings of the 16th International Conference, HCI International 2014 (Human-Computer Interaction. Applications and Services), 22-27 June 2014, Heraklion, Crete, Greece
This paper presents a research work on gesture recognition and feedback to reduce the learning time of new gestures and to augment user performance in a game application. A Wiimote controlled space shooter game, GeStar Wars, has been developed. The player controls a spaceship through the buttons in the controller, while forearm gestures can be used to perform special actions. Gesture strokes are mapped in a 3x3 grid and are differentiated according to the path of the covered grid cells. In-game visual feedback displays to the user the current gesture path and which cells were covered after the gesture is performed. The novelty of this research resides in the correlated gesture recognition methodology and feedback which helps the user to learn and correct the gestures. The evaluation, conducted with 12 users, showed that the users performed significantly better if feedback was provided.
Leonardo Angelini, Maurizio Caon, Stefano Carrino, Luc Bergeron, Nathalie Nyffeler, Mélanie Jean-Mairet, Elena Mugellini
Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication
In this paper, we present the design process of a smart bracelet that aims at enhancing the life of elderly people. The bracelet acts as a personal assistant during the user's everyday life, monitoring the health status and alerting him or her about abnormal conditions, reminding medications and facilitating the everyday life in many outdoor and indoor activities.
Leonardo Angelini, Maurizio Caon, Francesco Carrino, Stefano Carrino, Denis Lalanne, Omar Abou Khaled, Elena Mugellini
Proceedings of the 15th International Conference, HCI International 2013, 21-26 July 2013, Las Vegas, NV, USA, Pt. II
This paper presents WheelSense, a system for non-distracting and natural interaction with the In-Vehicle Information and communication System (IVIS). WheelSense embeds pressure sensors in the steering wheel in order to detect tangible gestures that the driver can perform on its surface. In this application, the driver can interact by means of four gestures that have been designed to allow the execution of secondary tasks without leaving the hands from the steering wheel. Thus, the proposed interface aims at minimizing the distraction of the driver from the primary task. Eight users tested the proposed system in an evaluation composed of three phases: gesture recognition test, gesture recognition test while driving in a simulated environment and usability questionnaire. The results show that the accuracy rate is 87% and 82% while driving. The system usability scale scored 84 points out of 100.