Suche

Bologna Guido

Professeur HES assistant

Hauptkompetenzen

Machine Learning

Hauptvertrag

Professeur HES assistant

Telefon-Nummer: +41 22 558 57 61

Büro: A410

Haute école du paysage, d'ingénierie et d'architecture de Genève
Rue de la Prairie 4, 1202 Genève, CH

Bereich
Technique et IT

Hauptstudiengang
Informatique et systèmes de communication

BSc HES-SO en Ingénierie des technologies de l'information - Haute école du paysage, d'ingénierie et d'architecture de Genève

Algorithmes et structures des données
apprentissage statistique

2020

Artificial intelligence within the interplay between natural and artificial computation: Advances in data science, trends and applications

Scientifique ArODES

Guido Bologna, Juan M. Gorriz, Javier Ramirez, Andrés Ortiz, Francisco J. Martinez-Murcia, Fermin Segovia, John Suckling, Matthew Leming, Yu-Dong Zhang, Jose Ramón Álvarez-Sánchez, Paula Bonomini, Fernando E. Casado, David Charte, Francisco Charte, Ricardo Contreras, Alfredo Cuesta-Infante, Richard J. Duro, Antonio Fernández-Caballero, Eduardo Fernández-Jover, Pedro Gomez-Vilda, Manuel Graña, Francisco Herrera, Roberto Iglesias, Anna Lekova, Javier de Lope, Ezequiel López-Rubio, Rafael Martínez-Tomás, Miguel A. Molina-Cabello, Antonio S. Montemayor, Paulo Novais, Daniel Palacios-Alonso, Juan J. Pantrigo, Bryson R. Payne, Félix de la Paz López, María Angélica Pinninghoff, Mariano Rincon, José Santos, Karl Thurnhofer-Hemsi, Athanasios Tsanas, Ramiro Varela, Jose M. Ferrández

Neurocomputing, 2020, vol. 410, pp. 237-270

Link zur Publikation

Zusammenfassung:

Artificial intelligence and all its supporting tools, e.g. machine and deep learning in computational intelligence-based systems, are rebuilding our society (economy, education, life-style, etc.) and promising a new era for the social welfare state. In this paper we summarize recent advances in data science and artificial intelligence within the interplay between natural and artificial computation. A review of recent works published in the latter field and the state the art are summarized in a comprehensive and self-contained way to provide a baseline framework for the international community in artificial intelligence. Moreover, this paper aims to provide a complete analysis and some relevant discussions of the current trends and insights within several theoretical and application fields covered in the essay, from theoretical models in artificial intelligence and machine learning to the most prospective applications in robotics, neuroscience, brain computer interfaces, medicine and society, in general.

A two-step rule-extraction technique for a CNN

Scientifique ArODES

Guido Bologna, Silvio Fossati

Electronics, 2020, vol. 9, no. 6, article no. 990

Link zur Publikation

Zusammenfassung:

The explanation of the decisions provided by a model are crucial in a domain such as medical diagnosis. With the advent of deep learning, it is very important to explain why a classification is reached by a model. This work tackles the transparency problem of convolutional neural networks(CNNs). We propose to generate propositional rules from CNNs, because they are intuitive to the way humans reason. Our method considers that a CNN is the union of two subnetworks: a multi-layer erceptron (MLP) in the fully connected layers; and a subnetwork including several 2D convolutional layers and max-pooling layers. Rule extraction exhibits two main steps, with each step generating rules from each subnetwork of the CNN. In practice, we approximate the two subnetworks by two particular MLP models that makes it possible to generate propositional rules. We performed the experiments with two datasets involving images: MNISTdigit recognition; and skin-cancer diagnosis. With high fidelity, the extracted rules designated the location of discriminant pixels, as well as the conditions that had to be met to achieve the classification. We illustrated several examples of rules by their centroids and their discriminant pixels.

2018

A rule extraction study from SVM on sentiment analysis

Scientifique ArODES

Guido Bologna, Yoichi Hayashi

Big Data and Cognitive Computing, 2018, vol. 2, no. 1, article no. 6

Link zur Publikation

Zusammenfassung:

A natural way to determine the knowledge embedded within connectionist models is to generate symbolic rules. Nevertheless, extracting rules from Multi Layer Perceptrons (MLPs) is NP-hard. With the advent of social networks, techniques applied to Sentiment Analysis show a growing interest, but rule extraction from connectionist models in this context has been rarely performed because of the very high dimensionality of the input space. To fill the gap we present a case study on rule extraction from ensembles of Neural Networks and Support Vector Machines (SVMs), the purpose being the characterization of the complexity of the rules on two particular Sentiment Analysis problems. Our rule extraction method is based on a special Multi Layer Perceptron architecture for which axis-parallel hyperplanes are precisely located. Two datasets representing movie reviews are transformed into Bag-of-Words vectors and learned by ensembles of neural networks and SVMs. Generated rules from ensembles of MLPs are less accurate and less complex than those extracted from SVMs. Moreover, a clear trade-off appears between rules’ accuracy, complexity and covering. For instance, if rules are too complex, less complex rules can be re-extracted by sacrificing to some extent their accuracy. Finally, rules can be viewed as feature detectors in which very often only one word must be present and a longer list of words must be absent.

A comparison study on rule extraction from neural network ensembles, boosted shallow trees, and SVMs

Scientifique ArODES

Guido Bologna, Yoichi Hayashi

Applied Computational Intelligence and Soft Computing, 2018, article no. 4084850

Link zur Publikation

Zusammenfassung:

One way to make the knowledge stored in an artificial neural network more intelligible is to extract symbolic rules. However, producing rules from Multilayer Perceptrons (MLPs) is an NP-hard problem. Many techniques have been introduced to generate rules from single neural networks, but very few were proposed for ensembles. Moreover, experiments were rarely assessed by 10-fold cross-validation trials. In this work, based on the Discretized Interpretable Multilayer Perceptron (DIMLP), experiments were performed on 10 repetitions of stratified 10-fold cross-validation trials over 25 binary classification problems. The DIMLP architecture allowed us to produce rules from DIMLP ensembles, boosted shallow trees (BSTs), and Support Vector Machines (SVM). The complexity of rulesets was measured with the average number of generated rules and average number of antecedents per rule. From the 25 used classification problems, the most complex rulesets were generated from BSTs trained by “gentle boosting” and “real boosting.” Moreover, we clearly observed that the less complex the rules were, the better their fidelity was. In fact, rules generated from decision stumps trained by modest boosting were, for almost all the 25 datasets, the simplest with the highest fidelity. Finally, in terms of average predictive accuracy and average ruleset complexity, the comparison of some of our results to those reported in the literature proved to be competitive.

2017

Characterization of symbolic rules embedded in deep DIMLP networks :

Scientifique ArODES

a challenge to transparency of deep learning

Guido Bologna, Yoichi Hayashi

Journal of Artificial Intelligence and Soft Computing Research, 2017, vol. 7, no. 4, pp. 266-286

Link zur Publikation

Zusammenfassung:

Rule extraction from neural networks is a fervent research topic. In the last 20 years many authors presented a number of techniques showing how to extract symbolic rules from Multi Layer Perceptrons (MLPs). Nevertheless, very few were related to ensembles of neural networks and even less for networks trained by deep learning. On several datasets we performed rule extraction from ensembles of Discretized Interpretable Multi Layer Perceptrons (DIMLP), and DIMLPs trained by deep learning. The results obtained on the Thyroid dataset and the Wisconsin Breast Cancer dataset show that the predictive accuracy of the extracted rules compare very favorably with respect to state of the art results. Finally, in the last classification problem on digit recognition, generated rules from the MNIST dataset can be viewed as discriminatory features in particular digit areas. Qualitatively, with respect to rule complexity in terms of number of generated rules and number of antecedents per rule, deep DIMLPs and DIMLPs trained by arcing give similar results on a binary classification problem involving digits 5 and 8. On the whole MNIST problem we showed that it is possible to determine the feature detectors created by neural networks and also that the complexity of the extracted rulesets can be well balanced between accuracy and interpretability.

2016

Recursive-rule extraction algorithm with J48graft and applications to generating credit scores

Scientifique ArODES

Guido Bologna, Yoichi Hayashi, Yuki Tanaka, Tomohiro Takagi, Takamichi Saito, Hideaki Iiduka, Hiroaki Kikuchi, Sushmita Mitra

Journal of artificial intelligence and soft computing research, 2016, vol. 6, no. 1, pp. 35-44

Link zur Publikation

Zusammenfassung:

The purpose of this study was to generate more concise rule extraction from the Recursive-Rule Extraction (Re-RX) algorithm by replacing the C4.5 program currently employed in Re-RX with the J48graft algorithm. Experiments were subsequently conducted to determine rules for six different two-class mixed datasets having discrete and continuous attributes and to compare the resulting accuracy, comprehensibility and conciseness. When working with the CARD1, CARD2, CARD3, German, Bene1 and Bene2 datasets, Re-RX with J48graft provided more concise rules than the original Re-RX algorithm. The use of Re-RX with J48graft resulted in 43.2%, 37% and 21% reductions in rules in the case of the German, Bene1 and Bene2 datasets compared to Re-RX. Furthermore, the Re-RX with J48graft showed 8.87% better accuracy than the Re-RX algorithm for the German dataset. These results confirm that the application of Re-RX in conjunction with J48graft has the capacity to facilitate migration from existing data systems toward new concise analytic systems and Big Data.

2014

See ColOr :

Scientifique ArODES

an extended sensory substitution device for the visually impaired

Guido Bologna, Juan D. Gomez, Thierry Pun

Journal of Assistive Technologies,, 2014, vol. 8, no. 2, pp. 77-94

Link zur Publikation

Zusammenfassung:

Purpose : The purpose of this paper is to overcome the limitations of sensory substitution methods (SSDs) to represent high-level or conceptual information involved in vision, which are mainly produced by the biological sensory mismatch between sight and substituting senses. Thus, provide the visually impaired with a more practical and functional SSD. Design/methodology/approach : Unlike any other approach, the SSD extends beyond a sensing prototype, by integrating computer vision methods to produce reliable knowledge about the physical world (at the lowest cost to the user). Importantly though, the authors do not abandon the typical encoding of low-level features into sound. The paper simply argues that any visual perception can be achieved through hearing needs to be reinforced or enhanced by techniques that lie beyond mere visual-to-audio mapping (e.g. computer vision, image processing). Findings : Experiments reported in this paper reveal that the See ColOr is learnable and functional, and provides easy interaction. In moderate time, participants were enabled to grasp visual information of the world out of which they could derive: spatial awareness, ability to find someone, location of daily objects and skill to walk safely avoiding obstacles. The encouraging results open a door toward autonomous mobility of the blind. Originality/value : The paper uses the “extended” approach to introduce and justify that the system is brand new, as well as the experimental studies on computer-vision extension of SSDs that are presented. Also, this is the first paper reporting on a terminated, integrated and functional system.

Walking behavior change detector for a “smart” walker

Scientifique ArODES

Guido Bologna, Viviana Weiss, Séverine Cloix

Procedia Computer Science, 2014, vol. 39, pp. 43-50

Link zur Publikation

Zusammenfassung:

This study investigates the design of a novel real-time system to detect walking behavior changes using an accelerometer on a rollator. No sensor is required on the user. We propose a new non-invasive approach to detect walking behavior based on the motion transfer by the user on the walker. Our method has two main steps; the first is to extract a gait feature vector by analyzing the three-axis accelerometer data in terms of magnitude, gait cycle and frequency. The second is to classify gait with the use of a decision tree of multilayer perceptrons. To assess the performance of our technique, we evaluated different sampling window lengths of 1, 3 an 5 seconds and four different Neural Network architectures. The results revealed that the algorithm can distinguish walking behavior such as normal, slow and fast with an accuracy of about 86%. This research study is part of a project aiming at providing a simple and non-invasive walking behavior detector for elderly who use rollators.

2011

Toward local and global perception modules for vision substitution

Scientifique ArODES

Guido Bologna, Benoît Deville, Juan Diego Gomez, Thierry Pun

Neurocomputing, 2011, vol. 74, pp. 1182-1190

Link zur Publikation

Zusammenfassung:

Although retinal neural implants have considerably progressed they raise a number of questions concerning user acceptance, risk rejection, and cost. For the time being we support a low cost approach based on the transmission of limited vision information by means of the auditory channel. The See ColOr mobility aid for visually impaired individuals transforms a small portion of a coloured video image into sound sources represented by spatialised musical instruments. Basically, the conversion of colours into sounds is achieved by quantisation of the HSL colour system. Our purpose is to provide blind people with a capability of perception of the environment in real time. In this work the novelty is the simultaneous sonification of colour and depth, the last parameter being coded by sound rhythm. The main drawback of our approach is that the sonification of a limited portion of a captured image involves limited perception. As a consequence, we propose to extend the local perception module by introducing a new global perception module aiming at providing the user with a clear picture of the entire scene characteristics. Finally, we present several experiments to illustrate the limited perception module, such as: (1) detecting an open door in order to go out from the office; (2) walking in a hallway and looking for a blue cabinet; (3) walking in a hallway and looking for a red tee shirt; (4) avoiding two red obstacles; (5) moving outside and avoiding a parked car. Videos of experiments are available on http://www.youtube.com/guidobologna.

2007

Filtering Documents with an Hybrid Neural Network Model

Scientifique

Guido Bologna, Paul Albuquerque, Mathieu Boretti

Proc. of the Int'l Work-Conference on the Interplay Between Natural and Artificial Computation, Spain, Lecture Notes in Computer Science, 2007 , vol. 4527-4528

2018

A rule extraction study based on a convolutional neural network

Conference ArODES

Guido Bologna

Lecture Notes in Computer Science ; Proceedings of International Cross-Domain Conference for Machine Learning and Knowledge Extraction (CD-MAKE 2018), 27-30 August 2018, Hamburg, Germany

Link zur Konferenz

Zusammenfassung:

Convolutional Neural Networks (CNNs) lack an explanation capability in the form of propositional rules. In this work we define a simple CNN architecture having a unique convolutional layer, then a Max-Pool layer followed by a full connected layer. Rule extraction is performed after the Max-Pool layer with the use of the Discretized Interpretable Multi Layer Perceptron (DIMLP). The antecedents of the extracted rules represent responses of convolutional filters, which are difficult to understand. However, we show in a sentiment analysis problem that from these “meaningless” values it is possible to obtain rules that represent relevant words in the antecedents. The experiments illustrate several examples of rules that represent n-grams.

2016

Accuracies and number of rules extracted using the Re-RX algorithm family from a pareto-optimal perspective

Conference ArODES

Guido Bologna, Yoichi Hayashi, Riku Hashiguchi

Proceedings of 25th International Conference on Artificial Neural Networks and Machine Learning, ICANN 2016, 6-9 September 2016, Barcelona, Spain

Link zur Konferenz

Zusammenfassung:

The Recursive-Rule eXtraction (Re-RX) algorithm family includes the Re-RX algorithm, the Re-RX algorithm with both discrete and continuous attributes (Continuous Re-RX [1]), the Re-RX algorithm with J48graft [2], Re-RX with J48graft combined with Sampling Selection Techniques (Sampling Re-RX with J48graft [4]), and the Re-RX algorithm with a trained neural network (Sampling Re-RX [3]). In this study, we compare the performance of the Re-RX algorithm family with various previous algorithms. One issue that always remains important in rule extraction is Pareto optimality, or in other words, an ideally balanced trade-off. In rule extraction, the trade-off is between the classification accuracy and interpretability of extracted rules. Our goal is to obtain a wider viable region for the Pareto optimal curve that will enable improvements in both the accuracy and interpretability of extracted rules. We vividly demonstrate Pareto-optimal curves between the accuracies and number of rules obtained for German and Australian datasets by 10 runs of 10-fold cross validation of the Re-RX algorithm family and those obtained using other algorithms. The Re-RX algorithm family has proven effective for extracting concise and interpretable rules from medical [1, 2, 4] and financial [3] datasets.

A rule extraction study on a neural network trained by deep learning

Conference ArODES

Guido Bologna, Yoichi Hayashi

Proceedings of 2016 International Joint Conference on Neural Networks (IJCNN), 24-29 July 2016, Vancouver, BC, Canada

Link zur Konferenz

Zusammenfassung:

Rule extraction from neural networks is a fervent research topic. In the last 20 years many authors presented a number of techniques showing how to extract symbolic rules from Multi Layer Perceptrons (MLPs). Nevertheless, very few were related to ensembles of neural networks and even less for networks trained by deep learning. In this work the Discretized Multi Layer Perceptron (DIMLP) was trained by deep learning, then symbolic rules were extracted in an easier way with respect to standard MLPs. We compared the accuracy of deep trained DIMLPs and DIMLP ensembles on a subset of the MNIST dataset. The former networks were more accurate than the latter. Moreover, the complexity of the rules extracted from deep trained DIMLPs was similar to that obtained by boosted ensembles of DIMLPs. Finally, we examined the generated rules with respect to the centroids of the covered samples. Qualitatively, no clear difference in the strategy of classification emerged between deep trained DIMLPs and the ensembles.

2015

Three-MLP Ensemble Re-RX algorithm and recent classifiers for credit-risk evaluation

Conference ArODES

Guido Bologna, Yoichi Haysahi, Yuki Tanaka, Shonosuke Yukita, Satoshi Nakano

Proceedings of 2015 Inernational Joint Conference on neural Networks (IJCNN), 12-17 July 2015, Killarney, Ireland

Link zur Konferenz

Zusammenfassung:

Credit-risk evaluation is a challenging and important task in the domain of financial analysis for which many classification methods have been suggested. In this paper, we present the results for eight real-life credit-risk two-class mixed datasets (i.e., discrete and continuous attributes) analyzed by the Three-MLP Ensemble Re-RX algorithm (shortened to “Three-MLP Ensemble”). Clarifying the neural network decisions by explanatory rules that capture the learned knowledge embedded in the networks can help a credit-risk manager explain why a particular applicant is classified as either bad or good. To compare the Three-MLP Ensemble performance, we executed comprehensive rule extraction experiments on eight two-class mixed datasets commonly used for benchmarking studies in credit-risk evaluation. The extremely high accuracy of the Three-MLP Ensemble outperformed the accuracies by the Re-RX algorithm and a variant. In this study, we also compared the accuracy of the Three-MLP Ensemble with that of classifiers recently proposed. It is concluded that neural network rule extraction by the Three-MLP Ensemble is a powerful management tool that allows us to build advanced, comprehensible, and accurate decision-support systems for credit-risk evaluation.

QSVM :

Conference ArODES

a support vector machine for rule extraction

Guido Bologna, Yoichi Hayashi

Lecture Notes in Computer Science ; Proceedings of International Work-Conference on Artificial Neural Networks 2015 (IWANN), 10-12 June 2015, Palama de Mallorca, Spain

Link zur Konferenz

Zusammenfassung:

Rule extraction from neural networks represents a difficult research problem, which is NP-hard. In this work we show how a special Multi Layer Perceptron architecture denoted as DIMLP can be used to extract rules from ensembles of DIMLPs and Quantized Support Vector Machines (QSVMs). The key idea for rule extraction is that the locations of discriminative hyperplanes are known, precisely. Based on ten repetitions of stratified 10-fold cross validation trials and with the use of default learning parameters we generated symbolic rules from five datasets. The obtained results compared favorably with respect to another state of the art technique applied to Support Vector Machines.

An embedded ground change detector for a “smart walker”

Conference ArODES

Guido Bologna, Viviana Weiss, Aleksandr Korolev, Séverine Cloix, Thierry Pun

Lecture Notes in Computer Science ; Proceedings of International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2015: Artificial Computation in Biology and Medicine, 1-5 June 20015, Elche, Spain

Link zur Konferenz

Zusammenfassung:

Millions of elderly people around the world use the walker for their mobility; nevertheless, these devices may lead to an accident. One of the cause of these accidents is misjudge the terrain. The main objective of this work is the implementation of a ground change detector in real time on a small and light embedded system that can be clipped on a rollator. As a long-term goal, this device will allow users to anticipate entering dangerous situations. We implemented an algorithm to detect ground changes based on color histograms and texture descriptor given as inputs to multi-layer perceptrons. Experiments were performed both off-line and with an embedded system. The obtained results indicated that it is possible to have an accurate detector which is able to distinguish ground changes in real-time.

2014

Descending stairs detection with low-power sensors

Conference ArODES

David Hasler, Guido Bologna, Séverine Cloix, Viviana Weiss, Thierry Pun

Lecture Notes in Computer Science; Proceedings of European Conference on Computer Vision: ECCV 2014: Computer Vision - ECCV 2014 Workshops, 6-12 September 2014, Zurich, Switzerland

Link zur Konferenz

Zusammenfassung:

With the increasing proportion of senior citizens, many mobility aid devices were developed such as the rollator. However among walker’s users, 87% of their falls is attributed to rollators. The EyeWalker project aims at developing a small device for rollators to protect elderly people from such dangers. Descending stairs are ones of the potential hazards rollator users have to daily face. We propose a method to detect them in real-time using a passive stereo camera. To meet the requirements of low-power consumption, we examined the performance of our stereo vision based detector with regard to the camera resolution. It succeeds in differentiating dangerously approaching stairs from safe situations at low resolutions. In the future, our detector will be ported on an embedded platform equipped with a pair of low-resolution and high dynamic range stereo camera for both indoor and outdoor usage with a battery-life of several days.

Obstacle and planar object detection using sparse 3D information for a smart walker

Conference ArODES

David Hasler, Guido Bologna, Séverine Cloix, Viviana Weiss, Thierry Pun

Proceedings of 2014 International Conference on Computer Vision Theory and Applications (VISAPP), 8-8 January 2014, Lisbon, Portugal

Link zur Konferenz

Zusammenfassung:

With the increasing proportion of senior citizens, many mobility aid devices have been developed such as the rollator. However, under some circumstances, the latter may cause accidents. The EyeWalker project aims to develop a small and autonomous device for rollators to help elderly people, especially those with some degree of visual impairment, avoiding common dangers like obstacles and hazardous ground changes, both outdoors and indoors. We propose amethod of real-time stereo obstacle detection using sparse 3D information. Working with sparse 3D points, in opposition to dense 3D maps, is computationally more efficient and more appropriate for a long battery-life. In our approach, 3D data are extracted from a stereo-rig of two 2D high dynamic range cameras developed at the CSEM (Centre Suisse d'Electronique et de Microtechnique) and processed to perform a boosting classification. We also present a deformable 3D object detector for which the 3D points are combined in several different ways and result in a set of pose estimates used to execute a less ill-posed classification. The evaluation, carried out on real stereo images of obstacles described with both 2D and 3D features, shows promising results for a future use in real-world conditions.

A robust, real-time ground change detector for a "smart" walker

Conference ArODES

David Hasler, Guido Bologna, Viviana Weiss, Séverine Cloix, Thierry Pun

Proceedings of 2014 International Conference on Computer Vision Theory and Applications (VISAPP), 8-8 January 2014, Lisbon, Portugal

Link zur Konferenz

Zusammenfassung:

Nowadays, there are many different types of mobility aids for elderly people. Nevertheless, these devices may lead to accidents, depending on the terrain where they are being used. In this paper, we present a robust ground change detector that will warn the user of potentially risky situations. Specifically, we propose a robust classification algorithm to detect ground changes based on colour histograms and texture descriptors. In our design, we compare the current frame and the average of the k previous frames using different colour systems and Local Edge Patterns. To assess the performance of our algorithm, we evaluated different Artificial Neural Networks architectures. The best results were obtained by representing in the input neurons measures related to Histogram Intersections, Kolmogorov-Smirnov distance, Cumulative Integrals and Earth mover's distance. Under real environmental conditions our results indicated that our proposed detector can accurately distinguish the grounds changes in real-time.

2012

Spatial awareness and intelligibility for the blind :

Conference ArODES

audio-touch interfaces

Guido Bologna, Juan Diego Gomez, Thierry Pun

Proceedings of CHIA EA'12, CHI'12, ACM SIGCHI Conference on Human Factors in Computing Systems, 5-10 May 2011, Austin, Texas, USA

Link zur Konferenz

Zusammenfassung:

Many people with visual disabilities mainly use audio feedback as a primary modality for interaction. Representing the visual environment with appropriate sounds contributes to make it intelligible to the blind. This audio-encoded environment still needs to be accessed in the same way as sighted people scan visual contents with their gaze. A finger-based scanning could be seen as a gaze-like strategy for those with visual impairments to be able of sensing an audio-represented context. We present in this work a computational interface that meets both, the visual-audio codification and the multi-touch interaction, so as to enlarge legibility of the environment for the blind and to facilitate navigating to desired locations, exploration, and serendipitous discovery. The core of this interface is the color and depth codification into musical instruments sounds, which effectively provides spatial awareness, audio revealing of boundaries and obstacles detection. The main contribution of our work is the assistance provided by this interface toward an active interaction of the user with his fingers that makes it possible to selectively explore, to discover points of interest, develop personalized strategies for navigating, and, in general, enjoy a greater sense of independence.

2011

Toward 3D scene understanding via audio-description :

Conference ArODES

kinect-iPad fusion for the visually impaired

Guido Bologna, Juan Diego Gomez, Sinan Mohammed, Thierry Pun

Proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility (ASSESTS'11), 24-26 October 2011, Dundee, Scotland, UK

Link zur Konferenz

Zusammenfassung:

Microsoft's Kinect 3-D motion sensor is a low cost 3D camera that provides color and depth information of indoor environments. In this demonstration, the functionality of this fun-only camera accompanied by an iPad's tangible interface is targeted to the benefit of the visually impaired. A computer-vision-based framework for real time objects localization and for their audio description is introduced. Firstly, objects are extracted from the scene and recognized using feature descriptors and machine-learning. Secondly, the recognized objects are labeled by instruments sounds, whereas their position in 3D space is described by virtual space sources of sound. As a result, the scene can be heard and explored while finger-triggering the sounds within the iPad, on which a top-view of the objects is mapped. This enables blindfolded users to build a mental occupancy grid of the environment. The approach presented here brings the promise of efficient assistance and could be adapted as an electronic travel aid for the visually-impaired in the near future.

Multisource sonification for visual substitution in an auditory memory game :

Conference ArODES

one, or two fingers?

Guido Bologna, Juan Diego Gomez, Benoît Deville, Thierry Pun

Proceedings of the 17th International Conference on Auditory Display (ICAD-2011), 20-24 June 2011, Budapest, Hungary

Link zur Konferenz

Zusammenfassung:

The See ColOr project aims at developing a mobility system for blind persons based on image color sonification. Within this project the present work addresses the optimal use of auditory-multi-touch interaction, and in particular the matter of the number of fingers needed for efficient exploration. To determine the actual significance of mono and multi-touch interaction onto the auditory feedback, a color matching memory game was implemented. Sounds of this game were generated by touching a tablet with one or two fingers. A group of 20 blindfolded users was tasked to find color matches into an image grid represented on the tablet by listening to their associated color-sound representation. Our results show that for an easy task aiming at matching few objects, the use of two fingers is moderately more efficient than the use of one finger. Whereas, against our intuition, this cannot be statistically confirmed in the case of similar tasks of increasing difficulty.

2010

Detecting objects and obstacles for visually impaired individuals using visual saliency

Conference ArODES

Guido Bologna, Benoît Deville, Thierry Pun

Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility, 25-27 October 2010, Orlando, FL, USA

Link zur Konferenz

Zusammenfassung:

In this demo, we present the detection module of the See ColOr (Seeing Colors with an Orchestra) mobility aid for visually impaired persons. This module points out areas that present either particular interest or potential threat. In order to detect object and obstacles, we propose a bottom-up approach based on visual saliency: objects that would attract the visual attention of a non-disabled individual are pointed out by the system as areas of interest for the user. The device uses a stereoscopic camera, a laptop, and standard headphones. Given the type of scene and/or scenario, specific feature maps are computed in order to indicate areas of interest in real-time. This demonstration shows that the module indicates objects and obstacles as accurately as a system using all available feature maps.

Color-audio encoding interface for visual substitution :

Conference ArODES

see color matlab-based demo

Guido Bologna, Juan Diego Gomez, Thierry Pun

Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility, 25-27 October 2010, Orlando, FL, USA

Link zur Konferenz

Zusammenfassung:

Providing the blind with substitute visual perception is a relentless challenge confronting researchers of diverse areas. The See ColOr (Seeing Color with an Orchestra) system translates aimed-at regions of a color scene into 2D-spatialized sound signals represented by musical instruments. Associating sounds with colors is achieved thanks to an efficient quantization of the regions' HSL representation (Hue, Saturation, Luminosity). See ColOr can be used both for exploring static images and as a mobility aid. In this report, we introduce its general framework for color-audio encoding of images to provide visually impaired individuals with a low-cost application for color perception substitution, using the sense of hearing instead of sight. The experimental tests using a demo implementation have shown the feasibility and usefulness of this approach.

Errungenschaften