Search

Bologna Guido

Professeur HES assistant

Main skills

Machine Learning

Main contract

Professeur HES assistant

Phone: +41 22 558 57 61

Desktop: A410

Haute école du paysage, d'ingénierie et d'architecture de Genève
Rue de la Prairie 4, 1202 Genève, CH

Faculty
Technique et IT

Main Degree Programme
Informatique et systèmes de communication

BSc HES-SO en Ingénierie des technologies de l'information - Haute école du paysage, d'ingénierie et d'architecture de Genève

Algorithmes et structures des données
apprentissage statistique

2023

Transferring CNN features maps to ensembles of explainable neural networks

Scientific paper ArODES

Guido Bologna

Information, 2023, vol. 14, no. 2, article no. 89

Link to the publication

Summary:

The explainability of connectionist models is nowadays an ongoing research issue. Before the advent of deep learning, propositional rules were generated from Multi Layer Perceptrons (MLPs) to explain how they classify data. This type of explanation technique is much less prevalent with ensembles of MLPs and deep models, such as Convolutional Neural Networks (CNNs). Our main contribution is the transfer of CNN feature maps to ensembles of DIMLP networks, which are translatable into propositional rules. We carried out three series of experiments; in the first, we applied DIMLP ensembles to a Covid dataset related to diagnosis from symptoms to show that the generated propositional rules provided intuitive explanations of DIMLP classifications. Then, our purpose was to compare rule extraction from DIMLP ensembles to other techniques using cross-validation. On four classification problems with over 10,000 samples, the rules we extracted provided the highest average predictive accuracy and fidelity. Finally, for the melanoma diagnostic problem, the average predictive accuracy of CNNs was 84.5% and the average fidelity of the top-level generated rules was 95.5%. The propositional rules generated from the CNNs were mapped at the input layer by squares in which the relevant data for the classifications resided. These squares represented regions of attention determining the final classification, with the rules providing logical reasoning.

2021

A rule extraction technique applied to ensembles of neural networks, random forests, and gradient-boosted trees

Scientific paper ArODES

Guido Bologna

Algorithms, 2021, vol. 14, no. 12, article no. 339

Link to the publication

Summary:

In machine learning, ensembles of models based on Multi-Layer Perceptrons (MLPs) or decision trees are considered successful models. However, explaining their responses is a complex problem that requires the creation of new methods of interpretation. A natural way to explain the classifications of the models is to transform them into propositional rules. In this work, we focus on random forests and gradient-boosted trees. Specifically, these models are converted into an ensemble of interpretable MLPs from which propositional rules are produced. The rule extraction method presented here allows one to precisely locate the discriminating hyperplanes that constitute the antecedents of the rules. In experiments based on eight classification problems, we compared our rule extraction technique to “Skope-Rules” and other state-of-the-art techniques. Experiments were performed with ten-fold cross-validation trials, with propositional rules that were also generated from ensembles of interpretable MLPs. By evaluating the characteristics of the extracted rules in terms of complexity, fidelity, and accuracy, the results obtained showed that our rule extraction technique is competitive. To the best of our knowledge, this is one of the few works showing a rule extraction technique that has been applied to both ensembles of decision trees and neural networks.

2020

Artificial intelligence within the interplay between natural and artificial computation:

Scientific paper ArODES

advances in data science, trends and applications

Juan M. Gorriz, Javier Ramirez, Andrés Ortiz, Francisco J. Martinez-Murcia, Fermin Segovia, John Suckling, Matthew Leming, Yu-Dong Zhang, Jose Ramón Álvarez-Sánchez, Guido Bologna, Paula Bonomini, Fernando E. Casado, David Charte, Francisco Charte, Ricardo Contreras, Alfredo Cuesta-Infante, Richard J. Duro, Antonio Fernández-Caballero, Eduardo Fernández-Jover, Pedro Gomez-Vilda, Manuel Graña, Francisco Herrera, Roberto Iglesias, Anna Lekova, Javier de Lope, Ezequiel López-Rubio, Rafael Martínez-Tomás, Miguel A. Molina-Cabello, Antonio S. Montemayor, Paulo Novais, Daniel Palacios-Alonso, Juan J. Pantrigo, Bryson R. Payne, Félix de la Paz López, María Angélica Pinninghoff, Mariano Rincon, José Santos, Karl Thurnhofer-Hemsi, Athanasios Tsanas, Ramiro Varela, Jose M. Ferrández

Neurocomputing, 2020, vol. 410, pp. 237-270

Link to the publication

Summary:

Artificial intelligence and all its supporting tools, e.g. machine and deep learning in computational intelligence-based systems, are rebuilding our society (economy, education, life-style, etc.) and promising a new era for the social welfare state. In this paper we summarize recent advances in data science and artificial intelligence within the interplay between natural and artificial computation. A review of recent works published in the latter field and the state the art are summarized in a comprehensive and self-contained way to provide a baseline framework for the international community in artificial intelligence. Moreover, this paper aims to provide a complete analysis and some relevant discussions of the current trends and insights within several theoretical and application fields covered in the essay, from theoretical models in artificial intelligence and machine learning to the most prospective applications in robotics, neuroscience, brain computer interfaces, medicine and society, in general.

A two-step rule-extraction technique for a CNN

Scientific paper ArODES

Guido Bologna, Silvio Fossati

Electronics, 2020, vol. 9, no. 6, article no. 990

Link to the publication

Summary:

The explanation of the decisions provided by a model are crucial in a domain such as medical diagnosis. With the advent of deep learning, it is very important to explain why a classification is reached by a model. This work tackles the transparency problem of convolutional neural networks(CNNs). We propose to generate propositional rules from CNNs, because they are intuitive to the way humans reason. Our method considers that a CNN is the union of two subnetworks: a multi-layer erceptron (MLP) in the fully connected layers; and a subnetwork including several 2D convolutional layers and max-pooling layers. Rule extraction exhibits two main steps, with each step generating rules from each subnetwork of the CNN. In practice, we approximate the two subnetworks by two particular MLP models that makes it possible to generate propositional rules. We performed the experiments with two datasets involving images: MNISTdigit recognition; and skin-cancer diagnosis. With high fidelity, the extracted rules designated the location of discriminant pixels, as well as the conditions that had to be met to achieve the classification. We illustrated several examples of rules by their centroids and their discriminant pixels.

2018

A rule extraction study from SVM on sentiment analysissvm

Scientific paper ArODES

Guido Bologna, Yoichi Hayashi

Big Data and Cognitive Computing, 2018, vol. 2, no. 1, article no. 6

Link to the publication

Summary:

A natural way to determine the knowledge embedded within connectionist models is to generate symbolic rules. Nevertheless, extracting rules from Multi Layer Perceptrons (MLPs) is NP-hard. With the advent of social networks, techniques applied to Sentiment Analysis show a growing interest, but rule extraction from connectionist models in this context has been rarely performed because of the very high dimensionality of the input space. To fill the gap we present a case study on rule extraction from ensembles of Neural Networks and Support Vector Machines (SVMs), the purpose being the characterization of the complexity of the rules on two particular Sentiment Analysis problems. Our rule extraction method is based on a special Multi Layer Perceptron architecture for which axis-parallel hyperplanes are precisely located. Two datasets representing movie reviews are transformed into Bag-of-Words vectors and learned by ensembles of neural networks and SVMs. Generated rules from ensembles of MLPs are less accurate and less complex than those extracted from SVMs. Moreover, a clear trade-off appears between rules’ accuracy, complexity and covering. For instance, if rules are too complex, less complex rules can be re-extracted by sacrificing to some extent their accuracy. Finally, rules can be viewed as feature detectors in which very often only one word must be present and a longer list of words must be absent.

A comparison study on rule extraction from neural network ensembles, boosted shallow trees, and SVMs

Scientific paper ArODES

Guido Bologna, Yoichi Hayashi

Applied Computational Intelligence and Soft Computing, 2018, article no. 4084850

Link to the publication

Summary:

One way to make the knowledge stored in an artificial neural network more intelligible is to extract symbolic rules. However, producing rules from Multilayer Perceptrons (MLPs) is an NP-hard problem. Many techniques have been introduced to generate rules from single neural networks, but very few were proposed for ensembles. Moreover, experiments were rarely assessed by 10-fold cross-validation trials. In this work, based on the Discretized Interpretable Multilayer Perceptron (DIMLP), experiments were performed on 10 repetitions of stratified 10-fold cross-validation trials over 25 binary classification problems. The DIMLP architecture allowed us to produce rules from DIMLP ensembles, boosted shallow trees (BSTs), and Support Vector Machines (SVM). The complexity of rulesets was measured with the average number of generated rules and average number of antecedents per rule. From the 25 used classification problems, the most complex rulesets were generated from BSTs trained by “gentle boosting” and “real boosting.” Moreover, we clearly observed that the less complex the rules were, the better their fidelity was. In fact, rules generated from decision stumps trained by modest boosting were, for almost all the 25 datasets, the simplest with the highest fidelity. Finally, in terms of average predictive accuracy and average ruleset complexity, the comparison of some of our results to those reported in the literature proved to be competitive.

2017

Characterization of symbolic rules embedded in deep DIMLP networks :

Scientific paper ArODES

a challenge to transparency of deep learning

Guido Bologna, Yoichi Hayashi

Journal of Artificial Intelligence and Soft Computing Research, 2017, vol. 7, no. 4, pp. 266-286

Link to the publication

Summary:

Rule extraction from neural networks is a fervent research topic. In the last 20 years many authors presented a number of techniques showing how to extract symbolic rules from Multi Layer Perceptrons (MLPs). Nevertheless, very few were related to ensembles of neural networks and even less for networks trained by deep learning. On several datasets we performed rule extraction from ensembles of Discretized Interpretable Multi Layer Perceptrons (DIMLP), and DIMLPs trained by deep learning. The results obtained on the Thyroid dataset and the Wisconsin Breast Cancer dataset show that the predictive accuracy of the extracted rules compare very favorably with respect to state of the art results. Finally, in the last classification problem on digit recognition, generated rules from the MNIST dataset can be viewed as discriminatory features in particular digit areas. Qualitatively, with respect to rule complexity in terms of number of generated rules and number of antecedents per rule, deep DIMLPs and DIMLPs trained by arcing give similar results on a binary classification problem involving digits 5 and 8. On the whole MNIST problem we showed that it is possible to determine the feature detectors created by neural networks and also that the complexity of the extracted rulesets can be well balanced between accuracy and interpretability.

2016

Recursive-rule extraction algorithm with J48graft and applications to generating credit scores

Scientific paper ArODES

Yoichi Hayashi, Yuki Tanaka, Tomohiro Takagi, Takamichi Saito, Hideaki Iiduka, Hiroaki Kikuchi, Guido Bologna, Sushmita Mitra

Journal of artificial intelligence and soft computing research, 2016, vol. 6, no. 1, pp. 35-44

Link to the publication

Summary:

The purpose of this study was to generate more concise rule extraction from the Recursive-Rule Extraction (Re-RX) algorithm by replacing the C4.5 program currently employed in Re-RX with the J48graft algorithm. Experiments were subsequently conducted to determine rules for six different two-class mixed datasets having discrete and continuous attributes and to compare the resulting accuracy, comprehensibility and conciseness. When working with the CARD1, CARD2, CARD3, German, Bene1 and Bene2 datasets, Re-RX with J48graft provided more concise rules than the original Re-RX algorithm. The use of Re-RX with J48graft resulted in 43.2%, 37% and 21% reductions in rules in the case of the German, Bene1 and Bene2 datasets compared to Re-RX. Furthermore, the Re-RX with J48graft showed 8.87% better accuracy than the Re-RX algorithm for the German dataset. These results confirm that the application of Re-RX in conjunction with J48graft has the capacity to facilitate migration from existing data systems toward new concise analytic systems and Big Data.

2014

See ColOr :

Scientific paper ArODES

an extended sensory substitution device for the visually impaired

Juan D. Gomez, Guido Bologna, Thierry Pun

Journal of Assistive Technologies,, 2014, vol. 8, no. 2, pp. 77-94

Link to the publication

Summary:

Purpose : The purpose of this paper is to overcome the limitations of sensory substitution methods (SSDs) to represent high-level or conceptual information involved in vision, which are mainly produced by the biological sensory mismatch between sight and substituting senses. Thus, provide the visually impaired with a more practical and functional SSD. Design/methodology/approach : Unlike any other approach, the SSD extends beyond a sensing prototype, by integrating computer vision methods to produce reliable knowledge about the physical world (at the lowest cost to the user). Importantly though, the authors do not abandon the typical encoding of low-level features into sound. The paper simply argues that any visual perception can be achieved through hearing needs to be reinforced or enhanced by techniques that lie beyond mere visual-to-audio mapping (e.g. computer vision, image processing). Findings : Experiments reported in this paper reveal that the See ColOr is learnable and functional, and provides easy interaction. In moderate time, participants were enabled to grasp visual information of the world out of which they could derive: spatial awareness, ability to find someone, location of daily objects and skill to walk safely avoiding obstacles. The encouraging results open a door toward autonomous mobility of the blind. Originality/value : The paper uses the “extended” approach to introduce and justify that the system is brand new, as well as the experimental studies on computer-vision extension of SSDs that are presented. Also, this is the first paper reporting on a terminated, integrated and functional system.

Walking behavior change detector for a “smart” walker

Scientific paper ArODES

Viviana Weiss, Guido Bologna, Séverine Cloix

Procedia Computer Science, 2014, vol. 39, pp. 43-50

Link to the publication

Summary:

This study investigates the design of a novel real-time system to detect walking behavior changes using an accelerometer on a rollator. No sensor is required on the user. We propose a new non-invasive approach to detect walking behavior based on the motion transfer by the user on the walker. Our method has two main steps; the first is to extract a gait feature vector by analyzing the three-axis accelerometer data in terms of magnitude, gait cycle and frequency. The second is to classify gait with the use of a decision tree of multilayer perceptrons. To assess the performance of our technique, we evaluated different sampling window lengths of 1, 3 an 5 seconds and four different Neural Network architectures. The results revealed that the algorithm can distinguish walking behavior such as normal, slow and fast with an accuracy of about 86%. This research study is part of a project aiming at providing a simple and non-invasive walking behavior detector for elderly who use rollators.

2011

Toward local and global perception modules for vision substitution

Scientific paper ArODES

Guido Bologna, Benoît Deville, Juan Diego Gomez, Thierry Pun

Neurocomputing, 2011, vol. 74, pp. 1182-1190

Link to the publication

Summary:

Although retinal neural implants have considerably progressed they raise a number of questions concerning user acceptance, risk rejection, and cost. For the time being we support a low cost approach based on the transmission of limited vision information by means of the auditory channel. The See ColOr mobility aid for visually impaired individuals transforms a small portion of a coloured video image into sound sources represented by spatialised musical instruments. Basically, the conversion of colours into sounds is achieved by quantisation of the HSL colour system. Our purpose is to provide blind people with a capability of perception of the environment in real time. In this work the novelty is the simultaneous sonification of colour and depth, the last parameter being coded by sound rhythm. The main drawback of our approach is that the sonification of a limited portion of a captured image involves limited perception. As a consequence, we propose to extend the local perception module by introducing a new global perception module aiming at providing the user with a clear picture of the entire scene characteristics. Finally, we present several experiments to illustrate the limited perception module, such as: (1) detecting an open door in order to go out from the office; (2) walking in a hallway and looking for a blue cabinet; (3) walking in a hallway and looking for a red tee shirt; (4) avoiding two red obstacles; (5) moving outside and avoiding a parked car. Videos of experiments are available on http://www.youtube.com/guidobologna.

2007

Filtering Documents with an Hybrid Neural Network Model

Scientific paper

Bologna Guido, Albuquerque Paul, Mathieu Boretti

Proc. of the Int'l Work-Conference on the Interplay Between Natural and Artificial Computation, Spain, Lecture Notes in Computer Science, 2007 , vol. 4527-4528

2022

Explaining CNN classifications by propositional rules generated from DCT feature maps

Conference ArODES

Guido Bologna

Proceedings of the Bio-inspired Systems and Applications: from Robotics to Ambient Intelligence, 9th International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2022, 31 May, 3 June 2022, Puerto de la Cruz, Tenerife, Spain