Search

Müller Henning

Professeur-e HES Ordinaire, Responsable de l'unité eHealth

Main contract

Professeur-e HES Ordinaire, Responsable de l'unité eHealth

Phone: +41 58 606 90 36

Desktop: TP108

HES-SO Valais-Wallis - Haute Ecole de Gestion
Route de la Plaine 2, Case postale 80, 3960 Sierre, CH

Faculty
Economie et services

Main Degree Programme
Informatique de gestion

MSc HES-SO en Business Administration - HES-SO Master

Méthodologie de recherche

BSc HES-SO en Informatique de gestion - HES-SO Valais-Wallis - Haute Ecole de Gestion

Software Lifecycle Management

Ongoing

H2020 EXAMODE-Valorisation

AGP

Role: Main Applicant

Description du projet : EXAMODE-Valorisation

Research team within HES-SO: Müller Henning

Partenaires académiques: VS - Institut Informatique

Durée du projet: 01.01.2019 - 31.12.2024

Statut: Ongoing

Completed

ProHand (3D printed prosthetic hand tests on Cybathlon tasks

AGP

Role: Main Applicant

Financement: Fondation Hasler

Description du projet : ProHand (3D printed prosthetic hand tests on Cybathlon tasks Résumé à compléter

Research team within HES-SO: Atzori Manfredo , Müller Henning , Reggiani Franco , Buosi Alessandro

Partenaires académiques: VS - Institut Informatique

Durée du projet: 01.11.2019 - 31.10.2020

Montant global du projet: 99'800 CHF

Statut: Completed

2022

Empowering digital pathology applications through explainable knowledge extraction tools

Scientific paper ArODES

Stefano Marchesin, Fabio Giachelle, Niccolò Marini, Manfredo Atzori, Svetla Boytcheva, Genziana Buttafuoco, Francesco Ciompi, Giorgio Maria Di Nunzio, Filippo Fraggetta, Ornella Irrera, Henning Müller, Todor Primov, Simona Vatrano, Gianmaria Silvello

Journal of pathology informatics, 2022, vol. 13, article no. 100139, pp. 1-14

Link to the publication

Summary:

Exa-scale volumes of medical data have been produced for decades. In most cases, the diagnosis is reported in free text, encoding medical knowledge that is still largely unexploited. In order to allow decoding medical knowledge included in reports, we propose an unsupervised knowledge extraction system combining a rule-based expert system with pre-trained Machine Learning (ML) models, namely the Semantic Knowledge Extractor Tool (SKET). Combining rule-based techniques and pre-trained ML models provides high accuracy results for knowledge extraction. This work demonstrates the viability of unsupervised Natural Language Processing (NLP) techniques to extract critical information from cancer reports, opening opportunities such as data mining for knowledge extraction purposes, precision medicine applications, structured report creation, and multimodal learning. SKET is a practical and unsupervised approach to extracting knowledge from pathology reports, which opens up unprecedented opportunities to exploit textual and multimodal medical information in clinical practice. We also propose SKET eXplained (SKET X), a web-based system providing visual explanations about the algorithmic decisions taken by SKET. SKET X is designed/developed to support pathologists and domain experts in understanding SKET predictions, possibly driving further improvements to the system.

A global taxonomy of interpretable AI :

Scientific paper ArODES

unifying the terminology for the technical and social sciences

Mara Graziani, Lidia Dutkiewicz, Davide Calvaresi, José Pereira Amorim, Katerina Yordanova, Mor Vered, Rahul Nair, Pedro Henriques Abreu, Tobias Blanke, Valeria Pulignano, John O. Prior, Lode Lauwaert, Wessel Reijers, Adrien Depeursinge, Vincent Andrearczyk, Henning Müller

Artificial intelligence review, To be published

Link to the publication

Summary:

Since its emergence in the 1960s, Artificial Intelligence (AI) has grown to conquer many technology products and their fields of application. Machine learning, as a major part of the current AI solutions, can learn from the data and through experience to reach high performance on various tasks. This growing success of AI algorithms has led to a need for interpretability to understand opaque models such as deep neural networks. Various requirements have been raised from different domains, together with numerous tools to debug, justify outcomes, and establish the safety, fairness and reliability of the models. This variety of tasks has led to inconsistencies in the terminology with, for instance, terms such as interpretable, explainable and transparent being often used interchangeably in methodology papers. These words, however, convey different meanings and are “weighted" differently across domains, for example in the technical and social sciences. In this paper, we propose an overarching terminology of interpretability of AI systems that can be referred to by the technical developers as much as by the social sciences community to pursue clarity and efficiency in the definition of regulations for ethical and reliable AI development. We show how our taxonomy and definition of interpretable AI differ from the ones in previous research and how they apply with high versatility to several domains and use cases, proposing a—highly needed—standard for the communication among interdisciplinary areas of AI.

Overview of the ImageCLEF 2022 :

Book chapter ArODES

multimedia retrieval in medical, social media and nature applications, in experimental IR meets multilinguality, multimodality, and interaction

Bogdan Ionescu, Henning Müller, Renaud Péteri, Johannes Rückert, Asma Ben Abacha, Alba García Seco de Herrera, Christoph M. Friedrich, Louise Bloch, Raphael Brüngel, Ahmad Idrissi-Yaghir, Henning Schäfer, Serge Kozlovski, Yashin Dicente Cid, Vassili Kovalev, Liviu-Daniel Stefan, Mihai Gabriel Constantin, Mihai Dogariu, Adrian Popescu, Jérôme Deshayes-Chossart, Hugo Schindler, Jon Chamberlain, Antonio Campello, Adrian Clark

Dans Barrón-Cedeño, Alberto, Experimental IR meets multilinguality, multimodality, and interaction (pp 541–564). 2022, Cham : Springer

Link to the publication

Summary:

This paper presents an overview of the ImageCLEF 2022 lab that was organized as part of the Conference and Labs of the Evaluation Forum – CLEF Labs 2022. ImageCLEF is an ongoing evaluation initiative (first run in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2022, the 20th edition of ImageCLEF runs four main tasks: (i) a medical task that groups two previous tasks, i.e., caption analysis and tuberculosis prediction, (ii) a social media aware task on estimating potential real-life effects of online image sharing, (iii) a nature coral task about segmenting and labeling collections of coral reef images, and (iv) a new fusion task addressing the design of late fusion schemes for boosting the performance, with two real-world applications: image search diversification (retrieval) and prediction of visual interestingness (regression). The benchmark campaign received the participation of over 25 groups submitting more than 258 runs.

Unleashing the potential of digital pathology data by training computer-aided diagnosis models without human annotations

Scientific paper ArODES

Niccolò Marini, Stefano Marchesin, Sebastian Otálora, Marek Wodzinski, Alessandro Caputo, Mart Van Rijthoven, Witali Aswolinskiy, John-Melle Bokhorst, Damian Podareanu, Edyta Petters, Svetla Boytcheva, Genziana Buttafuoco, Simona Vatrano, Filippo Fraggetta, Jeroen Van der Laak, Maristella Agosti, Francesco Ciompi, Gianmaria Silvello, Henning Müller, Manfredo Atzori

npj Digital Medicine, 2022, vol. 5, article no. 102, pp. 1-18

Link to the publication

Summary:

The digitalization of clinical workflows and the increasing performance of deep learning algorithms are paving the way towards new methods for tackling cancer diagnosis. However, the availability of medical specialists to annotate digitized images and free-text diagnostic reports does not scale with the need for large datasets required to train robust computer-aided diagnosis methods that can target the high variability of clinical cases and data produced. This work proposes and evaluates an approach to eliminate the need for manual annotations to train computer-aided diagnosis tools in digital pathology. The approach includes two components, to automatically extract semantically meaningful concepts from diagnostic reports and use them as weak labels to train convolutional neural networks (CNNs) for histopathology diagnosis. The approach is trained (through 10-fold cross-validation) on 3’769 clinical images and reports, provided by two hospitals and tested on over 11’000 images from private and publicly available datasets. The CNN, trained with automatically generated labels, is compared with the same architecture trained with manual labels. Results show that combining text analysis and end-to-end deep neural networks allows building computer-aided diagnosis tools that reach solid performance (micro-accuracy = 0.908 at image-level) based only on existing clinical data without the need for manual annotations.

Editorial :

Scientific paper ArODES

neuroimaging and informatics for successful aging

Toshiharu Nakai, Eric Tatt Wei Ho, Henning Müller, Fanpei G. Yang, Hanna Lu

Frontiers in human neuroscience, 2022, vol. 16, article no. 976524, pp. 1-3

Link to the publication

Evaluation of methods for the extraction spatial muscle synergies

Scientific paper ArODES

Kunkun Zhao, Haiying Wen, Zhisheng Zhang, Manfredo Atzori, Henning Müller, Zhongqu Xie, Alessandro Scano

Frontiers in neuroscience, 2022, vol. 16, article 732156

Link to the publication

Summary:

Muscle synergies have been largely used in many application fields, including motor control studies, prosthesis control, movement classification, rehabilitation, and clinical studies. Due to the complexity of the motor control system, the full repertoire of the underlying synergies has been identified only for some classes of movements and scenarios. Several extraction methods have been used to extract muscle synergies. However, some of these methods may not effectively capture the nonlinear relationship between muscles and impose constraints on input signals or extracted synergies. Moreover, other approaches such as autoencoders (AEs), an unsupervised neural network, were recently introduced to study bioinspired control and movement classification. In this study, we evaluated the performance of five methods for the extraction of spatial muscle synergy, namely, principal component analysis (PCA), independent component analysis (ICA), factor analysis (FA), nonnegative matrix factorization (NMF), and AEs using simulated data and a publicly available database. To analyze the performance of the considered extraction methods with respect to several factors, we generated a comprehensive set of simulated data (ground truth), including spatial synergies and temporal coefficients. The signal-to-noise ratio (SNR) and the number of channels (NoC) varied when generating simulated data to evaluate their effects on ground truth reconstruction. This study also tested the efficacy of each synergy extraction method when coupled with standard classification methods, including K-nearest neighbors (KNN), linear discriminant analysis (LDA), support vector machines (SVM), and Random Forest (RF). The results showed that both SNR and NoC affected the outputs of the muscle synergy analysis. Although AEs showed better performance than FA in variance accounted for and PCA in synergy vector similarity and activation coefficient similarity, NMF and ICA outperformed the other three methods. Classification tasks showed that classification algorithms were sensitive to synergy extraction methods, while KNN and RF outperformed the other two methods for all extraction methods; in general, the classification accuracy of NMF and PCA was higher. Overall, the results suggest selecting suitable methods when performing muscle synergy-related analysis.

Grading diabetic retinopathy and prostate cancer diagnostic images with deep quantum ordinal regression

Scientific paper ArODES

Santiago Toledo-Cortés, Diego H. Useche, Henning Müller, Fabio A. González

Computers in Biology and Medicine, 2022, vol. 145, article no. 105472

Link to the publication

Summary:

Although for many diseases there is a progressive diagnosis scale, automatic analysis of grade-based medical images is quite often addressed as a binary classification problem, missing the finer distinction and intrinsic relation between the different possible stages or grades. Ordinal regression (or classification) considers the order of the values of the categorical labels and thus takes into account the order of grading scales used to assess the severity of different medical conditions. This paper presents a quantum-inspired deep probabilistic learning ordinal regression model for medical image diagnosis that takes advantage of the representational power of deep learning and the intrinsic ordinal information of disease stages. The method is evaluated on two different medical image analysis tasks: prostate cancer diagnosis and diabetic retinopathy grade estimation on eye fundus images. The experimental results show that the proposed method not only improves the diagnosis performance on the two tasks but also the interpretability of the results by quantifying the uncertainty of the predictions in comparison to conventional deep classification and regression architectures. The code and datasets are available at https://github.com/stoledoc/DQOR.

Assessing radiomics feature stability with simulated CT acquisitions

Scientific paper ArODES

Kyriakos Flouris, Oscar Alfonso Jiménez del Toro, Christoph Aberle, Michael Bach, Roger Schaer, Markus M. Obmann, Bram Stieltjes, Henning Müller, Adrien Depeursinge, Ender Konukoglu

Scientific Reports, 2022, vol. 12, article 4732, pp. 1-11

Link to the publication

Summary:

Medical imaging quantitative features had once disputable usefulness in clinical studies. Nowadays, advancements in analysis techniques, for instance through machine learning, have enabled quantitative features to be progressively useful in diagnosis and research. Tissue characterisation is improved via the “radiomics” features, whose extraction can be automated. Despite the advances, stability of quantitative features remains an important open problem. As features can be highly sensitive to variations of acquisition details, it is not trivial to quantify stability and efficiently select stable features. In this work, we develop and validate a Computed Tomography (CT) simulator environment based on the publicly available ASTRA toolbox (www.astra-toolbox.com). We show that the variability, stability and discriminative power of the radiomics features extracted from the virtual phantom images generated by the simulator are similar to those observed in a tandem phantom study. Additionally, we show that the variability is matched between a multi-center phantom study and simulated results. Consequently, we demonstrate that the simulator can be utilised to assess radiomics features’ stability and discriminative power.

Improving robotic hand prosthesis control with eye tracking and computer vision :

Scientific paper ArODES

a multimodal approach based on the visuomotor behavior of grasping

Matteo Cognolato, Manfredo Atzori, Roger Gassert, Henning Müller

Frontiers in artificial intelligence, January 2022, vol. 4, article no. 744476, pp. 1-12

Link to the publication

Summary:

The complexity and dexterity of the human hand make the development of natural and robust control of hand prostheses challenging. Although a large number of control approaches were developed and investigated in the last decades, limited robustness in real-life conditions often prevented their application in clinical settings and in commercial products. In this paper, we investigate a multimodal approach that exploits the use of eye-hand coordination to improve the control of myoelectric hand prostheses. The analyzed data are from the publicly available MeganePro Dataset 1, that includes multimodal data from transradial amputees and able-bodied subjects while grasping numerous household objects with ten grasp types. A continuous grasp-type classification based on surface electromyography served as both intent detector and classifier. At the same time, the information provided by eye-hand coordination parameters, gaze data and object recognition in first-person videos allowed to identify the object a person aims to grasp. The results show that the inclusion of visual information significantly increases the average offline classification accuracy by up to 15.61 ± 4.22% for the transradial amputees and of up to 7.37 ± 3.52% for the able-bodied subjects, allowing trans-radial amputees to reach average classification accuracy comparable to intact subjects and suggesting that the robustness of hand prosthesis control based on grasp-type recognition can be significantly improved with the inclusion of visual information extracted by leveraging natural eye-hand coordination behavior and without placing additional cognitive burden on the user.

2021

The discriminative power and stability of radiomics features with computed tomography variations :

Scientific paper ArODES

task-based analysis in an anthropomorphic 3D-printed CT phantom

Oscar Alfonso Jiménez del Toro, Christoph Aberle, Michael Bach, Roger Schaer, Markus Obmann, Kyriakos Flouris, Ender Konukoglu, Bram Stieltjes, Henning Müller, Adrien Depeursinge

Investigative radiology, 2021, vol. 56, no. 12, pp. 820-825

Link to the publication

Summary:

Objectives : The aims of this study were to determine the stability of radiomics features against computed tomography (CT) parameter variations and to study their discriminative power concerning tissue classification using a 3D-printed CT phantom based on real patient data. Materials and Methods : A radiopaque 3D phantom was developed using real patient data and a potassium iodide solution paper-printing technique. Normal liver tissue and 3 lesion types (benign cyst, hemangioma, and metastasis) were manually annotated in the phantom. The stability and discriminative power of 86 radiomics features were assessed in measurements taken from 240 CT series with 8 parameter variations of reconstruction algorithms, reconstruction kernels, slice thickness, and slice spacing. Pairwise parameter group and pairwise tissue class comparisons were performed using Wilcoxon signed rank tests. Results : In total, 19,264 feature stability tests and 8256 discriminative power tests were performed. The 8 CT parameter variation pairwise group comparisons had statistically significant differences on average in 78/86 radiomics features. On the other hand, 84% of the univariate radiomics feature tests had a successful and statistically significant differentiation of the 4 classes of liver tissue. The 86 radiomics features were ranked according to the cumulative sum of successful stability and discriminative power tests. Conclusions : The differences in radiomics feature values obtained from different types of liver tissue are generally greater than the intraclass differences resulting from CT parameter variations.

Questioning domain adaptation in myoelectric hand prostheses control :

Scientific paper ArODES

an inter- and intra-subject study

Giulio Marano, Cristina Brambilla, Robert Mihai Mira, Alessandro Scano, Henning Müller, Manfredo Atzori

Sensors, 2021, vol. 21, no. 22, article no. 7500, pp. 1-13

Link to the publication

Summary:

One major challenge limiting the use of dexterous robotic hand prostheses controlled via electromyography and pattern recognition relates to the important efforts required to train complex models from scratch. To overcome this problem, several studies in recent years proposed to use transfer learning, combining pre-trained models (obtained from prior subjects) with training sessions performed on a specific user. Although a few promising results were reported in the past, it was recently shown that the use of conventional transfer learning algorithms does not increase performance if proper hyperparameter optimization is performed on the standard approach that does not exploit transfer learning. The objective of this paper is to introduce novel analyses on this topic by using a random forest classifier without hyperparameter optimization and to extend them with experiments performed on data recorded from the same patient, but in different data acquisition sessions. Two domain adaptation techniques were tested on the random forest classifier, allowing us to conduct experiments on healthy subjects and amputees. Differently from several previous papers, our results show that there are no appreciable improvements in terms of accuracy, regardless of the transfer learning techniques tested. The lack of adaptive learning is also demonstrated for the first time in an intra-subject experimental setting when using as a source ten data acquisitions recorded from the same subject but on five different days.

An AI-based algorithm for the automatic classification of thoracic radiographs in cats

Scientific paper ArODES

Tommaso Banzato, Marek Wodzinski, Federico Tauceri, Chiara Donà, Filippo Scavazza, Henning Müller, Alessandro Zotti

Frontiers in veterinary science, 2021, vol. 8, article no. 731936, pp. 1-7

Link to the publication

Summary:

An artificial intelligence (AI)-based computer-aided detection (CAD) algorithm to detect some of the most common radiographic findings in the feline thorax was developed and tested. The database used for training comprised radiographs acquired at two different institutions. Only correctly exposed and positioned radiographs were included in the database used for training. The presence of several radiographic findings was recorded. Consequenly, the radiographic findings included for training were: no findings, bronchial pattern, pleural effusion,mass, alveolar pattern, pneumothorax, cardiomegaly.Multi-label convolutional neural networks (CNNs) were used to develop the CAD algorithm, and the performance of two different CNN architectures, ResNet 50 and Inception V3, was compared. Both architectures had an area under the receiver operating characteristic curve (AUC) above 0.9 for alveolar pattern, bronchial pattern and pleural effusion, an AUC above 0.8 for no findings and pneumothorax, and an AUC above 0.7 for cardiomegaly. The AUC for mass was low (above 0.5) for both architectures. No significant differences were evident in the diagnostic accuracy of either architecture.

A large margin piecewise linear classifier with fusion of deep features in the diagnosis of COVID-19

Scientific paper ArODES

Neda Azouji, Ashkan Sami, Mohammad Taheri, Henning Müller

Computers in biology and medicine, 2021, vol. 139, article no. 104927, pp. 1-14

Link to the publication

Summary:

The world has experienced epidemics of coronavirus infections several times over the last two decades. Recent studies have shown that using medical imaging techniques can be useful in developing an automatic computer-aided diagnosis system to detect pandemic diseases with high accuracy at an early stage. In this study, a large margin piecewise linear classifier was developed to diagnose COVID-19 compared to a wide range of viral pneumonia, including SARS and MERS, using chest x-ray images. In the proposed method, a preprocessing pipeline was employed. Moreover, deep pre- and post-rectified linear unit (ReLU) features were extracted using the well-known VGG-Net19, which was fine-tuned to optimize transfer learning. Afterward, the canonical correlation analysis was performed for feature fusion, and fused deep features were passed into the LMPL classifier. The introduced method reached the highest performance in comparison with related state-of-the-art methods for two different schemes (normal, COVID-19, and typical viral pneumonia) and (COVID-19, SARS, and MERS pneumonia) with 99.39% and 98.86% classification accuracy, respectively.

Making radiomics more reproducible across scanner and imaging protocol variations :

Scientific paper ArODES

a review of harmonization methods

Shruti Atul Mali, Abdalla Ibrahim, Henry C. Woodruff, Vincent Andrearczyk, Henning Müller, Sergey Primakov, Zohaib Salahuddin, Avishek Chatterjee, Philippe Lambin

Journal of personalized medicine, 2021, vol. 11, no. 9, article no. 842, pp. 1-30

Link to the publication

Summary:

Radiomics converts medical images into mineable data via a high-throughput extraction of quantitative features used for clinical decision support. However, these radiomic features are susceptible to variation across scanners, acquisition protocols, and reconstruction settings. Various investigations have assessed the reproducibility and validation of radiomic features across these discrepancies. In this narrative review, we combine systematic keyword searches with prior domain knowledge to discuss various harmonization solutions to make the radiomic features more reproducible across various scanners and protocol settings. Different harmonization solutions are discussed and divided into two main categories: image domain and feature domain. The image domain category comprises methods such as the standardization of image acquisition, post-processing of raw sensor-level image data, data augmentation techniques, and style transfer. The feature domain category consists of methods such as the identification of reproducible features and normalization techniques such as statistical normalization, intensity harmonization, ComBat and its derivatives, and normalization using deep learning. We also reflect upon the importance of deep learning solutions for addressing variability across multi-centric radiomic studies especially using generative adversarial networks (GANs), neural style transfer (NST) techniques, or a combination of both. We cover a broader range of methods especially GANs and NST methods in more detail than previous reviews.

Multi_Scale_Tools :

Scientific paper ArODES

a Python library to exploit multi-scale whole slide images

Niccolò Marini, Sebastian Otálora, Damian Podareanu, Mart van Rijthoven, Jeroen van der Laak, Francesco Ciompi, Henning Müller, Manfredo Atzori

Frontiers in computer science, 2021, vol. 3, article no. 684521, pp. 1-12

Link to the publication

Summary:

Algorithms proposed in computational pathology can allow to automatically analyze digitized tissue samples of histopathological images to help diagnosing diseases. Tissue samples are scanned at a high-resolution and usually saved as images with several magnification levels, namely whole slide images (WSIs). Convolutional neural networks (CNNs) represent the state-of-the-art computer vision methods targeting the analysis of histopathology images, aiming for detection, classification and segmentation. However, the development of CNNs that work with multi-scale images such as WSIs is still an open challenge. The image characteristics and the CNN properties impose architecture designs that are not trivial. Therefore, single scale CNN architectures are still often used. This paper presents Multi_Scale_Tools, a library aiming to facilitate exploiting the multi-scale structure of WSIs. Multi_Scale_Tools currently include four components: a pre-processing component, a scale detector, a multi-scale CNN for classification and a multi-scale CNN for segmentation of the images. The pre-processing component includes methods to extract patches at several magnification levels. The scale detector allows to identify the magnification level of images that do not contain this information, such as images from the scientific literature. The multi-scale CNNs are trained combining features and predictions that originate from different magnification levels. The components are developed using private datasets, including colon and breast cancer tissue samples. They are tested on private and public external data sources, such as The Cancer Genome Atlas (TCGA). The results of the library demonstrate its effectiveness and applicability. The scale detector accurately predicts multiple levels of image magnification and generalizes well to independent external data. The multi-scale CNNs outperform the single-magnification CNN for both classification and segmentation tasks. The code is developed in Python and it will be made publicly available upon publication. It aims to be easy to use and easy to be improved with additional functions.

Semi-supervised training of deep convolutional neural networks with heterogeneous data and few local annotations :

Scientific paper ArODES

an experiment on prostate histopathology image classification

Niccolò Marini, Sebastian Otálora, Henning Müller, Manfredo Atzori

Medical image analysis, 2021, vol. 73, article no. 102165, pp. 1-16

Link to the publication

Summary:

Convolutional neural networks (CNNs) are state-of-the-art computer vision techniques for various tasks, particularly for image classification. However, there are domains where the training of classification models that generalize on several datasets is still an open challenge because of the highly heterogeneous data and the lack of large datasets with local annotations of the regions of interest, such as histopathology image analysis. Histopathology concerns the microscopic analysis of tissue specimens processed in glass slides to identify diseases such as cancer. Digital pathology concerns the acquisition, management and automatic analysis of digitized histopathology images that are large, having in the order of 100′0002 pixels per image. Digital histopathology images are highly heterogeneous due to the variability of the image acquisition procedures. Creating locally labeled regions (required for the training) is time-consuming and often expensive in the medical field, as physicians usually have to annotate the data. Despite the advances in deep learning, leveraging strongly and weakly annotated datasets to train classification models is still an unsolved problem, mainly when data are very heterogeneous. Large amounts of data are needed to create models that generalize well. This paper presents a novel approach to train CNNs that generalize to heterogeneous datasets originating from various sources and without local annotations. The data analysis pipeline targets Gleason grading on prostate images and includes two models in sequence, following a teacher/student training paradigm. The teacher model (a high-capacity neural network) automatically annotates a set of pseudo-labeled patches used to train the student model (a smaller network). The two models are trained with two different teacher/student approaches: semi-supervised learning and semi-weekly supervised learning. For each of the two approaches, three student training variants are presented. The baseline is provided by training the student model only with the strongly annotated data. Classification performance is evaluated on the student model at the patch level (using the local annotations of the Tissue Micro-Arrays Zurich dataset) and at the global level (using the TCGA-PRAD, The Cancer Genome Atlas-PRostate ADenocarcinoma, whole slide image Gleason score). The teacher/student paradigm allows the models to better generalize on both datasets, despite the inter-dataset heterogeneity and the small number of local annotations used. The classification performance is improved both at the patch-level (up to κ = 0.6127 ± 0.0133 from κ = 0.5667 ± 0.0285), at the TMA core-level (Gleason score) (up to κ = 0.7645 ± 0.0231 from κ = 0.7186 ± 0.0306) and at the WSI-level (Gleason score) (up to κ = 0.4529 ± 0.0512 from κ = 0.2293 ± 0.1350). The results show that with the teacher/student paradigm, it is possible to train models that generalize on datasets from entirely different sources, despite the inter-dataset heterogeneity and the lack of large datasets with local annotations.

On the scale invariance in state of the art CNNs trained on ImageNet

Scientific paper ArODES

Mara Graziani, Thomas Lompech, Henning Müller, Adrien Depeursinge, Vincent Andrearczyk

Machine learning and knowledge extraction, 2021, vol. 3, no. 2, pp. 374-391

Link to the publication

Summary:

The diffused practice of pre-training Convolutional Neural Networks (CNNs) on large natural image datasets such as ImageNet causes the automatic learning of invariance to object scale variations. This, however, can be detrimental in medical imaging, where pixel spacing has a known physical correspondence and size is crucial to the diagnosis, for example, the size of lesions, tumors or cell nuclei. In this paper, we use deep learning interpretability to identify at what intermediate layers such invariance is learned. We train and evaluate different regression models on the PASCAL-VOC (Pattern Analysis, Statistical modeling and ComputAtional Learning-Visual Object Classes) annotated data to (i) separate the effects of the closely related yet different notions of image size and object scale, (ii) quantify the presence of scale information in the CNN in terms of the layer-wise correlation between input scale and feature maps in InceptionV3 and ResNet50, and (iii) develop a pruning strategy that reduces the invariance to object scale of the learned features. Results indicate that scale information peaks at central CNN layers and drops close to the softmax, where the invariance is reached. Our pruning strategy uses this to obtain features that preserve scale information. We show that the pruning significantly improves the performance on medical tasks where scale is a relevant factor, for example for the regression of breast histology image magnification. These results show that the presence of scale information at intermediate layers legitimates transfer learning in applications that require scale covariance rather than invariance and that the performance on these tasks can be improved by pruning off the layers where the invariance is learned. All experiments are performed on publicly available data and the code is available on GitHub.

Combining weakly and strongly supervised learning improves strong supervision in Gleason pattern classification

Scientific paper ArODES

Sebastian Otálora, Niccolò Marini, Henning Müller, Manfredo Atzori

BMC Medical Imaging, 2021, vol. 21, article no. 77, pp. 1-14

Link to the publication

Summary:

One challenge to train deep convolutional neural network (CNNs) models with whole slide images (WSIs) is providing the required large number of costly, manually annotated image regions. Strategies to alleviate the scarcity of annotated data include: using transfer learning, data augmentation and training the models with less expensive image-level annotations (weakly-supervised learning). However, it is not clear how to combine the use of transfer learning in a CNN model when different data sources are available for training or how to leverage from the combination of large amounts of weakly annotated images with a set of local region annotations. This paper aims to evaluate CNN training strategies based on transfer learning to leverage the combination of weak and strong annotations in heterogeneous data sources. The trade-off between classification performance and annotation effort is explored by evaluating a CNN that learns from strong labels (region annotations) and is later fine-tuned on a dataset with less expensive weak (image-level) labels.

Process data infrastructure and data services

Scientific paper ArODES

Reginald Cushing, Onno Valkering, Adam Belloum, Souley Madougou, Martin Bobak, Ondrej Habala, Viet Tran, Jan Meizner, Piotr Nowakowski, Mara Graziani, Henning Müller

Computers and informatics, 2020, vol. 39, no. 4, pp. 724-756

Link to the publication

Summary:

Due to energy limitation and high operational costs, it is likely that exascale computing will not be achieved by one or two datacentres but will require many more. A simple calculation, which aggregates the computation power of the 2017 Top500 supercomputers, can only reach 418 petaflops. Companies like Rescale, which claims 1.4 exaflops of peak computing power, describes its infrastructure as composed of 8 million servers spread across 30 datacentres. Any proposed solution to address exascale computing challenges has to take into consideration these facts and by design should aim to support the use of geographically distributed and likely independent datacentres. It should also consider, whenever possible, the co-allocation of the storage with the computation as it would take 3 years to transfer 1 exabyte on a dedicated 100 Gb Ethernet connection. This means we have to be smart about managing data more and more geographically dispersed and spread across different administrative domains. As the natural settings of the PROCESS project is to operate within the European Research Infrastructure and serve the European research communities facing exascale challenges, it is important that PROCESS architecture and solutions are well positioned within the European computing and data management landscape namely PRACE, EGI, and EUDAT. In this paper we propose a scalable and programmable data infrastructure that is easy to deploy and can be tuned to support various data-intensive scientific applications.

Benchmarking image retrieval diversification techniques for social media

Scientific paper ArODES

Bogdan Ionescu, Maia Rohm, Bogdan Boteanu, Alexandru Lucian Gînsca, Mihai Lupu, Henning Müller

IEEE Transactions on multimedia, 2021, vol. 23, pp. 677-691

Link to the publication

Summary:

Image retrieval has been an active research domain for over 30 years and historically it has focused primarily on precision as an evaluation criterion. Similar to text retrieval, where the number of indexed documents became large and many relevant documents exist, it is of high importance to highlight diversity in the search results to provide better results for the user. The Retrieving Diverse Social Images Task of the MediaEval benchmarking campaign has addressed exactly this challenge of retrieving diverse and relevant results for the past years, specifically in the social media context. Multimodal data (e.g., images, text) was made available to the participants including metadata assigned to the images, user IDs, and precomputed visual and text descriptors. Many teams have participated in the task over the years. The large number of publications employing the data and also citations of the overview articles underline the importance of this topic. In this paper, we introduce these publicly available data resources as well as the evaluation framework, and provide an in-depth analysis of the crucial aspects of social image search diversification, such as the capabilities and the evolution of existing systems. These evaluation resources will help researchers for the coming years in analyzing aspects of multimodal image retrieval and diversity of the search results.

Breast histopathology with high-performance computing and deep learning

Scientific paper ArODES

Mara Graziani, Ivan Eggel, François Deligand, Martin Bobák, Vincent Andrearczyk, Henning Müller

Computing and informatics, 2020, vol. 39, no. 4, pp. 780-807

Link to the publication

Summary:

The increasingly intensive collection of digitalized images of tumor tissue over the last decade made histopathology a demanding application in terms of computational and storage resources. With images containing billions of pixels, the need for optimizing and adapting histopathology to large-scale data analysis is compelling. This paper presents a modular pipeline with three independent layers for the detection of tumoros regions in digital specimens of breast lymph nodes with deep learning models. Our pipeline can be deployed either on local machines or high-performance computing resources with a containerized approach. The need for expertise in high-performance computing is removed by the self-su_cient structure of Docker containers, whereas a large possibility for customization is left in terms of deep learning models and hyperparameters optimization. We show that by deploying the software layers in di_erent infrastructures we optimize both the data preprocessing and the network training times, further increasing the scalability of the application to datasets of approximatively 43 million images. The code is open source and available on Github.

DeepHistReg :

Scientific paper ArODES

unsupervised deep learning registration framework for differently stained histology samples

Marek Wodzinski, Henning Müller

Computer methods and programs in biomedicine, 2021, vol. 198, article no. 105799, pp. 1-11

Link to the publication

Summary:

Background and objective : The use of several stains during histology sample preparation can be useful for fusing complementary information about different tissue structures. It reveals distinct tissue properties that combined may be useful for grading, classification, or 3-D reconstruction. Nevertheless, since the slide preparation is different for each stain and the procedure uses consecutive slices, the tissue undergoes complex and possibly large deformations. Therefore, a nonrigid registration is required before further processing. The nonrigid registration of differently stained histology images is a challenging task because: (i) the registration must be fully automatic, (ii) the histology images are extremely high-resolution, (iii) the registration should be as fast as possible, (iv) there are significant differences in the tissue appearance, and (v) there are not many unique features due to a repetitive texture. Methods :In this article, we propose a deep learning-based solution to the histology registration. We describe a registration framework dedicated to high-resolution histology images that can perform the registration in real-time. The framework consists of an automatic background segmentation, iterative initial rotation search and learning-based affine/nonrigid registration. Results : We evaluate our approach using an open dataset provided for the Automatic Non-rigid Histological Image Registration (ANHIR) challenge organized jointly with the IEEE ISBI 2019 conference. We compare our solution to the challenge participants using a server-side evaluation tool provided by the challenge organizers. Following the challenge evaluation criteria, we use the target registration error (TRE) as the evaluation metric. Our algorithm provides registration accuracy close to the best scoring teams (median rTRE 0.19% of the image diagonal) while being significantly faster (the average registration time is about 2 seconds). Conclusions : The proposed framework provides results, in terms of the TRE, comparable to the best-performing state-of-the-art methods. However, it is significantly faster, thus potentially more useful in clinical practice where a large number of histology images are being processed. The proposed method is of particular interest to researchers requiring an accurate, real-time, nonrigid registration of high-resolution histology images for whom the processing time of traditional, iterative methods in unacceptable. We provide free access to the software implementation of the method, including training and inference code, as well as pretrained models. Since the ANHIR dataset is open, this makes the results fully and easily reproducible.

Reference exascale architecture (extended version)

Scientific paper ArODES

Martin Bobák, Ladislav Hluchý, Ondrej Habala, Viet Tran, Reginald Cushing, Onno Valkering, Adam Belloum, Mara Graziani, Henning Müller, Souley Madougou, Jason Maassen

Computing and informatics, 2020, vol. 34, no. 4, pp. 644-677

Link to the publication

Summary:

While political commitments for building exascale systems have been made, turning these systems into platforms for a wide range of exascale applications faces several technical, organisational and skills-related challenges. The key technical challenges are related to the availability of data. While the first exascale machines are likely to be built within a single site, the input data is in many cases impossible to store within a single site. Alongside handling of extreme-large amount of data, the exascale system has to process data from different sources, support accelerated computing, handle high volume of requests per day, minimize the size of data flows, and be extensible in terms of continuously increasing data as well as an increase in parallel requests being sent. These technical challenges are addressed by the general reference exascale architecture. It is divided into three main blocks: virtualization layer, distributed virtual file system, and manager of computing resources. Its main property is modularity which is achieved by containerization at two levels: 1) application containers - containerization of scientific workflows, 2) micro-infrastructure - containerization of extreme-large data service-oriented infrastructure. The paper also presents an instantiation of the reference architecture - the architecture of the PROCESS project (PROviding Computing solutions for ExaScale ChallengeS) and discusses its relation to the reference exascale architecture. The PROCESS architecture has been used as an exascale platform within various exascale pilot applications. This paper also presents performance modelling of exascale platform with its validation.

2020

BIAS :

Scientific paper ArODES

transparent reporting of biomedical image analysis challenges

Lena Maier-Hein, Annika Reinke, Michal Kozubek, Anne L. Martel, Tal Arbel, Matthias Eisenmann, Allan Hanbury, Pierre Jannin, Henning Müller, Sinan Onogur, Julio Saez-Rodriguez, Bram Van Ginneken, Annette Kopp-Schneider, Bennett A. Landman

Medical image analysis, 2020, vol. 66, article 101796, pp. 1-7

Link to the publication

Summary:

The number of biomedical image analysis challenges organized per year is steadily increasing. These international competitions have the purpose of benchmarking algorithms on common data sets, typically to identify the best method for a given problem. Recent research, however, revealed that common practice related to challenge reporting does not allow for adequate interpretation and reproducibility of results. To address the discrepancy between the impact of challenges and the quality (control), the Biomedical Image Analysis ChallengeS (BIAS) initiative developed a set of recommendations for the reporting of challenges. The BIAS statement aims to improve the transparency of the reporting of a biomedical image analysis challenge regardless of field of application, image modality or task category assessed. This article describes how the BIAS statement was developed and presents a checklist which authors of biomedical image analysis challenges are encouraged to include in their submission when giving a paper on a challenge into review. The purpose of the checklist is to standardize and facilitate the review process and raise interpretability and reproducibility of challenge results by making relevant information explicit.

The importance of feature aggregation in radiomics :

Scientific paper ArODES

a head and neck cancer study

Pierre Fontaine, Oscar Acosta, Joël Castelli, Renaud De Crevoisier, Henning Müller, Adrien Depeursinge

Scientific reports, 2020, vol. 10, article no. 19679 , pp. 1-11

Link to the publication

Summary:

In standard radiomics studies the features extracted from clinical images are mostly quantified with simple statistics such as the average or variance per Region of Interest (ROI). Such approaches may smooth out any intra-region heterogeneity and thus hide some tumor aggressiveness that may hamper predictions. In this paper we study the importance of feature aggregation within the standard radiomics workflow, which allows to take into account intra-region variations. Feature aggregation methods transform a collection of voxel values from feature response maps (over a ROI) into one or several scalar values that are usable for statistical or machine learning algorithms. This important step has been little investigated within the radiomics workflows, so far. In this paper, we compare several aggregation methods with standard radiomics approaches in order to assess the improvements in prediction capabilities. We evaluate the performance using an aggregation function based on Bags of Visual Words (BoVW), which allows for the preservation of piece-wise homogeneous information within heterogeneous regions and compared with standard methods. The different models are compared on a cohort of 214 head and neck cancer patients coming from 4 medical centers. Radiomics features were extracted from manually delineated tumors in clinical PET-FDG and CT images were analyzed. We compared the performance of standard radiomics models, the volume of the ROI alone and the BoVW model for survival analysis. The average concordance index was estimated with a five fold cross-validation. The performance was significantly better using the BoVW model 0.627 (95% CI: 0.616–0.637) as compared to standard radiomics0.505 (95% CI: 0.499–0.511), mean-var. 0.543 (95% CI: 0.536–0.549), mean0.547 (95% CI: 0.541–0.554), var.0.530 (95% CI: 0.524–0.536) or volume 0.577 (95% CI: 0.571–0.582). We conclude that classical aggregation methods are not optimal in case of heterogeneous tumors. We also showed that the BoVW model is a better alternative to extract consistent features in the presence of lesions composed of heterogeneous tissue.

A lung graph model for the radiological assessment of chronic thromboembolic pulmonary hypertension in CT

Scientific paper ArODES

Oscar Alfonso Jiménez del Toro, Yashin Dicente Cid, Alexandra Platon, Anne-Lise Hachulla, Frédéric Lador, Pierre-Alexandre Poletti, Henning Müller

Computers in biology and medicine, 2020, vol. 125, article 103962, pp. 1-8

Link to the publication

Summary:

Chronic thromboembolic pulmonary hypertension (CTEPH) is a possible complication of pulmonary embolism (PE), with poor prognosis if left untreated. Surgical curative treatment is available, particularly in the early stages of the disease. However, most cases are not diagnosed until specific symptoms become evident. A small number of computed tomography (CT) findings, such as a widened pulmonary artery and mosaicism in the lung parenchyma, have been correlated with pulmonary hypertension (PH). Quantitative texture analysis in the CT scans of these patients could provide complementary sub-visual information of the vascular changes taking place in the lungs. For this task, a lung graph model was developed with texture descriptors from 37 CT scans with confirmed CTEPH diagnosis and 48 CT scans from PE patients who did not develop PH. The probability of presenting CTEPH, computed with the graph model, outperformed a convolutional neural network approach using 10 different train/test splits of the data set. An accuracy of 0.76 was obtained with the proposed texture analysis, and was then compared to the visual assessment of CT findings, manually identified by a team of three expert radiologists, commonly associated with pulmonary hypertension. This graph-based score combined with the information attained from the radiological findings resulted in a Cohen’s kappa coefficient of 0.47 when differentiating patients with confirmed CTEPH from those with PE who did not develop the disease. The proposed texture quantification could be an objective measurement, complementary to the current analysis of radiologists for the early detection of CTEPH and thus improve patient outcome.

Concept attribution :

Scientific paper ArODES

explaining CNN decisions to physicians

Mara Graziani, Vincent Andrearczyk, Stéphane Marchand-Maillet, Henning Müller

Computers in biology and medicine, 2020, vol. 123, Article 103865, pp. 1-11

Link to the publication

Summary:

Deep learning explainability is often reached by gradient-based approaches that attribute the network output to perturbations of the input pixels. However, the relevance of input pixels may be difficult to relate to relevant image features in some applications, e.g. diagnostic measures in medical imaging. The framework described in this paper shifts the attribution focus from pixel values to user-defined concepts. By checking if certain diagnostic measures are present in the learned representations, experts can explain and entrust the network output. Being post-hoc, our method does not alter the network training and can be easily plugged into the latest state-of-the-art convolutional networks. This paper presents the main components of the framework for attribution to concepts, in addition to the introduction of a spatial pooling operation on top of the feature maps to obtain a solid interpretability analysis. Furthermore, regularized regression is analyzed as a solution to the regression overfitting in high-dimensionality latent spaces. The versatility of the proposed approach is shown by experiments on two medical applications, namely histopathology and retinopathy, and on one non-medical task, the task of handwritten digit classification. The obtained explanations are in line with clinicians’ guidelines and complementary to widely used visualization tools such as saliency maps.

Automating quality control for structured standardized radiology reports using text analysis

Book chapter ArODES

Anjani Dhrangadhariya, Sandy Millius, Cyril Thouly, Benoit Rizk, Dominique Fournier, Henning Müller, Hugues Brat

Dans Lovis, Christian, Madsen, Inge Cort, Nielsen, Per Hostrup, Pape-Haugaard, Louise B., Scott, Philip, Weber, Patrick, Digital personalized health and medicine (Pp. 58-62). 2020, Amsterdam : IOS press

Link to the publication

Summary:

Radiology reports describe the findings of a radiologist in an imaging examination, produced for another clinician in order to answer to a clinical indication. Sometimes, the report does not fully answer the question asked, despite guidelines for the radiologist. In this article, a system that controls the quality of reports automatically is described. It notably maps the free text onto MeSH terms and checks if the anatomy and disease terms match in the indication and conclusion of a report. The agreement between manual checks of experienced radiologists and the system is high with automatic checks requiring only a fraction of time. Being able to quality control all reports has the potential to improve report quality and thus limit misunderstandings, loosing time for requesting more information and possibly avoid medical mistakes.

Variability of muscle synergies in hand grasps :

Scientific paper ArODES

analysis of intra- and inter-session data

Una Pale, Manfredo Atzori, Henning Müller, Alessandro Scano

Sensors, 2020, vol. 20, no 15, article 4297

Link to the publication

Summary:

Background. Muscle synergy analysis is an approach to understand the neurophysiological mechanisms behind the hypothesized ability of the Central Nervous System (CNS) to reduce the dimensionality of muscle control. The muscle synergy approach is also used to evaluate motor recovery and the evolution of the patients’ motor performance both in single-session and longitudinal studies. Synergy-based assessments are subject to various sources of variability: natural trial-by-trial variability of performed movements, intrinsic characteristics of subjects that change over time (e.g., recovery, adaptation, exercise, etc.), as well as experimental factors such as di_erent electrode positioning. These sources of variability need to be quantified in order to resolve challenges for the application of muscle synergies in clinical environments. The objective of this study is to analyze the stability and similarity of extracted muscle synergies under the e_ect of factors that may induce variability, including inter- and intra-session variability within subjects and inter-subject variability di_erentiation. The analysis was performed using the comprehensive, publicly available hand grasp NinaPro Database, featuring surface electromyography (EMG) measures from two EMG electrode bracelets. Methods. Intra-session, inter-session, and inter-subject synergy stability was analyzed using the following measures: variance accounted for (VAF) and number of synergies (NoS) as measures of reconstruction stability quality and cosine similarity for comparison of spatial composition of extracted synergies. Moreover, an approach based on virtual electrode repositioning was applied to shed light on the influence of electrode position on inter-session synergy similarity. Results. Inter-session synergy similarity was significantly lower with respect to intra-session similarity, both considering coe_cient of variation of VAF (approximately 0.2–15% for inter vs. approximately 0.1% to 2.5% for intra, depending on NoS) and coe_cient of variation of NoS (approximately 6.5–14.5% for inter vs. approximately 3–3.5% for intra, depending on VAF) as well as synergy similarity (approximately 74–77% for inter vs. approximately 88–94% for intra, depending on the selected VAF). Virtual electrode repositioning revealed that a slightly di_erent electrode position can lower similarity of synergies from the same session and can increase similarity between sessions. Finally, the similarity of inter-subject synergies has no significant di_erence from the similarity of inter-session synergies (both on average approximately 84–90% depending on selected VAF). Conclusion. Synergy similarity was lower in inter-session conditions with respect to intra-session. This finding should be considered when interpreting results from multi-session assessments. Lastly, electrode positioning might play an important role in the lower similarity of synergies over di_erent sessions.

Prospects and challenges of radiomics by using nononcologic routine chest CT

Scientific paper ArODES

Sebastian Röhrich, Johannes Hofmanninger, Florian Prayer, Henning Müller, Helmut Prosch, Georg Langs

Radiology : cardiothoracic imaging, 2020, vol. 2, no. 4, pp. 1-11

Link to the publication

Summary:

Chest CT scans are one of the most common medical imaging procedures. The automatic extraction and quantification of imaging features may help in diagnosis, prognosis of, or treatment decision in cardiovascular, pulmonary, and metabolic diseases. However, an adequate sample size as a statistical necessity for radiomics studies is often difficult to achieve in prospective trials. By exploiting imaging data from clinical routine, a much larger amount of data could be used than in clinical trials. Still, there is only little literature on the implementation of radiomics in clinical routine chest CT scans. Reasons are heterogeneous CT scanning protocols and the resulting technical variability (eg, different slice thicknesses, reconstruction kernels or timings after contrast material administration) in routine CT imaging data. This review summarizes the recent state of the art of studies aiming to develop quantifiable imaging biomarkers at chest CT, such as for osteoporosis, chronic obstructive pulmonary disease, interstitial lung disease, and coronary artery disease. This review explains solutions to overcome heterogeneity in routine data such as the use of imaging repositories, the standardization of radiomic features, algorithmic approaches to improve feature stability, test-retest studies, and the evolution of deep learning for modeling radiomics features.

Machine learning assisted citation screening for systematic reviews

Book chapter ArODES

Anjani Dhrangadhariya, Roger Hilfiker, Roger Schaer, Henning Müller

Dans Pape-Haugaard, Louise B., Digital personalized health and medicine (5 p.). 2020, Amsterdam, The Netherlands : OIS Press

Link to the publication

Summary:

Evidence-based practice is highly dependent upon up-to-date systematic reviews (SR) for decision making. However, conducting and updating systematic reviews, especially the citation screening for identification of relevant studies, requires much human work and is therefore expensive. Automating citation screening using machine learning (ML) based approaches can reduce cost and labor. Machine learning has been applied to automate citation screening but not for the SRs with very narrow research questions. This paper reports the results and observations for an ongoing research that aims to automate citation screening for SRs with narrow research questions using machine learning. The research also sheds light on the problem of class imbalance and class overlap on the performance of ML classifiers when applied to SRs with narrow research questions.

The image biomarker standardization initiative :

Scientific paper ArODES

standardized quantitative radiomics for high-throughput image-based phenotyping

Alex Zwanenburg, Martin Vallières, Mahmoud A. Abdalah, Hugo J. W. L. Aerts, Vincent Andrearczyk, Adrien Depeursinge, Henning Müller

Radiology, 2020, vol. 295, no. 2, pp. 328-338

Link to the publication

Summary:

The Image Biomarker Standardization Initiative validated consensus-based reference values for 169 radiomics features, thus enabling calibration and verification of radiomics software. Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use. Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features. Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI). A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software.

Gaze, behavioral, and clinical data for phantom limbs after hand amputation from 15 amputees and 29 controls

Scientific paper ArODES

Gianluca Saetta, Matteo Cognolato, Manfredo Atzori, Diego Faccio, Katia Giacomino, Anne-Gabrielle Mittaz Hager, Cesare Tiengo, Franco Bassetto, Henning Müller, Peter Bruger

Scientific Data, 2020, vol. 7, article 60, pp. 1-14

Link to the publication

Summary:

Despite recent advances in prosthetics, many upper limb amputees still use prostheses with some reluctance. They often do not feel able to incorporate the artificial hand into their bodily self. Furthermore, prosthesis fitting is not usually tailored to accommodate the characteristics of an individual’s phantom limb sensations. These are experienced by almost all persons with an acquired amputation and comprise the motor and postural properties of the lost limb. This article presents and validates a multimodal dataset including an extensive qualitative and quantitative assessment of phantom limb sensations in 15 transradial amputees, surface electromyography and accelerometry data of the forearm, and measurements of gaze behavior during exercises requiring pointing or repositioning of the forearm and the phantom hand. The data also include acquisitions from 29 able-bodied participants, matched for gender and age. Special emphasis was given to tracking the visuo-motor coupling between eye-hand/eye-phantom during these exercises.

A large calibrated database of hand movements and grasps kinematics

Scientific paper ArODES

Néstor J. Jarque-Bou, Manfredo Atzori, Henning Müller

Scientific data, 2020, vol. 7, article 12, pp. 1-10

Link to the publication

Summary:

Modelling hand kinematics is a challenging problem, crucial for several domains including robotics, 3D modelling, rehabilitation medicine and neuroscience. Currently available datasets are few and limited in the number of subjects and movements. The objective of this work is to advance the modelling of hand kinematics by releasing and validating a large publicly available kinematic dataset of hand movements and grasp kinematics. The dataset is based on the harmonization and calibration of the kinematics data of three multimodal datasets previously released (Ninapro DB1, DB2 and DB5, that include electromyography, inertial and dynamic data). The novelty of the dataset is related to the high number of subjects (77) and movements (40 movements, each repeated several times) for which we release for the first time calibrated kinematic data, resulting in the largest available kinematic dataset. Differently from the previous datasets, the data are also calibrated to avoid sensor nonlinearities. The validation confirms that the data are not affected by experimental procedures and that they are similar to data acquired in real-life conditions.

Information fusion for medical data :

Scientific paper ArODES

early, late, and deep fusion methods for multimodal data

Inés Domingues, Henning Müller, Andres Ortiz, Belur V. Dasarathy, Pedro H. Abreu, Vince D. Calhoun

IEEE Journal of biomedical and health informatics, 2020, vol. 24, no. 1, pp. 14-16

Link to the publication

Summary:

The papers in this special section examine important current topics on multimodal data fusion in the medical context. All clinical data, including genomic and proteomic, play a role in the diagnosis and in particular in the treatment planning and follow-up. This is true for all types of data analyses whether in classification, regression, retrieval, clustering, or other. The interaction between several types of information is not always well understood. Experienced clinicians automatically and even unconsciously add multiple sources of information into their decision process, but machine learning tools often concentrate on single information sources. This special issue presents five examples where several data sources are fused. The papers give several examples of fusion techniques and also the results obtained in quite different application scenarios.

2019

Classification of diabetes-related retinal diseases using a deep learning approach in optical coherence tomography

Scientific paper ArODES

Oscar Perdomo, Hernán Rios, Francisco J. Rodríguez, Sebastián Otálora, Fabrice Meriaudeau, Henning Müller, Fabio A. González

Computer methods and programs in biomedicine, September 2019, vol. 178, pp. 181-189

Link to the publication

Summary:

Background and objectives: Spectral Domain Optical Coherence Tomography (SD-OCT) is a volumetric imaging technique that allows measuring patterns between layers such as small amounts of fluid. Since 2012, automatic medical image analysis performance has steadily increased through the use of deep learning models that automatically learn relevant features for specific tasks, instead of designing visual features manually. Nevertheless, providing insights and interpretation of the predictions made by the model is still a challenge. This paper describes a deep learning model able to detect medically interpretable information in relevant images from a volume to classify diabetes-related retinal diseases. Methods: This article presents a new deep learning model, OCT-NET, which is a customized convolutional neural network for processing scans extracted from optical coherence tomography volumes. OCT-NET is applied to the classification of three conditions seen in SD-OCT volumes. Additionally, the proposed model includes a feedback stage that highlights the areas of the scans to support the interpretation of the results. This information is potentially useful for a medical specialist while assessing the prediction produced by the model. Results: The proposed model was tested on the public SERI-CUHK and A2A SD-OCT data sets containing healthy, diabetic retinopathy, diabetic macular edema and age-related macular degeneration. The experimental evaluation shows that the proposed method outperforms conventional convolutional deep learning models from the state of the art reported on the SERI+CUHK and A2A SD-OCT data sets with a precision of 93% and an area under the ROC curve (AUC) of 0.99 respectively. Conclusions: The proposed method is able to classify the three studied retinal diseases with high accuracy. One advantage of the method is its ability to produce interpretable clinical information in the form of highlighting the regions of the image that most contribute to the classifier decision.

PaWFE :

Scientific paper ArODES

fast signal feature extraction using parallel time windows

Manfredo Atzori, Henning Müller

Frontiers in Neurorobotics, 2019, vol. 13, article 74

Link to the publication

Summary:

Motivation: Hand amputations can dramatically affect the quality of life of a person. Researchers are developing surface electromyography and machine learning solutions to control dexterous and robotic prosthetic hands, however long computational times can slow down this process. Objective: This paper aims at creating a fast signal feature extraction algorithm that can extract widely used features and allow researchers to easily add new ones. Methods: PaWFE (Parallel Window Feature Extractor) extracts the signal features from several time windows in parallel. The MATLAB code is publicly available and supports several time domain and frequency features. The code was tested and benchmarked using 1,2,4,8,16,32, and 48 threads on a server with four Xeon E7- 4820 and 128 GB RAM using the ﬁrst 5 datasets of the Ninapro database, that are recorded with different acquisition setups. Results: The parallel time window analysis approach allows to reduce the computational time up to 20 times when using 32 cores, showing a very good scalability. Signal features can be extracted in few seconds from an entire data acquisition and in <100ms from a single time window, easily reducing of up to over 15 times the feature extraction procedure in comparison to traditional approaches. The code allows users to easily add new signal feature extraction scripts, that can be added to the code and on the Ninapro website upon request. Signiﬁcance: The code allows researchers in machine learning and biosignals data analysis to easily and quickly test modern machine learning approaches on big datasets and it can be used as a resource for real time data analysis too.

Experimental IR meets multilinguality, multimodality, and interaction : 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, proceedings

Book ArODES

Fabio Crestani, Henning Müller, et al.

2019, Cham : Springer, 434 p.

Link to the publication

Summary:

This book constitutes the refereed proceedings of the 10th International Conference of the CLEF Association, CLEF 2019, held in Lugano, Switzerland, in September 2019. The conference has a clear focus on experimental information retrieval with special attention to the challenges of multimodality, multilinguality, and interactive search ranging from unstructured to semi structures and structured data. The 7 full papers and 8 short papers presented in this volume were carefully reviewed and selected from 30 submissions. This year, many contributions tackle the social networks with the detection of stances or early identiﬁcation of depression signs on Twitter in a cross-lingual context. Further this volume presents 7 “best of the labs” papers which were reviewed as a full paper submission with the same review criteria. The labs represented scientific challenges based on new data sets and real world problems in multimodal and multilingual information access. In addition to this, 9 benchmarking labs reported results of their yearlong activities in overview talks and lab sessions.

Biodiversity information retrieval through large scale content-based identification: a long-term evaluation

Book chapter ArODES

Alexis Joly, Hervé Goëau, Hervé Glotin, Concetto Spampinato, Pierre Bonnet, Willem-Pier Vellinga, Jean-Christophe Lombardo, Robert Planqué, Simone Palazzo, Henning Müller

Dans Ferro, Nicolas, Peters, Carol, Information retrieval evaluation in a changing world : lessons learned from 20 years of CLEF (pp. 389-413). 2019, Cham : Springer

Link to the publication

Summary:

Identifying and naming living plants or animals is usually impossible for the general public and often a difficult task for professionals and naturalists. Bridging this gap is a key challenge towards enabling effective biodiversity information retrieval systems. This taxonomic gap was actually already identified as one of the main ecological challenges to be solved during the Rio de Janeiro United Nations “Earth Summit” in 1992. Since 2011, the LifeCLEF challenges conducted in the context of the CLEF evaluation forum have been boosting and evaluating the advances in this domain. Data collections with an unprecedented volume and diversity have been shared with the scientific community to allow repeatable and long-term experiments. This paper describes the methodology of the conducted evaluation campaigns as well as providing a synthesis of the main results and lessons learned along the years.

Experiences from the ImageCLEF medical retrieval and annotation tasks

Book chapter ArODES

Henning Müller, Jayashree Kalpathy-Cramer, Alba García Seco de Herrera

Dans Ferro Nicola, Peters, Carol, Information Retrieval Evaluation in a Changing World : lessons learned from 20 Years of CLEF (19 p.). 2019, Cham : Springer

Link to the publication

Summary:

The medical tasks in ImageCLEF have been run every year from 2004- 2018 and many different tasks and data sets have been used over these years. The created resources are being used by many researchers well beyond the actual evaluation campaigns and are allowing to compare the performance of many techniques on the same grounds and in a reproducible way. Many of the larger data sets are from the medical literature, as such images are easier to obtain and to share than clinical data, which was used in a few smaller ImageCLEF challenges that are specifically marked with the disease type and anatomic region. This chapter describes the main results of the various tasks over the years, including data, participants, types of tasks evaluated and also the lessons learned in organizing such tasks for the scientific community.

Fusing learned representations from Riesz Filters and Deep CNN for lung tissue classification

Scientific paper ArODES

Ranveer Joyseeree, Sebastian Otálora, Henning Müller, Adrien Depeursinge

Medical image analysis, August 2019, vol. 56, pp. 172-183

Link to the publication

Summary:

A novel method to detect and classify several classes of diseased and healthy lung tissue in CT (Computed Tomography), based on the fusion of Riesz and deep learning features, is presented. First, discriminative parametric lung tissue texture signatures are learned from Riesz representations using a one–versus–one approach. The signatures are generated for four diseased tissue types and a healthy tissue class, all of which frequently appear in the publicly available Interstitial Lung Diseases (ILD) dataset used in this article. Because the Riesz wavelets are steerable, they can easily be made invariant to local image rotations, a property that is desirable when analyzing lung tissue micro–architectures in CT images. Second, features from deep Convolutional Neural Networks (CNN) are computed by fine–tuning the Inception V3 architecture using an augmented version of the same ILD dataset. Because CNN features are both deep and non–parametric, they can accurately model virtually any pattern that is useful for tissue discrimination, and they are the de facto standard for many medical imaging tasks. However, invariance to local image rotations is not explicitly implemented and can only be approximated with rotation–based data augmentation. This motivates the fusion of Riesz and deep CNN features, as the two techniques are very complementary. The two learned representations are combined in a joint softmax model for final classification, where early and late feature fusion schemes are compared. The experimental results show that a late fusion of the independent probabilities leads to significant improvements in classification performance when compared to each of the separate feature representations and also compared to an ensemble of deep learning approaches.

ImageCLEF 2019 :

Book chapter ArODES

multimedia retrieval in medicine, lifelogging, security and nature

Bogdan Ionescu, Henning Müller, Renaud Péteri, Yashin Dicente Cid, et al.

Dans Crestani, Fabio, et al., Müller, Henning, Experimental IR Meets Multilinguality, Multimodality, and Interaction : 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings (pp. 358-386). 2019, Cham : Springer

Link to the publication

Summary:

This paper presents an overview of the ImageCLEF 2019 lab, organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2019. ImageCLEF is an ongoing evaluation initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2019, the 17th edition of ImageCLEF runs four main tasks: (i) a medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with new data, (ii) a lifelog task (videos, images and other sources) about daily activities understanding, retrieval and summarization, (iii) a new security task addressing the problems of automatically identifying forged content and retrieve hidden information, and (iv) a new coral task about segmenting and labeling collections of coral images for 3D modeling. The strong participation, with 235 research groups registering, and 63 submitting over 359 runs, shows an important interest in this benchmark campaign.

Visualizing and interpreting feature reuse of pretrained CNNs for histopathology

Book chapter ArODES

Mara Graziani, Vincent Andrearczyk, Henning Müller

Dans Courtney, Jane, Deegan, Catherine, Leamy, Paul, MVIP 2019: Irish Machine Vision and Image Processing Conference Proceedings (4 p.). 2019, Dublin, Ireland : Irish Pattern Recognition and Classification Society

Link to the publication

Summary:

Reusing the parameters of networks pretrained on large scale datasets of natural images, such as ImageNet, is a common technique in the medical imaging domain. The large variability of objects and classes is, however, drastically reduced in most medical applications where images are dominated by repetitive patterns with, at times, subtle differences between the classes. This paper takes the example of finetuning a pretrained convolutional network on a histopathology task. Because of the reduced visual variability in this application domain, the network mostly learns to detect textures and simple patterns. As a result, the complex structures that maximize the channel activations of deep layers in the pretrained network are not present after finetuning. The learned features seem to be used by the network to spot atypical nuclei in the images, as shown by class activation maps. Finally, texture measures appear discriminative after finetuning, as shown by accurate Regression Concept Vectors.

Overview of LifeCLEF 2019: identification of Amazonian plants, south and north American birds, and niche prediction

Book chapter ArODES

Alexis Joly, Henning Müller, et al.

Dans Crestani, Fabio, et al., Müller, Henning, Experimental IR meets multilinguality, multimodality, and interaction : 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings (pp. 387-401). 2019, Cham : Springer

Link to the publication

Summary:

Building accurate knowledge of the identity, the geographic distribution and the evolution of living species is essential for a sustainable development of humanity, as well as for biodiversity conservation. Unfortunately, such basic information is often only partially available for professional stakeholders, teachers, scientists and citizens, and often incomplete for ecosystems that possess the highest diversity. In this context, an ultimate ambition is to set up innovative information systems relying on the automated identication and understanding of living organisms as a means to engage massive crowds of observers and boost the production of biodiversity and agro-biodiversity data. The LifeCLEF 2019 initiative proposes three data-oriented challenges related to this vision, in the continuity of the previous editions but with several consistent novelties intended to push the boundaries of the state-of-the-art in several research directions. This paper describes the methodology of the conducted evaluations as well as the synthesis of the main results and lessons learned.

An augmented reality environment to provide visual feedback to amputees during sEMG Data Acquisitions

Book chapter ArODES

Francesca Palermo, Matteo Cognolato, Ivan Eggel, Manfredo Atzori, Henning Müller

Dans Althoefer, Kaspar, Konstantinova, Jelizaveta, Zhang, Ketao, Towards autonomous robotic systems : 20th annual conference, TAROS 2019, London, UK, July 3–5, 2019, Proceedings, Part II (12 p.). 2019, Cham : Springer

Link to the publication

Summary:

Myoelectric hand prostheses have the potential to improve the quality of life of hand amputees. Still, the rejection rate of functional prostheses in the adult population is high. One of the causes is the long time for fitting the prosthesis and the lack of feedback during training. Moreover, prosthesis control is often unnatural and requires mental effort during the training. Virtual and augmented reality devices can help to improve these difficulties and reduce phantom limb pain. Amputees can start training the residual limb muscles with a weightless virtual hand earlier than possible with a real prosthesis. When activating the muscles related to a specific grasp, the subjects receive a visual feedback from the virtual hand. To the best of our knowledge, this work presents one of the first portable augmented reality environment for transradial amputees that combines two devices available on the market: the Microsoft HoloLens and the Thalmic labs Myo. In the augmented environment, rendered by the HoloLens, the user can control a virtual hand with surface electromyography. By using the virtual hand, the user can move objects in augmented reality and train to activate the right muscles for each movement through visual feedback. The environment presented represents a resource for rehabilitation and for scientists. It helps hand amputees to train using prosthetic hands right after the surgery. Scientists can use the environment to develop real time control experiments, without the logistical disadvantages related to dealing with a real prosthetic hand but with the advantages of a realistic visual feedback.

Deep learning-based retrieval system for gigapixel histopathology cases and the open access literature

Scientific paper ArODES

Roger Schaer, Oscar Alfonso Jiménez del Toro, Sebastian Otálora, Manfredo Atzori, Henning Müller

Journal of pathology informatics, 1 juillet 2019

Link to the publication

Summary:

Background: The introduction of digital pathology into clinical practice has led to the development of clinical workflows with digital images, in connection with pathology reports. Still, most of the current work is time‑consuming manual analysis of image areas at different scales. Links with data in the biomedical literature are rare, and a need for search based on visual similarity within whole slide images (WSIs) exists. Objectives: The main objective of the work presented is to integrate content‑based visual retrieval with a WSI viewer in a prototype. Another objective is to connect cases analyzed in the viewer with cases or images from the biomedical literature, including the search through visual similarity and text. Methods: An innovative retrieval system for digital pathology is integrated with a WSI viewer, allowing to define regions of interest (ROIs) in images as queries for finding visually similar areas in the same or other images and to zoom in/out to find structures at varying magnification levels. The algorithms are based on a multimodal approach, exploiting both text information and content‑based image features. Results: The retrieval system allows viewing WSIs and searching for regions that are visually similar to manually defined ROIs in various data sources (proprietary and public datasets, e.g., scientific literature). The system was tested by pathologists, highlighting its capabilities and suggesting ways to improve it and make it more usable in clinical practice. Conclusions: The developed system can enhance the practice of pathologists by enabling them to use their experience and knowledge to control artificial intelligence tools for navigating repositories of images for clinical decision support and teaching, where the comparison with visually similar cases can help to avoid misinterpretations. The system is available as open source, allowing the scientific community to test, ideate and develop similar systems for research and clinical practice.

A graph model of the lungs with morphology-based structure for tuberculosis type classification

Book chapter ArODES

Yashin Dicente Cid, Oscar Alfonso Jiménez del Toro, Pierre-Alexandre Poletti, Henning Müller

Dans Chung, Albert C. S., et al., Information processing in medical imaging : 26th International Conference, IPMI 2019, Hong Kong, China, June 2–7, 2019, proceedings (12 p.). 2019, Hong Kong, China : 2-7 June 2019

Link to the publication

Summary:

Pulmonary tuberculosis (TB) is still an important cause of death worldwide, even after being almost eradicated 40 years ago. Early identification of TB in computed tomography (CT) scans can influence therapeutical decisions, thus improving patient outcome. In this paper, a graph model of the lungs is proposed for the classification of TB types using local texture features in thorax CT scans of TB patients. Based on lung morphology, an automatic patient-specific lung field parcellation was initially computed. Local visual features were then extracted from each region and were used to build a personalized lung graph model. A new graph-based patient descriptor enables comparisons between lung graphs with a different number of nodes and edges, encoding the distribution of several node measures from graph theory. The proposed model was trained and tested on a public dataset of 1,513 CT scans of 994 TB patients. The evaluation was performed on data from a scientific challenge together with 39 participant algorithms, obtaining the best unweighted Cohen kappa coefficient of 0.24 in a 5-class setup with 505 test CTs. Even though each lung graph has a unique structure, the proposed method was able to identify key texture changes associated with the different manifestations of TB.

Kinematic synergies of hand grasps :

Scientific paper ArODES

a comprehensive study on a large publicly available dataset

Néstor J. Jarque-Bou, Alessandro Scano, Manfredo Aztori, Henning Müller

Journal of neuroengineering and rehabilitation, 2019, vol. 16, article no. 63

Link to the publication

Summary:

Background: Hand grasp patterns require complex coordination. The reduction of the kinematic dimensionality is a key process to study the patterns underlying hand usage and grasping. It allows to define metrics for motor assessment and rehabilitation, to develop assistive devices and prosthesis control methods. Several studies were presented in this field but most of them targeted a limited number of subjects, they focused on postures rather than entire grasping movements and they did not perform separate analysis for the tasks and subjects, which can limit the impact on rehabilitation and assistive applications. This paper provides a comprehensive mapping of synergies from hand grasps targeting activities of daily living. It clarifies several current limits of the field and fosters the development of applications in rehabilitation and assistive robotics. Methods: In this work, hand kinematic data of 77 subjects, performing up to 20 hand grasps, were acquired with a data glove (a 22-sensor CyberGlove II data glove) and analyzed. Principal Component Analysis (PCA) and hierarchical cluster analysis were used to extract and group kinematic synergies that summarize the coordination patterns available for hand grasps. Results: Twelve synergies were found to account for > 80% of the overall variation. The first three synergies accounted for more than 50% of the total amount of variance and consisted of: the flexion and adduction of the Metacarpophalangeal joint (MCP) of fingers 3 to 5 (synergy #1), palmar arching and flexion of the wrist (synergy #2) and opposition of the thumb (synergy #3). Further synergies refine movements and have higher variability among subjects. Conclusion: Kinematic synergies are extracted from a large number of subjects (77) and grasps related to activities of daily living (20). The number of motor modules required to perform the motor tasks is higher than what previously described. Twelve synergies are responsible for most of the variation in hand grasping. The first three are used as primary synergies, while the remaining ones target finer movements (e.g. independence of thumb and index finger). The results generalize the description of hand kinematics, better clarifying several limits of the field and fostering the development of applications in rehabilitation and assistive robotics.

Neural network training for cross-protocol radiomic feature standardization in computed tomography

Scientific paper ArODES

Vincent Andrearczyk, Adrien Depeursinge, Henning Müller

Journal of Medical Imaging, 2019, vol. 6, no. 2

Link to the publication

Summary:

Radiomics has shown promising results in several medical studies, yet it suffers from a limited discrimination and informative capability as well as a high variation and correlation with the tomographic scanner types, pixel spacing, acquisition protocol, and reconstruction parameters. We propose and compare two methods to transform quantitative image features in order to improve their stability across varying image acquisition parameters while preserving the texture discrimination abilities. In this way, variations in extracted features are representative of true physiopathological tissue changes in the scanned patients. A first approach is based on a two-layer neural network that can learn a nonlinear standardization transformation of various types of features including handcrafted and deep features. Second, domain adversarial training is explored to increase the invariance of the transformed features to the scanner of origin. The generalization of the proposed approach to unseen textures and unseen scanners is demonstrated by a set of experiments using a publicly available computed tomography texture phantom dataset scanned with various imaging devices and parameters.

ImageCLEF 2019 :

Book chapter ArODES

multimedia retrieval in lifelogging, medical, nature, and security applications

Bogdan Ionescu, Henning Müller, Renaud Péteri, Yashin Dicente Cid, et al.

Dans Azzopardi, Leif, et al., Advances in Information Retrieval : 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part II (pp. 301-308). 2019, Cham : Springer

Link to the publication

Summary:

This paper presents an overview of the foreseen ImageCLEF 2019 lab that will be organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2019. ImageCLEF is an ongoing evaluation initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2019, the 17th edition of ImageCLEF will run four main tasks: (i) a Lifelog task (videos, images and other sources) about daily activities understanding, retrieval and summarization, (ii) a Medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with newer data, (iii) a new Coral task about segmenting and labeling collections of coral images for 3D modeling, and (iv) a new Security task addressing the problems of automatically identifying forged content and retrieve hidden information. The strong participation, with over 100 research groups registering and 31 submitting results for the tasks in 2018 shows an important interest in this benchmarking campaign and we expect the new tasks to attract at least as many researchers for 2019.

ImageCLEF 2019 :

Book chapter ArODES

biodiversity identification and prediction challenges

Alexis Joly, Hervé Goëau, Christophe Botella, Henning Müller, et al.

Link to the publication

Summary:

Building accurate knowledge of the identity, the geographic distribution and the evolution of living species is essential for a sustainable development of humanity, as well as for biodiversity conservation. However, the burden of the routine identication of plants and animals in the eld is strongly penalizing the aggregation of new data and knowledge. Identifying and naming living plants or animals is actually almost impossible for the general public and often a dicult task for professionals and naturalists. Bridging this gap is a key challenge towards enabling eective biodiversity information retrieval systems. The LifeCLEF evaluation campaign, presented in this paper, aims at boosting and evaluating the advances in this domain since 2011. In particular, the 2019 edition proposes three data-oriented challenges related to the identication and prediction of biodiversity: (i) an image-based plant identication challenge, (ii) a bird sounds identication challenge and (iii) a location-based species prediction challenge based on spatial occurrence data and environmental tensors.

Revealing tumor habitats from texture heterogeneity analysis for classification of lung cancer malignancy and aggressiveness

Scientific paper ArODES

Dmitri Cherezo, Dmitri Goldgof, Lawrence Hall, Robert Gillies, Matthew Schabath, Henning Müller, Adrien Depeursinge

Scientific reports, Mars 2019, vol. 9, article no. 4500

Link to the publication

Summary:

We propose an approach for characterizing structural heterogeneity of lung cancer nodules using Computed Tomography Texture Analysis (CTTA). Measures of heterogeneity were used to test the hypothesis that heterogeneity can be used as predictor of nodule malignancy and patient survival. To do this, we use the National Lung Screening Trial (NLST) dataset to determine if heterogeneity can represent differences between nodules in lung cancer and nodules in non-lung cancer patients. 253 participants are in the training set and 207 participants in the test set. To discriminate cancerous from non-cancerous nodules at the time of diagnosis, a combination of heterogeneity and radiomic features were evaluated to produce the best area under receiver operating characteristic curve (AUROC) of 0.85 and accuracy 81.64%. Second, we tested the hypothesis that heterogeneity can predict patient survival. We analyzed 40 patients diagnosed with lung adenocarcinoma (20 short-term and 20 long-term survival patients) using a leave-one-out cross validation approach for performance evaluation. A combination of heterogeneity features and radiomic features produce an AUROC of 0.9 and an accuracy of 85% to discriminate long- and short-term survivors.

Heterogeneous exascale computing

Book chapter ArODES

Ladislav Hluchý, Martin Bobák, Henning Müller, Mara Graziani, Matti Heikkurinen, et al.

Dans Haidegger, Tamás, Kovács, Levente, Szakál, Anikó, Recent advances in intelligent engineering : volume dedicated to Imre J. Rudas’ Seventieth Birthday (pp. 81-110). 2019, Cham : Springer

Link to the publication

Summary:

Exascale services bring new unique challenges that the current computational, big data and workflow solutions are unable to meet. The chapter includes a detailed description of selected exascale services with known state of the art in extreme date solutions. The integration of requirements and the analysis of the state of the art in the exascale field is centered in on a description of a high-level architectural approach. The next main contribution of the paper is the description of the architecture capable to handle heterogeneous exascale services coming from both academic as well as industrial sphere. Those two models represent a (conceptual, and technological) design of a platform that addresses the requirements of the use cases. The resulting architecture will help us provide computing solutions to exascale challenges within the H2020 project PROCESS.

Medical decision support using increasingly large multimodal data sets

Book chapter ArODES

Henning Müller, Devrim Unay

Dans et al., Vrochidis, Stefanos, Big data analytics for large-scale multimedia search (20 p.). 2019, [Hoboken] : John Wiley & Sons

Link to the publication

Summary:

Medical decision support has traditionally been using model‐based approaches and small data sets for evaluation. This chapter analyses recent trends and techniques that make use of increasingly large data sets and thus more data‐driven approaches to medical decision support that have in some areas replaced the more traditional rule‐based approaches. It explains the challenge infrastructures and approaches as they are often essential to access data, and application scenarios give examples of existing applications and objectives. The chapter focuses on how to overcome current limitations and how to tackle the upcoming challenges of image‐based decision support for digital medicine. It reviews the literature of mainly the past five years in the field of medical visual decision support and highlights the use of multimodal data and data‐driven approaches. Machine learning is an indispensable part of medical decision support, especially in diagnostic and categorization applications.

A quantitative taxonomy of human hand grasps

Scientific paper ArODES

Francesca Stival, Stefano Michieletto, Matteo Cognolato, Enrico Pagello, Henning Müller, Manfredo Atzori

Journal of neuroengineering and rehabilitation, 2019, vol. 16, no. 28

Link to the publication

Summary:

A proper modeling of human grasping and of hand movements is fundamental for robotics, prosthetics, physiology and rehabilitation. The taxonomies of hand grasps that have been proposed in scientific literature so far are based on qualitative analyses of the movements and thus they are usually not quantitatively justified. Methods This paper presents to the best of our knowledge the first quantitative taxonomy of hand grasps based on biomedical data measurements. The taxonomy is based on electromyography and kinematic data recorded from 40 healthy subjects performing 20 unique hand grasps. For each subject, a set of hierarchical trees are computed for several signal features. Afterwards, the trees are combined, first into modality-specific (i.e. muscular and kinematic) taxonomies of hand grasps and then into a general quantitative taxonomy of hand movements. The modality-specific taxonomies provide similar results despite describing different parameters of hand movements, one being muscular and the other kinematic. Results The general taxonomy merges the kinematic and muscular description into a comprehensive hierarchical structure. The obtained results clarify what has been proposed in the literature so far and they partially confirm the qualitative parameters used to create previous taxonomies of hand grasps. According to the results, hand movements can be divided into five movement categories defined based on the overall grasp shape, finger positioning and muscular activation. Part of the results appears qualitatively in accordance with previous results describing kinematic hand grasping synergies. Conclusions The taxonomy of hand grasps proposed in this paper clarifies with quantitative measurements what has been proposed in the field on a qualitative basis, thus having a potential impact on several scientific fields.

2018

Evaluation-as-a-service for the computational sciences :

Scientific paper ArODES

overview and outlook

Frank Hopfgartner, Allan Hanbury, Henning Müller, Ivan Eggel, Balog Krisztian, Torben Brodt, Gordon V. Cormack, Jimmy Lin, Jayashree Kalpathy-Cramer, Noriko Kando, Makoto P. Kato, Anastasia Krithara, Tim Gollub, Martin Potthast, Evelyne Viegas, Simon Mercer

Journal of data and information quality, October 2018, vol. 10, no. 4, pp. 1-32

Link to the publication

Summary:

Evaluation in empirical computer science is essential to show progress and assess technologies developed. Several research domains such as information retrieval have long relied on systematic evaluation to measure progress: here, the Cranfield paradigm of creating shared test collections, defining search tasks, and collecting ground truth for these tasks has persisted up until now. In recent years, however, several new challenges have emerged that do not fit this paradigm very well: extremely large data sets, confidential data sets as found in the medical domain, and rapidly changing data sets as often encountered in industry. Also, crowdsourcing has changed the way that industry approaches problem-solving with companies now organizing challenges and handing out monetary awards to incentivize people to work on their challenges, particularly in the field of machine learning. This white paper is based on discussions at a workshop on Evaluation-as-a-Service (EaaS). EaaS is the paradigm of not providing data sets to participants and have them work on the data locally, but keeping the data central and allowing access via Application Programming Interfaces (API), Virtual Machines (VM) or other possibilities to ship executables. The objective of this white paper are to summarize and compare the current approaches and consolidate the experiences of these approaches to outline the next steps of EaaS, particularly towards sustainable research infrastructures. This white paper summarizes several existing approaches to EaaS and analyzes their usage scenarios and also the advantages and disadvantages. The many factors influencing EaaS are overviewed, and the environment in terms of motivations for the various stakeholders, from funding agencies to challenge organizers, researchers and participants, to industry interested in supplying real-world problems for which they require solutions.

Overview of LifeCLEF 2018 :

Book chapter ArODES

a large-scale evaluation of species identification and recommendation algorithms in the era of AI

Alexis Joly, Hervé Goëau, Christophe Botella, Hervé Glotin, Pierre Bonnet, Willem-Pier Vellinga, Robert Planqué, Henning Müller

Dans Bellot, P., Experimental IR meets multilinguality, multimodality, and interaction : 9th International Conference of the CLEF Association, CLEF 2018, Avignon, France, September 10-14, 2018, Proceedings (Pp. 247-266). 2018, Cham : Springer

Link to the publication

Summary:

Building accurate knowledge of the identity, the geographic distribution and the evolution of living species is essential for a sustain-able development of humanity, as well as for biodiversity conservation. Unfortunately, such basic information is often only partially available for professional stakeholders, teachers, scientists and citizens, and often incomplete for ecosystems that possess the highest diversity. In this con-text, an ultimate ambition is to set up innovative information systems relying on the automated identification and understanding of living or-ganisms as a means to engage massive crowds of observers and boost the production of biodiversity and agro-biodiversity data. The LifeCLEF 2018 initiative proposes three data-oriented challenges related to this vi-sion, in the continuity of the previous editions, but with several consis-tent novelties intended to push the boundaries of the state-of-the-art in several research directions. This paper describes the methodology of the conducted evaluations as well as the synthesis of the main results and lessons learned.

Overview of ImageCLEF 2018 :

Book chapter ArODES

challenges, datasets and evaluation

Bogdan Ionescu, Henning Müller, Mauricio Villegas, Alba García Seco de Herrera, Carsten Eickhoff, Vincent Andrearczyk, Yashin Dicente Cid, Vitali Liauchuk, Vassili Kovalev, Sadid A. Hasan, Yuan Ling, Oladimeji Farri, Joey Liu, Matthew Lungren, Duc-Tien Dang-Nguyen, Luca Piras, Michael Riegler, Liting Zhou, Mathias Lux, Cathal Gurrin

Link to the publication

Summary:

Abstract. This paper presents an overview of the ImageCLEF 2018 evaluation campaign, an event that was organized as part of the CLEF (Conference and Labs of the Evaluation Forum) Labs 2018. ImageCLEF is an ongoing initiative (it started in 2003) that promotes the evalua-tion of technologies for annotation, indexing and retrieval with the aim of providing information access to collections of images in various us-age scenarios and domains. In 2018, the 16th edition of ImageCLEF ran three main tasks and a pilot task: 1) a caption prediction task that aims at predicting the caption of a figure from the biomedical literature based only on the figure image; 2) a tuberculosis task that aims at detecting the tuberculosis type, severity and drug resistance from CT (Computed Tomography) volumes of the lung; 3) a LifeLog task (videos, images and other sources) about daily activities understanding and moment re-trieval, and 4) a pilot task on visual question answering where systems are tasked with answering medical questions. The strong participation, with over 100 research groups registering and 31 submitting results for the tasks, shows an increasing interest in this benchmarking campaign.

Muscle synergy analysis of a hand-grasp dataset :

Scientific paper ArODES

a limited subset of motor modules may underlie a large variety of grasps

Alessandro Scano, Andrea Chiavenna, Lorenzo Molinari Tosatti, Henning Müller, Manfredo Atzori

Frontiers in Neurorobotics, 2018, vol. 12

Link to the publication

Summary:

Kinematic and muscle patterns underlying hand grasps have been widely investigated in the literature. However, the identification of a reduced set of motor modules, generalizing across subjects and grasps, may be valuable for increasing the knowledge of hand motor control, and provide methods to be exploited in prosthesis control and hand rehabilitation. Methods: Motor muscle synergies were extracted from a publicly available database including 28 subjects, executing 20 hand grasps selected for daily-life activities. The spatial synergies and temporal components were analyzed with a clustering algorithm to characterize the patterns underlying hand-grasps. Results: Motor synergies were successfully extracted on all 28 subjects. Clustering orders ranging from 2 to 50 were tested. A subset of ten clusters, each one represented by a spatial motor module, approximates the original dataset with a mean maximum error of 5% on reconstructed modules; however, each spatial synergy might be employed with different timing and recruited at different grasp stages. Two temporal activation patterns are often recognized, corresponding to the grasp/hold phase, and to the pre-shaping and release phase. Conclusions: This paper presents one of the biggest analysis of muscle synergies of hand grasps currently available. The results of 28 subjects performing 20 different grasps suggest that a limited number of time dependent motor modules (shared among subjects), correctly elicited by a control activation signal, may underlie the execution of a large variety of hand grasps. However, spatial synergies are not strongly related to specific motor functions but may be recruited at different stages, depending on subject and grasp. This result can lead to applications in rehabilitation and assistive robotics.

How to exploit weaknesses in biomedical challenge design and organization

Book chapter ArODES

Annika Reinke, Matthias Eisenmann, Sinan Onogur, Marko Stankovic, Patrick Scholz, Peter M. Full, Hrvoje Bogunovic, Bennett A. Landman, Oskar Maier, Bjoern Menze, Gregory C. Sharp, Korsuk Sirinukunwattana, Stefanie Speidel, Fons Van der Sommen, Guoyan Zheng, Henning Müller, Michal Kozubek, Tal Arbel, Andrew P. Bradley, Pierre Jannin, Annette Kopp-Schneider, Lena Maier-Hein

Dans Frangi, Alejandro F., Medical image computing and computer assisted intervention – MICCAI 2018 : 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part IV (Pp. 388-395). 2018, Cham : Springer

Link to the publication

Summary:

Since the first MICCAI grand challenge organized in 2007 in Brisbane, challenges have become an integral part of MICCAI conferences. In the meantime, challenge datasets have become widely recognized as international benchmarking datasets and thus have a great influence on the research community and individual careers. In this paper, we show several ways in which weaknesses related to current challenge design and organization can potentially be exploited. Our experimental analysis, based on MICCAI segmentation challenges organized in 2015, demonstrates that both challenge organizers and participants can potentially undertake measures to substantially tune rankings. To overcome these problems we present best practice recommendations for improving challenge design and organization.

From local to global :

Book chapter ArODES

a holistic lung graph model

Yashin Dicente Cid, Oscar Alfonso Jiménez del Toro, Alexandra Platon, Henning Müller, Pierre-Alexandre Poletti

Link to the publication

Summary:

Lung image analysis is an essential part in the assessment of pulmonary diseases. Through visual inspection of CT scans, radiologists detect abnormal patterns in the lung parenchyma, aiming to establish a timely diagnosis and thus improving patient outcome. However, in a generalized disorder of the lungs, such as pulmonary hypertension, the changes in organ tissue can be elusive, requiring additional invasive studies to confirm the diagnosis. We present a graph model that quantifies lung texture in a holistic approach enhancing the analysis between pathologies with similar local changes. The approach extracts local state-of-the-art 3D texture descriptors from an automatically generated geometric parcellation of the lungs. The global texture distribution is encoded in a weighted graph that characterizes the correlations among neighboring organ regions. A data set of 125 patients with suspicion of having a pulmonary vascular pathology was used to evaluate our method. Three classes containing 47 pulmonary hypertension, 31 pulmonary embolism and 47 control cases were classified in a one vs. one setup. An area under the curve of up to 0.85 was obtained adding directionality to the edges of the graph architecture. The approach was able to identify diseased patients, and to distinguish pathologies with abnormal local and global blood perfusion defects.

Image magnification regression using DenseNet for exploiting histopathology open access content

Book chapter ArODES

Sebastian Otálora, Manfredo Atzori, Vincent Andrearczyk, Henning Müller

Dans Stoyanov, Danail, Computational pathology and ophthalmic medical image analysis (pp. 148-155). 2018, Cham : Springer

Link to the publication

Summary:

Open access medical content databases such as PubMed Central and TCGA offer possibilities to obtain large amounts of images for training deep learning models. Nevertheless, accurate labeling of large-scale medical datasets is not available and poses challenging tasks for using such datasets. Predicting unknown magnification levels and standardize staining procedures is a necessary preprocessing step for using this data in retrieval and classification tasks. In this paper, a CNN-based regression approach to learn the magnification of histopathology images is presented, comparing two deep learning architectures tailored to regress the magnification. A comparison of the performance of the models is done in a dataset of 34,441 breast cancer patches with several magnifications. The best model, a fusion of DenseNet-based CNNs, obtained a kappa score of 0.888. The methods are also evaluated qualitatively on a set of images from biomedical journals and TCGA prostate patches.

Deep multimodal classification of image types in biomedical journal figures

Book chapter ArODES

Vincent Adrearczyk, Henning Müller

Dans Bellot, P., Mothe, J, Trabelsi, C., Experimental IR meets multilinguality, multimodality, and interaction : 9th International Conference of the CLEF Association, CLEF 2018, Avignon, France, September 10-14, 2018, Proceedings (Pp. 3-14). 2018, Cham : Springer

Link to the publication

Summary:

This paper presents a robust method for the classification of medical image types in figures of the biomedical literature using the fusion of visual and textual information. A deep convolutional network is trained to discriminate among 31 classes including compound figures, diagnostic image types and generic illustrations, while another shallow convolutional network is used for the analysis of the captions paired with the images. Various fusion methods are analyzed as well as data augmentation approaches. The proposed system is validated on the ImageCLEF 2013 classification task, largely improving the currently best performance from 83.5% to 93.7% accuracy.

Glaucoma diagnosis from eye fundus images based on deep morphometric feature estimation

Book chapter ArODES

Oscar Perdomo, Vincent Andrearczyk, Fabrice Meriaudeau, Henning Müller, Fabio González

Dans Stoyanov, Danail, Computational pathology and ophthalmic medical image analysis (8 p.). 2018, Cham : Springer

Link to the publication

Summary:

Glaucoma is an ophthalmic disease related to damage in the optic nerve and it is without symptoms in its early stages. Left untreated, it can lead to vision limitation and blindness. Eye fundus images have been widely accepted by medical personnel to examine the morphology and texture of the optic nerve head and the physiologic cup but glaucoma diagnosis is still subjective and without clear consensus among experts. This paper presents a multi-stage deep learning model for glaucoma diagnosis based on a curriculum learning strategy. In curriculum learning, a model is sequentially trained to solve incrementally difficult tasks. Our proposed model includes the following stages: segmentation of the optic disc and physiological cup, prediction of morphometric features from segmentations, and prediction of disease level (healthy, suspicious and glaucoma). The experimental evaluation shows that our proposed method outperforms conventional convolutional deep learning models from the state of the art reported on the RIM-ONE-v1 and DRISHTI-GS1 datasets with an accuracy of 89.4% and an AUC of 0.82 respectively.

Textured graph-based model of the lungs :

Book chapter ArODES

application on tuberculosis type classification and multi-drug resistance detection

Yashin Dicente Cid, Kayhan Batmanghelich, Henning Müller

Dans Bello, Patrice, Mothe, Josiane, Murtagh, Fionn, Trabelsi, Chiraz, Experimental IR Meets Multilinguality, Multimodality, and Interaction : 9th International Conference of the CLEF Association, CLEF 2018, Avignon, France, September 10-14, 2018, Proceedings (pp. 157-168). 2018, Cham : Springer

Link to the publication

Summary:

Tuberculosis (TB) remains a leading cause of death worldwide. Two main challenges when assessing computed tomography scans of TB patients are detecting multi{drug resistance and di_erentiating TB types. In this article we model the lungs as a graph entity where nodes represent anatomical lung regions and edges encode interactions between them. This graph is able to characterize the texture distribution along the lungs, making it suitable for describing patients with different TB types. In 2017, the ImageCLEF benchmark proposed a task based on computed tomography volumes of patients with TB. This task was divided into two subtasks: multi{drug resistance prediction, and TB type classi_cation. The participation in this task showed the strength of our model, leading to best results in the competition for multi{drug resistance detection (AUC = 0.5825) and good results in the TB type classi_cation (Cohen's Kappa coe_cient = 0.1623).

Subdiv17 :

Book chapter ArODES

a dataset for investigating subjectivity in the visual diversification of image search results

Maia Rohm, Bogdan Ionescu, Alexandru Lucian Gînsca, Rodrygo Santos, Henning Müller

MMSys'18 9th ACM Multimedia Systems Conference (pp. 444-449). 2018, New York : ACM

Link to the publication

Summary:

In this paper, we present a new dataset that facilitates the comparison of approaches aiming at the diversification of image search results. The dataset was explicitly designed for general-purpose, multi-topic queries and provides multiple ground truth annotations to allow for the exploration of the subjectivity aspect in the general task of diversification. The dataset provides images and their metadata retrieved from Flickr for around 200 complex queries. Additionally, to encourage experimentations (and cooperations) from different communities such as information and multimedia retrieval, a broad range of pre-computed descriptors is provided. The proposed dataset was successfully validated during the MediaEval 2017 Retrieving Diverse Social Images task using 29 submitted runs.

Head-mounted eye gaze tracking devices :

Scientific paper ArODES

an overview of modern devices and recent advances

Matteo Cognolato, Manfredo Atzori, Henning Müller

Journal of rehabilitation and assistive technologies engineering, 2018, vol. 5, pp. 1-13

Link to the publication

Summary:

An increasing number of wearable devices performing eye gaze tracking have been released in recent years. Such devices can lead to unprecedented opportunities in many applications. However, staying updated regarding the continuous advances and gathering the technical features that allow to choose the best device for a specific application is not trivial. The last eye gaze tracker overview was written more than 10 years ago, while more recent devices are substantially improved both in hardware and software. Thus, an overview of current eye gaze trackers is needed. This review fills the gap by providing an overview of the current level of advancement for both techniques and devices, leading finally to the analysis of 20 essential features in six head-mounted eye gaze trackers commercially available. The analyzed characteristics represent a useful selection providing an overview of the technology currently implemented. The results show that many technical advances were made in this field since the last survey. Current wearable devices allow to capture and exploit visual information unobtrusively and in real time, leading to new applications in wearable technologies that can also be used to improve rehabilitation and enable a more active living for impaired persons.

Tumor proliferation assessment of whole slide images

Scientific paper ArODES

Mikael Rousson, Martin Hedlunda, Mats Anderssona, Ludwig Jacobsson, Gunnar Lathen, Bjorn Norella, Oscar Alfonso Jiménez del Toro, Henning Müller, Manfredo Atzori

Medical Imaging 2018 (SPIE) : Digital Pathology, March 2018, vol. 10581

Link to the publication

Summary:

Grading whole slide images (WSIs) from patient tissue samples is an important task in digital pathology, particularly for diagnosis and treatment planning. However, this visual inspection task, performed by pathologists, is inherently subjective and has limited reproducibility. Moreover, grading of WSIs is time consuming and expensive. Designing a robust and automatic solution for quantitative decision support can improve the objectivity and reproducibility of this task. This paper presents a fully automatic pipeline for tumor proliferation assessment based on mitosis counting. The approach consists of three steps: i) region of interest selection based on tumor color characteristics, ii) mitosis counting using a deep network based detector, and iii) grade prediction from ROI mitosis counts. The full strategy was submitted and evaluated during the Tumor Proliferation Assessment Challenge (TUPAC) 2016. TUPAC is the rst digital pathology challenge grading whole slide images, thus mimicking more closely a real case scenario. The pipeline is extremely fast and obtained the 2nd place for the tumor proliferation assessment task and the 3rd place in the mitosis counting task, among 17 participants. The performance of this fully automatic method is similar to the performance of pathologists and this shows the high quality of automatic solutions for decision support.

Rotation-covariant tissue analysis for interstitial lung diseases using learned steerable filters :

Scientific paper ArODES

performance evaluation and relevance for diagnostic aid

Ranveer Joyseeree, Henning Müller, Adrien Depeursinge

Computerized medical imaging and graphics, March 2018, vol. 64, pp. 1-11

Link to the publication

Summary:

A novel method to detect and classify several classes of diseased and healthy lung tissue of interstitial lung diseases is presented, as these diseases are hard to diagnose and differentiate. Local organizations of image directions at several scales drive the process of creating discriminative lung tissue texture signatures using spatial and Fourier domain information extracted from the images. The signatures are generated for four diseased tissue classes and healthy tissue, all of which appear in the Interstitial Lung Disease (ILD) database, using a novel one-versus-one approach for learning discriminative filter signatures. A multiclass tissue classification accuracy of 80.31% is observed using Radial Basis Function (RBF) Support Vector Machines (SVMs). The presented method compares well against a variety of state-of-the-art approaches. Another strong feature of our approach is the ability to access the individual class probabilities before a final classification decision is made. This enables an analysis of the causes of misclassification in this paper. We also make the case against total reliance on the accuracy of the ground truth given that the ILD database only contains a single label for a specific region and sometimes more than one pattern can be present, particularly for regions classified as healthy tissue. Measures to address misclassifications in this context are also proposed.

Large-scale retrieval for medical image analytic :

Scientific paper ArODES

a comprehensive review

Zhongyu Li, Xiaofan Zhang, Henning Müller, Shaoting Zhang

Medical image analysis, 2018, vol. 43, pp. 66-84

Link to the publication

Summary:

Over the past decades, medical image analytics was greatly facilitated by the explosion of digital imaging techniques, where huge amounts of medical images were produced with ever-increasing quality and diversity. However, conventional methods for analyzing medical images have achieved limited success, as they are not capable to tackle the huge amount of image data. In this paper, we review state-of-the-art approaches for large-scale medical image analysis, which are mainly based on recent advances in computer vision, machine learning and information retrieval. Specifically, we first present the general pipeline of large-scale retrieval, summarize the challenges/opportunities of medical image analytics on a large-scale. Then, we provide a comprehensive review of algorithms and techniques relevant to major processes in the pipeline, including feature representation, feature indexing, searching, etc. On the basis of existing work, we introduce the evaluation protocols and multiple applications of large-scale medical image retrieval, with a variety of exploratory and diagnostic scenarios. Finally, we discuss future directions of large-scale retrieval, which can further improve the performance of medical image analysis.

2017

Datasets column :

Scientific paper ArODES

diversity and credibility for social images and image retrieval

Bogdan Ionescu, Mihai Lupu, Maia Rohm, Alexandru Lucian Gînsca, Henning Müller

ACM SIGMultimedia Records, December 2017, vol. 9, no. 3

Link to the publication

Shangri–La :

Scientific paper ArODES

a medical case–based retrieval tool

Alba García Seco de Herrera, Roger Schaer, Henning Müller

Journal of the association for information science and technology, 2017, vol. 68, no 11, pp. 2587–2601

Link to the publication

Summary:

Large amounts of medical visual data are produced in hospitals daily and made available continuously via publications in the scientific literature, representing the medical knowledge. However, it is not always easy to find the desired information and in clinical routine the time to fulfil an information need is often very limited. Infor-mation retrieval systems are a useful tool to provide access to these documents/images in the biomedical literature related to information needs of medical professionals. Shangri-La is a medical retrieval sys-tem that can potentially help clinicians to make decisions on diﬃcult cases. It retrieves articles from the biomedical literature when query-ing a case description and attached images. The system is based on a multi-modal retrieval approach with a focus on the integration of visual information connected to text. The approach includes a query-adaptive multi-modal fusion criterion that analyses if visual features are suitable to be fused with text for the retrieval. Furthermore, image modality information is integrated in the retrieval step. The approach is evaluated using the ImageCLEFmed 2013 medical retrieval bench-mark and can thus be compared to other approaches. Results show that the final approach outperforms the best multi-modal approach submitted to ImageCLEFmed 2013.

Comparison of six electromyography acquisition setups on hand movement classification tasks

Scientific paper ArODES

Stefano Pizzolato, Luca Tagliapietra, Matteo Cognolato, Monica Reggiani, Henning Müller, Manfredo Atzori

Plos One, 2017, vol.

Link to the publication

Summary:

Hand prostheses controlled by surface electromyography are promising due to the non-invasive approach and the control capabilities offered by machine learning. Nevertheless, dexterous prostheses are still scarcely spread due to control difficulties, low robustness and often prohibitive costs. Several sEMG acquisition setups are now available, ranging in terms of costs between a few hundred and several thousand dollars. The objective of this paper is the relative comparison of six acquisition setups on an identical hand movement classification task, in order to help the researchers to choose the proper acquisition setup for their requirements. The acquisition setups are based on four different sEMG electrodes (including Otto Bock, Delsys Trigno, Cometa Wave + Dormo ECG and two Thalmic Myo armbands) and they were used to record more than 50 hand movements from intact subjects with a standardized acquisition protocol. The relative performance of the six sEMG acquisition setups is compared on 41 identical hand movements with a standardized feature extraction and data analysis pipeline aimed at performing hand movement classification. Comparable classification results are obtained with three acquisition setups including the Delsys Trigno, the Cometa Wave and the affordable setup composed of two Myo armbands. The results suggest that practical sEMG tests can be performed even when costs are relevant (e.g. in small laboratories, developing countries or use by children). All the presented datasets can be used for offline tests and their quality can easily be compared as the data sets are publicly available.

Training deep convolutional neural networks with active learning for exudate classification in eye fundus images

Book chapter ArODES

Sebastian Otálora, Oscar Perdomo, Fabio González, Henning Müller

LABELS 2017, CVII 2017, STENT 2017 : intravascular imaging and computer assisted stenting, and large-scale annotation of biomedical data and expert label synthesis (pp. 146-154). 2017, Cham : Springer

Link to the publication

Summary:

Training deep convolutional neural network for classification in medical tasks is often diﬃcult due to the lack of annotated data sam-ples. Deep convolutional networks (CNN) has been successfully used as an automatic detection tool to support the grading of diabetic retinopa-thy and macular edema. Nevertheless, the manual annotation of exu-dates in eye fundus images used to classify the grade of the DR is very time consuming and repetitive for clinical personnel. Active learning al-gorithms seek to reduce the labeling eﬀort in training machine learning models. This work presents a label-eﬃcient CNN model using the ex-pected gradient length, an active learning algorithm to select the most informative patches and images, converging earlier and to a better local optimum than the usual SGD (Stochastic Gradient Descent) strategy. Our method also generates useful masks for prediction and segments regions of interest.

Retrieval from and understanding of large–scale multi–modal medical datasets :

Scientific paper ArODES

a review

Henning Müller, Devrim Unay

IEEE transactions on multimedia, 2017, vol. 19, no. 9, pp. 2093-2104

Link to the publication

Summary:

Content–based multimedia retrieval has been an active research domain since the mid 1990s. In the medical domain visual retrieval started later and has mostly remained a research instrument and less a clinical tool, even though a few tools for retrieval are employed in clinical work. The limited size of data sets due to privacy constraints is often mentioned as a reason for these limitations. Nevertheless, much work has been done in medical visual information retrieval, including the availability of increasingly large data sets and scientific challenges. Annotated data sets and clinical data for the images have now become available and can be combined for multi– modal retrieval. Much has been learned on user behavior and application scenarios. This text is motivated by the advances in medical image analysis and the availability of more public data large data sets that often include clinical data that can be combined for multimodal retrieval based on the experience available in the multimedia community. This text is a systematic review of recent work (concentrating on the period between 2011-2017) on content–based multi–modal retrieval and image understanding in the medical domain, where image understanding includes techniques such as detection, localization, and classification for leveraging visual content. The main conferences in the field are screened for relevant articles and these are presented in a structured way, identifying current limitations and areas where work is still much required. Objective of the work is to summarize the current state of research for multimedia researchers not working in the medical field. It provides ways to get data sets and identify promising research directions. The text highlights the areas of advances in the past six years and particularly a trend to use larger scale training data sets as well as deep learning approaches that can replace or complement hand–crafted feature extraction. Using images alone will likely only work in limited sub domains but combining multiple sources of data for multi–modal retrieval has the biggest chances of success, particularly for clinical impact. Future fields of research are identified in the text, as there is a high research potential in the medical multimedia domain.

Deep multimodal case–based retrieval for large histopathology datasets

Book chapter ArODES

Oscar Alfonso Jiménez del Toro, Sebastian Otálora, Manfredo Atzori, Henning Müller

Patch-based techniques in medical imaging : third International Workshop, Patch-MI 2017, held in conjunction with MICCAI 2017, Quebec City, QC, Canada, September 14, 2017, Proceedings (pp. 149-157). 2017, Cham : Springer

Link to the publication

Summary:

The current gold standard for interpreting patient tissue samples is the visual inspection of whole–slide histopathology images (WSIs) by pathologists. They generate a pathology report describing the main findings relevant for diagnosis and treatment planning. Search-ing for similar cases through repositories for diﬀerential diagnosis is often not done due to a lack of efficient strategies for medical case–based re-trieval. A patch–based multimodal retrieval strategy that retrieves sim-ilar pathology cases from a large data set fusing both visual and text information is explained in this paper. By fine–tuning a deep convolu-tional neural network an automatic representation is obtained for the vi-sual content of weakly annotated WSIs (using only a global cancer score and no manual annotations). The pathology text report is embedded into a category vector of the pathology terms also in a non–supervised approach. A publicly available data set of 267 prostate adenocarcinoma cases with their WSIs and corresponding pathology reports was used to train and evaluate each modality of the retrieval method. A MAP (Mean Average Precision) of 0.54 was obtained with the multimodal method in a previously unseen test set. The proposed retrieval system can help in diﬀerential diagnosis of tissue samples and during the training of pathol-ogists, exploiting the large amount of pathology data already existing digital hospital repositories.

Analysis of histopathology images :

Book chapter ArODES

from traditional machine learning to deep learning

Oscar Alfonso Jiménez del Toro, Sebastian Otálora, Mats Andersson, Kristian Eurén, Martin Hedlund, Mikael Rousson, Henning Müller, Manfredo Atzori

Biomedical texture analysis : fundamentals, tools and challenges (pp. 281–314). 2017, [S. l.] : Elsevier

Link to the publication

Summary:

Digitizing pathology is a current trend that makes large amounts of visual data available for automatic analysis. It allows to visualize and interpret pathologic cell and tissue samples in high-resolution images and with the help of computer tools. This opens the possibility to develop image analysis methods that help pathologists and support their image descriptions (i.e., staging, grading) with objective quantification of image features. Numerous detection, classification and segmentation algorithms of the underlying tissue primitives in histopathology images have been proposed in this respect. To better select the most suitable algorithms for histopathology tasks, biomedical image analysis challenges have evaluated and compared both traditional feature extraction with machine learning and deep learning techniques. This chapter provides an overview of methods addressing the analysis of histopathology images, as well as a brief description of the tasks they aim to solve. It is focused on histopathology images containing textured areas of different types.

Semi-automatic training of an object recognition system in scene camera data using gaze tracking and accelerometers

Book chapter ArODES

Matteo Cognolato, Mara Graziani, Francesca Giordaniello, Gianluca Saetta, Bassetto, Peter Brugger, Barbara Caputo, Henning Müller, Manfredo Atzori

Computer Vision Systems : 11th International Conference, ICVS 2017, Shenzhen, China, July 10-13, 2017 (pp. 175-184). 2017, Cham : Springer

Link to the publication

Summary:

Object detection and recognition algorithms usually require large, annotated training sets. The creation of such datasets requires expensive manual annotation. Eye tracking can help in the annotation procedure. Humans use vision constantly to explore the environment and plan motor actions, such as grasping an object. In this paper we investigate the possibility to semi-automatically train object recognition with eye tracking, accelerometer in scene camera data, learning from the natural hand-eye coordination of humans. Our approach involves three steps. First, sensor data are recorded using eye tracking glasses that are used in combination with accelerometers and surface electromyography that are usually applied when controlling prosthetic hands. Second, a set of patches are extracted automatically from the scene camera data while grasping an object. Third, a convolutional neural network is trained and tested using the extracted patches. Results show that the parameters of eye-hand coordination can be used to train an object recognition system semi-automatically. These can be exploited with proper sensors to fine-tune a convolutional neural network for object detection and recognition. This approach opens interesting options to train computer vision and multi-modal data integration systems and lays the foundations for future applications in robotics. In particular, this work targets the improvement of prosthetic hands by recognizing the objects that a person may wish to use. However, the approach can easily be generalized.

Combining radiology images and clinical metadata for multimodal medical case-based retrieval

Book chapter ArODES

Oscar Alfonso Jiménez del Toro, Pol Cirujeda, Henning Müller

Cloud-based benchmarking of medical image analysis (pp. 221-236). 2017, Cham : Springer

Link to the publication

Summary:

As part of their daily workload, clinicians examine patient cases in the process of formulating a diagnosis. These large multimodal patient datasets stored in hospitals could help in retrieving relevant information for a differential diagnosis, but these are currently not fully exploited. The VISCERAL Retrieval Benchmark organized a medical case-based retrieval algorithm evaluation using multimodal (text and visual) data from radiology reports. The common dataset contained patient CT (Computed Tomography) or MRI (Magnetic Resonance Imaging) scans and RadLex term anatomy–pathology lists from the radiology reports. A content-based retrieval method for medical cases that uses both textual and visual features is presented. It defines a weighting scheme that combines the anatomical and clinical correlations of the RadLex terms with local texture features obtained from the region of interest in the query cases. The visual features are computed using a 3D Riesz wavelet texture analysis performed on a common spatial domain to compare the images in the analogous anatomical regions of interest in the dataset images. The proposed method obtained the best mean average precision in 6 out of 10 topics and the highest number of relevant cases retrieved in the benchmark. Obtaining robust results for various pathologies, it could further be developed to perform medical case-based retrieval on large multimodal clinical datasets.

Retrieval of medical cases for diagnostic decisions :

Book chapter ArODES

VISCERAL retrieval benchmark

Oscar Alfonso Jiménez del Toro, Henning Müller, Antonio Foncubierta-Rodriguez, Georg Langs, Allan Hanbury

Cloud-based benchmarking of medical image analysis (pp. 127-141). 2017, Cham : Springer

Link to the publication

Summary:

Health providers currently construct their differential diagnosis for a given medical case most often based on textbook knowledge and clinical experience. Data mining of the large amount of medical records generated daily in hospitals is only very rarely done, limiting the reusability of these cases. As part of the VISCERAL project, the Retrieval benchmark was organized to evaluate available approaches for medical case-based retrieval. Participant algorithms were required to find and rank relevant medical cases from a large multimodal dataset (including semantic RadLex terms extracted from text and visual 3D data) for common query topics. The relevance assessment of the cases was done by medical experts who selected cases that are useful for a differential diagnosis for the given query case. The approaches that integrated information from both the RadLex terms and the 3D volumes (mixed techniques) obtained the best results based on five standard evaluation metrics. The benchmark set up, dataset description and result analysis are presented.

Text- and content-based medical image retrievals in the VISCERAL retrieval benchmark

Book chapter ArODES

Fan Zhang, Yang Song, Weidong Cai, Adrien Depeursinge, Henning Müller

Cloud-based benchmarking of medical image analysis (pp. 237-249). 2017, Cham : Springer

Link to the publication

Summary:

Text- and content-based retrieval are the most widely used approaches for medical image retrieval. They capture the similarity between the images from different perspectives: text-based methods rely on manual textual annotations or captions associated with images; content-based approaches are based on the visual content of the images themselves such as colours and textures. Text-based retrieval can better meet the high-level expectations of humans but is limited by the time-consuming annotations. Content-based retrieval can automatically extract the visual features for high-throughput processing; however, its performance is less favourable than the text-based approaches due to the gap between low-level visual features and high-level human expectations. In this chapter, we present the participation from our joint research team of USYD/HES-SO in the VISCERAL retrieval task. Five different methods are introduced, of which two are based on the anatomy–pathology terms, two are based on the visual image content and the last one is based on the fusion of the aforementioned methods. The comparison results, given the different methods indicated that the text-based methods outperformed the content-based retrieval and the fusion of text and visual contents, generated the best performance overall.

VISCERAL :

Book chapter ArODES

evaluation-as-a-service for medical imaging

Allan Hanbury, Henning Müller

Cloud-based benchmarking of medical image analysis (pp. 3-13). 2017, Cham : Springer

Link to the publication

Summary:

Systematic evaluation has had a strong impact on many data analysis domains, for example, TREC and CLEF in information retrieval, ImageCLEF in image retrieval, and many challenges in conferences such as MICCAI for medical imaging and ICPR for pattern recognition. With Kaggle, a platform for machine learning challenges has also had a significant success in crowdsourcing solutions. This shows the importance to systematically evaluate algorithms and that the impact is far larger than simply evaluating a single system. Many of these challenges also showed the limits of the commonly used paradigm to prepare a data collection and tasks, distribute these and then evaluate the participants’ submissions. Extremely large datasets are cumbersome to download, while shipping hard disks containing the data becomes impractical. Confidential data can often not be shared, for example medical data, and also data from company repositories. Real-time data will never be available via static data collections as the data change over time and data preparation often takes much time. The Evaluation-as-a-Service (EaaS) paradigm tries to find solutions for many of these problems and has been applied in the VISCERAL project. In EaaS, the data are not moved but remain on a central infrastructure. In the case of VISCERAL, all data were made available in a cloud environment. Participants were provided with virtual machines on which to install their algorithms. Only a small part of the data, the training data, was visible to participants. The major part of the data, the test data, was only accessible to the organizers who ran the algorithms in the participants’ virtual machines on the test data to obtain impartial performance measures.

Using the cloud as a platform for evaluation and data preparation

Book chapter ArODES

Ivan Eggel, Roger Schaer, Henning Müller

Cloud-based benchmarking of medical image analysis (pp. 15-30). 2017, Cham : Springer

Link to the publication

Summary:

This chapter gives a brief overview of the VISCERAL Registration System that is used for all the VISCERAL Benchmarks and is released as open source on GitHub. The system can be accessed by both participants and administrators, reducing the direct participant–organizer interaction and handling the documentation available for each of the benchmarks organized by VISCERAL. Also, the upload of the VISCERAL usage and participation agreements is integrated, as well as the attribution of virtual machines that allow participation in the VISCERAL Benchmarks. In the second part, a summary of the various steps in the continuous evaluation chain mainly consisting of the submission, algorithm execution and storage as well as the evaluation of results is given. The final part consists of the cloud infrastructure detail, describing the process of defining requirements, selecting a cloud solution provider, setting up the infrastructure and running the benchmarks. This chapter concludes with a short experience report outlining the encountered challenges and lessons learned.

Making sense of large data sets without annotations :

Book chapter ArODES

analyzing age–related correlations from lung CT scans

Yashin Dicente Cid, Artem Mamonov, Andrew Beers, Armin Thomas, Vassili Kovalev, Jayashree Kalpathy-Cramer, Henning Müller

Proceedings Volume 10138, Medical Imaging 2017: Imaging Informatics for Healthcare, Research, and Applications (12 p.). 2017, Orlando, Florida : SPIE medical imaging

Link to the publication

Summary:

The analysis of large data sets can help to gain knowledge about specific organs or on specific diseases, just as big data analysis does in many non-medical areas. This article aims to gain information from 3D volumes, so the visual content of lung CT scans of a large number of patients. In the case of the described data set, only little annotation is available on the patients that were all part of an ongoing screening program and besides age and gender no information on the patient and the findings was available for this work. This is a scenario that can happen regularly as image data sets are produced and become available in increasingly large quantities but manual annotations are often not available and also clinical data such as text reports are often harder to share. We extracted a set of visual features from 12,414 CT scans of 9,348 patients that had CT scans of the lung taken in the context of a national lung screening program in Belarus. Lung fields were segmented by two segmentation algorithms and only cases where both algorithms were able to find left and right lung and had a Dice coefficient above 0.95 were analyzed. This assures that only segmentations of good quality were used to extract features of the lung. Patients ranged in age from 0 to 106 years. Data analysis shows that age can be predicted with a fairly high accuracy for persons under 15 years. Relatively good results were also obtained between 30 and 65 years where a steady trend is seen. For young adults and older people the results are not as good as variability is very high in these groups. Several visualizations of the data show the evolution patters of the lung texture, size and density with age. The experiments allow learning the evolution of the lung and the gained results show that even with limited metadata we can extract interesting information from large-scale visual data. These age-related changes (for example of the lung volume, the density histogram of the tissue) can also be taken into account for the interpretation of new cases. The database used includes patients that had suspicions on a chest X-ray, so it is not a group of healthy people, and only tendencies and not a model of a healthy lung at a specific age can be derived.

3D solid texture classification using locally-oriented wavelet transforms

Scientific paper ArODES

Yashin Dicente Cid, Henning Müller, Alexandra Platon, Pierre–Alexandre Poletti, Adrien Depeursinge

IEEE transactions on image processing, 2017, vol. 26, no. 4, pp. 1899-1910

Link to the publication

Summary:

Many image acquisition techniques used in biomedical imaging, material analysis, and structural geology are capable of acquiring 3D solid images. Computational analysis of these images is complex but necessary, since it is difficult for humans to visualize and quantify their detailed 3D content. One of the most common methods to analyze 3D data is to characterize the volumetric texture patterns. Texture analysis generally consists of encoding the local organization of image scales and directions, which can be extremely diverse in 3D. Current state-of-the-art techniques face many challenges when working with 3D solid texture, where most approaches are not able to consistently characterize both scale and directional information. 3D Riesz-wavelets can deal with both properties. One key property of Riesz filterbanks is steerability, which can be used to locally align the filters and compare textures with arbitrary (local) orientations. This paper proposes and compares three novel local alignment criteria for higher-order 3D Riesz-wavelet transforms. The estimations of local texture orientations are based on higher-order extensions of regularized structure tensors. An experimental evaluation of the proposed methods for the classification of synthetic 3D solid textures with alterations (such as rotations and noise) demonstrated the importance of local directional information for robust and accurate solid texture recognition. These alignment methods achieved an accuracy of 0.95 in the rotated data, three times more than the unaligned Riesz descriptor that achieved 0.32. The accuracy obtained is better than all other techniques that are published and tested on the same database.

Big data in medical imaging, forensics and beyond

Book chapter ArODES

Henning Müller

Forensigraphie : Möglichkeiten und Grenzen IT-gesützter klinisch-forensischer Bildgebung (10 p.). 2017, [S. l.] : J. Sramek

Link to the publication

Elettromiografia, protesica e robotica in rapido progresso verso l'amputazione funzionale :

Scientific paper ArODES

i risultati del progetto Ninapro

Manfredo Atzori, Cesare Tiengo, Franco Bassetto, Henning Müller

Rivista di chirurgia della mano,

Link to the publication

Summary:

Hand amputation can dramatically affect the capabilities of a person. Improving the functionality of robotic prosthetic hands is thus a challenge. The integration of advanced prosthetic and robotic technologies with functional amputations may bring to reality the non-invasive natural control of robotic hand prostheses in a near future. Scientific research and prosthetic market are rapidly advancing towards the natural control of dexterous robotic prosthetic hands. Myoelectric hand prostheses with many degrees of freedom are commercially available and recent advances in scientific research suggest that their natural control can be performed in real life through pattern recognition and the integration of multimodal data. However, robustness is still not sufficient to transfer scientific results to a real life. In this work we describe the Ninapro (Non Invasive Adaptive Prosthetics) database, which is aimed to study the relationships between sEMG, hand movement, force and clinical parameters. The data are publicly available to worldwide research groups. The Ninapro database allowed to obtain several important results including: showing that up to 11 hand movements can be recognized without any training in amputated subject; showing that multimodal data can strongly improve movement recognition; showing that several clinical parameters (including remaining forearm percentage and phantom limb sensation) are related to the capability of amputees to control the remnant muscles in the stump. 20 The Ninapro results, in combination with other scientific literature achievements, suggests that future "functional amputation" surgery procedures may better integrate with the prosthetic robotic limbs and contribute to solve natural control problems.

A lung graph–model for pulmonary hypertension and pulmonary embolism detection on DECT images

Book chapter ArODES

Yashin Dicente Cid, Henning Müller, Alexandra Platon, Jean-Paul Janssens, Frederic Lador, Pierre-Alexandre Poletti, Adrien Depeursinges

Dans Müller, Henning, Medical Computer Vision and Bayesian and Graphical Models for Biomedical Imaging MICCAI 2016 International Workshops, MCV and BAMBI, Athens, Greece, October 21, 2016, Revised Selected Papers (Pp. 58-68). 2017, Cham : Springer

Link to the publication

Summary:

This article presents a novel graph–model approach encoding the relations between the perfusion in several regions of the lung extracted from a geometry–based atlas. Unlike previous approaches that individually analyze regions of the lungs, our method evaluates the entire pulmonary circulatory network for the classification of patients with pulmonary embolism and pulmonary hypertension. An undirected weighted graph with fixed structure is used to encode the network of intensity distributions in Dual Energy Computed Tomography (DECT) images. Results show that the graph–model presented is capable of characterizing a DECT dataset of 30 patients affected with disease and 26 healthy patients, achieving a discrimination accuracy from 0.77 to 0.87 and an AUC between 0.73 and 0.86. This fully automatic graph–model of the lungs constitutes a novel and effective approach for exploring the various patterns of pulmonary perfusion of healthy and diseased patients.

2016

Cloud-based evaluation of anatomical structure segmentation and landmark detection algorithms :

Scientific paper ArODES

VISCERAL anatomy benchmarks

Oscar Alfonso Jiménez del Toro, Henning Müller, Ivan Eggel, Roger Schaer, Yashin Dicente Cid

IEEE transactions on medical imaging, 2016, vol. 35, no 11, pp. 2459-2475

Link to the publication

Summary:

Variations in the shape and appearance of anatomical structures in medical images are often relevant radiological signs of disease. Automatic tools can help automate parts of this manual process. A cloud-based evaluation framework is presented in this paper including results of benchmarking current state-of-the-art medical imaging algorithms for anatomical structure segmentation and landmark detection: the VISCERAL Anatomy benchmarks. The algorithms are implemented in virtual machines in the cloud where participants can only access the training data and can be run privately by the benchmark administrators to objectively compare their performance in an unseen common test set. Overall, 120 computed tomography and magnetic resonance patient volumes were manually annotated to create a standard Gold Corpus containing a total of 1295 structures and 1760 landmarks. Ten participants contributed with automatic algorithms for the organ segmentation task, and three for the landmark localization task. Different algorithms obtained the best scores in the four available imaging modalities and for subsets of anatomical structures. The annotation framework, resulting data set, evaluation setup, results and performance analysis from the three VISCERAL Anatomy benchmarks are presented in this article. Both the VISCERAL data set and Silver Corpus generated with the fusion of the participant algorithms on a larger set of non-manually-annotated medical images are available to the research community.

Reports on CBMI 16 and ICME 16

Scientific paper ArODES

Bogdan Ionescu, Henning Müller, Yiannis Kompatsiaris, Guillaume Gravier, Anthony Vetro

IEEE MultiMedia, 2016, vol. 23, issue 4, pp. 88 - 93

Link to the publication

Summary:

This issue features not just one but two conference reports. The first covers the 14th International Workshop on Content-Based Multimedia Indexing (CBMI 16), while the second covers the 2016 IEEE International Conference on Multimedia and Expo (ICME 2016). For both, find out what hot topics and key themes were discussed, which submissions earned the Best Paper awards, and more.

Medical images modality classification using discrete Bayesian Networks

Scientific paper ArODES

Jesus Martinez-Gomez, Jacinto Arias, Alba García Seco de Herrera, Henning Müller

Computer vision and image understanding, October 2016, vol. 151, pp. 61–71

Link to the publication

Summary:

In this paper we propose a complete pipeline for medical image modality classification focused on the application of discrete Bayesian network classifiers. Modality refers to the categorization of biomedical images from the literature according to a previously defined set of image types, such as X-ray, graph or gene sequence. We describe an extensive pipeline starting with feature extraction from images, data combination, pre-processing and a range of different classification techniques and models. We study the expressive power of several image descriptors along with supervised discretization and feature selection to show the performance of discrete Bayesian networks compared to the usual deterministic classifiers used in image classification. We perform an exhaustive experimentation by using the ImageCLEFmed 2013 collection. This problem presents a high number of classes so we propose several hierarchical approaches. In a first set of experiments we evaluate a wide range of parameters for our pipeline along with several classification models. Finally, we perform a comparison by setting up the competition environment between our selected approaches and the best ones of the original competition. Results show that the Bayesian Network classifiers obtain very competitive results. Furthermore, the proposed approach is stable and it can be applied to other problems that present inherent hierarchical structures of classes.

Deep learning with convolutional neural networks applied to electromyography data :

Scientific paper ArODES

a resource for the classification of movements for prosthetic hands

Manfredo Atzori, Matteo Cognolato, Henning Müller

Frontiers in Neurorobotics, Septembre 2016, vol. 10

Link to the publication

Summary:

Natural control methods based on surface electromyography (sEMG) and pattern recognition are promising for hand prosthetics. However, the control robustness offered by scientific research is still not sufficient for many real life applications, and commercial prostheses are capable of offering natural control for only a few movements. In recent years deep learning revolutionized several fields of machine learning, including computer vision and speech recognition. Our objective is to test its methods for natural control of robotic hands via sEMG using a large number of intact subjects and amputees. We tested convolutional networks for the classification of an average of 50 hand movements in 67 intact subjects and 11 transradial amputees. The simple architecture of the neural network allowed to make several tests in order to evaluate the effect of pre-processing, layer architecture, data augmentation and optimization. The classification results are compared with a set of classical classification methods applied on the same datasets. The classification accuracy obtained with convolutional neural networks using the proposed architecture is higher than the average results obtained with the classical classification methods, but lower than the results obtained with the best reference methods in our tests. The results show that convolutional neural networks with a very simple architecture can produce accurate results comparable to the average classical classification methods. They show that several factors (including pre-processing, the architecture of the net and the optimization parameters) can be fundamental for the analysis of sEMG data. Larger networks can achieve higher accuracy on computer vision and object recognition tasks. This fact suggests that it may be interesting to evaluate if larger networks can increase sEMG classification accuracy too.

A 3-D Riesz-covariance texture model for prediction of nodule recurrence in Lung CT

Scientific paper ArODES

Henning Müller, Adrien Depeursinge, et al.

IEEE Transactions on medical imaging, 2016, vol. 35, issue 12, pp. 2620 - 2630

Link to the publication

Summary:

This paper proposes a novel imaging biomarker of lung cancer relapse from 3–D texture analysis of CT images. Three–dimensional morphological nodular tissue properties are described in terms of 3–D Riesz–wavelets. The responses of the latter are aggregated within nodular regions by means of feature covariances, which leverage rich intra– and inter– variations of the feature space dimensions. When compared to the classical use of the average for feature aggregation, feature covariances preserve spatial co–variations between features. The obtained Riesz–covariance descriptors lie on a manifold governed by Riemannian geometry allowing geodesic measurements and differentiations. The latter property is incorporated both into a kernel for support vector machines (SVM) and a manifold–aware sparse regularized classifier. The effectiveness of the presented models is evaluated on a dataset of 110 patients with non–small cell lung carcinoma (NSCLC) and cancer recurrence information. Disease recurrence within a timeframe of 12 months could be predicted with an accuracy of 81.3–82.7%. The anatomical location of recurrence could be discriminated between local, regional and distant failure with an accuracy of 78.3–93.3%. The obtained results open novel research perspectives by revealing the importance of the nodular regions used to build the predictive models.

Optimized distributed hyperparameter search and simulation for Lung texture classification in CT Usi

Scientific paper ArODES

Roger Schaer, Henning Müller, Adrien Depeursinge

Journal of imaging, 2016, vol. 2, no. 2

Link to the publication

Summary:

Many medical image analysis tasks require complex learning strategies to reach a quality of image-based decision support that is sufficient in clinical practice. The analysis of medical texture in tomographic images, for example of lung tissue, is no exception. Via a learning framework, very good classification accuracy can be obtained, but several parameters need to be optimized. This article describes a practical framework for efficient distributed parameter optimization. The proposed solutions are applicable for many research groups with heterogeneous computing infrastructures and for various machine learning algorithms. These infrastructures can easily be connected via distributed computation frameworks. We use the Hadoop framework to run and distribute both grid and random search strategies for hyperparameter optimization and cross-validations on a cluster of 21 nodes composed of desktop computers and servers. We show that significant speedups of up to 364 compared to a serial execution can be achieved using our in-house Hadoop cluster by distributing the computation and automatically pruning the search space while still identifying the best-performing parameter combinations. To the best of our knowledge, this is the first article presenting practical results in detail for complex data analysis tasks on such a heterogeneous infrastructure together with a linked simulation framework that allows for computing resource planning. The results are directly applicable in many scenarios and allow implementing an efficient and effective strategy for medical (image) data analysis and related learning approaches.

Computational fluid dynamics as a tool to predict the air pollution dispersion in a neighborhood :

Scientific paper ArODES

a research project to improve the quality of life in cities

Gilles Triscone, Nabil Abdennadher, Christophe Balistreri, Olivier Donzé, D. Greco, Patrick Haas, H. Haas-Pekoz, T. Mohamed-Nour, Pierre Munier, Piero Pontelandolfo, Roberto Putzu, Jacques Richard, H. Sthioul, Nicolas Delley, D. Choffat, Elena-Lavinia Niederhäuser, Roger Schaer, Henning Müller, Jean Decaix, Sylvain Richard, Cécile Münch-Alligné, P. Kunz, F. Despot

International Journal of Sustainable Development and Planning, 2016, vol. 11, no. 4, pp. 546-557

Link to the publication

Summary:

In large cities, pollution composed of many different chemical components and small particles is an important public health problem that affects especially children and people presenting breathing difficulties. One challenge for public authorities is to respect the norms given by the central state, but how? Today, concrete methods for reducing pollution are perceived by the majority of citizens as constraints. However, the authorities have the possibility of modifying the wind’s action by imposing architectural constraints, such as building emplacement and roof structure. This is the main objective of the Geneva ‘Clean City’ project financed by the University of Applied Sciences Western Switzerland. ‘Clean City’ focuses its research on one of Geneva’s polluted neighborhood called Pâquis, which is situated directly on the Geneva lake front. The project attempts to understand the dispersion of pollution from an experimental and a numerical point of view. After validation of the technique for a simple case, we compare environmental measurements on a 1/500 3D scale model of the Pâquis installed in an instrumented wind tunnel with Computational Fluid Dynamics (CFD) simulation obtained with the help of cloud computing.

Forschungsanwendungen in der digitalen Radiologie

Scientific paper ArODES

Henning Müller, Allan Hanbury

Der Radiologe, Februar 2016, Bd. 56, Nr. 2, pp. 176–180

Link to the publication

Summary:

Medizinische Bildgebung produziert immer komplexere Bilder (höhere Auflösung, dünnere Schichten) mit einer zunehmenden Zahl unterschiedlicher Protokolle, sodass die Befundung schwieriger wird und mehr Informationen verarbeitet werden müssen, da die Zahl der Radiologen bei weitem nicht im selben Umfang wächst wie die zu analysierenden Daten. Ziel dieses Artikels ist es, aktuelle Forschungsergebnisse von Projekten vorzustellen, die medizinische Bilddaten zur Entscheidungsunterstützung benutzten. Eine Infrastruktur, die es Forschern erlaubt, Bilder zu analysieren und beste Algorithmen zu erkennen, ohne dass die Daten einen sicheren Server verlassen müssen, wird auch erklärt. Der Text stellt Resultate der EU-finanzierten Khresmoi- und VISCERAL-Projekte vor. Diese erlauben es, Daten vergangener Fälle in einem Bildarchiv zu benutzten, um Entscheidungen zu unterstützen und Aufgaben zu automatisieren. Die Resultate stellen eine Evaluationsumgebung für medizinische Bilddaten und andere Datensätze vor. Diese erlauben es, in einer sicheren Umgebung aus vergangenen Fällen, Daten für zukünftige Befundungen zu extrahieren. Die vorgestellten Prototypen erlauben es, aus Bilddaten direkt Information zu extrahieren und als Entscheidungsunterstützung zu benutzten. Die Forschungsprototypen sind allerdings noch nicht in klinischem Einsatz evaluiert worden, nur subjektive Benutzertests wurden unternommen, welche sehr positiv waren. Die Zukunft der Radiologie wird sicherlich z. T. davon abhängen, Prozesse zu automatisieren und Daten („big data“) aus institutionellen Bildarchiven zu benutzen, um bestmöglich Wissen zu extrahieren und damit die Arbeit der Radiologen auf die wichtigen, entscheidungsrelevanten Bereiche zu konzentrieren, und Routineaufgaben zu automatisieren, wo dies möglich ist.

Effect of clinical parameters on the control of myoelectric robotic prosthetic hands

Scientific paper ArODES

Manfredo Atzori, Henning Müller, et al.

Journal of rehabilitation research and development, 2016, vol. 53, no. 3, pp. 345-358

Link to the publication

Summary:

Improving the functionality of prosthetic hands with noninvasive techniques is still a challenge. Surface electromyography (sEMG) currently gives limited control capabilities; however, the application of machine learning to the analysis of sEMG signals is promising and has recently been applied in practice, but many questions still remain. In this study, we recorded the sEMG activity of the forearm of 11 male subjects with transradial amputation who were mentally performing 40 hand and wrist movements. The classification performance and the number of independent movements (defined as the subset of movements that could be distinguished with >90% accuracy) were studied in relationship to clinical parameters related to the amputation. The analysis showed that classification accuracy and the number of independent movements increased significantly with phantom limb sensation intensity, remaining forearm percentage, and time since amputation. The classification results suggest the possibility of naturally controlling up to 11 movements of a robotic prosthetic hand with almost no training. Knowledge of the relationship between classification accuracy and clinical parameters adds new information regarding the nature of phantom limb pain as well as other clinical parameters, and it can lay the foundations for future "functional amputation" procedures in surgery.

Medical information retrieval :

Scientific paper ArODES

introduction to the special issue

Henning Müller, et al.

Journal of information retrieval, April 2016, vol. 19, no. 1, pp. 1–5

Link to the publication

2015

Control capabilities of myoelectric robotic prostheses by hand amputees :

Scientific paper ArODES

a scientific research and market overview

Manfredo Atzori, Henning Müller

Frontiers in systems neuroscience,

Link to the publication

Summary:

Hand amputation can dramatically affect the capabilities of a person. Cortical reorganization occurs in the brain, but the motor and somatosensorial cortex can interact with the remnant muscles of the missing hand even many years after the amputation, leading to the possibility to restore the capabilities of hand amputees through myoelectric prostheses. Myoelectric hand prostheses with many degrees of freedom are commercially available and recent advances in rehabilitation robotics suggest that their natural control can be performed in real life. The first commercial products exploiting pattern recognition to recognize the movements have recently been released, however the most common control systems are still usually unnatural and must be learned through long training. Dexterous and naturally controlled robotic prostheses can become reality in the everyday life of amputees but the path still requires many steps. This mini-review aims to improve the situation by giving an overview of the advancements in the commercial and scientific domains in order to outline the current and future chances in this field and to foster the integration between market and scientific research.

Analyzing image search behaviour of radiologists :

Scientific paper ArODES

semantics and prediction of query results

Maria De-Arteaga, Ivan Eggel, Charles Kahn, Henning Müller

Journal of digital imaging, October 2015, vol. 28, Issue 5, pp. 537-546

Link to the publication

Summary:

Log files of information retrieval systems that record user behavior have been used to improve the outcomes of retrieval systems, understand user behavior, and predict events. In this article, a log file of the ARRS GoldMiner search engine containing 222,005 consecutive queries is analyzed. Time stamps are available for each query, as well as masked IP addresses, which enables to identify queries from the same person. This article describes the ways in which physicians (or Internet searchers interested in medical images) search and proposes potential improvements by suggesting query modifications. For example, many queries contain only few terms and therefore are not specific; others contain spelling mistakes or non-medical terms that likely lead to poor or empty results. One of the goals of this report is to predict the number of results a query will have since such a model allows search engines to automatically propose query modifications in order to avoid result lists that are empty or too large. This prediction is made based on characteristics of the query terms themselves. Prediction of empty results has an accuracy above 88 %, and thus can be used to automatically modify the query to avoid empty result sets for a user. The semantic analysis and data of reformulations done by users in the past can aid the development of better search systems, particularly to improve results for novice users. Therefore, this paper gives important ideas to better understand how people search and how to use this knowledge to improve the performance of specialized medical search engines.

User-oriented evaluation of a medical image retrieval system for radiologists

Scientific paper ArODES

Dimitrios Markonis, Markus Holzer, Frederic Baroz, Rafael Luis Ruiz De Castaneda, Célia Boyer, Georg Langs, Henning Müller

International journal of medical informatics, October 2015, vol. 84, no. 10, pp. 774-783

Link to the publication

Summary:

PURPOSE: This article reports the user-oriented evaluation of a text- and content-based medical image retrieval system. User tests with radiologists using a search system for images in the medical literature are presented. The goal of the tests is to assess the usability of the system, identify system and interface aspects that need improvement and useful additions. Another objective is to investigate the system's added value to radiology information retrieval. The study provides an insight into required specifications and potential shortcomings of medical image retrieval systems through a concrete methodology for conducting user tests. METHODS: User tests with a working image retrieval system of images from the biomedical literature were performed in an iterative manner, where each iteration had the participants perform radiology information seeking tasks and then refining the system as well as the user study design itself. During these tasks the interaction of the users with the system was monitored, usability aspects were measured, retrieval success rates recorded and feedback was collected through survey forms. RESULTS: In total, 16 radiologists participated in the user tests. The success rates in finding relevant information were on average 87% and 78% for image and case retrieval tasks, respectively. The average time for a successful search was below 3 min in both cases. Users felt quickly comfortable with the novel techniques and tools (after 5 to 15 min), such as content-based image retrieval and relevance feedback. User satisfaction measures show a very positive attitude toward the system's functionalities while the user feedback helped identifying the system's weak points. The participants proposed several potentially useful new functionalities, such as filtering by imaging modality and search for articles using image examples. CONCLUSION: The iterative character of the evaluation helped to obtain diverse and detailed feedback on all system aspects. Radiologists are quickly familiar with the functionalities but have several comments on desired functionalities. The analysis of the results can potentially assist system refinement for future medical information retrieval systems. Moreover, the methodology presented as well as the discussion on the limitations and challenges of such studies can be useful for user-oriented medical image retrieval evaluation, as user-oriented evaluation of interactive system is still only rarely performed. Such interactive evaluations can be limited in effort if done iteratively and can give many insights for developing better systems.

Using MapReduce for large-scale medical image analysis

Scientific paper ArODES

Dimitrios Markonis, Roger Schaer, Ivan Eggel, Henning Müller, Adrien Depeursinge

ArXiv,

Link to the publication

Summary:

The growth of the amount of medical image data produced on a daily basis in modern hospitals forces the adaptation of traditional medical image analysis and indexing approaches towards scalable solutions. The number of images and their dimensionality increased dramatically during the past 20 years. We propose solutions for large-scale medical image analysis based on parallel computing and algorithm optimization. The MapReduce framework is used to speed up and make possible three large-scale medical image processing use-cases: (i) parameter optimization for lung texture segmentation using support vector machines, (ii) content-based medical image indexing, and (iii) three-dimensional directional wavelet analysis for solid texture classification. A cluster of heterogeneous computing nodes was set up in our institution using Hadoop allowing for a maximum of 42 concurrent map tasks. The majority of the machines used are desktop computers that are also used for regular office work. The cluster showed to be minimally invasive and stable. The runtimes of each of the three use-case have been significantly reduced when compared to a sequential execution. Hadoop provides an easy-to-employ framework for data analysis tasks that scales well for many tasks but requires optimization for specific tasks.

Comparing image search behaviour in the ARRS GoldMiner search engine and a clinical PACS/RIS

Scientific paper ArODES

Henning Müller, Maria De-Arteaga, Ivan Eggel, Bao Do, Daniel Rubin, Charles Kahn Jr.

Journal of Biomedical Informatics, 2015, vol. 56, pp. 57-64

Link to the publication

Summary:

Information search has changed the way we manage knowledge and the ubiquity of information access has made search a frequent activity, whether via Internet search engines or increasingly via mobile devices. Medical information search is in this respect no different and much research has been devoted to analyzing the way in which physicians aim to access information. Medical image search is a much smaller domain but has gained much attention as it has different characteristics than search for text documents. While web search log files have been analysed many times to better understand user behaviour, the log files of hospital internal systems for search in a PACS/RIS (Picture Archival and Communication System, Radiology Information System) have rarely been analysed. Such a comparison between a hospital PACS/RIS search and a web system for searching images of the biomedical literature is the goal of this paper. Objectives are to identify similarities and differences in search behaviour of the two systems, which could then be used to optimize existing systems and build new search engines. Log files of the ARRS GoldMiner medical image search engine (freely accessible on the Internet) containing 222,005 queries, and log files of Stanford’s internal PACS/RIS search called radTF containing 18,068 queries were analysed. Each query was preprocessed and all query terms were mapped to the RadLex (Radiology Lexicon) terminology, a comprehensive lexicon of radiology terms created and maintained by the Radiological Society of North America, so the semantic content in the queries and the links between terms could be analysed, and synonyms for the same concept could be detected. RadLex was mainly created for the use in radiology reports, to aid structured reporting and the preparation of educational material (Lanlotz, 2006) [1]. In standard medical vocabularies such as MeSH (Medical Subject Headings) and UMLS (Unified Medical Language System) specific terms of radiology are often underrepresented, therefore RadLex was considered to be the best option for this task. The results show a surprising similarity between the usage behaviour in the two systems, but several subtle differences can also be noted. The average number of terms per query is 2.21 for GoldMiner and 2.07 for radTF, the used axes of RadLex (anatomy, pathology, findings, . . .) have almost the same distribution with clinical findings being the most frequent and the anatomical entity the second; also, combinations of RadLex axes are extremely similar between the two systems. Differences include a longer length of the sessions in radTF than in GoldMiner (3.4 and 1.9 queries per session on average). Several frequent search terms overlap but some strong differences exist in the details. In radTF the term ‘‘normal’’ is frequent, whereas in GoldMiner it is not. This makes intuitive sense, as in the literature normal cases are rarely described whereas in clinical work the comparison with normal cases is often a first step. The general similarity in many points is likely due to the fact that users of the two systems are influenced by their daily behaviour in using standard web search engines and follow this behaviour in their professional search. This means that many results and insights gained from standard web search can likely be transferred to more specialized search systems. Still, specialized log files can be used to find out more on reformulations and detailed strategies of users to find the right content.

Medical image retrieval :

Scientific paper ArODES

a multi-modal approach

Yu Cao, Shawn Steffey, Jianbiao He, Degui Xiao, Cui Tao, Ping Chen, Henning Müller

Cancer informatics, 2014, suppl. 3, pp. 125-136

Link to the publication

Summary:

Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.

A review of medical grids and their direction :

Scientific paper ArODES

a Swiss/Japanese perspective

Toshiharu Nakai, Henning Müller, Epifanio Bagarinao, Koji Tomida, Shiraishi Yoshiaki, Marko Niinimaki

International journal of research studies in computing, 2015, vol. 4, no. 1

Link to the publication

Summary:

This paper presents a general and comparative review of the advances of grid technologies in medical sciences since 2000, with a special emphasis on Europe and Japan. The EU has over the years funded several big projects in the Grid and now the cloud area, covering besides the high energy physics domain also bioinformatics and medical applications as in image analysis. Bioinformatics and pharmaceutical design have been the major targets of grid applications in Japan. Grids and clouds share many of the same goals and techniques, although clouds are more commercially oriented and privately provided services and grids are rather publicly initiated resource sharing platforms between institutions without strong economic objectives. In the future, merging these two may potentially solve inter-operation problems of grids, which have been limiting the propagation of biomedical grids.

Evaluating performance of biomedical image retrieval systems :

Scientific paper ArODES

an overview of the medical image retrieval task at ImageCLEF 2004-2013

Jayashree Kalpathy-Cramer, Alba García Seco de Herrera, Dina Demner-Fushman, Sameer Antani, Steven Bedrick, Henning Müller

Computerized medical imaging and graphics, January 2015, vol. 39. no. 55-61, 15 p.

Link to the publication

Summary:

Medical image retrieval and classification have been extremely active research topics over the past 15 years. With the ImageCLEF benchmark in medical image retrieval and classification a standard test bed was created that allows researchers to compare their approaches and ideas on increasingly large and varied data sets including generated ground truth. This article describes the lessons learned in ten evaluations campaigns. A detailed analysis of the data also highlights the value of the resources created.

Meaningful bags of words for medical image classification and retrieval

Book chapter ArODES

Antonio Foncubierta-Rodríguez, Alba García Seco de Herrera, Henning Müller

Health monitoring and personalized feedback using multimedia data (pp. 73-93). 2015, [s.l] : Springer

Link to the publication

Summary:

Content–based medical image retrieval has been proposed as a technique that allows not only for easy access to images from the relevant literature and electronic health records but also for training physicians, for research and clinical decision support. The bag–of–visual–words approach is a widely used technique that tries to shorten the semantic gap by learning meaningful features from the dataset and describing documents and images in terms of the histogram of these features. Visual vocabularies are often redundant, over–complete and noisy. Larger than required vocabularies lead to high–dimensional feature spaces, which present important disadvantages with the curse of dimensionality and computational cost being the most obviousones. In this work a visual vocabulary pruning and descriptor transformation technique is presented. It enormously reduces the amount of required words to describe a medical image dataset with no significant effect on the accuracy. Results show that a reduction of up to 90% can be achieved without impact on the system performance. Obtaining a more compact representation of a document enables multimodal description as well as using classifiers requiring low–dimensional representations.

Evaluating multimodal relevance feedback techniques for medical image retrieval

Scientific paper ArODES

Dimitrios Markonis, Roger Schaer, Henning Müller

Journal of information retrieval, Janvier 2015

Link to the publication

Summary:

Medical image retrieval can assist physicians in finding information supporting their diagnosis and fulfilling information needs. Systems that allow searching for medical images need to provide tools for quick and easy navigation and query refinement as the time available for information search is often short. Relevance feedback is a powerful tool in information retrieval. This study evaluates relevance feedback techniques with regard to the content they use. A novel relevance feedback technique that uses both text and visual information of the results is proposed. The two information modalities from the image examples are fused either at the feature level using the Rocchio algorithm or at the query list fusion step using a common late fusion rule. Results using the ImageCLEF 2012 benchmark database for medical image retrieval show the potential of relevance feedback techniques in medical image retrieval. The mean average precision (mAP) is used as the evaluation metric and the proposed method outperforms commonly-used methods. The baseline without feedback reached 16 % whereas the relevance feedback with 20 images reached up to 26.35 % with three steps and when using 100 images up to 34.87 % in four steps. Most improvements occur in the first two steps of relevance feedback and then results start to become relatively flat. This might also be due to only using positive feedback as negative feeback often also improves results after more steps. The effect of relevance feedback in automatically spelling corrected and translated queries is investigated as well. Results without mistakes were better than spell-corrected results but the spelling correction more than double results over non-corrected retrieval. Multimodal relevance feedback has shown to be able to help visual medical information retrieval. Next steps include integrating semantics into relevance feedback techniques to benefit from the structured knowledge of ontologies and experimenting on the fusion of text and visual information.

How users search and what they search for in the medical domain :

Scientific paper ArODES

understanding laypeople and experts through query logs

João Palotti, Allan Hanbury, Henning Müller, Charles Kahn

Journal of information retrieval, Janvier 2015

Link to the publication

Summary:

The internet is an important source of medical knowledge for everyone, from laypeople to medical professionals. We investigate how these two extremes, in terms of user groups, have distinct needs and exhibit significantly different search behaviour. We make use of query logs in order to study various aspects of these two kinds of users. The logs from America On- line (AOL), Health on the Net (HON), Turning Re- search Into Practice (TRIP) and American Roentgen Ray Society (ARRS) GoldMiner were divided into three sets: (1) laypeople, (2) medical professionals (such as physicians or nurses) searching for health content and (3) users not seeking health advice. Several analyses are made focusing on discovering how users search and what they are most interested in. One possible outcome of our analysis is a classifer to infer user expertise, which was built. We show the results and analyse the feature set used to infer expertise. We conclude that medical experts are more persistent, interacting more with the search engine. Also, our study reveals that, conversely to what is stated in much of the literature, the main focus of users, both laypeople and professionals, is on disease rather than symptoms. The results of this article, especially through the classifer built, could be used to detect specifc user groups and then adapt search results to the user group.

Medical visual information analysis and retrieval :

Scientific paper ArODES

special issue

Henning Müller, Jayashree Kalpathy-Cramer, Dina Demner-Fushman, Sameer Antani

Medical visual information analysis and retrieval, 2015, vol. 39, special issue, pp. 1-62

Link to the publication

A decade of community-wide efforts in advancing medical image understanding and retrieval

Scientific paper ArODES

Henning Müller, Dina Demner-Fushman, Sameer Antani, Jayashree Kalpathy-Cramer

Computerized Medical Imaging and Graphics : special issue, 2015, vol. 39, special issue, pp. 1-2

Link to the publication

Summary:

Access to images in scientific publications overall and in the biomedical domain specifically, has long been established a desirable feature for services providing access to scientific literature [2,6]. Likewise, automatic identification of regions of interest in clinical images and image-based assisted diagnosis are widely recognized as a means for bringing potentially important health issues to clinicians’ attention [11]. The individual research efforts in addressing these needs were faced with the lack of resources for development and evaluation of image search engines and tools for understanding image content.

Comparing fusion techniques for the ImageCLEF 2013 medical case retrieval task

Scientific paper ArODES

Henning Müller, Alba García Seco de Herrera, Roger Schaer, Dimitrios Markonis

Computerized Medical Imaging and Graphics : special issue, 2015, vol. 39, special issue, pp. 46-54

Link to the publication

Summary:

Retrieval systems can supply similar cases with a proven diagnosis to a new example case under observation to help clinicians during their work. The ImageCLEFmed evaluation campaign proposes a framework where research groups can compare case-based retrieval approaches. This paper focuses on the case-based task and adds results of the compound figure separation and modality classification tasks. Several fusion approaches are compared to identify the approaches best adapted to the heterogeneous data of the task. Fusion of visual and textual features is analyzed, demonstrating that the selection of the fusion strategy can improve the best performance on the case-based retrieval task.

Result diversification in social image retrieval :

Scientific paper ArODES

a benchmarking framework

Bogdan Ionescu, Adrian Popescu, Anca-Livia Radu, Henning Müller

Multimedia tools and applications, octobre 2015, vol. 74, no 19

Link to the publication

Summary:

This article addresses the diversification of image retrieval results in the context of image retrieval from social media. It proposes a benchmarking framework together with an annotated dataset and discusses the results achieved during the related task run in the MediaEval 2013 benchmark. 38 multimedia diversification systems, varying from graph-based representations, re-ranking, optimization approaches, data clustering to hybrid approaches that included a human in the loop, and their results are described and analyzed in this text. A comparison of the use of expert vs. crowdsourcing annotations shows that crowdsourcing results have a slightly lower inter-rater agreement but results are comparable at a much lower cost than expert annotators. Multimodal approaches have best results in terms of cluster recall. Manual approaches can lead to high precision but often lower diversity. With this detailed results analysis we give future insights into diversity in image retrieval and also for preparing new evaluation campaigns in related areas.

2014

Electromyography data for non-invasive naturally controlled robotic hand prostheses

Scientific paper ArODES

Manfredo Atzori, Arjan Gijsberts, Claudio Castellini, Anne-Gabrielle Mittaz Hager, Barbara Caputo, Simone Elsig, Giorgio Giatsidis, Franco Bassetto, Henning Müller

Scientific data, vol. 1, no. 140053, pp. 2-13

Link to the publication

Summary:

Recent advances in rehabilitation robotics suggest that it may be possible for hand-amputated subjects to recover at least a significant part of the lost hand functionality. The control of robotic prosthetic hands using non-invasive techniques is still a challenge in real life: myoelectric prostheses give limited control capabilities, the control is often unnatural and must be learned through long training times. Meanwhile, scientific literature results are promising but they are still far from fulfilling real-life needs. This work aims to close this gap by allowing worldwide research groups to develop and test movement recognition and force control algorithms on a benchmark scientific database. The database is targeted at studying the relationship between surface electromyography, hand kinematics and hand forces, with the final goal of developing non-invasive, naturally controlled, robotic hand prostheses. The validation section verifies that the data are similar to data acquired in real-life conditions, and that recognition of different hand tasks by applying state-of-the-art signal features and machine-learning algorithms is possible.

Electromyography low pass filtering effects on the classification of hand movements in amputated subjects

Scientific paper ArODES

Manfredo Atzori, Henning Müller

Scientific data, Vol. 1, no. 140053, pp. 1-13

Link to the publication

Summary:

People with transradial hand amputations can have control capabilities of prosthetic hands via surface electromyography (sEMG) but the control systems are limited and usually not natural. In the scientific literature, the application of pattern recognition techniques to classify hand movements in sEMG led to remarkable results but the evaluations are usually far from real life applications with all uncertainties and noise. Therefore, there is a need to improve the movement classification accuracy in real settings. Smoothing the signal with a low pass filter is a common pre– processing procedure to remove high–frequency noise. However, the filtering frequency modifies the signal strongly and can therefore affect the classification results. In this paper we analyze the dependence of the classification accuracy on the pre–processing low–pass filtering frequency in 3 hand amputated subjects performing 50 different movements. The results highlight two main interesting aspects. First, the filtering frequency strongly affects the classification accuracy, and choosing the right frequency between 1Hz–5Hz can improve the accuracy up to 5%. Second, different subjects obtain the best classification performance at different frequencies. Theoretically these facts could affect all the similar classification procedures re- ducing the classification uncertainity. Therefore, they contribute to set the field closer to real life applications, which could deeply change the life of hand amputated subjects.

L’échange des données de santé entre hôpitaux et acteurs de la santé :

Professional paper ArODES

le projet MediCoordination

Henning Müller, Sandrine Ding, Bruno Alves, David Godel, Omar Abou Khaled, Francois Mooser, Michael Schumacher

ASTRM actuel (Association Suisse es techniciens en radiologie médicales) - SVMTRA aktuell (Schweizerische Vereinigung der Fachleute für medizinisch technische Radiologie) - ASTRM attualità (Associazione Svizzera dei Tecnici di Radiologia Medica), Octobre 2014, pp. 18-23

Link to the publication

Summary:

Le projet MediCoordination a permis de contribuer à la stratégie e-Health. L’e-Health est en plein essor en Suisse même si un gros effort devra encore être fourni en termes d’interopérabilité et de standardisation dans la mesure où le dossier patient informatisé est alimenté par des informations provenant d’un nombre considérable de services différents avec des standards tout aussi divers.

Comparing fusion techniques for the imageCLEF 2013 medical case retrieval task

Scientific paper ArODES

Alba García Seco de Herrera, Roger Schaer, Dimitrios Markonis, Henning Müller

Computerized Medical Imaging and Graphics, Vol. 38, Issue 7, pp. 46-54

Link to the publication

Summary:

Retrieval systems can supply similar cases with a proven diagnosis to a new example case under observation to help clinicians during their work. The ImageCLEFmed evaluation campaign proposes a framework where research groups can compare case{based retrieval approaches. This paper focuses on the case{based task and adds results of the compound gure separation and modality classication tasks. Several fusion approaches are compared to identify the approaches best adapted to the heterogeneous data of the task. Fusion of visual and textual features is analyzed, demonstrating that the selection of the fusion strategy can improve the best performance on the case{based retrieval task.

The movement error rate for evaluation of machine learning methods for sEMG-based hand movement classification

Scientific paper ArODES

Arjan Gijsberts, Manfredo Atzori, Claudio Castellini, Henning Müller, Barbara Caputo

IEEE Transactions on neural systems and rehabiliation engineering, juillet 2014, vol. 22, issue 4, pp. 735-744

Link to the publication

Summary:

There has been increasing interest in applying learning algorithms to improve the dexterity of myoelectric prostheses. In this work, we present a large-scale benchmark evaluation on the second iteration of the publicly released NinaPro database, which contains surface electromyography data for 6 DOF force activations as well as for 40 discrete hand movements. The evaluation involves a modern kernel method and compares performance of three feature representations and three kernel functions. Both the force regression and movement classification problems can be learned successfully when using a non-linear kernel function, while the exp-χ2 kernel outperforms the more popular Radial Basis Function kernel in all cases. Furthermore, combining surface electromyography and accelerometry in a multimodal classifier results in significant increases in accuracy as compared to when either modality is used individually. Since window-based classification accuracy should not be considered in isolation to estimate prosthetic controllability, we also provide results in terms of classification mistakes and prediction delay. To this extent, we propose the Movement Error Rate as an alternative to the standard window-based accuracy. This error rate is insensitive to prediction delays and it allows therefore to quantify mistakes and delays as independent performance characteristics. This type of analysis confirms that the inclusion of accelerometry is superior, as it results in fewer mistakes while at the same time reducing prediction delay.

2009

Le projet de développement de l’interopérabilité entre hôpitaux et fournisseurs de soins externes :

Professional paper ArODES

MediCoordination

Sandrine Ding, Henning Müller, Bruno Alvesa, David Godel, Omar Abou Khaled, François Mooser, Michael Schumacher

Clinicum Romandie, 2009, vol. 1, p. 1-3

Link to the publication

2022

Attention-based interpretable regression of gene expression in histology

Conference ArODES

Mara Graziani, Niccolò Marini, Nicolas Deutschmann, Nikita Janakarajan, Henning Müller, María Rodríguez Martínez

Proceedings of the Workshop on the Interpretability of Machine Intelligence in Medical Image Computing (iMIMIC) at MICCAI 2022