Recherche

Müller Henning

Professeur-e HES Ordinaire, Responsable de l'unité eHealth

Contrat principal

Professeur-e HES Ordinaire, Responsable de l'unité eHealth

Téléphone: +41 58 606 90 36

Bureau: TP108

HES-SO Valais-Wallis - Haute Ecole de Gestion
Route de la Plaine 2, Case postale 80, 3960 Sierre, CH

Domaine
Economie et services

Filière principale
Informatique de gestion

MSc HES-SO en Business Administration - HES-SO Master

Méthodologie de recherche

BSc HES-SO en Informatique de gestion - HES-SO Valais-Wallis - Haute Ecole de Gestion

Software Lifecycle Management

En cours

H2020 EXAMODE-Valorisation

Rôle: Requérant(e) principal(e)

Description du projet : EXAMODE-Valorisation

Equipe de recherche au sein de la HES-SO: Müller Henning

Partenaires académiques: VS - Institut Informatique de gestion

Durée du projet: 29.04.2019 - 31.12.2024

Statut: En cours

Terminés

ProHand (3D printed prosthetic hand tests on Cybathlon tasks

Rôle: Requérant(e) principal(e)

Financement: Fondation Hasler

Description du projet : ProHand (3D printed prosthetic hand tests on Cybathlon tasks Résumé à compléter

Equipe de recherche au sein de la HES-SO: Atzori Manfredo, Müller Henning, Reggiani Franco, Buosi Alessandro

Partenaires académiques: VS - Institut Informatique de gestion

Durée du projet: 08.07.2019 - 24.05.2021

Montant global du projet: 99'180 CHF

Statut: Terminé

2021

An AI-based algorithm for the automatic classification of thoracic radiographs in cats

Scientifique ArODES

Banzato Tommaso, Wodzinski Marek, Tauceri Federico, Donà Chiara, Scavazza Filippo, Müller Henning, Zotti Alessandro

Frontiers in veterinary science, 2021, vol. 8, article no. 731936, pp. 1-7

Lien vers la publication

Résumé:

An artificial intelligence (AI)-based computer-aided detection (CAD) algorithm to detect some of the most common radiographic findings in the feline thorax was developed and tested. The database used for training comprised radiographs acquired at two different institutions. Only correctly exposed and positioned radiographs were included in the database used for training. The presence of several radiographic findings was recorded. Consequenly, the radiographic findings included for training were: no findings, bronchial pattern, pleural effusion,mass, alveolar pattern, pneumothorax, cardiomegaly.Multi-label convolutional neural networks (CNNs) were used to develop the CAD algorithm, and the performance of two different CNN architectures, ResNet 50 and Inception V3, was compared. Both architectures had an area under the receiver operating characteristic curve (AUC) above 0.9 for alveolar pattern, bronchial pattern and pleural effusion, an AUC above 0.8 for no findings and pneumothorax, and an AUC above 0.7 for cardiomegaly. The AUC for mass was low (above 0.5) for both architectures. No significant differences were evident in the diagnostic accuracy of either architecture.

Making radiomics more reproducible across scanner and imaging protocol variations :

Scientifique ArODES

a review of harmonization methods

Mali Shruti Atul, Ibrahim Abdalla, Woodruff Henry C., Andrearczyk Vincent, Müller Henning, Primakov Sergey, Salahuddin Zohaib, Chatterjee Avishek, Lambin Philippe

Journal of personalized medicine, 2021, vol. 11, no. 9, article no. 842, pp. 1-30

Lien vers la publication

Résumé:

Radiomics converts medical images into mineable data via a high-throughput extraction of quantitative features used for clinical decision support. However, these radiomic features are susceptible to variation across scanners, acquisition protocols, and reconstruction settings. Various investigations have assessed the reproducibility and validation of radiomic features across these discrepancies. In this narrative review, we combine systematic keyword searches with prior domain knowledge to discuss various harmonization solutions to make the radiomic features more reproducible across various scanners and protocol settings. Different harmonization solutions are discussed and divided into two main categories: image domain and feature domain. The image domain category comprises methods such as the standardization of image acquisition, post-processing of raw sensor-level image data, data augmentation techniques, and style transfer. The feature domain category consists of methods such as the identification of reproducible features and normalization techniques such as statistical normalization, intensity harmonization, ComBat and its derivatives, and normalization using deep learning. We also reflect upon the importance of deep learning solutions for addressing variability across multi-centric radiomic studies especially using generative adversarial networks (GANs), neural style transfer (NST) techniques, or a combination of both. We cover a broader range of methods especially GANs and NST methods in more detail than previous reviews.

Multi_Scale_Tools :

Scientifique ArODES

a Python library to exploit multi-scale whole slide images

Marini Niccolò, Otálora Sebastian, Podareanu Damian, Rijthoven Mart van, Laak Jeroen van der, Ciompi Francesco, Müller Henning, Atzori Manfredo

Frontiers in computer science, 2021, vol. 3, article no. 684521, pp. 1-12

Lien vers la publication

Résumé:

Algorithms proposed in computational pathology can allow to automatically analyze digitized tissue samples of histopathological images to help diagnosing diseases. Tissue samples are scanned at a high-resolution and usually saved as images with several magnification levels, namely whole slide images (WSIs). Convolutional neural networks (CNNs) represent the state-of-the-art computer vision methods targeting the analysis of histopathology images, aiming for detection, classification and segmentation. However, the development of CNNs that work with multi-scale images such as WSIs is still an open challenge. The image characteristics and the CNN properties impose architecture designs that are not trivial. Therefore, single scale CNN architectures are still often used. This paper presents Multi_Scale_Tools, a library aiming to facilitate exploiting the multi-scale structure of WSIs. Multi_Scale_Tools currently include four components: a pre-processing component, a scale detector, a multi-scale CNN for classification and a multi-scale CNN for segmentation of the images. The pre-processing component includes methods to extract patches at several magnification levels. The scale detector allows to identify the magnification level of images that do not contain this information, such as images from the scientific literature. The multi-scale CNNs are trained combining features and predictions that originate from different magnification levels. The components are developed using private datasets, including colon and breast cancer tissue samples. They are tested on private and public external data sources, such as The Cancer Genome Atlas (TCGA). The results of the library demonstrate its effectiveness and applicability. The scale detector accurately predicts multiple levels of image magnification and generalizes well to independent external data. The multi-scale CNNs outperform the single-magnification CNN for both classification and segmentation tasks. The code is developed in Python and it will be made publicly available upon publication. It aims to be easy to use and easy to be improved with additional functions.

Semi-supervised training of deep convolutional neural networks with heterogeneous data and few local annotations :

Scientifique ArODES

an experiment on prostate histopathology image classification

Marini Niccolò, Otálora Sebastian, Müller Henning, Atzori Manfredo

Medical image analysis, 2021, vol. 73, article no. 102165, pp. 1-16

Lien vers la publication

Résumé:

Convolutional neural networks (CNNs) are state-of-the-art computer vision techniques for various tasks, particularly for image classification. However, there are domains where the training of classification models that generalize on several datasets is still an open challenge because of the highly heterogeneous data and the lack of large datasets with local annotations of the regions of interest, such as histopathology image analysis. Histopathology concerns the microscopic analysis of tissue specimens processed in glass slides to identify diseases such as cancer. Digital pathology concerns the acquisition, management and automatic analysis of digitized histopathology images that are large, having in the order of 100′0002 pixels per image. Digital histopathology images are highly heterogeneous due to the variability of the image acquisition procedures. Creating locally labeled regions (required for the training) is time-consuming and often expensive in the medical field, as physicians usually have to annotate the data. Despite the advances in deep learning, leveraging strongly and weakly annotated datasets to train classification models is still an unsolved problem, mainly when data are very heterogeneous. Large amounts of data are needed to create models that generalize well. This paper presents a novel approach to train CNNs that generalize to heterogeneous datasets originating from various sources and without local annotations. The data analysis pipeline targets Gleason grading on prostate images and includes two models in sequence, following a teacher/student training paradigm. The teacher model (a high-capacity neural network) automatically annotates a set of pseudo-labeled patches used to train the student model (a smaller network). The two models are trained with two different teacher/student approaches: semi-supervised learning and semi-weekly supervised learning. For each of the two approaches, three student training variants are presented. The baseline is provided by training the student model only with the strongly annotated data. Classification performance is evaluated on the student model at the patch level (using the local annotations of the Tissue Micro-Arrays Zurich dataset) and at the global level (using the TCGA-PRAD, The Cancer Genome Atlas-PRostate ADenocarcinoma, whole slide image Gleason score). The teacher/student paradigm allows the models to better generalize on both datasets, despite the inter-dataset heterogeneity and the small number of local annotations used. The classification performance is improved both at the patch-level (up to κ = 0.6127 ± 0.0133 from κ = 0.5667 ± 0.0285), at the TMA core-level (Gleason score) (up to κ = 0.7645 ± 0.0231 from κ = 0.7186 ± 0.0306) and at the WSI-level (Gleason score) (up to κ = 0.4529 ± 0.0512 from κ = 0.2293 ± 0.1350). The results show that with the teacher/student paradigm, it is possible to train models that generalize on datasets from entirely different sources, despite the inter-dataset heterogeneity and the lack of large datasets with local annotations.

On the scale invariance in state of the art CNNs trained on ImageNet

Scientifique ArODES

Graziani Mara, Lompech Thomas, Müller Henning, Depeursinge Adrien, Andrearczyk Vincent

Machine learning and knowledge extraction, 2021, vol. 3, no. 2, pp. 374-391

Lien vers la publication

Résumé:

The diffused practice of pre-training Convolutional Neural Networks (CNNs) on large natural image datasets such as ImageNet causes the automatic learning of invariance to object scale variations. This, however, can be detrimental in medical imaging, where pixel spacing has a known physical correspondence and size is crucial to the diagnosis, for example, the size of lesions, tumors or cell nuclei. In this paper, we use deep learning interpretability to identify at what intermediate layers such invariance is learned. We train and evaluate different regression models on the PASCAL-VOC (Pattern Analysis, Statistical modeling and ComputAtional Learning-Visual Object Classes) annotated data to (i) separate the effects of the closely related yet different notions of image size and object scale, (ii) quantify the presence of scale information in the CNN in terms of the layer-wise correlation between input scale and feature maps in InceptionV3 and ResNet50, and (iii) develop a pruning strategy that reduces the invariance to object scale of the learned features. Results indicate that scale information peaks at central CNN layers and drops close to the softmax, where the invariance is reached. Our pruning strategy uses this to obtain features that preserve scale information. We show that the pruning significantly improves the performance on medical tasks where scale is a relevant factor, for example for the regression of breast histology image magnification. These results show that the presence of scale information at intermediate layers legitimates transfer learning in applications that require scale covariance rather than invariance and that the performance on these tasks can be improved by pruning off the layers where the invariance is learned. All experiments are performed on publicly available data and the code is available on GitHub.

Combining weakly and strongly supervised learning improves strong supervision in Gleason pattern classification

Scientifique ArODES

Otálora Sebastian, Marini Niccolò, Müller Henning, Atzori Manfredo

BMC Medical Imaging, 2021, vol. 21, article no. 77, pp. 1-14

Lien vers la publication

Résumé:

One challenge to train deep convolutional neural network (CNNs) models with whole slide images (WSIs) is providing the required large number of costly, manually annotated image regions. Strategies to alleviate the scarcity of annotated data include: using transfer learning, data augmentation and training the models with less expensive image-level annotations (weakly-supervised learning). However, it is not clear how to combine the use of transfer learning in a CNN model when different data sources are available for training or how to leverage from the combination of large amounts of weakly annotated images with a set of local region annotations. This paper aims to evaluate CNN training strategies based on transfer learning to leverage the combination of weak and strong annotations in heterogeneous data sources. The trade-off between classification performance and annotation effort is explored by evaluating a CNN that learns from strong labels (region annotations) and is later fine-tuned on a dataset with less expensive weak (image-level) labels.

Process data infrastructure and data services

Scientifique ArODES

Cushing Reginald, Valkering Onno, Belloum Adam, Madougou Souley, Bobak Martin, Habala Ondrej, Tran Viet, Meizner Jan, Nowakowski Piotr, Graziani Mara, Müller Henning

Computers and informatics, 2020, vol. 39, no. 4, pp. 724-756

Lien vers la publication

Résumé:

Due to energy limitation and high operational costs, it is likely that exascale computing will not be achieved by one or two datacentres but will require many more. A simple calculation, which aggregates the computation power of the 2017 Top500 supercomputers, can only reach 418 petaflops. Companies like Rescale, which claims 1.4 exaflops of peak computing power, describes its infrastructure as composed of 8 million servers spread across 30 datacentres. Any proposed solution to address exascale computing challenges has to take into consideration these facts and by design should aim to support the use of geographically distributed and likely independent datacentres. It should also consider, whenever possible, the co-allocation of the storage with the computation as it would take 3 years to transfer 1 exabyte on a dedicated 100 Gb Ethernet connection. This means we have to be smart about managing data more and more geographically dispersed and spread across different administrative domains. As the natural settings of the PROCESS project is to operate within the European Research Infrastructure and serve the European research communities facing exascale challenges, it is important that PROCESS architecture and solutions are well positioned within the European computing and data management landscape namely PRACE, EGI, and EUDAT. In this paper we propose a scalable and programmable data infrastructure that is easy to deploy and can be tuned to support various data-intensive scientific applications.

Benchmarking image retrieval diversification techniques for social media

Scientifique ArODES

Ionescu Bogdan, Rohm Maia, Boteanu Bogdan, Gînsca Alexandru Lucian, Lupu Mihai, Müller Henning

IEEE Transactions on multimedia, 2021, vol. 23, pp. 677-691

Lien vers la publication

Résumé:

Image retrieval has been an active research domain for over 30 years and historically it has focused primarily on precision as an evaluation criterion. Similar to text retrieval, where the number of indexed documents became large and many relevant documents exist, it is of high importance to highlight diversity in the search results to provide better results for the user. The Retrieving Diverse Social Images Task of the MediaEval benchmarking campaign has addressed exactly this challenge of retrieving diverse and relevant results for the past years, specifically in the social media context. Multimodal data (e.g., images, text) was made available to the participants including metadata assigned to the images, user IDs, and precomputed visual and text descriptors. Many teams have participated in the task over the years. The large number of publications employing the data and also citations of the overview articles underline the importance of this topic. In this paper, we introduce these publicly available data resources as well as the evaluation framework, and provide an in-depth analysis of the crucial aspects of social image search diversification, such as the capabilities and the evolution of existing systems. These evaluation resources will help researchers for the coming years in analyzing aspects of multimodal image retrieval and diversity of the search results.

Breast histopathology with high-performance computing and deep learning

Scientifique ArODES

Graziani Mara, Eggel Ivan, Deligand François, Bobák Martin, Andrearczyk Vincent, Müller Henning

Computing and informatics, 2020, vol. 39, no. 4, pp. 780-807

Lien vers la publication

Résumé:

The increasingly intensive collection of digitalized images of tumor tissue over the last decade made histopathology a demanding application in terms of computational and storage resources. With images containing billions of pixels, the need for optimizing and adapting histopathology to large-scale data analysis is compelling. This paper presents a modular pipeline with three independent layers for the detection of tumoros regions in digital specimens of breast lymph nodes with deep learning models. Our pipeline can be deployed either on local machines or high-performance computing resources with a containerized approach. The need for expertise in high-performance computing is removed by the self-su_cient structure of Docker containers, whereas a large possibility for customization is left in terms of deep learning models and hyperparameters optimization. We show that by deploying the software layers in di_erent infrastructures we optimize both the data preprocessing and the network training times, further increasing the scalability of the application to datasets of approximatively 43 million images. The code is open source and available on Github.

DeepHistReg :

Scientifique ArODES

unsupervised deep learning registration framework for differently stained histology samples

Wodzinski Marek, Müller Henning

Computer methods and programs in biomedicine, 2021, vol. 198, article no. 105799, pp. 1-11

Lien vers la publication

Résumé:

Background and objective : The use of several stains during histology sample preparation can be useful for fusing complementary information about different tissue structures. It reveals distinct tissue properties that combined may be useful for grading, classification, or 3-D reconstruction. Nevertheless, since the slide preparation is different for each stain and the procedure uses consecutive slices, the tissue undergoes complex and possibly large deformations. Therefore, a nonrigid registration is required before further processing. The nonrigid registration of differently stained histology images is a challenging task because: (i) the registration must be fully automatic, (ii) the histology images are extremely high-resolution, (iii) the registration should be as fast as possible, (iv) there are significant differences in the tissue appearance, and (v) there are not many unique features due to a repetitive texture. Methods :In this article, we propose a deep learning-based solution to the histology registration. We describe a registration framework dedicated to high-resolution histology images that can perform the registration in real-time. The framework consists of an automatic background segmentation, iterative initial rotation search and learning-based affine/nonrigid registration. Results : We evaluate our approach using an open dataset provided for the Automatic Non-rigid Histological Image Registration (ANHIR) challenge organized jointly with the IEEE ISBI 2019 conference. We compare our solution to the challenge participants using a server-side evaluation tool provided by the challenge organizers. Following the challenge evaluation criteria, we use the target registration error (TRE) as the evaluation metric. Our algorithm provides registration accuracy close to the best scoring teams (median rTRE 0.19% of the image diagonal) while being significantly faster (the average registration time is about 2 seconds). Conclusions : The proposed framework provides results, in terms of the TRE, comparable to the best-performing state-of-the-art methods. However, it is significantly faster, thus potentially more useful in clinical practice where a large number of histology images are being processed. The proposed method is of particular interest to researchers requiring an accurate, real-time, nonrigid registration of high-resolution histology images for whom the processing time of traditional, iterative methods in unacceptable. We provide free access to the software implementation of the method, including training and inference code, as well as pretrained models. Since the ANHIR dataset is open, this makes the results fully and easily reproducible.

Reference exascale architecture (extended version)

Scientifique ArODES

Bobák Martin, Hluchý Ladislav, Habala Ondrej, Tran Viet, Cushing Reginald, Valkering Onno, Belloum Adam, Graziani Mara, Müller Henning, Madougou Souley, Maassen Jason

Computing and informatics, 2020, vol. 34, no. 4, pp. 644-677

Lien vers la publication

Résumé:

While political commitments for building exascale systems have been made, turning these systems into platforms for a wide range of exascale applications faces several technical, organisational and skills-related challenges. The key technical challenges are related to the availability of data. While the first exascale machines are likely to be built within a single site, the input data is in many cases impossible to store within a single site. Alongside handling of extreme-large amount of data, the exascale system has to process data from different sources, support accelerated computing, handle high volume of requests per day, minimize the size of data flows, and be extensible in terms of continuously increasing data as well as an increase in parallel requests being sent. These technical challenges are addressed by the general reference exascale architecture. It is divided into three main blocks: virtualization layer, distributed virtual file system, and manager of computing resources. Its main property is modularity which is achieved by containerization at two levels: 1) application containers - containerization of scientific workflows, 2) micro-infrastructure - containerization of extreme-large data service-oriented infrastructure. The paper also presents an instantiation of the reference architecture - the architecture of the PROCESS project (PROviding Computing solutions for ExaScale ChallengeS) and discusses its relation to the reference exascale architecture. The PROCESS architecture has been used as an exascale platform within various exascale pilot applications. This paper also presents performance modelling of exascale platform with its validation.

2020

BIAS :

Scientifique ArODES

transparent reporting of biomedical image analysis challenges

Maier-Hein Lena, Reinke Annika, Kozubek Michal, Martel Anne L., Arbel Tal, Eisenmann Matthias, Hanbury Allan, Jannin Pierre, Müller Henning, Onogur Sinan, Saez-Rodriguez Julio, Van Ginneken Bram, Kopp-Schneider Annette, Landman Bennett A.

Medical image analysis, 2020, vol. 66, article 101796, pp. 1-7

Lien vers la publication

Résumé:

The number of biomedical image analysis challenges organized per year is steadily increasing. These international competitions have the purpose of benchmarking algorithms on common data sets, typically to identify the best method for a given problem. Recent research, however, revealed that common practice related to challenge reporting does not allow for adequate interpretation and reproducibility of results. To address the discrepancy between the impact of challenges and the quality (control), the Biomedical Image Analysis ChallengeS (BIAS) initiative developed a set of recommendations for the reporting of challenges. The BIAS statement aims to improve the transparency of the reporting of a biomedical image analysis challenge regardless of field of application, image modality or task category assessed. This article describes how the BIAS statement was developed and presents a checklist which authors of biomedical image analysis challenges are encouraged to include in their submission when giving a paper on a challenge into review. The purpose of the checklist is to standardize and facilitate the review process and raise interpretability and reproducibility of challenge results by making relevant information explicit.

The importance of feature aggregation in radiomics :

Scientifique ArODES

a head and neck cancer study

Fontaine Pierre, Acosta Oscar, Castelli Joël, De Crevoisier Renaud, Müller Henning, Depeursinge Adrien

Scientific reports, 2020, vol. 10, article no. 19679 , pp. 1-11

Lien vers la publication

Résumé:

In standard radiomics studies the features extracted from clinical images are mostly quantified with simple statistics such as the average or variance per Region of Interest (ROI). Such approaches may smooth out any intra-region heterogeneity and thus hide some tumor aggressiveness that may hamper predictions. In this paper we study the importance of feature aggregation within the standard radiomics workflow, which allows to take into account intra-region variations. Feature aggregation methods transform a collection of voxel values from feature response maps (over a ROI) into one or several scalar values that are usable for statistical or machine learning algorithms. This important step has been little investigated within the radiomics workflows, so far. In this paper, we compare several aggregation methods with standard radiomics approaches in order to assess the improvements in prediction capabilities. We evaluate the performance using an aggregation function based on Bags of Visual Words (BoVW), which allows for the preservation of piece-wise homogeneous information within heterogeneous regions and compared with standard methods. The different models are compared on a cohort of 214 head and neck cancer patients coming from 4 medical centers. Radiomics features were extracted from manually delineated tumors in clinical PET-FDG and CT images were analyzed. We compared the performance of standard radiomics models, the volume of the ROI alone and the BoVW model for survival analysis. The average concordance index was estimated with a five fold cross-validation. The performance was significantly better using the BoVW model 0.627 (95% CI: 0.616–0.637) as compared to standard radiomics0.505 (95% CI: 0.499–0.511), mean-var. 0.543 (95% CI: 0.536–0.549), mean0.547 (95% CI: 0.541–0.554), var.0.530 (95% CI: 0.524–0.536) or volume 0.577 (95% CI: 0.571–0.582). We conclude that classical aggregation methods are not optimal in case of heterogeneous tumors. We also showed that the BoVW model is a better alternative to extract consistent features in the presence of lesions composed of heterogeneous tissue.

A lung graph model for the radiological assessment of chronic thromboembolic pulmonary hypertension in CT

Scientifique ArODES

Jiménez del Toro Oscar Alfonso, Dicente Cid Yashin, Platon Alexandra, Hachulla Anne-Lise, Lador Frédéric, Poletti Pierre-Alexandre, Müller Henning

Computers in biology and medicine, 2020, vol. 125, article 103962, pp. 1-8

Lien vers la publication

Résumé:

Chronic thromboembolic pulmonary hypertension (CTEPH) is a possible complication of pulmonary embolism (PE), with poor prognosis if left untreated. Surgical curative treatment is available, particularly in the early stages of the disease. However, most cases are not diagnosed until specific symptoms become evident. A small number of computed tomography (CT) findings, such as a widened pulmonary artery and mosaicism in the lung parenchyma, have been correlated with pulmonary hypertension (PH). Quantitative texture analysis in the CT scans of these patients could provide complementary sub-visual information of the vascular changes taking place in the lungs. For this task, a lung graph model was developed with texture descriptors from 37 CT scans with confirmed CTEPH diagnosis and 48 CT scans from PE patients who did not develop PH. The probability of presenting CTEPH, computed with the graph model, outperformed a convolutional neural network approach using 10 different train/test splits of the data set. An accuracy of 0.76 was obtained with the proposed texture analysis, and was then compared to the visual assessment of CT findings, manually identified by a team of three expert radiologists, commonly associated with pulmonary hypertension. This graph-based score combined with the information attained from the radiological findings resulted in a Cohen’s kappa coefficient of 0.47 when differentiating patients with confirmed CTEPH from those with PE who did not develop the disease. The proposed texture quantification could be an objective measurement, complementary to the current analysis of radiologists for the early detection of CTEPH and thus improve patient outcome.

Concept attribution :

Scientifique ArODES

explaining CNN decisions to physicians

Graziani Mara, Andrearczyk Vincent, Marchand-Maillet Stéphane, Müller Henning

Computers in biology and medicine, 2020, vol. 123, Article 103865, pp. 1-11

Lien vers la publication

Résumé:

Deep learning explainability is often reached by gradient-based approaches that attribute the network output to perturbations of the input pixels. However, the relevance of input pixels may be difficult to relate to relevant image features in some applications, e.g. diagnostic measures in medical imaging. The framework described in this paper shifts the attribution focus from pixel values to user-defined concepts. By checking if certain diagnostic measures are present in the learned representations, experts can explain and entrust the network output. Being post-hoc, our method does not alter the network training and can be easily plugged into the latest state-of-the-art convolutional networks. This paper presents the main components of the framework for attribution to concepts, in addition to the introduction of a spatial pooling operation on top of the feature maps to obtain a solid interpretability analysis. Furthermore, regularized regression is analyzed as a solution to the regression overfitting in high-dimensionality latent spaces. The versatility of the proposed approach is shown by experiments on two medical applications, namely histopathology and retinopathy, and on one non-medical task, the task of handwritten digit classification. The obtained explanations are in line with clinicians’ guidelines and complementary to widely used visualization tools such as saliency maps.

Variability of muscle synergies in hand grasps :

Scientifique ArODES

analysis of intra- and inter-session data

Pale Una, Atzori Manfredo, Müller Henning, Scano Alessandro

Sensors, 2020, vol. 20, no 15, article 4297

Lien vers la publication

Résumé:

Background. Muscle synergy analysis is an approach to understand the neurophysiological mechanisms behind the hypothesized ability of the Central Nervous System (CNS) to reduce the dimensionality of muscle control. The muscle synergy approach is also used to evaluate motor recovery and the evolution of the patients’ motor performance both in single-session and longitudinal studies. Synergy-based assessments are subject to various sources of variability: natural trial-by-trial variability of performed movements, intrinsic characteristics of subjects that change over time (e.g., recovery, adaptation, exercise, etc.), as well as experimental factors such as di_erent electrode positioning. These sources of variability need to be quantified in order to resolve challenges for the application of muscle synergies in clinical environments. The objective of this study is to analyze the stability and similarity of extracted muscle synergies under the e_ect of factors that may induce variability, including inter- and intra-session variability within subjects and inter-subject variability di_erentiation. The analysis was performed using the comprehensive, publicly available hand grasp NinaPro Database, featuring surface electromyography (EMG) measures from two EMG electrode bracelets. Methods. Intra-session, inter-session, and inter-subject synergy stability was analyzed using the following measures: variance accounted for (VAF) and number of synergies (NoS) as measures of reconstruction stability quality and cosine similarity for comparison of spatial composition of extracted synergies. Moreover, an approach based on virtual electrode repositioning was applied to shed light on the influence of electrode position on inter-session synergy similarity. Results. Inter-session synergy similarity was significantly lower with respect to intra-session similarity, both considering coe_cient of variation of VAF (approximately 0.2–15% for inter vs. approximately 0.1% to 2.5% for intra, depending on NoS) and coe_cient of variation of NoS (approximately 6.5–14.5% for inter vs. approximately 3–3.5% for intra, depending on VAF) as well as synergy similarity (approximately 74–77% for inter vs. approximately 88–94% for intra, depending on the selected VAF). Virtual electrode repositioning revealed that a slightly di_erent electrode position can lower similarity of synergies from the same session and can increase similarity between sessions. Finally, the similarity of inter-subject synergies has no significant di_erence from the similarity of inter-session synergies (both on average approximately 84–90% depending on selected VAF). Conclusion. Synergy similarity was lower in inter-session conditions with respect to intra-session. This finding should be considered when interpreting results from multi-session assessments. Lastly, electrode positioning might play an important role in the lower similarity of synergies over di_erent sessions.

Automating quality control for structured standardized radiology reports using text analysis

Chapter ArODES

Dhrangadhariya Anjani, Millius Sandy, Thouly Cyril, Rizk Benoit, Fournier Dominique, Müller Henning, Brat Hugues

Dans Lovis, Christian, Madsen, Inge Cort, Nielsen, Per Hostrup, Pape-Haugaard, Louise B., Scott, Philip, Weber, Patrick, Digital personalized health and medicine (Pp. 58-62). 2020, Amsterdam : IOS press

Lien vers la publication

Résumé:

Radiology reports describe the findings of a radiologist in an imaging examination, produced for another clinician in order to answer to a clinical indication. Sometimes, the report does not fully answer the question asked, despite guidelines for the radiologist. In this article, a system that controls the quality of reports automatically is described. It notably maps the free text onto MeSH terms and checks if the anatomy and disease terms match in the indication and conclusion of a report. The agreement between manual checks of experienced radiologists and the system is high with automatic checks requiring only a fraction of time. Being able to quality control all reports has the potential to improve report quality and thus limit misunderstandings, loosing time for requesting more information and possibly avoid medical mistakes.

Prospects and challenges of radiomics by using nononcologic routine chest CT

Scientifique ArODES

Röhrich Sebastian, Hofmanninger Johannes, Prayer Florian, Müller Henning, Prosch Helmut, Langs Georg

Radiology : cardiothoracic imaging, 2020, vol. 2, no. 4, pp. 1-11

Lien vers la publication

Résumé:

Chest CT scans are one of the most common medical imaging procedures. The automatic extraction and quantification of imaging features may help in diagnosis, prognosis of, or treatment decision in cardiovascular, pulmonary, and metabolic diseases. However, an adequate sample size as a statistical necessity for radiomics studies is often difficult to achieve in prospective trials. By exploiting imaging data from clinical routine, a much larger amount of data could be used than in clinical trials. Still, there is only little literature on the implementation of radiomics in clinical routine chest CT scans. Reasons are heterogeneous CT scanning protocols and the resulting technical variability (eg, different slice thicknesses, reconstruction kernels or timings after contrast material administration) in routine CT imaging data. This review summarizes the recent state of the art of studies aiming to develop quantifiable imaging biomarkers at chest CT, such as for osteoporosis, chronic obstructive pulmonary disease, interstitial lung disease, and coronary artery disease. This review explains solutions to overcome heterogeneity in routine data such as the use of imaging repositories, the standardization of radiomic features, algorithmic approaches to improve feature stability, test-retest studies, and the evolution of deep learning for modeling radiomics features.

Machine learning assisted citation screening for systematic reviews

Chapter ArODES

Dhrangadhariya Anjani, Hilfiker Roger, Schaer Roger, Müller Henning

Dans Pape-Haugaard, Louise B., Digital personalized health and medicine (5 p.). 2020, Amsterdam, The Netherlands : OIS Press

Lien vers la publication

Résumé:

Evidence-based practice is highly dependent upon up-to-date systematic reviews (SR) for decision making. However, conducting and updating systematic reviews, especially the citation screening for identification of relevant studies, requires much human work and is therefore expensive. Automating citation screening using machine learning (ML) based approaches can reduce cost and labor. Machine learning has been applied to automate citation screening but not for the SRs with very narrow research questions. This paper reports the results and observations for an ongoing research that aims to automate citation screening for SRs with narrow research questions using machine learning. The research also sheds light on the problem of class imbalance and class overlap on the performance of ML classifiers when applied to SRs with narrow research questions.

The image biomarker standardization initiative :

Scientifique ArODES

standardized quantitative radiomics for high-throughput image-based phenotyping

Zwanenburg Alex, Vallières Martin, Abdalah Mahmoud A., Aerts Hugo J. W. L., Andrearczyk Vincent, Depeursinge Adrien, Müller Henning

Radiology, 2020, vol. 295, no. 2, pp. 328-338

Lien vers la publication

Résumé:

The Image Biomarker Standardization Initiative validated consensus-based reference values for 169 radiomics features, thus enabling calibration and verification of radiomics software. Radiomic features may quantify characteristics present in medical imaging. However, the lack of standardized definitions and validated reference values have hampered clinical use. Radiomic features were assessed in three phases. In phase I, 487 features were derived from the basic set of 174 features. Twenty-five research teams with unique radiomics software implementations computed feature values directly from a digital phantom, without any additional image processing. In phase II, 15 teams computed values for 1347 derived features using a CT image of a patient with lung cancer and predefined image processing configurations. In both phases, consensus among the teams on the validity of tentative reference values was measured through the frequency of the modal value and classified as follows: less than three matches, weak; three to five matches, moderate; six to nine matches, strong; 10 or more matches, very strong. In the final phase (phase III), a public data set of multimodality images (CT, fluorine 18 fluorodeoxyglucose PET, and T1-weighted MRI) from 51 patients with soft-tissue sarcoma was used to prospectively assess reproducibility of standardized features. Consensus on reference values was initially weak for 232 of 302 features (76.8%) at phase I and 703 of 1075 features (65.4%) at phase II. At the final iteration, weak consensus remained for only two of 487 features (0.4%) at phase I and 19 of 1347 features (1.4%) at phase II. Strong or better consensus was achieved for 463 of 487 features (95.1%) at phase I and 1220 of 1347 features (90.6%) at phase II. Overall, 169 of 174 features were standardized in the first two phases. In the final validation phase (phase III), most of the 169 standardized features could be excellently reproduced (166 with CT; 164 with PET; and 164 with MRI). A set of 169 radiomics features was standardized, which enabled verification and calibration of different radiomics software.

Gaze, behavioral, and clinical data for phantom limbs after hand amputation from 15 amputees and 29 controls

Scientifique ArODES

Saetta Gianluca, Cognolato Matteo, Atzori Manfredo, Faccio Diego, Giacomino Katia, Mittaz Hager Anne-Gabrielle, Tiengo Cesare, Bassetto Franco, Müller Henning, Bruger Peter

Scientific Data, 2020, vol. 7, article 60, pp. 1-14

Lien vers la publication

Résumé:

Despite recent advances in prosthetics, many upper limb amputees still use prostheses with some reluctance. They often do not feel able to incorporate the artificial hand into their bodily self. Furthermore, prosthesis fitting is not usually tailored to accommodate the characteristics of an individual’s phantom limb sensations. These are experienced by almost all persons with an acquired amputation and comprise the motor and postural properties of the lost limb. This article presents and validates a multimodal dataset including an extensive qualitative and quantitative assessment of phantom limb sensations in 15 transradial amputees, surface electromyography and accelerometry data of the forearm, and measurements of gaze behavior during exercises requiring pointing or repositioning of the forearm and the phantom hand. The data also include acquisitions from 29 able-bodied participants, matched for gender and age. Special emphasis was given to tracking the visuo-motor coupling between eye-hand/eye-phantom during these exercises.

A large calibrated database of hand movements and grasps kinematics

Scientifique ArODES

Jarque-Bou Néstor J., Atzori Manfredo, Müller Henning

Scientific data, 2020, vol. 7, article 12, pp. 1-10

Lien vers la publication

Résumé:

Modelling hand kinematics is a challenging problem, crucial for several domains including robotics, 3D modelling, rehabilitation medicine and neuroscience. Currently available datasets are few and limited in the number of subjects and movements. The objective of this work is to advance the modelling of hand kinematics by releasing and validating a large publicly available kinematic dataset of hand movements and grasp kinematics. The dataset is based on the harmonization and calibration of the kinematics data of three multimodal datasets previously released (Ninapro DB1, DB2 and DB5, that include electromyography, inertial and dynamic data). The novelty of the dataset is related to the high number of subjects (77) and movements (40 movements, each repeated several times) for which we release for the first time calibrated kinematic data, resulting in the largest available kinematic dataset. Differently from the previous datasets, the data are also calibrated to avoid sensor nonlinearities. The validation confirms that the data are not affected by experimental procedures and that they are similar to data acquired in real-life conditions.

Information fusion for medical data :

Scientifique ArODES

early, late, and deep fusion methods for multimodal data

Domingues Inés, Müller Henning, Ortiz Andres, Dasarathy Belur V., Abreu Pedro H., Calhoun Vince D.

IEEE Journal of biomedical and health informatics, 2020, vol. 24, no. 1, pp. 14-16

Lien vers la publication

Résumé:

The papers in this special section examine important current topics on multimodal data fusion in the medical context. All clinical data, including genomic and proteomic, play a role in the diagnosis and in particular in the treatment planning and follow-up. This is true for all types of data analyses whether in classification, regression, retrieval, clustering, or other. The interaction between several types of information is not always well understood. Experienced clinicians automatically and even unconsciously add multiple sources of information into their decision process, but machine learning tools often concentrate on single information sources. This special issue presents five examples where several data sources are fused. The papers give several examples of fusion techniques and also the results obtained in quite different application scenarios.

2019

Classification of diabetes-related retinal diseases using a deep learning approach in optical coherence tomography

Scientifique ArODES

Perdomo Oscar, Rios Hernán, Rodríguez Francisco J., Otálora Sebastián, Meriaudeau Fabrice, Müller Henning, González Fabio A.

Computer methods and programs in biomedicine, September 2019, vol. 178, pp. 181-189

Lien vers la publication

Résumé:

Background and objectives: Spectral Domain Optical Coherence Tomography (SD-OCT) is a volumetric imaging technique that allows measuring patterns between layers such as small amounts of fluid. Since 2012, automatic medical image analysis performance has steadily increased through the use of deep learning models that automatically learn relevant features for specific tasks, instead of designing visual features manually. Nevertheless, providing insights and interpretation of the predictions made by the model is still a challenge. This paper describes a deep learning model able to detect medically interpretable information in relevant images from a volume to classify diabetes-related retinal diseases. Methods: This article presents a new deep learning model, OCT-NET, which is a customized convolutional neural network for processing scans extracted from optical coherence tomography volumes. OCT-NET is applied to the classification of three conditions seen in SD-OCT volumes. Additionally, the proposed model includes a feedback stage that highlights the areas of the scans to support the interpretation of the results. This information is potentially useful for a medical specialist while assessing the prediction produced by the model. Results: The proposed model was tested on the public SERI-CUHK and A2A SD-OCT data sets containing healthy, diabetic retinopathy, diabetic macular edema and age-related macular degeneration. The experimental evaluation shows that the proposed method outperforms conventional convolutional deep learning models from the state of the art reported on the SERI+CUHK and A2A SD-OCT data sets with a precision of 93% and an area under the ROC curve (AUC) of 0.99 respectively. Conclusions: The proposed method is able to classify the three studied retinal diseases with high accuracy. One advantage of the method is its ability to produce interpretable clinical information in the form of highlighting the regions of the image that most contribute to the classifier decision.

PaWFE :

Scientifique ArODES

fast signal feature extraction using parallel time windows

Atzori Manfredo, Müller Henning

Frontiers in Neurorobotics, 2019, vol. 13, article 74

Lien vers la publication

Résumé:

Motivation: Hand amputations can dramatically affect the quality of life of a person. Researchers are developing surface electromyography and machine learning solutions to control dexterous and robotic prosthetic hands, however long computational times can slow down this process. Objective: This paper aims at creating a fast signal feature extraction algorithm that can extract widely used features and allow researchers to easily add new ones. Methods: PaWFE (Parallel Window Feature Extractor) extracts the signal features from several time windows in parallel. The MATLAB code is publicly available and supports several time domain and frequency features. The code was tested and benchmarked using 1,2,4,8,16,32, and 48 threads on a server with four Xeon E7- 4820 and 128 GB RAM using the ﬁrst 5 datasets of the Ninapro database, that are recorded with different acquisition setups. Results: The parallel time window analysis approach allows to reduce the computational time up to 20 times when using 32 cores, showing a very good scalability. Signal features can be extracted in few seconds from an entire data acquisition and in <100ms from a single time window, easily reducing of up to over 15 times the feature extraction procedure in comparison to traditional approaches. The code allows users to easily add new signal feature extraction scripts, that can be added to the code and on the Ninapro website upon request. Signiﬁcance: The code allows researchers in machine learning and biosignals data analysis to easily and quickly test modern machine learning approaches on big datasets and it can be used as a resource for real time data analysis too.

Experimental IR meets multilinguality, multimodality, and interaction : 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, proceedings

Livre ArODES

Crestani Fabio, Müller Henning, et al.

2019, Cham : Springer, 434 p.

Lien vers la publication

Résumé:

This book constitutes the refereed proceedings of the 10th International Conference of the CLEF Association, CLEF 2019, held in Lugano, Switzerland, in September 2019. The conference has a clear focus on experimental information retrieval with special attention to the challenges of multimodality, multilinguality, and interactive search ranging from unstructured to semi structures and structured data. The 7 full papers and 8 short papers presented in this volume were carefully reviewed and selected from 30 submissions. This year, many contributions tackle the social networks with the detection of stances or early identiﬁcation of depression signs on Twitter in a cross-lingual context. Further this volume presents 7 “best of the labs” papers which were reviewed as a full paper submission with the same review criteria. The labs represented scientific challenges based on new data sets and real world problems in multimodal and multilingual information access. In addition to this, 9 benchmarking labs reported results of their yearlong activities in overview talks and lab sessions.

Biodiversity information retrieval through large scale content-based identification: a long-term evaluation

Chapter ArODES

Joly Alexis, Goëau Hervé, Glotin Hervé, Spampinato Concetto, Bonnet Pierre, Vellinga Willem-Pier, Lombardo Jean-Christophe, Planqué Robert, Palazzo Simone, Müller Henning

Dans Ferro, Nicolas, Peters, Carol, Information retrieval evaluation in a changing world : lessons learned from 20 years of CLEF (pp. 389-413). 2019, Cham : Springer

Lien vers la publication

Résumé:

Identifying and naming living plants or animals is usually impossible for the general public and often a difficult task for professionals and naturalists. Bridging this gap is a key challenge towards enabling effective biodiversity information retrieval systems. This taxonomic gap was actually already identified as one of the main ecological challenges to be solved during the Rio de Janeiro United Nations “Earth Summit” in 1992. Since 2011, the LifeCLEF challenges conducted in the context of the CLEF evaluation forum have been boosting and evaluating the advances in this domain. Data collections with an unprecedented volume and diversity have been shared with the scientific community to allow repeatable and long-term experiments. This paper describes the methodology of the conducted evaluation campaigns as well as providing a synthesis of the main results and lessons learned along the years.

Experiences from the ImageCLEF medical retrieval and annotation tasks

Chapter ArODES

Müller Henning, Kalpathy-Cramer Jayashree, García Seco de Herrera Alba

Dans Ferro Nicola, Peters, Carol, Information Retrieval Evaluation in a Changing World : lessons learned from 20 Years of CLEF (19 p.). 2019, Cham : Springer

Lien vers la publication

Résumé:

The medical tasks in ImageCLEF have been run every year from 2004- 2018 and many different tasks and data sets have been used over these years. The created resources are being used by many researchers well beyond the actual evaluation campaigns and are allowing to compare the performance of many techniques on the same grounds and in a reproducible way. Many of the larger data sets are from the medical literature, as such images are easier to obtain and to share than clinical data, which was used in a few smaller ImageCLEF challenges that are specifically marked with the disease type and anatomic region. This chapter describes the main results of the various tasks over the years, including data, participants, types of tasks evaluated and also the lessons learned in organizing such tasks for the scientific community.

Fusing learned representations from Riesz Filters and Deep CNN for lung tissue classification

Scientifique ArODES

Joyseeree Ranveer, Otálora Sebastian, Müller Henning, Depeursinge Adrien

Medical image analysis, August 2019, vol. 56, pp. 172-183

Lien vers la publication

Résumé:

A novel method to detect and classify several classes of diseased and healthy lung tissue in CT (Computed Tomography), based on the fusion of Riesz and deep learning features, is presented. First, discriminative parametric lung tissue texture signatures are learned from Riesz representations using a one–versus–one approach. The signatures are generated for four diseased tissue types and a healthy tissue class, all of which frequently appear in the publicly available Interstitial Lung Diseases (ILD) dataset used in this article. Because the Riesz wavelets are steerable, they can easily be made invariant to local image rotations, a property that is desirable when analyzing lung tissue micro–architectures in CT images. Second, features from deep Convolutional Neural Networks (CNN) are computed by fine–tuning the Inception V3 architecture using an augmented version of the same ILD dataset. Because CNN features are both deep and non–parametric, they can accurately model virtually any pattern that is useful for tissue discrimination, and they are the de facto standard for many medical imaging tasks. However, invariance to local image rotations is not explicitly implemented and can only be approximated with rotation–based data augmentation. This motivates the fusion of Riesz and deep CNN features, as the two techniques are very complementary. The two learned representations are combined in a joint softmax model for final classification, where early and late feature fusion schemes are compared. The experimental results show that a late fusion of the independent probabilities leads to significant improvements in classification performance when compared to each of the separate feature representations and also compared to an ensemble of deep learning approaches.

ImageCLEF 2019 :

Chapter ArODES

multimedia retrieval in medicine, lifelogging, security and nature

Ionescu Bogdan, Müller Henning, Péteri Renaud, Dicente Cid Yashin, et al.

Dans Crestani, Fabio, et al., Müller, Henning, Experimental IR Meets Multilinguality, Multimodality, and Interaction : 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings (pp. 358-386). 2019, Cham : Springer

Lien vers la publication

Résumé:

This paper presents an overview of the ImageCLEF 2019 lab, organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2019. ImageCLEF is an ongoing evaluation initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2019, the 17th edition of ImageCLEF runs four main tasks: (i) a medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with new data, (ii) a lifelog task (videos, images and other sources) about daily activities understanding, retrieval and summarization, (iii) a new security task addressing the problems of automatically identifying forged content and retrieve hidden information, and (iv) a new coral task about segmenting and labeling collections of coral images for 3D modeling. The strong participation, with 235 research groups registering, and 63 submitting over 359 runs, shows an important interest in this benchmark campaign.

Visualizing and interpreting feature reuse of pretrained CNNs for histopathology

Chapter ArODES

Graziani Mara, Andrearczyk Vincent, Müller Henning

Dans Courtney, Jane, Deegan, Catherine, Leamy, Paul, MVIP 2019: Irish Machine Vision and Image Processing Conference Proceedings (4 p.). 2019, Dublin, Ireland : Irish Pattern Recognition and Classification Society

Lien vers la publication

Résumé:

Reusing the parameters of networks pretrained on large scale datasets of natural images, such as ImageNet, is a common technique in the medical imaging domain. The large variability of objects and classes is, however, drastically reduced in most medical applications where images are dominated by repetitive patterns with, at times, subtle differences between the classes. This paper takes the example of finetuning a pretrained convolutional network on a histopathology task. Because of the reduced visual variability in this application domain, the network mostly learns to detect textures and simple patterns. As a result, the complex structures that maximize the channel activations of deep layers in the pretrained network are not present after finetuning. The learned features seem to be used by the network to spot atypical nuclei in the images, as shown by class activation maps. Finally, texture measures appear discriminative after finetuning, as shown by accurate Regression Concept Vectors.

Overview of LifeCLEF 2019: identification of Amazonian plants, south and north American birds, and niche prediction

Chapter ArODES

Joly Alexis, Müller Henning, et al.

Dans Crestani, Fabio, et al., Müller, Henning, Experimental IR meets multilinguality, multimodality, and interaction : 10th International Conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9–12, 2019, Proceedings (pp. 387-401). 2019, Cham : Springer

Lien vers la publication

Résumé:

Building accurate knowledge of the identity, the geographic distribution and the evolution of living species is essential for a sustainable development of humanity, as well as for biodiversity conservation. Unfortunately, such basic information is often only partially available for professional stakeholders, teachers, scientists and citizens, and often incomplete for ecosystems that possess the highest diversity. In this context, an ultimate ambition is to set up innovative information systems relying on the automated identication and understanding of living organisms as a means to engage massive crowds of observers and boost the production of biodiversity and agro-biodiversity data. The LifeCLEF 2019 initiative proposes three data-oriented challenges related to this vision, in the continuity of the previous editions but with several consistent novelties intended to push the boundaries of the state-of-the-art in several research directions. This paper describes the methodology of the conducted evaluations as well as the synthesis of the main results and lessons learned.

An augmented reality environment to provide visual feedback to amputees during sEMG Data Acquisitions

Chapter ArODES

Palermo Francesca, Cognolato Matteo, Eggel Ivan, Atzori Manfredo, Müller Henning

Dans Althoefer, Kaspar, Konstantinova, Jelizaveta, Zhang, Ketao, Towards autonomous robotic systems : 20th annual conference, TAROS 2019, London, UK, July 3–5, 2019, Proceedings, Part II (12 p.). 2019, Cham : Springer

Lien vers la publication

Résumé:

Myoelectric hand prostheses have the potential to improve the quality of life of hand amputees. Still, the rejection rate of functional prostheses in the adult population is high. One of the causes is the long time for fitting the prosthesis and the lack of feedback during training. Moreover, prosthesis control is often unnatural and requires mental effort during the training. Virtual and augmented reality devices can help to improve these difficulties and reduce phantom limb pain. Amputees can start training the residual limb muscles with a weightless virtual hand earlier than possible with a real prosthesis. When activating the muscles related to a specific grasp, the subjects receive a visual feedback from the virtual hand. To the best of our knowledge, this work presents one of the first portable augmented reality environment for transradial amputees that combines two devices available on the market: the Microsoft HoloLens and the Thalmic labs Myo. In the augmented environment, rendered by the HoloLens, the user can control a virtual hand with surface electromyography. By using the virtual hand, the user can move objects in augmented reality and train to activate the right muscles for each movement through visual feedback. The environment presented represents a resource for rehabilitation and for scientists. It helps hand amputees to train using prosthetic hands right after the surgery. Scientists can use the environment to develop real time control experiments, without the logistical disadvantages related to dealing with a real prosthetic hand but with the advantages of a realistic visual feedback.

Deep learning-based retrieval system for gigapixel histopathology cases and the open access literature

Scientifique ArODES

Schaer Roger, Jiménez del Toro Oscar Alfonso, Otálora Sebastian, Atzori Manfredo, Müller Henning

Journal of pathology informatics, 1 juillet 2019

Lien vers la publication

Résumé:

Background: The introduction of digital pathology into clinical practice has led to the development of clinical workflows with digital images, in connection with pathology reports. Still, most of the current work is time‑consuming manual analysis of image areas at different scales. Links with data in the biomedical literature are rare, and a need for search based on visual similarity within whole slide images (WSIs) exists. Objectives: The main objective of the work presented is to integrate content‑based visual retrieval with a WSI viewer in a prototype. Another objective is to connect cases analyzed in the viewer with cases or images from the biomedical literature, including the search through visual similarity and text. Methods: An innovative retrieval system for digital pathology is integrated with a WSI viewer, allowing to define regions of interest (ROIs) in images as queries for finding visually similar areas in the same or other images and to zoom in/out to find structures at varying magnification levels. The algorithms are based on a multimodal approach, exploiting both text information and content‑based image features. Results: The retrieval system allows viewing WSIs and searching for regions that are visually similar to manually defined ROIs in various data sources (proprietary and public datasets, e.g., scientific literature). The system was tested by pathologists, highlighting its capabilities and suggesting ways to improve it and make it more usable in clinical practice. Conclusions: The developed system can enhance the practice of pathologists by enabling them to use their experience and knowledge to control artificial intelligence tools for navigating repositories of images for clinical decision support and teaching, where the comparison with visually similar cases can help to avoid misinterpretations. The system is available as open source, allowing the scientific community to test, ideate and develop similar systems for research and clinical practice.

A graph model of the lungs with morphology-based structure for tuberculosis type classification

Chapter ArODES

Dicente Cid Yashin, Jiménez del Toro Oscar Alfonso, Poletti Pierre-Alexandre, Müller Henning

Dans Chung, Albert C. S., et al., Information processing in medical imaging : 26th International Conference, IPMI 2019, Hong Kong, China, June 2–7, 2019, proceedings (12 p.). 2019, Hong Kong, China : 2-7 June 2019

Lien vers la publication

Résumé:

Pulmonary tuberculosis (TB) is still an important cause of death worldwide, even after being almost eradicated 40 years ago. Early identification of TB in computed tomography (CT) scans can influence therapeutical decisions, thus improving patient outcome. In this paper, a graph model of the lungs is proposed for the classification of TB types using local texture features in thorax CT scans of TB patients. Based on lung morphology, an automatic patient-specific lung field parcellation was initially computed. Local visual features were then extracted from each region and were used to build a personalized lung graph model. A new graph-based patient descriptor enables comparisons between lung graphs with a different number of nodes and edges, encoding the distribution of several node measures from graph theory. The proposed model was trained and tested on a public dataset of 1,513 CT scans of 994 TB patients. The evaluation was performed on data from a scientific challenge together with 39 participant algorithms, obtaining the best unweighted Cohen kappa coefficient of 0.24 in a 5-class setup with 505 test CTs. Even though each lung graph has a unique structure, the proposed method was able to identify key texture changes associated with the different manifestations of TB.

Kinematic synergies of hand grasps :

Scientifique ArODES

a comprehensive study on a large publicly available dataset

Jarque-Bou Néstor J., Scano Alessandro, Aztori Manfredo, Müller Henning

Journal of neuroengineering and rehabilitation, 2019, vol. 16, article no. 63

Lien vers la publication

Résumé:

Background: Hand grasp patterns require complex coordination. The reduction of the kinematic dimensionality is a key process to study the patterns underlying hand usage and grasping. It allows to define metrics for motor assessment and rehabilitation, to develop assistive devices and prosthesis control methods. Several studies were presented in this field but most of them targeted a limited number of subjects, they focused on postures rather than entire grasping movements and they did not perform separate analysis for the tasks and subjects, which can limit the impact on rehabilitation and assistive applications. This paper provides a comprehensive mapping of synergies from hand grasps targeting activities of daily living. It clarifies several current limits of the field and fosters the development of applications in rehabilitation and assistive robotics. Methods: In this work, hand kinematic data of 77 subjects, performing up to 20 hand grasps, were acquired with a data glove (a 22-sensor CyberGlove II data glove) and analyzed. Principal Component Analysis (PCA) and hierarchical cluster analysis were used to extract and group kinematic synergies that summarize the coordination patterns available for hand grasps. Results: Twelve synergies were found to account for > 80% of the overall variation. The first three synergies accounted for more than 50% of the total amount of variance and consisted of: the flexion and adduction of the Metacarpophalangeal joint (MCP) of fingers 3 to 5 (synergy #1), palmar arching and flexion of the wrist (synergy #2) and opposition of the thumb (synergy #3). Further synergies refine movements and have higher variability among subjects. Conclusion: Kinematic synergies are extracted from a large number of subjects (77) and grasps related to activities of daily living (20). The number of motor modules required to perform the motor tasks is higher than what previously described. Twelve synergies are responsible for most of the variation in hand grasping. The first three are used as primary synergies, while the remaining ones target finer movements (e.g. independence of thumb and index finger). The results generalize the description of hand kinematics, better clarifying several limits of the field and fostering the development of applications in rehabilitation and assistive robotics.

Neural network training for cross-protocol radiomic feature standardization in computed tomography

Scientifique ArODES

Andrearczyk Vincent, Depeursinge Adrien, Müller Henning

Journal of Medical Imaging, 2019, vol. 6, no. 2

Lien vers la publication

Résumé:

Radiomics has shown promising results in several medical studies, yet it suffers from a limited discrimination and informative capability as well as a high variation and correlation with the tomographic scanner types, pixel spacing, acquisition protocol, and reconstruction parameters. We propose and compare two methods to transform quantitative image features in order to improve their stability across varying image acquisition parameters while preserving the texture discrimination abilities. In this way, variations in extracted features are representative of true physiopathological tissue changes in the scanned patients. A first approach is based on a two-layer neural network that can learn a nonlinear standardization transformation of various types of features including handcrafted and deep features. Second, domain adversarial training is explored to increase the invariance of the transformed features to the scanner of origin. The generalization of the proposed approach to unseen textures and unseen scanners is demonstrated by a set of experiments using a publicly available computed tomography texture phantom dataset scanned with various imaging devices and parameters.

ImageCLEF 2019 :

Chapter ArODES

multimedia retrieval in lifelogging, medical, nature, and security applications

Ionescu Bogdan, Müller Henning, Péteri Renaud, Dicente Cid Yashin, et al.

Dans Azzopardi, Leif, et al., Advances in Information Retrieval : 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part II (pp. 301-308). 2019, Cham : Springer

Lien vers la publication

Résumé:

This paper presents an overview of the foreseen ImageCLEF 2019 lab that will be organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2019. ImageCLEF is an ongoing evaluation initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2019, the 17th edition of ImageCLEF will run four main tasks: (i) a Lifelog task (videos, images and other sources) about daily activities understanding, retrieval and summarization, (ii) a Medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with newer data, (iii) a new Coral task about segmenting and labeling collections of coral images for 3D modeling, and (iv) a new Security task addressing the problems of automatically identifying forged content and retrieve hidden information. The strong participation, with over 100 research groups registering and 31 submitting results for the tasks in 2018 shows an important interest in this benchmarking campaign and we expect the new tasks to attract at least as many researchers for 2019.

ImageCLEF 2019 :

Chapter ArODES

biodiversity identification and prediction challenges

Joly Alexis, Goëau Hervé, Botella Christophe, Müller Henning, et al.

Lien vers la publication

Résumé:

Building accurate knowledge of the identity, the geographic distribution and the evolution of living species is essential for a sustainable development of humanity, as well as for biodiversity conservation. However, the burden of the routine identication of plants and animals in the eld is strongly penalizing the aggregation of new data and knowledge. Identifying and naming living plants or animals is actually almost impossible for the general public and often a dicult task for professionals and naturalists. Bridging this gap is a key challenge towards enabling eective biodiversity information retrieval systems. The LifeCLEF evaluation campaign, presented in this paper, aims at boosting and evaluating the advances in this domain since 2011. In particular, the 2019 edition proposes three data-oriented challenges related to the identication and prediction of biodiversity: (i) an image-based plant identication challenge, (ii) a bird sounds identication challenge and (iii) a location-based species prediction challenge based on spatial occurrence data and environmental tensors.

Revealing tumor habitats from texture heterogeneity analysis for classification of lung cancer malignancy and aggressiveness

Scientifique ArODES

Cherezo Dmitri, Goldgof Dmitri, Hall Lawrence, Gillies Robert, Schabath Matthew, Müller Henning, Depeursinge Adrien

Scientific reports, Mars 2019, vol. 9, article no. 4500

Lien vers la publication

Résumé:

We propose an approach for characterizing structural heterogeneity of lung cancer nodules using Computed Tomography Texture Analysis (CTTA). Measures of heterogeneity were used to test the hypothesis that heterogeneity can be used as predictor of nodule malignancy and patient survival. To do this, we use the National Lung Screening Trial (NLST) dataset to determine if heterogeneity can represent differences between nodules in lung cancer and nodules in non-lung cancer patients. 253 participants are in the training set and 207 participants in the test set. To discriminate cancerous from non-cancerous nodules at the time of diagnosis, a combination of heterogeneity and radiomic features were evaluated to produce the best area under receiver operating characteristic curve (AUROC) of 0.85 and accuracy 81.64%. Second, we tested the hypothesis that heterogeneity can predict patient survival. We analyzed 40 patients diagnosed with lung adenocarcinoma (20 short-term and 20 long-term survival patients) using a leave-one-out cross validation approach for performance evaluation. A combination of heterogeneity features and radiomic features produce an AUROC of 0.9 and an accuracy of 85% to discriminate long- and short-term survivors.

Heterogeneous exascale computing

Chapter ArODES

Hluchý Ladislav, Bobák Martin, Müller Henning, Graziani Mara, Heikkurinen Matti, et al.

Dans Haidegger, Tamás, Kovács, Levente, Szakál, Anikó, Recent advances in intelligent engineering : volume dedicated to Imre J. Rudas’ Seventieth Birthday (pp. 81-110). 2019, Cham : Springer

Lien vers la publication

Résumé:

Exascale services bring new unique challenges that the current computational, big data and workflow solutions are unable to meet. The chapter includes a detailed description of selected exascale services with known state of the art in extreme date solutions. The integration of requirements and the analysis of the state of the art in the exascale field is centered in on a description of a high-level architectural approach. The next main contribution of the paper is the description of the architecture capable to handle heterogeneous exascale services coming from both academic as well as industrial sphere. Those two models represent a (conceptual, and technological) design of a platform that addresses the requirements of the use cases. The resulting architecture will help us provide computing solutions to exascale challenges within the H2020 project PROCESS.

Medical decision support using increasingly large multimodal data sets

Chapter ArODES

Müller Henning, Unay Devrim

Dans et al., Vrochidis, Stefanos, Big data analytics for large-scale multimedia search (20 p.). 2019, [Hoboken] : John Wiley & Sons

Lien vers la publication

Résumé:

Medical decision support has traditionally been using model‐based approaches and small data sets for evaluation. This chapter analyses recent trends and techniques that make use of increasingly large data sets and thus more data‐driven approaches to medical decision support that have in some areas replaced the more traditional rule‐based approaches. It explains the challenge infrastructures and approaches as they are often essential to access data, and application scenarios give examples of existing applications and objectives. The chapter focuses on how to overcome current limitations and how to tackle the upcoming challenges of image‐based decision support for digital medicine. It reviews the literature of mainly the past five years in the field of medical visual decision support and highlights the use of multimodal data and data‐driven approaches. Machine learning is an indispensable part of medical decision support, especially in diagnostic and categorization applications.

A quantitative taxonomy of human hand grasps

Scientifique ArODES

Stival Francesca, Michieletto Stefano, Cognolato Matteo, Pagello Enrico, Müller Henning, Atzori Manfredo

Journal of neuroengineering and rehabilitation, 2019, vol. 16, no. 28

Lien vers la publication

Résumé:

A proper modeling of human grasping and of hand movements is fundamental for robotics, prosthetics, physiology and rehabilitation. The taxonomies of hand grasps that have been proposed in scientific literature so far are based on qualitative analyses of the movements and thus they are usually not quantitatively justified. Methods This paper presents to the best of our knowledge the first quantitative taxonomy of hand grasps based on biomedical data measurements. The taxonomy is based on electromyography and kinematic data recorded from 40 healthy subjects performing 20 unique hand grasps. For each subject, a set of hierarchical trees are computed for several signal features. Afterwards, the trees are combined, first into modality-specific (i.e. muscular and kinematic) taxonomies of hand grasps and then into a general quantitative taxonomy of hand movements. The modality-specific taxonomies provide similar results despite describing different parameters of hand movements, one being muscular and the other kinematic. Results The general taxonomy merges the kinematic and muscular description into a comprehensive hierarchical structure. The obtained results clarify what has been proposed in the literature so far and they partially confirm the qualitative parameters used to create previous taxonomies of hand grasps. According to the results, hand movements can be divided into five movement categories defined based on the overall grasp shape, finger positioning and muscular activation. Part of the results appears qualitatively in accordance with previous results describing kinematic hand grasping synergies. Conclusions The taxonomy of hand grasps proposed in this paper clarifies with quantitative measurements what has been proposed in the field on a qualitative basis, thus having a potential impact on several scientific fields.

2018

Evaluation-as-a-service for the computational sciences :

Scientifique ArODES

overview and outlook

Hopfgartner Frank, Hanbury Allan, Müller Henning, Eggel Ivan, Krisztian Balog, Brodt Torben, Cormack Gordon V., Lin Jimmy, Kalpathy-Cramer Jayashree, Kando Noriko, Kato Makoto P., Krithara Anastasia, Gollub Tim, Potthast Martin, Viegas Evelyne, Mercer Simon

Journal of data and information quality, October 2018, vol. 10, no. 4, pp. 1-32

Lien vers la publication

Résumé:

Evaluation in empirical computer science is essential to show progress and assess technologies developed. Several research domains such as information retrieval have long relied on systematic evaluation to measure progress: here, the Cranfield paradigm of creating shared test collections, defining search tasks, and collecting ground truth for these tasks has persisted up until now. In recent years, however, several new challenges have emerged that do not fit this paradigm very well: extremely large data sets, confidential data sets as found in the medical domain, and rapidly changing data sets as often encountered in industry. Also, crowdsourcing has changed the way that industry approaches problem-solving with companies now organizing challenges and handing out monetary awards to incentivize people to work on their challenges, particularly in the field of machine learning. This white paper is based on discussions at a workshop on Evaluation-as-a-Service (EaaS). EaaS is the paradigm of not providing data sets to participants and have them work on the data locally, but keeping the data central and allowing access via Application Programming Interfaces (API), Virtual Machines (VM) or other possibilities to ship executables. The objective of this white paper are to summarize and compare the current approaches and consolidate the experiences of these approaches to outline the next steps of EaaS, particularly towards sustainable research infrastructures. This white paper summarizes several existing approaches to EaaS and analyzes their usage scenarios and also the advantages and disadvantages. The many factors influencing EaaS are overviewed, and the environment in terms of motivations for the various stakeholders, from funding agencies to challenge organizers, researchers and participants, to industry interested in supplying real-world problems for which they require solutions.

Overview of LifeCLEF 2018 :

Chapter ArODES

a large-scale evaluation of species identification and recommendation algorithms in the era of AI

Joly Alexis, Goëau Hervé, Botella Christophe, Glotin Hervé, Bonnet Pierre, Vellinga Willem-Pier, Planqué Robert, Müller Henning

Dans Bellot, P., Experimental IR meets multilinguality, multimodality, and interaction : 9th International Conference of the CLEF Association, CLEF 2018, Avignon, France, September 10-14, 2018, Proceedings (Pp. 247-266). 2018, Cham : Springer

Lien vers la publication

Résumé:

Building accurate knowledge of the identity, the geographic distribution and the evolution of living species is essential for a sustain-able development of humanity, as well as for biodiversity conservation. Unfortunately, such basic information is often only partially available for professional stakeholders, teachers, scientists and citizens, and often incomplete for ecosystems that possess the highest diversity. In this con-text, an ultimate ambition is to set up innovative information systems relying on the automated identification and understanding of living or-ganisms as a means to engage massive crowds of observers and boost the production of biodiversity and agro-biodiversity data. The LifeCLEF 2018 initiative proposes three data-oriented challenges related to this vi-sion, in the continuity of the previous editions, but with several consis-tent novelties intended to push the boundaries of the state-of-the-art in several research directions. This paper describes the methodology of the conducted evaluations as well as the synthesis of the main results and lessons learned.

Glaucoma diagnosis from eye fundus images based on deep morphometric feature estimation

Chapter ArODES

Perdomo Oscar, Andrearczyk Vincent, Meriaudeau Fabrice, Müller Henning, González Fabio

Dans Stoyanov, Danail, Computational pathology and ophthalmic medical image analysis (8 p.). 2018, Cham : Springer

Lien vers la publication

Résumé:

Glaucoma is an ophthalmic disease related to damage in the optic nerve and it is without symptoms in its early stages. Left untreated, it can lead to vision limitation and blindness. Eye fundus images have been widely accepted by medical personnel to examine the morphology and texture of the optic nerve head and the physiologic cup but glaucoma diagnosis is still subjective and without clear consensus among experts. This paper presents a multi-stage deep learning model for glaucoma diagnosis based on a curriculum learning strategy. In curriculum learning, a model is sequentially trained to solve incrementally difficult tasks. Our proposed model includes the following stages: segmentation of the optic disc and physiological cup, prediction of morphometric features from segmentations, and prediction of disease level (healthy, suspicious and glaucoma). The experimental evaluation shows that our proposed method outperforms conventional convolutional deep learning models from the state of the art reported on the RIM-ONE-v1 and DRISHTI-GS1 datasets with an accuracy of 89.4% and an AUC of 0.82 respectively.

Overview of ImageCLEF 2018 :

Chapter ArODES

challenges, datasets and evaluation

Ionescu Bogdan, Müller Henning, Villegas Mauricio, García Seco de Herrera Alba, Eickhoff Carsten, Andrearczyk Vincent, Dicente Cid Yashin, Liauchuk Vitali, Kovalev Vassili, Hasan Sadid A., Ling Yuan, Farri Oladimeji, Liu Joey, Lungren Matthew, Dang-Nguyen Duc-Tien, Piras Luca, Riegler Michael, Zhou Liting, Lux Mathias, Gurrin Cathal

Lien vers la publication

Résumé:

Abstract. This paper presents an overview of the ImageCLEF 2018 evaluation campaign, an event that was organized as part of the CLEF (Conference and Labs of the Evaluation Forum) Labs 2018. ImageCLEF is an ongoing initiative (it started in 2003) that promotes the evalua-tion of technologies for annotation, indexing and retrieval with the aim of providing information access to collections of images in various us-age scenarios and domains. In 2018, the 16th edition of ImageCLEF ran three main tasks and a pilot task: 1) a caption prediction task that aims at predicting the caption of a figure from the biomedical literature based only on the figure image; 2) a tuberculosis task that aims at detecting the tuberculosis type, severity and drug resistance from CT (Computed Tomography) volumes of the lung; 3) a LifeLog task (videos, images and other sources) about daily activities understanding and moment re-trieval, and 4) a pilot task on visual question answering where systems are tasked with answering medical questions. The strong participation, with over 100 research groups registering and 31 submitting results for the tasks, shows an increasing interest in this benchmarking campaign.

Muscle synergy analysis of a hand-grasp dataset :

Scientifique ArODES

a limited subset of motor modules may underlie a large variety of grasps

Scano Alessandro, Chiavenna Andrea, Molinari Tosatti Lorenzo, Müller Henning, Atzori Manfredo

Frontiers in Neurorobotics, 2018, vol. 12

Lien vers la publication

Résumé:

Kinematic and muscle patterns underlying hand grasps have been widely investigated in the literature. However, the identification of a reduced set of motor modules, generalizing across subjects and grasps, may be valuable for increasing the knowledge of hand motor control, and provide methods to be exploited in prosthesis control and hand rehabilitation. Methods: Motor muscle synergies were extracted from a publicly available database including 28 subjects, executing 20 hand grasps selected for daily-life activities. The spatial synergies and temporal components were analyzed with a clustering algorithm to characterize the patterns underlying hand-grasps. Results: Motor synergies were successfully extracted on all 28 subjects. Clustering orders ranging from 2 to 50 were tested. A subset of ten clusters, each one represented by a spatial motor module, approximates the original dataset with a mean maximum error of 5% on reconstructed modules; however, each spatial synergy might be employed with different timing and recruited at different grasp stages. Two temporal activation patterns are often recognized, corresponding to the grasp/hold phase, and to the pre-shaping and release phase. Conclusions: This paper presents one of the biggest analysis of muscle synergies of hand grasps currently available. The results of 28 subjects performing 20 different grasps suggest that a limited number of time dependent motor modules (shared among subjects), correctly elicited by a control activation signal, may underlie the execution of a large variety of hand grasps. However, spatial synergies are not strongly related to specific motor functions but may be recruited at different stages, depending on subject and grasp. This result can lead to applications in rehabilitation and assistive robotics.

How to exploit weaknesses in biomedical challenge design and organization

Chapter ArODES

Reinke Annika, Eisenmann Matthias, Onogur Sinan, Stankovic Marko, Scholz Patrick, Full Peter M., Bogunovic Hrvoje, Landman Bennett A., Maier Oskar, Menze Bjoern, Sharp Gregory C., Sirinukunwattana Korsuk, Speidel Stefanie, Van der Sommen Fons, Zheng Guoyan, Müller Henning, Kozubek Michal, Arbel Tal, Bradley Andrew P., Jannin Pierre, Kopp-Schneider Annette, Maier-Hein Lena

Dans Frangi, Alejandro F., Medical image computing and computer assisted intervention – MICCAI 2018 : 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part IV (Pp. 388-395). 2018, Cham : Springer

Lien vers la publication

Résumé:

Since the first MICCAI grand challenge organized in 2007 in Brisbane, challenges have become an integral part of MICCAI conferences. In the meantime, challenge datasets have become widely recognized as international benchmarking datasets and thus have a great influence on the research community and individual careers. In this paper, we show several ways in which weaknesses related to current challenge design and organization can potentially be exploited. Our experimental analysis, based on MICCAI segmentation challenges organized in 2015, demonstrates that both challenge organizers and participants can potentially undertake measures to substantially tune rankings. To overcome these problems we present best practice recommendations for improving challenge design and organization.

From local to global :

Chapter ArODES

a holistic lung graph model

Dicente Cid Yashin, Jiménez del Toro Oscar Alfonso, Platon Alexandra, Müller Henning, Poletti Pierre-Alexandre

Lien vers la publication

Résumé:

Lung image analysis is an essential part in the assessment of pulmonary diseases. Through visual inspection of CT scans, radiologists detect abnormal patterns in the lung parenchyma, aiming to establish a timely diagnosis and thus improving patient outcome. However, in a generalized disorder of the lungs, such as pulmonary hypertension, the changes in organ tissue can be elusive, requiring additional invasive studies to confirm the diagnosis. We present a graph model that quantifies lung texture in a holistic approach enhancing the analysis between pathologies with similar local changes. The approach extracts local state-of-the-art 3D texture descriptors from an automatically generated geometric parcellation of the lungs. The global texture distribution is encoded in a weighted graph that characterizes the correlations among neighboring organ regions. A data set of 125 patients with suspicion of having a pulmonary vascular pathology was used to evaluate our method. Three classes containing 47 pulmonary hypertension, 31 pulmonary embolism and 47 control cases were classified in a one vs. one setup. An area under the curve of up to 0.85 was obtained adding directionality to the edges of the graph architecture. The approach was able to identify diseased patients, and to distinguish pathologies with abnormal local and global blood perfusion defects.

Image magnification regression using DenseNet for exploiting histopathology open access content

Chapter ArODES

Otálora Sebastian, Atzori Manfredo, Andrearczyk Vincent, Müller Henning

Dans Stoyanov, Danail, Computational pathology and ophthalmic medical image analysis (pp. 148-155). 2018, Cham : Springer

Lien vers la publication

Résumé:

Open access medical content databases such as PubMed Central and TCGA offer possibilities to obtain large amounts of images for training deep learning models. Nevertheless, accurate labeling of large-scale medical datasets is not available and poses challenging tasks for using such datasets. Predicting unknown magnification levels and standardize staining procedures is a necessary preprocessing step for using this data in retrieval and classification tasks. In this paper, a CNN-based regression approach to learn the magnification of histopathology images is presented, comparing two deep learning architectures tailored to regress the magnification. A comparison of the performance of the models is done in a dataset of 34,441 breast cancer patches with several magnifications. The best model, a fusion of DenseNet-based CNNs, obtained a kappa score of 0.888. The methods are also evaluated qualitatively on a set of images from biomedical journals and TCGA prostate patches.

Deep multimodal classification of image types in biomedical journal figures

Chapter ArODES

Adrearczyk Vincent, Müller Henning

Dans Bellot, P., Mothe, J, Trabelsi, C., Experimental IR meets multilinguality, multimodality, and interaction : 9th International Conference of the CLEF Association, CLEF 2018, Avignon, France, September 10-14, 2018, Proceedings (Pp. 3-14). 2018, Cham : Springer

Lien vers la publication

Résumé:

This paper presents a robust method for the classification of medical image types in figures of the biomedical literature using the fusion of visual and textual information. A deep convolutional network is trained to discriminate among 31 classes including compound figures, diagnostic image types and generic illustrations, while another shallow convolutional network is used for the analysis of the captions paired with the images. Various fusion methods are analyzed as well as data augmentation approaches. The proposed system is validated on the ImageCLEF 2013 classification task, largely improving the currently best performance from 83.5% to 93.7% accuracy.

Textured graph-based model of the lungs :

Chapter ArODES

application on tuberculosis type classification and multi-drug resistance detection

Dicente Cid Yashin, Batmanghelich Kayhan, Müller Henning

Dans Bello, Patrice, Mothe, Josiane, Murtagh, Fionn, Trabelsi, Chiraz, Experimental IR Meets Multilinguality, Multimodality, and Interaction : 9th International Conference of the CLEF Association, CLEF 2018, Avignon, France, September 10-14, 2018, Proceedings (pp. 157-168). 2018, Cham : Springer

Lien vers la publication

Résumé:

Tuberculosis (TB) remains a leading cause of death worldwide. Two main challenges when assessing computed tomography scans of TB patients are detecting multi{drug resistance and di_erentiating TB types. In this article we model the lungs as a graph entity where nodes represent anatomical lung regions and edges encode interactions between them. This graph is able to characterize the texture distribution along the lungs, making it suitable for describing patients with different TB types. In 2017, the ImageCLEF benchmark proposed a task based on computed tomography volumes of patients with TB. This task was divided into two subtasks: multi{drug resistance prediction, and TB type classi_cation. The participation in this task showed the strength of our model, leading to best results in the competition for multi{drug resistance detection (AUC = 0.5825) and good results in the TB type classi_cation (Cohen's Kappa coe_cient = 0.1623).

Subdiv17 :

Chapter ArODES

a dataset for investigating subjectivity in the visual diversification of image search results

Rohm Maia, Ionescu Bogdan, Gînsca Alexandru Lucian, Santos Rodrygo, Müller Henning

MMSys'18 9th ACM Multimedia Systems Conference (pp. 444-449). 2018, New York : ACM

Lien vers la publication

Résumé:

In this paper, we present a new dataset that facilitates the comparison of approaches aiming at the diversification of image search results. The dataset was explicitly designed for general-purpose, multi-topic queries and provides multiple ground truth annotations to allow for the exploration of the subjectivity aspect in the general task of diversification. The dataset provides images and their metadata retrieved from Flickr for around 200 complex queries. Additionally, to encourage experimentations (and cooperations) from different communities such as information and multimedia retrieval, a broad range of pre-computed descriptors is provided. The proposed dataset was successfully validated during the MediaEval 2017 Retrieving Diverse Social Images task using 29 submitted runs.

Head-mounted eye gaze tracking devices :

Scientifique ArODES

an overview of modern devices and recent advances

Cognolato Matteo, Atzori Manfredo, Müller Henning

Journal of rehabilitation and assistive technologies engineering, 2018, vol. 5, pp. 1-13

Lien vers la publication

Résumé:

An increasing number of wearable devices performing eye gaze tracking have been released in recent years. Such devices can lead to unprecedented opportunities in many applications. However, staying updated regarding the continuous advances and gathering the technical features that allow to choose the best device for a specific application is not trivial. The last eye gaze tracker overview was written more than 10 years ago, while more recent devices are substantially improved both in hardware and software. Thus, an overview of current eye gaze trackers is needed. This review fills the gap by providing an overview of the current level of advancement for both techniques and devices, leading finally to the analysis of 20 essential features in six head-mounted eye gaze trackers commercially available. The analyzed characteristics represent a useful selection providing an overview of the technology currently implemented. The results show that many technical advances were made in this field since the last survey. Current wearable devices allow to capture and exploit visual information unobtrusively and in real time, leading to new applications in wearable technologies that can also be used to improve rehabilitation and enable a more active living for impaired persons.

Tumor proliferation assessment of whole slide images

Scientifique ArODES

Rousson Mikael, Hedlunda Martin, Anderssona Mats, Jacobsson Ludwig, Lathen Gunnar, Norella Bjorn, Jiménez del Toro Oscar Alfonso, Müller Henning, Atzori Manfredo

Medical Imaging 2018 (SPIE) : Digital Pathology, March 2018, vol. 10581

Lien vers la publication

Résumé:

Grading whole slide images (WSIs) from patient tissue samples is an important task in digital pathology, particularly for diagnosis and treatment planning. However, this visual inspection task, performed by pathologists, is inherently subjective and has limited reproducibility. Moreover, grading of WSIs is time consuming and expensive. Designing a robust and automatic solution for quantitative decision support can improve the objectivity and reproducibility of this task. This paper presents a fully automatic pipeline for tumor proliferation assessment based on mitosis counting. The approach consists of three steps: i) region of interest selection based on tumor color characteristics, ii) mitosis counting using a deep network based detector, and iii) grade prediction from ROI mitosis counts. The full strategy was submitted and evaluated during the Tumor Proliferation Assessment Challenge (TUPAC) 2016. TUPAC is the rst digital pathology challenge grading whole slide images, thus mimicking more closely a real case scenario. The pipeline is extremely fast and obtained the 2nd place for the tumor proliferation assessment task and the 3rd place in the mitosis counting task, among 17 participants. The performance of this fully automatic method is similar to the performance of pathologists and this shows the high quality of automatic solutions for decision support.

Rotation-covariant tissue analysis for interstitial lung diseases using learned steerable filters :

Scientifique ArODES

performance evaluation and relevance for diagnostic aid

Joyseeree Ranveer, Müller Henning, Depeursinge Adrien

Computerized medical imaging and graphics, March 2018, vol. 64, pp. 1-11

Lien vers la publication

Résumé:

A novel method to detect and classify several classes of diseased and healthy lung tissue of interstitial lung diseases is presented, as these diseases are hard to diagnose and differentiate. Local organizations of image directions at several scales drive the process of creating discriminative lung tissue texture signatures using spatial and Fourier domain information extracted from the images. The signatures are generated for four diseased tissue classes and healthy tissue, all of which appear in the Interstitial Lung Disease (ILD) database, using a novel one-versus-one approach for learning discriminative filter signatures. A multiclass tissue classification accuracy of 80.31% is observed using Radial Basis Function (RBF) Support Vector Machines (SVMs). The presented method compares well against a variety of state-of-the-art approaches. Another strong feature of our approach is the ability to access the individual class probabilities before a final classification decision is made. This enables an analysis of the causes of misclassification in this paper. We also make the case against total reliance on the accuracy of the ground truth given that the ILD database only contains a single label for a specific region and sometimes more than one pattern can be present, particularly for regions classified as healthy tissue. Measures to address misclassifications in this context are also proposed.

Large-scale retrieval for medical image analytic :

Scientifique ArODES

a comprehensive review

Li Zhongyu, Zhang Xiaofan, Müller Henning, Zhang Shaoting

Medical image analysis, 2018, vol. 43, pp. 66-84

Lien vers la publication

Résumé:

Over the past decades, medical image analytics was greatly facilitated by the explosion of digital imaging techniques, where huge amounts of medical images were produced with ever-increasing quality and diversity. However, conventional methods for analyzing medical images have achieved limited success, as they are not capable to tackle the huge amount of image data. In this paper, we review state-of-the-art approaches for large-scale medical image analysis, which are mainly based on recent advances in computer vision, machine learning and information retrieval. Specifically, we first present the general pipeline of large-scale retrieval, summarize the challenges/opportunities of medical image analytics on a large-scale. Then, we provide a comprehensive review of algorithms and techniques relevant to major processes in the pipeline, including feature representation, feature indexing, searching, etc. On the basis of existing work, we introduce the evaluation protocols and multiple applications of large-scale medical image retrieval, with a variety of exploratory and diagnostic scenarios. Finally, we discuss future directions of large-scale retrieval, which can further improve the performance of medical image analysis.

2017

Datasets column :

Scientifique ArODES

diversity and credibility for social images and image retrieval

Ionescu Bogdan, Lupu Mihai, Rohm Maia, Gînsca Alexandru Lucian, Müller Henning

ACM SIGMultimedia Records, December 2017, vol. 9, no. 3

Lien vers la publication

Shangri–La :

Scientifique ArODES

a medical case–based retrieval tool

García Seco de Herrera Alba, Schaer Roger, Müller Henning

Journal of the association for information science and technology, 2017, vol. 68, no 11, pp. 2587–2601

Lien vers la publication

Résumé:

Large amounts of medical visual data are produced in hospitals daily and made available continuously via publications in the scientific literature, representing the medical knowledge. However, it is not always easy to find the desired information and in clinical routine the time to fulfil an information need is often very limited. Infor-mation retrieval systems are a useful tool to provide access to these documents/images in the biomedical literature related to information needs of medical professionals. Shangri-La is a medical retrieval sys-tem that can potentially help clinicians to make decisions on diﬃcult cases. It retrieves articles from the biomedical literature when query-ing a case description and attached images. The system is based on a multi-modal retrieval approach with a focus on the integration of visual information connected to text. The approach includes a query-adaptive multi-modal fusion criterion that analyses if visual features are suitable to be fused with text for the retrieval. Furthermore, image modality information is integrated in the retrieval step. The approach is evaluated using the ImageCLEFmed 2013 medical retrieval bench-mark and can thus be compared to other approaches. Results show that the final approach outperforms the best multi-modal approach submitted to ImageCLEFmed 2013.

Comparison of six electromyography acquisition setups on hand movement classification tasks

Scientifique ArODES

Pizzolato Stefano, Tagliapietra Luca, Cognolato Matteo, Reggiani Monica, Müller Henning, Atzori Manfredo

Plos One, 2017, vol.

Lien vers la publication

Résumé:

Hand prostheses controlled by surface electromyography are promising due to the non-invasive approach and the control capabilities offered by machine learning. Nevertheless, dexterous prostheses are still scarcely spread due to control difficulties, low robustness and often prohibitive costs. Several sEMG acquisition setups are now available, ranging in terms of costs between a few hundred and several thousand dollars. The objective of this paper is the relative comparison of six acquisition setups on an identical hand movement classification task, in order to help the researchers to choose the proper acquisition setup for their requirements. The acquisition setups are based on four different sEMG electrodes (including Otto Bock, Delsys Trigno, Cometa Wave + Dormo ECG and two Thalmic Myo armbands) and they were used to record more than 50 hand movements from intact subjects with a standardized acquisition protocol. The relative performance of the six sEMG acquisition setups is compared on 41 identical hand movements with a standardized feature extraction and data analysis pipeline aimed at performing hand movement classification. Comparable classification results are obtained with three acquisition setups including the Delsys Trigno, the Cometa Wave and the affordable setup composed of two Myo armbands. The results suggest that practical sEMG tests can be performed even when costs are relevant (e.g. in small laboratories, developing countries or use by children). All the presented datasets can be used for offline tests and their quality can easily be compared as the data sets are publicly available.

Training deep convolutional neural networks with active learning for exudate classification in eye fundus images

Chapter ArODES

Otálora Sebastian, Perdomo Oscar, González Fabio, Müller Henning

LABELS 2017, CVII 2017, STENT 2017 : intravascular imaging and computer assisted stenting, and large-scale annotation of biomedical data and expert label synthesis (pp. 146-154). 2017, Cham : Springer

Lien vers la publication

Résumé:

Training deep convolutional neural network for classification in medical tasks is often diﬃcult due to the lack of annotated data sam-ples. Deep convolutional networks (CNN) has been successfully used as an automatic detection tool to support the grading of diabetic retinopa-thy and macular edema. Nevertheless, the manual annotation of exu-dates in eye fundus images used to classify the grade of the DR is very time consuming and repetitive for clinical personnel. Active learning al-gorithms seek to reduce the labeling eﬀort in training machine learning models. This work presents a label-eﬃcient CNN model using the ex-pected gradient length, an active learning algorithm to select the most informative patches and images, converging earlier and to a better local optimum than the usual SGD (Stochastic Gradient Descent) strategy. Our method also generates useful masks for prediction and segments regions of interest.

Retrieval from and understanding of large–scale multi–modal medical datasets :

Scientifique ArODES

a review

Müller Henning, Unay Devrim

IEEE transactions on multimedia, 2017, vol. 19, no. 9, pp. 2093-2104

Lien vers la publication

Résumé:

Content–based multimedia retrieval has been an active research domain since the mid 1990s. In the medical domain visual retrieval started later and has mostly remained a research instrument and less a clinical tool, even though a few tools for retrieval are employed in clinical work. The limited size of data sets due to privacy constraints is often mentioned as a reason for these limitations. Nevertheless, much work has been done in medical visual information retrieval, including the availability of increasingly large data sets and scientific challenges. Annotated data sets and clinical data for the images have now become available and can be combined for multi– modal retrieval. Much has been learned on user behavior and application scenarios. This text is motivated by the advances in medical image analysis and the availability of more public data large data sets that often include clinical data that can be combined for multimodal retrieval based on the experience available in the multimedia community. This text is a systematic review of recent work (concentrating on the period between 2011-2017) on content–based multi–modal retrieval and image understanding in the medical domain, where image understanding includes techniques such as detection, localization, and classification for leveraging visual content. The main conferences in the field are screened for relevant articles and these are presented in a structured way, identifying current limitations and areas where work is still much required. Objective of the work is to summarize the current state of research for multimedia researchers not working in the medical field. It provides ways to get data sets and identify promising research directions. The text highlights the areas of advances in the past six years and particularly a trend to use larger scale training data sets as well as deep learning approaches that can replace or complement hand–crafted feature extraction. Using images alone will likely only work in limited sub domains but combining multiple sources of data for multi–modal retrieval has the biggest chances of success, particularly for clinical impact. Future fields of research are identified in the text, as there is a high research potential in the medical multimedia domain.

Deep multimodal case–based retrieval for large histopathology datasets

Chapter ArODES

Jiménez del Toro Oscar Alfonso, Otálora Sebastian, Atzori Manfredo, Müller Henning

Patch-based techniques in medical imaging : third International Workshop, Patch-MI 2017, held in conjunction with MICCAI 2017, Quebec City, QC, Canada, September 14, 2017, Proceedings (pp. 149-157). 2017, Cham : Springer

Lien vers la publication

Résumé:

The current gold standard for interpreting patient tissue samples is the visual inspection of whole–slide histopathology images (WSIs) by pathologists. They generate a pathology report describing the main findings relevant for diagnosis and treatment planning. Search-ing for similar cases through repositories for diﬀerential diagnosis is often not done due to a lack of efficient strategies for medical case–based re-trieval. A patch–based multimodal retrieval strategy that retrieves sim-ilar pathology cases from a large data set fusing both visual and text information is explained in this paper. By fine–tuning a deep convolu-tional neural network an automatic representation is obtained for the vi-sual content of weakly annotated WSIs (using only a global cancer score and no manual annotations). The pathology text report is embedded into a category vector of the pathology terms also in a non–supervised approach. A publicly available data set of 267 prostate adenocarcinoma cases with their WSIs and corresponding pathology reports was used to train and evaluate each modality of the retrieval method. A MAP (Mean Average Precision) of 0.54 was obtained with the multimodal method in a previously unseen test set. The proposed retrieval system can help in diﬀerential diagnosis of tissue samples and during the training of pathol-ogists, exploiting the large amount of pathology data already existing digital hospital repositories.

Analysis of histopathology images :

Chapter ArODES

from traditional machine learning to deep learning

Jiménez del Toro Oscar Alfonso, Otálora Sebastian, Andersson Mats, Eurén Kristian, Hedlund Martin, Rousson Mikael, Müller Henning, Atzori Manfredo

Biomedical texture analysis : fundamentals, tools and challenges (pp. 281–314). 2017, [S. l.] : Elsevier

Lien vers la publication

Résumé:

Digitizing pathology is a current trend that makes large amounts of visual data available for automatic analysis. It allows to visualize and interpret pathologic cell and tissue samples in high-resolution images and with the help of computer tools. This opens the possibility to develop image analysis methods that help pathologists and support their image descriptions (i.e., staging, grading) with objective quantification of image features. Numerous detection, classification and segmentation algorithms of the underlying tissue primitives in histopathology images have been proposed in this respect. To better select the most suitable algorithms for histopathology tasks, biomedical image analysis challenges have evaluated and compared both traditional feature extraction with machine learning and deep learning techniques. This chapter provides an overview of methods addressing the analysis of histopathology images, as well as a brief description of the tasks they aim to solve. It is focused on histopathology images containing textured areas of different types.

Semi-automatic training of an object recognition system in scene camera data using gaze tracking and accelerometers

Chapter ArODES

Cognolato Matteo, Graziani Mara, Giordaniello Francesca, Saetta Gianluca, Bassetto , Brugger Peter, Caputo Barbara, Müller Henning, Atzori Manfredo

Computer Vision Systems : 11th International Conference, ICVS 2017, Shenzhen, China, July 10-13, 2017 (pp. 175-184). 2017, Cham : Springer

Lien vers la publication

Résumé:

Object detection and recognition algorithms usually require large, annotated training sets. The creation of such datasets requires expensive manual annotation. Eye tracking can help in the annotation procedure. Humans use vision constantly to explore the environment and plan motor actions, such as grasping an object. In this paper we investigate the possibility to semi-automatically train object recognition with eye tracking, accelerometer in scene camera data, learning from the natural hand-eye coordination of humans. Our approach involves three steps. First, sensor data are recorded using eye tracking glasses that are used in combination with accelerometers and surface electromyography that are usually applied when controlling prosthetic hands. Second, a set of patches are extracted automatically from the scene camera data while grasping an object. Third, a convolutional neural network is trained and tested using the extracted patches. Results show that the parameters of eye-hand coordination can be used to train an object recognition system semi-automatically. These can be exploited with proper sensors to fine-tune a convolutional neural network for object detection and recognition. This approach opens interesting options to train computer vision and multi-modal data integration systems and lays the foundations for future applications in robotics. In particular, this work targets the improvement of prosthetic hands by recognizing the objects that a person may wish to use. However, the approach can easily be generalized.

Combining radiology images and clinical metadata for multimodal medical case-based retrieval

Chapter ArODES

Jiménez del Toro Oscar Alfonso, Cirujeda Pol, Müller Henning

Cloud-based benchmarking of medical image analysis (pp. 221-236). 2017, Cham : Springer

Lien vers la publication

Résumé:

As part of their daily workload, clinicians examine patient cases in the process of formulating a diagnosis. These large multimodal patient datasets stored in hospitals could help in retrieving relevant information for a differential diagnosis, but these are currently not fully exploited. The VISCERAL Retrieval Benchmark organized a medical case-based retrieval algorithm evaluation using multimodal (text and visual) data from radiology reports. The common dataset contained patient CT (Computed Tomography) or MRI (Magnetic Resonance Imaging) scans and RadLex term anatomy–pathology lists from the radiology reports. A content-based retrieval method for medical cases that uses both textual and visual features is presented. It defines a weighting scheme that combines the anatomical and clinical correlations of the RadLex terms with local texture features obtained from the region of interest in the query cases. The visual features are computed using a 3D Riesz wavelet texture analysis performed on a common spatial domain to compare the images in the analogous anatomical regions of interest in the dataset images. The proposed method obtained the best mean average precision in 6 out of 10 topics and the highest number of relevant cases retrieved in the benchmark. Obtaining robust results for various pathologies, it could further be developed to perform medical case-based retrieval on large multimodal clinical datasets.

Retrieval of medical cases for diagnostic decisions :

Chapter ArODES

VISCERAL retrieval benchmark

Jiménez del Toro Oscar Alfonso, Müller Henning, Foncubierta-Rodriguez Antonio, Langs Georg, Hanbury Allan

Cloud-based benchmarking of medical image analysis (pp. 127-141). 2017, Cham : Springer

Lien vers la publication

Résumé:

Health providers currently construct their differential diagnosis for a given medical case most often based on textbook knowledge and clinical experience. Data mining of the large amount of medical records generated daily in hospitals is only very rarely done, limiting the reusability of these cases. As part of the VISCERAL project, the Retrieval benchmark was organized to evaluate available approaches for medical case-based retrieval. Participant algorithms were required to find and rank relevant medical cases from a large multimodal dataset (including semantic RadLex terms extracted from text and visual 3D data) for common query topics. The relevance assessment of the cases was done by medical experts who selected cases that are useful for a differential diagnosis for the given query case. The approaches that integrated information from both the RadLex terms and the 3D volumes (mixed techniques) obtained the best results based on five standard evaluation metrics. The benchmark set up, dataset description and result analysis are presented.

Text- and content-based medical image retrievals in the VISCERAL retrieval benchmark

Chapter ArODES

Zhang Fan, Song Yang, Cai Weidong, Depeursinge Adrien, Müller Henning

Cloud-based benchmarking of medical image analysis (pp. 237-249). 2017, Cham : Springer

Lien vers la publication

Résumé:

Text- and content-based retrieval are the most widely used approaches for medical image retrieval. They capture the similarity between the images from different perspectives: text-based methods rely on manual textual annotations or captions associated with images; content-based approaches are based on the visual content of the images themselves such as colours and textures. Text-based retrieval can better meet the high-level expectations of humans but is limited by the time-consuming annotations. Content-based retrieval can automatically extract the visual features for high-throughput processing; however, its performance is less favourable than the text-based approaches due to the gap between low-level visual features and high-level human expectations. In this chapter, we present the participation from our joint research team of USYD/HES-SO in the VISCERAL retrieval task. Five different methods are introduced, of which two are based on the anatomy–pathology terms, two are based on the visual image content and the last one is based on the fusion of the aforementioned methods. The comparison results, given the different methods indicated that the text-based methods outperformed the content-based retrieval and the fusion of text and visual contents, generated the best performance overall.

VISCERAL :

Chapter ArODES

evaluation-as-a-service for medical imaging

Hanbury Allan, Müller Henning

Cloud-based benchmarking of medical image analysis (pp. 3-13). 2017, Cham : Springer

Lien vers la publication

Résumé:

Systematic evaluation has had a strong impact on many data analysis domains, for example, TREC and CLEF in information retrieval, ImageCLEF in image retrieval, and many challenges in conferences such as MICCAI for medical imaging and ICPR for pattern recognition. With Kaggle, a platform for machine learning challenges has also had a significant success in crowdsourcing solutions. This shows the importance to systematically evaluate algorithms and that the impact is far larger than simply evaluating a single system. Many of these challenges also showed the limits of the commonly used paradigm to prepare a data collection and tasks, distribute these and then evaluate the participants’ submissions. Extremely large datasets are cumbersome to download, while shipping hard disks containing the data becomes impractical. Confidential data can often not be shared, for example medical data, and also data from company repositories. Real-time data will never be available via static data collections as the data change over time and data preparation often takes much time. The Evaluation-as-a-Service (EaaS) paradigm tries to find solutions for many of these problems and has been applied in the VISCERAL project. In EaaS, the data are not moved but remain on a central infrastructure. In the case of VISCERAL, all data were made available in a cloud environment. Participants were provided with virtual machines on which to install their algorithms. Only a small part of the data, the training data, was visible to participants. The major part of the data, the test data, was only accessible to the organizers who ran the algorithms in the participants’ virtual machines on the test data to obtain impartial performance measures.

Using the cloud as a platform for evaluation and data preparation

Chapter ArODES

Eggel Ivan, Schaer Roger, Müller Henning

Cloud-based benchmarking of medical image analysis (pp. 15-30). 2017, Cham : Springer

Lien vers la publication

Résumé:

This chapter gives a brief overview of the VISCERAL Registration System that is used for all the VISCERAL Benchmarks and is released as open source on GitHub. The system can be accessed by both participants and administrators, reducing the direct participant–organizer interaction and handling the documentation available for each of the benchmarks organized by VISCERAL. Also, the upload of the VISCERAL usage and participation agreements is integrated, as well as the attribution of virtual machines that allow participation in the VISCERAL Benchmarks. In the second part, a summary of the various steps in the continuous evaluation chain mainly consisting of the submission, algorithm execution and storage as well as the evaluation of results is given. The final part consists of the cloud infrastructure detail, describing the process of defining requirements, selecting a cloud solution provider, setting up the infrastructure and running the benchmarks. This chapter concludes with a short experience report outlining the encountered challenges and lessons learned.

Making sense of large data sets without annotations :

Chapter ArODES

analyzing age–related correlations from lung CT scans

Dicente Cid Yashin, Mamonov Artem, Beers Andrew, Thomas Armin, Kovalev Vassili, Kalpathy-Cramer Jayashree, Müller Henning

Proceedings Volume 10138, Medical Imaging 2017: Imaging Informatics for Healthcare, Research, and Applications (12 p.). 2017, Orlando, Florida : SPIE medical imaging

Lien vers la publication

Résumé:

The analysis of large data sets can help to gain knowledge about specific organs or on specific diseases, just as big data analysis does in many non-medical areas. This article aims to gain information from 3D volumes, so the visual content of lung CT scans of a large number of patients. In the case of the described data set, only little annotation is available on the patients that were all part of an ongoing screening program and besides age and gender no information on the patient and the findings was available for this work. This is a scenario that can happen regularly as image data sets are produced and become available in increasingly large quantities but manual annotations are often not available and also clinical data such as text reports are often harder to share. We extracted a set of visual features from 12,414 CT scans of 9,348 patients that had CT scans of the lung taken in the context of a national lung screening program in Belarus. Lung fields were segmented by two segmentation algorithms and only cases where both algorithms were able to find left and right lung and had a Dice coefficient above 0.95 were analyzed. This assures that only segmentations of good quality were used to extract features of the lung. Patients ranged in age from 0 to 106 years. Data analysis shows that age can be predicted with a fairly high accuracy for persons under 15 years. Relatively good results were also obtained between 30 and 65 years where a steady trend is seen. For young adults and older people the results are not as good as variability is very high in these groups. Several visualizations of the data show the evolution patters of the lung texture, size and density with age. The experiments allow learning the evolution of the lung and the gained results show that even with limited metadata we can extract interesting information from large-scale visual data. These age-related changes (for example of the lung volume, the density histogram of the tissue) can also be taken into account for the interpretation of new cases. The database used includes patients that had suspicions on a chest X-ray, so it is not a group of healthy people, and only tendencies and not a model of a healthy lung at a specific age can be derived.

3D solid texture classification using locally-oriented wavelet transforms

Scientifique ArODES

Dicente Cid Yashin, Müller Henning, Platon Alexandra, Poletti Pierre–Alexandre, Depeursinge Adrien

IEEE transactions on image processing, 2017, vol. 26, no. 4, pp. 1899-1910

Lien vers la publication

Résumé:

Many image acquisition techniques used in biomedical imaging, material analysis, and structural geology are capable of acquiring 3D solid images. Computational analysis of these images is complex but necessary, since it is difficult for humans to visualize and quantify their detailed 3D content. One of the most common methods to analyze 3D data is to characterize the volumetric texture patterns. Texture analysis generally consists of encoding the local organization of image scales and directions, which can be extremely diverse in 3D. Current state-of-the-art techniques face many challenges when working with 3D solid texture, where most approaches are not able to consistently characterize both scale and directional information. 3D Riesz-wavelets can deal with both properties. One key property of Riesz filterbanks is steerability, which can be used to locally align the filters and compare textures with arbitrary (local) orientations. This paper proposes and compares three novel local alignment criteria for higher-order 3D Riesz-wavelet transforms. The estimations of local texture orientations are based on higher-order extensions of regularized structure tensors. An experimental evaluation of the proposed methods for the classification of synthetic 3D solid textures with alterations (such as rotations and noise) demonstrated the importance of local directional information for robust and accurate solid texture recognition. These alignment methods achieved an accuracy of 0.95 in the rotated data, three times more than the unaligned Riesz descriptor that achieved 0.32. The accuracy obtained is better than all other techniques that are published and tested on the same database.

Big data in medical imaging, forensics and beyond

Chapter ArODES

Müller Henning

Forensigraphie : Möglichkeiten und Grenzen IT-gesützter klinisch-forensischer Bildgebung (10 p.). 2017, [S. l.] : J. Sramek

Lien vers la publication

Elettromiografia, protesica e robotica in rapido progresso verso l'amputazione funzionale :

Scientifique ArODES

i risultati del progetto Ninapro

Atzori Manfredo, Tiengo Cesare, Bassetto Franco, Müller Henning

Rivista di chirurgia della mano,

Lien vers la publication

Résumé:

Hand amputation can dramatically affect the capabilities of a person. Improving the functionality of robotic prosthetic hands is thus a challenge. The integration of advanced prosthetic and robotic technologies with functional amputations may bring to reality the non-invasive natural control of robotic hand prostheses in a near future. Scientific research and prosthetic market are rapidly advancing towards the natural control of dexterous robotic prosthetic hands. Myoelectric hand prostheses with many degrees of freedom are commercially available and recent advances in scientific research suggest that their natural control can be performed in real life through pattern recognition and the integration of multimodal data. However, robustness is still not sufficient to transfer scientific results to a real life. In this work we describe the Ninapro (Non Invasive Adaptive Prosthetics) database, which is aimed to study the relationships between sEMG, hand movement, force and clinical parameters. The data are publicly available to worldwide research groups. The Ninapro database allowed to obtain several important results including: showing that up to 11 hand movements can be recognized without any training in amputated subject; showing that multimodal data can strongly improve movement recognition; showing that several clinical parameters (including remaining forearm percentage and phantom limb sensation) are related to the capability of amputees to control the remnant muscles in the stump. 20 The Ninapro results, in combination with other scientific literature achievements, suggests that future "functional amputation" surgery procedures may better integrate with the prosthetic robotic limbs and contribute to solve natural control problems.

A lung graph–model for pulmonary hypertension and pulmonary embolism detection on DECT images

Chapter ArODES

Dicente Cid Yashin, Müller Henning, Platon Alexandra, Janssens Jean-Paul, Lador Frederic, Poletti Pierre-Alexandre, Depeursinges Adrien

Dans Müller, Henning, Medical Computer Vision and Bayesian and Graphical Models for Biomedical Imaging MICCAI 2016 International Workshops, MCV and BAMBI, Athens, Greece, October 21, 2016, Revised Selected Papers (Pp. 58-68). 2017, Cham : Springer

Lien vers la publication

Résumé:

This article presents a novel graph–model approach encoding the relations between the perfusion in several regions of the lung extracted from a geometry–based atlas. Unlike previous approaches that individually analyze regions of the lungs, our method evaluates the entire pulmonary circulatory network for the classification of patients with pulmonary embolism and pulmonary hypertension. An undirected weighted graph with fixed structure is used to encode the network of intensity distributions in Dual Energy Computed Tomography (DECT) images. Results show that the graph–model presented is capable of characterizing a DECT dataset of 30 patients affected with disease and 26 healthy patients, achieving a discrimination accuracy from 0.77 to 0.87 and an AUC between 0.73 and 0.86. This fully automatic graph–model of the lungs constitutes a novel and effective approach for exploring the various patterns of pulmonary perfusion of healthy and diseased patients.

2016

Cloud-based evaluation of anatomical structure segmentation and landmark detection algorithms :

Scientifique ArODES

VISCERAL anatomy benchmarks

Jiménez del Toro Oscar Alfonso, Müller Henning, Eggel Ivan, Schaer Roger, Dicente Cid Yashin

IEEE transactions on medical imaging, 2016, vol. 35, no 11, pp. 2459-2475

Lien vers la publication

Résumé:

Variations in the shape and appearance of anatomical structures in medical images are often relevant radiological signs of disease. Automatic tools can help automate parts of this manual process. A cloud-based evaluation framework is presented in this paper including results of benchmarking current state-of-the-art medical imaging algorithms for anatomical structure segmentation and landmark detection: the VISCERAL Anatomy benchmarks. The algorithms are implemented in virtual machines in the cloud where participants can only access the training data and can be run privately by the benchmark administrators to objectively compare their performance in an unseen common test set. Overall, 120 computed tomography and magnetic resonance patient volumes were manually annotated to create a standard Gold Corpus containing a total of 1295 structures and 1760 landmarks. Ten participants contributed with automatic algorithms for the organ segmentation task, and three for the landmark localization task. Different algorithms obtained the best scores in the four available imaging modalities and for subsets of anatomical structures. The annotation framework, resulting data set, evaluation setup, results and performance analysis from the three VISCERAL Anatomy benchmarks are presented in this article. Both the VISCERAL data set and Silver Corpus generated with the fusion of the participant algorithms on a larger set of non-manually-annotated medical images are available to the research community.

Reports on CBMI 16 and ICME 16

Scientifique ArODES

Ionescu Bogdan, Müller Henning, Kompatsiaris Yiannis, Gravier Guillaume, Vetro Anthony

IEEE MultiMedia, 2016, vol. 23, issue 4, pp. 88 - 93

Lien vers la publication

Résumé:

This issue features not just one but two conference reports. The first covers the 14th International Workshop on Content-Based Multimedia Indexing (CBMI 16), while the second covers the 2016 IEEE International Conference on Multimedia and Expo (ICME 2016). For both, find out what hot topics and key themes were discussed, which submissions earned the Best Paper awards, and more.

Medical images modality classification using discrete Bayesian Networks

Scientifique ArODES

Martinez-Gomez Jesus, Arias Jacinto, García Seco de Herrera Alba, Müller Henning

Computer vision and image understanding, October 2016, vol. 151, pp. 61–71

Lien vers la publication

Résumé:

In this paper we propose a complete pipeline for medical image modality classification focused on the application of discrete Bayesian network classifiers. Modality refers to the categorization of biomedical images from the literature according to a previously defined set of image types, such as X-ray, graph or gene sequence. We describe an extensive pipeline starting with feature extraction from images, data combination, pre-processing and a range of different classification techniques and models. We study the expressive power of several image descriptors along with supervised discretization and feature selection to show the performance of discrete Bayesian networks compared to the usual deterministic classifiers used in image classification. We perform an exhaustive experimentation by using the ImageCLEFmed 2013 collection. This problem presents a high number of classes so we propose several hierarchical approaches. In a first set of experiments we evaluate a wide range of parameters for our pipeline along with several classification models. Finally, we perform a comparison by setting up the competition environment between our selected approaches and the best ones of the original competition. Results show that the Bayesian Network classifiers obtain very competitive results. Furthermore, the proposed approach is stable and it can be applied to other problems that present inherent hierarchical structures of classes.

Deep learning with convolutional neural networks applied to electromyography data :

Scientifique ArODES

a resource for the classification of movements for prosthetic hands

Atzori Manfredo, Cognolato Matteo, Müller Henning

Frontiers in Neurorobotics, Septembre 2016, vol. 10

Lien vers la publication

Résumé:

Natural control methods based on surface electromyography (sEMG) and pattern recognition are promising for hand prosthetics. However, the control robustness offered by scientific research is still not sufficient for many real life applications, and commercial prostheses are capable of offering natural control for only a few movements. In recent years deep learning revolutionized several fields of machine learning, including computer vision and speech recognition. Our objective is to test its methods for natural control of robotic hands via sEMG using a large number of intact subjects and amputees. We tested convolutional networks for the classification of an average of 50 hand movements in 67 intact subjects and 11 transradial amputees. The simple architecture of the neural network allowed to make several tests in order to evaluate the effect of pre-processing, layer architecture, data augmentation and optimization. The classification results are compared with a set of classical classification methods applied on the same datasets. The classification accuracy obtained with convolutional neural networks using the proposed architecture is higher than the average results obtained with the classical classification methods, but lower than the results obtained with the best reference methods in our tests. The results show that convolutional neural networks with a very simple architecture can produce accurate results comparable to the average classical classification methods. They show that several factors (including pre-processing, the architecture of the net and the optimization parameters) can be fundamental for the analysis of sEMG data. Larger networks can achieve higher accuracy on computer vision and object recognition tasks. This fact suggests that it may be interesting to evaluate if larger networks can increase sEMG classification accuracy too.

A 3-D Riesz-covariance texture model for prediction of nodule recurrence in Lung CT

Scientifique ArODES

Müller Henning, Depeursinge Adrien, et al.

IEEE Transactions on medical imaging, 2016, vol. 35, issue 12, pp. 2620 - 2630

Lien vers la publication

Résumé:

This paper proposes a novel imaging biomarker of lung cancer relapse from 3–D texture analysis of CT images. Three–dimensional morphological nodular tissue properties are described in terms of 3–D Riesz–wavelets. The responses of the latter are aggregated within nodular regions by means of feature covariances, which leverage rich intra– and inter– variations of the feature space dimensions. When compared to the classical use of the average for feature aggregation, feature covariances preserve spatial co–variations between features. The obtained Riesz–covariance descriptors lie on a manifold governed by Riemannian geometry allowing geodesic measurements and differentiations. The latter property is incorporated both into a kernel for support vector machines (SVM) and a manifold–aware sparse regularized classifier. The effectiveness of the presented models is evaluated on a dataset of 110 patients with non–small cell lung carcinoma (NSCLC) and cancer recurrence information. Disease recurrence within a timeframe of 12 months could be predicted with an accuracy of 81.3–82.7%. The anatomical location of recurrence could be discriminated between local, regional and distant failure with an accuracy of 78.3–93.3%. The obtained results open novel research perspectives by revealing the importance of the nodular regions used to build the predictive models.

Optimized distributed hyperparameter search and simulation for Lung texture classification in CT Usi

Scientifique ArODES

Schaer Roger, Müller Henning, Depeursinge Adrien

Journal of imaging, 2016, vol. 2, no. 2

Lien vers la publication

Résumé:

Many medical image analysis tasks require complex learning strategies to reach a quality of image-based decision support that is sufficient in clinical practice. The analysis of medical texture in tomographic images, for example of lung tissue, is no exception. Via a learning framework, very good classification accuracy can be obtained, but several parameters need to be optimized. This article describes a practical framework for efficient distributed parameter optimization. The proposed solutions are applicable for many research groups with heterogeneous computing infrastructures and for various machine learning algorithms. These infrastructures can easily be connected via distributed computation frameworks. We use the Hadoop framework to run and distribute both grid and random search strategies for hyperparameter optimization and cross-validations on a cluster of 21 nodes composed of desktop computers and servers. We show that significant speedups of up to 364 compared to a serial execution can be achieved using our in-house Hadoop cluster by distributing the computation and automatically pruning the search space while still identifying the best-performing parameter combinations. To the best of our knowledge, this is the first article presenting practical results in detail for complex data analysis tasks on such a heterogeneous infrastructure together with a linked simulation framework that allows for computing resource planning. The results are directly applicable in many scenarios and allow implementing an efficient and effective strategy for medical (image) data analysis and related learning approaches.

Computational fluid dynamics as a tool to predict the air pollution dispersion in a neighborhood :

Scientifique ArODES

a research project to improve the quality of life in cities

Triscone Gilles, Abdennadher Nabil, Balistreri Christophe, Donzé Olivier, Greco D., Haas Patrick, Haas-Pekoz H., Mohamed-Nour T., Munier Pierre, Pontelandolfo Piero, Putzu Roberto, Richard Jacques, Sthioul H., Delley Nicolas, Choffat D., Niederhäuser Elena-Lavinia, Schaer Roger, Müller Henning, Decaix Jean, Richard Sylvain, Münch-Alligné Cécile, Kunz P., Despot F.

International Journal of Sustainable Development and Planning, 2016, vol. 11, no. 4, pp. 546-557

Lien vers la publication

Résumé:

In large cities, pollution composed of many different chemical components and small particles is an important public health problem that affects especially children and people presenting breathing difficulties. One challenge for public authorities is to respect the norms given by the central state, but how? Today, concrete methods for reducing pollution are perceived by the majority of citizens as constraints. However, the authorities have the possibility of modifying the wind’s action by imposing architectural constraints, such as building emplacement and roof structure. This is the main objective of the Geneva ‘Clean City’ project financed by the University of Applied Sciences Western Switzerland. ‘Clean City’ focuses its research on one of Geneva’s polluted neighborhood called Pâquis, which is situated directly on the Geneva lake front. The project attempts to understand the dispersion of pollution from an experimental and a numerical point of view. After validation of the technique for a simple case, we compare environmental measurements on a 1/500 3D scale model of the Pâquis installed in an instrumented wind tunnel with Computational Fluid Dynamics (CFD) simulation obtained with the help of cloud computing.

Forschungsanwendungen in der digitalen Radiologie

Scientifique ArODES

Müller Henning, Hanbury Allan

Der Radiologe, Februar 2016, Bd. 56, Nr. 2, pp. 176–180

Lien vers la publication

Résumé:

Medizinische Bildgebung produziert immer komplexere Bilder (höhere Auflösung, dünnere Schichten) mit einer zunehmenden Zahl unterschiedlicher Protokolle, sodass die Befundung schwieriger wird und mehr Informationen verarbeitet werden müssen, da die Zahl der Radiologen bei weitem nicht im selben Umfang wächst wie die zu analysierenden Daten. Ziel dieses Artikels ist es, aktuelle Forschungsergebnisse von Projekten vorzustellen, die medizinische Bilddaten zur Entscheidungsunterstützung benutzten. Eine Infrastruktur, die es Forschern erlaubt, Bilder zu analysieren und beste Algorithmen zu erkennen, ohne dass die Daten einen sicheren Server verlassen müssen, wird auch erklärt. Der Text stellt Resultate der EU-finanzierten Khresmoi- und VISCERAL-Projekte vor. Diese erlauben es, Daten vergangener Fälle in einem Bildarchiv zu benutzten, um Entscheidungen zu unterstützen und Aufgaben zu automatisieren. Die Resultate stellen eine Evaluationsumgebung für medizinische Bilddaten und andere Datensätze vor. Diese erlauben es, in einer sicheren Umgebung aus vergangenen Fällen, Daten für zukünftige Befundungen zu extrahieren. Die vorgestellten Prototypen erlauben es, aus Bilddaten direkt Information zu extrahieren und als Entscheidungsunterstützung zu benutzten. Die Forschungsprototypen sind allerdings noch nicht in klinischem Einsatz evaluiert worden, nur subjektive Benutzertests wurden unternommen, welche sehr positiv waren. Die Zukunft der Radiologie wird sicherlich z. T. davon abhängen, Prozesse zu automatisieren und Daten („big data“) aus institutionellen Bildarchiven zu benutzen, um bestmöglich Wissen zu extrahieren und damit die Arbeit der Radiologen auf die wichtigen, entscheidungsrelevanten Bereiche zu konzentrieren, und Routineaufgaben zu automatisieren, wo dies möglich ist.

Effect of clinical parameters on the control of myoelectric robotic prosthetic hands

Scientifique ArODES

Atzori Manfredo, Müller Henning, et al.

Journal of rehabilitation research and development, 2016, vol. 53, no. 3, pp. 345-358

Lien vers la publication

Résumé:

Improving the functionality of prosthetic hands with noninvasive techniques is still a challenge. Surface electromyography (sEMG) currently gives limited control capabilities; however, the application of machine learning to the analysis of sEMG signals is promising and has recently been applied in practice, but many questions still remain. In this study, we recorded the sEMG activity of the forearm of 11 male subjects with transradial amputation who were mentally performing 40 hand and wrist movements. The classification performance and the number of independent movements (defined as the subset of movements that could be distinguished with >90% accuracy) were studied in relationship to clinical parameters related to the amputation. The analysis showed that classification accuracy and the number of independent movements increased significantly with phantom limb sensation intensity, remaining forearm percentage, and time since amputation. The classification results suggest the possibility of naturally controlling up to 11 movements of a robotic prosthetic hand with almost no training. Knowledge of the relationship between classification accuracy and clinical parameters adds new information regarding the nature of phantom limb pain as well as other clinical parameters, and it can lay the foundations for future "functional amputation" procedures in surgery.

Medical information retrieval :

Scientifique ArODES

introduction to the special issue

Müller Henning, et al.

Journal of information retrieval, April 2016, vol. 19, no. 1, pp. 1–5

Lien vers la publication

2015

Control capabilities of myoelectric robotic prostheses by hand amputees :

Scientifique ArODES

a scientific research and market overview

Atzori Manfredo, Müller Henning

Frontiers in systems neuroscience,

Lien vers la publication

Résumé:

Hand amputation can dramatically affect the capabilities of a person. Cortical reorganization occurs in the brain, but the motor and somatosensorial cortex can interact with the remnant muscles of the missing hand even many years after the amputation, leading to the possibility to restore the capabilities of hand amputees through myoelectric prostheses. Myoelectric hand prostheses with many degrees of freedom are commercially available and recent advances in rehabilitation robotics suggest that their natural control can be performed in real life. The first commercial products exploiting pattern recognition to recognize the movements have recently been released, however the most common control systems are still usually unnatural and must be learned through long training. Dexterous and naturally controlled robotic prostheses can become reality in the everyday life of amputees but the path still requires many steps. This mini-review aims to improve the situation by giving an overview of the advancements in the commercial and scientific domains in order to outline the current and future chances in this field and to foster the integration between market and scientific research.

Analyzing image search behaviour of radiologists :

Scientifique ArODES

semantics and prediction of query results

De-Arteaga Maria, Eggel Ivan, Kahn Charles, Müller Henning

Journal of digital imaging, October 2015, vol. 28, Issue 5, pp. 537-546

Lien vers la publication

Résumé:

Log files of information retrieval systems that record user behavior have been used to improve the outcomes of retrieval systems, understand user behavior, and predict events. In this article, a log file of the ARRS GoldMiner search engine containing 222,005 consecutive queries is analyzed. Time stamps are available for each query, as well as masked IP addresses, which enables to identify queries from the same person. This article describes the ways in which physicians (or Internet searchers interested in medical images) search and proposes potential improvements by suggesting query modifications. For example, many queries contain only few terms and therefore are not specific; others contain spelling mistakes or non-medical terms that likely lead to poor or empty results. One of the goals of this report is to predict the number of results a query will have since such a model allows search engines to automatically propose query modifications in order to avoid result lists that are empty or too large. This prediction is made based on characteristics of the query terms themselves. Prediction of empty results has an accuracy above 88 %, and thus can be used to automatically modify the query to avoid empty result sets for a user. The semantic analysis and data of reformulations done by users in the past can aid the development of better search systems, particularly to improve results for novice users. Therefore, this paper gives important ideas to better understand how people search and how to use this knowledge to improve the performance of specialized medical search engines.

User-oriented evaluation of a medical image retrieval system for radiologists

Scientifique ArODES

Markonis Dimitrios, Holzer Markus, Baroz Frederic, De Castaneda Rafael Luis Ruiz, Boyer Célia, Langs Georg, Müller Henning

International journal of medical informatics, October 2015, vol. 84, no. 10, pp. 774-783

Lien vers la publication

Résumé:

PURPOSE: This article reports the user-oriented evaluation of a text- and content-based medical image retrieval system. User tests with radiologists using a search system for images in the medical literature are presented. The goal of the tests is to assess the usability of the system, identify system and interface aspects that need improvement and useful additions. Another objective is to investigate the system's added value to radiology information retrieval. The study provides an insight into required specifications and potential shortcomings of medical image retrieval systems through a concrete methodology for conducting user tests. METHODS: User tests with a working image retrieval system of images from the biomedical literature were performed in an iterative manner, where each iteration had the participants perform radiology information seeking tasks and then refining the system as well as the user study design itself. During these tasks the interaction of the users with the system was monitored, usability aspects were measured, retrieval success rates recorded and feedback was collected through survey forms. RESULTS: In total, 16 radiologists participated in the user tests. The success rates in finding relevant information were on average 87% and 78% for image and case retrieval tasks, respectively. The average time for a successful search was below 3 min in both cases. Users felt quickly comfortable with the novel techniques and tools (after 5 to 15 min), such as content-based image retrieval and relevance feedback. User satisfaction measures show a very positive attitude toward the system's functionalities while the user feedback helped identifying the system's weak points. The participants proposed several potentially useful new functionalities, such as filtering by imaging modality and search for articles using image examples. CONCLUSION: The iterative character of the evaluation helped to obtain diverse and detailed feedback on all system aspects. Radiologists are quickly familiar with the functionalities but have several comments on desired functionalities. The analysis of the results can potentially assist system refinement for future medical information retrieval systems. Moreover, the methodology presented as well as the discussion on the limitations and challenges of such studies can be useful for user-oriented medical image retrieval evaluation, as user-oriented evaluation of interactive system is still only rarely performed. Such interactive evaluations can be limited in effort if done iteratively and can give many insights for developing better systems.

Using MapReduce for large-scale medical image analysis

Scientifique ArODES

Markonis Dimitrios, Schaer Roger, Eggel Ivan, Müller Henning, Depeursinge Adrien

ArXiv,

Lien vers la publication

Résumé:

The growth of the amount of medical image data produced on a daily basis in modern hospitals forces the adaptation of traditional medical image analysis and indexing approaches towards scalable solutions. The number of images and their dimensionality increased dramatically during the past 20 years. We propose solutions for large-scale medical image analysis based on parallel computing and algorithm optimization. The MapReduce framework is used to speed up and make possible three large-scale medical image processing use-cases: (i) parameter optimization for lung texture segmentation using support vector machines, (ii) content-based medical image indexing, and (iii) three-dimensional directional wavelet analysis for solid texture classification. A cluster of heterogeneous computing nodes was set up in our institution using Hadoop allowing for a maximum of 42 concurrent map tasks. The majority of the machines used are desktop computers that are also used for regular office work. The cluster showed to be minimally invasive and stable. The runtimes of each of the three use-case have been significantly reduced when compared to a sequential execution. Hadoop provides an easy-to-employ framework for data analysis tasks that scales well for many tasks but requires optimization for specific tasks.

Comparing image search behaviour in the ARRS GoldMiner search engine and a clinical PACS/RIS

Scientifique ArODES

Müller Henning, De-Arteaga Maria, Eggel Ivan, Do Bao, Rubin Daniel, Kahn Jr. Charles

Journal of Biomedical Informatics, 2015, vol. 56, pp. 57-64

Lien vers la publication

Résumé:

Information search has changed the way we manage knowledge and the ubiquity of information access has made search a frequent activity, whether via Internet search engines or increasingly via mobile devices. Medical information search is in this respect no different and much research has been devoted to analyzing the way in which physicians aim to access information. Medical image search is a much smaller domain but has gained much attention as it has different characteristics than search for text documents. While web search log files have been analysed many times to better understand user behaviour, the log files of hospital internal systems for search in a PACS/RIS (Picture Archival and Communication System, Radiology Information System) have rarely been analysed. Such a comparison between a hospital PACS/RIS search and a web system for searching images of the biomedical literature is the goal of this paper. Objectives are to identify similarities and differences in search behaviour of the two systems, which could then be used to optimize existing systems and build new search engines. Log files of the ARRS GoldMiner medical image search engine (freely accessible on the Internet) containing 222,005 queries, and log files of Stanford’s internal PACS/RIS search called radTF containing 18,068 queries were analysed. Each query was preprocessed and all query terms were mapped to the RadLex (Radiology Lexicon) terminology, a comprehensive lexicon of radiology terms created and maintained by the Radiological Society of North America, so the semantic content in the queries and the links between terms could be analysed, and synonyms for the same concept could be detected. RadLex was mainly created for the use in radiology reports, to aid structured reporting and the preparation of educational material (Lanlotz, 2006) [1]. In standard medical vocabularies such as MeSH (Medical Subject Headings) and UMLS (Unified Medical Language System) specific terms of radiology are often underrepresented, therefore RadLex was considered to be the best option for this task. The results show a surprising similarity between the usage behaviour in the two systems, but several subtle differences can also be noted. The average number of terms per query is 2.21 for GoldMiner and 2.07 for radTF, the used axes of RadLex (anatomy, pathology, findings, . . .) have almost the same distribution with clinical findings being the most frequent and the anatomical entity the second; also, combinations of RadLex axes are extremely similar between the two systems. Differences include a longer length of the sessions in radTF than in GoldMiner (3.4 and 1.9 queries per session on average). Several frequent search terms overlap but some strong differences exist in the details. In radTF the term ‘‘normal’’ is frequent, whereas in GoldMiner it is not. This makes intuitive sense, as in the literature normal cases are rarely described whereas in clinical work the comparison with normal cases is often a first step. The general similarity in many points is likely due to the fact that users of the two systems are influenced by their daily behaviour in using standard web search engines and follow this behaviour in their professional search. This means that many results and insights gained from standard web search can likely be transferred to more specialized search systems. Still, specialized log files can be used to find out more on reformulations and detailed strategies of users to find the right content.

Medical image retrieval :

Scientifique ArODES

a multi-modal approach

Cao Yu, Steffey Shawn, He Jianbiao, Xiao Degui, Tao Cui, Chen Ping, Müller Henning

Cancer informatics, 2014, suppl. 3, pp. 125-136

Lien vers la publication

Résumé:

Medical imaging is becoming a vital component of war on cancer. Tremendous amounts of medical image data are captured and recorded in a digital format during cancer care and cancer research. Facing such an unprecedented volume of image data with heterogeneous image modalities, it is necessary to develop effective and efficient content-based medical image retrieval systems for cancer clinical practice and research. While substantial progress has been made in different areas of content-based image retrieval (CBIR) research, direct applications of existing CBIR techniques to the medical images produced unsatisfactory results, because of the unique characteristics of medical images. In this paper, we develop a new multimodal medical image retrieval approach based on the recent advances in the statistical graphic model and deep learning. Specifically, we first investigate a new extended probabilistic Latent Semantic Analysis model to integrate the visual and textual information from medical images to bridge the semantic gap. We then develop a new deep Boltzmann machine-based multimodal learning model to learn the joint density model from multimodal information in order to derive the missing modality. Experimental results with large volume of real-world medical images have shown that our new approach is a promising solution for the next-generation medical imaging indexing and retrieval system.

A review of medical grids and their direction :

Scientifique ArODES

a Swiss/Japanese perspective

Nakai Toshiharu, Müller Henning, Bagarinao Epifanio, Tomida Koji, Yoshiaki Shiraishi, Niinimaki Marko

International journal of research studies in computing, 2015, vol. 4, no. 1

Lien vers la publication

Résumé:

This paper presents a general and comparative review of the advances of grid technologies in medical sciences since 2000, with a special emphasis on Europe and Japan. The EU has over the years funded several big projects in the Grid and now the cloud area, covering besides the high energy physics domain also bioinformatics and medical applications as in image analysis. Bioinformatics and pharmaceutical design have been the major targets of grid applications in Japan. Grids and clouds share many of the same goals and techniques, although clouds are more commercially oriented and privately provided services and grids are rather publicly initiated resource sharing platforms between institutions without strong economic objectives. In the future, merging these two may potentially solve inter-operation problems of grids, which have been limiting the propagation of biomedical grids.

Evaluating performance of biomedical image retrieval systems :

Scientifique ArODES

an overview of the medical image retrieval task at ImageCLEF 2004-2013

Kalpathy-Cramer Jayashree, García Seco de Herrera Alba, Demner-Fushman Dina, Antani Sameer, Bedrick Steven, Müller Henning

Computerized medical imaging and graphics, January 2015, vol. 39. no. 55-61, 15 p.

Lien vers la publication

Résumé:

Medical image retrieval and classification have been extremely active research topics over the past 15 years. With the ImageCLEF benchmark in medical image retrieval and classification a standard test bed was created that allows researchers to compare their approaches and ideas on increasingly large and varied data sets including generated ground truth. This article describes the lessons learned in ten evaluations campaigns. A detailed analysis of the data also highlights the value of the resources created.

Meaningful bags of words for medical image classification and retrieval

Chapter ArODES

Foncubierta-Rodríguez Antonio, García Seco de Herrera Alba, Müller Henning

Health monitoring and personalized feedback using multimedia data (pp. 73-93). 2015, [s.l] : Springer

Lien vers la publication

Résumé:

Content–based medical image retrieval has been proposed as a technique that allows not only for easy access to images from the relevant literature and electronic health records but also for training physicians, for research and clinical decision support. The bag–of–visual–words approach is a widely used technique that tries to shorten the semantic gap by learning meaningful features from the dataset and describing documents and images in terms of the histogram of these features. Visual vocabularies are often redundant, over–complete and noisy. Larger than required vocabularies lead to high–dimensional feature spaces, which present important disadvantages with the curse of dimensionality and computational cost being the most obviousones. In this work a visual vocabulary pruning and descriptor transformation technique is presented. It enormously reduces the amount of required words to describe a medical image dataset with no significant effect on the accuracy. Results show that a reduction of up to 90% can be achieved without impact on the system performance. Obtaining a more compact representation of a document enables multimodal description as well as using classifiers requiring low–dimensional representations.

Evaluating multimodal relevance feedback techniques for medical image retrieval

Scientifique ArODES

Markonis Dimitrios, Schaer Roger, Müller Henning

Journal of information retrieval, Janvier 2015

Lien vers la publication

Résumé:

Medical image retrieval can assist physicians in finding information supporting their diagnosis and fulfilling information needs. Systems that allow searching for medical images need to provide tools for quick and easy navigation and query refinement as the time available for information search is often short. Relevance feedback is a powerful tool in information retrieval. This study evaluates relevance feedback techniques with regard to the content they use. A novel relevance feedback technique that uses both text and visual information of the results is proposed. The two information modalities from the image examples are fused either at the feature level using the Rocchio algorithm or at the query list fusion step using a common late fusion rule. Results using the ImageCLEF 2012 benchmark database for medical image retrieval show the potential of relevance feedback techniques in medical image retrieval. The mean average precision (mAP) is used as the evaluation metric and the proposed method outperforms commonly-used methods. The baseline without feedback reached 16 % whereas the relevance feedback with 20 images reached up to 26.35 % with three steps and when using 100 images up to 34.87 % in four steps. Most improvements occur in the first two steps of relevance feedback and then results start to become relatively flat. This might also be due to only using positive feedback as negative feeback often also improves results after more steps. The effect of relevance feedback in automatically spelling corrected and translated queries is investigated as well. Results without mistakes were better than spell-corrected results but the spelling correction more than double results over non-corrected retrieval. Multimodal relevance feedback has shown to be able to help visual medical information retrieval. Next steps include integrating semantics into relevance feedback techniques to benefit from the structured knowledge of ontologies and experimenting on the fusion of text and visual information.

How users search and what they search for in the medical domain :

Scientifique ArODES

understanding laypeople and experts through query logs

Palotti João, Hanbury Allan, Müller Henning, Kahn Charles

Journal of information retrieval, Janvier 2015

Lien vers la publication

Résumé:

The internet is an important source of medical knowledge for everyone, from laypeople to medical professionals. We investigate how these two extremes, in terms of user groups, have distinct needs and exhibit significantly different search behaviour. We make use of query logs in order to study various aspects of these two kinds of users. The logs from America On- line (AOL), Health on the Net (HON), Turning Re- search Into Practice (TRIP) and American Roentgen Ray Society (ARRS) GoldMiner were divided into three sets: (1) laypeople, (2) medical professionals (such as physicians or nurses) searching for health content and (3) users not seeking health advice. Several analyses are made focusing on discovering how users search and what they are most interested in. One possible outcome of our analysis is a classifer to infer user expertise, which was built. We show the results and analyse the feature set used to infer expertise. We conclude that medical experts are more persistent, interacting more with the search engine. Also, our study reveals that, conversely to what is stated in much of the literature, the main focus of users, both laypeople and professionals, is on disease rather than symptoms. The results of this article, especially through the classifer built, could be used to detect specifc user groups and then adapt search results to the user group.

Medical visual information analysis and retrieval :

Scientifique ArODES

special issue

Müller Henning, Kalpathy-Cramer Jayashree, Demner-Fushman Dina, Antani Sameer

Medical visual information analysis and retrieval, 2015, vol. 39, special issue, pp. 1-62

Lien vers la publication

A decade of community-wide efforts in advancing medical image understanding and retrieval

Scientifique ArODES

Müller Henning, Demner-Fushman Dina, Antani Sameer, Kalpathy-Cramer Jayashree

Computerized Medical Imaging and Graphics : special issue, 2015, vol. 39, special issue, pp. 1-2

Lien vers la publication

Résumé:

Access to images in scientific publications overall and in the biomedical domain specifically, has long been established a desirable feature for services providing access to scientific literature [2,6]. Likewise, automatic identification of regions of interest in clinical images and image-based assisted diagnosis are widely recognized as a means for bringing potentially important health issues to clinicians’ attention [11]. The individual research efforts in addressing these needs were faced with the lack of resources for development and evaluation of image search engines and tools for understanding image content.

Comparing fusion techniques for the ImageCLEF 2013 medical case retrieval task

Scientifique ArODES

Müller Henning, García Seco de Herrera Alba, Schaer Roger, Markonis Dimitrios

Computerized Medical Imaging and Graphics : special issue, 2015, vol. 39, special issue, pp. 46-54

Lien vers la publication

Résumé:

Retrieval systems can supply similar cases with a proven diagnosis to a new example case under observation to help clinicians during their work. The ImageCLEFmed evaluation campaign proposes a framework where research groups can compare case-based retrieval approaches. This paper focuses on the case-based task and adds results of the compound figure separation and modality classification tasks. Several fusion approaches are compared to identify the approaches best adapted to the heterogeneous data of the task. Fusion of visual and textual features is analyzed, demonstrating that the selection of the fusion strategy can improve the best performance on the case-based retrieval task.

Result diversification in social image retrieval :

Scientifique ArODES

a benchmarking framework

Ionescu Bogdan, Popescu Adrian, Radu Anca-Livia, Müller Henning

Multimedia tools and applications, octobre 2015, vol. 74, no 19

Lien vers la publication

Résumé:

This article addresses the diversification of image retrieval results in the context of image retrieval from social media. It proposes a benchmarking framework together with an annotated dataset and discusses the results achieved during the related task run in the MediaEval 2013 benchmark. 38 multimedia diversification systems, varying from graph-based representations, re-ranking, optimization approaches, data clustering to hybrid approaches that included a human in the loop, and their results are described and analyzed in this text. A comparison of the use of expert vs. crowdsourcing annotations shows that crowdsourcing results have a slightly lower inter-rater agreement but results are comparable at a much lower cost than expert annotators. Multimodal approaches have best results in terms of cluster recall. Manual approaches can lead to high precision but often lower diversity. With this detailed results analysis we give future insights into diversity in image retrieval and also for preparing new evaluation campaigns in related areas.

2014

Electromyography data for non-invasive naturally controlled robotic hand prostheses

Scientifique ArODES

Atzori Manfredo, Gijsberts Arjan, Castellini Claudio, Mittaz Hager Anne-Gabrielle, Caputo Barbara, Elsig Simone, Giatsidis Giorgio, Bassetto Franco, Müller Henning

Scientific data, vol. 1, no. 140053, pp. 2-13

Lien vers la publication

Résumé:

Recent advances in rehabilitation robotics suggest that it may be possible for hand-amputated subjects to recover at least a significant part of the lost hand functionality. The control of robotic prosthetic hands using non-invasive techniques is still a challenge in real life: myoelectric prostheses give limited control capabilities, the control is often unnatural and must be learned through long training times. Meanwhile, scientific literature results are promising but they are still far from fulfilling real-life needs. This work aims to close this gap by allowing worldwide research groups to develop and test movement recognition and force control algorithms on a benchmark scientific database. The database is targeted at studying the relationship between surface electromyography, hand kinematics and hand forces, with the final goal of developing non-invasive, naturally controlled, robotic hand prostheses. The validation section verifies that the data are similar to data acquired in real-life conditions, and that recognition of different hand tasks by applying state-of-the-art signal features and machine-learning algorithms is possible.

Electromyography low pass filtering effects on the classification of hand movements in amputated subjects

Scientifique ArODES

Atzori Manfredo, Müller Henning

Scientific data, Vol. 1, no. 140053, pp. 1-13

Lien vers la publication

Résumé:

People with transradial hand amputations can have control capabilities of prosthetic hands via surface electromyography (sEMG) but the control systems are limited and usually not natural. In the scientific literature, the application of pattern recognition techniques to classify hand movements in sEMG led to remarkable results but the evaluations are usually far from real life applications with all uncertainties and noise. Therefore, there is a need to improve the movement classification accuracy in real settings. Smoothing the signal with a low pass filter is a common pre? processing procedure to remove high?frequency noise. However, the filtering frequency modifies the signal strongly and can therefore affect the classification results. In this paper we analyze the dependence of the classification accuracy on the pre?processing low?pass filtering frequency in 3 hand amputated subjects performing 50 different movements. The results highlight two main interesting aspects. First, the filtering frequency strongly affects the classification accuracy, and choosing the right frequency between 1Hz?5Hz can improve the accuracy up to 5%. Second, different subjects obtain the best classification performance at different frequencies. Theoretically these facts could affect all the similar classification procedures re- ducing the classification uncertainity. Therefore, they contribute to set the field closer to real life applications, which could deeply change the life of hand amputated subjects.

L’échange des données de santé entre hôpitaux et acteurs de la santé :

Professionnel ArODES

le projet MediCoordination

Müller Henning, Ding Sandrine, Alves Bruno, Godel David, Abou Khaled Omar, Mooser Francois, Schumacher Michael

ASTRM actuel (Association Suisse es techniciens en radiologie médicales) - SVMTRA aktuell (Schweizerische Vereinigung der Fachleute für medizinisch technische Radiologie) - ASTRM attualità (Associazione Svizzera dei Tecnici di Radiologia Medica), Octobre 2014, pp. 18-23

Lien vers la publication

Résumé:

Le projet MediCoordination a permis de contribuer à la stratégie e-Health. L’e-Health est en plein essor en Suisse même si un gros effort devra encore être fourni en termes d’interopérabilité et de standardisation dans la mesure où le dossier patient informatisé est alimenté par des informations provenant d’un nombre considérable de services différents avec des standards tout aussi divers.

Comparing fusion techniques for the imageCLEF 2013 medical case retrieval task

Scientifique ArODES

García Seco de Herrera Alba, Schaer Roger, Markonis Dimitrios, Müller Henning

Computerized Medical Imaging and Graphics, Vol. 38, Issue 7, pp. 46-54

Lien vers la publication

Résumé:

Retrieval systems can supply similar cases with a proven diagnosis to a new example case under observation to help clinicians during their work. The ImageCLEFmed evaluation campaign proposes a framework where research groups can compare case{based retrieval approaches. This paper focuses on the case{based task and adds results of the compound gure separation and modality classication tasks. Several fusion approaches are compared to identify the approaches best adapted to the heterogeneous data of the task. Fusion of visual and textual features is analyzed, demonstrating that the selection of the fusion strategy can improve the best performance on the case{based retrieval task.

The movement error rate for evaluation of machine learning methods for sEMG-based hand movement classification

Scientifique ArODES

Gijsberts Arjan, Atzori Manfredo, Castellini Claudio, Müller Henning, Caputo Barbara

IEEE Transactions on neural systems and rehabiliation engineering, juillet 2014, vol. 22, issue 4, pp. 735-744

Lien vers la publication

Résumé:

There has been increasing interest in applying learning algorithms to improve the dexterity of myoelectric prostheses. In this work, we present a large-scale benchmark evaluation on the second iteration of the publicly released NinaPro database, which contains surface electromyography data for 6 DOF force activations as well as for 40 discrete hand movements. The evaluation involves a modern kernel method and compares performance of three feature representations and three kernel functions. Both the force regression and movement classification problems can be learned successfully when using a non-linear kernel function, while the exp-χ2 kernel outperforms the more popular Radial Basis Function kernel in all cases. Furthermore, combining surface electromyography and accelerometry in a multimodal classifier results in significant increases in accuracy as compared to when either modality is used individually. Since window-based classification accuracy should not be considered in isolation to estimate prosthetic controllability, we also provide results in terms of classification mistakes and prediction delay. To this extent, we propose the Movement Error Rate as an alternative to the standard window-based accuracy. This error rate is insensitive to prediction delays and it allows therefore to quantify mistakes and delays as independent performance characteristics. This type of analysis confirms that the inclusion of accelerometry is superior, as it results in fewer mistakes while at the same time reducing prediction delay.

2021

H&E-adversarial network :

Conference ArODES

a convolutional neural network to learn stain-invariant features through Hematoxylin & Eosin regression

Henning Müller, Niccolò Marini

Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)