Why is Fawad Khan so addicting

Stefan Wesarg

  • Vita
  • Publications
  • Projects

Stefan Wesarg studied physics at the Technical University of Berlin, the Ecole Nationale Supérieure de Physique de Marseille in France and the University of Heidelberg. The main topics of his studies were astrophysics and medical physics. He did his diploma thesis in 2001 at the German Cancer Research Center (DKFZ) in Heidelberg.

From 2001 to 2007 he was a research assistant at Fraunhofer IGD in Darmstadt in the department Cognitive Computing & Medical Imaging (Deputy Head of Department from 2006 to 2007). There he worked on various projects in the field of medical image processing: ultrasound, medical navigation systems and analysis of cardiac image data.

In 2007 he received his doctorate from the Department of Computer Science at the Technical University of Darmstadt (Dr.-Ing.). He then worked as a postdoc at the Technical University of Darmstadt from 2007 to the end of 2011 and built the new junior research group Medical computing on. As head of this group, he successfully obtained research funding from the German Research Foundation (DFG).

He has been back at Fraunhofer IGD since 2012 and is head of the department Visual Healthcare Technologies. The current work focuses on image-guided interventions and surgery, ultrasound applications for diagnosis and planning, statistical modeling of organs as well as simulation and planning for medical applications.

Stefan Wesarg was awarded the »Eurographics 2005 Medical Prize, 2ndnd«For his contribution»MEDARPA - An Augmented Reality System for Supporting Minimally Invasive Interventions" excellent. His paper »Localizing Calcifications in Cardiac CT Data Sets Using a New Vessel Segmentation Approach«With the» Best Paper 2006, 2ndnd" of Journal of Digital Imaging thoughtful. He regularly works as a reviewer for various peer-reviewed journals (e.g. IEEE Trans Med Img, Int.Journal for CARS, Journal of Digital Imaging) as well as for various conferences and workshops (e.g. IEEE ISBI, MICCAI workshops).

Show publication details

2021

Medical Imaging 2021: Image Processing

SPIE Medical Imaging Symposium

Proceedings of SPIE, 11596, 1

Misalignment of teeth or jaws can impact the ability to chew or speak, increase the risk of gum disease or tooth decay, and potentially inuence a person's (psychological) well-being. Orthodontic treatments of misaligned teeth are complex procedures that employ dental braces to apply forces in order to move the teeth or jaws to their correct position. Photographs are typically used to document the treatment. An automatic analysis of those photographs could support the decision making and monitoring process. In this paper, we propose an automatic model-based end-to-end 3-D reconstruction approach of the teeth from _ve photographs with prede_ned viewing directions (i.e. the photographs used in orthodontic treatment documentation). It uses photo- or view-speci_c 2- D coupled shape models to extract the teeth contours from the images. The shape reconstruction is then carried out by a deformation-based reconstruction approach that utilizes 3-D coupled shape models and minimizes a silhouette-based loss. The optimal model parameters are determined by an optimization which maximizes the overlaps between the projected 2-D outlines of individual teeth from the 3-D model and the contours extracted from the corresponding photograph. After that the point displacements between the projected outline and segmented contour are used to iteratively deform the 3-D shape model of each tooth for all views. Back-projection into shape space ensures that the 3-D coupled shape model consists of (statistically) valid teeth. Evaluation on 22 data sets shows promising results with an average symmetric surface distance of 0.848mm and an average DICE coe_cient of 0.659.

View article

Show publication details

Oyarzun Laura, Cristina; Hartwig, Katrin; Distergoft, Alexander; Hoffmann, Tim; Scheckenbach, Kathrin; Brussels, Melanie; Wesarg, Stefan

Automatic Segmentation of the Structures in the Nasal Cavity and the Ethmoidal Sinus for the Quantification of Nasal Septal Deviations

2021

Medical Imaging 2021: Computer-Aided Diagnosis

SPIE Medical Imaging Symposium

Proceedings of SPIE, 11597

Nasal septal deviations are a well-known and widespread problem. According to the American Academy of Otolaryngology, 80% of the population have a nasal septal deviation. Its level of severity can range from the person not being aware of it to respiratory obstruction and choking. It is therefore necessary to distinguish those patients at risk. For a proper diagnosis, the amount and location of the deviation have to be considered, but also the shape and changes in the surrounding turbinates. The segmentation of the structures of interest is an important step to reduce subjectivity in the diagnosis. Unfortunately, due to their variable and tortuous shape manual segmentation is time consuming. In this paper, the _rst method for the automatic segmentation of the structures in the nasal cavity and ethmoidal sinus is presented. A coupled shape model of the nasal cavity and paranasal sinus regions is trained and used to detect the corresponding regions in new CT images. The nasal septum is then segmented using a novel slice-based propagation technique. This segmentation allows the additional separation and segmentation of the left and right nasal cavities and ethmoidal sinuses and their structures by means of an adaptive thresholding with varying boundary sizes. The method has been evaluated in 10 CT images obtaining promising results for the nasal septum (DICE: 87.71%) and for the remaining structures (DICE: 72.01% - 73.01%). Based on the resulting segmentations, a web-based diagnosis tool has been designed to quantify the septal deviation using three metrics proposed by clinical experts.

View article

Show publication details

2020

Proceedings of the 2020 Annual Meeting of the German Society of Biomedical Engineering

Annual meeting of the German Society for Biomedical Technology in the VDE (BMT)

Current Directions in Biomedical Engineering

Cephalometric analysis is an important method in orthodontics for the diagnosis and treatment of patients. It is performed manually in clinical practice, therefore automation of this time consuming task would be of great assistance. In order to provide dentists with such tools, a robust and accurate identification of the necessary landmarks is required. However, poor image quality of lateral cephalograms like low contrast or noise make this task difficult. In this paper, an approach for automatic landmark localization is presented and used to find 19 landmarks in lateral cephalometric images. An initial predicting of the individual landmark locations is done by using a 2-D coupled shape model to utilize the spatial relation between landmarks and other anatomical structures. These predictions are refined with a Hough Forest to determine the final landmark location. The approach achieves competitive performance with a successful detection rate of 70.24% on 250 images for the clinically relevant 2mm accuracy range.

View article

Show publication details

2020

Spectrum of science

The discovery of X-rays heralded clinical imaging - and made the extremely powerful method of computed tomography possible.

Show publication details

Syeda-Mahmood, Tanveer [Ed.]; Oyarzun Laura, Cristina [Ed.]; Wesarg, Stefan [Ed.]; Erdt, Marius [Ed.]

Multimodal Learning for Clinical Decision Support and Clinical Image-Based Procedures. Proceedings

2020

Jumper

International Workshop on Clinical Image-based Procedures (CLIP)

Image Processing, Computer Vision, Pattern Recognition, and Graphics (LNIP), 12445
Lecture Notes in Computer Science (LNCS)
12445

On October 4, 2020, the 9th International Workshop on Clinical Image-based Procedures: From Planning to Intervention (CLIP 2020), was held in conjunction with the 23rd International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2020). Due to the COVID-19 pandemic, the workshop was held as an online-only meeting to contribute to slowing down the spread of the virus. Despite the challenges involved, we have continued to build on what we have successfully practiced over the past eight years: providing a platform for the dissemination of clinically tested, state-of-the-art methods for image-based planning, monitoring, and evaluation of medical procedures. A major focus of CLIP 2020 was on the creation of holistic patient models to better understand the need of the individual patient and thus provide better diagnoses and therapies. In this context, it is becoming increasingly important to not only base decisions on image data alone, but to combine these with non-image data, such as ‘omics’ data, electronic medical records, electroencephalograms, and others. This approach offers exciting opportunities to research. CLIP provides a platform to present and discuss these developments and work, centered on specific clinical applications already in use and evaluated by clinical users. In 2020, CLIP accepted nine original manuscripts from all over the world for oral presentation at the online event. Each of the manuscripts underwent a single-blind peer review by two members of the Program Committee, all of them prestigious experts in the field of medical image analysis and clinical translations of technology. We would like to thank our Program Committee for its invaluable contributions and continuous support of CLIP over the years. It is not always easy to find the time to support our workshop given full schedules and challenges due to the ongoing pandemic, and we are very grateful to all our members because CLIP 2020 would not have been possible without them. We would also like to thank all the authors for their high-quality contributions this year as well as their efforts to make CLIP 2020 a success. Finally, we would like to thank all MICCAI 2020 organizers for supporting the organization of CLIP 2020.

View article

Show publication details

2020

Current Directions in Biomedical Engineering

Proper treatment of prostate cancer is essential to increase the survival chance. In this sense, numerous studies show how important the communication between all stakeholders in the clinic is. This communication is difficult because of the lack of conventions while referring to the location where a biopsy for diagnosis was taken. This becomes even more challenging taking into account that experts of different fields work on the data and have different requirements. In this paper a web-based communication tool is proposed that incorporates a visualization of the prostate divided into 27 segments according to the PI-RADS protocol. The tool provides 2 working modes that consider the requirements of radiologist and pathologist while keeping it consistent. The tool comprises all relevant information given by pathologists and radiologists, such as, severity grades of the disease or tumor length. Everything is visualized using a color code for better undestanding.

Show publication details

Ma, Jingting; Wang, Anqi; Lin, Feng; Wesarg, Stefan; Erdt, Marius

A novel robust kernel principal component analysis for nonlinear statistical shape modeling from erroneous data

2019

Computerized Medical Imaging and Graphics

Statistical Shape Models (SSMs) have achieved considerable success in medical image segmentation. A high quality SSM is able to approximate the main plausible variances of a given anatomical structure to guide segmentation. However, it is technically challenging to derive such a quality model because: (1) the distribution of shape variance is often nonlinear or multi-modal which cannot be modeled by standard approaches assuming Gaussian distribution; (2) as the quality of annotations in training data usually varies, heavy corruption will degrade the quality of the model as a whole. In this work, these challenges are addressed by introducing a generic SSM that is able to model nonlinear distribution and is robust to outliers in training data. Without losing generality and assuming a sparsity in nonlinear distribution, a novel Robust Kernel Principal Component Analysis (RKPCA) for statistical shape modeling is proposed with the aim of constructing a low-rank nonlinear subspace where outliers are discarded. The proposed approach is validated on two different datasets: a set of 30 public CT kidney pairs and a set of 49 MRI ankle bones volumes. Experimental results demonstrate a significantly better performance on outlier recovery and a higher quality of the proposed model as well as lower segmentation errors compared to the state-of-the-art techniques.

View article

Show publication details

Montefiori, Erica; Modenese, Luca; Di Marco, Roberto; Magni-Manzoni, Silvia; Malattia, Clara; Petrarch, Maurizio; Ronchetti, Anna; Tanturri De Horatio, Laura; Dijkhuizen, Pieter van; Wang, Anqi; Wesarg, Stefan; Viceconti, Marco; Mazzà, Claudia

An Image-based Kinematic Model of the Tibiotalar and Subtalar Joints and its Application to Gait Analysis in Children with Juvenile Idiopathic Arthritis

2019

Journal of Biomechanics

In vivo estimates of tibiotalar and the subtalar joint kinematics can unveil unique information about gait biomechanics, especially in the presence of musculoskeletal disorders affecting the foot and ankle complex. Previous literature investigated the ankle kinematics on ex vivo data sets, but little has been reported for natural walking, and even less for pathological and juvenile populations. This paper proposes an MRI-based morphological fitting methodology for the personalized definition of the tibiotalar and the subtalar joint axes during gait, and investigated its application to characterize the ankle kinematics in twenty patients affected by Juvenile Idiopathic Arthritis (JIA). The estimated joint axes were in line with in vivo and ex vivo literature data and joint kinematics variation subsequent to inter-operator variability was in the order of 1 °. The model allowed to investigate, for the first time in patients with JIA, the functional response to joint impairment. The joint kinematics highlighted changes over time that were consistent with changes in the patient’s clinical pattern and notably varied from patient to patient. The heterogeneous and patient-specific nature of the effects of JIA was confirmed by the absence of a correlation between a semi-quantitative MRI-based impairment score and a variety of investigated joint kinematics indexes. In conclusion, this study showed the feasibility of using MRI and morphological fitting to identify the tibiotalar and subtalar joint axes in a non-invasive patient-specific manner. The proposed methodology represents an innovative and reliable approach to the analysis of the ankle joint kinematics in pathological juvenile populations.

View article

Show publication details

2019

2019 IEEE International Symposium on Biomedical Imaging

IEEE International Symposium on Biomedical Imaging (ISBI)

The nasal cavity and paranasal sinuses present large interpatient variabilities. Additional circumstances like for example, concha bullosa or nasal septum deviations complicate their segmentation. As in other areas of the body a previous multistructure detection could facilitate the segmentation task. In this paper an approach is proposed to individually detect all sinuses and the nasal cavity. For a better delimitation of their borders the use of an irregular polyhedron is proposed. For an accurate prediction the Darknet-19 deep neural network is used which combined with the You Only Look Once method has shown very promising results in other fields of computer vision. 57 CT scans were available of which 85% were used for training and the remaining 15% for validation.

View article

Show publication details

Montefiori, Erica; Modenese, Luca; Di Marco, Roberto; Magni-Manzoni, Silvia; Malattia, Clara; Petrarch, Maurizio; Ronchetti, Anna; Tanturri De Horatio, Laura; Dijkhuizen, Pieter van; Wang, Anqi; Wesarg, Stefan; Viceconti, Marco; Mazzà, Claudia

Linking Joint Impairment and Gait Biomechanics in Patients with Juvenile Idiopathic Arthritis

2019

Annals of Biomedical Engineering

Juvenile Idiopathic Arthritis (JIA) is a pediatric musculoskeletal disease of unknown aetiology, leading to walking alterations when the lower-limb joints are involved. Diagnosis of JIA is mostly clinical. Imaging can quantify impairments associated with inflammation and joint damage. However, treatment planning could be better supported using dynamic information, such as joint contact forces (JCFs). For this purpose, we used a musculoskeletal model to predict JCFs and investigate how JCFs varied as a result of joint impairment in eighteen children with JIA. Gait analysis data and magnetic resonance images (MRI) were used to develop patient-specific lower-limb musculoskeletal models, which were evaluated for operator-dependent variability (< 3.6°,="" 0.05="" n="" kg21="" and="" 0.5="" bw="" for="" joint="" angles,="" moments,="" and="" jcfs,="" respectively).="" gait="" alterations="" and="" jcf="" patterns="" showed="" high="" between-subjects="" variability="" reflecting="" the="" pathology="" heterogeneity="" in="" the="" cohort.="" higher="" joint="" impairment,="" assessed="" with="" mri-based="" evaluation,="" was="" weakly="" associated="" to="" overall="" joint="" overloading.="" a="" stronger="" correlation="" was="" observed="" between="" impairment="" of="" one="" limb="" and="" overload="" of="" the="" contralateral="" limb,="" suggesting="" risky="" compensatory="" strategies="" being="" adopted,="" especially="" at="" the="" knee="" level.="" this="" suggests="" that="" knee="" overloading="" during="" gait="" might="" be="" a="" good="" predictor="" of="" disease="" progression="" and="" gait="" biomechanics="" should="" be="" used="" to="" inform="" treatment="">

View article

Show publication details

Ma, Jingting; Lin, Feng [supervisor]; Erdt, Marius [supervisor]; Fellner, Dieter W. [co-supervisor]; Wesarg, Stefan [co-supervisor]

Self-learning Shape Recognition in Medical Images

2019

Singapore, Nanyang Technological University, Diss., 2019

A massive amount of medical image data, e.g. from Computed Tomography (CT) and Magnetic Resonance Imaging (MRI), is generated from hospitals every day. Biological structure segmentation is very useful to support surgery planning and treatments, as an ideal delineation of the outline of the target object can offer a precise location and quantitative analysis for further clinical diagnoses such as identification of tumorous tissues. However, the large dimension and complex patterns in medical image data make manual annotation extremely time-consuming and problematic. Accordingly, automatic biomedical image segmentation becomes a crucial pre-requisite in practice and has been a critical research issue over tens of years. However, major challenges exist in medical image segmentation such as the low intensity contrast to surrounding tissues and complex geometry of shape. Moreover, limited amounts of labeled training data give rise to difficulties as well. Numerous approaches have been proposed to mitigate these challenges, from low-level image processing to supervised machine learning techniques. It is worth mentioning that statistical shape models (SSMs) based segmentation approaches have achieved remarkable success in a wide spread of applications.SSMs are trained mostly using self-learning approaches to parameterize the significant variabilities of biological shapes, subsequently, the learned shape prior is adopted in image adaption to guide the shape fitting. Despite the success, SSMs-based segmentation approaches suffer from the limitation that the power of SSMs rises and falls with the quality of training data and geometrical complexity of the target shape. Furthermore, the existing image adaption may not be efficient in cases where the target object has a small and distorted structure. Therefore, this thesis aims to derive SSMs that are robust to training data corruption and are able to represent complex patterns, and address the problem of the poor image adaption to realize the challenging object segmentation. As training data is often corrupted by many factors like inherent noise / artifacts and non-ideal delineations in this thesis, many efforts have been devoted to developing SSMs that are robust to data corruption. First, early attempts proposing an imputation method and weighted Robust Principal Component Analysis (WRPCA) have been made to ad- dress arbitrary corruptions under the assumption of linear distribution. Nevertheless, deriving a quality model is still demanding as the shape variance of biological structures may not simply follow Gaussian distribution. To combat this, a kernelized RPCA is proposed to cope with outliers in a nonlinear distribution. The idea is performing the low-rank modeling on the kernel matrix to achieve nonlinear dimensionality reduction, and outlier recovery thereof. To increase the generality and feasibility, this thesis, furthermore, presents a general nonlinear data compression technique, the Robust Kernel PCA (RKPCA), with the aim of constructing a low-rank nonlinear subspace free of outliers. In terms of evaluation, the proposed RKPCA delivers high performance on not only creating SSMs but also on outlier recovery. Experiments are conducted using two representative datasets, a set of 30 public CT kidneys and a set of 49 internal MRI ankle bones. Embedded into an existing segmentation framework, experimental results show that SSM built with the proposed RKPCA outperforms the state-of-the-art modeling techniques in terms of model quality and segmentation accuracy. Since SSMs fail to adopt in cases where the target structure occupies a relatively small or distorted area, deep neural networks that remedy this shortcoming are considered thereof. However, redundant background contents in 3D volume may significantly influence the accuracy of deep deep neural networks. Aiming at challenging structures that occupy relatively small areas and have large variances, a novel unified segmentation framework is proposed that incorporates SSM on the top of deep neural network for detailed refinement. The motivation is aggregating both spatial and intensity based features from a limited amount of data. Globally optimized via Bayesian inference, the segmentation is driven by a dynamic weighted Gaussian Mixture Model integrating the probability scores from the deep neural network and the shape prior from the SSM. Under a public NIH dataset of CT pancreas, the proposed segmentation framework achieves the best average Dice Similarity Coefficient compared to the-state-of-the-art approaches. The majority of this work is based on public tools: the Medical Imaging Interaction Toolkit (MITK) for SSMs investigation and analysis and the public library Keras for deep neural networks development. All medical image datasets used in this thesis have been validated by clinical experts.

View article

Show publication details

Wang, Anqi; Franke, Andreas; Wesarg, Stefan

Semi-automatic Segmentation of JIA-Induced Inflammation in MRI Images of Ankle Joints

2019

Medical Imaging 2019: Image Processing

SPIE Medical Imaging Symposium

Proceedings of SPIE, 10949

The autoimmune disease Juvenile Idiopathic Arthritis (JIA) affects children of under 16 years and leads to the symptom of inflamed synovial membranes in affected joints. In clinical practice, characteristics of these inflamed membranes are used to stage the disease progression and to predict erosive bone damage. Manual outlining of inflammatory regions in each slide of a MRI dataset is still the gold standard for detection and quantification, however, this process is very tiresome and time-consuming. In addition, the inter- and intra-observer variability is a known problem of human annotators. We have developed the first method to detect inflamed regions in and around major joints in the human ankle. First, we use an adapted coupled shape model framework to segment the ankle bones in a MRI dataset. Based on these segmentations, joints are defined as locations where two bones are particularly close to each other. A number of potential inflammation candidates are generated using multi-level thresholding. Since it is known that inflamed synovial membranes occur in the proximity of joints, we filter out structures with similar intensities such as vessels and tendons sheaths using not only a vesselness filter, but also their distance to the joints and their size. The method has been evaluated on a set of 10 manually annotated clinical MRI datasets and achieved the following results: Precision 0.6785 ± 0.1584, Recall 0.5388 ± 0.1213, DICE 0.5696 ± 0.0976.

View article

Show publication details

Greenspan, Hayit [Ed.]; Tanno, Ryutaro [Ed.]; Erdt, Marius [Ed.]; Arbel, Tal [Ed.]; Baumgartner, Christian [Ed.]; Dalca, Adrian [Ed.]; Sudre, Carole H. [Ed.]; Wells, William M. [Ed.]; Drechsler, Klaus [Ed.]; Linguraru, Marius George [Ed.]; Oyarzun Laura, Cristina [Ed.]; Shekhar, Raj [Ed.]; Wesarg, Stefan [Ed.]; González Ballester, Miguel Angel [Ed.]

Uncertainty for Safe Utilization of Machine Learning in Medical Imaging and Clinical Image-Based Procedures. Proceedings

2019

International Workshop on Clinical Image-based Procedures (CLIP)

Image Processing, Computer Vision, Pattern Recognition, and Graphics (LNIP), 11840
Lecture Notes in Computer Science (LNCS), 11840

On October 17, 2019, the 8th International Workshop on Clinical Image-based Procedures: From Planning to Intervention (CLIP 2019) was held in Shenzhen, China in conjunction with the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2019) . Following the tradition set in the last seven years, this year's edition of the workshop was an exciting forum for the discussion and dissemination of clinically tested, state-of-the-art methods for image-based planning, monitoring, and evaluation of medical procedures . Nowadays, it has become more and more important for many clinical applications to base decisions not only on image data alone, thus a focus of CLIP 2019 was the creation of holistic patient models. Here, image data such as radiologic images, microscopy images, and photographs are combined with non-image information such as 'omics' data (eg genomics, proteomics), lifestyle data, demographics, EEG, and others to build a more complete picture of the individual patient and to subsequently provide better diagnosis and therapies. CLIP 2019 provided a forum for work centered on specific clinical applications, including techniques and procedures based on comprehensive clinical image and other data. Submissions related to applications already in use and evaluated by clinical users were particularly encouraged. Furthermore, novel techniques and applications that are looking at combining image analysis with clinical data mining and analytics, user studies, and other heterogeneous data were a focus as well. In CLIP’s 8th edition, world-class researchers and clinicians came together to present ways to strengthen links between computer scientists and engineers, and surgeons, interventional radiologists, and radiation oncologists.

View article

Show publication details

Ma, Jingting; Lin, Feng; Wesarg, Stefan; Erdt, Marius

A Novel Bayesian Model Incorporating Deep Neural Network and Statistical Shape Model for Pancreas Segmentation

2018

Medical Image Computing and Computer Assisted Intervention - MICCAI 2018: Part IV

International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)

Lecture Notes in Computer Science (LNCS), 11073

Deep neural networks have achieved significant success in medical image segmentation in recent years. However, poor contrast to surrounding tissues and high flexibility of anatomical structure of the interest object are still challenges. On the other hand, statistical shape model based approaches have demonstrated promising performance on exploiting complex shape variabilities but they are sensitive to localization and initialization. This motivates us to leverage the rich shape priorities learned from statistical shape models to improve the segmentation of deep neural networks. In this work, we propose a novel Bayesian model incorporating the segmentation results from both deep neural network and statistical shape model for segmentation. In evaluation, experiments are performed on 82 CT datasets of the challenging public NIH pancreas dataset. We report 85.32% of the mean DSC that outperforms the state-of-the-art and approximately 12% improvement from the predicted segment of deep neural network.

View article

Show publication details

2018

OR 2.0 Context-Aware Operating Theaters, Computer Assisted Robotic Endoscopy, Clinical Image-Based Procedures, and Skin Image Analysis

International Workshop on OR 2.0 Context-Aware Operating Theaters (OR 2.0)

Lecture Notes in Computer Science (LNCS), 11041

Cephalometric analysis is an important tool used by dentists for diagnosis and treatment of patients. Tools that could automate this time consuming task would be of great assistance. In order to provide the dentist with such tools, a robust and accurate identification of the necessary landmarks is required. However, poor image quality of lateral cephalograms like low contrast or noise as well as duplicate structures resulting from the way these images are acquired make this task difficult. In this paper, a fully automatic approach for teeth segmentation is presented that aims to support the identification of dental landmarks. A 2-D coupled shape model is used to capture the statistical knowledge about the teeth’s shape variation and spatial relation to enable a robust segmentation despite poor image quality. 14 individual teeth are segmented and labeled using gradient image features and the quality of the generated results is compared to manually created gold-standard segmentations. Experimental results on a set of 14 test images show promising results with a DICE overlap of 77.2% and precision and recall values ​​of 82.3% and 75.4%, respectively.

View article

Show publication details

Wirtz, Andreas; Mirashi, Sudesh Ganapati; Wesarg, Stefan

Automatic Teeth Segmentation in Panoramic X-Ray Images Using a Coupled Shape Model in Combination with a Neural Network

2018

Medical Image Computing and Computer Assisted Intervention - MICCAI 2018: Part IV

International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)

Lecture Notes in Computer Science (LNCS), 11073

Dental panoramic radiographs depict the full set of teeth in a single image and are used by dentists as a popular first tool for diagnosis. In order to provide the dentist with automatic diagnostic support, a robust and accurate segmentation of the individual teeth is required. However, poor image quality of panoramic x-ray images like low contrast or noise as well as teeth variations in between patients make this task difficult. In this paper, a fully automatic approach is presented that uses a coupled shape model in conjunction with a neural network to overcome these challenges. The network provides a preliminary segmentation of the teeth region which is used to initialize the coupled shape model in terms of position and scale. Then the 28 individual teeth (excluding wisdom teeth) are segmented and labeled using gradient image features in combination with the model’s statistical knowledge about their shape variation and spatial relation. The segmentation quality of the approach is assessed by comparing the generated results to manually created gold standard segmentations of the individual teeth. Experimental results on a set of 14 test images show average precision and recall values ​​of 0.790 and 0.827, respectively and a DICE overlap of 0.744.

View article

Show publication details

Noll, Matthias; Noa-Rudolph, Werner; Wesarg, Stefan; Kraly, Michael; Stoffels, Ingo; Klode, Joachim; Fun, Cédric; Have fun, Gerrit

ICG based Augmented Reality System for Sentinel Lymph Node Biopsy

2018

Eurographics Workshop on Visual Computing for Biology and Medicine

Eurographics Workshop on Visual Computing for Biology and Medicine (VCBM)

In this paper we introduce a novel augmented-reality (AR) system for the sentinel lymph node (SLN) biopsy. The AR system consists of a cubic recording device with integrated stereo near-infrared (NIR) and stereo color cameras, a head-mounted display (HMD) for visualizing the SLN information directly into the physicians view and a controlling software application. The labeling of the SLN is achieved using the fluorescent dye indocyanine green (ICG). The dye accumulates in the SLN where it is excited to fluorescence by applying infrared light. The fluorescence is recorded from two directions by the NIR stereo cameras using appropriate filters. Applying the known rigid camera geometry, an ICG depth map can be generated from the camera images, thus creating a live 3D representation of the SLN. The representation is then superimposed to the physicians field of view, by applying a series of coordinate system transformations, that are determined in four separate system calibration steps. To compensate for the head motion, the recording systems is continuously tracked by a single camera on the HMD using fiducial markers. Because the system does not require additional monitors, the physicians attention is kept solely on the operation site. This can potentially decrease the intervention time and render the procedure safer for the patient.

View article

Show publication details

2018

Current Directions in Biomedical Engineering

After a liver tumor intervention the medical doctor has to compare both pre and postoperative CT acquisitions to ensure that all carcinogenic cells are destroyed. A correct assessment of the intervention is of vital importance, since it will reduce the probability of tumor recurrence. Some methods have been proposed to support the medical doctors during the assessment process, however, all of them focus on secondary tumors. In this paper a tool is presented that enables the outcome validation for both primary and secondary tumors. Therefore, a multiphase registration (preoperative arterial and portal phases) followed by a registration between the pre and postoperative CT images is carried out. The first registration is in charge of the primary tumors that are only visible in the arterial phase. The secondary tumors will be incorporated in the second registration step. Finally, the part of the tumor that was not covered by the necrosis is quantified and visualized. The method has been tested in 9 patients, with an average registration error of 1.41 mm.

View article

Show publication details

Modenese, Luca; Montefiori, Erica; Wang, Anqi; Wesarg, Stefan; Viceconti, Marco; Mazzà, Claudia

Investigation of the Dependence of Joint Contact Forces on Musculotendon Parameters Using a Codified Workflow for Image-based Modeling

2018

Journal of Biomechanics

The generation of subject-specific musculoskeletal models of the lower limb has become a feasible task thanks to improvements in medical imaging technology and musculoskeletal modeling software. Nevertheless, clinical use of these models in pediatric applications is still limited for what concerns the estimation of muscle and joint contact forces. Aiming to improve the current state of the art, a methodology to generate highly personalized subject-specific musculoskeletal models of the lower limb based on magnetic resonance imaging (MRI) scans was codified as a step-by-step procedure and applied to data from eight juvenile individuals. The generated musculoskeletal models were used to simulate 107 gait trials using stereophotogrammetric and force platform data as input. To ensure completeness of the modeling procedure, muscles ’architecture needs to be estimated. Four methods to estimate muscles ’maximum isometric force and two methods to estimate musculotendon parameters (optimal fiber length and tendon slack length) were assessed and compared, in order to quantify their influence on the models’ output. Reported results represent the first comprehensive subject-specific model-based characterization of juvenile gait biomechanics, including profiles of joint kinematics and kinetics, muscle forces and joint contact forces. Our findings suggest that, when musculotendon parameters were linearly scaled from a reference model and the muscle force-length-velocity relationship was accounted for in the simulations, realistic knee contact forces could be estimated and these forces were not sensitive the method used to compute muscle maximum isometric force.

View article

Show publication details

Stoyanov, Danail [Ed.]; Taylor, Zaike [Ed.]; Sarikaya, Duygu [Ed.]; McLeod, Jonathan [Ed.]; González Ballester, Miguel Angel [Ed.]; Codella, Noel C.F. [Ed.]; Martel, Anne L. [Ed.]; Maier-Hein, Lena [Ed.]; Malpani, Anand [Ed.]; Zenati, Marco A. [Ed.]; De Ribaupierre, Sandrine [Ed.]; Xiongbiao, Luo [Ed.]; Collins, Toby [Ed.]; Reichl, Tobias [Ed.]; Drechsler, Klaus [Ed.]; Erdt, Marius [Ed.]; Linguraru, Marius George [Ed.]; Oyarzun Laura, Cristina [Ed.]; Shekhar, Raj [Ed.]; Wesarg, Stefan [Ed.]; Celebi, M. Emre [Ed.]; Dana, Kristin [Ed.]; Halpern, Allan [Ed.]

OR 2.0 Context-Aware Operating Theaters, Computer Assisted Robotic Endoscopy, Clinical Image-Based Procedures, and Skin Image Analysis

2018

Jumper

International Workshop on OR 2.0 Context-Aware Operating Theaters (OR 2.0)

Lecture Notes in Computer Science (LNCS), 11041
Image Processing, Computer Vision, Pattern Recognition, and Graphics (LNIP), 11041

The OR 2.0 papers cover a wide range of topics such as machine vision and perception, robotics, surgical simulation and modeling, multi-modal data fusion and visualization, image analysis, advanced imaging, advanced display technologies, human-computer interfaces, sensors. The CARE papers cover topics to advance the field of computer-assisted and robotic endoscopy. The CLIP papers cover topics to fill gaps between basic science and clinical applications.The ISIC papers cover topics to facilitate knowledge dissemination in the field of skin image analysis, as well as to host a melanoma detection challenge, raising awareness and interest for these socially valuable tasks.

View article

Show publication details

2018

2017 IEEE Workshop on Visual Analytics in Healthcare

IEEE Workshop on Visual Analytics in Healthcare (VAHC)

The visualization and analysis of electronic health records (EHRs) are becoming increasingly relevant for clinical researchers. While the digitization of medical images is general practice today, many clinics are just starting to build up database with the related patient data, patient histories, and treatment outcomes. This paper reports on a project with a medical group of ear, nose, and throat (ENT) specialists. It combines medical image analysis and radiomics with visual analytics of patient data to build, analyze, and evaluate patient cohorts. The combined visual interface for both browsing and analyzing patient data was developed in collaboration with the medical researchers. In addition to offering a new way of cohort building, our approach also provides a first comprehensive view on the EHR, including the relevant anatomy of patients. This project triggered a new effort to extend the digitized patient database from around 100 patients to the entire patient population at our partner’s clinic.

Show publication details

2017

IEEE Transactions on Biomedical Engineering

The purpose of this paper is to present an outcome validation tool for tumor radiofrequency (RF) ablation and resection. Methods: Intervention assessment tools require an accurate registration of both pre- and postoperative computed tomographies able to handle big deformations. Therefore, a physics-based method is proposed with that purpose. To increase the accuracy both automatically detected internal and surface physical landmarks are incorporated in the registration process. Results: The algorithm has been evaluated in 25 clinical datasets containing RF ablations, resections, and patients with recurrent tumors. The achieved accuracy is 1.2 mm measured as mean internal distance between vessel landmarks and a positive predictive value of 0.95. The quantitative and qualitative results of the outcome validation tool show that in 50% of the cases tumors were only partially covered by the treatment. Conclusion: The use of internal and surface landmarks combined with a physics-based registration method increases the accuracy of the results compared to the accuracy of state of the art methods. An accurate outcome validation tool is important in order to certify that the tumor and its safety margin were fully covered by the treatment. Significance: An accurate outcome validation tool can result in a decrease of the tumor recurrence rate.

View article

Show publication details

2017

Imaging for Patient-Customized Simulations and Systems for Point-of-Care Ultrasound

International Workshop on Point-of-Care Ultrasound (POCUS)

Ultrasound provides a useful and readily available imaging tool. The big challenge in acquiring a good ultrasound image are possible shadow artefacts that hide anatomical structures. This applies in particular to 3D ultrasound acquisitions, because shadow artefacts may be recorded outside the visualized image plane. There are only a few automatic methods for shadow artefact detection. In our work we like to introduce a new shadow detection method that is based on an adaptive thresholding approach. The development was attempted, after existing methods had been extended to separate shadow and fluid regions. The entire detection procedure utilizes only the ultrasound scan line information and some basic knowledge about the ultrasound propagation inside the human body. Applying our method, the ultrasound operator can retrieve combined information about shadow and fluid locations, that may be invaluable for image acquisition or diagnosis. The method can be applied to conventional 2D as well as 3D ultrasound images.

View article

Show publication details

Jung, Florian; Biebl-Rydlo, Medea; Daisne, Jean-François; Wesarg, Stefan

Automatic Sentinel Lymph Node Localization in Head and Neck Cancer Using a Coupled Shape Model Algorithm

2017

Computer Assisted and Robotic Endoscopy and Clinical Image-Based Procedures

The localization and analysis of the sentinel lymph node for patients diagnosed with cancer, has significant influence on the prognosis, outcome and treatment of the disease. We present a fully automatic approach to localize the sentinel lymph node and additional active nodes and determine their lymph node level on SPECT-CT data. This is a crucial prerequisite for the planning of radiation therapy or a surgical neck dissection. Our approach was evaluated on 17 lymph nodes. The detection rate of the lymph nodes was 94%; and 88% of the lymph nodes were correctly assigned to their corresponding lymph node level. The proposed algorithm targets a very important topic in clinical practice. The first results are already very promising. The next step has to be the evaluation on a larger data set.

View article | PDF

Show publication details

Raudaschl, Patrik F .; Zaffino, Paolo; Spadea, Maria Francesca; Chen, Antong; Dawant, Benoit M .; Albrecht, Thomas; Gass, Tobias; Langgut, Christoph; Lüthi, Marcel; Jung, Florian; Knapp, Oliver; Wesarg, Stefan; Mannion-Haworth, Richard; Bowes, Mike; Ashman, Annaliese; Guillard, Gwenael; Brett, Alan; Vincent, Graham; Orbes-Arteaga, Mauricio; Cárdenas-Peña, David; Castellanos-Dominguez, German; Aghdasi, Nava; Li, Yangming; Berens, Angelique; Moe, Kris; Hannaford, Blake; Schubert, Rainer; Fritscher, Karl D.

Evaluation of Segmentation Methods on Head and Neck CT: Auto-segmentation Challenge 2015

2017

Medical Physics

Purpose automated delineation of structures and organs is a key step in medical imaging. However, due to the large number and diversity of structures and the large variety of segmentation algorithms, a consensus is lacking as to which automated segmentation method works best for certain applications. Segmentation challenges are a good approach for unbiased evaluation and comparison of segmentation algorithms. Methods In this work, we describe and present the results of the Head and Neck Auto-Segmentation Challenge 2015, a satellite event at the Medical Image Computing and Computer Assisted Interventions (MICCAI) 2015 conference. Six teams participated in a challenge to segment nine structures in the head and neck region of CT images: brainstem, mandible, chiasm, bilateral optic nerves, bilateral parotid glands, and bilateral submandibular glands. Results This paper presents the quantitative results of this challenge using multiple established error metrics and a well-defined ranking system. The strengths and weaknesses of the different auto-segmentation approaches are analyzed and discussed. Conclusions The Head and Neck Auto-Segmentation Challenge 2015 was a good opportunity to assess the current state-of-the-art in segmentation of organs at risk for radiotherapy treatment. Participating teams had the possibility to compare their approaches to other methods under unbiased and standardized circumstances. The results demonstrate a clear tendency toward more general purpose and fewer structure-specific segmentation algorithms.

View article

Show publication details

Ma, Jingting; Wang, Anqi; Lin, Feng; Wesarg, Stefan; Erdt, Marius

Nonlinear Statistical Shape Modeling for Ankle Bone Segmentation Using a Novel Kernelized Robust PCA

2017

Medical Image Computing and Computer Assisted Intervention - MICCAI 2017: Part I

International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)

Statistical shape models (SSMs) are widely employed in medical image segmentation. However, an inferior SSM will degenerate the quality of segmentations. It is challenging to derive an efficient model because: (1) often the training datasets are corrupted by noise and / or artifacts; (2) Conventional SSM is not capable of capturing nonlinear variabilities of a population of shape. Addressing these challenges, this work aims to create SSMs that are not only robust to abnormal training data but also satisfied with nonlinear distribution. As Robust PCA is an efficient tool to seek a clean low-rank linear subspace, a novel kernelized Robust PCA (KRPCA) is proposed to cope with nonlinear distribution for statistical shape modeling. In evaluation, the built nonlinear model is used in ankle bone segmentation where 9 bones are separately distributed. Evaluation results show that the model built with KRPCA has a significantly higher quality than other state-of-the-art methods.

View article

Show publication details

Prinold, Joe A. I .; Mazzà, Claudia; Di Marco, Roberto; Hannah, Iain; Malattia, Clara; Magni-Manzoni, Silvia; Petrarch, Maurizio; Ronchetti, Anna; Tanturri De Horatio, Laura; van Dijkhuizen, Pieter E.H .; Wesarg, Stefan; Viceconti, Marco

A Patient-Specific Foot Model for the Estimate of Ankle Joint Forces in Patients with Juvenile Idiopathic Arthritis

2016

Annals of Biomedical Engineering

Juvenile idiopathic arthritis (JIA) is the leading cause of childhood disability from a musculoskeletal disorder. It generally affects large joints such as the knee and the ankle, often causing structural damage. Different factors contribute to the damage onset, including altered joint loading and other mechanical factors, associated with pain and inflammation. The prediction of patients' joint loading can hence be a valuable tool in understanding the disease mechanisms involved in structural damage progression. A number of lower-limb musculoskeletal models have been proposed to analyze the hip and knee joints, but juvenile models of the foot are still lacking. This paper presents a modeling pipeline that allows the creation of juvenile patient-specific models starting from lower limb kinematics and foot and ankle MRI data. This pipeline has been applied to data from three children with JIA and the importance of patient-specific parameters and modeling assumptions has been tested in a sensitivity analysis focused on the variation of the joint reaction forces. This analysis highlighted the criticality of patient-specific definition of the ankle joint axes and location of the Achilles tendon insertions. Patient-specific detection of the Tibialis Anterior, Tibialis Posterior, and Peroneus Longus origins and insertions were also shown to be important.

View article | PDF

Show publication details

Colter, Lena; Scheckenbach, Kathrin; Stenin, Igor; Wesarg, Stefan; Klenzner, Thomas; Schipper, Jörg; Young, Florian

Acoustic neuroma segmentation: application of the radial-ray-based 3D methodology

2016

GMS Current Posters in Otorhinolaryngology - Head and Neck Surgery

Introduction: Acoustic neuromas are benign tumors of the vestibular nerve in the area of ​​the cerebellopontine angle or the internal auditory canal. In the case of slow growth, in addition to surgical removal or radiation therapy, a "wait and scan" strategy with regular MRI controls is possible. Objective tumor volume determinations can be carried out by means of time-consuming segmentations. By automating the segmentation process, this method can be used quickly, precisely and objectively. Method: The radial-ray-based 3D segmentation sends rays radially in all directions starting from a manually specified seed point and generates a segmentation using image information and local shape knowledge. Within a few seconds the axes and the volume of the tumor are displayed. Within a project, the method was specifically developed for acoustic neuromas and validated on our patient population. Measurements in manual and automated segmentation were carried out by various examiners in order to evaluate the reliability, speed and suitability for everyday use of the method. Results: The volume of acoustic neuromas can also be reproducibly automated by different examiners with high accuracy within a few seconds and thus segmented faster than manually. Conclusion: The automated radial-ray-based 3D segmentation is a very suitable method for the objective determination of the volume of acoustic neuromas. It reduces the inter-observer variability and reduces the time required for image assessment. In this respect, this method has good potential, especially in the "<.wait and="" scan"-methode="" in="" den="" klinischen="" alltag="" eingeführt="" zu="">

View article

Show publication details

2016

Clinical Image-Based Procedures. Translational Research in Medical Imaging

International Workshop on Clinical Image-based Procedures (CLIP)

Ultrasound provides a useful and readily available imaging tool to detect free fluids in blunt abdominal trauma patients. However, applying conventional 2D ultrasound to diagnose the patient requires a well trained physician. In this paper we describe a fully automatic free fluid detection pipeline for the hepathorenal recess or Morrison's pouch using 3D ultrasound acquisitions. The image data is collected using the standardized "Focused Assessment with Sonography for Trauma" (FAST) exam. Our method extracts key structures like the kidney and the liver from the image data and uses their relative positions to search and detect free fluids between the organ interfaces. To evaluate our method we have developed a free fluid simulation that allows us to generate free fluid images using acquisitions of healthy volunteers. Our intentions are to enable even untrained ultrasound operators to perform a free fluid diagnosis of an injured person. In order to do this, our method additionally provides basic image acquisition guidance information.

View article

Show publication details

2016

Medical Imaging 2016: Ultrasonic Imaging and Tomography