Simulation of brain tumors in MR images for evaluation of segmentation efficacy

doi:10.1016/j.media.2008.11.002

Medical Image Analysis

Volume 13, Issue 2, April 2009, Pages 297-311

https://doi.org/10.1016/j.media.2008.11.002 Get rights and content

Abstract

Obtaining validation data and comparison metrics for segmentation of magnetic resonance images (MRI) are difficult tasks due to the lack of reliable ground truth. This problem is even more evident for images presenting pathology, which can both alter tissue appearance through infiltration and cause geometric distortions. Systems for generating synthetic images with user-defined degradation by noise and intensity inhomogeneity offer the possibility for testing and comparison of segmentation methods. Such systems do not yet offer simulation of sufficiently realistic looking pathology. This paper presents a system that combines physical and statistical modeling to generate synthetic multi-modal 3D brain MRI with tumor and edema, along with the underlying anatomical ground truth, Main emphasis is placed on simulation of the major effects known for tumor MRI, such as contrast enhancement, local distortion of healthy tissue, infiltrating edema adjacent to tumors, destruction and deformation of fiber tracts, and multi-modal MRI contrast of healthy tissue and pathology. The new method synthesizes pathology in multi-modal MRI and diffusion tensor imaging (DTI) by simulating mass effect, warping and destruction of white matter fibers, and infiltration of brain tissues by tumor cells. We generate synthetic contrast enhanced MR images by simulating the accumulation of contrast agent within the brain. The appearance of the the brain tissue and tumor in MRI is simulated by synthesizing texture images from real MR images. The proposed method is able to generate synthetic ground truth and synthesized MR images with tumor and edema that exhibit comparable segmentation challenges to real tumor MRI. Such image data sets will find use in segmentation reliability studies, comparison and validation of different segmentation methods, training and teaching, or even in evaluating standards for tumor size like the RECIST criteria (response evaluation criteria in solid tumors).

Introduction

The segmentation of brain tumor from magnetic resonance (MR) images is a vital process for treatment planning, monitoring of therapy, examining efficacy of radiation and drug treatments, and studying the differences of healthy subjects and subjects with tumor. The process of automatically extracting tumors from MR images is a challenging process. This leads to many different approaches for automatic tumor segmentation (Clark et al., 1998, Kaus et al., 2001, Prastawa et al., 2004). The usual standard used for validating segmentation results of the automatic methods is the manual segmentation results done by human experts. However, different investigators are likely to employ different image acquisition parameters and different manual segmentation techniques. A compounding issue is that any manual segmentation method suffers from lack of reliability and reproducibility. Even if a rich set of manual segmentations are available, they may not reflect the ground truth and the true gold standard may need to be estimated (Warfield et al., 2004). Furthermore, validation is typically not performed for the segmentations of non-tumor structures since manual segmentations of edema and the healthy brain tissue are very challenging tasks and have a high degree of variability.

Brain MRI exhibiting tumor is difficult to segment due to a combination of the following factors:

•
The deformation of brain tissue due to tumor mass effect or volume expansion.
•
The infiltration of brain tissue by tumor and edema (swelling). Edema appears around tumor mainly in the white matter regions and may also contain infiltrative tumor cells.
•
The gradual transition between tumor, edema, and surrounding brain tissue. This results in the ambiguity of the structural boundaries.
•
The T1w MRI with contrast enhancement, typically using a gadolinium agent, is the standard modality for identifying tumors. This modality results in active tumor tissue appearing with bright intensity. Unfortunately, blood vessels also appear bright while parts of tumor that are necrotic do not have higher levels of intensity. Therefore, the information provided by the intensities in this modality is not always consistent, and it is generally impossible to segment the tumor by thresholding the intensities in this image modality.

In order to provide objective assessments of segmentation performance, there is a need for an objective 3D ground truth with associated MR images that exhibit the same major segmentation challenges as that of common, realistic scans of a tumor patient. A database of real brain tumor MR images, along with their segmentations, may provide the means to measure the performance of an algorithm by comparing the results against the variability of the expert raters’ judgements. However, an objective evaluation to systematically compare different methodologies also needs a ground truth with little or no variability. An example of such a ground truth is the synthetic brain MRI database provided by the Montreal Neurological Institute¹ that is currently considered to be the common standard for evaluating the segmentations of healthy brain MR images. For this purpose, we propose a method that generates realistic looking MR images with the associated ground truth by approximating the brain tumor generation process.

Rexilius et al. (2004) proposed a framework for generating digital brain phantoms with tumor. They used a biomechanical linear elastic finite element model to simulate the tumor mass effect. In their method, the MRI of a healthy subject is deformed and a tumor structure from a real subject is inserted into the MRI. Their model for edema is computed from the distances to the tumor boundary and the white matter mask. This is insufficient to simulate real edema infiltration properties since infiltration can occur in regions away from tumor. Such regions are typically connected through white matter fibers. The framework of Rexilius et al. only considered contrast enhancement inside tumors, without contrast enhancement of blood vessels.

Models for brain tumor expansion and edema have been proposed by Nagashima et al., 1990a, Clatz et al., 2004, Clatz et al., 2005, Mohamed and Davatzikos, 2005, Mohamed et al., 2006. More recently, Clatz et al. developed a realistic tumor growth model that explicitly simulates the main effects of tumor growth (mass effect and infiltration) using simple computational models. Clatz et al. used a linearized biomechanical finite element model to simulate mass effect and they used a reaction–diffusion process that is modulated by the diffusion tensor field to simulate the infiltration by tumor cells and edema. The simple computational models used by Clatz et al. are ideal for generating realistic tumor models in an efficient manner. We propose a method for generating new pathological ground truth by applying their mass effect and infiltration model to a well defined ground truth for normal brains. Additionally, we propose to extend the Clatz et al. model by using random pressure directions, and by simulating the effect of volume expansion on the white matter fibers by warping the diffusion tensors and making them more isotropic depending on the magnitude of local deformations.

We develop a method for generating realistic-appearing contrast enhanced T1 weighted MR images (a standard modality for diagnosis) by simulating the accumulation of contrast agents in the brain. The corresponding multi-modal MR images (contrast enhanced T1w, T1w, and T2) are generated from the simulated ground truth and from textures that are synthesized using samples of real tumor MRI data. Fig. 1 shows an overview of the proposed method. Our method is capable of generating 3D whole brain ground truth that exhibits the primary effects of tumor on normal brains, along with simulated multi-modal MR images that are challenging to segment.

The proposed method does not attempt to simulate the complete process of real tumor growth and the true MR image generation process. Instead, our aim is to generate a database of synthetic brain tumor MR images that have similar challenges for segmentation as in real tumors, along with the associated anatomical ground truth. The simulated brain tumor MR images can function as test data for any segmentation method and the ground truth can provide the means for objective assessment of segmentation performance. We do not aim to create a database of simulated brain tumor MR images that are indistinguishable from real brain tumor MR images. Such an effort requires the faithful modeling of the anatomical, chemical, and vascular changes in the brain due to tumor. It would also require the exact formulation of what neuroradiologists and neurosurgeons define as tumor. Currently, this definition involves a large degree of intuition and cannot be formulated algorithmically. Our simulated data provides a standard benchmark for different tumor segmentation methods that is currently not available to the community.

Section snippets

Generation of pathological ground truth

Tumor and edema growth involves many concurrently occurring processes. As proposed by Wasserman et al. (1996), the growth model may involve biomechanics, nutrient distribution, and metabolic processes. Since our goal is not to model tumor growth per se, we have chosen to simplify the model and use three separate sequential processes for efficiency, as shown in Fig. 2. First, we simulate the deformation that is due to tumor mass effect using a biomechanical model. It is then followed by the

Generation of MR images

For the purpose of validating segmentation methods, we need a set of synthetic MR images that appear reasonably realistic and that correspond to the 3D pathological ground truth. These images serve as test data for the evaluation of segmentation methods. The generation of synthetic tumor MRI involves the simulation of two processes: contrast enhancement in T1w MRI due to the use of contrast agents (the standard modality for tumor diagnosis), and generation of intensity patterns similar to those

Results

We generated five synthetic MR datasets with varying tumor location, tumor count, levels of tumor expansion, and extent of edema. Fig. 8, Fig. 9 show MR images of observed clinical cases that demonstrate the true variations of tumor appearance. The five synthetic brain tumor MRI datasets with similar variations are shown in Fig. 10, Fig. 11. SimTumor001 shows a tumor with significant mass effect and large surrounding edema. SimTumor002 shows a tumor that displaces the right ventricle from below

Discussion and conclusions

We presented a new method for generating modified ground truth with tumor and edema from a normal brain ground truth, along with a method for generating synthetic multi-modal MR images that present similar segmentation challenges as real tumor MRI. The process for generating a synthetic brain tumor dataset is summarized in Fig. 14. Our proposed simulation scheme introduces a tensor model for the warping and destruction of white matter fibers (demyelination). The scheme also introduces a

Acknowledgements

The authors would like to thank Sarang Joshi (University of Utah) and Stephen Pizer (University of North Carolina) for providing useful feedback on parts of the manuscript, and the maintainers and contributors of the open source packages Insight Toolkit (Kitware, 2008a) and Visualization Toolkit (Kitware, 2008b) for providing the framework for the software system described in this paper.

References (42)

A. Mohamed et al.
Deformable registration of brain tumor images via a statistical model of tumor-induced deformation
Medical Image Analysis
(2006)
M. Prastawa et al.
A brain tumor segmentation framework based on outlier detection
Med. Image Anal.
(2004)
M. Prastawa et al.
Automatic segmentation of MR images of the developing newborn brain
Med. Image Anal.
(2005)
R. Wasserman et al.
A patient-specific in vivo tumor model
Math. Biosci.
(1996)
X. Zheng et al.
Nonlinear simulation of tumor necrosis, neo-vascularization and tissue invasion via and adaptive finite-element/level-set method
Bull. Math. Biol.
(2005)
D.C. Alexander et al.
Spatial transformations of diffusion tensor magnetic resonance images
IEEE Trans. Med. Imaging
(2001)
Arsigny, V., Fillard, P., Pennec, X., Ayache, N., 2005. Fast and simple calculus on tensors in the log-euclidean...
B. Aubert-Broche et al.
Twenty new digital brain phantoms for creation of validation of image data bases
IEEE Trans. Med. Imaging
(2006)
O. Axelsson et al.
Finite Element Solution of Boundary Value Problems
(1984)
C.B. Barber et al.
The Quickhull algorithm for convex hulls
ACM Trans. Math. Software
(1996)

E. Bullitt et al.

Measuring tortuosity of the intracerebral vasculature from MRA images

IEEE Trans. Med. Imaging

(2003)

M.C. Clark et al.

Automatic tumor-segmentation using knowledge-based techniques

IEEE Trans. Med. Imaging

(1998)

Clatz, O., Bondiau, P.-Y., Delingette, H., Malandain, G., Sermesant, M., Warfield, S.K., Ayache, N., 2004. In silico...

O. Clatz et al.

Realistic simulation of the 3D growth of brain tumors in MR images including diffusion and mass effect

IEEE Trans. Med. Imaging

(2005)

C.A. Cocosco et al.

BrainWeb: Online interface to a 3D MRI simulated brain database

NeuroImage

(1997)

L.R. Dice

Measures of the amount of ecologic association between species

Ecology

(1945)

M. Ferrant et al.

Registration of 3D interoperative MR images of the brain using finite element biomechanical model

IEEE Trans. Med. Imaging

(2001)

G. Gerig et al.

VALMET: A new validation tool for assessing and improving 3D object segmentation

M.E. Gurtin

An Introduction to Continuum Mechanics

(1981)

Ho, S., Bullitt, E., Gerig, G., 2002. Level set evolution with region competition: Automatic 3-D segmentation of brain...

T.J.R. Hughes

The Finite Element Method: Linear Static and Dynamic Finite Element Analysis

(2000)

Cited by (120)

Brain MR image simulation for deep learning based medical image analysis networks
2024, Computer Methods and Programs in Biomedicine
As large sets of annotated MRI data are needed for training and validating deep learning based medical image analysis algorithms, the lack of sufficient annotated data is a critical problem. A possible solution is the generation of artificial data by means of physics-based simulations. Existing brain simulation data is limited in terms of anatomical models, tissue classes, fixed tissue characteristics, MR sequences and overall realism.
We propose a realistic simulation framework by incorporating patient-specific phantoms and Bloch equations-based analytical solutions for fast and accurate MRI simulations. A large number of labels are derived from open-source high-resolution T1w MRI data using a fully automated brain classification tool. The brain labels are taken as ground truth (GT) on which MR images are simulated using our framework. Moreover, we demonstrate that the T1w MR images generated from our framework along with GT annotations can be utilized directly to train a 3D brain segmentation network. To evaluate our model further on larger set of real multi-source MRI data without GT, we compared our model to existing brain segmentation tools, FSL-FAST and SynthSeg.
Our framework generates 3D brain MRI for variable anatomy, sequence, contrast, SNR and resolution. The brain segmentation network for WM/GM/CSF trained only on T1w simulated data shows promising results on real MRI data from MRBrainS18 challenge dataset with a Dice scores of $0.818 / 0.832 / 0.828$ . On OASIS data, our model exhibits a close performance to FSL, both qualitatively and quantitatively with a Dice scores of $0.901 / 0.939 / 0.937$ .
Our proposed simulation framework is the initial step towards achieving truly physics-based MRI image generation, providing flexibility to generate large sets of variable MRI data for desired anatomy, sequence, contrast, SNR, and resolution. Furthermore, the generated images can effectively train 3D brain segmentation networks, mitigating the reliance on real 3D annotated data.
Efficient brain tumor segmentation using OTSU and K-means clustering in homomorphic transform
2023, Biomedical Signal Processing and Control
In computer vision, image segmentation technology plays a vital role in computer-aided diagnostic systems to precisely and accurately identify the area to be treated. Practically, it is usually not interested in all image parts, but only the required characteristics. Therefore, image segmentation algorithms can bound the cancerous cells in boxes to determine the severity of the cancer and get more clear results. This paper examined an efficient approach based on local contrast enhancement using the OTSU segmentation and K-means clustering segmentation in different transform domains for localizing the brain tumor area at a competitive time. The transform domains are providing better representation for images to be segmented and enabling much feature information for extraction. Moreover, adaptive histogram equalization with the local contrast enhancement has been improved the whole performance of the proposed approach. The proposed enhanced segmentation approaches have been tested with different datasets and evaluated using various quality metrics such as; accuracy, sensitivity, precision, specificity, MCC, F-measure, Dice, Jaccard, and the processing time. The proposed segmentation approaches in homomorphic transform and adaptive histogram equalization have conducted an efficient and reliable performance with high precision in the brain tumor localization. Moreover, the proposed algorithms have implemented for both 2D and 3D images and can provide higher efficiency and better image quality with both 2D and 3D image types achieving better robustness and proving that the proposed algorithms don’t depend on the input datasets.
MRI-based brain tumour image detection using CNN based deep learning method
2022, Smart Agricultural Technology
In modern days, checking the huge number of MRI (magnetic resonance imaging) images and finding a brain tumour manually by a human is a very tedious and inaccurate task. It can affect the proper medical treatment of the patient. Again, it can be a hugely time-consuming task as it involves a huge number of image datasets. There is a good similarity between normal tissue and brain tumour cells in appearance, so segmentation of tumour regions become a difficult task to do. So there is an essentiality for a highly accurate automatic tumour detection method.
In this paper, we proposed an algorithm to segment brain tumours from 2D Magnetic Resonance brain Images (MRI) by a convolutional neural network which is followed by traditional classifiers and deep learning methods. We have taken various MRI images with diverse Tumour sizes, locations, shapes, and different image intensities to train the model well. Furthermore, we have applied SVM classifier and other activation algorithms (softmax, RMSProp, sigmoid, etc) to cross-check our work. We implement our proposed method using “TensorFlow” and “Keras” in “Python” as it is an efficient programming language to perform fast work.
In our work, CNN gained an accuracy of 99.74%, which is better than the state of the result obtained so far.
Our CNN based model will help the doctors to detect brain tumours in MRI images accurately, so that the speed in treatment will increase a lot.
A novel minimum generalized cross entropy-based multilevel segmentation technique for the brain MRI/dermoscopic images
2022, Computers in Biology and Medicine
One of the challenging and the primary stages of medical image examination is the identification of the source of any disease, which may be the aberrant damage or change in tissue or organ caused by infections, injury, and a variety of other factors. Any such condition related to skin or brain sometimes advances in cancer and becomes a life-threatening disease. So, an efficient automatic image segmentation approach is required at the initial stage of medical image analysis.
To make a segmentation process efficient and reliable, it is essential to use an appropriate objective function and an efficient optimization algorithm to produce optimal results.
The above problem is resolved in this paper by introducing a new minimum generalized cross entropy (MGCE) as an objective function, with the inclusion of the degree of divergence. Another key contribution is the development of a new optimizer called opposition African vulture optimization algorithm (OAVOA). The proposed optimizer boosted the exploration, skill by inheriting the opposition-based learning.
The experimental work in this study starts with a performance evaluation of the optimizer over a set of standards (23 numbers) and IEEE CEC14 (8 numbers) Benchmark functions. The comparative analysis of test results shows that the OAVOA outperforms different state-of-the-art optimizers. The suggested OAVOA-MGCE based multilevel thresholding approach is carried out on two different types of medical images – Brain MRI Images (AANLIB dataset), and dermoscopic images (ISIC 2016 dataset) and found superior than other entropy-based thresholding methods.
Machine learning applications to neuroimaging for glioma detection and classification: An artificial intelligence augmented systematic review
2021, Journal of Clinical Neuroscience
Glioma is the most common primary intraparenchymal tumor of the brain and the 5-year survival rate of high-grade glioma is poor. Magnetic resonance imaging (MRI) is essential for detecting, characterizing and monitoring brain tumors but definitive diagnosis still relies on surgical pathology. Machine learning has been applied to the analysis of MRI data in glioma research and has the potential to change clinical practice and improve patient outcomes. This systematic review synthesizes and analyzes the current state of machine learning applications to glioma MRI data and explores the use of machine learning for systematic review automation. Various datapoints were extracted from the 153 studies that met inclusion criteria and analyzed. Natural language processing (NLP) analysis involved keyword extraction, topic modeling and document classification. Machine learning has been applied to tumor grading and diagnosis, tumor segmentation, non-invasive genomic biomarker identification, detection of progression and patient survival prediction. Model performance was generally strong (AUC = 0.87 ± 0.09; sensitivity = 0.87 ± 0.10; specificity = 0.0.86 ± 0.10; precision = 0.88 ± 0.11). Convolutional neural network, support vector machine and random forest algorithms were top performers. Deep learning document classifiers yielded acceptable performance (mean 5-fold cross-validation AUC = 0.71). Machine learning tools and data resources were synthesized and summarized to facilitate future research. Machine learning has been widely applied to the processing of MRI data in glioma research and has demonstrated substantial utility. NLP and transfer learning resources enabled the successful development of a replicable method for automating the systematic review article screening process, which has potential for shortening the time from discovery to clinical application in medicine.
Towards effective machine learning in medical imaging analysis: A novel approach and expert evaluation of high-grade glioma ‘ground truth’ simulation on MRI
2021, International Journal of Medical Informatics
Citation Excerpt :
Such datasets however are labour-intensive and time-consuming to create and subjective with ground-truth labels being manually created, while still insufficient to meet the increasing need for data in machine learning studies, e.g., brain tumour deep learning [15–17]. An alternative approach involves creating simulated tumours on the brain MRI of healthy subjects [18–20]. In addition to the fast and automated nature, the known label of the images is advantageous.
Gliomas are uniformly fatal brain tumours with significant neurological and quality of life detriment to patients. Improvement in outcomes has remained largely unchanged in nearly 20 years. MRI (magnetic resonance imaging) is often used in diagnosis and management. Machine learning analyses of large-scale MRI data are pivotal in advancing the diagnosis, management and improve outcomes in neuro-oncology. A common challenge to robust machine learning approaches is the lack of large ‘ground truth’ datasets in supervised learning for building classification and prediction models. The creation of these datasets relies on human-expert input and is time-consuming and subjective error-prone, limiting effective machine learning applications. Simulation of mechanistic aspects such as geometry, location and physical properties of brain tumours can generate large-scale ground-truth datasets allowing for comparison of analysis techniques in clinical applications. We aimed to develop a transparent and convenient method for building ‘ground truth’ presentations of simulated glioma lesions on anatomical MRI.
The simulation workflow was created using the Feature Manipulation Engine (FME®), a data integration platform specializing in the spatial data processing. By compiling and integrating FME’s functions to read, integrate, transform, validate, save, and display MRI data, and experimenting with ways to manipulate the parameters concerning location, size, shape, and signal intensity with the presentations of glioma, we were able to generate simulated appearances of high-grade gliomas on gadolinium-based high-resolution 3D T1-weighted MRI (1 mm³). Data of patients with canonical high-grade tumours were used as real-world tumours for validating the accuracy of the simulation. Twenty raters who are experienced with brain tumour interpretation on MRI independently completed a survey, designed to distinguish simulated and real-world brain tumours. Sensitivity and specificity were calculated for assessing the performance of the approach with the binary classification of simulated vs real-world tumours. Correlation and regression were used in run time analysis, assessing the software toolset’s efficiency in producing different numbers of simulated lesions. Differences in the group means were examined using the non-parametric Kruskal-Wallis test.
The simulation method was developed as an interpretable and useful workflow for the easy creation of tumour simulations and incorporation into 3D MRI. A linear increase in the running time and memory usage was observed with an increasing number of generated lesions. The respondents' accuracy rate ranged between 33.3 and 83.3 %. The sensitivity and specificity were low for a human expert to differentiate simulated lesions from real gliomas (0.43 and 0.58) or vice versa (0.65 and 0.62). The mean scores ranking the real-world gliomas did not differ between the simulated and real tumours.
The reliable and user-friendly software method can allow for robust simulation of high-grade glioma on MRI. Ongoing research efforts include optimizing the workflow for generating glioma datasets as well as adapting it to simulating additional MRI brain changes.

View all citing articles on Scopus

^☆: This work was supported by the NIH NIBIB under grant R01 EB000219.

View full text

Simulation of brain tumors in MR images for evaluation of segmentation efficacy☆

Abstract

Introduction

Section snippets

Generation of pathological ground truth

Generation of MR images

Results

Discussion and conclusions

Acknowledgements

Medical Image Analysis

Med. Image Anal.

Med. Image Anal.

Math. Biosci.

Bull. Math. Biol.

Spatial transformations of diffusion tensor magnetic resonance images

IEEE Trans. Med. Imaging

Twenty new digital brain phantoms for creation of validation of image data bases

IEEE Trans. Med. Imaging

Finite Element Solution of Boundary Value Problems

The Quickhull algorithm for convex hulls

ACM Trans. Math. Software

Measuring tortuosity of the intracerebral vasculature from MRA images

IEEE Trans. Med. Imaging

Automatic tumor-segmentation using knowledge-based techniques

IEEE Trans. Med. Imaging

Realistic simulation of the 3D growth of brain tumors in MR images including diffusion and mass effect

IEEE Trans. Med. Imaging

BrainWeb: Online interface to a 3D MRI simulated brain database

NeuroImage

Measures of the amount of ecologic association between species

Ecology

Registration of 3D interoperative MR images of the brain using finite element biomechanical model

IEEE Trans. Med. Imaging

VALMET: A new validation tool for assessing and improving 3D object segmentation

An Introduction to Continuum Mechanics

The Finite Element Method: Linear Static and Dynamic Finite Element Analysis