Elsevier

Medical Image Analysis

Volume 14, Issue 1, February 2010, Pages 1-12
Medical Image Analysis

Automatic segmentation of colon glands using object-graphs

https://doi.org/10.1016/j.media.2009.09.001Get rights and content

Abstract

Gland segmentation is an important step to automate the analysis of biopsies that contain glandular structures. However, this remains a challenging problem as the variation in staining, fixation, and sectioning procedures lead to a considerable amount of artifacts and variances in tissue sections, which may result in huge variances in gland appearances. In this work, we report a new approach for gland segmentation. This approach decomposes the tissue image into a set of primitive objects and segments glands making use of the organizational properties of these objects, which are quantified with the definition of object-graphs. As opposed to the previous literature, the proposed approach employs the object-based information for the gland segmentation problem, instead of using the pixel-based information alone. Working with the images of colon tissues, our experiments demonstrate that the proposed object-graph approach yields high segmentation accuracies for the training and test sets and significantly improves the segmentation performance of its pixel-based counterparts. The experiments also show that the object-based structure of the proposed approach provides more tolerance to artifacts and variances in tissues.

Introduction

Histopathological examination includes examining a biopsy tissue under a microscope for the identification of tissue changes associated with disease. In the current practice of medicine, this examination is the most important tool for routine clinical diagnosis of a large group of diseases including cancer. However, as it mainly relies on the visual interpretation of a pathologist, it may lead to a certain level of subjectivity (Thomas et al., 1983, Andrion et al., 1995). To help pathologists in diagnosis, and hence, to reduce the subjectivity level, it has been proposed to use computational methods that provide objective measures (Wolberg et al., 1995, Thiran and Macq, 1996, Choi et al., 1997, Hamilton et al., 1997, Esgiar et al., 1998, Spyridonos et al., 2001, Wiltgen et al., 2003, Nielsen et al., 1999, Esgiar et al., 2002, Weyn et al., 1999, Keenan et al., 2000, Demir et al., 2005, Gunduz-Demir, 2007). These computational methods extract a set of mathematical features (e.g., morphological (Wolberg et al., 1995, Thiran and Macq, 1996), textural (Hamilton et al., 1997, Choi et al., 1997, Esgiar et al., 1998, Spyridonos et al., 2001, Wiltgen et al., 2003), fractal (Nielsen et al., 1999, Esgiar et al., 2002), and structural (Choi et al., 1997, Weyn et al., 1999, Keenan et al., 2000, Demir et al., 2005, Gunduz-Demir, 2007)) from a tissue image for its quantification and use these mathematical features to objectively measure the degree of the tissue changes associated with a disease of the interest. Different types of features might be necessary to quantify the tissue changes as these changes show differences from one tissue type to another as well as from one disease type to another. For example, soft tissue tumors change the cell distribution in the tissue whereas adenocarcinomas change the architecture of glands1 as well. To identify the latter type of neoplastic diseases, which cause changes in gland architectures, the very first step is to segment the tissue into its gland structures.

In literature, there are few studies that focus on the problem of automatic gland segmentation for tissues that contain gland structures (Wu et al., 2005a, Wu et al., 2005b, Naik et al., 2007, Farjam et al., 2007). These studies make use of the fact that glands are characterized by their luminal areas surrounded by epithelial cells; an example of the histopathological image of a colon tissue is given in Fig. 1. In order to capture this characterization, these studies first identify the pixels of different classes (e.g., nucleus, stroma, and lumen classes) and then form gland regions using this class information of pixels. For example, in Wu et al. (2005a), nucleus pixels are identified applying a threshold to the intensities of pixels after they are convolved with a composition of directional filters. The regions surrounded by these pixels are then determined as glands, provided that their areas are larger than a threshold. In another work of the same authors (Wu et al., 2005b), nucleus and lumen pixels are first determined also applying a threshold to their intensities. Subsequently, large enough connected components of lumen pixels are identified as gland seeds and these seeds are then iteratively grown until a barrier of nuclei chain is reached. In another work (Naik et al., 2007), a Bayesian classifier is used to classify the pixels into nucleus, lumen, and cytoplasm classes based on their intensity values. Then candidate gland regions are defined as the connected components of pixels for which the classifier outputs posteriors greater than a threshold for the lumen class. Finally, false glands are eliminated according to their sizes and the probability of their surrounding pixels belonging to the cytoplasm class. In Farjam et al. (2007), after clustering the pixels into nucleus, stroma, and lumen classes based on their textural properties, the glands are obtained excluding the regions containing nucleus pixels from those containing stroma and lumen pixels.

These studies yield promising results for especially tissues in which the glands appear in more regular structures showing less variations. However, due to staining, fixation, and sectioning procedures, there is a considerable amount of artifacts and variances in tissue sections, which may result in huge variances in gland appearances. First, glands could be of different sizes, depending on the orientation of the tissue at the time of sectioning. For example, although they are taken with the same magnification, the images shown in Fig. 2a–c have glands of different sizes. Furthermore, the improper orientation of the tissue produces tangential sectioning, which results in glands of different sizes within the same tissue image (Fig. 2d). Therefore, in false gland elimination, it is almost impossible to find an area threshold that applies for all images. Second, because of the density difference between the glandular and connective tissue structures, the fixation and sectioning procedures may result in large white artifacts on the boundaries of the glands (some of these artifacts are shown with red2 arrows in Fig. 2e and f). Considering only the pixel-based information, it is more difficult to distinguish such white artifacts from luminal regions. Third, the thickness of a tissue section and the freshness of dye cause variations in the intensity distribution of a tissue image. Moreover, stain fades in time. Therefore, a single threshold value could not be found for all images to determine their nucleus pixels. Even such a threshold is manually selected or automatically determined (e.g., by the Otsu method (Otsu, 1979)) for each image, the resulting nucleus pixels do not usually form a closed component even after postprocessing the pixels (e.g., using mathematical morphology (Serra, 1982)). Thus, it is rare to find continuous nucleus pixels that surround the luminal area. For example, although it is more possible to find such nucleus pixels in the tissue shown in Fig. 2g, it is much more difficult to find them for tissues shown in Fig. 2h and i. Because of all these issues, using only the pixel-based information leads to incorrect gland segmentations for especially tissues with artifacts and variations.

In this paper, we report a new gland segmentation algorithm that relies on decomposing the image into a set of primitive objects (nucleus and lumen objects) and then making use of the organizational properties of these objects instead of using the pixel-based information alone. This object-based algorithm is a three-step region growing approach. First, it constructs a graph on all of its objects and determines gland seeds based on the features extracted from this object-graph. Then, it constructs another graph, this time on its nucleus objects, and uses this second object-graph for growing the gland seeds. Finally, it determines the final boundary of glands based on the locations of the nucleus objects. After this region-growing process, false glands are eliminated based on the cluster information of the grown regions. Working with colon tissues of 36 different patients, our experiments show that the region-growing process leads to 82.57% average segmentation accuracy on the test set and that this accuracy increases to 87.59% after the false gland elimination step. These results (both before and after false gland elimination) demonstrate that the proposed object-based algorithm significantly improves the segmentation performance of its pixel-based counterparts. To the best of our knowledge, this is the first demonstration of the use of object-graphs for the purpose of gland segmentation.

Section snippets

Overview

The proposed object-graph approach relies on modeling the regular structure of glands. For this purpose, it decomposes a tissue image into a set of objects, which represent different tissue components, and uses the way that they distribute within the tissue in a region-growing process to determine the locations of gland structures. Compared to pixel-based information, the use of object-based information in a region-growing process yields more robust segmentations as pixel intensities are

Experiments

The experiments are conducted on 72 microscopic images of colon biopsy samples of 36 randomly chosen patients (two randomly selected images for each patient) from the Pathology Department archives in Hacettepe University School of Medicine. Each sample consists of 5 μm-thick tissue section and is stained with the hematoxylin-and-eosin technique. The images of these samples are taken using a Nikon Coolscope Digital Microscope with 20× microscope objective lens. These images are taken in the RGB

Discussions

In this work, we introduce an object-based approach for the purpose of gland segmentation. This approach decomposes the tissue image into a set of primitive objects and segments glands making use of the spatial distributions of these objects, which are quantified with the definition of object-graphs. In this work, the experiments are conducted on the images of 72 colon tissues of 36 different patients. Experimental results demonstrate that the proposed object-based approach yields high

Acknowledgment

This work has been supported by the Scientific and Technological Research Council of Turkey under the project number TÜBİTAK 106E118.

References (29)

  • C. Demir et al.

    Learning the topological properties of brain tumors

    IEEE-ACM Transactions on Computational Biology and Bioinformatics

    (2005)
  • A.N. Esgiar et al.

    Microscopic image analysis for quantitative measurement and feature identification of normal and cancerous colonic mucosa

    IEEE Transactions on Information Technology in Biomedicine

    (1998)
  • A.N. Esgiar et al.

    Fractal analysis in the detection of colonic cancer images

    IEEE Transactions on Information Technology in Biomedicine

    (2002)
  • R. Farjam et al.

    An image analysis approach for automatic malignancy determination of prostate pathological images

    Clinical Cytometry

    (2007)
  • Cited by (144)

    • Detail preserving conditional random field as 2-D RNN for gland segmentation in histology images

      2022, Pattern Recognition Letters
      Citation Excerpt :

      Finally, the paper is concluded in Section 5. While there were a few prior works [13,14], interest in automated gland segmentation increased significantly following the gland segmentation challenge contest [3] of 2015. A stochastic polygon model for gland segmentation was proposed in [9].

    View all citing articles on Scopus
    View full text