A non-local approach for image super-resolution using intermodality priors

Abstract

Image enhancement is of great importance in medical imaging where image resolution remains a crucial point in many image analysis algorithms. In this paper, we investigate brain hallucination (Rousseau, 2008), or generating a high-resolution brain image from an input low-resolution image, with the help of another high-resolution brain image. We propose an approach for image super-resolution by using anatomical intermodality priors from a reference image. Contrary to interpolation techniques, in order to be able to recover fine details in images, the reconstruction process is based on a physical model of image acquisition. Another contribution to this inverse problem is a new regularization approach that uses an example-based framework integrating non-local similarity constraints to handle in a better way repetitive structures and texture. The effectiveness of our approach is demonstrated by experiments on realistic Brainweb Magnetic Resonance images and on clinical images from ADNI, generating automatically high-quality brain images from low-resolution input.

Introduction

Image processing algorithm performances are often limited by the image resolution. Resolution is for instance a key point in brain segmentation in Magnetic Resonance (MR) imaging for which partial volume effect is a limiting factor for fine image analysis. Then, it clearly appears that improving image resolution is still one of the main challenges in medical image processing. In medical imaging, a so-called low-resolution (LR) 3D image is usually a stack of 2D thick slices. As a result, 3D data are generally not isotropic. Fig. 1 shows for instance a high-resolution (HR) T1-weighted image and a LR T2-weighted image of the same patient (in this case, these images come from the Brainweb database Cocosco et al., 1997). In many cases, it may be necessary to put HR and LR images into a same coordinate system. In medical imaging, this is usually done by applying interpolation techniques (Lehmann et al., 1999). Image interpolation is a very common image processing technique in medical imaging pipelines and it may have a strong impact on other processing steps such as segmentation or registration. Interpolation methods can be divided into two groups: scene-based and object-based methods (Grevera and Udupa, 1998). Scene-based approaches use only image intensities to determine the interpolated intensity (for instance: nearest-neighbor, linear interpolation, spline-based interpolation). Such scene-based methods produce perceptually unsatisfactory results with blurred edges and textures. Many edge-preserving interpolation techniques have been reported to handle this problem. However, these techniques rely on accurate edge information that is not obtainable from coarse data. In order to guide the interpolation process, object-based methods make use of additional information extracted from images. As an example, non-rigid registration techniques have been used to drive interpolation methods between adjacent slices (Frakes et al., 2008, Penney et al., 2004). However, scene-based and object-based techniques do not take advantage of a model of the imaging process.

Super-resolution (SR) is another model-based technique which relies on modeling the imaging processes and using regularization methods describing a priori constraints. The principle of this approach is usually combining LR images to produce an image that has a higher spatial resolution than the original images (Bose et al., 2004). In medical imaging, several SR methods have been proposed to combine LR images to reconstruct one HR image (Peeters et al., 2004, Carmi et al., 2006, Rousseau et al., 2006) (usually by modifying the acquisition protocol). SR is a large research field encompassing many applications. The work we present in this paper is related to the single-frame SR framework (van Ouwerkerk, 2006), meaning that only one LR image is used to generate an HR image. More specifically, we focus on studies involving MR imaging for which an anatomical HR image and several other LR images are acquired to keep acquisition time at an acceptable level for the patient (see Fig. 1). This is the case for routinely performed clinical MR acquisitions such as follow-up of neurodegenerative diseases, brain tumor evolution or diffusion tensor imaging. Typically, one isotropic HR T1-weighted image and several anisotropic LR images (such as T2-weighted, FLAIR or proton density images) are acquired. One can note that this issue of relative difference in image resolution may also occur in other clinical settings. For instance, patient follow-up may require an analysis of scans taken at different magnetic field strength, potentially at different clinical sites.

In such context, we propose a new approach for image SR by using information from an HR MR image to drive the image reconstruction of the LR MR image. The general framework developed in this paper can be applied to other domains where image resolution is an important issue, such as remote sensing. This paper is built on previously published work (Rousseau, 2008). Many experiments have been added in order to provide further insight to the proposed approach. In Section 2, we present the image SR problem using a model-based framework and some recently proposed example-based approaches. Section 3 details our non-local approach for image SR using an HR reference MR image. In Section 4, results obtained on realistic brainweb images are presented and discussed.