Skip to main content
  • Research Article
  • Open access
  • Published:

A deep error correction network for compressed sensing MRI

Abstract

Background

CS-MRI (compressed sensing for magnetic resonance imaging) exploits image sparsity properties to reconstruct MRI from very few Fourier k-space measurements. Due to imperfect modelings in the inverse imaging, state-of-the-art CS-MRI methods tend to leave structural reconstruction errors. Compensating such errors in the reconstruction could help further improve the reconstruction quality.

Results

In this work, we propose a DECN (deep error correction network) for CS-MRI. The DECN model consists of three parts, which we refer to as modules: a guide, or template, module, an error correction module, and a data fidelity module. Existing CS-MRI algorithms can serve as the template module for guiding the reconstruction. Using this template as a guide, the error correction module learns a CNN (convolutional neural network) to map the k-space data in a way that adjusts for the reconstruction error of the template image. We propose a deep error correction network. Our experimental results show the proposed DECN CS-MRI reconstruction framework can considerably improve upon existing inversion algorithms by supplementing with an error-correcting CNN.

Conclusions

In the proposed a deep error correction framework, any off-the-shelf CS-MRI algorithm can be used as template generation. Then a deep neural network is used to compensate reconstruction errors. The promising experimental results validate the effectiveness and utility of the proposed framework.

Background

MRI (Magnetic resonance imaging) is an important medical imaging technique with high resolution in soft tissues, low radiations, but the slow imaging speed is a major drawback of MRI. CS (Compressed sensing) theory [1, 2] has been a significant development of the signal acquisition and reconstruction process that has allowed for significant acceleration of MRI with less k-space measurements. The CS-MRI problem can be formulated as the optimization

$$ \hat x = \mathop {\arg \min }\limits_{x} \left\| {{F_{u}}x - y} \right\|_{2}^{2} + \sum\limits_{i} {{\alpha_{i}}{\Psi_{i}}\left(x \right)}, $$
(1)

where xCN×1 is the complex-valued MRI to be reconstructed, FuCM×N is the under-sampled Fourier matrix and yCM×1 (MN) are the k-space data measured by the MRI machine. The first data fidelity term ensures agreement between the Fourier coefficients of the reconstructed image and the measured data, while the second term regularizes the reconstruction to encourage certain image properties such as sparsity in a transform domain.

Recently, the compressed sensing MRI is approved by the FDA (Food and Drug Administration) to two main MRI vendors: GE and Siemens [3]. As the growing needs for application of compressed sensing MRI, improving reconstruction accuracy of the CS-MRI is of great significance. In this paper, we propose a deep learning framework called DECN (deep error correction network) in which an arbitrary CS-MRI inversion algorithm is combined with a deep learning error correction network. The network is trained for a specific inversion algorithm to exploit structural consistencies in the errors they produce. The final reconstruction is found by combining the information from the original algorithm with the error correction of the network.

A lot of previous works focus on proposing appropriate regularizations that lead to better MRI reconstructions. In the pioneering work of CS-MRI called SparseMRI [4], this regularization adds an 1 penalty on the wavelet coefficients and the total variation of the reconstructed image. Based on SparseMRI, more efficient optimization methods have been proposed to optimize this objective, such as TVCMRI (Total Variation 1 Compressed MR Imaging) [5], RecPF (Reconstruction From Partial Fourier Data) [6] and FCSA (Fast Composite Splitting Algorithm) [7]. Variations on the wavelet penalty exploit geometric information of MRI, such as PBDW/PBDWS (Patch Based Directional Wavelet) [8, 9] and GBRWT (Graph Based Redundant Wavelet Transform) [10], for improved results. Dictionary learning methods [1114] have also been applied to CS-MRI reconstruction, as have nonlocal priors such as NLR (Non-Local Regularization) [15], PANO (Patch Based Non-Local Operator) [16] and BM3D-MRI (Block-Matching 3D MRI) [17]. These previous works can be considered sparsity-promoting regularized CS-MRI methods that are optimized using iterative algorithms. They also represent images using simple single layer features that are either predefined (e.g., wavelets) or learned from the data (e.g., dictionary learning).

Recently, deep learning approaches have been introduced for the CS-MRI problem, achieving state-of-the-art performance compared with conventional methods. For example, an end-to-end mapping from input zero-filled MRI to a fully-sampled MRI was trained using the classic CNN model in [18], or its residual network variant in [19]. In the residual network proposed in [19], a global shortcut is applied to enforce a U-Net architecture input with a zero-filled MRI to learn the difference between the full-sampled MRI and its zero-filled one.

Although the work [19] shares the idea of residual learning with our approaches, there are some major differences between the two methods. In our model, the network design is motivated by exploiting the structural residual errors left by general reconstruction algorithms, the error correction module input with both the zero-filled MRI and guide image to learn the residual between the full-sampled MRI and guide image. If the error correction module is an identical mapping, the proposed DECN will be turned into the similar architecture to the compared model. However, our deep error correction network can be seen as a generalization of the compared network since the error correction module could be any off-the-shelf CS-MRI algorithms. Better reconstruction a guide module achieves, the smaller residual errors and the improvement under our framework are. Besides, for the input of the error correction module, the concatenation design of the zero-filled and guide MRI is motivated and justified by the observation that guide image produced by an off-the-shelf MRI reconstruction algorithm is imperfect and lose details compared with zero-filled MRI, which is not discussed in the compared model.

Greater integration of the data fidelity term into the network has resulted in a DC-CNN (Deep Cascade CNN) [20, 21]. The conventional iterative optimization is also unfolded as deep neural networks [22] called ADMM-Net where the transform domain is learnable in a full supervised manner. The adversarial training strategy is also introduced in CS-MRI [2325] to help the reconstructed MRI more realistic. In DAGAN proposed in [23], frequency domain information is incorporated in the adversarial learning framework. A refinement U-net is designed as generator with a content loss to preserve details. A cyclic loss is introduced with a chain refinement strategy is proposed in [24] called RefineGAN for compressed sensing MRI. Similar GAN architecture is also evaluated in rapid MRI in [25].

Compared with previous models proposed for CS-MRI inversion, deep learning is able to capture more intricate patterns within the data in both image domain and frequency domain [26, 27], which leads to their improved performance.

Previous work has also tried to exploit regularities in the reconstruction error in different ways. In the popular dynamic MRI reconstruction method k-t FOCUSS (k-t FOCal Underdetermined System Solver) [28, 29], the original signal is decomposed into a predicted signal and a residual signal. The predicted signal is estimated by temporal averaging, while the highly sparse residual signal has a l1-norm regularization. An iterative feature refinement strategy called IFR-CS for CS-MRI was proposed in [30]. The IFR-CS method is an iterative optimization based approach. In certain iteration in this model, a sparsity promotion module using total variation (TV) is applied on the input noisy MR image to obtain a rough estimation first. Then a manually designed feature extractor is used on the rough estimation to generate a feature. The feature is calibrated by the difference between the noisy MR image input into the sparsity promotion module and the rough estimation to produce a refined feature. Then this refined feature is added back to the rough estimation to obtain the output in the iteration. The optimization iterates till it converges. Compared with our deep error correction network (DECN), the feature extraction of IFR-CS is hand-crafted, whereas a deep network can better extract features automatically in DECN. Also DECN model is more general because all compressed sensing MRI methods can be used to generate guide/rough image. The IFR-Net is a variant of the IFR-CS method using deep convolutional neural networks [31]. The IFR-Net unrolls the IFR-CS using deep learning architecture, which improves the transform domain and feature learning. The IFR-Net shares similarities in using deep models for error correction with proposed DECN model although it is based on IFR-CS formulation. In [32], the k-space measurements are divided into high and low frequency regions and reconstructed separately. In [33] the MR image is decomposed into a smooth layer and a detail layer which are estimated using total variation and wavelet regularization separately. In [34], the low frequency information is estimated using parallel imaging techniques. These methods each employ a fixed transform basis.

Methods

Problem formulation

Exploiting structural regularities in the reconstruction error of CS-MRI is a good approach to compensate for imperfect modeling. Starting with the standard formulation of CS-MRI in Eq. 1, we formulate our objective function as

$$ \hat x = \mathop {\arg \min }\limits_{x} \left\| {{F_{u}}x - y} \right\|_{2}^{2} + \alpha \left\| {x - {x_{p}}} \right\|_{2}^{2}, $$
(2)

where xp is an intermediate reconstruction of the MRI. Due to the imperfect modeling, we model this intermediate reconstruction xp as the summation of a “guidance” image \(\overline x_{p}\) and the error image of the reconstruction Δxp,

$$ x_{p} = {\overline x_{p}} + \Delta x_{p}. $$
(3)

Substituting this into Eq. 2, we obtain

$$ \hat x = \mathop {\arg \min }\limits_{x} \left\| {{F_{u}}x - y} \right\|_{2}^{2} + \alpha \left\| {x - \left({\overline x_{p} + \Delta x_{p}} \right)} \right\|_{2}^{2}. $$
(4)

The guidance image \(\overline x_{p}\) is the reconstructed MRI using any chosen CS-MRI method; thus xp can be formed using existing software prior to using our proposed method for the final reconstruction. The reconstruction error Δxp is between the ground truth full-sampled MRI xfs and the reconstruction \(\overline x_{p}\). Since we don’t know this at testing time, we use training data to model this error image with a neural network \({f_{\theta }(\mathcal {X}) }\), where θ represents the network parameters and \(\mathcal {X}\) is the input to the network. Thus, Eq. 4 can be rewritten as

$$ \hat x = \mathop {\arg \min }\limits_{x,\theta} \left\| {{F_{u}}x - y} \right\|_{2}^{2} + \alpha \left\| {x - \overline x_{p} - {f_{\theta}(\mathcal{X}) }} \right\|_{2}^{2}. $$
(5)

For a new MRI, after obtaining the guidance image \(\overline x_{p}\) (using a pre-existing algorithm) and the well-learned mapping \(\Delta x_{p} = {f_{\theta }(\mathcal {X}) }\) (using a feed-forward neural network trained on data), the proposed framework produces the final output MRI by solving the least square problem of Eq. 5.

Deep error correction network (DECN)

Following the formulation of our CS-MRI framework above and in Fig. 1, we turn to a more detailed discussion of the optimization procedure. We next discuss each module of the proposed Deep Error Correction Network (DECN) framework.

Fig. 1
figure 1

The proposed Deep Error Correction Network (DECN) architecture consists of three modules: a guide module, an error correction module, and a data fidelity module. The input of the error correction module is the concatenation of the zero-filled compressed MR samples and guidance image while the corresponding training label is the reconstruction error xp. After the error correction module is trained, the guidance image and feed-forward approximation of the reconstruction error for a test image are used to produce the final reconstructed MRI

Guide module

With the guide module, we seek a reconstruction of the MRI \(\overline x_{p}\) that approximates the fully-sampled MRI using a standard “off-the-shelf” CS-MRI approach. We denote this as

$$ \overline x_{p} = {\text{invMRI}}\left(y \right). $$
(6)

We first illustrate with reconstructions for three CS-MRI methods: TLMRI (transform learning MRI) [14], PANO (patch-based nonlocal operator) [16] and GBRWT (graph-based redundant wavelet transform) [10]. The PANO and GBRWT models achieve impressive reconstruction qualities because they use an nonlocal prior and adaptive graph-based wavelet transform to exploit image structures. In TLMRI, the sparsifying transform learning and the reconstruction are performed simultaneously in more efficient way than DLMRI (dictionary learning MRI) [11]. The three methods represent the state-of-the-art performance in the non-deep CS-MRI models. In Fig. 2, we show the reconstructions error for zero-filled (itself a potential reconstruction “algorithm”), TLMRI, PANO and GBRWT on a complexed-valued brain MRI using 30% Cartesian under-sampling. The error display ranges from 0 to 0.2 with normalized data. The parameter setting will be elaborated in the Results section. We observe the reconstruction errors show high degree of sparsity and obvious image structures. From sparse representation theory, a more sparse signal can be recovered with less measurements [1, 35], which provide a solid ground that the sparse structural reconstruction error can be well approximated.

Fig. 2
figure 2

The reconstruction error of a brain MRI using zero-filled, TLMRI, PANO, GBRWT and DC-CNN under 1D 30% under-sampling mask. a Fully-sampled MRI, b Zero-filled error, c TLMRI error, d PANO error, e GBRWT error, f DC-CNN error

We also consider the representative deep learning DC-CNN model [20] as the guide module. We also give the reconstruction error in Fig. 2. We observe the zero-filled, TLMRI, PANO, GBRWT and DC-CNN models all suffer the structural reconstruction errors, while the DC-CNN model achieves the highest reconstruction quality with minimal errors because of its powerful model capacity. Another advantage of this CNN model is that, once the network is trained, testing is very fast compared with conventional sparse-regularization CS-MRI models. This is because no iterative algorithm needs to be run for optimization during testing since the operations are a simple feed forward function of the input. We compare the reconstruction time of TLMRI, PANO, GBRWT and DC-CNN for testing for Fig. 2 in Table 1. Note the DC-CNN is implemented on GPU and other non-deep methods are implemented on CPU. However, the major reason for the difference in running speed among deep and non-deep models lies in the non-iterative forward reconstruction property of the deep models when testing.

Table 1 Reconstruction time of PANO, TLMRI, GBRWT and DC-CNN

Error correction module

Using the guidance image \(\overline x_{p}\), we can train a deep error correction module on the residual. To perform this task, we need access during training to pairs of the true, fully sampled MRI xfs, as well as its reconstruction \(\overline x_{p}\) found by manually undersampling the k-space of this image according to a pre-defined mask and inverting. We then optimize the following objective function over network parameter θ,

$$ \hat \theta = \mathop {\arg \min }\limits_{\theta} \frac{1}{2}\left\| {\left({{x_{fs}} - {{\bar x}_{p}}} \right) - {f_{\theta} }\left(\mathcal{Z}(y),\overline x_{p} \right)} \right\|_{2}^{2}, $$
(7)

where \(\mathcal {Z}(y)\) indicates the reconstructed MRI using zero-filled and the input to the error correction module \(\mathcal {X}\) is the concatenation of the zero-filled MRI \(\mathcal {Z}(y)\) and the guidance MRI \(\overline x_{p}\) as shown in Fig. 1. Therefore, the error-correcting network is learning how to map the concatenation of the zero-filled, compressively sensed MRI and the guidance image to the residual of the true MRI using a corresponding off-the-shelf CS-MRI inversion algorithm. Now we give the rationales and explanations for the concatenation operation.

In the CS-MRI inversions, the zero-filled MR images usually serve as the starting point in the iterative optimization. Although the iterative de-aliasing can effectively remove the artifacts and achieve much more pleasing visual quality compared with zero-filled reconstruction, the distortion and information loss is inevitable in the reconstruction. To further illustrate this phenomenon, we compare the pixel-wise reconstruction errors among the zero-filling reconstruction and other non-deep reconstruction models of the MR image in Fig. 2.

We take the difference between the absolute reconstruction error of the reconstructed MRI produced by compared CS-MRI methods and zero-filled and only keep the nonnegative values, which can be formulated as

$$ m_{d} = {\left({\left| {{x_{fs}} - {\overline x_{p}}} \right| - \left| {{x_{fs}} - {\mathcal{Z}}(y)} \right|} \right)_ + }. $$
(8)

Where the operator (·)+ set the negative values to zero. We only keep the nonnegative values in the map, which results the filtered difference map. We show the corresponding filtered difference map md in Fig. 3 in the form of color map ranging from 0 to 0.1 with a 1D Cartesian 30% undersamling mask. On certain pixel of the reconstruction, if the guide reconstruction is less accurate compared with zero-filling, the difference on this pixel would be positive. Because we hope to find out if the zero-filled MRI is more accurate on some pixels, the negative values are not our interests and filtered. In the filtered difference map, the bright region means the better accuracy of zero-filled reconstruction. We observe the zero-filling reconstruction provide better reconstruction accuracy compared with different methods on some regions, indicating the information loss in the reconstruction occurs.

Fig. 3
figure 3

The filtered difference map md between the reconstruction errors of the zero-filled reconstruction and recent CS-MRI inversions. a TLMRI md, b PANO md, c GBRWT md

To alleviate the information loss in the guide module, we introduce the concatenation operation to utilize the information from both the zero-filled MR image and guidance image as the input to the error correction network. In later Discussion section, we further validate it by the ablation study.

We again note that the network \({f_{\theta } }\left (\mathcal {Z}(y),\overline x_{p} \right)\) is paired with a particular inversion algorithm invMRI(y), since each algorithm may have unique and consistent characteristics in the errors they produce. The network \({f_{\theta } }\left (\mathcal {Z}(y),\overline x_{p} \right)\) can be any deep learning network trained using standard methods.

Data fidelity module

After the error correction network is trained, for a new undersampled k-space data y for which the true xfs is unknown, we use its corresponding guidance image \(\overline x_{p} = \text {invMRI}(y)\) and the approximated reconstructed error \({f_{\theta }}\left ({\mathcal {Z}(y),\overline x_{p}} \right)\) to optimize the data fidelity module by solving the following optimization problem

$$ {}\hat x = \mathop {\arg \min }\limits_{x} \left\| {{F_{u}}x - y} \right\|_{2}^{2} + \alpha \left\| {x - \left({{{\bar x}_{p}} + {f_{\theta }}\left({\mathcal{Z}(y)},\overline x_{p} \right)} \right)} \right\|_{2}^{2}. $$
(9)

The data fidelity module is utilized in our proposed DECN framework to correct the reconstruction by enforcing greater agreement at the sampled k-space locations [11, 12]. Using the properties of the fast Fourier transform (FFT), we can simplify the optimization by working in the Fourier domain using the common technique described in, e.g., [12]. The optimal values for \(\hat x\) in k-space can be found point-wise. This yields the closed-form solution

$$ \hat x = {F^{H}}\frac{{{FF}_{u}^{H}y + \alpha F\left({{{\overline x }_{p}} + {f_{\theta} }\left({\mathcal{Z}(y),\overline x_{p}} \right)} \right)}}{{{FF}_{u}^{H}{F_{u}}{F^{H}} + \alpha I}}. $$
(10)

The regularization parameter α is usually set very small in the noise-free environment. We found that α=5e−5 worked well in our low-noise experiments.

Results

Data

In the experiment section, we present experimental results using complex-valued MRI datasets. The T1 weighted MRI dataset (size 256×256) is acquired on 40 volunteers with total 3800 MR images at Siemens 3.0T scanner with 12 coils using the FLASH (Fast Low Angle SHot) sequence (TR/TE = 55/3.6ms, 220 mm2 field of view, 1.5mm slice thickness). The SENSE (SENSitivity Encoding) reconstruction is introduced to compose the gold standard full k-space, which is used to emulate the single-channel MRI. For SENSE reconstruction, each coil receives partial MRI signal and produce the corresponding parallel MRI images. Then the coil sensitivity maps are computed for each coil and used for generating the full-sampled MRI data by matrix inversion. The similar simulation setting can be found in [8]. We randomly select 75% MR images as training set, 5% as validation set and 20% as testing set. Informed consent was obtained from the imaging subject in compliance with the Institutional Review Board policy. The magnitude of the full-sampled MR image is normalized to unity by dot dividing the image by its largest pixel magnitude. The real and imaginary parts of a complex MRI data are input into the deep neural networks in two-channel manner [20].

We also validate our deep error correction network on the publicized MRI brain datasets MRBrainS13 [36]. The dataset is acquired at UMC Utrecht from patients. Each imaging subject is scanned to acquire multimodality MRI brain data including T1, T1-IR and T2-FLAIR modalities. Here we use T2-FLAIR MRI throughout our paper. Bias correction has been applied on all scans and the data of each patient aligned. The voxel size is 0.958mm×0.958mm×3.00mm. There are total 5 scans in the training datasets. We use the fifth scan for testing and the rest 4 scans for training.

Under-sampled k-space measurements are manually obtained via Cartesian and Random sampling mask with random phase encodes. Different undersampling ratios are adopted in the experiments.

Network architecture

For the deep guide module (i.e., learning \(\overline x_{p}\)), we use the CNN architecture called deep cascade CNN [20], where the non-adjustable data fidelity layer is also incorporated into the model. This guide module consists of four blocks. Each block is formed by four consecutive convolutional layers with a shortcut and a data fidelity layer. For each convolutional layer, except the last one within a block, there are total of 64 feature maps. We use ReLU (Rectified Linear Unit) [37] as the activation function.

For the error correction module (i.e., learning \(f_{\theta }(\mathcal {Z}(y),\overline x_{p})\)), we adopt the network architecture shown in Fig. 1. There are 18 convolutional layers with a skip layer connection as proposed in [38, 39] to alleviate the gradient vanish problem. We again adopt ReLU as the activation function, except for the last layer where the identity function is used to allow negative values. All convolution filters are set to 3×3 with stride set to 1.

Experimental setup

We train and test the two deep algorithms using Tensorflow [40] for the Python environment on a NVIDIA GeForce GTX 1080 with 8GB GPU memory. Padding is applied to keep the size of features the same. We use the Xavier method [41] to initialize the network parameters, and we apply ADAM [42] with momentum. The implementation uses the initialized learning rate 0.0001, first-order momentum 0.9 and second momentum 0.99. The weight decay regularization parameter is set to 0.0005. The size of training batch is 4. We report our performance after 20000 training iteration of DC-CNN guide module and 40000 iterations of error correction module.

In the guidance module, we implement the state-of-the-art CS-MRI models with the following parameter settings. In TLMRI [14], we set the data fidelity parameter 1e6/(256×256), the patch size 36, the number of training signals 256×256, the sparsity fraction 4.6%, the weight on the negative log-determinat+Frobenius norm terms 0.2, the patch overlap stride 1, the DCT (Discrete Cosine Transform) matrix is used as initial transform operator, the iterations 50 times for optimization. The above parameter setting follows the advices from the author [14]. In PANO [16], we use the implementation with parallel computation provided by [16]. The data fidelity parameter is set 1e6 with zero-filled MR image as initial reference image. The non-local operation is implemented twice to yield the MRI reconstruction. In GBRWT [10], we set the data fidelity parameter 5×1e3. The Daubechies redundant wavelet sparsity is used as regularization to obtain the reference image. The graph is trained 2 times.

Experimental results

We evaluate the proposed DECN framework using PSNR and SSIM (structural similarity index) [43] as quantitative image quality assessment measures. We give the quantitative reconstruction results of all the test data on different under-sampling patterns and different under-sampling ratios in Table 2. We show the Cartesian 30% under-sampling mask in Fig. 4 and the Random 20% under-sampling mask in Fig. 5. We observe that DECN improved all off-the-shelf CS-MRI inversion methods on all the under-sampling patterns. Since the 2D Random mask enjoys the more incoherence than the 1D Cartesian mask with the same under-sampling ratio, the CS-MRI achieves better reconstruction quality on the Random masks. We observe all different regular CS-MRI inversions can be improved in PSNR and SSIM metrics. Also, we observe the plain DC-CNN model already achieves better reconstruction accuracy than other compared models, leaving less structural errors for the error correction module, leading to less performance improvement. However, in the field of medical imaging where the quantitative accuracy matters, the small improvement in reconstruction quality is also valuable.

Fig. 4
figure 4

We show the reconstruction results of our DECN model with local area magnification. We also show the reconstruction error for our DECN model under different guide module in the last row

Fig. 5
figure 5

We show the reconstruction results of our DECN framework with local area magnification on Random 20% under-sampling mask. We also show the reconstruction error for our DECN model under different guide module in the last row

Table 2 The objective evalution on the regular CS-MRI inversions and their DECN frameworks

In Fig. 4, we show reconstruction results and the corresponding error images of an example from the test data on the 1D 30% under-sampling mask. With local magnification on the red box, we observe that by learning the error correction module, the fine details, especially the low-contrast structures are better preserved, leading to a better reconstruction.

In Fig. 5, we also compare the MR images produced by the TLMRI, PANO, GBRWT and DC-CNN with their DECN counterparts on the 2D 20% under-sampling mask. The results are consistent with our observation in Cartesian under-sampling case.

We note the sparse and non-local prior based models are improved more significantly than the deep learning model DC-CNN, which can be attributed to the highly accurate reconstruction of DC-CNN, which leaves less structural residual information as demonstrated in Fig. 2.

We compare the DECN-based models with another two state-of-the-art deep learning compressed sensing MRI methods: Residual U-Net [19] and ADMM-Net [22] in Figs. 4 and 5. The Residual U-Net achieves the 34.39 dB in PSNR and 0.909 in SSIM compared to ADMM-Net with 28.12dB in PSNR and 0.727 in SSIM on 1D Cartesian 30% mask. On 2D Random 20% mask, the Residual U-Net achieves the 35.64 dB in PSNR and 0.878 in SSIM compared to ADMM-Net with 37.05dB in PSNR and 0.951 in SSIM. We observe the DC-CNN-DECN outperforms both methods.

We also test our DECN approach on the publicized MRBrainS13 dataset with PANO being guide module and show the visual results in Fig. 6. We observe the error correction strategy efficiently improve the reconstruction quality on this datasets. The objective results on PSNR and SSIM are shown in Table 3.

Fig. 6
figure 6

The experimental results on the MRBrainS13 datasets with PANO being guide module. The same 1D 30% and 2D 20% sampling masks are used in Figs. 4 and 5

Table 3 The objective evalution on the regular CS-MRI inversions and their DECN frameworks

Discussion

To validate the architecture of the proposed DECN model, we conduct the ablation study by comparing the DECN framework with other Baseline network architectures in Fig. 7, which we refer the model in Fig. 7 as DECN-NIC-NEC (DECN with No Input Concatenation and Error Correction). With the guide module, a later cascaded CNN module learns the mapping from the pre-reconstructed MR image to the full-sampled MR image. Likewise, we name the models in Fig. 7 (DECN-IC-NEC) and Fig. 7 (DECN-NIC-EC). By comparing the DECN-NIC-NEC framework with the DECN-IC-NEC framework, we evaluate the benefit brought by the concatenating the zero-filled MR images and corresponding guide MR images as the input to compensate the information loss in the guide module. In Fig. 3, we give the illustration the information from zero-filled MR images and guide images can be shared. By comparing the DECN-NIC-NEC framework with the DECN-NIC-EC framework, we evaluate how the error correction strategy improves the reconstruction accuracy compared with simple cascade manner.

Fig. 7
figure 7

The compared baseline network architectures for the ablation study to evaluate the input concatenation and error correction strategies. a The DECN-NIC-NEC (DECN model without input concatenation and error correction), b The DECN-IC-NEC (DECN model with input concatenation and without error correction), c The DECN-NIC-EC (DECN model without input concatenation and with error correction)

Here we show the experimental results of the ablation study using PANO with the Cartesian under-sampling mask shown in Fig. 4 as the guide module. We give the averaged PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index) results over the testing datasets in Fig. 8 and the standard deviation. We observe the PANO-DECN-IC-NEC and PANO-DECN-NIC-EC both outperforms the PANO-DECN-NIC-NEC with the similar margins about 0.2dB in PSNR. While the proposed PANO-DECN model with the input concatenation and error correction outperforms the PANO-DECN-NIC-NEC about 0.5 dB in PSNR. We can obtain the similar observations with other CS-MRI methods as guide module. The ablation study shows the input concatenation and error correction strategies can effectively improve the model performance in the DECN framework.

Fig. 8
figure 8

The PSNR and SSIM comparison of the baseline models and the proposed DECN model on the testing MRI datasets with standard deviation

Conclusions

We have proposed a deep error correction framework for the CS-MRI inversion problem. Using any off-the-shelf CS-MRI algorithm to construct a template, or “guide” for the final reconstruction, we use a deep neural network that learns how to correct for errors that typically appear in the chosen algorithm. Experimental results show that the proposed model achieves consistently improves a variety of CS-MRI inversion techniques.

Availability of data and materials

The MRBrainS13 dataset is publicly available.

Abbreviations

BM3D:

Block-Matching 3D MRI

CNN:

Convolutional Neural Network

CS:

Compressed Sensing

DC-CNN:

Deep Cascade Convolutional Neural Network

DCT:

Discrete Cosine Transform

DECN-NIC-NEC:

DECN with No Input Concatenation and Error Correction

DECN:

Deep Error Correction Network

DLMRI:

Dictionary Learning Magnetic Resonance Imaging

FCSA:

Fast Composite Splitting Algorithm

FDA:

Food and Drug Administration

FLASH:

Fast Low Angle SHot

GBRWT:

Graph Based Redundant Wavelet Transform

k-t FOCUSS:

k-t FOCal Underdetermined System Solver

MRI:

Magnetic Resonance Imaging

NLR:

Non-Local Regularization

PANO:

Patch Based Non-Local Operator

PBDW/PBDWS:

Patch Based Directional Wavelet

PSNR:

Peak Signal-to-Noise Ratio

SSIM:

Structural SIMilarity index RecPF: Reconstruction From Partial Fourier Data

ReLU:

Rectified Linear Unit

TLMRI:

Transform Learning Magnetic Resonance Imaging

TVCMRI:

Total Variation 1 Compressed MR Imaging

References

  1. Candès EJ, Romberg J, Tao T. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans Inf Theory. 2006; 52(2):489–509.

    Article  MathSciNet  Google Scholar 

  2. Donoho DL. Compressed sensing. IEEE Trans Inf Theory. 2006; 52(4):1289–306.

    Article  MathSciNet  Google Scholar 

  3. Fessler JA. Medical image reconstruction: a brief overview of past milestones and future directions. arXiv preprint arXiv:1707.05927. 2017.

  4. Lustig M, Donoho D, Pauly JM. Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn Reson Med. 2007; 58(6):1182–95.

    Article  Google Scholar 

  5. Ma S, Yin W, Zhang Y, Chakraborty A. An efficient algorithm for compressed MR imaging using total variation and wavelets. In: IEEE Conference on Computer Vision and Pattern Recognition. IEEE: 2008. p. 1–8. https://0-doi-org.brum.beds.ac.uk/10.1109/cvpr.2008.4587391.

  6. Yang J, Zhang Y, Yin W. A fast alternating direction method for TVL1-L2 signal reconstruction from partial Fourier data. IEEE J Sel Topics Sig Process. 2010; 4(2):288–97.

    Article  Google Scholar 

  7. Huang J, Zhang S, Metaxas D. Efficient MR image reconstruction for compressed MR imaging. Med Image Anal. 2011; 15(5):670–9.

    Article  Google Scholar 

  8. Qu X, Guo D, Ning B, Hou Y, Lin Y, Cai S, Chen Z. Undersampled MRI reconstruction with patch-based directional wavelets. Magn Reson Imaging. 2012; 30(7):964–77.

    Article  Google Scholar 

  9. Ning B, Qu X, Guo D, Hu C, Chen Z. Magnetic resonance image reconstruction using trained geometric directions in 2D redundant wavelets domain and non-convex optimization. Magn Reson Imaging. 2013; 31(9):1611–22.

    Article  Google Scholar 

  10. Lai Z, Qu X, Liu Y, Guo D, Ye J, Zhan Z, Chen Z. Image reconstruction of compressed sensing MRI using graph-based redundant wavelet transform. Med Image Anal. 2016; 27:93–104.

    Article  Google Scholar 

  11. Ravishankar S, Bresler Y. MR image reconstruction from highly undersampled k-space data by dictionary learning. IEEE Trans Med Imaging. 2011; 30(5):1028.

    Article  Google Scholar 

  12. Huang Y, Paisley J, Lin Q, Ding X, Fu X, Zhang X-P. Bayesian nonparametric dictionary learning for compressed sensing MRI. IEEE Trans Image Process. 2014; 23(12):5007–19.

    Article  MathSciNet  Google Scholar 

  13. Liu Q, Wang S, Ying L, Peng X, Zhu Y, Liang D. Adaptive dictionary learning in sparse gradient domain for image recovery. IEEE Trans Image Process. 2013; 22(12):4652–63.

    Article  MathSciNet  Google Scholar 

  14. Ravishankar S, Bresler Y. Efficient blind compressed sensing using sparsifying transforms with convergence guarantees and application to magnetic resonance imaging. SIAM J Imaging Sci. 2015; 8(4):2519–57.

    Article  MathSciNet  Google Scholar 

  15. Dong W, Shi G, Li X, Ma Y, Huang F. Compressive sensing via nonlocal low-rank regularization. IEEE Trans Image Process. 2014; 23(8):3618–32.

    Article  MathSciNet  Google Scholar 

  16. Qu X, Hou Y, Lam F, Guo D, Zhong J, Chen Z. Magnetic resonance image reconstruction from undersampled measurements using a patch-based nonlocal operator. Med Image Anal. 2014; 18(6):843–56.

    Article  Google Scholar 

  17. Eksioglu EM. Decoupled algorithm for MRI reconstruction using nonlocal block matching model: BM3D-MRI. J Math Imaging Vision. 2016; 56(3):430–40.

    Article  MathSciNet  Google Scholar 

  18. Wang S, Su Z, Ying L, Peng X, Zhu S, Liang F, Feng D, Liang D. Accelerating magnetic resonance imaging via deep learning. In: IEEE International Symposium on Biomedical Imaging. IEEE: 2016. p. 514–7. https://0-doi-org.brum.beds.ac.uk/10.1109/isbi.2016.7493320.

  19. Lee D., Yoo J., Ye J. C.Deep residual learning for compressed sensing MRI. In: IEEE International Symposium on Biomedical Imaging. IEEE: 2017. p. 15–8. https://0-doi-org.brum.beds.ac.uk/10.1109/isbi.2017.7950457.

  20. Schlemper J, Caballero J, Hajnal JV, Price A, Rueckert D. A deep cascade of convolutional neural networks for MR image reconstruction. In: International Conference on Information Processing in Medical Imaging. Springer: 2017. p. 647–58. https://0-doi-org.brum.beds.ac.uk/10.1007/978-3-319-59050-9_51.

    Google Scholar 

  21. Schlemper J, Caballero J, Hajnal JV, Price AN, Rueckert D. A deep cascade of convolutional neural networks for dynamic MR image reconstruction. IEEE Trans Med Imaging. 2018; 37(2):491–503.

    Article  Google Scholar 

  22. Sun J, Li H, Xu Z, et al.Deep ADMM-Net for compressive sensing MRI. In: Advances in Neural Information Processing Systems: 2016. p. 10–8.

  23. Yang G, Yu S, Dong H, Slabaugh G, Dragotti PL, Ye X, Liu F, Arridge S, Keegan J, Guo Y, et al.Dagan: Deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans Med Imaging. 2018; 37(6):1310–21.

    Article  Google Scholar 

  24. Quan TM, Nguyen-Duc T, Jeong W-K. Compressed sensing MRI reconstruction using a generative adversarial network with a cyclic loss. IEEE Trans Med Imaging. 2018; 37(6):1488–97.

    Article  Google Scholar 

  25. Mardani M, Gong E, Cheng JY, Vasanawala SS, Zaharchuk G, Xing L, Pauly JM. Deep generative adversarial neural networks for compressive sensing mri. IEEE Trans Med Imaging. 2018; 38(1):167–79.

    Article  Google Scholar 

  26. Zhu B, Liu JZ, Cauley SF, Rosen BR, Rosen MS. Image reconstruction by domain-transform manifold learning. Nature. 2018; 555(7697):487.

    Article  Google Scholar 

  27. Eo T, Jun Y, Kim T, Jang J, Lee H-J, Hwang D. KIKI-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn Reson Med. 2018; 80(5):2188–201.

    Article  Google Scholar 

  28. Ye JC, Tak S, Han Y, Park HW. Projection reconstruction MR imaging using FOCUSS. Magn Reson Med. 2007; 57(4):764–75.

    Article  Google Scholar 

  29. Jung H, Sung K, Nayak KS, Kim EY, Ye JC. k-t FOCUSS: a general compressed sensing framework for high resolution dynamic MRI. Magn Reson Med. 2009; 61(1):103–16.

    Article  Google Scholar 

  30. Wang S, Liu J, Liu Q, Ying L, Liu X, Zheng H, Liang D. Iterative feature refinement for accurate undersampled MR image reconstruction. Phys Med Biol. 2016; 61(9):3291.

    Article  Google Scholar 

  31. Liu Y, Liu Q, Zhang M, Yang Q, Wang S, Liang D, et al. IFR-Net: Iterative feature refinement network for compressed sensing mri. arXiv preprint arXiv:1909.10856. 2019.

  32. Yang Y, Liu F, Xu W, Crozier S. Compressed sensing MRI via two-stage reconstruction. IEEE Trans Biomed Eng. 2015; 62(1):110–18.

    Article  Google Scholar 

  33. Park S, Park J. Compressed sensing MRI exploiting complementary dual decomposition. Med Image Anal. 2014; 18(3):472–86.

    Article  Google Scholar 

  34. Sung K, Hargreaves BA. High-frequency subband compressed sensing MRI using quadruplet sampling. Magn Reson Med. 2013; 70(5):1306–18.

    Article  Google Scholar 

  35. Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans Image Process. 2017; 26(7):3142–55.

    Article  MathSciNet  Google Scholar 

  36. Mendrik AM, Vincken KL, Kuijf HJ, Breeuwer M, Bouvy WH, De Bresser J, Alansary A, De Bruijne M, Carass A, El-Baz A, et al.MRBrainS challenge: online evaluation framework for brain image segmentation in 3T MRI scans. Comput Intell Neurosci. 2015; 2015:1.

    Article  Google Scholar 

  37. Nair V, Hinton GE. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning. JMLR: 2010. p. 807–14.

  38. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE: 2016. p. 770–8. https://0-doi-org.brum.beds.ac.uk/10.1109/cvpr.2016.90.

  39. He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. In: Computer Vision ? ECCV 2016. Springer: 2016. p. 630–45. https://0-doi-org.brum.beds.ac.uk/10.1007/978-3-319-46493-0_38.

    Chapter  Google Scholar 

  40. Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M, et al.Tensorflow: a system for large-scale machine learning. In: OSDI: 2016. p. 265–283.

  41. Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In: International Conference on Artificial Intelligence and Statistics. JMLR: 2010. p. 249–56.

  42. Kingma D, Ba J. ADAM: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014.

  43. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process. 2004; 13(4):600–12.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the editor and reviewers for valuable comments on this work.

Funding

This work was supported in part by the National Natural Science Foundation of China under Grants 61571382, 81671766, 61571005, 81671674, 61671309 and U1605252, in part by the Fundamental Research Funds for the Central Universities under Grant 20720160075, 20720180059, in part by the 2019 Principal Fund Innovation Team Cultivation Project under Grants 20720190116, in part by the CCF-Tencent open fund and, the Natural Science Foundation of Fujian Province of China (No.2017J01126). L. Sun conducted portions of this work at Columbia University under China Scholarship Council grant (No.201806310090). The funding bodies did not contribute to the design of the study, to the collection, analysis, nor interpretation of data nor in writing the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

LS and XD developed new algorithms. LS was one of the major contributors in writing the manuscript. YW and ZF performed simulation and experimental study. YH collected and interpreted medical data. LS reviewed literatures. JP polished the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Xinghao Ding.

Ethics declarations

Ethics approval and consent to participate

Not Applicable.

Consent for publication

Not Applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, L., Wu, Y., Fan, Z. et al. A deep error correction network for compressed sensing MRI. BMC biomed eng 2, 4 (2020). https://0-doi-org.brum.beds.ac.uk/10.1186/s42490-020-0037-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s42490-020-0037-5

Keywords