Advanced Photonics

Main > JournalMain > Publication Highlights

End-to-end deep learning framework for digital holographic reconstruction

  • Publication Highlights
  • Mar. 26, 2019

Digital holography is a widely-used imaging technique that can record the entire wavefront information, including amplitude and phase, of a 3D object. With an interferometer and an image sensor, a 2D hologram can be acquired and stored in a computer. Due to the noninvasive and label-free properties, it has been applied to biological imaging, air/water quality monitoring, and quantitative surface characterization measurement.

After capturing a digital hologram, appropriate algorithms are utilised to reconstruct the object numerically. Although effective, conventional approaches require prior knowledge and cumbersome operations for an in-focus and successful reconstruction. In addition, for quantitative phase imaging, the inevitable phase aberration has to be compensated by physical or numerical means, and subsequently an unwrapping step is needed to recover the true object profile. The former either requires additional hardware or strong assumptions, whereas the phase unwrapping algorithms are often sensitive to noise and distortion. Furthermore, for a 3D object, an all-in-focus image and a depth map are particularly desired for many applications, but current approaches tend to be computationally demanding.

In recent years, as the development of deep learning, it has been shown to be useful to holography. Nguyen et al. proposed a simplified U-net for phase aberration compensation in digital holographic microscopy (Opt. Express 25, 13, 2017). Deep learning works as an intermediate tool to preprocess the unwrapped aberrated phase images. Rivenson et al. demonstrated a deep neural network that is trained for twin-image and self-interference artifacts elimination in lens-free in-line DH (Light Sci. Appl. 7, 17141, 2018). The in-focus back-propagation of the hologram is fed into the network for training. Wu et al. demonstrated the use of deep learning for performing autofocusing and phase-recovery to extend the depth-of-focus in an on-chip holographic microscope (Optica 5, 6, 2018). An autofocusing method as well as a conventional reconstruction method are employed and the back-propagated hologram is fed into the network as input .

In this paper, we propose an “all-in-one” method that can tackle these holographic reconstruction problems by simply training a neural network with appropriate data. By constructing and training an end-to-end learning framework, which is motivated by the residual learning scheme, cumbersome operations in conventional reconstruction approaches are thus avoided and system parameters become unnecessary. Qualitative visualization and quantitative measurements demonstrate the superior performance of learning-based method over conventional ones. The intensive computational demand is also significantly alleviated. Through this data-driven approach, we show that it is possible to reconstruct a noise-free image that does not require any prior knowledge and can handle phase imaging as well as depth map generation. We believe that this method is universal to various digital holographic configurations and is potentially applicable to biological and industrial applications.

Figure 1. (a) Schematic of the deep learning workflow and the structure of HRNet. It consists of three functional blocks: input, feature extraction and reconstruction. In the first block, the input is a hologram of either an amplitude object (top), a phase object (middle) or a two-sectional object (bottom). The third block is the reconstructed output image according to the specific input. The second block shows the structure of HRNet. (b) and (c) elaborate the detailed structures of the residual unit and the sub-pixel convolutional layer, respectively.

Read More

Ren Zhenbo, Xu Zhimin, Lam Edmund Y.. End-to-end deep learning framework for digital holographic reconstruction[J]. Advanced Photonics, 2019, 1(1):016004