AI Uses Deep Learning to Convert 2D Images into 3D Images

2019-11-11

A research team at the University of California, Los Angeles (UCLA) has developed a technology that enhances the functionality of fluorescence microscopy. This technology allows scientists to precisely label different parts of living cells and tissues using fluorescent dyes under specific illumination conditions. By utilizing artificial intelligence, the researchers transformed 2D images into virtual stacks of 3D slices, which revealed the activities within biological organisms. In a study published on November 4, 2019, in the journal "Nature Methods," the scientists also reported that their framework, called "Deep-Z," can correct errors or aberrations in images when samples are tilted or curved. Furthermore, they demonstrated that the system can capture 2D images from a microscope and virtually create 3D images of the samples, as if they were obtained by a more advanced microscope. "This is a very powerful new method that allows 3D imaging of live specimens with minimal light exposure, minimizing the toxicity of the sample," said Aydogan Ozcan, the Chancellor's Professor at UCLA. In addition to avoiding harmful irradiation of the samples, this system provides a new 3D imaging tool for biologists and life science researchers that is simpler, faster, and cheaper than previous methods. The opportunity to correct aberrations may enable scientists studying living organisms to collect data from images that would otherwise be unusable. This research builds upon earlier techniques developed by Ozcan and his colleagues, which allowed them to render 2D fluorescence microscope images with super-resolution. Both techniques rely on deep learning to enhance the level of microscopy imaging technology and utilize data to train neural networks, which are computer systems inspired by the human brain. Deep-Z was trained using experimental images from a scanning fluorescence microscope, which can achieve 3D imaging of samples at multiple depths of focus. Through thousands of training samples, the neural network learned how to capture 2D images and infer accurate 3D slices at different depths within the samples. The framework was then blindly tested by inputting images that were not part of its training content and comparing the virtual images with actual 3D slices obtained from the scanning microscope, resulting in excellent matches. Ozcan and his colleagues applied Deep-Z to images of the nematode Caenorhabditis elegans, a common model in neuroscience due to its simple and well-understood nervous system. By converting the worm's 2D images frame by frame into 3D, the team was able to track the activity of individual neurons within the worm's body. Starting from one or two 2D images captured at different depths, Deep-Z generated virtual 3D images that allowed the team to identify individual neurons, matching the output of the scanning microscope's 3D images. The researchers also found that Deep-Z could generate 3D images from the 2D surface of samples that were tilted or curved, even when the neural network was trained only with 3D slices parallel to the sample's surface. In other experiments, Deep-Z was trained using images from two types of fluorescence microscopes: wide-field, which exposes the entire sample to light; and confocal, which scans the sample section by section using lasers. Ozcan and his team demonstrated that their framework could subsequently generate 3D images that were almost identical to those captured by confocal microscopy using 2D wide-field microscope images of the sample. This transformation is valuable because confocal microscopy produces clearer and higher-contrast images compared to wide-field microscopy. On the other hand, wide-field microscopy captures images at lower cost and with fewer technical requirements. "This is a platform that can be generally applicable to various pairs of microscopes, not just the wide-field to confocal conversion. Each microscope has its own advantages and disadvantages. With this framework, you can digitally connect different types of microscopes using AI, achieving the best of both worlds," said Yair Rivenson, an assistant professor of electrical and computer engineering at UCLA.

RECENT AI NEWS

RECENT AI TOOLS