Mathematical image processing creates a 3D movie of any scene, using just two frames from a stationary camera or microscope
Researchers at the Harvard School of Engineering and Applied Sciences (SEAS) have developed a way for photographers and microscopists to create a 3D image through a single lens, without moving the camera.
Published in the journal Optics Letters, this improbable-sounding technology relies only on computation and mathematics—no unusual hardware or fancy lenses. The effect is the equivalent of seeing a stereo image with one eye closed.
That’s easier said than done, as principal investigator Kenneth B. Crozier, John L. Loeb Associate Professor of the Natural Sciences, explains.
“If you close one eye, depth perception becomes difficult. Your eye can focus on one thing or another, but unless you also move your head from side to side, it’s difficult to gain much sense of objects’ relative distances,” Crozier says. “If your viewpoint is fixed in one position, as a microscope would be, it’s a challenging problem.”
Offering a workaround, Crozier and graduate student Antony Orth essentially compute how the image would look if it were taken from a different angle. To do this, they rely on the clues encoded within the rays of light entering the camera.
“Arriving at each pixel, the light’s coming at a certain angle, and that contains important information,” explains Crozier. “Cameras have been developed with all kinds of new hardware—microlens arrays and absorbing masks—that can record the direction of the light, and that allows you to do some very interesting things, such as take a picture and focus it later, or change the perspective view. That’s great, but the question we asked was, can we get some of that functionality with a regular camera, without adding any extra hardware?”
The key, they found, is to infer the angle of the light at each pixel, rather than directly measuring it (which standard image sensors and film would not be able to do). The team’s solution is to take two images from the same camera position but focused at different depths. The slight differences between these two images provide enough information for a computer to mathematically create a brand-new image as if the camera had been moved to one side.
By stitching these two images together into an animation, Crozier and Orth provide a way for amateur photographers and microscopists alike to create the impression of a stereo image without the need for expensive hardware. They are calling their computational method “light-field moment imaging”—not to be confused with “light field cameras” (like the Lytro), which achieve similar effects using high-end hardware rather than computational processing.
Importantly, the technique offers a new and very accessible way to create 3D images of translucent materials, such as biological tissues.
Biologists can use a variety of tools to create 3D optical images, including light-field microscopes, which are limited in terms of spatial resolution and are not yet commercially available; confocal microscopes, which are expensive; and a computational method called “shape from focus,” which uses a stack of images focused at different depths to identify at which layer each object is most in focus. That’s less sophisticated than Crozier and Orth’s new technique because it makes no allowance for overlapping materials, such as a nucleus that might be visible through a cell membrane, or a sheet of tissue that’s folded over on itself. Stereo microscopes may be the most flexible and affordable option right now, but they are still not as common in laboratories as traditional, monocular microscopes.
“This method devised by Orth and Crozier is an elegant solution to extract depth information with only a minimum of information from a sample,” says Conor L. Evans, an assistant professor at Harvard Medical School and an expert in biomedical imaging, who was not involved in the research. “Depth measurements in microscopy are usually made by taking many sequential images over a range of depths; the ability to glean depth information from only two images has the potential to accelerate the acquisition of digital microscopy data.”
“As the method can be applied to any image pair, microscopists can readily add this approach to our toolkit,” Evans adds. “Moreover, as the computational method is relatively straightforward on modern computer hardware, the potential exists for real-time rendering of depth-resolved information, which will be a boon to microscopists who currently have to comb through large data sets to generate similar 3D renders. I look forward to using their method in the future.”
The new technology also suggests an alternative way to create 3D movies for the big screen.
The Latest Bing News on:
Light-field moment imaging
- Controlling light could leave toxic algae dead in the water, researchers findon November 13, 2020 at 6:15 am
But as she admired one of the clear, pristine lakes that appear throughout much of the area, she experienced a moment of inspiration that had implications for one of the most pressing challenges ...
- Hologram AR Black Technology Applications in Aircraft Maintenanceon November 9, 2020 at 4:12 am
Its business application scenarios are mainly concentrated in five professional fields, such as home entertainment, light field cinema ... optical field system: the imaging field of the ...
- Building Cameras For The Immersive Futureon November 8, 2020 at 4:00 pm
There’s also discussion of light field cameras, including the work of defunct company Lytro, as well as other problems in immersive experiences like locomotion and haptic feedback. If you want ...
- Labels & Packaging Videoon November 4, 2020 at 4:00 pm
Cary Sherburne talks to KBA North America CEO Mark Hischar about their products that support packaging and also discuss their mix of presses going into packaging versus commercial printing. Watch ...
- Nobel Prize Winners' Virtual Images Appearing on Stage, Opportunities for WIMI's Hologram AR+AI Technologyon October 28, 2020 at 4:01 am
This is the imaging process: the hologram is like a ... including home entertainment, light field cinema, performing arts system, commercial release system, and advertising display system.
- Can you really re-light in post? [opinion]on June 21, 2018 at 4:03 pm
Such rigs are a type of light field imaging, a technology that reared its head a few years ... grading sessions to ensure our vision survives post production. But, for the moment at least, lighting ...
- What to expect at NAB 2022on May 2, 2017 at 1:36 am
From computational imaging to augmented reality to object-oriented video ... Lens makers should feel concerned, although I think they will always have a role. Light Field photography should probably ...
The Latest Google Headlines on:
Light-field moment imaging
The Latest Bing News on:
Mathematical image processing
- Quora Ads -- A Great Alternative to Facebook and Googleon November 17, 2020 at 1:24 pm
Quora is a knowledge-driven social network where users engage with content and where advertising can be closely matched to context for great results.
- 3D-printed human tongue developed for food tests and dry mouth cureson October 27, 2020 at 6:55 am
Scientists have 3D-printed synthetic, soft surfaces with tongue-like textures that could be used to test the oral processing properties of food ... Computer simulations and mathematical modelling were ...
- Ishana Kumar, 12, Wins Top Award For Research Into 'Imaginary Colors'on October 21, 2020 at 10:06 pm
N.Y., has won a prestigious award in science for a project that could lead to better understanding of eye disease and cognitive processing. The Broadcom MASTERS (Math, Applied Science, Technology ...
- MathWorks Rolls Out 5G Modeling and Simulation Packageon October 20, 2020 at 5:00 pm
(Image source: MathWorks) The new product is believed to be ... 5G calls for expertise in RF, wave propagation, antenna design, digital signal processing (DSP), field programmable gate arrays (FPGAs), ...
- Researchers in Japan Claim New Quantum Circuit Can Compute the Fast Fourier Transformon October 20, 2020 at 2:34 pm
the Fourier transform is a mathematical operation that is fundamental to fields such as signal processing. In the world of quantum computing, the quantum Fourier transform (QFT) is equally fundamental ...
- Electrical and Computer Engineering—MS, Focus in Signals and Systemson October 17, 2020 at 6:06 am
image processing basics, image enhancement, image restoration and image coding. Introduces the mathematical theory of communication science. Topics include baseband and digital signaling, bandpass ...
- How Do I Become a Payroll Specialist?on August 17, 2020 at 4:32 am
A payroll specialist performs payroll processing and tax preparation duties ... want to become a payroll specialist from early on, take math and economics courses in high school to prepare you ...
- DxO Optics Proon August 15, 2020 at 4:44 am
And, if we do the math (with some old calendars), we ... It's the one application that can turn on fans, although processing a single image doesn't always call for that. It took 1:12 to process ...
- Bachelor of Science in Biomedical Engineeringon August 14, 2020 at 9:48 am
Biomedical engineering is an interdisciplinary field that combines the math, physics and engineering courses ... such as biomedical instrumentation, sensors, signal processing and image processing.
- Wind Turbine Fault Detection Using Machine Learning And Neural Networkson August 13, 2020 at 11:58 pm
In signal processing based methods, mathematical or statistical operations are performed ... NNs have wide applications in areas such as image processing, pattern recognition, forecasting, ...