A picture is analyzed by the chip, which then provides the appropriate output signal.
Graphik: Joanna Symonowicz, TU Wien
An ultra-fast image sensor with a built-in neural network has been developed at TU Wien (Vienna). It can be trained to recognize certain objects. It has now been presented in “Nature”.
Automatic image recognition is widely used today: There are computer programs that can reliably diagnose skin cancer, navigate self-driving cars, or control robots. Up to now, all this has been based on the evaluation of image data as delivered by normal cameras – and that is time-consuming. Especially when the number of images recorded per second is high, a large volume of data is generated that can hardly be handled.
Scientists at TU Wien therefore took a different approach: using a special 2D material, an image sensor was developed that can be trained to recognize certain objects. The chip represents an artificial neural network capable of learning. The data does not have to be read out and processed by a computer, but the chip itself provides information about what it is currently seeing – within nanoseconds. The work has now been presented in the scientific journal “Nature”.
Neural networks are artificial systems that are similar to our brain: Nerve cells are connected to many other nerve cells. When one cell is active, this can influence the activity of neighbouring nerve cells. Artificial learning on the computer works according to exactly the same principle: A network of neurons is simulated digitally, and the strength with which one node of this network influences the other is changed until the network shows the desired behaviour.
“Typically, the image data is first read out pixel by pixel and then processed on the computer,” says Thomas Mueller. “We, on the other hand, integrate the neural network with its artificial intelligence directly into the hardware of the image sensor. This makes object recognition many orders of magnitude faster.”
The chip was developed and manufactured at the TU Vienna. It is based on photodetectors made of tungsten diselenide – an ultra-thin material consisting of only three atomic layers. The individual photodetectors, the “pixels” of the camera system, are all connected to a small number of output elements that provide the result of object recognition.
Learning through variable sensitivity
“In our chip, we can specifically adjust the sensitivity of each individual detector element – in other words, we can control the way the signal picked up by a particular detector affects the output signal,” says Lukas Mennel, first author of the publication. “All we have to do is simply adjust a local electric field directly at the photodetector.” This adaptation is done externally, with the help of a computer program. One can, for example, use the sensor to record different letters and change the sensitivities of the individual pixels step by step until a certain letter always leads exactly to a corresponding output signal. This is how the neural network in the chip is configured – making some connections in the network stronger and others weaker.
Once this learning process is complete, the computer is no longer needed. The neural network can now work alone. If a certain letter is presented to the sensor, it generates the trained output signal within 50 nanoseconds – for example, a numerical code representing the letter that the chip has just recognized.
Object detection when things have to go fast
“Our test chip is still small at the moment, but you can easily scale up the technology depending on the task you want to solve,” says Thomas Mueller. “In principle, the chip could also be trained to distinguish apples from bananas, but we see its use more in scientific experiments or other specialized applications.”
The technology can be usefully applied wherever extremely high speed is required: “From fracture mechanics to particle detection – in many research areas, short events are investigated,” says Thomas Mueller. “Often it is not necessary to keep all the data about this event, but rather to answer a very specific question: Does a crack propagate from left to right? Which of several possible particles has just passed by? This is exactly what our technology is good for.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Ultra-fast image sensor
- Sony Unveils PXW-Z750 Flagship XDCAM Shoulder Camcorder, with 4K 2/3-type 3-chip CMOS Sensor System with Global Shutteron May 1, 2020 at 5:00 pm
Sony's premier 3-chip CMOS image sensor system is mounted on a wide ... and SBP-240F (240GB), ultra-fast transfer speeds of up to 10 Gbps* are supported allowing the camera to capture high bit ...
- OnePlus 8 Pro review: OnePlus seriously raises the bar ...and the priceon April 28, 2020 at 1:58 am
ONEPLUS unveiled its all-singing, all-dancing OnePlus 8 Pro during an online-only event on April 14, 2020. How does the new flagship smartphone - the priciest one yet from the six-year-old company - ...
- Best Buy has so many limited-time deals today – here are the 10 best oneson April 27, 2020 at 8:59 am
Best Buy knows that people would rather not visit its brick and mortar stores right now, even with a curbside pickup option now available. That’s why the consumer electronics retailer has ...
- Best cheap action camera 2020: Feature-packed cameras at bargain priceson April 16, 2020 at 10:38 am
The package also includes a carry case, an array of mounts and accessories, an ultra-fast 64GB MicroSD ... a 16 megapixel sensor if stills are your bag. Built-in electronic image stabilisation ...
- OPPO revolutionizes smartphone cameras with SmartSensor stabilizationon April 14, 2020 at 7:10 pm
In other words, it can eliminate many more photo flaws than standard lens-based image stabilization systems, and it does it ultra-fast, also reducing your cam’s power consumption to as little as ...
Go deeper with Google Headlines on:
Ultra-fast image sensor
Go deeper with Bing News on:
- Remote working and learning should be enabled with free/low-cost hardware and softwareon May 10, 2020 at 7:01 am
The current situation of workplaces running on reduced manpower on-site with many people working from home via email and voice/video-conferencing is likely to persist for the year or so that will ...
- Smarter hardware to make artificial intelligence more energy efficienton May 7, 2020 at 7:44 am
Artificial intelligence requires a lot of energy. Simply solving a puzzle can require the equivalent of the energy produced by three nuclear plants in a single hour.
- Artificial intelligence is energy-hungry -- new hardware could curb its appetiteon May 7, 2020 at 2:03 am
A team of engineers has created hardware that can learn skills using a type of AI that currently runs on software platforms. Sharing intelligence features between hardware and software would offset ...
- Distance Learning a Signpost to the Futureon May 6, 2020 at 9:37 pm
Distance Learning a Signpost to the FuturePR NewswireSAN DIEGO,SAN DIEGO, /PRNewswire/ -- Distance learning has exploded onto the American stage like a firestorm. One second it was the last thing on ...
- Smart Learning Market Growing at a CAGR 19.5% | Key Player IBM, Samsung, Adobe, Oracle, Microsofton May 6, 2020 at 1:42 pm
The Smart Learning Market size is projected to grow from USD 23.2 billion in 2019 to USD 56.5 billion by 2024, at ...