AI generates images from (almost) no data
More and more elements of a conventional camera are proving to be dispensable. Thanks to artificial intelligence, even more can be omitted than the optics seem to allow.
Scientists have been competing for some time to remove more and more elements from the classic structure of a camera and still produce correct images. For example, two-dimensional images can be captured with just a single sensor pixel, provided certain tricks are used. It is also theoretically possible to take photos without a lens.
A team led by Alex Turpin from the University of Glasgow is now taking this idea to the extreme: they are creating images with a sensor that can only count how many photons it receives and when. To capture an image, Turpin and his colleagues first generate a laser flash. Its photons spread out in space, are reflected by objects in the room and return to the camera, where they are focussed on the sensor by a lens.
Assume there is only one person in the room. Immediately after the laser flash, the sensor initially registers nothing, then the "echo" reflected by the person arrives at the sensor, which manifests itself in a striking peak in the number of registered photons. And finally, the photons reflected by the background arrive.
In order to reconstruct an image from this sequence of peaks, Turpin and his colleagues used a comparatively simple neural network. They also used a conventional 3D camera to record how the scene was actually structured. They then had the network's learning algorithm reconcile the two.
Surveillance camera use case with data protection
Although the objects in the room are clearly recognisable, the results are nowhere near as high-resolution as the original material filmed with the 3D camera. This weakness can be turned into a strength: Turpin and colleagues suggest using such a system to strengthen data protection. Unlike any real camera, it would not be able to provide an image from the outset in which, for example, individual people could be recognised. It could therefore take the place of conventional surveillance cameras. The extremely high speed at which images can be recorded could also potentially be utilised technically.
Another disadvantage of the system is that it only works in a known environment - the one in which the training was carried out. At the same time, however, once a network has been trained, it proves to be robust enough to deliver images even if the entire hardware is replaced and a radio wave transmitter and receiver, i.e. radar, is used instead of laser light and a photodiode. A skilful hobbyist could use his WLAN adapter to monitor a room, Turpin told the science magazine "Science".
The background of the scene proved to be a crucial element, the researchers write in the scientific journal "Optica". Without a background, the artificial neural network also failed, as it is simply impossible to distinguish between a scene and its exact mirror image using only the photon travel times. However, a background in which left, right, top and bottom are differentiated makes it possible to resolve these ambiguities - an object placed on the right obscures different parts of the background than one placed on the left.
Spectrum of science
We are partners of Spektrum der Wissenschaft and want to make well-founded information more accessible to you. Follow Spektrum der Wissenschaft if you like the articles.
[[small:]]
(CUT-OUT)
Experts from science and research report on the latest findings in their fields – competent, authentic and comprehensible.