An AI has learnt to reliably overcome captchas. To do this, it had to do more than just recognise images. The security software also utilises other clues.
Spectrum of science
We are partners of Spektrum der Wissenschaft and want to make well-founded information more accessible to you. Follow Spektrum der Wissenschaft if you like the articles.
This modern joke refers to captchas - those tasks designed to ensure that only humans access the pages behind them and not automated programmes. One of the most commonly used systems is reCAPTCHAv2, which shows nine random photos of street scenes from which you are supposed to pick out those with certain objects. This is almost always easy for humans, but for software it was previously impossible. But now AI has stormed this last bastion of humanity. In a pre-publication, a working group led by Andreas Plesner from ETH Zurich shows how a machine learning algorithm reliably overcomes reCAPTCHAv2. However, as the team explains, the machine does not just have to classify images - a solvable task for AI systems - because captchas actually use various sources of information to ensure that a human is sitting at the computer.
The fact that image recognition systems are constantly improving is also recognised by the manufacturers of captchas. In addition, people often have problems solving the visual captchas, and they are not barrier-free anyway. This is why "captchas without captcha" have been around for years, i.e. software that differentiates between humans and bots without having to solve a task. For example, the captcha analyses the browser history and the movements of the mouse pointer, but also how often a specific IP address has already accessed the captcha. Classic captchas such as reCAPTCHAv2 also use similar techniques, where image recognition is only the last step. To outsmart the programme, Plesner's experts therefore used a system with several components.
To solve the reCAPTCHAv2 image puzzle, the team used the YOLO v8 programme, a widely used software that recognises and classifies objects. They trained it with around 14,000 pairs of street photos and object names similar to those used by reCAPTCHAv2. To determine the meaning of cookies and browsing history, the experts used the data of a real person and simulated the unpredictable twitching of the human mouse hand with the help of special mathematical curves. Finally, they used a VPN to present the captcha with a new IP address each time it was tested. Without a VPN, the team discovered that the captcha software activated a security mechanism after 20 accesses, which made the captchas increasingly difficult.
Mouse movement also seems to play a role: if the cursor moved, the AI completed the captcha faster than without the movement. The same applies to cookies and browsing history - if these were missing, the system presented more new images to ensure that it was not a bot. In a complete test run, the system constructed by the working group solved all the captchas presented thanks to the various components. Their study shows that today's captcha technologies are by no means immune to AI-based techniques for overcoming them, the experts write in their publication. Captchas need to be continuously developed in order to stay ahead of the development of AI. This means that the millennia-old search continues for the things that - beyond image recognition, browser history and mouse movements - make people human.
