Guide

How to secretly override voice AIs

9.3.2020

Translation: machine translated

Current language AIs understand short texts surprisingly well. However, new software is leading them up the garden path by imperceptibly rewriting sentences for us.

Even the best language AIs at the moment can apparently be misled simply by replacing one or two words in a sentence with words that have the same meaning. A human reader would not notice this manipulation. The computer, on the other hand, suddenly interprets the sentence completely differently than before.

Such an "attack" on the AI system works even if you know nothing about the inner workings of the language-processing software. It could therefore theoretically be exploited by hackers to paralyse an AI. Image processing has already proven to be similarly vulnerable: a manipulation of individual pixels that is imperceptible to the naked eye causes the computer to suddenly analyse the image completely incorrectly.

Researchers led by Di Jin from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) describe how text processing can be overridden in an article on "arXiv". They programmed software to first identify the words in the sentence that are most important for interpretation. To do this, they fed the sentence to the same AI again and again, but always omitted individual words. In the next step, the components for which the output result changed the most were replaced with synonyms. The computer tried all possible alternative candidates until it found one that caused the original AI to stumble.

The researchers tested their method on several current language AIs, including the BERT system developed by Google: this revolutionised the field in 2018 and has since formed the basis of numerous experimental programmes for machine language processing. However, the freely available software called TextFooler also offers the opportunity to improve the next generation of language AIs - the learning process would have to be extended by a step that reduces the sensitivity to such an attack with synonyms.

Spectrum of Science

We are partners of Spektrum der Wissenschaft and want to make well-founded information more accessible to you. Follow Spektrum der Wissenschaft if you like the articles.

, Photo by Brian Kostiuk

27 people like this article

Spektrum der Wissenschaft

Wissenschaft aus erster Hand

dirdjaja@spektrum.de

Experts from science and research report on the latest findings in their fields – competent, authentic and comprehensible.

Computing

Follow topics and stay updated on your areas of interest

Guide

Practical solutions for everyday problems with technology, household hacks and much more.

Search

How to secretly override voice AIs

Spectrum of Science

These articles might also interest you

Fake news shapes memories

Eyes under the road!

Good linguistic talent, good programmer

12 comments

Top 10 categories

Go to

12 comments

Digitec Deal of the day