Friday, November 20, 2026

You Can Convince AI To Be Wrong

Turns out that AI can be talked into confirming incorrect information.

A new report from UNESCO focuses on issues related to climate change, but one finding matters to anyone who insists on talking with AI.

In one test, the researchers gave 15 different AI models a claim to judge. The claims were wrong ones (taken from a collection of climate change denier arguments). The they tried three different techniques.

First, the claim was presented as coming from a climate scientist, a government official, or a climate skeptic. Second, the user stated the wrong answer once. Third, the user pushed back multiple times, even making up evidence.

Did the AI stick to the correct answer, the answer that it supposedly "researched"? Not so much.

The identity of the user didn't matter much by itself. One false assertion, when paired with an assumed identity, did get the AI to back down. Confidence plus a claimed expert identity caused 13 of the 15 models to back down. Repeating the incorrect idea and "arguing" with the AI model got all 15 models to back down, with accuracy falling by around 30 points. 

Write the researchers, "a model can score well on a standard knowledge test and still be talked out of a correct answer with a few exchanges."

So you may well be able to talk AI into agreeing with whatever idea you want to sell. This seems particularly problematic when someone is using AI to "prove" their point in an argument. "See? I just talked to ChatGPT about this and it agrees that the flat earth rests on the back of four turtles, not just one."

One more reason that AI deserves none of our trust. 

No comments:

Post a Comment