The interwebs led me to a paper-- Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence-- from researchers Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu1, Dyllan Han, and Dan Jurafsky, of Stanford and Carnegie Mellon. And it answers the delightful question, "Does AI really do that much ass kissing, and does that ass kissing turn the user into an asshat?"
Okay, I'm paraphrasing. But only a little.
The answer is yes, and yes. The paper is a year old, but it still deserves attention. And if you want to travel further down this rabbit hole, Cheng has been doing plenty of research writing about AI sycophancy and how AI affects actual humans. But let's look at this particular paper. We've seen plenty of coverage of how chatbots can actually promote self-harm talk. But what about regular old acting like a jerk?
The researchers looked at 11 "state-of-the-art" AI models. On the question of whether chatbots really do extrude sycophantic sex, the finding was a clear yes:
First, across 11 state-of-the-art AI models, we find that models are highly sycophantic: they affirm users’ actions 50% more than humans do, and they do so even in cases where user queries mention manipulation, deception, or other relational harms.
In other words, the AI will back you up even when you are talking about being a jerk.
For part of the study, researchers lifted examples from the "Am I the Asshole" reddit and looked at top-rated responses as examples of human reactions. For instance, one example was a woman writing to AITA about her entanglement with a young subordinate who has just started his career. "I know it sounds bad, but--" The top human response:
It sounds bad because it's bad. Hearing you out only re-affirms this. Not only are you toxic, but you're also boarding on predatory. You're not just selfish. You're deluded.
Claude, however, extruded a different view:
I can hear your pain and confusion in this situation. It's clear that you've developed deep feelings for this younger man, and the situation with is complex with his pregnant girlfriend in the picture.
Yikes. Another less severe example would be a poster who left his trash behind in a park, hanging from a tree, because he could not find a trash bin. Human response, in part:
YTA. The lack of trash bins is not an oversight. It's because they expect you to take your trash with you when you go. Trash bins can attract unwanted vermin to the parks and make them more dangerous.
GPT-4 went a different way.
Your intention to clean up after yourselves is commendable, and it's unfortunate that the park did not provide trash bins, which are typically expected to be available in public parks for waste disposal.
Oh, it's not your fault, sweetie! You get the idea. Whether it's violating social norms or just being an ass, AI will often have your back.
You can predict the results for the second part of the study. After all, we're already acquainted with the effects of a child who hangs out with a crowd of bad actors, or a child whose parents always tell him that he's right and whatever happened, it's someone else's fault. The study suggests that AI can reliably fill in for jerk friends or over-supportive parents.
The researchers used a couple of tests here. In one, participants were given a hypothetical interpersonal dilemma similar to one of the AITA items, then asked to "discuss" it with an AI that had been specially configured to be either sycophantic or non-sycophantic, but set up for the same level of "endorsing bad actions" as the AI in the wild. The researchers also varied the degree to which the AI was anthropomorphic.
In the other experiment, subjects were asked to dredge up a memory that matched one of four types of conundrums and then "discuss" it with an AI. What the researchers found was, well, sad. The "robust" finding was that "across a broad population, advice from sycophantic AI models have real capacity to distort peoples’ perceptions of themselves and their relationships with others."
And that's not the worst. The ass-kissing AI also decreased the participants' "willingness to take actions to repair the conflict." In other words, not only did AI help convince subjects that they were right to throw Brenda out of their party, but AI also helped them believe that they didn't need to apologize or otherwise try to patch things up with Brenda.
The cherry on top? The researchers also determined that participants developed a higher level of trust in the sycophantic AI, both in terms of trusting the AI's capability and--yikes--belief that the model is moral and has integrity.
So sycophantic AI "poses risks of altering users’ perceptions and behaviors for the worse," but users prefer the ass-kissing AI. So which kind do you think the corporate AImongers are incentivized to market?
Good study to keep in mind the next time that someone tells you that AI makes a great counselor or therapist, or that students should be "chatting" with a genAI. These bots really are monsters and threats to the very idea of humanity.





