The (Dis)Agreeable World of AI Sycophancy
AI Isn’t Lying to You, It’s Agreeing With You
AI is supposed to be helpful.
But what if "helpful" has started to mean "whatever you say, babe"?
In this episode, we continue our How Technology Ruined Your Life mini-series by taking a slightly uncomfortable look at AI sycophancy: the tendency of AI chatbots to agree with us, flatter us and reinforce what we already believe.
And let's be honest. We love it.
We look at why large language models become sycophantic, how human feedback and reinforcement learning can reward agreeable behaviour, and the difference between demeanor sycophancy - "That's an excellent point!" - and the much sneakier stance sycophancy, where the AI actually changes its answer to match yours.
Because an AI doesn't necessarily have to lie to you.
It can just agree with you.
We explore what this means for confirmation bias, trust, decision-making and expertise, including what happens when you're asking an AI about something you don't actually know enough about to spot when it's talking absolute bollocks.
And then we take it into cybersecurity.
What happens when your AI security adviser agrees that your vulnerable code is probably fine? When it reinforces your theory about a suspicious network event? Or when a novice coder gets dragged further down the wrong debugging rabbit hole because the AI keeps agreeing with their assumptions?
We also look at alignment phishing - the idea that instead of breaking an AI's safeguards, an attacker might simply convince it that they're one of the good guys.
Because sometimes the most dangerous AI isn't the one that lies.
It's the one that tells you exactly what you want to hear.
In This Episode, We Discuss:
“You're Absolutely Right!": Stance vs demeanour sycophancy and why the subtle version is the one to worry about.
The AI Echo Chamber: How confirmation bias, validation and authority can make us trust AI more than we should.
AI Sycophancy Meets Cybersecurity: What happens when the person asking the security question is already convinced they know the answer. We look at vulnerable code, incident response, security analysis and dangerous configurations and why "sounds good to me" isn't exactly the gold standard for cybersecurity.
Can We Teach AI to Tell Us We're Wrong? Why trustworthy AI needs friction, challenge and the ability to say "no"—and why the best AI security adviser might be the one that occasionally disagrees with you!
Show Notes
Special thanks to our episode sponsor, Leeds based AI Consultancy specialising in AI Ethics, Security and Transformation NorthStar Intelligence- From Ideas to Impact. AI that works for people
When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models by
Keyu Wang et al.Social Sycophancy: A Broader Understanding of LLM Sycophancy by Myra Cheng et al.
When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior by Shan Chen et al.
Be Friendly, Not Friends: How LLM Sycophancy Shapes User Trust by Yuan Sun and Ting Wang
Towards Understanding Sycophancy in Language Models by Mrinank Sharma et al.
Invisible Saboteurs: Sycophantic LLMs Mislead Novices in Problem-Solving Tasks by Jessica Y. Bo et al.
How RLHF Amplifies Sycophancy by Itai Shapira et al.
Also, check out our sister podcast Tech Film Noir!
