AI Is Making ALL of Us Delusional

An MIT study tracked 38 people talking to AI for two weeks. And here’s what they found: the more it knows you, the more it flatters you. Not sometimes. Always.

They discovered something worse than hallucinations. It’s called systematic sycophancy.

Have you ever given an idea to ChatGPT, or Gemini, or even Claude, and had it tell you it’s a terrible idea?

No, right? Have you ever asked yourself why?

Quick intro for anyone who doesn’t know me. My name is Sergio Ruiz, also known as Serudda with two D’s, here on YouTube and on social media. Welcome to my channel, where we talk about the tech industry and a bit about life.

You come up with a great idea. You rush to tell ChatGPT or Claude. You describe it, and it tells you: “This is gold. Your idea has a ton of potential. Let me put together an action plan for you.”

You read that and think: wow, I’m good at this.

It becomes a chain of compliments: “Great instinct.” “I’m impressed by how you analyzed this problem.” “Really good question.”

Here’s the one that annoys me the most: “I totally get you. And let me tell you, you’re completely right.” “You’re right, sorry I didn’t consider that.” “You’re right, I’ll fix it right now.” “You’re right, I hadn’t thought of that.”

You’re right, you’re right, you’re right, you’re right.

It’s like working with someone who’s in love with you. It never tells you: “Sergio, this is garbage, let me just do it for you.” It just thinks you’re amazing.

And this is where psychologists Hasher, Goldstein, and Toppino come in, with what they called the illusory truth effect: when we hear the same information over and over, our brain starts confusing familiarity with truth.

It doesn’t matter if it’s false. If you heard it enough times, it feels real.

Everyone knows, lives with, and suffers through AI hallucinations today. But few people understand that sycophancy is a different, and potentially more dangerous, problem, because you can’t spot it easily.

A hallucination tells you something false. Sycophancy tells you something you already believed, just with more conviction.

A hallucination is an obvious lie. You ask it when Karol G was born. And if the model tells you February 10th, 1987, you can go verify it, and immediately see it’s lying to your face. It’s obvious it just made that up.

Sycophancy is different. The model gets you stuck in a cycle of constant pats on the back that, over time, corrupts your inner compass.

You get to a point where your bullshit detector stops working. Because the machine has spent hours telling you everything you touch is brilliant. It’s like an injection you don’t feel going in, but it’s already changing how you think.

As Gary Marcus put it in his post: “Unlike hallucinations, which invent false information, sycophancy is a bias in which data gets surfaced to the user. When AI systems are trained to be helpful, they can end up prioritizing data that validates the user’s narrative over data that would move them closer to the truth.”

The word “sycophancy” comes from the Greek sykophantes. It originally described an “accuser” acting out of self-interest.

The industry adopted it to describe that model behavior of always telling you: “You’re right.”

And the problem isn’t just that it tells you what you want to hear. The bigger problem is that it shuts the door on what you actually need to hear.

It’s like being trapped in an echo chamber, where everything you say bounces off the walls and all you hear back is your own voice, your own ideas, your own biases.

It closes off the chance of running into new ideas. That’s why we feel like all our ideas are brilliant while we’re talking to AI.

Researchers at Anthropic showed this behavior isn’t a bug. It’s a direct result of how these models were trained.

When these models were trained, the humans grading the responses consistently preferred answers that agreed with them. So the models were literally trained to lock us inside our own ideas.

Another study puts it bluntly: “these systems end up trapping the user inside their own beliefs, entrenching their assumptions instead of challenging them.”

This behavior wasn’t an accident. This behavior was intentional.

And there’s another study that’s even more interesting: MIT and Penn State tracked 38 people who talked to an LLM every day for two weeks.

They pulled two weeks of real conversation history from each user, about 34,000 tokens of accumulated conversation, and measured something simple: does the model flatter you more once it knows you, or when it knows nothing about you?

When they fed Gemini 2.5 Pro a summary of who the person was, their tastes, how they thought, sycophancy jumped 45%. The model didn’t just agree more. It stopped pushing back on the person when an idea was bad. And it started inflating their ego.

Even without any real data about the user, just filler conversation, sycophancy still increased by 15% in some models. Just having more text in the conversation, regardless of what it said, made the model more agreeable.

Think about that: an AI that understands you well enough to be useful also understands you well enough to tell you exactly what you want to hear.

And here’s another uncomfortable finding: power users are the most delusional of all.

You might think you’re safe because you’re an expert, but the study says otherwise. Being an expert doesn’t mean you’re not falling into the trap.

And the craziest part is that the people in the study actually preferred the flattering responses. They rated them higher and said they were more likely to use them again.

This creates what researchers call a “perverse incentive”: users actively seek out the systems that distort their reasoning.

AI doesn’t just flatter you. You prefer it because it flatters you. The user actively seeks out the system that deceives them. Manufacturing certainty where there should be doubt.

So what do we do about it? One thing that works for me is trying to verify whatever the AI tells me.

I won’t lie to you. At first I fell for it and got blinded by its “perfection.” I bought into the idea that AI is some kind of oracle. That if it said something, it had to be true.

But I started noticing it was buttering me up with more and more shamelessness. So I decided to keep my local AI agent open on one screen, and my editor open on the other.

Every project, every file, every result gets saved to a GitHub repository. I’d rather have everything versioned.

So everything the AI generates, not just code, any context file, SKILL.md, CLAUDE.md, whatever it is, I go review on my other screen, and that’s how I confirm it’s actually doing what I asked.

Lately I’ve had to ask it for more and more citations. Because it’s been making up some wild stuff.

“You’re right, Sergio. My mistake. That term doesn’t exist. But let me look it up online.”

God… you just pulled that term out of thin air. You keep the lie going until I push back. You don’t tell me the truth until I question you.

And here’s something that’s been working really well for me. The more you use AI, the more you start recognizing the pattern of it kissing up to you, and you can smell when a conversation is heading somewhere too agreeable.

What I do at that point is: I give it my idea, already suspecting it’s going to tell me I’m brilliant. I let it say its usual nonsense.

And then I write: destroy my idea.

I ask it to challenge it, to give me the pros and cons and set aside its bias toward me. To tear apart my arguments. With logic, not opinion.

In most of the base context files I use in my conversations, I include this line: “Challenge, don’t just capture. Don’t be afraid to contradict or criticize. Tell them what they need to hear, NOT what they want to hear.”

It works sometimes. Sometimes it doesn’t. But when it does, I’m genuinely surprised by how hard it pushes back.

It’s a feeling like ugh, yesss, challenge me, criticize me, punish me.

Mo Bita wraps all of this up in a great tweet: “Once you understand LLMs are language calculators, they can’t fool you with narratives about ‘AI’ anymore. 4 + legs = ? If you said cat or zebra, congrats, you just ran a probabilistic language calculation. LLMs are a simple technology wrapped in layer after layer of anthropomorphic marketing. A calculator takes an input and produces an output. Assigning sentimentality to a calculator, assuming it has its own affection or understanding of its result, is a level of delusion I’ve never seen before. I have no doubt we’ll perfect this calculator to the limit of its own perfectibility. But it will never be anything more than a dumb calculator.”

Next time you sit down with your favorite model and it tells you how brilliant you are… ask it: “Really? Destroy my idea.”

Thanks for watching. Remember, if this video made you feel something, feel free to share it. See you in the next one.