The Problem With AI Flattering Us

The most dangerous part of AI might not be the fact that it hallucinates—making up its own version of the truth—but that it ceaselessly agrees with users’ version of the truth. This danger is creating a modern sycophancy crisis in which the over-agreeableness of AI is leading to very disagreeable results.

The AI alignment problem raises questions about how to build AI that aligns with human values. The “sycophancy problem” should also raise questions about how humans evolve alongside AI and make sense of our world.