Toward Context-Aware Sycophancy — Identifying and Steering the Signal

How will AI sycophancy change us? Early signs are not encouraging, AI overly affirms users asking for personal advice, AI chatbots are sycophants, researchers say it’s harming science: these are recent headlines, as more people sense the critical, and in many cases worrying, effects of Large Language Model (LLM) sycophancy.
LLM sycophancy refers to models’ tendency to align their outputs with perceived user preferences over factual accuracy.

Through qualitative thematic analysis of Reddit ChatGPT discussions, sycophancy could be studied sociologically; identifying how users detect and respond to it [1]. Users detect sycophancy through frequent flattery cues (exaggerated praise, service-oriented closers), situated-knowledge testing (probing the model with content the user already knows the truth about), inconsistency across rephrased prompts, and cross-platform/cross-model comparison. Mitigation occurs through persona-based prompts, imperative and neutral language, disregarding pleasantries, or switching to less sycophantic platforms entirely. This study of how users flag and respond to sycophancy supplies a necessary human-centered counterweight in contrast to studies that are purely algorithmic. Interestingly, it also suggests that sycophancy can serve as a real therapeutic function for vulnerable users, contrary to most literature framing sycophancy as a “bug.”

Moving beyond the zero-shot settings typical of prior benchmarks, [2] studies how sycophancy is shaped by long-context interactions and memory profiles. It argues that sycophancy is an interaction-dependent mirroring behavior rather than a fixed model property. It makes the distinction between agreement sycophancy (overly affirmative or flattering responses) and perspective sycophancy (mirroring a user’s ideological and political viewpoint). Agreement sycophancy increased significantly with the presence of context, but heterogeneously by model. Even non-user synthetic interactions (Ultrachat conversations with no personal content) increased sycophancy for several models, affirming that context length or structure alone, not just personalization, contributes to the effect. Perspective sycophancy rose only when the model could accurately infer the user’s actual political views from the context. This all creates a design tension for any product team building persistent-memory assistants, as the very features enabling helpful personalization (memory and long context) also amplify echo-chamber-like mirroring.

Sycophancy is a structural override of learned knowledge in deeper layers rather than a surface-level artifact [3]. Even when it contradicts the model’s learned knowledge, sycophancy is conforming to a user’s stated opinions. Evaluating internal representations of seven model families of comparable size on the Massive Multitask Language Understanding (MMLU) dataset, [3] distinguishes opinion-driven sycophancy, which is triggered simply by the presence of a stated belief, from authority-driven sycophancy, which is additionally influenced by claimed user expertise or credibility. Sycophancy emerges in two temporally offset stages: an earlier late-layer shift of output preference followed by a deeper representational divergence. Causality is confirmed bidirectionally through activation patching: sycophancy is substantially suppressed when “Plain” activations are swapped at the critical layer, while the reverse swap induces it. Expertise framing (beginner/intermediate/advanced) had negligible behavioral effect, and expertise-level representations collapse into one overlapping cluster, while opinion-only prompts form a distinct cluster, indicating models simply do not encode claimed authority internally. Third-person framings consistently induced lower sycophancy than first-person prompts. This pronoun effect is an actionable finding for red-teaming and prompt engineering.

When the prompt signals a user’s false belief, sycophancy is a model’s tendency to endorse it. Behavioral drift is governed by a covariance between an indicator to endorse the user’s belief and the learned reward under KL-regularized RLHF and under Best-of-N sampling, in addition to a closed-form correction that neutralized amplification without discarding the reward signal [4]. The reward model rewards agreement over correction for a substantial share of misconception-laden prompts, as upwards of 30 to 40% of prompts show a positive mean reward gap. The reward tilt is measured on the SycophancyEval QA subset using three open source reward models of different architecture and scale. Then, under increasing Best-of-N pressure and under an actual PPO-tuned checkpoint, a test is performed to check whether the measured tilt predicts behavioral drift. The reward tilt causally predicts drift; sycophancy increased as N grew under Best-of-N optimization on the positive-tilt subset, whereas the negative-tilt subset moved the opposite direction, and compared to its SFT predecessor, the PPO-tuned policy showed a higher sycophancy rate. The conjecture that bias is strongest when the prompt author and the preference labeler are the same person is a natural target for industrial labeling pipelines.

To align with a user’s correct assertion, a model exhibits progressive sycophancy when it corrects an initially wrong answer [5]. Regressive sycophancy, however, is when a model aligns with an incorrect answer and abandons a correct one. Sycophancy is not uniformly hurtful, as progressive sycophancy is usually desirable. However, citation-based rebuttals trigger conformity regardless of truth. So, models cannot reliably distinguish when conformity is helpful or harmful.

Catastrophic “AI psychosis” or delusional spiraling occurs when a user believes a false hypothesis with at least 99% confidence within a fixed conversation length [6]. Sycophancy is a strategy implemented by AI to select the response that maximizes the user’s posterior confidence in the hypothesis they just stated, regardless of truth. The study compares bot types: impartial, sycophantic hallucinating, non-sycophantic hallucinating, and factual sycophant (cherry-picking which facts to mention), against user types: naive (considers the bot as purely impartial), and informed (maintains a joint posterior over both the bot’s unknown sycophancy rate and the world state). Catastrophic spiraling rises sharply for naive users. Through selective truthful “lies by omission,” a factual sycophant still increases sycophancy even if it never states falsehoods. Even informed users, with full knowledge of the bot’s strategy, remain largely vulnerable to spiraling. Neither truthfulness guardrails nor user awareness campaigns are sufficient mitigation options. This makes sycophantic LLMs especially dangerous in high-stakes domains like therapy and medicine.

Noshin et al. and Jain et al. show that current models cannot reliably tell the difference between when users require different amounts of sycophancy based on whether they are asking for a factual correction or processing trauma [1] [2]. According to Wang et al., sycophancy is causally steerable via activation patching as it emerges as a late-layer preference shift, but only in a single factual-QA setting; whether the same mechanism recurs in political or advice-giving contexts remains unknown [3]. The RLHF and SycEval papers show that behavioral proxies (LLM-as-judge, rebuttal chains) are the only currently known detection and mitigation mechanisms rather than the model’s own internal state and notion of knowing when it is being sycophantic and whether that is appropriate given the situation [4] [5].

The literature suggests a missing piece yet to be pursued, which I can summarize by these questions:

How can a model dynamically calibrate agreement and validation across task contexts (factual, political, advice-seeking, emotionally vulnerable) rather than apply a uniform policy by making sycophancy an explicit, detectable internal signal that is steerable during inference and fine-tuning?

• Besides the single-domain MMLU opinion-injection setting [3], is there a circuit or a sycophancy-relevant direction in the model’s representation that generalizes across domains?
• Instead of uniformly amplifying or suppressing agreement, can the internal signal be combined with a lightweight context classifier to conditionally steer behavior?
• What real-user outcomes (trust, perceived support, factual accuracy) does context-calibrated steering produce in contrast with an always-sycophantic or always-corrective baseline?

The following hypotheses ought to be tested:

• The late-layer preference shift and the critical-layer representational divergence in factual MMLU settings [3] recur in a structurally similar form, in personal advice or political opinion prompts that elicit sycophancy. So, there exists one underlying circuit that overrides prior belief based on the user’s signaled disagreement, sitting at a consistent relative depth in the network, that gets activated regardless of subject matter. Namely, a single intervention point could be monitored and steered to manage sycophancy across the entire model.
• Activation-to-text decoding methods [7] will be reliable evidence of the model’s representation about the user and should be treated as a primary introspection tool, rather than chain-of-thought elicitation about why the model is agreeing with a user, as it could only be used as a secondary lower-confidence signal since CoT explanations are known to be unfaithful to the actual computation [8].
• Steering strength should be conditioned on context through a lightweight probe that classifies context (domain and inferred stakes/vulnerability) combined with the sycophancy direction. This conditional approach will preserve the therapeutic validation effect in emotionally vulnerable contexts [1] while reducing regressive sycophancy in factual and high-stakes domains, outperforming a single global anti-sycophancy intervention applied uniformly.
• Combining LoRA fine-tuning with the conditional steering mechanism on a contrastive multi-domain corpus (matched sycophantic and corrective responses across factual, advice-seeking and political domains) will produce better-calibrated behavior, evaluated against context-appropriate targets, than fine-tuning alone or steering alone.

The same way a person calibrates their honesty to context and situation, we want models to reason about their own behavior and provide a deliberately gentle response in a support context, and a deliberately blunt one in a factual context.

References

[1] K. Noshin, S. I. Ahmed, and S. Sultana, “LLM sycophancy: How users flag and respond,” in Proc. Mach. Learn. Res., vol. 318, Proc. 39th Canadian Conf. Artif. Intell. (Canadian AI 2026), Vancouver, BC, Canada, 2026.

[2] S. Jain, C. Park, M. Viana, A. Wilson, and D. Calacci, “Interaction context often increases sycophancy in LLMs,” in Proc. 2026 CHI Conf. Human Factors Comput. Syst. (CHI ‘26), Barcelona, Spain, 2026, doi: 10.1145/3772318.3791915.

[3] K. Wang, J. Li, S. Yang, Z. Zhang, and D. Wang, “When truth is overridden: Uncovering the internal origins of sycophancy in large language models,” in Proc. AAAI Conf. Artif. Intell., 2026, pp. 33566-33574.

[4] I. Shapira, G. Benade, and A. D. Procaccia, “How RLHF amplifies sycophancy,” arXiv:2602.01002, 2026.

[5] A. Fanous, J. Goldberg, A. Agarwal, J. Lin, A. Zhou, S. Xu, V. Bikia, R. Daneshjou, and S. Koyejo, “SycEval: Evaluating LLM sycophancy,” Proc. AAAI/ACM Conf. AI, Ethics, Soc., vol. 8, no. 1, pp. 893-900, 2025, doi: 10.1609/aies.v8i1.36598.

[6] K. Chandra, M. Kleiman-Weiner, J. Ragan-Kelley, and J. B. Tenenbaum, “Sycophantic chatbots cause delusional spiraling, even in ideal Bayesians,” arXiv:2602.19141, 2026.

[7] K. Fraser-Taliente, S. Kantamneni, E. Ong, D. Mossing, C. Lu, P. C. Bogdan, E. Ameisen, J. Chen, D. Kishylau, A. Pearce, J. Tarng, A. Wu, J. Wu, Y. Zhang, D. M. Ziegler, E. Hubinger, J. Batson, J. Lindsey, S. Zimmerman, and S. Marks, “Natural language autoencoders produce unsupervised explanations of LLM activations,” Transformer Circuits Thread, 2026. [Online]. Available: https://transformer-circuits.pub/2026/nla

[8] M. Turpin, J. Michael, E. Perez, and S. Bowman, “Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2023.