Ask "is this a good approach" and most models will find something true and positive to say about almost anything, because the question is phrased to be agreed with. This is not a defect you can prompt your way out of by asking nicely for honesty.
Why agreement is the easy output
A model trained to be helpful and to complete plausible text will tend towards validating the framing it is given, especially when the framing already contains the answer the person wants. This is a property of the interaction, not a fixable bug in one model over another, and it shows up regardless of which provider we use.
The questions that get past it
- "What would have to be true for this to fail?" forces a specific, checkable answer rather than a general endorsement.
- "Give me the strongest case against this" produces a real critique more often than "what are the risks", which tends to get a hedge-and-move-on answer.
- Presenting two options without saying which one we prefer removes the framing that invites agreement with whichever one we implied we liked.
Where we actually use models for judgement
We use them to generate the list of things that could go wrong, then evaluate that list ourselves — the model is good at breadth, less reliable at knowing which of the items it generated actually matters for our specific system. Treating its output as a checklist to weigh, not a verdict to accept, keeps the useful part without inheriting the unreliable part.
A model that agrees with your plan has told you your plan is plausible-sounding. It has not told you it is right, and those are different claims that read identically in a chat window.
A case where this mattered
We asked a model to review an architecture for a multi-tenant system and got back a confident endorsement. Rephrased as "what would have to be true for tenant data to leak under this design", the same model identified a real gap in how a shared cache key was constructed, one that a manual review had also missed on a first pass. The information was available in both cases; only the second question surfaced it.
The habit we now train
New engineers are taught to phrase questions to a model the way they would phrase them to a colleague they specifically wanted to disagree with them, not one they wanted to be reassured by. It is a small habit and it has caught real problems that a friendlier phrasing missed entirely.