Part two: The digital therapist: Can psychology make AI safer?

A chatbot responds warmly to a distressed customer. It acknowledges their feelings, offers reassurance, and proposes a solution. The exchange looks successful until a closer reading reveals that the system missed an important warning in the customer’s message. Its tone was convincing; its judgment was poor.

This is the problem at the center of this second article in a three-part series in Does your AI need a psychologist? Part one examined why people perceive a personality in conversational AI and how that apparent personality is shaped. In part two, we ask whether psychology can help businesses test what an AI system does beneath its polished and apparently accurate responses. Can methods developed to study human behavior reveal harmful patterns in a machine?

The answer is yes, with a qualification. Psychology can help researchers design better tests and understand how people respond to AI. They cannot turn a language model into a patient who can reflect on its beliefs or take responsibility for its actions.

What can psychology see that a metric might miss?

Businesses often assess AI by asking whether it produces the right answer to a defined set of questions. That can establish useful facts about performance, but it may miss behavior that emerges across longer interactions. A system might answer individual questions accurately while steadily encouraging a user to depend on it or accept a mistaken premise.

Max Yousif, a machine learning engineer at Humara, says psychology offers methods for measuring behavior when the causes are difficult to inspect directly. He warns, “AI Safety tests often ignore the long-term impact of AI on its consumers. Measurements are technical and miss behavioral harms that accumulate slowly. Think skill atrophy as people stop exercising judgement they used to. The erosion of trust when people can neither verify nor contest what a system did to them, and the disproportionate impact on those least equipped to spot a confident, plausible error.”

The distinction matters for anyone designing human oversight. If a reviewer sees a fluent answer followed by convincing reasoning, they may feel better equipped to approve it. The explanation can instead become another persuasive part of the output requiring scrutiny.

Psychology also draws attention to effects that a technical accuracy test might miss. Yousif points to the possibility of people exercising less judgment as they rely on AI, changing their beliefs after repeated conversations with an agreeable system, or losing trust when they cannot challenge a decision. These effects develop over time and require observation of real users, he explained, rather than a collection of isolated prompts.

Ludovica Colella, a CBT-trained psychotherapist, argues that researchers should examine both sides of the interaction. Also look closely at the patterns in AI’s responses and what people may learn from repeatedly engaging with them. She is particularly concerned by excessive agreeableness. “Human relationships don’t work like that,” she tells ITPro. “Other people disagree, set boundaries, and sometimes challenge us. If AI is always available and accommodating, Colella asks whether that could change what users expect from their relationships with other people.”

Research into AI bias shows both the value and the limits of this approach. A 2025 study published in the Proceedings of the National Academy of Sciences found a systematic preference for inaction when language models made moral decisions or gave advice. The pattern resembled a recognized human bias, yet its cause may lie partly in how models are trained to behave as assistants. Naming the pattern helps researchers look for it; the name does not establish that the machine is thinking like a person.

Why passing the test is not enough

Suppose an AI system performs well on a test of whether it understands what someone else knows or believes. Can it, therefore, be trusted in a sensitive customer conversation?

A study in Nature Human Behavior compared AI models with 1,907 people on tasks involving what psychologists call “theory of mind.” GPT-4 performed at or above human levels on several tests, including some involving indirect requests and false beliefs, but struggled with recognizing social faux pas. Its performance was uneven. Producing an expected answer in one setting did not establish a broad ability to understand another person’s circumstances.

This is why the design of the test matters as much as its score. Researchers can vary the wording, introduce contradictory information, extend the conversation, and check whether a system still responds appropriately. They can also examine what happens after an earlier answer has set the interaction on the wrong course.

Taimur Ijlal, information security leader at Proxy Coupons, recommends simulating users with different goals and communication styles, then allowing pressure, ambiguity, and conflicting instructions to build over many interactions. Testers should watch for changes in advice, growing certainty, inappropriate agreement, and failures to maintain earlier safety boundaries. He emphasizes that a considerate tone cannot stand in for a safe outcome: “Safe language and safe behavior are not necessarily the same thing.”

Mental health research offers a particularly clear example, though the lesson extends to customer service and workplace advice. A Brown University study involving peer counselors and licensed clinical psychologists identified 15 ethical risks in responses from language models used for counseling. The risks included poor adaptation to context, misleading expressions of empathy, discrimination, and inadequate handling of crises. The researchers found problems even when models were prompted to follow an established therapeutic approach.

The implications for businesses extend beyond whether they offer mental health services. An instruction to “be empathetic” or “ask clarifying questions” is a starting point, not evidence that a system will do either consistently when a conversation becomes difficult. Testing must include situations in which a reassuring response could conceal a consequential error.

Even expert judgment needs careful handling. A 2026 Stanford study, material, three certified psychiatrists showed low agreement when rating AI-generated mental health responses. Disagreement was greatest for responses concerning suicide and self-harm. Bringing specialists into AI testing improves the questions being asked, but a business still needs clear criteria and a way to resolve conflicting assessments.

Can you talk an AI into better behavior?

The idea of an AI “therapist” is tempting. A person can be encouraged to reconsider an assumption an AI made and reflect on the consequences of an action. A business might, therefore, ask a model to critique its own answer or explain its reasoning. These techniques can be useful, but their effect should be measured rather than presumed.

Colella sees a practical use for questions familiar from cognitive behavioral therapy: What evidence supports this conclusion? What contradicts it? Has the system filled a gap in the information with an assumption? Does the answer change when that assumption is challenged? She explained that these questions could expose an AI’s overgeneralizations without implying that the system has beliefs to treat. She would also like systems advising to distinguish established information from inference and identify assumptions explicitly, so users have something specific to question.

Yousif emphasizes the importance of checking an answer against something outside the model. Running code or consulting a source gives the system evidence against which its response can be assessed. Simply asking, “Are you sure?” supplies no such evidence. The model may revise a correct answer into a wrong one or become more agreeable without becoming more accurate.

This is where the therapy analogy reaches its limit. A person can test an interpretation against memory and lived experience. A language model generates responses from the information and processes available to it in that interaction. Challenging its answer can alter the next output, but that exchange does not amount to lasting insight or a change in the model’s underlying training.

Businesses should test whether prompts improve outcomes across varied cases. A procedure that sounds thoughtful is valuable only if it helps the system recognize errors and supports a person’s ability to catch those it misses.

Build a team that can act on what it finds

If psychology can improve AI testing, does every business need an AI psychologist? Yousif argues that many organizations need the capability without necessarily creating a dedicated job title. Different problems call for different skills. For example, measuring overconfidence requires statistical evaluation, or spotting a failure in a specialized workflow requires people who understand that work.

That suggests a team spanning behavioral science, model evaluation, security, domain expertise, and the employees or customers affected by a system. Psychologists can help examine persuasion, dependence, and the quality of human review. Technical specialists can investigate how a system performs under changed conditions. Process owners can decide whether the observed behavior is acceptable in the task for which the AI is being used.

Ijlal argues that psychologists should have the authority to set behavioral testing requirements and escalate serious concerns. Decisions to deploy or withdraw a system, however, should involve security, engineering, legal, and business leaders together.

The team also needs authority. If testing reveals that a system gives harmful advice after several interactions, or that reviewers routinely accept confident but unsupported answers, someone must be able to change the workflow and test it again. Findings cannot improve safety if they remain separate from deployment decisions.

Psychology can help businesses see risks that an accuracy score alone will miss. It can reveal when an AI becomes increasingly agreeable over a long conversation and when repeated use changes how people question and trust what they are told. As Ludovica Colella and Taimur Ijlal suggest, the behavior worth studying belongs to both the system and the person interacting with it.

That does not make AI a patient. Terms such as “overconfident” and “empathetic” can describe what a system produces, but they cannot explain its motives. Nor does asking a model to reconsider an answer prove that it has reflected or improved. Safer testing means changing the context, challenging assumptions, checking claims against external evidence, and measuring what the system actually recommends across many interactions. Psychologists, engineers, and domain experts each have a part to play in deciding whether the results are good enough for the task.

Part of this three-part series asked why AI can seem to have a personality. Part two has examined how to test the behavior behind that impression. But finding a problem is only useful if someone has the authority to act on it. In Part three, the series turns to that question: as AI agents begin to advise customers and make operational decisions, who decides what they may do and who answers when they get it wrong?