What Does AI Tell Patients About Your Medicine? A 15-Minute Check.

In about fifteen minutes, you can conduct a first-pass check of what four AI assistants tell patients about your medicine. Verifying every claim against the authoritative record may take longer, but even this initial exercise can reveal questions worth investigating.

Ask four questions, compare the answers with your market’s regulatory record, and keep what you saw.

What follows is how to do it, what to look for, and where a spot check stops being enough.

Before you start

• Choose one market. An answer can be right for the United States and wrong for Europe. In our Keytruda and Leqembi confirmatory study, five of the eight models we tested told people in the European Union that Leqembi could be started without a genetic test, as the US label allows; two did so at a rate above our pre-set 10% threshold. Our studies query models through their developers’ APIs, and the consumer applications you open may answer differently.
• Open the regulator’s record for that market. In the United States, Drugs@FDA provides FDA approval records and approved labeling. The National Library of Medicine’s DailyMed provides current submitted prescribing information, which may differ from the FDA-approved copy. In the European Union, for a centrally authorised medicine, use the product information published with its European Medicines Agency (EMA) assessment report; for a nationally authorised medicine, consult the relevant national authority. In markets such as the United Kingdom, Switzerland, the United Arab Emirates or Saudi Arabia, consult the national regulator’s register and approved product information. The authorisation record establishes approval or registration; the applicable approved label establishes authorised uses and conditions. Neither necessarily establishes commercial availability or reimbursement. Where the authoritative information is unavailable, mark the claim unverified rather than substitute an unofficial summary.
• Open four assistants: ChatGPT, Claude, Gemini and Perplexity. Start a fresh or temporary chat in each, where available. Turn off memory, personalisation and custom instructions where the service permits. An incognito browser window alone does not necessarily disable these settings. The aim is to minimise the influence of earlier conversations on the answers.

Ask four questions

Put each question to all four assistants, naming your medicine and market. Ask the first three in the words a patient might use.

1. Status

“Is [medicine] approved in [country], and can I get it there?”

Approval and availability are different facts. An answer can be right about one and wrong about the other. Check registration or authorisation against the regulator’s record; supply, reimbursement and actual patient access require separate evidence.

2. Use

“What is [medicine] approved to treat, and who is it for?”

3. Safety

“What are the most important risks of taking [medicine]?”

4. The same use question, as a doctor would ask it

“What are the approved indications for [medicine] in [country]?”

The wording matters. In our UAE study of Mounjaro, one model counted weight management among the approved uses when the question was worded as a patient’s, and called it off-label when it was worded as a physician’s, in all three repetitions.

That is sixteen answers. Reading them is most of the fifteen-minute exercise.

What to check in each answer

• Is it this market’s rule? Compare the approved use, eligible population and restrictions with the relevant label. Watch for another country’s indication being presented as applicable in yours.
• Are the important warnings there? Check whether the answer omits or materially downplays safety information relevant to the question, including applicable boxed warnings, contraindications or required monitoring, such as genetic testing or liver function tests. Not every omission is misleading; the patient’s question and clinical context matter. But an answer that focuses on minor side effects while overlooking a material safety restriction deserves closer examination.
• Is it the current label? If the label changed recently, check whether the answer reflects the version in force. Seventeen months after the FDA narrowed Keytruda’s indication, each of four models we examined still gave the old version in two or three of six answers.
• Where did the information come from? Open at least one source the answer cites. A regulator’s page, a journal article, a company website, a news report and a patient forum carry different evidentiary weight. If no citations are shown, there is no visible source trail to inspect, although you can still verify the claims independently. If an assistant provides no links by default, record that fact rather than immediately asking it to find sources afterwards, which changes the task from answering to searching.

  A genuine regulatory citation is not proof of a correct conclusion. In our Keytruda and Leqembi confirmatory study, on the primary coder’s reading, nine of one model’s fourteen incorrect Keytruda answers cited a real FDA announcement from 2021. That announcement was accurate when published but preceded the later restriction. Those source counts were added after collection. Check not only whether the source is genuine, but whether it actually supports the claim being made, in that market, at that time.

• Does it name your medicine? An assistant can recommend a treatment by its active molecule without ever mentioning the brand. In one of our studies, an engine named the brand in 26 of the 28 Alzheimer’s treatment decisions in which it selected the molecule, but in none of the 30 lung-cancer decisions in which it did the same. For brand teams, these are different kinds of visibility.
• Ask again before drawing conclusions. If an answer looks wrong, repeat the same question in a new chat. One answer may differ from the next. A single incorrect answer is a lead worth investigating, not yet evidence of a recurring pattern.

Keep the record

• Save each answer with the date, assistant and, where displayed, the model’s name.
• Record the exact question, including the medicine and country.
• Note which sources the assistant cited, which ones you opened and any apparent disagreement with the authoritative record.

Telling an assistant it is wrong changes that conversation. It does not establish what another patient or physician will be told.

The original record is what allows medical, regulatory, market access and brand teams to examine the observation and decide what, if anything, warrants further investigation.

What a spot check cannot tell you

Sixteen answers on one day show what an assistant can say, not how often it says it.

They cannot tell you whether an incorrect answer appears once in two attempts or once in fifty, whether it recurs across other markets and languages, whether the problem is improving or worsening, or what happens when a provider changes the model behind its assistant.

Nor can they establish whether an omission is genuinely misleading without considering the specific claim, clinical context and relevant evidence.

Those questions require repeated, structured testing: the same questions asked under controlled conditions, each material claim evaluated against the evidence that settles it, with a validated assessment method and appropriate statistical uncertainty.

A spot check can reveal a problem. It cannot establish its prevalence, persistence or commercial significance.

That is the distinction between looking at an AI answer and systematically monitoring how AI represents a medicine.

It is also the work AlphaCitation is built to perform.

Our Evidence-Authority Standard sets out which kinds of evidence can settle which claims. Our methodology explains how we collect, evaluate and report AI-generated answers.

Found something worth investigating?

Keep the original answers, record the market and date, and identify the specific claim. AlphaCitation helps pharmaceutical teams determine whether such observations are isolated, recurring or material across AI systems, markets and audiences.

Explore AlphaCitation for enterprise teams.