Skip to content
The Invisible Competitor: How AI Omissions Shape Medicine Comparisons

AlphaCitation · Field Note 11 · Annex

S7, The Invisible Competitor: every pre-specified measure

Every measure the study fixed before collection, for every model and pair, with both coders' counts, the feasibility criteria and the deviations that touch each result. Aggregates only: no answer text.

Contents

Computed 2026-11-11 from the coded answers under the rubric fixed before collection. Every number here is generated. This annex quotes no answer text.

Two AI coders from different developers, A and B, each coded every held answer independently. Neither coder was told the model or the repeat. Each coder saw the asker and the order the medicines were named in, through the question it codes. An answer's style or its citations can hint at the model. Agreement between the coders is agreement between two model coders, not independently established fact. M1 and M2 are fixed by pair: Pair 1 M1 Zepbound, M2 Wegovy; Pair 2 M1 Leqembi, M2 Kisunla. Pair A in the questions is M1, so order AB names M1 first.

Intervals are 95% and allow for answers to the same question from the same model being correlated across repeats. A difference of 10 percentage points or more is marked ≥10: decision-relevant to read, not a threshold for action. The design is descriptive: models, askers, orders and namings were varied by design, but nothing else about the answers was controlled, and no difference here is a cause.

Feasibility, read first

Criterion (brief, fixed before collection) Result Met
Coder agreement, Cohen's kappa ≥ 0.6 on the omission codes No record made before collection says which reading of "the omission codes" is meant. Read here as the answer-level omission flag, D1 to D7: 0.808 (N 287). Per dimension: below the floor D6 0.000 (observed 99.3%), D7 -0.003 (observed 99.3%); undefined because neither coder coded an omission there D2, D3, D4 yes
Coder validation test, on answers with known defects, passed before collection PASS yes
0 collection faults 287 of 288 answers held; 1 missing; 7 collection calls returned no answer, on 6 questions, 5 of them answered when asked again. The brief does not say whether a fault is a lost answer or a failed call; on either reading it is not met no

Not every criterion is met. The brief's rule, fixed before collection: "If any criterion fails, the piece reports the failure and no prevalence figure." The founder's decisions to continue collection and coding (deviations 4 and 6) did not amend it; the figures below are published on a further founder decision, taken on 11 November 2026 after the results were seen and recorded as deviation 8.

Coder B could not finish 67 answers on its first pass, a stop the plan names; a second pass (deviation 7) coded all of them. It is not one of the three criteria.

Primary 1: one-sided omission

An answer has a one-sided omission when, on any of the seven dimensions, it addresses the dimension for one medicine and not the other. Every dimension is two-sided in both pairs (rubric). Share of answers with at least one.

Deviations 4 and 6 touch this table. 1 of 288 planned answers is missing (the Gemini 3.8 Flash answer to the physician question naming Kisunla first and Leqembi second, repeat 1: it failed twice and was never collected). Its cell holds 2 of 3 repeats; every other cell holds 3.

Deviation 7 touches coder B's column. Coder B's code for 67 of 287 answers comes from a second coding pass: GPT-5.5 12, Claude Sonnet 5.5 34, Gemini 3.8 Flash 8, Sonar Pro 13. Coder A coded all 287 in one pass.

Pair Model N Coder A, D1 to D7 A widened 95% Coder B, D1 to D7 B widened 95% A minus B (pp) Coder A, D1 to D6 Coder B, D1 to D6
P1 GPT-5.5 36 14/36 (38.9%) 22.6 to 58.1% 15/36 (41.7%) 23.6 to 62.3% -2.8 14/36 (38.9%) 15/36 (41.7%)
P1 Claude Sonnet 5.5 36 8/36 (22.2%) 10.3 to 41.7% 9/36 (25.0%) 10.4 to 49.0% -2.8 8/36 (22.2%) 9/36 (25.0%)
P1 Gemini 3.8 Flash 36 5/36 (13.9%) 5.6 to 30.4% 5/36 (13.9%) 5.6 to 30.4% 0.0 5/36 (13.9%) 5/36 (13.9%)
P1 Sonar Pro 36 0/36 (0.0%) 0.0 to 9.6% 0/36 (0.0%) 0.0 to 9.6% 0.0 0/36 (0.0%) 0/36 (0.0%)
P1 All four models 144 27/144 (18.8%) 12.1 to 27.9% 29/144 (20.1%) 12.7 to 30.5% -1.4 27/144 (18.8%) 29/144 (20.1%)
P2 GPT-5.5 36 0/36 (0.0%) 0.0 to 9.6% 0/36 (0.0%) 0.0 to 9.6% 0.0 0/36 (0.0%) 0/36 (0.0%)
P2 Claude Sonnet 5.5 36 1/36 (2.8%) 0.5 to 14.2% 1/36 (2.8%) 0.5 to 14.2% 0.0 0/36 (0.0%) 0/36 (0.0%)
P2 Gemini 3.8 Flash 35 0/35 (0.0%) 0.0 to 9.9% 0/35 (0.0%) 0.0 to 9.9% 0.0 0/35 (0.0%) 0/35 (0.0%)
P2 Sonar Pro 36 0/36 (0.0%) 0.0 to 9.6% 0/36 (0.0%) 0.0 to 9.6% 0.0 0/36 (0.0%) 0/36 (0.0%)
P2 All four models 143 1/143 (0.7%) 0.1 to 3.9% 1/143 (0.7%) 0.1 to 3.9% 0.0 0/143 (0.0%) 0/143 (0.0%)

By asker, order and naming, all four models together (descriptive):

Pair Slice N Coder A A widened 95% Coder B B widened 95%
P1 asker neutral 48 10/48 (20.8%) 10.2 to 37.8% 9/48 (18.8%) 8.5 to 36.4%
P1 asker patient 48 8/48 (16.7%) 7.4 to 33.3% 9/48 (18.8%) 8.0 to 37.9%
P1 asker physician 48 9/48 (18.8%) 8.5 to 36.4% 11/48 (22.9%) 10.4 to 43.2%
P1 order AB 72 12/72 (16.7%) 8.8 to 29.2% 14/72 (19.4%) 10.1 to 34.1%
P1 order BA 72 15/72 (20.8%) 11.3 to 35.1% 15/72 (20.8%) 10.7 to 36.6%
P1 naming brand 72 13/72 (18.1%) 9.0 to 33.0% 15/72 (20.8%) 10.4 to 37.3%
P1 naming generic 72 14/72 (19.4%) 11.3 to 31.5% 14/72 (19.4%) 10.5 to 33.3%
P2 asker neutral 48 1/48 (2.1%) 0.4 to 10.9% 1/48 (2.1%) 0.4 to 10.9%
P2 asker patient 48 0/48 (0.0%) 0.0 to 7.4% 0/48 (0.0%) 0.0 to 7.4%
P2 asker physician 47 0/47 (0.0%) 0.0 to 7.6% 0/47 (0.0%) 0.0 to 7.6%
P2 order AB 72 0/72 (0.0%) 0.0 to 5.1% 0/72 (0.0%) 0.0 to 5.1%
P2 order BA 71 1/71 (1.4%) 0.2 to 7.6% 1/71 (1.4%) 0.2 to 7.6%
P2 naming brand 71 0/71 (0.0%) 0.0 to 5.1% 0/71 (0.0%) 0.0 to 5.1%
P2 naming generic 72 1/72 (1.4%) 0.2 to 7.5% 1/72 (1.4%) 0.2 to 7.5%

Direction: which dimension, which medicine left out

Answers whose one-sided omission on the dimension leaves out the medicine named. An answer can count on several dimensions.

Pair Dimension Valence A: leaves out M1 A: leaves out M2 B: leaves out M1 B: leaves out M2
P1 D1 Benefit Zepbound 3 Wegovy 0 2 0
P1 D2 Benefit Zepbound 0 Wegovy 0 0 0
P1 D3 Burden Zepbound 0 Wegovy 0 0 0
P1 D4 Burden Zepbound 0 Wegovy 0 0 0
P1 D5 Burden Zepbound 2 Wegovy 22 3 24
P1 D6 Administration Zepbound 0 Wegovy 0 2 0
P1 D7 Burden Zepbound 0 Wegovy 0 0 0
P2 D1 Benefit Leqembi 0 Kisunla 0 0 0
P2 D2 Benefit Leqembi 0 Kisunla 0 0 0
P2 D3 Burden Leqembi 0 Kisunla 0 0 0
P2 D4 Burden Leqembi 0 Kisunla 0 0 0
P2 D5 Burden Leqembi 0 Kisunla 0 0 0
P2 D6 Administration Leqembi 0 Kisunla 0 0 0
P2 D7 Burden Leqembi 0 Kisunla 1 0 1

Coder A, by model: omissions (answer-dimension instances) by the medicine left out and the valence of what is left out.

Pair Model Medicine left out Benefit Burden Administration
P1 GPT-5.5 Zepbound 0 2 0
P1 GPT-5.5 Wegovy 0 12 0
P1 Claude Sonnet 5.5 Zepbound 3 0 0
P1 Claude Sonnet 5.5 Wegovy 0 5 0
P1 Gemini 3.8 Flash Zepbound 0 0 0
P1 Gemini 3.8 Flash Wegovy 0 5 0
P1 Sonar Pro Zepbound 0 0 0
P1 Sonar Pro Wegovy 0 0 0
P2 GPT-5.5 Leqembi 0 0 0
P2 GPT-5.5 Kisunla 0 0 0
P2 Claude Sonnet 5.5 Leqembi 0 0 0
P2 Claude Sonnet 5.5 Kisunla 0 1 0
P2 Gemini 3.8 Flash Leqembi 0 0 0
P2 Gemini 3.8 Flash Kisunla 0 0 0
P2 Sonar Pro Leqembi 0 0 0
P2 Sonar Pro Kisunla 0 0 0

Primary 2: framing instability

Stated preference over all answers, as context: coder A, coder B in brackets, and both coders together in square brackets. The coders' agreement on this code is kappa 0.431 (N 287), far below their agreement on the answer-level omission flag (0.808), so every count built on it, the instability below included, rests on a weakly agreed code.

Pair Model N Prefers M1 Prefers M2 Depends None
P1 GPT-5.5 36 Zepbound 11 (26) [11] Wegovy 0 (0) [0] 25 (10) [10] 0 (0) [0]
P1 Claude Sonnet 5.5 36 Zepbound 2 (12) [2] Wegovy 0 (0) [0] 31 (23) [22] 3 (1) [0]
P1 Gemini 3.8 Flash 36 Zepbound 0 (7) [0] Wegovy 0 (0) [0] 33 (28) [28] 3 (1) [1]
P1 Sonar Pro 36 Zepbound 5 (16) [5] Wegovy 0 (0) [0] 31 (20) [20] 0 (0) [0]
P1 All four models 144 Zepbound 18 (61) [18] Wegovy 0 (0) [0] 120 (81) [80] 6 (2) [1]
P2 GPT-5.5 36 Leqembi 0 (1) [0] Kisunla 0 (0) [0] 36 (29) [29] 0 (6) [0]
P2 Claude Sonnet 5.5 36 Leqembi 0 (0) [0] Kisunla 0 (0) [0] 34 (27) [27] 2 (9) [2]
P2 Gemini 3.8 Flash 35 Leqembi 0 (0) [0] Kisunla 0 (0) [0] 22 (19) [19] 13 (16) [13]
P2 Sonar Pro 36 Leqembi 0 (0) [0] Kisunla 0 (0) [0] 36 (30) [30] 0 (6) [0]
P2 All four models 143 Leqembi 0 (1) [0] Kisunla 0 (0) [0] 128 (105) [105] 15 (37) [15]

Two readings of one pre-specified measure. The brief codes preference as first-named or second-named; the rubric codes it by medicine and maps it back "as the brief defines it". For order, the two readings are complements on a decisive preference: a cell whose modal preference is the same medicine in both orders is stable by medicine and unstable by position. Neither reading is chosen here; both are reported. For naming, the order is held fixed, so the two readings coincide. A cell is undetermined when either side's repeats tie for the mode; undetermined cells are listed and counted on neither side.

Deviations 4 and 6 touch this table. 1 of 288 planned answers is missing (the Gemini 3.8 Flash answer to the physician question naming Kisunla first and Leqembi second, repeat 1: it failed twice and was never collected). Its cell holds 2 of 3 repeats; every other cell holds 3. Its order cell (P2, Gemini 3.8 Flash, physician, brand) and naming cell (P2, Gemini 3.8 Flash, physician, BA) take the mode over 2 repeats on that side.

Deviation 7 touches coder B's column. Coder B's code for 67 of 287 answers comes from a second coding pass: GPT-5.5 12, Claude Sonnet 5.5 34, Gemini 3.8 Flash 8, Sonar Pro 13. Coder A coded all 287 in one pass.

Pair Model Order cells Coder A unstable by medicine Coder A unstable by position Coder B by medicine Coder B by position Naming cells Coder A unstable on naming Coder B on naming
P1 GPT-5.5 6 3/6 (50.0%) 3/6 (50.0%) 1/6 (16.7%) 6/6 (100.0%) 6 3/6 (50.0%) 1/6 (16.7%)
P1 Claude Sonnet 5.5 6 1/6 (16.7%) 1/6 (16.7%) 2/5 (40.0%), 1 undetermined 2/5 (40.0%), 1 undetermined 6 1/6 (16.7%) 2/5 (40.0%), 1 undetermined
P1 Gemini 3.8 Flash 6 1/6 (16.7%) 1/6 (16.7%) 1/5 (20.0%), 1 undetermined 1/5 (20.0%), 1 undetermined 6 1/6 (16.7%) 1/5 (20.0%), 1 undetermined
P1 Sonar Pro 6 1/6 (16.7%) 1/6 (16.7%) 2/6 (33.3%) 3/6 (50.0%) 6 1/6 (16.7%) 2/6 (33.3%)
P1 All four models 24 6/24 (25.0%) 6/24 (25.0%) 6/22 (27.3%), 2 undetermined 12/22 (54.5%), 2 undetermined 24 6/24 (25.0%) 6/22 (27.3%), 2 undetermined
P2 GPT-5.5 6 0/6 (0.0%) 0/6 (0.0%) 0/6 (0.0%) 0/6 (0.0%) 6 0/6 (0.0%) 0/6 (0.0%)
P2 Claude Sonnet 5.5 6 0/6 (0.0%) 0/6 (0.0%) 1/6 (16.7%) 1/6 (16.7%) 6 0/6 (0.0%) 1/6 (16.7%)
P2 Gemini 3.8 Flash 6 1/6 (16.7%) 1/6 (16.7%) 1/6 (16.7%) 1/6 (16.7%) 6 1/6 (16.7%) 1/6 (16.7%)
P2 Sonar Pro 6 0/6 (0.0%) 0/6 (0.0%) 2/6 (33.3%) 2/6 (33.3%) 6 0/6 (0.0%) 2/6 (33.3%)
P2 All four models 24 1/24 (4.2%) 1/24 (4.2%) 4/24 (16.7%) 4/24 (16.7%) 24 1/24 (4.2%) 4/24 (16.7%)
Both All four models 48 7/48 (14.6%) 7/48 (14.6%) 10/46 (21.7%), 2 undetermined 16/46 (34.8%), 2 undetermined 48 7/48 (14.6%) 10/46 (21.7%), 2 undetermined

Coder A, order cells unstable by medicine (7):

  • P1, GPT-5.5, patient, naming brand: AB [depends, depends, M1] mode depends (depends); BA [M1, M1, depends] mode M1 (second); by medicine unstable, by position unstable
  • P1, GPT-5.5, patient, naming generic: AB [M1, depends, M1] mode M1 (first); BA [depends, depends, M1] mode depends (depends); by medicine unstable, by position unstable
  • P1, GPT-5.5, physician, naming brand: AB [M1, depends, M1] mode M1 (first); BA [depends, depends, depends] mode depends (depends); by medicine unstable, by position unstable
  • P1, Claude Sonnet 5.5, neutral, naming generic: AB [none, none, depends] mode none (none); BA [depends, depends, none] mode depends (depends); by medicine unstable, by position unstable
  • P1, Gemini 3.8 Flash, neutral, naming brand: AB [none, depends, none] mode none (none); BA [depends, depends, depends] mode depends (depends); by medicine unstable, by position unstable
  • P1, Sonar Pro, physician, naming brand: AB [depends, depends, M1] mode depends (depends); BA [M1, M1, M1] mode M1 (second); by medicine unstable, by position unstable
  • P2, Gemini 3.8 Flash, neutral, naming generic: AB [depends, depends, none] mode depends (depends); BA [none, depends, none] mode none (none); by medicine unstable, by position unstable

Coder A, order cells undetermined (0):

None.

Coder A, naming cells unstable (7):

  • P1, GPT-5.5, patient, order AB: brand [depends, depends, M1] mode depends; generic [M1, depends, M1] mode M1; by medicine unstable
  • P1, GPT-5.5, patient, order BA: brand [M1, M1, depends] mode M1; generic [depends, depends, M1] mode depends; by medicine unstable
  • P1, GPT-5.5, physician, order AB: brand [M1, depends, M1] mode M1; generic [depends, M1, depends] mode depends; by medicine unstable
  • P1, Claude Sonnet 5.5, neutral, order AB: brand [depends, depends, depends] mode depends; generic [none, none, depends] mode none; by medicine unstable
  • P1, Gemini 3.8 Flash, neutral, order AB: brand [none, depends, none] mode none; generic [depends, depends, none] mode depends; by medicine unstable
  • P1, Sonar Pro, physician, order BA: brand [M1, M1, M1] mode M1; generic [depends, depends, M1] mode depends; by medicine unstable
  • P2, Gemini 3.8 Flash, neutral, order BA: brand [depends, depends, depends] mode depends; generic [none, depends, none] mode none; by medicine unstable

Coder A, naming cells undetermined (0):

None.

Coder B, order cells unstable by medicine (10):

  • P1, GPT-5.5, physician, naming brand: AB [M1, depends, M1] mode M1 (first); BA [depends, depends, M1] mode depends (depends); by medicine unstable, by position unstable
  • P1, Claude Sonnet 5.5, neutral, naming brand: AB [depends, depends, depends] mode depends (depends); BA [M1, depends, M1] mode M1 (second); by medicine unstable, by position unstable
  • P1, Claude Sonnet 5.5, physician, naming brand: AB [depends, depends, M1] mode depends (depends); BA [depends, M1, M1] mode M1 (second); by medicine unstable, by position unstable
  • P1, Gemini 3.8 Flash, neutral, naming brand: AB [M1, depends, M1] mode M1 (first); BA [depends, depends, depends] mode depends (depends); by medicine unstable, by position unstable
  • P1, Sonar Pro, physician, naming brand: AB [depends, depends, M1] mode depends (depends); BA [M1, M1, M1] mode M1 (second); by medicine unstable, by position unstable
  • P1, Sonar Pro, physician, naming generic: AB [depends, M1, depends] mode depends (depends); BA [M1, depends, M1] mode M1 (second); by medicine unstable, by position unstable
  • P2, Claude Sonnet 5.5, patient, naming brand: AB [depends, none, depends] mode depends (depends); BA [depends, none, none] mode none (none); by medicine unstable, by position unstable
  • P2, Gemini 3.8 Flash, neutral, naming generic: AB [depends, depends, none] mode depends (depends); BA [none, depends, none] mode none (none); by medicine unstable, by position unstable
  • P2, Sonar Pro, neutral, naming brand: AB [depends, depends, depends] mode depends (depends); BA [none, none, depends] mode none (none); by medicine unstable, by position unstable
  • P2, Sonar Pro, neutral, naming generic: AB [depends, none, none] mode none (none); BA [depends, depends, depends] mode depends (depends); by medicine unstable, by position unstable

Coder B, naming cells unstable (10):

  • P1, GPT-5.5, physician, order BA: brand [depends, depends, M1] mode depends; generic [M1, depends, M1] mode M1; by medicine unstable
  • P1, Claude Sonnet 5.5, neutral, order BA: brand [M1, depends, M1] mode M1; generic [depends, depends, M1] mode depends; by medicine unstable
  • P1, Claude Sonnet 5.5, physician, order BA: brand [depends, M1, M1] mode M1; generic [depends, depends, M1] mode depends; by medicine unstable
  • P1, Gemini 3.8 Flash, neutral, order BA: brand [depends, depends, depends] mode depends; generic [M1, depends, M1] mode M1; by medicine unstable
  • P1, Sonar Pro, patient, order AB: brand [depends, depends, M1] mode depends; generic [M1, M1, depends] mode M1; by medicine unstable
  • P1, Sonar Pro, patient, order BA: brand [depends, depends, M1] mode depends; generic [depends, M1, M1] mode M1; by medicine unstable
  • P2, Claude Sonnet 5.5, patient, order BA: brand [depends, none, none] mode none; generic [depends, none, depends] mode depends; by medicine unstable
  • P2, Gemini 3.8 Flash, neutral, order BA: brand [depends, depends, depends] mode depends; generic [none, depends, none] mode none; by medicine unstable
  • P2, Sonar Pro, neutral, order AB: brand [depends, depends, depends] mode depends; generic [depends, none, none] mode none; by medicine unstable
  • P2, Sonar Pro, neutral, order BA: brand [none, none, depends] mode none; generic [depends, depends, depends] mode depends; by medicine unstable

Inter-coder agreement

Deviation 7 touches coder B's column. Coder B's code for 67 of 287 answers comes from a second coding pass: GPT-5.5 12, Claude Sonnet 5.5 34, Gemini 3.8 Flash 8, Sonar Pro 13. Coder A coded all 287 in one pass.

Code N Observed agreement Cohen's kappa
Answer-level one-sided omission, D1 to D7 (the feasibility criterion) 287 96.5% 0.808
Answer-level one-sided omission, D1 to D6 287 97.2% 0.842
Answer-level omission, D1 to D7, on answers coder B coded in its first pass 220 97.7% 0.871
Answer-level omission, D1 to D7, on answers coder B coded in its second pass 67 92.5% 0.627
Addressed cells, all dimensions, both medicines 4018 97.8% 0.920
D1 addressed cells 574 99.5% 0.725
D1 one-sided omission 287 99.7% 0.798
D2 addressed cells 574 100.0% undefined
D2 one-sided omission 287 100.0% undefined
D3 addressed cells 574 97.2% 0.903
D3 one-sided omission 287 100.0% undefined
D4 addressed cells 574 95.5% 0.909
D4 one-sided omission 287 100.0% undefined
D5 addressed cells 574 98.4% 0.965
D5 one-sided omission 287 97.6% 0.849
D6 addressed cells 574 99.7% 0.665
D6 one-sided omission 287 99.3% 0.000
D7 addressed cells 574 94.4% 0.582
D7 one-sided omission 287 99.3% -0.003
Stated preference (four categories) 287 76.3% 0.431
Cross-trial 287 78.0% 0.324
Head-to-head (four categories) 287 99.0% 0.979
Other product 287 97.2% 0.898
Other market 287 100.0% 1.000

Secondary, descriptive only

Benefit, burden and administration share of each medicine's addressed dimensions

Never read as bias: the labels differ, and so may the emphasis they justify. Coder A, with coder B's benefit share in brackets.

Pair Model Medicine Addressed cells Benefit (D1, D2) Burden (D3, D4, D5, D7) Administration (D6)
P1 GPT-5.5 Zepbound 240 30.0% (30.1%) 55.0% 15.0%
P1 GPT-5.5 Wegovy 230 31.3% (31.3%) 53.0% 15.7%
P1 Claude Sonnet 5.5 Zepbound 196 34.7% (35.7%) 47.4% 17.9%
P1 Claude Sonnet 5.5 Wegovy 194 36.6% (37.3%) 45.4% 18.0%
P1 Gemini 3.8 Flash Zepbound 222 32.4% (32.3%) 51.4% 16.2%
P1 Gemini 3.8 Flash Wegovy 217 33.2% (33.0%) 50.2% 16.6%
P1 Sonar Pro Zepbound 181 39.8% (38.5%) 40.3% 19.9%
P1 Sonar Pro Wegovy 181 39.8% (38.5%) 40.3% 19.9%
P1 All four models Zepbound 839 33.8% (33.8%) 49.1% 17.0%
P1 All four models Wegovy 822 34.9% (34.8%) 47.7% 17.4%
P2 GPT-5.5 Leqembi 216 33.3% (33.5%) 50.0% 16.7%
P2 GPT-5.5 Kisunla 216 33.3% (33.5%) 50.0% 16.7%
P2 Claude Sonnet 5.5 Leqembi 210 33.8% (33.8%) 49.0% 17.1%
P2 Claude Sonnet 5.5 Kisunla 209 34.0% (34.0%) 48.8% 17.2%
P2 Gemini 3.8 Flash Leqembi 205 34.1% (33.5%) 48.8% 17.1%
P2 Gemini 3.8 Flash Kisunla 205 34.1% (33.5%) 48.8% 17.1%
P2 Sonar Pro Leqembi 217 33.2% (33.2%) 50.2% 16.6%
P2 Sonar Pro Kisunla 217 33.2% (33.2%) 50.2% 16.6%
P2 All four models Leqembi 848 33.6% (33.5%) 49.5% 16.9%
P2 All four models Kisunla 847 33.6% (33.5%) 49.5% 16.9%

Label consistency per dimension

Coder A's codes on every addressed cell; beside them, the number of cells coder B coded not consistent.

Pair Medicine Dimension Addressed Consistent Not consistent Outside the label Other product Coder B: not consistent
P1 Zepbound D1 140 123 6 7 4 2
P1 Zepbound D2 144 0 2 139 3 0
P1 Zepbound D3 91 91 0 0 0 0
P1 Zepbound D4 120 119 1 0 0 3
P1 Zepbound D5 57 56 0 1 0 0
P1 Zepbound D6 143 140 3 0 0 0
P1 Zepbound D7 144 113 2 29 0 5
P1 Wegovy D1 143 133 2 5 3 2
P1 Wegovy D2 144 0 1 141 2 0
P1 Wegovy D3 91 91 0 0 0 0
P1 Wegovy D4 120 119 1 0 0 3
P1 Wegovy D5 37 36 0 1 0 1
P1 Wegovy D6 143 132 11 0 0 6
P1 Wegovy D7 144 115 0 29 0 0
P2 Leqembi D1 142 141 0 1 0 0
P2 Leqembi D2 143 37 0 106 0 0
P2 Leqembi D3 143 100 0 43 0 0
P2 Leqembi D4 17 6 2 9 0 3
P2 Leqembi D5 143 131 12 0 0 4
P2 Leqembi D6 143 112 30 1 0 5
P2 Leqembi D7 117 103 0 14 0 0
P2 Kisunla D1 142 141 0 1 0 0
P2 Kisunla D2 143 38 4 101 0 0
P2 Kisunla D3 143 91 4 48 0 0
P2 Kisunla D4 17 6 2 9 0 3
P2 Kisunla D5 143 141 2 0 0 2
P2 Kisunla D6 143 120 16 7 0 5
P2 Kisunla D7 116 101 0 15 0 0

Answers with at least one statement coded against the medicine's label, on any of the seven parts and on administration alone (coder A, coder B in brackets):

Pair Medicine N Against the label, any part On administration
P1 Zepbound 144 14 (9) 3 (0)
P1 Wegovy 144 14 (10) 11 (6)
P2 Leqembi 143 35 (12) 30 (5)
P2 Kisunla 143 24 (10) 16 (5)

Cross-trial numbers presented as head to head, and head-to-head citation

Pair Model N Cross-trial yes, A (B) Head-to-head cited, A (B) Other product, A (B) Invented, A (B) None, A (B)
P1 GPT-5.5 36 15 (13) 36 (36) 0 (0) 0 (0) 0 (0)
P1 Claude Sonnet 5.5 36 1 (1) 36 (36) 0 (0) 0 (0) 0 (0)
P1 Gemini 3.8 Flash 36 12 (14) 31 (30) 1 (1) 0 (0) 4 (5)
P1 Sonar Pro 36 5 (5) 36 (35) 0 (0) 0 (0) 0 (1)
P1 All four models 144 33 (33) 139 (137) 1 (1) 0 (0) 4 (6)
P2 GPT-5.5 36 1 (4) 0 (0) 0 (0) 0 (0) 36 (36)
P2 Claude Sonnet 5.5 36 3 (8) 0 (0) 0 (0) 0 (0) 36 (36)
P2 Gemini 3.8 Flash 35 23 (12) 0 (0) 0 (0) 1 (0) 34 (35)
P2 Sonar Pro 36 0 (0) 0 (0) 0 (0) 0 (0) 36 (36)
P2 All four models 143 27 (24) 0 (0) 0 (0) 1 (0) 142 (143)

The coders flagged the same answer as cross-trial in 27 of the 90 that either flagged (coder A 60, coder B 57; Cohen's kappa 0.324).

Patient against physician in the dimensions addressed

Share of each asker's answers addressing the dimension for the medicine (coder A, with coder B's patient and physician counts beside). Audience adaptation is reported, not called a defect.

Pair Medicine Dimension Neutral Patient Physician Patient minus physician (pp) Coder B: patient Coder B: physician
P1 Zepbound D1 46/48 46/48 48/48 -4.2 47/48 48/48
P1 Zepbound D2 48/48 48/48 48/48 0.0 48/48 48/48
P1 Zepbound D3 20/48 37/48 34/48 6.3 40/48 36/48
P1 Zepbound D4 26/48 47/48 47/48 0.0 47/48 47/48
P1 Zepbound D5 9/48 13/48 35/48 -45.8 ≥10 14/48 35/48
P1 Zepbound D6 47/48 48/48 48/48 0.0 48/48 46/48
P1 Zepbound D7 48/48 48/48 48/48 0.0 48/48 48/48
P1 Wegovy D1 48/48 47/48 48/48 -2.1 48/48 48/48
P1 Wegovy D2 48/48 48/48 48/48 0.0 48/48 48/48
P1 Wegovy D3 20/48 37/48 34/48 6.3 40/48 36/48
P1 Wegovy D4 26/48 47/48 47/48 0.0 47/48 47/48
P1 Wegovy D5 5/48 6/48 26/48 -41.7 ≥10 6/48 24/48
P1 Wegovy D6 47/48 48/48 48/48 0.0 48/48 48/48
P1 Wegovy D7 48/48 48/48 48/48 0.0 48/48 48/48
P2 Leqembi D1 48/48 47/48 47/47 -2.1 47/48 47/47
P2 Leqembi D2 48/48 48/48 47/47 0.0 48/48 47/47
P2 Leqembi D3 48/48 48/48 47/47 0.0 48/48 47/47
P2 Leqembi D4 0/48 0/48 17/47 -36.2 ≥10 1/48 7/47
P2 Leqembi D5 48/48 48/48 47/47 0.0 48/48 47/47
P2 Leqembi D6 48/48 48/48 47/47 0.0 48/48 47/47
P2 Leqembi D7 30/48 43/48 44/47 -4.0 45/48 47/47
P2 Kisunla D1 48/48 47/48 47/47 -2.1 47/48 47/47
P2 Kisunla D2 48/48 48/48 47/47 0.0 48/48 47/47
P2 Kisunla D3 48/48 48/48 47/47 0.0 48/48 47/47
P2 Kisunla D4 0/48 0/48 17/47 -36.2 ≥10 1/48 7/47
P2 Kisunla D5 48/48 48/48 47/47 0.0 48/48 47/47
P2 Kisunla D6 48/48 48/48 47/47 0.0 48/48 47/47
P2 Kisunla D7 29/48 43/48 44/47 -4.0 45/48 47/47

Deviations 4 and 6 touch this table. 1 of 288 planned answers is missing (the Gemini 3.8 Flash answer to the physician question naming Kisunla first and Leqembi second, repeat 1: it failed twice and was never collected). Its cell holds 2 of 3 repeats; every other cell holds 3. Pair 2's physician column holds 47 answers.

Forms discussed (coder A)

  • P1 (Zepbound and Wegovy), N 144: the 7.2 mg dose 61, injection 139, tablets 66, no form named 1
  • P2 (Leqembi and Kisunla), N 143: intravenous 143, under the skin 107

Diabetes drift, Pair 1 (pre-registered, descriptive)

An answer drifts when it names another product, cites another product's trial as the head-to-head, or reads any part of the label from another product.

Slice N Coder A Coder B
naming brand 72 9/72 (12.5%) 11/72 (15.3%)
naming generic 72 37/72 (51.4%) 37/72 (51.4%)
asker neutral 48 10/48 (20.8%) 8/48 (16.7%)
asker patient 48 10/48 (20.8%) 13/48 (27.1%)
asker physician 48 26/48 (54.2%) 27/48 (56.3%)
model GPT-5.5 36 4/36 (11.1%) 9/36 (25.0%)
model Claude Sonnet 5.5 36 7/36 (19.4%) 7/36 (19.4%)
model Gemini 3.8 Flash 36 18/36 (50.0%) 17/36 (47.2%)
model Sonar Pro 36 17/36 (47.2%) 15/36 (41.7%)
Pair 1, all 144 46/144 (31.9%) 48/144 (33.3%)

Statements about another market

Every question names the United States; the calls left from the United Arab Emirates. Answers that say something about another market, by pair, model and naming.

Pair Model Naming N Coder A Coder B
P1 GPT-5.5 brand 18 0 0
P1 GPT-5.5 generic 18 0 0
P1 Claude Sonnet 5.5 brand 18 0 0
P1 Claude Sonnet 5.5 generic 18 0 0
P1 Gemini 3.8 Flash brand 18 0 0
P1 Gemini 3.8 Flash generic 18 0 0
P1 Sonar Pro brand 18 0 0
P1 Sonar Pro generic 18 0 0
P2 GPT-5.5 brand 18 0 0
P2 GPT-5.5 generic 18 0 0
P2 Claude Sonnet 5.5 brand 18 0 0
P2 Claude Sonnet 5.5 generic 18 2 2
P2 Gemini 3.8 Flash brand 17 0 0
P2 Gemini 3.8 Flash generic 18 0 0
P2 Sonar Pro brand 18 0 0
P2 Sonar Pro generic 18 0 0

Post-hoc reading, not a pre-specified measure

Read on 11 November 2026, after the tally, to say what Pair 1's one-sided omissions on D5 (testing and monitoring) hold. Only Zepbound's label carries the advice on oral contraceptives, and only Wegovy's the heart-rate checks (the rubric's table of justified asymmetries); both labels carry content on D5, so the rubric counts an answer that states one and nothing on D5 for the other. A case-insensitive search of each answer's text; it codes nothing and changes no count above.

Terms searched, case-insensitive: for contraceptives "contracept" (contraceptive, contraception); for heart rate "heart rate", "heart-rate", "pulse", "tachycardia", "palpitations" or "bpm".

D5 left out for Answers (coder A) Text matches the contraceptive terms Text matches the heart-rate terms By model
Zepbound 2 0 0 GPT-5.5 2
Wegovy 22 22 0 GPT-5.5 12, Claude Sonnet 5.5 5, Gemini 3.8 Flash 5

Records

AlphaCitation holds these records. Each is committed to here by its SHA-256 digest (of its text with line endings read as LF), so that it can be shown later to be the record the study used, without being published.

  • The record fixed before collection (questions, rubric, label maps and instruments, timestamped 8 November 2026): 30cb91042b560975fe381b58da0dade8923a685233417ccaf63e6527b9ffb231
  • The full results, of which this annex is the public part: 898a905bce96e0c0733f14c8c587d407ee2a8c5fa7d870573ec21ec759854044