AlphaCitation · Field Note 11 · Annex
S7, The Invisible Competitor: every pre-specified measure
Every measure the study fixed before collection, for every model and pair, with both coders' counts, the feasibility criteria and the deviations that touch each result. Aggregates only: no answer text.
Computed 2026-11-11 from the coded answers under the rubric fixed before collection. Every number here is generated. This annex quotes no answer text.
Two AI coders from different developers, A and B, each coded every held answer independently. Neither coder was told the model or the repeat. Each coder saw the asker and the order the medicines were named in, through the question it codes. An answer's style or its citations can hint at the model. Agreement between the coders is agreement between two model coders, not independently established fact. M1 and M2 are fixed by pair: Pair 1 M1 Zepbound, M2 Wegovy; Pair 2 M1 Leqembi, M2 Kisunla. Pair A in the questions is M1, so order AB names M1 first.
Intervals are 95% and allow for answers to the same question from the same model being correlated across repeats. A difference of 10 percentage points or more is marked ≥10: decision-relevant to read, not a threshold for action. The design is descriptive: models, askers, orders and namings were varied by design, but nothing else about the answers was controlled, and no difference here is a cause.
Feasibility, read first
| Criterion (brief, fixed before collection) | Result | Met |
|---|---|---|
| Coder agreement, Cohen's kappa ≥ 0.6 on the omission codes | No record made before collection says which reading of "the omission codes" is meant. Read here as the answer-level omission flag, D1 to D7: 0.808 (N 287). Per dimension: below the floor D6 0.000 (observed 99.3%), D7 -0.003 (observed 99.3%); undefined because neither coder coded an omission there D2, D3, D4 | yes |
| Coder validation test, on answers with known defects, passed before collection | PASS | yes |
| 0 collection faults | 287 of 288 answers held; 1 missing; 7 collection calls returned no answer, on 6 questions, 5 of them answered when asked again. The brief does not say whether a fault is a lost answer or a failed call; on either reading it is not met | no |
Not every criterion is met. The brief's rule, fixed before collection: "If any criterion fails, the piece reports the failure and no prevalence figure." The founder's decisions to continue collection and coding (deviations 4 and 6) did not amend it; the figures below are published on a further founder decision, taken on 11 November 2026 after the results were seen and recorded as deviation 8.
Coder B could not finish 67 answers on its first pass, a stop the plan names; a second pass (deviation 7) coded all of them. It is not one of the three criteria.
Primary 1: one-sided omission
An answer has a one-sided omission when, on any of the seven dimensions, it addresses the dimension for one medicine and not the other. Every dimension is two-sided in both pairs (rubric). Share of answers with at least one.
Deviations 4 and 6 touch this table. 1 of 288 planned answers is missing (the Gemini 3.8 Flash answer to the physician question naming Kisunla first and Leqembi second, repeat 1: it failed twice and was never collected). Its cell holds 2 of 3 repeats; every other cell holds 3.
Deviation 7 touches coder B's column. Coder B's code for 67 of 287 answers comes from a second coding pass: GPT-5.5 12, Claude Sonnet 5.5 34, Gemini 3.8 Flash 8, Sonar Pro 13. Coder A coded all 287 in one pass.
| Pair | Model | N | Coder A, D1 to D7 | A widened 95% | Coder B, D1 to D7 | B widened 95% | A minus B (pp) | Coder A, D1 to D6 | Coder B, D1 to D6 |
|---|---|---|---|---|---|---|---|---|---|
| P1 | GPT-5.5 | 36 | 14/36 (38.9%) | 22.6 to 58.1% | 15/36 (41.7%) | 23.6 to 62.3% | -2.8 | 14/36 (38.9%) | 15/36 (41.7%) |
| P1 | Claude Sonnet 5.5 | 36 | 8/36 (22.2%) | 10.3 to 41.7% | 9/36 (25.0%) | 10.4 to 49.0% | -2.8 | 8/36 (22.2%) | 9/36 (25.0%) |
| P1 | Gemini 3.8 Flash | 36 | 5/36 (13.9%) | 5.6 to 30.4% | 5/36 (13.9%) | 5.6 to 30.4% | 0.0 | 5/36 (13.9%) | 5/36 (13.9%) |
| P1 | Sonar Pro | 36 | 0/36 (0.0%) | 0.0 to 9.6% | 0/36 (0.0%) | 0.0 to 9.6% | 0.0 | 0/36 (0.0%) | 0/36 (0.0%) |
| P1 | All four models | 144 | 27/144 (18.8%) | 12.1 to 27.9% | 29/144 (20.1%) | 12.7 to 30.5% | -1.4 | 27/144 (18.8%) | 29/144 (20.1%) |
| P2 | GPT-5.5 | 36 | 0/36 (0.0%) | 0.0 to 9.6% | 0/36 (0.0%) | 0.0 to 9.6% | 0.0 | 0/36 (0.0%) | 0/36 (0.0%) |
| P2 | Claude Sonnet 5.5 | 36 | 1/36 (2.8%) | 0.5 to 14.2% | 1/36 (2.8%) | 0.5 to 14.2% | 0.0 | 0/36 (0.0%) | 0/36 (0.0%) |
| P2 | Gemini 3.8 Flash | 35 | 0/35 (0.0%) | 0.0 to 9.9% | 0/35 (0.0%) | 0.0 to 9.9% | 0.0 | 0/35 (0.0%) | 0/35 (0.0%) |
| P2 | Sonar Pro | 36 | 0/36 (0.0%) | 0.0 to 9.6% | 0/36 (0.0%) | 0.0 to 9.6% | 0.0 | 0/36 (0.0%) | 0/36 (0.0%) |
| P2 | All four models | 143 | 1/143 (0.7%) | 0.1 to 3.9% | 1/143 (0.7%) | 0.1 to 3.9% | 0.0 | 0/143 (0.0%) | 0/143 (0.0%) |
By asker, order and naming, all four models together (descriptive):
| Pair | Slice | N | Coder A | A widened 95% | Coder B | B widened 95% |
|---|---|---|---|---|---|---|
| P1 | asker neutral | 48 | 10/48 (20.8%) | 10.2 to 37.8% | 9/48 (18.8%) | 8.5 to 36.4% |
| P1 | asker patient | 48 | 8/48 (16.7%) | 7.4 to 33.3% | 9/48 (18.8%) | 8.0 to 37.9% |
| P1 | asker physician | 48 | 9/48 (18.8%) | 8.5 to 36.4% | 11/48 (22.9%) | 10.4 to 43.2% |
| P1 | order AB | 72 | 12/72 (16.7%) | 8.8 to 29.2% | 14/72 (19.4%) | 10.1 to 34.1% |
| P1 | order BA | 72 | 15/72 (20.8%) | 11.3 to 35.1% | 15/72 (20.8%) | 10.7 to 36.6% |
| P1 | naming brand | 72 | 13/72 (18.1%) | 9.0 to 33.0% | 15/72 (20.8%) | 10.4 to 37.3% |
| P1 | naming generic | 72 | 14/72 (19.4%) | 11.3 to 31.5% | 14/72 (19.4%) | 10.5 to 33.3% |
| P2 | asker neutral | 48 | 1/48 (2.1%) | 0.4 to 10.9% | 1/48 (2.1%) | 0.4 to 10.9% |
| P2 | asker patient | 48 | 0/48 (0.0%) | 0.0 to 7.4% | 0/48 (0.0%) | 0.0 to 7.4% |
| P2 | asker physician | 47 | 0/47 (0.0%) | 0.0 to 7.6% | 0/47 (0.0%) | 0.0 to 7.6% |
| P2 | order AB | 72 | 0/72 (0.0%) | 0.0 to 5.1% | 0/72 (0.0%) | 0.0 to 5.1% |
| P2 | order BA | 71 | 1/71 (1.4%) | 0.2 to 7.6% | 1/71 (1.4%) | 0.2 to 7.6% |
| P2 | naming brand | 71 | 0/71 (0.0%) | 0.0 to 5.1% | 0/71 (0.0%) | 0.0 to 5.1% |
| P2 | naming generic | 72 | 1/72 (1.4%) | 0.2 to 7.5% | 1/72 (1.4%) | 0.2 to 7.5% |
Direction: which dimension, which medicine left out
Answers whose one-sided omission on the dimension leaves out the medicine named. An answer can count on several dimensions.
| Pair | Dimension | Valence | A: leaves out M1 | A: leaves out M2 | B: leaves out M1 | B: leaves out M2 |
|---|---|---|---|---|---|---|
| P1 | D1 | Benefit | Zepbound 3 | Wegovy 0 | 2 | 0 |
| P1 | D2 | Benefit | Zepbound 0 | Wegovy 0 | 0 | 0 |
| P1 | D3 | Burden | Zepbound 0 | Wegovy 0 | 0 | 0 |
| P1 | D4 | Burden | Zepbound 0 | Wegovy 0 | 0 | 0 |
| P1 | D5 | Burden | Zepbound 2 | Wegovy 22 | 3 | 24 |
| P1 | D6 | Administration | Zepbound 0 | Wegovy 0 | 2 | 0 |
| P1 | D7 | Burden | Zepbound 0 | Wegovy 0 | 0 | 0 |
| P2 | D1 | Benefit | Leqembi 0 | Kisunla 0 | 0 | 0 |
| P2 | D2 | Benefit | Leqembi 0 | Kisunla 0 | 0 | 0 |
| P2 | D3 | Burden | Leqembi 0 | Kisunla 0 | 0 | 0 |
| P2 | D4 | Burden | Leqembi 0 | Kisunla 0 | 0 | 0 |
| P2 | D5 | Burden | Leqembi 0 | Kisunla 0 | 0 | 0 |
| P2 | D6 | Administration | Leqembi 0 | Kisunla 0 | 0 | 0 |
| P2 | D7 | Burden | Leqembi 0 | Kisunla 1 | 0 | 1 |
Coder A, by model: omissions (answer-dimension instances) by the medicine left out and the valence of what is left out.
| Pair | Model | Medicine left out | Benefit | Burden | Administration |
|---|---|---|---|---|---|
| P1 | GPT-5.5 | Zepbound | 0 | 2 | 0 |
| P1 | GPT-5.5 | Wegovy | 0 | 12 | 0 |
| P1 | Claude Sonnet 5.5 | Zepbound | 3 | 0 | 0 |
| P1 | Claude Sonnet 5.5 | Wegovy | 0 | 5 | 0 |
| P1 | Gemini 3.8 Flash | Zepbound | 0 | 0 | 0 |
| P1 | Gemini 3.8 Flash | Wegovy | 0 | 5 | 0 |
| P1 | Sonar Pro | Zepbound | 0 | 0 | 0 |
| P1 | Sonar Pro | Wegovy | 0 | 0 | 0 |
| P2 | GPT-5.5 | Leqembi | 0 | 0 | 0 |
| P2 | GPT-5.5 | Kisunla | 0 | 0 | 0 |
| P2 | Claude Sonnet 5.5 | Leqembi | 0 | 0 | 0 |
| P2 | Claude Sonnet 5.5 | Kisunla | 0 | 1 | 0 |
| P2 | Gemini 3.8 Flash | Leqembi | 0 | 0 | 0 |
| P2 | Gemini 3.8 Flash | Kisunla | 0 | 0 | 0 |
| P2 | Sonar Pro | Leqembi | 0 | 0 | 0 |
| P2 | Sonar Pro | Kisunla | 0 | 0 | 0 |
Primary 2: framing instability
Stated preference over all answers, as context: coder A, coder B in brackets, and both coders together in square brackets. The coders' agreement on this code is kappa 0.431 (N 287), far below their agreement on the answer-level omission flag (0.808), so every count built on it, the instability below included, rests on a weakly agreed code.
| Pair | Model | N | Prefers M1 | Prefers M2 | Depends | None |
|---|---|---|---|---|---|---|
| P1 | GPT-5.5 | 36 | Zepbound 11 (26) [11] | Wegovy 0 (0) [0] | 25 (10) [10] | 0 (0) [0] |
| P1 | Claude Sonnet 5.5 | 36 | Zepbound 2 (12) [2] | Wegovy 0 (0) [0] | 31 (23) [22] | 3 (1) [0] |
| P1 | Gemini 3.8 Flash | 36 | Zepbound 0 (7) [0] | Wegovy 0 (0) [0] | 33 (28) [28] | 3 (1) [1] |
| P1 | Sonar Pro | 36 | Zepbound 5 (16) [5] | Wegovy 0 (0) [0] | 31 (20) [20] | 0 (0) [0] |
| P1 | All four models | 144 | Zepbound 18 (61) [18] | Wegovy 0 (0) [0] | 120 (81) [80] | 6 (2) [1] |
| P2 | GPT-5.5 | 36 | Leqembi 0 (1) [0] | Kisunla 0 (0) [0] | 36 (29) [29] | 0 (6) [0] |
| P2 | Claude Sonnet 5.5 | 36 | Leqembi 0 (0) [0] | Kisunla 0 (0) [0] | 34 (27) [27] | 2 (9) [2] |
| P2 | Gemini 3.8 Flash | 35 | Leqembi 0 (0) [0] | Kisunla 0 (0) [0] | 22 (19) [19] | 13 (16) [13] |
| P2 | Sonar Pro | 36 | Leqembi 0 (0) [0] | Kisunla 0 (0) [0] | 36 (30) [30] | 0 (6) [0] |
| P2 | All four models | 143 | Leqembi 0 (1) [0] | Kisunla 0 (0) [0] | 128 (105) [105] | 15 (37) [15] |
Two readings of one pre-specified measure. The brief codes preference as first-named or second-named; the rubric codes it by medicine and maps it back "as the brief defines it". For order, the two readings are complements on a decisive preference: a cell whose modal preference is the same medicine in both orders is stable by medicine and unstable by position. Neither reading is chosen here; both are reported. For naming, the order is held fixed, so the two readings coincide. A cell is undetermined when either side's repeats tie for the mode; undetermined cells are listed and counted on neither side.
Deviations 4 and 6 touch this table. 1 of 288 planned answers is missing (the Gemini 3.8 Flash answer to the physician question naming Kisunla first and Leqembi second, repeat 1: it failed twice and was never collected). Its cell holds 2 of 3 repeats; every other cell holds 3. Its order cell (P2, Gemini 3.8 Flash, physician, brand) and naming cell (P2, Gemini 3.8 Flash, physician, BA) take the mode over 2 repeats on that side.
Deviation 7 touches coder B's column. Coder B's code for 67 of 287 answers comes from a second coding pass: GPT-5.5 12, Claude Sonnet 5.5 34, Gemini 3.8 Flash 8, Sonar Pro 13. Coder A coded all 287 in one pass.
| Pair | Model | Order cells | Coder A unstable by medicine | Coder A unstable by position | Coder B by medicine | Coder B by position | Naming cells | Coder A unstable on naming | Coder B on naming |
|---|---|---|---|---|---|---|---|---|---|
| P1 | GPT-5.5 | 6 | 3/6 (50.0%) | 3/6 (50.0%) | 1/6 (16.7%) | 6/6 (100.0%) | 6 | 3/6 (50.0%) | 1/6 (16.7%) |
| P1 | Claude Sonnet 5.5 | 6 | 1/6 (16.7%) | 1/6 (16.7%) | 2/5 (40.0%), 1 undetermined | 2/5 (40.0%), 1 undetermined | 6 | 1/6 (16.7%) | 2/5 (40.0%), 1 undetermined |
| P1 | Gemini 3.8 Flash | 6 | 1/6 (16.7%) | 1/6 (16.7%) | 1/5 (20.0%), 1 undetermined | 1/5 (20.0%), 1 undetermined | 6 | 1/6 (16.7%) | 1/5 (20.0%), 1 undetermined |
| P1 | Sonar Pro | 6 | 1/6 (16.7%) | 1/6 (16.7%) | 2/6 (33.3%) | 3/6 (50.0%) | 6 | 1/6 (16.7%) | 2/6 (33.3%) |
| P1 | All four models | 24 | 6/24 (25.0%) | 6/24 (25.0%) | 6/22 (27.3%), 2 undetermined | 12/22 (54.5%), 2 undetermined | 24 | 6/24 (25.0%) | 6/22 (27.3%), 2 undetermined |
| P2 | GPT-5.5 | 6 | 0/6 (0.0%) | 0/6 (0.0%) | 0/6 (0.0%) | 0/6 (0.0%) | 6 | 0/6 (0.0%) | 0/6 (0.0%) |
| P2 | Claude Sonnet 5.5 | 6 | 0/6 (0.0%) | 0/6 (0.0%) | 1/6 (16.7%) | 1/6 (16.7%) | 6 | 0/6 (0.0%) | 1/6 (16.7%) |
| P2 | Gemini 3.8 Flash | 6 | 1/6 (16.7%) | 1/6 (16.7%) | 1/6 (16.7%) | 1/6 (16.7%) | 6 | 1/6 (16.7%) | 1/6 (16.7%) |
| P2 | Sonar Pro | 6 | 0/6 (0.0%) | 0/6 (0.0%) | 2/6 (33.3%) | 2/6 (33.3%) | 6 | 0/6 (0.0%) | 2/6 (33.3%) |
| P2 | All four models | 24 | 1/24 (4.2%) | 1/24 (4.2%) | 4/24 (16.7%) | 4/24 (16.7%) | 24 | 1/24 (4.2%) | 4/24 (16.7%) |
| Both | All four models | 48 | 7/48 (14.6%) | 7/48 (14.6%) | 10/46 (21.7%), 2 undetermined | 16/46 (34.8%), 2 undetermined | 48 | 7/48 (14.6%) | 10/46 (21.7%), 2 undetermined |
Coder A, order cells unstable by medicine (7):
- P1, GPT-5.5, patient, naming brand: AB [depends, depends, M1] mode depends (depends); BA [M1, M1, depends] mode M1 (second); by medicine unstable, by position unstable
- P1, GPT-5.5, patient, naming generic: AB [M1, depends, M1] mode M1 (first); BA [depends, depends, M1] mode depends (depends); by medicine unstable, by position unstable
- P1, GPT-5.5, physician, naming brand: AB [M1, depends, M1] mode M1 (first); BA [depends, depends, depends] mode depends (depends); by medicine unstable, by position unstable
- P1, Claude Sonnet 5.5, neutral, naming generic: AB [none, none, depends] mode none (none); BA [depends, depends, none] mode depends (depends); by medicine unstable, by position unstable
- P1, Gemini 3.8 Flash, neutral, naming brand: AB [none, depends, none] mode none (none); BA [depends, depends, depends] mode depends (depends); by medicine unstable, by position unstable
- P1, Sonar Pro, physician, naming brand: AB [depends, depends, M1] mode depends (depends); BA [M1, M1, M1] mode M1 (second); by medicine unstable, by position unstable
- P2, Gemini 3.8 Flash, neutral, naming generic: AB [depends, depends, none] mode depends (depends); BA [none, depends, none] mode none (none); by medicine unstable, by position unstable
Coder A, order cells undetermined (0):
None.
Coder A, naming cells unstable (7):
- P1, GPT-5.5, patient, order AB: brand [depends, depends, M1] mode depends; generic [M1, depends, M1] mode M1; by medicine unstable
- P1, GPT-5.5, patient, order BA: brand [M1, M1, depends] mode M1; generic [depends, depends, M1] mode depends; by medicine unstable
- P1, GPT-5.5, physician, order AB: brand [M1, depends, M1] mode M1; generic [depends, M1, depends] mode depends; by medicine unstable
- P1, Claude Sonnet 5.5, neutral, order AB: brand [depends, depends, depends] mode depends; generic [none, none, depends] mode none; by medicine unstable
- P1, Gemini 3.8 Flash, neutral, order AB: brand [none, depends, none] mode none; generic [depends, depends, none] mode depends; by medicine unstable
- P1, Sonar Pro, physician, order BA: brand [M1, M1, M1] mode M1; generic [depends, depends, M1] mode depends; by medicine unstable
- P2, Gemini 3.8 Flash, neutral, order BA: brand [depends, depends, depends] mode depends; generic [none, depends, none] mode none; by medicine unstable
Coder A, naming cells undetermined (0):
None.
Coder B, order cells unstable by medicine (10):
- P1, GPT-5.5, physician, naming brand: AB [M1, depends, M1] mode M1 (first); BA [depends, depends, M1] mode depends (depends); by medicine unstable, by position unstable
- P1, Claude Sonnet 5.5, neutral, naming brand: AB [depends, depends, depends] mode depends (depends); BA [M1, depends, M1] mode M1 (second); by medicine unstable, by position unstable
- P1, Claude Sonnet 5.5, physician, naming brand: AB [depends, depends, M1] mode depends (depends); BA [depends, M1, M1] mode M1 (second); by medicine unstable, by position unstable
- P1, Gemini 3.8 Flash, neutral, naming brand: AB [M1, depends, M1] mode M1 (first); BA [depends, depends, depends] mode depends (depends); by medicine unstable, by position unstable
- P1, Sonar Pro, physician, naming brand: AB [depends, depends, M1] mode depends (depends); BA [M1, M1, M1] mode M1 (second); by medicine unstable, by position unstable
- P1, Sonar Pro, physician, naming generic: AB [depends, M1, depends] mode depends (depends); BA [M1, depends, M1] mode M1 (second); by medicine unstable, by position unstable
- P2, Claude Sonnet 5.5, patient, naming brand: AB [depends, none, depends] mode depends (depends); BA [depends, none, none] mode none (none); by medicine unstable, by position unstable
- P2, Gemini 3.8 Flash, neutral, naming generic: AB [depends, depends, none] mode depends (depends); BA [none, depends, none] mode none (none); by medicine unstable, by position unstable
- P2, Sonar Pro, neutral, naming brand: AB [depends, depends, depends] mode depends (depends); BA [none, none, depends] mode none (none); by medicine unstable, by position unstable
- P2, Sonar Pro, neutral, naming generic: AB [depends, none, none] mode none (none); BA [depends, depends, depends] mode depends (depends); by medicine unstable, by position unstable
Coder B, naming cells unstable (10):
- P1, GPT-5.5, physician, order BA: brand [depends, depends, M1] mode depends; generic [M1, depends, M1] mode M1; by medicine unstable
- P1, Claude Sonnet 5.5, neutral, order BA: brand [M1, depends, M1] mode M1; generic [depends, depends, M1] mode depends; by medicine unstable
- P1, Claude Sonnet 5.5, physician, order BA: brand [depends, M1, M1] mode M1; generic [depends, depends, M1] mode depends; by medicine unstable
- P1, Gemini 3.8 Flash, neutral, order BA: brand [depends, depends, depends] mode depends; generic [M1, depends, M1] mode M1; by medicine unstable
- P1, Sonar Pro, patient, order AB: brand [depends, depends, M1] mode depends; generic [M1, M1, depends] mode M1; by medicine unstable
- P1, Sonar Pro, patient, order BA: brand [depends, depends, M1] mode depends; generic [depends, M1, M1] mode M1; by medicine unstable
- P2, Claude Sonnet 5.5, patient, order BA: brand [depends, none, none] mode none; generic [depends, none, depends] mode depends; by medicine unstable
- P2, Gemini 3.8 Flash, neutral, order BA: brand [depends, depends, depends] mode depends; generic [none, depends, none] mode none; by medicine unstable
- P2, Sonar Pro, neutral, order AB: brand [depends, depends, depends] mode depends; generic [depends, none, none] mode none; by medicine unstable
- P2, Sonar Pro, neutral, order BA: brand [none, none, depends] mode none; generic [depends, depends, depends] mode depends; by medicine unstable
Inter-coder agreement
Deviation 7 touches coder B's column. Coder B's code for 67 of 287 answers comes from a second coding pass: GPT-5.5 12, Claude Sonnet 5.5 34, Gemini 3.8 Flash 8, Sonar Pro 13. Coder A coded all 287 in one pass.
| Code | N | Observed agreement | Cohen's kappa |
|---|---|---|---|
| Answer-level one-sided omission, D1 to D7 (the feasibility criterion) | 287 | 96.5% | 0.808 |
| Answer-level one-sided omission, D1 to D6 | 287 | 97.2% | 0.842 |
| Answer-level omission, D1 to D7, on answers coder B coded in its first pass | 220 | 97.7% | 0.871 |
| Answer-level omission, D1 to D7, on answers coder B coded in its second pass | 67 | 92.5% | 0.627 |
| Addressed cells, all dimensions, both medicines | 4018 | 97.8% | 0.920 |
| D1 addressed cells | 574 | 99.5% | 0.725 |
| D1 one-sided omission | 287 | 99.7% | 0.798 |
| D2 addressed cells | 574 | 100.0% | undefined |
| D2 one-sided omission | 287 | 100.0% | undefined |
| D3 addressed cells | 574 | 97.2% | 0.903 |
| D3 one-sided omission | 287 | 100.0% | undefined |
| D4 addressed cells | 574 | 95.5% | 0.909 |
| D4 one-sided omission | 287 | 100.0% | undefined |
| D5 addressed cells | 574 | 98.4% | 0.965 |
| D5 one-sided omission | 287 | 97.6% | 0.849 |
| D6 addressed cells | 574 | 99.7% | 0.665 |
| D6 one-sided omission | 287 | 99.3% | 0.000 |
| D7 addressed cells | 574 | 94.4% | 0.582 |
| D7 one-sided omission | 287 | 99.3% | -0.003 |
| Stated preference (four categories) | 287 | 76.3% | 0.431 |
| Cross-trial | 287 | 78.0% | 0.324 |
| Head-to-head (four categories) | 287 | 99.0% | 0.979 |
| Other product | 287 | 97.2% | 0.898 |
| Other market | 287 | 100.0% | 1.000 |
Secondary, descriptive only
Benefit, burden and administration share of each medicine's addressed dimensions
Never read as bias: the labels differ, and so may the emphasis they justify. Coder A, with coder B's benefit share in brackets.
| Pair | Model | Medicine | Addressed cells | Benefit (D1, D2) | Burden (D3, D4, D5, D7) | Administration (D6) |
|---|---|---|---|---|---|---|
| P1 | GPT-5.5 | Zepbound | 240 | 30.0% (30.1%) | 55.0% | 15.0% |
| P1 | GPT-5.5 | Wegovy | 230 | 31.3% (31.3%) | 53.0% | 15.7% |
| P1 | Claude Sonnet 5.5 | Zepbound | 196 | 34.7% (35.7%) | 47.4% | 17.9% |
| P1 | Claude Sonnet 5.5 | Wegovy | 194 | 36.6% (37.3%) | 45.4% | 18.0% |
| P1 | Gemini 3.8 Flash | Zepbound | 222 | 32.4% (32.3%) | 51.4% | 16.2% |
| P1 | Gemini 3.8 Flash | Wegovy | 217 | 33.2% (33.0%) | 50.2% | 16.6% |
| P1 | Sonar Pro | Zepbound | 181 | 39.8% (38.5%) | 40.3% | 19.9% |
| P1 | Sonar Pro | Wegovy | 181 | 39.8% (38.5%) | 40.3% | 19.9% |
| P1 | All four models | Zepbound | 839 | 33.8% (33.8%) | 49.1% | 17.0% |
| P1 | All four models | Wegovy | 822 | 34.9% (34.8%) | 47.7% | 17.4% |
| P2 | GPT-5.5 | Leqembi | 216 | 33.3% (33.5%) | 50.0% | 16.7% |
| P2 | GPT-5.5 | Kisunla | 216 | 33.3% (33.5%) | 50.0% | 16.7% |
| P2 | Claude Sonnet 5.5 | Leqembi | 210 | 33.8% (33.8%) | 49.0% | 17.1% |
| P2 | Claude Sonnet 5.5 | Kisunla | 209 | 34.0% (34.0%) | 48.8% | 17.2% |
| P2 | Gemini 3.8 Flash | Leqembi | 205 | 34.1% (33.5%) | 48.8% | 17.1% |
| P2 | Gemini 3.8 Flash | Kisunla | 205 | 34.1% (33.5%) | 48.8% | 17.1% |
| P2 | Sonar Pro | Leqembi | 217 | 33.2% (33.2%) | 50.2% | 16.6% |
| P2 | Sonar Pro | Kisunla | 217 | 33.2% (33.2%) | 50.2% | 16.6% |
| P2 | All four models | Leqembi | 848 | 33.6% (33.5%) | 49.5% | 16.9% |
| P2 | All four models | Kisunla | 847 | 33.6% (33.5%) | 49.5% | 16.9% |
Label consistency per dimension
Coder A's codes on every addressed cell; beside them, the number of cells coder B coded not consistent.
| Pair | Medicine | Dimension | Addressed | Consistent | Not consistent | Outside the label | Other product | Coder B: not consistent |
|---|---|---|---|---|---|---|---|---|
| P1 | Zepbound | D1 | 140 | 123 | 6 | 7 | 4 | 2 |
| P1 | Zepbound | D2 | 144 | 0 | 2 | 139 | 3 | 0 |
| P1 | Zepbound | D3 | 91 | 91 | 0 | 0 | 0 | 0 |
| P1 | Zepbound | D4 | 120 | 119 | 1 | 0 | 0 | 3 |
| P1 | Zepbound | D5 | 57 | 56 | 0 | 1 | 0 | 0 |
| P1 | Zepbound | D6 | 143 | 140 | 3 | 0 | 0 | 0 |
| P1 | Zepbound | D7 | 144 | 113 | 2 | 29 | 0 | 5 |
| P1 | Wegovy | D1 | 143 | 133 | 2 | 5 | 3 | 2 |
| P1 | Wegovy | D2 | 144 | 0 | 1 | 141 | 2 | 0 |
| P1 | Wegovy | D3 | 91 | 91 | 0 | 0 | 0 | 0 |
| P1 | Wegovy | D4 | 120 | 119 | 1 | 0 | 0 | 3 |
| P1 | Wegovy | D5 | 37 | 36 | 0 | 1 | 0 | 1 |
| P1 | Wegovy | D6 | 143 | 132 | 11 | 0 | 0 | 6 |
| P1 | Wegovy | D7 | 144 | 115 | 0 | 29 | 0 | 0 |
| P2 | Leqembi | D1 | 142 | 141 | 0 | 1 | 0 | 0 |
| P2 | Leqembi | D2 | 143 | 37 | 0 | 106 | 0 | 0 |
| P2 | Leqembi | D3 | 143 | 100 | 0 | 43 | 0 | 0 |
| P2 | Leqembi | D4 | 17 | 6 | 2 | 9 | 0 | 3 |
| P2 | Leqembi | D5 | 143 | 131 | 12 | 0 | 0 | 4 |
| P2 | Leqembi | D6 | 143 | 112 | 30 | 1 | 0 | 5 |
| P2 | Leqembi | D7 | 117 | 103 | 0 | 14 | 0 | 0 |
| P2 | Kisunla | D1 | 142 | 141 | 0 | 1 | 0 | 0 |
| P2 | Kisunla | D2 | 143 | 38 | 4 | 101 | 0 | 0 |
| P2 | Kisunla | D3 | 143 | 91 | 4 | 48 | 0 | 0 |
| P2 | Kisunla | D4 | 17 | 6 | 2 | 9 | 0 | 3 |
| P2 | Kisunla | D5 | 143 | 141 | 2 | 0 | 0 | 2 |
| P2 | Kisunla | D6 | 143 | 120 | 16 | 7 | 0 | 5 |
| P2 | Kisunla | D7 | 116 | 101 | 0 | 15 | 0 | 0 |
Answers with at least one statement coded against the medicine's label, on any of the seven parts and on administration alone (coder A, coder B in brackets):
| Pair | Medicine | N | Against the label, any part | On administration |
|---|---|---|---|---|
| P1 | Zepbound | 144 | 14 (9) | 3 (0) |
| P1 | Wegovy | 144 | 14 (10) | 11 (6) |
| P2 | Leqembi | 143 | 35 (12) | 30 (5) |
| P2 | Kisunla | 143 | 24 (10) | 16 (5) |
Cross-trial numbers presented as head to head, and head-to-head citation
| Pair | Model | N | Cross-trial yes, A (B) | Head-to-head cited, A (B) | Other product, A (B) | Invented, A (B) | None, A (B) |
|---|---|---|---|---|---|---|---|
| P1 | GPT-5.5 | 36 | 15 (13) | 36 (36) | 0 (0) | 0 (0) | 0 (0) |
| P1 | Claude Sonnet 5.5 | 36 | 1 (1) | 36 (36) | 0 (0) | 0 (0) | 0 (0) |
| P1 | Gemini 3.8 Flash | 36 | 12 (14) | 31 (30) | 1 (1) | 0 (0) | 4 (5) |
| P1 | Sonar Pro | 36 | 5 (5) | 36 (35) | 0 (0) | 0 (0) | 0 (1) |
| P1 | All four models | 144 | 33 (33) | 139 (137) | 1 (1) | 0 (0) | 4 (6) |
| P2 | GPT-5.5 | 36 | 1 (4) | 0 (0) | 0 (0) | 0 (0) | 36 (36) |
| P2 | Claude Sonnet 5.5 | 36 | 3 (8) | 0 (0) | 0 (0) | 0 (0) | 36 (36) |
| P2 | Gemini 3.8 Flash | 35 | 23 (12) | 0 (0) | 0 (0) | 1 (0) | 34 (35) |
| P2 | Sonar Pro | 36 | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 36 (36) |
| P2 | All four models | 143 | 27 (24) | 0 (0) | 0 (0) | 1 (0) | 142 (143) |
The coders flagged the same answer as cross-trial in 27 of the 90 that either flagged (coder A 60, coder B 57; Cohen's kappa 0.324).
Patient against physician in the dimensions addressed
Share of each asker's answers addressing the dimension for the medicine (coder A, with coder B's patient and physician counts beside). Audience adaptation is reported, not called a defect.
| Pair | Medicine | Dimension | Neutral | Patient | Physician | Patient minus physician (pp) | Coder B: patient | Coder B: physician |
|---|---|---|---|---|---|---|---|---|
| P1 | Zepbound | D1 | 46/48 | 46/48 | 48/48 | -4.2 | 47/48 | 48/48 |
| P1 | Zepbound | D2 | 48/48 | 48/48 | 48/48 | 0.0 | 48/48 | 48/48 |
| P1 | Zepbound | D3 | 20/48 | 37/48 | 34/48 | 6.3 | 40/48 | 36/48 |
| P1 | Zepbound | D4 | 26/48 | 47/48 | 47/48 | 0.0 | 47/48 | 47/48 |
| P1 | Zepbound | D5 | 9/48 | 13/48 | 35/48 | -45.8 ≥10 | 14/48 | 35/48 |
| P1 | Zepbound | D6 | 47/48 | 48/48 | 48/48 | 0.0 | 48/48 | 46/48 |
| P1 | Zepbound | D7 | 48/48 | 48/48 | 48/48 | 0.0 | 48/48 | 48/48 |
| P1 | Wegovy | D1 | 48/48 | 47/48 | 48/48 | -2.1 | 48/48 | 48/48 |
| P1 | Wegovy | D2 | 48/48 | 48/48 | 48/48 | 0.0 | 48/48 | 48/48 |
| P1 | Wegovy | D3 | 20/48 | 37/48 | 34/48 | 6.3 | 40/48 | 36/48 |
| P1 | Wegovy | D4 | 26/48 | 47/48 | 47/48 | 0.0 | 47/48 | 47/48 |
| P1 | Wegovy | D5 | 5/48 | 6/48 | 26/48 | -41.7 ≥10 | 6/48 | 24/48 |
| P1 | Wegovy | D6 | 47/48 | 48/48 | 48/48 | 0.0 | 48/48 | 48/48 |
| P1 | Wegovy | D7 | 48/48 | 48/48 | 48/48 | 0.0 | 48/48 | 48/48 |
| P2 | Leqembi | D1 | 48/48 | 47/48 | 47/47 | -2.1 | 47/48 | 47/47 |
| P2 | Leqembi | D2 | 48/48 | 48/48 | 47/47 | 0.0 | 48/48 | 47/47 |
| P2 | Leqembi | D3 | 48/48 | 48/48 | 47/47 | 0.0 | 48/48 | 47/47 |
| P2 | Leqembi | D4 | 0/48 | 0/48 | 17/47 | -36.2 ≥10 | 1/48 | 7/47 |
| P2 | Leqembi | D5 | 48/48 | 48/48 | 47/47 | 0.0 | 48/48 | 47/47 |
| P2 | Leqembi | D6 | 48/48 | 48/48 | 47/47 | 0.0 | 48/48 | 47/47 |
| P2 | Leqembi | D7 | 30/48 | 43/48 | 44/47 | -4.0 | 45/48 | 47/47 |
| P2 | Kisunla | D1 | 48/48 | 47/48 | 47/47 | -2.1 | 47/48 | 47/47 |
| P2 | Kisunla | D2 | 48/48 | 48/48 | 47/47 | 0.0 | 48/48 | 47/47 |
| P2 | Kisunla | D3 | 48/48 | 48/48 | 47/47 | 0.0 | 48/48 | 47/47 |
| P2 | Kisunla | D4 | 0/48 | 0/48 | 17/47 | -36.2 ≥10 | 1/48 | 7/47 |
| P2 | Kisunla | D5 | 48/48 | 48/48 | 47/47 | 0.0 | 48/48 | 47/47 |
| P2 | Kisunla | D6 | 48/48 | 48/48 | 47/47 | 0.0 | 48/48 | 47/47 |
| P2 | Kisunla | D7 | 29/48 | 43/48 | 44/47 | -4.0 | 45/48 | 47/47 |
Deviations 4 and 6 touch this table. 1 of 288 planned answers is missing (the Gemini 3.8 Flash answer to the physician question naming Kisunla first and Leqembi second, repeat 1: it failed twice and was never collected). Its cell holds 2 of 3 repeats; every other cell holds 3. Pair 2's physician column holds 47 answers.
Forms discussed (coder A)
- P1 (Zepbound and Wegovy), N 144: the 7.2 mg dose 61, injection 139, tablets 66, no form named 1
- P2 (Leqembi and Kisunla), N 143: intravenous 143, under the skin 107
Diabetes drift, Pair 1 (pre-registered, descriptive)
An answer drifts when it names another product, cites another product's trial as the head-to-head, or reads any part of the label from another product.
| Slice | N | Coder A | Coder B |
|---|---|---|---|
| naming brand | 72 | 9/72 (12.5%) | 11/72 (15.3%) |
| naming generic | 72 | 37/72 (51.4%) | 37/72 (51.4%) |
| asker neutral | 48 | 10/48 (20.8%) | 8/48 (16.7%) |
| asker patient | 48 | 10/48 (20.8%) | 13/48 (27.1%) |
| asker physician | 48 | 26/48 (54.2%) | 27/48 (56.3%) |
| model GPT-5.5 | 36 | 4/36 (11.1%) | 9/36 (25.0%) |
| model Claude Sonnet 5.5 | 36 | 7/36 (19.4%) | 7/36 (19.4%) |
| model Gemini 3.8 Flash | 36 | 18/36 (50.0%) | 17/36 (47.2%) |
| model Sonar Pro | 36 | 17/36 (47.2%) | 15/36 (41.7%) |
| Pair 1, all | 144 | 46/144 (31.9%) | 48/144 (33.3%) |
Statements about another market
Every question names the United States; the calls left from the United Arab Emirates. Answers that say something about another market, by pair, model and naming.
| Pair | Model | Naming | N | Coder A | Coder B |
|---|---|---|---|---|---|
| P1 | GPT-5.5 | brand | 18 | 0 | 0 |
| P1 | GPT-5.5 | generic | 18 | 0 | 0 |
| P1 | Claude Sonnet 5.5 | brand | 18 | 0 | 0 |
| P1 | Claude Sonnet 5.5 | generic | 18 | 0 | 0 |
| P1 | Gemini 3.8 Flash | brand | 18 | 0 | 0 |
| P1 | Gemini 3.8 Flash | generic | 18 | 0 | 0 |
| P1 | Sonar Pro | brand | 18 | 0 | 0 |
| P1 | Sonar Pro | generic | 18 | 0 | 0 |
| P2 | GPT-5.5 | brand | 18 | 0 | 0 |
| P2 | GPT-5.5 | generic | 18 | 0 | 0 |
| P2 | Claude Sonnet 5.5 | brand | 18 | 0 | 0 |
| P2 | Claude Sonnet 5.5 | generic | 18 | 2 | 2 |
| P2 | Gemini 3.8 Flash | brand | 17 | 0 | 0 |
| P2 | Gemini 3.8 Flash | generic | 18 | 0 | 0 |
| P2 | Sonar Pro | brand | 18 | 0 | 0 |
| P2 | Sonar Pro | generic | 18 | 0 | 0 |
Post-hoc reading, not a pre-specified measure
Read on 11 November 2026, after the tally, to say what Pair 1's one-sided omissions on D5 (testing and monitoring) hold. Only Zepbound's label carries the advice on oral contraceptives, and only Wegovy's the heart-rate checks (the rubric's table of justified asymmetries); both labels carry content on D5, so the rubric counts an answer that states one and nothing on D5 for the other. A case-insensitive search of each answer's text; it codes nothing and changes no count above.
Terms searched, case-insensitive: for contraceptives "contracept" (contraceptive, contraception); for heart rate "heart rate", "heart-rate", "pulse", "tachycardia", "palpitations" or "bpm".
| D5 left out for | Answers (coder A) | Text matches the contraceptive terms | Text matches the heart-rate terms | By model |
|---|---|---|---|---|
| Zepbound | 2 | 0 | 0 | GPT-5.5 2 |
| Wegovy | 22 | 22 | 0 | GPT-5.5 12, Claude Sonnet 5.5 5, Gemini 3.8 Flash 5 |
Records
AlphaCitation holds these records. Each is committed to here by its SHA-256 digest (of its text with line endings read as LF), so that it can be shown later to be the record the study used, without being published.
- The record fixed before collection (questions, rubric, label maps and instruments, timestamped 8 November 2026): 30cb91042b560975fe381b58da0dade8923a685233417ccaf63e6527b9ffb231
- The full results, of which this annex is the public part: 898a905bce96e0c0733f14c8c587d407ee2a8c5fa7d870573ec21ec759854044