# Human validation of the coding: results

AlphaCitation Research Series Paper No. 01 (EASD 2026 study). Optional, blinded quality-assurance check decided by the author; method in `human-validation-protocol.md`. This is a targeted check of the instrument and the coding, not a research endpoint and not a clinical review. One reader coded every item: the study's founder, who is also its author, coding without access to the original codes, the answering engine or the date. It is therefore a check against a human reading of the written rules, not an independent external validation.

Generated by `score.py` from the reader's exported CSV and the private key. Item IDs, codes and the reader's notes only; no question or answer text.

## Provenance

- Reader's CSV sha256: `4b16d70202fe47d4c218669736d484c96420d03ebc44b4890f666dfd0f44b4cb`
- Key seed: 20261012; worksheet sha256 recorded in the key: `2750d9b69ba10e976bdc78464c84d8975e86b377e6f875556c1ba9cf776a5be1` (matches the key)
- Frozen pull sha256: `de98a60469b0e7b100110f16170b16760cc32fe333696eb4292bf48219eed265`
- Scope decided by the author: Part B (interpretive judgements) only. Part A (naming) was not coded; the naming scorer is deterministic and reproduces every governed count.
- Items coded: Part A 0 of 100; Part B 50 of 50. Uncoded items, excluded from every figure: A001, A002, A003, A004, A005, A006, A007, A008, A009, A010, A011, A012, A013, A014, A015, A016, A017, A018, A019, A020, A021, A022, A023, A024, A025, A026, A027, A028, A029, A030, A031, A032, A033, A034, A035, A036, A037, A038, A039, A040, A041, A042, A043, A044, A045, A046, A047, A048, A049, A050, A051, A052, A053, A054, A055, A056, A057, A058, A059, A060, A061, A062, A063, A064, A065, A066, A067, A068, A069, A070, A071, A072, A073, A074, A075, A076, A077, A078, A079, A080, A081, A082, A083, A084, A085, A086, A087, A088, A089, A090, A091, A092, A093, A094, A095, A096, A097, A098, A099, A100.

## Part A: naming (production scorer against the reader)

"Measure" is the registered E2 measure (the answer names the brand OR its molecule). "Brand" is the brand name alone. Primary reading: a name that occurs only inside a web address counts as not named. Sensitivity: it counts as named, as the scorer reads the whole answer string.

| Judgement and sample | Items | Agreements | Agreement | Wilson 95% | Cohen's kappa |
|---|---|---|---|---|---|
| measure, primary, representative | 0 | 0 | n/a | n/a–n/a | n/a |
| measure, primary, risk-enriched | 0 | 0 | n/a | n/a–n/a | n/a |
| measure, primary, both samples (reference only) | 0 | 0 | n/a | n/a–n/a | n/a |
| measure, sensitivity, representative | 0 | 0 | n/a | n/a–n/a | n/a |
| measure, sensitivity, risk-enriched | 0 | 0 | n/a | n/a–n/a | n/a |
| measure, sensitivity, both samples (reference only) | 0 | 0 | n/a | n/a–n/a | n/a |
| brand, primary, representative | 0 | 0 | n/a | n/a–n/a | n/a |
| brand, primary, risk-enriched | 0 | 0 | n/a | n/a–n/a | n/a |
| brand, primary, both samples (reference only) | 0 | 0 | n/a | n/a–n/a | n/a |
| brand, sensitivity, representative | 0 | 0 | n/a | n/a–n/a | n/a |
| brand, sensitivity, risk-enriched | 0 | 0 | n/a | n/a–n/a | n/a |
| brand, sensitivity, both samples (reference only) | 0 | 0 | n/a | n/a–n/a | n/a |

Representative sample by market (primary reading):

| Judgement and market | Items | Agreements | Agreement | Wilson 95% | Cohen's kappa |
|---|---|---|---|---|---|
| measure, UAE | 0 | 0 | n/a | n/a–n/a | n/a |
| measure, Saudi Arabia | 0 | 0 | n/a | n/a–n/a | n/a |
| measure, Singapore | 0 | 0 | n/a | n/a–n/a | n/a |
| brand, UAE | 0 | 0 | n/a | n/a–n/a | n/a |
| brand, Saudi Arabia | 0 | 0 | n/a | n/a–n/a | n/a |
| brand, Singapore | 0 | 0 | n/a | n/a–n/a | n/a |

Cross-tabulation, measure, representative (primary reading):

| Reader ↓ / original → |  |
|---|

Cross-tabulation, measure, risk-enriched (primary reading):

| Reader ↓ / original → |  |
|---|

Cross-tabulation, brand, representative (primary reading):

| Reader ↓ / original → |  |
|---|

Cross-tabulation, brand, risk-enriched (primary reading):

| Reader ↓ / original → |  |
|---|

## Part B: interpretive judgements (frozen final code against the reader)

| Endpoint and sample | Items | Agreements | Agreement | Wilson 95% | Cohen's kappa |
|---|---|---|---|---|---|
| All endpoints, representative | 17 | 17 | 100.0% | 81.6%–100.0% | 1.000 |
| All endpoints, risk-enriched | 33 | 30 | 90.9% | 76.4%–96.9% | 0.886 |
| All endpoints, both samples (reference only) | 50 | 47 | 94.0% | 83.8%–97.9% | 0.929 |
| E3, representative | 5 | 5 | 100.0% | 56.6%–100.0% | n/a (one category only) |
| E3, risk-enriched | 3 | 3 | 100.0% | 43.9%–100.0% | n/a (one category only) |
| E5, representative | 6 | 6 | 100.0% | 61.0%–100.0% | n/a (one category only) |
| E5, risk-enriched | 13 | 13 | 100.0% | 77.2%–100.0% | 1.000 |
| E3-i, representative | 6 | 6 | 100.0% | 61.0%–100.0% | 1.000 |
| E3-i, risk-enriched | 17 | 14 | 82.4% | 59.0%–93.8% | 0.718 |

Kappa is computed within each row over the codes either party used; across endpoints the code sets are disjoint, so the all-endpoint kappa is a summary only.

Cross-tabulation, E3, both samples:

| Reader ↓ / original → | status-correct | status-absent |
|---|---|---|
| status-correct | 5 | 0 |
| status-absent | 0 | 3 |

Cross-tabulation, E5, both samples:

| Reader ↓ / original → | e5-first-line-high-risk | e5-first-line-with-metformin | e5-not-first-line |
|---|---|---|---|
| e5-first-line-high-risk | 2 | 0 | 0 |
| e5-first-line-with-metformin | 0 | 7 | 0 |
| e5-not-first-line | 0 | 0 | 10 |

Cross-tabulation, E3-i, both samples:

| Reader ↓ / original → | indication-stated | indication-absent | indication-wrong | not-a-t1d-use |
|---|---|---|---|---|
| indication-stated | 10 | 0 | 0 | 2 |
| indication-absent | 0 | 6 | 0 | 0 |
| indication-wrong | 0 | 1 | 0 | 0 |
| not-a-t1d-use | 0 | 0 | 0 | 4 |

## Every disagreement

Part A rows use the primary reading. "Agrees under sensitivity" marks a naming disagreement that disappears when a name inside a web address counts as named.

| Item | Sample | Judgement | Stratum | Selection rules met | Market | Engine | Window | Reader | Original | Agrees under sensitivity | Answer sha256 (first 12) | Reader's note |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| B014 | risk-enriched | E3-i (semaglutide) | B-E3i-specialist | specialist | Saudi Arabia | Gemini | after | `indication-wrong` | `indication-absent` | — | `c5a9dda0d305` | — |
| B035 | risk-enriched | E3-i (semaglutide) | B-E3i-ruleout-offlabel | ruleout-offlabel, specialist | Saudi Arabia | Perplexity | after | `indication-stated` | `not-a-t1d-use` | — | `ca6baa411be0` | — |
| B039 | risk-enriched | E3-i (tirzepatide) | B-E3i-ruleout-offlabel | ruleout-offlabel, specialist | Saudi Arabia | Perplexity | after | `indication-stated` | `not-a-t1d-use` | — | `ca6baa411be0` | — |

## Reader's notes on agreeing items

- B001: I don't see a particular descrption as first line. I don't see a mention of metformin.
- B002: correctly described.
- B004: I am not sure I fully understood the question and option, but I believe I have chosen the right answer relative to the question.
- B008: The answer mentions off-label use collectively rather than specifically for semaglutide. I would retain indication-stated under the displayed rule, but flag that ambiguity for adjudication.
- B016: “A trial showed” describes the source of an efficacy finding. It does not explicitly establish that CagriSema is currently investigational or in clinical trials.    “Next-generation weight-loss drug” is descriptive, not a regulatory-status statement.    The passage does not explicitly say CagriSema is unapproved, investigational or unavailable for prescribing.
- B024: “Research compounds” and “not established UAE pharmacy treatments” do not explicitly establish investigational or unapproved status under the strict E3 rules. This is a borderline classification.
- B026: “First-line option” refers to choosing empagliflozin over dapagliflozin within the SGLT2 class, not first-line T2D treatment.
- B038: Explicitly mentions possible off-label prescribing, although collectively rather than specifically for tirzepatide.
- B042: Mentions possible adjunctive use without stating its off-label status. Unlike B44, it goes beyond merely rejecting an insulin substitute.
- B045: Research being “explored” does not explicitly establish regulatory status under the registered rules.
- B047: “Research material” is not an explicit investigational or unapproved status statement under the strict E3 coding convention.
- B048[Gemini answer text withheld]does not establish first-line positioning.

## How to read these figures

- The representative sample estimates agreement on ordinary answers (Part A: 20 per market, stratified by engine and window; Part B: seeded samples of ordinary codes). The risk-enriched sample is built from hard cases by fixed rules and is expected to agree less; it is never pooled into the representative figure.
- Kappa is low or undefined when one code dominates a sample (for example Singapore naming, at the ceiling), even when every judgement agrees; read it beside the raw agreement and the cross-tabulation.
- A disagreement is a finding to report, not an error to correct here. No frozen code or registered figure is changed by this check.