# Pre-registration — PPN World Research Paper No. 3
## Do drug-trial press releases report the outcome the trial registered?

**Registered: 2026-11-11, approved by the author.** This file, the code it
names (§6, §9), the pilot's two agreement tables and a manifest of their
SHA-256 hashes (`paper-3-SHA256SUMS.txt`) are committed together, and the
manifest is timestamped with OpenTimestamps (`paper-3-SHA256SUMS.txt.ots`), as
Paper No. 2's was. No release in the frame (§2) is read, and no frame release
is coded, before that commit.

**What had been looked at before registration** (feasibility only):

- **a count**: on 2026-11-09 a script searched the 204,440 saved pages of Paper
  No. 2's census (§2) for the pattern `NCT0` + 7 digits and counted 774 pages
  and 1,324 identifiers. It printed counts only; no page text, headline or
  identifier list was read;
- **the pilot**: 823 English releases published 2026-10-07 → 11-08, *after*
  the frame's window, so none can be in the frame, found with nine trial words
  in the archive's stored headlines and summaries, fetched 2026-11-09 by
  Paper No. 2's fetcher. 668 pages were readable; 199 named a ClinicalTrials.gov
  identifier (313 records looked up). The author's assistant (Claude Opus 5.5)
  coded 10 of them by hand with the draft codebook. **Their outcomes were
  seen**: one release called a different outcome "primary" than the one
  registered, one failed Phase 3 trial's release led with secondary and
  subgroup results, one release said a Phase 3 trial met its primary endpoint
  and gave no number, and four early-phase releases led with efficacy while
  the registered primary outcome was safety. The pilot changed the design in
  three ways, each disclosed: the main analysis is restricted to randomized
  trials with an efficacy primary outcome (§3); releases not issued by the
  trial's side are excluded (§3); the registry record is taken as it stood on
  the release date, not as it stands today (§5), because 124 of the 312 pilot
  records had been edited since the pilot window opened.
- **pilot coding**: two model coders then coded all 199 pilot releases with
  codebook version 1 (2026-11-11) — Claude Sonnet 5.5 and, in that first
  round only, OpenAI's GPT-5.5; the owner then ruled Claude only, so coder B
  is Claude Opus 5.5 (§6). Agreement was high on who issued the release,
  results or not, the primary outcome's type and "not met" (α .89–1.00) but
  below .667 on the verdict, the headline and the spin questions, where
  categories overlapped. Version 2 — the registered codebook — merged or made
  binary those questions, and the pilot was coded again with it by the
  registered coders (round 2: α .98 verdict stated, 1.00 not met, .68 headline,
  .66 any spin, .69–.75 each spin type). Both rounds' agreement tables are
  deposited with this file.
- **the pilot's results were seen.** The registered analysis code (§9) was run
  on the pilot's round-2 codings to test it. On the pilot's 92 randomized,
  efficacy results pairs: the release said whether the registered primary
  outcome was met in 65%; the headline stated its result in 17%; one release
  called another outcome primary; 7 pairs were "not met"; spin appeared in 33
  of the 40 pairs without a met verdict; a number accompanied 81% of "met"
  verdicts. **H1 did not hold in the pilot** (−3.6 points, 90% interval −37.5
  to 28.0, on 7 not-met pairs); H2 could not be computed (no sponsor sizes).
  H1 and H2 were written before any pilot coding; after round 1, H1's measure
  changed from "the headline leads with the registered primary outcome" to
  "the headline states its result" (codebook version 2). Both are kept as
  written here.

### 1. Question and prior work

When a company, university or hospital announces a clinical trial's results in
a press release, does the release report the outcome the trial registered as
primary, does it say whether that outcome was met, and does it present another
outcome as the primary one?

Spin in trial press releases was measured by Yavchitz et al. (2012, PLOS
Medicine): 70 EurekAlert releases on randomized trials, compared with the
published article; spin in 47%. Purohit et al. (2023, J Gen Intern Med) started
from 988 drug trials published in NEJM, JAMA and The Lancet and found 172
releases issued before publication. Spin in published articles is well
documented (Boutron et al., 2010; Chiu et al., 2017), and so are differences
between registered and published primary outcomes (Goldacre et al., 2019,
COMPare). We found no study that compares press releases with the registry
record. Releases matter because they are, for most trials, the first and often
only public account of the result, and they move share prices before any peer
review (Purohit et al., 2023).

### 2. Population (the frame)

Every release **measured** in Paper No. 2's census — English releases published
2026-07-01 → 2026-10-06 in PPN World's archive, whose page answered Paper
No. 2's fetcher on 2026-10-11 → 10-13 with at least 50 words of English
carrying the archive's headline — **whose saved page contains at least one
ClinicalTrials.gov identifier** (`NCT` + 8 digits). The pages are not fetched
again: the frame is the copy saved then (gzipped HTML), so every release is
read as it stood 5 days to 3½ months after publication.

What the frame does not cover, stated in the paper: releases in other
languages; releases that name a trial only by acronym or by another registry
(EU CTR, ISRCTN, ChiCTR); the census's losses (pages walled, gone or behind an
invalid certificate — Paper No. 2, §4); trials that issued no release. The
paper describes press releases that cite a ClinicalTrials.gov record, not all
trial announcements.

### 3. Units, inclusion and the main analysis set

- **Release–trial pair**: a frame release and one identifier on its page.
- **Results pair** (codebook Q1): the release reports any efficacy or safety
  finding of that trial. Other pairs (a trial named in a pipeline list, an
  enrolment announcement) are counted and set aside.
- **Excluded releases** (codebook Q0): releases not issued by the trial's side
  — the lead sponsor, a collaborator or licensing partner, or an investigator's
  institution. A news-style rewrite of another organization's release, a law
  firm's investor alert and an analyst note are excluded.
- **Main analysis set**: results pairs whose trial is **randomized**
  (registry `allocation = RANDOMIZED`) and has **at least one efficacy primary
  outcome** (codebook R1: an outcome that measures benefit, not only safety,
  tolerability, pharmacokinetics or dose finding).
- **Early-phase set**, reported separately: results pairs whose registered
  primary outcomes are all safety, tolerability, pharmacokinetic or
  dose-finding outcomes.
- Pairs whose registry record cannot be retrieved are counted and set aside.

### 4. Census

Every release in the frame and every identifier on its page are coded; no
sampling, no weights.

### 5. The registry record

For each identifier, the record **in force on the release's publication date**:
the outcomes, design, phase and lead sponsor in the AACT monthly snapshot
(Clinical Trials Transformative Initiative's copy of ClinicalTrials.gov) taken
on or before that date, and in the next snapshot after it. The snapshots are
the flat-file exports of 2026-07-01, 08-01, 09-01, 10-01 and 11-01 (AACT ids
4243, 4297, 4359, 4419, 4481), downloaded 2026-11-11 before registration and
read before it only for the pilot's trials (to test the code). If the registered
primary outcomes are the same in both, that is the record. If they differ, the
pair is flagged *registry changed around the release*; it is analysed with the
earlier record and also reported on its own.

**If the AACT snapshots cannot be obtained**, the record is the current one
(ClinicalTrials.gov API v2, fetched once), and every pair whose record was
last updated on or after the release date is flagged *registry changed around
the release* — a superset of the pairs whose outcomes changed. The paper then
says which source was used.

Sponsor size: the number of studies the lead sponsor has registered (in the
same AACT snapshot; else a ClinicalTrials.gov API count on the day of the
registry pull). **Small sponsor** < 50 registered studies; **large** ≥ 50.

### 6. Measures and coders

The codebook (`scripts/research/trials/codebook.md`, version 2, registered
with this file) defines R1 and Q0–Q10. Two Claude models code independently
(the author's company uses Claude only), each given the codebook, the release
text (Mozilla Readability's reader view, as in Paper No. 2) and the registry
record (§5), never the publisher's name or website:

- **Coder A — the measurement**: Claude Sonnet 5.5 (`claude-sonnet-5-5`)
  codes every release in the frame;
- **Coder B — the reliability check**: Claude Opus 5.5 (`claude-opus-5-5`)
  codes a random 150 releases of the frame (seeded shuffle, seed 20261111,
  in `5-code.mjs`) — enough to estimate agreement, at a fraction of the cost;

one request per release covering every trial it names, sent through
Anthropic's Message Batches API with the codebook as a cached system prompt,
the answer constrained to a JSON schema, each answer with the release's own
words as evidence. Both
models run at their default sampling (neither accepts a temperature setting)
with reasoning at "medium" effort. A refusal is recorded as a missing coding
and never re-routed to another model. Both coders come from one company, so
their errors may be correlated; agreement between them is therefore an upper
bound on reliability, and the paper says so. The code that builds the
prompts and calls the models (`scripts/research/trials/5-code.mjs`) is
registered with this file.

Agreement between the coders is reported per question as Krippendorff's α
and raw agreement, on the 150 releases both coded, and every estimate is
also computed on those 150 with each coder (R1). **A measure with α < .667 does not decide a hypothesis**: its
estimates are reported as undecided, as Paper No. 2 did.

Main measures (codebook version 2):
- **M1** — the release names the registered primary outcome (Q3 = yes);
- **M2** — the release says whether it was met (Q4 ∈ met / not met / mixed);
- **M3** — the release calls an outcome this trial's primary (or main,
  principal) that is not a registered primary outcome (Q5 = yes);
- **M4** — spin, any of three types, when the release gives no verdict or the
  registered primary outcome was not met or mixed (Q8);
- **M5** — the headline states a result of the registered primary outcome
  (Q6 = yes);
- **M6** — a release that says the primary outcome was met gives at least one
  number for it (effect, rate, p-value or interval) (Q9 = yes).

### 7. Estimates and hypotheses

**Estimates** (main analysis set; share with a 95% interval from a bootstrap
that resamples lead sponsors, 10,000 draws, seed 20261111): M1, M2, M3, M4
(among pairs without a "met" verdict, and per spin type), M5, M6 (among "met"
pairs). These are the paper's findings; no threshold is
set for them. The comparison with Yavchitz et al.'s 47% is descriptive only
(different populations, different reference document).

**H1 — the headline hides a failure.** Among results pairs in the main set,
the headline states a result of the registered primary outcome (M5) **less
often** when that outcome was not met (Q4 = not met) than when it was met
(Q4 = met). Test: difference in shares,
one-sided, sponsor-bootstrap interval; supported if the 90% two-sided interval
excludes 0 in the predicted direction (equivalently one-sided α = .05).

**H2 — small sponsors are less complete.** Releases from small sponsors (§5)
say whether the registered primary outcome was met (M2) **less often** than
releases from large sponsors. Same test.

Holm adjustment over H1 and H2.

### 8. Robustness (reported; none decides a hypothesis)

- R1 the 150 releases both coders coded: each estimate with coder A's and
  with coder B's codings.
- R2 pairs flagged *registry changed around the release* removed.
- R4 one pair per trial (its earliest release), so a trial announced many
  times counts once.
- R5 Phase 3 trials only.
- R6 releases on distributors' websites only; organizations' own websites only.
- R7 (count only) main-set pairs whose trial has results posted on
  ClinicalTrials.gov.

### 9. Analysis code

Registered with this file and run unchanged, in this order
(`scripts/research/trials/`): `2-extract.mjs` (pages → release text and
identifiers), `1b-dates.mjs` (publication dates), `3c-aact.mjs` and
`3d-aact-extract.mjs` (snapshots → records), `3b-records.mjs` with
`SOURCE=aact` (the record in force on each release date), `5-code.mjs`
(coders), `6-reliability.mjs` (agreement) and `7-analyze.mjs` (estimates,
hypotheses, robustness). `3-registry.mjs` (the current record, API v2) is
used only for §5's fallback. Any change after registration is a numbered deviation with its
reason, as in Paper No. 2.

### 10. Conflicts of interest

The author's company (Bollé Communications inc.) operates PPN World, which
aggregates press releases, including from the distributors whose pages are in
the frame, and Hastewire, which sells AI writing tools. No pharmaceutical,
biotechnology or distribution company funds or reviews this work. The paper
names no company or trial in its text; the deposited data identify releases
and trials, which are public documents.

### 11. Ethics and data protection

Public documents only. No person is studied. People named in releases
(executives, investigators) are not coded; the deposited data carry no names
other than those in the public registry record.

### 12. Reporting commitments

All estimates and both hypotheses are reported whatever their direction; every
deviation is listed with its date; data, codings, prompts and code are
deposited on Zenodo with the paper; the cost and model time of the AI coding
are reported (pilot: $21.78 of Claude and about $10 of GPT-5.5 in round 1;
the main coding is estimated at $12–15 at batch prices).

### References

- Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of
  randomized controlled trials with statistically nonsignificant results for
  primary outcomes. JAMA. 2010;303(20):2058–2064. doi:10.1001/jama.2010.651
- Chiu K, Grundy Q, Bero L. 'Spin' in published biomedical literature: a
  methodological systematic review. PLoS Biol. 2017;15(9):e2002173.
  doi:10.1371/journal.pbio.2002173
- Goldacre B, Drysdale H, Dale A, et al. COMPare: a prospective cohort study
  correcting and monitoring 58 misreported trials in real time. Trials.
  2019;20:118. doi:10.1186/s13063-019-3173-2
- Purohit U, Sacks CA, Raissi A, et al. Press releases of drug-related
  randomized trial results prior to publication in high-impact journals: an
  observational study. J Gen Intern Med. 2023;38(14):3107–3114.
  doi:10.1007/s11606-023-08313-1
- Yavchitz A, Boutron I, Bafeta A, et al. Misrepresentation of randomized
  controlled trials in press releases and news coverage: a cohort study. PLoS
  Med. 2012;9(9):e1001308. doi:10.1371/journal.pmed.1001308
