# Pre-registration — PPN World Research Paper No. 3, Study 2
## Drug-trial press releases filed with the SEC, 2017–2026: three findings of Study 1, tested on new data

**Registered: 2026-11-12, approved by the author.** This file, the code it
names (§10), the two agreement tables cited below, and a manifest of their
SHA-256 hashes (`paper-3-study-2-SHA256SUMS.txt`, which also hashes the
candidate list `candidates.jsonl`, the pilot's document ids `pilot-ids.json`
and AACT's snapshot list `snapshots.json`, deposited with the data) are
committed together, and the manifest is timestamped with OpenTimestamps
(`paper-3-study-2-SHA256SUMS.txt.ots`), as for Study 1. No release in the
frame (§2) is fetched, read or coded before that commit, except the pilot's
(below).

**What had been looked at before registration:**

- **Study 1** (registered 2026-11-11, commit 15dc3a17; PPN World's archive,
  releases of 2026-07-01 → 10-06) and its results. After its registered
  analysis, three **exploratory** analyses were run on 2026-11-12. They are the
  reason for this study:
  - by kind of announcement (codebook Q2), the share of randomized, efficacy
    results pairs whose release says whether the registered primary outcome
    was met: first announcements ("topline") 96% (n = 48); regulatory 82%
    (72); publication 45% (22); other 33% (40); interim 31% (16); conference
    30% (61);
  - among first announcements, the headline stated the primary result for 17
    of 31 trials that met it and 3 of 15 that did not;
  - among releases that gave a verdict, "not met" was 15 of 46 first
    announcements and 8 of 107 other releases;
  - for 22 silent releases whose trial had results posted on
    ClinicalTrials.gov, 16 of the 17 with a p-value had p < .05 on a primary
    outcome (silence did not hide failure there).
- **Feasibility counts** (2026-11-12): EDGAR full-text search hit counts for
  four phrases in 8-K filings 2016–2026 ("met its primary endpoint" 1,281;
  "did not meet its primary endpoint" 335; "topline" + "primary endpoint"
  4,642; "clinicaltrials.gov" + "primary endpoint" 1,252), and the candidate
  pull of §2 (10,791 documents, counts by year, form and document type).
- **The pilot**: a seeded random draw of 140 candidates (seed 20261112, from
  every year 2016 → 2026-06), fetched 2026-11-12. 125 named a ClinicalTrials.gov
  identifier. The classifier (§3) was run twice: version 1 put sub-headings
  into 4 headlines and called one Form 6-K report page a press release;
  version 2 (registered) separates sub-headings and classes report pages as
  "other" — 86 press releases. They were coded with codebook version 2 by
  Claude Sonnet 5.5 and by Claude Haiku 5.5 (coder H, §6), with the current
  registry record. **Their results were seen.** Sonnet (classifier version 1,
  87 releases): 76 results pairs, 37 in the main set, 6 first announcements
  (2 not met, 1 met). Haiku (version 2, 86 releases): 35 main-set pairs, 10
  first announcements (4 met, 4 not met). The registered analysis code (§10)
  was run on Haiku's pilot codings to test it: H1 0 points (no first
  announcement's headline stated the result, met or not, n = 4 and 4), H2 +30
  points (90% from first announcements vs 60% from follow-ups, n = 10 each),
  H3 +26 points (n = 9 and 16). Haiku and Sonnet agreed on the pilot at α 1.00
  (verdict stated), .79 (headline), .73 (first announcement or not), 1.00
  (not met) (`paper-3-study-2-pilot-reliability-haiku-sonnet.json`). The
  pilot's documents, and any document with the same text, are
  not in the main run.
- **Coder H was chosen on Study 1's data.** Claude Haiku 5.5 coded the 150
  Study 1 releases that both registered coders had coded (2026-11-12, $0.10).
  Agreement with Claude Opus 5.5 (Krippendorff's α; Sonnet–Opus in
  brackets): verdict stated 1.00 (.97), headline states the primary result
  .92 (.77), not met 1.00 (1.00), met .97 (.93), primary named .69 (.73),
  number given with "met" .69 (.92), "trend as benefit" .54 (.84); first
  announcement or not .93, follow-up or not .88
  (`paper-3-study-2-haiku-test-reliability.json`). The owner asked for the
  cheaper model; it is used because it agreed with Opus at least as well as
  Sonnet did on every measure the hypotheses use.
- **Registry snapshots**: AACT's list of monthly flat-file snapshots
  (2017-01-05 → 2026-11-01, 116) and the 2017-01-05 snapshot, read only for the
  pilot's trials, to test the extraction code on the older file layout.

### 1. Question

Study 1 measured how press releases report a trial's registered primary
outcome. Its registered estimates stand on their own; its exploratory
analyses (above) suggested that the answer depends on **when** in a trial's
life the release comes: the first announcement nearly always says whether the
trial worked, and failures are announced then; later releases — conference
presentations, journal publications — rarely repeat the verdict, and failures
rarely get them. Those analyses were not registered and rest on 15 failed
first announcements. Study 2 tests them, as hypotheses written before any frame
release is read, on ten years of releases from a different source.

### 2. Population (the frame)

Companies listed in the United States file their material news with the
Securities and Exchange Commission, usually as the press release itself
attached to a Form 8-K (domestic issuers) or a Form 6-K (foreign issuers).

The frame: every document that

1. EDGAR full-text search returned for the query `NCT*` in filings on Forms
   8-K, 6-K, 8-K/A or 6-K/A, one query per month and form group, run on
   2026-11-12 (the search's index as of that day);
2. was filed **2017-01-01 → 2026-06-30** (the AACT snapshots of §5 start on
   2017-01-05; Study 1 covers releases from 2026-07-01);
3. is an exhibit of type EX-99.x or a Form 6-K's own document (9,954; 911
   filers; 134 of them in the pilot);
4. contains at least one ClinicalTrials.gov identifier (`NCT` + 8 digits);
5. is a press release with a headline, by the classifier of §3;

one copy per identical text (the earliest filing), without the pilot's
documents or copies of them. Documents are fetched from sec.gov under the
SEC's fair-access rules (a declared User-Agent with a contact address, about
3 requests a second).

What the frame does not cover, stated in the paper: companies not registered
with the SEC; releases a company did not file (large companies file only the
news they consider material, so their trial releases are under-represented);
releases naming a trial only by acronym or another registry; documents the
search index missed. The filing date stands in for the release date:
companies usually file a release the day they issue it, but not always, and
the paper calls it the filing date.

### 3. Units, inclusion and the main analysis set

**Is it a press release, and what is its headline?** Companies also file slide
decks, posters and letters as exhibits, and an exhibit's headline sits among
logos, exchange labels and sub-headings. One Claude Haiku 5.5 request per
document (its opening 700 words; prompt in `sec/4-classify.mjs`) returns the
kind (press release / presentation / scientific / other), the headline copied
verbatim and any sub-headings separately. The code checks that the headline is
in the text character for character (whitespace aside); the paper reports how
many are not (0 of 86 in the pilot) and they are kept.

Units, exclusions and sets are Study 1's (its §3): release–trial pairs; results
pairs (Q1); releases not issued by the trial's side excluded (Q0); **main
analysis set** = results pairs whose trial is randomized and has an efficacy
primary outcome; early-phase pairs reported separately; pairs without a
registry record counted and set aside.

**Kinds of announcement** (codebook Q2): **first announcement** = `topline`;
**follow-up** = `conference` or `publication`.

### 4. Census

Every frame release is coded by coder H; no sampling, no weights.

### 5. The registry record

As in Study 1, the record in force on the release date — from AACT flat-file
snapshots, here the **earliest snapshot of each calendar quarter**, 2017-Q1 →
2026-Q3 (the 2026-07-01 snapshot is the "next" one for releases of 2026-Q2):
the last of them on or before the filing date is the record; if the next one
lists different registered primary outcomes, the pair is flagged *registry
changed around the release*. Quarterly, not monthly, snapshots: ~38 downloads
(~60 GB) instead of ~114; a record edited and edited back within a quarter is
missed, and the paper says so.

### 6. Measures and coders

The codebook is Study 1's, version 2, unchanged (`scripts/research/trials/codebook.md`).

- **Coder H — the measurement**: Claude Haiku 5.5 (`claude-haiku-5-5`) codes
  every frame release;
- **Coder B — the reliability check**: Claude Opus 5.5 (`claude-opus-5-5`)
  codes a random 200 frame releases (seeded shuffle, seed 20261111, in
  `sec/5-code.mjs`).

Everything else is Study 1's §6: one request per release covering every trial
it names, Message Batches API, the codebook as a cached system prompt, a JSON
schema, reasoning at "medium" effort, default sampling, refusals recorded as
missing, never re-routed. Both coders come from one company, so agreement is
an upper bound on reliability.

Agreement is reported per question (Krippendorff's α and raw agreement) on the
releases both coded. **A hypothesis is undecided if any measure it uses has
α < .667** on those releases: H1 uses Q2 = topline, Q4 met / not met and Q6;
H2 uses Q2 = topline, Q2 ∈ {conference, publication} and M2; H3 uses Q2 =
topline and Q4 not met / verdict.

### 7. Estimates and hypotheses

**Estimates** (main set; share with a 95% interval from a bootstrap that
resamples **filers** — the company that filed the release — 10,000 draws, seed
20261112): Study 1's M1–M6 and "not met", and each part of H1–H3 below.
Descriptive, no test: the same by filing year.

**H1 — at the first announcement, a failure stays out of the headline.** Among
first announcements in the main set, the headline states a result of the
registered primary outcome (Q6 = yes) **less often** when the outcome was not
met (Q4 = not met) than when it was met (Q4 = met). Study 1, exploratory: 55%
vs 20%.

**H2 — the verdict fades after the first announcement.** In the main set, the
release says whether the registered primary outcome was met (M2) **less often**
in follow-up releases (conference, publication) than in first announcements.
Study 1, exploratory: 30–45% vs 96%.

**H3 — after the first announcement, the releases are about winners.** Among
main-set pairs whose release gives a verdict (met / not met / mixed), "not
met" is a **smaller** share in releases that are not first announcements than
in first announcements. Study 1, exploratory: 7% vs 33%.

Each test: the difference in shares (first minus second), one-sided,
filer-bootstrap interval; supported if the 90% two-sided interval excludes 0 in
the predicted direction and the Holm-adjusted one-sided p (over H1–H3) is
below .05.

### 8. Robustness (reported; none decides a hypothesis)

- R1 the 200 releases both coders coded: every estimate and test with coder H
  and with coder B.
- R2 pairs flagged *registry changed around the release* removed.
- R4 one pair per trial (its earliest release).
- R5 Phase 3 trials only.
- R6 lead sponsors (from the registry) as the bootstrap clusters instead of
  filers.
- R7 Form 8-K filers only; Form 6-K filers only.
- R8 (count only) main-set pairs whose trial has results posted.

### 9. Exploratory (labelled as such in the paper)

The day and hour of filing (EDGAR acceptance time) of first announcements, met
vs not met — whether failures are filed on Friday evenings more often; the
comparison of Study 2's estimates with Study 1's; any further split by year,
phase or therapeutic area. Code written for these after registration is
deposited and labelled exploratory.

### 10. Analysis code

Registered with this file and run unchanged, in this order
(`scripts/research/trials/`): `sec/1-search.mjs` (the candidate pull — run
before registration; its output `candidates.jsonl` is hashed in the manifest),
`sec/2-fetch.mjs` (documents), `sec/3-extract.mjs` (text and identifiers),
`sec/4-classify.mjs` with `PILOT_WORK` set (press release or not, headline,
duplicates), `sec/0-aact-snapshots.mjs get` (quarterly snapshots),
`sec/3d-aact-extract.mjs` (Study 1's step 3d plus the 2017 column names),
`3b-records.mjs` with `SOURCE=aact` (Study 1's, unchanged), `sec/5-code.mjs`
(coders H and B; Study 1's step 5 plus coder H), `sec/6-reliability.mjs` and
`sec/7-analyze.mjs`. Study 1's registered files are not changed. Any change
after registration is a numbered deviation with its reason.

### 11. Conflicts of interest

As Study 1: the author's company (Bollé Communications inc.) operates PPN World,
which aggregates press releases, and Hastewire, which sells AI writing tools.
No pharmaceutical, biotechnology or distribution company funds or reviews this
work. The paper names no company or trial in its text; the deposited data
identify filings and trials, which are public documents.

### 12. Ethics and data protection

Public documents only. No person is studied; people named in releases are not
coded.

### 13. Reporting commitments and cost

All estimates and the three hypotheses are reported whatever their direction,
next to Study 1's; every deviation is listed with its date; data, codings,
prompts and code are deposited on Zenodo with the paper. Cost of the AI work,
at batch prices: pilot and coder test $1.17 (Sonnet $0.96, Haiku $0.21); main
run estimated at about $11 (coder H ~$5, coder B ~$4.40, classifier ~$1.50).
The registry snapshots are free (AACT account).

### References

As Study 1, plus:

- U.S. Securities and Exchange Commission. Accessing EDGAR Data (fair-access
  policy). https://www.sec.gov/search-filings/edgar-search-assistance/accessing-edgar-data
