ChronicleBench by ChronicleMethodology Raw data Open source — GitHubExplore Chronicle

ChronicleBench v1.1 · an open benchmark by Chronicle

Can AI write a great novel?

Twelve frontier models from 9 labs, commissioned to write complete 80–90,000-word novels — scored window-by-window against 36 published human novels, under one frozen, preregistered protocol. And one enabling fact underneath every result: no model finishes a novel on its own.

0 / 13models finished unassisted
+12.9ending-quality lift from Chronicle's full engine, same model
p86Chronicle × Opus 5 vs published novels
Explore Chronicle books → See the full ranking
01 · Every book on one scale

Real novels included — the full range.

The mean of the 36 published public-domain reference novels is 65.3 on our instrument. The honest way to test AI prose is to stop scoring AI against AI: below, every AI entrant sits in one ranking with all 36 published novels — same pinned instrument, same five windows (opening · 25K · 50K · 75K · ending), same scale. GPT-5.6 Sol running on Chronicle scored 73.4 — the 78th percentile of the published reference.

What is this score? — OBR & the ChronicleBench Score in 30 seconds

Every book on this page — human and AI — is scored by OBR, one pinned evaluation instrument: an LLM judge (Claude Sonnet 4.6, temperature 0, three runs, medians) that must cite quoted evidence from the text for every score, cross-checked against deterministic text metrics it cannot contradict.

Eight judged dimensions, each 1–10 with anchored definitions:

  • Momentum — hook strength, pull-through, no mid-book sag
  • Voice & Craft — consistent voice, sentence variety, no AI-isms
  • Characters — distinct voices, desires, growth
  • Psychological Specificity — inner lives that couldn't belong to anyone else
  • Emotional Arc — escalation, stakes, payoff
  • Coherence — timeline and cause-effect integrity, no dropped threads
  • Promise Delivery — the book delivers what it promised
  • Causal Inevitability — surprising yet inevitable turns, no deus ex machina

Plus two computed dimensions: Canon Integrity (factual self-consistency — contradictions cap the score) and Prose Authenticity (AI-pattern density). Composite = genre-weighted mean × 10, on a 0–100 scale. ChronicleBench Score = the mean of five 10,000-word windows (opening · 25K · 50K · 75K · ending), so late-book collapse can't hide. Full detail, weights and caps: methodology.

Chronicle × modelpublished novel (Gutenberg, seeded random)
human median 64.3
PersuasionAusten
93.3 p97
The Scarlet LetterHawthorne
86.0 p94
What Maisie KnewJames
84.8 p92
KidnappedStevenson
84.5 p89
My ÁntoniaCather
81.8 p86
Chronicle × Opus 5
81.577.8–83.5 p86
Zuleika DobsonBeerbohm
80.5 p83
Chronicle × Fable 5*
78.5 p83*
Alice AdamsTarkington
77.3 p81
Two on a TowerHardy
76.0 p78
Chronicle × GPT-5.6 Sol
73.470.5–79.0 p78
Chronicle × GPT-6 Astra*
73.3 p78*
Journey to a womanBannon
72.5 p75
This Side of ParadiseFitzgerald
71.0 p72
Antic HayHuxley
70.5 p69
Chronicle × GPT-5.6 Sol Pro
70.368.5–72.0 p69
MessiahVidal
70.3 p67
Chronicle × Sonnet (Production)
69.262.8–71.8 p67
The Mercy of AllahBelloc
69.0 p64
The Black MothHeyer
68.0 p61
Chronicle × Qwen 3.8 Max
67.763.0–71.3 p61
The gods of MarsBurroughs
65.8 p58
The Power and the GloryCooke
65.5 p56
MEAN OF THE 36 REFERENCE NOVELSpublished public-domain corpus · median 64.3
65.3 p56
Erewhon Revisited Twenty Years La…Butler
65.0 p53
Rose of Dutcher's CoollyGarland
64.5 p50
The White MollPackard
64.0 p47
ErewhonButler
63.0 p44
The Adventures of Sir Launcelot G…Smollett
62.8 p42
The Wings of the MorningTracy
62.5 p39
The blood of the vampireMarryat
60.0 p36
Chronicle × Gemini 3.1 Pro
59.256.0–62.3 p36
Love in ExcessHaywood
59.0 p33
Thorstein of the mereCollingwood
58.8 p31
The Secret of the ReefBindloss
58.5 p28
Paradise GardenGibbs
57.0 p25
From Kingdom to ColonyDevereux
56.8 p22
The Mystery of the LocksHowe
56.0 p19
The Tickencote TreasureLe Queux
54.5 p17
The Royal Pawn of VeniceTurnbull
54.0 p14
The Mysterious Mr. MillerLe Queux
51.3 p11
The Red WidowLe Queux
49.3 p8
The Great God GoldLe Queux
48.3 p6
Monte-Cristo's DaughterFlagg
47.8 p3
The Witch of SalemMusick
39.0 p0
Chronicle × DeepSeek V4 Pro*
39.0 p0*
Chronicle × MiniMax M3
32.725.3–38.5 p0

pNN = percentile within the 36 published reference novels · * small sample

Reference corpus: published public-domain novels, three genre strata, deterministic seeded selection committed before scoring — nobody picked flattering comparisons. Historical style, survivorship and judge-recognition caveats in the methodology.

02 · The catch

None of these novels happens without Chronicle.

Same commission, no assistance: “write a complete 80–90,000-word novel.” Six attempts per model, every voluntary stop recorded. Left alone, every model quits early — most within the first act.

Chronicle is the leading harness for long-form writing. Every entrant on this board runs on it — unassisted, none of them finish.

80K — where a novel begins
Claude Sonnet 4.6
36K median
Claude Opus 5
30K median
MiniMax M3
22K median
DeepSeek V4 Pro
19K median
GPT-6 Astra
18K median
Claude Fable 5
17K median
GPT-5.6 Sol
13K median
GPT-5.6 Sol Pro
12K median
Qwen 3.8 Max
10K median
Mistral Medium 3.5
9K median
Gemini 3.1 Pro
5K median
Grok 4.6
4K median
Llama 4 Maverick
1K median

0 of 13 models delivered the novel they were asked to write. The median stop was 13K words — more novella than novel. The missing 72K words are the problem Chronicle was built to solve.

In this cohort, every eligible AI novel exists because the model ran inside Chronicle's harness.

03 · The Chronicle effect

Same model. Chronicle added.

Chronicle does more than make the model continue.

The controlled comparison holds the underlying model constant — Claude Sonnet 4.6, the same briefs, the same instrument. The architecture is the treatment.

MetricRaw Sonnet 4.6Full Chronicle engineChronicle effect
Delivered length~30K words, then stops70–76K words2.4× longer
Opening quality71.9278.15+6.23
Ending quality53.9866.90+12.92
Full-length score65.6070.96+5.36 †

† The full-length comparison rests on two eligible Chronicle books, scored under the v1.0 window contract (four positions; v1.1 uses five). The window-for-window opening and ending comparisons are the cleaner results. The largest improvement arrives at the ending — the point where raw models fail hardest. Full data on the archived v1.0 deep-dive.

Two Chronicle layers — read the board correctly.

Raw modelWrites strong short-form prose, then stops.
↓
Chronicle long-form harnessThe standardized, model-agnostic execution layer that makes an 80–90K attempt possible at all. Identical for every entrant — this is what the leaderboard measures.
↓
Full Chronicle engineThe production architecture: narrative planning, persistent state, pacing, continuity repair, editorial passes and quality gates on top of the harness. This is what the controlled comparison above measures.

Benchmark harness — the identical runtime used to test each frontier model at novel length. Full Chronicle engine — Chronicle's production stack, which adds its complete planning, memory, pacing, repair and quality machinery.

Full engine — measured
Chronicle × Sonnet (Production)
attainment + fresh-context ending (K=2 selection) · pre-production
69.2

p67 vs humans · 103.2% of the 85K commission delivered · endings 64.6 vs 58.2 raw

Preregistered before generation; every book and every attempt published. Same briefs, same frozen five-window instrument as every row on this board.

The baseline — without the engine
Raw Sonnet 4.6
the same model, benchmark harness only (control)
64.4

p50 vs published novels · +4.8 with Chronicle's engine → 69.2

04 · How Chronicle works

A book is not a sequence of prompts — it is persistent state.

A novel runs far past any model's usable attention, so systems replace context. Chronicle carries the book; the model writes the prose. Four capabilities do the carrying:

PLANStory architecture firstDramatic beats, promises, characters and the ending are constructed before a word of prose is drafted.
REMEMBERPersistent narrative stateCharacter state, timeline, promises, motifs and recent prose carried across tens of thousands of words.
PACELength-aware structureActs and sections budgeted against the requested length — no novella endings, no padded exhausted stories.
REPAIRAudit before shippingContinuity, repetition, prose quality and structure audited and repaired before a book ships.

The full pipeline — planning, stateful drafting, continuity management, editorial passes and a final quality gate — is documented in the methodology.

05 · The leaderboard

Every entrant, powered by Chronicle.

The identical harness ran every model. Without Chronicle, there is no novel-length race — and even with it, 4 of 13 models couldn't sustain a novel to 80K.

EntrantScoreOpeningEndingPercentile vs human novels
1 Chronicle × Opus 5 FULL ENGINE 81.582.082.7 p86
2 Chronicle × Fable 5 FULL ENGINE · n=1 small sample*; Fable transport adaptations disclosed in methodology notes 78.5*83.871.3 p83*
3 Chronicle × GPT-5.6 Sol BEST RELIABLE RESULT 73.481.367.1 p78
4 Chronicle × GPT-6 Astra 73.3*85.068.8 p78*
5 Chronicle × GPT-5.6 Sol Pro 70.379.564.8 p69
6 Chronicle × Sonnet (Production) FULL ENGINE production engine, end-to-end 69.278.664.6 p67
7 Chronicle × Qwen 3.8 Max 67.775.763.0 p61
8 Chronicle × Gemini 3.1 Pro 59.261.955.0 p36
9 Chronicle × DeepSeek V4 Pro 39.0*55.035.0 p0*
10 Chronicle × MiniMax M3 32.763.220.6 p0
—Chronicle × Claude Fable 5 stalled short of 80K on every attemptDNS
—Chronicle × Grok 4.6 stalled short of 80K on every attemptDNS
—Chronicle × Mistral Medium 3.5 stalled short of 80K on every attemptDNS
—Chronicle × Llama 4 Maverick stalled short of 80K on every attemptDNS

Each score averages a model's eligible 80–95K novels over all of its harnessed attempts (six in the base cohort; reliability extensions per protocol A7); ineligible and stalled runs publish as DNF in the complete record. GPT-6 Astra entered after cohort close under the protocol §6 re-entry procedure (2026-09-08); its runs used the identical frozen harness and instrument. * small sample — treat as provisional. Score = mean of five 10K-word window scores over eligible novels (the ChronicleBench Score). DNS = did not sustain despite the harness. p-numbers are percentiles: pNN = scores above NN% of the 36-novel published reference corpus. One pinned instrument judges every row — an 8-dimension, evidence-cited rubric with deterministic cross-checks, temperature 0, 3-run medians: full methodology. The Chronicle × Opus row is a preregistered model-swap: the frozen production engine with only the writer model changed — bare Opus finished a minority of its attempts; on Chronicle it delivered every commissioned novel.

Chronicle's row is the production system measured end-to-end (6 books through the real pipeline). A preregistered harness-level cohort of the same treatment, generated first, scored 69.5 (endings 66.5, 102.5% completion) — published in full in the raw data; both cohorts, every book, no selection.

The First Editions — five complete Chronicle × Opus novels, with covers, excerpts & immersive-audio samples
06 · Methodology & complete record

Every run, nothing hidden.

163 headline-cohort runs · 187 total recorded generation runs including withdrawn models. Every run, failure and refusal publishes here — preregistered before generation, never selected after.

Protocol frozen before generationRoster, briefs, arms, eligibility and scoring pinned and published before the first run; every amendment dated and disclosed.
Every failure publishedDNFs, stalls, provider failures and withdrawn models stay in the record — visible failures, never silent retries.
Everything downloadableScores, manuscripts hashes, per-run costs and provider receipts, in machine-readable form.
Download JSON Download CSV Frozen protocol Open-source repo (GitHub)
View all 187 generation runs — incl. DNFs, failures and withdrawn models
RunArmStatusWordsScoreSHA-256
astra-B2-free-r1freecomplete1699383.7491acd43ee434460
astra-B2-free-r2freecomplete1820785a61bd778ca42e79f
astra-B2-free-r3freecomplete1667083.7d20b3b6d8560f932
astra-B2-sustained-r1sustainedcomplete8068073.28190455139bbd030c
astra-B2-sustained-r2sustainedcomplete70833DNF0942c3f6739c77a2
astra-B2-sustained-r3sustainedcomplete74957DNF10e006ba96a46bc1
astra-N1-free-r1freecomplete2143981.35d0af55ebfe0be9e
astra-N1-free-r2freecomplete1701881.34a2a73f6b6d6d902
astra-N1-free-r3freecomplete1945082.5013f1d7242ee4a94
astra-N1-sustained-r1sustainedcomplete55852DNFb1f9ee76814af9ec
astra-N1-sustained-r2sustainedcomplete67972DNFa0a44da6ef22af7b
astra-N1-sustained-r3sustainedcomplete64473DNFb475e6a277e32f2c
dsv4pro-B2-free-r1freecomplete2192455831f1f1dafa33484
dsv4pro-B2-free-r2freecomplete1525355c309c49f0e4b92c6
dsv4pro-B2-free-r3freecomplete1155953.8015ba501142c3092
dsv4pro-B2-sustained-r1sustainedcomplete70979DNF8f7eb52c4bdacdce
dsv4pro-B2-sustained-r2sustainedcomplete8275539.028fa6476dbeb44b33
dsv4pro-B2-sustained-r3sustainedcomplete59897DNFec68bf7410e36047
dsv4pro-N1-free-r1freecomplete1741860c1d64edec37e7713
dsv4pro-N1-free-r2freecomplete2029363.829b4923a443454a9
dsv4pro-N1-free-r3freecomplete2240061.3f63eb6742023fb94
dsv4pro-N1-sustained-r1sustainedcomplete74779DNF4c0d0d6e04fde574
dsv4pro-N1-sustained-r2sustainedcomplete46404DNF5d179433d90db238
dsv4pro-N1-sustained-r3sustainedcomplete59479DNF9d75171f6991c4bd
fable5-B2-free-r1freecomplete1799079ad057c4a3f4dca57
fable5-B2-free-r2freecomplete15186852043d1866a3d7e4c
fable5-B2-free-r3freecomplete1470879ca159189feff9596
fable5-B2-sustained-r1sustainedcomplete9291DNF44904322ff085f4f
fable5-B2-sustained-r2sustainedcomplete21969DNF47ad80238bd318e5
fable5-B2-sustained-r3sustainedcomplete41977DNF983583b41673865b
fable5-N1-free-r1freecomplete2827079ac6a7fa1237a0e04
fable5-N1-free-r2freecomplete2058679b6e4a37a0241101a
fable5-N1-free-r3freecomplete14568797e02a4d40e6518be
fable5-N1-sustained-r1sustainedcomplete22596DNFaaaa297265b0a467
fable5-N1-sustained-r2sustainedcomplete35831DNFd895914ad1d3d7b4
fable5-N1-sustained-r3sustainedcomplete32830DNF2655e4b11bb51fc5
gemini31-B2-free-r1freecomplete372268.8e8b04692d516905c
gemini31-B2-free-r2freecomplete649863.82e352cb012bf841a
gemini31-B2-free-r3freecomplete523367.55e66bc63482fe5d8
gemini31-B2-sustained-r1sustainedcomplete72941DNF8a8d7807bbfab6f3
gemini31-B2-sustained-r2sustainedcomplete8528656.043433d9c902210f29
gemini31-B2-sustained-r3sustainedcomplete8713362.2887e36441702946d0
gemini31-N1-free-r1freecomplete675067.527757a63a54ce08f
gemini31-N1-free-r2freecomplete569062.5cfd92ec05ad12d24
gemini31-N1-free-r3freecomplete476562.55efa204919a72224
gemini31-N1-sustained-r1sustainedcomplete40822DNF9b9b661a6375dec0
gemini31-N1-sustained-r2sustainedcomplete47887DNF952fbab3926d6245
gemini31-N1-sustained-r3sustainedcomplete36639DNFd1a3f53dae154d5a
glm53-B2-free-r1freefailed (withdrawn — see methodology)6887—
glm53-B2-free-r2freecomplete (withdrawn — see methodology)265511570e0122662d8a27d
glm53-B2-free-r3freecomplete (withdrawn — see methodology)84147955a1f9fde2074454
glm53-B2-sustained-r1sustainedfailed (withdrawn — see methodology)0—
glm53-B2-sustained-r2sustainedfailed (withdrawn — see methodology)0—
glm53-B2-sustained-r3sustainedfailed (withdrawn — see methodology)0—
glm53-N1-free-r1freefailed (withdrawn — see methodology)24962—
glm53-N1-free-r2freefailed (withdrawn — see methodology)0—
glm53-N1-free-r3freefailed (withdrawn — see methodology)0—
glm53-N1-sustained-r1sustainedfailed (withdrawn — see methodology)0—
glm53-N1-sustained-r2sustainedfailed (withdrawn — see methodology)0—
glm53-N1-sustained-r3sustainedfailed (withdrawn — see methodology)0—
grok46-B2-free-r1freecomplete385265b6a4a9fa37fdb0f3
grok46-B2-free-r2freecomplete521778.8592e97e2acdf4a27
grok46-B2-free-r3freecomplete223673.8b6085d080343e08d
grok46-B2-sustained-r1sustainedcomplete63DNF72fa1ab6c07b2c90
grok46-B2-sustained-r2sustainedcomplete34735DNFd8af71be3398a61c
grok46-B2-sustained-r3sustainedcomplete41913DNF6285d84ecda66782
grok46-N1-free-r1freecomplete318676.34cbc86dbf35c6f9e
grok46-N1-free-r2freecomplete502076.38f9be458643228ca
grok46-N1-free-r3freecomplete403370bb091c8c1ba1a135
grok46-N1-sustained-r1sustainedcomplete42932DNF6492c06da4302a93
grok46-N1-sustained-r2sustainedcomplete45214DNF0f365f1cd0359dfc
grok46-N1-sustained-r3sustainedcomplete41702DNFe9dbec8b61d0e32f
kimik3-B2-free-r1freefailed (withdrawn — see methodology)0—
kimik3-B2-free-r2freefailed (withdrawn — see methodology)0—
kimik3-B2-free-r3freefailed (withdrawn — see methodology)834—
kimik3-B2-sustained-r1sustainedfailed (withdrawn — see methodology)0—
kimik3-B2-sustained-r2sustainedfailed (withdrawn — see methodology)0—
kimik3-B2-sustained-r3sustainedfailed (withdrawn — see methodology)0—
kimik3-N1-free-r1freefailed (withdrawn — see methodology)0—
kimik3-N1-free-r2freefailed (withdrawn — see methodology)0—
kimik3-N1-free-r3freefailed (withdrawn — see methodology)0—
kimik3-N1-sustained-r1sustainedfailed (withdrawn — see methodology)0—
kimik3-N1-sustained-r2sustainedfailed (withdrawn — see methodology)0—
kimik3-N1-sustained-r3sustainedfailed (withdrawn — see methodology)0—
maverick-B2-free-r1freecomplete1240—d9c0ed14daf4f478
maverick-B2-free-r2freecomplete1297—0ad31dc179abbeb4
maverick-B2-free-r3freecomplete1472—2fa57d8742df1af3
maverick-B2-sustained-r1sustainedcomplete10283DNFe313cd28cda4de41
maverick-B2-sustained-r2sustainedcomplete17354DNF14bc0590abcb3f15
maverick-B2-sustained-r3sustainedcomplete14376DNFf00acf68cfa3aefe
maverick-N1-free-r1freecomplete1749—4f081efc26cd6273
maverick-N1-free-r2freecomplete1244—6273704de3d253e3
maverick-N1-free-r3freecomplete1489—955fd8b292a74400
maverick-N1-sustained-r1sustainedcomplete13890DNF1e070ef730c382f8
maverick-N1-sustained-r2sustainedcomplete14253DNF4501fa6e82dad19d
maverick-N1-sustained-r3sustainedcomplete15970DNFd9d395473ab90936
minimax3-B2-free-r1freecomplete2509363.84de22a28ea6862d1
minimax3-B2-free-r2freecomplete2045557.57b03e501f348c197
minimax3-B2-free-r3freecomplete1535266.3def3e707c309ad40
minimax3-B2-sustained-r1sustainedcomplete8519225.285a44da76f610c3c5
minimax3-B2-sustained-r2sustainedcomplete64563DNF2be3d74925f6863a
minimax3-B2-sustained-r3sustainedcomplete8602732.0286ec18e7ed97a0f8
minimax3-N1-free-r1freecomplete2369168.8dacf6c0dd5810f00
minimax3-N1-free-r2freecomplete2236263.83c2d17b040b09b13
minimax3-N1-free-r3freecomplete2250465b37abcdd33ffa55a
minimax3-N1-sustained-r1sustainedcomplete9135938.546ba904b3aa339489
minimax3-N1-sustained-r2sustainedcomplete99408DNFe6ad3f28b9103f8d
minimax3-N1-sustained-r3sustainedcomplete9081035.04af1aefca34428002
mistral35-B2-free-r1freecomplete1037046.3206a02f7866b7ff5
mistral35-B2-free-r2freecomplete770050b675c0c2b32af1af
mistral35-B2-free-r3freecomplete1264248.81a3cc1de67bda48f
mistral35-B2-sustained-r1sustainedcomplete54385DNF464a3fd4c75adbb0
mistral35-B2-sustained-r2sustainedcomplete51279DNFc1c91f60ede43f75
mistral35-B2-sustained-r3sustainedcomplete43343DNF93af51c63c17cbfa
mistral35-N1-free-r1freecomplete472056.308f4d57addff0970
mistral35-N1-free-r2freecomplete996247.567960f59fbbf8d28
mistral35-N1-free-r3freecomplete343656.38cdcd242b6b4f2a9
mistral35-N1-sustained-r1sustainedcomplete48962DNF4c0c9ea5d0bd9c71
mistral35-N1-sustained-r2sustainedcomplete50987DNFa835951849d3c096
mistral35-N1-sustained-r3sustainedcomplete47364DNF1a7ba2cb6f238fe6
opus5-B2-free-r1freecomplete3127979e1c1e666afad4e0b
opus5-B2-free-r2freecomplete2780379db4898690e62d410
opus5-B2-free-r3freecomplete2491483.70b0078ec572c4415
opus5-B2-sustained-r1sustainedcomplete62570DNFa2ac3d3d5348da50
opus5-B2-sustained-r2sustainedcomplete79540DNF8cd31f0bdb04168b
opus5-B2-sustained-r3sustainedcomplete37173DNF6fd8d0393f0e2a41
opus5-B2-sustained-r4sustainedcomplete53415DNFc73fa94211ee2063
opus5-B2-sustained-r5sustainedcomplete59382DNFed58118776d26fda
opus5-B2-sustained-r6sustainedcomplete33298DNFbafa279e27cc02e6
opus5-N1-free-r1freecomplete296577901e4e587145539ae
opus5-N1-free-r2freecomplete4064180fb6e9744bad22feb
opus5-N1-free-r3freecomplete3416679dd2054496501a7c6
opus5-N1-sustained-r1sustainedcomplete8043382.824403653235038a14
opus5-N1-sustained-r2sustainedcomplete33638DNFd9e2323cc6f1f606
opus5-N1-sustained-r3sustainedcomplete68736DNF436f1b1a5cf91102
opus5-N1-sustained-r4sustainedcomplete37923DNF4f21d97d90865133
opus5-N1-sustained-r5sustainedcomplete8025983.264e9c40ec82fd2fd5
opus5-N1-sustained-r6sustainedcomplete8644883.089b3ca8e1f43e0346
qwen38-B2-free-r1freecomplete1175771.36d39a39bd9fe2876
qwen38-B2-free-r2freecomplete10812794bce24d659022bb7
qwen38-B2-free-r3freecomplete78797996de0d531072c06e
qwen38-B2-sustained-r1sustainedcomplete8583463.022da1a57d429af4d2
qwen38-B2-sustained-r2sustainedcomplete8785367.5211b128803667ac52
qwen38-B2-sustained-r3sustainedcomplete8696969.020b5255f3db93380b
qwen38-N1-free-r1freecomplete831773.83d082ebb86184ed6
qwen38-N1-free-r2freecomplete659175eeadde0bbefe66f4
qwen38-N1-free-r3freecomplete1150775e63dac21caeb4874
qwen38-N1-sustained-r1sustainedcomplete8785071.2866ba40e7e24eda53
qwen38-N1-sustained-r2sustainedcomplete9083967.04ab2c80fcf1a32927
qwen38-N1-sustained-r3sustainedcomplete8773368.28b6b983e6064154ef
sol-B2-free-r1freecomplete13295801d09ffb3233942a2
sol-B2-free-r2freecomplete1288281.3310110e694a64b01
sol-B2-free-r3freecomplete1070983.7e9e626daea9becc0
sol-B2-sustained-r1sustainedcomplete8810772.24eb2da41f05490a25
sol-B2-sustained-r2sustainedcomplete9076872.28d01b30b8112eb27d
sol-B2-sustained-r3sustainedcomplete8894779.02b3313ceb133f9b1e
sol-N1-free-r1freecomplete1366078.841c5d86c9dd130db
sol-N1-free-r2freecomplete1453376.37833e63d694c7c0b
sol-N1-free-r3freecomplete1240977.5996c52fe82276328
sol-N1-sustained-r1sustainedcomplete8658073.85a9d682c8b9a74a6
sol-N1-sustained-r2sustainedcomplete8761572.521154b15d224d106b
sol-N1-sustained-r3sustainedcomplete9183870.52ccb1b5d225a98462
solpro-B2-free-r1freecomplete1007385354a452597c6ff1e
solpro-B2-free-r2freecomplete1218982.5c72e47018848b82c
solpro-B2-free-r3freecomplete1209978.828f05f7a3c53b3b6
solpro-B2-sustained-r1sustainedcomplete9280370.78622c16b8232091e3
solpro-B2-sustained-r2sustainedcomplete8766872.04397d27546d1b0330
solpro-B2-sustained-r3sustainedcomplete95748DNF534548fcb025a673
solpro-B2-sustained-r4sustainedcomplete97324DNFfd72f79882672f0d
solpro-N1-free-r1freecomplete1306176.3eb15cd8434130b6d
solpro-N1-free-r2freecomplete113828092a05341cc24c120
solpro-N1-free-r3freecomplete1358877.5e27294e7a34eb67e
solpro-N1-sustained-r1sustainedcomplete8810269.26705722e343c88956
solpro-N1-sustained-r2sustainedcomplete9206868.52a9b8664c97f6bd0d
solpro-N1-sustained-r3sustainedcomplete8992070.785bff11d412764a3c
sonnet46-B2-free-r1freecomplete3146573.8b7497e7412be7375
sonnet46-B2-free-r2freecomplete4210676.326800018adada850
sonnet46-B2-free-r3freecomplete3156876.3335fdaa16a10f044
sonnet46-B2-sustained-r1sustainedcomplete8781966.521f72eb168a8c4384
sonnet46-B2-sustained-r2sustainedcomplete8659458.02d7e7ac51afe6b375
sonnet46-B2-sustained-r3sustainedcomplete8811468.764ef9a6cf34836127
sonnet46-N1-free-r1freecomplete3786373.8cf2e1503455c02e8
sonnet46-N1-free-r2freecomplete3327873.880ba6a58136c7c57
sonnet46-N1-free-r3freecomplete3848273.89e8f7edb0bbd0e94
sonnet46-N1-sustained-r1sustainedcomplete8827564.04b0c7d3e04f92c888
sonnet46-N1-sustained-r2sustainedcomplete9159564.8d8f4eb90478f6b88
sonnet46-N1-sustained-r3sustainedcomplete8930464.263c6089e124bd2a22
Human reference corpus — 36 novels, per-stratum

Stratum medians: plot 56.4 (n=12) · character 74.3 (n=12) · control 62.9 (n=12)

TitleAuthorStratumWordsScore
The Tickencote TreasureLe Queuxplot8719654.54
The Black Moth: A Romance of the XVIIIth CenturyHeyerplot9287168.02
The Great God GoldLe Queuxplot8210848.28
The gods of MarsBurroughsplot8273165.78
The Mystery of the LocksHoweplot9201856.04
Monte-Cristo's DaughterFlaggplot8479947.78
The White MollPackardplot8627464.02
From Kingdom to ColonyDevereuxplot9487856.78
The Mysterious Mr. MillerLe Queuxplot8471251.28
KidnappedStevensonplot8019984.5
The Secret of the ReefBindlossplot9228958.54
The Red Widow; or, The Death-Dealers of LondonLe Queuxplot8199249.28
Alice AdamsTarkingtoncharacter8449177.28
Two on a TowerHardycharacter9395376.02
This Side of ParadiseFitzgeraldcharacter8085371.02
PersuasionAustencharacter8332993.28
Love in Excess; or, the Fatal Enquiry: A Novel in Three PartHaywoodcharacter8889659.02
The Power and the GloryCookecharacter9118965.54
My ÁntoniaCathercharacter8134181.78
Rose of Dutcher's CoollyGarlandcharacter8700464.54
Zuleika Dobson; Or, An Oxford Love StoryBeerbohmcharacter8066080.54
Journey to a womanBannoncharacter8434472.52
What Maisie KnewJamescharacter9321684.78
The Wings of the MorningTracycharacter8889762.52
Thorstein of the mere : $b A saga of the Northmen in LakelanCollingwoodcontrol8354658.78
The Royal Pawn of Venice: A Romance of CyprusTurnbullcontrol8934554.04
Paradise Garden: The Satirical Narrative of a Great ExperimeGibbscontrol9229157.02
Antic HayHuxleycontrol8803770.54
MessiahVidalcontrol8336970.26
The Adventures of Sir Launcelot GreavesSmollettcontrol9074762.78
The Witch of Salem; or, Credulity Run MadMusickcontrol8041139.02
The Mercy of AllahBelloccontrol8208269.02
The blood of the vampireMarryatcontrol8775260.02
Erewhon; Or, Over the RangeButlercontrol8419463.02
Erewhon Revisited Twenty Years Later, Both by the Original DButlercontrol8508265.04
The Scarlet LetterHawthornecontrol8290186.02
Appendix A — other writing systems on the same commission (exploratory, n=1, never ranked)

One-run exploratory tests of third-party writing systems on the frozen ChronicleBench commission, scored with the same pinned instrument. Exploratory: n=1, operator-driven where the product requires it, never ranked. Product limits are recorded as results.

SystemBriefStatusWordsOpeningSagaScoreGenerationSHA-256
Sherpa (Pocket FM)native workflow (concept chat → storyboard → per-episode idea/outline/draft/review), operator takes first-listed option everywhere, zero prose typedB2DNFProduct limit: "You can't have more than 3 unpublished episodes at a time. Publish one to start a new one." Continuing requires publishing episodes live on the Pocket FM app; not done. Separately, Sherpa auto-planned a 100-episode serial (ending scheduled for episodes 76–100) for the 85K commission.4371no ending56.3opening window · 4330 wDNF59 min wall · 20 min operator · 5 interventions24a13ada927c6310

Same fragment, same instrument: Chronicle's B2 books cut to Sherpa's length

Sherpa produced 4330 scoreable words before its product limit. To compare like with like, the six Chronicle B2 books (Chronicle × Opus 5 ×3, Chronicle production ×3) were cut to their first ~4,300 words with the benchmark's own cutter and scored with the identical pinned instrument (Sonnet 4.6, temp 0, 3-run median). Below that, each book's Chapter 1 as written, plus Sherpa's Episode 1 — fragments under 2,000 words are below the instrument floor and are indicative only. Equal-length means: Chronicle × Opus 80.4, Chronicle production 77.1, Sherpa 56.3. Chapter-1 means: Opus 79.2, production 78.8, Sherpa Episode 1 63.8. Scored 2026-09-19; n=1 per fragment; one exploratory comparison, not a board result.

FragmentWordsComposite3 runsNote
Sherpa (Pocket FM) — B2, all 3 episodes433056.357.5 / 56.3 / 58.8the whole Sherpa output
Chronicle × Opus 5 — B2 r1 — first 4,371 words432578.878.8 / 78.8 / 80same cutter (cut-positions.py, paragraph boundaries), same length as the Sherpa output
Chronicle × Opus 5 — B2 r2 — first 4,371 words43658078.8 / 78.8 / 80same cutter (cut-positions.py, paragraph boundaries), same length as the Sherpa output
Chronicle × Opus 5 — B2 r3 — first 4,371 words432982.578.8 / 85 / 82.5same cutter (cut-positions.py, paragraph boundaries), same length as the Sherpa output
Chronicle production (Sonnet) — B2 r1 — first 4,371 words428378.878.8 / 78.8 / 78.8same cutter (cut-positions.py, paragraph boundaries), same length as the Sherpa output
Chronicle production (Sonnet) — B2 r2 — first 4,371 words432872.571.3 / 72.5 / 72.5same cutter (cut-positions.py, paragraph boundaries), same length as the Sherpa output
Chronicle production (Sonnet) — B2 r3 — first 4,371 words43228080 / 80 / 81.3same cutter (cut-positions.py, paragraph boundaries), same length as the Sherpa output
Sherpa — Episode 1 only148863.862.5 / 65 / 63.8below the 2,000-word instrument floor — indicative only
Chronicle × Opus 5 — B2 r1 — Chapter 1302281.382.5 / 78.8 / 82.5
Chronicle × Opus 5 — B2 r2 — Chapter 1113476.378.8 / 77.5 / 76.3below the 2,000-word instrument floor — indicative only
Chronicle × Opus 5 — B2 r3 — Chapter 17248080 / 80 / 80below the 2,000-word instrument floor — indicative only
Chronicle production (Sonnet) — B2 r1 — Chapter 124378080 / 80 / 78.8
Chronicle production (Sonnet) — B2 r2 — Chapter 1187378.878.8 / 78.8 / 78.8below the 2,000-word instrument floor — indicative only
Chronicle production (Sonnet) — B2 r3 — Chapter 1578377.577.5 / 75 / 77.5

Raw: sherpa-b2-001 score · sherpa-b2-001 summary · sherpa-b2-001 log · sherpa-b2-001 protocol · sherpa-b2-001 report

R&D frontier: in a preregistered off-benchmark validation, regenerating a novel's final act from a fresh, compact context recovered +9.7 ending-quality points at full length. It enters production through the standard gated release process, then a new benchmark round.