The LLM writes prose. The code owns the facts.
In a clinical study report, every serious adverse event needs a patient narrative: what happened, when it happened, what the patient was taking and how it ended. Medical writers spend days on these. A language model can draft one in seconds, and it will sometimes invent a date to make the paragraph flow.
In most products a made-up detail is just embarrassing. Here it's a regulatory finding. So NaraAI, where I’m the technical advisor, is built on one rule:
The model writes the prose. Deterministic code owns every fact.
1. Facts first, no model involved
The first stage reads the trial data (SDTM) and builds a fact sheet for each event that needs a narrative. Each fact gets a deterministic ID, like AE4.onset_date, and cites exactly one source row. If a value is missing, there is no fact, so the model has nothing to state. Partial dates stay partial. If a medication might overlap the event, it's included and marked “possible”. It's never silently dropped.
2. The model may only cite
The composer sends the model one section at a time, along with only that section's facts. It has to return sentences as JSON, with fact IDs cited in each one. A deterministic rules-based writer sits behind the same interface. It runs offline, and it sets the quality floor the model has to beat.
3. Verify, don't fix
A verifier checks every sentence: uncited claims, unknown fact IDs, wrong dates, imputed partial dates, invented numbers, omitted facts, and contradictions in seriousness, causality or outcome. That's 14 checks today. Each check has an adversarial test that plants the error on purpose and asserts the verifier catches it. When the source data itself is inconsistent, the verifier raises a data query for a human. It never quietly fixes the data.
On the public CDISC pilot study this is already paying off. All three fatal adverse events were coded “not serious” in the source data, and each one raised a query.
4. Evals are the release gate
Golden narratives approved by a clinical lead are the target. The eval harness measures fact recall, edit effort and the verifier pass rate, and CI blocks a release that regresses. Prompts, templates and model IDs are versioned config, and changing them means bumping a version.
Why this matters outside pharma
Any AI feature has the same split. Some parts have to be true (a price, a schedule time, a follower count) and some parts are allowed to be flexible (the wording). If you keep the true parts in code, make the model cite them, and test the failure you're afraid of, you can ship AI features users can trust, and engineers can change without fear.
See the early demo (public CDISC Pilot 01 data only, still in development) · Back to home