Stop Curating Medical Data. Start Making It Available.

For the last decade, the playbook for unstructured data was the same: build a schema, extract the fields, normalize everything, query the clean version. It made sense when models couldn't read.

Models can read now. GPT-4 hit 99.8% accuracy extracting structured data from unstructured pathology reports in an NIH study: 369 patients, 4,797 data points, 13 variables. GPT-4o hit 91.4% retrieving patient demographics, diagnostics, and pharmacological data from unstructured medical reports. Both on raw clinical text.

Context windows have crossed 1 million tokens. A full patient file fits in a single pass.

If frontier models can read the original record better than an extraction pipeline can summarize it, continuing to force data through rigid extraction schemas becomes counterproductive.

The schema was always the bias

Extract, normalize, model, query. Four stages, each one deciding what matters before the next stage ever sees the data. That felt like rigor. It was opinion, encoded as infrastructure.

A schema is a list of things you already know to look for: ICD codes, provider names, dates of service, prescribed medications. All important, and all anticipated.

The signal that changes outcomes is rarely the anticipated signal. It is the offhand provider note, the inconsistent symptom timeline, or the medication history that contradicts the diagnosis three visits later.

When you curate a clinical note into a structured field, you encode one interpretation as fact. The model needs the raw test results, the vitals, and the lab work. Give it the source data and let it reason directly over the record.

Schemas drop unmapped signal by design.

Adding LLMs to the extraction pipeline can compound this issue. Research published in Applied Sciences (2026) documents how LLM-assisted knowledge graph construction introduces compounded bias through hallucinated triples, like (Drug X, cures, Disease Y), from text that only indicated symptom reduction.

Schema-based paradigms force normalization that overstates evidence and strips nuance before the model ever reasons over the full record.

The real challenge

Medical records are long, spanning thousands of pages across dozens of providers and years of care. Volume used to be the primary barrier. Context windows have expanded to handle over 1 million tokens in a single pass, allowing a complete patient file to fit.

The remaining engineering challenge is temporal reasoning across a patient timeline: connecting a provider note on page 12 to a medication change on page 247 to an outcome documented six months later. Clinical benchmarks are starting to test for this.

Longer context windows improve clinical prediction on complex patient records: a Mamba-based model beat prior state of the art on 9 of 14 tasks (ICLR, 2025). That study surfaced three specific failure modes in clinical records:

LLMs tested on long clinical documents show strong performance on extraction and sorting, but consistent failure on identifying missing information (LongHealth, 2025). Schema-first extraction shares this blind spot because a pipeline cannot flag what it was never built to inspect.

Effective long-context use has improved over 250x in nine months (Epoch AI, 2026). The volume problem is effectively addressed. The central question now is whether the architecture makes the record easy to reason through: page order preserved, visits sequenced, providers attributed, and timelines intact.

This is an architecture problem, not a curation problem.

From filtering to fidelity

Filtering takes a model capable of reading raw clinical text and pre-filters what it can see through an extraction schema.

The alternative is the Full Record Principle: stop deciding what the model gets to see, and guarantee it sees the complete source material, structured for reasoning.

Where this leaves data engineering

The question for anyone building on medical data is not "what fields do you extract?" It is "can the model re-read the actual record when you ask it a new question?" The first question measures curation; the second measures whether the model has access to the full evidence.

Frontier models read clinical text better than most extraction pipelines summarize it. The accuracy gap between a model reading the original text and a pipeline extracting a summary continues to widen.

Medical records are clinical histories first. The same temporal reasoning needed to understand patient outcomes applies directly to legal and clinical workflows. Fidelity serves both.

Sources: NEJM AI: LLM-Mediated Data Extraction from Pathology Reports (2025), JMIR AI: LLM Retrieval of Patient Information from Medical Reports (2025), Context Clues: Long Context Models for Clinical Prediction on EHRs (ICLR, 2025), LongHealth: QA Benchmark with Long Clinical Documents (Springer, 2025), MDPI: Analyzing Bias in LLM-Augmented Knowledge Graph Systems (2026), Epoch AI: Context Windows Data Insights (2026).