The 433-Record Study
We compared 433 expert human Medical Summaries to AI-generated indexes of the same records. The AI recovered 1,318 clinician-documented findings the humans missed, across 92% of cases.
Study by Yoelvis Orozco Gonzalez, PhD, Principal Machine Learning Engineer, Gemini Legal
433 processed medical records · Semantic review methodology
Coverage
of records had AI-only valued information
398 of 433 records
Recovered
clinician-documented findings missed by human summaries
Records
records with clinician-documented gaps
89% of all records
Safety
safety-critical findings
allergies, opioids, anticoagulants, severe BP
Methodology
How we judged “missing”: semantically, not mechanically
Findings by category
What humans missed, by type
Of the 1,318 clinician-documented findings the AI recovered, here is how they break down by clinical category:
| Category | Share of 1,318 | Count | What this means |
|---|---|---|---|
| Clinical findings | 430 | Positive exam signs, imaging results, abnormal vitals documented by treating clinicians | |
| Conditions | 428 | Diagnoses: hypertension, trigeminal neuralgia, sleep apnea, diabetes, frozen shoulder, etc. | |
| Medications | 167 | Prescribed drugs including opioids (38), anticoagulants (5), antibiotics, corticosteroids, neuropathic agents | |
| Procedures | 113 | Surgeries, injections, nerve blocks, imaging studies, electrodiagnostic tests, ERCP | |
| Symptoms | 77 | Clinician-documented patient symptoms (not self-reported questionnaire items) | |
| Lab results | 74 | Abnormal thyroid, renal, glycemic, lipid, infectious disease, and vitamin panels | |
| Clinical history | 13 | Prior medical history entries relevant to current treatment | |
| Drug allergies | 7 | Sulfa, penicillin, amlodipine, latex, each one a prescribing hazard | |
| Other | 9 | Imaging findings, substance use, vital signs, and work status entries |
167 missed medications included 38 opioids or controlled substances and 5 anticoagulants, drug classes where a gap means duplicate prescribing risk or unmanaged bleeding risk.
Prevalence
How widespread were the gaps?
Of 433 records reviewed, 384 (89%) contained at least one clinician-documented finding absent from the human summary. Only 49 records (11%) had complete coverage.
Record coverage
Of 433 records reviewed
The 1,318 findings, by priority
Clinician-documented findings only
Safety-critical findings
69 findings where the gap means patient risk
Not all missed findings carry equal weight. We identified 69 findings across 54 records (12.5% of the dataset) where the absence from the human summary creates immediate, concrete patient safety risk. These are findings where a downstream clinician making a decision without this information could directly harm the patient.
| Category | Count | Examples |
|---|---|---|
| Opioids and controlled substances | 38 | Fentanyl 100 mcg, Dilaudid, Norco, Percocet, Tramadol, Oxycodone, Hydrocodone, Methadone, Codeine |
| Severely elevated blood pressure | 11 | BP 233/121 (hypertensive crisis), BP 179/112, BP 174/117, BP 166/98, BP 162/104 (pre-operative) |
| Drug allergies | 7 | Sulfa allergy, amlodipine allergy, penicillin allergy (with respiratory component), latex allergy |
| Acute life-threatening conditions | 7 | Right hemiparesis status post t-PA (stroke requiring clot-busting medication), pulmonary embolism, DVT |
| Anticoagulants | 5 | Xarelto (rivaroxaban), Enoxaparin, Plavix 75 mg, Clopidogrel |
| Cardiac emergency medication | 1 | Nitroglycerin / Nitrostat |
| Total | 69 |
A pre-operative blood pressure of 162/104 that doesn’t appear in the summary means the surgeon has no signal to re-check or manage blood pressure before operating. A missed sulfa allergy before prescribing a sulfonamide is not a theoretical risk. It is an anaphylaxis risk.
The bandwidth ceiling
The gaps don’t shrink with effort
The intuitive assumption is that longer, more thorough human summaries miss less. They don’t.
One record had a human Medical Summary of 112,000 characters, roughly 56 pages of single-spaced text. An exhaustive document by any measure. The semantic review still found 9 clinician-documented items the summary omitted, including diabetes mellitus type 2, acute right hemiparesis status post t-PA, symptomatic cholelithiasis, revision to laparoscopic sleeve gastrectomy, H. pylori gastritis, and plantar fasciitis.
In another record (a workers’ compensation forearm-injury case), the human summary captured the work injury perfectly. But it missed a positive Finkelstein test documented in a treating physician’s note.
When even a 112,000-character summary leaves out a stroke and a prior bariatric surgery, the lesson is clear: the bottleneck isn’t human effort, it’s human bandwidth. A person reading hundreds of pages cannot hold every detail, and the details that slip through are unpredictable.
Clinician-documented misses per record
Distribution across 433 records
Most records (187) had 3 to 5 clinician-documented gaps. Only 2 exceeded 10, and 49 had none.
“The bottleneck isn’t human effort, it’s human bandwidth.”
Case studies
Four records, in detail
Each case study below examines a single record from the 433-record dataset, what the human summary covered, what the AI found that was missing, and why the gap matters clinically.
6 findings missed
The prescribing hazard
A sulfa allergy, tramadol, and nitroglycerin, all documented in clinical notes, none in the human summary. A missed allergy before prescribing is not a theoretical risk.
Read case study →6 findings missed
The missing chapter
An entire hospitalization (gallstone disease, ERCP, robotic cholecystectomy, post-op infection) reconstructed by the AI from operative notes the human never summarized.
Read case study →9 findings missed
The 112,000-character summary
The longest, most exhaustive human summary in the dataset. The AI still found a stroke, a prior bariatric surgery, and diabetes that weren’t in it.
Read case study →5 findings missed
The perioperative blind spot
Obstructive sleep apnea, morbid obesity (BMI 59.1), and left foot drop, none in the summary. Each one changes surgical planning.
Read case study →