TL;DR
ChartInsight ran a 433-record study that placed expert human medical summaries side by side with AI-reviewed indexes of the same records. ChartInsight's AI recovered 1,318 clinician-documented findings that the human summaries missed, spread across 92% of the records (398 of 433). Of those, 69 were safety-critical (missed opioids, anticoagulants, drug allergies, and hypertensive-crisis blood pressures). The takeaway is not that human reviewers are careless. It is that the constraint is bandwidth: when a person reads hundreds or thousands of pages, some documented findings slip through, and which ones slip is unpredictable. AI indexing does not read the record for you, but it catches what a single pass overlooks and links every finding to its source page so it can be verified.
The real problem with reviewing large records
Anyone who reviews medical records for a living, a workers' compensation paralegal building a chronology, a QME or AME writing a medical-legal report, a defense attorney stress-testing a claim, works from the same raw material: a multi-provider PDF that runs from a few hundred to several thousand pages. A typical workers' comp package is 500 to 5,000 pages from treating physicians, imaging centers, physical therapists, pharmacies, and surgeons. The reviewer's job is to pull out the chronology, the causation timeline, the medications, the provider contradictions, and the key diagnostic findings, and to do it in a way that holds up when an adjuster, opposing counsel, or judge challenges it.
The quiet assumption in that workflow is that a careful, experienced reviewer captures everything that matters. The 433-record study set out to test that assumption directly. The result should change how anyone who relies on a medical summary thinks about what "complete" means.
What the study measured
The study compared 433 processed medical records, each with an expert human "Medical Summary" and an AI-generated index of the same underlying record. It was conducted by Yoelvis Orozco Gonzalez, PhD, Principal Machine Learning Engineer at Gemini Legal, and the full methodology and findings are published on the ChartInsight accuracy page.
The review was semantic, not mechanical. Each record was read by comparing the full AI index against the full human summary. A finding was counted as "missing" only when it was a clinically meaningful, documented item that appeared in the AI index but did not appear, by any name, synonym, or paraphrase, in the human summary. This is the opposite of a keyword diff, and the study applied deliberately conservative standards so the count would understate rather than overstate the gap.
| Conservative standard | What it means |
|---|---|
| Synonym matching | "Osteoarthritis" in the summary covers "arthritis" in the index. HTN = hypertension. Norco = hydrocodone. CTS = carpal tunnel syndrome. |
| Negation exclusion | "No fracture" or "denies chest pain" is not counted as a missed finding. Only affirmative, documented findings count. |
| Hypothetical exclusion | "In case of gastritis" or "if symptoms recur" is not counted as a diagnosis. Only confirmed findings count. |
| AI-error exclusion | When the AI extracted something the source contradicted (for example "breast cancer" when all imaging read benign), it was removed from the count. |
| Questionnaire separation | Self-reported intake-questionnaire items are counted separately. The 1,318 headline figure refers only to clinician-documented findings. |
Two points matter for reading the numbers honestly. First, this measures coverage (recall), how much of the documented record each summary captured, not a blind "accuracy percentage" of the AI. Second, it is a single-reviewer semantic comparison rather than a multi-rater clinical trial. Those are real limitations. They are also why the standards were set to favor the human summary at every ambiguous call.
The headline result
Across the 433 records, the AI index surfaced 1,318 clinician-documented findings absent from the human summaries. The gaps were not confined to a few weak summaries:
- 92% of records (398 of 433) contained at least one AI-only clinician-documented finding.
- 384 records (89%) had a clinician-documented gap; only 49 (11%) had complete coverage.
- The mean was 3.04 missed findings per record, with a median of 3.
Here is how the 1,318 findings break down by clinical category, per the study data:
| Category | Count | What it covers |
|---|---|---|
| Clinical findings | 430 | Positive exam signs, imaging results, abnormal vitals documented by treating clinicians |
| Conditions | 428 | Diagnoses: hypertension, trigeminal neuralgia, sleep apnea, diabetes, frozen shoulder, and more |
| Medications | 167 | Prescribed drugs, including 38 opioids or controlled substances and 5 anticoagulants |
| Procedures | 113 | Surgeries, injections, nerve blocks, imaging studies, electrodiagnostic tests |
| Symptoms | 77 | Clinician-documented symptoms (not self-reported questionnaire items) |
| Lab results | 74 | Abnormal thyroid, renal, glycemic, lipid, infectious-disease, and vitamin panels |
| Clinical history | 13 | Prior medical-history entries relevant to current treatment |
| Drug allergies | 7 | Sulfa, penicillin, amlodipine, latex, each one a prescribing hazard |
| Other | 9 | Imaging findings, substance use, vital signs, work-status entries |
Conditions and clinical findings dominate the count, but the smallest categories carry outsized risk. Seven missed drug allergies and 167 missed medications, including opioids and anticoagulants, are exactly the items a downstream decision-maker cannot afford to work without.
The 69 findings where a gap means patient risk
Not every missed finding carries equal weight. The study flagged 69 findings across 54 records (12.5% of the dataset) where the absence from the human summary creates concrete, immediate risk, the kind of finding where a clinician making a decision without it could directly harm the patient.
| Safety-critical category | Count | Examples |
|---|---|---|
| Opioids and controlled substances | 38 | Fentanyl 100 mcg, Dilaudid, Norco, Percocet, Tramadol, Oxycodone, Methadone |
| Severely elevated blood pressure | 11 | BP 233/121 (hypertensive crisis), 179/112, 174/117, 162/104 (pre-operative) |
| Drug allergies | 7 | Sulfa, amlodipine, penicillin (with respiratory component), latex |
| Acute life-threatening conditions | 7 | Right hemiparesis status post t-PA (stroke), pulmonary embolism, DVT |
| Anticoagulants | 5 | Xarelto (rivaroxaban), Enoxaparin, Plavix, Clopidogrel |
| Cardiac emergency medication | 1 | Nitroglycerin / Nitrostat |
Consider one documented case: a competent human summary captured the patient's chest pain, atrial fibrillation, prior myocardial infarction, echocardiogram findings, and metoprolol, but omitted a sulfa allergy recorded in the clinical notes. Sulfonamide antibiotics are prescribed routinely for urinary and skin infections. Nothing in that summary would stop the next prescriber. As the study puts it, a missed allergy before prescribing is not a theoretical risk, it is an anaphylaxis risk. These are not summary-quality problems. They are bandwidth problems.
Effort does not close the gap
The intuitive fix is "write a longer, more thorough summary." The data says that does not work.
One record in the dataset had a human summary of 112,000 characters, roughly 56 pages of single-spaced text, an exhaustive document by any measure. The semantic review still found nine clinician-documented items the summary omitted, including diabetes mellitus type 2, an acute right hemiparesis status post t-PA (a stroke treated with clot-busting medication), symptomatic cholelithiasis, a revision to laparoscopic sleeve gastrectomy, H. pylori gastritis, and plantar fasciitis. Length was not the differentiator.
"The bottleneck isn't human effort, it's human bandwidth." (Findings of the 433-record study, Gemini Legal)
Most records (187) had three to five clinician-documented gaps. Only two records exceeded ten. And the 49 records with complete coverage prove the point in the other direction: perfect capture is possible, it is just not reliably repeatable across hundreds of pages by manual reading alone. This tracks a long-standing patient-safety theme. The landmark Institute of Medicine report To Err Is Human established that most clinical errors trace to systems and process limits rather than individual negligence. Its successor report, Improving Diagnosis in Health Care, found that gaps in synthesizing and communicating information across a patient's record are a leading contributor to diagnostic error. And medication reconciliation, matching a patient's full medication list across care transitions, remains one of the most error-prone steps in care, per AHRQ PSNet. A record review that depends on one person holding every detail in working memory is running into the same wall.
Why this matters for your work
The study is clinical in its examples, but its consequences land squarely on the people who build documents from these records.
For workers' comp attorneys and paralegals, the chronology, demand letter, deposition prep, and MSC brief are only as strong as the findings underneath them. A missed medication, a missed prior condition, or a missed positive exam finding is exactly what opposing counsel or an adjuster probes for. In one forearm-injury record, the human summary captured the work injury perfectly but missed a positive Finkelstein test documented in a treating physician's note, the kind of detail that shapes causation and apportionment arguments. Defensibility depends on catching those details and being able to point to the exact page they came from.
For QME, AME, and IME physicians, the medical-legal report has to address diagnosis, causation, apportionment, and permanent disability, and every clinical finding it asserts must cite its source. A reviewer is legally required to review the entire record, so the value is not reading less. It is removing the manual grind of assembling and citing findings, and having a second, systematic pass that flags the abnormal blood pressure or the anticoagulant a first read did not register.
How ChartInsight closes the gap
The 433-record study used ChartInsight's index as the comparison against human summaries. ChartInsight, built by Gemini Legal (a legal-support company that has processed more than 100 million pages over 20+ years), turns a large PDF record set into a structured, source-linked review:
- Live citations on every finding. Every chronology entry, summary sentence, extracted vital, and medication is one click from the exact source page in the built-in PDF viewer. Nothing is asserted without a citation, which is precisely what makes the output defensible.
- Structured chronologies across all providers, sortable by date or grouped by provider.
- Narrative summaries, vitals, and medications pre-extracted on every record, each cited to its page.
- A per-record research assistant that answers a question about the record with a cited answer.
The point is not that AI replaces the reviewer's judgment. It is that a systematic index plus human review catches more than either alone, and every finding stays traceable to the page it came from.
Key takeaways
| Question | Answer |
|---|---|
| What did the study compare? | 433 expert human medical summaries vs AI indexes of the same records |
| What did it find? | 1,318 clinician-documented findings the human summaries missed |
| How widespread? | 92% of records (398/433) had at least one AI-only finding; mean 3.04 per record |
| How risky? | 69 safety-critical findings (opioids, anticoagulants, allergies, crisis BP) across 54 records |
| Does more effort help? | No, a 112,000-character summary still missed 9 findings, including a stroke |
| What actually helps? | A systematic, source-cited index reviewed by a human, so nothing depends on one pass of memory |
FAQ
How accurate are AI medical record reviews?
In ChartInsight's 433-record study, the AI index recovered 1,318 clinician-documented findings that expert human summaries had missed, across 92% of records. Measured as coverage of the documented record, the AI index captured findings a single human pass overlooked in the large majority of cases. Accuracy still depends on human verification, which is why every ChartInsight finding links to its source page.
Do human medical summaries miss findings?
Yes, and more often than most reviewers expect. In the study, 89% of records (384 of 433) had at least one clinician-documented finding absent from the human summary, with a mean of 3.04 misses per record. Only 11% of summaries had complete coverage. The cause is bandwidth, not carelessness.
Is AI more accurate than a human at reviewing medical records?
The study does not claim AI beats humans outright. It shows the AI index caught documented findings that human summaries missed, while noting that the AI occasionally extracted errors (which were excluded from the count). The strongest result comes from combining a systematic index with human review, each catches what the other misses.
Does using AI mean reviewing less of the record?
No. A QME, AME, or IME is legally required to review the entire record, and ChartInsight does not change that. It removes the manual work of assembling findings and building page citations by hand, and adds a systematic pass that surfaces items a first read can miss. The reviewer reads the whole record, with less mechanical overhead.
Recommended links
Nicola Riker
Senior Full-Stack Engineer
Nicola is a founding engineer for ChartInsight and Senior Software Engineer at Gemini Legal. She helped build ChartInsight from scratch alongside Alex Solo, drawing on the firm's 20 years of workers' comp experience to design a tool that actually fits how attorneys and physicians work.