Disclosures
- How we make money: Some links may earn us a referral fee at no cost to you. Fees never affect scores, rankings, or what we cover, and we accept no paid placement.
- Status: Pilot results. Heidi scored under CAW Clinical Accuracy Benchmark v1.0.1; Blueprint, Freed, JotPsych and Mentalyc are pending. Affiliate links never change scores.
- Not advice: General information only, not medical, legal, or compliance advice. Synthetic encounters only; not a clinical-validation study.
Cohort leaderboard
Equal-weight mean of three frozen synthetic encounters (Linda CAW-102, Marcus CAW-103, Angela CAW-109). Critical-error count is reported separately from the point score. Checklist: CAW Clinical Accuracy Benchmark v1.0.1.
| Rank | Vendor | Mode | Overall | Linda | Marcus | Angela | Criticals | Status |
|---|---|---|---|---|---|---|---|---|
| 1 | Heidi | Goldilocks | 96.0 | 96 | 96 | 96 | 0 | Scored |
| - | Blueprint | - | - | - | - | - | - | Pending |
| - | Freed | - | - | - | - | - | - | Pending |
| - | JotPsych | - | - | - | - | - | - | Pending |
| - | Mentalyc | - | - | - | - | - | - | Pending |
Heidi at a glance
Overall mean: 96.0 · Critical errors: 0 (No critical errors identified) · Mode: Goldilocks · Note date: 2026-10-26 (America/Los_Angeles)
Ruling applied: L-MT09 Full (refills “sent” vs planned “I’ll send” judged too nitpicky): +1 Linda, overall 95.7 to 96.0. Same checklist rulings apply when later vendors are scored.
- Linda (CAW-102): 96 · CA 30/30, CO 23/25, MT 25/25, SR 18/20
- Marcus (CAW-103): 96 · CA 30/30, CO 25/25, MT 22/25, SR 19/20
- Angela (CAW-109): 96 · CA 30/30, CO 22/25, MT 24/25, SR 20/20
Case scores
| Case | Final | Clinical | Complete | Meds | Safety | Criticals | Safety status |
|---|---|---|---|---|---|---|---|
| Linda (CAW-102) | 96 | 30/30 | 23/25 | 25/25 | 18/20 | 0 | No critical errors identified |
| Marcus (CAW-103) | 96 | 30/30 | 25/25 | 22/25 | 19/20 | 0 | No critical errors identified |
| Angela (CAW-109) | 96 | 30/30 | 22/25 | 24/25 | 20/20 | 0 | No critical errors identified |
Case final = domain sum (Clinical 30 + Completeness 25 + Medication and treatment 25 + Safety 20) minus unsupported-addition deductions. Overall = equal-weight mean of the three case finals. Heidi had 0 unsupported-addition deductions on every case.
Remaining non-full items
Six non-full checklist items remain across the trio. None are critical. Published as scored; we do not invent additional misses.
| Case | Item | Capture | Points | Note |
|---|---|---|---|---|
| Linda | L-CO09 | Absent | 0 / 2 | Fewer-meds / stop-trazodone request not documented |
| Linda | L-SR02 | Partial | 2 / 4 | Orthostatic counseling incomplete (lightheadedness + no falls present) |
| Marcus | M-MT05 | Absent | 0 / 3 | Outpatient-unsafe judgment not stated |
| Marcus | M-SR08 | Absent | 0 / 1 | Not left alone while arranging transport/eval not documented |
| Angela | A-CO01 | Absent | 0 / 3 | At-home 3x homework course not documented |
| Angela | A-MT06 | Partial | 1 / 2 | Beach Saturday tentativeness partially lost |
Separate from research scores
These clinical-accuracy results are not the documentation-based research score. Heidi’s research score is unchanged. The two are never combined. Methodology: How we test AI scribes for psychiatry.
Pilot framing
- Audio is synthetic and fictional. No real patient information is used.
- Voices are text-to-speech.
- The number of scenarios and runs is small.
- This is a pilot protocol, not a clinical-validation claim, and results may not generalize to your practice.