Changelog
- 2026-11-05: Published.
Two careful rankings of therapy AI scribes currently give opposite answers about the same two vendors, and both answers are honest. Scribechoice, an independent comparison site that ranks therapy scribes from vendor documentation, puts Mentalyc at the top of its ranking at 9.2 and Upheal next at 8.9 [1]. Our clinical pilot, which scored the notes the two tools actually produced from standardized synthetic psychiatric encounters, flagged both vendors "Unsafe as written" on two of three encounters. The contradiction is not a mistake in either ranking. It is a lesson in asking what a ranking measures before you buy on the strength of it.
What each ranking actually measures
Scribechoice ranks and compares therapy scribes on note formats, pricing, compliance posture, and EHR fit, with every fact sourced from the vendors' own public pages and re-checked monthly [1]. Its page states that rankings are editorial and independent, and that no vendor can pay for placement [1]. That is a documentation ranking: it answers whether a vendor publishes a strong feature set, clear pricing, and credible compliance terms.
Our pilot asks a different question. It ran three frozen synthetic psychiatric encounters (a depression and anxiety follow-up, an acute suicide-risk assessment, and a PTSD medication-management plus psychotherapy visit) through each vendor under disclosed settings and scored the resulting notes against a locked checklist for clinical accuracy, completeness, medication and treatment detail, and safety. Documentation quality and note accuracy are different properties; a vendor can be strong on one and weak on the other, as here.
What the pilot found on Mentalyc
Mentalyc scored 79.0 overall in the pilot under its intake, longer-note, and online settings, with two critical flags. The flagged notes were the depression and anxiety follow-up (78.5) and the suicide-risk assessment (84.5); the PTSD visit scored 74 with no critical errors.
- Wrong medication identity. The follow-up note documented Lexapro as citalopram and continued it as citalopram. Lexapro is escitalopram, not citalopram, and the error was scored Incorrect and flagged critical.
- Contradictory diagnosis. The suicide-risk note diagnosed major depressive disorder with psychotic features while its mental status examination denied hallucinations.
- Invented details alongside the criticals. The same notes also labeled a wife's sleeping pills "Collided" (the audio named zolpidem/Ambien) and invented tearful in-session exam language for the PTSD visit.
None of this erases what Mentalyc documents well. On our documentation-based research scorecard, Mentalyc (75/100, #5) among vendors with a documented BAA has a perfect 20/20 on privacy and security and the lowest published paid entry price in its group, at $19.99 per month for Mini or $14.99 billed annually. Our Mentalyc review still calls it the strongest documented fit for individual therapists and therapy groups. The pilot adds a separate fact: on these test encounters, its notes needed correction before clinical use.
What the pilot found on Upheal
Upheal scored 63.0 overall in the pilot under its default settings, with three critical flags, the most in the six-vendor cohort. Its follow-up scored 50.5 and its suicide-risk assessment 70.5, both flagged "Unsafe as written"; the PTSD visit scored 68 with no critical errors.
- Omitted medication details. The follow-up note never documented vitamin D or occasional ibuprofen, both predefined critical items for that encounter.
- Wrong medication ownership. The suicide-risk note grouped zolpidem with the patient's own bupropion, without attributing the zolpidem to his wife.
- Sparse capture beyond the criticals. Fifty-four checklist items were non-full across the three notes, which is why the overall score sits well below the rest of the cohort even apart from the critical flags.
Here, too, the documentation picture is different from the note-accuracy picture. Upheal (71/100, #6) among vendors with a documented BAA on our research scorecard, and our Upheal review calls it the lowest-cost way to start with AI notes, with a permanent free plan and AI Notes at $39 per month. A free plan and a strong feature list are real advantages. They do not answer whether a given draft note is safe to sign.
Why the disagreement is useful, not confusing
Three points keep this comparison honest. First, the pilot is a pilot: three synthetic encounters, one scored run per vendor per scenario, text-to-speech audio, and evaluations adjudicated by a single psychiatrist scorer with no second independent psychiatrist review yet. Its full protocol, including those limits, is published in the pilot methodology. Treat the flags as a reason to test carefully, not as a verdict on every note either vendor will ever produce.
Second, vendor settings differed, as the pilot discloses: Mentalyc was tested as an intake note in longer form online, Upheal on default settings, and the other four cohort vendors under their own named settings. Settings are part of the result, which is why the pilot page reports them next to every score rather than presenting a single leaderboard.
Third, the rest of the cohort shows the two kinds of evidence can agree. Heidi scored 96.0 with zero critical flags, Freed 91.0, JotPsych 90.2, and Blueprint 83.2, all with zero critical flags. The full results, with annotated evidence for each vendor, are on the clinical pilot results page. When documentation strength and note accuracy point the same way, a buying decision is easy. Mentalyc and Upheal are the cases where they point in different directions, and those are the cases where the choice of evidence matters most.
How to use two rankings that disagree
When a ranking and a pilot disagree, do not average them. Our site deliberately never combines the two scores, and for the same reason you should not either: a combined number would hide exactly the information you need. Instead, use each source for what it measures.
- Use a documentation ranking to build the shortlist. Feature coverage, pricing, BAA terms, and EHR fit decide which vendors are worth trialing at all. That is the job Scribechoice's ranking does well [1].
- Use note-accuracy evidence to test the shortlist. Run your own encounters through each finalist, and read the drafts the way the pilot reads them: medication names and ownership, diagnosis against the documented examination, and anything the note asserts that the session did not support.
- Check the failure mode, not just the score. A wrong SSRI identity and an omitted interacting medication are different risks from a missing scale score, and they call for different safeguards in your review routine.
- Price the review step, not just the subscription. If a vendor's drafts need heavier correction, that correction time is part of its cost. If you are weighing a standalone scribe against an EHR-bundled note tool, our psychiatric EHR comparison prices the bundled alternative across 13 EHRs, and the wider Practice Stack section covers the prescribing and billing workflow the note has to live inside.
The bottom line: Scribechoice's top two are the documentation ranking's answer, and our pilot's "Unsafe as written" flags are the note-accuracy answer, and a clinician buying a scribe needs both. Shortlist on documentation, decide on the drafts, and never let a ranking built from vendor pages substitute for reading what the tool actually writes.
Sources
- Scribechoice, "AI therapy scribes, ranked," https://www.scribechoice.com/ (opened 2026-11-05)