epocrates

Kaiser Permanente study suggests ambient AI may alleviate physician burnout, improve patient care

March 7, 2024

Ambient artificial intelligence (AI), a newer technology that enhances a doctor’s note taking capabilities, could soon be the invisible third party in the exam room. Companies behind this technology, like Abridge, the Pittsburgh-based startup that just secured $150 million in a series C funding round, claim that their AI scribes can unshackle physicians from their increasing clerical burdens and after-hours EHR work. (Landi, 2024)

The Permanente Medical Group (TPMG), Kaiser’s Northern California physician group, put AI scribe technology to the test. Early results from its regional pilot study suggest that both physicians and patients benefited from the technology, although authors stress that long-term development and wider deployment will require a rigorous evaluation framework that tracks engagement, effectiveness, quality, and safety. (Tierney, 2024)

How does it work?

To test whether the technology could reduce the burden of documentation and initial editing required by clinicians, TPMG worked with an external vendor that provided and trained an AI scribe. The ambient AI technology produced real-time transcripts of clinician-patient encounters by rapidly converting speech collected from microphones on the clinicians’ TPMG-provided smartphones into text and applied natural language processing techniques to summarize key clinical content. (Tierney, 2024)

TPMG has a track record of deploying cutting-edge medical software and had already deployed medical dictation technology to improve clinical documentation and workflows. The authors explained that because the newer ambient AI technology represented a potentially transformative improvement over existing software, TPMG embarked on a limited, 2-week pilot with 47 physician test users. The limited pilot study was successful enough that the company secured licenses for 10,000 physicians and staff and prepared to undergo a more in-depth study and implementation. (Pines, 2024)

What does the study show?

In October 2023, TPMG enabled ambient AI technology for its 10,000 physicians with the goal of identifying optimal approaches to effectively and safely using ambient AI scribes to support clinician workflow. Virtual interactive training sessions, as well a recorded webinar, were offered to users. Because of the fast pace of implementation, complete EHR integration wasn’t possible. Instead, documentation from patient encounters was recorded on smartphones and uploaded to a secure cloud, accessible by computer and interfaced for incorporation and editing within the EHR. Clinicians were instructed to gain permission from patients to use the AI scribe and patients’ verbal consent was recorded in the EHR. Trainers reinforced the ongoing responsibility of the physician to review all clinical documentation. (Tierney, 2024)

In the following 10 weeks, the ambient AI tool was used by 3,442 physicians and 303,266 patient encounters across an array of medical specialties and locations. Doctors cited many benefits of the technology, including its capability to facilitate more meaningful interactions and reduce the burden of after-hours note writing. Patient feedback was generally positive as well, with some describing their physicians as more attentive, possibly because ambient AI allows doctors to not have to record notes during the visit. (Tierney, 2024)

To assess effectiveness of the technology, the team used several aggregates to determine workload metrics, including time spent outside of working hours and time spent in notes per chart to assess changes before and after AI use for clinicians participating in the pilot study as compared to nonusers. Researchers found a statistically significant (P <0.05) association in primary care physicians between the use of the ambient AI scribe with larger decreases in time spent in the EHR outside of working hours and in time spent in notes during appointments. What researchers termed “high use” physicians appeared to exhibit the most time reductions. (Tierney, 2024)

While effective at note taking, the AI scribe was not perfect. Researchers identified two significant examples of “hallucinations,” or bits of conversation that didn’t occur in the patient encounter. In one instance, the doctor and patient discussed scheduling a prostate exam, but the AI scribe incorrectly documented that it had already happened. In another example, a physician mentioned concerns about a patient’s hands, feet, and mouth, and the AI scribe incorrectly indicated that the patient had been diagnosed with hand foot mouth disease. Such findings underscore the importance of reviewing AI-generated content for accuracy. (Tierney, 2024)

Sources:

Landi, H. (2024, February 23). Fierce Healthcare. Abridge clinches $150M to build out generative AI for medical documentation. https://www.fiercehealthcare.com/ai-and-machine-learning/abridge-clinches-150m-build-out-generative-ai-medical-documentation

Pines, J. (2024, March 4). Forbes. Your doctor’s office might be bugged. Here’s why. https://www.forbes.com/sites/jessepines/2024/03/04/your-doctors-office-might-be-bugged-heres-why/?sh=750200f124e7

Tierney, A., et al. (2024, February 21). NEJM Catal Innov Care Deliv. Ambient artificial intelligence scribes to alleviate the burden of clinical documentation. https://catalyst.nejm.org/doi/full/10.1056/CAT.23.0404