Clinician portal replies slowed after AI rollout

Clinical takeaway: Don’t count on AI drafts to lighten after-hours portal work. Replying outside working hours didn’t fall overall, and the faster turnaround appears limited to messages where clinicians used a draft. 

Patient portal messages now crowd the clinical day, and many clinicians answer them outside working hours. Primary care message volume doubled from 2018 to 2021. Health systems have since turned to an Epic tool that uses a large language model to write a first reply for clinicians to edit and send or discard. 

In a prior randomized study, primary care physicians had no significant change in time spent replying, because longer reading time canceled out the drafting benefit. Reports of faster turnaround came from comparing messages where clinicians chose a draft with messages where they didn’t, a design that favors simpler, more routine inquiries. Whether drafts also move reply work out of after-hours time was less studied. This retrospective cohort study compared each clinician before and after adopting the AI response tool. 

Clinicians answered portal messages 8.6% slower after they began using the tool, yet messages answered with a draft went out 29.5% faster than messages the same clinicians answered without one. The gain was largest for laboratory result messages, at 35.9%. Clinicians used a draft on about 5% of messages after they began using the tool, and more experience with it did not clearly shorten replies. 

Replying outside working hours did not fall, as neither adoption nor draft use was linked to a change in the share of after-hours replies. Draft use was tied to decreased after-hours replies for laboratory result messages that arrived during working hours and increased after-hours replies for messages that arrived after hours. Neither shift held when working hours were widened to 8 AM to 6 PM. 

The study drew on 983,154 first responses to patient portal messages from 1,542 clinicians and staff at one academic medical center, Columbia University Irving Medical Center, between March 2023 and May 2025. Turnaround ran from patient message to first reply. Each clinician served as their own comparison. Clinicians chose message by message whether to use a draft, so unmeasured differences between messages could remain. Message categories were estimated by classifier. 

“The binding constraint is not only how well the AI models perform but the organizational complements: local tuning, clinician trust in the system, message routing, role design, and cultural norms around both AI and the inbox,” said A Jay Holmgren, PhD, MHI, of the Division of Clinical Informatics and Digital Transformation at the University of California, San Francisco, in an accompanying commentary. 

The authors propose that a draft may lower the barrier to starting a reply rather than shorten the writing, and measuring time to drafting would test that. Patient message volume also needs to be measured. If easier replies just bring more messages, clinician response behavior could shift in ways this analysis did not capture. 

“Realizing the potential of AI will require more than bolting new technology onto existing workflows,” Holmgren concluded. “It will require discovering how clinicians, teams, and organizations should work differently as new technology arrives.” 

Source: Jeanselme V, et al. (2026 Oct 30) JAMA Netw Open. AI-Generated Draft Response Tool, Message Turnaround Time, and After-Hours Replies Among Health Care Professionals