As Utah gives the AI power to prescribe some drugs, physicians warn of patient risks
State officials see the pilot as a way to expand healthcare access, but the American Medical Association warns that removing physicians from clinical decisions puts patients at risk.
An AI system is now prescribing medication to patients in Utah—no doctor required. The state has recently launched a pilot program that lets AI prescribe repeat medication, marking the first time in the U.S. that an AI has been given clinical authority without human oversight.
The program, which began last month, allows an AI system to verify patient prescription histories, walk them through clinical questions, and send approved renewals directly to pharmacies. State officials say the program is a way to reduce costs, prevent medication lapses, and ease the burden on doctors, particularly in rural areas where clinicians are already stretched thin. However, physician groups say they were worried about the risks that come with a lack of human oversight in even minor clinical decisions.
“Any time clinical decisions are made without appropriate physician oversight, patients are put at risk. Medicine isn’t just about arriving at an answer; it’s about judgment, context, and accountability,” John Whyte, CEO of the American Medical Association, told Fortune.
AI gets clincial authority
The state’s commerce department has waived certain rules for a year-long trial, which is run in partnership with health-tech startup Doctronic. Margaret Busse, Executive Director at Utah Department of Commerce, told Fortune that the waiver was part of a “regulatory mitigation” program designed to safely test innovative AI tools.
“We really hope that this can shed light on where AI can be used responsibly in low-risk interactions in medicine that can help drive down costs and increase access. Because we all know we have a crisis…we desperately need technological solutions to help drive those costs down,” she said.
Doctronic, which was founded in 2023 by co-founders Matt Pavelle and Dr. Adam Oskowitz, said the system could be used by tens of thousands of patients in Utah during its first year. If it’s proven to be safe, the co-founders said they are hoping to scale the operation to include new prescriptions in low-risk scenarios, such as antibiotic prescriptions. They say the system can help speed up the prescription process and help patients, doctors, and pharmacists.
“The whole process from starting a conversation with the AI to having a prescription waiting at a pharmacy for you can take less than 30 minutes, and frankly, almost always takes less than 30 minutes. So it’s super efficient,” Pavelle, one of the co-founders and co-CEO of Doctronic, said.
To ensure safety, the phased rollout requires that the first 250 renewals per medication category are reviewed by a doctor before the AI can process them independently. After this, 10% of subsequent interactions will be randomly sampled for safety, Adam Oskowitz, Doctronic AI co-founder, said. Pavelle added that the company also has doctors available if the patient or pharmacist has questions.