Caution

The first 250 renewals for each drug class will be reviewed by real doctors, but after that, the AI chatbot will be on its own. Adam Oskowitz, Doctronic co-founder and a professor at the University of California, San Francisco, told Politico that the AI chatbot is designed to err on the side of safety and escalate any case with uncertainty to a real doctor.

“Utah’s approach to regulatory mitigation strikes a vital balance between fostering innovation and ensuring consumer safety,” Margaret Woolley Busse, executive director of the Utah Department of Commerce, said in a statement.

For now, it’s unclear if the Food and Drug Administration will step in to regulate AI prescribing. On the one hand, prescription renewals are a matter of practicing medicine, which falls under state governance. However, Politico notes that the FDA has said that it has the authority to regulate medical devices used to diagnose, treat, or prevent disease.

In a statement, Robert Steinbrook, health research group director at watchdog Public Citizen, blasted Doctronic’s program and the lack of oversight. “AI should not be autonomously refilling prescriptions, nor identifying itself as an ‘AI doctor,'” Steinbrook said.

“Although the thoughtful application of AI can help to improve aspects of medical care, the Utah pilot program is a dangerous first step toward more autonomous medical practice,” he said. “The FDA and other federal regulatory agencies cannot look the other way when AI applications undermine the essential human clinician role in prescribing and renewing medications.”