Effective use of AI in research
Artificial intelligence is changing scientific research. Learning how to best use it could benefit your career and the research itself.
To use AI or not, that was the question. Panelists at Monday’s fireside chat, “The AI-Power-User vs. the Editor-in-Chief: Responsible Acceleration of Science in the Age of Generative Intelligence,” answered with a resounding yes. The real question, they said, is how to use it to supplement and improve scientific research in anesthesiology.
Although AI should not be used to write research papers, session panelist Franklin Dexter, MD, PhD, FASA, said it can be used to do some of the work behind the research.
“For the kind of statistical work I do, I cannot imagine doing it without AI,” said Dr. Dexter, who is Director of the Division of Management Consulting and Professor in Anesthesia at the University of Iowa College of Medicine in Iowa City, Iowa. “It comes up with techniques that are entirely different in terms of the algorithms that I would consider, and it saves time.”
However, he added, the human element should never be replaced. Specifically, he said AI should never be allowed to do any coding on its own, because most large language models (LLMs) are not advanced enough and thus increase the risk for errors.
“I don’t find that it works without having large amounts of my involvement day-by-day, step-by-step,” he said. “But for my scientific work, I would not be nearly as productive as I am without using it.”
Panelist Steven Shafer, MD, FASA, disagreed with Dr. Dexter about letting AI create code. In fact, he advocated for having AI be used to both write and test code.
“I am empirical. I just want to know, does it work? Or does it not work?” said Dr. Shafer, who is Emeritus Professor of Anesthesiology, Perioperative and Pain Medicine at Stanford University and former Editor-in-Chief of the ASA Monitor. “I use (AI service) Claude to write code and ChatGPT to rip it apart. Adversarial review is essential to the process.”
Dr. Dexter said he doesn’t use Claude for adversarial reviews, instead preferring to use Gemini and ChatGPT. Both panelists encouraged the audience to avoid the free versions of both services because they are not as advanced and cannot handle the in-depth science.
Despite the virtues being extolled by the panel, session moderator Michael Fettiplace, MD, PhD, said most scientific journals still resist allowing AI in research papers. Dr. Shafer said he believes the issue is money.
“(The publishers) are creating their own in-house AI models that they can monetize,” he said. “The problem is that the LLMs will always outperform them because (models created by publishers) have a smaller database to draw from."
Once publishers see the benefits, including the amount of time that can be saved, Dr. Dexter believes they will engage. They will also come to understand that a human presence is a necessity.
“Reviewing is so much work for these journals,” he said. “The problem is it needs to be done well. If you have someone who doesn’t know the science, (the review) will fail miserably. The goal is to increase the production of very knowledgeable editors.”
Dr. Fettiplace, who is Assistant Professor of Anesthesiology at the University of Illinois College of Medicine in Chicago, asked the two panelists where they see the future headed when it comes to AI in research. Both had diverging views.
“My job is to implement stuff at hospitals,” said Dr. Dexter. “The difference (with AI) is that the cost of coding is so much cheaper, so the number of projects keeps going up. I don’t think there’s going to be any reduction in people because of that increased number of projects.”
“I think the math will change,” added Dr. Shafer. “Mathematicians will go from pushing lead around on a piece of paper to trying to figure out what AI has put forward.”
For his part, Dr. Fettiplace said he believes AI will be a boon because of the valuable medical data generated by research in areas like anesthesiology.
“The real value for (anesthesiology) now is patient data, but it’s all regulated by HIPAA,” he said. “But the other piece for us is randomized controlled data. As a specialty, we should be pushing for more quality randomized trials that are going to create data to go into these LLMs.”
