An AI early-warning system that predicts which patients are at risk of deteriorating while in hospital was associated with a decrease in unexpected deaths, a new study says.
The study, published Monday in the Canadian Medical Association Journal, found a 26 per cent reduction in non-palliative deaths among patients in St. Michael’s Hospital’s general internal medicine unit when the AI tool was used.
“We’ve seen that there is a lot of hype and excitement around artificial intelligence in medicine. We’ve also seen not as much actual deployment of these tools in real clinical environments,” said lead author Dr. Amol Verma, a general internal medicine specialist and scientist at the hospital in Toronto.
“This is an early example of a tool that’s deployed that was rigorously tested and evaluated and where it’s showing promise for actually helping improve patient care,” Verma, who is also a professor of AI research and education in medicine at the University of Toronto, said in an interview.
The technology called CHARTwatch continuously analyzed more than 100 different pieces of information about each patient in the unit, Verma said.
When the AI tool predicted that a patient was deteriorating, it sent an alert to physicians and nurses, prompting them to quickly intervene.
“The machine learning tool gathers the information that’s already routinely collected in a patient’s electronic medical record,” he said.
That includes information such as age and medical history, as well as measurements such as vital signs, blood pressure, heart rate and lab test results.
“It gathers all of that information to make a prediction about their risk of becoming more seriously ill going forward and then it updates its model predictions every hour based on how all of those things are changing over time,” Verma said.
Get weekly health news
Receive the latest medical news and health information delivered to you every Sunday.
If the clinician agreed with the AI prediction after examining the patient, necessary action was taken. That included transferring the patient to the intensive care unit, giving them antibiotics for serious infections such as sepsis, or monitoring the patient more frequently.
If a patient’s death was inevitable they would receive end-of-life care earlier than they might have otherwise, easing their suffering, Verma said.
“Importantly, the AI doesn’t tell the clinician ‘prescribe this drug, you know, intervene with this test or this treatment.’ That’s all still up to the judgment of the nurses and the doctors doing the care,” he said.
“It’s a signal that says, ‘hey, pay attention to this patient.’”
Those early-warning signals are important in busy hospital settings where each nurse or doctor is caring for many patients who are getting multiple lab tests, medical imaging and other interventions that could change their prognosis, Verma said.
“It’s just not possible for humans to keep their eyes on 20, 30 patients at the same time, all the time,” he said.
Muhammad Mamdani, co-senior author of the study, said AI can process large amounts of data about patients — and combining that with a human clinician’s judgment can lead to better care.
Physicians and nurses should always err on the side of caution when using the tool, said Mamdani, who is the vice president of data science and advanced analytics at Unity Health Toronto, which includes St. Michael’s Hospital.
“What we tell our clinicians is, ‘if you think this patient’s going to die, but the algorithm says, the AI says, ‘no, they’re fine,’ don’t believe the AI. Believe your gut,” he said.
“But if the AI says ‘this patient’s going to die’ and you don’t think so, don’t believe your gut. Believe the AI.’”
The study looked at non-palliative patient deaths in the general internal medicine unit between Nov. 1, 2020 and June 1, 2022 when the AI tool was used and compared them to a previous time period — Nov. 1, 2016 to June 1, 2020 — when the technology wasn’t used.
The researchers found a 2.1 per cent non-palliative death rate when AI wasn’t used, compared to 1.6 per cent when it was.
To reduce the possibility that the results could be attributed to the different time periods, the researchers used the hospital’s cardiology, respirology and nephrology units — which didn’t have the AI tool — as a comparison. None of those units showed a difference in non-palliative deaths between the two time periods.
The researchers controlled for potential confounding factors such as age. In addition, because the study period coincided with the COVID-19 pandemic, which could be another variable affecting the results, the researchers excluded data about COVID patients.
In total, the study included 13,649 patient admissions in the general internal medicine unit and 8,470 patient admissions in the cardiology, respiratory and nephrology comparison units.
Verma said although the results are promising, they should be interpreted with caution and a randomized control trial is needed to hold the AI research up to the same standard as drug and medication studies.
Ross Mitchell, a professor in the faculty of medicine at the University of Alberta who was not involved in the study, said the research was “very encouraging.”
“This specific technology, CHARTwatch, needs to be studied in a broader sense,” said Mitchell, who is the Alberta Health Services chair in AI in health.
“It needs to be deployed in more hospitals across Canada so that we can get more than one hospital involved.”