Artificial intelligence (AI) has potential uses in health care and palliative care. It may help clinicians recognize weakening earlier, analyze imaging, identify important information buried in enormous medical records, identify patients who could benefit from palliative care, predict prognosis, and help clinicians and patients understand which treatments are most likely to be beneficial. However, researchers in palliative care are already asking what happens when AI algorithms are involved in decisions surrounding serious illness and dying.
Oh, Demiris, and Ulrich (2025) examined artificial intelligence in palliative care using the ethical principles of beneficence, nonmaleficence, autonomy, justice, and explicability. They discussed applications that include mortality prediction, identification of psychological symptoms, and artificial intelligence systems designed to support caregivers. Their work highlights an important issue because, while palliative care is person-and family-centered, AI has never been a part of the process.
Artificial intelligence may help us understand what is likely to happen to a patient, but it cannot determine what should matter to that patient. Imagine a person with advanced cancer and an AI system analyzing thousands of pieces of information. It predicts that another hospitalization has only a small probability of extending that person’s life. While that prediction could be extremely useful information, it cannot determine what that probability means to the person living that life.
One person might say that if there is a reasonable possibility of gaining another month, that they want to try another treatment. Another might say that spending another day in an intensive care unit is unacceptable. A third person might say that remaining alert enough to recognize and talk with family is more important than living longer. Someone else might say that remaining at home is the most important goal. Those differences are not failures of mathematics but rather they reflect human values.
An algorithm may eventually become good at predicting what is likely to happen medically. But a probability is not the same thing as a preference, a prognosis is not the same thing as a person’s goal, and an algorithm is not the same thing as a person’s values. A prediction based on thousands or millions of previous patients also cannot tell us anything about the person sitting in front of us. Most importantly, the algorithm cannot tell us what that individual considers an acceptable quality of life. Artificial intelligence can inform the conversation, but it should not become the source of the values governing that conversation.
Now imagine that an artificial intelligence system predicts that a critically ill patient has a very small probability of surviving intensive treatment. There may be good reasons for clinicians to ask for that information. A more accurate prediction might help them explain prognosis to the patient and family, and it might help everyone begin an important conversation about goals of care. However, allowing an algorithm to independently determine that treatment will not be offered is an entirely different proposition. We already have insurance companies using data to either approve or deny surgeries and medical treatments despite the recommendations of health care practitioners.
Autonomy is one of the central principles of health-care ethics. People have the right to participate in decisions about their own bodies and medical care. For that participation to be meaningful, accurate information, an understanding of options, and the opportunity to make decisions consistent with their values is needed. Statistical predictions describe probabilities, but they do not determine the worth of an individual human life or individual values.
Artificial intelligence could strengthen autonomy if it helps people understand complicated information. It could, however, weaken autonomy if its recommendations are presented in ways that make them appear unquestionable. Imagine being told that “the computer says” another treatment will not work. That statement sounds very different from a health care practitioner saying that the available evidence suggests the treatment has a small chance of helping and then asking the patient what matters most.
The first statement may close a conversation, the second can begin one. As artificial intelligence becomes increasingly involved in healthcare, we need to make certain that AI probabilities and predictions support shared decision-making and not replace it.
| Key questions for examining the ethical dimensions of AI integration examples in palliative care. Oh, Demiris, and Ulrich (2025) | |||
| Predicting mortality using machine learning | Detecting psychological symptoms using natural language processing | AI chatbots | |
| Beneficence | Can mortality prediction based on machine learning improve patients’ well-being at the end of life? | Will using natural language processing for automated symptom screening promote the well-being of seriously ill individuals? | Will AI chatbots improve caregivers’ access to healthcare information? |
| Nonmaleficence | Will palliative care consultations initiated by machine learning algorithms provoke harm to patients and their families? | Will using natural language processing to review clinical notes infringe on patients’ personal privacy? | What harm could arise from the outputs generated by AI chatbots? |
| Autonomy | To what extent will the computed mortality shape providers’ decisions about whom to recommend for palliative care? | Does automated symptom screening respect patients’ autonomy to determine their own emotional state? | Would the use and advice of a chatbot augment one’s decisions? |
| Justice | Will the use of these machine learning algorithms contribute to equitable distribution of palliative care resources? | How can we use natural language processing algorithms to reduce health disparities within palliative care? | Will AI chatbots ensure equal distribution of information for all? |
| Explicability | Is the decision-making process of the algorithm transparent, and who is responsible for algorithm errors and biases? | Is the dataset representative of the population of interest? | Where does the data for the AI chatbot come from? How does AI handle errors and bias? |
References
Oh, O., Demiris, G., & Ulrich, C. M. (2025). The ethical dimensions of utilizing Artificial Intelligence in palliative care. Nursing Ethics, 32(4), 1285-1296. https://doi.org/10.1177/09697330241296874
Parikh, R. B., Manz, C. R., Nelson, M. N., Evans, C. N., Regli, S. H., O’Connor, N., Schuchter, L. M., Shulman, L. N., Patel, M. S., Paladino, J., & Shea, J. A. (2022). Clinician perspectives on machine learning prognostic algorithms in the routine care of patients with cancer: a qualitative study. Supportive care in cancer: official journal of the Multinational Association of Supportive Care in Cancer, 30(5), 4363–4372. https://doi.org/10.1007/s00520-021-06774-w
Storick, V., O’Herlihy, A., Abdelhafeez, S., Ahmed, R., & May, P. (2019). Improving palliative and end-of-life care with machine learning and routine data: a rapid review.
HRB open research, 2, 13. https://doi.org/10.12688/hrbopenres.12923.2
Windisch, P., Hertler, C., Blum, D., Zwahlen, D., & Förster, R. (2020). Leveraging Advances in Artificial Intelligence to Improve the Quality and Timing of Palliative Care. Cancers, 12(5), 1149. https://doi.org/10.3390/cancers12051149
Note. AI: artificial intelligence.
Podcast: www.every1dies.org
Children’s Book: www.everyonediesthebook.com
(Also available at Amazon and Barnes & Noble)
