Table 2 Core arguments for and against the default requirement of explainability

The use of xAI or ad-hoc explainable models is not a default requirement for AI-DSS to be ethically permissible in healthcare

Medical decisions are commonly atheoretic as well (double standard argument)

[1, 23,24,25]

Post-Hoc explainability methods add new levels of uncertainty and may cause false confidence

[23, 26,27,28,29,30]

There can be a trade-off between accuracy and explainability

[1, 24, 31]

Explainability is not required to resolve problems of responsibility

[30, 32, 33]

Explainability is not required and not sufficient for the detection of biases

[23, 34, 35]

Trust, acceptance, and uptake are feasible by transparency

[30, 34,35,36,37, 39]

Shared decision-making, informed consent, and patient autonomy are feasible with transparency

[24, 27,28,29, 35, 36, 38]

The duty of HCPs to explain risks and benefits of the medical procedures is satisfiable by transparency

[36, 38, 39]

The associated risks of AI-DSS determine the requirement of explainability standards

[40,41,42,43,44,45]

The capacities and values of the patients and HCPs determine the requirement of explainability standards

[40, 42]

Potential benefits and lack of alternatives may outweigh the concerns associated with less explainable decisions

[2, 39, 46, 47]

The use of xAI or ad-hoc explainable models is a default requirement for AI-DSS to be ethically permissible in healthcare

The double-standard argument is an inapt comparison

[48, 52, 61]

Explainability reduces the risk of false hope and inappropriate interventions

[50]

The accuracy/explainability trade-off is only claimed but not substantiated

[52, 58]

Explainability is a requirement for accountability or the attribution of responsibility

[45, 48, 51, 56, 59, 60]

Explainability can help to find biases

[50, 52, 53, 57]

A lack of explainability threatens trust, acceptance, and uptake

[34, 53, 55,56,57,58, 63]

Explainability is a requirement for shared decision-making, informed consent, and patient autonomy

[10, 50, 51, 54,55,56, 60, 62]

Explainability increases HCP autonomy

[54]

Explainability is required to account for patient-values

[50, 51, 55, 63]