Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

A data science team can build a model on its own, but it cannot tell on its own whether nurses will trust the alert, whether patients accept being scored, or whether the hospital lawyer will sign off. Those answers come from other people, and you need them before the design is fixed.

Illustration of participatory research showing diverse people collaborating to co-create and shape a research project together.

Figure 1:The Turing Way Community. This illustration is created by Scriberia with The Turing Way community, used under a CC-BY 4.0 licence. DOI: 10.5281/zenodo.3332807.

Who are the stakeholders?

A stakeholder is anyone who affects or is affected by the AI system. In a hospital project, the list usually includes the groups below.

Stakeholder groupRole in AI development
Clinicians (doctors, nurses, allied health professionals)Domain knowledge, workflow knowledge; often the end users
Patients and communitiesAffected by the system; can spot harms the development team cannot see
Data managers and ITData access, infrastructure, integration with the EHR
Data protection officerLawful basis, data protection impact assessment, patient rights
Legal and complianceRegulatory requirements, liability, contracts
EthicistsEthical review, value conflicts
Hospital management and boardFunding, procurement, governance, final responsibility
Medical technology or clinical physicsMedical device requirements and safe use of technology
Developers and data scientistsTechnical design and development
RegulatorsSupervision and certification

When to engage

Plan engagement for every phase. In design, stakeholders help define the problem, the intended use, the data strategy and the ethical questions. In development, they react to model outputs and to prototypes of the interface. In evaluation, they take part in usability tests, judge clinical usefulness and review explanations. In deployment, they receive training, report problems and give feedback on how the tool performs in daily work.

If you first involve clinicians or patients at the usability-testing stage, the problem, the data and the output format have already been chosen without them, and changing those choices is costly.

Patient and community engagement

Patients are directly affected by healthcare AI, yet they are rarely involved in building it. They can point to data that developers would not think to collect, to care situations that limit how a tool can be used, to kinds of harm or unfairness that clinical evaluation misses, and to whether the promised benefit is one they care about.

Methods for patient engagement

Think about the burden on participants. Patients with serious illness may have little time and energy, so make participation accessible and flexible, and pay people for their time where you can.

Interdisciplinary team composition

The team needs access to clinical expertise in the target domain, data science and machine learning, healthcare law and regulation, medical ethics, human factors and user experience design, and health informatics and data governance. Smaller teams will not have all of these in-house. Record which expertise you consulted from outside, and when, so that reviewers can see who shaped which decision.

The Quadruple Helix model Carayannis & Campbell (2009) brings together government, academia, industry and civil society. You can use it as a check that no single group dominates the design and that legal, ethical and social perspectives sit at the table alongside technical ones.

References
  1. Carayannis, E. G., & Campbell, D. F. (2009). “Mode 3” and “Quadruple Helix”: toward a 21st century fractal innovation ecosystem. International Journal of Technology Management, 46(3/4), 201–234. 10.1504/IJTM.2009.023374