1.1 The Impact of AI on Clinical Trials

AI is at the forefront of revolutionizing clinical trials through its use in enhancing various aspects of the research process. Its ability to process large datasets, identify patterns, and optimize trial designs positions it as a powerful tool for researchers. So what are the benefits and challenges of deploying AI solutions and what could a governance model for this look like?

Key benefits of integrating AI

• Enhanced data analysis: AI can analyze massive volumes of data quickly, identifying correlations and patterns that inform better decision-making. This capacity allows researchers to derive insights from complex datasets and to improve the quality and reliability of trial outcomes. For clinical trials, this also allows greater flexibility in consuming vendor-agnostic data from both structured (e.g., Electronic Data Capture (EDC); Electronic Health Records (EHRs); and Clinical Trial Management Systems (CTMS)) and unstructured (e.g., Electronic Clinical Outcome Assessments (eCOA); Application Programming Interfaces (APIs), or RWD) sources for empowered aggregate insights.

• Streamlined patient recruitment: By utilizing AI algorithms that analyze EHRs, claims data and demographic data, researchers can effectively identify suitable candidates for clinical studies. This targeted approach increases the likelihood of both meeting inclusion criteria and improving enrolment rates for clinical trials.

• Optimized trial design: AI can assist in crafting protocols that reflect real-world patient experiences. Through simulations and predictive analytics, researchers can refine trial parameters, ultimately leading to more adaptable and meaningful clinical studies that are tailored to not only the regulatory guidelines for analysis but also to the specific Diversity, Equity and Inclusion (DEI) considerations for the patient population.

1.2 Ethical Considerations in AI Utilization

While the integration of AI in clinical trials offers numerous benefits, certain ethical considerations must be addressed to preserve the integrity of the research process. Figure 1 shows key considerations to be included when assessing the usability of AI technologies. The categories provide an overview of the strengths and challenges that will surface as points of interest in an evaluation of the appropriateness of implementation of an AI technology.

Figure 1

Balancing automation and human oversight in the selection of AI technologies.

Ethical implications and challenges of integrating AI

• Bias and fairness: AI systems are susceptible to the biases present in the data they are trained on. Historical data often lacks diversity and can be skewed. AI algorithms may reinforce these disparities, leading to biased outcomes in patient selection and treatment efficacy. Ensuring diversity in datasets is crucial for fairness and equity in clinical research. To achieve this, as the AI Agent, you must understand and actively adjust or account for the known biases inherent in your training datasets.

• Transparency in AI decision-making: Transparency remains a critical challenge in AI implementation, as many algorithms function as “black boxes” with opaque decision-making processes. This opacity becomes particularly problematic in healthcare settings where patient safety depends on understanding how AI systems reach their conclusions. To address these concerns, researchers must develop clearer explanations of AI model operations and decision pathways, while also establishing robust accountability frameworks that preserve stakeholder confidence. Regulatory bodies are beginning to address these challenges through structured validation approaches. The US Food and Drug Administration (FDA)’s draft Credibility Assessment Framework, released in January 2025, represents a significant step toward systematic AI oversight.¹ However, effective validation requires careful attention to reproducibility issues that are unique to AI systems, including the potential for hallucinations, the presence of embedded biases, and the challenges posed by continuously evolving datasets. A fundamental question emerges from this validation process: are our data collection methods inadvertently amplifying existing biases, or are they successfully reducing them? This consideration must remain central to any comprehensive AI validation strategy, as the quality and representativeness of training data directly influence both the reliability and fairness of AI-driven healthcare decisions.

1.3 Navigating Regulatory Compliance

As AI technology evolves, regulatory frameworks must adapt accordingly to maintain compliance while fostering innovation in clinical trials.

Strategies for ensuring regulatory compliance

• Engagement with regulatory authorities: It is imperative to facilitate ongoing dialogue with regulatory bodies to shape policies that support responsible AI deployment while safeguarding public health. Collaborative efforts can lead to the formulation of guidelines that promote ethical AI use in clinical settings. The European Union’s AI Act² is the world’s first comprehensive legal framework on AI and provides a conduit for these discussions as our industry works towards implementation of its prohibitions, governance rules and obligations for high-risk AI systems through 2027.³

• Real-time monitoring and reporting: Organizations should implement systems for the continuous monitoring of AI-driven processes. This human-in-the-loop methodology provides timely performance metrics and outcomes for regulators. Organizations can ensure demonstrated adherence to compliance requirements, facilitating both transparency and ongoing risk-based management of emerging trends or considerations.

• Training and development of stakeholders: As AI continues to advance, ongoing education for researchers and stakeholders will be necessary. This training should encompass ethical considerations, compliance standards, and best practices for AI implementation in clinical trials.

1.4 The AI Agentic Movement – Agents Controlling Agents

A new concept emerging in the context of AI and regulatory compliance is the AI Agentic movement. This movement suggests that regulatory bodies may also adopt an “agentic” role in the future, in which they oversee the quality and performance of AI systems used by businesses.

Agents controlling agents

• Regulatory oversight of AI: Under this framework, agents from regulatory authorities would ensure that AI systems deployed in clinical trials adhere to the highest standards of quality and compliance. This oversight could involve monitoring AI algorithms to prevent bias, ensure transparency, and uphold ethical standards.

• Emergence of “Agentic-like” roles: As AI becomes more sophisticated, roles within research organizations may evolve to become more agentic. This means that both regulatory agents and business agents will assume responsibilities that transcend traditional boundaries, focusing on collaboration and cooperation to foster ethical AI deployment in clinical trials, but always with a human in the loop.

1.5 Moving from data to decision – a deployment model

Frameworks are being developed by the clinical trials industry to manage AI lifecycles and associated governance structures. A three-tiered model is proposed here to enable the effective planning, deployment, and oversight of AI technology within regulatory requirements.

Foundation Layer

Technology Infrastructure: This foundational tier requires deep technical expertise in software and hardware solutions capable of supporting a unified data architecture. The infrastructure must seamlessly integrate both structured and unstructured data from diverse vendor platforms, enabling comprehensive data aggregation while breaking down cross-functional silos and data barriers. This technical foundation is essential for generating meaningful, higher-order insights from complex clinical datasets.

Strategic Layer

Implementation and Risk Assessment: The middle tier focuses on intelligent deployment through systematic evaluations of ethical considerations, training dataset biases, and potential quality issues inherent in both structured and unstructured data sources. This layer employs risk-based critical analysis to identify potential pitfalls and presents findings across organizational functions, enabling informed decision-making throughout the implementation process.

Oversight Layer

Governance and Compliance: Drawing insights from both the foundational and strategic layers, the governance tier provides comprehensive oversight of AI deployment, clinical insight generation, and regulatory compliance through a risk-centered approach. This layer establishes and maintains policies that reinforce essential guardrails across DEI initiatives, regulatory compliance, data quality standards, information integrity, and patient safety protocols.

This hierarchical model ensures that AI implementation in clinical trials maintains both technological robustness and regulatory alignment while prioritizing patient welfare and data integrity (see Figure 2).

Figure 2

Moving from Data to Decision – a three-tiered approach to AI implementation.

2.1 The Shift Towards Decentralized Clinical Trials

The traditional model of clinical trials, characterized by centralized data collection and consistent in-person visits, is increasingly becoming obsolete. The COVID-19 pandemic has accelerated this transition to decentralized and hybrid models that provide more flexibility and accessibility for patients.

Key characteristics of decentralized trials

• Patient empowerment: Decentralized trials place patients at the center of the research process, allowing them to participate remotely. This approach not only mitigates the burden of travel but also enhances engagement. Patients can contribute to clinical research without the constraints of traditional trial designs.

• Leveraging technology: The integration of telehealth services, remote monitoring devices, and mobile applications is critical in facilitating decentralized trials. These technological advancements enable real-time data collection and enhance communication between researchers and participants, improving the overall efficiency of the trial process.

2.2 Harnessing RWD

RWD presents a valuable opportunity to enhance clinical trials. By augmenting traditional clinical trial data with RWD, researchers can gain insights into patient experiences and treatment outcomes that reflect real-world conditions.

Benefits of utilizing RWD in clinical trials

• Deepening patient insights: RWD can provide researchers with a more comprehensive understanding of patient behavior, treatment responses, and health outcomes outside of controlled clinical environments. These insights can inform trial designs that better align with actual clinical practices.

• Informing treatment protocols: Analyzing RWD enables researchers to tailor protocols based on diverse patient populations. Such adaptability enhances patient retention and satisfaction throughout the study, making trials more relevant and responsive to individual patient needs.

2.3 Challenges in Implementing Decentralized Trials and RWD

The transition to decentralized trials and the integration of RWD present certain challenges that need to be addressed to maximize their potential.

Key challenges include:

• Data privacy and security: Protecting data privacy is paramount. Organizations must implement robust data security measures and comply with privacy regulations, such as the General Data Protection Regulation (GDPR), to build trust among participants and stakeholders.

• Standardization of data: Integrating data from various sources, including RWD, necessitates the establishment of standardized protocols. Consistency and reliability are crucial for ensuring that data is captured, stored, and analyzed across platforms effectively.

• Navigating regulatory landscapes: The regulatory environment surrounding decentralized trials and RWD is constantly evolving. Organizations must remain adaptable and proactive in ensuring compliance with local and international regulatory standards.