1. AI & Clinical Trials: Revolutionizing Research with Ethics at the Core
1.1 The Impact of AI on Clinical Trials
AI is at the forefront of revolutionizing clinical trials through its use in enhancing various aspects of the research process. Its ability to process large datasets, identify patterns, and optimize trial designs positions it as a powerful tool for researchers. So what are the benefits and challenges of deploying AI solutions and what could a governance model for this look like?
Key benefits of integrating AI
Enhanced data analysis: AI can analyze massive volumes of data quickly, identifying correlations and patterns that inform better decision-making. This capacity allows researchers to derive insights from complex datasets and to improve the quality and reliability of trial outcomes. For clinical trials, this also allows greater flexibility in consuming vendor-agnostic data from both structured (e.g., Electronic Data Capture (EDC); Electronic Health Records (EHRs); and Clinical Trial Management Systems (CTMS)) and unstructured (e.g., Electronic Clinical Outcome Assessments (eCOA); Application Programming Interfaces (APIs), or RWD) sources for empowered aggregate insights.
Streamlined patient recruitment: By utilizing AI algorithms that analyze EHRs, claims data and demographic data, researchers can effectively identify suitable candidates for clinical studies. This targeted approach increases the likelihood of both meeting inclusion criteria and improving enrolment rates for clinical trials.
Optimized trial design: AI can assist in crafting protocols that reflect real-world patient experiences. Through simulations and predictive analytics, researchers can refine trial parameters, ultimately leading to more adaptable and meaningful clinical studies that are tailored to not only the regulatory guidelines for analysis but also to the specific Diversity, Equity and Inclusion (DEI) considerations for the patient population.
1.2 Ethical Considerations in AI Utilization
While the integration of AI in clinical trials offers numerous benefits, certain ethical considerations must be addressed to preserve the integrity of the research process. Figure 1 shows key considerations to be included when assessing the usability of AI technologies. The categories provide an overview of the strengths and challenges that will surface as points of interest in an evaluation of the appropriateness of implementation of an AI technology.
Balancing automation and human oversight in the selection of AI technologies.
Ethical implications and challenges of integrating AI
Bias and fairness: AI systems are susceptible to the biases present in the data they are trained on. Historical data often lacks diversity and can be skewed. AI algorithms may reinforce these disparities, leading to biased outcomes in patient selection and treatment efficacy. Ensuring diversity in datasets is crucial for fairness and equity in clinical research. To achieve this, as the AI Agent, you must understand and actively adjust or account for the known biases inherent in your training datasets.
Transparency in AI decision-making: Transparency remains a critical challenge in AI implementation, as many algorithms function as “black boxes” with opaque decision-making processes. This opacity becomes particularly problematic in healthcare settings where patient safety depends on understanding how AI systems reach their conclusions. To address these concerns, researchers must develop clearer explanations of AI model operations and decision pathways, while also establishing robust accountability frameworks that preserve stakeholder confidence. Regulatory bodies are beginning to address these challenges through structured validation approaches. The US Food and Drug Administration (FDA)’s draft Credibility Assessment Framework, released in January 2025, represents a significant step toward systematic AI oversight.1 However, effective validation requires careful attention to reproducibility issues that are unique to AI systems, including the potential for hallucinations, the presence of embedded biases, and the challenges posed by continuously evolving datasets. A fundamental question emerges from this validation process: are our data collection methods inadvertently amplifying existing biases, or are they successfully reducing them? This consideration must remain central to any comprehensive AI validation strategy, as the quality and representativeness of training data directly influence both the reliability and fairness of AI-driven healthcare decisions.
1.3 Navigating Regulatory Compliance
As AI technology evolves, regulatory frameworks must adapt accordingly to maintain compliance while fostering innovation in clinical trials.
Strategies for ensuring regulatory compliance
Engagement with regulatory authorities: It is imperative to facilitate ongoing dialogue with regulatory bodies to shape policies that support responsible AI deployment while safeguarding public health. Collaborative efforts can lead to the formulation of guidelines that promote ethical AI use in clinical settings. The European Union’s AI Act2 is the world’s first comprehensive legal framework on AI and provides a conduit for these discussions as our industry works towards implementation of its prohibitions, governance rules and obligations for high-risk AI systems through 2027.3
Real-time monitoring and reporting: Organizations should implement systems for the continuous monitoring of AI-driven processes. This human-in-the-loop methodology provides timely performance metrics and outcomes for regulators. Organizations can ensure demonstrated adherence to compliance requirements, facilitating both transparency and ongoing risk-based management of emerging trends or considerations.
Training and development of stakeholders: As AI continues to advance, ongoing education for researchers and stakeholders will be necessary. This training should encompass ethical considerations, compliance standards, and best practices for AI implementation in clinical trials.
1.4 The AI Agentic Movement – Agents Controlling Agents
A new concept emerging in the context of AI and regulatory compliance is the AI Agentic movement. This movement suggests that regulatory bodies may also adopt an “agentic” role in the future, in which they oversee the quality and performance of AI systems used by businesses.
Agents controlling agents
Regulatory oversight of AI: Under this framework, agents from regulatory authorities would ensure that AI systems deployed in clinical trials adhere to the highest standards of quality and compliance. This oversight could involve monitoring AI algorithms to prevent bias, ensure transparency, and uphold ethical standards.
Emergence of “Agentic-like” roles: As AI becomes more sophisticated, roles within research organizations may evolve to become more agentic. This means that both regulatory agents and business agents will assume responsibilities that transcend traditional boundaries, focusing on collaboration and cooperation to foster ethical AI deployment in clinical trials, but always with a human in the loop.
1.5 Moving from data to decision – a deployment model
Frameworks are being developed by the clinical trials industry to manage AI lifecycles and associated governance structures. A three-tiered model is proposed here to enable the effective planning, deployment, and oversight of AI technology within regulatory requirements.
Foundation Layer
Technology Infrastructure: This foundational tier requires deep technical expertise in software and hardware solutions capable of supporting a unified data architecture. The infrastructure must seamlessly integrate both structured and unstructured data from diverse vendor platforms, enabling comprehensive data aggregation while breaking down cross-functional silos and data barriers. This technical foundation is essential for generating meaningful, higher-order insights from complex clinical datasets.
Strategic Layer
Implementation and Risk Assessment: The middle tier focuses on intelligent deployment through systematic evaluations of ethical considerations, training dataset biases, and potential quality issues inherent in both structured and unstructured data sources. This layer employs risk-based critical analysis to identify potential pitfalls and presents findings across organizational functions, enabling informed decision-making throughout the implementation process.
Oversight Layer
Governance and Compliance: Drawing insights from both the foundational and strategic layers, the governance tier provides comprehensive oversight of AI deployment, clinical insight generation, and regulatory compliance through a risk-centered approach. This layer establishes and maintains policies that reinforce essential guardrails across DEI initiatives, regulatory compliance, data quality standards, information integrity, and patient safety protocols.
This hierarchical model ensures that AI implementation in clinical trials maintains both technological robustness and regulatory alignment while prioritizing patient welfare and data integrity (see Figure 2).
Moving from Data to Decision – a three-tiered approach to AI implementation.