In the regulatory assessment process for market authorisation applications (MAAs) within the European Medicines Agency (EMA), there are currently no requirements for including individual patient data (IPD) in the submission (see current guidance on eCTD submissions in EU),1 and the review does not make use of these but relies on the aggregated results from the trial reports of the clinical and non-clinical trials and the integrated summaries. Although IPD usually are listed as part of the trial reports and thus available as single values, they are usually not further utilised in any analysis by regulators.
Presented here are experiences made by the Data Analytics Centre (DAC) at the Danish Medicines Agency (DKMA) from a series of pilots utilising IPD from clinical trials included in the dossiers for MAAs. The data used were from two phase III trials provided by two different applicants used in MAAs, an application within a rare disease with a high unmet medical need and a series of bioequivalence trials from other applications. A proposed process to support inclusion of IPD in DKMA reviews was created. Key steps in the proposed process are presented.
To identify possible challenges in the use of IPD in the review of MAAs, DKMA liaised with two different applicants on pilots providing trial data to DAC that conformed to the Clinical Data Interchange Standards Consortium (CDISC) Submission Data Tabulation Model (SDTM) standards for two phase III clinical trials. The primary objective was to evaluate the ability of DKMA to receive, process and analyse the data to recreate the results from the statistical analyses provided in the clinical trial report. A secondary objective was to identify areas in which DKMA had shortcomings of a legal or technical kind or lacked the relevant skills in the current organisation to use IPD submitted in the CDISC SDTM. The experiences were later utilised in analyses with a third applicant toward improving the benefit-risk assessment for an MMA for a drug targeting a rare disease and to investigate data anomalies in IPD received for other clinical trials.
Data from the two clinical phase III trials were provided to DKMA’s private cloud using a secure FTP server and processed according to the steps shown in Figure 1. Although not included in the initial version of the process, Pinnacle 21 Community 3.1.2 was later used to ascertain compliance with CDISC standards for the SDTMs received. Data were analysed using SAS JMP Clinical 7.1 and SAS 9.4.
Analysis reproducibility (Case 1)
Data from a large clinical trial for an oral treatment for type 2 diabetes were provided to DKMA in the SDTM and Analysis Data Model (ADaM), together with the SAS programs used for creating the results that were reported for the trial. The trial was a phase III, randomised, double-blinded, double-dummy trial with a built-in dose-escalation period for three different doses of a drug, against an active control. The trial was designed to assess efficacy and safety over more than one year of treatment. However, the primary endpoint, HbA1c, a lab-value indicative of disease severity in diabetics, was collected after six months.
The SAS programs were not directly executable within the analysis environment available to DKMA due to missing dependencies to macro code and other issues. However, the specification of the population to analyse and the statistical model used was obtained with ease from the provided SAS programs, such that consultation with the protocol and statistical analysis plan and their amendments was not necessary.
The results of the analyses of the primary endpoint performed by DKMA, together with results shown in the trial report, are seen to be identical in all but the number of digits used (Table 1). These findings are representative of all of the analyses performed by DKMA for this MAA.
|Treatment difference compared to Control Treatment||DKMA estimate (95% CI)||DKMA p-value||Reported estimate (95% CI)||Reported p-value|
|IMP dose 1||0.189 (0.063, 0.316)||0.0856||0.2 (0.1, 0.3)||0.0856|
|IMP dose 2||–0.189 (–0.376, –0.121)||<.0001||–0.2 (–0.4, –0.1)||<.0001|
|IMP dose 3||–0.499 (–0.626, –0.373)||<.0001||–0.5 (–0.6, –0.4)||<.0001|
Analysis reproducibility (Case 2)
Data from a middle-sized cancer clinical trial was provided to DKMA in the SDTM, ADaM datasets and SAS programs used to create the results reported for the trial. The trial was a phase III randomised, open-label trial comparing a novel combination treatment with an active control until no clinical benefit or unacceptable toxicity was observed. Overall survival and progression-free survival were the primary endpoints. The analyses conducted by DKMA were performed by directly applying the SAS programs provided by the applicant. Results identical to those reported by the applicant, with the exception that headers, footers and other informative text and visuals found in the study report, were not generated by the provided SAS programs.
The plots included in Figure 2 indicate that, with minor efforts, the results can be shown with increased granularity compared to that included in the report, which could have been of relevance during the assessment.
Assistance during submission (Case 3)
With experiences from the above two pilots, DKMA engaged in a constructive dialogue with a third applicant for a specific MAA for a rare disease with an urgent medical need. This included additional analyses on the data from the clinical trials to provide more robust evidence on the efficaciousness of a drug, which was added to the application assessment.
Investigation of data anomalies (Case 4)
DKMA has also engaged in exploring the possibilities of analysing trials for signs of possible fraud. Specifically, a series of bioequivalence trials could be investigated through an R based shiny tool,2 courtesy of the US Food & Drug Administration (FDA). These trials were conducted to evaluate equivalence of two versions of a drug in terms of the exposure measured by two pharmacokinetic parameters, peak concentration (Cmax) and area under the curve of the exposure profile (AUC). Part of the analysis included investigating the similarity of the exposure profiles (i.e., drug concentration over time) following administration of the study drug and assessing the likelihood of having near-identical profiles.
Assessing the likelihood of nearly identical profiles was accomplished by calculating the linear correlations between all possible pairs of profiles (i.e., each exposure profile was compared to all the other exposure profiles collected). An example pair of profiles, for subject 1 treated with drug A versus subject 2 treated with drug B, is shown in Figure 3. The exposure profile for subject 1 treated with drug A exhibits an unlikely high correlation, 0.9964, with the exposure profile of subject 2 treated with drug B. The time matched paired observations, shown to the right, indicate that the two profiles are not simple copies of one another. Thus, this finding could be coincidental. Such resemblance between two exposure profiles, even had it been for two similar treatments within the same subject, is unlikely. Such similarities must be assessed within the context of the rest of the trial data and considering variation in pharmacokinetic parameters and sample size.
Learnings from the pilots
While there were some discrepancies between elements produced by DKMA compared to those contained within the two clinical trial reports from applicants 1 and 2, the differences did not involve the statistical results; all the essential aspects were reproduced. This means that the results, such as test probabilities, estimates and confidence limits, were identical with the exception of the choice of significant digits, major differences in layout, and missing headers, footnotes and versioning information. Besides the data, DKMA was also provided with the SAS programs used for creating ADaM datasets and the statistical output. The purpose of this was to explore the extent to which DKMA could utilise the applicant´s source code. For one of the two trials, these programs were directly usable in DKMA’s analysis environment; however, for the other trial, there were challenges on execution of the programs due to missing values of macro variables and unknown references to SAS libraries and file locations. The recreation of the results from the report had to be done by following the instructions in the provided SAS programs or could have been obtained from the protocol and the statistical analysis plan. This demonstrates that DKMA was able to receive and use the data and SAS codes provided. While both applicants provided CDISC-compliant data submissions, there were differences in the packages that required some agility by DKMA in using the data. These differences are likely indicative of submissions that would be received for other applications.
A better understanding of the benefit-risk assessment and the data used for the treatment of the rare disease was part of the regulatory assessment and adds further insight in the evaluation of the evidence of drug effects. While this kind of activity most likely will seldom be for regulatory engagement in similar activities in the EMA, the ability to do so is seen as valuable.
Analysing bioequivalence trials, DKMA has gained insight on what to look for and when to react concerning potential data anomalies. This might be seen as an addition to the aspects included in future applications and a means to exclude applications with insufficient quality in the data collected and analysed.
Benefits of using IPD in the assessment
Potential benefits of including IPD in the assessments would be to increase the overall quality of the regulatory assessment by providing increased granularity of the results over that presented in the trial reports and integrated summaries. Opportunities to perform supportive analyses or to pursue simple questions in the assessments, not sufficiently answered in the provided reports and summaries (e.g., subgroup descriptive statistics), could readily be obtained by the inquisitive assessor, and thus might lead to a reduction in questions to the applicant and the associated clock stop(s) in the EMA MAA procedure. Thus, shortening this period. For this, visualisation tools supported by the embedded statistical analyses could be versatile tools to support the assessment. The apparent verification of the quality of the provided results from the analyses is yet another benefit. Another possible benefit identified through the pilots was the opportunity to engage with an applicant, increasing the level of evidence for the efficacy of the drug and providing a better basis for assessing the benefit-risk rate. Lastly, receipt of IPD and analysis datasets facilitates probing the plausibility of the data on which the reported results are based.
Although it is possible to request IPD for the assessment of MAAs, it is currently not a standard in the EMA process; as such, many aspects are yet not defined. Currently, if IPD become needed for regulatory assessment, they would be requested during the assessment phase, most likely in the clock stop, and specifications such as extent, data definition, format, means for delivery and receipt would need to be agreed. Doing so would extend the clock stop. A series of activities investigating various aspects of submitting IPD to the EMA are expected to be initiated by the EMA within the near future.3 Figure 4 shows the likely transition from the current state towards a future state in which the inclusion of IPD in applications is standard.
A procedure that resembles how IPD is provided to other regulatory bodies is most likely ideal, as these are tested, likely to function well and will require the least extra effort from the applicants.
The experiences from the different pilots described above exploring the use of IPD demonstrates that DKMA has (or has established along the way) the technical and legal foundations, as well as the competency, to make use of IPD from clinical trials in SDTM and ADaM formats, thus adding to the quality of the regulatory assessment of the MAAs. The systems and procedures are in place, although most likely there are improvements that could increase efficiency. Including IPD in the assessment of a market authorisation application allows for a more detailed understanding of the benefit-risk assessment and appreciation of the evidence provided, with the potential for a faster review and with increased understanding of drug effects.
The views presented in this text are those of the authors and not necessarily representative for DKMA or other organisations.
- The EU Harmonised technical eCTD guidance version 5.0, 1 February 2022, available at https://esubmission.ema.europa.eu/. [^]
- Cf. https://shiny.rstudio.com/. [^]
- See more at https://www.ema.europa.eu/en/about-us/how-we-work/big-data. [^]
The authors have no competing interests to declare.