Introduction
Since its inception in 1997, the Clinical Data Interchange Standards Consortium (CDISC) has worked to ensure that data for clinical trials are interoperable and quickly interpretable by organizations seeking to understand the safety and efficacy of clinical interventions. Adoption of CDISC standards for clinical trial data became ubiquitous when the submission of all trial data in CDISC’s Study Data Tabulation Model (SDTM) was required by the US Food and Drug Administration (FDA) in 2006 and by Japan’s Pharmaceuticals and Medical Devices Agency (PMDA) in 2020.1,2 The National Medical Products Administration (NMPA) in China also prefers data to be submitted in accordance with CDISC standards.3 The increased adoption by regulatory agencies has also resulted in growth in CDISC membership. In 2022, there were over 500 CDISC members, including most major pharmaceutical companies and clinical research organizations (CROs).4
Although CDISC standards are important in clinical research, implementing CDISC compliant data collection, formatting, and submission can be a time consuming and costly process. By nature, clinical trials data are necessarily detailed and complex in order to capture the nuances of study design, participant characteristics, intervention delivery, safety outcomes, and efficacy outcomes that might influence a regulatory agency’s decision on whether to approve a medical drug or device. Drug companies and CROs often employ teams of experts or hire consultants to assist with the creation of electronic case report forms (eCRFs) at the start of a trial and creation of submission datasets during and after the trial. Electronic data capture (EDC) software that pharmaceutical companies use for clinical trials such as Medidata Rave and Oracle Health Sciences Clinical One are well tuned for CDISC compliance and regulatory agency submissions. However, due to their high costs, the use of such software by academic investigator-initiated clinical trials is limited.5,6 Recently, CDISC published results of the CDISC Real World Data Connect project, which outlined a number of recommendations to make the CDISC standards easier to use in settings outside clinical research for regulatory submission.7 These recommendations included taking steps to support academic and public health researchers in the use of data coming from observational studies and registries.
Research Electronic Data Capture (REDCap) is an online data capture system designed for clinical and translational research.8 It is available at no cost to academic, non-profit, and government institutions. As of February 2022, the REDCap Consortium counted 5743 institutions in 145 countries under its umbrella (see Figure 1).9 The REDCap marker paper has been cited 23,659 times by researchers in many clinical research disciplines.
Map of REDCap consortium members.
One of the guiding principles for developing REDCap was attention to workflow design that allows easy implementation of best practices for academic clinical and translational researchers. What makes REDCap successful is that all institutions who use the software are part of the REDCap Consortium, which provides feedback on how to make REDCap better serve clinical researchers.10 Because standards are essential to conducting safe, high-quality research, REDCap has developed several tools to promote standards adoption based on demand from REDCap Consortium members. REDCap has the ability to import common data elements (CDE) and standard instruments from the US National Institutions of Health CDE Repository.11 REDCap has also collaborated with PhenX to host REDCap instrument files on their website.12 Most importantly for CDISC standards adoption, REDCap hosts a shared library of instruments that consortium members can import directly into their projects.13 Previous partnerships with Patient-Reported Outcomes Measurement Information System (PROMIS) that have their surveys posted on the REDCap Shared Data Instrument Library (SDIL) have resulted in speedy implementation of standard forms and high uptake from users.14,15
While new drug applications to regulatory agencies are typically filed by large companies with the resources to comply with CDISC standards, academic institutions also perform clinical trials that contribute information about the safety or efficacy of drugs. These may take the form of drug repurposing trials or studies where the frequency, sequence, or duration of interventions may vary and not require electronic regulatory submission and review.16 Nevertheless, because these trials share interventions with new drug trials, their datasets should interoperate.17 Despite CDISC offering a low-cost membership to academic institutions, academic medical centers make up just 7% of CDISC membership.4 One reason for their low participation in CDISC is the high barrier of harmonizing data to a standard that is not required for the types of studies normally done by these researchers. In 2021, CDISC and the REDCap team at Vanderbilt University Medical Center formed a partnership with a goal to close the gap of implementing CDISC standards at academic institutions.18 As the first step toward simplifying this implementation process, we sought to make all Clinical Data Acquisition Standards Harmonization (CDASH) Foundational eCRFs available on CDISC’s eCRF portal available in the REDCap SDIL.19 CDISC also has a host of disease-specific implementation guides and eCRFs.20 As a test case for disease-specific eCRFs, and with the support of The Leona M. and Harry B. Helmsley Charitable Trust and the CDISC community, we sought to make Crohn’s Disease-specific eCRFs available on the SDIL.21
Methods
The instrument metadata describing both the CDASH Foundational eCRFs and the Crohn’s Disease eCRFs are in a CDISC format called the Operational Data Model (ODM-XML). ODM-XML is a vendor-neutral machine-readable format for describing case report forms.22 Meanwhile, REDCap case report form metadata is created and shared in a comma separated file called a data dictionary. It is also possible to create, export, and upload individual forms in a project with a .zip file. We created a reusable script that parsed the ODM-XML and created a REDCap instrument .zip file. This script was based on existing REDCap code that creates a new project from an ODM-XML file. We modified this code to convert individual eCRF ODM-XML files to REDCap .zip files with some additional customizations. Some of these customizations included the ability to prefill fields with preset text such as units, links to terminologies for response options, and branching logic. The program can also take an ODM-XML file and create an eCRF in a REDCap project directly from within the eCRF designer user interface as shown in Figure 2.
Screenshot of eCRF creation with ODM-XML file from within REDCap’s online designer.
This script was written in PHP and implemented as a REDCap external module. REDCap external modules are add-ons programs that can supplement REDCap functionality. System administrators can download external modules from the REDCap External Modules Repository.23 We also created explanatory text, licensing information, and disclosures to include with the instrument when it was added to the SDIL.
We followed a two-week cycle iterative design process to develop the eCRF converter and descriptive text. CDISC experts spent a week reviewing the eCRFs that were produced from the eCRF converter. CDISC experts and the REDCap team then met to report deficiencies in how the eCRFs rendered in REDCap or inconsistencies with other EDC systems. The REDCap team would then spend the following week incorporating those changes into the external module. The external module would then be available to the CDISC experts again and the process repeated. All identified issues and their resolutions were tracked in a shared spreadsheet that CDISC and REDCap team members updated. This cycle started in November 2021 and continued through March 2022.
Implementation
The process of developing the external module required 190 hours of REDCap developer time and 352 hours of CDISC expert time to complete. In June 2022, we published the CDISC ODM-XML to REDCap instrument zip external module to the REDCap External Module Repository.23 We also posted all 34 CDASH Foundational and 20 CDASH Crohn’s Disease eCRFs to the SDIL. Version 1.1 of four CDASH instruments (demographics, common identifiers, adverse events, and protocol deviations) were already available on the SDIL prior to June 2022 and had been downloaded by REDCap Consortium members 4024 times. At the time of publication there were insufficient data to evaluate the uptake of the newly posted eCRFs in the SDIL. However, the SDIL framework has the ability to track the number of downloads and institutions using the eCRFs.13 Those statistics will be shared at a later time.
Discussion
With CDASH instruments easily accessible from the SDIL, academic institutions in the REDCap consortium can benefit from faster eCRF creation and shorter overall study startup time, particularly for Crohn’s Disease studies. Applying standards at eCRF creation as opposed to after data collection has also been shown to improve data completeness by decreasing the number of missing values in SDTM datasets.24 There is also the benefit that data from clinical trials can be more easily combined for secondary uses. The external module for conversion of ODM-XML to REDCap instrument zip files is available for reuse in future versions of these eCRFs and new CDASH eCRFs for other therapeutic areas and disease areas where CDISC has implementation guides.20
Limitations
Much of the CRF customizations specified in the ODM-XML were easily translated into the REDCap equivalent of that functionality. Prefilled fields, field validation (such as numeric values of dates), branching logic, and controlled terminologies all had REDCap equivalents. Some features in the ODM-XML, however, proved harder to replicate. For example, some forms had nested repeating subsections (e.g., enter information for all surgeries during the study) within non-repeating forms. While REDCap does support repeating instruments, these are used in cases where the entire form, not parts of the form, repeat. We overcame this limitation by hard coding in several fields, hidden by branching logic, into forms where repeating subsections were needed. We also included instructions to the user for what to do if they needed more repeating instances.
Another challenge we faced was that many form names, variable names, and response options contained special characters, which were not supported in REDCap. While we did consider simply removing these special characters, having the CDASH standard name unavailable would pose a problem for future SDTM mapping. To overcome this limitation, we put full variable names in the field annotation text area of REDCap. Other metadata will accompany variable name and response option codes to inform SDTM mapping.
Future Directions
The CDASH foundational and Crohn’s Disease eCRFs are just the beginning of an ongoing collaboration between CDISC and REDCap. In developing the eCRFs, we recognized that simply making forms available for reuse may not be sufficient for academic researchers to understand and implement CDISC standards. We intend for the eCRF documentation to become part of online guidance for academic researchers hosted on the CDISC website and linked from the REDCap SDIL CDASH instrument descriptions. The CRF descriptions will provide more details about how to implement the eCRFs and other design modifications and considerations that researchers could take into account when using the eCRFs. Building on the work to distribute Crohn’s Disease eCRFs, we plan to expand to other therapeutic areas. This includes existing CDISC Therapeutic Area User Guides (TAUGs) and future CDISC standards development projects.
The CDASH eCRF we developed can already facilitate data aggregation and sharing within a multi-site study or between similar studies due to standardized variable names, units, response options, formatting, and terminologies. The next major technical project in the CDISC and REDCap collaboration is to create REDCap functionality to export study data in the SDTM format to facilitate regulatory group submission. We will modify code and metadata formatting developed by Yamamoto et al. to enable SDTM file exports from within REDCap as an external module.25 Like the ODM-XML converter, this external module will be available to the REDCap consortium through the External Modules Repository. We will then populate the mapping metadata in field annotation for all the CDASH eCRFs we create so that anyone who downloads these instruments from the SDIL will be ready, with some study customization, to export data to SDTM. Again, instructions for how to make study customizations will be part of the guidance for academics on the CDISC website that will link from the SDIL.
Conclusion
This collaboration between CDISC and REDCap has resulted in CDASH eCRF tools that are readily accessible to academic researchers in the REDCap Consortium at no cost. This work and ongoing development is a result of mutual demand between CDISC and REDCap Consortium members and the recognition that building a bridge between data collected in an academic setting and clinical trial-generated data will benefit all stakeholders. And while CDISC is focused on standards development and consensus building and REDCap on developing a technology platform, both consortia rely on diverse and international coalitions of members for continuous product improvement with the purpose of improving the quality and efficiency of clinical research data collection and analysis. The availability of these new CDASH eCRF tools provides a new opportunity for academic medical researchers to contribute their drug safety and efficacy data to the body of work that agencies rely on for making regulatory decisions while facilitating data aggregation and sharing.
Acknowledgements
The authors would like to thank the National Center for Advancing Translational Sciences (5 UL1 TR002243-05) and The Leona M. and Harry B. Helmsley Charitable Trust for funding this work.
Competing Interests
The authors have no competing interests to declare.
References
1. Unified Agenda of Federal Regulatory and Deregulatory Actions – Department of Health and Human Services Semiannual Regulatory Agenda. Accessed February 17, 2022. https://www.govinfo.gov/content/pkg/GPO-UA-2003-05-27/html/GPO-UA-2003-05-27-7.htm
2. Nakajima Y, Kitahara T, Hara R. Awareness from Electronic Data Submission to PMDA and FDA: Lesson & Learnt from hands-on experiences. PharmaSUG 2017. https://www.pharmasug.org/proceedings/2017/SS/PharmaSUG-2017-SS06.pdf
3. National Drug Administration Drug Evaluation Center. Clinical trial database and related data in eCTD Declaration requirements. September 2019.
4. Membership. Accessed February 17, 2022. https://www.cdisc.org/membership
5. Rave Electronic Data Capture (EDC) System. Medidata Solutions. September 17, 2021. Accessed February 26, 2022. https://www.medidata.com/en/clinical-trial-products/clinical-data-management/edc-systems.
6. Health Sciences Clinical One Data Collection. Accessed March 11, 2022. https://www.oracle.com/industries/life-sciences/data-collection/
7. Facile R, Muhlbradt EE, Gong M, et al. Use of Clinical Data Interchange Standards Consortium (CDISC) Standards for Real-world Data: Expert Perspectives From a Qualitative Delphi Survey. JMIR Med Inform. 2022; 10(1): e30363. DOI: http://doi.org/10.2196/30363
8. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap): A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009; 42(2): 377–381. DOI: http://doi.org/10.1016/j.jbi.2008.08.010
9. REDCap. Accessed February 19, 2022. https://projectredcap.org/
10. Harris PA, Taylor R, Minor BL, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019; 95: 103208. DOI: http://doi.org/10.1016/j.jbi.2019.103208
11. Harris PA, Taylor R, Jagtap V, Conway D, Duda SN, Cheng AC. Promoting Use of Common Data Elements in Research Studies. 2021 AMIA Annual Symposium Conference.
12. Hendershot T, Pan H, Haines J, et al. Using the PhenX Toolkit to Add Standard Measures to a Study. Curr Protoc Hum Genet. 2015; 86: 1.21.1–1.21.17. DOI: http://doi.org/10.1002/0471142905.hg0121s86
13. Obeid JS, McGraw CA, Minor BL, et al. Procurement of shared data instruments for Research Electronic Data Capture (REDCap). J Biomed Inform. 2013; 46(2): 259–265. DOI: http://doi.org/10.1016/j.jbi.2012.10.006
14. Lizzio VA, Gulledge CM, Meta F, Franovic S, Makhni EC. Using a Web-Based Data Collection Platform to Implement an Effective Electronic Patient-Reported Outcome Registry. Arthrosc Tech. 2019; 8(6): e535–e539. DOI: http://doi.org/10.1016/j.eats.2019.01.012
15. Levin SN, Riley CS, Dhand A, et al. Association of social network structure and physical function in patients with multiple sclerosis. Neurology. 2020; 95(11): e1565–e1574. DOI: http://doi.org/10.1212/WNL.0000000000010460
16. Center for Drug Evaluation, Research. Electronic Regulatory Submission and Review. Accessed February 25, 2022. https://www.fda.gov/drugs/forms-submission-requirements/electronic-regulatory-submission-and-review
17. Natanegara F, Zariffa N, Buenconsejo J, et al. Statistical Opportunities to Accelerate Development for COVID-19 Therapeutics. Stat Biopharm Res. 2022; 14(1): 5–21. DOI: http://doi.org/10.1080/19466315.2020.1865195
18. CDISC and REDCap Work Together to Foster More Meaningful Clinical Research for Academics. Accessed March 11, 2022. https://www.cdisc.org/news/cdisc-and-redcap-work-together-foster-more-meaningful-clinical-research-academics
19. eCRF Portal. Accessed February 17, 2022. https://www.cdisc.org/kb/ecrf
20. Therapeutic Areas by Disease Area. Accessed February 18, 2022. https://www.cdisc.org/standards/therapeutic-areas/disease-area
21. CDISC to Develop Data Standards for Crohn’s Disease With Support from the Helmsley Charitable Trust. Accessed March 11, 2022. https://www.cdisc.org/news/cdisc-develop-data-standards-crohns-disease-support-helmsley-charitable-trust
22. ODM-XML. Accessed February 19, 2022. https://www.cdisc.org/standards/data-exchange/odm
23. REDCap External Module Repository. Accessed February 18, 2022. https://redcap.vanderbilt.edu/consortium/modules/index.php
24. Lin CH, Chou HI, Yang UC. A standard-driven approach for electronic submission to pharmaceutical regulatory authorities. J Biomed Inform. 2018; 79: 60–70. DOI: http://doi.org/10.1016/j.jbi.2018.01.006
25. Yamamoto K, Ota K, Akiya I, Shintani A. A pragmatic method for transforming clinical research data from the research electronic data capture “REDCap” to Clinical Data Interchange Standards Consortium (CDISC) Study Data Tabulation Model (SDTM): Development and evaluation of REDCap2SDTM. J Biomed Inform. 2017; 70: 65–76. DOI: http://doi.org/10.1016/j.jbi.2017.05.003