Electronic data capture (EDC) has become a common and proven tool for data collection and management in clinical trials. Thus, understanding the principles and methods for EDC use has become a major component of clinical data management (CDM) professional practice. This chapter focuses on using the EDC system and accruing data to support study conduct, maintaining an EDC system during a study, and concluding active data collection through database lock. The regulatory basis for minimum standards and recommended best practices are discussed.
After reading this chapter, the reader should understand
the regulatory basis for practices in EDC study conduct, maintenance, and closeout
special considerations for ongoing data surveillance and use when using web-based EDC
common processes used in study conduct of EDC-based studies
methods for managing system access and privileges
common practices for EDC system maintenance and change control during studies
special considerations for close-out of EDC-based studies
Electronic resources for clinical data management have developed over the last 40 years as a suite of processes and tools to enhance the management, quality control, quality assurance, and archiving of clinical trial research data. This development has led to a major paradigm shift in CDM, with form-based data capture available via the internet to investigator sites.
Pre-production activities and planning such as those covered in Chapter 2, “Electronic Data Capture – Implementation and Study Start-up” are necessary for a study employing EDC technology. Equally important is how the technology is used every day to support the ongoing conduct, maintenance, and ultimately closeout of a study. Well-executed conduct and closeout activities are crucial, especially for clinical research data used to support research conclusions, evidence changes in clinical practice, or presented for use in regulatory decision-making.
The integrated addendum to Good Clinical Practice (R2) recommends a Quality Management System (QMS) approach to assuring the safety of human subjects and the integrity of study data.
This chapter focuses on EDC-assisted methods for active study management such as problem detection, data reviews, trend analyses, use of work lists, and using the EDC system for common and high-volume communication. Maintaining data security and change control are also addressed as are mid-study data requests and interim analyses. EDC study closeout practices such as final source document review, managing database lock, audits, generation of archival copies, site close-out, and hardware disposal complete the treatment of EDC in clinical studies.
Many of the tasks described in this chapter may be joint responsibilities or performed by individuals outside the Clinical Data Management group. The CDM profession is defined ultimately by the tasks necessary to assure that data are capable of supporting research conclusions. As such, while responsibilities vary across organizations, where CDMs do not directly perform these tasks, the role usually has some organizational responsibility for assuring that data-related tasks have been performed with the rigor necessary to maintain data integrity and support reuse of the data during or after the study.
Detailed information describing design, planning, and pre-production activities for EDC-based studies can be found in Chapter 2, “Electronic Data Capture – Implementation and Study Start-up”. This chapter picks up after study start-up and covers aspects specific to web-based EDC of the active enrollment and data collection phase through to database lock and archival. The Data Management Planning chapter covers general aspects of data collection and management applicable to EDC and other data sources. This chapter covers issues pertaining to study conduct, maintenance, and closeout that are specific to EDC.
As a mode of data collection and management in clinical studies, EDC systems have the potential to impact human subject protection as well as the reliability of trial results. Regulation and guidance are increasingly vocal on the topic.
The
Similar to ICH E6 R2,
Recommendations in
The Medicines & Healthcare products Regulatory Agency (MHRA)
Similar to ICH E2 (R2), MHRA
The General Principles of Software Validation; Final Guidance for Industry and FDA Staff (2002) provides guidance regarding generally recognized software validation principles which can be applied to any software, inclusive of that used to support clinical trials.
The FDA guidance,
Similarly, the FDA’s guidance,
Section III.A.3 elaborates on Title 21 CFR Part 11 and states, “The eCRF should include the capability to record who entered or generated the data [i.e., the originator] and when it was entered or generated.” and “Changes to the data must not obscure the original entry, and must record who made the change, when, and why.”
Section III.A.5 states that the FDA encourages “the use of electronic prompts, flags, and data quality checks in the eCRF to minimize errors and omissions during data entry”.
Section III.C states, “The clinical investigator(s) should retain control of the records (i.e., completed and signed eCRF or certified copy of the eCRF).” In other words, eSource data cannot be in sole control of the sponsor.
As such, we state the following minimum standards for study conduct, maintenance, and closeout using EDC systems.
Best practices were identified by both the review and the writing group. Best practices do not have a strong requirement based in regulation or recommended approach based in guidance, but do have supporting evidence either from the literature or consensus of the writing group. As such best practices, like all assertions in GCDMP chapters, have a literature citation where available and are always tagged with a roman numeral indicating the strength of evidence supporting the recommendation. Levels of Evidence are outlined in Table
Minimum Standards.
1. | Establish and follow SOPs that include EDC specific aspects of study conduct, maintenance, and closeout. |
2. | Document the source for data and changes in sources of data at each site including explicit statement that the EDC system is used as the source where this is the case. |
3. | Ensure data values can be traced from the data origination through all changes and that the audit trail is immutable and readily available for review. |
4. | Establish and follow SOPs for change control (and documentation thereof) for changes to the underlying EDC system and the study-specific EDC application including the eCRF, data processing, and other dynamic system behavior. |
5. | Assure procedures for use of computerized systems at clinical sites are in place and available to the site personnel at all times, including procedures for system setup or installation, data collection and handling, system maintenance, data backup, recovery, and contingency plans, computer security, and change control. |
6. | Complete testing prior to implementation and deployment to sites. |
7. | Establish and follow SOPs to ensure that all users have documented education, experience, or training supporting their qualification for functions relevant to their role prior to using the system; assure that site users receive training on significant changes to the study-specific EDC application. |
8. | Establish and follow SOPs to limit data access and permissions to authorized individuals and to document data access and permissions. |
Best Practices.
1. | Data should be entered by the site staff most familiar with the patients and data so that error is reduced and discrepancies identified during entry can be resolved during entry. |
2. | Data flow should be immediate and continuous. |
3. | Establish and follow procedures to continually surveil and mine study data and metadata using alerts or reports to detect trends and aberrant events, to prioritize and direct correct and for preventative action. |
4. | Leverage EDC technology to provide decision support, process automation, and connectivity with other information systems used on the study where practical and useful. |
5. | The EDC system and all intended data operations such as edit checks and dynamic behavior should be in production prior to enrollment of the first patient. |
6. | Establish and maintain all-stakeholder study team communication to manage data entry, query resolution, and change control throughout the study. [VI] |
7. | The EDC system should be tightly coupled to the safety data collection and handling process with potential events triggered from the EDC system and ongoing synchronicity. |
8. | Take advantage of opportunities to solicit feedback and provide additional training at investigator meetings, study coordinator teleconferences, and monitoring visits, as well as through communications such as splash screens or other notification functionality available in the EDC. [VII] |
9. | Employ active management processes for data acquisition and processing through communication of data entry and query timelines, frequent reporting of status, and regular follow-up to resolve lagging activities so that data are available rapidly to support trial management. |
10. | Employ ongoing surveillance when new systems or new functions are implemented to assure that the EDC system continues to operate as intended. |
11. | Manage activities from data origination to “clean” and use an incremental form or casebook lock strategy to lock-as-you-go to reduce the amount of data review and locking needed upon study completion. |
Grading Criteria.
Evidence Level | Evidence Grading Criteria |
---|---|
I | Large controlled experiments, meta, or pooled analysis of controlled experiments, regulation or regulatory guidance |
II | Small controlled experiments with unclear results |
III | Reviews or synthesis of the empirical literature |
IV | Observational studies with a comparison group |
V | Observational studies including demonstration projects and case studies with no control |
VI | Consensus of the writing group including GCDMP Executive Committee and public comment process |
VII | Opinion papers |
After all of the work to design, develop, test, and implement an EDC study application, the move to production, also called go-live, can be quite a relief. However, the active enrollment and data collection phase of a study is no time to rest. In fact, when recruitment starts, data management tasks not only change, but in a large and well-managed study they often accelerate. In studies where high data volume is accompanied by high enrollment, managing and controlling the collection and processing of data can consume multiple dedicated people. Without sufficient resources, backlogs of data entry, other data processing, or Source Data Verification (SDV) develop quickly. Backlogs delay access to and use of the data not only for interim analyses but also for monitoring and managing the study, ultimately eroding the value of EDC.
Provided good operational design, the next major step is to follow through on that design by conducting the study as planned.
The vast majority of benefits of web-based EDC accrue during the active data collection phase of the study. Web-based information systems make centralization of information from multiple locations possible and in real time and, likewise, support decentralized, simultaneous access use of that same information. With web-based EDC systems, immediate oversight and coordination became possible for the first time in clinical trials.
The rapid availability of study data in an EDC system allows project teams to detect problems such as delays, discrepancies, and deviations and make decisions earlier than in paper-based studies.
Interim efficacy and safety data reviews can be performed earlier in an EDC-based study using the most current, near real-time patient information. Non-serious adverse events and other pertinent patient information can be also be reviewed earlier in the study, ensuring that a Data and Safety Monitoring Board (DSMB) has a current, complete picture of the patient safety profile. Decisions by a DSMB to stop a study because of safety concerns or lack of efficacy can be made much more quickly, ensuring better subject protection and lower costs.
Given the advantages of having data immediately after a study participant is enrolled, it is surprising that about one-third of companies responding to the eClinical Landscape survey reported “often” or “always” releasing the study-specific database after the First Patient First Visit (FPFV).
Having the processes and tools in place at study start are necessary but not sufficient for gaining benefit from EDC. The data have to be up-to-date to be of maximal value. Failure to manage and maintain a study in an up-to-date state is a common point of failure for data managers and organizations – as the time from the patient visit to entry of data increases, the advantage of EDC decreases and the risk increases (
Key Relationships Between Cycle-time Metrics and Benefits of EDC.
Keeping data current requires active management. Active management entails more than identifying important metrics, setting performance expectations and action limits, and tooling to detect when and where action limits have been reached. Examples of action limits include how late is a late query, too many answers to a question, too many missing fields. Follow-through of prompt and consistent intervention is the all-important last step. In other words, processes and tools to assure that the operational design of a study is implemented with high fidelity are important, but without systematic surveillance and prompt response when action limits are reached processes will degrade. “Ironically, there is a major difference between a process that is presumed through inaction to be error-free and one that monitors mistakes. The so-called error-free process will often fail to note mistakes when they occur.”
Active management for up-to-date data starts with setting expectations for data receipt and discrepancy resolution timeliness with sites and vendors during contracting. Paper studies commonly used cycle-time expectations in weeks; e.g., data should be submitted within one week of the visit and queries should be resolved within one week of their generation. In an early two study EDC pilot, Dimenas reported 69% and 54% of visits entered same or next day and 23% and 24% of queries resolved same or next day.
Active management of data acquisition and processing requires three things: (1) real-time or daily reporting of status and late items such as late entry or outstanding query responses in a format that facilitates follow-up as demonstrated in
Reporting to Facilitate Late Entry/Resolution of Queries.
While some standard system status reports list only counts of cumulative or by-period completed items, others facilitate the work. For example, a CRF page status report that provides the number of complete and incomplete CRFs brings visibility to the data entry status. Doing so and following-up on outstanding pages as patient enrollment progresses avoids a large number of incomplete CRF page counts. The same can be done for any data collected for a study and for managing resolution of data discrepancies as well. Many EDC systems are able to link a user directly to the outstanding or late item so that the resolution of data discrepancies can be completed using the report as a work list to address the outstanding items, working sequentially down the list. Such directly actionable reports and within-system connectivity facilitates completion of work by eliminating steps. Directionally actionable work list reports should be available for all units of work such as visits or procedures, data from such collected via eCRFs, central labs, devices, or reading centers, as well as data discrepancies. Others have emphasized the importance of patient-level, site-level, and study-level reporting to actively manage data collection and processing.
An insightful industry leader mined data from over 100 EDC studies conducted by his organization.
Three Patterns of Data Responsiveness. Adapted from Summa 2004.
a) Ideal data responsiveness behavior. The majority of data are entered, resolved, or otherwise responded to at the earliest possible time. The peak should be highest at the earliest time point (zero days from the prior workflow step), should taper off precipitously, and the tail should be small. Further improvement in this ideal scenario is pursued through root cause analysis (understanding the root causes) for data entry, resolution, or response occurring later in time. Curves can be displayed by study, site, study coordinator, data manager, monitor, or any other actionable unit of analysis to help identify opportunities for improvement.
b) Likely systematic delay in responsiveness to the availability of data. The majority of data activity of interest (data entry in the figure) does not take place till around 14 days after the patient visit. This behavior is indicative of responding to the data just before some deadline. Importantly, data displayed on the graphs should be homogeneous with respect to deadlines; where deadlines are different for different sites, regions of the world, or in-house versus contract monitors, the differences in expectations will obscure the signal.
c) Waiting till the end. The majority of the data are responded to at the end of the period of interest.
The behavior in graph (A) in blue takes full advantage of immediate availability of data possible with EDC. The behavior in graph (B) in orange has room for improvement. The behavior in graph (C) squanders the opportunity for early detection and correction of problems possible with EDC.
“With EDC the batch processing has been replaced by the continuous flow of data.”
Percentage of Patients With Data Entered At Each Day Following the Visit.
Active management strategies have demonstrated through elapsed time metrics such as time from visit to entry and time from visit to clean data
Complete data management equally includes systematic screening to detect unexpected problems.
Data submission and query response timeliness
visit date to data entered
data entered to data queried
data queried to query answered
query answered to query resolved
patient out to patient record locked
counts of automated queries to detect over-firing or under-firing rules
number of queries per patient
frequency of queries by field to detect problems with the eCRF
CRA review status of the eCRF forms
Metrics for data changes
help desk calls, tickets, or error reports
system and support response times specified in the Service Level Agreement
system downtime
Percentages of subjects meeting key inclusion and exclusion criteria
Protocol violation percentages
Elapsed time windows for visits and study procedures
Differences in distributional characteristics of data values
Percentage of subjects with use of concomitant or rescue medications
Percentage of subjects with reported adverse events
Percentages, proportions, or rates of early terminations and reasons for termination
Many metrics of interest are aggregates of different aspects of units of work, for example, average data entry latency, query response times, number of queries per form, and percent of time points for which ePRO data are missing. Additionally reported adverse event patient visit should be available and run early in a trial to identify potential problems. Remediation can then be taken to reduce or even eliminate underlying problems as the study progresses. Remediation may include revisions to CRF forms or completion guidelines, retraining site staff, retraining CDM staff regarding query wording, adding alerts or other supportive workflow, or reiterating expectations through special topic newsletters and teleconferences.
All of these allow comparison of data across sites to identify unwanted process differences at sites. In fact, one of the newer CDM competencies is, “Applies analytics to identify data and operational problems and opportunities.”
Such screening requires making risk-based and potentially statistically-based decisions about how different is different enough to look into, and the extent of investigation and attempts at resolution. Examples of resolution may include retraining the site concerning the protocol, the EDC system, eCRF, or eCRF completion guidelines; explaining problems to sites in an auto-notification message, reminder, or banner via the EDC communication functionality; or reviewing top problems and corrective action in periodic meetings and presentations to sites. Similarly, screening query frequencies may prompt the project team to re-examine edit check specifications and narrow or broaden ranges, change logic, or eliminate certain checks altogether.
The sequence and timing of initiating surveillance must be decided early. Data are usually entered into an EDC system before they have been source reviewed. Data validation and review activities are usually performed before data are source verified, if the latter is planned for the study. In a Risk-Based Monitoring (RBM) paradigm, results of surveillance can be used to trigger or prioritize monitoring calls, visits, or SDV. To the extent that data cleaning, active surveillance, and SDV are intertwined, communication and workflow between clinical research associates (CRAs) and CDMs should be established within the EDC system. For example, many EDC systems have the functionality to trigger SDV or to track SDV and indicate completion by form, visit, or subject.
The Mitchel, et al. report details their approach to implementing RBM. Briefly, the trial team identified risks to patient safety and the trial results and scored the likelihood and severity of each.
Operationalizing active surveillance requires CDM such as
encoding them in written procedures
developing tools to screen data and report the results
dedicating resources to manage the process and work the reports
development and delivery of role-based training for the study team on those processes and tools
Many EDC systems have existing reports for some of these. However, today, not all data are integrated within the EDC system. Some data are managed in other systems or organizations. Comprehensive data surveillance as described above requires timely access to all of the study data; thus, obtaining the full benefit from EDC would be best with interoperability and data integration with all data sources. Metric reports from other data sources should also be considered. Studies integrating data from automated equipment such as an electrocardiogram (ECG), personal digital assistant (PDA), Interactive Voice/Web Response System (IVRS/IWRS), or other electronic devices for electronic patient-reported outcomes (ePRO) often necessitates development of custom reports.
Application of advanced statistical techniques to filter and detect more significant anomalies often necessitate use of a statistical analysis software package such as SAS® or R®. Such ongoing surveillance has also been reported when new systems or new functions were implemented to ensure that the EDC system continues to operate as intended.
Variability in site-specific practices is inevitable. However, site-specific procedures such as insertion of additional data processing steps can interfere with the advantages of EDC. Reports of sites using paper CRF-like forms or worksheets as source or as an intermediate step on which to record data as it is abstracted from the source and from which data are subsequently entered into the EDC system have been reported in the literature.
Many EDC systems offer functionality to support communication between the central study team members and site personnel. Where possible and practical, such system functionality should be leveraged for communication about trial conduct tasks. Computers perform such deterministic and repetitive tasks with higher reliability than humans and without the delay of waiting for humans to read and respond to email. For example, flagging discrepant data in the EDC system, by applying a workflow for discrepancy management, handles the “communication” regarding discrepancy resolution through the system as a by-product of the different roles undertaking actions toward discrepancy identification and resolution and obviates the need for emails to request responses or notify that resolutions have been completed. Likewise for other workflows such as safety event notification and tracking and prompting collection of source documents for Clinical Event Classification (CEC) review.
Provision of trial information and job aids to clinical investigational sites is also an important type of communication and can be managed through portal functionality in many EDC systems. Similarly, many EDC systems are able to provide status and other reports within the system. Doing so at a regular frequency or real-time obviates email distribution, drives site personnel and central team members to the EDC system where addressing outstanding issues may be only a few clicks away, and promotes the EDC system as the source of up-to-date information.
When it is not possible or practical to leverage the EDC system for needed communication, traditional communication vehicles such as calls, meetings, and email are needed. The purpose, frequency, distribution, and content of each should be considered and documented.
Privacy regulation and guidance, such as the HIPAA privacy rule, ICH Guidelines E6 Sections 2.11, and 4.8.10, and Article 8 of EU Directive 95/46/EC generally expect that access to an EDC system should be limited to authorized staff.
Security cannot be achieved without user compliance. As such, site user training is often provided by data managers as part of study start-up as described in the GCDMP chapter titled EDC – Implementation and Study Start-up. During training all users should be informed about system access rules. During monitoring visits, the sponsor or designee should remind site staff of the importance of confidentiality for each user’s ID and password and should be vigilant for signs of security weaknesses such as user identifiers or passwords posted in plain sight or sharing login credentials. Suspected noncompliance with access rules should be reported to the assigned system administrator as appropriate.
EDC systems generally use role-based security, where different system actions can be performed or accessed for specific user roles. System privileges such as entering or updating data, adding a new patient, responding to queries, or performing SDV are assigned to roles. When users are granted system access, usually through provision of login credentials, they are usually also assigned to a role in the system. The role then confers privileges for what can and cannot be accomplished within the system by the user. Study-specific role and responsibility definitions and corresponding user access and privileges are often maintained in a matrix showing privileges corresponding to each role. Documentation of system access must comprehensively represent all system access throughout the study.
Managing user accounts and permissions is a time-consuming task, requiring diligence to ensure security and confidentiality are maintained throughout the duration of a trial. Open communication with clinical operations is necessary to keep track of site and contract research organization (CRO) staff changes so as to activate or deactivate corresponding user accounts as needed. User access to the EDC system and all other study data should be periodically reviewed.
Each user of an EDC system should have an individual account, consisting of unique credentials, often a login ID and password. Typically, the initial login ID and password can be sent to the individual user using his or her e-mail address, or through traditional methods such as mail or courier. The system administrator should only grant a user access to the system once the user’s role-specific training has been completed and documented. [VI]
If credentials were supplied to a user or otherwise known by others, the user should be required to change their initial login ID and/or password when a user first logs into the EDC system.
Turnover of site and study team members is likely. Therefore, management of user access will be an ongoing task throughout the course of an EDC study. Procedures should be in place for revoking access when users change roles or leave the project or organization.
Procedures should be established to define processes for disabling or revoking access to the system as needed.
The sponsor should define appropriate lock-out rules in the event of unauthorized access, whether attempted or successful. [VI] If a user enters an incorrect ID or password, an alternative method, as specified through established standard operating procedures (SOPs) or work instructions, should be employed to quickly reauthenticate the user and to quickly reestablish system access for reauthenticated users. [VI]
Throughout the course of a trial, it will become necessary to add new users or modify access privileges for existing users. Procedures should be established to ensure these tasks occur without disruption of ongoing study activities. [VI] These procedures should detail training prerequisites, steps for requesting access, and the staff members who are responsible for ensuring all site staff and study team members have appropriate access. [VI] Documentation of completed training should be provided to the appropriate personnel so they know which users may be granted new or modified access rights. [VI] Documentation should be maintained throughout the course of the study and archived with study documentation.
When available, reports (which may include surveys) detailing the responsiveness and effectiveness of software support (e.g., the average length of time the help desk takes to assist a user) should be reviewed regularly to ensure support is effective. Several factors are important to ensure assistance is provided efficiently and expeditiously, including easy access to support staff, ability to address users’ questions, and the availability of support when needed.
Although language needs for the help desk should be determined during the pre-production phase of a study, CDM staff should be sensitive to complaints regarding communication problems during the study conduct phase. [VI] The problems may be, in part or in whole, related to an inability of the help desk to provide the language support needed, and may require a revision to the original translation needs of the study.
As with multiple language support, help desk availability should be determined prior to the start of a study. However, during the conduct of the study CDM should evaluate feedback from users to ensure that the availability of support is adequate for the study. [VI] Reports detailing the responsiveness and effectiveness of software support should be reviewed regularly to ensure that front-line software support is effective. [VI] Front-line software support is the lowest level of support needed and includes activities such as unlocking user accounts and resetting user passwords. Information gained from reports and feedback may involve reevaluating the original decisions regarding the level of support needed. For example, if 24 × 7 × 365 support was not originally set up, it may be necessary to reconsider the cost.
Where the EDC technology is supporting subject safety or regulatory requirements, down-time procedures for operational continuity when the system is down as well as back-up and recovery procedures must be in place.
EDC-related training should be provided to anyone who uses the system. [VI] Training is most effective when provided as close as possible to the time when the newly learned skills will be used. If a significant time lapse occurs between training and use of the learned skills, retraining should be considered.
EDC system training is an important part of proper study management. Training is dependent on the study and target audience; therefore, training materials should be developed with these considerations in mind to make the training as effective and appropriate as possible. [VI] Moreover, training should be an ongoing process, not just a one-time event. [VI] An EDC system can provide the sponsor with the ability to identify a need for retraining users. Some EDC systems can also be used by the study team to deliver updated training materials and communications to users in a timely manner. For example, updated CRF instructions can be immediately provided to all sites and study team members, and newsletters can be provided through a dedicated website to communicate updates or changes.
Identifying users’ needs for retraining is an important activity of both CDM and clinical operations team members who interact with the site regularly. A mechanism should be in place for the CDM to become aware of situations at a site that may present challenges and a need for retraining, such as coordinator inexperience, isolation, turnover, or competing priorities. [VI] Available information, such as help desk reports, query frequency reports, and protocol deviation reports, can be used to identify materials that need to be updated or users requiring new or additional training. For example, with query frequency reports, one site can have a misunderstanding of
A common occurrence in clinical research is turnover of both site and sponsor staff. New staff should receive required training. [VI] A plan should be established for new users to be trained in a timely manner so they will have the benefit of access to data on the EDC system. [VI] If new site staff are not trained and do not have access to the system, they cannot enter data, and study timelines can be negatively affected. To ensure regulatory compliance, controls should be put into place to ensure untrained users do not have access to the system.
A mid-study request for subject data can occur for many reasons, including, but not limited to
A scheduled interim statistical analysis based on study design and protocol, which typically focuses on efficacy data
An interim review of data focusing on safety data, such as adverse events and other data that indicated safety issues in earlier studies (e.g., ECG data, lab panels)
DSMB or Clinical Endpoint Committee (CEC) regularly scheduled meetings
A submission package or other type of update (e.g., 120-day safety update) for regulatory purposes
Any other planned or unplanned data lock
A major factor affecting delivery of mid-study subject data is whether the data are stored by the sponsor or a vendor. If data are stored by the sponsor, the data should be readily available, thereby reducing costs and resources needed. If a vendor’s hosted system (Application Service Provider (ASP) model) is used, the timing and frequency of deliveries are more important.
Whether a sponsor or vendor system is used, the required subject data should be clearly identified. [VI] Examples of prerequisite identification for exporting subject data include but are not limited to.
An interim analysis planned to occur at a particular milestone (e.g., the 100th randomized patient) or of a particular module (e.g., dosing data)
A safety review planned to occur at a particular milestone (e.g., 25% patients enrolled, 50% enrolled)
A mid-study efficacy analysis based on statistical design of the protocol
Regularly scheduled DSMB/CEC meeting
In addition to determining which subjects are to be included in an export, the sponsor should identify which records are to be included in the delivery. [VI] The simplest solution is to include all study data, regardless of its status. However, delivery could be restricted to data verified by the CRA or monitor, or to locked (clean) data, which requires close coordination with the CRA for scheduling monitoring visits. If data are to be used for an interim safety analysis, reconciliation of SAEs and Medical Coding may require additional attention.
Any external data that is to be integrated into the database prior to providing any subject data mid-study (e.g., laboratory data or ECGs) should be planned in advance of the study team’s timeline for reporting. [VI] As necessary, the completeness and accuracy of such data should be ensured by reconciliation before the data delivery occurs. [VI]
The recipients of requested study data and the impact to study blinding should also be considered. [VI] For interim analyses, datasets are typically provided to a biostatistician or statistical programmer, who subsequently creates tables or listings from that data. Timing of the delivery (e.g., planned or on demand) is also an important component to consider. If required data deliveries are scheduled, necessary procedures can be planned in detail. [VI] However, if ad hoc requests for data are anticipated, the process for exporting and delivering data should be defined in the SOPs or Data Management Plan. [VI] When ad hoc requests are received, programs should be tested and validated to ensure timely delivery.
Regulatory agencies such as the Food and Drug Administration (FDA) require CRFs from subjects to meet certain criteria. As required by CFR 314.50(f), for any new drug application (NDA), individual CRFs are to be provided for any subject who withdrew from the study due to an adverse event, or who died during the study.
The sponsor should be prepared to transfer CRFs at any time during the study such as for an NDA periodic safety update or integrated safety summary. One possible solution is to provide electronic copies of CRF images. When working with a vendor, the sponsor should factor the process for obtaining CRFs in the contract’s study timelines and expectations (e.g., maximum number of requests).
Web-based EDC brings unique data quality measurement challenges. The process of getting data into the EDC system may be handled differently at clinical sites. Some sites will use paper CRF-like worksheets on which to initially transcribe data abstracted from source documents such as the medical record. The paper form will then subsequently be entered into the EDC system. Other sites will abstract the needed data and enter it directly into the EDC system. In both cases there is a source document. The difference in the two approaches is the use (or not) of the paper form as an intermediate step. Either way, the error rate between the source and the EDC system (1) is needed (per ICH E6 R2 ss 5.1.3) and (2) can only be obtained by SDV.
With the transition to web-based EDC, this additional task necessary to measure the error rate was not routinely added to monitoring responsibilities; thus, few organizations measure the source to EDC error rate.
Further, use of EDC does not change the fact that medical record abstraction is a manual process. As such, ongoing measurement of the error rate and feedback to sites is strongly recommended to maintain the error rate within acceptable limits for the study.
The EDC data audit process may differ between organizations. The auditing process will be impacted by how the data can be extracted from the EDC system and the source used for comparison. Any additional programming required to transform study data into SAS® datasets could affect how data are displayed. Additional EDC issues to consider for auditing include, but are not limited to, reconciling medical coding, data management plan comparison, external data import, the extent of process auditing, query verification, and completion of all queries that required data changes.
An audit plan should be established in advance to identify approaches to be taken, including sampling, establishing sources for comparison, acceptable error rates, how audit findings will be reported, and expectations for corrective and preventative action.
Any EDC system may undergo changes during the conduct of a study because of changes in EDC software and/or changes in the study itself. Efficient change control processes are needed to quickly address issues that arise during a study to prevent further occurrences and to implement corrective and preventative action with minimal disruption.
Because clinical trials may occur over the course of several years, software changes and upgrades will inevitably have an impact on EDC studies. These changes or upgrades are not just limited to core EDC software, but could also include upgrades to the operating system, back-end database software, or any auxiliary software integrated with the EDC system, such as reporting or extracting software. The differences in change control strategies and processes depend on whether the system is developed internally by the sponsor or purchased from a vendor.
If software was purchased, the sponsor may decide to rely on the vendor’s system validation package for the software, including all releases or upgrades, and maintain the system as a “qualified” platform rather than performing system validation upon each release. [VI] However, “qualified” software platforms should not be customized by the sponsor unless validation of the customized platform will also be performed.
In order to implement upgrades to the software system (whether it is a new release, a minor version update or a SAS® update), CDM representatives should make a complete assessment of the software changes and obtain input from other functional areas that may be impacted, including a thorough risk assessment. [VI] Documentation of such should be maintained for each version release.
The first step in performing an assessment is to gain a clear understanding of all changes or additions that will be made to the software. For software purchased from a vendor, this task can be accomplished by ensuring that the software release notes are reviewed and well understood by appropriate staff. Release notes should include documentation of all changes, any known issues in the new release, and instructions for upgrading the software from previous versions.
For software produced internally by the sponsor, a well-developed change control process should be established. This process should include steps for reviewing change requests, grouping multiple change requests together as appropriate, updating requirements and design documentation, build, testing, and implementation.
To determine whether a software system should be upgraded, the sponsor should consider the following issues:
Impact on data – Assess if any changes in software functionality could potentially impact data integrity. For example, if certain characters or functions will no longer be supported, the sponsor must make sure data integrity will be preserved after the software upgrade. [VI]
Impact on existing code – The software upgrade may require you to make changes to existing programming code. [VI]
Auxiliary systems – The sponsor should assess how related systems or programs will be affected by the software upgrade. Will other systems require corresponding upgrades or modifications? [VI]
Impact on sites – Will the study be inaccessible during the software upgrade? Is the site required to perform certain tasks such as installing software on their local PCs or changing browser settings? Will the site require additional training? How will the sites be notified of the impact? [VI]
Comparison of cost and value – The costs of implementing and validating a software upgrade should be compared with the business value to be gained. [VI]
The impact on ongoing studies – Considering the impact on the study database and remaining duration, is it worth upgrading software to a new version? Does the software for ongoing studies need to be upgraded simultaneously? [VI]
SOPs and training materials – Will the software upgrade require revision of the sponsor’s SOPs or training materials? [VI]
For internally produced or customized EDC software, new requirements documentation should be created.
In addition to the requirements documentation, clinical data management will need to develop a test strategy that documents the testing and validation required for the new software. Depending on the type of upgrade, intensive testing is not always necessary. The following guidelines can be used to determine required testing efforts:
For a minor version (bug fix or upgrade), limited testing may suffice. [VI]
For a new release or major version upgrade, moderate to intensive testing is usually advisable. [VI]
For purchased EDC systems, the vendor should be able to provide and maintain testing plans and results. [VI] For internally produced EDC software, test scripts or test cases based on new user requirements should be produced.
After validation of the new release has been successfully completed, the new version or changes can be implemented in production. Please refer to the “Database Validation, Programming and Standards” chapter of
While a minor upgrade to software is likely to go unnoticed by users, a new release or major upgrade to software could require additional training. The sponsor should determine the required level of training, which users should receive training, and the method of providing the training. [VI]
Typically, sponsor staff (CRAs/CDM) and site staff will require training, which can be delivered in person, from durable media, or over the internet. Presentations using screen images can be particularly beneficial for training purposes, as they can be reused for later training sessions. Sponsor staff should be trained first on the software’s new or modified functionality and then the site staff. [VI]
Before making new software available to staff, the impact of the revised software needs to be assessed. [VI] For example, if software revisions will require modification of approved CRFs, the sponsor should identify Institutional Review Board (IRB) issues to be addressed (IRB issues are not likely to apply to a software upgrade but may apply to CRF revisions). [VI] The sponsor should determine whether new software should be upgraded in stages or all at one time.
Sites should be informed of the rollout of new software with sufficient time given for necessary preparations. In case the upgrade does not occur as expected, a clearly defined rollback or fallback plan should be established prior to software implementation. [VI] For international studies, the time available to perform software upgrades can be limited and may require the upgrade to be completed during normal business hours.
Software vendors typically maintain all software versions for a defined period of time. The sponsor should be aware of the level of support provided by these vendors. When a vendor rolls out a new system, they may not continue to offer the same level of support for earlier versions of the software system and may eventually retire earlier versions. Typically, once a version is retired, the software vendor no longer provides support for that version.
Because of ongoing development of software systems, the sponsor should plan for future changes, and determine when it is appropriate to upgrade or retire an existing system. Some factors to consider include
Software upgraded during study conduct
Vendor’s support commitment to previous versions of software
Software or hardware that becomes obsolete
Decreased system performance
Trial timelines
In contrast to software changes, a trial could also be affected by study-specific changes, such as protocol amendments or development issues. As a result, CRFs, edit checks, and/or reports may need to be modified. As change control is a heavily regulated process, any change should be documented, analyzed, and tested.
Version-control software should be used for CRFs. [VI] Electronic files may be organized by study in a hierarchical directory structure. For each study, initial release of and subsequent changes to CRFs should be indicated, e.g., by directory naming conventions or labels. [VI] Identifying the date and time files were released must be possible so the release timeline is clear for regulatory purposes or to troubleshoot issues with CRFs.
Before rolling out the new version of a CRF, CDM may need to assess who the changes will impact. If the changes resulted in revisions to already approved CRFs, the reviewer should determine if these changes will impact the sites and IRBs and inform all appropriate study team members and sites well in advance. [VI] For international studies, the time to deploy an updated version is more limited and may require deployment during normal business hours.
CDM should consult with clinical operations to determine whether to roll out the new version in stages or all at once. If changes are rolled out mid-study, the study team should first be notified when the changes will be available, and whether the study team will have an opportunity to review changes prior to deployment to sites. [VI] Site staff should be notified of training and rollout before changes are released to the production system. [VI] Once changes have been made, all parties should be notified. [VI] To ensure appropriate sites are moved to the new version of the CRF, CDM should create a log that will keep track of when each site was moved to the new CRF version. [VI] Proper logging will ensure no sites are missed in the process. Not all sites may require the new version, such as in cases where the changes are related to a protocol addendum. Target dates should be set and tracked for upgrading sites to the new system, which should be closely monitored and tracked. [VI]
Data entered in the previous version of the EDC study should be made available to the sites and study team.
The final review precedes the database lock process. A list of all close-out activities and what triggers them should be used.
Before a form can be locked and the database closed, source documentation and data review should be completed as required for the study for all forms and fields. [VI] It is beneficial if the EDC system can indicate SDV status and data review activity. Prior to locking a database, ensure all required SDV has been completed by the CRAs.
Frequent communication between the clinical study team, including the CRAs and clinical data management team, is critical. Changes to data in the CRFs are possible as late as the day of database lock. A plan should be established to ensure CRAs are available to source verify any changes, if necessary. [VI]
All reasonable efforts should be made to ensure all queries are answered or closed prior to database lock, particularly for those queries that may impact the analysis or outcome of study results. [VI] Depending on the details of the Data Management Plan (DMP) it may be acceptable to lock noncritical forms with open queries. During study startup, conditions for locking a CRF should have been defined thereby ensuring, for example, that the CRF cannot be locked with open query status. Prior to locking a database, CDM should ensure all queries have been answered or closed, including automatic queries and manual queries created by CRAs and clinical data managers. [VI] This task may be accomplished through use of varied system reports and status indicators, according to the EDC system’s features.
It may be necessary to verify that all CRFs have been electronically signed by the investigators (principal and/or sub) responsible for a particular site. The investigators are responsible for reviewing each subject’s CRF to confirm that data entered are complete and accurate. Sponsor organizations must determine the level of detail required for investigators’ signatures. [VI] For example, some organizations will decide that the investigator’s e-signature must be applied to every individual CRF, while others may decide one investigator signature on the final page of each subject’s CRF is acceptable.
Ensure investigators and other site staff members are educated in the signature-break process long before study closeout so there is no confusion on the topic of closeout. [VI] Signature-break may occur when data has been changed post investigator signature. Should signature-break occur, this information will help avoid confusion, and will ensure the investigator is available to re-sign if necessary.
While there are many studies using e-Signatures, some are also still working with paper-based signatures, even with those studies utilizing EDC.
Regardless of the final signature method being used, a process should be established for notifying sites that CRFs are ready for investigator signature. Policies and processes related to re-signing a CRF should also be defined and adhered to. [VI] If a site changes a data point on a CRF already signed by the investigator, the rules must be in place to decide whether the data change “breaks” the signature. If the signature is broken due to the change, the investigator must re-sign the CRF.
It is expected that the CRA will track the investigator’s progress signing the CRFs. However, the clinical data manager may be responsible for conclusively verifying that all CRFs have been signed by the investigator prior to database lock. For any data changes that “break” the signature, the clinical data manager must verify those CRFs are re-signed. [VI]
The term “lock” may refer to not only locking a study or database, but may also refer to locking specific forms or casebooks. While not all topics discussed in this EDC-specific chapter actually occur during study closeout, they are common components of the database lock process and closeout of a study. For detailed information on a database lock, reference the GCDMP chapter on Database Closure. It is recommended that a checklist be established to ensure completion of required tasks prior to database lock.
Tasks on such a checklist may include, but are not limited to
Identifying personnel responsible for carrying out database close-out and lock activities
Ensuring all data have been received and outstanding data have been addressed.
Ensuring required medical coding of adverse events, prior and/or concomitant medications, and medical history verbatim terms has been completed; i.e., that all uncoded terms have been addressed and any quality review of the coding has been completed and the results are within established acceptance criteria.
Ensuring that all data discrepancies including those performed within and outside the EDC system and identified during data listing reviews have been resolved
Importing and/or reconciling all external data (and listing external data reconciled), for example, serious adverse event (SAE) reconciliation
Exceptions identified during final reviews and final checks sometimes are unanticipated, are not covered by pre-defined acceptance criteria, and require decisions be made and documented. Preparing for this eventuality by calling close-out meetings of decision-makers to review and decide such exceptions while data are still blinded (in the case of a blinded study) helps to keep close-out on track. [VI]
Once all database lock tasks have been completed, the database can be closed and soft- or hard-locked according to the sponsor-approved definition. This step implies that all forms in the study have been locked according to defined procedures, all tasks are completed, all conditions have been met as defined in the data management plan, and the final data transfer has been received or extracted according to data extraction specifications defined at the start of the study.
Planned QA and QC audits should be performed before database lock. [VI] Should an audit be performed on locked data? Based on findings from the audit of final data, further data corrections may be requested of the site. Once corrections have been made and verified, and additional investigator signatures have been obtained as necessary, another data transfer or extraction should occur. It is recommended that a comparison program be run to determine if requested changes to the database were successfully executed, as well as any other changes that were made to data other than what was expected. [VI]
Typically, records that are soft-locked cannot be updated by sites, but the sites may still be able to respond to open queries. Many EDC systems support soft locks at a visit, page, or data point level. This capability supports a process of “rolling” or gradual soft locking of data throughout the course of a study, reducing the effort required by the CDM to lock data at the end of the study. Incremental soft locks can also be an effective approach to supporting mid-study or interim data review activities.
Typically, a database hard lock occurs when all data have achieved soft lock status and all other study closeout activities have been completed. After a database has been hard-locked, tightly controlled procedures for unlocking the database must be employed, and only a few privileged users should be able to modify data. [VI] Once the database has undergone a hard lock, the data are considered ready for final analysis and archiving.
At the conclusion of a study, user access rights must be modified. During study closeout activities, rights should be modified to allow the site only to read data but not enter or change data. Once the required archives have been created, sent, and confirmed received by the site, all access (including view access) to the corresponding data in the EDC system can be removed. Prior to this revocation, the site must continue to have access to the study database in the event of a site inspection or audit. [VI]
Understandably in early EDC studies, evaluation questionnaires were frequently performed.
At the conclusion of a study, the sites must be provided a copy of data they collected throughout the study.
A final review of all study documentation should be performed to ensure defined processes have been adhered to such that end of study deliverables from the EDC system meet the requirements specified in applicable regulatory guidance documents. This review should ensure that the study documentation address expectations from regulatory agencies. Some of the guidance and specifications for this review include
Subject data should present the data collected in a CRF system organized by subject identifier, visit, and form. The data are typically presented in a manner that allows for effective navigation of the subject profile in a format such as a PDF file or a similar format. The subject data should also include an audit trail, electronic signature, and query information, which allows a reviewer the ability to see all data entry and modification that have occurred since it was created. Subject data should be provided on durable media. A master copy of the durable media should be created to contain all the subject profile data. In addition to this master copy, individual copies or other durable media with site-specific data should be created and forwarded accordingly.
After the study is complete and a copy of subject data have been generated and securely (e.g., password protected) distributed to sites successfully, a complete copy of the study database should be created for archival purposes.
Data Management Plan
User Training
User Management and Security
Study/CRF and Edit Check Change Control
Software System Change Control
Maintenance of Coding Dictionaries
Data Acquisition and Processing
Study Closeout
Database Lock
Generation and Review of Archive Media
System Setup/Installation
Data Collection and Handling
System Maintenance
Data Backup, Recovery, and Contingency Plans
Security
Change Control
Rusnak provides a good outline for a site SOP covering these topics.
This revision is based on a systematic review of the peer-reviewed literature indexed for retrieval. The goals of this literature review were to (1) identify published research results and reports of EDC methods and evaluation and (2) identify, evaluate, and summarize evidence capable of informing the practice of management and closeout of studies using web-based EDC.
The following PubMed query was used:
The search query was customized for and executed on the following databases: PubMed (777 results); CINAHL (230 results); EMBASE (257 results); Science Citation Index/Web of Science (393 results); Association for Computing Machinery (ACM) Guide to the Computing Literature (115 results). A total of 1772 works were identified through the searches. The latest search was conducted on February 8, 2017. Search results were consolidated to obtain a list of 1368 distinct articles. Because this was the first review for this chapter, the searches were not restricted to any time range. Literature review and screening details are included in the PRISMA diagram for the chapter.
Two reviewers used inclusion criteria to screen all abstracts. Disagreements were adjudicated by the writing groups. Forty-nine meeting inclusion criteria were selected for review. The selected articles were read by the writing group and 104 additional sources identified through the review. Each of the 153 articles was read for mention of explicit practice recommendations or research results informing practice. A total of 82 articles were identified as relevant to the history of EDC or to one or more of the EDC GCDMP Chapters. Fourteen articles provided evidence for this EDC Chapter. Relevant findings from these 14 articles have been included in the chapter and graded according to the GCDMP evidence grading criteria in the table below. This synthesis of the literature relevant to web-based EDC was performed to support the transition of the EDC chapters to an evidence-based guideline.
Articles supporting assertions or otherwise informing practice recommendations in GCDMP chapters are graded according to the strength of the evidence. The GCDMP has adopted the following grading criteria.
Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) For Web-based EDC Study Conduct, Maintenance, and Closeout.
Date | Revision description |
---|---|
September 2003 | Initial publication as Electronic Data Capture Principles. |
May 2007 | Revised for style, grammar, and clarity. Substance of chapter content unchanged. |
September 2008 | Revised to reflect the orientation of chapter towards the conduct phase of EDC. Content updated and organization of material revised. Study concept and start up, and study closeout content moved to separate chapters. |
May 2020 | Content updated and organization of material revised. Study conduct was combined with study closeout into one comprehensive chapter. |
The authors have no competing interests to declare.