Web-based electronic data capture (EDC) has become the preferred method for capture of key-entered data in clinical studies. This chapter reviews the considerations for selecting an EDC system including evaluation of systems and vendors, user requirements, and process change, as well as initial implementation of systems within organizations. The goal of system selection is to assure that organizational needs are identified and documented and ultimately that that the desired functionality is available and appropriately supports clinical studies conducted by the organization. Multiple roles on study teams use the EDC system and should be involved in software selection and initial implementation.
After reading this chapter, the reader should understand
The regulatory basis for practices in selecting an EDC system
Common requirements and functionality domains of EDC systems
Key domains and criteria for pre-selection evaluation of EDC systems
Process impact and redesign considerations at evaluation and selection time
Initial system implementation within an organization
Historically, data for clinical trials have been manually abstracted from medical records, electronically extracted from medical records, or recorded directly on Case Report Forms.
Reports of EDC predecessor systems and process, called Remote Data Entry (RDE) started appearing in the 1970s with the then increasing accessibility of computers.
Choosing an EDC system can and should be a significant decision for an organization. EDC system selection decisions are complex. Such decisions involve choices about processes for collecting and managing data that involve the entire trial team. Adopting new information technology offers new opportunity for process redesign such as workflow automation and other decision support. It has been argued that the real gains from EDC come with use of the technology to re-engineer processes.
While sometimes EDC systems are chosen for an individual study, more frequently the chosen system will be used for multiple studies conducted by an organization and will impact operating procedures at clinical investigational sites and throughout clinical study operations. As the primary system for data collection and management in clinical studies today, EDC systems can be a cornerstone of, integral component of, and leaping-off point toward greater safety, quality, and efficiency in clinical studies.
This first of three chapters on web-based Electronic Data Capture (hereafter EDC) covers considerations in and criteria and processes for selection of software for web-based EDC in clinical studies. Topics covered include common EDC system functionality, evaluation of candidate systems and vendors, and consideration of process impact and potential for process redesign at the time of system selection. The primary focus in this chapter is the identification of requirements for the EDC platform itself. These include functionality important to human subject protection, data integrity, study conduct – including needs of system users, and Title 21 Part 11 compliance. Other important aspects of choosing an EDC system such as software delivery, acquisition models, vendor stability, and cost estimation are addressed as are initial software implementation considerations.
Recommendations for building a study within an EDC system, testing a built study, study start-up, and provisions for change control for an EDC study are covered in the second EDC Chapter titled “Electronic Data Capture –Study Implementation and Start-up.” Aspects of study conduct and study closeout are addressed in the third and final EDC Chapter entitled “Electronic Data Capture – Study Conduct, Maintenance, and Closeout”. The EDC chapters contain applications of good clinical data management practices
The
“The sponsor should implement a system to manage quality throughout all stages of the trial process.
Sponsors should focus on trial activities essential to ensuring human subject protection and the reliability of trial results. Quality management includes the design of efficient clinical trial protocols, tools, and procedures for data collection and processing, as well as the collection of information that is essential to decision making.
The methods used to assure and control the quality of the trial should be proportionate to the risks inherent in the trial and the importance of the information collected. The sponsor should ensure that all aspects of the trial are operationally feasible and should avoid unnecessary complexity, procedures, and data collection. Protocols, case report forms (CRFs), and other operational documents should be clear, concise, and consistent.
The quality management system should use a risk-based approach.”
A Quality Management System necessitates that executive leadership articulate and support a quality policy that documents leadership intent with respect to quality management. Because methods used to collect and manage data impact quality, executive leadership support for EDC selection and use is imperative. Further, leadership should assure that the quality management system extends throughout the organization and to vendors, suppliers, and sub-contractors where appropriate through a vendor qualification and management program.
The March 2018
The
Section 5.1 “Systems and processes should be designed in a way that facilitates compliance with the principles of data integrity.”
Section 6.9 of the MHRA guidance states, “There should be adequate traceability of any user-defined parameters used within data processing activities to the raw data, including attribution to who performed the activity.”
The
The FDA guidance,
Further echoing the eSource guidance Section V.C.2 states, “If a potential for unblinding is identified, sponsors should determine whether the use of interoperable systems is appropriate or whether other appropriate controls should be in place to prevent unblinding.”
Similarly, the FDA’s
In the background section, the guidance states, “Source data should be attributable, legible, contemporaneous, original, and accurate (ALCOA) and must meet the regulatory requirements for recordkeeping.”
The same section states, “clinical investigator(s) should have the ability to enter comments about issues associated with the data”.
With these requirements in mind, in Table
Minimum Standards.
1. Secure upper management support for the selection of an EDC system and the functionality to be sought in the selection. |
2. Identify system requirements to be used as software evaluation and selection criteria including but not limited to functionality needed for human subject protection, data integrity, study conduct, and regulatory compliance. |
3. The identified functionality should be comprehensive for the organization and serve as the starting point of the traceability matrix used in software validation. |
4. If a commercial product is to be used, identify vendor selection criterion; these include functionality, business, Quality Management System, or other characteristics to be assessed as part of vendor qualification. |
5. Perform and document a risk assessment with respect to the EDC system as intended to be used. |
6. Undertake and complete software validation activities and other controls commensurate with the risk. |
7. Assess and complete changes to Standard Operating Procedures (SOPs) and other governance policy and procedures necessitated by EDC-enabled processes. |
8. Create and maintain SOPs covering the software evaluation, selection, and implementation processes. |
Best practices, as stated in Table
Best Practices.
1. Identify appropriate personnel participation in the evaluation and selection of EDC systems. [VI] |
2. Ensure that internal or external skills are a match for the chosen EDC software acquisition model and are present to support the system and studies for which the system is used. [VI] |
3. Separate infrastructure development from the set-up and conduct of individual studies. |
4. Identify and leverage EDC functionality to improve data collection and management; e.g., decision support, process automation, and interoperability with other information systems. |
5. Where software is acquired as a service or with services, prepare a thorough Scope of Work (SOW) to document and ensure complete understanding of expectations and deliverables. |
Grading Criteria.
Evidence Level | Evidence Grading Criteria |
---|---|
I | Large controlled experiments, meta, or pooled analysis of controlled experiments, regulation or regulatory guidance |
II | Small controlled experiments with unclear results |
III | Reviews or synthesis of the empirical literature |
IV | Observational studies with a comparison group |
V | Observational studies including demonstration projects and case studies with no control |
VI | Consensus of the writing group including GCDMP Executive Committee and public comment process |
VII | Opinion papers |
Choosing an EDC system involves multiple considerations including strategy, goals, business relationships, and capability of the selecting organization and their desired provisioning models, software functionality, system performance, and available funds. Thus, the list of stakeholders in EDC selection and implementation can be extensive. Operationally, data collection and management decisions such as whether EDC should be used and, if so, for which data and through what types of processes can impact most operational groups involved in clinical trials. Impacted operational groups include, but are not limited to, project management, data management, clinical operations, research pharmacy, biostatistics, information technology, and contracting. Individuals from all of these functional areas are possible stakeholders in the EDC selection and implementation process. [VI] The likelihood that all stakeholder needs will be reflected in the selection of an EDC system increases when those stakeholders and their responsibilities are identified early in the decision making process. [VI]
There are over 60 EDC system vendors in the EDC market.
The cost of licensing an EDC system ranges from free to hundreds of thousands or millions of dollars. Pricing models vary among vendor organizations and historically have included charging by data value, data element, or CRF screen while others charge by study or even implementation. Other factors in pricing may include number of simultaneous or named user licenses and the services provided by the vendor. In addition, consideration should be given to service level agreements for availability of platform, system response time, and accessibility of support. The purchasing organization may prefer making an upfront one-time investment versus paying by study or incurring monthly fees. Organizational vendor contracting processes may not allow for the type of service contract offered by the vendor. Organizational processes may only authorize paying invoices for services that have been provided already, but a vendor may require a quarter payment upfront. For some organizations budget constraints may be a large factor in a decision.
The implementation timelines will also drive the selection process. The timing for the installation and validation of the EDC system must be considered to ensure that the system is ready to start a study build when needed and in full compliance with applicable regulations. Additional timing considerations include training, length of time to build a study specific application, and the extent of changes to organizational processes and SOPs required.
If the vendor is a public company, their financial and historical performance can be obtained freely. Knowing about the vendor is essential; information such as the following may provide insights:
experience with the type or types of studies of interest
experience in one or more therapeutic areas of interest
experience in one or more regions of the world
number of current/past customers
software development experience
aspects of the vendors software development quality management system
number of employees
ratio of development personnel to entire staff
length of time in business
financial stability
past performance on similar volume and scope of functionality or services
Obtaining recent customer references is also strongly advised.
Current SOPs may dictate a specific process that the system may or may not support. For example, organizational SOPs may allow a data manager to review and close queries, regardless of how they were generated, whereas a system may only allow the “monitor” role to close queries generated during the source document verification process. Such a difference would require a work-around or change to organizational process. There are many opportunities for role-based functionality like that employed in most EDC systems to conflict with current organizational practices, roles, and workflow. These conflicts necessitate changes in organizational practices, generation of work-arounds, or changes in software functionality prior to implementation. These organizational and business considerations may become selection criteria.
There is a core set of common EDC functions such as entering data and identifying data discrepancies covered by most EDC systems. See early functionality lists by McFadden et al., Kush et al., El Emam et al., and Franklin et al.
Common EDC product features are described but are not limited to those listed below.
Interactive Voice Response Systems (IVRS), Interactive Web Response Systems (IWRS), or Interactive Response Technology (IRT) systems may be used for randomization in trials. Some EDC systems have built-in randomization functionality that is fully integrated as part of the baseline configuration Other systems do not have this feature or do not support the type of randomization chosen for a study necessitating use of an external randomization system. In either case, the combination of this functionality integrated with the EDC provides a powerful solution that reduces data entry for site staff and ensures no transcription errors in research subject identifiers.
Integration of this data from external vendors involves building a secured pathway via a secure File Transfer Protocol (sFTP), Web Services (SOAP/REST), or other secure mechanism that will transfer the files generated from the vendor system and place these files in the EDC compatible host. This process is sometimes referred to as developing an IVR/IWRS program.
The point at which data are integrated should also be informed by reporting needs. Data from EDC, ePRO, vendors, or other sources often need to be viewed together to assess data status, completeness, or payment milestones. Where such data integration is not required for study operations, data from disparate sources may be merged prior to analysis.
When considering features and functionality requirements, data managers should be aware that most vendors are in the continual process of improving/updating their system functionality. Therefore it may be important to understand features and their development timelines, such as whether the feature is a standard feature of the product in current release; will be a standard feature of the product in future release within the next 12 months; the feature needs a specific modification to the software; or the feature will not supported within the next 12 months. These are included in the sample Request for Proposals in the Appendix.
A good starting point for feature understanding is to think about needed system features in the context of a specific or typical project. The high-level business needs and desired system features form the requirements used in EDC system selection process. Although much older than the list above, McFadden et al. provide a pragmatic, basic list of needed functionality.
Other specialized needs may include functions such as the following:
Document management such as providing a webpage for making current versions of study documents available to sites
Automatic reporting, sending out populated forms (e.g., CIOMS/SAE) to external Pharmacovigilance (PV) systems
Collection and management of reference ranges by site for local labs
Collecting data for or triggering milestone base for Site Payments.
El Emam et al. offers a hierarchical list of EDC functionality where systems with more advanced functionality could be assumed to have more basic functionality.
There are multiple business models for delivery of the EDC software or a study application. These include technical transfer, software as a service, and software as a service with services. The most used models today are software as a service and software as a service with services.
This is when an organization purchases a license to use software, acquires the software, and installs the software in an environment of their choosing. Requirements include having or having access to the appropriate hardware and network connectivity. Where Title 21 Part 11 compliance is required, the new installation has to be qualified, validated, and maintained under change control. There should also be procedures for the use and support of the system as well as training for system support staff. In this model, all study builds and upkeep are done in-house.
The SaaS option is sometimes referred to as hosting, or more recently as cloud computing and was formerly known as an Application Service Provider (ASP) model. Software as a service, e.g., provided via the internet, involves providing the hosted application as a service. The EDC vendor has all the software in their environment and it is accessed via the internet. A SaaS model may provide early benefits when starting out utilizing an EDC system. The EDC vendor can offer support and expertise for early trials based on previous experience. EDC trials may be initiated faster since they are not dependent on a technical implementation. The pricing models differ between vendors; however, typically there will be the licensing fee to use the software, per study fees based on the time of trials, and per service costs. Study builds can be done by the sponsor or designee, e.g., a CRO or the EDC vendor.
Software as a service with services includes the vendor additionally providing services such as building the study application within the EDC system, training, or data management. Newer EDC applications that utilize cloud computing may offer “per-study” models. In this scenario the sponsor or designee pays for each study to be hosted. There will often be a one-time fee for building the study, the EDC vendor will build the study, there will often be a monthly subscription charge to maintain and support the study, and there may or may not be a per user fee or per site fee. In this model, the agreement with the vendor may be limited to one study or may cover multiple studies.
With new technology, organizations tend to progress from fully outsourcing through various stages of bringing services and sometimes the technology itself in-house.
Software and support services delivery models play a large part in setting roles and responsibilities between a vendor and customer. Determining and clearly articulating the roles and relationship(s) among the EDC vendor, sponsor, and CRO is a fundamental step in selecting an EDC vendor. This is particularly important in situations, where a sponsor uses multiple EDC systems with multiple models across multiple projects.
New technology is being delivered to the marketplace at a fast pace. Vendors are expanding existing functionality and developing partnerships for augmented offerings. New technology to handle new sources of data are also emerging. Organizations need a forum for ongoing dialog about the current strategy, environmental scanning for new opportunities, assessment of need, value offered, and risk, and decision-making regarding piloting or integrating a new technology into the organizational tool box.
At some point, decision-makers want to know and weigh the additional value added by a new technology. Waife points out that doing so requires organizational awareness and discipline to accurately calculate the cost of the current process as a baseline against which possible advances can be compared.
There are multiple activities associated with selecting and contracting an EDC vendor.
Specific project needs should be identified and a Request for Information (RFI) and/or Request for Proposal (RFP) should be sent to specific EDC vendors for consideration. An RFI is a document that an organization sends to prospective vendors to ask for specific information or clarification on services or products. Often the information obtained will be used to shorten the list of vendors or contractors from which proposals will be requested. A request for proposal, or RFP, is the document that an organization sends to prospective vendors to formally request a proposal and associated cost.
Both documents are normally used in the early stages of vendor selection, with the RFI typically sent earlier than the RFP. The proposals received in response to an RFP are often used as the basis for selecting the vendor outright or for creating a short-list of vendors from whom presentations will be requested. Using a structured RFP such as that in the Appendix helps to collect the desired information from vendors and in comparing them along factors and criteria important to the organization.
Often the vendors will have the opportunity to demonstrate the EDC functionality as well as the vendor capabilities. Key vendor personnel will be present. It is important to include key stakeholders in these presentations and to request demonstration of functionality required for your organization. Often organizations will rate proposals and functionality against a grid such as the example provided in the Appendix. Once all of the presentations have concluded, additional selection criteria may include past, ongoing, or future projects within the program, vendor performance history, vendor experience with previous industry studies, and references from current/former customers. See the Vendor Selection and Management Chapter of the GCDMP for more information on vendor selection processes.
The Statement of Work or Scope of Work (SOW) is often an attachment to the contract with a vendor. The SOW describes in detail the work to be performed, for example, implementation support or study set-up and execution, often with estimated timelines and milestones. The SOW is usually negotiated and the negotiations on the SOW or contract language may go for several rounds before the final version is signed by all parties. Upon finalization of the SOW, the sponsor project team may choose to kick-off the project or relationship with a meeting between the EDC vendor and sponsor or CRO teams to introduce team members, review SOW highlights, discuss vendor processes and templates, identify anticipated complexities and risks, and agree on the implementation timelines and a meeting schedule. See the Vendor Selection and Management Chapter of the GCDMP for more information on contracting and management processes.
In EDC studies the data collection method is much more than a piece of paper – it is an entire software application that includes workflow automation, decision support, and other study conduct aids designed to work as data are generated. Therefore the entire scope of validation activities should be completed up-front before a study starts. One of the biggest benefits of utilizing EDC is that the data are cleaned at entry and are available for review in real time. If the software is not fully validated or the study set-up is not complete when the first patient is enrolled, these advantages are lost. Refer to the GCDMP’s Chapter, “Electronic Data Capture – Implementation and Study Start-up” for discussion and evidence supporting this best practice.
When using an EDC application there are important considerations that the sponsor needs to consider regarding the validation of the chosen system. The EDC system itself needs to be validated
Whether developed by the sponsor or not, it is the responsibility of the sponsor to ensure that the system has been developed and is maintained per an acceptable Software Development Life Cycle (SDLC) process, including validation. Where a hosted vended system is used, to ensure the vendor’s system meets the requirements, an in-person or virtual audit of the vendor’s Quality Management System, including SDLC processes and documentation, is usually performed. Where software is installed locally, a vendor audit is usually conducted and is followed by installation and installation qualification and validation in the local environment. The validation process usually follows contract execution.
After a decision has been made to implement a new EDC system, organizational infrastructure for appropriate use of the system must be developed. A key recommendation is to separate infrastructure development from the set-up and conduct of a study.
EDC and most new technology offers an advance of some sort. The value added from new technology including EDC is realized when the new technology enables things that were not possible or were difficult prior to the technological advance. Mechanisms through which such benefits are realized include automation, connectivity, decision support, and generating knowledge from data, such as mining data to identify new trends or undetected problems.
Multiple people have articulated frameworks for thinking through organizational changes needed to support EDC implementation. Richardson categorizes the key implementation and support functions where change is to be expected with EDC adoption into: capable processes, human resources, qualified staff, and, implicit in his model, the technology itself.
Capable processes are required for study build and conduct tasks. Processes documentation describes the sequence of process steps, when they are performed, and the roles that perform them and supports EDC or any new technology to be used as intended. Processes interacting with EDC impact data collection and processing, and, as stated explicitly now in ICH E6(R2), are required to be documented. Process documentation, job aids, and training facilitates consistency across sites and staff. In addition, documented processes evidence that data collection and handling procedures were established a priori and not decided in an ad hoc manner that invites bias. Capable processes are those that have demonstrated desired and consistent behavior and quality. Process capability is established and maintained by measuring and monitoring key aspects of the process. This documented evidence demonstrates that all study functions leveraging or interacting with the software are operating as intended. Richardson refers to processes as validated processes.
Human resources includes understanding what types of professionals and support staff will be needed to run an EDC study and gaining the commitment that they will be dedicated for the required percentage of time and duration.
As described by Richardson, qualified staff means that staff are trained in the study procedures for the particular study and are prepared to execute that study.
Spink offers a similar framework that includes technology, logistics, process, and organization.
Though not explicitly stated in the frameworks, would-be technology adopters in organizations should assure that the technology adoption and associate process re-engineering is aligned with organizational goals and has the support of top leadership. Leadership should be briefed on the cost, benefit, and risks with new technology adoption so that decisions are well-informed by the risks and potential advantage with initiating new technology adoption and that the timing for organizational adoption is at a point in the technology-adoption curve that leadership supports. Informed decisions help assure enduring support through the challenges and set-backs inherent in new technology adoption.
Once the EDC system has been validated in the local environment or a vendor’s validation has been accepted for a hosted platform, after organizational processes around study set-up and conduct using the system have been documented and tested, after professionals and support staff have been allocated and are “EDC ready”, and after the study and site personnel have been trained in use of the EDC system for the study, then the study can proceed. At this point, the new technology is considered implemented. These aspects apply to set-up of a study within an EDC system, i.e., set-up can proceed after validation acceptance, process finalization, and allocation and training of professionals in their set-up tasks.
Even though pilots indicated benefit, information systems need to be evaluated in production to know if the anticipated gains have been achieved, to identify and assess any unintended consequences, and to identify opportunities for improvement.
This will occur prior to sites being trained on the study application. The second EDC chapter, Electronic Data Capture – Implementation and Study Start-up, picks up at this point – after EDC software is implemented with the appropriate surrounding infrastructure and ready for use on a clinical study.
In section 5.0.1, ICH E6 states, “During protocol development the Sponsor should identify processes and data that are critical to ensure human subject protection and the reliability of trial results.”
Vendor or EDC system selection
EDC System Setup, Installation and Support
Software Development Lifecycle including maintenance and change control
Vendor auditing and oversight
This revision is based on a systematic review of the peer-reviewed literature indexed for retrieval. The goals of this literature review were to (1) identify published research results and reports of EDC methods and evaluation and (2) identify, evaluate, and summarize evidence capable of informing the practice of selecting and implementing a web-based EDC system.
The following PubMed query was used:
The search query was customized for and executed on the following databases: PubMed (777 results); CINAHL (230 results); EMBASE (257 results); Science Citation Index/Web of Science (393 results); Association for Computing Machinery (ACM) Guide to the Computing Literature (115 results). A total of 1772 works were identified through the searches. The latest search was conducted on February 8, 2017. Search results were consolidated to obtain a list of 1368 distinct articles. Because this was the first review for this chapter, the searches were not restricted to any time range.
Two reviewers used inclusion criteria to screen all abstracts. Disagreements were adjudicated by the writing group. Forty-nine sources (mostly articles) meeting inclusion criteria were selected for review. The selected sources were read by the writing group and 109 additional sources identified through the review. Each of these 158 (49 + 109) sources was read for mention of explicit practice recommendations or research results informing practice. A total of 85 sources were deemed relevant to EDC and 73 were excluded by the full text review as not relevant to EDC. Of the 85 relevant sources, 53 were identified as informative for practice in one or more of the EDC GCDMP chapters and 32 were relevant but not informative of practice in any of the three EDC chapters. Sixteen articles provided evidence for this EDC chapter (
Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) Diagram for Web-based EDC – Selecting an EDC System.
Publication Date | Comments |
---|---|
September 2003 | Initial publication. |
May 2007 | Revised for style, grammar, and clarity. Substance of chapter content unchanged. |
September 2008 | Revised to reflect the orientation of chapter towards the concept and start-up phase of EDC. |
January 2021 | Content updated and organization of material revised to reflect orientation of the material to system selection. Study start-up material moved to separate chapter. |
The additional file for this article can be found as follows:
Sample Request for Proposal (RFP). DOI:
The authors have no competing interests to declare.
The chapter authors would like to thank Emily Patridge, Katrina M. Donovan, Jennifer Price, and Sean Berryman for their invaluable help with the literature review for this text.