1) Learning Objectives
After reading this chapter, the reader should understand
The purpose of and regulatory basis for vendor selection and management
Differences in outsourcing and contracting models
Processes for vendor selection
Contents and organization of a vendor scope of work (SOW)
Processes for vendor oversight
Governance models for strategic partnerships
2) Introduction
In the mid to late 1990s outsourcing in clinical studies, in particular clinical trials, saw significant increases. Today, vendors are used in all aspects of clinical studies and have particular relevance in clinical data management (CDM) processes. Examples of vendors relevant to CDM include contract research organizations (CROs), electronic patient reporting tool providers, clinical laboratories, specialty labs, central readers, imaging vendors, pharmacokinetics (PK) vendors, immunogenicity vendors, interactive web/voice response system (IxRS) providers, electronic data capture (EDC) and other software suppliers, and off-site storage and data hosting facilities. Before a vendor is selected, the deliverable or result desired from the vendor should be clearly defined and described.
Title 21 CFR Part 312 Responsibilities of Sponsors and Investigators requires official transfer obligations to a Contract Research Organization (CRO).1 Therefore, CROs should expect sponsors’ oversight and should be prepared to perform vendor oversight for responsibilities the CRO contracts to others. Regulation and guidance, such as ICH E6(R2) Section 5.0–5.2 are clear that “Ultimate responsibility for the quality and integrity of the trial data always resides with the sponsor.”2 Thus, sponsors must manage vendors and the vendors of vendors in a way that ensures quality, integrity, and reliability.
3) Scope
The scope of vendor services differs widely across the industry, ranging from protocol development to assistance with a regulatory submission. This chapter examines the communication of clear expectations between the vendor and the sponsor and some strategies for clearly documenting various areas of vendor oversight. The chapter also includes considerations for vendor qualification and the appropriate level of oversight needed, depending on the vendor’s scope of work and risks identified. Details and discussions regarding relationship management are beyond the scope of this chapter.
Disciplines such as operations engineering have developed methods for assuring quality of goods and services. These have been encoded into standards for Quality Management Systems through the International Organization for Standardization (ISO) and maturity models for such systems through the Software Engineering Institute (SEI). In project driven industries like therapeutic development, Quality Management Systems rely in part and heavily on project management to plan and manage work within and across organizations to successful conclusion.3 The Project Management Institute has developed and currently maintains the Project Management Body of Knowledge (PMBOK) and a professional certification program for project management. Project management fundamentals are not covered here. This chapter instead focuses on Data Management-relevant content for common project management tools. As described in ICH E6 section 5.0, these methods and practices together are foundational to managing the “quality throughout all stages of the trial process” and “ensuring human subject protection and the reliability of trial results.”2
Some of the tasks described in this chapter may be joint responsibilities among or performed by different groups. However, clinical data managers must be knowledgeable about performance of tasks relevant to data quality and integrity.
4) Minimum Standards
Both regulation and guidance address transfer of Sponsor obligations. In particular, the United States Code of Federal Regulations Title 21, subpart D, Part 312 states the following:1
Part 312.52 Transfer of obligations to a contract research organization states, “(a) A sponsor may transfer responsibility for any or all of the obligations set forth in this part to a contract research organization. Any such transfer shall be described in writing. If not all obligations are transferred, the writing is required to describe each of the obligations being assumed by the contract research organization. If all obligations are transferred, a general statement that all obligations have been transferred is acceptable. Any obligation not covered by the written description shall be deemed not to have been transferred.” And that, “(b) A contract research organization that assumes any obligation of a sponsor shall comply with the specific regulations in this chapter applicable to this obligation and shall be subject to the same regulatory action as a sponsor for failure to comply with any obligation assumed under these regulations. Thus, all references to “sponsor” in this part apply to a contract research organization to the extent that it assumes one or more obligations of the sponsor.”
The International Council for Harmonization E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R1) Guidance for Industry further elaborates on transfer of obligations.2
Section 5.0 states that, “The sponsor should implement a system to manage quality throughout all stages of the trial process.” Section 5.0.1 goes on to specify that, “During protocol development the Sponsor should identify processes and data that are critical to ensure human subject protection and the reliability of trial results.”
Section 5.0.2 describes risk identification, evaluation, control, communication, review, and reporting with section 5.0.2 stating that, “The sponsor should identify risks to critical trial processes and data. Risks should be considered at both the system level (e.g., standard operating procedures, computerized systems, personnel) and clinical trial level (e.g., trial design, data collection, informed consent process).”
Section 5.2.1 echoes 21CFR312.52 stating that, “A sponsor may transfer any or all of the sponsor’s trial-related duties and functions to a CRO” and further adds that, “the ultimate responsibility for the quality and integrity of the trial data always resides with the sponsor” and that, “The CRO should implement quality assurance and control.”
Section 5.2.2 specifies that, “Any trial-related duty and function that is transferred to and assumed by a CRO should be specified in writing.”
The addendum in section 5.2 states that, “The sponsor should ensure oversight of any trial-related duties and functions carried out on its behalf, including trial-related duties and functions that are subcontracted to another party by the sponsor’s contracted CRO(s).” With section 5.2.3 stating that, “Any trial-related duties and functions not specifically transferred to and assumed by a CRO are retained by the sponsor.” Similarly, ICH E6(R2), section 5.2.4 echoes 21, subpart D, Part 312(b), stating, “All references to a sponsor in this guidance also apply to a CRO to the extent that a CRO has assumed the trial-related duties and functions of a sponsor.”2
Thus, with respect to selection and management of vendors, we state minimum standards as indicated in Table 1.
Minimum Standards.
| 1 | Sponsors should assess a vendor’s Quality Management System and deem it appropriate prior to receiving goods or services toward a clinical study. |
| 2 | Decisions made in the course of such an assessment may be risk-based. |
| 3 | Sponsors should assess potential impact of contracted work on human subject protection and reliability of trial results. |
| 4 | Sponsor responsibilities delegated to a vendor should be documented in writing. |
| 5 | Sponsors should establish a vendor-auditing program including plans to and criteria for re-audit. |
| 6 | Sponsors who contract goods or services toward a clinical study should provide adequate oversight. |
5) Best Practices
Best practices were identified by both the review and the writing group and are presented in Table 2. Best practices do not have a strong requirement based in regulation or recommended approach based in guidance, but do have supporting evidence either from the literature or consensus of the writing group. As such best practices, like all assertions in GCDMP chapters, have a literature citation where available and are always tagged with a roman numeral indicating the strength of evidence supporting the recommendation. GCDMP Levels of Evidence are outlined in Table 3.
Best Practices.
| 1 | Obtain a confidentiality agreement with the vendor prior to exchange of proprietary information. [VI] |
| 2 | Document the sponsor’s process and support functions needed to evaluate the use of vendor services. [VI] |
| 3 | Evaluate and qualify, e.g., by capacity, capability, qualifications, experience, regulatory compliance, company stability, vendors prior to contracting for their services or products. [VI] |
| 4 | Create a contacts list that is centrally accessible to study team members. [VI] |
| 5 | Determine and document whether the sponsor’s, the vendor’s, or a combination thereof standard operating procedures (SOPs) are to be followed. [VI] |
| 6 | Clearly define expectations, deliverables, and responsibilities.3 [III] Both the sponsor and the vendor must participate in defining expectations, deliverables, and responsibilities. [VI] |
| 7 | Conduct ongoing management of vendor activities by communicating and assessing the vendor’s performance throughout the study.3 [III] |
| 8 | Review data transfer agreement for all the third party vendors. [VI] |
| 9 | Where feasible, evaluate from a CDM perspective the risk of utilizing or not utilizing vendor services related to the conduct and outcome of the study. [VI] |
| 10 | Maintain an internally approved vendor list (Template available in Appendix A) with regular evaluations (e.g., preferred vendor list or prequalified vendor list). This may be risk-based. [VI] |
| 11 | The vendor-auditing program should be cross-functional based on contracted services. [VI] |
| 12 | Processes for vendor evaluation, vendor audits, issue resolution, and escalation should be informed by cross-functional subject matter experts within a centralized organizational team. Use a subject matter consultant if your organization lacks this expertise in-house. [VI] |
| 13 | Define and document a detailed statement of work and project plans that delineate each task, role or person responsible, task timing and dependencies, related documentation, role or person responsible for reviewing and approving related documentation or other task results, and reporting related to the task. [VI] |
| 14 | Define and document detailed sponsor/vendor communication plans that clearly address the expected communication tools and frequency, as well as establish who is responsible for communications and how to escalate issues when deemed necessary. [VI] |
| 15 | In high risk situations, identify other possible vendors or options as part of a contingency plan in case the vendor relationship is deemed unsatisfactory at any point during the course of the study. [VI] |
| 16 | Establish a collaborative relationship based on partnership, trust, and coownership of the project. [VI] |
| 17 | Hold frequent one-on-one meetings or teleconferences with the vendor lead to share concerns, provide mutual feedback, plan for success, and ensure activities are on track without any red flags. [VI] |
| 18 | If the vendor is providing services that involve computerized systems, ensure system support documentation is in place. For example, establish a service level agreement (SLA), that describes in detail how much time it will take the vendor to respond to support inquiries, how long it will take to get a database back online in case of a system failure, and other details related to supporting the sponsor’s business requirements. [VI] |
| 19 | Study teams should engage early to begin a study-level sourcing strategy and vendor identification. [VI] |
| 20 | SLAs should be defined for vendors with whom your organization works frequently. [VI] |
Grading Criteria.
| Evidence Level | Criteria |
|---|---|
| I | Large controlled experiments, meta, or pooled analysis of controlled experiments, regulation or regulatory guidance |
| II | Small controlled experiments with unclear results |
| III | Reviews or synthesis of the empirical literature |
| IV | Observational studies with a comparison group |
| V | Observational studies including demonstration projects and case studies with no control |
| VI | Consensus of the writing group including GCDMP Executive Committee and public comment process |
| VII | Opinion papers |
6) Types of Vendors and Vendor Services Commonly Used in CDM
Required vendor services will vary from study to study depending on the needs of the study and resources already available within the organization. Below are some of the key types of vendors often utilized during the course of a clinical study.
a) Contract Research Organization (CRO)
Over the past several decades, CROs have gradually evolved from organizations providing limited services in clinical trial management into organizations that have expertise across a wide spectrum of the clinical development process. Today, CROs provide a broad spectrum of services, from full service contracts to only data management services.
A full service contract would include a wide range of services from a single CRO, such as study management, site monitoring, data management, biostatistics, or medical writing. The Sponsor determines the services needed and CROs respond with a plan for meeting those requirements. Multiple CROs might propose different services for their participation in a study. For example, if the contract is for data management only, some CROs might choose to conduct all aspects of data management in-house while others might not. Some types of services included by CROs providing data management services include:
Data Management (DM) project management
Development of electronic case report form (eCRF) specification based on protocol needs
Development of paper case report form for additional data collection if required by the protocol. For example, adjudication committee, medical review by a third party specialist, etc.
Creation of Case report form (CRF) completion guidelines
Creation of the data management plan (DMP) or equivalent documentation
Database design and programming
Database validation and testing
Edit check specification development and programming
Data entry, review, coding, and cleaning
Third party vendor management including vendors such as laboratory and eDiary
Serious adverse event (SAE) reporting and reconciliation
External data transfer and integration
Quality control audits
Database Lock
b) Third Party Vendors
Third-party vendors are subcontractors who are independent from the customer and supplier. They are individuals or organizations hired to perform services in obligation to and as a separate entity from the supplier. Common examples are the third party vendors that a CRO contracts for additional services such as interactive voice/web response system (IXRS), electronic diary (eDiary) for home visit/diaries, sites, central lab, specialty lab, material printing, face to face meeting organizers, monitors, patient travels, home visit nurse, PK/PD, immunogenicity, imaging, coding, and translations services. Typically, these vendors are selected shortly after the decision to conduct the study. Services to be subcontracted for a study should be decided prior to protocol finalization and before the first patient in (FPI). Different vendors may be desired for different functions, and preferred vendors may differ from organization to organization.
c) Independent consultants/contractors
Independent consultants (or contractors) are individuals who often do not have their own infrastructure or work for themselves. Any role on a study can be filled by an independent consultant. Independent consultants could be hired by the sponsor, a vendor, or a staffing firm working for a Sponsor or vendor and are usually hired for a specific project or program and for a short duration.
7) Contracting Business Models
The business model followed by a vendor can significantly affect the relationship between the vendor and the sponsor. The following are some of the more frequently encountered business models that could affect clinical data management.
a) Transactional Model
The transactional model, in which a sponsor contracts vendors on a per-project or per-study basis, could be considered the traditional outsourcing model for clinical studies. The payment could be based on per unit (e.g., per hour or per unique CRF) or it could be fixed cost per project based on the nature of the contract. Transactional relationships may be more likely than other models to perform out-of-scope activities, resulting in cost overrun.
b) Strategic Partnerships
Strategic partnerships are usually formed between a sponsor and a vendor with complementary resources and expertise. Strategic partnerships could be between a sponsor and a company providing EDC services or other electronic tools for clinical studies, or may be between a sponsor and a full-service CRO. Strategic partnerships may also be formed to gain location-specific resources needed for studies that span multiple countries or regions.
Before forming a strategic partnership, the sponsor should carefully evaluate the potential partner. Ensure there are no significant differences between corporate cultures, philosophies, or SOPs among sponsor, the strategic partner vendor, and other vendors involved that could potentially lead to conflicts. Ensure that any identified issues can be rectified to the satisfaction of all parties involved prior to the initiation of the stated contract.
Some of the main reasons that sponsors opt for strategic partnerships are reduced cost, improved efficiency in the use of internal staff, access to operational expertise, improved quality, reduced contracting effort, reduced effort for provider selection, process improvement, access to therapeutic expertise, access to experienced staff members, and access to innovation and technology. While the benefits of a strategic partnership are many, these relationships require investments of time and resources from both parties to maximize the outcome.4
c) Functional Service Provider (FSP) Models
In contrast to outsourcing all data management aspects of a study to a single CRO, an FSP model may involve outsourcing only select activities. “Because project ownership remains in-house, companies that use functional outsourcing may experience higher levels of quality control yet have access to specific services at a lower overall cost. Sponsor companies benefit from being able to ramp up and draw down resources relative to their development activity levels without affecting their internal head count.”5 Using an FSP model allows sponsors to focus on their core competencies and outsource certain activities (such as CRF design or system validation) to niche vendors, rather than needing to hire additional personnel or provide additional training to existing personnel. An example of an FSP is a sponsor that hires a company to deliver 10 statistical programmers for the following year.
d) Application Service Provider (ASP) Models
An ASP is a vendor that leases a software application to clients, and can involve contracts that are for the duration of a study, for a set amount of time, or on a per-use basis, such as per user, per study, per CRF, etc. Using an ASP shifts much of the responsibility to the vendor for implementing, hosting, validating, maintaining, upgrading, and supporting the software. However, because sponsors are ultimately responsible for data integrity and quality, a risk-based approach should be used to determine the scope and depth of any additional software testing and validation. Examples include EDC or electronic Patient Reported Outcomes (ePRO) vendors. Some sponsors may oversee and manage these vendors directly while others may delegate this responsibility to the CRO.
8) Vendor Qualification, Initial Evaluation and Selection
There is no “set in stone process” for vendor evaluation and selection; but what follows are common and generally accepted processes. This approach may be modified depending on whether the service provider is new or has past experience with the sponsor. There also may be further modification based on the sponsor’s experience with the vendor and with the service to be provided.
Example process for vendor evaluation and selection.
a) Request for Information (RFI)
The objective of RFI is to discuss the proposed strategy for vendor selection and gather relevant information ahead of the resource evaluation and planning processes. It is a best practice that study teams engage in early to begin a study-level sourcing strategy and vendor identification. [VI] Based on the design of the study under consideration, study specific needs and possible vendors can be identified. A template RFI that includes specific services required for the study or program is a best practice. [VI] A sample RFI template is available in Appendix B
Some of the general contents of the RFI include:
Company information including history, financial stability trend, past or planned merger and acquisitions, and organizational structure
Indication that the potential vendor provides products and services relevant to the project
Number and types of sponsors or similar studies supported
Number of qualified personnel in key roles
Experience and expertise of current staff relevant to the project in the therapeutic area, study phase, or type of study, etc.
The vendor’s geographic capabilities
The number of sponsors or studies currently supported by the available vendor staff
Capacity to take on the work to be contracted within the needed start-up time and for the project duration
Indication of services for which third-party vendors will be used
Availability for a pre-award survey/Quality Management System audit in the timeframe needed to cover the items indicated below
Indication of required accreditation in the vendor’s field of work (e.g., lab certifications)
Indication of vendor’s ability within the vendor’s QMS to adapt to sponsor’s SOPs if required
Indication of information system validation for regulated processes
Indication of information system change control processes
Indication that vendor can meet Service Level Agreement (SLA) requirements for the project
Indication of required physical and logical security practices; e.g., controlled facility access, server rooms, file rooms, information system authentication and role-based security, independent backup procedures, secure data transfer processes
Indication of disaster/contingency plan(s) to protect business operations
Indication of a vendor audit program
Description of vendor’s experience with relevant business models
Description of the vendor’s quality management system, including dates and scopes of ten most recent internal QMS audits
Dates and outcomes of previous regulatory inspections, as permitted
Description of the vendor’s process for identifying and managing changes in scope
Description of the vendor’s process for projecting, assigning, and managing project resources including the organization’s succession planning process for project team members
Indication that vendor can meet any other project requirements such as an audit schedule, performance indicators, availability of documentation to regulators, etc.
References from previous customers
b) Cross-Functional Team Discussion
After obtaining information on different vendors, appropriate cross-functional teams meet to review vendor’s information to ensure that the services offered meet the needs of the upcoming clinical study. Often at this stage a short list of the most attractive vendors is developed. If all the functions are satisfied with the information, sponsor requests the short-listed vendors for a proposal.
c) Request for Proposal (RFP)
A request for proposal (RFP) is a document shared by a company interested in procurement of a service to potential suppliers requesting submission of business proposals. Typically, the sponsor maintains a template for the RFP and bid grid that is sent to the interested vendors with specific study information such as the protocol or the protocol synopsis.
A bid grid is a tool provided by a purchaser (i.e., a Sponsor or delegate) to potential vendors such as CROs usually along with an RFP. The bid grid categorizes and standardizes the services or products being bid so that bids can be compared. The categorization often informs contracting and expense reconciliation throughout the contract. Most large pharmaceutical companies use bid grids.6 A bid grid may have added columns indicating responsibility assignment or task ownership. In such cases, the bid grid may be referred to as a roles and responsibilities (R&R) matrix. Typically, the bid grid is maintained by a procurement or vendor management office, though this may vary among organizations.7
A bid grid serves two primary purposes:
A bid grid captures the sponsor’s predefined study-specific cost drivers
A bid grid allows the outsourced partner to assign prices to specific tasks associated with cost drivers
For all CDM cost drivers, a bid grid should include definitions of units, cost per unit, the estimated number of units expected, and total anticipated costs for each row. The structure of a bid grid should cover the scope of work and all tasks and units should be clearly defined, meaningful, and measurable. In addition to specific CDM cost drivers, a bid grid may also aggregate costs into high-level categories, such as all costs associated with an investigator meeting or data cleaning.
Some high-level categories for pricing consideration include
CRF/eCRF design
Database development (including edit check specifications)
Data management plan development
Data cleaning
Management of local lab reference ranges
Dictionary coding and up-versioning
Management of external data
SAE reconciliation
Quality control audit(s)
Data transfers
Database finalization/lock
Some examples of CDM cost drivers include
Number of unique CRFs (paper or electronic)
Number of total CRF pages (paper or electronic)
Number of edit checks to be programmed
Number of subjects to be enrolled
Number of cleanup listings
Number of external data sources (e.g., central labs, electronic diaries, etc.)
Number of local labs
Number of queries expected
Number of terms to be encoded
Number of SAEs to be reconciled
Number of data review rounds
Number and types of data transfers/integrations
Number of unique status reports
Frequency of status reports
Frequency of teleconferences
Number of interim database locks
Number of Patients
Patient Profiles reviews if utilized
d) Evaluation of the RFP and Bid grid
Once the vendor receives and completes the request, the bid will be submitted. Bid defense, a face-to-face or teleconference meeting where vendor presents the contents of the proposal, is usually a part of this process. However, strategic partnership or functional service provider relationships do not usually require a bid grid and bid defense for each project.
Once the completed RFP/Bid grid is received, it is circulated to an appropriate cross-functional team for function specific review. Senior personnel from each function usually conduct the review. This functional assessment may include questions in a bid defense, review of documentation provided by a prospective vendor such as the vendor quality manual, SOPs, key team member qualifications, or example project plans. Some assessments require a visit to vendor’s location. On-site assessments, also called pre-award surveys or pre-award audits may include the following:
Review of the vendor’s SOPs and work instructions to ensure soundness of processes and proof of regulatory and industry standards compliance
Confirmation of QMS audits indicated in the RFI
Evaluation of the vendor’s QC/QA processes
Personnel qualification (through a review of, for example, curriculum vitae (CVs) of company personnel, job descriptions, organizational charts, training plans and documentation, etc.)
Evidence of clearly defined project-specific training plans for new team members, and adequate transition processes to address staffing changes that occur during a study
Sufficient staffing, including documented adherence to training and retraining plans
Security of physical locations where services are provided (controlled facility access, security of server rooms and file rooms, independent backup procedures, etc.)
Physical conditions of server and file rooms (limited access, fireproof, controlled temperature and humidity, etc.)
Disaster/contingency plan(s) to protect business operations
Evaluation of subcontractors and the vendor’s management processes for those subcontractors, if applicable
After relevant cost drivers have been shared with the vendor, the sponsor and vendor should discuss variables that could affect pricing prior to the vendor completing the bid. This discussion should include determination of which organization’s SOPs will be followed. If the sponsor’s SOPs are to be followed, the sponsor will determine training requirements for the vendor. Both parties should also consider which systems would be used and if any standards or efficiencies can be applied to the project(s). During this phase of the relationship, clear expectations should be agreed upon and documented. Expectations to be discussed and documented should include the following:
Communications (project status updates, escalation path, etc.)
Quality (documents and data)
Timelines and turnaround times
Final deliverables
When working with a CRO, the final bid grid should be shared with the CRO parties in charge of managing the study and study deliverables. Both parties (sponsor and CRO) should review each task on the bid grid, line by line, to confirm understanding of the task and confirm the responsibility and accountability (responsible party, approving party, etc.) Each task should be explained to the CRO in sufficient detail prior to completion of the bid so that both parties fully understand what is to be included and priced. See Appendix C for an abbreviated sample bid grid.
9) Approval
At this stage, the RFP is finalized and work is awarded to vendors. Each applicable study team member will review the budget and approve the scope of work generated by the vendor. Once the official contract is signed, the vendor can begin work towards the discussed study.
A Statement of Work (SOW) is comprised of legal language, which will define the high-level services to be provided, deliverables and timeline for services being performed, the scope with all the study specific details on services performed by the vendor, cost per service item, and the signature of both parties
10) Development of Contract and Scope of Work (SOW)
Once potential vendors have been evaluated and vendor selections have been made, a Contract as well as a Statement of the Scope of Work must be prepared and agreed upon by the sponsor and the vendor. Many large companies have separate departments that handle these details, but CDM personnel may be involved with these processes in some organizations.
a) Considerations for Sponsors, Vendors, and Data Managers
The type of outsourcing business model to be used is a very important consideration in preparing the contract and the scope of work. Because numerous variations can exist between outsourcing relationships even when following the same outsourcing business model, the contract and the scope of work for each vendor relationship may also have unique variations.
When using models that involve more organizational integration, such as strategic partnerships or an FSP relationship, both organizations should commit to several levels of oversight (executive committees, operational committees, etc.) that focus on strategy and implementation to ensure that the partnership is successful. Each level of oversight should also be associated with a clear escalation path in case issues are not able to be resolved at a particular level. Governance models should ensure long-term senior management commitment from both sides.
For organizations using a transactional outsourcing business model, costs and scope of work are typically based on certain assumptions. Because some of these assumptions may be incorrect or based on changing information, the contract and scope of work should include provisions detailing how changes will be handled. These provisions should include a description of how changes to underlying assumptions may result in change orders, as well as mitigation plans to resolve situations where the scope of work evolves slowly over time (i.e., scope creep).
Although typically the responsibility of a legal department, CDM personnel should be aware that contracts may include special clauses such as penalty clauses or bonus clauses. These clauses are intended to give vendors incentives for exceeding expectations, or disincentives for not meeting expectations.
b) Common Components of a Contract
Outsourcing relationships frequently start with a Master Services Agreement (MSA). The MSA outlines the overarching agreement between the signing parties (e.g., Vendor and Sponsor). Its purpose is to simplify future contracts. The MSA may contain details on payment terms, indemnification, confidentiality, delivery requirements, intellectual property rights, dispute resolutions, limitations, and work standards. In some cases, as is often the case in a Strategic Partnership, a rate card or bid grid may be a part of the MSA. Specific Scopes of Work (SOW) fall under the MSA. In most cases, this is where CDM personnel will begin their involvement as MSAs are often negotiated by a legal team or executive staff.
c) Scope of Work
In many cases the SOW looks similar to the Transfer of Regulatory Obligations (TORO) as it details which party has responsibility for an activity; however, the SOW should contain additional details for deliverables describing how the work will be done; resources needed; assumptions made by the vendor; and costs for each deliverable. Costs may be detailed in the SOW or in an attached Bid Grid.
It is important to review the SOW from multiple perspectives in order to make sure it is complete. Changes to any aspect of the SOW usually result in a Change-In-Scope (CIS). A CIS can take time to negotiate and can stop the progress of deliverables.
d) Task Ownership Matrix
A task ownership matrix identifies all tasks that may arise during execution of a clinical study. It often goes a step beyond the TORO or SOW to include project team roles. The matrix is intended to ensure all tasks are accounted for and to reduce the potential for duplication of effort. Failure to develop a task ownership matrix, or developing one poorly, can defeat the anticipated benefits that drove the parties to enter into an agreement in the first place. For example, if both parties duplicate efforts with a task because responsibility for the task is not clearly defined, duplicate costs are incurred and the desired monetary savings of the relationship may never be realized. The matrix should clearly identify the following four ownership responsibilities that occur with any task or document:
Who is responsible for this task or document (e.g., creation, revision, approval)
Who is accountable for this task or document (e.g., the single individual held accountable for the decision or task)
Who is consulted for this task or document
Who is informed for this task or document
The end result of a well-documented task ownership matrix, also known as a RACI (responsible, accountable, consulted, informed) table, will be a better relationship between the sponsor and vendor, as well as provide clear one-party accountability for success or remediation of various tasks. Both parties should mutually agree upon the task ownership matrix prior to study startup. A sample RACI Chart Template is provided in the chapter entitled Project Management for the Clinical Data Manager Appendix C.
In addition to the bid grid, it is advised to consider both writing acceptance criteria and service level agreements within the contract. An example of acceptance criteria may mean a database with no open queries for a subset of subjects at a particular study milestone. Service level agreements are time-based agreed upon expectations for a service. An example of a service level agreement would be that all calls to the helpdesk would be returned within 24 hours.
11) Vendor Oversight
Organizational vendor oversight and management is a key part of an overall Quality Management System. How much effort your organization puts into a vendor oversight and management program will vary depending on your organization, the extent to which your organization outsources to vendors, the risk of the activity they are outsourcing, and the volume of work outsourced to a single vendor. This topic is also discussed in the Assuring Data Quality Chapter of the GCDMP.
It is important for your organization to have SOPs that address vendor selection and management. Your organization’s Quality Assurance Group likely has this. If not, it is recommended that you either create one or hire a qualified quality assurance or compliance company to do so.
An example of a risk based vendor oversight and management program is described in Table 4 below.
Example of risk-based vendor oversight.
| Vendor Name | Experience with Vendor | Vendor Activity | Vendor Volume | Oversight |
|---|---|---|---|---|
| A+ CRO | New | Entire Clinical Trial | 1 Phase 2 US trial | RFP, Bid defense, Vendor qualification by third party |
| Labs-R-Us | Used vendor for 4 studies over past 3 years with good results | All clinical labs for Phase 1–2 studies | Approx. 2 studies/year | Vendor audit by internal QA every other year |
a) Managing a vendor within the scope of a contract
Your organization’s vendor governance program should be risk-based, meaning new vendors for a Phase 3 study that contribute data to the primary endpoint will most likely require more oversight than a frequent vendor partner that contributes exploratory data for a post-marketing study. According to ICH E6(R2), the project team should conduct the steps listed in the Quality Management Program beginning with Critical Process and Data Identification and moving through the subsequent steps in order.2 These are the following: Risk Identification, Risk Evaluation, Risk Control, Risk Communication, Risk Review and Risk Reporting. A detailed explanation for each of these steps can be found in the ICH E6(R2): Guideline for Good Clinical Practice.2
b) Governance documents that may be used
Several helpful documents may be used. Some common governance documents are:
Service Level Agreement
A Service Level Agreement, SLA, may be outlined in the MSA, SOW, or a separate Vendor Governance Plan. It is commonly found in strategic partnership governance plans. It is considered best practice to have SLAs defined for vendors with whom your organization works frequently. An example of a CDM SLA is a mutually agreed upon time from last query closed to database lock.
Timelines, task, and deliverables list
The clinical project manager will likely maintain a project timeline for high-level deliverables such as First Patient First Visit (FPFV), site initiation dates, database lock, etc.; however, there are many granular tasks and deliverables that must be tracked that support these higher-level milestones. CDM staff may be asked to track vendor deliverables such as data transfers, query resolution, data entered from data of subject visit, etc. This is frequently referred to as a Deliverables List or they may be sub-items in overall Project Management Plan using software such as Microsoft Project or an equivalent document.
Resource Management Plan
A Resource Management plan will help you describe how many staff and at what percent FTE each staff member is needed and for the duration that a deliverable must be met. Such plans are often found in project management software or one can be developed using Excel. An example Resource Management Plan to complete the first draft DMP is Table 5 below.
Example first draft DMP Resource Management Plan.
| Task | Who | Estimated Hours | Target Date |
|---|---|---|---|
| Draft DMP | Lead CDM | 8.0 | 01Dec2018 |
| Internal QC DMP | Director, CDM | 3.0 | 03Dec2018 |
| Incorporate comments and send to sponsor | Lead CDM | 4.0 | 05Dec2018 |
Key Performance Indicators
Key Performance Indicators (KPIs) are another frequently used tool to gauge the health of a partnership. They are often collaboratively developed between parties. An example of a KPI that affects vendor oversight is “Rate of Key Staff Turnover.” KPIs may be categorized by study stage such as Study Startup, Study Conduct, and Study Closeout since each stage has its own set of risks. This list should be modified as best fits your study, clinical program, or vendor partnership. Some example KPIs, listed in Table 6 below, are from the 2012 SCDM Annual Conference presentation entitled “Outsourcing DM: How to Get the Most Value out of a Partnership.”8 You will want to fine tune KPIs to fit your organization and clinical studies.
Examples of Key Performance Indicators.
| Service | Service Level Description | Service Level Measurement | Minimum Service Level Expected by Sponsor | Target Service Level |
|---|---|---|---|---|
| Listing Review | All listings reviewed and queries generated | % of listings reviewed within 2 weeks of schedule | 80% | 95% |
| Queries Handling | Queries correctly generated | % of errors | 90% | 95% |
Performance metrics such as SLAs and KPIs should be reviewed regularly with the vendor on a mutually agreed upon basis. An honest and collaborative review will provide insight for ways both parties can improve the overall effectiveness of their relationship.
Communication Plan
Appendix B in the chapter Project Management for the Clinical Data Manager provides an example Communication Plan Template. In addition to this plan, it is recommended that data management have regular biweekly (once every two weeks) or weekly meetings with a set agenda followed by meeting minutes to discuss upcoming milestones, plan resources, set priorities, and to build a respectful, collaborative relationship. Such meetings ensure risks to timeline or quality are identified early when such problems are more easily solved. In addition to Data Management meetings, one-to-one Sponsor/CRO meetings are very beneficial to plan and keep focus on the Data Management agenda.
The schedule for regular meetings should be specific to the protocol. For example, a study that is expected to be slow enrolling may have weekly data management meetings to oversee study start-up activities then reduce the schedule to monthly during the maintenance phase. It is recommended that status updates and study metrics be reviewed at these meetings.
Occasionally, unfortunate events such as missed deadlines or a high degree of errors requires escalation. According to the Glossary of Communication Terminology escalation is “The process which details how conflicts and issues will be passed up the management chain for resolution as well as the timeframe to achieve resolution.”9 It is recommended that an Escalation Plan be developed for any vendor relationship where there is a significant amount of work (i.e., Strategic Partnerships; Functional Service Provider relationships). Development and approval of the plan early in the relationship is important before any escalation issues need to be addressed. An escalation plan should be based on impact of the issue at hand with the goal of addressing low impact issues within the first level of escalation. The sample Communication Escalation Process in Table 7 below was obtained via www.ProjectManagementDocs.com, within the Communications Management Plan Template.
Sample Communication Escalation Process.
| Priority | Definition | Decision Authority | Timeframe for Resolution |
|---|---|---|---|
| Priority 1 | Major impact to project or business operations. If not resolved quickly there will be a significant adverse impact to revenue and/or schedule. | Vice President or higher | Within 4 hours |
| Priority 2 | Medium impact to project or business operations that may result in some adverse impact to revenue and/or schedule. | Project Sponsor | Within one business day |
| Priority 3 | Slight impact that may cause some minor scheduling difficulties with the project but no impact to business operations or revenue. | Project Manager | Within two business days |
| Priority 4 | Insignificant impact to project but there may be a better solution. | Project Manager | Work continues and any recommendations are submitted via the project change control process |
Unfortunately, there are times when a decision is made to terminate a vendor relationship. Again, it is recommended to develop and approve an exit plan for any vendor relationship where there is a significant amount of work at the start of the relationship. Doing so will mitigate risks.
Detailing the criteria for terminating a relationship is best outlined at the beginning of the relationship so that there is a mutual understanding.
The process for termination of work is usually detailed in the Master Services Agreement (MSA) between the vendor and sponsor. Clinical Data Managers should familiarize themselves with this section of the MSA; however, the exit plan duties for closing out the scope of work may fall to the data manager. In that event, an example Exit Plan for data management activities using the RACI model is suggested below, in Table 8.
Example Exit Plan using RACI.
| Task | Responsible | Accountable | Consulted | Informed |
|---|---|---|---|---|
| Communication to StakeholdersVendorProject LeadProject Team members | Project Manager | VP or Executive | DM Lead | Study Team members |
| Acknowledgement of Receipt for Stop Work | Project Manager | VP or Executive | DM Lead | Study Team members |
| Financial reconciliation of Scope of Work | DM Lead (for DM activities) | Project Manager | DM Study Team Members | DM Study Team Members |
| Archival of Work Completed | DM Lead (for DM activities) | Project Manager | DM Study Team Members | DM Study Team Members |
c) What to do when you have to oversee a vendor you did not select?
Ideally, the data manager will be involved in the development of the SOW; however, many times CDM staff is not involved in the development of the SOW but often ends up with the responsibility of managing the vendors who contribute to the development of the clinical database. If this is your situation, the first thing to do is to thoroughly review and understand the SOW. If you notice any aspect of the SOW that could lead to miscommunication, missed timelines, or incorrect assumptions, it is best to alert the project manager and senior management within data management as soon as possible. Dealing with potential problems sooner rather than later is usually less expensive and less of a headache.
d) Inheriting vendor oversight mid-stream in a project
Sometimes a data manager will inherit vendor oversight in the middle of a project. This may happen for any number of reasons. The primary project manager may realize that vendor oversight needs more care and attention to ensure the integrity of the trial’s data and safety of the subjects, or perhaps the previous data manager was reassigned or left the organization. In any case, the steps to get up to speed are the same and are outlined below:
Read the contract with the vendor. Know the assumptions that drive the vendor’s budget and the components for each deliverable.
Familiarize yourself with the protocol and any associated study plans where the vendor plays a part.
Do not assume that because vendor oversight was not originally scoped in study plans that it is not needed. Decide for yourself using your own quick assessment of risk identification and assessment taken from the ICH E6(R2) guideline.2
Make a plan and communicate your plan to the rest of the study team. What you do from there will depend on project team decisions.
Whatever the outcome of the project team, responsibility for vendor oversight should not be considered as an afterthought. If an activity is important enough to outsource, it is important enough to make sure it is done correctly.
12) Recommended Standard Operating Procedures
Vendor Qualification
Vendor Oversight
Appendices
Vendor List Template.
| Vendor Name | Vendor Address | Vendor Contact Name | Vendor Title | Vendor Phone Number | Vendor Email | Services Provided Previously Provided | Services Approved for use by QA | Date of Last Qualifi-cation Audit | Auditor for Last Qualifi-cation Audit | Date of Requalifi-cation Needed |
|---|---|---|---|---|---|---|---|---|---|---|
| Just Right Labs | xxxx | xxxx | xxxx | xxxx | xxxx | Clinical Lab Services | Clinical Lab Services; PK Lab Services | 15-Feb-18 | Internal QA | 15-Feb-20 |
Sample Request for Information (RFI).
| Company Information | |
|---|---|
| 1. | Provide a brief description of the company’s history, including length of tune in the industry, origins of the company, mission statement and vision. |
| 2. | Provide an organizational chart. Include position and number of employees in each department (senior management, technical support, user support, technical and client service managers, sales and marketing, development, recruiting, quality assurance, training, etc.). |
| 3. | Describe quality assurance processes and roles. Is the quality assurance organization independent of the operational organization? |
| 4. | Describe the current level of company funding. |
| 5. | Describe the company’s pricing model. |
| 6. | Describe the quality management system adopted by the company. |
| 7. | Describe the validation/change control processes of the computerized systems. |
| 8. | Describe results of prior audits. |
| 9. | Describe quality oversight on contractors (if applicable). |
| 10. | If applicable, provide the results of previous regulatory inspections. |
| Products/Services | |
| 1. | Describe the evolution of your product or service. |
| 2. | How many clients are currently using your product or service? |
| 3. | Describe your user support services (IT, helpdesk, IVRS, etc.). |
| 4. | Describe the company’s interpretation of 21 CFR 11 and how your product is in compliance with this regulation. |
| 5. | Can your company produce Clinical Data Interchange Standards Consortium (CDISC) compliant data? If so, which model or models? |
| 6. | Describe the company’s involvement and specific recommendations for user training. Differentiate between clinical studies with a few sites and those with a large number of sites, if appropriate. |
| 7. | What other products or services do you offer? |
| Experience | |
| 1. | How many studies has your company supported in the past years? |
| 2. | What is the largest clinical study completed to date with respect to number of sites, number of subjects? What lessons did you learn? |
| 3. | What are some of the qualities of your company from a human resource perspective? (e.g., What is your rate of turnover? What percentage of your employees are contract versus permanent? What are your training procedures?) |
| 3. | What user feedback have you solicited or received from study site personnel or clients about your product or services? How was the feedback addressed? |
| 4. | Provide references. |
| 5. | Provide CVs and training plans for the proposed personnel. |
Sample Bid Grid.
| CRO SERVICES | Unit | Cost/Unit | Estimated Number of Units | Item Cost |
|---|---|---|---|---|
| DATA MANAGEMENT | ||||
| Project Management | Month | $0.00 | ||
| CRF Creation | Per Unique Page | $0.00 | ||
| CRF Guidelines | Per Unique Page | $0.00 | ||
| Create Data Management Plan | Plan | $0.00 | ||
| Design Database | Per Unique Page | $0.00 | ||
| Program Derived Fields | Per Unique Page | $0.00 | ||
| Program Data Edit Specifications (to include number of edit checks to be developed) | Per Edit check/Per Unique Page | $0.00 | ||
| Query Rate (queries X pages X subjects | Page | $0.00 | ||
| Line Listing Review (for Safety, Sponsor, etc) | Listing | $0.00 | ||
| Data Management Review | Page | $0.00 | ||
| Data Coding | Code | $0.00 | ||
| Provide Coding Dictionaries | Dictionary | $0.00 | ||
| SAE Reconciliation | SAE | $0.00 | ||
| Lab Normal Maintenance | Lab Site | $0.00 | ||
| External Data Loads | Load | $0.00 | ||
| QC Audit | Page | $0.00 | ||
| Database Lock | Lock | $0.00 | ||
| Database Transfer(s) | Transfer | $0.00 | ||
| TOTAL | $0.00 | |||
13) Literature Review details and References
This chapter is based predominately on the Project Management Body of Knowledge (PMBOK), which is an authoritative source. For this reason a full systematic search of the literature was not undertaken. Assertions within the text are based on the evidence level described in Table 3.
14) Revision History
| Date | Revision description |
|---|---|
| December 2019 | Complete revision |
Competing Interests
The authors have no competing interests to declare.
References
Food and Drug Administration. US Department of Health and Human Services. Code of Federal Regulations Title 21 Part 312 Subpart D. Responsibilities of Sponsors and Investigators. Available at https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/cfrsearch.cfm.
Food and Drug Administration. US Department of Health and Human Services. ICH E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R1), March 2018. Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/e6r2-good-clinical-practice-integrated-addendum-ich-e6r1.
A guide to the project management body of knowledge. 5th ed.: Chapter 12 Project procurement management, pp. 355–390. Newtown Square, PA. Project Management Institute, Inc., 2013.
The Avoca Group. A Renewed Look at the State of Clinical Outsourcing, DIA presentation 2016. Available at http://theavocagroup.com/wp-content/uploads/2017/04/2016-Avoca-Industry-Survey-Report.pdf.
Lucas KM. Steps to functional service provider success: outsourcing fundamentals for this pick-and-choose model so that both sponsors and service providers win. Appl Clin Trials. August, 2008. Available from: http://www.appliedclinicaltrialsonline.com/steps-functional-service-provider-success.
Glass HE. An outsourcing necessity. Appl Clin Trials, April, 2009. Available from http://www.appliedclinicaltrialsonline.com/outsourcing-necessity-0.
Hudgens S, Hill R. Managing vendor relationships. Pharm Outsourcing, January/February 2010; 11(1). Available from https://www.pharmoutsourcing.com/Featured-Articles/37566-Managing-Vendor-Relationships/.
Outsourcing DM: How to get the most value out of a partnership, In: SCDM 2012 Annual Conference; 2012.
Communications Management Plan page of Project Management Docs (). Available from https://www.projectmanagementdocs.com/template/project-planning/communications-management-plan/#axzz67rSXzdbe. Accessed December 11, 2019.