The following is based on the Question and Answer portion of “Document Management Success: A Guide to Part 11 System Validation,” a Complion webcast featuring Michelle Grienauer, JD, MPH, Senior Regulatory Attorney at Kinetiq, and Cristina Ferrazzano Yaussy, MPH, CCRP, VP of Professional Services, Complion, Inc. The webinar explored what clinical research sites should know about validating eRegulatory and document management systems.
366 individuals from the clinical research industry attended: 31% from an Academic Medical Center, 23% from a Hospital or Health System, 12% from a Dedicated Research Site, 11% from a Cancer Center, 7% from a Physician Practice, and 5% from a Sponsor or CRO. Prior to the webinar, the audience was polled to collect feedback about the current state of document management and validation.
“How does your site/organization currently manage electronic regulatory and trial documents?”
- 65% said “In-house system (Shared drives, USBs, CDs)”
- 39% said “None, we still use paper”
- 22% said “Vendor/3rd party system (Complion, Box, Dropbox, SharePoint)”
- 4% said “I’m not sure”
“Does your site/organization currently have a Validation Plan?”
- 47% said “I’m not sure”
- 33% said “No”
- 20% said “Yes”
According to FDA 21CFR Part 11 and ICH-GCP E6 (R2), clinical research sites that rely on electronic records or electronic signatures are required to perform validation to ensure their system operates correctly. While the FDA does not indicate how to meet this requirement, you still must demonstrate how you intend to perform validation.
Continue reading to review answers to common questions about when regulations apply. Click here to watch the full webinar and receive a free validation worksheet.
Q. Does Part 11 apply if we're keeping both electronic AND paper copies (i.e. wet ink copies are scanned and saved to our shared drive)?
A. Simply put, Part 11 applies to your site if you’re relying on electronic records or electronic signatures to meet an FDA requirement. The FDA’s draft guidance on 21 CFR Part 11 (Part 11) indicates that if “regulated entity intends to use an electronic copy in place of the paper source data (i.e., intends to destroy the paper source data), then part 11 regulations apply to the electronic system used to create the copy (see §§ 11.10 and 11.30)).
Q. Do network shared drives really need to be validated?
A. Any essential documents stored electronically for FDA regulated research fall under Part 11 regulation. So if you are storing important documents (e.g. 1572, ICF, Protocol, etc.) in a shared drive, and relying on this information to make study decision, that shared drive should be validated.
Q. How do you decide what is "critical" and should be tested?
A. The FDA recommends a risk-based approach to validation, so it depends on the impact the system will have on quality, safety, and record integrity. Critical components are those that have impact on the integrity of the study. So components related to data and systems that an IRB may use to make their determinations when there are critical outcomes that will result from using these systems. In the draft guidance, the FDA provides examples of critical records such as records containing laboratory and study endpoint data, information on serious adverse events and study participant deaths, information on drug and device accountability and administration.
When using a risk-based approach for validating electronic systems, entities should consider (1) the purpose and significance of the record, including the extent of error that can be tolerated without compromising the reliability and utility of the record for its regulatory purpose and (2) the attributes and intended use of the electronic system used to produce the record.
The following diagram illustrates a risk-based approach for validation:
Q. How does this apply to resident/fellow research being tracked on in-house excel files?
A. Since resident research is not typically FDA regulated, as they are often retrospective reviews, observational studies fall outside of the FDA rules, therefore Part 11 wouldn’t apply. However, compliance to Part 11 is case-specific and can depend on the type of research they being conducted and what’s being done.
Q. Does this presentation is refer to general computer systems or the electronic health records (EHR)? My understanding is that the FDA does not oversee the EHR?
A. The presentation refers to electronic systems used to create, modify, maintain, archive, retrieve, or transmit an electronic record required for FDA regulated clinical investigations.
Regarding EHR, the draft guidance indicates that, “electronic systems used in the provision of medical care (e.g., electronic health records (EHRs)) generally are systems that are (1) designed for medical care of patients not enrolled in a clinical investigation and (2) owned and managed by the institutions providing medical care. FDA does not intend to assess compliance of these systems
These electronic systems may produce additional electronic records during the course of patients’ care (e.g., hospital admission records, electronic health records, pharmacy records, laboratory records, imaging records, electronic consultation records) that may be useful for providing data in clinical investigations. As provided in the guidance for industry, “Electronic Source Data in Clinical Investigations”, FDA does not intend to assess compliance of these systems with part 11.24 For more information on best practices for using data from EHRs in FDA-regulated clinical investigations, see the draft guidance for industry “Use of Electronic Health Records Data in Clinical Investigations.”
Q. I often hear the terms Validation and Qualification when referring to computerized systems. What's the difference between those two terms, if any?
A. From an IT perspective, the terms qualification or installation qualification, are actually very different concepts. Installation qualification refers to ensuring the software or the system being implemented is installed correctly. Qualification refers to systematic testing of the system.
Q. I've never had a sponsor send Part 11 compliance documentation to the site. Is this a standard thing sites are asking for in the study start up process?
A. If the site is relying on the sponsor system to create, modify, maintain, archive, retrieve, or transmit an electronic record required for FDA regulated clinical investigations and these records are considered the source for the site, this may be considered, “outsourced electronic service” and should be validated as appropriate.
In other words, if your site is relying on the sponsor system to be the system of record to meet FDA compliance, then Part 11 applies. However, most sponsor systems are intended to be helpful tools, and are not the sites system of record. Therefore it is not common for sponsors to send Part 11 documentation.
However, if the sponsor system is the system of record, then your site should obtain documentation that includes, but is not limited to, a description of standard operating procedures and results of testing and validation to establish that the outsourced electronic service functions in the manner intended.
Q. Not all cloud solutions are necessarily Part 11 compliant. How would one know?
A. Ask the vendor if they’re Part 11 compliant, and ask them to provide documentation. Ask if they have undergone a third-party assessment for validation, or are willing to. If they haven’t, you may want to audit the vendor. Keep in mind at the end of the day, even though you may be purchasing a system, you are still responsible at the site to make sure that the system is Part 11 compliant. So, even if the vendor says it’s Part 11 compliant, you’re still responsible for doing your due diligence to make sure and documenting that it is so.
Q. Once a system is validated, does it need to be re-validated annually?
A. This is a great question. Change control relates to making modifications to the system. So, unlike protocols, that go through an annual review, systems should be validated or revalidated when there are significant changes. There may be minor modifications such as minor tweaks such as fixing bugs that don’t warrant full revalidation; whereas new features or critical updates would require revalidation or validation of those new components. So, I wouldn’t view it from an annual basis perspective. It’s really about the system, confirming integrity of the system, and if there are any changes, those changes are tested prior to implementation.
Q. Regarding the application of Part 11 to our site, we utilize a cloud based CTMS which houses patient demographics (not procedure/health records). We have subjects who participate in NIH studies in the CTMS. Is the CTMS vendor responsible for being part 11 compliant? Is our site accountable in this case?
A. Simply put, Part 11 applies to your site if you’re relying on electronic records or electronic signatures to meet an FDA requirement. Additionally, you can’t outsource your regulatory compliance obligations, so you’re still on the hook. You’re still the one that has to answer to the FDA about whether your data is reliable. So, that means that you need to take due diligence and properly vet any potential vendor and ensure they can produce documentation of validation and Part 11 compliance.
Q. If sponsor is providing software, i.e. EDC software, are they responsible for validation? Or is the site required to have their own separate policy as well?
A. It depends on what the intended purpose is for that particular system. If you intend to use it as your source of the truth, there should be some type of process in place. Whether you’re gathering their validation plan or the outcomes of their validation for your records because if you intend to rely on those systems, then you will be held accountable for making sure that that system complies with your requirements. So it really depends on your intentions for those systems, but if you intend to rely on it, then you need to do your due diligence and gather that information from the site’s perspective.
Q. Is there anything in the regs that we can use to go to vendors with asking for their validation documents? Most vendors I've spoken to are protective of their systems and validation documents.
A. Reference the draft guidance document. If the vendor is not willing to provide adequate documentation, you may want to consider another vendor.
Q. Is Part 11 applicable if site uses a secure drive (limited access) to archive a portion of the regulatory binder?
A. If you intend to use an electronic copy in place of the paper source data (i.e., intends to destroy the paper source data), then part 11 regulations would apply to the electronic system used to create the copy. Using an electronic means, such as a durable electronic storage device is an acceptable method to archive study-related records at the end of the study. You should ensure that the integrity of the original data and the content and meaning of the record are preserved. If the electronic records are archived in such a way that the records can be searched, sorted, or analyzed, sponsors and other regulated entities should provide electronic copies with the same capability to FDA during inspection if it is reasonable and technically feasible.
Q. What documentation can a site reasonably expect a vendor to release to them, since most vendors consider their documentation proprietary.
A. For commercial off the shelf systems (COTS) that perform functions beyond office utilities such as COTS EDC systems, validation should include a description of standard operating procedures and documentation from the vendor that includes, but is not limited to, results of their testing and validation to establish that the electronic system functions in the manner intended.
Q. If a site used RedCAP to collect data for investigator initiated studies that will later be sent to the FDA for approval, must the site have a System Validation process for their RedCAP?
A. If any of the following hold true, then yes.
The FDA identified the following records as being applicable to Part 11:
- Records required for clinical investigations of medical products that are maintained in electronic format in place of paper format, including all records that are necessary for FDA to reconstruct a study
- Records required for clinical investigations of medical products that are maintained in electronic format and where the electronic record is relied on to perform regulated activities
- Records for clinical investigations submitted to FDA in electronic format under predicate rules, even if such records are not specifically identified in FDA regulations (see 122 § 11.1(b))
- Electronic signatures required for clinical investigations intended to be the equivalent of handwritten signatures, initials, and other general signings
For example, an investigator must maintain records of drug disposition and case histories for any individual that receives the investigational product. If your site keeps an electronic version of those disposition logs and case histories, then Part 11 applies to you. Part 11 applies to you regardless of whether you’re using an in-house system or a vendor system.
Q. If the vendor provides validation documents as evidence and upon review internally, it meets our requirements, do we (as the investigator site) still need a validation document?
A. This would depend on your Standard Operating Procedures to define what you deem acceptable. The extent of any validation procedures completed by the site should be tailored to the nature of the system and its intended use.
Q. If you validate in a test/validation system, should you re-validate once you're in the live/production system? Can we skip the test/validation site and validate directly in production?
A. From a software perspective, system testing should occur prior live utilization. It is best practice to validate and revalidate any critical system changes in a validation environment prior to utilization and deployment of the critical updates to the live environment.
Q. Does Complion have validation tracking included and are there templates for normal items to validate?
A. Complion does support sites with developing a plan and facilitating initial and ongoing validation including templates, documentation and reports. To learn more, please contact us.