Automated Validation—Does one size have to fit all?

Eva Kelly is a Director at ERA Sciences Ltd and has more than 25 years of experience as a Data Integrity (DI) subject matter expert specialising in 21 CFR Part 11 and Annex 11 compliance and has held DI lead and quality assurance information technology lead positions for pharma and biotech companies utilising GxP software as a service, hosted and on-premise solutions. She has been involved with many international DI remediation projects with significant positive outcomes.

Opinions expressed by CSols Inc. contributors are their own.

In life sciences, including pharmaceuticals, biotechnology, and medical technology, when considering the impact of good practices (GxP) activities and their supporting records and data, the most important risk is the risk to the patient.

When we move away from a patient-centric approach, our efforts may no longer be commensurate with the risk. Validation is one of the controls we apply to systems and processes to ensure that data and systems remain fit for purpose and to mitigate risk to patients, product quality, or overall data reliability and integrity. 

Whether the validation process is manual (predominantly paper-based) or the digitization journey has already started, it continues to throw up challenges and often causes heartache for those involved. Relying on paper-based validation processes may already be impacting organizational efficiency and on-time delivery of GxP software solutions. 

When writing this blog post I had to take a step back and consider the difference between digital and automated validation solutions. In my opinion, digitization loosely means the conversion of analog information into texts, photographs, etc., whereas automation implies a move toward the execution of tasks with limited personnel inputs.

I believe it’s the automation of validation processes and documentation that will offer long-term organizational benefits, including the availability of accurate and complete validation records (ALCOA+).

To the naysayers of automated validation solutions, I would say, when the validation and documentation evidence has a bigger physical footprint than the products produced and the business requirements continue to change at a rapid rate, it’s time to look at automated validation.

Unfortunately, sometimes it takes a catastrophic event to precipitate change. Since the COVID-19 pandemic started, the ability to be on site executing, reviewing, or approving protocols has been massively challenged and many have made, or are now contemplating, the move to an automated solution as the only resilient option.

Buy In

Transitioning from paper-based validation approaches likely sits well with those already on the digitization journey! But full-blown automation may throw up questions and concerns, including the following:

• How do we start to automate our validation approach?
• How do we get large teams trained up on automated test solutions without impacting business as usual?
• Who will need to have access to the automated validation solution?
• Can we use an automated validation solution to manage all validation documents or just our test execution protocols?
• Can we link automated validation solutions to standalone GxP applications?
• Is there a migration tool or export/import tool for a seamless movement of protocols?
• Will protocols be version controlled? 
• Should we park the past and decide to only create new protocols going forward? (This is a much bigger question and is really a question of strategy.)
• Can I link executed protocols to a requirement traceability matrix (RTM)? 

Understanding that automated validation can be implemented on a phased basis, with test protocol construction and use as a suggested first phase, needs to be popularized. It’s not so much a question of one size fits all (as with some clothing) but rather, what size addresses our needs now and will it sustain our needs as they change? Is this one size fits forever? Even I don’t believe that one!

Communication and stakeholder engagement will be important in managing organizational understandings and expectations and will help to address many of the questions mentioned in the previous paragraph. Once an organization identifies a need for an automated validation solution, it should be treated the same as any other GxP application starting with IT supplier assessments and definitely a demo or two. User requirements for the automated solution should involve cross-functional stakeholders to understand current process gaps, process constraints, validation holdups, and approvals management to deliver a sustainable, resilient solution.

Don’t forget that the automated validation solution will likely be the repository of your future executed validation protocols, test scripts, and test steps; all of which should be considered GxP records—and the solution will also require validation. The vendor should have all validation documents available on request. For on-premise solutions, these may be provided electronically or as a bound volume (more paper). For SaaS-based solutions, these are likely access controlled and available for review or download via a shared link. To stick with the clothing analogy, check the care and use instructions as this solution when used for GxP purposes, will need to be 21 CFR Part 11 and/or Eudralex Volume 4 Annex 11 compliant.

Is it likely that one size can fit all? Solutions are flexible and many are built with Jira software interfaces, well known in software development. Solutions range widely in the number of available licenses, configuration, expansion capability, and even functionality, so it’s wise to shop around for your organization’s best fit. One consideration from the outset is the availability of signature approvals—whatever the solution, it will either need embedded electronic signature approvals or a bolt-on option that allows this functionality. 

Understanding the Validation Ask

So, let’s look at the requirement for validation and consider GxP computerized systems and their fitness for purpose.

Referring here to Eudralex Volume 4 Annex 11, as it is informative in its requirements for computerized system validation and its definitions do not swamp us in legalese:

A computerized system is a set of software and hardware components that together fulfill certain functionalities.

1. The application should be validated; IT infrastructure should be qualified. 
2. As part of a risk management system, decisions on the extent of validation and data integrity controls should be based on a justified and documented risk assessment of the computerized system.

So, 1, the application should be validated; this means we can validate it or we can outsource this activity to, for example, a third-party consultant like CSols or the IT solution vendor.

And 2, we need to take a risk-based approach—Annex 11 and 21 CFR Part 11 certainly do not dictate whether validation should be automated or manual or generate paper, electronic, or even hybrid records. Newer technology vendors offer flexible, paperless solutions.

Additional Annex 11 requirements clearly call out that validation is required not only at the project phase prior to initial implementation and system go-live but also throughout the operational life cycle of the system when changes are required and even at retirement (if migration of records is indicated). Furthermore, Annex 11 requires validation controls to be applied such that the data remains fit for purpose.

Should we equate mountains of paper records generated during validation protocol test execution as the proof of fitness for purpose? Do you believe that [more paper = better assurance!] across the system lifecycle? Should our efforts always require extensive screen captures, even when systems fall into a patient-risk spectrum ranging from direct impact to no impact? Would it not be better to ensure that the effort is commensurate with risk and use screenshots only as appropriate, with testing efforts directed more toward higher risk direct impact systems? Validation approaches, just like solutions, need flexibility. Computer Software Assurance, i.e., a risk-based approach to validation, advocates such thinking. Most automated validation solutions would facilitate scripted and ad hoc testing with all records generated electronically (no more paper!) and screenshots captured as and when indicated.

For those of you who have been engaged in the screenshot process, screenshots often fail or lack ALCOA+ elements that we expect as a minimum data integrity control for our records.

I often find myself asking ‘Who captured the screenshot and when was it taken?’ Would it not be better to automate the screenshot capture in the environment of execution, with an automatic attachment to the test step?

Validation Frameworks

Industry-recognized best practices such as GAMP 5 and Agile methodologies and frameworks can direct your validation effort. These methodologies can both leverage automated validation testing.

Agile is certainly becoming more prevalent as it allows us to rapidly respond to changing business requirements and push release cycles required by many of our SaaS- and cloud-based GxP solutions.

But if you are looking for validation guidance on how many screenshots, how many scripted or unscripted test cases, or how much sampling needs to be performed to assure a data migration effort, you will be looking for a very long time. Explicit guidance isn’t there because it is up to the business to make a risk-based decision as to what validation must include, to ensure the effort is commensurate with system risk. 

A Move to Automation

Is it possible to enhance the overall documentation and process evidence without increasing the burden on the teams involved? Enter, stage right—Automated Validation

Leveraging automated validation means that your end-to-end validation activity can be fully managed across all phases of the system life cycle. It has such flexibility to include all documents, tests, approvals, screenshots, and deviation records and to automate as much or as little as the business needs.

If we refer to the GAMP risk-based approach (below) and the project stages and supporting processes within the system lifecycle, automated validation could be leveraged from Concept to Retirement.

We often use a simplified diagram (below) to further capture the transition from one system to another that reflects the changing business need and again all life cycle phases where validation is required to ensure the system and the data remain fit for purpose.

Now we’ll look at the documentation suite that might control the end-to-end validation activity. Let’s take a typical suite based on a GAMP 5 category 4 system informed by a completed System Level Impact Assessment (SLIA):

• User Requirement Specification
• Validation Plan
• More detailed specification documents such as Functional Requirements (FRS) or Design Configuration (DS)
• Test Plan and Testing Protocols (IQ/OQ/PQ) based on the outcome of the SLIA and the validation approach
• Requirements Traceability Matrix
• Validation Report

Now consider what an automated solution could allow you to do in an efficient and centralized electronic location.

1. Create and manage the entire validation suite of documents including version control
2. Users can execute electronic test cases, and direct capture of screenshots or video evidence is automated
3. Deviations can be automatically recorded and ready for review
4. Direct user test execution effort is reduced or eliminated 
5. Test cases are automatically scheduled to run as a full validation or as a form of regression testing 

Realized Gains and Concerns

A company could schedule test executions that could directly integrate with the computerized system and run IQ, OQ, and even PQ tests without any impact on business resources until the review and approval stage. Screenshots would automatically append to the protocol test steps with no messy cutting, pasting, or attribution checks—all completed by the automated solution itself. 

Deviations could be directly managed in the system until they are resolved or further investigated the test execution would be awaiting approval. Once all deviations are closed, the validation is completed and all electronic documents are available in the one system with a final validation report.

Are there any cons?

Why haven’t we all bought into automated validation?

The biggest perceived burdens include:

• movement of existing protocols and documents
• creation of new documents
• ownership

There are loads of import tool options, templates, and more than can be leveraged; so, in my opinion, it’s at the very least worth a demo. Bolting on third-party signing solutions such as DocuSign is not automated validation, and it never will be. 

Pricing, and I have purposely left this for last, will always be a consideration, but based on this author’s experience, the return on investment is undeniable. Paper mountains no more! And, an electronic document management system (EDMS) purpose-built for end-to-end validation with full traceability of all validation activities and GxP records.

---

What are your thoughts on Automated Validation Software?