Most of us feel comfortable with paper records. We can see them, feel them, touch them (apologies to Peter Townshend). We like to think they’re the same as the electronic records. But we hear rumblings from GMP inspections that FDA doesn’t always share that view (see Finding 4).
What are the 2 keys?
1. Content
2. Meaning
Let’s explore a simplified, hypothetical example to give you ideas on what to look for, the kinds of tactics you can use, and how to communicate effectively with your auditee.
Hypothetical flow cytometry example
You’re auditing Acme Labs, a central lab performing flow cytometry for one of your company’s studies. Your company will use the results to support safety claims.
Do your homework
Before the audit
- Review the instrument manufacturer’s manuals on-line
- Identify the outputs you should see
- Note potential pitfalls
You identify 4 different files created by the flow cytometer
- Experiment
- Data
- Statistics
- Instrument settings
You note the manufacturer recommends storing files on the instrument hard drive and moving them to other locations “later.” Hmmm. Wonder what that will look like?
Conduct the audit
The cytometry lab manager explains their SOPs require printing the test results, reviewing the printouts, and identifying the printout as the raw data.
Start by looking at the most recent notebook of the printouts. You note they are all in black and white, including the plots. In discussions with lab personnel, you confirm the colors displayed by the flow cytometer are very important and play a role in interpreting the results and conducting a meaningful review.
Select a sample of printouts.
Move back to the instrument and ask the operator to show you the experiment, data, statistics, and instrument settings files for your sample. (If for some reason, they can’t, try a different angle. Ask to see the files for the most recent run.) You and the operator confirm the contents of the settings file are not included in the printout.
Discussion confirms the lab follows the manufacturer’s recommendation not to save files directly to external storage. However, the lab only saves the settings files locally on the instrument hard drive and never moves them to external storage. The instrument automatically deletes older settings files as space on the hard drive is used up.
You have collaborated with the lab to help them understand paper printouts are not equivalent in content (instrument setting files are missing) or meaning (color is important).
Write the finding(s)
“Acme Lab failed to maintain complete and accurate laboratory data generated in the course of establishing the safety of the investigational product. Acme’s SOPs did not include instructions for the retention of electronic raw data, and instrument settings files had been deleted. Printed flow cytometer results did not include all required information for adequate review. For example, instrument settings were not included, and color plots were printed in black and white.”
What about the predicate rule?
GLP Study
Consider starting with 58.81 Standard operating procedures and 58.190 Storage and retrieval of records and data. See also the Colorado Histo-Prep Warning Letter for a recent example, including citations against the Quality Assurance Unit.
GCP Study
Current FDA podium-speak is that the Centers for Medicare and Medicaid Services (CMS) are responsible for clinical labs. That does not absolve sponsors or clinical labs of the responsibility to ensure the quality and integrity of clinical trial lab data. Consider 312.22 General principles of the IND submission. This requires assurance of subject safety and that the quality of the scientific evaluation be “adequate to permit an evaluation of the drug’s effectiveness and safety” and “yield data capable of meeting statutory standards for marketing approval.”
Your Takeaway
When an auditee tells you they use paper records instead of electronic records,
- Look for differences in content and meaning
- Work with the auditee to understand the differences you find
Thank you for this piece Jamie – interesting to see how we are moving from a world where we justify replacing paper with electronic records to where paper is inadequate compared to the electronic record. In the FDA finding the problem seemed to be relying on inadequate paper records due to in effect destroying the electronic records (like telling the inspector you burn your paper every two years to make space in your filing room!). That’s the part I don’t get – not investing in relatively cheap IT solutions to render your electronic records sufficient! Am I understanding that correctly?
Nic – That is definitely part of the problem. In some quarters there is a genuine misunderstanding about the relationship between paper and electronic records. It would be interesting to see some actual case studies comparing the costs of maintaining paper records and maintaining electronic records. I have to imagine that electronic records also beat paper records in terms of costs.
I love the analogy you make to burning paper records. Flashback to another state, another time when my neighbor (now deceased) was a defense lawyer dealing with addicts and dealers. He burned his records on a routine basis in the backyard. It got so we could sniff the air and say, “Oh there’s *****, burning client records again.”