Krouwer Consulting
 

Unit-use devices, POC, and Quality Control

Unit-use devices have existed for many years. It may be useful to consider two types of unit use devices – those that are used in the main clinical laboratory and Point of Care (POC) devices. Since POC devices are often operated outside of the clinical laboratory, one challenge is the difficulty for non clinical laboratory personnel to perform external quality control (QC). CMS proposed reducing the frequency of external QC (for any assay) – called equivalent QC – provided certain criteria were met (1).

There has been some confusion with respect to unit use and non unit use (called here continuous flow) devices. It is often suggested that external QC is of no value in unit use devices, because whatever the outcome of external QC with the unit use device, that specific device has been used up and the next specimen will see a new unit use device. There is in fact not that much difference between unit use and continuous flow devices. Consider external QC in four cases. 

1. Continuous flow device – Reagent lot is bad, external QC detects the failure in all samples.
2. Unit use device – Reagent lot is bad. If  a clinical laboratory receives a shipment of unit use devices all from the same lot, which is likely, external QC will detect the failure in all samples.
3. Continuous flow device – Random failures occur, external QC will not detect the failure in all samples.
4. Unit use device – Random failures occur, external QC will not detect the failure in all samples.

There are some differences between what happens at the manufacturing plant vs. the clinical laboratory. That is, for either device type, the reagent is made and tested by the manufacturer. However, recalibration occurs at the clinical laboratory only for continuous flow devices, but one should not think that procedures performed at a manufacturing plant are immune to problems or that the only issues that occur are due to shipping and storage.

It may be helpful to understand quality tools as related to failures in the clinical laboratory (for all devices). Failures may be considered to be of three types:

• reliability – an error occurs preventing the device from reporting a result. This may be a hardware error such as a failed power supply or the result of a detection algorithm, whereby software has detected that there is something wrong with the response signal, so the result is suppressed. Note that reliability errors can be considered to be either persistent (failed power supply) or non persistent (isolated response signal problem).

• persistent performance errors – an error in a result that repeats across several samples. For example a calibration error is persistent because each sample will have a bias until the system is recalibrated.

• non persistent performance (random) errors – an error in a result that occurs in an apparently random fashion. Examples include interference in a patient sample or a noisy signal (and the noisy signal escapes the detection algorithm).

These failures may also be classified as:

The following table shows the effectiveness of various quality tools to deal with failure types.

FMEA=Failure Mode Effects Analysis, FRACAS=Failure Review And Corrective Action System

One tool that has been omitted is attribute (also called acceptance) sampling. This technique can detect non persistent errors (both reliability or performance) but it is impractical for the clinical laboratory. This is because to guarantee with high confidence a high proportion of a lot of materials will not exhibit non persistent errors, usually requires very large samples sizes.

This can be shown using the hypergeometric distribution. However, the use of this distribution could be questioned since it involves knowledge of lot attributes that clinical laboratories are unlikely to have. The binomial distribution is a good approximation. For example, if one sampled 10 units and found 0 defectives, one could only guarantee with 95% confidence that no more than 25.9% of units are defective. To obtain better results, one has to sample many more units (see: http://krouwerconsulting.com/Essays/Outliers.htm). 

The table above shows the importance of internal QC, which unlike some recent suggestions is not new but has been in virtually all systems since assays were automated. However, internal QC methods are largely proprietary and thus details are generally not known to clinical laboratory users.

FMEA and FRACAS represent tools that the clinical laboratory can carry out and are effective for all errors.

A final table shows how each of the quality tools works.

The meaning of prevention, detection, and recovery is explained in reference 2.

References.

1. See http://www.cms.hhs.gov/CLIA/downloads/6066bk.pdf
2. Managing risk in hospitals using integrated Fault Trees / FMECAs. Jan S. Krouwer, AACC Press, Washington DC, 2004.