שיעור מס’ 10 17.02.2021
Validation and Verification in the Clinical Microbiology Laboratory
השאלה החשובה ביותר: האם השיטה/הערכה/המכשיר מתאימים לשימוש במעבדה שלי? האם אני בטוח שאתן תשובות נכונות ואורך הזמן?
Literature
Cumitech 31 A
Procedures for validation of diagnostic methods in clinical laboratory (ISO 15189
Validation Policy for test methods
שאלה
אם שיטה עובדת לפני שמשתמשים
אם שיטה עדיין עובדת (בזמן שיש שינוים)
קודם מתחילים תיקוף עם ATSS אם שיטה לא עובד עם ATSS אז לא מתאים למעבדה קלינית
Gold standard A test method currently accepted as reasonably, but not necessarily 100%, accurate The new test (new and improved technologies) may be more accurate than the accepted gold standard.
Study Components
Accuracy
Precision
Analytical sensitivity
Analytical specificity
Reportable range of patient test results
Reference range(s)
Any other characteristic required for test performance and interpretation of results
Analytic Accuracy Agreement between test result and “true” result
Can be done in two main ways: 1. Comparison of results between new method and “reference” method 2. Results using new method on certified reference materials (Recovery) First approach almost always used
Precision (repeatability) …replicate analyses of a homogeneous analyte agree with each other Precision does not guarantee accuracy Precision (quantitative assay) = reproducibility (qualitative)
Sensitivity (positive accuracy) Analytic sensitivity measures the smallest quantity of an analyte that can be reproducibly distinguished from background levels Clinical sensitivity is the proportion of people with the disease who have a positive test
Specificity (negative accuracy) Analytic specificity is the ability of an analytical method to detect only the analyte that it was designed to measure Clinical specificity is the proportion of people
PPV – probability of disease in a patient with a positive test result. NPV – probability of not having disease when the test result is negative.
בודקים אם יש אישור של FDA
כמה חיידקים צריך להריץ בתוקוך?
תלוי בשיטה: שיטה פשוטה – צביעת גרם 20 הרצות אם שיטה FDA CLEARED
אם NOT FDA CLEARED – 150-200 הרצות
צביעת גרם צריך תירוף
1 צריך לקבוע רמת דקולוריזאציה בתנאים של מעבדה אם חומרים שונים
כול מצע עובר תיקוף
תיקוף תרביות דם.
אם מצע מתאים לחיידקים
אם מכשיר מתאים
בודקים 15-20 ISOLATES מה שבדרך כלל יש במעבדה.
Seeded Blood Culture Studies Select a minimum of 15 to 20 isolates representative of blood culture isolates normally seen in the institution Prepare seeded blood cultures with isolates of each of the above species The minimum amount of sterile, antibiotic-free human blood recommended by the manufacturer should be placed in each bottle. The numbers of organisms placed in each bottle should approximate those found in cases of septicemia. This can be done with serial dilutions of the organisms before inoculation to achieve approximately 5 to 30 CFU per bottle.
גם בודקים TIME TO DETECTION לא יותר מ5 ימים
Parallel Blood Culture Studies Actual patient and laboratory conditions. Duplicate sets of blood cultures Minimum of 20 positive blood cultures (not including contaminants) The new method is considered verified if the sensitivity is at least 95%, relative to the reference method, and times to detection are not significantly different
תיקוף של רגישות
כשאר קבלנו דיסקים של חברה אחרת להשוות בין שני חברות או מול שיטות אחרות
יותר טוב לבדוק מול GOLD STANDRD
Verification of AST Systems
Method #1 – Precision
Test five isolates in triplicate for 3 to 5 days
Use ATCC, QC strains or proficiency samples
Acceptable results are >95%, including both essential and categorical agreement
Verification of AST Systems
Method #2– Accuracy
Bacteria from clinical specimens ◦commonly isolated at the institution ◦represent resistant phenotypes observed in the institution
- aureus (MRSA) D-test positive S. aureus
VRE – Vancomycin Resistant Enterococci
Coagulase negative staphylococci
ESBLs KPC, Amp-C producers
MDR Pseudomonas aeruginosa and Acinetobacter
E-TEST אבדל אמור ליהות לא יותר מ 2 ניולים
Discrepancies
Very Major Errors (false-positive susceptibility if the result was sensitive with the new system but resistant with the current method) – use in validation vs gold standard method only
Major Errors (false-positive resistance if the result was resistant with the new system but sensitive with current method)
minor Errors (a susceptible or resistant result with the new method with an intermediate result with current method).
Accuracy New vs reference (gold standard) method
VME rate should be less than 3% (minimum of 35 resistant isolates)
ME rate should be less than 3% ME and
MinE combined (a minimum of 100 strains), the error rate should be a combined less than
7%
Accuracy Two methods, not one is gold standard
MinE – Minor error (discrepancy) – one AST is I and the other is S or R.
ME – Major error (discrepancy) – one AST is S and the other is R. Should be less than 5% MinE + ME should be <10%
זיהוי חיידקים
ATCC, QC strains or proficiency samples Clinical isolates – at least 20 At least 90% agreement with the existing system or reference method
DATA BASE Phoenix – יותר מ-1000 חיידקים MALDI – אלפים!!!
The results of test verification
The test is acceptable for routine use
Further verification studies are required
Immediate corrective action is required by the manufacturer (if commercially obtained), the user, or both.
The test is unsuitable for routine use until its performance parameters can be verified
S.pneumoniae *Supplemental (confirmatory) method = OPTOXIN
Molecular method
Detection of carbapenem-resistant bacteria by culture DNA extraction from perianal/rectal swabs using the Roche MagNA Pure LC instrument. DNA extraction from perianal/rectal swabs using the bioMe´rieux NucliSENS easyMAG system blaKPC detection by q-PCR
The detection limit of the real-time blaKPC assay was 1 CFU.
A test method currently accepted as reasonably, but not necessarily 100%, accurate The new test (new and improved technologies) may be more accurate than the accepted gold standard. Further investigated by other tests or by monitoring the patient’s condition to determine if disease develops.
by other tests Repeating the extraction from the original tubes using both extraction systems and repeating the q-PCR assay for both new and old extractions. Agarose gel electrophoresis, PCR amplicons of appropriate sizes were sequenced using the ABI Prism Dye Deoxy Terminator cycle sequencing kit (Applied Biosystems, Foster City, CA) as previously described
Does the test still work?” Quality Assurance Personnel training Proficiency testing Parallel testing of duplicate instruments Instrument maintenance and calibration Specimen identification and integrity QC/QA Prevent, detect and address errors and performance deviations
