Disclaimer: This post is for academic purposes only. Please read the original document if you intend to use them for clinical purposes.
This document summarizes the key themes and important ideas presented in the Indian Council of Medical Research (ICMR) guidance document, “Validation of Rapid Diagnostics for pathogen identification and Antimicrobial Susceptibility Testing (AST), 2025“. It aims to assist innovators and testing laboratories in India with the validation of diagnostic tests for pathogen identification and antimicrobial susceptibility, supplementing existing Central Drugs Standard Control Organization (CDSCO) guidelines.
Main Themes:
- Addressing Antimicrobial Resistance (AMR): AMR is a critical global health concern, particularly in countries like India. The guidance is framed within the context of needing reliable and validated diagnostics to combat this growing threat.
- Comprehensive Validation Framework: The core theme revolves around providing a structured and detailed framework for validating rapid diagnostics. This framework encompasses assay development, analytical performance evaluation, clinical performance evaluation, and assay maintenance.
- Regulatory Alignment: The guidance emphasizes the importance of aligning validation processes with the regulatory requirements of CDSCO for In Vitro Diagnostics (IVDs) in India. It outlines the different regulatory pathways and licenses required for new and predicate device-based diagnostics.
- Focus on Both Pathogen Identification and AST: The document provides specific considerations and criteria for validating diagnostics aimed at both identifying pathogenic microorganisms and determining their susceptibility to antimicrobial agents.
- Importance of Analytical and Clinical Performance: The guidance stresses the need to rigorously evaluate both the analytical performance (accuracy, precision, sensitivity, specificity, etc.) and the clinical performance (clinical validity, utility, sensitivity, and specificity) of the diagnostic tests.
- Emphasis on Quality and Standards: Throughout the document, there is a strong emphasis on the use of quality control measures, reference standards, and adherence to established guidelines from organizations like CLSI, EUCAST, ISO, and others.
- Practical Guidance for Developers and Laboratories: The document is designed to be practical, providing step-by-step guidance, considerations for study design, sample size, statistical evaluation, and the content of a validation dossier.
Advertisement:
Most Important Ideas and Facts:
1. Introduction and regulatory context:
- AMR is a critical health concern in India due to population density and infectious disease burden.
- The document aims to provide guidance on validating indigenous AMR diagnostics, supplementing existing CDSCO guidelines.
- It outlines the regulatory process in India for IVD approval, including Test Licenses, Clinical Performance Evaluation (CPE) permission, clinical data approval, and manufacturing licenses.
- The regulatory pathway differs for new IVDs (no predicate device) and those with existing predicate devices.
- IVDs are classified based on risk levels (Class A to D).
2. General considerations for validation:
- Validation studies should ideally be multi-site (minimum two geographically diverse sites complying with ISO15189).
- The type of specimen used must be specified, and isolates from sterile sites are considered more clinically significant.
- Sample size determination requires statistical expertise.
- Reference materials (standard strains, antimicrobials, etc.) must be of certified quality.
- Interpretation of AST results relies on breakpoints set by CLSI and EUCAST. Indian labs primarily use CLSI breakpoints.
- The Minimum Inhibitory Concentration (MIC) is a key metric in AST, interpreted against breakpoints.
- A recognized reference method (often the “gold standard“) is crucial for comparison to demonstrate equivalent performance.
- Test kits and reagents must be stored under conditions specified by the developer, with temperature monitoring.
3. Assay development and validation criteria:
- Phase 1: Assay development:
- Step 1: Define the purpose of validation, including whether it’s for pathogen identification, AST, or AMR marker detection, and whether qualitative or quantitative results are needed. For pathogen identification, consider the disease, result type, and whether it’s a screening or confirmatory test. Sample size depends on the target population (asymptomatic vs. symptomatic). For AST, specify genotypic or phenotypic basis, differentiation of susceptibility categories (S/R or S/I/R), and whether a single test or algorithm is required.
- Steps 2 & 3: Optimize reagents and protocols, and establish quality control (QC) and quality assurance (QA) procedures. QC for the test and reference method should follow guidelines (e.g., CLSI). Reference strains should be well-characterized and stored optimally.
- Phase 2: Assay validation:
- Step 4 (Analytical Performance): which requires Standard Operating Procedures (SOPs) for validation. Key elements include study design, statistically determined sample size, selection of a reference method (gold standard), defined specimen types, detailed test method, shelf-life assessment, and statistical evaluation of results. Exceptions in reference methods for AST (e.g., D-zone test, Agar Dilution for specific organisms, molecular detection for specific resistance mechanisms) are highlighted. Analytical and performance studies should be repeated for each sample type if the diagnostic uses multiple types. Linearity studies are needed for each specimen type in quantitative assays. Shelf-life (stability) must be assessed through real-time or accelerated studies following guidelines.
- Step 5 (Analytical Performance Testing): which identify relevant performance parameters (accuracy, precision, sensitivity, specificity, reportable range, etc.) and define formulas. Justification is needed for non-applicable parameters (e.g., linearity for qualitative tests). The number of parameters tested depends on the diagnostic type (biochemical, molecular, microbiological) and format (method, device, kit). Detailed guidance on assessing:
- Accuracy: Comparison to a reference method using methods like Barnett’s test, intra-class correlation, Bland-Altman, regression, and calculating diagnostic sensitivity, specificity, PPV, NPV, and Likelihood Ratios.
- Reproducibility: Consistency of results when tested twice on the same subject (Cohen’s kappa, duplicate standard errors, repeatability coefficients, intra-class correlations).
- Precision: Agreement of repeated measurements on the same sample (Confidence intervals, SD, CV within and between runs, day-to-day, total variation).
- Reportable Range (Linearity): For quantitative assays, using 7-11 concentrations across the anticipated range.
- Analytical Sensitivity (Limit of Detection): Using multiple replicates near the detection limit and statistical analysis (probit regression).
- Analytical Specificity (Interference): Testing for interfering substances and genetically similar organisms.
- Reference Interval: Verification using known normal samples.
- Step 6 {Clinical Performance Evaluation (CPE)}: Conducted to evaluate the diagnostic’s ability to correctly identify the clinical condition. Requires a clinical performance evaluation plan and ethical approval. Elements include selection of target population with inclusion/exclusion criteria, study settings and number of sites, clear test purpose, defined specimen collection and handling, ethical considerations (ISO 20916), and clinical performance evaluation criteria (clinical validity, utility, sensitivity, and specificity). Clinical sensitivity and specificity are defined and their calculation formulas provided. Regulatory permission is required from CDSCO for CPE, and registration with the Clinical Trial Registry of India is mandatory.
- Phase 3: Assay maintenance:
- Step 7: Establish criteria for ongoing maintenance and extension validation.
- Step 8: Compile a comprehensive validation dossier
- Step 9: Outcome and decision based on the validation results.
4. Antimicrobial Susceptibility Testing (AST) specifics:
- For AST diagnostics, accuracy (Categorical Agreement – CA, Essential Agreement – EA), error rates (minor Errors – mEs, Major Errors – MEs, Very Major Errors – VMEs), precision, and reproducibility are key parameters.
- Acceptance criteria for these parameters are provided, referencing ISO and FDA guidelines.
- Error rates are calculated based on discrepancies with the reference method, with VMEs (false susceptible) being the most critical. FDA’s VME acceptance limit is <1.5%.
- The “error-rate-bound method” and “model-based approach” for breakpoint evaluation are mentioned for specific conditions.
- “Void testing bias” should be addressed, and discrepant analysis techniques avoided.
5. Clinical Performance Evaluation (CPE) details:
- CPE aims to demonstrate clinical validity and utility.
- Study design should be appropriate for the intended use (diagnosis, screening, monitoring).
- Multi-site validation is preferred to ensure accuracy and avoid bias, with sites ideally accredited.
- Ethical considerations are paramount, requiring ethics committee approval.
- Sample size for CPE depends on the test purpose and prevalence of the condition. Statistical expertise is needed for sample size calculation
- Clinical validity refers to the diagnostic’s ability to predict the presence or absence of a clinical condition.
- Clinical utility considers the overall benefit of the test in improving patient outcomes and guiding clinical decisions. Factors like turnaround time, cost, and ease of use are relevant.
Conclusion:
This document serves as a crucial guidance document for bolstering the development and implementation of reliable AMR diagnostics in India. By providing a detailed and comprehensive framework aligned with regulatory requirements and international standards, it aims to empower innovators and laboratories to produce high-quality diagnostic tools essential for effective AMR surveillance and management. The emphasis on both analytical and clinical validation, coupled with practical considerations, underscores the commitment to ensuring the utility and impact of these diagnostics in the Indian healthcare setting.
