88 Inviting comments on standard IVD evaluation protocol drafted by ICMR and CDSCO 2 2025-Jan-29 1704 KB

ICMR-CDSCO/IVD/GD/PROTOCOLS/04/2024 Page 1 of 1

Indian Council of Medical Research and Central Drugs Standard Control Organization Department of Health Research and Drugs Controller General of India Ministry of Health and Family Welfare Government of India Document No.: ICMR-CDSCO/IVD/GD/PROTOCOLS/04/2024

Subject: Inviting comments on standard IVD evaluation protocol drafted by ICMR and CDSCO

Licensure of In-Vitro Diagnostics (IVDs) under Medical Devices Rules 2017 requires a detailed evaluation protocol for the performance evaluation of IVDs to evaluate their quality and performance. To facilitate this process, the Indian Council of Medical Research (ICMR) and CDSCO have come together to draft standard evaluation protocols for use by IVD manufacturers testing labs in India. Currently, the HMPV real time PCR IVD evaluation protocol has been developed by ICMR and CDSCO.

The protocol is now being placed in the public domain for comments from relevant stakeholders. This window of opportunity will close on 15th March 2025, and, once finalized, there will be minimal scope for change in these documents. Therefore, all interested stakeholders are requested to provide their comments before 15th March 2025, at ivdevaluation@gmail.com as per the enclosed format. Once the public consultation period concludes, all comments will be reviewed and considered in finalizing the draft protocols before final clearance by ICMR and CDSCO.

Dated: 28th January 2025
Place: New Delhi

STANDARD IVD PERFORMANCE EVALUATION PROTOCOL STAKEHOLDER FEEDBACK FORM

S.N. Name of the Protocol Document No. Page No. Line No. Current Text Proposed Text Explanation/Reference

Name:

Designation and Affiliation:

1 HUMAN METAPNEUMOVIRUS REAL-TIME PCR 2 ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 3

4

5 STANDARD PERFORMANCE EVALUATION PROTOCOL
DRAFT FOR STAKEHOLDER COMMENTS JANUARY, 2025 New Delhi, India

HMPV IVD Performance Evaluation Protocols ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 1 | Page

Human Metapneumovirus Real Time PCR Performance Evaluation Protocol 6 Table of Contents 7 S.No. Content Page Number 1.
Performance evaluation protocol for Human Metapneumovirus real- time PCR kit 2 2.
Information on Operational and Test Performance Characteristics Required from Manufacturers 12

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 2 | Page

Performance evaluation protocol for Human Metapneumovirus real-time PCR kit 31 I. Background:
32 CDSCO and ICMR, New Delhi, have aimed at facilitating the availability of Quality-Assured 33 Diagnostics kits appropriate for use in India. Hence the following guidelines shall establish the 34 uniformity in performance evaluation of in-vitro diagnostic kits (IVD). The performance 35 evaluation is to independently verify the manufacturer’s claim regarding in-vitro diagnostic kit 36 (IVD) performance. 37 This recommendation focuses on the laboratory performance evaluation of Human 38 Metapneumovirus (hMPV) virus real time PCR kit. All clinical samples tested in the study should 39 be evaluated in accordance with the candidate test’s instructions for use.
40

41 II. Purpose: 42 To evaluate the performance characteristics of hMPV real-time PCR kits in the diagnosis of hMPV 43 infection/ disease. 44 III. Requirements:
45

1. Supply of kits under evaluation (Along with batch/lot No. Expiry & required details). If the 46 kit to be evaluated works in a closed system format, the manufacturer needs to supply 47 the required equipment. 48
2. Evaluation sites/laboratories (With required equipment) 49
3. Reference test kits 50
4. Characterised Evaluation panel 51
5. Laboratory supplies
   52 IV. Ethical approvals:
   53 Exempted from Ethics approval as per ICMR’s Guidance on Ethical Requirements for Laboratory 54 Validation Testing, 2024. A self-declaration form as provided in ICMR guidelines to be submitted 55 by the investigators to the institutional authorities and ethics committee for information. 56 V. Procedure: 57
6. Study design/type: Diagnostic accuracy study using clinical/spiked samples
   58
7. Preparation of Evaluation sites/laboratories:
   59 Identified IVD kit evaluation laboratories should be well-equipped and establish their 60 proficiency through ALL of the following: 61

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 3 | Page

A. Accreditation from NABL for at least one of the Quality management systems for at least one 62 respiratory viral pathogen molecular testing (NABL accreditation for testing Lab / calibration 63 lab as per ISO/IES 17025, Medical Lab as per ISO 15189, PT provider as per ISO/IEC 17043), or 64 CDSCO approved Reference laboratory.
65 B. Staff training: All the staff involved in hMPV virus IVD evaluation should undergo hands-on 66 training and competency testing on following
67 ⮚ Preparation & characterization of reference sample panel (at least 2 staff) 68 ⮚ Handling of hMPV RT-PCR kits received for performance evaluation 69 (Verification/Storage/Unpacking etc). 70 ⮚ Testing, interpreting, recording of results & reporting 71 ⮚ Data handling, data safety & confidentiality 72 3. Preparation of hMPV RNA evaluation panel 73 A well characterised panel of hMPV positive human samples is a critical requirement for 74 evaluation of these RT-PCR IVD kits. A statistically significant number of clinical samples should 75 be used for the evaluation. 76 The sample type for hMPV detection is nasopharyngeal/oropharyngeal swab. If a kit claims to 77 detect hMPV across several sample types, attempt should be made to evaluate the assay across 78 all the sample types. In case all the sample types mentioned in the IFU are not available with the 79 lab, the performance evaluation report should clearly mention the sample type against which the 80 kit is evaluated. There should be no ambiguity about the type of sample used for evaluation. 81 4. RNA extraction 82 RNA extraction should be performed using standard techniques. If the manufacturer of the index 83 test recommends a specific RNA extraction kit, the same needs to be provided by the 84 manufacturer if the evaluation lab is unable to procure the same.
85 5. Real-Time PCR System 86 PCR should be performed using IVD-approved machines. If any equipment(s) is specified in the 87 IFU of the index test, it should be used for the evaluation, and it should be provided by the 88 manufacturer if not available within the lab’s IVD evaluation scope.
89 Real-time closed systems/devices awaiting evaluation should be provided by the manufacturer 90 along with all necessary components, supplies and reagents. 91 6. Internal control/Extraction control 92 The index test must have an internal control (housekeeping gene), with or without an extraction 93 control (RNA added before extraction to a sample). 94

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 4 | Page

7. Reference assay:
   95 FDA approved real-time PCR assay/ ICMR-NIV Pune in-house Real Time PCR Assay should be used 96 as the Reference Assay.
   97 All positive samples should be confirmed positive by the reference assay. 98 All negative samples should be confirmed negative by the reference assay. 99

100 8. Sample size for performance evaluation: Sample size is calculated assuming 95% 101 sensitivity and specificity of the index test, 95% confidence level, absolute precision of 5% and 102 ≤5% invalid test rate. A minimum of 77 (rounded to 80) positive clinical samples and a minimum 103 of 77 (rounded to 80) negative clinical samples are required for performance evaluation. 104 However, for negative samples, a minimum of 115 specimens are suggested to account for a 105 rigorous cross reactivity panel. 106

107

108 9. Sample panel composition: 109 A. Human samples 110 A.1 Positive samples (n=80): Clinical samples positive by the reference real-time PCR 111 assay
112 A.1.1 Strong positive (Ct value <25) = 20 samples
113 A.1.2. Moderate positive (Ct value between 25-30) = 40 samples 114 A.1.3 Weak positive (Ct value >30-35) = 20 samples
115 Note: 116 If possible, attempt should be made to include all lineages of hMPV in the positive sample panel. 117

118 A.2 Negative samples (n=115): All negative samples should be negative by reference real- 119 time PCR assay. Distribution of the negative samples should be as follows: 120 A.2.1 NP/OP swab from individuals with respiratory infection that are negative for hMPV 121 RNA = 30 samples 122 A.2.2 NP/OP swab from apparently healthy individuals with no respiratory symptoms = 123 20 samples 124 A.2.3 Cross reactivity panel (Table 1): Samples negative for hMPV RNA but positive for 125 other common respiratory viruses = 65 samples
126

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 5 | Page

Table 1: Cross reactivity panel for performance evaluation of HMPV real time PCR kit 127 S.N. Pathogen Minimum no. of positive samples needed (n=65) Additional comments i.

RSV A 5 In case adequate number of one RSV type is unavailable, supplement with the available RSV type ii.

RSV B 5 iii.

Measles 5

iv.

Mumps 5 Buccal swab is the preferred sample type for Mumps, and the same (or throat swab) should be used for evaluation v.

Seasonal Influenza A (H1N1pdm09 and H3N2) 10 (5 of each)

vi.

Seasonal Influenza B (Victoria, with/without Yamagata) 5

vii.

SARS-CoV-2 5

viii.

Respiratory Adenovirus
5 Representation from all respiratory types is desirable ix.

Human Respiroviruses 1 and 3, Human Rubulaviruses 2 and 4
5 Representation from all types is desirable x.

Rhinovirus 5 In case samples available with the lab are not typed into Rhinovirus and non-Rhinovirus Enteroviruses, please use 10 such samples to represent these 2 pathogens xi.

Enterovirus 5

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 6 | Page

xii.

Seasonal coronaviruses 3 OC43 AND 229E
xiii.

Cytomegalovirus 2 Lower respiratory specimen positive for CMV is acceptable

128 If available, samples positive for relevant bacterial pathogens and other relevant viruses 129 (with which majority of the population is likely to be infected), should also be included in 130 the cross-reactivity panel.
131

132 10. Evaluation method: 133 The index test and the reference assay should be run simultaneously on the sample panel, 134 and results should be recorded.
135 11. Test reproducibility 136 A. Sample size for lot-to-lot reproducibility 137 Three lots of an assay should be evaluated. Sample size for lot-to-lot reproducibility should 138 be as follows:
139  First lot of the assay: should be tested on statistically significant number of positive 140 and negative samples as calculated in the protocol.
141  Second lot of the assay: should be tested on 25 samples (15 positive samples 142 comprising 10 low positive AND 5 moderate/high positive samples, and 10 negative 143 samples).
144  Third lot of the assay: should be tested on 25 samples (15 positive samples comprising 145 10 low positive AND 5 moderate/high positive samples, and 10 negative samples). 146  There should be no lot-to-lot variation. 147

148 Refer the flowchart below (Fig. 1): 149

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 7 | Page

150

151

152 12. Blinding of laboratory staff 153 To ensure rigor of the evaluation process, laboratory staff performing the evaluation 154 should be blinded to the status of the clinical samples. The PI of the evaluation exercise 155 should remain unblinded, i.e., privy to the status of the samples. Another senior 156 laboratory staff selected by the PI may remain unblinded and carry out coding of samples 157 and dispensing them into similar-looking vials to be used for testing, and maintaining the 158 database of results. Staff performing the reference test and the test under evaluation, 159 interpretation of the test result, and entering the results against the coded samples in the 160 database, should remain blinded to the status of samples till the completion of evaluation. 161 The data should be analyzed only by the PI of the evaluating lab. Refer to Fig. 2. 162

163 Fig.2: Blinding in evaluation exercise 164

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 8 | Page

165

166

167 13. Acceptance Criteria 168 Expected sensitivity: ≥95% 169 Expected specificity: ≥98% 170 Cross reactivity with other viruses as outlined in the negative sample panel: Nil 171 Invalid test rate: ≤5% 172

173 13. Publication Rights: 174 The PI(s) of the evaluating labs shall retain publication rights of the evaluation as lead author(s). 175

176 After following due procedure as defined in this document, once any kit is found to be Not of 177 Standard Quality, thereafter, no request for repeat testing of the same kit will be acceptable. 178 Any request of re-validation from the same manufacturer for the same test type will only be 179 entertained if valid proof of change in the kit composition is submitted. 180

181

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 9 | Page

VI. References: 182

1. U.S. Food and Drug Administration: Testing for Human Metapneumovirus (hMPV) Using Nucleic Acid 183 Assays - Class II Special Controls Guidance for Industry and FDA Staff. 2009. Available at: 184 https://www.fda.gov/medical-devices/guidance-documents-medical-devices-and-radiation-emitting- 185 products/testing-human-metapneumovirus-hmpv-using-nucleic-acid-assays-class-ii-special-controls- 186 guidance#3 [Accessed on January 11, 2025] 187
2. Amarasinghe, G.K., Ayllón, M.A., Bào, Y. et al. Taxonomy of the order Mononegavirales: update 188
3. Arch Virol 164, 1967–1980 (2019). https://doi.org/10.1007/s00705-019-04247-4 189

190 VII. Performance evaluation report format 191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 10 | Page

PERFORMANCE EVALUATION REPORT FOR HUMAN METAPNEUMOVIRUS (HMPV) REAL-TIME 211 PCR KITS 212 Name of the product (Brand /generic) Name and address of the legal manufacturer Name and address of the actual manufacturing site Name and address of the Importer Name of supplier: Manufacturer/Importer/Port office of CDSCO/State licensing Authority Lot No / Batch No.: Product Reference No/ Catalogue No Type of Assay Kit components Manufacturing Date Expiry Date Pack size (Number of tests per kit) Intended Use Number of Tests Received Regulatory Approval: Import license / Manufacturing license/ Test license License Number: Issue date:

Valid Up to: Application No. Sample Panel Positive samples (provide details: clinical/spiked, strong, moderate, weak)
Negative samples (provide details (clinical/spiked), including cross reactivity panel)

213 Results 214

Reference assay ……….……………… (name)

Positive Negative Total Name of HMPV virus real-time PCR Positive

Negative

Total

215

Estimate (%) 95% CI Sensitivity

Specificity

216 ● Details of cross reactivity with other viruses: 217 ● Conclusions: 218

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 11 | Page

o Sensitivity, specificity 219 o Performance: Satisfactory / Not satisfactory 220 (Sensitivity and specificity have been assessed in controlled lab setting using kits provided by the manufacturer from 221 the batch mentioned above using ….. sample. Results should not be extrapolated to other sample types.) 222 Disclaimers 223

1. This validation process does not approve / disapprove the kit design 224
2. This validation process does not certify user friendliness of the kit / assay 225 Note:
   226 This report is exclusively for Human Metapneumovirus………….. Kit (Lot No……) manufactured by …………… 227 (supplied by ……….) 228 The kit has been validated against the pathogen (as a whole) with statistically significant sample size, and 229 NOT against different lineages of the pathogen. 230 Evaluation Done on …………………… 231 Evaluation Done by …………………………. 232 Signature of Director/ Director-In-charge …………………… Seal …………………………. 233 ****End of the Report 234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 12 | Page

Annexure-1: Information on Operational and Test Performance Characteristics Required from 249 Manufacturers 250 The manufacturer should provide the following details about the IVD: 251

1. Instructions for Use 252
2. Scope of the IVD: to diagnose hMPV. 253
3. Intended Use Statement 254
4. Principle of the assay 255
5. Intended testing population (cases of ARI/ILI/SARI) 256
6. Intended user (laboratory professional and/or health care worker at point-of-care) 257
7. Lot/batch No. 258
8. Date of manufacture 259
9. Date of Expiry 260
10. Information on operational Characteristics 261 i. Configuration of the kit/device 262 ii. Requirement of any additional equipment, device 263 iii. Requirement of any additional reagents 264 iv. Operation conditions 265 v. Storage and stability before and after opening 266 vi. Internal control provided or not 267 vii. Quality control and batch testing data 268 viii. Biosafety aspects- waste disposal requirements 269
11. Information on Test Performance Characteristics 270 i. Type of sample-NP/OP swab, other respiratory specimen 271 ii. Volume of sample 272 iii. Any specific sample NOT to be tested 273 iv. Any additional sample processing required 274

ICMR-CDSCO/IVD/GD/PROTOCOLS/03/2024 13 | Page

v. Any additional device/consumable like sample transfer device, pipette, tube, etc required 275 vi. Name of analyte to be detected 276 vii. Pathogens targeted by the kit 277 viii. Time taken for testing 278 ix. Time for result reading and interpretation 279 x. Manual or automated(equipment)reading 280 xi. Limit of detection 281 xii. Diagnostic sensitivity 282 xiii. Diagnostic specificity 283 xiv. Stability and reproducibility 284 xv. Training required for testing 285 xvi. If yes, duration 286 xvii. Details of Cut-off and /or Equivocal Zone for interpretation of test 287 xviii. Interpretation of invalid and indeterminate results to be provided 288 xix. It is recommended to provide data demonstrating the precision 289

290 *Please mention “Not applicable” against sections not pertaining to the kit. 291

292

293

294

295

296

297

298

299