Performance evaluation: a guide to clinical performance

Clinical evidence for in vitro diagnostic (IVD) devices is based on the three pillars of scientific validity, analytical performance, and clinical performance. For most manufacturers, it is clear how to demonstrate scientific validity and analytical performance for their in vitro diagnostic devices (IVDs). However, demonstrating clinical performance in the performance evaluation process is a source of uncertainty for many IVD manufacturers. With a view to the EU In Vitro Diagnostic Medical Device Regulation (EU) 2017/746 (IVDR) and the MedTech Europe e-book Clinical Evidence Requirements under the EU In Vitro Diagnostics Regulation (IVDR) (Third Edition, February 2023), we show in this article what is important. We use three key questions to illustrate how you can demonstrate clinical performance for a particular product.

1 Scientific validity

Evidence of scientific validity refers to "the association of an analyte with a clinical condition or physiological state" (IVDR, Article 2 (38)). This is usually based on scientific, published literature (peer-reviewed!) on the analyte and the medical condition or disease. The use of the analyte and/or the technology should also be additionally supported by consensual expert opinions or guidelines from relevant professional organizations. What if the literature search for new analytes/markers/technologies provides no or insufficient data? In these cases, scientific validity must be demonstrated by means of own proof of concept studies and/or results of clinical performance studies.

2 Analytical performance

Analytical performance refers to "the ability of a device to correctly detect or measure a particular analyte" (IVDR, Article 2 (40)). For the proof of analytical performance, the parameters according to IVDR (Annex I, Section 9.1. a) must be demonstrated or, if not applicable, justifiably omitted. In order to demonstrate analytical performance, you must conduct your own analytical performance studies. The conduct of analytical performance studies should be guided by common specifications and/or product-specific standards, if available. Common, accepted protocols for determining analytical performance parameters are provided by the appropriate CLSI guidelines.

Make sure that you work according to the latest versions of the standards and guidelines. The Notified Bodies attach importance to the fact that your data have been generated and evaluated according to the current state of the art (SOTA).

3 Clinical performance

While the procedure up to this point is clear and there is little room for interpretation, demonstrating clinical performance is somewhat more complicated as several factors need to be considered. Clinical performance refers to "the ability of a device to yield results that are correlated with a particular clinical condition or physiological or pathological process or state in accordance with the population and intended user" (IVDR, Article 2 (41)). At first glance, the demonstration of clinical performance also appears to be clearly defined, since the IVDR, as with analytical performance, specifies parameters that must be demonstrated (Annex I. Section 9.1. b). Again, in case of non-applicability, parameters can be justifiably omitted.

However, unlike the analytical performance, it is not mandatory to conduct own clinical performance studies: "Clinical performance studies in accordance with Section 2 of Part A of Annex XIII shall be carried out unless it is duly justified to rely on other sources of clinical performance data" (IVDR, Article 56 (4)).
Other sources for demonstration of clinical performance may include:

• peer-reviewed scientific literature; and/or
• published experience gained by routine diagnostic testing.

In the following, we will highlight for you which clinical performance parameters you need to determine for your product and under which circumstances conducting you can forego conducting your own clinical performance study.

3.1 What is the clinical function of my IVD?

Take a close look at your intended purpose. What clinical function (e.g. diagnosis/aid in diagnosis or monitoring) is promised? Is this clinical function linked to a specific disease (e.g. rheumatoid arthritis) or merely to a (patho-)physiological condition (e.g. iron deficiency)? You should keep in mind that any additional claims in your marketing materials and on your website will expand the intended purpose. The precise definition of your in vitro diagnostics (IVD's) clinical function is critical for the selection of the applicable clinical performance parameters and for the study design to determine those parameters. For initial guidance on which parameters should be demonstrated in terms of the clinical function, please refer to the MedTech Europe E-Book (Third Edition, February 2023). Basically, it always depends on the individual product which parameters are appropriate. Please do not hesitate to contact us if you have any questions.

Figure 1: Clinical performance (Source: Clinical Evidence Requirements under the EU In Vitro Diagnostics Regulation (IVDR); Third Edition, published in February 2023; page 25)

3.2 What is the risk class of my IVD?

The strength and robustness of clinical evidence should follow a risk-based approach:
Class A < Class B < Class C < Class D

The risk class is critical to the level of evidence of clinical performance, or in other words, as the risk class increases, so do the requirements for clinical performance data. The higher the risk class, the more likely you are to need a clinical performance study for your product. Conversely, MDCG 2022-2 even acknowledges that for some Class A devices, clinical performance data are not required at all:

"The IVDR sets out that clinical performance may not be required for certain devices in Article 2 (39). For example, clinical performance data may not be expected for non-sterile specimen receptacles, microscopy glass slides, or some general reagents. In such cases and where due justification is given, a clinical performance report would not be expected. Nevertheless, the remaining aspects of the performance evaluation report including other elements of clinical evidence would still be required unless the justification is given."

3.3 Is my IVD already established on the market?

Another important factor that influences the type and scope of proof for clinical performance is the novelty of the in vitro diagnostic device (IVD):

• Is your product established and/or already authorized on the market under the In Vitro Diagnostic Device Directive (IVDD, replaced by IVDR) (= legacy device)?
• Is the product to be placed on the market for the first time?
• Is the technology behind the product new?
• Are there already many comparable products on the market?

The Global Harmonization Task Force (GHTF) defines established in vitro diagnostic devices (IVDs) in its publication GHTF/SG5/N7:2012 entitled "Clinical Evidence for IVD medical devices - Scientific Validity Determination and Performance Evaluation" (from 2012!) as tests for which there are clinical guidelines or consensus on use, and also more than one commercially available test. If your IVD is an established test, the next question is whether it is also standardized. If so, an international standard or accepted reference material for the analyte is available which needs to be used to adjust the test (see also GHTF/SG5/N7:2012 and MedTech E-Book Third Edition, February 2023). The GHTF emphasizes that established and standardized tests must provide comparable results for the analyte, regardless of the method or manufacturer.

If your in vitro diagnostic device (IVD) is neither established nor standardized, your test falls into the "novel tests" category. Novel tests, according to GHTF/SG5/N7:2012, include:

• a new analyte,
• a new technology,
• a new application of an established technology,
• a new target group, or
• a new intended purpose.

4 Evidence of clinical performance: New versus legacy products

If your product is novel, you must conduct your own prospective and/or retrospective clinical performance study to demonstrate clinical performance or at least the "novel" part of clinical performance (e.g. extension of indication). Likewise, for products in the highest risk classes (Class C and D), clinical performance studies should always be considered. Your own clinical performance study is the most valuable source for demonstration of clinical performance, as the study design can be precisely adjusted to the parameters to be determined.

If your product is already on the market, data and results on the clinical performance of the product from peer-reviewed literature can also be used.

Caution: Here it must be analyzed exactly whether the study design, the study conduct, the number of subjects included, the methodology, the data analysis, etc. are suitable or allow the use of the data for the proof of the clinical performance of the product. These published studies are usually conducted and published independently of the manufacturer. For this reason, they do not always conform to the optimal study design required by the manufacturer to demonstrate clinical performance. Although these studies may not always cover all the parameters you need to demonstrate for your IVD, they can be used to support and/or substantiate clinical performance.

For products that have been on the market for some time or were already on the market under IVDD (= legacy device; legacy product), it is quite common to support own study data generated under IVDD with data from peer-reviewed literature. This means that no new clinical performance study needs to be conducted under the IVDR. The prerequisite for using such "other" study data is that the study design, the conduct, and the number of samples still correspond to the state of the art (SOTA) and thus meet the quality requirements.

It depends on the data situation, whether you conduct clinical performance studies yourself or rely on a mixture of your own old data and published, peer-reviewed literature on your legacy device. Start with a gap analysis of your existing clinical data. If you find gaps, you can set up a well-structured and comprehensive literature search with well-considered search terms. This will help you decide whether you can close your gaps via the published literature, or show you which aspects of clinical relevance (e.g. indications, target groups) are not yet sufficiently covered. For aspects not covered, you would have to generate your own data.

5 Evidence of clinical performance by method comparison

If your in vitro diagnostic device (IVD) is an established test, you do not necessarily need to conduct a prospective or retrospective performance study. You can also perform a method comparison between your product and a CE-marked reference product to demonstrate clinical performance. This applies to both high and low-risk classes. It is essential that the reference method is state-of-the-art (SOTA) and that the clinical performance of the reference method, including applicable clinical performance parameters, is presented in the performance evaluation. The clinical performance of the reference product should be known and published accordingly (e.g. through the Instructions for Use; IFU).

5.1 Beware of equivalence considerations for in vitro diagnostic devices!

According to the authors of the MedTech E-Book (Third Edition, February 2023), if your IVD is not only established but also standardized, it may be sufficient to demonstrate the clinical performance parameters via the literature or by means of published data on a standardized reference product. This would apply insofar as the determination of the analytical performance of your in vitro diagnostic device (IVD) has been performed using a standardized product and reference material.

Our recommendation: Always carry out a method comparison with a CE-marked reference product. So far, we are not aware of any case in which proof of clinical performance was provided exclusively via the literature on a reference product AND this was accepted by Notified Bodies.
As a general rule, proof purely via the equivalence route should be avoided. Feedback and exchanges with Notified Bodies have shown us that equivalence is very difficult to prove for IVDs and is therefore almost never accepted.

5.2 Hint for conducting method comparisons for clinical performance evidence.

For retrospective clinical performance studies, it is very important that the samples you use reflect the target population as stated in the purpose statement. Example: If an IVD is used to diagnose iron deficiency in adults, samples must be used from adults where it is clear whether iron deficiency was present. Linking samples to clinical information is fundamental and ultimately distinguishes analytical from clinical performance studies. For analytical performance studies, sample material can also be purchased, e.g. from biobanks, in order to determine the accuracy of one's own in vitro diagnostic (IVD) in comparison with a reference method. This is an analytical method comparison in which, for example, the coefficient of determination is determined.

In the case of a method comparison within the framework of clinical performance demonstration, in which one's own IVD is compared with a CE-marked reference method, the positive and negative agreement (PNA) between two in vitro diagnostics (IVD) should be determined at the end of the study, for example. Only with this determination is the link to the clinical information in accordance with the intended purpose given.

Conclusion

To determine how to provide clinical evidence for your in vitro diagnostic device (IVD), first, look closely at the clinical function in the intended use to find out which clinical performance parameters are applicable to your IVD. Based on the risk class and market establishment, as well as any standardization, determine whether direct proof (proof of clinical performance parameters with data on your own product) of clinical performance is required or whether indirect proof is sufficient (e.g. method comparison and presentation of the applicable clinical performance parameters of the CE-marked reference method).

Our IVD experts are happy to support you in all questions related to performance evaluation. Our support can be limited to the development of a strategy or also include concrete implementation. We are also happy to answer questions outside Clinical Affairs – arrange your free initial consultation now!

Best Regards
Dr. Ulrike Kolbe und Dr. Sandra Reuter

Dr. Ulrike Kolbe	Dr. Sandra Reuter
Expert Regulatory Affairs    & Clinical Affairs	IVD Expert Regulatory Affairs & Technical Documentation
ulrike.kolbe@metecon.de	sandra.reuter@metecon.de