Literature Search, SOTA Review process and Clinical Evaluation

A literature search & review is a vital component of the Clinical Evaluation of medical devices. It is also challenging to perform and is an aspect of Clinical Evaluation that can be a source of confusion, difficulty and non-conformity.

In this article, we help to demystify the process of systematic review of literature for Clinical Evaluation.

What is MedDev 2.7/1 rev 4?

MedDev 2.7/1 rev 4 is one of a series of guidance documents published by the European Commission intended to assist medical device manufacturers and Notified Bodies in meeting medical device regulatory obligations. MedDev stands for MEDical DEVices documents. MedDev 2.7/1 rev 4 is a detailed document which sets out comprehensive guidelines for medical device manufacturers and notified bodies for the conduct of Clinical Evaluation. MedDev 2.7/1 rev 4 is a technical document which can be challenging to digest.

Appendix 5 of the MedDev guideline sets out requirements for the identification and review of literature in the context of Clinical Evaluation. It is therefore central to performing a compliant literature review.

What is the role of a literature review in medical device Clinical Evaluation?

The review of published literature is a critical component of medical device Clinical Evaluation. A literature review is performed to inform two major elements of a Clinical Evaluation:

• A review of the current knowledge/ the state of the art (SOTA) – this enables the identification of safety and performance benchmarks against which performance of the subject (and, if relevant, equivalent) device may be evaluated
• A review of literature relating to the subject (and equivalent) device.

It is a requirement that literature searches are performed according to a documented protocol and that they include both favourable and unfavourable data. They must be representative and not be biased or selective; therefore, methods must be set out clearly for inclusion, exclusion, appraisal and analysis of sources.

What guidance is offered by MedDev 2.7/1 rev 4 Appendix 5?

Appendix 5 of MedDev 2.7/1 rev 4 provides a range of guidance for Clinical Evaluation literature searches & reviews, including:

• Offering general instructions on literature searching
• Outlining methods for constructing protocols for undertaking a literature review
• Defining the key elements of literature reviews that are required in order to deliver a compliant Clinical Evaluation Report.

Understanding the literature search & review process explained in Appendix 5 can be made easier by dividing it into four stages:

Stage 1: Background to the literature search

Background to the literature search stage sets the scene for the substantive literature search, review, appraisal and report to follow. Performed properly, a rationale for performing the literature review should be clearly documented.

The background to the literature search should include information such as:

1. Name of the device/model.
2. Importance of literature review in the risk management process. The review should provide data on current therapies, diagnostic measures, and measures for the management of medical conditions for the targeted patient population. This will help in order to assess the acceptable benefits/risks.
3. Outcomes of previous literature reviews and literature searches conducted by manufacturer.
4. Identification of any equivalent or benchmark devices to be included in the evaluation, including the name and model of these devices.

Carefully formulated research question(s) should be set out to guide the selection of search terms. Selected search terms will then be subjected to an objective, non-biased, systematic search and review. The inputs required for this section are the device description and the expected device performance; these can be applied for the formation of relevant research questions. Methods or protocols adopted for the conduct of literature review should set out the search methodology, sources of data, selection & exclusion criteria and appraisal/critique methodology.

Stage 2: Literature Screen

The next step is to screen initial results for relevance to the research questions. This stage can be thought of as a ‘triage’ of sources, with the objective being to limit full appraisal to only those results that are pertinent.

MedDev 2.7/1 rev 4 requires the use of carefully formulated inclusion / exclusion criteria, augmented by an objective, non-biased, systematic screening and review process such as PICO (patient characteristics, type of intervention, control, and outcome queries), or PRISMA (The Preferred Reporting Items for Systematic Reviews and Meta-Analyses). Whatever method is used, it should align with the method specified in the search protocol (or review protocol) developed prior to commencing the review.

In a practical sense, begin by reviewing the titles and abstracts – the full text is not required at this stage. Due to the number of results often found by search terms, screening needs to be efficient while maintaining appropriate scrutiny. It may be helpful to use exclusion codes to codify reasons for exclusions (which may include, for example, “unrelated topic” or “article not available in English” etc).

Remember — this stage is all about identifying relevant sources.

Stage 3: Appraisal & Analysis

Before considering how to perform appraisal and analysis, it’s important to understand the difference between them:

• Appraisal means to assess the quality of a source, methodological quality, scientific validity, reliability, relevance and weighting to the evaluation.
• Analysis is the post-appraisal stage of evaluating what the results mean (or what they tell us). Appraisal and weighting should be considered during analysis.

According to MedDev 2.7/1 rev 4, an appraisal plan should form part of the search protocol. It is advisable to create an appraisal framework covering multiple domains (such as relevance, clinical significance, study type, etc) and it can be useful to assign a code to each appraisal element. As part of appraisal, a score can be applied against each appraisal domain, forming a structured and codified appraisal summary of each source and the ‘body’ of sources as a whole.

Weighting is a key concept in appraisal that’s worth considering independently. The weighting of a source indicates how much importance should be placed on the results of that source when compared to others. A high weighting indicates that the results should be deemed more relevant to the evaluation than data from a lower weighting source.

Following appraisal, the next stage is to analyse the data. The analysis will depend on the reason for conducting the literature review; in a clinical evaluation, the appraisal may consider (for example):

1. Does the data demonstrate that the device is suitable for its intended purpose?
2. Does the data demonstrate compliance with the relevant General Safety and Performance Requirements (GSPRs)?

During appraisal, ensure that an objective approach is taken and that both favourable and unfavourable data is treated equally.

Stage 4: Literature Review Report

The fourth and final stage of the literature analysis and review process is preparing a literature review report. While writing the report, especially if performed in the context of a clinical evaluation, it is important to identify quantifiable safety and performance parameters, whether in relation to the subject device or comparable alternatives.

Ensure the report includes enough information for a reviewer to understand how the search was conducted and what appraisal and analysis criteria were used. Present a summary of each source (outlining a rationale for the appraisal and weighting) in the body of the document and outline the analysis towards the end.

The bottom line

Performing a systematic literature review to the required standard is not easy. While the framework in Appendix 5 of MedDev 2.7/1 rev 4 is not without its limitations, it is useful as a starting point and in ensuring that the review has an appropriate structure without major omissions.

At Mantra Systems, our team of medical experts are on hand to support your literature search & review requirements. If you need any information or advice, please contact us for a free discussion.

Note: A shortened version of this article was previously published on 20th October, 2021.