SaMD Clinical Evidence Package: A Single-File Map

What Clinical Evidence Is and Is Not

The most common framing mistake medical-AI teams make is to mistake a performance metric for evidence. "AUC 0.94, sensitivity 92 percent" is a metric; it is not evidence. Evidence is a traceable and auditable chain showing that a claim holds in a specific context. The chain starts with the intended use statement (for whom is the device, in which clinical decision, what does it claim), continues with a study design, and closes with a document set.

The difference between metric and evidence is, in practice, this: of two models with the same AUC value, one can be usable and the other unusable. The difference is in which population the metric was produced, against which reference standard, and at a threshold chosen in advance or after the fact. The regulator and the notified body ask exactly this: not the number, but how and where the number was produced. That is why an evidence package is not a sum of individual metrics but a coherent map running from claim to document.

The Intended Use Statement: The Root Document of the Entire Package

The root document of the evidence package is the intended use (intended use / intended purpose) statement. This statement fixes the clinical claim of the device, the target population, the clinical setting, and the user profile in a single sentence. The reason it is the root document is that this sentence determines both the classification and the entire evidence requirement.

Under EU MDR Annex VIII Rule 11, software that provides information used for diagnostic or therapeutic decisions is, as a rule, at least Class IIa. If the information can lead to death or irreversible deterioration it rises to Class III, and if it can lead to serious deterioration or surgical intervention it rises to Class IIb. The principal guidance for qualification and classification decisions, clarifying modular software, predictive function, and borderline cases, is MDCG 2019-11 (European Commission, 2019). The intended use statement is therefore not marketing copy but a lever: if you claim too much, your class and your evidence burden rise; if you claim too little, your product is confined to a box that cannot be advertised. The sentence must be frozen before data collection begins, because every claim that changes afterward invalidates the evidence behind it.

The Evidence Map: Which Document Closes Which Reader-Question

The evidence package has different readers (the regulator, the notified body, the hospital procurement committee), and each asks a different question. The matrix below shows which question the core documents for a Class IIa/IIb SaMD close.

The value of this map is that it shows the whole instead of solving the pieces one by one. If a team performs the standalone performance study perfectly but skips the scientific validity file, the evidence package remains incomplete; the regulator asks, "the number is good, but where did you show the link between this output and the clinical condition?"

Standalone Performance or Clinical Effect: The Evidence Layers

Evidence can be built at two different levels, and confusing the two is a common mistake.

The first is standalone performance: how accurately the software produces output, independent of a human user, against a predefined reference standard. Here, three design rules keep the evidence standing:

• The reference standard (ground truth) must be predefined and blinded to the index test. A gold standard such as multi-reader consensus or histopathology is preferably used. Ground-truth quality is the ceiling of the entire performance claim: a poor reference refutes a good model claim.
• External/independent validation is mandatory. Performance must be reported on a test set fully separate from training; from a different center, a different device, and a different population. Internal (split) validation alone does not substitute for clinical validation. Patient-level data leakage and selection bias are also controlled for.
• The analysis plan must be locked in advance. The primary endpoint, the acceptance threshold, subgroups, and the sample size calculation are frozen before data lock and analysis. Post-hoc threshold selection invalidates the evidence.

The second is clinical effect: whether the software actually achieves the clinical purpose in the intended use context. For AI-assisted reading claims, the gold-standard design is the fully-crossed MRMC (Multi-Reader Multi-Case) study, which models reader and case variability together: the same readers read with and without the software, and the difference is measured via ROC/AUC. This distinction is critical because a model with high standalone accuracy may not produce benefit in the clinical workflow; whichever your claim is, your evidence must be built at that layer.

One Study, Three Regions: TITCK, EU MDR, and FDA

The most expensive mistake is to design three separate studies for the same product. A single, correctly designed standalone/clinical study satisfies most of the overlapping expectations of the three regions, because they all rest on the same IMDRF triad.

• Turkey (TITCK): Turkey does not publish a separate SaMD/CDSS guidance. TITCK transposes EU MDR 2017/745 and the UTS infrastructure into national legislation directly. Therefore the binding reference for TR is the EU framework, and evidence prepared for a TR application can be reused for EU MDR as well.
• EU MDR: Clinical evidence is presented within the CER using the MDCG 2020-1 (European Commission, 2020) methodology. The CER structure runs from scope and the clinical evaluation plan, to identification and weighting (appraisal) of relevant data, to data synthesis and benefit-risk, to the writing of the final report. Under MDR, clinical evaluation is not one-off but continuous through the PMS/PMCF cycle.
• FDA: SaMD mostly proceeds via the 510(k) or De Novo pathway; the clinical-validation expectation is again aligned with the IMDRF triad. The most important life-cycle tool on the FDA side is the PCCP (Predetermined Change Control Plan), which allows the model to be changed at each update without requiring a new submission.

Where they diverge is mainly life-cycle management: FDA's PCCP final guidance (FDA, December 2024) requires three components (change description, change protocol, impact assessment); the EU counterpart is continuous clinical evaluation and the PMCF cycle. If you plan for this difference from the outset, one study feeds three files.

How Reporting Guidelines Make Evidence Acceptable

A good study can be rejected when reported poorly. Reporting guidelines ensure that the evidence is assessable by the auditor and the reviewer; which one you use depends on the study type.

• If you developed a diagnostic/prognostic prediction model: TRIPOD+AI (BMJ, 2024) is the current standard for reporting regression or machine-learning-based clinical prediction models.
• If you conducted an AI-based diagnostic-accuracy study: STARD-AI (Nature Medicine, 2025) is the definitive reporting guideline for artificial-intelligence-centered diagnostic accuracy studies.
• If you conducted an early-stage, live-clinical-setting human-AI evaluation: DECIDE-AI (Nature Medicine, 2022) is used for the early clinical evaluation of decision-support systems.
• If you are running a randomized controlled trial involving an AI intervention: CONSORT-AI (Nature Medicine, 2020) for the report, and SPIRIT-AI (Nature Medicine, 2020) for the protocol.

Conformance to a guideline is not a formality: a reviewer or auditor reads an under-reported item as "no evidence." If the calibration, subgroup analysis, and how the reference standard was established are not in the report, the study, even if performed, counts as unshown.

Where Reviewers and Auditors Reject Most

In independent assessment, the same gaps return again and again. Going through these six items before submitting the evidence package prevents the most expensive delays.

1. No external validation, only split validation. Internal validation does not substitute for clinical validation; an independent center/population is essential.
2. The threshold or endpoint was chosen after the fact. Post-hoc threshold selection is the most common cause of invalidation; the analysis plan must be locked before data lock.
3. The reference standard is weak or not blinded to the index test. Ground-truth quality is the ceiling of the entire claim.
4. The intended use does not match the study population. The device claims for one population but was validated in another.
5. The scientific validity pillar was skipped. There is a performance number but the link between the output and the clinical condition is not documented.
6. No PMCF/PMS plan. Because clinical evaluation is continuous under MDR, a file without a field-monitoring plan is considered incomplete.

In-House or External Methodologist: The Delegable Pieces

Not every piece of the evidence package is equally delegable. The intended use statement and the clinical claim are inevitably the team's own decision (clinical and product ownership); these cannot be handed off externally. In contrast, locking the analysis plan in advance, the statistical design of standalone performance, the construction of the external validation set, the reference-standard protocol, calibration and subgroup analyses, and reporting-guideline conformance are the pieces that can be delegated to an independent methodologist, and that produce the most value when they are. Independence itself is a quality of the evidence: an analysis plan locked by a party with an interest in the product and one locked by an independent party carry different weight in the auditor's eyes, even if they are the same document.

From Pilot Data to Regulatory Evidence: When the Study Begins

Pre-clinical pilot data is valuable but is not, on its own, regulatory evidence. The pilot is used to clarify the intended use, to try out the reference standard, and to produce a preliminary estimate for sample size. Pilot data becomes regulatory evidence only when it turns into a study conducted with a pre-locked analysis plan, an independent test set, and pre-defined endpoints. The practical threshold is clear: if you are choosing the threshold and the endpoint after seeing the data, you are still in the pilot; if you freeze both before seeing the data and test them on an independent set, you are in the study.

This distinction is the summary of the entire evidence package. Evidence is not about collecting good numbers; it is about fixing the claim first and then testing it independently.

To independently assess which parts of your SaMD evidence package are file-ready, which pillars (scientific validity, analytical validation, clinical validation) remain missing, and how you can satisfy three regions with a single study, you can request a 15-minute free scoping call.

Related Posts

• Is Our SaMD Class IIa or IIb? Software as a Medical Device Classification
• How to Design a SaMD Clinical Validation Study?
• Clinical AI Validation Pipeline: TRIPOD and DECIDE-AI
• How to Write the Statistics Section of the Clinical Evaluation Report (CER)
• How to Prepare a TITCK CDSS Clinical Validation Report?
• EU AI Act Article 10: Data Governance and Representativeness in Medical-AI
• RWE (Real-World Evidence) Medical-AI Protocol