The FDA is asking a foundational question for generative AI medical devices: how can a regulator evaluate a product whose inputs are open ended, outputs can vary, and underlying foundation model may change? The agency has not issued a new rule. It has published a 31-page discussion paper and asked the public to help define the evidence problem.
Comments are due October 19, 2026, under docket FDA-2026-N-7874. The distinction matters. The paper explicitly says it is not draft or final guidance, does not implement policy, and does not state future submission requirements. Builders should read it as a map of unresolved regulatory questions.
Why generative AI changes device evaluation
| Device property | Evidence challenge | Builder record |
|---|---|---|
| Open-ended input | Test space can become extremely broad | Supported and prohibited input classes |
| Variable output | Same input may not produce the same answer | Repeated-run distribution and failure modes |
| Multiple subtasks | One product may cross several clinical functions | Task-level intended use and acceptance criteria |
| Third-party foundation model | Developer may lack training and evaluation detail | Model version, contract, monitoring, and fallback |
| Evolving components | Prompt, retrieval, guardrail, or model changes can shift behavior | Change log and impact assessment |
| Real-world drift | Deployment conditions differ from test conditions | Postmarket performance and incident signals |
The agency highlights confabulation, unclear intended-use boundaries, limited visibility into third-party models, and performance degradation between evaluation and deployment. These are not unique to medicine, but the consequence of a plausible wrong answer can be materially higher.
Start with intended use, not model capability
Define who uses the device, for which patient population, with which inputs, and what decision follows the output. State whether the system drafts, recommends, prioritizes, measures, or acts. Then state what it must not do.
“Answers clinical questions” is too broad to test. “Drafts a discharge summary from specified chart fields for clinician review” creates a more defensible evaluation frame. A product can use a general model while maintaining a narrow device claim.
Test distributions, not one golden response
Variable output means a single pass does not establish performance. Repeat representative cases, record the range of answers, and measure both clinical correctness and whether the system follows its role. Include adversarial wording, incomplete data, conflicting evidence, unusual populations, and requests outside intended use.
Separate model error from product error. Retrieval, prompt assembly, guardrails, user interface, and post-processing can each create or prevent harm. Keep traces sufficient to reconstruct the path without exposing unnecessary patient data.
Use an independent clinical reviewer for high-consequence cases and document disagreements. A benchmark can summarize average performance while hiding a small class of dangerous outputs. Our high-assurance AI boundary guide explains why environment, access, monitoring, and human authority must be evaluated together.
Treat the foundation model as a changing dependency
A vendor model update can alter a device even when your application code does not change. Record the exact model revision, region, safety settings, prompt template, retrieval index, and evaluation suite. Require notification for material provider changes where contracts allow it.
Build a fallback for endpoint retirement, performance regression, and loss of access. Our model retirement migration checklist is written for creative AI, but its core lesson applies here: preserve a test fixture before swapping the dependency.
Premarket evidence needs a postmarket loop
The FDA points toward a total product life-cycle approach because no premarket test can cover every open-ended input. Define what you will monitor after launch: output quality, abstention, override, escalation, subgroup performance, provider complaints, patient impact, model drift, and near misses.
Set thresholds that trigger investigation, rollback, or suspension. Do not wait for aggregate accuracy to move if a rare error is severe. A monitoring plan should name the owner, review cadence, evidence source, and action for each signal.
Twelve questions to answer before submission
- What exact medical purpose does the product claim?
- Who is the intended user?
- Which inputs are supported and prohibited?
- What human decision follows the output?
- When must the system abstain or escalate?
- How variable are repeated outputs?
- Which patient groups and environments were tested?
- What does the foundation-model provider disclose?
- Which component changes require reevaluation?
- What does the user see about uncertainty and limitations?
- Which postmarket signals trigger action?
- How can an incident be reconstructed and contained?
This is a preparation checklist, not a prediction of FDA requirements. The agency is seeking input precisely because the final approach is unresolved.
My verdict: build the evidence system before the feature list
Before the October deadline, organize feedback around a specific question in the paper. State the device context, evidence available, limitation observed, and the regulatory decision that evidence could support. Distinguish a foundation-model capability from the performance of the complete medical device. FDA is asking for practical input across risk assessment, premarket review, and postmarket monitoring, so a narrow documented example is more useful than a broad forecast about AI in medicine.
The discussion paper rewards teams that can explain the whole product rather than point at a model benchmark. Intended use, system-level evaluation, change control, and real-world monitoring must connect.
Submit comments when you have concrete evidence about where current methods work or fail. General claims that regulation helps or hurts innovation add less value than a reproducible example, dataset limitation, monitoring result, or proposed decision rule.
Read and respond to the FDA
- Read the FDA’s full discussion paper.
- Review the FDA announcement and comment deadline.
- Use docket FDA-2026-N-7874 to inspect and submit comments.
Checked September 3, 2026. The paper’s status, scope, risks, and deadline come from the FDA. The twelve-question checklist and implementation records are Musthave.ai analysis, not legal or regulatory advice.
