Medical AI Compliance: Don’t Wait Until It Hurts

AI is moving fast in healthcare. But when AI is built into medical devices, one mistake can do more than damage a company’s reputation. It can affect real patients. 

For private companies developing medical devices in the European Union, the General Data Protection Regulation (GDPR), the EU Artificial Intelligence Act (AI Act), and the Medical Devices Regulation (MDR) creates a complex but critical framework for ensuring patient safety, data integrity, and ethical AI use. These regulations directly impact manufacturers, as their products often fall under the scope of all three framework.

Waiting until something goes wrong is not a compliance strategy. It's a risk to patients, to trust, and to the company's future, one manufacturers can no longer afford to take.

Why Compliance Can’t Wait

In 2020, a ransomware attack on Düsseldorf University Hospital crippled IT systems, forcing a life threatening patient to be redirected 30 kilometers away.[1] The one hour delay in treatment contributed to her death, the first known fatality linked to a cyberattack on a healthcare facility. This tragedy shows the real world cost of failing to secure medical devices.

Given these high stakes, compliance is essential for establishing trust, enabling market access, and safeguarding patient data. Research shows that:

• 77% of the public are concerned about the privacy of their medical information provided to AI tools.[2]
• 68% of physicians see value in AI tools, but 47% cite increased oversight as the most important step to build trust in AI-generated recommendations.[3]
• Trust in AI healthcare systems increases significantly when patient privacy concerns are addressed and robust security measures are in place.[4]

Non-compliance also carries severe risks: fines up to 35 million or 7% of global annual revenue under the AI Act, 20 million or 4% under GDPR, or product recalls and market access suspensions under MDR. Beyond financial penalties, issues like bias in diagnostics or data leaks can lead to misdiagnoses and reputational harm for medical manufacturers.

Navigating the Three Key Regulations

1.    GDPR: Guarding Patient Data

The GDPR classifies health data as special category data under Article 9, attracting the highest level of protection. Processing health data requires both a lawful basis under Article 6 and an additional condition under Article 9, such as necessity for medical diagnosis or explicit consent. For medical device manufacturers, this applies to any personal data processed by their devices or associated systems.

Companies must also ensure compliance with other requirements, such as data minimization, purpose limitation, and transparency, while respecting patients data subject rights and maintaining a high standard of security.

2.    AI Act: Ensuring Safe and Transparent AI

The AI Act classifies AI systems by risk level, with high risk categories such as AI-driven medical devices facing stringent obligations for safety, transparency, and human oversight. For manufacturers, this means that any AI enabled medical device will be classified as high risk.

These systems must undergo conformity assessments, maintain comprehensive technical documentation, and implement robust risk management systems to address potential hazards such as bias, misdiagnosis, or system failures. Additionally, these systems must ensure accuracy, robustness, and cybersecurity throughout their lifecycle. Furthermore, high risk AI systems must establish post market monitoring plans, maintain automatic logging for traceability, and provide clear instructions for use to ensure safe and effective deployment.

3.    MDR: Guaranteeing Device Safety and Performance

The MDR ensures that medical devices, including AI-driven tools, meet safety, performance, and traceability standards. The MDR requires classification based on risk, conformity assessments, CE marking, post market surveillance, and Unique Device Identification.

Under the MDR, software with a medical intended purpose, such as diagnosing, monitoring, or treating disease, is classified as Software as a Medical Device SaMD and must comply with the regulation. For manufacturers, this means that AI enabled devices must meet both MDR and AI Act requirements.

 Where the Rules Align and Where They Clash

 

Overlapping Requirements

1.    Risk Management

The AI Act and MDR both require risk management systems to identify, evaluate, and mitigate risks such as bias, misdiagnosis, or system failures, ensuring patient safety and compliance.

2.    Technical Documentation

The AI Act and MDR both mandate comprehensive technical documentation. Additionally, manufacturers can combine AI Act and MDR documentation into a single file, streamlining compliance efforts. This documentation must include details such as system design, training data, validation methodologies, and performance metrics.

3.    Quality Management System QMS

The AI Act requires manufacturers to implement a Quality Management System for high risk AI systems. However, manufacturers already subject to MDRs QMS requirements (e.g., ISO 13485) can integrate AI Act requirements into their existing QMS, avoiding duplication of efforts.

4.    Conformity Assessment

Both the AI Act and MDR require conformity assessments to verify compliance before placing a device on the market. Manufacturers can use the same Notified Body for both AI Act and MDR assessments, simplifying the process.

5.    Post Market Surveillance

The AI Act and MDR both require post market surveillance. Manufacturers can merge AI Act post market monitoring with MDR post market surveillance to create a unified process for tracking device performance, reporting serious incidents, and taking corrective actions.

 

 Key Tensions

1.    Data Retention vs. Minimization

The AI Act requires manufacturers to retain data for audit purposes, ensuring traceability and accountability. However, the GDPR emphasizes data minimization, requiring that only necessary data be collected and stored. Manufacturers must resolve this tension by implementing purpose specific retention schedules and using anonymization techniques where feasible.

2.    Transparency vs. Proprietary Secrets

Both the AI Act and the GDPR require transparency about how AI systems operate and make decisions. However, vendors may resist disclosing proprietary algorithms or models to protect intellectual property. Manufacturers must strike a balance by providing meaningful transparency, such as disclosing capabilities, limitations, and risks, while safeguarding trade secrets.

3.    Notified Body Shortages

A Notified Body is an independent organization designated by EU authorities to assess the conformity of products with applicable regulations. Currently, there is a shortage of designated Notified Bodies for higher risk devices, which can delay market access for manufacturers.

 Five Actions to Start Off Compliance

 

1.    Audit Your AI Inventory: Identify all AI tools in use, including those embedded in medical devices, and map them to the MDR, GDPR, and AI Act requirements.

2.    Invest in Privacy Enhancing Technologies: Use encryption and anonymization to protect sensitive patient data.

3.    Establish Cross Functional Oversight: Create teams with regulatory experts, engineers, legal advisors, and data protection officers to embed responsible AI principles into development and deployment.

4.    Conduct Regular AI Audits: Audit training data, consent tracking, and model outputs to ensure continuous compliance.

5.    Engage with Regulators and Industry: Stay ahead of regulatory changes and contribute to industry standards to position your company as a leader in compliant medical device innovation

 The Winning Formula

For medical device manufacturers, the path is clear: Make AI safe. Make compliance simple. Make trust your competitive advantage.

 How Anove Supports Medical Device Manufacturers

Navigating the complexities of GDPR, AI Act, and MDR compliance can be challenging for medical device manufacturers. Anove offers comprehensive solutions to simplify this process and help companies focus on innovation rather than administrative burdens.

With insAIght, manufacturers can:

·      Automate compliance documentation by generating reports and evidence, reducing manual workload.

·      Automatically scan for shadow AI use, identifying unauthorized or unmanaged AI tools.

·      Conduct risk assessments to identify and evaluate risks related to AI models, data privacy, and ethics, with actionable recommendations.

·      Monitor AI systems continuously for regulatory compliance, including open source AI models.

·      Ensure AI infrastructure security by managing security assets and aligning them with best practices.

·      Streamline business processes and workflows to align AI applications with organizational goals.

·      Utilize real time risk and compliance management with customizable dashboards.

·      Integrate seamlessly with existing systems via APIs for efficient compliance.

·      Define and track ownership of AI assets and associated risks for enhanced accountability.

Find out more about insAIght here: https://www.anove.ai/en/product/insaight