European Commission Guidelines on Prohibited Artificial Intelligence Practices

Dr Clare Dixon
An illustration of a brain-shaped object on an abstract background.

The Regulation (EU) 2024/1689 artificial intelligence (AI) Act aims to promote innovation and uptake of AI while ensuring high protection of health, safety, and fundamental rights. It classifies AI systems into four risk categories which are unacceptable risk, high risk, transparency risk and minimal to no risk.

On 4th February 2025, the European Commission issued guidelines on prohibited AI practices established by regulation (EU) 2024/1689 (AI Act). The guidelines are important to organisations and businesses with AI systems in the EU and detail which AI practices are unacceptable.

A summary of the eight prohibited AI practices, examples of AI systems to which they apply and systems that are out of scope are listed below.

Prohibited AI practices:

1. Harmful Manipulation and Deception - Article 5 (1)(a):

AI systems that employ subliminal, manipulative, or deceptive techniques that distort behaviour and cause significant harm are prohibited.

  • Subliminal techniques: Examples include AI systems that use visual and auditory subliminal messages, subvisual and subaudible cueing and embedded images that can unconsciously influence users.
  • Manipulative techniques: Examples include AI systems that purposefully manipulate users through background audio or images to induce mood changes such as increased anxiety or mental suffering resulting in significant harm.
  • Deceptive techniques: Examples include an AI chatbot that impersonates a friend or relative with a synthetic voice leading to scams and significant harm.

2. Harmful exploitation of vulnerabilities - Article 5(1)(b):

AI systems that exploit vulnerabilities due to age, disability, or socio-economic situations resulting in distortion of behaviour and significant harm are prohibited.

  • An example is an AI powered toy that encourages children to complete increasingly risky challenges for digital rewards and virtual praise, potentially leading to dangerous behaviour and physical harm.
  • AI systems that use lawful persuasion rather than manipulation and that are not likely to cause significant harm are outside of the scope of Article 5(1)(a) and (b).
    • For example, an AI system that uses personalised recommendations based on transparent algorithms and user preferences engages in lawful persuasion.
    • For example, a therapeutic chatbot that uses subliminal techniques to guide users towards a healthier lifestyle and to quit bad habits such as smoking is not likely to cause significant harm even if users experience physical discomfort and psychological stress because of the effort made to quit smoking.

3. Social Scoring - Article 5(1)(c)

AI systems that evaluate or classify individuals based on social behaviour or personal characteristics resulting in unjustified treatment are prohibited.

  • An example is an AI predictive tool that analyses taxpayers’ tax returns to select tax returns for closer inspection. The AI tool uses relevant variables such as yearly income as well as unrelated data such as a taxpayer’ social habits or internet connections to select individuals for closer inspection, leading to potential discrimination.
  • Out of scope legitimate scoring practices include financial credit scoring systems which are used by creditors and credit information agencies to determine a customer’s ability to repay debts by analysing a range of financial data such as the customer’s income and expenses.

4. Individual Criminal Offence Risk Assessment and Prediction - Article 5(1)(d)

AI systems that predict criminal behaviour based solely on profiling or personality traits, without objective human assessment are prohibited.

  • An example is a law enforcement authority that uses an AI system to predict criminal behaviour based on a personal characteristic such as age, nationality, address and marital status leading to unjust profiling.
  • Out of scope AI systems include predictive policing systems which generate a score for the likelihood of criminality in different neighbourhoods based on previous criminality rates and other supporting information such as street maps, allowing law enforcement to identify areas that require additional police presence.

5. Untargeted Scraping of Facial Images - Article 5(1)(e)

Creating or expanding facial recognition databases through untargeted scraping from the internet or CCTV footage is prohibited.

  • In terms of the internet, if a person has published facial images of themselves on social media it does not mean that the person has given permission for the images to be included in a facial recognition database. An example of scraping images from CCTV is an AI tool that uses images from surveillance cameras in public spaces like airports and streets without consent.
  • Out of scope AI systems include databases that contain facial images but do not associate them with identifiable individuals e.g. datasets used solely for training or testing AI models without any intent to recognise or identify the persons in the images.

6. Emotion Recognition - Article 5(1)(f)

AI systems inferring emotions in workplaces and educational institutions, except for medical or safety reasons are prohibited.

  • An example of ‘emotion recognition’ is an AI system that infers that an employee is unhappy, sad or angry to customers using body gestures, a frown or the absence of a smile.
  • Out of scope are AI systems that infer emotions not on the basis of biometric data e.g. from written text and AI systems that infer physical states e.g. pain and tiredness.

7. Biometric Categorisation - Article 5(1)(g)

Categorising individuals based on biometric data to infer sensitive characteristics like race, political opinions, or sexual orientation are prohibited.

  • An example is an AI system that categorises social media users by their presumed political orientation based on biometric data from uploaded photos to send them targeted political messages.
  • Examples of permissible filtering include the categorisation of patients using images according to skin or eye colour which may be important for a medical diagnosis such as cancer.

8. Real-time Remote Biometric Identification (RBI) - Article 5(1)(h)

The use of real-time RBI systems in publicly accessible spaces for law enforcement is prohibited with exceptions only for serious threats and criminal investigations.

Safeguards and conditions for the exceptions (Article 5(2)-(7) AI Act) are documented in the guidelines which aim to ensure the responsible and ethical use of AI technologies while safeguarding fundamental rights and promoting trust in AI systems.

A diagram showing 8 prohibited AI practices.svg
Overview of Prohibited AI Practices

Enforcement of Article 5 AI Act

The prohibitions in Article 5 AI Act became applicable after 2 February 2025 and penalties, governance and confidentiality will apply from 2 August 2025.

Market Surveillance Authorities are responsible for the enforcement of rules in the AI Act for AI systems including prohibitions. The AI Act employs a tiered response to determine penalties for non-compliance. This system is designed to ensure that the severity of the infringement is appropriately matched with the corresponding penalty. Non-compliance with prohibitions in Article 5 AI Act are classified as the most serious infringement and subject to the highest fine which can be up to EUR 35,000,000 or up to 7% of worldwide turnover for the previous financial year, if the offender is an undertaking, whichever is higher.

Conclusion

AI in healthcare, such as software as a medical device (SaMD), must be developed with stringent ethical standards to ensure patient safety and protection of fundamental rights. The guidelines remind developers and regulators alike of the importance of maintaining transparency and safeguarding against AI misuse. As the healthcare sector continues to integrate AI into medical devices, these guidelines will serve as a key framework for ensuring that AI-driven solutions prioritise the well-being of patients while promoting innovation and trust in the healthcare system.

If you need guidance on navigating AI challenges, contact us today to arrange a free, no-obligation discussion.

Related articles

  1. A man carefully steps across a cliff-face. An analogy for assessing risk.

    Top 5 Common Pitfalls to Avoid During Risk Assessment

    Learn how to sidestep costly mistakes which manufacturers commonly make. From hazard ID to post-market surveillance, we help you improve safety and speed up approvals.

    Kamiya Crabtree Kamiya Crabtree Regulatory Medical Writer
  2. A label maker printing bar-code labels.

    Labelling 101: A Comprehensive Overview for Medical Device Manufacturers

    Labelling and packaging are critical elements to ensuring safety, compliance, and ease of use.

    Kamiya Crabtree Kamiya Crabtree Regulatory Medical Writer
  3. A compass being used to navigate across mountainous countryside.

    Navigating Non-Conformities in Technical Documentation

    We explore how to manage non-conformities effectively and implement Corrective and Preventive Actions (CAPAs).

    Kamiya Crabtree Kamiya Crabtree Regulatory Medical Writer
  4. A photograph of a literal maze that we're using as a clever metaphor.

    Mastering the EU MDR: Essential Steps for Compliance-Ready Docs

    If you're uncertain about the readiness of your EU MDR documentation, this article provides an overview of the essential steps to ensure you’re on track.

    Kamiya Crabtree Kamiya Crabtree Regulatory Medical Writer
  5. An illustration showing scientists at work.

    A Guide to Electronic Instructions for Use (eIFU)

    Electronic Instructions for Use (eIFUs) are set to revolutionise how medical device instructions are delivered. We explore what this means for you.

    Dr Will Brambley Dr Will Brambley Lead Medical Writer
  6. Two helicopters look as if they are about to collide: An analogy for risk.

    Navigating Risk Management Requirements under the EU MDR

    This is a cornerstone of EU MDR 2017/745, requiring a continuous, well-documented approach. We unpack key requirements and provide actionable strategies.

    Peter Boxall Peter Boxall Lead Medical Writer
  7. A doctor operates a tablet computer.

    Beyond the Acronyms: Understanding SaMD and SiMD

    As software advancements continue, the line between traditional hardware-centric medical devices and software-driven solutions becomes increasingly blurred.

    Kamiya Crabtree Kamiya Crabtree Regulatory Medical Writer
  8. A team of profesional-looking people sit around a table, congratulating themselves.

    Extending the Validity of your IVDD Certificates – Key Dates

    The EU and the MHRA have extended the validity of IVDD certificates, allowing you more time to transition to the IVDR. We explain what this means for manufacturers.

    Kamiya Crabtree Kamiya Crabtree Regulatory Medical Writer
  9. A team of profesional-looking people sit around a table, congratulating themselves.

    GSPR 1: A New Era of Performance with Safety at the Core

    This regulation emphasizes risk management, durable design & biocompatibility to ensure medical devices are safe and effective. GSPR 1 protects users while driving innovation in medical technology.

    Kamiya Crabtree Kamiya Crabtree Regulatory Medical Writer
  10. Cybersecurity Vulnerabilities in Medical Devices: FDA Alerts on Contec and Epsimed Monitors

    Patients can be exposed to risks when devices are online. We explore implications for EU MDR/IVDR cybersecurity requirements, including MDCG guidance

    Dr Clare Dixon Dr Clare Dixon Regulatory Specialist
  11. A futuristic-looking factory full of labelled cardboard boxes.

    Decoding UDI: Your Ultimate Guide to Smarter Medical Device Labelling

    The Unique Device Identifier (UDI) ensures medical device traceability and compliance. We break down its structure, Device Identifier (UDI-DI), Production Identifier (UDI-PI) and its role in EUDAMED.

    Kamiya Crabtree Kamiya Crabtree Regulatory Medical Writer
  12. A hospital room full of equipment with futuristic user interfaces.

    IMDRF Sets the Standard: 10 Key Principles for AI-enabled Medical Devices

    Good Machine Learning Practice (GMLP) principles ensure safe devices, covering intended use, clinical evaluation & Human-AI Interaction (HAII).

    Ron Sangal Ron Sangal Lead Medical Writer
  13. A medical team discuss performance data at their desktop computer.

    Key Updates for Navigating EMDN: MDCG 2024-2 Rev.1 & 2021-12 Rev.1

    Release of the updated guidance helps manufacturers navigate the EMDN system for accurate device classification, ensuring market access.

    Ron Sangal Ron Sangal Lead Medical Writer
  14. A dated monitor for medical equipment.

    Understanding Clinical Evidence Requirements with MDCG 2020-6

    How can manufacturers ensure legacy devices meet MDR's stringent requirements? Discover how MDCG 2020-6 guidance simplifies the path to compliance.

    Dr Clare Dixon Dr Clare Dixon Regulatory Specialist
  15. A stethoscope laid on a desk of regulatory documentation.

    Clinical benefits of an in vitro diagnostic medical device

    How to determine the clinical benefit of an IVD and successfully incorporate it into regulatory documentation.

    Dr Gayle Buchel Dr Gayle Buchel Chief Medical Writer
  16. EU flags

    Regulation (EU) 2024/1860 - Its impact on EU MDR and IVDR

    How does the recent Regulation (EU) 2024/1860 amendment affect the EU MDR & IVDR?

    Shona Richardson Shona Richardson Regulatory Medical Writer
  17. EU flag

    MDCG 2024-10 - Orphan medical devices

    How to apply MDR pre-market clinical evidence requirements to medical devices intended for limited usage.

    Dr Simon Cumiskey Dr Simon Cumiskey Senior Lead Medical Writer
  18. Considering a medical device's intended purpose

    A medical device's intended purpose - what is the point?

    How do you define intended purpose, indication for use, intended clinical benefits, and claims?

    Dr Simon Cumiskey Dr Simon Cumiskey Senior Lead Medical Writer
  19. Mantra Systems presents EnableChat, your AI-powered MDR & MDCG chatbot

    EnableChat - Your AI-powered MDR and MDCG chatbot

    Search the MDR and MDCG documents in seconds by asking EnableChat your questions.

    Dr Simon Cumiskey Dr Simon Cumiskey Senior Lead Medical Writer
  20. Searching adverse event databases for vigilance data

    Staying vigilant - A guide to searching for adverse events data

    We discuss the pros and cons of existing adverse event databases for vigilance data searching.

    Dr Simon Cumiskey Dr Simon Cumiskey Senior Lead Medical Writer
  21. A doctor reading an SSCP document with a patient

    What is Summary of Safety and Clinical Performance (SSCP)?

    We explain what the SSCP is, when you'll need it and what its objectives are.

    Sandra Gopinath Sandra Gopinath Chief Regulatory Officer
  22. A pile of question marks

    Medical Device 'Significant Changes' – Navigating EU MDR Article 120(3) using MDCG 2020-3 rev. 1

    Understand what changes to your medical device are considered 'significant' under EU MDR (2017/745).

    Shen May Khoo Shen May Khoo Junior Regulatory Specialist
  23. A signpost giving unsure directions

    MDR or IVDR - A sibling rivalry?

    A guide to easily understanding whether your device is a medical device or an in vitro diagnostic medical device (IVD).

    Dr Gayle Buchel Dr Gayle Buchel Chief Medical Writer
  24. An EU and UK flag

    What the latest Brexit U-turn means for CE Marking of medical devices in Great Britain

    Will Great Britain continue to allow the use of the CE mark for medical devices beyond the 2024 deadline?

    Dr Hanna Gul Dr Hanna Gul Lead Medical Writer
  25. A woman writing her own medical device regulation documentation

    Gain confidence, reassurance and control over your EU MDR strategy

    Find out how to build your own technical files within a guided framework while minimising financial outlays.

    Dr Gayle Buchel Dr Gayle Buchel Chief Medical Writer
  26. Racing to achieve MDR compliance

    Still racing to achieve MDR compliance? A transition period update

    On January 6th 2023, the EU commission has adopted the proposal to extend the transition rules of the EU MDR.

    Sandra Gopinath Sandra Gopinath Chief Regulatory Officer
  27. A 7-step guide to navigating regulatory requirements for medical device start-ups

    A medical device regulations guide for start-up companies

    We present a 7-step guide to navigating regulatory requirements on a budget.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  28. An update on UKCA Marking of Medical Devices

    UKCA Marking of Medical Devices – An update on the status quo

    We review recently updated requirements for UKCA marking and what it means for your regulatory strategy.

    Dr Hanna Gul Dr Hanna Gul Lead Medical Writer
  29. How to choose a CER writer for your MDR Clinical Evaluation

    Choosing a CER writer for your MDR Clinical Evaluations

    We've compiled a list of considerations that will help you make the right choice when choosing a CER writer.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  30. Achieving MDR Compliance for Class I medical devices

    How to achieve MDR Compliance for Class I medical devices

    We outline a strategy for the regulatory compliance of Class I medical devices.

    Sandra Gopinath Sandra Gopinath Chief Regulatory Officer
  31. Literature Search, SOTA Review and Clinical Evaluation

    Literature Search, SOTA Review process and Clinical Evaluation

    We help to demystify the process of systematic search & review of literature for Clinical Evaluation.

    Sandra Gopinath Sandra Gopinath Chief Regulatory Officer
  32. Literature Search Protocols & SOTA Reviews for medical devices and what to know before you start

    Literature searches and reviews for medical devices - what to know before you start

    We explain what you should know before beginning a literature search & review for your medical device.

    Sandra Gopinath Sandra Gopinath Chief Regulatory Officer
  33. Five useful resources when writing a medical device CER

    Five useful resources when writing a medical device CER

    We outline five of the most useful and trustworthy Clinical Evaluation Report writing resources.

    Dr Victoria Cartwright Dr Victoria Cartwright Relationship Manager
  34. Avoid pitfalls when writing a Clinical Evaluation Report

    Five common pitfalls when writing a Clinical Evaluation Report

    We illustrate five pitfalls when writing CERs and give you some tips to overcome them.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  35. How to make a medical device equivalence claim under the MDR

    Five tips for making a medical device equivalence claim under the MDR

    We'll show you what to keep in mind with regards to equivalance and Clinical Evaluation.

    Sandra Gopinath Sandra Gopinath Chief Regulatory Officer
  36. Keeping medical devices in market and maintaining CE-marks - a guide to effective data collection

    Keeping medical devices in market and maintaining CE-marks

    The 4 golden rules to drive regulatory compliance with PMCF and vigilance data collection.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  37. How PMCF goes beyond simple compliance - improving products and engaging customers

    How PMCF goes beyond simple compliance

    The wider benefits of a well-designed PMCF system include improving your products and your relationship with your clients.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  38. PMCF systems for medical devices

    Why you'll almost certainly need a PMCF system for your medical devices

    We tell you what to be aware of under the EU MDR regarding PMCF and your medical devices.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  39. Ensure medical device regulatory compliance of your devices through Brexit

    The impact of Brexit on medical device regulatory compliance

    How to ensure regulatory alignment of your devices in the territories affected by Brexit.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  40. Use medical device regulatory consulting services to supercharge your MDR transition

    Is outside consulting support the answer to your MDR transition?

    Getting ready for the MDR is a demanding process. Outsourcing might be your solution.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  41. Increasing data entry compliance in PMCF studies

    Increasing data entry compliance in PMCF studies

    5 methods every medical device manufacturer should know to improve their Post-Market Clinical Follow-up studies.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  42. Why medical doctors can drive MDR compliance

    Why medical doctors can drive MDR compliance

    Working with the MDR requires knowing how to work with clinical evidence. Medical doctors are perfectly positioned to meet this requirement.

    Dr Victoria Cartwright Dr Victoria Cartwright Relationship Manager
  43. Software as a Medical Device

    Software as a Medical Device

    Unless you have spent time working with medical device legislation in the past, the idea that software could be a medical device may be rather unexpected.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  44. clinical investigator for pmcf eu mdr compliance

    Ensuring that clinical investigations work in practice

    How can medical device manufacturers ensure valid clinical investigations when access to medical expertise remains limited?

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  45. Coronavirus and medical device regulations

    Relaxing medical device regulatory requirements during a healthcare crisis

    During the coronavirus pandemic, how far should we go when relaxing medical device regulatory requirements?

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  46. The new MDR compliance challenge

    The new MDR compliance challenge

    Across the industry, medical device companies are facing challenges in meeting the demands of the new Medical Device Regulations (MDR) 2017/745 framework.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer
  47. Sources of Real World Evidence for MDR compliance

    Sources of Real World Evidence for MDR compliance

    At Mantra Systems our objective is to make sure that our clients choose the method of real world data harvesting that is right for them.

    Dr Paul Hercock Dr Paul Hercock Chief Executive Officer

More articles

Do you need support with your medical device approval strategy?

Contact us today