Rules for medical AI applications
What legislation applies to algorithms developed for internal use? In 2019, an overview was prepared at the request of the Ministry of Health, Welfare and Sport, some of which are highlighted here.[1] The rules mainly hit two topics:
- The application of the algorithm and the processes around it, and
- The use of personal data.
APPLICATION OF THE ALGORITHM
For software that is developed and used internally, the Medical Device Regulation (MDR) may apply.[2] Software that supports a medical device may be a medical device. There is an enlightened regime for healthcare facilities within the MDR for internally developed medical devices.[3] A healthcare facility is defined as an organization whose primary purpose is the care of patients or the promotion of public health. Healthcare facilities are not required to apply for certification for their medical device as long as it remains in-house. However, a number of requirements listed in Article 5.5 of the MDR must be met. For example, the healthcare facility must be able to argue why a new application is needed and why an existing application cannot be used.
In addition to the MDR, there are also national rules for the application of medical technology such as the Dutch Care Quality, Complaints and Disputes Act (wet kwaliteit, klachten en geschillen zorg) and related decrees and regulations. There is also the Medical Technology Covenant which applies when a healthcare provider uses a device.
PERSONAL DATA
When developing algorithms in a healthcare environment, it is almost inevitable that sensitive personal data will be processed. A patient's data must be properly protected, and a range of general and specific legislation applies here. First, the General Data Protection Regulation (GDPR) and the Dutch implementation act of the GDPR (UAVG) apply. The processing of personal data must always be lawful, proper and transparent.
With sensitive personal data such as data about someone's health, even stricter requirements apply. For example, consent must be sought from patients before using their personal data to develop an algorithm [4] or measures must be taken to minimize the risk to patients if it is not possible to seek consent [5].
Also, depending on the purpose of the algorithm, it may be necessary to do a Data Protection Impact Assessment (DPIA) before starting to develop the algorithm. A DPIA is mandatory if a processing of personal data poses a high privacy risk to data subjects. Whether there is a high risk can be determined based on the guidance from the GDPR and the responsible authority (like the AP in the Netherlands).[6] Large-scale processing of health data is on the list of processing operations requiring a DPIA from the AP.[7]
In addition to the general rules of the GDPR, there are rules specific to healthcare, such as the Decree on Electronic Data Processing by Healthcare Providers (het Besluit elektronische gegevensverwerking door zorgaanbieders) and a number of NEN and ISO standards that are included as harmonized standards in that Decree or in the MDR.
AI ACT
The EU is working on a regulation specifically aimed at AI applications.[8] When developing algorithms to be used in healthcare, the rules of the AI Act may become important. If a particular algorithm falls under the MDR, the chances are quite high that it is a High-Risk AI application.[9] This means that the developer of the application will have to meet a number of requirements in terms of documentation, control and development methodology.[10]
In addition to requiring developers of AI systems to comply with the AI Act, there are also requirements for users of systems covered by the regulation. For example, researchers who research an application but do not develop it themselves are included.
CONCLUSION
Currently, there are actually no specific additional rules when developing an AI application versus ordinary software. However, the processing of personal data is becoming more important and thus the role of the GDPR is increasing. Once the new AI Act enters into force, there will be greater differences between the rules for ordinary software and AI applications.
[1] Ministry of Health, Welfare and Sport (2019) Handout legal and normative frameworks around AI in healthcare.
[2] M.F. van der Mersch (2022) Guide AI in healthcare.
[3] Article 5(5) MDR.
[4] Article 9 GDPR.
[5] Article 24 UAVG (the Dutch implementation act of the GDPR)
[6] https://autoriteitpersoonsgegevens.nl/nl/zelf-doen/data-protection-impact-assessment-dpia
[7] https://autoriteitpersoonsgegevens.nl/sites/default/files/atoms/files/stcrt-2019-64418.pdf
[8] https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52021PC0206
[9] Article 6(1)(a) AI Act.
[10] Articles 8 to 15 AI Act.
Details
More questions?
If you were not able to find an answer to your question, contact us via our member-only helpdesk or our contact page.
