Privacy, governance and regulation · Cybersecurity

Regulation turns good security and privacy practice into obligations with deadlines and fines. For AI work in Germany, four regimes matter most: GDPR (any personal data), the EU AI Act (AI systems and models), the Cyber Resilience Act (software and hardware products), and NIS2 (critical organizations). Voluntary frameworks such as the NIST AI RMF and ISO/IEC 42001 are how organizations show they manage the risk. This part is orientation, not legal advice; specific cases need a lawyer.

GDPR applies whenever personal data is processed - and in ML that includes training data, prompts, conversation logs, embeddings and evaluation sets.

Principles. You need a legal basis (consent, contract, legitimate interest and others). Data may be used only for its stated purpose, kept to the minimum needed, deleted when no longer needed, and protected with appropriate security (Article 32). You must be able to demonstrate all of this.
Pseudonymous is not anonymous. Hashed or tokenized identifiers are still personal data (Part 4). And because models can memorize, a model trained on personal data is not automatically “anonymous”; European data-protection authorities expect a case-by-case assessment.
Rights. People can request access, correction and erasure. Erasure is easy in a database, hard in a vector index you forgot about, and very hard inside trained weights (Part 9’s unlearning).
DPIAs. High-risk processing - common with AI - requires a data protection impact assessment beforehand.
Automated decisions. Article 22 gives people rights around decisions made solely by automated means that significantly affect them.
Privacy by design and default (Article 25), and breach notification within 72 hours (Part 7).
Third-party model APIs. Sending prompts to a provider is a data transfer. Check the processing agreement, where data is processed, how long it is retained, and whether it is used for training.

The EU AI Act

The AI Act regulates by risk and by role.

Prohibited practices - manipulative techniques, social scoring, untargeted facial-image scraping, emotion recognition at work or school, and others.
High-risk systems - uses listed in Annex III (employment, credit, education, biometrics, critical infrastructure and more) and AI as a safety component of regulated products (Annex I). These face requirements for risk management, data governance, documentation, logging, human oversight, accuracy, robustness and cybersecurity.
Transparency duties - people must be told they are talking to an AI, generated content must carry machine-readable marking, and deepfakes must be disclosed.
General-purpose AI (GPAI) models - documentation, a copyright policy and a training-content summary; models with systemic risk also need evaluations, adversarial testing, incident reporting and cybersecurity protection.

The two main roles are provider (develops the system and places it on the market under its name) and deployer (uses it professionally). Their duties differ.

The security link. Article 15 requires high-risk systems to be resilient against attacks specific to AI, and explicitly names data poisoning, model poisoning, adversarial examples, confidentiality attacks and model flaws. Part 9 is, in effect, the technical background to that article.

Timeline. The Digital Omnibus on AI (Regulation (EU) 2026/1744) entered into force on 27 July 2026 and postponed the high-risk deadlines. It did not remove any category of obligation.

Date	What applies
2 Feb 2025	Prohibited practices; AI literacy measures
2 Aug 2025	Obligations for general-purpose AI models
2 Aug 2026	Transparency duties (Article 50); Commission fines for GPAI providers
2 Dec 2026	Machine-readable marking for generative systems already on the market; two new prohibitions (non-consensual intimate imagery, child sexual abuse material)
2 Dec 2027	High-risk requirements for Annex III uses
2 Aug 2028	High-risk requirements for AI in regulated products (Annex I)

In Germany, the Bundesnetzagentur is the market surveillance authority under the national implementing law, in force since 29 July 2026. Maximum fines reach €35 million or 7% of worldwide turnover for prohibited practices (timeline and status as of October 2026).

The Cyber Resilience Act (CRA)

The CRA sets cybersecurity requirements for “products with digital elements” sold in the EU: connected hardware and software, which includes AI-powered software products. Core ideas: secure by design, vulnerability handling for the whole support period, security updates, and an SBOM.

Its reporting duties have applied since 11 September 2026, even for products already on the market. Manufacturers must report actively exploited vulnerabilities and severe incidents through ENISA’s Single Reporting Platform: an early warning within 24 hours of becoming aware, a fuller notification within 72 hours, and a final report later. Most other obligations apply from 11 December 2027 (Ireland’s NCSC summary). Building the incident process from Part 7 is now a legal necessity for product makers.

NIS2

NIS2 is the EU directive for “essential” and “important” organizations in sectors such as energy, health, transport and digital infrastructure. It requires risk-management measures, supply-chain security and fast incident reporting, and it makes management personally accountable. In Germany it is implemented through national law, with the BSI as the central authority. If you build AI for such organizations, their NIS2 duties flow down to you as a supplier.

Voluntary frameworks

NIST AI Risk Management Framework - four functions: Govern (culture, roles, policies), Map (context and risks), Measure (test and track), Manage (prioritize and act). A companion profile covers generative AI.
ISO/IEC 42001 - a certifiable AI management system standard, usually combined with ISO/IEC 27001 for information security.
Harmonised standards for the AI Act are still being published; following them will create a presumption of conformity.

How it fits together

Security engineering produces the evidence, frameworks organize it, and regulation demands it. For an AI product, a threat model, red-team reports, logs, an AIBOM, data-protection assessments and a tested incident process cover most of what auditors and regulators will ask to see.

Questions to ask

Does this system touch personal data anywhere - training, prompts, logs, embeddings? On what legal basis?
Can we honour access and deletion requests for data inside models, indexes and logs?
Under the AI Act, are we provider or deployer, and which risk category does this use fall into?
If we sell a product in the EU, could we report an exploited vulnerability within 24 hours?
Which framework structures our AI risk management, and where is the evidence?
When we call a third-party model API, where does the data go, how long is it kept, and is it used for training?

GDPR through an ML engineer’s eyes