Showing Posts From

Ai architecture

Building a Privacy-Safe AI Pipeline: What Goes Between Your Data and the Model

Building a Privacy-Safe AI Pipeline: What Goes Between Your Data and the Model

Most European companies running AI programmes have a data protection policy that says something like: "personal data should not be sent to external AI models without appropriate controls." What most of those same companies don't have is a technical architecture that enforces this — at the point where data actually leaves the organisation, before it reaches the model, in a way that doesn't depend on every employee making the right decision every time. Policy without technical enforcement is an aspiration. I've seen teams build careful data governance frameworks, train employees on what can and cannot enter a prompt, and roll out usage guidelines. Then I've seen the same teams, six months later, discover that customer records, HR files, and contract documents had been flowing into an external model because the work was faster that way and nobody had built anything to stop it. The technical answer is a processing layer between your data and the model — a pipeline that intercepts data before it reaches the LLM, applies privacy controls, and only passes through what meets the organisation's data protection requirements. Building it is an architecture decision, not a compliance decision. This article covers what that architecture looks like, the components that make it work, and where its limits are. Why policy-only approaches fail at the prompt The problem is structural. Employees use AI tools to get work done faster. The faster path almost always involves sending more context — a full customer record rather than selected fields, a complete document rather than the relevant section, a full email thread rather than the key paragraph. The model produces better outputs with richer input, so the incentive runs directly against the minimisation principle. Enforcing minimisation through policy means asking people to voluntarily produce worse outputs to protect data they don't directly manage. That is a losing proposition. The violation isn't usually malicious — it is the product of a rational trade-off that the person making the decision doesn't see as a trade-off at all. Architectural enforcement removes the trade-off. The pipeline intercepts data before it reaches the model and applies one of two controls: stripping personal data entirely where the use case doesn't require specific identity references, or replacing identifiers with on-premise-keyed tokens where the model genuinely needs to track the same entity across a document. In both cases, what reaches the model contains no original personal data. This is the design goal: make the compliant path identical to the convenient path. But it is important to be precise about what that compliance achieves. Neither stripping nor tokenisation makes data anonymous under GDPR. Stripping is the stronger control — no key exists, no reversal is possible — and it is what the pipeline should default to. Tokenisation is a fallback for use cases that require entity consistency, and tokenised data remains personal data under GDPR, as covered in detail in the anonymisation article in this series. The pipeline reduces risk significantly. It does not eliminate the organisation's GDPR obligations for what reaches the model. The core architecture: a processing layer the LLM never bypasses The pipeline has five logical components. All personal data must pass through all five before reaching the model. The LLM has no direct access to raw data — only to what the pipeline has processed and approved. Data ingestion: Documents, database records, API responses, or user prompts enter the pipeline. This is the boundary where classification begins — what type of data is this, what sensitivity level does it carry, and what processing rules apply. PII detection: A detection layer identifies personal data in the ingested content. This covers direct identifiers (names, email addresses, phone numbers, national identification numbers, passport numbers, IBAN and financial account numbers, IP addresses, biometric identifiers) and indirect identifiers that could contribute to re-identification (age combined with location, job title combined with department size, medical codes combined with demographic data). Detection uses a combination of Named Entity Recognition (NER) models and pattern-matching rules. NER models identify contextual entities — a name in a sentence, a location in a paragraph. Pattern-matching rules catch structured identifiers — a correctly formatted IBAN, an email address, a German Steuernummer. PII removal and tokenisation layer: This is the core control, and it operates in two modes depending on what the task requires. Stripping (default): Detected identifiers are removed entirely. A name disappears. An email address is deleted. No key is created, no mapping is stored, no reversal is possible. This is the stronger control and should be the default for use cases where specific identity is not required — summarisation, document classification, general drafting, sentiment analysis. Stripped data is the closest the pipeline can bring data to the GDPR anonymisation standard, though it still may not meet the full legal threshold if indirect re-identification from remaining attributes remains possible. Tokenisation (exception): Where the AI task genuinely requires consistent entity references across a document — contract analysis referencing the same counterparty throughout, an HR review tracking a specific employee's record — detected identifiers are replaced with consistent tokens (PERSON_001, EMAIL_001, ORG_002). The token mapping is stored in a key vault that lives entirely on-premise and is never transmitted. The model sees consistent references without knowing the real identity. This is pseudonymisation. Pseudonymised data is still personal data under GDPR. The pipeline's GDPR obligations — transfer mechanisms, DPA requirements, breach notification — apply to any tokenised content that reaches the external model. LLM processing: The stripped or tokenised content is sent to the external model. No original personal data is present. The model has no access to the key vault. Output de-tokenisation (tokenisation mode only): The model's response is returned to the pipeline. Where tokenisation was applied, the pipeline replaces tokens with the original identifiers using the on-premise key vault before delivery to the user. The key vault never left the organisation. What the vendor receives, what any external disclosure would expose, and what any CLOUD Act order could compel them to produce is the stripped or tokenised version. The reduction in exposure is real. It is not, however, a GDPR exemption. Building the PII detection layer The detection layer is where most implementations have their weakest point. NER models trained on general text perform well on common identifiers — names, locations, organisations in standard formats. They perform poorly on domain-specific identifiers, non-standard formats, and contextual indirect identifiers that require understanding of the data environment. For enterprise deployments, the detection layer needs to be tuned for the organisation's specific data. A healthcare provider will have medical record identifiers that a general NER model won't recognise. A financial services firm will have account reference formats specific to its systems. A logistics company will have shipment identifiers tied to individual customers. Microsoft Presidio is an open-source PII detection and anonymisation framework that provides a base NER capability with extensible recogniser architecture — custom recognisers can be added for organisation-specific identifier formats. spaCy provides the underlying NER model that can be fine-tuned on domain data. GLiNER is a more recent generalist NER model that performs well on zero-shot entity recognition, useful for catching entity types that weren't anticipated in the original configuration. The detection layer should be treated as imperfect by design. False negatives — personal data that is not detected and therefore passes through to the model as-is — will occur. The residual risk from false negatives needs to be part of the organisation's data protection impact assessment. For high-sensitivity data categories (health data, financial data, data on minors), a higher detection threshold or additional validation step is warranted. For lower-sensitivity data, the residual risk from occasional false negatives may be acceptable. Logging is essential. Every item that passes through the pipeline should generate a log entry: what data type was detected, what action was taken (stripped, tokenised, blocked), what was transmitted to the model, and when. This log is the audit trail that GDPR and the June 2025 DSK guidance require. It also lets the organisation identify patterns where specific data types are consistently slipping through the detection layer. Synthetic data as an alternative for AI development For organisations building, fine-tuning, or testing AI models on internal data, the pipeline approach described above is a runtime control — it operates on data flowing to the model during inference. A different problem arises at development time: the organisation needs representative data to develop, test, and fine-tune AI systems, and real personal data carries risk even in development environments. Synthetic data generation creates datasets that have the same statistical properties as real data without containing records about real individuals. For a customer dataset, the synthetic version preserves the distribution of ages, purchase patterns, geographic spread, and behavioural signals — but every record is generated, not derived from an actual customer. The GDPR position on genuinely synthetic data is more favourable than on pseudonymised data, because data generated from statistical distributions rather than from individual records may qualify as anonymous if the generation process cannot be reversed to recover the underlying individuals. This requires that the generative process genuinely produces new data rather than perturbing or recombining existing records — a meaningful technical distinction that affects both the data's legal status and its usefulness as a development dataset. For model testing and development environments, synthetic data removes the need to apply runtime privacy controls and reduces the risk that development data leaks through version control, shared notebooks, or development tooling. For production fine-tuning, synthetic data has quality limitations — complex behavioural patterns and rare edge cases are difficult to preserve through synthesis — but it is a viable alternative for many use cases where the fine-tuning goal is improving general performance rather than capturing rare behaviours. What this architecture doesn't solve The pipeline addresses the most direct GDPR exposure in AI: personal data flowing to an external model without appropriate controls. It does not address all exposure. Usage metadata is not affected. The vendor still sees query frequency, document structure patterns, workflow characteristics, and feature usage — the competitive intelligence risk described in the first article in this series. The pipeline controls what is in the data; it doesn't control that data is being sent. Transfer mechanisms still apply. Pseudonymised data is still personal data under GDPR. The pipeline reduces the sensitivity and impact of a disclosure event, but it does not remove the organisation's obligation to have a valid cross-border transfer mechanism in place for the pseudonymised content that does reach the external model. Fine-tuning risk remains for the training phase. If the organisation fine-tunes a model on pseudonymised data, membership inference risk applies to the trained model — the model may encode patterns that allow indirect re-identification, as covered in the anonymisation article in this series. The runtime pipeline prevents personal data from reaching the model during inference. It does not address patterns the model has already learned from training data. Homomorphic encryption is the emerging technical approach that would allow data to be processed by the model while remaining encrypted throughout — the model would never see plaintext data at any point. This would represent a fundamentally stronger privacy guarantee than pseudonymisation. In practice, homomorphic encryption adds significant computational overhead and is not yet deployable at enterprise scale for LLM inference. It is being actively researched and is recognised in EU policy frameworks as a potential compliance tool for the medium term. It is not a practical option for most enterprise AI programmes today. What to take from thisBuild the privacy pipeline before deploying AI to end users. The decision to deploy an AI interface to employees without technical privacy controls in place is a decision to rely on policy enforcement alone — which consistently fails. The pipeline is the enforcement mechanism. Start PII detection with a general-purpose NER model and extend it with custom recognisers for your organisation's specific identifier formats. General models will miss domain-specific and proprietary identifiers. Tuning for your data is required, not optional. Default to stripping, not tokenisation. Stripping removes PII entirely — no key, no reversal, no personal data transmitted. Use tokenisation only when the AI task genuinely requires entity consistency across a document, and treat tokenised data as personal data throughout, because under GDPR it is. Where tokenisation is applied, the key vault must live on-premise, isolated from all network-facing components, and never transmitted. If the key vault is in the same cloud environment as the model API calls, the architectural separation is broken. Treat the detection layer as imperfect and log everything. Build the DPIA assumption that some personal data will pass through undetected. Log all pipeline activity so you can identify detection gaps and demonstrate the audit trail regulators will ask for. Use synthetic data for AI development and testing environments. Remove real personal data from development pipelines entirely where possible — synthetic data preserves enough statistical structure for most development use cases while eliminating the exposure. Understand what the pipeline does and doesn't cover. Transfer mechanisms, metadata exposure, and fine-tuning data risk sit outside what a runtime pseudonymisation pipeline addresses. The pipeline closes the most direct exposure. The other risks need separate controls.The gap between "we have a data protection policy for AI" and "our AI systems are technically enforcing that policy" is where most European enterprises currently sit. Policy documents don't intercept a prompt. Architecture does. The pipeline is not complex to build in its basic form — a PII detection layer, a tokeniser, and an on-premise key vault are available as open-source components or commercial services. What requires investment is tuning detection for your specific data environment, integrating the pipeline into every AI touchpoint in the organisation, and maintaining it as the AI programme grows. That investment is also what makes the compliance position defensible when examined — because a working technical control is a fundamentally different answer to a regulatory question than a policy that assumed people would follow it.

Read full article