Showing Posts From

Enterprise ai

Building a Privacy-Safe AI Pipeline: What Goes Between Your Data and the Model

Building a Privacy-Safe AI Pipeline: What Goes Between Your Data and the Model

Most European companies running AI programmes have a data protection policy that says something like: "personal data should not be sent to external AI models without appropriate controls." What most of those same companies don't have is a technical architecture that enforces this — at the point where data actually leaves the organisation, before it reaches the model, in a way that doesn't depend on every employee making the right decision every time. Policy without technical enforcement is an aspiration. I've seen teams build careful data governance frameworks, train employees on what can and cannot enter a prompt, and roll out usage guidelines. Then I've seen the same teams, six months later, discover that customer records, HR files, and contract documents had been flowing into an external model because the work was faster that way and nobody had built anything to stop it. The technical answer is a processing layer between your data and the model — a pipeline that intercepts data before it reaches the LLM, applies privacy controls, and only passes through what meets the organisation's data protection requirements. Building it is an architecture decision, not a compliance decision. This article covers what that architecture looks like, the components that make it work, and where its limits are. Why policy-only approaches fail at the prompt The problem is structural. Employees use AI tools to get work done faster. The faster path almost always involves sending more context — a full customer record rather than selected fields, a complete document rather than the relevant section, a full email thread rather than the key paragraph. The model produces better outputs with richer input, so the incentive runs directly against the minimisation principle. Enforcing minimisation through policy means asking people to voluntarily produce worse outputs to protect data they don't directly manage. That is a losing proposition. The violation isn't usually malicious — it is the product of a rational trade-off that the person making the decision doesn't see as a trade-off at all. Architectural enforcement removes the trade-off. The pipeline intercepts data before it reaches the model and applies one of two controls: stripping personal data entirely where the use case doesn't require specific identity references, or replacing identifiers with on-premise-keyed tokens where the model genuinely needs to track the same entity across a document. In both cases, what reaches the model contains no original personal data. This is the design goal: make the compliant path identical to the convenient path. But it is important to be precise about what that compliance achieves. Neither stripping nor tokenisation makes data anonymous under GDPR. Stripping is the stronger control — no key exists, no reversal is possible — and it is what the pipeline should default to. Tokenisation is a fallback for use cases that require entity consistency, and tokenised data remains personal data under GDPR, as covered in detail in the anonymisation article in this series. The pipeline reduces risk significantly. It does not eliminate the organisation's GDPR obligations for what reaches the model. The core architecture: a processing layer the LLM never bypasses The pipeline has five logical components. All personal data must pass through all five before reaching the model. The LLM has no direct access to raw data — only to what the pipeline has processed and approved. Data ingestion: Documents, database records, API responses, or user prompts enter the pipeline. This is the boundary where classification begins — what type of data is this, what sensitivity level does it carry, and what processing rules apply. PII detection: A detection layer identifies personal data in the ingested content. This covers direct identifiers (names, email addresses, phone numbers, national identification numbers, passport numbers, IBAN and financial account numbers, IP addresses, biometric identifiers) and indirect identifiers that could contribute to re-identification (age combined with location, job title combined with department size, medical codes combined with demographic data). Detection uses a combination of Named Entity Recognition (NER) models and pattern-matching rules. NER models identify contextual entities — a name in a sentence, a location in a paragraph. Pattern-matching rules catch structured identifiers — a correctly formatted IBAN, an email address, a German Steuernummer. PII removal and tokenisation layer: This is the core control, and it operates in two modes depending on what the task requires. Stripping (default): Detected identifiers are removed entirely. A name disappears. An email address is deleted. No key is created, no mapping is stored, no reversal is possible. This is the stronger control and should be the default for use cases where specific identity is not required — summarisation, document classification, general drafting, sentiment analysis. Stripped data is the closest the pipeline can bring data to the GDPR anonymisation standard, though it still may not meet the full legal threshold if indirect re-identification from remaining attributes remains possible. Tokenisation (exception): Where the AI task genuinely requires consistent entity references across a document — contract analysis referencing the same counterparty throughout, an HR review tracking a specific employee's record — detected identifiers are replaced with consistent tokens (PERSON_001, EMAIL_001, ORG_002). The token mapping is stored in a key vault that lives entirely on-premise and is never transmitted. The model sees consistent references without knowing the real identity. This is pseudonymisation. Pseudonymised data is still personal data under GDPR. The pipeline's GDPR obligations — transfer mechanisms, DPA requirements, breach notification — apply to any tokenised content that reaches the external model. LLM processing: The stripped or tokenised content is sent to the external model. No original personal data is present. The model has no access to the key vault. Output de-tokenisation (tokenisation mode only): The model's response is returned to the pipeline. Where tokenisation was applied, the pipeline replaces tokens with the original identifiers using the on-premise key vault before delivery to the user. The key vault never left the organisation. What the vendor receives, what any external disclosure would expose, and what any CLOUD Act order could compel them to produce is the stripped or tokenised version. The reduction in exposure is real. It is not, however, a GDPR exemption. Building the PII detection layer The detection layer is where most implementations have their weakest point. NER models trained on general text perform well on common identifiers — names, locations, organisations in standard formats. They perform poorly on domain-specific identifiers, non-standard formats, and contextual indirect identifiers that require understanding of the data environment. For enterprise deployments, the detection layer needs to be tuned for the organisation's specific data. A healthcare provider will have medical record identifiers that a general NER model won't recognise. A financial services firm will have account reference formats specific to its systems. A logistics company will have shipment identifiers tied to individual customers. Microsoft Presidio is an open-source PII detection and anonymisation framework that provides a base NER capability with extensible recogniser architecture — custom recognisers can be added for organisation-specific identifier formats. spaCy provides the underlying NER model that can be fine-tuned on domain data. GLiNER is a more recent generalist NER model that performs well on zero-shot entity recognition, useful for catching entity types that weren't anticipated in the original configuration. The detection layer should be treated as imperfect by design. False negatives — personal data that is not detected and therefore passes through to the model as-is — will occur. The residual risk from false negatives needs to be part of the organisation's data protection impact assessment. For high-sensitivity data categories (health data, financial data, data on minors), a higher detection threshold or additional validation step is warranted. For lower-sensitivity data, the residual risk from occasional false negatives may be acceptable. Logging is essential. Every item that passes through the pipeline should generate a log entry: what data type was detected, what action was taken (stripped, tokenised, blocked), what was transmitted to the model, and when. This log is the audit trail that GDPR and the June 2025 DSK guidance require. It also lets the organisation identify patterns where specific data types are consistently slipping through the detection layer. Synthetic data as an alternative for AI development For organisations building, fine-tuning, or testing AI models on internal data, the pipeline approach described above is a runtime control — it operates on data flowing to the model during inference. A different problem arises at development time: the organisation needs representative data to develop, test, and fine-tune AI systems, and real personal data carries risk even in development environments. Synthetic data generation creates datasets that have the same statistical properties as real data without containing records about real individuals. For a customer dataset, the synthetic version preserves the distribution of ages, purchase patterns, geographic spread, and behavioural signals — but every record is generated, not derived from an actual customer. The GDPR position on genuinely synthetic data is more favourable than on pseudonymised data, because data generated from statistical distributions rather than from individual records may qualify as anonymous if the generation process cannot be reversed to recover the underlying individuals. This requires that the generative process genuinely produces new data rather than perturbing or recombining existing records — a meaningful technical distinction that affects both the data's legal status and its usefulness as a development dataset. For model testing and development environments, synthetic data removes the need to apply runtime privacy controls and reduces the risk that development data leaks through version control, shared notebooks, or development tooling. For production fine-tuning, synthetic data has quality limitations — complex behavioural patterns and rare edge cases are difficult to preserve through synthesis — but it is a viable alternative for many use cases where the fine-tuning goal is improving general performance rather than capturing rare behaviours. What this architecture doesn't solve The pipeline addresses the most direct GDPR exposure in AI: personal data flowing to an external model without appropriate controls. It does not address all exposure. Usage metadata is not affected. The vendor still sees query frequency, document structure patterns, workflow characteristics, and feature usage — the competitive intelligence risk described in the first article in this series. The pipeline controls what is in the data; it doesn't control that data is being sent. Transfer mechanisms still apply. Pseudonymised data is still personal data under GDPR. The pipeline reduces the sensitivity and impact of a disclosure event, but it does not remove the organisation's obligation to have a valid cross-border transfer mechanism in place for the pseudonymised content that does reach the external model. Fine-tuning risk remains for the training phase. If the organisation fine-tunes a model on pseudonymised data, membership inference risk applies to the trained model — the model may encode patterns that allow indirect re-identification, as covered in the anonymisation article in this series. The runtime pipeline prevents personal data from reaching the model during inference. It does not address patterns the model has already learned from training data. Homomorphic encryption is the emerging technical approach that would allow data to be processed by the model while remaining encrypted throughout — the model would never see plaintext data at any point. This would represent a fundamentally stronger privacy guarantee than pseudonymisation. In practice, homomorphic encryption adds significant computational overhead and is not yet deployable at enterprise scale for LLM inference. It is being actively researched and is recognised in EU policy frameworks as a potential compliance tool for the medium term. It is not a practical option for most enterprise AI programmes today. What to take from thisBuild the privacy pipeline before deploying AI to end users. The decision to deploy an AI interface to employees without technical privacy controls in place is a decision to rely on policy enforcement alone — which consistently fails. The pipeline is the enforcement mechanism. Start PII detection with a general-purpose NER model and extend it with custom recognisers for your organisation's specific identifier formats. General models will miss domain-specific and proprietary identifiers. Tuning for your data is required, not optional. Default to stripping, not tokenisation. Stripping removes PII entirely — no key, no reversal, no personal data transmitted. Use tokenisation only when the AI task genuinely requires entity consistency across a document, and treat tokenised data as personal data throughout, because under GDPR it is. Where tokenisation is applied, the key vault must live on-premise, isolated from all network-facing components, and never transmitted. If the key vault is in the same cloud environment as the model API calls, the architectural separation is broken. Treat the detection layer as imperfect and log everything. Build the DPIA assumption that some personal data will pass through undetected. Log all pipeline activity so you can identify detection gaps and demonstrate the audit trail regulators will ask for. Use synthetic data for AI development and testing environments. Remove real personal data from development pipelines entirely where possible — synthetic data preserves enough statistical structure for most development use cases while eliminating the exposure. Understand what the pipeline does and doesn't cover. Transfer mechanisms, metadata exposure, and fine-tuning data risk sit outside what a runtime pseudonymisation pipeline addresses. The pipeline closes the most direct exposure. The other risks need separate controls.The gap between "we have a data protection policy for AI" and "our AI systems are technically enforcing that policy" is where most European enterprises currently sit. Policy documents don't intercept a prompt. Architecture does. The pipeline is not complex to build in its basic form — a PII detection layer, a tokeniser, and an on-premise key vault are available as open-source components or commercial services. What requires investment is tuning detection for your specific data environment, integrating the pipeline into every AI touchpoint in the organisation, and maintaining it as the AI programme grows. That investment is also what makes the compliance position defensible when examined — because a working technical control is a fundamentally different answer to a regulatory question than a policy that assumed people would follow it.

Read full article
Data Anonymisation for AI: Why the GDPR Standard Is Harder Than European Companies Think

Data Anonymisation for AI: Why the GDPR Standard Is Harder Than European Companies Think

A European data team prepares a dataset for processing by an external AI model. Before it goes, they run the standard anonymisation process: names removed, email addresses stripped, identification numbers deleted. The team confirms the data is anonymous. The dataset is sent. It isn't anonymous. What most organisations call anonymisation is pseudonymisation. The distinction matters — not as a legal technicality, but because GDPR treats them entirely differently, the actual standard for true anonymisation is far higher than most teams have achieved, and AI models introduce a specific re-identification risk that didn't exist in conventional data processing. European companies making data flow decisions based on an anonymisation claim they haven't actually met are carrying GDPR exposure they believe they've eliminated. The gap between the two terms is where most enterprise AI programmes are currently sitting. The difference between pseudonymisation and anonymisation — and why it changes everything Pseudonymisation replaces identifying fields with artificial identifiers. A name becomes a token. An ID number becomes a different ID number. The original identity can be restored using a key held by the originating organisation. The data looks de-identified. Under GDPR, it is explicitly classified as personal data. GDPR Recital 26 makes this clear: pseudonymised data — data that could be attributed to a natural person by the use of additional information — remains within the scope of the regulation. Cross-border transfers of pseudonymised data still require a valid transfer mechanism. Data subject rights still apply to it. Breach notification obligations apply if it is compromised along with the re-identification key. Pseudonymisation is valuable risk reduction — GDPR Article 32 explicitly recommends it as a technical security measure — but it does not remove data from GDPR scope. True anonymisation is the irreversible kind. Data is genuinely anonymous when re-identification is not reasonably possible, taking into account all the means likely to be used by any party. That is the GDPR standard. It is an extremely high bar. Truly anonymous data falls entirely outside GDPR scope — it can be transferred to AI models outside the EU without triggering the transfer restrictions in Chapter V, it carries no breach notification obligations, and it is not subject to data subject access rights. The problem is that meeting this standard for real-world enterprise datasets is genuinely difficult. And most organisations claiming anonymisation have not met it. Why the re-identification standard is harder than it appears Remove a name, remove an email address, remove a national identification number. The dataset looks clean. Now consider what remains: age, job title, department, location, salary band, tenure, performance rating, and a timestamp on when the record was created. For a company with three hundred employees in a single city, how many people does that description fit? Often fewer than five. Sometimes one. This is the re-identification problem. It doesn't require the re-identifier to have the original data. It requires them to have other data — which may be publicly available, commercially available, or deducible from the context — that allows them to match records back to individuals. The more detailed and rich the dataset, the smaller the group that any given record could describe, and the easier the re-identification. Study after study has demonstrated this in practice. Researchers have re-identified individuals in supposedly anonymous medical datasets by cross-referencing with public records. Anonymous mobility data has been linked back to named individuals by correlating location patterns with public social media. Netflix user records, released as part of an anonymisation research dataset, were re-identified by cross-referencing with public film ratings platforms. The European Data Protection Board is working on updated guidelines on anonymisation — this was announced in its 2024-2025 work programme. The updated guidance hasn't resolved the standard; it has reinforced the position that organisations asserting anonymisation should be able to demonstrate it, not assert it as a processing decision taken internally without external validation. For European enterprises, the practical implication is this: if the dataset is complex, if the records are detailed, if the organisation holds a re-identification key anywhere in its systems, or if the data could be cross-referenced with external sources to recover identities — the data is pseudonymous at best, not anonymous. And pseudonymous data requires all the GDPR protections that come with personal data. How AI models add a re-identification risk that didn't exist before Conventional data processing treats data as data. An analytics platform runs queries. A reporting system aggregates records. The data doesn't learn from its own processing. AI models are different. A model trained on pseudonymised data can, under certain conditions, learn patterns correlated with the identity of specific individuals — patterns derived not from the identifiers (which were removed) but from the combination of other attributes in the records. This creates what is known as a membership inference risk: an attacker — or the model itself, under specific prompting conditions — can determine whether a specific individual was present in the training set and, in some cases, infer characteristics of that individual's record. The model hasn't stored the record. It has encoded patterns from it during training. Those patterns are queryable. This is not a theoretical attack vector — membership inference has been demonstrated on real models trained on real datasets, and it is an active area of regulatory concern at both the EDPB and national supervisory authority level. For European companies, this creates a specific additional problem. An organisation might correctly conclude that sending pseudonymised data to an external processor for conventional analytics is acceptable. Sending the same pseudonymised data to an AI model for training or fine-tuning is a materially different decision — because the model learns from the data in ways a conventional processor doesn't, and those learned patterns can be queried in ways that don't look like data access requests. The re-identification key doesn't need to be compromised. The model output can, under the right conditions, function as a partial re-identification mechanism — without any key ever leaving the organisation. What the November 2025 ECJ ruling actually says — and what it doesn't The European Court of Justice issued a ruling in November 2025 that clarified a specific scenario: pseudonymised data held by a recipient who does not have access to the re-identification key, and who cannot reasonably obtain it, may fall outside GDPR scope from that recipient's perspective. This ruling has been widely reported as a significant relaxation of the pseudonymisation standard. It is useful clarification. It is not the broad clearance it has sometimes been described as. The ruling applies to the recipient's position, not the sender's. The organisation sending pseudonymised data still holds the re-identification key. From the sending organisation's perspective, it is still processing personal data, and the transfer still requires a valid mechanism under GDPR Chapter V. The question of whether the recipient — the AI vendor — is processing personal data is a separate analysis that requires examining whether the vendor can reasonably obtain or derive the means to re-identify individuals. For major AI vendors with access to large external datasets, broad internet data, and models capable of cross-referencing patterns at scale, answering that question is not straightforward. A vendor that cannot access the specific organisation's re-identification key may nonetheless be capable of re-identifying individuals by cross-referencing the pseudonymised data with other sources in their possession. The ECJ ruling doesn't provide clearance in that scenario — it requires case-by-case analysis. The safe position for European companies is not to cite the November 2025 ruling as a blanket clearance for pseudonymised AI transfers. It is to conduct the case-by-case analysis the ruling prescribes — which requires understanding what external data the AI vendor has access to, what their re-identification capabilities are, and whether the specific pseudonymised dataset could reasonably be linked back to identifiable individuals by someone in the vendor's position. What this means in practice for European AI programmes The path forward is not to avoid AI models or to conclude that no data can be transferred. It is to be accurate about what has and hasn't been achieved. Pseudonymisation before data leaves the organisation is the correct technical control for reducing exposure when processing data on external AI models. It should be implemented properly — direct identifiers removed, re-identification key held on-premise and not transmitted, pseudonymisation applied consistently and auditably. It significantly reduces the impact of any disclosure event. It reduces but does not eliminate re-identification risk. What it does not do is remove the organisation's GDPR obligations. The transfer still requires a valid mechanism. The data still counts as personal data for GDPR purposes. The organisation still needs a lawful basis for the processing. These obligations persist alongside the pseudonymisation. For fine-tuning or training AI models on internal data, the additional membership inference risk needs to be part of the architecture decision — not assessed after the model is in production. The question of what personal data is in the training dataset, what patterns an attacker could derive from it, and what technical controls reduce the inference risk is an architecture question, not a post-deployment compliance question. What to take from thisAudit what your organisation actually does when it calls data "anonymous" before sending it to an AI model. In most cases, direct identifiers have been removed but the data remains pseudonymous. Update the data flow documentation accordingly — GDPR obligations follow from the accurate classification, not the assumed one. Treat pseudonymisation as risk reduction, not GDPR elimination. It is valuable and recommended under Article 32. It does not take data out of GDPR scope. Cross-border transfer mechanisms still apply. Do not cite the November 2025 ECJ ruling as a blanket clearance. It applies to recipients who genuinely cannot re-identify individuals. Establish whether your AI vendor meets that condition before relying on it — because a vendor with access to large external datasets may not. Include membership inference risk in AI architecture reviews. For any model trained or fine-tuned on internal data containing personal information, assess whether the model could be queried in ways that allow re-identification of individuals in the training set. Do this before the model is built, not after it's running. Keep the re-identification key on-premise. This sounds obvious. In practice, I have seen organisations pseudonymise data and then include a lookup table in the data package sent to the vendor — which converts pseudonymisation into a form of weak encryption rather than a genuine privacy control. Document the anonymisation or pseudonymisation process with enough detail to support a regulatory assessment. If a supervisory authority asks how the data was anonymised, the answer needs to be a documented process, not a verbal account of what the data team did six months ago.The word "anonymisation" has been in data governance policies for years as shorthand for "we processed the data carefully before sharing it." That shorthand has never fully survived GDPR scrutiny. It survives even less well when the sharing is with an AI model that learns from what it receives. The question isn't whether the organisation intended to comply with the regulation. The question is whether the data classification was accurate — and for most European enterprises running AI programmes, the honest answer is that it hasn't been tested against the standard that actually applies.

Read full article
Sending European Data to US AI Models: The Risks That Outlast the Contract

Sending European Data to US AI Models: The Risks That Outlast the Contract

The legal team signed off. The data processing agreement is in place. Standard contractual clauses are attached to the vendor agreement. The European company is operating under the EU-US Data Privacy Framework. From a compliance documentation standpoint, the boxes are ticked. Three specific risks persist that none of those documents fix. This isn't an argument against using external AI models. European companies are running AI programmes on US-hosted infrastructure because those models are currently the most capable available, and the business case for using them is real. The argument here is narrower: the compliance documentation that most European companies have in place for cross-border AI data transfers is necessary but not sufficient. Understanding what it doesn't cover — and why — is the starting point for an honest risk assessment. What signing the contract actually achieves Standard contractual clauses (SCCs) create binding obligations on the data importer — the US-based AI vendor receiving European data. The vendor commits to handling the data in accordance with GDPR requirements, to notifying the European company of any requests from authorities, and to applying appropriate technical and organisational security measures. This is genuine legal protection. It creates enforceable obligations. It is also the minimum requirement for a cross-border transfer to be lawful under GDPR, not the complete solution. What SCCs do not do: they do not change the jurisdiction of the vendor's infrastructure. They do not change the legal obligations a US-incorporated company is subject to under US law. A US-incorporated AI vendor that receives a lawful demand from US law enforcement is subject to that demand whether or not it has signed an SCC with every European customer it serves. The SCC creates a contractual conflict — the vendor may be obliged to notify the European customer and resist disclosure where legally possible. It does not create a legal mechanism that overrides US law. The data processing agreement operates in the same space. It defines how the vendor handles the data, what it can do with it, what happens in a breach. It is a contract between two private parties. It cannot bind US law enforcement. The political fragility underneath current EU-US transfers The EU-US Data Privacy Framework (DPF) is the current mechanism that provides an adequacy basis for transfers of personal data from the EU to US companies that have self-certified under the framework. Most major US AI vendors have self-certified. This is what allows European companies to transfer data to those vendors without relying solely on SCCs. The DPF was adopted in July 2023. It is the third iteration of a transatlantic data transfer agreement. The first two — Safe Harbor and Privacy Shield — were both invalidated by the European Court of Justice, in 2015 and 2020 respectively. The conditions that produced those rulings — US surveillance laws, specifically FISA Section 702 and Executive Order 12333, which permit broad data collection on non-US persons — have not materially changed. The DPF was adopted alongside a US executive order (EO 14086) that introduced new redress mechanisms for EU data subjects, but whether those mechanisms are sufficient to satisfy the ECJ's adequacy requirements remains contested. A challenge to the DPF is already working through the European legal system. Privacy rights organisations have filed complaints in several EU member states. The pattern from Safe Harbor and Privacy Shield — challenge filed, referred to ECJ, adequacy decision struck down — is a known sequence. European companies whose entire cross-border transfer strategy depends on the DPF holding are carrying a risk they may not have quantified. The practical position: SCCs should be in place independently of the DPF, as a fallback mechanism. But SCCs alone don't resolve the structural issues the ECJ has identified in prior rulings — which brings us to the third problem. The US CLOUD Act: a structural gap no SCC closes The Clarifying Lawful Overseas Use of Data Act (CLOUD Act), enacted in 2018, allows US law enforcement to compel US-incorporated companies to produce data stored on their servers — regardless of where that data is physically stored. A US-based AI vendor with a data centre in Frankfurt is still subject to a CLOUD Act order for data held in that Frankfurt facility, because the obligation runs to the company, not to the data's location. This is not a theoretical risk or an unlikely scenario. It is the structural architecture of US data law. European companies that have negotiated data residency commitments from their US AI vendors — contractual guarantees that data stays in EU data centres — have obtained a meaningful operational control and a genuine security benefit. They have not obtained protection from CLOUD Act orders. SCCs require vendors to notify the European customer of government access requests and to resist such requests where legally possible. In practice, where US law enforcement has obtained a valid CLOUD Act order, the vendor's ability to resist is limited and the notification may be subject to a gag order that prevents it. The SCC provision exists; the real-world protection it provides against a valid CLOUD Act order is narrow. The only structural approach that avoids CLOUD Act exposure entirely is infrastructure that is not subject to US jurisdiction — EU-incorporated companies, with EU-domiciled infrastructure, and no US parent company with control over the entity. Several European AI infrastructure providers market this explicitly as a selling point for regulated industries. It is a legitimate differentiator for use cases where the CLOUD Act risk is assessed as unacceptable. What actually reduces exposure — and what just creates paperwork For European companies that need the capability of major US-hosted foundation models and have concluded the business case outweighs the residual risk, the realistic risk reduction available is a layered approach, not a single mechanism. The most practical technical control is pseudonymisation before data leaves the organisation. If the data reaching the AI model doesn't contain directly identifying information — names, identification numbers, email addresses, and other direct identifiers have been replaced with tokens, and the re-identification key stays on-premise — then what the vendor receives, and what any CLOUD Act order would compel them to produce, is pseudonymised data. Pseudonymised data is still personal data under GDPR, so this doesn't eliminate GDPR obligations. But it significantly reduces the impact of a disclosure event. Combined with this: SCCs in place as a fallback, a documented assessment of the DPF's continued validity as a transfer mechanism, a clear record of what data categories are permitted to enter the AI model and which are not, and a written acceptance of the residual CLOUD Act risk at the appropriate seniority level in the organisation. That last point matters. The risk assessment is not complete until someone with authority has reviewed and signed off on the residual risk — not because sign-off eliminates the risk, but because undocumented accepted risks are a different regulatory problem from documented ones. What doesn't reduce exposure: signing the SCC addendum in the vendor's standard contract pack without reading it; obtaining a data residency commitment without understanding it doesn't address CLOUD Act; and treating DPF certification as equivalent to a permanent adequacy finding. These create documentation. They don't close the exposure. What to take from thisHave SCCs in place as a baseline, but treat them as necessary rather than sufficient. Understand what they actually obligate the vendor to do and where US law creates limits on what those obligations can achieve. Assess the EU-US Data Privacy Framework as a risk position, not a permanent mechanism. Build SCCs as a fallback and document the contingency position for the scenario where the DPF is challenged successfully — which is not an unlikely scenario given the historical pattern. Understand that data residency commitments from US vendors do not provide CLOUD Act protection. A contractual guarantee that data stays in EU infrastructure is a meaningful operational control; it is not a jurisdictional change. Implement pseudonymisation before data exits the organisation for AI processing. This is the most practical technical control available for reducing the impact of any disclosure event — whether that's a CLOUD Act order, a vendor breach, or a DPF invalidation. Document the accepted residual risk at the appropriate executive level. A risk that has been assessed, quantified, and signed off by someone with authority is a different compliance position from a risk that has been ignored. Regulators distinguish between the two. For use cases where CLOUD Act exposure is genuinely unacceptable — particularly in regulated industries, defence-adjacent work, or cases involving highly sensitive personal data — evaluate EU-incorporated AI infrastructure providers as the structural answer, not a contractual workaround.Cross-border data transfers in AI are a live compliance question with no clean answers at present. The honest position is that European companies using US-hosted AI models are operating in a legal environment that is less stable than the documentation in their procurement files suggests. That's not a reason to stop — it's a reason to understand the actual risk clearly, document what has been accepted and why, and hold a position that survives scrutiny when examined. The organisations that have done that work are in a defensible position. The ones that assumed the vendor's standard agreement covered everything are not.

Read full article
European Company Data in US AI Models: What Your Executives Don't Know Is Already Out

European Company Data in US AI Models: What Your Executives Don't Know Is Already Out

Six months after a European CTO signed off on an AI programme, the DPO sent a single-line email: which data has been sent to the model, where is it stored, and how long is it kept there? Nobody had an answer. The programme had been running on a standard enterprise API agreement. The vendor was reputable. The legal team had reviewed the terms. But the question of what data was leaving the organisation — in what form, to which infrastructure, under what retention policy — had never been mapped. I've seen versions of this conversation more times than I'd like. The AI programme gets approved at the business case level. The vendor agreement goes through procurement. The DPO gets looped in late, or not at all. By the time anyone asks the data flow question seriously, the model has been running for months and the answer is complicated. This isn't a scare piece. It's a map of what actually leaves a European organisation when it runs AI workloads on external models — because the gap between what executives believe is happening and what the contract actually governs is wider than most teams realise. GDPR has been in force since 2018. The enforcement precedent in the AI space is now live. And the organisations that haven't done the data flow mapping are running programmes that haven't been assessed for compliance. What enters a prompt has left the organisation Every query sent to an AI model — every instruction, every document, every piece of text — is transmitted to and processed on the vendor's infrastructure. This is obvious in principle and consistently underestimated in practice. The problem isn't that employees use AI tools. The problem is that the data flowing through those tools hasn't been classified before it flows. When a salesperson pastes a customer contract into a prompt to generate a summary, that contract has left the organisation. When an HR manager uploads a performance review to ask for a draft response, that review has left the organisation. When a finance analyst feeds a quarterly forecast into a model to restructure the narrative, that forecast has left the organisation. None of this requires a policy violation. It happens in ordinary, productive use — which is exactly why it's difficult to control retrospectively. Most enterprise AI APIs retain prompt data for abuse monitoring purposes unless the customer has explicitly opted out — and opted out in writing, in a way the contract records. Retention periods vary by vendor. They are subject to change. I have reviewed enterprise AI agreements where the data retention section was three lines long and referred to a separate policy document that had been updated four times in eighteen months. The business owner who approved the tool had never read either version. Data classification before any AI tool goes into production is not optional — it is the prerequisite. Without knowing what data is sensitive, personal, confidential, and commercially restricted, there is no basis for deciding what can enter a prompt. Most European enterprises have data classification policies. Most of those policies were written before AI prompt data was a category that needed to be governed. Personal data in prompts — minimisation is not enough When the data entering a prompt includes names, email addresses, identification numbers, health records, or financial information, the transfer isn't just a data governance concern — it is a GDPR event. Personal data has moved to an external processor, potentially outside the EU, under whatever transfer mechanism (or lack of one) applies to that vendor relationship. The standard response inside most European enterprises is to invoke data minimisation: GDPR's Article 5(1)(c) principle that personal data should be adequate, relevant, and limited to what is necessary. In theory, this means sending only the fields required for the AI task. In practice, it almost never works that way. Teams send full customer records because the model produces better outputs with more context. HR workflows send complete personnel files because partial information leads to incomplete responses. Customer service AI is fed entire interaction histories because isolated records produce generic answers. The business logic for sending more is always compelling, and the minimisation principle gets applied at the policy level while the actual data flows ignore it. Even when minimisation is applied properly, it is not a compliance answer for personal data. A dataset that contains only a customer's name, email address, and account status is minimal — and it is still fully personal data under GDPR. Minimising what you send does not change the legal classification of what arrives at the model. The transfer still requires a lawful basis. The vendor still needs to be a compliant data processor. The cross-border transfer rules still apply. The only way personal data falls outside GDPR scope when it reaches an external AI model is if it has been genuinely anonymised — not minimised, not pseudonymised, but anonymised to the standard where re-identification is not reasonably possible for anyone, including the model provider. That standard is far harder to meet than most teams assume, and AI models introduce a specific re-identification risk through pattern learning that conventional processing does not. This is covered in full in a separate post in this series. The practical position for European enterprises: names, email addresses, and any other direct identifiers should not enter an external AI model unless a valid GDPR lawful basis, a compliant data processing agreement, and a valid cross-border transfer mechanism are all in place — and verified, not assumed. The metadata picture your vendor is accumulating Prompt content is the obvious exposure. Usage metadata is the one that organisations consistently miss. AI vendors collect usage patterns: query frequency, document types, feature usage, workflow structures, which teams use the model for what. Individually, these data points look low-sensitivity. Collectively, over time, they build a detailed picture of how the organisation operates — what it's working on, where it's directing AI resources, which functions are scaling their usage and in what direction. This is a competitive intelligence risk that exists independently of whether any personal data is involved. A vendor who can see that a company's legal team started using the model heavily in March, its M&A function spiked usage in April, and its HR team has been running high volumes of termination-related drafts since May has learned something meaningful — without any named individual appearing anywhere in the data. Enterprise agreements typically don't restrict vendors from using aggregated, anonymised usage data for product improvement and benchmarking. The data processing agreement and the main commercial contract often address prompt data and metadata under different provisions, negotiated at different points in the contracting process, with different protections attached to each. The metadata protections are almost always weaker. Read both documents carefully — not just the DPA summary the vendor sends as a compliance checkbox. The training data assumption most enterprise teams get wrong There is a persistent assumption in European enterprises that the enterprise version of an AI tool doesn't use customer data for training. Sometimes this is correct. Often it is an assumption that hasn't been verified against the current version of the data processing agreement. Consumer-grade AI tools — the non-enterprise versions of products your employees are also using — may by default use user inputs to improve the underlying model. This is processing for a new purpose. Under GDPR, processing for a new purpose requires a separate lawful basis. When an employee uses the consumer version of a tool on work data because it's faster or the enterprise licence hasn't been set up on their machine, the organisation has a problem it didn't create deliberately but owns entirely. Enterprise agreements typically include data processing agreements that prohibit training on customer data. Three things determine whether that prohibition is actually protecting the organisation: whether it applies to all data types or only certain categories; whether it covers all infrastructure the vendor uses, including subprocessors; and whether the DPA reflects the current model architecture. I have seen enterprise agreements where the DPA predated a major vendor platform change by over a year. The prohibition on training was still in the original terms — which no longer accurately described what was happening with the data. The check is straightforward. Request the current, signed DPA. Confirm it covers training, all subprocessors, and the current model version. Don't assume continuity because the vendor's name hasn't changed. When model outputs carry a breach risk This risk sits with the deployer — and it is the one most European organisations haven't included in their AI risk registers. If a model has been trained on data that included personal information, it can under certain conditions reproduce elements of that information in outputs. Not reliably, not on demand — but under specific prompting patterns, models have been shown to surface fragments of training data. If European employee or customer data was part of the training set, and the model surfaces it in a response, the deploying organisation has a data breach on its hands. Under GDPR, that breach sits with the deployer. The model provider may carry liability, but the deployer cannot transfer its own. For foundation models from major providers, the deployer typically has no visibility into training data composition. The risk is difficult to quantify and impossible to eliminate entirely. For systems where the deployer has fine-tuned or augmented a foundation model with internal data — retrieval-augmented generation, fine-tuning on internal documents, embedding proprietary content — the deployer has direct control over what went in. That control increases responsibility. Before augmenting or fine-tuning a model with internal data, the architecture review needs to answer: what personal data is in these documents, what happens if the model later surfaces fragments of it, and what technical controls reduce that risk? This is a question most teams are not asking at the design stage. It becomes the question they wish they had asked after an incident. What the enterprise agreement covers — and what it doesn't The enterprise agreement creates contractual obligations. It doesn't create infrastructure controls. The data processing agreement requires the vendor to handle data in certain ways. It doesn't prevent US law enforcement from issuing a lawful demand to a US-incorporated vendor under US law — a structural problem I address in the next article in this series, focused specifically on cross-border transfer risk. What the enterprise agreement also doesn't replace is the audit trail that European regulators now expect to exist. The June 2025 guidance from Germany's data protection conference (DSK) requires documentation across the AI system lifecycle. That documentation needs to exist before regulators ask for it — not in the forty-eight hours after they do. For European companies with German operations, this requirement is not optional and it is not the vendor's responsibility to create. The enforcement precedent is live. A major AI provider was fined €15 million by Italian regulators in December 2024 for GDPR violations — failure to establish a lawful processing basis and inadequate transparency around how user data was handled. European regulators now have precedent, process, and political pressure to continue in this space. The organisations currently running AI programmes without a data flow map are running programmes that have not been assessed for compliance. The gap between those two states is not a legal technicality — it is the gap between a defensible position and one that cannot be defended when examined. What to take from thisMap data flows before deployment, not after. Every AI tool processing external data should have a documented flow — what enters, what is retained, where it's stored, for how long — completed before the tool goes live, not reconstructed months later. Read the current DPA, not the version in the procurement file. Data processing agreements are updated. Request the current signed version and confirm it covers training, subprocessors, and the current model architecture. Classify data before it enters a prompt. Sensitive, personal, commercially restricted, and confidential data should be classified before it enters any AI tool — not assessed after six months of use. Do not treat data minimisation as a substitute for anonymisation. Sending fewer fields does not change the legal classification of what arrives at the model. Personal data that has been minimised is still personal data — names, email addresses, and identifiers stripped down to the "minimum necessary" still carry full GDPR obligations. If personal data cannot be genuinely anonymised before it reaches an external model, the transfer needs a lawful basis, a compliant DPA, and a valid cross-border mechanism — not a smaller dataset. Treat usage metadata as a competitive risk, not just a data protection issue. Understand what your vendor collects on usage patterns, what restrictions apply, and what they're permitted to do with aggregated data. Enforce a clear policy on consumer versus enterprise tool versions. If employees have access to both, the policy needs to specify which applies to which data — because the data handling terms are materially different. Include model output re-identification in your AI architecture review. For any system fine-tuned or augmented with internal data, answer the re-identification risk question before build — not after the system is in production.GDPR has been in force since 2018. The data protection gap in AI programmes isn't a new legal problem — it is an existing problem that the pace of AI adoption is making visible at scale. Most of what European companies need to do to close this gap doesn't require waiting for new regulation. It requires applying existing obligations to a class of tools that moved faster than the governance frameworks around them.

Read full article
What AI Adoption Does to Your Existing Technology Contracts

What AI Adoption Does to Your Existing Technology Contracts

Deploying AI into an enterprise technology stack does not happen in isolation. It happens into an existing web of contracts: software licenses, SaaS agreements, data processing terms, and vendor relationships that were written before AI capabilities were relevant and that were not designed to accommodate what an AI program requires. The collision between AI deployment and existing contracts produces a category of problem that most organizations encounter somewhere in the middle of delivery, after commitments have been made and timelines are set. The CIO and general counsel who review the contract landscape before deployment starts are in a substantially better position than those who discover the issues under delivery pressure. There are five areas where existing contracts tend to create friction for AI programs. Software licensing terms and data use Many enterprise software licenses include restrictions on how data within the system can be used beyond its primary purpose. The typical language covers authorized users, permitted use cases, and sometimes explicit restrictions on automated processing or data extraction. When an AI system is connected to a licensed software platform to extract, process, or train on the data within it, those restrictions may be relevant. A CRM contract that limits data use to direct customer relationship management may not automatically permit the creation of an AI training dataset from CRM records. A document management system license that covers authorized human users may not straightforwardly cover an AI agent that queries the system as part of an automated workflow. The likelihood that existing enterprise software licenses explicitly address AI use cases is low — most were written before those use cases were anticipated. The risk is that implicit prohibitions, generic restrictions on automated processing, or data use limitations apply in ways that neither party anticipated. Before connecting AI systems to existing licensed platforms, general counsel should review the relevant license terms for restrictions on data use and automated access, and engage with vendors where the position is unclear. SaaS data portability and processing terms SaaS agreements typically govern what data is held in the platform, how it can be exported, and what the vendor can do with it. The standard SaaS agreement, particularly for products that predate the current AI era, was written with human-facing use in mind. When an AI program requires bulk data extraction from a SaaS platform — to populate a training dataset, to build a knowledge index, to migrate data to an AI-ready format — the agreement may not straightforwardly permit this. Data export limitations, API rate limits, and format restrictions may be in the contract in ways that constrain what the AI program needs. The practical issue: discovering mid-implementation that a bulk data extraction is contractually restricted by an existing SaaS agreement is a problem that takes time to resolve. Vendor negotiations to expand export rights, technical workarounds, or alternative data sourcing each add delay and cost that was not in the original plan. Review SaaS agreements for any data that the AI program will need to process, extract, or migrate before the implementation schedule is set. Existing AI vendor agreements and scope creep Organizations that have been using AI tools for some period often have existing vendor agreements that define the scope of permitted use cases. As the AI program expands, new use cases may not be covered under the existing agreement. This matters specifically in two ways. First, using an AI vendor for use cases outside the defined scope of the agreement — even with the same tool and the same vendor — may create data handling situations the original agreement did not contemplate. Second, enterprise AI agreements often include pricing that is tied to defined use parameters. Expanding use significantly beyond those parameters may trigger renegotiation on terms less favorable than the original agreement. Audit the scope of existing AI vendor agreements against the planned AI program before expansion. Know what is and is not covered before the program is designed around assumptions about what the vendor relationship permits. Data processing agreements for third-party integrations AI programs frequently involve connecting internal data to third-party AI systems through integrations: an AI tool connected to the CRM, an AI analytics layer over the data warehouse, an AI agent with access to internal APIs. Each of these integrations creates a data flow that may require a data processing agreement. Where the integrated party processes personal data on behalf of the organization, a data processing agreement is a regulatory requirement under data protection law in most jurisdictions. For integrations that existed before the AI layer was added, the original data processing agreement may not cover the additional processing the AI component involves. Before adding AI components to existing integrations, review whether existing data processing agreements need to be updated to reflect the expanded processing scope. The integration may be technically unchanged from the data flow perspective while creating a materially different processing activity for regulatory purposes. Third-party data in AI training and indexes Organizations often use third-party data sources — market data, industry benchmarks, licensed research, external databases — in their operations. When an AI program wants to use this data as training material or as content in a retrieval index, the license for that third-party data may not permit it. Third-party data licenses typically specify permitted use cases: internal analysis, reporting, specific product use. Training an AI model on licensed data, or including it in an AI knowledge base that generates outputs for a broad user population, may constitute a use case that the license does not cover. The risk is real, the issue is common, and the discovery process for finding all the affected data sources takes time. Conduct a data sourcing review for any data that will go into AI training sets or retrieval indexes. Identify all third-party licensed content, review the license terms for AI use restrictions, and either obtain the necessary permissions or exclude the content. Practical approach for the CIO and general counsel The scale of this review problem varies significantly by organization. For most enterprises, the contract review is manageable if it is structured and approached systematically before the AI program enters delivery. Start with the data sources the AI program will use. Map every system, database, and data feed the AI system will connect to. For each, identify the governing contract and whether it has been reviewed for AI-relevant terms. Prioritize by data volume and sensitivity. The highest-volume data sources feeding the AI program, and the data categories with the most regulatory and contractual complexity, deserve the most thorough review. Engage vendors early where the position is unclear. Vendors generally prefer to resolve license ambiguity before it becomes a dispute. A proactive conversation about AI use cases typically produces better outcomes than a post-hoc assertion that something was permitted. What to take from thisSoftware licenses and SaaS agreements often contain restrictions on automated processing and data use that predate AI and may apply to AI use cases in ways neither party anticipated. Review them before deployment. Bulk data extraction requirements for AI programs may be restricted by existing SaaS agreements. Discover this before the implementation schedule depends on it. Expanding AI use cases beyond the scope defined in existing AI vendor agreements can create data handling and pricing issues. Audit current agreements against planned use. Data processing agreements need to be updated to reflect AI components added to existing integrations. The regulatory obligation does not adjust automatically when the technical architecture changes. Third-party licensed data in AI training sets or retrieval indexes may require explicit permission that the existing license does not provide. Conduct a data sourcing review before building the training set.

Read full article
The Knowledge That Walks Out the Door When Employees Use AI on Client Work

The Knowledge That Walks Out the Door When Employees Use AI on Client Work

Professional services firms — consulting, legal, accounting, advisory — have a specific relationship with client data that is different from most enterprise AI contexts. The data they handle belongs to their clients. The confidentiality obligations around it are contractual and, in many cases, professional and regulatory. The consequences of a breach are not limited to regulatory exposure; they extend to client trust, which is the fundamental asset in any advisory relationship. AI tools are now deeply embedded in how professional services work gets done. Analysts use them to accelerate research. Consultants use them to draft documents. Lawyers use them to review contracts. The productivity benefits are real and the competitive pressure to use them is significant. The risk accumulation that comes with that use is largely unaddressed. What client data actually flows through AI tools in professional services The volume and sensitivity of client data flowing through AI tools in professional services contexts tends to be higher than in most enterprise settings, because the work itself involves processing and analyzing client-owned information. Project research and analysis. Analysts feed client financial data, market analysis, and competitive benchmarks into AI tools to accelerate synthesis. The client's internal data, which the firm has received under a confidentiality agreement, enters a third-party AI system. Document drafting. Consultants use AI writing assistants to draft recommendations, presentations, and reports. The source material that informs the drafting — interview outputs, internal data, strategic context — is included as context for the tool. Contract review and legal analysis. Legal and advisory professionals use AI tools to review and summarize contracts, due diligence materials, and transaction documents. These materials contain some of the most sensitive information clients possess. Meeting summaries and communication assistance. Client meeting recordings processed through meeting AI tools. Client correspondence drafted with AI assistance. Internal discussions about client situations entered as context for AI queries. Each of these flows involves client-confidential data entering a third-party AI system. Most firms have not mapped this systematically. Many assume it falls under the general confidentiality terms in their client agreements without having verified that the AI tool's data processing terms are compatible with those obligations. The contractual gap that most firms have not closed Professional services firms operate under engagement letters and master services agreements that include confidentiality provisions. These provisions were written before AI tools existed in their current form. They typically cover how the firm handles client confidential information: where it is stored, who has access, what the firm's obligations are around disclosure. What they almost never address: whether the firm can process client confidential information using third-party AI tools, and if so under what conditions. This creates a gap. The firm has agreed to keep client information confidential. The firm's employees are feeding that information to third-party AI systems. Whether that constitutes a breach of the confidentiality provisions depends on the specific language and how it would be interpreted, which is not a comfortable analysis to be doing reactively. Some clients are now asking about this proactively in RFPs and at the start of engagements. Firms that have a clear, honest answer to the question "do your employees use AI tools when working on our engagement, and if so how is our data handled" are in a better position than those who have not worked out an answer. The knowledge residue problem There is a second dimension to this risk that is less obvious than the direct data exposure question. When an employee works with client information through an AI tool over the course of an engagement, the contextual knowledge they develop about the client's situation is richer and more detailed than it would be if they had processed the same information manually. The AI tool allows them to work across more data, make more connections, and develop a more comprehensive understanding than time would have permitted through manual analysis. This enriched understanding lives in the employee's head when they walk out the door. When that employee moves to a competitor or, in certain conflict situations, works on a client in a similar competitive situation, the depth of knowledge they carry creates an exposure that goes beyond the normal knowledge transfer risk. The firm cannot fully control what employees internalize through their work. That has always been true. AI tools increase the depth and breadth of what an employee can internalize over a fixed period of time. The risk management implications are subtle but real. What governance looks like in practice The minimum governance framework for a professional services firm using AI tools on client work: An explicit AI use policy that covers client work. This should specify which AI tools are approved for use on client matters, what categories of client data can be processed through AI tools under what conditions, and what the data handling terms are for approved tools. This is different from the general employee AI policy — it needs to address the confidentiality obligations that are specific to client engagements. Client engagement agreement updates. The confidentiality provisions in engagement letter and master services agreement templates need to be updated to address AI tool use. At minimum, the provisions should not preclude AI use in ways that are inconsistent with how work is actually being delivered. Better than that: the provisions should address AI tool use explicitly, with appropriate confidentiality protections around how client data is handled within those tools. Client disclosure for high-sensitivity matters. For engagements involving particularly sensitive information — M&A transactions, regulatory matters, litigation, restructuring — the engagement team should have a protocol for discussing AI tool use with the client and obtaining explicit confirmation about what is acceptable. Employee education that is specific to client work. General AI use training does not address the confidentiality implications specific to professional services. Employees handling client confidential information need to understand what AI tools they can use, with what data, under what terms, and what their obligations are when in doubt. The question clients are starting to ask The most direct signal that this needs to be addressed now: clients are beginning to ask about it. Not often, but the frequency is increasing, and the questions are getting more specific. "Does your team use AI tools when working on our matters?" "If so, does our confidential information enter those AI systems?" "What are the data handling terms for the AI tools you use, and how do they interact with your confidentiality obligations to us?" A firm that has thought about these questions and has clear answers is in a different position from one that has to formulate an answer under client scrutiny. The latter tends to produce either an evasive answer that damages trust or a defensive answer that raises more questions than it resolves. What to take from thisMap what client data is flowing through AI tools on active engagements. The volume is almost certainly higher than any single partner or manager would estimate. Review whether existing client confidentiality provisions are consistent with how AI tools are actually being used in client delivery. The gap is likely to be meaningful. Update engagement agreement templates to address AI tool use explicitly, before clients start asking for it in contract negotiations. Develop a protocol for client disclosure on high-sensitivity matters. The default should be proactive transparency, not reactive disclosure. Train client-facing staff specifically on the confidentiality implications of AI tool use in their context. Generic AI training is not sufficient for professional services.The firms that handle this well are not necessarily the most cautious ones. They are the ones that have been honest about how AI is being used in client delivery, have updated their agreements to reflect that, and can answer client questions about it clearly and without hesitation.

Read full article
Training AI on Proprietary Data: What You Gain and What You Give Up

Training AI on Proprietary Data: What You Gain and What You Give Up

At some point in most enterprise AI programs, someone proposes using the organization's own data to improve the model. The pitch is compelling: a model trained on your internal documentation, your historical decisions, your domain-specific knowledge will perform better on your actual work than a general-purpose model that knows nothing about your context. The pitch is not wrong. Done well, training or fine-tuning a model on proprietary data can produce a genuinely better system. The problem is that "done well" requires a set of decisions that most organizations have not thought through, and the downside of doing it carelessly — from data exposure, from quality degradation, from regulatory exposure — is substantial. This is not an argument against using proprietary data to improve AI systems. It is an argument for approaching that decision with the same rigor you would apply to any decision about where your most valuable data goes. What training on proprietary data actually involves There are a few different mechanisms by which proprietary data can improve model performance. The distinctions matter for understanding the exposure. Retrieval-augmented generation (RAG). The model is not trained on proprietary data at all. Instead, a retrieval system fetches relevant internal documents at query time and provides them as context for the model to work from. The proprietary data sits in a controlled index that the organization manages. The model itself stays unchanged. This approach avoids most of the training data risks while providing significant performance improvement on domain-specific tasks. Fine-tuning. The model's internal weights are adjusted using proprietary data. The model itself changes — it becomes better at tasks that reflect the patterns in the training data. The proprietary data has, in a meaningful sense, been absorbed into the model. This is more powerful than RAG for certain tasks, and considerably more complex from a data governance standpoint. Full training from scratch. Building a model entirely on proprietary data, without starting from a pre-trained foundation. This is rare at the enterprise level — the compute and data requirements are significant — but it does happen for organizations with specialized domain requirements and the resources to invest. The data exposure implications are very different across these approaches. RAG keeps proprietary data in an index the organization controls. Fine-tuning moves it into the model weights. The latter is where the tricky questions live. What gets encoded in a fine-tuned model When data is used to fine-tune a model, the specific documents and content do not get stored inside the model as retrievable files. The model cannot be queried to reproduce its training data verbatim — in most cases. But the training data has shaped the model's behavior in ways that are difficult to audit or reverse. This matters in two respects. First, sensitive information can influence the model's outputs in ways that are hard to anticipate. A model fine-tuned on internal strategy documents may, when asked questions that touch on those topics, produce outputs that reflect internal strategic positions without anyone intending to reveal them. The model is not leaking documents — it is producing outputs shaped by the patterns in documents it has seen. The effect is subtle and hard to trace. Second, fine-tuned models can, under certain adversarial conditions, be prompted to surface information that reflects their training data more directly than normal. This is not a theoretical risk — it is an active research area, and the techniques for eliciting training data from fine-tuned models are improving. Organizations that fine-tune on genuinely sensitive data on models hosted by third parties should factor this risk into their assessment. The vendor relationship in fine-tuning When an organization fine-tunes a model that is hosted by a cloud AI vendor, several questions about the proprietary data need to be answered clearly before proceeding. Does the vendor use the fine-tuning data to improve their base models? Most enterprise agreements exclude this, but it should be a named contractual commitment. Who can access the fine-tuning data? During the fine-tuning process, the data is processed on the vendor's infrastructure. Access controls, the circumstances under which vendor employees can access training content, and the security measures around the fine-tuning pipeline should all be part of the assessment. What happens to the fine-tuning data after training is complete? Is it retained, for how long, and can it be deleted? What does deletion mean in the context of a model that has already been trained on it — a question vendors are not always able to answer cleanly, because once data has influenced model weights, the effect cannot be fully reversed. Where is the fine-tuned model stored, and who controls it? The fine-tuned model is organizational IP. The vendor relationship needs to be clear about ownership, portability, and what happens to the fine-tuned model if the organization ends the vendor relationship. Data quality is the amplification lever One risk that gets less attention than the security questions: the quality and composition of the training data determine whether fine-tuning improves or degrades model performance. Organizations that rush fine-tuning treat it as an input optimization problem — more data is better. That is incorrect. Noisy, inconsistent, or poorly representative training data produces a fine-tuned model that performs worse than the base model on the tasks that matter, and sometimes produces dangerous outputs in domains where the training data was biased or incomplete. I have seen organizations fine-tune models on documentation that contained outdated processes, contradictory guidance, and examples of known-bad decisions that were documented as cautionary tales rather than as positive examples. The resulting model incorporated those patterns along with the useful ones. The outputs were confidently wrong in ways that were hard to trace back to the training data quality issue. Before committing to fine-tuning, the data curation question deserves as much attention as the technical process. What data will be included, what will be excluded, who reviews the training set for quality and appropriateness, and how will the fine-tuned model's behavior be evaluated against the base model are all part of a credible fine-tuning program. When it is worth it Fine-tuning on proprietary data is worth the complexity when two conditions are both true: the performance improvement on the target task is significant and demonstrable, and the data exposure risks can be managed through a combination of vendor agreement terms, data classification, and governance. RAG should be the default approach for most enterprise use cases. It provides substantial performance improvement without the fine-tuning data exposure risk, and it is operationally simpler to maintain and update. Fine-tuning becomes worth the investment when the task requires deep pattern recognition across the organization's specific domain that a retrieval approach cannot fully replicate. The decision to fine-tune should not be made by the AI team alone. It needs the CTO's visibility into the data exposure implications, legal review of the vendor terms around training data, and a clear data classification decision about which content is and is not appropriate as training material. What to take from thisRAG and fine-tuning are different. RAG keeps proprietary data in an index the organization controls. Fine-tuning moves patterns from that data into model weights. They have different data exposure profiles. A model fine-tuned on sensitive data can surface that information in ways that are hard to predict or audit. This is not a reason to avoid fine-tuning, but it is a reason to be deliberate about what goes into the training set. When fine-tuning on a third-party hosted model, get contractual clarity on training data use, retention, and what happens to the data and model at contract end. Data quality in the training set is as important as security. Noisy, inconsistent, or outdated data produces a fine-tuned model that may perform worse than the base model. Make fine-tuning a CTO-level decision with legal review, not a technical team default. The data exposure implications require organizational sign-off, not just technical execution.

Read full article
Defining AI Success Before Anyone Commits a Budget

Defining AI Success Before Anyone Commits a Budget

I've sat in enough AI program kick-off meetings to have stopped being surprised by this: twenty people in a room, a board mandate to "move fast on AI," a vendor selected, a team forming — and nobody has yet agreed on what success looks like. Not in specific terms. Not in a way that would allow someone to come back in twelve months and determine whether the program worked. This isn't unusual. It's the norm. And it's the single failure mode I see most consistently across enterprise AI programs, regardless of industry, company size, or technical maturity. The reasons are understandable. There's pressure to start. Defining success precisely feels constraining when the technology is new and the possibilities feel open. Executives are comfortable with directional goals — "improve customer experience," "reduce operational cost," "increase fraud detection" — and less comfortable with the kind of specificity that creates a clear accountability line. Specificity means someone owns the number. Owning the number means someone can be wrong. The cost of this discomfort is high. Why technical metrics aren't the answer The first place program teams turn when asked to define success is model performance metrics. Accuracy, F1 score, AUC, precision-recall tradeoffs. These are measurable, they're familiar, and they can be calculated before the model is in production. They're also not what the business cares about. A model with a 94% accuracy rate that nobody uses isn't a success. A model with 87% accuracy that improves a critical business process by a measurable amount is. The gap between technical performance and business outcome is where most AI programs lose the narrative — and where the CFO eventually starts asking whether the investment was worth it. Technical metrics are necessary for model development and monitoring. They're not sufficient for program success definition. What the business cares about is downstream: decisions improved, costs reduced, revenue generated, risk reduced. Those are different measurements, and the relationship between model performance and business outcome needs to be made explicit at the start, not assumed. What a complete success definition contains A success definition that can be used to evaluate the program twelve months from now has four components. A specific metric. Not "improve fraud detection" — a number the business can measure. False negative rate, dollar value of fraud losses prevented, percentage of fraud alerts requiring manual review. The metric needs to be something that exists in a system the business actually maintains, not something that has to be constructed after the fact. A documented baseline. What is the current value of that metric, measured against the same methodology that will be used to measure the AI-driven result? Without a baseline, you can't measure improvement. Without a consistent measurement methodology, comparisons are arbitrary. Getting the baseline documented before the program starts is more important than it sounds — it eliminates a whole category of disagreement about whether the program worked. A numeric target. A direction is not a target. "Improve" is not a target. "Reduce false negative rate from 4.2% to below 2.5%" is a target. The target should be challenging enough to justify the investment and specific enough to be unambiguous. A timeframe. By when? The timeframe determines the evaluation rhythm and gives the program team a real deadline against which to calibrate pace. Without it, the target floats indefinitely. Without all four components, the success definition isn't complete. It's a directional goal dressed up as a commitment. The traps Several common patterns make success definitions look complete when they aren't. Vanity metrics are things the program team can control that don't connect to outcomes the business cares about — models built, data sources integrated, team size, features shipped. These are useful operational metrics. They're not success metrics. A program that reports these as evidence of success has redefined success to be about activity rather than outcome. Unmeasurable outcomes are aspirations that can't be tracked. "Become an AI-native organization." "Embed AI into our culture." These may represent genuine long-term goals. They cannot be evaluated in a twelve-month program review, and including them as success criteria gives the program team permanent cover against accountability. Metrics the business can't track are a trap that sounds technical but isn't. If measuring the success metric requires access to data the business doesn't actually maintain, or calculations the business doesn't currently run, the metric will be reported inconsistently or not at all. Success metrics need to be things the business can measure on a monthly or quarterly basis with existing data infrastructure. Proxy metrics that drift from the target outcome are the hardest to catch. An AI program for customer service might measure success by handle time reduction. But if the model reduces handle time by routing calls to hold queues rather than resolving queries, the metric looks good while the outcome is bad. The connection between the proxy and the real outcome needs to be validated, not assumed. Running the conversation with executives who prefer ambiguity The executives most resistant to specific success definitions are usually the ones with the most at stake. Specificity creates accountability, and accountability creates risk. Understanding that the resistance is rational rather than evasive changes how you approach the conversation. The approach that tends to work: starting not from "what does success look like" but from "what would change your mind." Ask the sponsor what result, at what point in time, would cause them to question whether the program was working. Ask the CFO what the program would need to show at month twelve to be considered a good investment. Ask the business unit head what their team would need to see to start relying on the model's outputs. These questions come at the success definition from the outside — from what a skeptic would need to see to be convinced — rather than from the inside, where optimism tends to inflate targets and round off the hard edges. They also surface the implicit assumptions about what success looks like that are already in the room, undiscussed. The success definition document that comes out of this conversation doesn't need to be complex. It needs to be signed — literally. An agreed definition, documented and acknowledged by the program sponsor, the business lead, and the CFO or their representative. The act of signing matters because it makes the definition a commitment rather than a suggestion. What happens when you skip this step Programs without defined success criteria don't fail suddenly. They drift. Twelve months in, there's a review. The program has made progress — models are built, pilots are running, the team has learned a lot. Nobody can agree on whether the program is working because nobody agreed at the start on what "working" would mean. The sponsor points to the positive signals. The CFO points to the cost. The business unit says the outputs aren't quite what they needed. The program continues — not because it's succeeding, but because it's not clearly failing. Two years in, the program has consumed significant investment with ambiguous returns. The next budget cycle is where it gets cut — not in a formal review, but quietly, when the sponsor doesn't go to bat for it. The program closes with a retrospective report that describes what was learned rather than what was delivered. That's not a technical failure. It's a governance failure that started on day one, and it was entirely preventable.

Read full article
Shadow AI: What's Already Running in Your Organization

Shadow AI: What's Already Running in Your Organization

Before any formal AI program exists, before the steering committee has its first meeting, before the CIO has signed off on a single vendor — your employees are already using AI. Not because they are reckless. Because they have work to finish. I have walked into organizations that spent six months building an AI governance framework and discovered, halfway through the engagement, that the finance team had been using a consumer large language model to draft board reports for the past year. The legal team was using a different one to summarize contracts. Neither team thought they were doing anything wrong. They were getting work done faster. That is the shape of shadow AI. It does not announce itself. It does not appear in your IT asset register. It shows up in the gap between the work people need to do and the tools they have been given to do it. What shadow AI actually looks like The term sounds covert. It rarely is. Most shadow AI use is visible if you look for it — people pasting client briefs into ChatGPT, uploading PDFs to summarization tools, running AI writing assistants over strategy documents. Nobody is hiding anything. They just do not think of it as an IT procurement decision. The categories I see most often: Consumer AI assistants used for drafting, summarizing, researching, and explaining technical material. These are usually the first tools employees reach for because they already use them outside work. The friction is zero. AI features embedded in software people already have — writing assistants in productivity suites, AI tools inside project management and communication platforms. These arrive via a product update, not a procurement decision, and most organizations do not notice until they are already in use. Specialist tools for specific functions: AI contract review, AI coding assistants, AI research tools, AI presentation builders. These typically start as a free trial one person tries. Three months later the whole team is using them, usually without telling anyone. The common thread: easy to access, fast to start, and they solve a real problem. Nobody waits for procurement when they have a real problem. The data that leaves when they do Here is the question I put to leadership teams: what data has left your organization in the last 90 days through AI tools? Almost nobody has an answer. Every prompt sent to a third-party AI tool is data that has left the building. Every document uploaded for summarization. Every contract pasted in for analysis. Every email thread fed to an assistant for context. This is not theoretical exposure. It is live, ongoing, and unmeasured. The specific risk depends on the tool. Some consumer AI products train future models on user inputs by default unless the user has explicitly opted out — a setting most enterprise users have never opened. Some retain query data for extended periods for internal product improvement. Some have data residency terms that have no relationship to the organization's regulatory obligations. Most employees have read none of this. What makes shadow AI data exposure different from other shadow IT is the combination of volume and sensitivity. When someone uses an unsanctioned SaaS product, they tend to generate new data inside that system. When someone uses an unsanctioned AI tool, they are typically feeding existing sensitive material — client information, financial projections, internal strategy — into a system with unknown retention, processing, and training terms. A CTO told me once: "We spent a year tightening our cloud storage permissions, and the whole time people were copying strategy documents into a consumer AI chat." He was not exaggerating. He had run the discovery work. The decision quality problem The data risk is one issue. The decision quality risk is a separate one. When employees use AI tools that have not been evaluated or approved, the outputs they receive carry no governance. There is no audit trail of what the model was asked, what it returned, or how that output influenced a decision. Nobody has tested whether the tool performs reliably on the organization's specific data domain. Nobody has checked whether outputs are factually accurate, whether knowledge cutoffs create blind spots, or whether model behavior introduces biases relevant to the use case. I have seen board presentations drafted with consumer AI assistance that contained subtly incorrect market figures — the kind of error that is hard to catch if you are not already deeply familiar with the material. I have seen contract summaries that missed jurisdiction-specific clauses because the model lacked coverage of that legal context. None of these produced immediate disasters. But they were invisible errors that reached decision-makers before anyone thought to check the source. The problem is not that the tools are necessarily unreliable. The problem is that nobody defined what "reliable enough" looks like for this use case, nobody validated that the tool clears that bar, and nobody has visibility into which decisions have been shaped by which tools. Why a policy does not solve this on its own Every organization that discovers shadow AI responds the same way: they draft a policy. Employees must not use AI tools without prior approval. All AI tools must go through procurement. No sensitive data should be uploaded to external AI systems. These policies are not wrong. They are just insufficient on their own. A policy without detection capability is a statement of intent, not a control. If you cannot observe what tools are in use, you cannot enforce anything. And the detection problem is real — consumer AI tools operate over standard encrypted web traffic that looks indistinguishable from any other browser activity on most network monitoring setups. A policy without a usable alternative is a speed bump, not a barrier. If employees are turning to shadow AI because the approved tooling is slow, limited, expensive, or simply does not exist yet, a policy telling them to stop will reduce usage briefly and then have no effect. People optimize for their work. A policy without a clear explanation of why tends to generate resentment rather than behavior change. If employees do not understand what the actual risk is — not just that "data security is important" but specifically what could happen to the specific data they are handling — they will weigh an abstract policy against a concrete productivity benefit and find the policy unconvincing. What actually works There are four interventions that change the situation. Not instead of policy — alongside it. Run a discovery exercise before making any decisions. You need to know what is actually in use before designing controls. This means endpoint monitoring, network traffic analysis, and honest conversations with department heads. Expect surprises. The goal is not to catch anyone — it is to understand your real exposure before you design a response to it. Move quickly on a sanctioned alternative. The fastest way to reduce shadow AI use is to provide a better approved option. This does not require the best enterprise AI platform with a six-month procurement timeline. It means the minimum viable approved tool that addresses the main use cases driving shadow adoption — often that is simply a properly configured, privacy-compliant version of the same tool people are already using. Create a fast path for new tool requests. The reason shadow AI persists is that the formal route takes too long. Teams wait months for IT to evaluate a tool they need now. Make the process faster and more transparent. Most requests should get a decision within two weeks. The ones that cannot should at least get a clear explanation of why not. Treat the people using shadow AI as a signal. They are telling you where your official tooling is falling short. Employees using unauthorized AI tools are often the highest performers trying to work better. Treating them as a compliance problem to be managed misreads what is happening. Their behavior is product feedback. What to take from thisAudit shadow AI use before designing governance for it. You need to know what is already running before you build controls around it. Consumer AI data terms are not written for enterprise compliance. Read them — specifically the sections on input retention, training use, and data residency — before employees continue uploading sensitive material. A policy without detection is not a control. Invest in observability first, then communicate the policy. The fastest fix is a sanctioned alternative that works. Prohibition without substitution creates resentment, not compliance. The employees using shadow AI are showing you where your approved tooling has gaps. Use that information when planning what to procure next.The organizations I see handle this well are not the ones that moved fastest to write a policy. They are the ones that ran the discovery work, understood their actual exposure, and moved quickly to close the gap between what employees needed and what they were officially allowed to use. That gap is where shadow AI lives.

Read full article
When AI Gets It Wrong at Scale: Incident Response for Executive Teams

When AI Gets It Wrong at Scale: Incident Response for Executive Teams

Most enterprise AI incident response planning amounts to "we will deal with it when it happens." Sometimes that is stated explicitly. More often it is implicit — the program plan does not include an incident response section, the governance framework describes oversight without describing what happens when oversight catches something wrong, and the executive team has not run a tabletop exercise that covers AI failure modes. The absence of preparation is not reckless. It reflects where attention goes when building an AI program: toward capability, delivery, and adoption. Incident response feels like a problem for later. It becomes a problem for now at the worst possible time. AI incidents are different enough from traditional software incidents to warrant explicit preparation. The failure modes are different. The scope assessment is harder. The communications are more complex. And the pressure to continue operating while containing the incident — because the AI system may be deeply embedded in workflows that cannot simply be paused — creates tradeoffs that need to be worked out in advance. Here is what the first 72 hours of an AI incident should look like for an executive team that has done the preparation. The AI incident categories that require executive involvement Not every AI malfunction requires executive attention. A model performance degradation that is caught by monitoring, addressed by the AI team, and resolved within hours without affecting users or external parties is an operational incident, not an executive incident. The categories that require executive involvement are those where the impact exceeds what the AI team can contain operationally: Output failures at scale. The AI system produces incorrect, harmful, or biased outputs that have been acted on by a significant population of users, customers, or decision-makers before the problem is identified. The scope of the downstream impact is not immediately clear. Data exposure. The AI system has caused data to be accessed, transmitted, or disclosed in ways that exceed what was authorized — whether through a prompt injection attack, a misconfiguration, or a vendor incident. The regulatory notification question is immediately live. Regulatory or legal trigger. An AI system output has been identified as potentially discriminatory, as having violated an applicable regulation, or as the subject of a legal claim. The legal and regulatory response needs to begin immediately. Reputational incidents. An AI system failure has become externally visible — through media coverage, customer complaints reaching public forums, or regulatory inquiry. The communications response is time-sensitive. Each of these requires a different primary response, and each has a different set of executives in the lead role. The incident response plan should specify who is in the lead for each category, not assume that a single generic incident response structure covers all of them. Hours 0 to 4: initial assessment The first hours of any significant AI incident involve two parallel tracks: containment and assessment. These need to happen simultaneously, because the decision to maintain, restrict, or shut down the AI system needs to be made quickly and on the basis of the best available information. Containment decision. The immediate question is whether the AI system should continue operating. The options range from full shutdown through restricted operation (limiting to lower-risk use cases) to continued operation with enhanced monitoring. The decision depends on the nature of the incident, the criticality of the AI system to business operations, and the risk of further harm if operation continues. This decision needs to be made by the executive sponsor with input from the CTO and, where relevant, the CISO and general counsel. It should not default to the AI team alone, because the business continuity implications extend beyond the technical assessment. Scope assessment initiation. What has the AI system done, to whom, and with what data? The scope assessment for an AI incident is harder than for a traditional data breach because AI systems do not maintain the same kind of access logs that database systems do. The query and output logs of the AI system are the starting point. Reconstructing the impact from those logs requires AI-specific expertise. Assign the scope assessment as a named responsibility to the CTO's function. Set a first checkpoint time — four hours or less for an initial estimate of scope — and a full report timeline. Hours 4 to 24: assessment and notification decisions By the end of the first 24 hours, the executive team needs to have answered several questions that drive subsequent decisions. Regulatory notification. If the incident involves personal data, has the assessment produced enough clarity to determine whether the incident triggers regulatory notification obligations? In most jurisdictions, the notification clock starts at discovery of a breach, and the notification period is 72 hours in many regulatory regimes. This timeline is unforgiving. If the incident involves personal data and there is a reasonable possibility of notification obligation, engage the data protection officer and external legal counsel immediately — do not wait for full scope clarity. Customer and partner notification. Separate from regulatory obligations, does the incident require proactive notification to affected customers or partners? This decision involves commercial judgment as well as regulatory judgment. The general counsel and the CTO together need to produce a recommendation for the executive sponsor. Internal communications. Who inside the organization needs to know about the incident, at what level of detail, and at what stage of the investigation? The communications approach should be deliberate — too narrow creates governance failures; too broad before facts are established creates confusion and external leak risk. Hours 24 to 72: remediation and external response By 48 hours, the executive team should have enough understanding of the incident to make the decisions that need to be made. Remediation plan. What is the AI team doing to fix the underlying problem, what is the timeline, and what assurance does the business have that the fix is complete before the system returns to full operation? The AI team provides the technical answer; the executive team validates that the assurance standard is sufficient. External communications. For incidents that have external visibility — regulatory inquiry, media coverage, significant customer impact — the external communications response needs to be coordinated. The communications team needs to work from accurate, verified information. Premature statements that are later contradicted damage credibility significantly. Regulatory response. If a regulatory notification has been made, assign the regulatory liaison role clearly. The communications with regulators should be managed through a single channel, should be accurate and complete, and should not outrun the organization's actual knowledge of the incident. Building the plan before you need it An incident response plan that is written during an incident is worse than one written in advance. The decisions about who leads which type of incident, what the shutdown criteria are, what the notification thresholds are, and who the external legal and regulatory advisors are need to be made without the time pressure and reputational stakes that an active incident creates. Specifically: run an AI incident tabletop exercise before the first significant AI system goes live. The exercise should cover at least one of the major incident categories — scale output failure, data exposure, regulatory trigger — and should involve the executive team, not just the AI and security functions. The outputs of the tabletop should be a tested incident response plan that reflects the organization's actual structure and decision-making processes. What to take from thisAI incidents are distinct enough from traditional software incidents to require their own response planning. The scope assessment, containment decision, and regulatory notification analysis all require AI-specific expertise and approaches. The containment decision — whether to maintain, restrict, or shut down the AI system — needs to be made by the executive sponsor, not delegated to the AI team. The business continuity implications require that level of authority. For incidents involving personal data, the regulatory notification clock starts at discovery. Engage legal immediately if there is a reasonable possibility of notification obligation — do not wait for full scope clarity. Run a tabletop exercise covering AI incident scenarios before the first significant AI system goes live. Test the plan with the people who will execute it, under conditions that approximate the time pressure of a real incident. External communications for AI incidents need to be coordinated and accurate. Premature statements corrected later are more damaging than a short delay for verification.

Read full article
AI and Third-Party Risk: What Supplier Due Diligence Isn't Covering

AI and Third-Party Risk: What Supplier Due Diligence Isn't Covering

Third-party risk management programs have been through several cycles of expansion in the past decade. After a wave of supply chain security incidents, procurement and legal functions added cybersecurity assessments. After data protection regulation arrived, they added data processing reviews. After financial resilience became a concern, they added operational continuity checks. AI has opened a new gap in the same process, and most supplier due diligence frameworks have not caught up. Suppliers are using AI in their operations, in the products they provide, and in the services they deliver. This creates three categories of risk that the standard supplier review does not currently assess: risk from AI-generated errors affecting deliverables, risk from supplier AI systems processing the organization's data, and risk from supplier AI dependency creating operational continuity exposure. Each is worth understanding in detail. Risk category one: AI errors in supplier outputs Suppliers increasingly use AI tools to produce deliverables: reports, analysis, content, code, legal documents, financial models, regulatory submissions. In many cases, the organization receiving these deliverables has no visibility into how they were produced or whether AI was involved. The risk is not that AI-assisted work is inherently inferior — in many contexts it produces better outputs than manual work alone. The risk is that AI-assisted work with inadequate review processes introduces errors that are harder to spot than typical human errors: errors that are internally consistent and plausible-sounding, that require domain expertise to identify, and that can propagate through downstream decisions if not caught. A legal firm that uses AI to draft contract language without adequate attorney review. A financial advisory firm that uses AI for market analysis without the analyst verification that catches model hallucinations. An engineering firm that uses AI-assisted code generation in deliverables that the receiving organization will run in production. Most supplier due diligence frameworks assess whether the supplier has adequate quality management processes. They do not assess whether those quality management processes have been updated to account for AI involvement. This is a gap that procurement and legal need to close. Risk category two: supplier AI systems and organizational data When the organization shares data with a supplier — as part of a service engagement, as integration data, as content for processing — and the supplier uses AI tools in their operations, that data may flow through the supplier's AI systems in ways the organization did not anticipate when it established the supplier relationship. The data processing agreement the organization has with the supplier may govern how the supplier handles the organization's data in general terms. It almost certainly does not address specifically how the supplier may use that data in AI processing: whether it may be used as prompt context in a large language model, whether it may flow through the supplier's AI-powered productivity tools, or whether it may be incorporated into a supplier AI training dataset. The regulatory implications are the same as for any unauthorized processing of personal or confidential data, but the vector is the supplier rather than the organization's own systems. The organization bears the consequences, including regulatory notification obligations and liability to affected individuals, even though the breach occurred at the supplier. Data processing agreement templates need to be updated to include explicit terms about supplier AI use of customer data. This is not a standard clause in most current DPA templates. Risk category three: supplier AI dependency and operational continuity Suppliers that have deeply integrated AI tools into their operations have created a concentration of operational dependency that the organization should understand as part of continuity planning. If a supplier's core delivery capability depends on an AI system — for routing, for quality assessment, for decision-making in their operational processes — and that AI system experiences an outage, a model performance degradation, or a vendor relationship disruption, the supplier's ability to deliver may be materially impaired. This is a new category of operational risk in supplier relationships. Traditional continuity assessments look at infrastructure resilience, financial stability, and key personnel dependency. AI platform dependency needs to be added to the list. A supplier that uses a single AI vendor for critical operational processes, without fallback capability for the underlying task, has a concentration risk that the organization's continuity planning should reflect. This may not be a reason to avoid the supplier, but it is information that belongs in the continuity assessment. What to add to supplier due diligence The practical additions to a supplier due diligence framework for AI risk are not complex in structure, but they require the organization to decide on its risk appetite for each category. For AI in supplier deliverables: Add a disclosure requirement in supplier questionnaires: does the supplier use AI tools in producing deliverables for this engagement, and if so what quality management processes govern AI-assisted outputs? For high-stakes deliverables — legal, financial, technical, regulatory — set a minimum quality management standard that includes specific review requirements for AI-assisted content. For supplier AI use of organizational data: Update data processing agreement templates to include explicit terms on supplier AI use. Specifically: whether the supplier may process the organization's data through AI tools, under what conditions, with what data handling terms, and with what notification obligations if AI processing practices change. For supplier AI operational continuity: Add AI platform dependency to the operational resilience section of supplier due diligence questionnaires. Understand which of the supplier's core capabilities depend on AI systems, which AI vendors those systems run on, and what the fallback position is if the AI capability is unavailable. The sequencing question Not all suppliers need the same level of AI risk assessment. The effort should be calibrated to risk. The highest priority for AI-specific assessment: suppliers who produce high-stakes analytical or technical deliverables, suppliers with whom the organization shares personal or commercially sensitive data, and suppliers who are operationally critical to the organization's delivery capability. The second tier: suppliers involved in content, communications, or advisory services where AI-assisted work is common and quality control matters. Standard suppliers with limited data access and limited operational criticality can be addressed through a lighter-touch questionnaire update rather than a full assessment. What to take from thisUpdate supplier due diligence frameworks to include AI-specific risk categories. The standard framework does not cover AI errors in deliverables, AI processing of shared data, or AI operational dependency. Add AI disclosure requirements to supplier questionnaires for high-stakes deliverables. Know whether AI is involved before the deliverable arrives, not after. Update data processing agreement templates to include explicit terms on supplier AI use of customer data. Most current DPA templates are silent on this. Add AI platform dependency to operational continuity assessments for critical suppliers. Concentrate risk on AI platforms creates a new category of continuity exposure. Prioritize the AI risk assessment effort by supplier tier — full assessment for high-stakes, light questionnaire for standard. The whole supplier base does not need the same level of scrutiny.

Read full article
Why AI Proof of Concepts Keep Failing to Reach Production

Why AI Proof of Concepts Keep Failing to Reach Production

The statistic that gets quoted most often in enterprise AI discussions is that somewhere between 70 and 85 percent of AI proof of concepts never make it to production. The number varies by survey and by how you define "production," but the underlying phenomenon is consistent enough that I've stopped being surprised by it. What still surprises me is how the failure gets explained. The common story is that POCs fail because of technical complexity — the model doesn't generalize, the infrastructure isn't ready, the data is messier than expected. Sometimes that's true. More often, it isn't. The POC-to-production gap is primarily a governance failure, a funding failure, and an ownership failure. The technical problems are real but solvable. The structural problems are what actually kill programs. Why POC success can make things harder There's a version of this problem that's counterintuitive. A POC that performs well in a controlled environment can actually make production harder, not easier. When a POC succeeds, it generates expectations anchored to demonstration conditions. The data was curated. The use case was selected because it would work. The team was focused exclusively on making the thing perform. Production conditions are none of those things. The data is messier, the scope is broader, the team is split across other priorities, and the infrastructure needs to support real volumes and real latency requirements. The expectation gap between a successful POC and a production deployment is where a lot of programs die quietly. The business saw the demo, was impressed, approved funding — and then watched the production timeline slip while the performance benchmarks eroded. By the time the program is asking for additional time and budget, the credibility built by the POC has been spent. The five root causes Funding cliff. Most POCs are funded as experiments. A fixed budget, a fixed timeline, a specific deliverable: a working model that demonstrates feasibility. When the POC ends, the project budget closes. The team moves on to the next experiment. Production deployment isn't a continuation of the POC — it's a different program with different requirements and a different cost structure. Data infrastructure needs to handle production volumes. The model needs serving infrastructure. Monitoring needs to be built. Documentation needs to exist. Integration with production systems needs to happen. None of this was in the POC budget. Organizations that fund AI as a series of POCs never get to production. The model sits in a notebook, technically demonstrated, operationally useless. Ownership vacuum. A POC has natural owners: the data science team that built it and the business function that requested it. When the POC ends, ownership becomes ambiguous. The data science team has moved on. The business function owns the use case but not the model. IT owns the infrastructure but not the model logic. Nobody owns the whole thing. A production model needs a named owner — someone accountable for performance monitoring, retraining cadence, incident response, and the decision to decommission if performance degrades. That person and role need to be identified before the POC even starts, not after. Infrastructure gap. Most enterprise AI infrastructure decisions get deferred until after a POC has proven the concept. The logic is reasonable — don't invest in infrastructure for something that might not work. The consequence is that every successful POC immediately runs into a queue of infrastructure decisions that take months to resolve: model serving, feature engineering pipelines, data integration, security review, cloud provisioning. The gap between "POC complete" and "infrastructure ready for production deployment" is often six to twelve months in large enterprises. During that window, the team disperses, the business loses momentum, and the case for continued investment weakens. Governance mismatch. Enterprise governance processes were designed for traditional software. They weren't designed for AI systems that change over time, produce probabilistic outputs, and can generate systematically wrong answers without producing an error code. When a production-bound AI model hits the enterprise change management process, risk assessment, security review, and compliance sign-off, it often encounters requirements that weren't anticipated in the POC design. The model may need to be redesigned to meet explainability requirements. Data sourcing may need to change to meet compliance requirements. The security review may identify risks that require architectural changes. Each of these adds time and cost the original program budget didn't include. Success metric drift. POCs are typically evaluated on model performance metrics: accuracy, F1, AUC. Production is evaluated on business metrics: decisions improved, costs reduced, revenue generated. Those are different measurements, and the relationship between them is not guaranteed. A model that achieves 92% accuracy in testing may produce business outcomes that are difficult to attribute to the model specifically. Or the business metric assumed in the business case turns out to be hard to measure in practice. When production performance can't be clearly connected to business value, the investment becomes hard to defend. What production-ready actually means "Production-ready" in enterprise AI means more than a model that performs well on test data. It means: A serving infrastructure that handles the required throughput at the required latency, with defined behavior when the model fails or is unavailable. A monitoring system that tracks performance against defined thresholds and alerts when drift occurs. A retraining process that is documented, tested, and owned. An audit trail that captures model inputs, outputs, and decisions for the retention period required by relevant regulations. An explainability layer where required by regulation or business process. A decommissioning plan. Most POCs deliver none of these. Getting from a POC to a production-ready system is the bulk of the actual engineering work — which is why the common estimate that a POC represents 10 to 20 percent of the total production cost is roughly right in most programs I've seen. The playbook The decisions that close the gap need to happen before the POC starts, not after it proves the concept. Define the production requirements before building the POC. What infrastructure will the production system run on? What monitoring will it require? What governance processes will it need to pass? Building the POC against these requirements costs slightly more upfront and dramatically reduces the cost of moving to production. Name the production owner before the POC is approved. Who will own this model when it's live? What role is that person in? What resources will they have? If there's no good answer, the POC shouldn't start — because even if it succeeds, there's nowhere for it to go. Fund build-to-production, not build-to-POC. The funding model needs to include the full cost of production deployment: infrastructure, integration, monitoring, governance sign-offs, and the first year of operational costs. Approving POC budgets without production budgets produces a portfolio of successful experiments with nowhere to go. Run production governance in parallel with POC development. Security review, compliance assessment, and explainability requirements shouldn't be surprises at the end of the POC. They should be running in parallel so the production path is clear before the model is ready to move. None of this is complicated. Most organizations know it's the right approach. The reason it doesn't happen is that POCs are easier to approve than production programs — they're smaller, faster, and lower risk. The problem is that a series of successful POCs is not an AI program. It's an expensive set of demonstrations. The gap between those two things is what most enterprises are currently living in.

Read full article
What a Data Breach Looks Like When AI Is in the Middle of It

What a Data Breach Looks Like When AI Is in the Middle of It

Most enterprise data breach response plans were written for a specific type of incident: unauthorized external access to a database, a misconfigured cloud storage bucket, a stolen credential, a ransomware attack. The response playbook is well understood. Contain the breach, assess the scope, notify regulators, notify affected individuals, remediate the vulnerability. When an AI system is in the middle of a breach — as a vector, as an amplifier of exposure, or as the primary source of the incident — the playbook breaks down in several places. The scope assessment is harder. The cause is less obvious. The regulatory notification may require analysis the organization has not done. And the communications with affected parties need to account for AI involvement in ways that the standard template does not anticipate. Organizations that have AI systems in production and have not updated their incident response plans are carrying risk they have not quantified. The ways AI changes the breach scenario AI as a vector. A prompt injection attack — where malicious content in the AI system's input causes the system to execute unintended actions — is a category of attack that did not exist before AI systems were connected to organizational data. The technical mechanics are different from a SQL injection or a credential attack, but the organizational response involves the same triage: what did the attacker access, what did they exfiltrate, what actions did the AI system take on their behalf? Prompt injection is not theoretical. It has been demonstrated against production AI systems across multiple vendors. Organizations that have not evaluated their AI systems against this class of attack have a gap in their security assessment. AI as an amplifier. An attacker who compromises credentials to an account with AI system access may be able to extract substantially more information than they could from the underlying data systems alone. The AI system's ability to query, synthesize, and summarize across data sources means that a single compromised session can produce outputs equivalent to weeks of manual data extraction. The scope of a breach involving AI access is likely to be larger than the scope of a breach involving equivalent access to the underlying data without AI. This matters for the scope assessment, for regulatory notification thresholds, and for the volume of affected records. AI as the source. Misconfigurations in AI systems — incorrectly permissioned data access, insecure output handling, improperly sandboxed tool use — can themselves cause data exposure without any external attacker. An AI system that surfaces information it should not have had access to in response to a user query, or that exposes data through an incorrectly configured output channel, has caused a data exposure incident even in the absence of a security breach. These incidents are less dramatic than external attacks but potentially more common. And they are harder to detect because the behavior looks like normal AI system use rather than an anomalous external access pattern. Where the standard response plan fails Scope assessment. The standard scope assessment for a data breach identifies which records were accessed. When an AI system was involved, the relevant question is not which records were accessed but which outputs were generated — what did the AI synthesize from the records it could reach, and what information was contained in those outputs? This is a harder problem. AI outputs are not automatically logged in the way that database queries are. The organization may not have complete records of what the AI system produced during the breach window. Reconstructing the scope requires different methods than a traditional database access log analysis. Cause determination. Traditional breaches have identifiable technical causes: a vulnerability, a misconfigured permission, a phishing attack. AI incidents often have more diffuse causes — a combination of permissive access, insufficient output monitoring, and system behavior that was technically within parameters but produced an unintended result. Root cause analysis for AI incidents requires understanding of the AI system's architecture and behavior that most incident response teams do not have. Regulatory notification. Data breach notification requirements typically specify notification timelines and the content of notifications. When an AI system is involved, determining what categories of personal data were exposed requires understanding what the AI could access and what it may have surfected — an analysis that takes longer and requires more specialized input than a direct database access log review. Communication with affected parties. Breach notification communications are standardized around the concept of "your data was accessed by an unauthorized party." When an AI system was the mechanism, the communication needs to explain something more complex: what the AI system could access, what it may have produced, and why that creates risk for the affected individual. Most breach communication templates are not equipped for this. What the CFO and CIO need to prepare now Update the incident response plan. The plan needs to include AI-specific scenarios: prompt injection, AI-amplified credential breach, misconfiguration-driven data exposure. Each scenario should have a defined response team (which needs to include AI system expertise), assessment methodology, and escalation path. Establish AI audit logging requirements. If the organization does not have comprehensive logging of AI system queries and outputs, it cannot conduct a complete scope assessment for an AI-involved incident. The logging requirement needs to be part of AI system deployment standards, not something added after an incident. Define who owns AI incidents. Traditional breach response has clear ownership — typically the CISO and legal team with CFO involvement for material incidents. AI incidents may involve technical characteristics the CISO team does not have expertise in. Define who the AI-specific escalation path involves and ensure that person or team is part of incident response planning. Test the plan. Incident response plans for traditional breaches are tested through tabletop exercises. AI-specific scenarios should be part of the tabletop exercise inventory. The scenario of an AI system producing outputs it should not have, or being used as a vector by an attacker, is sufficiently different from traditional scenarios to warrant explicit testing. Understand regulatory notification requirements. Check whether the data protection officer's understanding of notification thresholds and timelines accounts for AI-involved incidents. In particular: the scope determination for an AI breach may take longer than for a traditional breach, and the notification timeline starts from discovery of the breach, not from completion of scope determination. What to take from thisUpdate the incident response plan to include AI-specific scenarios before an incident occurs. The scenarios are different enough from traditional breaches to require explicit planning. Require comprehensive logging of AI system queries and outputs as a deployment standard. Without it, scope assessment for an AI-involved incident is incomplete. Define AI-specific escalation paths within the incident response structure. The expertise required to assess an AI incident is different from traditional breach response expertise. Test AI breach scenarios in tabletop exercises. The behavior of an AI system during and after an attack is counterintuitive enough to warrant practice. The scope of an AI-amplified breach is likely larger than an equivalent breach without AI involvement. Build this into the material incident threshold assessment.The organizations that handle AI-involved incidents well are not the ones that were lucky enough to avoid them. They are the ones that updated their preparedness before the first incident, so that when it happened — and it will happen — the response was organized rather than improvised.

Read full article
The Internal Data Access Problem That AI Makes Suddenly Visible

The Internal Data Access Problem That AI Makes Suddenly Visible

Access controls in most organizations work on a document-by-document basis. You have permission to read a file or you do not. The logic has been sufficient for most purposes because humans navigate information deliberately — they go looking for specific things and find what they have access to. AI tools have broken that model without anyone changing any permissions. When an AI system with broad read access is asked a question, it does not navigate to a specific document. It queries across everything it can reach, synthesizes what is relevant, and produces an answer. The access controls determine what the system can read. They do not determine what combinations it can surface, what inferences it can draw, or what aggregated view of the organization's data it can present to the user. The result is a category of access control failure that most organizations have not addressed, because the access controls themselves are technically correct — and still inadequate. The gap between technical access and intended visibility The cleanest way to describe the problem: in most organizations, there is a meaningful difference between what an employee technically has access to and what they were intended to be able to see. This gap exists because access management is messy in practice. Permissions accumulate over time as people join projects, take on new roles, and inherit access from reorganizations. Revocation processes lag behind changes. Distribution lists include people who should have rotated off. Shared drives created for one purpose get used for another. The intended access model and the actual permissions diverge, and in normal day-to-day work the gap is largely invisible because people go looking for things they need rather than systematically browsing everything they can reach. AI tools systematically browse everything they can reach. That is their function. An employee asking an AI assistant "what do we know about the performance review process for the engineering team" may receive an answer drawn from documents they technically have access to but were never intended to be the audience for — HR process documentation, individual feedback templates, comparative data that lives in a folder from an organizational design project two years ago that nobody cleaned up. The employee has not circumvented any security control. But they have seen something the access model was not designed to permit. The categories where this matters most HR and compensation data. Salary information, performance ratings, disciplinary records, and individual feedback exist throughout organizations in documents with permissions that were set for a specific purpose and have often drifted since. AI systems connected to broad document repositories will find this material and surface it in response to queries that touch on it. Legal and privileged material. Legal advice, litigation strategy, settlement terms, and attorney-client communications often exist in places that technically-authorized users can access for one purpose but should not be able to aggregate for another. The privilege protection may be legally intact — the employee can read the document — but the ability to synthesize across years of legal communications is a different kind of access. Financial data beyond role scope. Budget holders can typically access their own budget data. AI systems may surface aggregate financial data by drawing on individual documents each of which was appropriately accessible, producing a consolidated view that nobody intended to give the employee. Client and partner confidential information. Client files shared within engagement teams are accessible to all team members for legitimate work purposes. An AI system that can search across all engagement files simultaneously may surface patterns about client relationships, deal economics, or strategic situations that no single team member was supposed to see in aggregate. Why the standard response does not work The first response most organizations reach for is tightening access controls. If AI is exposing the problem, fix the permissions. This is not wrong, but it is not sufficient. The problem has two parts that require different responses. The first part is genuine permission drift that should be corrected regardless of AI. Employees who have retained access to systems and documents they no longer need it for should have that access revoked. This is an overdue access hygiene exercise, and AI deployment is a reasonable forcing function for doing it. The second part is structurally different. Even with clean, intentional permissions, an employee with access to many documents across an organization will technically have access to combinations of data that, when synthesized by an AI, reveal more than the permission model was designed to permit. You cannot solve this purely by tightening access, because the individual access grants may all be correct. The solution to the second part requires building constraints into the AI system itself: what categories of data it can include in synthesis across user queries, what aggregation rules apply, and what escalation or approval processes apply to queries that touch the highest-sensitivity categories. Building the right architecture Three things need to happen in parallel, not sequentially. Access control remediation. Run an access review scoped to the data sources the AI system will connect to. Specifically look for: permissions that predate current roles, broad read access granted for historical projects that is no longer needed, distribution list membership that has not been reviewed in over a year. This will not solve the problem completely, but it reduces the surface area. AI-specific access boundaries. Define, at the AI system configuration level, what categories of data the system can use for synthesis in response to user queries. HR data, compensation data, legal documents, and individual performance information may be categories where even technically authorized access should not be available to the AI synthesis function. These boundaries need to be implemented as technical constraints in the AI system, not just as policy guidance. Query monitoring and anomaly detection. The AI system's query logs are, for the first time, making the access control problem visible. An employee who systematically queries for compensation data across a broad population, or who extracts patterns from legal files, shows up in the query logs in ways they would not show up in document access logs. This monitoring capability is new and should be used. What the CIO needs to drive The access control gap in AI deployments is fundamentally a CIO problem, not an AI team problem. The AI team can build a capable system. The CIO needs to ensure that the system's access to organizational data is deliberately configured rather than broadly permissive by default. Broadly permissive by default is the path of least resistance. It makes the AI system more capable and easier to demonstrate. It also creates the access control failures described above, and the first incident involving inadvertent disclosure of HR or financial data through an AI tool is going to be a painful conversation. The access architecture needs to be designed before the AI system goes live. The conversation about what categories of data the system should not be able to synthesize — even if individual documents in those categories are technically accessible — needs to happen with legal, HR leadership, and the CFO, not just the AI team. What to take from thisTechnical access controls determine what an AI system can read. They do not determine what it will synthesize or surface. The gap between these is where the access control problem lives. Run an access control remediation exercise scoped to the AI system's data access before deployment. Clean up permission drift even if the AI deployment were not happening — AI just makes the urgency visible. Build AI-specific access boundaries into the system configuration. Some data categories should not be available for AI synthesis even if individual documents within them are technically accessible. Use AI query logs as an access monitoring tool. The visibility into what the system is being asked to surface is new and valuable. The CIO needs to own the access architecture decision, not delegate it to the AI team. The decisions about what data categories the AI should not aggregate require organizational input that the AI team is not positioned to provide alone.

Read full article
Stakeholder Management for AI Programs: What Nobody Tells You

Stakeholder Management for AI Programs: What Nobody Tells You

Every AI program has a stakeholder deck. Roles, interests, influence levels, communication plans. It's usually a PowerPoint slide with a 2x2 matrix, produced at program inception and never updated again. The deck doesn't tell you what you need to know. What you need to know is who the informal blockers are, which executive will withdraw support when results are mixed, what the CMO told the CFO about AI in the corridor last week, and whether the sponsor who signed off at the start is still paying attention. Stakeholder management for AI programs is different from stakeholder management for other kinds of programs because the stakes are higher, the uncertainty is more sustained, and the gap between what executives expect and what AI actually delivers in the early phases is systematically wide. The stakeholder map you're not building The standard stakeholder map captures formal power: titles, reporting lines, decision authority. It misses informal power — the relationships and reputations that determine whether an organization actually moves. The CFO who approved the budget but doesn't attend reviews. The head of operations who has a long history of skepticism about technology programs and whose quiet opposition can drain energy from a program without ever appearing in meeting minutes. The board member who had a bad experience with an AI vendor three years ago and is looking for evidence the same thing is happening again. These people don't appear in the stakeholder matrix because their influence isn't visible in an org chart. But they shape outcomes. Missing them is one of the most common and most expensive mistakes in AI program management. The map that actually works starts not from the org chart but from the question: who can make this fail? For each person who could derail the program — through active opposition, budget cuts, organizational changes, or simply withdrawing attention — the question is what they need to see, hear, and experience to stay aligned. The conversations that don't go in the deck A lot of the real work in stakeholder management happens outside formal review structures. The sponsor who asks you in the hallway whether the team is actually making progress or whether the weekly updates are theater. The CFO's chief of staff who is quietly assessing whether this program will be on the budget cut list. The senior engineer in the business unit who has been telling their colleagues the model won't work, and who is now looking at every output for evidence they were right. These conversations require a different kind of preparation than a status update. They require knowing what each person is actually worried about — not what they're saying in meetings — and being able to address that concern directly without appearing defensive or over-reassuring. The pattern I've found most useful: regular informal contact with the people who matter most to the program's survival, calibrated to where they are in their thinking. Not updates — conversations. What are they hearing? What questions are they carrying from other conversations? What would change their view? This sounds like politics. It is politics. Programs that pretend otherwise don't last. The mixed results problem Early AI program performance is almost never as good as the initial business case implied. The data is messier than expected. The use case turns out to be harder than it looked in the POC. The business integration takes longer. The performance metrics are trending in the right direction but aren't there yet. This is normal. It's also the period where programs are most at risk of losing stakeholder support. Three communication failure modes are predictable in this period. Over-promising is the most common. The program team, under pressure to demonstrate progress, frames results more positively than they are — emphasizing the metrics that are improving while not addressing the ones that aren't. This buys time, but it creates a credibility deficit that's hard to recover from when reality catches up with the framing. Over-qualifying is the inverse error. Every update comes with so many caveats, so many technical explanations for why the results aren't yet representative, that stakeholders stop believing anything the team says. Uncertainty is real in early AI programs, but there's a difference between acknowledging it and using it as a shield. Going silent is the most dangerous. The program team, sensing that results are below expectation, starts avoiding the conversations. Updates get more infrequent, meetings get canceled, access to the actual performance data becomes difficult. Stakeholders who are getting less information don't assume things are fine. They assume the opposite. The communication approach that actually works is also the hardest one: saying clearly what is and isn't working, what the team has learned, what has changed about the approach as a result, and what the revised timeline looks like. Early in an AI program, honesty about setbacks doesn't lose stakeholder confidence if it's paired with credible adaptation. What loses confidence is the impression that the team doesn't know what's happening or isn't telling you. When something goes visibly wrong At some point in most large AI programs, something goes wrong in a way that's visible: a model produces outputs that embarrass the business, a performance metric collapses, a system that was supposed to be in production isn't. How the program team handles that moment determines whether the program survives it. The instinct is to minimize and explain. This is usually wrong. A stakeholder who learns about a failure from someone other than the program team, or who hears the team's explanation and feels it's incomplete, will not recover trust easily. What works: telling the story before it's told to you, owning the root cause rather than distributing blame, presenting the specific changes the team is making as a result, and establishing a timeline for when confidence should be re-evaluated. Not "we're on track," but "this is what happened, here's what it means, here's what we're changing, here's when you should assess whether the change is working." The six-month alignment problem Stakeholder alignment established at program inception is not alignment that holds at month six. It's a starting position. Organizations change. Priorities shift. The executive who sponsored the program gets a new role. The business unit that was most engaged finds itself under budget pressure. The board committee that approved the investment wants to see different metrics than the ones the program was optimized for. Programs that maintain alignment do so through continuous work, not a one-time alignment exercise. That means regular recalibration with key stakeholders — not just reporting to them — and willingness to adapt the program's framing, pace, or scope when the external context changes. The hardest version of this is when the program itself needs to change course — when the original use case isn't working as planned and a pivot is necessary. Executing that pivot without losing stakeholder confidence requires having built the kind of trust where stakeholders believe the team is telling them the truth, including the uncomfortable parts. That trust is built over months of consistent, honest communication. It's not something you can manufacture when you need it. And the organizations that skip the groundwork — that assume formal sign-off equals ongoing support — are usually the ones scrambling at month nine to explain why the program needs more time, to stakeholders who have already stopped listening.

Read full article
What Your AI Vendor Knows About Your Business After Six Months

What Your AI Vendor Knows About Your Business After Six Months

When an organization signs an enterprise AI agreement, the focus is almost always on what the vendor will provide — model capabilities, performance benchmarks, uptime commitments, support terms. The less examined side of the exchange is what the vendor learns about the organization over the course of the relationship. This is not a question of whether the vendor is misusing data. Most enterprise AI vendors have robust commitments around data use and treat customer data with appropriate care. The question is subtler: what does the accumulated pattern of the organization's AI usage tell a sophisticated observer about how the business operates, and what are the implications of that information sitting with a third party for years? The implications are not obvious until you think them through. What usage data reveals An AI vendor with access to enterprise usage data can observe, at scale and over time, patterns that individual data points do not reveal. What the organization focuses on. The topics, domains, and question types that generate the highest AI usage volume reveal where the organization is directing attention. A spike in queries about regulatory compliance in a specific jurisdiction signals a business development or risk management concern before it shows up in any public disclosure. A sustained pattern of usage around a particular product area signals strategic investment before any announcement. How the organization works. The workflows AI tools are used in reveal process patterns: how decisions are prepared, what information sources are consulted, how different functions interact, where bottlenecks exist. This is the kind of operational picture that management consultants spend weeks building in client engagements. AI vendors accumulate it as a byproduct of normal usage. Where the organization's capabilities are strong and where they are not. The questions an organization asks of an AI system reflect, to some degree, what the people asking cannot do themselves. Heavy usage of AI tools for a specific type of analysis suggests that internal capability is limited in that area. A pattern of AI-assisted communication drafting in certain functions suggests communication capability constraints. Who the organization interacts with. Queries that reference client names, partner organizations, or market contexts — even in enterprise agreements where input content is excluded from training — create metadata about the organization's relationship network and market focus. None of this requires the vendor to actively analyze any specific piece of content. Aggregate usage patterns make these inferences available without individual query inspection. Why this accumulates over time The picture that emerges after six months of enterprise AI usage is qualitatively different from what was visible at month one. The accumulation of patterns across thousands of interactions, across multiple functions, across different business cycles reveals consistency and change in ways that a snapshot does not. Organizations change focus, enter new markets, encounter new challenges, and invest in new capabilities. All of those shifts are visible in AI usage patterns before they are visible elsewhere. The vendor relationship, if it persists, captures the strategic trajectory of the organization over time. This is particularly relevant for multi-year AI vendor relationships, which are increasingly common as organizations embed AI tools into core workflows. An AI vendor that has maintained an enterprise relationship for three or four years has accumulated a longitudinal view of the organization's strategic and operational evolution that very few parties outside the organization have. The vendor concentration dimension The question of what a single AI vendor knows about an organization becomes more significant when that vendor also serves the organization's competitors, its clients, or its industry peers. This does not mean the vendor is sharing information between customers — contractual commitments and practical self-interest both constrain that. But it does mean the vendor has a vantage point on industry-wide patterns that individual organizations lack. Aggregate insights about what questions enterprises in a specific industry are asking of AI systems, what capabilities they are developing, where they are investing — this is a form of competitive intelligence that accrues to the vendor in ways that have no clean analog in traditional software relationships. For organizations in sectors where competitive intelligence matters — financial services, pharmaceuticals, technology — the accumulation of strategic signal at a shared AI vendor is worth thinking about explicitly. What the CFO should factor into vendor relationship management The financial relationship with an AI vendor needs to account for switching costs that go beyond the cost of migrating to a new platform. The accumulated organizational context — the conversation history, the fine-tuned models, the usage patterns and metadata that have built up over years — creates a real switching cost that is not always visible at contract negotiation. Organizations that have deeply embedded a single AI vendor into core workflows may find that switching is more expensive than they anticipated, not because the technology cannot be replicated but because the years of accumulated context cannot easily be transferred. This is relevant to contract renewal negotiations, where vendors understand the switching cost dynamic better than most customers. It is also relevant to how the organization structures its AI vendor portfolio — whether to consolidate around a single vendor for maximum integration, or to distribute across vendors in ways that limit the strategic depth of any single relationship. What to do about it This is not an argument for avoiding AI vendors or maintaining zero-depth relationships. The value of AI tools requires meaningful integration, and meaningful integration creates the usage patterns described above. The practical response is to understand what the relationship accumulates and manage it deliberately. Conduct a periodic vendor relationship review that includes, alongside performance and cost, an assessment of what the vendor relationship has revealed about the organization through usage. This is not paranoia — it is the same kind of vendor relationship management organizations apply to any strategic supplier relationship. Review data minimization options. Many AI vendor agreements include options to limit usage data retention, opt out of certain analytics, or configure how interaction metadata is handled. These options are not always publicized, but they are often available in enterprise agreements. Understand them before defaulting to whatever the vendor's standard configuration produces. Consider the vendor concentration question explicitly in AI strategy. The organization that routes all AI usage through a single vendor is building a deeper relationship than the one that distributes across vendors. Both approaches have merits. The decision should be deliberate rather than a byproduct of procurement timing. Build contract terms around usage data explicitly. What the vendor can do with aggregate usage data — not just input content — should be addressed in the enterprise agreement, not assumed from the default terms. What to take from thisEnterprise AI usage creates an aggregate picture of the organization's focus, workflows, and capabilities over time. Understand what that picture contains. Multi-year AI vendor relationships accumulate strategic signal about the organization's trajectory. The longer the relationship, the more the vendor knows. Switching costs for deeply embedded AI vendors include the loss of accumulated context, not just migration effort. Factor this into vendor relationship management. Review data minimization options in enterprise agreements. They are often available and not actively surfaced. Address how the vendor may use aggregate usage data — distinct from input content — in the enterprise agreement terms.The organizations that handle this thoughtfully are not the ones who avoid AI vendor relationships. They are the ones who understand what those relationships accumulate and manage them with the same care they apply to any strategic supplier holding significant organizational knowledge.

Read full article
The Competitive Intelligence Risk Nobody Is Discussing in the Boardroom

The Competitive Intelligence Risk Nobody Is Discussing in the Boardroom

Most enterprise risk conversations about AI center on what happens to the organization's data when it flows through AI systems. That is the right conversation to be having. But there is an adjacent risk that gets far less attention: the question of what AI tools make visible to outsiders from data the organization has already published, disclosed, or inadvertently made accessible. This is not a theoretical scenario. AI tools have fundamentally changed the economics of information aggregation. Tasks that previously required significant analyst effort — synthesizing public disclosures, identifying patterns across procurement records, cross-referencing job postings with product announcements — are now within reach of any competitor with access to a capable AI tool and a few hours of time. The organization's competitive exposure through this channel is probably larger than the board has considered. Here is what that risk profile actually looks like. The data surface you have already published Organizations publish more information than they realize. Some of it is intentional. Much of it is not. Regulatory filings disclose financial structure, revenue composition, operational dependencies, and strategic priorities in more detail than executives typically remember. Job postings reveal technology stack, team composition, expansion plans, and capability gaps. Press releases and case studies describe products, customers, partnerships, and methodologies. Conference presentations and white papers lay out strategic thinking. Patent applications describe technical approaches before they are in production. None of this is secret. Most of it is searchable. But aggregating it at scale, identifying patterns, and drawing inferences about competitive position and strategy has historically been expensive. It required analysts, time, and a systematic process. These barriers meant that most competitors did not maintain a comprehensive, continuously updated picture of each other. AI tools have eliminated most of that friction. A capable AI system can ingest years of public disclosures, synthesize patterns across data types, and surface inferences about competitive position in minutes. The barrier to maintaining detailed competitive intelligence on any organization has dropped substantially. What AI-powered competitive analysis can surface The outputs of this kind of analysis are more specific than the broad category of "public information" might suggest. Strategic priorities and timing. The combination of leadership statements, hiring patterns, product announcements, and partnership disclosures can reveal a significant amount about where the organization is investing and on what timeline. A competitor who can identify that your organization has been hiring AI infrastructure talent in a specific geography for the past 18 months can reasonably infer an expansion play. Technology stack and vendor relationships. Job postings are one of the most underappreciated sources of competitive intelligence. The technical requirements in engineering roles reveal which tools, frameworks, and platforms the organization is using. Vendor relationships disclosed in case studies and partnership announcements fill in the picture further. An AI system processing this data at scale can construct a reasonably accurate technology map. Customer relationships and vertical focus. Case studies and client announcements, conference panels, award submissions, and procurement filings (for public sector clients) disclose customer relationships in detail that organizations often do not track systematically. An AI tool can aggregate this to build a picture of the customer base that the organization itself might not have in one place. Organizational structure and decision-making. Leadership announcements, departures, restructuring communications, and employee updates on professional networks tell a story about organizational priorities and political dynamics that is more readable in aggregate than in individual data points. The inadvertent disclosure layer Beyond what organizations publish deliberately, there is a layer of inadvertent disclosure that AI tools make significantly more accessible. Metadata in documents and presentations shared externally. Employee behavior on professional networks — what they share, who they follow, what they comment on — that in aggregate reveals organizational sentiment and priorities. Procurement and vendor records in public databases that disclose vendor relationships more completely than any press release would. Customer reviews and reference lists that reveal implementation approaches and satisfaction levels. These are individually innocuous. In aggregate, processed by a capable AI system with good instructions, they can surface patterns that executives would not have chosen to disclose. The counter-argument — that this information is technically public and therefore fair game — is correct as a matter of law but misses the practical point. The question is not what is legally protectable. The question is whether the organization understands its actual information surface and has made deliberate decisions about what it wants to be visible. What this means for the organization's own AI use There is a symmetry here that boards should find clarifying: the same AI tools that make the organization's public information more analyzable are the tools the organization itself is using to analyze others. The competitive intelligence advantage of AI is available to everyone. The organizations that are ahead in using AI tools for competitive analysis are gathering more and better intelligence about their competitors. The organizations behind in AI adoption are, conversely, being analyzed more thoroughly than they are analyzing others. This is one of the competitive dynamics of AI adoption that does not show up in the typical ROI analysis. The cost of AI underdevelopment is not just operational inefficiency — it includes an information asymmetry in competitive intelligence. The specific risks for different data categories Client relationships. If the organization's client list is reconstructable from public sources — which for most organizations it largely is — then the client targeting strategies of competitors can be informed by that data. This matters for retention strategy and for protecting long-term client relationships. Pricing and deal structure. Pricing information disclosed in competitive bids, procurement filings, or case study economics creates a data trail that AI tools can use to inform competitor pricing strategy. Organizations that have been disciplined about what deal economics they allow to become public are in a better position than those that have not. Technical approaches and intellectual property. Patent filings and technical publications are the most obvious source here, but the combination of job descriptions, technical conference presentations, and open source contributions can paint a detailed picture of technical methodology. What to actually do about this The goal is not to eliminate the organization's public information surface — that is neither possible nor desirable. The goal is to understand it and make deliberate decisions about what to protect. Run an AI-powered analysis of your own public information surface. This is the most direct way to understand what a sophisticated competitor with AI tools could learn about your organization. Hire someone to do it or do it internally, but see the output before deciding how to respond. Review what the organization chooses to disclose in non-mandatory contexts: case studies, conference talks, award submissions, technical publications. These disclosures often carry more competitive intelligence value than they add in marketing or recruiting value. Build intelligence aggregation into the competitive monitoring process. If the organization is not using AI tools to monitor competitor public information at scale, it is falling behind in the intelligence competition. What to take from thisAI tools have made the aggregation of public competitive intelligence far cheaper and more comprehensive. Assume sophisticated competitors have done this analysis of your organization. Run an AI-powered review of your own public information surface. The output will show you what a competitor with good tools and reasonable instructions can learn about your strategy, customer base, and technology. Evaluate non-mandatory disclosures — case studies, conference presentations, technical publications — through a competitive intelligence lens before publication. The competitive intelligence advantage of AI is available to everyone. The organizations ahead in AI adoption are gathering better competitive intelligence. This is a real asymmetry, not a theoretical one. Build AI-powered competitive monitoring into the standard intelligence process. Point-in-time competitive analysis is less useful than a continuously updated picture.

Read full article
The Confidentiality Risk in Your AI Productivity Rollout

The Confidentiality Risk in Your AI Productivity Rollout

The business case for an organization-wide AI productivity rollout usually focuses on time saved — hours of drafting, summarizing, and searching that employees no longer have to do manually. The productivity math is often compelling. The confidentiality implications rarely make it into the same document. I am not arguing against AI productivity tools. Most of the organizations I work with benefit from deploying them thoughtfully. I am arguing that the deployment decision and the confidentiality assessment need to happen together, not sequentially, because by the time the confidentiality issues surface in a live deployment, they are substantially harder to address. There are four areas where confidentiality exposure tends to materialize in AI productivity rollouts, and none of them are obvious from the outside. The permissions inheritance problem Most enterprise AI productivity tools integrate with the organization's existing data. A writing assistant that can access email and calendar content. A search tool that can query across the organization's document repositories. A meeting assistant that processes conversation recordings. The integration is the point — the tool needs access to data to provide value. The confidentiality problem is that the access often inherits existing permissions without anyone reviewing what those permissions actually cover. Organizational data permissions are almost never clean. Documents shared broadly during a project and never restricted afterward. Distribution lists with members who should have rotated off. Legacy permissions on systems that predate the current structure. This is normal; access controls accumulate over time and rarely get regularly pruned. When an AI productivity tool indexes the content that a user can access, it indexes everything they can access — including the content they technically have access to but were never meant to see in its entirety. When the tool then uses that content to answer queries, generate summaries, or surface relevant information, it may surface content in ways that exceed what the original permission model was designed to permit. I have seen this manifest in practice: an AI assistant that could search across an organization's document repositories began surfacing salary data in response to queries about a particular team, because the underlying HR documents were stored in a folder the user had access to for an unrelated historical reason. The user was not trying to find that data. The AI found it for them. The aggregation problem Individual pieces of information that are harmless in isolation can be sensitive in combination. AI productivity tools are particularly good at making the combination visible. An employee with legitimate access to sales pipeline data, client meeting notes, and internal budget discussions does not normally see all of that information together in a synthesized form. They encounter it in different contexts, at different times, through different systems. The totality is there, but the cognitive effort required to combine it provides a natural friction. An AI tool that can aggregate, summarize, and cross-reference across all of those sources removes that friction. The same employee can now, with a single query, see a synthesized view of their organization's client relationships, deal economics, and strategic priorities that no single document or system would have surfaced. This is not a bug in the tool — it is often the primary selling point. The confidentiality question is whether there are categories of information where that aggregation creates an exposure that the access control model did not anticipate. The answer is usually yes, but nobody looked. The meeting and conversation record Meeting AI tools — platforms that transcribe, summarize, and make conversation content searchable — have become common in enterprise deployments. The confidentiality implications deserve explicit attention before rollout. Conversations that participants understood to be informal or confidential in the moment become searchable records. This matters in three contexts that are not always considered during rollout planning. Board and leadership discussions processed by meeting AI tools create records of deliberations that may need to be protected under legal privilege or governance confidentiality obligations. Whether the tool's data handling terms are compatible with those obligations is often not reviewed. Client and partner conversations. Many organizations use meeting AI tools for external calls without explicitly disclosing this to the other party. Depending on jurisdiction, recording requirements vary, but the confidentiality implication extends beyond recording law: the content of client conversations is typically covered by confidentiality obligations in the client relationship. Where that content is stored, who can access it, and what the tool does with it are questions the client may reasonably want answered. HR and sensitive personnel conversations. Performance discussions, disciplinary matters, and sensitive employee conversations processed by meeting AI tools create records that carry additional obligations around storage, access, and deletion. The external output risk AI productivity tools help employees produce external outputs faster. That productivity benefit creates a confidentiality exposure that tends to get overlooked: the risk that AI-assisted drafting incorporates confidential context that the author did not intend to share. When an employee drafts a client proposal using an AI writing tool that has access to their full communication and document history, the tool may draw on that context in ways the author does not fully control or review. A proposal drafted with AI assistance might reflect information about the organization's pricing strategy, competitive positioning, or internal deliberations that no single author would have consciously included. This is harder to observe than the other risks because it manifests in outputs that look normal and are not obviously different from what the employee would have written manually. The signal is subtle: slight reveals of internal context, references to information the recipient was not meant to have, framing that reflects internal discussions the author forgot they had consulted. Running the confidentiality assessment before rollout The practical steps that matter: Review the permission state before enabling AI access to existing content. Specifically: which users have access to what, and are the existing permissions consistent with what was intended? The AI rollout is a good forcing function for an access control review that should have happened anyway. Identify the sensitive data categories in scope. For each category — client data, HR data, financial data, legal and privileged content — assess whether AI tool access is appropriate and under what controls. Check whether meeting recording disclosure is required. For external calls, understand the legal and relationship requirements in the relevant jurisdictions and configure the tool accordingly. Establish a content review process for AI-assisted external documents. This does not have to be comprehensive — it should focus on the document types where inadvertent disclosure risk is highest. Set explicit expectations with employees about what the tool is and is not appropriate for. Not a policy document nobody reads — a short, specific briefing that describes the actual confidentiality risks and what to do about them. What to take from thisAI productivity tools inherit existing permissions. Review the permission state of the content they will access before enabling the rollout — you will find problems. Aggregation risk is real and is not obvious from reviewing individual access controls. Think about what combinations of accessible content look like when synthesized. Meeting AI tools create records of conversations that may carry confidentiality obligations the tool's data terms do not satisfy. Assess this before deployment, not after. AI-assisted external drafting can inadvertently incorporate confidential context. Build a light-touch review step into the document production workflow for the highest-risk document types. The business case and the confidentiality assessment need to happen simultaneously. Running the confidentiality review after the deployment decision has been made tends to surface problems at the wrong point in the process.

Read full article
How AI Vendors Use Your Data: Contract Versus Reality

How AI Vendors Use Your Data: Contract Versus Reality

I have read a lot of AI vendor contracts in the past few years. Not because contract review is interesting in itself, but because the gap between what vendors say in sales conversations and what their agreements actually commit to has consequences. Organizations that do not close that gap before signing end up discovering what the contract actually says at the worst possible time. The general shape of an AI vendor data agreement is worth understanding at the executive level — not because the CFO or CIO needs to redline individual clauses, but because the strategic choices about which vendors to use and under what conditions flow directly from what those agreements permit and exclude. Here is what I see consistently. The default terms favor the vendor This should not be surprising. The default data processing terms in any commercial agreement are written to minimize the vendor's liability and maximize their operational flexibility. AI vendor agreements are no different, and in some respects they are more aggressively drafted than traditional software agreements because the stakes around data use are higher and the regulatory landscape is still evolving. The standard structure of a consumer or early-stage enterprise AI agreement typically includes: A broad grant to the vendor to use interaction data for service improvement, model training, and product development purposes, subject to anonymization or aggregation. In practice, what "anonymization" means and how consistently it is applied is rarely specified. Retention periods that are defined by the vendor's operational needs rather than the customer's preferences, often without a customer-initiated deletion right. Liability limitations that cap the vendor's exposure in the event of a data incident at amounts that bear no relationship to the potential harm to the customer — typically limited to fees paid rather than the value of the data or the cost of a breach. Unilateral modification rights that allow the vendor to change the data processing terms with notice, sometimes as short as 30 days, without requiring the customer's affirmative consent. None of these are unusual in commercial software agreements. But when the agreement governs how your organization's strategic data, client information, and proprietary content is handled, they warrant closer attention than a standard SaaS contract. What changes in a properly negotiated enterprise agreement The distinction between a default agreement and a properly negotiated enterprise agreement is significant. When procurement and legal have done their job, the enterprise agreement should include at minimum: Exclusion from training data. A clear, contractually binding commitment that the customer's interaction data will not be used to train or fine-tune the vendor's models. This is the single most important data term and the one that organizations should refuse to proceed without. Data processing agreement compliant with applicable regulations. For any processing of personal data of EU residents, a GDPR-compliant data processing agreement is a legal requirement. Increasingly, other jurisdictions impose similar requirements. This agreement specifies the purposes for which data is processed, the retention periods, the data subject rights the vendor will support, and the security measures in place. Defined retention and deletion terms. The agreement should specify how long the vendor retains interaction data, under what circumstances, and what deletion looks like — with confirmation that deletion is complete and irreversible. Sub-processor disclosure and control. AI platforms often rely on cloud infrastructure, third-party safety tooling, and other sub-processors. The enterprise agreement should disclose who these are and give the customer the ability to object to new sub-processors. Breach notification terms. The timeframe within which the vendor will notify the customer of a security incident affecting customer data. Thirty days is common in default agreements; 72 hours is what most regulatory regimes require you to provide to your own regulators. Make sure the vendor's notification obligation to you is faster than your notification obligation to regulators. The clauses that cause problems later In practice, the clauses that create the most problems are not the ones organizations focus on during negotiation. Aggregated and anonymized data carve-outs. Most agreements carve out "aggregated and anonymized data" from the restrictions on training use, with the rationale that anonymized data cannot be traced back to the customer. The problem is that what counts as "anonymized" is not usually defined with precision, and for certain types of content — queries about niche industries, specialized technical topics, or specific organizational patterns — re-identification is more feasible than the carve-out implies. Operational necessity language. Agreements often include broad permissions for the vendor to process customer data "as necessary to provide and improve the service." The scope of "improve the service" is frequently contested. Make sure this language is defined, not left open. Right to audit provisions. The ability to verify that the vendor is actually complying with the data processing commitments they have made. Many agreements include an audit right that is functionally unusable — limited to once per year, requiring 90 days notice, subject to the vendor's approval of the auditor. An audit right with those conditions provides limited practical assurance. Termination data handling. What happens to your data when the contract ends. How long does the vendor retain it after termination? What format is it returned in? Is deletion from backup systems addressed? Organizations that have ended vendor relationships often discover that "deletion" in practice means deletion from active systems, with indefinite retention in backup infrastructure. The sales conversation versus the signed contract The gap I see most often is between what the sales team communicates during the evaluation — "we never use your data for training," "your data is completely private," "you retain full ownership of everything" — and what the signed agreement actually commits to. This is not always deliberate misrepresentation. Sales teams are not contract lawyers, and they often communicate what they believe to be true without knowing the precise legal scope of the commitments they are describing. The problem is that verbal assurances do not create contractual obligations. What matters is what the signed agreement says. The practical implication: have the conversation about data handling before the procurement decision, but validate every assurance by finding the corresponding contractual language. If the vendor says they do not train on customer data, ask them to point to the specific clause that says so. If the clause does not exist, or if it is qualified in ways that limit its practical scope, that is important information. What the CFO should be looking at The CFO's lens on AI vendor data terms is different from the CIO's. Beyond the data handling questions, the financial and liability exposure matters. Liability caps that are set at fees paid rather than harm caused mean that in the event of a serious data incident, the vendor's contractual exposure is often a fraction of the cost the organization incurs — in regulatory fines, breach notification costs, customer notification, and reputational damage. This does not mean the vendor relationship is unworkable, but it does mean the organization is bearing most of the downside risk and should price that accordingly. Insurance coverage. Some AI vendor incidents may fall into gaps between the organization's existing cyber insurance policy and the vendor's coverage. This is worth reviewing explicitly before the program goes live. Renewal and price terms. AI vendor agreements increasingly include significant pricing flexibility — unilateral price changes, usage-based components that scale in ways that are hard to predict, and renewal terms that are less favorable than the initial agreement. Understanding the financial exposure over a three-to-five year horizon matters for the investment case. What to take from thisDefault AI vendor data terms are written for the vendor's benefit. Do not assume they protect customer interests without reviewing them. The training exclusion is non-negotiable for any enterprise deployment handling sensitive data. Get it as a contractual commitment, not a sales assurance. Aggregated and anonymized data carve-outs are often broader than they appear. Define what anonymization means in the specific context of your data. Audit rights that are functionally unusable provide no real assurance. Push for meaningful audit provisions. Verify every data handling assurance the sales team makes by finding the contractual language that supports it. If it is not in the contract, it is not a commitment.The organizations that manage AI vendor relationships well are not the ones with the longest or most restrictive agreements. They are the ones that understood what they were agreeing to before they signed, addressed the material gaps, and built a vendor relationship on actual commitments rather than assumed ones.

Read full article
The AI Tools Your Employees Are Using With Your Data

The AI Tools Your Employees Are Using With Your Data

The standard framing for AI governance starts with the question of which tools to approve. That is the wrong starting point. The better question is: which tools are already in use, with what data, under what terms? By the time most organizations start building an AI governance framework, their employees have already made a fairly coherent set of tool choices. They have picked the tools that solve their immediate problems. They have not, in most cases, read the privacy policies or data processing terms. And because nobody told them not to, they have been using company data freely. A CIO who wants to get ahead of this — or who wants to manage it after the fact — needs a clear picture of the tool landscape and an honest assessment of where the data risk actually concentrates. Not all unsanctioned AI tools carry the same risk. Understanding the difference is where governance work should start. The tool categories and what they mean for data General-purpose AI assistants This is the highest-volume category. Consumer versions of large language model interfaces are used daily by employees across functions — drafting communications, summarizing documents, answering domain-specific questions, structuring thinking. The use is frequent, the content fed in is varied, and the data handling terms depend entirely on whether the employee is using a consumer or enterprise account. The specific risk here: consumer-tier accounts with default settings often permit the vendor to use interaction data for product improvement. The same vendor's enterprise tier typically does not. Most organizations have no visibility into whether employees using these tools are on a consumer or enterprise tier, and many are on a consumer tier simply because it was free and faster to start. Productivity AI features in existing software Word processing, spreadsheets, presentation tools, email clients, and project management platforms increasingly include AI features — often activated via a premium license or a setting employees can enable without IT involvement. The risk here is different from standalone AI tools: because these features exist inside software the organization already uses, they often fly under the radar of any AI tool review. The data handling terms for AI features embedded in existing software are usually governed by the same agreement covering the base product, but with additional clauses for the AI component that many organizations have not reviewed since they were added. These clauses deserve explicit attention. Specialist function tools Legal AI tools, sales intelligence platforms, HR tools, finance automation assistants, coding assistants, market research tools — these are purpose-built AI products targeting specific professional functions. They tend to be adopted department by department, often through a free trial that converts to a team subscription without going through central IT. The data risk with specialist tools is often higher than with general-purpose ones, for a specific reason: the content fed into specialist tools tends to be more consistently sensitive. Legal teams feed contracts. Finance teams feed financial models. Sales teams feed client data and deal structures. The tool is designed for that content, which means employees use it confidently and at volume. AI-powered integrations and automation platforms Workflow automation tools, AI connectors between SaaS platforms, and integration layers that use AI for data transformation or decision-making sit in a category that CIOs are least likely to have visibility into. These tools often operate in the background — processing data as part of an automated flow rather than through a direct user interaction — and their data handling terms are buried inside integration documentation that nobody reads. The risk with automation platforms is not necessarily higher than with interactive tools, but the visibility is lower. When a human pastes text into an AI tool, there is at least a moment of conscious choice. When an automated workflow passes data through an AI component as part of processing, there is no such moment. The risk factors that actually matter When assessing the data risk of any specific tool category, there are four factors that determine how much it matters. What data flows through it. The highest risk is where the most sensitive data concentrates: client information, financial projections, legal material, personal data. This varies by tool and by how a specific team uses it. What tier the organization is on. Enterprise agreements typically include data processing terms, exclusions from training use, and deletion rights that consumer tiers do not. A tool is not inherently high-risk or low-risk — the tier and the agreement terms are what determine the actual data handling. Whether a data processing agreement exists. For any tool processing personal data of EU residents, a data processing agreement is a legal requirement under data protection regulation, not a nice-to-have. Many organizations are operating without these agreements in place for tools their employees use every day. How much volume is flowing through it. A low-use tool with poor data terms is a lower priority than a high-use tool with poor data terms. Volume matters. The tools employees reach for first, most often, at highest volume are where the exposure is concentrated. What a CIO needs to do before writing a policy Policies written without a clear picture of the current state tend to be wrong in two ways: too restrictive in areas where the risk is manageable, and silent on areas where the risk is real. Getting the picture right first makes the policy more useful. That means running a discovery exercise that goes beyond the IT procurement system. Talk to department heads about what their teams use. Survey employees. Analyze network traffic for connections to known AI tool endpoints. The goal is a realistic list of tools in active use, categorized by function and frequency. For each tool, determine what tier the organization is on — enterprise or consumer — and whether a data processing agreement exists. This is the most important variable in understanding the actual data handling exposure. From there, prioritize remediation by volume and sensitivity. The tools that process the highest volume of the most sensitive data under the least favorable terms are the first order of business. That might mean migrating employees from a consumer tier to an enterprise tier of the same tool. It might mean negotiating a data processing agreement with a vendor. It might mean replacing a tool with an approved alternative. The classification that comes out of this exercise — which tools are approved at which tier for which data types — is what the policy should be based on. Policies that precede this exercise tend to produce compliance theater rather than actual risk reduction. The conversation with department heads This is where the process usually gets uncomfortable. When a CIO discovers that a department has been using an unsanctioned AI tool with client data for the past year, the instinct is often to shut it down immediately. That is rarely the right response. Abrupt prohibition creates resistance and drives use underground. It also signals that the governance process is about compliance rather than risk management, which damages the working relationship the CIO needs to make future governance effective. The better approach: treat the discovery as information rather than a violation. Understand what the tool is being used for, what problem it solves, and what the actual data exposure has been. If the tool can be moved to an enterprise tier with appropriate terms, do that quickly. If it needs to be replaced with an approved alternative, make the transition timeline reasonable and the approved alternative usable. Department heads whose teams are using shadow AI tools are not adversaries. They are telling you, through their behavior, what the organization's official tooling is failing to provide. The policy conversation goes much better when it starts from that acknowledgment. What to take from thisMap what tools are in active use before designing any AI tool governance policy. The gap between what IT has approved and what employees are actually using is almost always larger than expected. For each tool, determine whether it is in use on an enterprise or consumer tier. That distinction drives most of the material data handling difference. Check whether data processing agreements exist for tools processing personal data. This is a current legal obligation, not a future aspiration. Prioritize remediation by volume and sensitivity: high-use tools handling sensitive data under weak terms first. Treat departments using unsanctioned tools as providing product feedback. Understand why they chose the tool before deciding how to respond.The CIOs who manage this well are not the ones with the strictest policies. They are the ones who ran the discovery work, understood what was actually happening, and built governance around the real picture rather than the one they assumed existed.

Read full article
What Data Leaves Your Organization Every Time Someone Uses an AI Tool

What Data Leaves Your Organization Every Time Someone Uses an AI Tool

Most organizations operate under a working assumption that their data is contained. Files live on approved systems. Emails go through monitored infrastructure. Cloud storage is access-controlled. The perimeter is imperfect, but it is at least visible. AI tools have quietly dismantled that assumption. Not through a breach. Through normal, sanctioned-feeling use. Every time an employee types a prompt into a large language model, attaches a document for summarization, or pastes a block of text for analysis, that content leaves the organization's infrastructure and enters a third-party system. The employee does not experience this as data transfer. They experience it as using a tool. But the data has moved, and where it goes, how long it stays, and what is done with it depends entirely on terms most organizations have never reviewed. What "data leaving the building" actually means The framing matters here, so I want to be precise. When I say data leaves the organization, I mean three distinct things. First, the input reaches the vendor's infrastructure. The prompt, the document, the pasted text — all of it travels to servers the organization does not control, under security and access policies the organization did not set, in jurisdictions the organization may not have mapped. Second, the vendor processes and stores that input for some period. The length and purpose of storage varies dramatically by product and by the specific agreement in place. Some vendors retain inputs for a defined period for abuse prevention. Some retain them longer for product improvement. Some will, under certain terms, use them to improve future model versions. The defaults on this vary and are not always what organizations assume. Third, the output the model generates may itself be derived from patterns the model learns over time. This is the mechanism that tends to unsettle executives most when they understand it, though the practical risk here is more nuanced than the headline version usually suggests. The part that matters most in practice is the first two: the content reaches third-party infrastructure, and its fate is governed by the vendor's policies, not yours. The content that tends to flow through AI tools This is worth spending time on, because organizations that have audited actual AI tool usage consistently find that the content flowing through consumer and productivity AI tools is more sensitive than they assumed. Strategy and planning documents. Employees use AI tools to refine presentations, summarize options, and draft documents for leadership review. The source material they feed in frequently includes internal plans, financial projections, and competitive analysis. Client and customer information. Sales teams use AI assistants to draft proposals and account summaries. Support teams use them to summarize case histories. Analysts use them to structure reports. Client data is routinely included, often without a deliberate decision to include it. Legal and contractual material. Lawyers and procurement teams use AI tools to summarize contracts, identify key clauses, and compare terms. Contract text often contains commercially sensitive information that neither party intended to share beyond the two signatories. HR and personnel data. Managers use AI tools to draft performance reviews, restructuring communications, and offer letters. The inputs frequently include specific salary information, performance ratings, and personal circumstances. None of these employees are being careless. They are using AI to do their jobs. The exposure is a product of normal behavior, not negligence. Where the data goes: the three mechanisms Processing for the immediate request. This happens in every interaction, by definition. The data reaches the model, the model generates a response, and the exchange is complete from the user's perspective. What happens after that depends on the vendor. Retention for operational purposes. Most AI services retain some record of interactions for a period — to detect abuse, to provide conversation history to the user, or to meet regulatory requirements in certain jurisdictions. The retention period and what the organization can do about it (deletion requests, data portability) varies significantly and is usually defined in the data processing agreement or privacy policy. Use for model training and improvement. This is the term that gets the most attention, and for good reason. Some AI products, particularly consumer-grade versions of enterprise tools, include default settings that allow the vendor to use interaction data to improve the model. The important nuance: enterprise agreements frequently exclude this, while consumer free tiers often include it. The problem in most organizations is that employees are using a mix of both, and nobody has mapped which is which. The distinction between enterprise and consumer tiers on this specific point is where most of the real exposure sits. An employee using an enterprise-licensed product with a properly negotiated data processing agreement is in a materially different position than an employee using the same vendor's free consumer product with default settings. The output is functionally identical. The data treatment is not. What the CTO and CIO actually need to understand The question is not whether AI tools create data exposure — they do, by design, in the same way any cloud service does. The question is whether the organization's data exposure through AI tools is understood, consented to, and consistent with its regulatory and contractual obligations. That requires knowing three things you probably do not know right now. What tools are actually in use. Not just the ones IT has approved — all of them. This means running discovery before designing governance. Most organizations that do this discovery find a longer list than they expected. What tier of each tool is in use. The enterprise agreement and the free consumer version of the same product often have dramatically different data processing terms. This distinction matters for training data use, retention, and deletion rights. What the data processing terms actually say. Not the marketing language about being "privacy-first" or "enterprise-grade" — the actual data processing agreement. Specifically: what the vendor can do with inputs, how long they retain them, what the organization's rights are around deletion, and where the data is processed. Most organizations have answered none of these questions systematically. The CIO knows what is in the procurement system. The CTO knows what is in production. Neither has a complete picture of what is happening between individual employees and third-party AI services. The regulatory and contractual layer Data flowing to AI tools does not exist in a vacuum. It intersects with existing obligations. If the organization operates under data protection regulation, any transfer of personal data to a third-party processor requires a legal basis and, in many jurisdictions, a data processing agreement that specifies how the processor may use the data. AI tools that process personal data — and most enterprise use cases involve at least some personal data — need to be assessed against these requirements. If the organization has contractual confidentiality obligations to clients, those obligations typically extend to how client data is handled regardless of the tool involved. A consultant uploading client strategy documents to an AI summarization tool without a data processing agreement in place may be in breach of their client agreement, regardless of whether the AI tool's terms are otherwise acceptable. These are not hypothetical risks. They are existing obligations that most organizations have not mapped against their AI tool usage. What to take from thisAudit what AI tools are in active use across the organization before designing any data governance response. The list will be longer than IT's approved toolset. Distinguish between enterprise and consumer tiers. The same tool can have dramatically different data processing implications depending on which version employees are using. Read the data processing agreements — specifically the sections on input retention, training use, and deletion rights. Do not rely on the vendor's marketing language. Map AI tool usage against existing data protection and client confidentiality obligations. The intersection is almost certainly not clean. Build a disclosure and classification step into any AI tool approval process: what categories of data can employees use with this tool, under what conditions?The data exposure from AI tools is not a future problem to prepare for. It is a current condition to understand. The organizations that handle this well are not the ones with the most restrictive policies — they are the ones that ran the discovery work, understood what was actually flowing through which tools, and made deliberate decisions about what that meant for their obligations.

Read full article
The AI Audit Your Board Should Be Asking For (But Probably Isn't)

The AI Audit Your Board Should Be Asking For (But Probably Isn't)

When organizations commission audits, they tend to know what they're looking for. A financial audit looks for misstatements. A cybersecurity audit looks for vulnerabilities. Both have established methodologies, credentialed practitioners, and a clear output format. No equivalent exists yet for AI — and the absence is starting to matter. Most boards that have approved AI investments can answer some questions about them: what budget was committed, which vendor was selected, whether the program is on schedule. Very few can answer the questions that actually determine whether the organization's AI exposure is understood and managed: which AI systems are currently making decisions that affect customers, employees, or revenue, who is accountable when those decisions are wrong, and would anyone be able to explain a specific bad outcome if asked to by a regulator or a plaintiff's attorney. The audit that answers those questions isn't a technical audit. It's a strategic one — a systematic review of what the organization is actually doing with AI, whether the governance structures in place are real rather than documented, and whether the accountability chains hold under scrutiny. What a strategic AI audit is A strategic AI audit is a decision audit. It asks: what decisions is AI making in this organization, and is the governance around those decisions adequate? This is different from a technical audit, which asks whether AI systems are built correctly. It's different from a compliance audit, which asks whether documentation requirements have been met. And it's different from a security audit, which asks whether AI infrastructure is protected from external threats. The strategic audit asks the governance question: if something goes wrong with an AI-driven decision, does the organization know what happened, does someone own it, and is the board in a position to account for it? In my experience, the answer to at least one of those three questions is "no" in most organizations that haven't specifically designed for it. AI systems accumulate in organizations faster than governance frameworks evolve to cover them. A use case that started as an internal productivity tool is now influencing hiring decisions. A model deployed for one market is being used in another where the regulatory context is different. A vendor has updated an underlying model and the organization's internally built layer is now operating on a different foundation than it was when it was approved. None of these are necessarily failures. All of them are things the board should know about — and typically doesn't. The three questions it needs to answer What AI systems are making consequential decisions, and do we have a complete inventory? Most organizations do not have a comprehensive inventory of their production AI systems. AI proliferates in ways that other technology doesn't — it's embedded in vendor products, built by business units operating outside central governance, and updated by vendors without explicit notification to the client. The first deliverable of any AI audit is an accurate map of what exists. "Consequential" is a meaningful threshold here. Not every AI system making recommendations needs the same governance treatment. An internal tool that suggests email response drafts is different from a model that scores customer loan applications or determines which job candidates advance to interview. The audit should focus governance energy on decisions that affect customers, employees, or material financial outcomes. Who is accountable when an AI-driven decision is wrong? This is the question that most AI governance documentation fails to answer concretely. Organizations have AI ethics policies, responsible AI frameworks, and model risk management guidelines. Very few of them name a specific person who is accountable for a specific model's outputs in production. The audit should resolve this to a named individual for each consequential AI system. Not a team. Not a committee. A person, in a role, with defined responsibilities for performance monitoring, incident escalation, and the decision to pause or decommission the system. Could anyone explain a specific bad outcome if required to? This is the forensics question. If a customer was denied credit by an AI model and files a complaint, can the organization trace the specific inputs that drove the decision, explain why the model weighted those inputs the way it did, and demonstrate that the decision was consistent with the model's approved use case and the organization's stated policies? In many organizations, the honest answer is no. The model exists in production, but the audit trail, explainability layer, and documentation necessary to reconstruct a specific decision either don't exist or aren't maintained in a format accessible to anyone outside the technical team. Why internal audit isn't equipped to run it alone Internal audit functions have the independence and mandate to commission this work. They typically don't have the domain expertise to execute it without specialist support — and that's worth being explicit about rather than papering over. An internal auditor assessing whether AI governance documentation is complete can do that independently. An internal auditor assessing whether the documentation reflects what's actually happening in production models, whether monitoring thresholds are set appropriately for the use case, or whether a model's training data is representative of the population it's scoring — that requires someone with operational AI experience. The practical answer is a co-sourced approach: internal audit drives the process and maintains ownership of findings, specialist external support provides the domain expertise for the technical evaluation components. The independence of the finding sits with internal audit. The technical credibility sits with the specialist. This is how most mature compliance functions handle domains where internal expertise is thin — it's not a novel structure, just one that AI hasn't yet been systematically included in. The business case Boards sometimes resist commissioning audits because the output is uncertain and the cost is visible. The AI audit case is stronger than that framing suggests. Regulatory exposure is real and increasing. The EU AI Act creates conformity assessment requirements for high-risk AI systems. Sector-specific AI guidance from financial regulators, the FDA, and employment regulators creates audit trails that organizations will need to produce. An AI audit conducted proactively is a fraction of the cost of a regulatory examination that finds governance gaps the organization didn't know it had. Operational risk is also material. A model that has been quietly degrading for months is a liability that doesn't appear on anyone's radar until it affects enough decisions to produce visible business consequences — customer complaints, adverse outcomes at scale, regulatory notice. An audit that finds this early is worth more than its cost. The D&O angle is worth raising directly with board members. Directors who approve AI strategies and investments are making decisions they will be held accountable for if something goes wrong at scale. An independent, documented review of whether the AI governance is adequate is meaningful protection. Approving an AI investment without it is a risk that sits with the individual director, not just the organization. Frequency and triggers For organizations with material AI exposure — models in production affecting customers, employees, or revenue at volume — an AI strategic audit should be an annual activity, structured similarly to other assurance reviews. Out-of-cycle triggers worth defining: a significant change to a production AI system or its underlying model; entry into a new market or use case with AI involvement; a regulatory examination or enforcement action involving AI anywhere in the industry; a visible AI failure in a comparable organization that prompts questions about whether a similar pattern exists internally; and any M&A that brings new AI systems into the organization. The audit doesn't need to be comprehensive every year. A rolling program that covers the highest-risk systems annually and lower-risk systems on a longer cycle is a practical approach for large organizations with many AI deployments. What it should never be is one-time. The AI landscape inside an organization changes faster than any other technology domain. A clean finding from two years ago is not evidence of a clean position today.

Read full article
When Your AI Vendor Fails: What Enterprise Continuity Planning Misses

When Your AI Vendor Fails: What Enterprise Continuity Planning Misses

Enterprise business continuity planning has matured significantly over the past decade. Most large organizations have detailed plans covering data center failure, key supplier disruption, network outages, and critical SaaS dependency. The plans are tested, updated annually, and presented to the board as evidence of operational resilience. Almost none of them cover AI vendor failure. This is a gap that is growing faster than organizations are closing it, because AI infrastructure has accumulated as a strategic dependency faster than any previous technology category — and the concentration of that infrastructure in a small number of providers creates a risk profile that most business continuity frameworks weren't designed to address. The concentration reality Most enterprise AI programs now run on infrastructure from three to five large providers. The foundational models — the large language models, the vision models, the embedding models — come from a handful of companies. The infrastructure to fine-tune, serve, and monitor those models runs predominantly on two or three cloud platforms. The tooling layer for MLOps, vector databases, and AI observability has its own concentrations. This isn't a failure of procurement strategy. It reflects the current state of the market: the compute requirements for training large models are only viable at a few companies, and the organizational dependencies compound over time as internal teams build workflows, integrations, and institutional knowledge around specific providers. The result is that a significant portion of enterprise AI capability now has a concentrated dependency structure that looks nothing like the diversified supply chains those same organizations maintain for physical goods or traditional software. The failure scenarios that aren't hypothetical None of the scenarios I'm going to describe are theoretical. All of them have happened to some organization or category of organization in the past few years. Pricing changes. API pricing for foundational models has changed multiple times since commercial availability began. Organizations that built business cases on a specific cost-per-call structure have seen those economics shift materially. At low volumes this is manageable. At production scale, a significant pricing increase is a P&L event that has to be absorbed or responded to — and the response options are limited when the alternative is rebuilding on a different model. API deprecation. Model versions get deprecated. The model version that a production system was built on, fine-tuned on, and evaluated against may be removed from availability on a timeline driven by the provider's product roadmap, not the client's operational needs. This forces either a migration under time pressure or an extended period of running on a deprecated version with no support and potential security exposure. Performance degradation. Foundation model behavior changes with version updates, even when providers describe updates as improvements. A model that performed reliably on a specific task may behave differently after an update that was not communicated as a breaking change. For AI systems that have been through a conformity assessment or regulatory approval process based on specific model behavior, this creates a compliance problem as well as an operational one. Regulatory shutdown. Regulators in multiple jurisdictions have shown willingness to restrict or suspend AI capabilities. This risk is higher for providers with significant regulatory exposure and for capabilities that sit in legally ambiguous territory. Acquisition and pivot. The AI infrastructure market is consolidating. A provider that is an independent company today may be acquired by a larger platform company whose strategic priorities don't include the specific capabilities your organization depends on. Post-acquisition product decisions are made by the acquirer. What business continuity plans currently miss Standard business continuity frameworks are built around the concept of recovery time objective (RTO) and recovery point objective (RPO): how quickly can we restore service, and how much data can we afford to lose? These are the right questions for infrastructure failures — a data center going down, a primary database becoming unavailable. They're the wrong questions for AI vendor risk, because the failure modes are different in character. A data center failure is abrupt and typically temporary. An AI vendor pricing change is gradual and potentially permanent. A model version deprecation has a defined timeline but requires significant engineering work to respond to. A performance shift may not be immediately detectable and may require revalidation of systems that were previously approved. Business continuity planning for AI needs to address a different set of scenarios: what does the organization do if this model version is deprecated in 90 days? What does the organization do if this provider's pricing increases by 40%? What does the organization do if a capability we depend on is restricted by a regulator? These scenarios require capability-based responses, not infrastructure failover. The portability question The honest answer for most organizations is that they are significantly more locked in than they realize. Fine-tuned models, when the fine-tuning has been done on a provider's infrastructure, may not be portable in a form that can be redeployed elsewhere without significant rework. Embeddings and vector indexes built against one model's embedding space are not compatible with a different model's embedding space without recalculation. Prompts engineered for one model's behavior may produce degraded outputs on a different model without re-optimization. The integration layer — the application code, the data pipelines, the orchestration logic — is generally portable. The model-specific components often aren't, and those are frequently where the most time and expertise have been invested. Understanding your actual portability position requires a dependency mapping exercise that most organizations haven't done: for each production AI system, what would it take to migrate to an alternative model, and how long would that take with current team capability? Contract provisions that actually matter Standard enterprise software contracts are not adequate for AI vendor relationships. The provisions that should be in place, and often aren't: SLA definitions for model behavior. Standard SLAs cover uptime and response time. They don't cover model performance consistency. If the provider can change model behavior without notification and without SLA consequence, the client has no contractual protection against the performance degradation scenario. Data portability rights. Any fine-tuning data, output data, or evaluation data held by the provider should be contractually accessible and exportable. This matters both for migration and for regulatory compliance — organizations may need to produce this data in response to a regulator. Deprecation notice periods. Minimum notice periods before model version deprecation should be defined contractually. Ninety days is common in SaaS contracts for feature deprecation — AI model deprecation warrants at least the same, and often longer given migration complexity. Audit rights. The right to audit model behavior, training data provenance (where relevant), and security practices. This matters for regulatory conformity — particularly for organizations subject to the EU AI Act's high-risk system requirements. Successor model provisions. What obligations does the provider have when a model version is deprecated? Is there a migration support commitment? Are there protections against the replacement model failing to meet the performance specifications of the deprecated version? The resilient procurement position No procurement position eliminates AI vendor risk. But the gap between organizations that have thought about this and those that haven't is significant. The elements of a resilient position: Multi-vendor architecture for critical systems. For AI capabilities that support critical business processes, architectural design should account for vendor substitution. This doesn't mean parallel deployment of everything — it means understanding which systems could be migrated within an acceptable timeframe and what would be required to do it. Open-weight model capability as insurance. Maintaining the internal capability to run open-weight models — Llama, Mistral, and similar — creates an option that most organizations are currently not exercising. These models may not match proprietary model performance for all use cases, but for some use cases they're adequate, and the ability to fall back to self-hosted capability removes one category of vendor dependency. Internal capability as a floor. The organization should maintain sufficient internal AI capability that it can evaluate vendor claims, understand what it's using, and make migration decisions based on technical judgment rather than pure vendor dependency. This doesn't require building everything internally. It requires not outsourcing understanding. Documentation for migration. For each production AI system, maintaining documentation of what the system does, what inputs it requires, what performance benchmarks it meets, and what alternatives were considered creates a foundation for migration planning that doesn't require starting from scratch under pressure. The business continuity plan that doesn't include AI vendor failure scenarios is a plan with a known gap. The question is whether the gap is closed before or after the scenario that reveals it.

Read full article
What the C-Suite Gets Wrong When Briefed by Their Own AI Teams

What the C-Suite Gets Wrong When Briefed by Their Own AI Teams

I've sat on both sides of the AI executive briefing. I've given them, and I've prepared executives to receive them. The gap between what gets presented and what's actually happening in an AI program is not unique to any particular organization — it's structural, and it runs in one direction consistently. The direction is optimism. Not because AI teams are dishonest. Because they're humans operating in an organizational context where progress is rewarded, setbacks are uncomfortable, and the executives receiving the briefing are rarely equipped to distinguish between a meaningful demonstration and a controlled one. The incentive structure produces a particular kind of briefing, and the C-suite needs to understand that structure to extract accurate information from it. The incentive problem An AI team's relationship with executive leadership is shaped by several pressures that all point toward positive framing. The team secured budget by promising something. Every briefing is an implicit progress report against that promise. Acknowledging that the promise was wrong — that the timeline was too short, the use case was harder than expected, the production environment is more constrained than the POC assumed — is a career-adjacent risk. The team works in a domain that most of its executive audience doesn't understand deeply. This creates both an opportunity and a temptation. The opportunity: a technically sophisticated team can explain complex tradeoffs honestly and build genuine understanding. The temptation: a technically sophisticated team can use that complexity to obscure problems that would be obvious in plain language. The team has invested months — sometimes years — in a program. The sunk cost effect is real. Acknowledging that the program is not working as designed, or that the architecture needs to change, or that the use case selection was wrong, requires a level of intellectual honesty that is harder when you've built your professional identity around the thing you're assessing. None of this is malicious. All of it is human. The C-suite needs to account for it. The three things that get consistently obscured The gap between demo performance and production performance. A live demo is not a production system. This distinction sounds obvious. In executive briefings, it consistently isn't. A demo is run on curated data, in a controlled environment, with known good inputs, by the person who built it. Production is run on real data — messier, more varied, more adversarial — in an environment with different latency, different system interactions, and different edge cases than the demo accounted for. The performance gap between demo and production in AI systems is often 15–30 percentage points on key metrics. A model that achieves 94% accuracy in a demo may achieve 78% accuracy in production against the real distribution of inputs. The team knows this, or suspects it, and the demo is generally not where they surface it. When a briefing leads with a demo, the question that matters is: what does this look like against the real production input distribution, over the last 30 days? Not "can you show me it working" — "what's the monitored performance over real traffic?" The timeline the team actually believes vs. the timeline in the deck. Project timelines in AI executive briefings are almost always optimistic. The reasons are predictable: timeline estimates are produced under pressure to show momentum, AI programs have more unknown unknowns than most program types, and the cost of presenting a longer timeline (reduced budget enthusiasm, increased scrutiny) is visible while the cost of presenting an optimistic one (eventual overrun) is future. The tell is usually in the dependency language. "This timeline assumes the data infrastructure work completes in Q1" — where is the data infrastructure work currently? "This assumes we have the ML engineer hired by month three" — what's the current hiring status? Dependencies that are "assumed" in a timeline slide are often dependencies that are behind or at risk but not presented as such. The useful question: what is the most likely single point of failure in this timeline, and what's the contingency if it doesn't resolve? The production failure rate. AI programs accumulate failures — model predictions that were wrong, system behaviors that didn't match expectations, user adoption that didn't develop as projected. In executive briefings, these are typically either absent or characterized as "learnings" without the quantitative dimension that would allow an executive to assess their significance. A briefing that describes a "challenging quarter with good learnings" but doesn't specify what the model's production accuracy was over that quarter, what percentage of outputs were overridden by human reviewers, or how far business outcomes deviated from projection is a briefing that has converted failure information into narrative. The useful request: show me the model performance trend over the last six months, in actual numbers, against the performance targets that were set at program start. The benchmark trap AI teams report model performance using benchmark metrics. The most common are accuracy, precision/recall, and AUC. These are meaningful for comparing models and for tracking technical progress. They are not the same as business outcomes, and the relationship between them is often assumed rather than demonstrated. A model that improved AUC from 0.83 to 0.89 over the quarter has made genuine technical progress. Whether that progress translates into better business outcomes — more accurate fraud detection, better loan decisions, fewer customer service escalations — requires a different measurement entirely, one that connects model output to downstream business process. That connection is frequently not in the briefing. The question: for each technical metric in this briefing, what is the business metric it's supposed to drive, what is the current value of that business metric, and what is the target? If the AI team can answer that question clearly, the program has good outcome alignment. If the answer is complicated or deferred — "we're still working on the measurement framework" — the program may be optimizing for technical progress without a clear line to business value. Reading the language Certain language patterns in AI executive briefings are diagnostic of underlying program health. "We're making good progress" without specific metrics usually means the program is moving but metrics aren't where they need to be. If progress were specifically good, the specific numbers would be in the briefing. "The model is performing well in testing" without production performance data means the team is presenting test performance because production performance is worse or unmeasured. "We're seeing strong adoption" without adoption rate numbers means adoption is incomplete. Strong adoption would be presented as a number. "The data quality challenges are being addressed" means the data quality challenges have not been resolved and are affecting model performance. "Addressed" and "resolved" are different things. "We're on track" against a milestone that was previously described as at risk means the milestone was re-scoped to make it achievable, or the team has decided to declare it complete at a lower quality level than originally intended. None of these are lies. They're organizational language patterns that absorb uncertainty and make things sound more resolved than they are. Reading them accurately requires pattern recognition that executives build over multiple briefing cycles, or that they can shortcut by asking for the underlying numbers. When to get a second opinion There are situations where the C-suite should commission an independent assessment rather than relying solely on internal briefing. When a program has been running for more than 12 months and the production deployment is still described as upcoming. When the metrics in briefings have changed categories over time — when the team starts reporting different metrics than it started with, it's worth asking whether the metrics changed because the original ones showed the wrong trend. When the business unit that was supposed to benefit from the AI program is not prominently represented in the briefings as an active champion. When a specific milestone has slipped more than twice. These aren't definitive indicators of a failing program. They are indicators that the C-suite doesn't have a fully accurate picture and should find out why before making the next resource allocation decision. An independent technical advisor who can review program documentation, interview the team, and assess what's actually in production — without career stake in the outcome — produces a different quality of information than an internal briefing. The cost of commissioning one is small relative to the cost of continuing to invest in a program based on an inaccurate picture of its health.

Read full article