What a Data Breach Looks Like When AI Is in the Middle of It

What a Data Breach Looks Like When AI Is in the Middle of It

Most enterprise data breach response plans were written for a specific type of incident: unauthorized external access to a database, a misconfigured cloud storage bucket, a stolen credential, a ransomware attack. The response playbook is well understood. Contain the breach, assess the scope, notify regulators, notify affected individuals, remediate the vulnerability. When an AI system is in the middle of a breach — as a vector, as an amplifier of exposure, or as the primary source of the incident — the playbook breaks down in several places. The scope assessment is harder. The cause is less obvious. The regulatory notification may require analysis the organization has not done. And the communications with affected parties need to account for AI involvement in ways that the standard template does not anticipate. Organizations that have AI systems in production and have not updated their incident response plans are carrying risk they have not quantified. The ways AI changes the breach scenario AI as a vector. A prompt injection attack — where malicious content in the AI system's input causes the system to execute unintended actions — is a category of attack that did not exist before AI systems were connected to organizational data. The technical mechanics are different from a SQL injection or a credential attack, but the organizational response involves the same triage: what did the attacker access, what did they exfiltrate, what actions did the AI system take on their behalf? Prompt injection is not theoretical. It has been demonstrated against production AI systems across multiple vendors. Organizations that have not evaluated their AI systems against this class of attack have a gap in their security assessment. AI as an amplifier. An attacker who compromises credentials to an account with AI system access may be able to extract substantially more information than they could from the underlying data systems alone. The AI system's ability to query, synthesize, and summarize across data sources means that a single compromised session can produce outputs equivalent to weeks of manual data extraction. The scope of a breach involving AI access is likely to be larger than the scope of a breach involving equivalent access to the underlying data without AI. This matters for the scope assessment, for regulatory notification thresholds, and for the volume of affected records. AI as the source. Misconfigurations in AI systems — incorrectly permissioned data access, insecure output handling, improperly sandboxed tool use — can themselves cause data exposure without any external attacker. An AI system that surfaces information it should not have had access to in response to a user query, or that exposes data through an incorrectly configured output channel, has caused a data exposure incident even in the absence of a security breach. These incidents are less dramatic than external attacks but potentially more common. And they are harder to detect because the behavior looks like normal AI system use rather than an anomalous external access pattern. Where the standard response plan fails Scope assessment. The standard scope assessment for a data breach identifies which records were accessed. When an AI system was involved, the relevant question is not which records were accessed but which outputs were generated — what did the AI synthesize from the records it could reach, and what information was contained in those outputs? This is a harder problem. AI outputs are not automatically logged in the way that database queries are. The organization may not have complete records of what the AI system produced during the breach window. Reconstructing the scope requires different methods than a traditional database access log analysis. Cause determination. Traditional breaches have identifiable technical causes: a vulnerability, a misconfigured permission, a phishing attack. AI incidents often have more diffuse causes — a combination of permissive access, insufficient output monitoring, and system behavior that was technically within parameters but produced an unintended result. Root cause analysis for AI incidents requires understanding of the AI system's architecture and behavior that most incident response teams do not have. Regulatory notification. Data breach notification requirements typically specify notification timelines and the content of notifications. When an AI system is involved, determining what categories of personal data were exposed requires understanding what the AI could access and what it may have surfected — an analysis that takes longer and requires more specialized input than a direct database access log review. Communication with affected parties. Breach notification communications are standardized around the concept of "your data was accessed by an unauthorized party." When an AI system was the mechanism, the communication needs to explain something more complex: what the AI system could access, what it may have produced, and why that creates risk for the affected individual. Most breach communication templates are not equipped for this. What the CFO and CIO need to prepare now Update the incident response plan. The plan needs to include AI-specific scenarios: prompt injection, AI-amplified credential breach, misconfiguration-driven data exposure. Each scenario should have a defined response team (which needs to include AI system expertise), assessment methodology, and escalation path. Establish AI audit logging requirements. If the organization does not have comprehensive logging of AI system queries and outputs, it cannot conduct a complete scope assessment for an AI-involved incident. The logging requirement needs to be part of AI system deployment standards, not something added after an incident. Define who owns AI incidents. Traditional breach response has clear ownership — typically the CISO and legal team with CFO involvement for material incidents. AI incidents may involve technical characteristics the CISO team does not have expertise in. Define who the AI-specific escalation path involves and ensure that person or team is part of incident response planning. Test the plan. Incident response plans for traditional breaches are tested through tabletop exercises. AI-specific scenarios should be part of the tabletop exercise inventory. The scenario of an AI system producing outputs it should not have, or being used as a vector by an attacker, is sufficiently different from traditional scenarios to warrant explicit testing. Understand regulatory notification requirements. Check whether the data protection officer's understanding of notification thresholds and timelines accounts for AI-involved incidents. In particular: the scope determination for an AI breach may take longer than for a traditional breach, and the notification timeline starts from discovery of the breach, not from completion of scope determination. What to take from thisUpdate the incident response plan to include AI-specific scenarios before an incident occurs. The scenarios are different enough from traditional breaches to require explicit planning. Require comprehensive logging of AI system queries and outputs as a deployment standard. Without it, scope assessment for an AI-involved incident is incomplete. Define AI-specific escalation paths within the incident response structure. The expertise required to assess an AI incident is different from traditional breach response expertise. Test AI breach scenarios in tabletop exercises. The behavior of an AI system during and after an attack is counterintuitive enough to warrant practice. The scope of an AI-amplified breach is likely larger than an equivalent breach without AI involvement. Build this into the material incident threshold assessment.The organizations that handle AI-involved incidents well are not the ones that were lucky enough to avoid them. They are the ones that updated their preparedness before the first incident, so that when it happened — and it will happen — the response was organized rather than improvised.

Read full article
Enterprise ML Model Selection: How to Choose Without Getting It Wrong

Enterprise ML Model Selection: How to Choose Without Getting It Wrong

Every enterprise ML project I've seen go wrong had one thing in common: the team picked the model before they understood the problem. Not the business problem — they usually had that written down somewhere. I mean the operational problem: what does production actually look like, who consumes the output, what happens when the model is wrong, and what does the organisation's tolerance for that look like. Model selection in enterprise settings is treated, too often, as a technical decision. The data scientists run a few benchmarks, the most accurate model wins, and the project moves forward. Six months later the model is sitting in a notebook because no one can explain its outputs to the risk committee, or it's running too slowly at inference to be useful, or it requires a retraining cadence no one budgeted for. This is the article I wish existed when I was doing this work early in my career. It's not about which model architecture is theoretically best for a given data type. It's about how to think through the selection decision in a way that holds up when the project hits the real world — regulatory scrutiny, budget conversations, operational constraints, and the thousand other things that don't appear in benchmark papers. The framework here is built from real projects, primarily in financial services, insurance, and logistics. The patterns apply more broadly. Where I've seen exceptions, I'll say so. The question you're not asking at the start Most teams start model selection by asking: what model performs best on our data? That's the third question you should ask. The first two are:What does the model's output need to do in the system it's deployed into? What are the non-negotiable constraints on how the model operates?Output requirements determine what "performance" even means. A credit risk model that predicts default probability doesn't just need to be accurate — it needs to produce a calibrated probability, because downstream systems are making threshold decisions based on it. An object detection model for warehouse automation doesn't need the highest mAP score; it needs consistent latency under 50ms with a failure mode that's recoverable. A demand forecasting model for supply chain might need interpretable feature contributions so planners can override it credibly. Once you know what the output actually needs to do, you can define a meaningful evaluation metric. Before that, you're benchmarking against a proxy that may have nothing to do with operational success. Non-negotiable constraints come in two flavors: hard and soft. Hard constraints eliminate entire model classes before you've run a single experiment. Soft constraints shape the selection decision once the hard constraints narrow the field. Common hard constraints in enterprise:Inference latency — if you need sub-100ms response in a user-facing API, large transformer models are off the table unless you have a serious serving infrastructure budget Explainability requirements — if a regulator requires feature-level explanations for every decision (common in credit, insurance underwriting, and healthcare), black-box models require an explanation layer that adds its own failure modes Data residency — if training data cannot leave a specific jurisdiction, any model requiring cloud-based training infrastructure is constrained Retraining frequency — if your data distribution shifts fast and you cannot support frequent retraining, you need a model that degrades gracefully or one that incorporates online learningThe soft constraints — deployment complexity, team familiarity, tooling compatibility — shape the shortlist once the hard constraints have done their filtering work.Why model complexity is a separate axis from model performance There's a prevailing assumption in ML that a more complex model is always better if you have enough data. In research settings, this is often true. In enterprise settings, it creates a class of problems that don't show up until the model is in production. Complexity has costs that compound over time: Debugging cost. When a complex model produces an unexpected output — a prediction that triggers an alert, a recommendation that contradicts business logic — the investigation takes longer. With a gradient boosting model, a skilled analyst can often trace the output back to specific feature values within minutes. With a deep neural network, you may be running attribution methods that give you approximate explanations, not definitive ones. Retraining cost. A neural network that takes six hours to train on a GPU cluster has a different operational footprint than a gradient boosting model that trains in twenty minutes on CPU. If your use case requires weekly retraining — common in fraud detection, recommendation, and demand forecasting — that cost is real and recurring. Serving cost. Large models have larger memory footprints and higher per-inference compute requirements. At low request volumes this is invisible. At scale it becomes a line item that someone will eventually want to reduce. Team dependency. A model that only one or two people on the team can work with is a risk. It's not a model risk — it's a key-person risk disguised as a technical choice. None of this means you should always pick the simpler model. It means complexity should be justified by the performance gain in terms that the business actually cares about. "Our test AUC went from 0.87 to 0.91" is not a justification. "This 4-point AUC improvement translates to £2.3M in annual fraud loss reduction, and here's the operational cost of running the more complex model" is. The four model classes you'll actually choose between In practice, most enterprise ML problems resolve to one of four model families. There are edge cases and exceptions, but if you've been in the field long enough, you know that 80% of production ML at enterprise scale runs on:Gradient boosting (XGBoost, LightGBM, CatBoost) Linear and regularised linear models (logistic regression, ridge, lasso, elastic net) Ensembles on structured data (stacking, voting classifiers) Neural networks (feedforward networks, CNNs, RNNs — for image, audio, and dense sequential data)The fifth category — time series models (ARIMA, Prophet, temporal fusion transformers) — is real but specific enough that it tends to self-select based on problem type. Here's how these four map to enterprise contexts. Gradient boosting This is the default model for structured/tabular data in enterprise settings. If your problem involves tabular data — transaction records, customer attributes, sensor readings, operational logs — and you have tens of thousands to tens of millions of rows, gradient boosting is where you start unless a hard constraint rules it out. It handles missing values natively, is tolerant of feature scale differences, produces good out-of-the-box performance with reasonable hyperparameter defaults, and has mature tooling around SHAP-based explainability. XGBoost and LightGBM in particular have inference speeds that make real-time deployment practical on standard infrastructure. The limitations are real. Gradient boosting doesn't generalise well to image or text inputs without heavy feature engineering. It requires more hyperparameter tuning than linear models to get to peak performance. And it can overfit on small datasets in ways that aren't always obvious from standard train/test splits. Linear and regularised linear models Underused in many teams, and it's a mistake. For high-stakes decisions where regulatory explainability is a hard constraint — credit decisions under GDPR's right to explanation, insurance pricing, medical risk scoring — a well-engineered logistic regression model is often the right answer. The performance gap between a well-featured linear model and a complex model is frequently smaller than practitioners expect. I've seen logistic regression models with carefully engineered features outperform gradient boosting on small datasets, and match it on medium ones. The feature engineering work is harder, but it forces a discipline around understanding your predictors that benefits the entire project. Inference is cheap, retraining is fast, and the model coefficients are directly auditable. In regulated industries, these properties are worth a lot. Ensembles on structured data Useful when you have a stable problem where you've exhausted single-model performance and the complexity cost is manageable. Stacking approaches in particular can squeeze meaningful performance gains from combining models that have different error patterns. In practice I find ensembles most useful as a late-stage optimisation rather than a first-pass choice. Start with a single model, understand its failure modes, then consider whether an ensemble addresses those failure modes specifically. Neural networks These have two distinct enterprise use cases that should not be conflated. The first is unstructured data — images, audio, and dense time-series signals. For these inputs, neural networks aren't a choice so much as a requirement. No tabular model is going to process a satellite image or a raw sensor waveform. The architecture question — CNN, RNN, transformer encoder — follows from the data structure, not from a benchmark. The second is structured data where the relationship between features is complex enough that tree-based models consistently underperform. This is less common than practitioners assume. When it does occur, it's usually in problems with dense numerical features, long sequential patterns, or data volumes that warrant the training cost. Fraud detection at very large scale sometimes hits this case. So does next-item recommendation for very large catalogues. The mistake I see most often is applying neural networks to structured data problems not because there's evidence they'll perform better, but because they feel more sophisticated. That instinct is worth interrogating before you commit to the infrastructure requirements.Evaluation: what your benchmark isn't telling you Offline evaluation — splitting data, training, measuring test performance — is necessary but not sufficient. It tells you how the model performs on historical data under the assumption that the future looks like the past. That assumption fails in ways that matter in enterprise settings. Class imbalance and operational thresholds. Most enterprise ML problems are imbalanced — fraud is rare, defaults are rare, equipment failures are rare. Optimising for AUC on an imbalanced test set tells you something about discriminatory power but nothing about what threshold to operate the model at in production. Operating threshold decisions depend on the relative cost of false positives and false negatives, and those costs are business decisions, not statistical ones. Make sure your evaluation simulates the threshold decision you'll actually make, not just the ranking performance. Distribution shift. Your test set is a sample of historical data. Production data arrives from a distribution that drifts. This is obvious in theory and consistently underestimated in practice. At minimum, evaluate your model on a held-out time slice that's more recent than your training data — not a random split. Better still, run a temporal cross-validation scheme that simulates how the model will be retrained and evaluated over time. Data leakage. In enterprise datasets, features are constructed from operational databases that can contain subtle leakage — information that was available at prediction time in the training set but wouldn't be available at inference time in production. Timestamp-based feature construction, lookback windows, and aggregated customer history features are all common leakage sources. I've seen models that test at 0.95 AUC and operate at 0.72 because of leakage that wasn't caught in evaluation. Review every feature construction for temporal validity. Shadow scoring. Before promoting a model to production decision-making, run it in shadow mode — score live data, store the predictions, but don't act on them. Compare shadow predictions against actual outcomes over a meaningful time period. This catches distribution shift that didn't show up in historical evaluation and gives you confidence intervals on live performance before it has operational consequences. # Temporal cross-validation — evaluate across time slices, not random splits from sklearn.model_selection import TimeSeriesSplit import numpy as npdef temporal_cv_score(model, X, y, timestamps, n_splits=5): """ Evaluate model across time-ordered folds. Assumes X and y are sorted by timestamp. """ tscv = TimeSeriesSplit(n_splits=n_splits) scores = [] for fold, (train_idx, test_idx) in enumerate(tscv.split(X)): X_train, X_test = X.iloc[train_idx], X.iloc[test_idx] y_train, y_test = y.iloc[train_idx], y.iloc[test_idx] train_end = timestamps.iloc[train_idx[-1]].date() test_end = timestamps.iloc[test_idx[-1]].date() model.fit(X_train, y_train) score = model.score(X_test, y_test) scores.append(score) print(f"Fold {fold + 1} | Train up to: {train_end} | Test up to: {test_end} | Score: {score:.4f}") print(f"\nMean: {np.mean(scores):.4f} | Std: {np.std(scores):.4f}") return scoresExplainability is not optional — it's a deployment requirement In enterprise settings, explainability is usually treated as a nice-to-have that gets deferred to the end of the project. This is consistently a mistake. The downstream consequences of explainability gaps are severe:A risk committee that can't understand model outputs will not approve deployment A customer facing a declined application has legal rights in many jurisdictions to a meaningful explanation An operations team that can't diagnose why the model is producing unusual outputs cannot respond to incidents effectivelyExplainability requirements should be captured at the problem definition stage, not retrofitted after model selection. For gradient boosting models, SHAP values are the current best practice. They're computationally tractable, locally accurate, and the tooling (the shap library) is mature. Tree SHAP specifically runs in polynomial time and is fast enough for batch scoring workflows. For neural networks, SHAP and LIME both apply but with limitations. SHAP DeepExplainer and GradientExplainer work for many architectures but can be slow at scale. Integrated Gradients is a solid alternative for differentiable models. The important thing is to define what quality of explanation is acceptable before you choose the model — not the other way around. For linear models, the model is the explanation. Coefficients with appropriate standardisation give you feature contributions directly. This is why linear models are undervalued in regulated industries: the explanation isn't an approximation layer built on top of the model, it's intrinsic to the model structure. One distinction worth making explicit: global explainability (which features drive the model overall) is useful for model validation and stakeholder communication. Local explainability (why did the model produce this output for this instance) is what's required for operational incident response and, in many jurisdictions, regulatory compliance. Make sure you have both. A note on language models and unstructured text If your problem involves raw text — contract analysis, document classification, clinical notes — the model selection conversation shifts. NLP is its own decision space, and the selection logic there is different enough to warrant separate treatment. What I will say here: the most common mistake I see is reaching for a language model on a problem that is fundamentally a classification or extraction task on structured fields. If the data is structured and the target is a label or a number, stay in the framework above. Language models applied to structured data problems almost always lose on latency, cost, and operational maintainability compared to a well-featured gradient boosting model. Governance, monitoring, and the model's operational lifetime A model selection decision is not just a decision about which model to train. It's a decision about what you're committing to maintain. Every enterprise model needs: Performance monitoring. Track your target metric on a holdout sample that continues to be labelled over time, or use proxy metrics that correlate with model performance where ground truth labels are delayed. For fraud models, ground truth arrives quickly. For churn models, it can take months. Design your monitoring for your label latency. Data drift detection. The distribution of your input features shifts over time. Monitor input distributions using statistical tests (KS test, PSI — Population Stability Index is the standard in financial services) and alert when drift exceeds a defined threshold. Prediction drift. Monitor the distribution of model outputs independently of input drift. Prediction drift without input drift often indicates a feature engineering issue. Input drift without prediction drift may mean your model is more robust than expected — or that your monitoring isn't sensitive enough. A defined retraining trigger. Don't retrain on a fixed schedule unless your problem domain specifically justifies it. Retrain when monitored metrics fall below defined thresholds. This avoids unnecessary retraining when the model is still performing, and avoids delayed response when it isn't. # Population Stability Index — standard financial services drift metric import numpy as npdef psi(expected, actual, buckets=10): """ Calculate PSI between expected (training) and actual (production) distributions. PSI < 0.1 → no significant change PSI 0.1–0.2 → moderate change, monitor PSI > 0.2 → significant shift, investigate retraining """ breakpoints = np.linspace(0, 1, buckets + 1) expected_pct = np.histogram(expected, bins=breakpoints)[0] / len(expected) actual_pct = np.histogram(actual, bins=breakpoints)[0] / len(actual) # Avoid log(0) expected_pct = np.where(expected_pct == 0, 0.0001, expected_pct) actual_pct = np.where(actual_pct == 0, 0.0001, actual_pct) psi_value = np.sum((actual_pct - expected_pct) * np.log(actual_pct / expected_pct)) return psi_valueThe governance question is also about who owns the model after it ships. In most enterprises, models are built by a data science or ML engineering team and then handed to an operational team that doesn't have the expertise to manage them. The selection decision should account for this. A model that requires specialist intervention to retrain or debug is a higher operational risk than a model that can be maintained by the team who'll own it long-term. What to take from thisDefine output requirements and hard constraints before shortlisting any model class. What the output needs to do in your system determines what "good performance" means — before you run a single experiment. Justify complexity in business terms. A performance improvement is only meaningful if it translates to an operational outcome. Calculate the cost of increased model complexity — debugging, retraining, serving, team dependency — and weigh it against the gain. Use temporal cross-validation, not random splits. Historical performance evaluated on random splits overstates expected production performance for almost all enterprise ML problems. Evaluate on time-ordered folds that simulate your actual retraining cycle. Treat explainability as a deployment constraint, not a post-hoc feature. Capture explainability requirements at the problem definition stage. If a regulator or risk function requires feature-level explanations, that requirement should filter your model selection, not be solved with a wrapper after the fact. Run shadow scoring before production promotion. Score live data with the new model for a meaningful period before it drives decisions. This catches distribution gaps that offline evaluation misses. Design monitoring before you deploy. Define your drift thresholds, label latency handling, and retraining triggers as part of the deployment design. A model with no monitoring is not a production model — it's a ticking clock. Don't confuse team capability with model capability. A model the team cannot maintain, debug, or retrain without specialist support is a liability dressed as a technical choice. Ownership needs to be part of the selection criteria from day one.Model selection in enterprise settings is a systems problem, not a benchmark problem. The model you choose is the model you'll maintain, explain, monitor, and retrain for years. That timeline should be visible in the decision you make on day one. I've watched teams spend two weeks optimising a model's AUC by 0.02 points and zero time asking whether anyone had modelled the retraining cost, the explanation requirement, or what the failure mode looked like at inference. The technical work was excellent. The project stalled at deployment review because the answers to those questions weren't ready. Getting model selection right at enterprise scale is mostly a matter of asking the boring questions early — and insisting on answers before you write any training code.

Read full article
When Your AI System Becomes a Source of Competitive Disadvantage

When Your AI System Becomes a Source of Competitive Disadvantage

The business case for enterprise AI is almost always framed as an upside story. Productivity gains, quality improvements, faster decision-making, competitive advantage over slower competitors. The framing is not wrong — there are real benefits and they are significant. What tends to be absent from the business case is a honest assessment of the downside scenarios: the ways in which an AI system, poorly designed or inadequately governed, can actively harm the organization's competitive position rather than improve it. This is not a reason to avoid AI investment. It is a reason to think more carefully about the specific failure modes, because they are not obvious and the organizations that encounter them are often surprised by the channel through which the harm arrived. The pricing exposure problem Pricing logic is one of the most commercially sensitive forms of knowledge an organization holds. The rules that govern how deals are priced, what flexibility exists, where the floor is, and how different customer profiles are segmented represent years of market learning that competitors would pay substantially to understand. AI systems connected to CRM data, deal management systems, and pricing tools learn those patterns in the course of normal use. The risk is not necessarily that the AI reveals pricing logic externally — although that is a risk if the system interacts with clients or partners. The risk is that the system, if its access is not carefully controlled, makes the pricing logic accessible in ways that would not otherwise exist. An employee with access to a deal management system could, with effort, reconstruct pricing patterns from individual deals. An AI system with access to the same data can answer "what are the pricing thresholds we typically use for mid-market accounts in this vertical" in seconds. The information was always technically accessible. The AI made it effectively accessible. If that employee later joins a competitor, the information they have internalized about the organization's pricing approach is substantially richer if they worked with an AI system that made it easily queryable than if they worked with raw data that required effort to interpret. Strategy document proliferation Every AI system that helps with document drafting, summarization, and analysis leaves a trail of artifacts: intermediate drafts, summary documents, synthesized analysis, and conversation histories that reflect the strategic content fed into the system. These artifacts accumulate. In most organizations, nobody is managing them. The conversation history from a strategy planning session assisted by an AI tool lives in the tool's logs or in a chat interface export that gets saved to a shared drive with broader permissions than the original strategy documents. The proliferation of strategy artifacts through AI-assisted work is a real exposure. The discipline of handling strategic content carefully — compartmentalized access, appropriate distribution, secure storage — tends to dissolve when people are working fluidly with AI tools and producing artifacts as a natural byproduct. The client relationship surface area Organizations that use AI tools to assist with client work create a specific category of exposure: the AI system's access to client relationship context becomes a surface area through which client-sensitive information can migrate. This matters most in two scenarios. First, when employees who have worked with client information through an AI tool leave the organization — the contextual knowledge they take with them is richer because the AI made it more accessible and easier to process. Second, when the AI tool itself, through the mechanism of vendor data handling, creates a record of client relationship context that exists outside the organization's control. Neither of these is a dramatic failure. They are the kind of slow-building exposure that does not create a single incident but changes the risk profile of the organization's competitive position over time. The output channel problem AI systems increasingly generate content that goes directly to external audiences: customer communications, partner correspondence, market-facing materials. When the prompts that generate this content incorporate internal context, and when the review process is lighter than it would be for human-drafted content, the outputs can inadvertently reveal internal information. I have seen this manifest specifically in three ways. AI-drafted client proposals that reflected internal pricing rationale in the justification language. AI-generated market commentary that incorporated internal strategic positioning that had not been publicly disclosed. AI-assisted responses to procurement questionnaires that revealed internal capability assessments that were intended to be held back. In each case, the AI was using available context to produce more relevant output. That is the tool doing what it was designed to do. The failure was in the review process — human review was lighter because the AI-generated output looked professional and well-structured, and nobody caught the inadvertent disclosure. The dependency risk and what it does to negotiating position An organization that has deeply integrated a single AI vendor into core business workflows has a different negotiating position with that vendor than one that has maintained optionality. The vendor knows this. This is not unique to AI — the same dynamic applies to any deeply integrated enterprise technology relationship. But AI integration tends to be faster and deeper than traditional enterprise software, and the switching costs can accumulate before anyone has explicitly thought about what the dependency looks like. The competitive disadvantage here is not in what the AI system reveals — it is in the negotiating position the organization finds itself in at contract renewal, and in the operational exposure if the vendor relationship is disrupted. Turning the analysis into a practical question The practical question for a CTO and CFO is not "does AI create competitive risk" — the answer is yes in the ways described, and also yes it creates competitive advantage. The question is whether the specific deployment decisions being made have been evaluated against both sides. A few questions worth asking before the next AI deployment decision: What internal knowledge does this system have access to, and what would a competitor pay to know it? This is the most direct framing for pricing, strategy, and client relationship exposure. What artifacts does this system produce, how are they stored, and who has access to them? The artifact proliferation risk is almost never considered in deployment planning. What external outputs does this system generate, and what review process exists for catching inadvertent disclosures? The review discipline for AI-assisted outputs tends to be lower than for human-drafted equivalents. What would the organization's competitive position look like if a key employee who worked with this system extensively moved to a direct competitor? The answer to that question reflects the degree of competitive exposure the system creates. What to take from thisPricing logic made easily queryable through AI is more vulnerable to retention and misuse than pricing logic that required effort to extract. Scope AI access to pricing systems deliberately. AI-assisted work produces artifacts — conversation histories, intermediate summaries, synthesized documents — that tend not to be managed with the care applied to primary strategy documents. Build artifact handling into the governance model. AI-generated external content requires review discipline that is often lower than human-drafted content gets. The professional appearance of AI output does not mean it is free of inadvertent disclosure. Deep AI vendor integration creates switching costs and dependency that affect negotiating position. Evaluate this explicitly in vendor strategy. Ask explicitly: what would a competitor need to know about this system's data access to understand our strategic position? The answer identifies the highest-priority access controls.

Read full article
The Internal Data Access Problem That AI Makes Suddenly Visible

The Internal Data Access Problem That AI Makes Suddenly Visible

Access controls in most organizations work on a document-by-document basis. You have permission to read a file or you do not. The logic has been sufficient for most purposes because humans navigate information deliberately — they go looking for specific things and find what they have access to. AI tools have broken that model without anyone changing any permissions. When an AI system with broad read access is asked a question, it does not navigate to a specific document. It queries across everything it can reach, synthesizes what is relevant, and produces an answer. The access controls determine what the system can read. They do not determine what combinations it can surface, what inferences it can draw, or what aggregated view of the organization's data it can present to the user. The result is a category of access control failure that most organizations have not addressed, because the access controls themselves are technically correct — and still inadequate. The gap between technical access and intended visibility The cleanest way to describe the problem: in most organizations, there is a meaningful difference between what an employee technically has access to and what they were intended to be able to see. This gap exists because access management is messy in practice. Permissions accumulate over time as people join projects, take on new roles, and inherit access from reorganizations. Revocation processes lag behind changes. Distribution lists include people who should have rotated off. Shared drives created for one purpose get used for another. The intended access model and the actual permissions diverge, and in normal day-to-day work the gap is largely invisible because people go looking for things they need rather than systematically browsing everything they can reach. AI tools systematically browse everything they can reach. That is their function. An employee asking an AI assistant "what do we know about the performance review process for the engineering team" may receive an answer drawn from documents they technically have access to but were never intended to be the audience for — HR process documentation, individual feedback templates, comparative data that lives in a folder from an organizational design project two years ago that nobody cleaned up. The employee has not circumvented any security control. But they have seen something the access model was not designed to permit. The categories where this matters most HR and compensation data. Salary information, performance ratings, disciplinary records, and individual feedback exist throughout organizations in documents with permissions that were set for a specific purpose and have often drifted since. AI systems connected to broad document repositories will find this material and surface it in response to queries that touch on it. Legal and privileged material. Legal advice, litigation strategy, settlement terms, and attorney-client communications often exist in places that technically-authorized users can access for one purpose but should not be able to aggregate for another. The privilege protection may be legally intact — the employee can read the document — but the ability to synthesize across years of legal communications is a different kind of access. Financial data beyond role scope. Budget holders can typically access their own budget data. AI systems may surface aggregate financial data by drawing on individual documents each of which was appropriately accessible, producing a consolidated view that nobody intended to give the employee. Client and partner confidential information. Client files shared within engagement teams are accessible to all team members for legitimate work purposes. An AI system that can search across all engagement files simultaneously may surface patterns about client relationships, deal economics, or strategic situations that no single team member was supposed to see in aggregate. Why the standard response does not work The first response most organizations reach for is tightening access controls. If AI is exposing the problem, fix the permissions. This is not wrong, but it is not sufficient. The problem has two parts that require different responses. The first part is genuine permission drift that should be corrected regardless of AI. Employees who have retained access to systems and documents they no longer need it for should have that access revoked. This is an overdue access hygiene exercise, and AI deployment is a reasonable forcing function for doing it. The second part is structurally different. Even with clean, intentional permissions, an employee with access to many documents across an organization will technically have access to combinations of data that, when synthesized by an AI, reveal more than the permission model was designed to permit. You cannot solve this purely by tightening access, because the individual access grants may all be correct. The solution to the second part requires building constraints into the AI system itself: what categories of data it can include in synthesis across user queries, what aggregation rules apply, and what escalation or approval processes apply to queries that touch the highest-sensitivity categories. Building the right architecture Three things need to happen in parallel, not sequentially. Access control remediation. Run an access review scoped to the data sources the AI system will connect to. Specifically look for: permissions that predate current roles, broad read access granted for historical projects that is no longer needed, distribution list membership that has not been reviewed in over a year. This will not solve the problem completely, but it reduces the surface area. AI-specific access boundaries. Define, at the AI system configuration level, what categories of data the system can use for synthesis in response to user queries. HR data, compensation data, legal documents, and individual performance information may be categories where even technically authorized access should not be available to the AI synthesis function. These boundaries need to be implemented as technical constraints in the AI system, not just as policy guidance. Query monitoring and anomaly detection. The AI system's query logs are, for the first time, making the access control problem visible. An employee who systematically queries for compensation data across a broad population, or who extracts patterns from legal files, shows up in the query logs in ways they would not show up in document access logs. This monitoring capability is new and should be used. What the CIO needs to drive The access control gap in AI deployments is fundamentally a CIO problem, not an AI team problem. The AI team can build a capable system. The CIO needs to ensure that the system's access to organizational data is deliberately configured rather than broadly permissive by default. Broadly permissive by default is the path of least resistance. It makes the AI system more capable and easier to demonstrate. It also creates the access control failures described above, and the first incident involving inadvertent disclosure of HR or financial data through an AI tool is going to be a painful conversation. The access architecture needs to be designed before the AI system goes live. The conversation about what categories of data the system should not be able to synthesize — even if individual documents in those categories are technically accessible — needs to happen with legal, HR leadership, and the CFO, not just the AI team. What to take from thisTechnical access controls determine what an AI system can read. They do not determine what it will synthesize or surface. The gap between these is where the access control problem lives. Run an access control remediation exercise scoped to the AI system's data access before deployment. Clean up permission drift even if the AI deployment were not happening — AI just makes the urgency visible. Build AI-specific access boundaries into the system configuration. Some data categories should not be available for AI synthesis even if individual documents within them are technically accessible. Use AI query logs as an access monitoring tool. The visibility into what the system is being asked to surface is new and valuable. The CIO needs to own the access architecture decision, not delegate it to the AI team. The decisions about what data categories the AI should not aggregate require organizational input that the AI team is not positioned to provide alone.

Read full article
Stakeholder Management for AI Programs: What Nobody Tells You

Stakeholder Management for AI Programs: What Nobody Tells You

Every AI program has a stakeholder deck. Roles, interests, influence levels, communication plans. It's usually a PowerPoint slide with a 2x2 matrix, produced at program inception and never updated again. The deck doesn't tell you what you need to know. What you need to know is who the informal blockers are, which executive will withdraw support when results are mixed, what the CMO told the CFO about AI in the corridor last week, and whether the sponsor who signed off at the start is still paying attention. Stakeholder management for AI programs is different from stakeholder management for other kinds of programs because the stakes are higher, the uncertainty is more sustained, and the gap between what executives expect and what AI actually delivers in the early phases is systematically wide. The stakeholder map you're not building The standard stakeholder map captures formal power: titles, reporting lines, decision authority. It misses informal power — the relationships and reputations that determine whether an organization actually moves. The CFO who approved the budget but doesn't attend reviews. The head of operations who has a long history of skepticism about technology programs and whose quiet opposition can drain energy from a program without ever appearing in meeting minutes. The board member who had a bad experience with an AI vendor three years ago and is looking for evidence the same thing is happening again. These people don't appear in the stakeholder matrix because their influence isn't visible in an org chart. But they shape outcomes. Missing them is one of the most common and most expensive mistakes in AI program management. The map that actually works starts not from the org chart but from the question: who can make this fail? For each person who could derail the program — through active opposition, budget cuts, organizational changes, or simply withdrawing attention — the question is what they need to see, hear, and experience to stay aligned. The conversations that don't go in the deck A lot of the real work in stakeholder management happens outside formal review structures. The sponsor who asks you in the hallway whether the team is actually making progress or whether the weekly updates are theater. The CFO's chief of staff who is quietly assessing whether this program will be on the budget cut list. The senior engineer in the business unit who has been telling their colleagues the model won't work, and who is now looking at every output for evidence they were right. These conversations require a different kind of preparation than a status update. They require knowing what each person is actually worried about — not what they're saying in meetings — and being able to address that concern directly without appearing defensive or over-reassuring. The pattern I've found most useful: regular informal contact with the people who matter most to the program's survival, calibrated to where they are in their thinking. Not updates — conversations. What are they hearing? What questions are they carrying from other conversations? What would change their view? This sounds like politics. It is politics. Programs that pretend otherwise don't last. The mixed results problem Early AI program performance is almost never as good as the initial business case implied. The data is messier than expected. The use case turns out to be harder than it looked in the POC. The business integration takes longer. The performance metrics are trending in the right direction but aren't there yet. This is normal. It's also the period where programs are most at risk of losing stakeholder support. Three communication failure modes are predictable in this period. Over-promising is the most common. The program team, under pressure to demonstrate progress, frames results more positively than they are — emphasizing the metrics that are improving while not addressing the ones that aren't. This buys time, but it creates a credibility deficit that's hard to recover from when reality catches up with the framing. Over-qualifying is the inverse error. Every update comes with so many caveats, so many technical explanations for why the results aren't yet representative, that stakeholders stop believing anything the team says. Uncertainty is real in early AI programs, but there's a difference between acknowledging it and using it as a shield. Going silent is the most dangerous. The program team, sensing that results are below expectation, starts avoiding the conversations. Updates get more infrequent, meetings get canceled, access to the actual performance data becomes difficult. Stakeholders who are getting less information don't assume things are fine. They assume the opposite. The communication approach that actually works is also the hardest one: saying clearly what is and isn't working, what the team has learned, what has changed about the approach as a result, and what the revised timeline looks like. Early in an AI program, honesty about setbacks doesn't lose stakeholder confidence if it's paired with credible adaptation. What loses confidence is the impression that the team doesn't know what's happening or isn't telling you. When something goes visibly wrong At some point in most large AI programs, something goes wrong in a way that's visible: a model produces outputs that embarrass the business, a performance metric collapses, a system that was supposed to be in production isn't. How the program team handles that moment determines whether the program survives it. The instinct is to minimize and explain. This is usually wrong. A stakeholder who learns about a failure from someone other than the program team, or who hears the team's explanation and feels it's incomplete, will not recover trust easily. What works: telling the story before it's told to you, owning the root cause rather than distributing blame, presenting the specific changes the team is making as a result, and establishing a timeline for when confidence should be re-evaluated. Not "we're on track," but "this is what happened, here's what it means, here's what we're changing, here's when you should assess whether the change is working." The six-month alignment problem Stakeholder alignment established at program inception is not alignment that holds at month six. It's a starting position. Organizations change. Priorities shift. The executive who sponsored the program gets a new role. The business unit that was most engaged finds itself under budget pressure. The board committee that approved the investment wants to see different metrics than the ones the program was optimized for. Programs that maintain alignment do so through continuous work, not a one-time alignment exercise. That means regular recalibration with key stakeholders — not just reporting to them — and willingness to adapt the program's framing, pace, or scope when the external context changes. The hardest version of this is when the program itself needs to change course — when the original use case isn't working as planned and a pivot is necessary. Executing that pivot without losing stakeholder confidence requires having built the kind of trust where stakeholders believe the team is telling them the truth, including the uncomfortable parts. That trust is built over months of consistent, honest communication. It's not something you can manufacture when you need it. And the organizations that skip the groundwork — that assume formal sign-off equals ongoing support — are usually the ones scrambling at month nine to explain why the program needs more time, to stakeholders who have already stopped listening.

Read full article
What Your AI Vendor Knows About Your Business After Six Months

What Your AI Vendor Knows About Your Business After Six Months

When an organization signs an enterprise AI agreement, the focus is almost always on what the vendor will provide — model capabilities, performance benchmarks, uptime commitments, support terms. The less examined side of the exchange is what the vendor learns about the organization over the course of the relationship. This is not a question of whether the vendor is misusing data. Most enterprise AI vendors have robust commitments around data use and treat customer data with appropriate care. The question is subtler: what does the accumulated pattern of the organization's AI usage tell a sophisticated observer about how the business operates, and what are the implications of that information sitting with a third party for years? The implications are not obvious until you think them through. What usage data reveals An AI vendor with access to enterprise usage data can observe, at scale and over time, patterns that individual data points do not reveal. What the organization focuses on. The topics, domains, and question types that generate the highest AI usage volume reveal where the organization is directing attention. A spike in queries about regulatory compliance in a specific jurisdiction signals a business development or risk management concern before it shows up in any public disclosure. A sustained pattern of usage around a particular product area signals strategic investment before any announcement. How the organization works. The workflows AI tools are used in reveal process patterns: how decisions are prepared, what information sources are consulted, how different functions interact, where bottlenecks exist. This is the kind of operational picture that management consultants spend weeks building in client engagements. AI vendors accumulate it as a byproduct of normal usage. Where the organization's capabilities are strong and where they are not. The questions an organization asks of an AI system reflect, to some degree, what the people asking cannot do themselves. Heavy usage of AI tools for a specific type of analysis suggests that internal capability is limited in that area. A pattern of AI-assisted communication drafting in certain functions suggests communication capability constraints. Who the organization interacts with. Queries that reference client names, partner organizations, or market contexts — even in enterprise agreements where input content is excluded from training — create metadata about the organization's relationship network and market focus. None of this requires the vendor to actively analyze any specific piece of content. Aggregate usage patterns make these inferences available without individual query inspection. Why this accumulates over time The picture that emerges after six months of enterprise AI usage is qualitatively different from what was visible at month one. The accumulation of patterns across thousands of interactions, across multiple functions, across different business cycles reveals consistency and change in ways that a snapshot does not. Organizations change focus, enter new markets, encounter new challenges, and invest in new capabilities. All of those shifts are visible in AI usage patterns before they are visible elsewhere. The vendor relationship, if it persists, captures the strategic trajectory of the organization over time. This is particularly relevant for multi-year AI vendor relationships, which are increasingly common as organizations embed AI tools into core workflows. An AI vendor that has maintained an enterprise relationship for three or four years has accumulated a longitudinal view of the organization's strategic and operational evolution that very few parties outside the organization have. The vendor concentration dimension The question of what a single AI vendor knows about an organization becomes more significant when that vendor also serves the organization's competitors, its clients, or its industry peers. This does not mean the vendor is sharing information between customers — contractual commitments and practical self-interest both constrain that. But it does mean the vendor has a vantage point on industry-wide patterns that individual organizations lack. Aggregate insights about what questions enterprises in a specific industry are asking of AI systems, what capabilities they are developing, where they are investing — this is a form of competitive intelligence that accrues to the vendor in ways that have no clean analog in traditional software relationships. For organizations in sectors where competitive intelligence matters — financial services, pharmaceuticals, technology — the accumulation of strategic signal at a shared AI vendor is worth thinking about explicitly. What the CFO should factor into vendor relationship management The financial relationship with an AI vendor needs to account for switching costs that go beyond the cost of migrating to a new platform. The accumulated organizational context — the conversation history, the fine-tuned models, the usage patterns and metadata that have built up over years — creates a real switching cost that is not always visible at contract negotiation. Organizations that have deeply embedded a single AI vendor into core workflows may find that switching is more expensive than they anticipated, not because the technology cannot be replicated but because the years of accumulated context cannot easily be transferred. This is relevant to contract renewal negotiations, where vendors understand the switching cost dynamic better than most customers. It is also relevant to how the organization structures its AI vendor portfolio — whether to consolidate around a single vendor for maximum integration, or to distribute across vendors in ways that limit the strategic depth of any single relationship. What to do about it This is not an argument for avoiding AI vendors or maintaining zero-depth relationships. The value of AI tools requires meaningful integration, and meaningful integration creates the usage patterns described above. The practical response is to understand what the relationship accumulates and manage it deliberately. Conduct a periodic vendor relationship review that includes, alongside performance and cost, an assessment of what the vendor relationship has revealed about the organization through usage. This is not paranoia — it is the same kind of vendor relationship management organizations apply to any strategic supplier relationship. Review data minimization options. Many AI vendor agreements include options to limit usage data retention, opt out of certain analytics, or configure how interaction metadata is handled. These options are not always publicized, but they are often available in enterprise agreements. Understand them before defaulting to whatever the vendor's standard configuration produces. Consider the vendor concentration question explicitly in AI strategy. The organization that routes all AI usage through a single vendor is building a deeper relationship than the one that distributes across vendors. Both approaches have merits. The decision should be deliberate rather than a byproduct of procurement timing. Build contract terms around usage data explicitly. What the vendor can do with aggregate usage data — not just input content — should be addressed in the enterprise agreement, not assumed from the default terms. What to take from thisEnterprise AI usage creates an aggregate picture of the organization's focus, workflows, and capabilities over time. Understand what that picture contains. Multi-year AI vendor relationships accumulate strategic signal about the organization's trajectory. The longer the relationship, the more the vendor knows. Switching costs for deeply embedded AI vendors include the loss of accumulated context, not just migration effort. Factor this into vendor relationship management. Review data minimization options in enterprise agreements. They are often available and not actively surfaced. Address how the vendor may use aggregate usage data — distinct from input content — in the enterprise agreement terms.The organizations that handle this thoughtfully are not the ones who avoid AI vendor relationships. They are the ones who understand what those relationships accumulate and manage them with the same care they apply to any strategic supplier holding significant organizational knowledge.

Read full article