What AI Adoption Does to Your Existing Technology Contracts

What AI Adoption Does to Your Existing Technology Contracts

Deploying AI into an enterprise technology stack does not happen in isolation. It happens into an existing web of contracts: software licenses, SaaS agreements, data processing terms, and vendor relationships that were written before AI capabilities were relevant and that were not designed to accommodate what an AI program requires. The collision between AI deployment and existing contracts produces a category of problem that most organizations encounter somewhere in the middle of delivery, after commitments have been made and timelines are set. The CIO and general counsel who review the contract landscape before deployment starts are in a substantially better position than those who discover the issues under delivery pressure. There are five areas where existing contracts tend to create friction for AI programs. Software licensing terms and data use Many enterprise software licenses include restrictions on how data within the system can be used beyond its primary purpose. The typical language covers authorized users, permitted use cases, and sometimes explicit restrictions on automated processing or data extraction. When an AI system is connected to a licensed software platform to extract, process, or train on the data within it, those restrictions may be relevant. A CRM contract that limits data use to direct customer relationship management may not automatically permit the creation of an AI training dataset from CRM records. A document management system license that covers authorized human users may not straightforwardly cover an AI agent that queries the system as part of an automated workflow. The likelihood that existing enterprise software licenses explicitly address AI use cases is low — most were written before those use cases were anticipated. The risk is that implicit prohibitions, generic restrictions on automated processing, or data use limitations apply in ways that neither party anticipated. Before connecting AI systems to existing licensed platforms, general counsel should review the relevant license terms for restrictions on data use and automated access, and engage with vendors where the position is unclear. SaaS data portability and processing terms SaaS agreements typically govern what data is held in the platform, how it can be exported, and what the vendor can do with it. The standard SaaS agreement, particularly for products that predate the current AI era, was written with human-facing use in mind. When an AI program requires bulk data extraction from a SaaS platform — to populate a training dataset, to build a knowledge index, to migrate data to an AI-ready format — the agreement may not straightforwardly permit this. Data export limitations, API rate limits, and format restrictions may be in the contract in ways that constrain what the AI program needs. The practical issue: discovering mid-implementation that a bulk data extraction is contractually restricted by an existing SaaS agreement is a problem that takes time to resolve. Vendor negotiations to expand export rights, technical workarounds, or alternative data sourcing each add delay and cost that was not in the original plan. Review SaaS agreements for any data that the AI program will need to process, extract, or migrate before the implementation schedule is set. Existing AI vendor agreements and scope creep Organizations that have been using AI tools for some period often have existing vendor agreements that define the scope of permitted use cases. As the AI program expands, new use cases may not be covered under the existing agreement. This matters specifically in two ways. First, using an AI vendor for use cases outside the defined scope of the agreement — even with the same tool and the same vendor — may create data handling situations the original agreement did not contemplate. Second, enterprise AI agreements often include pricing that is tied to defined use parameters. Expanding use significantly beyond those parameters may trigger renegotiation on terms less favorable than the original agreement. Audit the scope of existing AI vendor agreements against the planned AI program before expansion. Know what is and is not covered before the program is designed around assumptions about what the vendor relationship permits. Data processing agreements for third-party integrations AI programs frequently involve connecting internal data to third-party AI systems through integrations: an AI tool connected to the CRM, an AI analytics layer over the data warehouse, an AI agent with access to internal APIs. Each of these integrations creates a data flow that may require a data processing agreement. Where the integrated party processes personal data on behalf of the organization, a data processing agreement is a regulatory requirement under data protection law in most jurisdictions. For integrations that existed before the AI layer was added, the original data processing agreement may not cover the additional processing the AI component involves. Before adding AI components to existing integrations, review whether existing data processing agreements need to be updated to reflect the expanded processing scope. The integration may be technically unchanged from the data flow perspective while creating a materially different processing activity for regulatory purposes. Third-party data in AI training and indexes Organizations often use third-party data sources — market data, industry benchmarks, licensed research, external databases — in their operations. When an AI program wants to use this data as training material or as content in a retrieval index, the license for that third-party data may not permit it. Third-party data licenses typically specify permitted use cases: internal analysis, reporting, specific product use. Training an AI model on licensed data, or including it in an AI knowledge base that generates outputs for a broad user population, may constitute a use case that the license does not cover. The risk is real, the issue is common, and the discovery process for finding all the affected data sources takes time. Conduct a data sourcing review for any data that will go into AI training sets or retrieval indexes. Identify all third-party licensed content, review the license terms for AI use restrictions, and either obtain the necessary permissions or exclude the content. Practical approach for the CIO and general counsel The scale of this review problem varies significantly by organization. For most enterprises, the contract review is manageable if it is structured and approached systematically before the AI program enters delivery. Start with the data sources the AI program will use. Map every system, database, and data feed the AI system will connect to. For each, identify the governing contract and whether it has been reviewed for AI-relevant terms. Prioritize by data volume and sensitivity. The highest-volume data sources feeding the AI program, and the data categories with the most regulatory and contractual complexity, deserve the most thorough review. Engage vendors early where the position is unclear. Vendors generally prefer to resolve license ambiguity before it becomes a dispute. A proactive conversation about AI use cases typically produces better outcomes than a post-hoc assertion that something was permitted. What to take from thisSoftware licenses and SaaS agreements often contain restrictions on automated processing and data use that predate AI and may apply to AI use cases in ways neither party anticipated. Review them before deployment. Bulk data extraction requirements for AI programs may be restricted by existing SaaS agreements. Discover this before the implementation schedule depends on it. Expanding AI use cases beyond the scope defined in existing AI vendor agreements can create data handling and pricing issues. Audit current agreements against planned use. Data processing agreements need to be updated to reflect AI components added to existing integrations. The regulatory obligation does not adjust automatically when the technical architecture changes. Third-party licensed data in AI training sets or retrieval indexes may require explicit permission that the existing license does not provide. Conduct a data sourcing review before building the training set.

Read full article
What Happens to Your Data Inside a Large Language Model

What Happens to Your Data Inside a Large Language Model

One of the questions I get most often from executive teams when they start getting serious about AI governance is some version of: "If we send data to an AI model, does that data end up in the model? Can the model then use our data to answer questions for our competitors?" It is a reasonable question. The answer is more nuanced than the headlines around AI and data privacy usually suggest, and getting the nuance right matters for making sound decisions about vendor selection, data handling, and acceptable use. This is not a technical explanation. It is an executive one. I want to give you the conceptual framework that lets you ask the right questions and evaluate the answers vendors give you. The key distinction: training versus inference There are two fundamentally different things that can happen to data when it touches an AI model. Inference is what happens during normal use. You send a prompt. The model processes it using the knowledge and patterns it already has. It generates a response. Your data was processed, but it did not change the model. The model is no more or less capable after your interaction than it was before. Think of it like asking an expert a question: they used their knowledge to answer you, but they did not become a different expert because you asked. Training is different. Training is when data is used to update the model's internal parameters — to change what the model knows or how it responds. This is what actually shapes the model's behavior and capabilities. Training happens periodically, using large datasets, through a deliberate process. It is not what happens every time a user sends a prompt. The confusion between training and inference is responsible for most of the anxiety executives have about sending data to AI vendors. When an employee pastes a strategy document into an AI assistant, that document is used for inference — to generate the response. It is not, in that moment, training the model or making the model more likely to surface that information to other users. The question of whether your data is used for training is a separate one, governed by the vendor's policies and your agreement with them. When data does influence the model The concern about data "ending up in the model" is legitimate in one specific scenario: when the vendor uses interaction data to train future versions of the model. This practice is more common in consumer products than enterprise ones. Many consumer AI tools, under default settings, retain interaction data and may use it as part of the training pipeline for future model versions. This does not mean a competitor can directly query the model and retrieve your document. Training does not work like storing files in a searchable database. But your data, if used for training, has influenced the model's patterns in ways that are effectively irreversible and non-auditable. Enterprise agreements typically exclude this. When an organization purchases an enterprise license with a proper data processing agreement, the vendor generally commits to not using that organization's data for training purposes. This is one of the most important terms to verify in any AI vendor agreement, and one of the strongest reasons to ensure employees are using enterprise tiers rather than consumer accounts. The practical implication: the risk of your data influencing the model is primarily a function of which tier you are on and what your agreement says — not of using AI tools in general. What retention actually means Even when a vendor does not train on your data, they may retain it for a period. Understanding what retention means in practice matters for two reasons: regulatory compliance and the question of who can access the retained data. Vendors retain interaction data for different reasons: abuse prevention, conversation history for the user, debugging and quality assurance, and in some cases legal holds. The retention period varies from days to years depending on the product and the settings. What the retained data can be used for is defined in the vendor's privacy policy and data processing agreement. The key questions are: Can vendor employees access the content of retained interactions? Under what circumstances? Are there audit logs of such access? What are the deletion terms — can you request deletion, and is it complete? These are not abstract questions. An employee sending sensitive content to an AI tool is creating a record that exists in the vendor's infrastructure for some period. If that infrastructure is breached, or if the vendor is subject to legal process, that record is potentially accessible. The same employee would not dream of emailing that content to a stranger. But the AI tool does not feel like an external party — it feels like a private tool. The retention question is also where GDPR and similar regulations create specific obligations. Any interaction containing personal data is a transfer of personal data to a third-party processor. That transfer requires a legal basis, a data processing agreement, and compliance with data subject rights including deletion. Most organizations have not mapped their AI tool usage against these obligations. The questions a CTO should ask every AI vendor The framework above translates into a specific set of questions that should be part of any AI vendor evaluation: Is interaction data used for training future models? Under what conditions? What controls does the customer have over this? This is the most important question. Get the answer in writing, as a contractual commitment, not as a verbal assurance. What is the data retention period for interaction data? Can this be configured? What are the deletion rights and processes? What confirmation is provided when deletion is complete? Who within the vendor organization can access the content of customer interactions? Under what circumstances? Are there access logs? What are the procedures if vendor employees need to access content for support or debugging? Where is the data processed? This matters for regulatory compliance. Data about EU residents processed in jurisdictions without an adequacy decision creates specific compliance obligations that need to be managed. What happens to retained data in the event of the vendor being acquired, going out of business, or being subject to legal process? Where does customer data fall in those scenarios? What is the vendor's certification posture? SOC 2 Type II, ISO 27001, and similar certifications do not answer all of these questions, but they provide a baseline for security practices that matters for any serious enterprise evaluation. The honest assessment No AI tool is risk-free from a data perspective. Sending data to any third-party system involves some degree of information leaving your infrastructure, under terms you did not write, in systems you do not control. That is true of cloud storage, email services, and every other third-party tool the organization uses. The question is whether the risk is understood, whether the terms are acceptable given the regulatory and contractual context, and whether the data classification of what is being sent is appropriate for the tier and agreement in place. The worst outcome is not using AI tools with enterprise data under a proper enterprise agreement with a reputable vendor. The worst outcome is using consumer-tier products with default settings, with sensitive data, without any of the contractual protections that make enterprise use manageable. Most organizations are currently somewhere in between. The CTO's job is to understand exactly where on that spectrum the organization sits, and to move deliberately toward the part of the spectrum that is defensible. What to take from thisTraining and inference are different. Using an AI tool to process data does not automatically mean that data trains the model. Whether it does depends on the vendor's policies and your agreement. The training exclusion is one of the most important terms in an enterprise AI agreement. Verify it explicitly — a verbal assurance is not sufficient. Retention means your data exists on vendor infrastructure for some period. Understand the retention period, access controls, and deletion rights for every tool in active use. Consumer and enterprise tiers of the same product often have materially different data handling terms. The tier distinction matters more than the vendor selection in many cases. Map AI tool usage against data protection obligations before the next regulatory review, not during it.The executives who handle this well are the ones who moved past the surface-level anxiety about "AI knowing your data" and got specific about the mechanisms: what are the actual terms, what does retention mean, and what commitments can the vendor make in writing?

Read full article
The Knowledge That Walks Out the Door When Employees Use AI on Client Work

The Knowledge That Walks Out the Door When Employees Use AI on Client Work

Professional services firms — consulting, legal, accounting, advisory — have a specific relationship with client data that is different from most enterprise AI contexts. The data they handle belongs to their clients. The confidentiality obligations around it are contractual and, in many cases, professional and regulatory. The consequences of a breach are not limited to regulatory exposure; they extend to client trust, which is the fundamental asset in any advisory relationship. AI tools are now deeply embedded in how professional services work gets done. Analysts use them to accelerate research. Consultants use them to draft documents. Lawyers use them to review contracts. The productivity benefits are real and the competitive pressure to use them is significant. The risk accumulation that comes with that use is largely unaddressed. What client data actually flows through AI tools in professional services The volume and sensitivity of client data flowing through AI tools in professional services contexts tends to be higher than in most enterprise settings, because the work itself involves processing and analyzing client-owned information. Project research and analysis. Analysts feed client financial data, market analysis, and competitive benchmarks into AI tools to accelerate synthesis. The client's internal data, which the firm has received under a confidentiality agreement, enters a third-party AI system. Document drafting. Consultants use AI writing assistants to draft recommendations, presentations, and reports. The source material that informs the drafting — interview outputs, internal data, strategic context — is included as context for the tool. Contract review and legal analysis. Legal and advisory professionals use AI tools to review and summarize contracts, due diligence materials, and transaction documents. These materials contain some of the most sensitive information clients possess. Meeting summaries and communication assistance. Client meeting recordings processed through meeting AI tools. Client correspondence drafted with AI assistance. Internal discussions about client situations entered as context for AI queries. Each of these flows involves client-confidential data entering a third-party AI system. Most firms have not mapped this systematically. Many assume it falls under the general confidentiality terms in their client agreements without having verified that the AI tool's data processing terms are compatible with those obligations. The contractual gap that most firms have not closed Professional services firms operate under engagement letters and master services agreements that include confidentiality provisions. These provisions were written before AI tools existed in their current form. They typically cover how the firm handles client confidential information: where it is stored, who has access, what the firm's obligations are around disclosure. What they almost never address: whether the firm can process client confidential information using third-party AI tools, and if so under what conditions. This creates a gap. The firm has agreed to keep client information confidential. The firm's employees are feeding that information to third-party AI systems. Whether that constitutes a breach of the confidentiality provisions depends on the specific language and how it would be interpreted, which is not a comfortable analysis to be doing reactively. Some clients are now asking about this proactively in RFPs and at the start of engagements. Firms that have a clear, honest answer to the question "do your employees use AI tools when working on our engagement, and if so how is our data handled" are in a better position than those who have not worked out an answer. The knowledge residue problem There is a second dimension to this risk that is less obvious than the direct data exposure question. When an employee works with client information through an AI tool over the course of an engagement, the contextual knowledge they develop about the client's situation is richer and more detailed than it would be if they had processed the same information manually. The AI tool allows them to work across more data, make more connections, and develop a more comprehensive understanding than time would have permitted through manual analysis. This enriched understanding lives in the employee's head when they walk out the door. When that employee moves to a competitor or, in certain conflict situations, works on a client in a similar competitive situation, the depth of knowledge they carry creates an exposure that goes beyond the normal knowledge transfer risk. The firm cannot fully control what employees internalize through their work. That has always been true. AI tools increase the depth and breadth of what an employee can internalize over a fixed period of time. The risk management implications are subtle but real. What governance looks like in practice The minimum governance framework for a professional services firm using AI tools on client work: An explicit AI use policy that covers client work. This should specify which AI tools are approved for use on client matters, what categories of client data can be processed through AI tools under what conditions, and what the data handling terms are for approved tools. This is different from the general employee AI policy — it needs to address the confidentiality obligations that are specific to client engagements. Client engagement agreement updates. The confidentiality provisions in engagement letter and master services agreement templates need to be updated to address AI tool use. At minimum, the provisions should not preclude AI use in ways that are inconsistent with how work is actually being delivered. Better than that: the provisions should address AI tool use explicitly, with appropriate confidentiality protections around how client data is handled within those tools. Client disclosure for high-sensitivity matters. For engagements involving particularly sensitive information — M&A transactions, regulatory matters, litigation, restructuring — the engagement team should have a protocol for discussing AI tool use with the client and obtaining explicit confirmation about what is acceptable. Employee education that is specific to client work. General AI use training does not address the confidentiality implications specific to professional services. Employees handling client confidential information need to understand what AI tools they can use, with what data, under what terms, and what their obligations are when in doubt. The question clients are starting to ask The most direct signal that this needs to be addressed now: clients are beginning to ask about it. Not often, but the frequency is increasing, and the questions are getting more specific. "Does your team use AI tools when working on our matters?" "If so, does our confidential information enter those AI systems?" "What are the data handling terms for the AI tools you use, and how do they interact with your confidentiality obligations to us?" A firm that has thought about these questions and has clear answers is in a different position from one that has to formulate an answer under client scrutiny. The latter tends to produce either an evasive answer that damages trust or a defensive answer that raises more questions than it resolves. What to take from thisMap what client data is flowing through AI tools on active engagements. The volume is almost certainly higher than any single partner or manager would estimate. Review whether existing client confidentiality provisions are consistent with how AI tools are actually being used in client delivery. The gap is likely to be meaningful. Update engagement agreement templates to address AI tool use explicitly, before clients start asking for it in contract negotiations. Develop a protocol for client disclosure on high-sensitivity matters. The default should be proactive transparency, not reactive disclosure. Train client-facing staff specifically on the confidentiality implications of AI tool use in their context. Generic AI training is not sufficient for professional services.The firms that handle this well are not necessarily the most cautious ones. They are the ones that have been honest about how AI is being used in client delivery, have updated their agreements to reflect that, and can answer client questions about it clearly and without hesitation.

Read full article
How to Classify Your Data Before Your AI Program Does It for You

How to Classify Your Data Before Your AI Program Does It for You

Data classification is one of those governance practices that most organizations have in some form and almost none have in a form that is adequate for AI. The gap matters because AI deployment without a working classification framework creates a specific category of problem: the system treats all accessible data as equivalent input, and the outputs reflect that indiscriminateness in ways that are difficult to predict and costly to remediate after the fact. The CIO who gets this right before the AI program starts is in a very different position from the one who inherits a classification gap when the first incident surfaces. Here is what a practical classification approach looks like when AI deployment is the specific forcing function. Why existing classification frameworks usually fall short Most organizations have some form of data classification. The typical structure is a four-level hierarchy: public, internal, confidential, and restricted. Documents get tagged — or are supposed to get tagged — at one of these levels. Access controls are set accordingly. This framework was designed for a world where humans navigate information deliberately. You look for a document, you find it, you read it. The sensitivity of what you see is a function of where you went to look. AI tools do not navigate information that way. They can process everything they have access to simultaneously, surface connections between data sources that were never designed to be combined, and produce outputs that reflect the aggregate of what they have seen rather than any single document. The sensitivity classification of individual documents does not translate cleanly into the sensitivity of an AI system's outputs. There are three specific failure modes I see in organizations that try to apply existing classification frameworks to AI deployment. Permission-level accuracy. Existing classification may reflect the intention of who should access what, but actual permissions often diverge from the classification framework over time. Documents move between folders. Projects end and access is not revoked. Distribution lists grow and are not pruned. When an AI system is given access to everything a user can access, it inherits this divergence between intended and actual permissions. Output sensitivity. A document classified as "internal" might, in combination with five other documents also classified as "internal," produce an AI output that reveals information that would have been classified "confidential" if anyone had written it down directly. The classification framework addresses individual document sensitivity but not the sensitivity of AI-generated synthesis. Dynamic content. AI systems that connect to live data sources — CRMs, financial systems, email archives — encounter content that has never been classified at all, because classification was designed for documents rather than data records. Building a classification framework for AI specifically A classification framework that works for AI deployment needs to answer three questions that the standard framework typically does not. What can this data type be used for in AI context? Rather than a single sensitivity level, each data category needs a set of permitted AI use cases. Client financial data might be appropriate for internal analytics AI but not for a tool that produces externally shared outputs. Personal data might be appropriate for a tool with data processing agreement coverage but not for one without it. The permitted use case dimension is specific to AI and does not exist in traditional classification frameworks. What combinations create elevated sensitivity? Certain combinations of data categories produce outputs that are more sensitive than any individual category. A practical classification framework for AI should identify the high-risk combinations and set explicit controls around AI systems that can access both. What is the real-time classification status? For live data sources, the classification question is not just "what is this data type" but "what is the current state of this specific record, and does that affect what AI can do with it." A client record that includes active litigation flags, for example, may need to be treated differently than a standard client record even if the data type is classified the same way. The practical approach Doing this well does not require a multi-year data governance program. It requires a focused exercise tied directly to the AI deployment timeline. Here is what that looks like. Start with the AI system's data access scope. Before classifying anything, define what data sources the AI system will be connected to. The classification exercise is scoped to those sources. Everything else can wait. Map the sensitive data categories within scope. For each data source the AI will access, identify what sensitive categories exist: personal data, commercially sensitive data, legally privileged material, client confidential data, regulated financial data. This is an inventory exercise, and it usually reveals data in places people did not expect it. Define permitted use cases for each category. For each sensitive category, specify what the AI system is and is not permitted to do with it. This becomes the basis for technical controls — what data the system can retrieve, what it can include in outputs, and what it should exclude or flag. Build the combination rules. Identify the high-risk combinations and set rules for how the AI system handles them. This is the hardest part and the one most often skipped. Spending a day on this with the CIO, the data protection officer, and the AI system owner is worth it. Implement classification tags as technical controls. The classification decisions need to be expressed as technical constraints that the AI system respects, not just as policy documentation. A policy that says "the AI should not include client financial data in externally visible outputs" is unenforceable unless the system is technically configured to prevent it. The CIO's role in making this work Data classification for AI is not a project the technical team can own independently. The decisions about which data categories can be used for which AI purposes require input from legal, compliance, and the business functions that own the data. The CIO's role is to convene those conversations and drive them to decisions before the AI system goes live, not after. The alternative — deploying the AI system and addressing classification issues as they surface — is more expensive and more disruptive. When an AI system produces an output that reveals information it should not have had access to, the response involves technical remediation, incident investigation, potential regulatory notification, and organizational credibility damage. All of which are harder than running the classification exercise before deployment. The time required for a focused data classification exercise scoped to a specific AI deployment is typically two to four weeks for a system with well-defined data access scope. That is a reasonable investment given the alternative. What to take from thisExisting data classification frameworks were designed for human navigation of information. They do not translate directly to AI access, which aggregates and synthesizes rather than navigates. Classification for AI needs to address permitted use cases, high-risk combinations, and live data — three dimensions that standard frameworks typically do not cover. Scope the classification exercise to the AI system's data access, not the organization's entire data estate. A focused exercise is achievable in weeks; an organization-wide program is not. Classification decisions need to be expressed as technical constraints, not just policy documentation. A policy without technical enforcement is not a control. The CIO needs to convene legal, compliance, and business data owners in the classification exercise. The decisions require input from all of them, and making them without that input produces gaps.The organizations that get AI deployment right are not the ones with the most comprehensive data governance programs. They are the ones that did the focused, practical work of understanding their data before they connected it to a model, and made deliberate decisions about what that meant for acceptable use.

Read full article
Training AI on Proprietary Data: What You Gain and What You Give Up

Training AI on Proprietary Data: What You Gain and What You Give Up

At some point in most enterprise AI programs, someone proposes using the organization's own data to improve the model. The pitch is compelling: a model trained on your internal documentation, your historical decisions, your domain-specific knowledge will perform better on your actual work than a general-purpose model that knows nothing about your context. The pitch is not wrong. Done well, training or fine-tuning a model on proprietary data can produce a genuinely better system. The problem is that "done well" requires a set of decisions that most organizations have not thought through, and the downside of doing it carelessly — from data exposure, from quality degradation, from regulatory exposure — is substantial. This is not an argument against using proprietary data to improve AI systems. It is an argument for approaching that decision with the same rigor you would apply to any decision about where your most valuable data goes. What training on proprietary data actually involves There are a few different mechanisms by which proprietary data can improve model performance. The distinctions matter for understanding the exposure. Retrieval-augmented generation (RAG). The model is not trained on proprietary data at all. Instead, a retrieval system fetches relevant internal documents at query time and provides them as context for the model to work from. The proprietary data sits in a controlled index that the organization manages. The model itself stays unchanged. This approach avoids most of the training data risks while providing significant performance improvement on domain-specific tasks. Fine-tuning. The model's internal weights are adjusted using proprietary data. The model itself changes — it becomes better at tasks that reflect the patterns in the training data. The proprietary data has, in a meaningful sense, been absorbed into the model. This is more powerful than RAG for certain tasks, and considerably more complex from a data governance standpoint. Full training from scratch. Building a model entirely on proprietary data, without starting from a pre-trained foundation. This is rare at the enterprise level — the compute and data requirements are significant — but it does happen for organizations with specialized domain requirements and the resources to invest. The data exposure implications are very different across these approaches. RAG keeps proprietary data in an index the organization controls. Fine-tuning moves it into the model weights. The latter is where the tricky questions live. What gets encoded in a fine-tuned model When data is used to fine-tune a model, the specific documents and content do not get stored inside the model as retrievable files. The model cannot be queried to reproduce its training data verbatim — in most cases. But the training data has shaped the model's behavior in ways that are difficult to audit or reverse. This matters in two respects. First, sensitive information can influence the model's outputs in ways that are hard to anticipate. A model fine-tuned on internal strategy documents may, when asked questions that touch on those topics, produce outputs that reflect internal strategic positions without anyone intending to reveal them. The model is not leaking documents — it is producing outputs shaped by the patterns in documents it has seen. The effect is subtle and hard to trace. Second, fine-tuned models can, under certain adversarial conditions, be prompted to surface information that reflects their training data more directly than normal. This is not a theoretical risk — it is an active research area, and the techniques for eliciting training data from fine-tuned models are improving. Organizations that fine-tune on genuinely sensitive data on models hosted by third parties should factor this risk into their assessment. The vendor relationship in fine-tuning When an organization fine-tunes a model that is hosted by a cloud AI vendor, several questions about the proprietary data need to be answered clearly before proceeding. Does the vendor use the fine-tuning data to improve their base models? Most enterprise agreements exclude this, but it should be a named contractual commitment. Who can access the fine-tuning data? During the fine-tuning process, the data is processed on the vendor's infrastructure. Access controls, the circumstances under which vendor employees can access training content, and the security measures around the fine-tuning pipeline should all be part of the assessment. What happens to the fine-tuning data after training is complete? Is it retained, for how long, and can it be deleted? What does deletion mean in the context of a model that has already been trained on it — a question vendors are not always able to answer cleanly, because once data has influenced model weights, the effect cannot be fully reversed. Where is the fine-tuned model stored, and who controls it? The fine-tuned model is organizational IP. The vendor relationship needs to be clear about ownership, portability, and what happens to the fine-tuned model if the organization ends the vendor relationship. Data quality is the amplification lever One risk that gets less attention than the security questions: the quality and composition of the training data determine whether fine-tuning improves or degrades model performance. Organizations that rush fine-tuning treat it as an input optimization problem — more data is better. That is incorrect. Noisy, inconsistent, or poorly representative training data produces a fine-tuned model that performs worse than the base model on the tasks that matter, and sometimes produces dangerous outputs in domains where the training data was biased or incomplete. I have seen organizations fine-tune models on documentation that contained outdated processes, contradictory guidance, and examples of known-bad decisions that were documented as cautionary tales rather than as positive examples. The resulting model incorporated those patterns along with the useful ones. The outputs were confidently wrong in ways that were hard to trace back to the training data quality issue. Before committing to fine-tuning, the data curation question deserves as much attention as the technical process. What data will be included, what will be excluded, who reviews the training set for quality and appropriateness, and how will the fine-tuned model's behavior be evaluated against the base model are all part of a credible fine-tuning program. When it is worth it Fine-tuning on proprietary data is worth the complexity when two conditions are both true: the performance improvement on the target task is significant and demonstrable, and the data exposure risks can be managed through a combination of vendor agreement terms, data classification, and governance. RAG should be the default approach for most enterprise use cases. It provides substantial performance improvement without the fine-tuning data exposure risk, and it is operationally simpler to maintain and update. Fine-tuning becomes worth the investment when the task requires deep pattern recognition across the organization's specific domain that a retrieval approach cannot fully replicate. The decision to fine-tune should not be made by the AI team alone. It needs the CTO's visibility into the data exposure implications, legal review of the vendor terms around training data, and a clear data classification decision about which content is and is not appropriate as training material. What to take from thisRAG and fine-tuning are different. RAG keeps proprietary data in an index the organization controls. Fine-tuning moves patterns from that data into model weights. They have different data exposure profiles. A model fine-tuned on sensitive data can surface that information in ways that are hard to predict or audit. This is not a reason to avoid fine-tuning, but it is a reason to be deliberate about what goes into the training set. When fine-tuning on a third-party hosted model, get contractual clarity on training data use, retention, and what happens to the data and model at contract end. Data quality in the training set is as important as security. Noisy, inconsistent, or outdated data produces a fine-tuned model that may perform worse than the base model. Make fine-tuning a CTO-level decision with legal review, not a technical team default. The data exposure implications require organizational sign-off, not just technical execution.

Read full article
Defining AI Success Before Anyone Commits a Budget

Defining AI Success Before Anyone Commits a Budget

I've sat in enough AI program kick-off meetings to have stopped being surprised by this: twenty people in a room, a board mandate to "move fast on AI," a vendor selected, a team forming — and nobody has yet agreed on what success looks like. Not in specific terms. Not in a way that would allow someone to come back in twelve months and determine whether the program worked. This isn't unusual. It's the norm. And it's the single failure mode I see most consistently across enterprise AI programs, regardless of industry, company size, or technical maturity. The reasons are understandable. There's pressure to start. Defining success precisely feels constraining when the technology is new and the possibilities feel open. Executives are comfortable with directional goals — "improve customer experience," "reduce operational cost," "increase fraud detection" — and less comfortable with the kind of specificity that creates a clear accountability line. Specificity means someone owns the number. Owning the number means someone can be wrong. The cost of this discomfort is high. Why technical metrics aren't the answer The first place program teams turn when asked to define success is model performance metrics. Accuracy, F1 score, AUC, precision-recall tradeoffs. These are measurable, they're familiar, and they can be calculated before the model is in production. They're also not what the business cares about. A model with a 94% accuracy rate that nobody uses isn't a success. A model with 87% accuracy that improves a critical business process by a measurable amount is. The gap between technical performance and business outcome is where most AI programs lose the narrative — and where the CFO eventually starts asking whether the investment was worth it. Technical metrics are necessary for model development and monitoring. They're not sufficient for program success definition. What the business cares about is downstream: decisions improved, costs reduced, revenue generated, risk reduced. Those are different measurements, and the relationship between model performance and business outcome needs to be made explicit at the start, not assumed. What a complete success definition contains A success definition that can be used to evaluate the program twelve months from now has four components. A specific metric. Not "improve fraud detection" — a number the business can measure. False negative rate, dollar value of fraud losses prevented, percentage of fraud alerts requiring manual review. The metric needs to be something that exists in a system the business actually maintains, not something that has to be constructed after the fact. A documented baseline. What is the current value of that metric, measured against the same methodology that will be used to measure the AI-driven result? Without a baseline, you can't measure improvement. Without a consistent measurement methodology, comparisons are arbitrary. Getting the baseline documented before the program starts is more important than it sounds — it eliminates a whole category of disagreement about whether the program worked. A numeric target. A direction is not a target. "Improve" is not a target. "Reduce false negative rate from 4.2% to below 2.5%" is a target. The target should be challenging enough to justify the investment and specific enough to be unambiguous. A timeframe. By when? The timeframe determines the evaluation rhythm and gives the program team a real deadline against which to calibrate pace. Without it, the target floats indefinitely. Without all four components, the success definition isn't complete. It's a directional goal dressed up as a commitment. The traps Several common patterns make success definitions look complete when they aren't. Vanity metrics are things the program team can control that don't connect to outcomes the business cares about — models built, data sources integrated, team size, features shipped. These are useful operational metrics. They're not success metrics. A program that reports these as evidence of success has redefined success to be about activity rather than outcome. Unmeasurable outcomes are aspirations that can't be tracked. "Become an AI-native organization." "Embed AI into our culture." These may represent genuine long-term goals. They cannot be evaluated in a twelve-month program review, and including them as success criteria gives the program team permanent cover against accountability. Metrics the business can't track are a trap that sounds technical but isn't. If measuring the success metric requires access to data the business doesn't actually maintain, or calculations the business doesn't currently run, the metric will be reported inconsistently or not at all. Success metrics need to be things the business can measure on a monthly or quarterly basis with existing data infrastructure. Proxy metrics that drift from the target outcome are the hardest to catch. An AI program for customer service might measure success by handle time reduction. But if the model reduces handle time by routing calls to hold queues rather than resolving queries, the metric looks good while the outcome is bad. The connection between the proxy and the real outcome needs to be validated, not assumed. Running the conversation with executives who prefer ambiguity The executives most resistant to specific success definitions are usually the ones with the most at stake. Specificity creates accountability, and accountability creates risk. Understanding that the resistance is rational rather than evasive changes how you approach the conversation. The approach that tends to work: starting not from "what does success look like" but from "what would change your mind." Ask the sponsor what result, at what point in time, would cause them to question whether the program was working. Ask the CFO what the program would need to show at month twelve to be considered a good investment. Ask the business unit head what their team would need to see to start relying on the model's outputs. These questions come at the success definition from the outside — from what a skeptic would need to see to be convinced — rather than from the inside, where optimism tends to inflate targets and round off the hard edges. They also surface the implicit assumptions about what success looks like that are already in the room, undiscussed. The success definition document that comes out of this conversation doesn't need to be complex. It needs to be signed — literally. An agreed definition, documented and acknowledged by the program sponsor, the business lead, and the CFO or their representative. The act of signing matters because it makes the definition a commitment rather than a suggestion. What happens when you skip this step Programs without defined success criteria don't fail suddenly. They drift. Twelve months in, there's a review. The program has made progress — models are built, pilots are running, the team has learned a lot. Nobody can agree on whether the program is working because nobody agreed at the start on what "working" would mean. The sponsor points to the positive signals. The CFO points to the cost. The business unit says the outputs aren't quite what they needed. The program continues — not because it's succeeding, but because it's not clearly failing. Two years in, the program has consumed significant investment with ambiguous returns. The next budget cycle is where it gets cut — not in a formal review, but quietly, when the sponsor doesn't go to bat for it. The program closes with a retrospective report that describes what was learned rather than what was delivered. That's not a technical failure. It's a governance failure that started on day one, and it was entirely preventable.

Read full article