Showing Posts From
Ai accountability boardroom
- 19 Dec, 2025
The Liability Question Nobody Is Answering in the Boardroom
Every consequential AI deployment eventually produces a wrong decision. A loan gets denied that should have been approved. A job candidate is filtered out by a biased model. A medical summary contains an error that affects clinical judgment. A fraud detection system flags a legitimate transaction and freezes a customer's account. None of these are failures in the sense of the system breaking down. The system functioned as designed. It produced an output. The output was wrong, and the output had consequences. The question that follows — who is accountable for that wrong decision — is one that most boards and executive teams have not answered before the deployment, and have to scramble to answer after. That sequence is backward. Accountability for AI-driven decisions is something that needs to be designed in, not litigated into existence. And the design work is a governance decision, not a technical one. The legal gap AI systems occupy a strange position in the existing liability framework. They make decisions — consequential ones, at scale — but they aren't legal entities. They can't be sued. They don't have directors or officers. When something goes wrong, liability has to land somewhere in the human and organizational structure around the system. The current legal landscape draws from several established frameworks, none of which maps cleanly onto AI: Product liability — the manufacturer of a defective product is liable for harm it causes. AI systems can be analogized to products, and there's active litigation testing this framework in multiple jurisdictions. The challenge is that AI systems produce outputs that vary based on input, and the boundary between a "defect" and an "intended behavior that produced an unintended outcome" is genuinely ambiguous. Professional liability — practitioners in regulated professions (doctors, lawyers, financial advisors) are liable for negligent advice. AI systems are increasingly used to support or substitute for that professional judgment. The question of whether the liability stays with the professional when AI was involved in the recommendation — and how the professional's duty of care applies to AI-assisted decisions — is being litigated and regulated in parallel. Negligence — failing to exercise reasonable care in deploying or monitoring an AI system that then causes harm. This framework is probably the most applicable to enterprise AI in the near term, and it places significant weight on what processes the deploying organization had in place. Sector-specific regulation — in financial services, healthcare, employment, and other regulated industries, AI decisions may trigger specific regulatory liability that operates independently of common law. The EU AI Liability Directive, which is moving through the European legislative process, aims to make it easier for individuals harmed by AI to access compensation — including through a presumption of causality that shifts the burden of proof in certain cases. The direction of travel in regulation is toward easier liability attribution, not harder. Three scenarios, one question Consider three concrete scenarios and what the liability analysis looks like for each under current frameworks. A denied loan. A bank uses an AI model for credit decisions. A customer is denied a mortgage. The customer believes the denial was based on factors that correlate with protected characteristics. Under ECOA and the EU AI Act's high-risk classification for credit AI, the bank has obligations to explain the decision and to demonstrate that the model doesn't produce discriminatory outcomes. Liability sits with the bank as the deploying organization. The model vendor's liability is limited unless the bank can show the vendor misrepresented the model's properties or failed to disclose known issues. A flawed medical summary. A hospital uses an AI-generated clinical summary tool. A physician relies on a summary that omits a critical finding. The patient experiences harm as a result. Liability analysis here is complex: the physician has a professional duty of care that doesn't disappear because AI was involved; the hospital may have organizational liability for deploying a system without adequate oversight; the AI vendor may have product liability exposure if the system was marketed as clinically reliable. The physician-patient relationship doesn't dissolve because there's an AI in the loop. A biased hiring filter. A company uses an AI screening tool that disadvantages candidates from certain demographic groups. Multiple candidates who should have advanced are screened out. Under employment discrimination law, the company is liable for discriminatory outcomes regardless of whether the discrimination was intentional or was produced by an AI system. "We didn't know the model was biased" is not a defense that has worked in US employment discrimination cases, and the EU AI Act's high-risk classification for hiring AI creates additional conformity assessment obligations. In all three cases, the deploying organization carries significant liability. In none of the three cases does "the AI did it" operate as a meaningful defense. Why "the vendor is responsible" doesn't hold The most common liability assumption I encounter in enterprise boardrooms is that the AI vendor carries the primary liability for wrong decisions. This assumption is wrong in most deployment contexts, and acting on it creates governance gaps that become expensive. Vendor contracts are typically written to limit liability, often to the contract value or a multiple of it. This is standard commercial practice and not specific to AI, but the magnitude of AI-driven decisions can exceed contract liability caps by orders of magnitude. A vendor that is liable up to the annual contract value for a model used in credit decisions that affect thousands of customers has not transferred the real economic exposure. More fundamentally, courts and regulators in most jurisdictions have been clear that the organization deploying the AI system — not the vendor supplying the model — is responsible for the decisions made using it. The deploying organization chose to use the system, chose how to integrate it, chose what human oversight to maintain, and chose to act on its outputs. Those are deployment decisions, and they carry accountability. There are narrow scenarios where vendor liability is real and significant: if the vendor misrepresented the system's capabilities, if the vendor knew of material deficiencies and didn't disclose them, or if the system had a defect that was not discoverable through reasonable testing. These are exceptions, not the general rule. What good organizational liability design looks like Liability for AI decisions can't be eliminated, but it can be managed through organizational design that makes the accountability chain clear, auditable, and defensible. Named accountability for each production AI system. Someone in the organization should be designated as accountable for each AI system's decisions and performance. Not a team — a named individual in a named role. This person is responsible for monitoring performance, escalating anomalies, and making the decision to pause or decommission if performance degrades. Their accountability should be documented, and the documentation should be accessible if a decision is ever challenged. Human-in-the-loop requirements for consequential decisions. For decisions with significant impact on individuals — credit, employment, medical — maintaining a human review stage creates both a check on AI outputs and a clearer accountability structure. The human reviewer is accountable for the final decision in a way that's legally cleaner than pure AI automation. This doesn't require reviewing every decision — it requires designing the process so that consequential decisions involve human judgment at the point of decision. Audit trails for individual decisions. The ability to reconstruct a specific decision — what inputs the model received, what output it produced, what threshold it applied, what human review occurred — is essential for responding to challenges. This is an engineering requirement that has to be designed in before deployment, not retrofitted after a complaint. Documentation of known limitations. AI systems have known failure modes. Documenting these honestly — and documenting what controls are in place for each — creates a record of due diligence that is relevant in negligence analysis. An organization that knew a model performed poorly on a specific demographic and deployed it anyway is in a very different position from one that wasn't aware of the limitation. The board's role The liability question is a board-level question because the accountability structure for AI decisions is a governance design decision, not a management decision. Management designs the systems. The board should satisfy itself that the design is adequate — that accountability is named, that audit trails exist, that human oversight requirements are defined, and that the organization has a clear answer to the question of who is responsible when an AI decision is wrong. This isn't a theoretical exercise. The litigation and regulatory enforcement that will define enterprise AI liability over the next decade is starting now, and the decisions organizations make today about how to structure accountability will determine whether their position in that litigation is defensible or not. The board that has approved an AI investment and cannot answer "who is accountable if this system produces a wrong decision at scale" has a governance gap. Closing it is not a technical task. It's a decision the board needs to make, and it needs to make it before the scenario that tests it.
Read full article