Showing Posts From

Ai consulting evaluation

Why Every Management Consultancy Now Has an AI Practice (And What That Means for You as a Client)

Why Every Management Consultancy Now Has an AI Practice (And What That Means for You as a Client)

Sometime in 2022 and 2023, every major strategy and management consulting firm released a version of the same announcement: a new AI practice, a significant investment in AI capability, a commitment to helping clients navigate the AI transition. The numbers varied — some firms claimed hundreds of AI specialists, others thousands — but the message was consistent. AI is the next big thing, and we are ready to lead you through it. This was not primarily a capability announcement. It was a competitive positioning announcement. The trigger was existential. If AI was going to reshape how organizations make decisions, operate processes, and build competitive advantage, then firms whose primary product is advice on those things faced a direct threat to their relevance. The AI practice wasn't built because the firms had developed deep AI capability. It was built because not having one was a risk to the business model. This distinction matters when you're on the receiving end of an AI advisory pitch. What actually happened Major consulting firms do not build capability quickly. They build narrative quickly. The firms that announced AI practices in 2022 and 2023 didn't have AI practices in the operational sense — they had strategy practices that could speak to AI, some technology practices with data and analytics capability, and a rapidly growing collection of AI-themed slide decks. The staffing reality in most AI practices, particularly in the first two to three years, was strategy consultants who had taken machine learning courses, technology consultants who had moved into AI positioning from adjacent areas, and a smaller number of genuine practitioners — people who had actually built and deployed AI systems in production — concentrated at senior levels where they were primarily used to credentialize proposals rather than do delivery work. This is not unique to AI. It's the standard consulting model for every new technology category: the firm establishes a practice, builds the marketing narrative, and races to build actual capability behind it before clients realize the gap. The firms that invested most heavily in genuine AI capability — in hiring practitioners with production experience, in building internal AI tools, in running their own AI programs — do exist. The quality gap between them and the firms running strategy work with AI rebranding is real, and it's visible if you know what to look for. What genuine AI capability looks like A consulting firm with genuine AI capability can show you production AI systems they've delivered — not demos, not prototypes, not internal tools. Systems in production at clients, operating at scale, monitored and maintained by the client after the engagement ended. They can tell you specifically what went wrong in delivery and what they learned from it. AI programs that have never failed in delivery either haven't been through delivery at the scale and complexity they're claiming, or they're not being honest about the history. The senior practitioners on the engagement — not the people who pitched it, the people who will be in the room — should have direct experience owning AI programs in production. That means having been accountable for model performance, retraining decisions, production incidents, and stakeholder relationships when results were mixed. They should have a clear view on where they add value and where you shouldn't hire them. A firm that claims to do everything in AI — strategy, delivery, engineering, operations — is almost certainly overstating capability in at least one of those areas. Genuine practitioners are typically clear-eyed about scope. The cover slide test Ask the firm to show you an AI strategy they developed for a comparable client. Redacted is fine. Then ask yourself: how different is this from a digital transformation strategy from five years ago? The AI strategy documents I've seen from repositioned strategy practices tend to have the same structure as every other technology transformation strategy: current state assessment, capability gap analysis, use case portfolio, operating model recommendations, investment roadmap. The AI content is in the use cases — different applications, different tools — but the strategic logic is identical to the framework the same firm was applying to cloud transformation or data analytics five years earlier. That's not necessarily wrong. Some strategic frameworks are durable. But it's diagnostic. If the firm's AI strategy is structurally indistinguishable from their prior digital strategy work, the AI expertise is probably in the examples rather than in the methodology. And examples without methodology give you a document, not a capability. The POC incentive problem Consulting firms have a structural incentive toward proof-of-concept work and away from production delivery — and it's worth understanding why, because it shapes what you're buying when you hire them for AI. A POC engagement is bounded, visible, and politically low-risk. The client sees a working model. The engagement team gets credit for a demonstration. The timeline is short enough to maintain senior partner attention. When it succeeds, it becomes a case study. When it fails, it's an experiment rather than a program failure. Production delivery is the opposite. It's longer, more expensive, politically messier, and the credit is diffuse. The consultants are one of several parties involved. Problems surface slowly. The partners have moved on to the next client by the time the model is in production, so the reputational upside is limited. This means a consulting firm left to its own advice will systematically recommend more POCs and more strategy work — and less production delivery — than is actually in the client's interest. Not from bad faith, but from incentive alignment that runs counter to the client's objective of getting AI into production. If you're hiring for AI, be explicit about what you're buying: strategy, delivery, or both. And if it's delivery, be specific about what "delivery" means — a deployed model in production, monitored and owned by your team after the engagement, not a handoff package that requires the same firm to operate. What to demand from an AI advisory engagement Before signing an AI advisory engagement, the questions worth asking: Who are the senior practitioners on this engagement — not the partners who pitched it, the people who will be working on it day to day — and what production AI systems have they specifically delivered? What does the engagement end state look like? At completion, what does the client own, can operate without external support, and has the internal capability to maintain? What is the firm's model for capability transfer? Is knowledge transfer written into the SOW as a deliverable, or is it a by-product of working alongside the team? What has gone wrong in comparable engagements, and what did you learn? Can you speak to a client where the engagement ended and the client is now operating independently? The answers to these questions distinguish firms with genuine delivery capability from firms with genuine strategy capability. Both are useful. Neither is a substitute for the other, and conflating them is where most AI advisory relationships go wrong for the client. What good looks like A good AI advisory engagement leaves the client more capable than when it started. The client team understands the models being used, can maintain and retrain them without external support, and has developed internal judgment about AI investment decisions. This is achievable. It requires an engagement structure that prioritizes knowledge transfer — embedding alongside client teams rather than working in parallel, documentation that is maintained by the client rather than produced by the consultant, and handoff criteria that verify internal capability before the engagement closes. It also requires a firm that has an economic model compatible with building client capability rather than client dependency. Those firms exist. Finding them requires knowing what to look for and being willing to ask the uncomfortable questions before the contract is signed.

Read full article