LinkedIn, Walmart, Zendesk found the real AI agent bottleneck: infrastructure speed and control

AI agents are fast in demos. They move in milliseconds. Enterprise systems? They often run like they are still waiting for humans to click buttons and file tickets. That mismatch is exactly what LinkedIn, Walmart, and Zendesk said they ran into when they tried to scale agents from pilot to production at VB Transform 2026.

On the panel, Animesh Singh, senior director of AI platform and infrastructure at LinkedIn, Desiree Gosby, SVP of corporate technology services and technology strategy at Walmart, and Sami Ghoche, VP of applied AI at Zendesk, each walked through the bottleneck they hit. Their common thread was blunt: “None of the bottlenecks they hit were model problems.” The gap between how agents work and how legacy infrastructure was built to support humans is where engineering battle scars live.

To understand why this matters now, zoom out for a second. When enterprises evaluate AI agents, they usually start with the obvious: which model to use, what it can generate, how it reasons, whether it hallucinates. But once you give an agent real operational control, the system stops being a chatbot and becomes a workflow engine with tight timing, audit trails, and lots of “what happened next” logic. In that world, infrastructure turns from background to lead actor.

LinkedIn described two distinct infrastructure walls. The first wasn’t the model. It was Kubernetes. Kubernetes is designed for on-demand container spin-up, which Singh said takes seconds. Seconds is an eternity when your agent assumes rapid, repeated switching between tasks. LinkedIn’s fix was to move to pre-provisioned pools of containers, so agentic workloads can be swapped in and out in real time.

The second wall was subtler: it appeared once LinkedIn let agents orchestrate their own loop. Singh described a five-point evaluation system that initially looked clean, yet hallucinations kept showing up anyway. The structural issue, he said, is that an LLM evaluating another LLM shares the same failure mode as the thing it is evaluating. In other words, if both components are vulnerable to the same kind of errors, you are not actually closing the risk gap.

LinkedIn’s response was to change the control plane. Singh said they “built our own harness, our own control flow,” and pushed the LLMs “to the leaf instead of them orchestrating the loop.” Roughly 80% of the workflow is now scripted, deterministic code, with LLMs used only where reasoning is required. Evidence gets committed to disk before the system moves on, which matters because agents need a durable record of what they saw and decided, not just a narrative output.

Walmart’s bottleneck took the form of success. Gosby said an agent harness put directly into employees' hands went viral internally. “Citizen developers” began building their own agents to solve problems that once required a formal engineering roadmap. The upside was real innovation. The downside was duplication: dozens of overlapping agents with no coordination.

Here, the infrastructure isn’t just compute speed. It’s governance and release mechanics. Gosby said the fix wasn’t to pull the harness back. It was to build governance that can spot duplication, promote the best version of an agent, and get it into production without engineering becoming a chokepoint. That’s a board-level issue disguised as a technical one: if you let the organization ship autonomously, you also have to manage platform standards, security posture, and operational ownership.

Zendesk’s bottleneck came from data and context. Ghoche, who joined via Zendesk’s acquisition of Forethought, which closed in March 2026, described sitting on a “public figure of 20 billion customer conversations” in Zendesk’s repository. The natural instinct, he said, is to hand that history to a large language model with a big context window and generate the agents a business needs. But Ghoche argued that doesn’t work.

“You can’t really do that,” he said, and the alternative is to invest in the underlying data pipelines and all the data infrastructure that comes with that. This is the less glamorous part of agent scaling: data plumbing, lineage, transformations, and the ability to produce consistent context at runtime. It’s also where compliance and auditability tend to live. Even if the source doesn’t say “regulation,” the underlying theme is clear: agents that act in enterprise settings need traceable inputs and controllable outputs.

All three also aligned on open source strategy, but with an important nuance. Their common instinct was to own what they can and lean on frontier labs only where there’s a clear edge. Ghoche said enterprises would generally prefer to own their models and infrastructure wherever possible, and that reasoning drives Zendesk’s approach. The exception is frontier “reasoning work,” though he said that slice of use cases is shrinking compared to everything else enterprises do with AI.

Singh described LinkedIn’s push for independence with two subsystems. First is an AI gateway, a single interface so every outbound call to a model goes through the same layer regardless of provider. Second is a memory subsystem designed to hold context independent of any model provider. Singh said that every outbound call to an LLM, whether on public cloud or on-prem in Zendesk? Sorry, in LinkedIn’s own data centers, follows the same semantics and API calls, so they can switch providers quickly. Walmart’s approach was similar in spirit: an internal gateway built to stay vendor agnostic across three workload types, from fully deterministic workflows to planner-and-reasoner workflows and a hybrid of the two, with compliance-heavy work staying deterministic by design.

Finally, the panel offered modernization advice that maps directly to the walls they hit. Ghoche urged “Invest in evals before anything else,” calling evals the common element across internal and customer-facing use cases, because they force you to break the problem down and let you move faster once you have robust coverage. Gosby said “Own your agent harness from day one,” putting it into employees’ hands early while pairing it with infrastructure to monitor what it produces. Singh emphasized building for independence so enterprises can keep context within the enterprise and reuse it whether they ship a frontier model today or an open source model tomorrow.

If you run an enterprise platform, manage an AI budget, or sit on an AI steering committee, this is the uncomfortable takeaway: your bottleneck might not be your model vendor. It might be Kubernetes provisioning assumptions, the evaluation architecture itself, governance for internal agent proliferation, and the data pipelines that can actually feed an agent reliably. Agents may think in milliseconds, but your organization still has to move like it can.