Skip to content
LIVE
The Executives BriefThe Executives Brief

T-Mobile fed its AI every support doc. It invented hold music.

CIO Jeff Simon says dumping all support documentation into agents backfired, and a new survey finds 78% of firms have shipped a confident wrong answer.

ByHessa Al-FalehBusiness Desk, The Executives Brief
·6 min read
T-Mobile fed its AI every support doc. It invented hold music.
Executive summary

T-Mobile CIO Jeff Simon told VentureBeat's Agentic Infrastructure Stack Summit that loading all customer support documentation into its AI agents backfired, forcing a retreat to a plain lookup table and a second model that catches bad answers. New VB Pulse data shows 78% of organizations running agents on company data hit at least one confident-but-wrong answer from missing context in six months, making context curation a board-level risk.

T-Mobile tried to make its customer service AI smarter by giving it everything. The result was worse answers. The company loaded its entire body of customer support documentation into its agents' context window, and the agents went off the rails, CIO Jeff Simon said at VentureBeat's Agentic Infrastructure Stack Summit in October. "We said, 'Oh, we've finally got a big context window model. Let's load all of our customer support documentation into that agent, and it's gonna be the expert, right? It's gonna know everything,'" Simon said. "That didn't work at all." The problem was conflict, not volume. Support documentation at a company T-Mobile's size contains contradictory guidance, and one customer phrase can map to more than one process: ask for a "new device" and you might mean a new phone on an existing line, or an additional line entirely. "When you load up the context window really large, the AI is gonna be forced to make a decision between the paths and the documentation, and it may start answering from the wrong ones," Simon said.

The obvious fix failed too. T-Mobile built a knowledge graph. "The knowledge graph didn't work largely because the agents had to search across this big graph, and it took time, and we don't have a lot of time," Simon said. Speed was only half of it: customers ask the same question a dozen ways, and when agents misread a request they could take the wrong route through the graph and get lost, even when T-Mobile tried to guide them. So the company went backward on purpose. It now runs a simple cache in a key-value database that works as a lookup table, mapping the common ways customers phrase questions to the path an agent should follow. An orchestrator agent confirms which path the customer wants and routes the request only to the agent holding context for that path. The table is built for agents, not people, and the team updates it as customers invent new phrasings. "It is actually very basic, but it's fast and responsive, it scales really well, and so far the performance from our agents is good," Simon said.

Then there is the hold music. T-Mobile runs a small language model alongside its production model on every chat and voice session, acting as an LLM judge that flags sessions where an answer went wrong or a customer is getting angry so humans can review them. The volume is far too high for people to check all sessions, or even a large sample. The judge surfaced a strange pattern: hold music playing on many calls. T-Mobile does not play hold music. Engineers spent a couple of weeks digging through the calling and interactive voice response systems before tracing the cause to the customer-facing model, which was generating hold music when it paused to look something up or run a tool call. "It was completely invented," Simon said. T-Mobile now tells the model in its prompts not to hallucinate hold music. Running a second model on every interaction adds cost, Simon acknowledged, but T-Mobile considers it worthwhile because no team can anticipate every odd way a customer phrases a request.

The stakes extend well past one carrier. In VentureBeat Intelligence's October VB Pulse survey on agentic context layers, 78% of the 97 respondents whose organizations run AI agents on company data traced at least one confident but wrong answer to missing or inconsistent business context in the past six months. Among the 76 who reported a context failure, 42% said their most significant case ended with incorrect information reaching an outside party or an official report, or with a decision or action being reversed. The breakdown: 58% caught their worst case internally with no external impact, 21% had a customer, partner or supplier receive incorrect information, 12% saw an incorrect figure reach a financial, regulatory or board report, and 9% had to reverse a decision or action. Respondents could select more than one outcome.

How companies find out matters as much as what they find. Automated checks comparing agent output against a trusted source were the most common detection method at 57%, followed by subject-matter expert sample review at 55% and dashboard or monitoring alerts at 52%. A customer, partner or supplier noticing came in at 19%. Automated checks travel with root-cause discipline: all 55 respondents who named them also track whether a wrong answer came from business context or from the model. Of the 42 who did not name them, 10, or 24%, do not track that distinction at all.

Adobe frames the same problem as a precision question, not a volume question. Anjul Bhambhri, senior vice president of engineering at Adobe, described Adobe CX Enterprise Coworker, the company's marketing agent, as aiming for precise context, with the amount depending on the task. A European email marketer needs the email requirements, the EU's consent policies and the company's own rules on whom to contact. "If you feed both the EU and the US consent policies, now there are chances of errors," Bhambhri said. Refining a holiday campaign is the opposite: it calls for years of history. "There's a lot more context that has to be given because if that is not given, they may get a very generic response," she said. Bhambhri said historical data in one case showed free shipping converted better than a 20% discount for loyal customers during a Black Friday sale. Adobe narrows context before the model sees a prompt: seed context fixed per company, a prefetching step drawing on keywords, semantic indexes and knowledge graphs, then a merging and ranking step that decides how to fill the context window. Natural language understanding and a cheaper model do that work ahead of the main model.

Uber's knowledge graph tells a different story. In an August 27 engineering blog post, the company said an agent grounded in its AI Context Graph found a data table in 38 seconds, while the same model without the graph spent more than 20 minutes and wrongly concluded the data could not be queried. Uber's engineering agents use the graph to locate relevant systems and retrieve documents or code as needed. The difference is the job: Uber's graph is a map for finding things, while T-Mobile's agents had to navigate theirs mid-conversation, under time pressure.

The pattern across all three companies is that context is now an architecture decision, not a prompt tweak. T-Mobile's answer was to shrink what the model sees, route requests to narrow specialists, and spend on a second model to catch what slips through. T-Mobile has been building agents that answer customers over chat, text and voice since 2024, and it has focused on orchestrating those agents rather than training its own models. As the models improved at coordinating multiple agents and tool calls, T-Mobile added more agents, Simon said. Its agents can also take limited action on accounts, such as purchasing an international data pass or adding services, with checks confirming the customer intended the purchase. Before a request reaches the model, T-Mobile removes content that may not be relevant, a step Simon called one of its most important learnings. For executives scaling agents into customer-facing and financial workflows, the lesson is blunt: more context is not more intelligence. Conflicting documents, slow retrieval and untracked failures are the quiet ways an agent program stalls, and the fix usually starts with subtracting, not adding.

Executive ActionsLocked

This story's Key Insights and Take-aways are locked.

Create a free account to unlock Executive Actions for one credit.

Register to Unlock

Always free for Executives Club members. Join the Club

More in Business