86% of enterprise GPU operators run at half capacity or less

Enterprise GPUs are idle at exactly the wrong moment. In VentureBeat Research’s June survey of 573 technical leaders at companies with 100+ employees, 86% of enterprises that run their own GPUs report utilization of 50% or less. In other words: the most expensive hardware in the building is doing at most half the work it could do.

And here’s why Wall Street’s “AI buildout” debate matters to operators and boards: enterprises keep shopping even with that utilization gap. The research says 45% of these enterprises say the emerging compute option they are most likely to evaluate in the next 12 months is an AI-specialized cloud (CoreWeave, Lambda, Crusoe, Nebius). Yet under 2% report using one of these neoclouds today, and roughly one in three companies is considering a hedge against Nvidia. Specifically, 32% named non-Nvidia accelerators (AWS Trainium, Google TPUs, AMD) while 28% named next-generation Nvidia GPUs. The buying process is moving faster than measurement.

The measurement gap is the quiet accelerant. Only 44% of enterprises rigorously track what their AI compute actually costs and returns; everyone else is estimating. That matters because utilization, cost per workload, and real performance are the difference between “we need more capacity” and “we need better orchestration.” If you are not measuring properly, it’s easy to misdiagnose bottlenecks as shortages. So when the compute conversation turns to new clouds, new accelerators, or more GPUs, the risk is that budgets chase capacity instead of outcomes.

This underutilization is happening alongside another adoption mismatch. Most deployed “agents” are not truly multi-step agents in production. Seventy-one percent of enterprises say a quarter or fewer of their deployed “agents” can complete multi-step work on their own; the rest are single-prompt chatbots. Only 10% say true agents are the majority of what they run. Respondents are positioned to know (81% said they recommend or decide AI purchases), and the survey’s point is blunt: adoption claims across the industry may be outpacing what enterprise systems actually do.

Labels also control risk. The report highlights that if you are running chatbots where humans read each answer, you need fewer identity, evaluation, and cost controls than you would for true multi-step agents. But many enterprises are deploying forward while lagging on those controls anyway. Roughly six in 10 enterprises plan to switch or add vendors in each of five control layers within the next 12 months, and roughly a third (depending on the layer) plan to move within the quarter. The five layers are: identity for agents (who can do what, under whose credentials), evaluation of agent output, cost telemetry, the context layer (business data and definitions agents draw on), and the orchestration control plane (the software coordinating multi-step agent work).

Enterprises are already paying for moving ahead. Fifty-four percent of companies had an agent security incident or near-miss caught before harm in the past 12 months. On cost controls, 27% only exercise reactive control of agent spend, learning what an agent costs when the invoice arrives, with no per-agent budget or ceiling in place. That creates a second-order problem for finance and governance: even if the technical team is “successful,” the finance org might be flying blind until the bill lands.

The evaluation and safety gaps are even more consequential. Sixty-six percent of enterprises allow an AI agent to push a code or system change to production based on automated evaluation results alone, with no human reviewing it, or are engineering toward that within 12 months. Only five percent fully trust those automated evaluations. The distrust is earned: half of enterprises shipped an agent that passed internal evaluations and then caused a customer-facing failure in the past year, and a quarter watched that happen more than once. The biggest weakness in current evaluations, chosen by 29% of respondents, was “poor alignment with real-world outcomes.” And after agents are live, quality checks thin out: only 23% run real-time quality checks on agent answers; 51% monitor system health (uptime, request traces, gateway logs) which tells them the agent is running, not whether its answers are right.

Security gets worse where identities are shared. Sixty-nine percent of companies allow agent credential sharing somewhere in their agent fleet during runtime (multiple agents operating under one API key or service account). Those companies experienced a security incident or near-miss at a 63.5% rate (47 of 74), compared with 40.9% (9 of 22) where every agent has its own scoped identity. The report’s takeaway is specific: give every agent its own scoped identity, starting with agents that touch production systems. And context mistakes keep showing up. Fifty-seven percent of enterprises traced at least one confident, wrong agent answer in the past six months to missing or inconsistent business context (wrong metrics, stale definitions, absent documents), often more than once. Some companies are already fixing this: 25% already run a governed semantic layer, or one governed definition of the business that every AI reads from, in production.

So the strategic stakes are clear for executives: the compute story is not just about whether the AI buildout is overbuilt. It is about whether enterprises can measure utilization and cost, run real multi-step agents safely, and validate outcomes against production realities. With 86% underutilizing GPUs and 45% planning new compute evaluation while only 44% track cost and returns rigorously, boards should treat “AI spend” as a governance problem, not only a tech build problem. The winners will be the orgs that retro-fit control layers fast, test evaluations against production outcomes, and make sure the next budget tranche improves real work, not just hardware occupancy.