On this page · 10 sections
Summary. Gartner predicts that over 40% of agentic AI projects will be cancelled by the end of 2027, and its three named causes are escalating costs, unclear business value, and inadequate risk controls. Two of those three are measurement problems. In the same June 2025 release, Gartner's January 2025 poll of 3,412 webinar attendees found only 19% had made significant investments in agentic AI against 42% conservative and 31% waiting, and Gartner estimated that of the thousands of agentic AI vendors, only about 130 are real. By 1 July 2026 the firm put $234 billion of enterprise application software spend at risk from agentic AI. The awkward part is that a failing agent looks exactly like a working one. It returns 200 OK with a confident, plausible, wrong answer. No exception, no red dashboard, no page. Traditional monitoring was built to catch systems that stop. Agents do not stop. eCorpIT builds the evals and observability layer that catches the failures your APM cannot see.
You cannot prove business value you never measured. That is why the projects get cancelled.
Why agentic projects actually die
Gartner's Senior Director Analyst Anushree Verma is direct about the state of the field: "Most agentic AI projects right now are early stage experiments or proof of concepts that are mostly driven by hype and are often misapplied. This can blind organizations to the real cost and complexity of deploying AI agents at scale, stalling projects from moving into production."
Her second observation is the one that should shape your roadmap: "Most agentic AI propositions lack significant value or return on investment (ROI), as current models don't have the maturity and agency to autonomously achieve complex business goals or follow nuanced instructions over time. Many use cases positioned as agentic today don't require agentic implementations."
Read that against Gartner's three cancellation causes and a pattern appears. Escalating costs, unclear business value, inadequate risk controls. A team that instruments token spend per outcome can answer the first. A team with a scored eval suite tied to a business metric can answer the second. A team with traces and guardrails can answer the third. A team with none of these has only vibes when the CFO asks whether the agent is working, and vibes lose budget reviews.
Gartner still expects the capability to land: at least 15% of day-to-day work decisions made autonomously through agentic AI by 2028, up from 0% in 2024, and 33% of enterprise software applications including agentic AI by 2028, up from less than 1% in 2024. The technology is not the question. Proving it works is.
The failure modes traditional monitoring misses
| Failure mode | What it looks like in production | What catches it |
|---|---|---|
| Confident wrong answer | HTTP 200, fluent prose, plausible citation, factually incorrect | Scored eval against a golden set, groundedness check |
| Silent retrieval miss | Agent answers from parametric memory when RAG returned nothing | Retrieval hit-rate metric, context-in-prompt trace |
| Tool call never fired | Model narrates the action instead of taking it | Span assertions on tool invocation |
| Prompt or model drift | Quality decays after a provider updates a model behind the same name | Scheduled regression evals on a fixed set |
| Cost creep | Same task, more tokens per run, bill grows without traffic growing | Token and cost per successful outcome |
| Broken tenant filter | Retrieval returns another customer's documents | Filtered-path tests, tenant assertions in traces |
| Loop and stall | Agent burns turns without converging, session times out | Turn-count and session-duration metrics |
Every row here returns a 200. That is the entire problem. Your existing APM measures whether the service responded, and the service always responds.
What to instrument
Start with traces, not dashboards. A production agent needs one trace per run that carries the prompt, the retrieved context, every tool call and its arguments, the model and version, token counts, and the final output. Without the retrieved context in the trace, you cannot later tell a bad answer from a bad retrieval, and those have opposite fixes.
The standard to build on is OpenTelemetry's GenAI semantic conventions, which define shared attribute names for LLM calls, agent steps, vector database queries, token usage and cost. OpenTelemetry's own write-up covers the model, and Datadog now natively supports the conventions from v1.37 onward. The conventions are still evolving, so pin your versions and expect iteration. They are still a better foundation than a bespoke logging schema you will maintain forever.
The engineering judgement, from doing this repeatedly: instrument the retrieval context and the tool arguments on day one. Teams that skip those two fields spend the next quarter unable to debug their own incidents, and re-instrumenting a live agent is far more expensive than doing it at build time.
What to score
Evals are tests for non-deterministic systems. They differ from unit tests in one way that matters: the assertion is a judgement, not an equality.
| Eval type | Runs when | Catches |
|---|---|---|
| Golden-set regression | Every pull request, every model change | Prompt edits and model swaps that quietly reduce quality |
| Groundedness and citation | Continuously on sampled production traffic | Answers not supported by retrieved context |
| Tool-use assertion | CI, on every agent change | Narrated actions, wrong arguments, skipped calls |
| Filtered retrieval | CI plus scheduled | Tenant leaks and post-filter recall collapse |
| Cost per outcome | Scheduled, weekly | Token creep and unbounded loops |
| Human review sample | Weekly, small n | Everything your automated scorers have not learned yet |
The golden set is the asset. Fifty to two hundred real cases with agreed correct outcomes, drawn from your actual traffic and labelled by people who know the domain, will find more regressions than any generic benchmark. Build it from production traces once you have them, which is why instrumentation comes first.
Then wire it to a gate. An eval suite that nobody blocks a deploy on is a report, and reports get skimmed. We covered the CI side of this in our notes on evals catching silent failures in CI/CD.
Tie it to one business number
This is where projects are saved or lost. Pick the metric the agent exists to move, before you build it: deflection rate for a support agent, cycle time for a document workflow, conversion for a shopping assistant, cost per resolved ticket. Instrument that metric in the same trace as the technical ones, so a quality regression and a business dip are visible in one place.
When Gartner names "unclear business value" as a top cancellation cause, this is the gap. Teams report latency and token spend to a board that wants to know whether the thing paid for itself. Verma's advice points the same way: "To get real value from agentic AI, organizations must focus on enterprise productivity, rather than just individual task augmentation. They can start by using AI agents when decisions are needed, automation for routine workflows and assistants for simple retrieval. It's about driving business value through cost, quality, speed and scale."
Note the sequencing in that quote. Agents for decisions, automation for routine work, assistants for retrieval. Some of what is being built as an agent should be a scheduled job with a template. An honest eval suite tells you that early, and cancelling a bad use case in week three is a win, not a failure.
What eCorpIT builds
eCorp Information Technologies Private Limited, trading as eCorpIT, is a CMMI Level 5 certified, MSME certified technology organisation founded in 2021 and based in Gurugram. We are an AWS, Microsoft and Google partner, and our senior engineering teams work on production AI systems rather than demos.
Our evals and observability engagement runs in four stages.
- Trace audit, roughly one to two weeks. We instrument your agent to OpenTelemetry GenAI conventions, or fix what is there, so every run carries prompt, retrieved context, tool calls, model version, tokens and outcome. This stage is usually where a team sees a complete agent run for the first time, which is why it tends to be where the surprises surface.
- Golden set and scorers, roughly two to three weeks. We build the eval set from your real traces with your domain experts, define scorers for groundedness, tool use and task success, and calibrate them against human judgement rather than trusting a model grader blindly.
- CI gate and regression suite. The suite runs on pull requests and on a schedule, blocks merges that regress quality, and re-runs when a provider changes a model behind an unchanged name.
- Business metric wiring and handover. We connect the technical signal to the one number your leadership cares about, then hand the whole thing to your team with the runbook. We would rather you operate it than retain us to.
Engagement model: a fixed-scope discovery, then a build phase sized to your agent's surface area, then optional managed support. We design systems aligned with Digital Personal Data Protection Act 2023 requirements where personal data is in scope, and we will tell you plainly when a use case does not need an agent at all.
Related work: enterprise AI agent development, agent security guardrails, and governance layers for enterprise agents.
India-specific considerations
Two points for Indian teams and for GCCs running agents from India.
Tenancy is the sharp edge under the Digital Personal Data Protection Act 2023. Most Indian B2B deployments filter retrieval by tenant on every query. When a filter silently under-returns, that is a quality bug; when it over-returns, one data principal sees another's personal data, and that is an incident with a regulator attached. Filtered-path evals are not a nice-to-have here. Our notes on prompt injection and guardrails cover the adjacent attack surface, and the isolation choices underneath are in our comparison of agent session isolation across AWS, Azure, Google and Anthropic.
Cost discipline reads differently here too. Indian services teams have historically competed on people cost, and token spend does not respect that advantage: a wasteful agent loop bills the same in Gurugram as in California. Cost per successful outcome, tracked weekly, is the metric that keeps an agent economically defensible. See our FinOps guidance on AI cloud spend.
FAQ
How eCorpIT can help
eCorpIT is a CMMI Level 5 certified technology organisation in Gurugram, founded in 2021, and our senior engineering teams instrument and score production AI agents for a living. We start by making one full agent run visible end to end, build an eval suite from your real traffic rather than a public benchmark, gate it in CI so regressions cannot ship, and wire the whole thing to the business number your leadership actually asks about. If your agent works in the demo and you cannot prove it works in production, talk to us at /contact-us/.
References
- Gartner, Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027, 25 June 2025 (includes the January 2025 poll of 3,412 attendees and the Anushree Verma quotes)
- Gartner, Gartner Says $234 Billion in Enterprise Application Software Spend Is at Risk from Agentic AI, 1 July 2026
- Gartner, Gartner Says Applying Uniform Governance Across AI Agents Will Lead to Enterprise AI Agent Failure, 26 May 2026
- Gartner, 2026 Hype Cycle for Agentic AI
- Gartner, Gartner Predicts 40% of Enterprise Apps Will Feature Task-Specific AI Agents by 2026, 26 August 2025
- Gartner, Gartner Expects Most Enterprises to Abandon Assistive AI for Outcome-Focused Workflow by 2028, 2 April 2026
- OpenTelemetry, Inside the LLM Call: GenAI Observability with OpenTelemetry
- Janakiram MSV, The New Stack, AWS, Microsoft, and Google agree the session is the new unit of compute, 26 June 2026
Last updated: 16 July 2026.
