The vendor pitch sounds clean: one unified, AI-enabled observability platform that gives you full visibility across your entire environment. The enterprise reality looks nothing like it.
ETR Insights recently spoke with senior technology leaders spanning technology services, retail, and regulated industries (a CIO, a CISO, and a Head of Technology) about how they actually manage observability today. What they describe is a market that has broadened significantly but remains structurally fragmented, strained by AI-driven demands that today's tools were simply not built to handle.
Key Takeaways
Despite years of consolidation messaging from major vendors, not one of the panelists operates with a single-platform observability strategy, and none believes that's achievable in the near term. "Yes, the single integrated platform would be ideal," one executive says, "but we're so far away from that that we've had to add different products together with overlap, because one doesn't do the job."
Amazon CloudWatch and Microsoft Azure Monitor form the operational backbone for most environments, with Google Cloud Observability used more selectively. But native hyperscaler tools alone do not close the gap. Leaders consistently layer in commercial platforms (Datadog, Dynatrace, Splunk, New Relic, and Elastic) alongside open source components such as Grafana, Prometheus, the ELK stack, and OpenSearch. "We've had to add all these different products together because one doesn't do the job."
The comparison between the two dominant hyperscalers is pointed. "CloudWatch obviously makes sense. Azure Monitor, not so much from my experience. Relatively speaking, I could be biased, but CloudWatch does a better job." That gap forces third-party platforms to normalize cross-environment visibility. "Therefore, we are forced to use, whether it's Grafana, whether it is New Relic or Datadog, one of those."
ETR Data: In an October 2025 drill down survey on open source and AI/GPU observability (N=103), Microsoft Azure Monitor (55%), Cisco's Splunk & AppDynamics (54%), Amazon CloudWatch (42%), and Grafana (41%) had the highest rates of current use among listed observability tools.
Open source tools are valued for flexibility, licensing control, and cost advantages that are genuinely attractive at the architecture level. The operational reality complicates that picture. "Open source sounds great, it's free," one executive says, "but the problem with open source is you need multiple FTEs to look after the solution." Skill turnover compounds the challenge: "Staff will quickly turn over, and that skill set leaves."
The CISO of a technology and business services firm with defense and critical infrastructure customers frames AWS as the gravitational center of their observability strategy, with open source layered around it. "We're big on open source products. Amazon CloudWatch is for us number one, followed by Elastic ELK/OpenSearch, and obviously Grafana and Prometheus that go with that stack." Commitment to that stack deepened after Elasticsearch's licensing changes led the firm to Amazon OpenSearch Service as a stable long-term alternative.
Without sufficient internal expertise, however, a heavy reliance on open source architectures can evolve into what previous ETR panelists have called a "Frankenstein stack," bringing integration challenges, fragmented visibility, and maintenance burdens that quietly erode the cost advantages that made open source appealing in the first place.
OpenTelemetry represents the clearest attempt to standardize how observability data is collected across environments, and practitioners are watching it closely. One organization is "heavily using OpenTelemetry with the AWS suite," particularly for long-term log collection and search. Elsewhere, evaluation is ongoing. Internal architects have raised concerns around data integration and scalability "when it comes to massive data." In retail environments, OpenTelemetry is not yet in production, though vendors such as Splunk are actively promoting adoption.
ETR Data: In the October 2025 drill down survey on open source and AI/GPU observability (N=103), more than a third (37%) of respondents indicated they partially used OpenTelemetry for metrics, logs, or trace collection, with another 6% indicating extensive use. More than half (53%) said they did not currently use OpenTelemetry, though 30% from that group said they were evaluating it.
Licensing cost is the headline complaint, but the sharper operational pressure comes from storage economics. "The problem has remained what it's been for decades," one executive says. "That is the data quality and the cleansing of the data." More data is not the issue. It is the wrangling that comes with it, which cascades into poor model performance and unreliable outputs.
One CISO provides concrete benchmarks from a regulated environment, describing a tiered architecture across L1, L2, and L3 storage. "We offer essentially seven and 30 days of live data" for L1 and L2 tiers where logs must remain immediately searchable. "OpenSearch anywhere might be $10,000 a month to $100,000 a month just for having that hot data available." Data beyond that window moves into L3 long-term storage, often subject to compliance validation requirements. "Then you put it into something like Snowflake, and that's not cheap either, especially when you're talking about five- to seven-year retention."
These economics are prompting some organizations to make deliberate build-versus-buy decisions. One firm assigned three FTEs over one year, roughly a million dollars of investment, to build an internal lakehouse using inexpensive disks. The framing was strategic, not tactical: "That was to buy down long-term vendor cost. Now that cost is gone."
Panelists describe a recurring disconnect between what vendor sales organizations promise and what engineering teams can actually deliver. "The salespeople will say they do it, and the pre-sales architects will say yes. But when you actually deal with the techs, they'll say it's on the roadmap." The conclusion is blunt: "Sales are selling things that don't exist yet."
Even where consolidation is technically feasible, leaders want modularity, not monolithic bundles. Some organizations are actively negotiating to strip out embedded AI components they do not use, asking for base pricing that excludes unused functionality. "It would be nice to have one platform, but it has to be very modular, because I would not like to subscribe to everything." Observability needs break down into discrete operational categories (hardware, applications, databases, security telemetry, and AI systems), each with different relevance depending on the business. "If a platform can come up with something I can subscribe to a la carte, that'd be great."
On Splunk specifically, the ETR community's assessment is consistent: a mature, capable product that has priced itself out of broader adoption. "Splunk is an awesome product. It's one of the most mature, but it's just too expensive."
ETR Data: In the 2025 ETR Observatory for Observability Tools (N=319), Datadog, Grafana, and Google Cloud Observability had the highest rates of expected return on investment within the first three years. IBM Instana Observability, SolarWinds Observability, Cisco's Splunk and AppDynamics, and Dynatrace had the lowest expected ROI within three years.
The deepest frustration in the discussion is not fragmentation, cost, or vendor credibility. It is the inability of existing observability platforms to do what AI workloads actually require. "These are all traditional tools which are trying to incorporate AI monitoring," the CIO of a midsized technology company says. "However many you have, none of them does a good job when it comes to true AI monitoring."
Current platforms remain infrastructure-centric. "These tools define your infrastructure monitoring. But for AI, what we want to know is much more than that: is the model output correct?" Drift detection, hallucination analysis, and model behavior visibility remain largely unresolved. Once data enters a large language model, practitioners describe limited visibility into how it is used or prioritized, a governance problem that surfaces acutely during audits.
When AI-specific monitoring tools are proposed as potential solutions, the business case hesitation is immediate. "Is it worth it? Is the business case there?" That question remains open, and by the panelists' own assessment, the tools are not ready to answer it. "I'm talking about true AI model insights" and today's observability market is materially behind that expectation.
Observability remains structurally fragmented not because technology leaders haven't tried to consolidate, but because no platform has yet delivered the coverage, modularity, and AI-native monitoring that enterprise environments actually require. Hyperscalers dominate the foundation. Open source provides flexibility at a real staffing cost. Commercial platforms compete on depth and usability. And AI model monitoring, the most consequential emerging requirement, is the gap the entire market has yet to close.
This article is based on an ETR Insights Interview with senior enterprise technology leaders from technology services, retail, and regulated industries. Contact the ETR Insights team to discuss the research in detail or to request custom research.
Vendors mentioned: Amazon (AWS, CloudWatch), Chronosphere, Cisco (AppDynamics, Splunk), ClickHouse, Databricks, Datadog, Dynatrace, Elastic (ELK), Google, Grafana Labs, Microsoft (Azure Monitor), New Relic, OpenSearch, Oracle, OVHcloud, Prometheus, Snowflake.