Replacing proprietary monitoring with open-source observability

CHALLENGE

A defense-sector enterprise was running a proprietary monitoring platform — a Datadog-equivalent stack — costing thousands per month. The cost was painful, but the bigger problem was visibility. The platform generated a constant flood of alerts, most of them noise. On-call engineers had stopped trusting the alerts, which meant real incidents got buried under false positives. There were blind spots too — critical systems that the proprietary agent couldn't instrument properly, leaving gaps in coverage that nobody noticed until something broke.

The organization needed full observability — metrics, logs, and visualization in one coherent stack — without the enterprise licensing tax and without the alert fatigue. And because this was defense, the solution had to be self-hosted and fully under their control. No SaaS. No data leaving the network.

APPROACH

The approach was a three-layer open-source observability stack, each tool handling what it does best. Codyssey embedded with the customer operations team, defined the observability architecture and the hard calls on alert governance and data retention, and transferred day-to-day runbook and tuning ownership to the team. Runbooks, escalation criteria, and alert-correlation logic were captured in the customer's knowledge base so the team could own and improve the monitoring. Codyssey stayed responsible for the architecture and the hard calls; the customer team took ownership of routine monitoring maintenance and incremental tuning.

• Zabbix for metrics collection and infrastructure monitoring — agent-based for deep host telemetry, SNMP for network devices, covering all 142 hosts.
• Graylog for log aggregation — centralized log ingestion, parsing, and search across every service, with retention policies tuned to compliance requirements.
• Grafana for visualization — unified dashboards pulling from both Zabbix and Graylog, giving on-call engineers a single pane of glass instead of three tabs.
• Custom alert tuning — the existing alert rules were rebuilt from scratch. Every alert was reviewed, correlated, and assigned an escalation path. Noise was eliminated at the source, not suppressed after the fact.

SOLUTION

The full open-source observability stack was deployed across the environment. Zabbix agents were rolled out to all 142 hosts — servers, network devices, and critical services — with custom templates for the defense-specific workloads that the proprietary platform couldn't instrument. Graylog was configured as the central log pipeline, ingesting structured logs from every application and system service, with parsing rules that normalized formats across the estate.

Grafana dashboards were built for each operational domain: infrastructure health, application performance, security events, and capacity trends. Every dashboard pulls live data from both Zabbix and Graylog, so an engineer investigating an incident sees metrics and logs side by side without switching tools.

The alert rules were the hardest part — and the highest-impact. The old stack had hundreds of rules, most firing independently with no correlation. The new ruleset was rebuilt around escalation procedures: an alert only fires when a condition persists and correlates across signals. Each alert has a defined owner, a defined response, and a defined escalation path. Noise was designed out, not filtered out.

OUTCOME

All 142 hosts are now monitored end-to-end — including the systems the proprietary platform couldn't see. Alert noise dropped by 87%, measured by alert volume before and after the cutover. On-call engineers trust the alerts again, because when one fires, it means something. The blind spots are gone; every critical system has telemetry, logs, and a dashboard.

The cost savings were real: $6,200/month reclaimed from the proprietary bill — and that's with broader coverage, not less. The stack is fully self-hosted, under the organization's control, with no data leaving the network. Compliance requirements met. Budget reclaimed.

The customer operations team now owns the dashboards, alert rules, and escalation playbooks, with the knowledge base enriched by runbooks and correlation rationale. Codyssey remains responsible for the observability architecture and the hard calls around alert governance and retention. Because alerts are now trustworthy and correlated, incidents are caught earlier and escalated properly — a shift from alert fatigue to proactive incident prevention.

"For the first time in years, when my phone rings at 2 AM, it's actually something I need to look at. The noise is gone. We see everything now — including the systems we were flying blind on before."— Operations Lead, anonymized

STACK

Zabbix (metrics & infrastructure monitoring) · Graylog (log aggregation & search) · Grafana (unified visualization) · Custom alert rules with escalation procedures · Self-hosted, fully under customer control