Posts Taged zabbix

Vibe Coding with AI: Building an Automatic Zabbix Service Map

Vibe Coding with AI: Building an Automatic Zabbix Service Map

Good morning everyone! Dimitri Bellini here, back on the Quadrata channel, your spot for diving into the open-source world and IT topics I love – and hopefully, you do too!

If you enjoy these explorations, please give this video a thumbs up and subscribe if you haven’t already. Today, we’re venturing into the exciting intersection of coding, Artificial Intelligence, and Zabbix. I wanted to tackle a real-world challenge I faced: using AI to enhance Zabbix monitoring, specifically for event correlation.

Enter “Vibe Coding”: A Different Approach

Lately, there’s been a lot of buzz around “Vibe Coding.” What is it exactly? Honestly, it feels like a rather abstract concept! The general idea is to write code, perhaps in any language, by taking a more fluid, iterative approach. Instead of meticulous planning, designing flowcharts, and defining every function upfront, you start by explaining your goal to an AI and then, through a process of trial, error, and refinement (“kicks and punches,” as I jokingly put it), you arrive at a solution.

It’s an alternative path, potentially useful for those less familiar with specific languages, although I believe some foundational knowledge is still crucial to avoid ending up with a complete mess. It’s the method I embraced for the project I’m sharing today.

What You’ll Need for Vibe Coding:

  • Time and Patience: It’s an iterative process, sometimes frustrating!
  • Money: Accessing powerful AI models often involves API costs.
  • Tools: I used VS Code along with an AI coding assistant plugin. The transcript mentions “Cline” and “Roocode”; a popular and evolving option in this space is Continue.dev (which might be what was referred to, as tool names evolve). These tools integrate AI directly into the IDE.

The Zabbix Challenge: Understanding Service Dependencies

My initial goal was ambitious: leverage AI within Zabbix for better event correlation to pinpoint the root cause of problems faster. However, Zabbix presents some inherent challenges:

  • Standard Zabbix configurations (hosts, groups, tags) don’t automatically define the intricate dependencies *between* services running on different hosts.
  • Knowledge about these dependencies is often siloed within different teams in an organization, making manual mapping difficult and often incomplete.
  • Zabbix, by default, doesn’t auto-discover applications and their communication pathways across hosts.
  • Existing correlation methods (time-based, host groups, manually added tags) are often insufficient for complex scenarios.

Creating and maintaining a service map manually is incredibly time-consuming and struggles to keep up with dynamic environments. My objective became clear: find a way to automate service discovery and map the communications between them automatically.

My Goal: Smarter Event Correlation Through Auto-Discovery

Imagine a scenario with multiple Zabbix alerts. My ideal outcome was to automatically enrich these events with tags that reveal their relationships. For example, an alert on a CRM application could be automatically tagged as dependent on a specific database instance (DB Instance) because the system detected a database connection, or perhaps linked via an NFS share. This context is invaluable for root cause analysis.

My Vibe Coding Journey: Tools and Process

To build this, I leaned heavily on VS Code and the AI assistant plugin (let’s refer to it as Continue.dev for clarity, acknowledging the transcript’s terms “Cline/Clean”). The real power came from the Large Language Model (LLM) behind it.

AI Model Choice: Claude 3 Sonnet

While local, open-source models like Llama variants exist, I found they often lack the scale or require prohibitive resources for complex coding tasks. The most effective solution for me was using Claude 3 Sonnet via its API (provided by Anthropic). It performed exceptionally well, especially with the “tool use” features supported by Continue.dev, which seemed more effective than with other models I considered.

I accessed the API via OpenRouter, a handy service that acts as a broker for various AI models. This provides flexibility, allowing you to switch models without managing separate accounts and billing with each provider (like Anthropic, Google, OpenAI).

Lessons Learned: Checkpoints and Context Windows

  • Use Checkpoints! Continue.dev offers a “Checkpoint” feature. Vibe coding can lead you down wrong paths. Checkpoints let you revert your codebase. I learned the hard way that this feature relies on Git. I didn’t have it set up initially and had to restart significant portions of work. My advice: Enable Git and use checkpoints!
  • Mind the Context Window: When interacting with the AI, the entire conversation history (the context) is crucial. If the context window of the model is too small, it “forgets” earlier parts of the code or requirements, leading to errors and inconsistencies. Claude 3 Sonnet has a reasonably large context window, which was essential for this project’s success.

The Result: A Dynamic Service Map Application

After about three hours of work and roughly $10-20 in API costs (it might have been more due to some restarts!), I had a working proof-of-concept application. Here’s what it does:

  1. Connects to Zabbix: It fetches the list of hosts monitored by my Zabbix server.
  2. Discovers Services & Connections: For selected hosts, it retrieves information about running services and their network connections.
  3. Visualizes Dependencies: It generates a dynamic, interactive map showing the hosts and the communication links between them.

The “Magic Trick”: Using Netstat

How did I achieve the automatic discovery? The core mechanism is surprisingly simple, albeit a bit brute-force. I configured a Zabbix item on all relevant hosts to run the command:

netstat -ltunpa

This command provides a wealth of information about listening ports (services) and established network connections, including the programs associated with them. I added some preprocessing steps (initially aiming for CSV, though the core data comes from netstat) to make the data easier for the application to parse.

Live Demo Insights

In the video, I demonstrated the application live. It correctly identified:

  • My Zabbix server host.
  • Another monitored host (Graph Host).
  • My own machine connecting via SSH to the Zabbix host (shown as an external IP since it’s not monitored by Zabbix).
  • Connections between the hosts, such as the Zabbix agent communication (port 10050) and web server access (port 80).
  • Clicking on hosts or connections reveals more details, like specific ports involved.

While the visual map is impressive (despite some minor graphical glitches that are typical of the rapid Vibe Coding process!), the truly valuable output is the underlying relationship data. This data is the key to achieving the original goal: enriching Zabbix events.

Next Steps and Your Thoughts?

This application is a proof-of-concept, demonstrating the feasibility of automatic service discovery using readily available data (like netstat output) collected by Zabbix. The “wow effect” of the map is nice, but the real potential lies in feeding this discovered dependency information back into Zabbix.

My next step, time permitting, is to tackle the event correlation phase – using these discovered relationships to automatically tag Zabbix problems, making root cause analysis much faster and more intuitive.

What do you think? I’d love to hear your thoughts, ideas, and suggestions in the comments below!

  • Have you tried Vibe Coding or similar AI-assisted development approaches?
  • Do you face similar challenges with service dependency mapping in Zabbix or other monitoring tools?
  • Are there specific use cases you’d like me to explore further?

Don’t forget to like this post if you found it interesting, share it with others who might benefit, and subscribe to the Quadrata YouTube channel for more content like this!

You can also join the discussion on the ZabbixItalia Telegram Channel.

Thanks for reading, have a great week, and I’ll see you in the next video!

Bye from Dimitri.

Read More
Exploring Zabbix 7.4 Beta 1: What’s New and What I’m Hoping For

Exploring Zabbix 7.4 Beta 1: What’s New and What I’m Hoping For

Good morning everyone! Dimitri Bellini here, back on the Quadrata channel – your spot for everything Open Source and the IT topics I find fascinating (and hopefully, you do too!). Thanks for tuning in each week. If you haven’t already, please consider subscribing and hitting that like button; it really helps the channel!

This week, we’re diving into something exciting: the latest Zabbix 7.4 Beta 1 release. This is a short-term support (STS) version, meaning it’s packed with new features that pave the way for the next Long-Term Support (LTS) release, expected later this year. With Beta 1 out and release candidates already tagged in the repositories, the official 7.4 release feels very close – likely within Q2 2024. So, let’s break down what’s new based on this first beta.

Key Features Introduced in Zabbix 7.4 Beta 1

While we don’t have that dream dashboard I keep showing (maybe one day, Zabbix team!), Beta 1 brings several practical and technical improvements.

Performance and Internals: History Cache Management

A significant technical improvement is the enhanced management of the history cache. Sometimes, items become disabled (manually or via discovery) but still occupy space in the cache, potentially causing issues. Zabbix 7.4 introduces:

  • Automatic Aging: Zabbix will now automatically clean up these inactive items from the cache.
  • Manual Aging Command: For environments with many frequently disabled objects, you can now manually trigger this cleanup at runtime using a command. This helps free up resources and maintain stability.
  • Cache Content Analysis: For troubleshooting, there are now better tools to analyze cache content and adjust log verbosity in real-time, which is invaluable in critical production environments.

UI and Widget Enhancements

  • Item History Widget Sorting: The Item History widget (introduced in 7.0) gets a much-needed update. When displaying logs, you can now choose to show the newest entries first, making log analysis much more intuitive than the old default (oldest first).
  • Test Item Value Copy Button: A small but incredibly useful UI tweak! When testing items, especially those returning large JSON payloads, you no longer need to manually select the text. There’s now a dedicated copy icon. Simple, but effective!
  • User Notification Management: Finally! Users can now manage their own notification media types (like email addresses) directly from their user settings via a dedicated menu. Previously, this required administrator intervention.

New Monitoring Capabilities

  • New ICMP Ping Item (`icmpping`): A new item key for ICMP checks includes a crucial `retry` option. This helps reduce noise and potential engine load caused by transient network issues. Instead of immediately flagging an object as unreachable/reachable and potentially triggering unnecessary actions or internal retries, you can configure it to try, say, 3 times before marking the item state as unreachable. This should lead to more stable availability monitoring.
  • New Timestamp Functions & Macros: We have new functions (like `item.history.first_clock`) that return timestamps of the oldest/newest values within an evaluation period. While the exact use case isn’t immediately obvious to me (perhaps related to upcoming event correlation or specific Windows monitoring scenarios?), having more tools for time-based analysis is interesting. Additionally, new built-in timestamp macros are available for use in notifications.

Major Map Enhancements

Maps receive some fantastic updates in 7.4, making them much more powerful and visually appealing:

  • Item Value Link Indicators: This is huge! Previously, link status (color/style) could only be tied to triggers. Now, you can base link appearance on:

    • Static: Just a simple visual link.
    • Trigger Status: The classic method.
    • Item Value: Define thresholds for numeric item values (e.g., bandwidth usage) or specific strings for text items (e.g., “on”/”off”) to change the link’s color and line style. This opens up possibilities for visualizing performance directly on the map without relying solely on triggers.

  • Auto-Hiding Labels: Tired of cluttered maps with overlapping labels? You can now set labels to hide by default and only appear when you hover over the element. This drastically improves readability for complex maps.
  • Scalable Background Images: Map background images will now scale proportionally to fit the map widget size, preventing awkward cropping or stretching.

One thing I’d still love to see, maybe before the final 7.4 release, is the ability to have multiple links between two map objects (e.g., representing aggregated network trunks).

New Templates and Integrations

Zabbix continues to expand its out-of-the-box monitoring:

  • Pure Storage FlashArray Template: Monitoring for this popular enterprise storage solution is now included.
  • Microsoft SQL for Azure Template: Enhanced cloud monitoring capabilities.
  • MySQL/Oracle Agent 2 Improvements: Simplifications for running custom queries directly via the Zabbix Agent 2 plugins.

What I’m Hoping For (Maybe 7.4, Maybe Later?)

Looking at the roadmap and based on some code movements I’ve seen, here are a couple of features I’m particularly excited about and hope to see soon, possibly even in 7.4:

  • Nested Low-Level Discovery (LLD): This would be a game-changer for dynamic environments. Imagine discovering databases, and then, as a sub-task, discovering the tables within each database using discovery prototypes derived from the parent discovery. This structured approach would simplify complex auto-discovery scenarios (databases, Kubernetes, cloud resources). I have a strong feeling this might make it into 7.4.
  • Event Correlation: My big dream feature! The ability to intelligently link related events, identifying the root cause (e.g., a failed switch) and suppressing the symptoms (all the hosts behind it becoming unreachable). This would significantly reduce alert noise and help focus on the real problem. It’s listed on the roadmap, but whether it lands in 7.4 remains to be seen.
  • Alternative Backend Storage: Also on the roadmap is exploring alternative backend solutions beyond traditional SQL databases (like potentially TimescaleDB alternatives, though not explicitly named). This is crucial groundwork for Zabbix 8.0 and beyond, especially for handling the massive data volumes associated with full observability (metrics, logs, traces).
  • New Host Wizard: A guided wizard for adding new hosts is also in development, which should improve the user experience.

Wrapping Up

Zabbix 7.4 is shaping up to be another solid release, bringing valuable improvements to maps, performance, usability, and monitoring capabilities. The map enhancements based on item values and the history cache improvements are particularly noteworthy from this Beta 1.

I’ll definitely keep you updated as we get closer to the final release and if features like Nested LLD or Event Correlation make the cut!

What do you think? Are these features useful for you? What are you hoping to see in Zabbix 7.4 or the upcoming Zabbix 8.0? Let me know in the comments below – I’m always curious to hear your thoughts and often pass feedback along (yes, I’m known for being persistent with the Zabbix team, like Alexey Vladyshev!).

Don’t forget to check out the Quadrata YouTube channel for more content like this.

And if you’re not already there, join the conversation in the ZabbixItalia Telegram Channel – it’s a great place to connect with other Italian Zabbix users.

That’s all for today. Thanks for reading, and I’ll catch you in the next one!

– Dimitri Bellini

Read More
Unlock Powerful Visualizations: Exploring Zabbix 7.0 Dashboards

Unlock Powerful Visualizations: Exploring Zabbix 7.0 Dashboards

Good morning everyone! Dimitri Bellini here, back with another episode on Quadrata, my channel dedicated to the world of open source and IT. Today, we’re diving back into our favorite monitoring tool, Zabbix, but focusing on an area we haven’t explored much recently: **data visualization and dashboarding**, especially with the exciting improvements in Zabbix 7.0.

For a long time, many of us (myself included!) might have leaned towards Grafana for sophisticated dashboards, and rightly so – it set a high standard. However, Zabbix has been working hard, taking inspiration from the best, and Zabbix 7.0 introduces some fantastic new widgets and capabilities that significantly boost its native dashboarding power, pushing towards better observability of our collected metrics.

Why Zabbix Dashboards Now Deserve Your Attention

Zabbix 7.0 marks a significant step forward in visualization. The web interface’s dashboard section has received substantial upgrades, introducing new widgets that make creating informative and visually appealing dashboards easier than ever. Forget needing a separate tool for basic visualization; Zabbix now offers compelling options right out of the box.

Some of the key additions in 7.0 include:

  • Gauge Widgets: For clear, immediate visualization of single metrics against thresholds.
  • Pie Chart / Donut Widgets: Classic ways to represent proportions.
  • On-Icon Widget: Excellent for compactly displaying the status of many items (like host availability).
  • Host Navigator & Item Navigator: Powerful tools for creating dynamic, interactive dashboards where you can drill down into specific hosts and their metrics.
  • Item Value Widget: Displays a single metric’s value with trend indication.

Building a Dynamic Dashboard in Zabbix 7.0: A Walkthrough

In the video, I demonstrated how to leverage some of these new features. Let’s recap the steps to build a more dynamic and insightful dashboard:

Step 1: Creating Your Dashboard

It all starts in the Zabbix interface under Dashboards -> All dashboards. Click “Create dashboard”, give it a meaningful name (I used “test per video”), and you’ll be presented with an empty canvas, ready for your first widget.

Step 2: Adding the Powerful Graph Widget

The standard graph widget, while not brand new, has become incredibly flexible.

  • Host/Item Selection: You can use wildcards (*) for both hosts (e.g., Linux server*) and items (e.g., Bits received*) to aggregate data from multiple sources onto a single graph.
  • Aggregation: Easily aggregate data over time intervals (e.g., show the sum or average traffic every 3 minutes).
  • Stacking: Use the “Stacked” option combined with aggregation to visualize total resource usage (like total bandwidth across multiple servers).
  • Multiple Datasets: Add multiple datasets (like ‘Bits received’ and ‘Bits sent’) to the same graph for comprehensive views.
  • Customization: Control line thickness, fill transparency, handling of missing data, axis limits (e.g., setting a max bandwidth), legend display, and even overlay trigger information or working hours.

This allows for creating dense, informative graphs showing trends across groups of systems or interfaces.

Step 3: Introducing Interactivity with Navigators

This is where Zabbix 7.0 dashboards get really dynamic!

Host Navigator Setup

Add the “Host Navigator” widget. Configure it to target a specific host group (e.g., Linux Servers). You can further filter by host status (enabled/disabled), maintenance status, or tags. This widget provides a clickable list of hosts.

Item Navigator Setup

Next, add the “Item Navigator” widget. The key here is to link it to the Host Navigator:

  • In the “Host” selection, choose “From widget” and select your Host Navigator widget.
  • Specify the host group again.
  • Use “Item tags” to filter the list of items shown (e.g., show only items with the tag component having the value network).
  • Use “Group by” (e.g., group by the component tag) to organize the items logically within the navigator. (Note: In the video, I noticed a slight confusion where the UI might label tag value filtering as tag name, something to keep an eye on).

Now, clicking a host in the Host Navigator filters the items shown in the Item Navigator – the first step towards interactive drill-down!

Step 4: Visualizing Single Metrics (Gauge & Item Value)

With the navigators set up, we can add widgets that react to our selections:

Gauge Widget

Add a “Gauge” widget. Configure its “Item” setting to inherit “From widget” -> “Item Navigator”. Now, when you select an item in the Item Navigator (after selecting a host), this gauge will automatically display that metric’s latest value. Customize it with:

  • Min/Max values and units (e.g., %, BPS).
  • Thresholds (defining ranges for Green, Yellow, Red) for instant visual feedback.
  • Appearance options (angles, decimals).

Item Value Widget

Similarly, add an “Item Value” widget, also inheriting its item from the “Item Navigator”. This provides a simple text display of the value, often with a trend indicator (up/down arrow). You can customize:

  • Font size and units.
  • Whether to show the timestamp.
  • Thresholds that can change the background color of the widget for high visibility.

Step 5: Monitoring Multiple Hosts with On-Icon

The “On-Icon” widget is fantastic for a compact overview of many similar items across multiple hosts.

  • Configure it to target a host group (e.g., Linux Servers).
  • Select a specific item pattern relevant to status (e.g., agent.ping).
  • Set thresholds (e.g., Red if value is 0, Green if value is 1) to color-code each host’s icon based on the item value.

This gives you an immediate “at-a-glance” view of the health or status of all hosts in the group regarding that specific metric. The icons automatically resize to fit the widget space.

Putting It All Together

By combining these widgets – graphs for trends, navigators for interactivity, and gauges/item values/on-icons for specific states – you can build truly powerful and informative dashboards directly within Zabbix 7.0. The ability to dynamically filter and drill down without leaving the dashboard is a massive improvement.

Join the Conversation!

So, that’s a first look at the enhanced dashboarding capabilities in Zabbix 7.0. There’s definitely a lot to explore, and these new tools significantly improve how we can visualize our monitoring data.

What do you think? Have you tried the new Zabbix 7.0 dashboards? Are there specific widgets or features you’d like me to cover in more detail? Let me know in the comments below!

If you found this useful, please give the video a thumbs up and consider subscribing to the Quadrata YouTube channel for more content on open source and IT.

And don’t forget to join the conversation in the ZabbixItalia Telegram Channel – it’s a great place to ask questions and share knowledge with fellow Zabbix users.

Thanks for reading, and I’ll see you in the next one!

– Dimitri Bellini

Read More
Zabbix 7.0 Synthetic Monitoring: A Game Changer for Web Performance Testing

Zabbix 7.0 Synthetic Monitoring: A Game Changer for Web Performance Testing

Good morning everyone, I’m Dimitri Bellini, and welcome back to Quadrata! This is my channel dedicated to the open-source world and the IT topics I’m passionate about – and hopefully, you are too.

In today’s episode, we’re diving back into our friend Zabbix, specifically version 7.0, to explore a feature that I genuinely believe is a game-changer in many contexts: Synthetic Monitoring. This powerful capability allows us to simulate and test complex user interaction scenarios on our websites, corporate pages, and web applications. Ready to see how it works? Let’s jump in!

What Exactly is Synthetic Monitoring in Zabbix 7.0?

As I briefly mentioned, synthetic monitoring is a method used to simulate how a real user interacts with a web page or application. This isn’t just about checking if a page loads; it’s about mimicking multi-step journeys – like logging in, navigating through menus, filling out forms, or completing a checkout process.

While this concept might seem straightforward, having it seamlessly integrated into a monitoring solution like Zabbix is incredibly valuable and not always a given in other tools. (1:09-1:16)

The Key Ingredients: Zabbix & Selenium

To make this magic happen, we need a couple of key components:

  • Zabbix Server or Proxy (Version 7.0+): This is our central monitoring hub.
  • Selenium: This is the engine that drives the browser simulation. I strongly recommend running Selenium within a Docker container, ideally on a machine separate from your Zabbix server for better resource management.

Selenium is a well-established framework (known for decades!) that allows us to automate browsers. One of its strengths is the ability to test using different browsers like Chrome, Edge, Firefox, and even Safari, ensuring your site works consistently across platforms. Zabbix interacts with Selenium via the WebDriver API, which essentially acts as a remote control for the browser, allowing us to send commands without writing complex, browser-specific code.

For our purposes, we’ll focus on the simpler Selenium Standalone setup, specifically using the Chrome browser container, as Zabbix currently has the most robust support for it.

How the Architecture Works

The setup is quite logical:

  1. Your Zabbix Server (or Zabbix Proxy) needs to know where the Selenium WebDriver is running.
  2. Zabbix communicates with the Selenium container (typically on port 4444) using the WebDriver protocol.
  3. Selenium receives instructions from Zabbix, executes them in a real browser instance (running inside the container), and sends the results back.

If you need to scale your synthetic monitoring checks, using Zabbix Proxies is an excellent approach. You can dedicate specific proxies to handle checks for different environments.

The Selenium Docker container also provides useful endpoints:

  • Port 4444: Besides being the WebDriver endpoint, it often hosts a web UI to view the status and current sessions.
  • Port 7900 (often): Provides a VNC/web interface to visually watch the browser automation in real-time – fantastic for debugging!

Setting Up Your Environment

Getting started involves a couple of configuration steps:

1. Zabbix Configuration

You’ll need to edit your zabbix_server.conf or zabbix_proxy.conf file and set these parameters:

  • WebDriverURL=http://:4444/wd/hub (Replace with the actual IP/DNS of your Selenium host)
  • StartBrowserPollers=1 (Start with 1 and increase based on workload)

Remember to restart your Zabbix server or proxy after making these changes.

2. Installing the Selenium Docker Container

Running the Selenium Standalone Chrome container is straightforward using Docker. Here’s a typical command:

docker run -d -p 4444:4444 -p 7900:7900 --shm-size="2g" --name selenium-chrome selenium/standalone-chrome:latest

  • -d: Run in detached mode.
  • -p 4444:4444: Map the WebDriver port.
  • -p 7900:7900: Map the VNC/debug view port.
  • --shm-size="2g": Allocate shared memory (important for browser stability, especially Chrome).
  • --name selenium-chrome: Give the container a recognizable name.
  • selenium/standalone-chrome:latest: The Docker image to use. You can specify older versions if needed.

Crafting Your Monitoring Scripts with JavaScript

The heart of synthetic monitoring in Zabbix lies in JavaScript. Zabbix utilizes its familiar JavaScript engine, now enhanced with a new built-in object: browser.

This browser object provides methods to interact with the web page via Selenium:

  • browser.Navigate('https://your-target-url.com'): Opens a specific URL.
  • browser.FindElement(by, target): Locates an element on the page. The by parameter can be methods like browser.By.linkText('Click Me'), browser.By.tagName('button'), browser.By.xpath('//div[@id="login"]'), browser.By.cssSelector('.submit-button').
  • element.Click(): Clicks on a previously found element.
  • browser.CollectPerfEntries(): Gathers performance metrics and, crucially, takes a screenshot of the current page state.

The output of these scripts is a JSON object containing performance data (response times, status codes, page weight) and the screenshot encoded in Base64.

Testing Your Scripts

You can test your JavaScript code before deploying it in Zabbix:

  • Zabbix Web Interface: The item configuration page has a “Test” button.
  • zabbix_js Command-Line Tool: Useful for quick checks and debugging.

    zabbix_js --webdriver http://:4444/wd/hub -i your_script.js -p 'dummy_input' -t 60

    (Remember to provide an input parameter -p even if your script doesn’t use it, and set a reasonable timeout -t. Piping the output to jq (| jq .) makes the JSON readable.

Visualizing the Results in Zabbix

Once your main “Browser” item is collecting data (including the Base64 screenshot), you can extract specific pieces of information using:

  • Dependent Items: Create items that depend on your main Browser item.
  • Preprocessing Steps: Use JSONPath preprocessing rules within the dependent items to pull out specific metrics (e.g., $.steps[0].responseTime).
  • Binary Item Type: Zabbix 7.0 introduces a binary item type specifically designed to handle data like Base64 encoded images.

This allows you to not only graph performance metrics but also display the actual screenshots captured during the check directly in your Zabbix dashboards using the new Item History widget. Seeing a visual snapshot of the page, especially when an error occurs, is incredibly powerful for troubleshooting!

Looking Ahead

Synthetic monitoring in Zabbix 7.0 is a fantastic addition, opening up new possibilities for ensuring application availability and performance from an end-user perspective.

Here at Quadrata, we’re actively exploring ways to make creating these JavaScript scenarios even easier for everyone. We might even have something to share at the Zabbix Summit this year, so stay tuned!

I hope this overview gives you a solid start with Zabbix 7’s synthetic monitoring. It’s a feature with immense potential.

What do you think? Have you tried it yet, or do you have specific use cases in mind? Let me know in the comments below!

If you found this video helpful, please give it a thumbs up and consider subscribing to the Quadrata channel for more open-source and IT content.

Don’t forget to join our Italian Zabbix community on Telegram: ZabbixItalia!

Thanks for watching, and see you next week. Bye everyone!

Read More
Visualizing Your Infrastructure: A Deep Dive into Zabbix Maps

Visualizing Your Infrastructure: A Deep Dive into Zabbix Maps

Good morning everyone, Dimitri Bellini here, and welcome back to Quadrata – your go-to channel for open source and IT solutions! Today, I want to dive into a feature of our good friend Zabbix that we haven’t explored much yet: Zabbix Maps.

Honestly, I was recently working on some maps, and while it might not always be the most glamorous part of Zabbix, it sparked an idea: why not share what these maps are truly capable of, and perhaps more importantly, what they aren’t?

What Zabbix Maps REALLY Are: Your Digital Synoptic Panel

Think of Zabbix Maps as the modern, digital equivalent of those old-school synoptic panels with blinking lights. They provide a powerful graphical way to represent your infrastructure and its status directly within Zabbix. Here’s what you can achieve:

  • Real-time Host Status: Instantly see the overall health of your hosts based on whether they have active problems.
  • Real-time Event Representation: Visualize specific problems (triggers) directly on the map. Imagine a specific light turning red only when a critical service fails.
  • Real-time Item Metrics: Display actual data values (like temperature, traffic throughput, user counts) directly on your map, making data much more intuitive and visually appealing.

The core idea is to create a custom graphical overview tailored to your specific infrastructure, giving you an immediate understanding of what’s happening at a glance.

Clearing Up Misconceptions: What Zabbix Maps Are NOT

It’s crucial to understand the limitations to use maps effectively. Often, people hope Zabbix Maps will automatically generate network topology diagrams.

  • They are NOT Automatic Network Topology Maps: While you *could* manually build something resembling a network diagram, Zabbix doesn’t automatically discover devices and map their connections (who’s plugged into which switch port, etc.). Tools that attempt this often rely on protocols like Cisco’s CDP or the standard LLDP (both usually SNMP-based), which aren’t universally available across all devices. Furthermore, in large environments (think thousands of hosts and hundreds of switches), automatically generated topology maps quickly become an unreadable mess of tiny icons and overlapping lines. They might look cool initially but offer little practical value day-to-day.
  • They are NOT Application Performance Monitoring (APM) Relationship Maps (Yet!): Zabbix Maps don’t currently visualize the intricate relationships and data flows between different application components in the way dedicated APM tools do. While Zabbix is heading towards APM capabilities, the current map function isn’t designed for that specific purpose.

For the nitty-gritty details, I always recommend checking the official Zabbix documentation – it’s an invaluable resource.

Building Blocks of a Zabbix Map

When constructing your map, you have several element types at your disposal:

  • Host: Represents a monitored device. Its appearance can change based on problem severity.
  • Trigger: Represents a specific problem condition. You can link an icon’s appearance directly to a trigger’s state.
  • Map: Allows you to create nested maps. The icon for a sub-map can reflect the most severe status of the elements within it – great for drilling down!
  • Image: Use custom background images or icons to make your map visually informative and appealing.
  • Host Group: Automatically display all hosts belonging to a specific group within a defined area on the map.
  • Shape: Geometric shapes (rectangles, ellipses) that can be used for layout, grouping, or, importantly, displaying text and real-time data.
  • Link: Lines connecting elements. These can change color or style based on a trigger’s status, often used to represent connectivity or dependencies.

Zabbix also provides visual cues like highlighting elements with problems or showing small triangles to indicate a recent status change, helping you focus on what needs attention.

Bringing Maps to Life with Real-Time Data

One of the most powerful features is embedding live data directly onto your map. Instead of just seeing if a server is “up” or “down,” you can see its current CPU load, network traffic, or application-specific metrics.

This is typically done using Shapes and a specific syntax within the shape’s label. In Zabbix 6.x and later, the syntax looks something like this:

{?last(/Your Host Name/your.item.key)}

This tells Zabbix to display the last received value for the item your.item.key on the host named Your Host Name. You can add descriptive text around it, like:

CPU Load: {?last(/MyWebServer/system.cpu.load[,avg1])}

Zabbix is smart enough to often apply the correct unit (like Bps, %, °C) automatically if it’s defined in the item configuration.

Let’s Build a Simple Map (Quick Guide)

Here’s a condensed walkthrough based on what I demonstrated in the video (using Zabbix 6.4):

  1. Navigate to Maps: Go to Monitoring -> Maps.
  2. Create New Map: Click “Create map”. Give it a name (e.g., “YouTube Test”), set dimensions, and optionally choose a background image.

    • Tip: You can upload custom icons and background images under Administration -> General -> Images. I uploaded custom red/green icons and a background for the demo.

  3. Configure Map Properties: Decide on options like “Icon highlighting” (the colored border around problematic hosts) and “Mark elements on trigger status change” (the triangles for recent changes). You can also filter problems by severity or hide labels if needed. Click “Add”.
  4. Enter Constructor Mode: Open your newly created map and click “Constructor”.
  5. Add a Trigger-Based Icon:

    • Click “Add element” (defaults to a server icon).
    • Click the new element. Change “Type” to “Trigger”.
    • Under “Icons”, select your custom “green” icon for the “Default” state and your “red” icon for the “Problem” state.
    • Click “Add” next to “Triggers” and select the specific trigger you want this icon to react to.
    • Click “Apply”. Position the icon on your map.

  6. Add Real-Time Data Display:

    • Click “Add element” and select “Shape” (e.g., Rectangle).
    • Click the new shape. In the “Label” field, enter your data syntax, e.g., Temp: {?last(/quadrata-test-host/test.item)} (replace with your actual host and item key).
    • Customize font size, remove the border (set Border width to 0), etc.
    • Click “Apply”. Position the shape.
    • Important: In the constructor toolbar, toggle “Expand macros” ON to see the live data instead of the syntax string.

  7. Refine and Save: Adjust element positions (you might want to turn off “Snap to grid” for finer control). Remove default labels if they clutter the view (Map Properties -> Map element label type -> Nothing). Click “Update” to save your changes.

Testing with `zabbix_sender`

A fantastic tool for testing maps (especially with trapper items) is the zabbix_sender command-line utility. It lets you manually push data to Zabbix items.

Install the `zabbix-sender` package if you don’t have it. The basic syntax is:

zabbix_sender -z -s -k -o

For example:

zabbix_sender -z 192.168.1.100 -s quadrata-test-host -k test.item -o 25

Sending a value that crosses a trigger threshold will change your trigger-linked icon on the map. Sending a different value will update the real-time data display.

Wrapping Up

So, there you have it – a look into Zabbix Maps. They aren’t magic topology generators, but they are incredibly flexible and powerful tools for creating meaningful, real-time visual dashboards of your infrastructure’s health and performance. By combining different elements, custom icons, backgrounds, and live data, you can build truly informative synoptic views.

Don’t be afraid to experiment! Start simple and gradually add complexity as you get comfortable.

What are your thoughts on Zabbix Maps? Have you created any cool visualizations? Share your experiences or ask questions in the comments below!

If you found this helpful, please give the video a thumbs up, share it, and subscribe to Quadrata for more content on Zabbix and open source solutions.

Also, feel free to join the conversation in the Zabbix Italia Telegram channel – it’s a great community!

Thanks for reading, and I’ll see you in the next post!

– Dimitri Bellini

Read More
Deep Dive into Zabbix Database Monitoring with ODBC

Deep Dive into Zabbix Database Monitoring with ODBC

Good morning everyone, and welcome back to my channel, Quadrata! I’m Dimitri Bellini, and this week, we’re diving deep into Zabbix again. Why? Because many of you in our fantastic Zabbix Italia Telegram channel asked for a closer look at database monitoring with Zabbix.

Zabbix offers a few ways to monitor databases – using Zabbix Agent 2, User Parameters, and ODBC. While Agent 2 is great for standard metrics and User Parameters offer script-based flexibility (which I personally enjoy!), today we’re focusing on what I consider the most elegant and powerful method for many scenarios: **ODBC (Open Database Connectivity)**.

What is ODBC and Why Use It with Zabbix?

ODBC is a standard API (Application Programming Interface) born years ago, designed to allow applications to communicate with various databases without needing to know the specifics of each database system. Think of it as a universal translator for databases.

Here’s the basic idea:

  • Your application (in our case, Zabbix Server or Proxy) talks to the ODBC layer using standard SQL commands.
  • The ODBC layer uses specific **drivers** for each database type (MySQL, Oracle, Postgres, etc.).
  • These drivers handle the native communication with the target database.

This decoupling means Zabbix doesn’t need built-in drivers for every database, making it more flexible. On Linux systems, the common implementation is UnixODBC.

Setting Up ODBC for Zabbix

Installation (Linux Example)

Getting UnixODBC installed is usually straightforward as it’s often in standard repositories. For Red Hat-based systems (like Rocky Linux, AlmaLinux), you’d typically run:

dnf install unixodbc unixodbc-devel

(The `devel` package might not always be necessary, but can be helpful).

After installing UnixODBC, you need the specific ODBC driver for the database you want to monitor.

Crucial Driver Note for MySQL/MariaDB

Important! If you’re monitoring MySQL or MariaDB with Zabbix, you currently need to use the MariaDB ODBC connector. Due to licensing complexities, Zabbix Server/Proxy binaries are often compiled against the MariaDB libraries.

Install it like this (on Red Hat-based systems):

dnf install MariaDB-connector-odbc

While the official MySQL ODBC driver might work fine with command-line tools like `isql`, Zabbix itself likely won’t be able to use it directly. Stick with the MariaDB connector for compatibility.

Configuration Files: odbcinst.ini and odbc.ini

UnixODBC typically uses two main configuration files:

  • odbcinst.ini: Defines the available drivers, giving them an alias and pointing to the driver library file.
  • odbc.ini: Defines Data Source Names (DSNs). A DSN is a pre-configured connection profile containing the driver alias, server address, port, user, password, and database name.

Two Approaches to Connection: DSN vs. Connection String

1. The DSN Method (Using odbc.ini)

You can define all your connections in odbc.ini and then simply refer to the DSN name in Zabbix. You can test this setup from the command line using the isql utility:

isql your_dsn_name

If it connects successfully, you’ll get a prompt, confirming your ODBC setup works.

However, I personally find this method less flexible for Zabbix. Managing static entries in odbc.ini for potentially hundreds of databases can become cumbersome.

2. The Connection String Method (My Preferred Way!)

Instead of relying on odbc.ini for connection details, you can provide all the necessary information directly within Zabbix using a **connection string**. This bypasses the need for DSN entries in odbc.ini (though odbcinst.ini is still needed to define the driver itself).

You can test this approach from the command line too:

isql -k "Driver=YourDriverAlias;Server=your_db_host;Port=3306;User=your_user;Password=your_password;Database=your_db;"

(Replace the placeholders with your actual details. The exact parameters might vary slightly depending on the driver).

This method offers much greater flexibility, especially when combined with Zabbix templates and macros, as we’ll see.

Configuring Zabbix for ODBC Monitoring

Creating the Zabbix Item

To monitor a database using ODBC in Zabbix:

  1. Create a new item for your host.
  2. Set the **Type** to Database monitor.
  3. The **Key** format is crucial: db.odbc.select[,].

    • : A unique name for this specific check within the host (e.g., `mysql_version`, `user_count`). This ensures the key is unique.
    • : Here you either put your DSN name (if using odbc.ini) OR the full connection string (if bypassing odbc.ini).

  4. If using the connection string method, you leave the DSN part empty but include the connection string within the key’s parameters, often enclosed in quotes if needed, or directly if simple. *Correction from video explanation: The Zabbix key structure is slightly different. It’s `db.odbc.select[unique_name,dsn]` or `db.odbc.select[unique_name,,connection_string]`. Notice the double comma when omitting the DSN.*
  5. The most important field is the **SQL query** field. This is where you put the actual SQL query you want Zabbix to execute.
  6. You can optionally provide Username and Password in the dedicated fields, which might override or complement details in the DSN/connection string depending on the driver and configuration.
  7. Set the **Type of information** based on the expected query result (Numeric, Text, etc.).

Example with Connection String

Here’s how an item key might look using the connection string method (notice the empty DSN parameter indicated by the double comma):

db.odbc.select[CountZabbixTables,, "Driver=MariaDB;Server=127.0.0.1;Port=3306;Database=zabbix;User={$ZABBIX_DB_USER};Password={$ZABBIX_DB_PASSWORD};"]

And in the **SQL query** field for this item, I might put:

SELECT count(*) FROM information_schema.tables WHERE table_schema = 'zabbix';

Leveraging User Macros for Flexibility

The real power of the connection string method shines when you use Zabbix User Macros (like {$ZABBIX_DB_USER}, {$ZABBIX_DB_PASSWORD}, {$DB_HOST}, {$DB_NAME}) within the string. This allows you to create generic templates and customize the connection details per host via macros – incredibly useful for large or complex environments!

ODBC for Custom Queries vs. Agent 2 for System Health

It’s important to understand the typical use cases:

  • Standard Zabbix Templates (Agent 2 or ODBC): These usually focus on monitoring the *health and performance* of the database system itself (e.g., queries per second, buffer usage, connection counts, uptime).
  • Manual ODBC Items (like we’re discussing): This method is **perfect for running custom SQL queries**. Need to check the number of rows in a specific table? Verify if a critical configuration value exists? Confirm application data is being populated correctly? ODBC monitoring configured this way is your go-to solution.

While you *could* potentially use Agent 2 with User Parameters and scripts for custom queries, ODBC often provides a cleaner, more centralized, and integrated way to achieve this directly from the Zabbix Server or Proxy.

Putting It All Together: A Practical Example

In the video, I demonstrated creating an item to count the number of tables in my Zabbix database using the connection string method. The key steps were:

  1. Define the item with Type `Database monitor`.
  2. Construct the key using `db.odbc.select` and the connection string.
  3. Enter the `SELECT count(*)…` query in the SQL query field.
  4. Use the fantastic **Test** button in Zabbix! This lets you immediately check if the connection works and the query returns the expected data (in my case, 173 tables) without waiting for the item’s update interval.

This confirms the connection from Zabbix to the database via ODBC is working correctly.

Troubleshooting Tips

If things don’t work right away (which happens!), follow these steps:

  • Test Connectivity First: Always use the `isql` command-line tool (either with the DSN or the `-k “connection_string”`) to verify basic ODBC connectivity *before* configuring Zabbix.
  • Check Logs: Examine the Zabbix Server or Zabbix Proxy logs for detailed error messages related to database monitors.
  • Consult Documentation: The official Zabbix documentation has a dedicated section on ODBC monitoring.
  • Verify Driver Path: Ensure the driver path specified in `odbcinst.ini` is correct.
  • Permissions: Make sure the database user configured for Zabbix has the necessary permissions to connect and execute the query.
  • Take a Breath: Rushing leads to mistakes. Double-check configurations, read errors carefully, and approach it methodically.

Conclusion and Next Steps

ODBC monitoring in Zabbix is a highly flexible and powerful tool, especially when you need to go beyond standard system metrics and execute custom SQL queries to validate data or check specific application states. While the initial setup requires careful attention to drivers and connection details, the connection string method combined with user macros offers excellent scalability.

What are your experiences with Zabbix database monitoring? Do you prefer ODBC, Agent 2, or User Parameters? Share your thoughts and questions in the comments below!

If you found this helpful, please give the video a thumbs up and subscribe to Quadrata for more Zabbix content. Don’t forget to join our Zabbix Italia Telegram channel!

Stay tuned – with Zabbix 6.4 just around the corner, I’ll likely be covering the “What’s New” very soon!

Thanks again for watching and reading. Happy monitoring!

Ciao,
Dimitri Bellini

Read More
Taming the SNMP Beast: Custom Monitoring with Zabbix Discovery

Taming the SNMP Beast: Custom Monitoring with Zabbix Discovery

Good morning everyone, and welcome back! It’s Dimitri Bellini here on Quadrata, your channel for open-source tech insights. Today, we’re diving back into the world of Zabbix, specifically tackling what many consider a ‘black beast’: SNMP monitoring.

SNMP (Simple Network Management Protocol) is incredibly common for monitoring network devices like routers and switches (think Mikrotik, Cisco), but it’s also used for applications and servers running SNMP daemons. Zabbix comes packed with pre-built templates for many SNMP devices, which makes life easy – apply the template, set a few parameters, and data starts flowing. But what happens when you have a device with no template, or you only have the manufacturer’s MIB file? That’s when things get trickier, and you need to build your monitoring from scratch.

In this post, I want to share my approach to creating new SNMP monitoring templates or discovering SNMP data when starting from zero. Let’s demystify this process together!

Understanding the SNMP Basics: MIBs and OIDs

Before jumping into Zabbix, we need to understand what we *can* monitor. This information lives in MIB (Management Information Base) files provided by the device vendor. These files define the structure of manageable data using OIDs (Object Identifiers) – unique numerical addresses for specific metrics or pieces of information.

Your Essential Toolkit: MIB Browser and snmpwalk

To explore these MIBs and test SNMP communication, I rely on a couple of key tools:

  • MIB Browser: I often use the iReasoning MIB Browser. It’s free, multi-platform (Java-based), and lets you load MIB files visually. You can navigate the OID tree, see descriptions, data types, and even potential values (which helps later with Zabbix Value Maps). For example, you can find the OID for interface operational status (ifOperStatus) and see that ‘1’ means ‘up’, ‘2’ means ‘down’, etc.
  • snmpwalk: This command-line utility (part of standard SNMP tools on Linux) lets you query a device directly. It’s crucial for verifying that the device responds and seeing the actual data returned for a specific OID.

Finding Your Way with OIDs

Let’s say we want to monitor network interfaces on a device (like the pfSense appliance I use in the video). Using the MIB browser, we find the OID for interface descriptions, often IF-MIB::ifDescr. We can then test this with snmpwalk:

snmpwalk -v2c -c public 192.168.1.1 IF-MIB::ifDescr

(Replace public with your device’s SNMP community string and 192.168.1.1 with its IP address. We’re using SNMP v2c here for simplicity, though v3 offers better security).

This command might return something like:


IF-MIB::ifDescr.1 = STRING: enc0

IF-MIB::ifDescr.2 = STRING: ovpns1

...

Sometimes, especially when Zabbix might not have the MIB loaded, it’s easier to work with the full numerical OID. Use the -On flag:

snmpwalk -v2c -c public -On 192.168.1.1 1.3.6.1.2.1.2.2.1.2

This will output the full numerical OIDs, like .1.3.6.1.2.1.2.2.1.2.1, .1.3.6.1.2.1.2.2.1.2.2, etc.

The Power of SNMP Indexes

Notice the numbers at the end of the OIDs (.1, .2)? These are **indexes**. SNMP often organizes data in tables. Think of it like a spreadsheet: each row represents an instance (like a specific interface or disk), identified by its index. Different columns represent different metrics (like description, status, speed, octets in/out) for that instance.

So, ifDescr.1 is the description for interface index 1, and ifOperStatus.1 (OID: .1.3.6.1.2.1.2.2.1.8.1) would be the operational status for that *same* interface index 1. This index is the key to correlating different pieces of information about the same logical entity.

Automating with Zabbix Low-Level Discovery (LLD)

Manually creating an item in Zabbix for every single interface and every metric (status, traffic in, traffic out…) is tedious and static. If a new interface appears, you have to add it manually. This is where Zabbix’s Low-Level Discovery (LLD) shines for SNMP.

LLD allows Zabbix to automatically find entities (like interfaces, disks, processors) based on SNMP indexes and then create items, triggers, and graphs for them using prototypes.

Setting Up Your Discovery Rule

Let’s create a discovery rule for network interfaces:

  1. Go to Configuration -> Templates -> Your Template -> Discovery rules -> Create discovery rule.
  2. Name: Something descriptive, e.g., “Network Interface Discovery”.
  3. Type: SNMP agent.
  4. Key: A unique key you define, e.g., net.if.discovery.
  5. SNMP OID: This is the core. Use the Zabbix discovery syntax: discovery[{#MACRO_NAME1}, OID1, {#MACRO_NAME2}, OID2, ...].

    • Zabbix automatically provides {#SNMPINDEX} representing the index found.
    • We define custom macros to capture specific values. For interface names, we can use {#IFNAME}.

    So, to discover interface names based on their index, the OID field would look like this:
    discovery[{#IFNAME}, 1.3.6.1.2.1.2.2.1.2]
    (Using the numerical OID for ifDescr).

  6. Configure other settings like update interval as needed.

Zabbix will periodically run this rule, perform an SNMP walk on the specified OID (ifDescr), and generate a list mapping each {#SNMPINDEX} to its corresponding {#IFNAME} value.

Pro Tip: Use the “Test” button in the discovery rule configuration! It’s incredibly helpful to see the raw data Zabbix gets and the JSON output with your macros populated before saving.

Creating Dynamic Items with Prototypes

Now that Zabbix can discover the interfaces, we need to tell it *what* to monitor for each one using Item Prototypes:

  1. Within your discovery rule, go to the “Item prototypes” tab -> Create item prototype.
  2. Name: Use the macros found by discovery for dynamic naming, e.g., Interface {#IFNAME}: Operational Status.
  3. Type: SNMP agent.
  4. Key: Must be unique per host. Use a macro to ensure this, e.g., net.if.status[{#SNMPINDEX}].
  5. SNMP OID: Specify the OID for the metric you want, appending the index macro. For operational status (ifOperStatus, numerical OID .1.3.6.1.2.1.2.2.1.8), use: 1.3.6.1.2.1.2.2.1.8.{#SNMPINDEX}. Zabbix will automatically replace {#SNMPINDEX} with the correct index (1, 2, 3…) for each discovered interface.
  6. Type of information: Numeric (unsigned) for status codes.
  7. Units: N/A for status.
  8. Value mapping: Select or create a value map that translates the numerical status (1, 2, 3…) into human-readable text (Up, Down, Testing…). This uses the information we found earlier in the MIB browser.
  9. Configure other settings like update interval, history storage, etc.

Once saved, Zabbix will use this prototype to create an actual item for each interface discovered by the LLD rule. If a new interface appears on the device, Zabbix will discover it and automatically create the corresponding status item!

A Practical Example: Monitoring Disk Storage

We can apply the same logic to other SNMP data, like disk storage. In the video, I showed discovering disk types and capacities on my pfSense box.

Discovering Disk Types with Preprocessing

I created another discovery rule targeting the OID for storage types (e.g., from the HOST-RESOURCES-MIB or a vendor-specific MIB). This OID often returns numbers (like 3 for Hard Disk, 5 for Optical Disk).

To make the discovered macro more readable (e.g., {#DISKTYPE}), I used Zabbix’s **Preprocessing** feature within the discovery rule itself:

  • Add a preprocessing step of type “Replace”.
  • Find `^5$` (regex for the number 5) and replace with `Optical Disk`.
  • Add another step to find `^3$` and replace with `Hard Disk`.

Now, the {#DISKTYPE} macro will contain “Hard Disk” or “Optical Disk” instead of just a number.

Monitoring Disk Capacity with Unit Conversion

Then, I created an item prototype for disk capacity:

  • Name: `Disk {#DISKTYPE}: Capacity`
  • Key: `storage.capacity[{#SNMPINDEX}]`
  • SNMP OID: `[OID_for_storage_size].{#SNMPINDEX}`
  • Units: `B` (Bytes)
  • Preprocessing (in the Item Prototype): The SNMP device reported capacity in Kilobytes (or sometimes in allocation units * block size). To normalize it to Bytes, I added a “Custom multiplier” preprocessing step with a value of `1024`.

Putting It All Together

By combining MIB exploration, `snmpwalk` testing, Zabbix LLD rules with custom macros, and item prototypes with appropriate OIDs and preprocessing, you can build powerful, dynamic SNMP monitoring for almost any device, even without off-the-shelf templates.

It might seem a bit daunting initially, especially understanding the OID structure and LLD syntax, but once you grasp the concept of indexes and macros, it becomes quite manageable. The key is to break it down: find the OIDs, test them, set up discovery, and then define what data to collect via prototypes.


I hope this walkthrough helps demystify custom SNMP monitoring in Zabbix! It’s a powerful skill to have when dealing with diverse infrastructure.

What are your biggest challenges with SNMP monitoring? Have you built custom LLD rules? Share your experiences, questions, or tips in the comments below! I’ll do my best to answer any doubts you might have.

And if you have more Zabbix questions, feel free to join the Italian Zabbix community on Telegram: Zabbix Italia.

If you found this post helpful, please give the original video a ‘Like’ on YouTube, share this post, and subscribe to Quadrata for more open-source and Zabbix content.

Thanks for reading, and see you next week!

– Dimitri Bellini

Read More
Demystifying AI in Zabbix: Can AI Correlate Events?

Demystifying AI in Zabbix: Can AI Really Correlate Events?

Good morning, everyone! Dimitri Bellini here, back with you on Quadrata, my YouTube channel dedicated to the open-source world and the IT topics I’m passionate about. This week, I wanted to tackle a question that I, and many members of the Zabbix community, get asked all the time: Why doesn’t Zabbix have more built-in AI?

It seems like every monitoring product out there is touting its AI capabilities, promising to solve all your problems with a touch of magic. But is it all hype? My colleagues and I have been digging deep into this, exploring whether an AI engine can truly correlate events within Zabbix and make our lives easier. This blog post, based on my recent video, will walk you through our thought process.

The AI Conundrum: Monitoring Tools and Artificial Intelligence

Let’s be honest: integrating AI into a monitoring tool isn’t a walk in the park. It requires time, patience, and a willingness to experiment with different technologies. More importantly, it demands a good dose of introspection to understand how all the pieces of your monitoring setup fit together. But why even bother?

Anyone who’s managed a complex IT environment knows the struggle. You can be bombarded with hundreds, even thousands, of alerts every single day. Identifying the root cause and prioritizing issues becomes a monumental task, even for seasoned experts. Severity levels help, but they often fall short.

Understanding the Challenges

Zabbix gives us a wealth of metrics – CPU usage, memory consumption, disk space, and more. We typically use these to create triggers and set alarm thresholds. However, these metrics, on their own, often don’t provide enough context when a problem arises. Here are some key challenges we face:

  • Limited Metadata: Event information and metadata, like host details, aren’t always comprehensive enough. We often need to manually enrich this data.
  • Lack of Visibility: Monitoring teams often lack a complete picture of what’s happening across the entire organization. They might not know the specific applications running on a host or the impact of a host failure on the broader ecosystem.
  • Siloed Information: In larger enterprises, different departments (e.g., operating systems, databases, networks) might operate in silos, hindering the ability to connect the dots.
  • Zabbix Context: While Zabbix excels at collecting metrics and generating events, it doesn’t automatically discover application dependencies. Creating custom solutions to address this is possible but can be complex.

Our Goals: Event Correlation and Noise Reduction

Our primary goal is to improve event correlation using AI. We want to:

  • Link related events together.
  • Reduce background noise by filtering out less important alerts.
  • Identify the true root cause of problems, even when buried beneath a mountain of alerts.

Possible AI Solutions for Zabbix

So, what tools can we leverage? Here are some solutions we considered:

  • Time Correlation: Analyzing the sequence of events within a specific timeframe to identify relationships.
  • Host and Host Group Proximity: Identifying correlations based on the physical or logical proximity of hosts and host groups.
  • Semantic Similarities: Analyzing the names of triggers, tags, and hosts to find connections based on their meaning.
  • Severity and Tag Patterns: Identifying correlations based on event severity and patterns in tags.
  • Metric Pattern Analysis: Analyzing how metrics evolve over time to identify patterns associated with specific problems.

Leveraging scikit-learn

One promising solution we explored involves using scikit-learn, an open-source machine learning library. Our proposed pipeline looks like this:

  1. Event Processing: Collect events from our Zabbix server using streaming capabilities.
  2. Encoding Events: Use machine learning techniques to vectorize and transform events into a usable format.
  3. Cluster Creation: Apply algorithms like DBSCAN to create clusters of related events (e.g., network problems, operating system problems).
  4. Merging Clusters: Merge clusters based on identified correlations.

A Simple Example

Imagine a scenario where a router interface goes down and host B becomes unreachable. It’s highly likely that the router issue is the root cause, and host B’s unreachability is a consequence.

Implementation Steps

To implement this solution, we suggest a phased approach:

  1. Temporal Regrouping: Start by grouping events based on their timing.
  2. Host and Group Context: Add context by incorporating host and host group information.
  3. Semantic Analysis: Include semantic analysis of problem names to identify connections.
  4. Tagging: Enrich events with tags to define roles and provide additional information.
  5. Iterated Feedback: Gather feedback from users to fine-tune the system and improve its accuracy.
  6. Scaling Considerations: Optimize data ingestion and temporal window size based on Zabbix load.

Improvements Using Existing Zabbix Features

We can also leverage existing Zabbix features:

  • Trigger Dependencies: Utilize trigger dependencies to define static relationships.
  • Low-Level Discovery: Use low-level discovery to gather detailed information about network interfaces and connected devices.
  • Enriched Tagging: Encourage users to add more informative tags to events.

The Reality Check: It’s Not So Simple

While the theory sounds great, real-world testing revealed significant challenges. The timing of events in Zabbix can be inconsistent due to update intervals and threshold configurations. This can create temporary discrepancies and make accurate correlation difficult.

Consider this scenario:

  • File system full
  • CRM down
  • DB instance down
  • Unreachable host

A human might intuitively understand that a full file system could cause a database instance to fail, which in turn could bring down a CRM application. However, a machine learning algorithm might struggle to make these connections without additional context.

Exploring Large Language Models (LLMs)

To address these limitations, we explored using Large Language Models (LLMs). LLMs have the potential to understand event descriptions and make connections based on their inherent knowledge. For example, an LLM might know that a CRM system typically relies on a database, which in turn requires a file system.

However, even with LLMs, challenges remain. Identifying the root cause versus the symptoms can be tricky, and LLMs might not always accurately correlate events. Additionally, using high-end LLMs in the cloud can be expensive, while local models might not provide sufficient accuracy.

Conclusion: The Complex Reality of AI in Monitoring

In conclusion, integrating AI into Zabbix for event correlation is a complex challenge. A one-size-fits-all solution is unlikely to be effective. Tailoring the solution to the specific needs of each client is crucial. While LLMs offer promise, the cost and complexity of using them effectively remain significant concerns.

We’re continuing to explore this topic and welcome your thoughts and ideas!

Let’s Discuss!

What are your thoughts on using AI in monitoring? Have you had any success with similar approaches? Share your insights in the comments below or join the conversation on the ZabbixItalia Telegram Channel! Let’s collaborate and find new directions for our reasoning.

Thanks for watching! See you next week!

Bye from Dimitri!

Watch the original video: Quadrata Youtube Channel

Read More
Automate Your Zabbix Reporting with Scheduled Reports: A Step-by-Step Guide

Automate Your Zabbix Reporting with Scheduled Reports: A Step-by-Step Guide

Hey everyone, Dimitri Bellini here from Quadrata, your go-to channel for open source and IT insights! It’s fantastic to have you back with me. If you’re enjoying the content and haven’t subscribed yet, now’s a great time to hit that button and help me bring you even more valuable videos. 😉

Today, we’re diving deep into a Zabbix feature that’s been around for a while but is now truly shining – Scheduled Reports. Recently, I’ve been getting a lot of questions about this from clients, and it made me realize it’s time to shed light on this often-overlooked functionality. So, let’s talk about automating those PDF reports from your Zabbix dashboards.

Why Scheduled Reports? The Power of Automated Insights

Scheduled reports might not be brand new to Zabbix (they’ve been around since version 5.2!), but honestly, I wasn’t completely sold on them until recently. In older versions, they felt a bit… incomplete. But with Zabbix 7 and especially 7.2, things have changed dramatically. Now, in my opinion, scheduled reports are becoming a genuinely useful tool.

What are we talking about exactly? Essentially, scheduled reports are a way to automatically generate PDFs of your Zabbix dashboards and have them emailed to stakeholders – think bosses, team leads, or anyone who needs a regular overview without logging into Zabbix directly. We all know that stakeholder, right? The one who wants to see a “green is good” PDF report every Monday morning (or Friday afternoon!). While dashboards are great for real-time monitoring, scheduled reports offer that convenient, digestible summary for those who need a quick status update.

Sure, everyone *could* log into Zabbix and check the dashboards themselves. But let’s be real, sometimes pushing the information directly to them in a clean, professional PDF format is just more efficient and impactful. And that’s where Zabbix Scheduled Reports come in!

Key Features of Zabbix Scheduled Reports

Let’s break down the main advantages of using scheduled reports in Zabbix:

    • Automation: Define parameters to automatically send specific dashboards on a schedule (daily, weekly, monthly) to designated users.
    • Customization: Leverage your existing Zabbix dashboards. The reports are generated directly from the dashboards you design with widgets.
    • PDF Format: Reports are generated in PDF, the universally readable and versatile format.
    • Access Control: Control who can create and manage scheduled reports using user roles and permissions within Zabbix (Admin and Super Admin roles with specific flags).

For more detailed information, I highly recommend checking out the official Zabbix documentation and the Zabbix blog post about scheduled reports. I’ll include links in the description below for your convenience!

Setting Up Zabbix Scheduled Reports: A Step-by-Step Guide

Ready to get started? Here’s how to set up scheduled reports in Zabbix. Keep in mind, this guide is based on a simplified installation for demonstration purposes. For production environments, always refer to the official Zabbix documentation for best practices and advanced configurations.

Prerequisites

Before we begin, make sure you have the following:

    • A running Zabbix server (version 7.0 or higher recommended, 7.2+ for the best experience).
    • Configured dashboards in Zabbix that you want to use for reports.
    • Email media type configured in Zabbix for sending reports.

Installation of Zabbix Web Service and Google Chrome

The magic behind Zabbix scheduled reports relies on a separate component: Zabbix Web Service. This service handles the PDF generation and needs to be installed separately. It also uses Google Chrome (or Chromium) in headless mode to take screenshots of your dashboards and convert them to PDF.

Here’s how to install them on a Red Hat-based system (like Rocky Linux) using YUM/DNF:

    1. Install Zabbix Web Service:
      sudo yum install zabbix-web-service

      Make sure you have the official Zabbix repository configured.

    1. Install Google Chrome Stable:
      sudo yum install google-chrome-stable

      This will install Google Chrome and its dependencies. Be aware that Chrome can pull in quite a few dependencies, which is why installing the web service on a separate, smaller machine can be a good idea for cleaner Zabbix server environments.

Configuring Zabbix Server

Next, we need to configure the Zabbix server to enable scheduled reports and point it to the web service.

    1. Edit the Zabbix Server Configuration File:
      sudo vi /etc/zabbix/zabbix_server.conf
    1. Modify the following parameters:
        • StartReportWriters=1 (Change from 0 to 1 or more, depending on your reporting needs. Start with 1 for testing.)
        • WebServiceURL="http://localhost:10053/report" (Adjust the IP address and port if your web service is running on a different machine or port. 10053 is the default port for Zabbix Web Service).
    1. Restart Zabbix Server:
      sudo systemctl restart zabbix-server
    1. Start Zabbix Web Service:
      sudo systemctl start zabbix-web-service
    1. Enable Zabbix Web Service to start on boot:
      sudo systemctl enable zabbix-web-service

Configuring Zabbix Frontend

One last crucial configuration step in the Zabbix web interface!

    1. Navigate to Administration -> General -> GUI.
    1. Modify “Frontend URL”: Set this to the full URL of your Zabbix frontend (e.g., http://your_zabbix_server_ip/zabbix). This is essential for Chrome to access the dashboards correctly for PDF generation.
    1. Click “Update”.

Creating a Scheduled Report

Now for the fun part – creating your first scheduled report!

    1. Go to Reports -> Scheduled reports.
    1. Click “Create scheduled report”.
    1. Configure the report:
        • Name: Give your report a descriptive name (e.g., “Weekly Server Health Report”).
        • Dashboard: Select the dashboard you want to use for the report.
        • Period: Choose the time period for the report data (e.g., “Previous week”).
        • Schedule: Define the frequency (daily, weekly, monthly), time, and start/end dates for report generation.
        • Recipients: Add users or user groups who should receive the report via email. Make sure they have email media configured!
        • Generated report by: Choose if the report should be generated based on the permissions of the “Current user” (the admin creating the report) or the “Recipient” of the report.
        • Message: Customize the email message that accompanies the report (you can use Zabbix macros here).
    1. Click “Add”.

Testing and Troubleshooting

To test your setup, you can use the “Test” button next to your newly created scheduled report. If you encounter issues, double-check:

    • Email media configuration for recipients.
    • Zabbix Web Service and Google Chrome installation.
    • Zabbix server and web service configuration files.
    • Frontend URL setting.
    • Permissions: In the video, I encountered a permission issue related to the /var/lib/zabbix directory. You might need to create this directory and ensure the Zabbix user has write permissions if you face similar errors. sudo mkdir /var/lib/zabbix && sudo chown zabbix:zabbix /var/lib/zabbix

Why Zabbix 7.x Makes a Difference

I really started to appreciate scheduled reports with Zabbix 7.0 and 7.2. Why? Because these versions brought significant improvements:

    • Multi-page Reports: Finally, reports can span multiple pages, making them much more comprehensive.
    • Enhanced Dashboard Widgets: Zabbix 7.x introduced richer widgets like Top Hosts, Top Items, Pie charts, and Donut charts. These make dashboards (and therefore reports) far more visually appealing and informative.
    • Custom Widgets: With the ability to create custom widgets, you can tailor your dashboards and reports to very specific needs.

These enhancements make scheduled reports in Zabbix 7.x and above a truly valuable tool for delivering insightful and professional monitoring summaries.

Conclusion

Zabbix Scheduled Reports are a fantastic way to automate the delivery of key monitoring insights to stakeholders. While they’ve been around for a while, the improvements in Zabbix 7.x have made them significantly more powerful and user-friendly. Give them a try, experiment with your dashboards, and start delivering automated, professional PDF reports today!

I hope you found this guide helpful! If you did, please give this post a thumbs up (or share!) and let me know in the comments if you have any questions or experiences with Zabbix Scheduled Reports. Don’t forget to subscribe to Quadrata for more open source and IT tips and tricks.

And if you’re in the Zabbix community, be sure to join the ZabbixItalia Telegram channel – a great place to connect with other Zabbix users and get your questions answered. A big thank you for watching, and I’ll see you in the next video!

Bye from Dimitri!

P.S. Keep exploring Zabbix – there’s always something new and cool to discover!


Keywords: Zabbix, Scheduled Reports, PDF Reports, Automation, Dashboards, Monitoring, IT Reporting, Zabbix Web Service, Google Chrome, Tutorial, Guide, Dimitri Bellini, Quadrata, Zabbix 7.2, Zabbix 7.0, Open Source, IT Infrastructure, System Monitoring

Read More