Posts Taged open-source-monitoring

Home Posts Tagged "Open Source Monitoring"
NetBox and Zabbix: Creating the Ultimate Source of Truth for Your IT Infrastructure
By Dimitri Bellini29 October, 2025Blog

NetBox and Zabbix: Creating the Ultimate Source of Truth for Your IT Infrastructure

Good morning everyone! Dimitri Bellini here, and welcome back to Quadrata, my channel dedicated to the world of open source and all the IT that I, and hopefully you, find fascinating.

This week, we’re diving into a powerful solution that addresses a common and persistent challenge in IT management: the lack of a single, reliable source of information. In large infrastructures, it’s often difficult to know what objects exist, where they are, what their relationships are, and how to even begin monitoring them. Many turn to a Configuration Management Database (CMDB), but keeping it manually updated is a struggle. What if we could automate this process?

That’s where a fantastic open-source project called NetBox comes in. And thanks to our friends at Open Source ICT Solutions, there’s a brilliant integration that connects it directly to Zabbix. Let’s explore how to build a true “source of truth” for our network.

What is NetBox and Why Do You Need It?

For those who may not know it, NetBox is a well-established and solid open-source tool designed to be the central repository for your entire IT environment. It’s more than just a spreadsheet; it’s a structured database for everything from your network devices to your data center cabling. It’s designed to be the single source of truth.

NetBox helps you model and document your infrastructure with incredible detail. Its core functionalities include:

  • IP Address Management (IPAM): A robust module for managing all your IP spaces, prefixes, and addresses.

  • Data Center Infrastructure Management (DCIM): Model your physical infrastructure, including data center layouts, rack elevations, and the exact placement of devices within them.

  • Cabling and Connections: Document and visualize every cable connection between your devices, allowing you to trace the entire path of a circuit.

  • Automation and Integration: With a powerful REST API and support for custom scripts, NetBox is built for automation, allowing you to streamline processes and integrate with other tools—like Zabbix!

While maintaining this level of documentation might seem daunting, the benefits are immense, especially when you can automate parts of the workflow.

The NetBox-Zabbix Integration: How It Works

The concept behind this integration is simple yet crucial to understand. The flow of information is one-way: from NetBox to Zabbix.

This is fundamental. NetBox acts as the source of truth, the master record. When you add or update a device in NetBox, the plugin provisions that device in Zabbix. It creates the host, assigns templates, sets up interfaces, and applies tags. It is not Zabbix sending information back to NetBox. This ensures your documentation remains the authoritative source.

I got all my inspiration for this setup from a fantastic blog post on Zabbix.com by the team at Open Source ICT Solutions. They created this plugin and provided an excellent guide, so a huge thank you to them!

Getting Started: A Step-by-Step Guide

For my setup, I wanted to simplify things, so I used a non-official Docker repository designed for NetBox plugin development. It makes getting up and running much faster.

H3: Setting Up the Environment

Here are the commands I used to get the environment ready:

# 1. Clone the unofficial Docker repo for NetBox plugin development

> git clone https://github.com/dkraklan/netbox-plugin-development-env.git

> cd netbox-plugin-development-env



# 2. Clone the nbxsync plugin from OpensourceICTSolutions into the plugins directory

> cd plugins/

> git clone https://github.com/OpensourceICTSolutions/nbxsync.git

> cd ..



# 3. Add the plugin to the NetBox configuration

> vi configuration/configuration.py



# Add 'nbxsync' to the PLUGINS list:

PLUGINS = [

    'nbxsync'

]



# 4. Build and launch the Docker containers

> docker-compose build

> docker-compose up -d

And just like that, you should have a running NetBox instance with the Zabbix plugin installed!

Configuring the Zabbix Plugin in NetBox

Once NetBox is up, the configuration is straightforward.

  1. Add Your Zabbix Server: In the NetBox UI, you’ll see a new “Zabbix” menu at the bottom. Navigate there and add your Zabbix server. You’ll need to provide a name, the server URL (just the base URL, e.g., http://zabbix.example.com), and an API token from a Zabbix user with sufficient permissions.

  2. Sync Templates: After adding the server, you can click “Sync Templates.” The plugin will connect to your Zabbix instance and pull in all available templates, proxies, macros, and host groups. This is incredibly useful for later steps.

  3. Define Your Infrastructure: Before adding a device, you need to define some core components in NetBox. This is standard NetBox procedure:

    • Create a Site (e.g., your main office or data center).

    • Define a Manufacturer (e.g., Cisco).

    • Create a Device Role (e.g., Core Switch).

    • Create a Device Type, which is a specific model (e.gne., Cisco CBS350-24P). Here, you can go to the “Zabbix” tab and pre-assign a default Zabbix template for this device type, which is a huge time-saver!

Provisioning a New Device to Zabbix

Now for the magic. Let’s add a new switch and watch it appear in Zabbix.

  1. Create the Device: Create a new device in NetBox, assigning the Site, Device Role, and Device Type you created earlier.

  2. Add an IP Address: Go to the IPAM section and create an IP address that you will assign to the switch’s management interface.

  3. Configure the Zabbix Interface: Navigate back to your newly created device and click on the “Zabbix” tab.

    • Add a Host Interface. Select your Zabbix server, the interface type (e.g., SNMP), and assign the IP address you just created.

    • Add a Host Group. Assign the Zabbix host group where you want this device to appear.

    • Add any Tags you want. I created a “netbox” tag for easy identification.

  4. Sync to Zabbix: With all the information in place, simply click the “Sync to Zabbix” button. A background job will be queued.

If you switch over to your Zabbix frontend, you’ll see the new host has been created automatically, complete with the correct IP address, assigned to the right host group, linked to the correct template, and with the tags we defined. It’s that simple!

Even better, the integration also pulls some data back for viewing. In the “Zabbix Operation” tab within NetBox, you can see the latest problems for that specific device directly from Zabbix, giving you a unified view without leaving the NetBox interface.

Final Thoughts

I have to say, this is a truly impressive product. Of course, it requires a disciplined workflow to maintain the data in NetBox, but the payoff in consistency, automation, and having a single, reliable source of truth is enormous. From one dashboard, you can control your entire documented infrastructure and its monitoring configuration.

This project is actively developed, and the community is already making requests on GitHub. If you have ideas for new features or find any bugs, I encourage you to contribute. This is a tool that can be incredibly useful for anyone managing a Zabbix instance, from a small lab to a large server farm.

That’s all for today! I hope you found this overview interesting. It’s a powerful combination that can really level up your infrastructure management game.

What do you think? Have you used NetBox before? Let me know your thoughts on this integration in the comments below. As always, if you enjoyed the video and this post, please give it a thumbs up and subscribe to the channel if you haven’t already. See you next week!

Follow my work and join the community:

Read More

0

Back from Zabbix Summit: An Exclusive Look at the Future with Zabbix 8.0 LTS
By Dimitri Bellini22 October, 2025Blog

Back from Zabbix Summit: An Exclusive Look at the Future with Zabbix 8.0 LTS

Good morning everyone, Dimitri Bellini here! It’s great to be back with you on Quadrata, my channel dedicated to the world of open source and IT. It’s been a little while since my last video, as I’ve been quite busy traveling between the Zabbix Summit in Riga and Gitex in Dubai. The energy at the Summit was incredible, and I returned not just with great memories but with a clear vision of the future of monitoring. And today, I want to share that vision with you.

So, grab a coffee, and let’s dive into a recap of the fantastic Zabbix Summit and, more importantly, the groundbreaking features coming in the next major release: Zabbix 8.0 LTS.

The Zabbix Summit Experience: A Global Community United

This year’s Zabbix Summit in Riga, Latvia, was special. Not only did it mark the 20th anniversary of Zabbix, but it brought together a massive, passionate community from all corners of the globe—from Japan to South America to the Middle East. It’s always amazing to connect with so many people, share use cases, and discuss what we all love about Zabbix.

We had the chance to tour the new Zabbix headquarters—a beautiful, modern space with floors dedicated to development, support, and commercial teams. I even had the pleasure of presenting a use case with my colleague Francesco, which was a fantastic experience. But the main event, the moment we were all waiting for, was Alexey Vladyshev’s keynote on what’s next for Zabbix.

The Main Event: What’s Coming in Zabbix 8.0 LTS

Zabbix 8.0 is the next Long-Term Support (LTS) release, which means it’s designed for enterprise environments that need stability and support for up to five years. This release isn’t just an incremental update; it’s a monumental leap forward, addressing many of the current limitations and pushing Zabbix firmly into the realm of a full-fledged observability platform.

Let’s break down the most exciting pillars of this upcoming release.

Revolutionizing Event Management with Complex Event Processing (CEP)

One of the biggest game-changers is the introduction of a native Complex Event Processing (CEP) engine. This is designed to drastically reduce monitoring noise and help us focus on the root cause of issues, not just the symptoms. The CEP will operate on two levels:

  • Single Event Processing: This allows for fine-grained control over individual events as they come in. We’ll be able to perform operations like filtering irrelevant events, normalizing data to a standard format, manipulating tags and severity on the fly, and even anonymizing sensitive information.

  • Multiple Event Processing: This is where the real magic happens. By analyzing events over specific time windows, Zabbix will be able to perform advanced correlation, deduplicate redundant alerts, and distinguish between a root cause and its resulting symptoms.

Best of all, we’ll be able to implement custom logic using JavaScript. Imagine enriching an incoming event with data from your CMDB before it even becomes a problem in Zabbix. The possibilities are endless!

Embracing Observability: APM and OpenTelemetry Integration

Zabbix is officially stepping into the world of Application Performance Monitoring (APM). To do this, it’s fundamentally changing how it handles data. Instead of simple time-series data, Zabbix 8.0 will be built to handle complex, structured JSON data natively.

This architectural shift opens the door for seamless integration with modern observability standards like OpenTelemetry. We will finally be able to ingest traces, logs, and metrics from applications directly into Zabbix and visualize them. During the presentation, we saw a mockup of a request trace, broken down step-by-step, allowing for deep root cause analysis right within the Zabbix UI. This is a massive step forward.

A New Engine for Unprecedented Scale

With all this new data, how will Zabbix scale? While standard databases like PostgreSQL and MySQL will still be supported for smaller setups, the focus for large-scale deployments is shifting to high-performance backends. The star of the show here is ClickHouse.

The new architecture will offload the ingestion process to Zabbix Proxies, which will write data directly to ClickHouse. The Zabbix Server will then query this data for visualization and processing. This design allows Zabbix to handle millions of values per second, making it suitable for even the most demanding environments.

A Fresh Face and Enhanced Usability

Let’s be honest, the Zabbix UI, while functional, could use a modern touch. The Zabbix team knows this, and a complete UI overhaul is planned for 8.0! We saw mockups of a cleaner, fresher, and more intuitive interface.

But perhaps one of the most requested features of all time is finally coming: customizable table views. In the “Problems” view and other tables, you will be able to show, hide, reorder, and sort columns as you see fit. It might seem like a small change, but it’s a huge quality-of-life improvement that we’ve been waiting for.

Monitoring on the Go: The Official Zabbix Mobile App

Finally, Zabbix is developing an official mobile application! This will bring essential monitoring capabilities right to your phone, including:

  • Push notifications for alerts.

  • Problem management and collaboration tools.

  • Aggregated views from multiple Zabbix servers.

  • Integration with both on-premise and Zabbix Cloud instances.

A Glimpse into the Future

Zabbix 8.0 LTS is shaping up to be the most significant release in the product’s 20-year history. It’s evolving from a best-in-class monitoring tool into a comprehensive observability platform ready to meet the challenges of modern IT infrastructures. The expected release date is around mid-2026, and I, for one, cannot wait.

I’ll be keeping a close eye on the public roadmap and will keep you updated as these features move through development. But now, I want to hear from you!

What feature are you most excited about? Is there something else you’d love to see in Zabbix? Let me know in the comments below!

That’s all for today. Thanks for joining me, and I’ll see you in the next video. Bye everyone!

Stay Connected:

Read More

0

Revolutionize Your Zabbix Dashboards: RME Essential Custom Widgets
By Dimitri Bellini12 September, 2025Blog

Revolutionize Your Zabbix Dashboards: RME Essential Custom Widgets

Good morning and welcome, everyone! It’s Dimitri Bellini, back again on Quadrata, my channel dedicated to the open-source world and the IT that I love. It’s been a little while since we talked about our good friend Zabbix, and I’m excited to share something I stumbled upon that I think you’re going to love.

While browsing the Zabbix support portal, I came across a community member, Ryan Eberle, who has developed an incredible set of custom widgets. His GitHub repository is a goldmine of enhancements that bring a whole new level of functionality and clarity to our Zabbix dashboards. These aren’t just minor tweaks; they are game-changing improvements that address many of the limitations we’ve all faced.

So, let’s dive in and see how you can supercharge your monitoring setup!

Getting Started: How to Install These Custom Widgets

Installing these widgets is surprisingly simple. Just follow these steps, and you’ll be up and running in no time.

Important Note: These modules are designed for Zabbix 7.2 and 7.4. They leverage new functions not available in the 7.0 LTS version, so they are not backward compatible.

  1. Clone the Repository: First, head over to the developer’s GitHub repository. Find the widget you want to install (for example, the Graph widget), click on “Code,” and copy the clone URL.

  2. Download to Your Server: SSH into your Zabbix server console. In a temporary directory, use the `git clone` command to download the widget files. For example:
    git clone [paste the copied URL here]

  3. Copy to the Zabbix Modules Directory: This is a crucial step. In recent Zabbix versions, the path for UI modules has changed. You need to copy the downloaded widget directory into:
    /usr/share/zabbix/ui/modules/

  4. Scan for New Modules: Go to your Zabbix frontend and navigate to Administration → General → Modules. Click the “Scan directory” button. This is a step many people forget! If you don’t do this, Zabbix won’t see the new widgets you just added.

  5. Enable the Widgets: Once the scan is complete, you will see the new modules listed, authored by Ryan Eberle. By default, they will be disabled. Simply click to enable each one you want to use.

A Deep Dive into the New Widget Capabilities

Now for the fun part! Let’s explore what these new widgets bring to the table. I’ve been testing the enhanced Graph, Table, and Host/Group Navigator widgets, and they are phenomenal.

The Graph Widget We’ve Always Wanted

The default vector graph in Zabbix is good, but Ryan’s version is what it should have been. It introduces features that dramatically improve usability.

  • Interactive Legend: You can now click on a metric in the legend to toggle its visibility on the graph. Want to focus on just one or two data series? Simply click to hide the others. Hold the Ctrl key to select multiple items. This is fantastic for decluttering complex graphs.

  • Sorted Tooltip/Legend: No more hunting through a messy tooltip! The legend now automatically sorts metrics, placing the ones with the highest current value at the top. When you hover over the graph, you get a clean, ordered list, making it instantly clear which metric is which.

  • Hide Zero-Value Metrics: You can configure the widget to automatically hide any metrics that have a value of zero. This cleans up the tooltip immensely, allowing you to focus only on the data that matters.

  • Advanced Label Customization: Using built-in macros and regular expressions, you can customize the data set labels. If you have very long item names, you can now extract just the part you need to keep your graphs clean and readable.

  • Data Multiplication: Need to convert a value on the fly? You can now apply a multiplier directly within the widget’s data set configuration. This is perfect for when you need to change units of measurement for display purposes without creating a new calculated item.

The difference is night and day. A cluttered, hard-to-read Zabbix graph becomes a clean, interactive, and insightful visualization.

The Ultimate Table Widget

While Zabbix has widgets like “Top hosts,” they’ve always felt a bit rigid. The new Table widget is incredibly flexible and allows you to build the exact views you need for any scenario.

One of my favorite features is the “Column per pattern” mode. Imagine you want to see the incoming and outgoing traffic for all network interfaces on a host, side-by-side. With this widget, you can!

Here’s how it works:

  • You define an item pattern for your rows (e.g., the interface name using tags).

  • You then define a pattern for each column (e.g., one for `bits.sent` and another for `bits.recv`).

  • The widget intelligently organizes the data into a clean table with interfaces as rows and your metrics as columns.

You can also add a footer row to perform calculations like sum or average. This is incredibly useful for getting an overview of a cluster. For instance, you can display the average CPU and memory utilization across all nodes in a single, elegant table.

Improved Navigation Widgets

The new Host/Group Navigator and Item Navigator also bring welcome improvements. The Host Navigator provides better filtering and a more intuitive way to navigate through host group hierarchies, which is especially helpful for complex environments. The Item Navigator includes a search box that works on tags, allowing you to quickly find and display specific groups of metrics in another widget, like our new super-graph!

Final Thoughts and a Call to Action

These custom widgets have genuinely enhanced my Zabbix experience. They add a layer of polish, usability, and power that was sorely missing from the default dashboards. It’s a testament to the strength of the open-source community, and I hope the Zabbix team takes inspiration from this work for future official releases.

Now, I want to hear from you. What do you think of these widgets? Are there any features you’ve been desperately wanting for your Zabbix dashboards? Let me know in the comments below! Perhaps if we gather enough feedback, we can share it with the developer and help make these tools even better.

If you enjoyed this video and found it helpful, please give it a nice like and subscribe for more content. See you next week!

Stay Connected with Quadrata:

Read More

0

The Ultimate Guide to Open Source Backups: Kopia vs. Minarca
By Dimitri Bellini20 June, 2025Blog

The Ultimate Guide to Open Source Backups: Kopia vs. Minarca

Good morning, everyone! It’s Dimitri Bellini, and welcome back to Quadrata, my channel dedicated to the world of open source. A few weeks ago, I had one of those heart-stopping moments we all dread: a chunk of my data suddenly became inaccessible. While I had some backup systems in place, they weren’t as reliable as I thought. That scare sent me on a mission to find the best, most robust open-source backup solution out there.

We often don’t think about backups until it’s too late. Whether it’s a broken hard drive, corrupted files, or the dreaded ransomware attack, having a solid backup strategy is the only thing that stands between you and disaster. But remember, as the saying goes, “The backup is guaranteed, but the restore is another story.” A good tool needs to handle both flawlessly.

My Hunt for the Perfect Backup Tool: The Checklist

Before diving into the options, I set some clear requirements for my ideal solution. It needed to be more than just a simple file-copying utility. Here’s what I was looking for:

  • Truly Open Source: The solution had to be freely available and transparent.

  • Cross-Platform: It must work seamlessly across Windows, Linux, and macOS.

  • Powerful Features: I was looking for advanced capabilities like:

    • Deduplication: A smart feature that saves significant space by not storing the same file or block of data more than once. This is different from compression, which just shrinks individual files.

    • Encryption: End-to-end encryption is a must, especially if I’m sending my data to a cloud provider. My data should be for my eyes only.

  • Flexible & Versatile: It needed a Command Line Interface (CLI) for automation and a Graphical User Interface (GUI) for ease of use. It also had to support a wide range of storage backends, from a local NAS (using protocols like SSH, NFS, SFTP) to cloud storage like Amazon S3 and Backblaze B2.

  • Simplicity: Finally, it couldn’t be a convoluted mess of scripts. The setup and daily management had to be straightforward.

The Contenders: Kopia vs. Minarca

My research led me to several interesting projects like Restic, Borg, and UrBackup, but two stood out for meeting almost all my criteria: Kopia and Minarca. They represent two fundamentally different philosophies in the backup world, and I put both to the test.

Deep Dive #1: Kopia – The Power User’s Choice

Kopia is a modern, feature-rich, agent-based backup tool. This means all the intelligence—deduplication, encryption, scheduling—resides in the agent you install on your computer. It directly manages the data in your chosen storage repository without needing a server component on the other end.

What I Love About Kopia (Pros):

  • Advanced Features: It offers fantastic cross-device deduplication, strong end-to-end encryption, compression, and error correction to ensure your data is always valid.

  • Snapshot-Based: Kopia creates point-in-time “snapshots” of your directories. This is great for versioning and ensures a consistent state for each backup.

  • Incredibly Versatile: It supports virtually every storage backend you can think of.

  • Resilient by Design: All your configuration policies are stored within the remote repository itself. If your computer dies, you can install Kopia on a new machine, connect to your repository, and all your settings and backup history are immediately available for a restore.

Potential Hurdles (Cons):

  • Learning Curve: While the GUI simplifies things, the underlying policy system for managing snapshot retention, scheduling, and exclusions is incredibly detailed and can be a bit overwhelming for beginners.

  • No Simultaneous Multi-Repository: It doesn’t natively support backing up to multiple repositories at the exact same time in a single job.

A Quick Kopia Walkthrough

Using Kopia involves three main steps:

  1. Connect to a Repository: You start by pointing the Kopia UI to your storage location, whether it’s a local folder, an SFTP server, or a cloud bucket like Backblaze B2. You’ll set a password that encrypts everything.

  2. Create a Snapshot: You then select a folder you want to back up and Kopia will create a snapshot. The first run will upload everything, but subsequent snapshots are incredibly fast, as it only processes and uploads the changes, thanks to deduplication.

  3. Restore: To restore, you browse your snapshots by date and time, find the files or folders you need, and either download them or “mount” the entire snapshot as a local drive to browse through it.

Deep Dive #2: Minarca – Simplicity and Centralized Control

Minarca takes a more traditional client-server approach. You install a lightweight agent on your machines and a server component on a central server (which can be a simple Linux box in your home or office). This architecture is fantastic for managing multiple devices.

Where Minarca Shines (Pros):

  • Incredibly User-Friendly: Both the agent and the server have beautiful, simple interfaces. It’s very intuitive to set up and manage.

  • Centralized Management: The web-based server dashboard is the star of the show. It gives you a complete overview of all your users and devices, with stats on backup duration, space used, and new or modified files. You can even restore files for any user directly from the web interface!

  • Multi-User/Multi-Device: It’s built from the ground up to support multiple users, each with multiple devices, all backing up to one central location.

  • Easy Installation: Setting up the Minarca server on an Ubuntu or Debian machine takes just a few simple commands.

What’s Missing (Cons):

  • No Built-in Advanced Features: Minarca currently leaves features like deduplication and encryption to the underlying storage system (for example, using a ZFS filesystem on your server).

  • File-Based, Not Snapshot-Based: It works by copying files (incrementally) rather than creating atomic snapshots like Kopia.

A Quick Minarca Walkthrough

Getting started with Minarca is a breeze:

  1. Set Up the Server: Install the Minarca server package on a Linux machine.

  2. Configure the Agent: On your client PC, install the Minarca agent and point it to your server’s address with the credentials you created.

  3. Create a Backup Job: In the agent, you can easily select which folders to include or exclude and set a simple backup schedule.

  4. Restore: The agent’s restore interface presents a calendar. You just click a date to see the available backup and browse the files you want to recover. Or, as an admin, you can do this from the central web dashboard.

Kopia vs. Minarca: Which One Is for You?

After testing both extensively, it’s clear they serve different needs.

Choose Kopia if: You are a power user who wants the most advanced features like end-to-end encryption and high-efficiency deduplication built right in. You are comfortable with a more technical setup and prefer a decentralized, agent-only approach.

Choose Minarca if: You value simplicity, ease of use, and centralized control. It’s the perfect solution if you need to manage backups for your family or a small office, and you want a clean dashboard to monitor everything at a glance.

My Final Thoughts and Your Turn

Both Kopia and Minarca are fantastic, robust open-source solutions that put you in control of your data. They are both miles ahead of just hoping for the best. The most important takeaway is this: don’t be like me and wait for a scare. Set up your backups today!

I’d love to hear from you. Which of these tools sounds more appealing to you? Are you using another open-source solution that you love? Let me know in the comments below. Your feedback is always welcome!

Thank you for reading, and don’t forget to subscribe to my YouTube channel for more deep dives into the world of open source.

Until next time, this is Dimitri. Ciao!

Read More

0

SigNoz: A Powerful Open Source APM and Observability Tool
By Dimitri Bellini12 June, 2025Blog

Diving Deep into SigNoz: A Powerful Open Source APM and Observability Tool

Good morning everyone, I’m Dimitri Bellini, and welcome back to Quadrata, the channel where we explore the fascinating world of open source and IT. As I always say, I hope you enjoy these videos, and if you haven’t already, please consider subscribing and hitting that like button if you find the content valuable!

While Zabbix always holds a special place in our hearts for monitoring, today I want to introduce something different. I’ve been getting requests from customers about how to monitor their applications, and for that, you typically need an Application Performance Monitor (APM), or as it’s sometimes fancily called, an “Observability Tool.”

Introducing SigNoz: Your Open Source Observability Hub

The tool I’m excited to share with you today is called SigNoz. It’s an open-source solution designed for comprehensive observability, which means it helps you monitor metrics, traces (the calls made within your application), and even logs. This last part is a key feature of SigNoz, as it aims to incorporate everything you might need to keep a close eye on your applications.

One of its core strengths is that it’s built natively on OpenTelemetry. OpenTelemetry is becoming an industry standard for collecting telemetry data (metrics, traces, logs) from your applications and transmitting it to a backend like SigNoz. We’ll touch on the advantages of this later.

Why Consider SigNoz?

SigNoz positions itself as an open-source alternative to paid, proprietary solutions like Datadog or New Relic, which can be quite expensive. Of course, choosing open source isn’t just about avoiding costs; it’s also about flexibility and community. For home labs, small projects, or even just for learning, SigNoz can be incredibly useful.

Key Features of SigNoz

  • Application Performance Monitoring (APM): Out-of-the-box, you get crucial metrics like P99 latency, error rates, requests per second, all neatly presented in dashboards.

  • Distributed Tracing: This allows you to follow the path of a request as it travels through your application, helping you pinpoint bottlenecks and errors.

  • Log Management: A relatively recent but powerful addition, SigNoz can ingest logs, allowing you to search and analyze them, similar to tools like Greylog (though perhaps with fewer advanced log-specific features for now).

  • Metrics and Dashboards: SigNoz provides a user-friendly interface with customizable dashboards and widgets.

  • Alerting: You can set up alerts, much like triggers in Zabbix, to get notified via various channels when something goes wrong.

Under the Hood: The Architecture of SigNoz

Understanding how SigNoz is built is fundamental to appreciating its capabilities:

  • OpenTelemetry: As mentioned, this is the core component for collecting and transmitting data from your applications.

  • ClickHouse: This is the database SigNoz uses. ClickHouse is an open-source, column-oriented database management system that’s incredibly efficient for handling and querying millions of data points very quickly. It also supports high availability and horizontal scaling even in its open-source version, which isn’t always the case with other databases.

  • SigNoz UI: The web interface that allows you to visualize and interact with the data collected by OpenTelemetry and stored in ClickHouse.

For those wanting to try it out at home, you can easily get this all running with Docker.

The Power of OpenTelemetry

OpenTelemetry is a game-changer. It’s becoming a de facto standard, with even tools like Dynatrace now able to use OpenTelemetry as a data source. The community around it is very active, making it a solid foundation for a product like SigNoz.

Key advantages of OpenTelemetry include:

  • Standardization: It provides a consistent way to instrument applications.

  • Libraries and Agents: It offers out-of-the-box libraries and agents for most major programming languages, simplifying instrumentation.

  • Auto-Instrumentation (Monkey Patching): Theoretically, OpenTelemetry can automatically inject the necessary code into your application to capture telemetry data without you needing to modify your application’s source code significantly. You just invoke your application with certain environment parameters. I say “theoretically” because while I tried it with one of my Python applications, I couldn’t get it to trace anything. Let me know in the comments if you’d like a dedicated video on this; I’m curious to dig deeper into why it didn’t work for me straight away!

Getting Started: Installing SigNoz with Docker and a Demo App

For my initial tests, I used a demo application suggested by the SigNoz team. Here’s a rundown of how you can get started with a standalone Docker setup:

1. Install SigNoz

It’s straightforward:

  1. Clone the SigNoz repository: git clone https://github.com/SigNoz/signoz.git (or the relevant path from their docs).

  2. Navigate into the directory and run Docker Compose. This will pull up four containers:

    • SigNoz Hotel Collector (OpenTelemetry Collector): Gathers data from OpenTelemetry agents.

    • SigNoz Query Service/Frontend: The graphical interface.

    • ClickHouse Server: The database.

    • Zookeeper: Manages ClickHouse instances (similar to etcd).

You can usually find the exact commands in the official SigNoz documentation under the Docker deployment section.

2. Set Up the Sample FastAPI Application

To see SigNoz in action, I used their “Sample FastAPI App”:

  1. Clone the demo app repository: (You’ll find this on the SigNoz GitHub or documentation).

  2. Create a Python 3 virtual environment: It’s always good practice to isolate dependencies. 
    python3 -m venv .venv
    
source .venv/bin/activate

  3. Install dependencies: 
    pip install -r requirements.txt

  4. Install OpenTelemetry components for auto-instrumentation: 
    pip install opentelemetry-distro opentelemetry-exporter-otlp

  5. Bootstrap OpenTelemetry (optional, for auto-instrumentation): 
    opentelemetry-bootstrap --action=install

    This attempts to find requirements for your specific application.

  6. Launch the application with OpenTelemetry instrumentation:

    You’ll need to set a few environment variables:

    • OTEL_RESOURCE_ATTRIBUTES: e.g., service.name=MyFastAPIApp (This name will appear in SigNoz).

    • OTEL_EXPORTER_OTLP_ENDPOINT: The address of your SigNoz collector (e.g., http://localhost:4317 if running locally).

    • OTEL_EXPORTER_OTLP_TRACES_EXPORTER: Set to otlp.

    • OTEL_EXPORTER_OTLP_PROTOCOL: Can be grpc or http/protobuf.

    Then, run your application using the opentelemetry-instrument command:

    OTEL_RESOURCE_ATTRIBUTES=service.name=FastApp OTEL_EXPORTER_OTLP_ENDPOINT="http://:4317" OTEL_EXPORTER_OTLP_TRACES_EXPORTER=otlp OTEL_EXPORTER_OTLP_PROTOCOL=grpc opentelemetry-instrument uvicorn main:app --host 0.0.0.0 --port 8000

    (Replace with the actual IP where SigNoz is running).
    The opentelemetry-instrument part is what attempts the “monkey patching” or auto-instrumentation. The application itself (uvicorn main:app...) starts as it normally would.

A Quick Look at SigNoz in Action

Once the demo app was running and sending data, I could see traces appearing in my terminal (thanks to console exporter settings). To generate some load, I used Locust with a simple configuration to hit the app’s HTTP endpoint. This simulated about 10 users.

Navigating to the SigNoz UI (typically on port 3301, or as configured, if you’re using the Docker setup that forwards to 8080 or another port for the frontend, but the collector often listens on 4317/4318), the dashboard immediately showed my “FastApp” service. Clicking on it revealed:

  • Latency, request rate, and error rate graphs.

  • A list of endpoints called.

Drilling down into the traces, I could see individual requests. For this simple “Hello World” app, the trace was trivial, just showing the HTTP request. However, if the application were more complex, accessing a database, for example, OpenTelemetry could trace those interactions too, showing you the queries and time taken. This is where it gets really interesting for debugging and performance analysis.

The SigNoz interface felt responsive and well-designed. I was quite impressed with how smoothly it all worked.

Final Thoughts and What’s Next

I have to say, SigNoz seems like a very capable and well-put-together tool. It’s definitely worth trying out, especially if you’re looking for an open-source observability solution.

I plan to test it further with a more complex application, perhaps one involving a database, to see how it handles more intricate call graphs and to really gauge if it can be a strong contender against established players for more demanding scenarios.

It’s also interesting to note that Zabbix has APM features on its roadmap, potentially for version 8. So, the landscape is always evolving! But for now, SigNoz is a noteworthy project, especially for those interested in comprehensive observability that includes metrics, traces, AND logs in one package. This log management capability could make it a simpler alternative to setting up a separate, more complex logging stack for many use cases, particularly in home labs or smaller environments.

So, what do you think? Have you tried SigNoz or other APM tools? Let me know in the comments below! If there’s interest, I can certainly make more videos exploring its features or trying out more complex scenarios.

Thanks for watching, and I’ll see you next week. A greeting from me, Dimitri!

—

Stay Connected with Quadrata:

📺 Subscribe to Quadrata on YouTube

💬 Join the Zabbix Italia Telegram Channel (Also great for general monitoring discussions!)

Read More

0