Diving Deep into SigNoz: A Powerful Open Source APM and Observability Tool

Good morning everyone, I’m Dimitri Bellini, and welcome back to Quadrata, the channel where we explore the fascinating world of open source and IT. As I always say, I hope you enjoy these videos, and if you haven’t already, please consider subscribing and hitting that like button if you find the content valuable!

While Zabbix always holds a special place in our hearts for monitoring, today I want to introduce something different. I’ve been getting requests from customers about how to monitor their applications, and for that, you typically need an Application Performance Monitor (APM), or as it’s sometimes fancily called, an “Observability Tool.”

Introducing SigNoz: Your Open Source Observability Hub

The tool I’m excited to share with you today is called SigNoz. It’s an open-source solution designed for comprehensive observability, which means it helps you monitor metrics, traces (the calls made within your application), and even logs. This last part is a key feature of SigNoz, as it aims to incorporate everything you might need to keep a close eye on your applications.

One of its core strengths is that it’s built natively on OpenTelemetry. OpenTelemetry is becoming an industry standard for collecting telemetry data (metrics, traces, logs) from your applications and transmitting it to a backend like SigNoz. We’ll touch on the advantages of this later.

Why Consider SigNoz?

SigNoz positions itself as an open-source alternative to paid, proprietary solutions like Datadog or New Relic, which can be quite expensive. Of course, choosing open source isn’t just about avoiding costs; it’s also about flexibility and community. For home labs, small projects, or even just for learning, SigNoz can be incredibly useful.

Key Features of SigNoz

• Application Performance Monitoring (APM): Out-of-the-box, you get crucial metrics like P99 latency, error rates, requests per second, all neatly presented in dashboards.
• Distributed Tracing: This allows you to follow the path of a request as it travels through your application, helping you pinpoint bottlenecks and errors.
• Log Management: A relatively recent but powerful addition, SigNoz can ingest logs, allowing you to search and analyze them, similar to tools like Greylog (though perhaps with fewer advanced log-specific features for now).
• Metrics and Dashboards: SigNoz provides a user-friendly interface with customizable dashboards and widgets.
• Alerting: You can set up alerts, much like triggers in Zabbix, to get notified via various channels when something goes wrong.

Under the Hood: The Architecture of SigNoz

Understanding how SigNoz is built is fundamental to appreciating its capabilities:

• OpenTelemetry: As mentioned, this is the core component for collecting and transmitting data from your applications.
• ClickHouse: This is the database SigNoz uses. ClickHouse is an open-source, column-oriented database management system that’s incredibly efficient for handling and querying millions of data points very quickly. It also supports high availability and horizontal scaling even in its open-source version, which isn’t always the case with other databases.
• SigNoz UI: The web interface that allows you to visualize and interact with the data collected by OpenTelemetry and stored in ClickHouse.

For those wanting to try it out at home, you can easily get this all running with Docker.

The Power of OpenTelemetry

OpenTelemetry is a game-changer. It’s becoming a de facto standard, with even tools like Dynatrace now able to use OpenTelemetry as a data source. The community around it is very active, making it a solid foundation for a product like SigNoz.

Key advantages of OpenTelemetry include:

• Standardization: It provides a consistent way to instrument applications.
• Libraries and Agents: It offers out-of-the-box libraries and agents for most major programming languages, simplifying instrumentation.
• Auto-Instrumentation (Monkey Patching): Theoretically, OpenTelemetry can automatically inject the necessary code into your application to capture telemetry data without you needing to modify your application’s source code significantly. You just invoke your application with certain environment parameters. I say “theoretically” because while I tried it with one of my Python applications, I couldn’t get it to trace anything. Let me know in the comments if you’d like a dedicated video on this; I’m curious to dig deeper into why it didn’t work for me straight away!

Getting Started: Installing SigNoz with Docker and a Demo App

For my initial tests, I used a demo application suggested by the SigNoz team. Here’s a rundown of how you can get started with a standalone Docker setup:

1. Install SigNoz

It’s straightforward:

1. Clone the SigNoz repository: git clone https://github.com/SigNoz/signoz.git (or the relevant path from their docs).
2. Navigate into the directory and run Docker Compose. This will pull up four containers:

   • SigNoz Hotel Collector (OpenTelemetry Collector): Gathers data from OpenTelemetry agents.
   • SigNoz Query Service/Frontend: The graphical interface.
   • ClickHouse Server: The database.
   • Zookeeper: Manages ClickHouse instances (similar to etcd).

You can usually find the exact commands in the official SigNoz documentation under the Docker deployment section.

2. Set Up the Sample FastAPI Application

To see SigNoz in action, I used their “Sample FastAPI App”:

1. Clone the demo app repository: (You’ll find this on the SigNoz GitHub or documentation).
2. Create a Python 3 virtual environment: It’s always good practice to isolate dependencies.

   python3 -m venv .venv
   
   source .venv/bin/activate

3. Install dependencies:

   pip install -r requirements.txt

4. Install OpenTelemetry components for auto-instrumentation:

   pip install opentelemetry-distro opentelemetry-exporter-otlp

5. Bootstrap OpenTelemetry (optional, for auto-instrumentation):

   opentelemetry-bootstrap --action=install

   This attempts to find requirements for your specific application.

6. Launch the application with OpenTelemetry instrumentation:

   You’ll need to set a few environment variables:

   • OTEL_RESOURCE_ATTRIBUTES: e.g., service.name=MyFastAPIApp (This name will appear in SigNoz).
   • OTEL_EXPORTER_OTLP_ENDPOINT: The address of your SigNoz collector (e.g., http://localhost:4317 if running locally).
   • OTEL_EXPORTER_OTLP_TRACES_EXPORTER: Set to otlp.
   • OTEL_EXPORTER_OTLP_PROTOCOL: Can be grpc or http/protobuf.

   Then, run your application using the opentelemetry-instrument command:

   OTEL_RESOURCE_ATTRIBUTES=service.name=FastApp OTEL_EXPORTER_OTLP_ENDPOINT="http://:4317" OTEL_EXPORTER_OTLP_TRACES_EXPORTER=otlp OTEL_EXPORTER_OTLP_PROTOCOL=grpc opentelemetry-instrument uvicorn main:app --host 0.0.0.0 --port 8000

   (Replace with the actual IP where SigNoz is running).
   The opentelemetry-instrument part is what attempts the “monkey patching” or auto-instrumentation. The application itself (uvicorn main:app...) starts as it normally would.

A Quick Look at SigNoz in Action

Once the demo app was running and sending data, I could see traces appearing in my terminal (thanks to console exporter settings). To generate some load, I used Locust with a simple configuration to hit the app’s HTTP endpoint. This simulated about 10 users.

Navigating to the SigNoz UI (typically on port 3301, or as configured, if you’re using the Docker setup that forwards to 8080 or another port for the frontend, but the collector often listens on 4317/4318), the dashboard immediately showed my “FastApp” service. Clicking on it revealed:

• Latency, request rate, and error rate graphs.
• A list of endpoints called.

Drilling down into the traces, I could see individual requests. For this simple “Hello World” app, the trace was trivial, just showing the HTTP request. However, if the application were more complex, accessing a database, for example, OpenTelemetry could trace those interactions too, showing you the queries and time taken. This is where it gets really interesting for debugging and performance analysis.

The SigNoz interface felt responsive and well-designed. I was quite impressed with how smoothly it all worked.

Final Thoughts and What’s Next

I have to say, SigNoz seems like a very capable and well-put-together tool. It’s definitely worth trying out, especially if you’re looking for an open-source observability solution.

I plan to test it further with a more complex application, perhaps one involving a database, to see how it handles more intricate call graphs and to really gauge if it can be a strong contender against established players for more demanding scenarios.

It’s also interesting to note that Zabbix has APM features on its roadmap, potentially for version 8. So, the landscape is always evolving! But for now, SigNoz is a noteworthy project, especially for those interested in comprehensive observability that includes metrics, traces, AND logs in one package. This log management capability could make it a simpler alternative to setting up a separate, more complex logging stack for many use cases, particularly in home labs or smaller environments.

So, what do you think? Have you tried SigNoz or other APM tools? Let me know in the comments below! If there’s interest, I can certainly make more videos exploring its features or trying out more complex scenarios.

Thanks for watching, and I’ll see you next week. A greeting from me, Dimitri!

—

Stay Connected with Quadrata:

📺 Subscribe to Quadrata on YouTube

💬 Join the Zabbix Italia Telegram Channel (Also great for general monitoring discussions!)