I switched from Linux to MacOs: My First Month Experience

Good morning everyone, Dimitri Bellini here! Welcome back to my channel, Quadrata, where we dive into the world of open source and IT. If you’ve been following along, you’ll know I recently discussed my decision to potentially move from my trusty Linux setup to the world of Apple laptops.

Well, the temptation won. I bought it. After over 20 years deeply rooted in the Linux ecosystem, primarily using Fedora on various ThinkPads for the last decade, I took the plunge and acquired a MacBook Pro M4. It was a step I took with some apprehension, mainly driven by a quest for increased productivity and hardware that felt truly integrated and powerful, especially after my ThinkPad T14S started showing its age with battery life and overheating issues during video calls and rendering.

The Machine: Why the MacBook Pro M4?

I didn’t go for the Air as initially considered. I found a fantastic deal on a MacBook Pro M4 (around €1650), which made the decision easier. Here’s what I got:

• CPU/GPU: 12 cores (4 performance, 8 efficiency) + GPU cores + Neural Engine
• RAM: 24GB Unified Memory (great for VMs and local AI models)
• Storage: 512GB SSD (the compromise for the price – 1TB was significantly more expensive)
• Display: A stunning high-resolution display with excellent brightness.

Coming from years of using used ThinkPads – which were workhorses, true Swiss Army knives – this felt like a significant hardware upgrade, especially considering the price point I managed to secure.

Hardware Impressions: The Good, The Bad, and The Workarounds

Hardware: The Good Stuff

• Battery Life: This is a game-changer. I’m easily getting 10-12 hours of normal use (coding, web browsing, conferences, shell usage). Standby time is phenomenal; I barely turn it off anymore. This was simply unattainable on my previous x86 Linux laptops.
• Display: Truly gorgeous. It’s hard to compare with most laptops in the same price range I found this Mac in.
• Touchpad: Exceptional. The haptic feedback and precision are on another level. It genuinely enhances the daily user experience.
• Speakers: Finally! Decent built-in audio. I can actually listen to music with bass and clarity. This also translates to much better web call experiences – the microphone and speaker combination works so well I often don’t need a headset.
• Performance (CPU/GPU/Neural Engine): It handles my workload smoothly. The real surprise was running local AI models. I tested Gemma 3 (around 10GB, 12 billion parameters) and got around 23 tokens/second. This opens up possibilities for local AI experimentation without needing a dedicated GPU rig.
• USB-C Flexibility: Having three USB-C/Thunderbolt ports is adequate, and the ability to charge via any of them using a Power Delivery hub (which also connects my peripherals) is incredibly convenient. One cable does it all.

Hardware: The Not-So-Good

• Keyboard: This is my biggest hardware gripe. The key travel is very shallow. While better than some cheap laptops, it feels like typing on plastic compared to the ThinkPad keyboards I’m used to.
• Weight: At 1.6kg, the MacBook Pro is noticeably heavier than my old T14S. Quality materials add weight, I suppose.
• Non-Upgradeable Storage: 512GB isn’t huge, and knowing I can’t upgrade it later means careful storage management is essential. You *must* choose your storage size wisely at purchase.

Hardware Workarounds

To address the storage limitation for non-critical files, I found a neat gadget: the BaseQi MicroSD Card Adapter. It sits flush in the SD card slot, allowing me to add a high-capacity MicroSD card (I used a SanDisk Extreme Pro) for documents and media. It’s not fast enough for active work or applications due to latency, but perfect for expanding storage for less performance-sensitive data. I sync these documents to the cloud as a backup.

For the keyboard, since I mostly work docked, I bought an external mechanical keyboard: the Epomaker EK68 (or possibly a similar model like the AJAZZ K820 Pro mentioned). It’s a 75% layout keyboard with great tactile feedback that I got for around €50 – a worthwhile investment for comfortable typing.

Diving into macOS: A Linux User’s Perspective

Okay, let’s talk software. Coming from the freedom and structure of Linux, macOS feels… different. Sometimes simple, sometimes restrictive.

The Frustrations

• No Native Package Manager: This was jarring. Hunting for software online, downloading DMG files – it felt archaic compared to `apt` or `dnf`. The App Store exists, but often has weird pricing discrepancies compared to direct downloads, and doesn’t have everything.
• The Dock: I used to admire it from afar on Linux. Now that I have it? I find it mostly useless. It takes up space and doesn’t offer the workflow benefits I expected.
• Finder (File Manager): Oh, Finder. It feels incredibly basic. Simple tasks like moving files often default to copy-paste. Customizing it to show path bars or folder info requires digging into options. Searching defaults to the entire Mac instead of the current folder, which is maddening. It feels permeated throughout the OS and hard to escape.
• Application Closing: Clicking the ‘X’ often just minimizes the app instead of closing it. You need to explicitly Quit (Cmd+Q) or Force Quit. It’s a different paradigm I’m still adjusting to.
• Monotasking Feel: The OS seems optimized for focusing on one application at a time. While this might benefit single-app workflows (video editing, music production), it feels less efficient for my typical multi-tasking sysadmin/developer style. The strong single-core performance seems to reflect this philosophy.
• System Settings/Control Center: There are *so many* options, and finding the specific setting you need can feel like a maze.

The Silver Linings & Essential Tools

It’s not all bad, of course. The UI, while sometimes frustrating, is generally coherent. And thankfully, the community has provided solutions:

• Homebrew: This is **ESSENTIAL**. It brings a proper package manager experience to macOS (`brew install `). It makes installing and updating software (especially open-source tools) sane. Install this first!
• iTerm2: A vastly superior terminal emulator compared to the default Terminal. Highly customizable and brings back a familiar Linux-like terminal experience.
• Oh My Zsh (or Oh My Bash): Customizes the shell environment for a better look, feel, and useful shortcuts/plugins. Works great with iTerm2.
• Forklift: A paid, dual-pane file manager. It’s my current replacement for Finder, offering features like tabs, sync capabilities (Google Drive, etc.), and a more productive interface. Still evaluating, but much better than Finder for my needs.
• Zed: A fast, modern code and text editor. It starts quickly and handles my text editing needs well.
• LibreOffice: My go-to office suite. Works perfectly via Homebrew (`brew install libreoffice`).
• Inkscape & GIMP: Open-source staples for vector and raster graphics. Both easily installable via Homebrew (`brew install inkscape gimp`) and cover my needs perfectly.
• Latest: A handy utility (installable via Brew) that scans your applications (even those not installed via Brew or the App Store) and notifies you of available updates. Helps manage the entropy of different installation methods.
• WireGuard & Tunnelblick: Essential VPN clients. WireGuard has an official client, and Tunnelblick is my preferred choice for OpenVPN connections on macOS.

Key System Setting Tweaks

After watching countless videos, here are a few settings I changed immediately:

• Window Tiling (Sequoia+): Enabled tiling but *removed the margins* between windows to maximize screen real estate.
• Touchpad: Enabled “Secondary Click” (two-finger tap) for right-click functionality.
• Dock: Enabled “Show battery percentage”. Set the Dock to “Automatically hide and show”. Removed unused default app icons and recent application suggestions to minimize clutter.

The Verdict After One Month

So, am I happy? It’s complicated.

The hardware is undeniably premium. The performance, display, battery life, and touchpad are fantastic. For the price I paid, it feels like great value in that regard.

However, productivity isn’t magically perfect. macOS has its own quirks, bugs, and limitations. Using third-party (especially open-source) applications doesn’t always feel as seamless as on Linux. The “it just works” mantra isn’t universally true.

The software experience requires adaptation and, frankly, installing several third-party tools to replicate the workflow I was comfortable with on Linux. Homebrew is the saving grace here.

Overall, it’s a high-quality machine with some frustrating software paradigms for a long-time Linux user. The experience is *coherent*, which is better than the sometimes fragmented feel of Windows, but coherence doesn’t always mean *better* for my specific needs.

Will I stick with it? Time will tell. Maybe in another month, I’ll be fully converted. Or maybe I’ll be cheering even louder for the Asahi Linux project to bring full Linux support to the M4 chips!

What Are Your Thoughts?

This is just my experience after one month. I’m still learning! What are your tips for a Linux user transitioning to macOS? What essential apps or settings have I missed? Let me know in the comments below!

If you found this useful, please give the video a thumbs up, share it, and subscribe to the Quadrata YouTube channel if you haven’t already!

Also, feel free to join the discussion on the ZabbixItalia Telegram Channel.

Thanks for reading, and see you next week!

– Dimitri Bellini

Automating My Video Workflow with N8N and AI: A Real-World Test

Good morning everyone, Dimitri Bellini here! Welcome back to Quadrata, my channel dedicated to the open-source world and the IT topics I find fascinating – and hopefully, you do too.

This week, I want to dive back into artificial intelligence, specifically focusing on a tool we’ve touched upon before: N8N. But instead of just playing around, I wanted to tackle a real problem I face every week: automating the content creation that follows my video production.

The Challenge: Bridging the Gap Between Video and Text

Making videos weekly for Quadrata is something I enjoy, but the work doesn’t stop when the recording ends. There’s the process of creating YouTube chapters, writing blog posts, crafting LinkedIn announcements, and more. These tasks, while important, can be time-consuming. My goal was to see if AI, combined with a powerful workflow tool, could genuinely simplify these daily (or weekly!) activities.

Could I automatically generate useful text content directly from my video’s subtitles? Let’s find out.

The Toolkit: My Automation Stack

To tackle this, I assembled a few key components:

• N8N: An open-source workflow automation tool that uses a visual, node-based interface. It’s incredibly versatile and integrates with countless services. We’ll run this using Docker/Docker Compose.
• AI Models: I experimented with two approaches:

  • Local AI with Ollama: Using Ollama to run models locally, specifically testing Gemma 3 (27B parameters). The latest Ollama release (0.1.60 at the time of recording, though versions update) offers better support for models like Gemma.
  • Cloud AI with Google AI Studio: Leveraging the power of Google’s models via their free API tier, primarily focusing on Gemini 2.5 Pro due to its large context window and reasoning capabilities.

• Video Transcripts: The raw material – the subtitles generated for my YouTube videos.

Putting it to the Test: Automating Video Tasks with N8N

I set up an N8N workflow designed to take my video transcript and process it through AI to generate different outputs. Here’s how it went:

1. Getting the Transcript

The first step was easy thanks to the N8N community. I used a community node called “YouTube Transcript” which, given a video URL, automatically fetches the subtitles. You can find and install community nodes easily via the N8N settings.

2. Generating YouTube Chapters

This was my first major test. I needed the AI to analyze the transcript and identify logical sections, outputting them in the standard YouTube chapter format (00:00:00 - Chapter Title).

• Local Attempt (Ollama + Gemma 3): I configured an N8N “Basic LLM Chain” node to use my local Ollama instance running Gemma 3. I set the context length to 8000 tokens and the temperature very low (0.1) to prevent creativity and stick to the facts. The prompt was carefully crafted to explain the desired format, including examples.

  Result: Disappointing. While it generated *some* chapters, it stopped very early in the video (around 6 minutes for a 25+ minute video), missing the vast majority of the content. Despite the model’s theoretical capabilities, it failed this task with the given transcript length and my hardware (RTX 8000 GPUs – good, but maybe not enough or Ollama/model limitations).

• Cloud Attempt (Google AI Studio + Gemini 2.5 Pro): I switched the LLM node to use the Google Gemini connection, specifically targeting Gemini 2.5 Pro with a temperature of 0.2.

  Result: Much better! Gemini 2.5 Pro processed the entire transcript and generated accurate, well-spaced chapters covering the full length of the video. Its larger context window and potentially more advanced reasoning capabilities handled the task effectively.

For chapter generation, the cloud-based Gemini 2.5 Pro was the clear winner in my tests.

3. Crafting the Perfect LinkedIn Post

Next, I wanted to automate the announcement post for LinkedIn. Here, the prompt engineering became even more crucial. I didn’t just want a generic summary; I wanted it to sound like *me*.

• Technique: I fed the AI (Gemini 2.5 Pro again, given the success with chapters) a detailed prompt that included:

  • The task description (create a LinkedIn post).
  • The video transcript as context.
  • Crucially: Examples of my previous LinkedIn posts. This helps the AI learn and mimic my writing style and tone.
  • Instructions on formatting and including relevant hashtags.
  • Using N8N variables to insert the specific video link dynamically.

• Result: Excellent! The generated post was remarkably similar to my usual style, captured the video’s essence, included relevant tags, and was ready to be published (with minor review).

4. Automating Blog Post Creation

The final piece was generating a draft blog post directly from the transcript.

• Technique: Similar to the LinkedIn post, but with different requirements. The prompt instructed Gemini 2.5 Pro to:

  • Generate content in HTML format for easy pasting into my blog.
  • Avoid certain elements (like quotation marks unless necessary).
  • Recognize and correctly format specific terms (like “Quadrata”, my name “Dimitri Bellini”, or the “ZabbixItalia Telegram Channel” – https://t.me/zabbixitalia).
  • Structure the text logically with headings and paragraphs.
  • Include basic SEO considerations.

• Result: Success again! While it took a little longer to generate (likely due to the complexity and length), the AI produced a well-structured HTML blog post draft based on the video content. It correctly identified and linked the channels mentioned and formatted the text as requested. This provides a fantastic starting point, saving significant time.

Key Takeaways and Challenges

This experiment highlighted several important points:

• Prompt Engineering is King: The quality of the AI’s output is directly proportional to the quality and detail of your prompt. Providing examples, clear formatting instructions, and context is essential. Using AI itself (via web interfaces) to help refine prompts is a valid strategy!
• Cloud vs. Local AI Trade-offs:

  • Cloud (Gemini 2.5 Pro): Generally more powerful, handled long contexts better in my tests, easier setup (API key). However, subject to API limits (even free tiers have them, especially for frequent/heavy use) and potential costs.
  • Local (Ollama/Gemma 3): Full control, no API limits/costs (beyond hardware/electricity). However, requires capable hardware (especially GPU RAM for large contexts/models), and smaller models might struggle with complex reasoning or very long inputs. Performance was insufficient for my chapter generation task in this test.

• Model Capabilities Matter: Gemini 2.5 Pro’s large context window and reasoning seemed better suited for processing my lengthy video transcripts compared to the 27B parameter Gemma 3 model run locally (though further testing with different local models or configurations might yield different results).
• Temperature Setting: Keeping the temperature low (e.g., 0.1-0.2) is vital for tasks requiring factual accuracy and adherence to instructions, minimizing AI “creativity” or hallucination.
• N8N is Powerful: It provides the perfect framework to chain these steps together, handle variables, connect to different services (local or cloud), and parse outputs (like the Structured Output Parser node for forcing JSON).

Conclusion and Next Steps

Overall, I’m thrilled with the results! Using N8N combined with a capable AI like Google’s Gemini 2.5 Pro allowed me to successfully automate the generation of YouTube chapters, LinkedIn posts, and blog post drafts directly from my video transcripts. While the local AI approach didn’t quite meet my needs for this specific task *yet*, the cloud solution provided a significant time-saving and genuinely useful outcome.

The next logical step is to integrate the final publishing actions directly into N8N using its dedicated nodes for YouTube (updating descriptions with chapters) and LinkedIn (posting the generated content). This would make the process almost entirely hands-off after the initial video upload.

This is a real-world example of how AI can move beyond novelty and become a practical tool for automating tedious tasks. It’s not perfect, and requires setup and refinement, but the potential to streamline workflows is undeniable.

What do you think? Have you tried using N8N or similar tools for AI-powered automation? What are your favourite use cases? Let me know in the comments below! And if you found this interesting, give the video a thumbs up and consider subscribing to Quadrata for more content on open source and IT.

Thanks for reading, and see you next week!

Bye everyone,
Dimitri

Zabbix 7.0 LTS is Almost Here: A Deep Dive into the Next Generation of Monitoring

Good morning everyone, Dimitri Bellini here from the Quadrata channel! It’s fantastic to have you back on my corner of the internet dedicated to the world of open source and IT. First off, a huge thank you – we’ve recently crossed the 800 subscriber mark, which is amazing! My goal is to hit 1000 this year, so if you haven’t already, please consider subscribing!

This week, we’re diving into something truly exciting: the upcoming release of Zabbix 7.0. It’s just around the corner, and while we wait for the final release, the Release Candidate 1 already packs all the features we expect to see. So, let’s explore what makes this version so special.

Why Zabbix 7.0 LTS is a Game Changer

Zabbix 7.0 isn’t just another update; it’s an LTS (Long Term Support) version. This is crucial because LTS releases are designed for stability and longevity, typically receiving support for three years (extendable to five for security). This makes them the ideal choice for production environments where reliability and long-term planning are paramount. Updating complex systems isn’t always easy, so an LTS version provides that much-needed stability.

The previous LTS, version 6.0, was released back in February 2022. While the usual cycle is about 1.5 years between LTS versions, 7.0 took a bit longer. This extended development time was necessary to incorporate significant changes, both under the hood and in terms of user-facing features. Trust me, the wait seems worth it!

Don’t panic if you’re on 6.0 – full support continues until February 28, 2025. You have ample time to plan your migration. In fact, I often suggest waiting for a few minor point releases (like 7.0.3 or 7.0.5) to let the dust settle before deploying in critical environments.

Bridging the Gap: Key Enhancements Since Zabbix 6.0

Since many users stick with LTS versions in production and might have skipped the intermediate 6.2 and 6.4 releases, I want to cover the major improvements introduced between 6.0 and 7.0. There’s a lot to unpack!

Performance, Scalability, and Architecture Boosts

• Automatic Configuration Sync (Server-Proxy-Agent): This is huge! Previously, configuration changes on the GUI could take time (often a minute or more by default) to propagate through the server, proxies, and agents due to database polling cycles. Now, changes trigger near-instantaneous updates across the chain using a differential sync mechanism. This means faster deployments of new hosts and checks, and less load as only the *changes* are pushed.
• SNMP Bulk Monitoring: Instead of multiple individual `get` requests, Zabbix can now group SNMP requests, reducing load on both the Zabbix server and the monitored devices.
• Asynchronous Polling: Passive checks (SNMP, HTTP, Zabbix agent passive) are now asynchronous. The server sends the request and moves on, processing the data as it arrives. This significantly improves scalability, especially for large environments.
• Proxy High Availability (HA) and Load Balancing: Finally! You can group proxies together for automatic failover and load balancing. If a proxy in the group fails, its assigned hosts are automatically redistributed to other available proxies in the group. We’ll look at a demo of this shortly!
• Customizable Item Timeouts: The previous global 30-second timeout limit is gone. Now you can set timeouts per item (up to 10 minutes), providing flexibility for checks that naturally take longer.
• Proxy Backward Compatibility: Upgrading is now less stressful. A Zabbix 7.0 server can work with 6.0 proxies in an “outdated” state. They’ll continue sending data and executing remote commands, giving you time to upgrade the proxies without a complete “big bang” cutover. Configuration updates for hosts on outdated proxies won’t work, however.

Enhanced Integrations and User Management

• LDAP/AD Just-in-Time (JIT) User Provisioning: A highly requested feature for enterprise environments. Zabbix can now automatically create and update user accounts based on your Active Directory or LDAP server upon their first login. You can map attributes like email and phone numbers, and even assign Zabbix roles based on AD/LDAP groups. Plus, support for multiple LDAP servers is included.
• Expanded Vault Support: Alongside HashiCorp Vault, Zabbix 7.0 now integrates with CyberArk for external secret management.
• Real-time Data Streaming: Push metrics and events efficiently to external systems like Kafka or Splunk. Crucially, you can filter what gets sent based on tags (e.g., send only high-priority events, or only network-related metrics), allowing for sophisticated data routing to different data lakes or analysis tools.

Improved Monitoring & User Experience

• Advanced Web Monitoring with Selenium: This is potentially revolutionary for Zabbix. The web monitoring capabilities have been rebuilt using Selenium, allowing you to simulate real user interactions (clicking buttons, filling forms) and monitor user experience, page performance, and functionality directly within Zabbix.
• Manual Problem Suppression: Acknowledge known issues or problems occurring during maintenance windows by suppressing them temporarily (indefinitely or for a defined period). They’ll reappear if the issue persists after the suppression window.
• Clearer Agent Availability: The agent availability status is now more intuitive, incorporating a heartbeat mechanism to clearly show if an agent (active or passive) is truly up and running. The corresponding dashboard widget has also been revamped.
• UI and Template Upgrades: Continuous improvements to the graphical interface, new widgets (like improved pie charts/gauges), and significantly enhanced templates, especially for VMware, AWS, Azure, and Google Cloud monitoring.
• OS Prototype Functionality Extension: More flexibility in managing discovered hosts, including adding templates and tags more easily.

Hands-On: Exploring Proxy High Availability (Demo Recap)

I set up a quick Docker environment with a Zabbix 7.0 RC1 server and three proxies to test the new Proxy HA feature. Here’s a summary of how it works:

1. Create a Proxy Group: Under `Administration -> Proxy groups`, define a group (e.g., “MyHAProxies”). Set the `Failover period` (how quickly hosts move after a proxy failure – I used 1 minute for the demo) and the `Minimum available proxies` (the minimum number needed for the group to be considered operational).
2. Assign Proxies to the Group: Edit each proxy you want in the group (`Administration -> Proxies`) and assign it to the created Proxy Group. You also need to specify the proxy address for active agents, as agent configuration changes with this feature.
3. Assign Hosts to the Proxy Group: When configuring a host, instead of selecting a specific proxy, select the Proxy Group. Zabbix automatically assigns the host to the least loaded proxy within that group initially.
4. Simulate Failure: I stopped one of the proxy containers.
5. Observe Failover: After the configured failover period (1 minute), Zabbix detected the proxy was down. The hosts monitored by that proxy were automatically redistributed among the remaining two active proxies in the group. Crucially, checking the `Latest data` for a moved host showed minimal interruption in data collection.
6. Recovery: When I restarted the stopped proxy, Zabbix detected it coming back online. After a stabilisation period (to avoid flapping), it automatically started rebalancing the hosts back across all three proxies.
7. Monitoring the Group: A new internal item `zabbix[proxy_group,,state]` allows you to monitor the health of the proxy group itself (e.g., online, degraded). This is essential for alerting! (Note: I’ve asked the Zabbix team to add this to the default server health template).

This feature significantly enhances the resilience of your Zabbix monitoring infrastructure, especially in distributed environments.

Planning Your Upgrade to Zabbix 7.0

Upgrading requires careful planning. Here’s a checklist:

• Read the Release Notes: Understand all the new features and changes.
• Check Requirements: Ensure your OS, database, and library versions meet the new requirements for Zabbix 7.0. You might need to upgrade your underlying infrastructure (e.g., moving from RHEL 7 to RHEL 8 or 9).
• Review Breaking Changes & Known Issues: The official Zabbix documentation has sections dedicated to these. Pay close attention to avoid surprises.
• CRITICAL: Primary Keys: Zabbix 6.0 introduced optional primary keys for history tables. If you upgraded from 5.x to 6.x and didn’t manually add them (it could be a slow process), **now is the time**. While Zabbix 7.0 *might* run without them, future versions *will require* them. Adding primary keys also provides a significant performance boost (often 20%+). Factor this database maintenance into your upgrade plan if needed.
• Follow the Official Upgrade Procedure: Stick to the step-by-step guide in the Zabbix documentation for your specific environment.
• Backup Everything: Before you start, ensure you have reliable backups of your Zabbix database and configuration files.

Conclusion: Get Ready for Zabbix 7.0!

Zabbix 7.0 LTS is shaping up to be a monumental release. The focus on scalability, high availability, usability, and deeper integration makes it incredibly compelling for both existing users and those considering Zabbix for their monitoring needs. Features like Proxy HA, LDAP JIT provisioning, and the new Selenium-based web monitoring are truly exciting developments.

I’ll definitely be exploring the new web monitoring capabilities in more detail once the documentation is fully available, so stay tuned for that!

Unlock Your Documents Potential with Ragflow: An Open-Source RAG Powerhouse

Good morning, everyone! Dimitri Bellini here, back on the Quadrata channel – your spot for diving into the exciting world of open source and IT tech. Today, we’re tackling a topic some of you have asked about: advanced solutions for interacting with your own documents using AI.

I sometimes wait to showcase software until it’s a bit more polished, and today, I’m excited to introduce a particularly interesting one: Ragflow.

What is RAG and Why Should You Care?

We’re diving back into the world of RAG solutions – Retrieval-Augmented Generation. It sounds complex, but the core idea is simple and incredibly useful: using your *own* documents (manuals, reports, notes, anything on your disk) as a private knowledge base for an AI.

Instead of relying solely on the general knowledge (and potential inaccuracies) of large language models (LLMs), RAG lets you get highly relevant, context-specific answers based on *your* information. This is a practical, powerful use case for AI, moving beyond generic queries to solve specific problems using local data.

Introducing Ragflow: A Powerful Open-Source RAG Solution

Ragflow (find it on GitHub!) stands out from other RAG tools I’ve explored. It’s not just a basic framework; it’s shaping up to be a comprehensive, business-oriented platform. Here’s why it caught my eye:

• Open Source: Freely available and community-driven.
• Complete Solution: Offers a wide range of features out-of-the-box.
• Collaboration Ready: Designed for teams to work on shared knowledge bases.
• Easy Installation: Uses Docker Compose for a smooth setup.
• Local First: Integrates seamlessly with local LLM providers like Ollama (which I use).
• Rapid Development: The team is actively adding features and improvements.
• Advanced Techniques: Incorporates methods like Self-RAG and Raptor for better accuracy.
• API Access: Allows integration with other applications.

Diving Deeper: How Ragflow Enhances RAG

Ragflow isn’t just about basic document splitting and embedding. It employs sophisticated techniques:

• Intelligent Document Analysis: It doesn’t just grab text. Ragflow performs OCR and analyzes document structure (understanding tables in Excel, layouts in presentations, etc.) based on predefined templates. This leads to much better comprehension and more accurate answers.
• Self-RAG: A framework designed to improve the quality and factuality of the LLM’s responses, reducing the chances of the AI “inventing” answers (hallucinations) when it doesn’t know something.
• Raptor: This technique focuses on the document processing phase. For long, complex documents, Raptor builds a hierarchical summary or tree of concepts *before* chunking and embedding. This helps the AI maintain context and understand the overall topic better.

These aren’t trivial features; they represent significant steps towards making RAG systems more reliable and useful.

Getting Started: Installing Ragflow (Step-by-Step)

Installation is straightforward thanks to Docker Compose. Here’s how I got it running:

1. Clone the Repository (Important Tip!): Use the `–branch` flag to specify a stable release version. This saved me some trouble during testing. Replace `release-branch-name` with the desired version (e.g., `0.7.0`).

   git clone --branch release-branch-name https://github.com/infiniflow/ragflow.git

2. Navigate to the Docker Directory:

   cd ragflow/docker

3. Make the Entrypoint Script Executable:

   chmod +x entrypoint.sh

4. Start the Services: This will pull the necessary images (including Ragflow, MySQL, Redis, MinIO, Elasticsearch) and start the containers.

   docker-compose up -d

   Note: Be patient! The Docker images, especially the main Ragflow one, can be quite large (around 9GB in my tests), so ensure you have enough disk space.

Once everything is up, you can access the web interface (usually at `http://localhost:80` or check the Docker Compose file/logs for the exact port).

A Look Inside: Configuring and Using Ragflow

The web interface is clean and divided into key sections: Knowledge Base, Chat, File Manager, and Settings.

Setting Up Your AI Models (Ollama Example)

First, you need to tell Ragflow which AI models to use. Go to your profile settings -> Model Providers.

• Click “Add Model”.
• Select “Ollama”.
• Choose the model type: “Chat” (for generating responses) or “Embedding” (for analyzing documents). You’ll likely need one of each.
• Enter the **exact** model name as it appears in your Ollama list (e.g., `mistral:latest`, `nomic-embed-text:latest`).
• Provide the Base URL for your Ollama instance (e.g., `http://your-ollama-ip:11434`).
• Save the model. Repeat for your embedding model if it’s different. I used `nomic-embed-text` for embeddings and `weasel-lm-7b-v1-q5_k_m` (a fine-tuned model) for chat in my tests.

Creating and Populating a Knowledge Base (Crucial Settings)

This is where your documents live.

• Create a new Knowledge Base and give it a name.
• Before Uploading: Go into the KB settings. This is critical! Define:

  • Language: The primary language of your documents.
  • Chunking Method: How documents are split. Ragflow offers templates like “General”, “Presentation”, “Manual”, “Q&A”, “Excel”, “Resume”. Choose the one that best fits your content. I used “Presentation” for my Zabbix slides.
  • Embedding Model: Select the Ollama embedding model you configured earlier.
  • Raptor: Enable this for potentially better context handling on complex docs.

• Upload Documents: Now you can upload files or entire directories.
• Parse Documents: Click the “Parse” button next to each uploaded document. Ragflow will process it using the settings you defined (OCR, chunking, embedding, Raptor analysis). You can monitor the progress.

Building Your Chat Assistant

This connects your chat model to your knowledge base.

• Create a new Assistant.
• Give it a name and optionally an avatar.
• Important: Set an “Empty Response” message (e.g., “I couldn’t find information on that in the provided documents.”). This prevents the AI from making things up.
• Add a welcome message.
• Enable “Show Citation”.
• Link Knowledge Base: Select the KB you created.
• Prompt Engine: Review the system prompt. The default is usually quite good, instructing the AI to answer based *only* on the documents.
• Model Setting: Select the Ollama chat model you configured. Choose a “Work Mode” like “Precise” to encourage focused answers.
• (Optional) Re-ranking Model: I skipped this in version 0.7 due to some issues, but it’s a feature to watch.
• Confirm and save.

Putting Ragflow to the Test (Zabbix Example)

I loaded my Zabbix presentation slides and asked the assistant some questions:

• Explaining Zabbix log file fields.
• Identifying programming languages used in Zabbix components.
• Differentiating between Zabbix Agent Passive and Active modes.
• Describing the Zabbix data collection flow.

The results were genuinely impressive! Ragflow provided accurate, detailed answers, citing the specific slides it drew information from. There was only one minor point where I wasn’t entirely sure if the answer was fully grounded in the text or slightly inferred, but overall, the accuracy and relevance were excellent, especially considering it was analyzing presentation slides.

Integrating Ragflow with Other Tools via API

A standout feature is the built-in API. For each assistant you create, you can generate an API key. This allows external applications to query that specific assistant and its associated knowledge base programmatically – fantastic for building custom integrations.

Final Thoughts and Why Ragflow Stands Out

Ragflow is a compelling RAG solution. Its focus on accurate document analysis, integration of advanced techniques like Self-RAG and Raptor, ease of use via Docker and Ollama, and the inclusion of collaboration and API features make it feel like a mature, well-thought-out product, despite being relatively new.

While it’s still evolving (as seen with the re-ranking feature I encountered), it’s already incredibly capable and provides a robust platform for anyone serious about leveraging their own documents with AI.

PFSense vs. OPNsense: Choosing Your Open Source Firewall Champion

Good morning everyone! Dimitri Bellini here, back with you on Quadrata, my channel dedicated to the fascinating world of open source and IT. You might notice a new backdrop today – still working on the aesthetics, but the content is what matters!

Today, we’re shifting gears slightly from our usual Zabbix discussions to explore another critical area in the open-source landscape: firewall solutions. I’m not just talking about the basic firewall tools within a Linux distro, but dedicated distributions that create powerful network appliances.

Why Dedicated Open Source Firewalls?

Why opt for something like PFSense or OPNsense instead of simpler solutions? Well, whether you’re running a home lab, a small office, or even looking for robust solutions without an enterprise price tag, these distributions offer incredible value. They are designed to be:

• Robust and Reliable: Often running for years without issues, especially on dedicated hardware (like those small, multi-port appliances you often see).
• Feature-Rich: Going beyond basic packet filtering to offer comprehensive network management.
• Versatile: Capable of handling routing, complex NAT scenarios, port forwarding, secure VPN access (site-to-site or remote user), intrusion detection, content filtering, and much more.

Essentially, if you need to manage traffic between your LAN and the internet, securely expose services, connect different locations, or simply have granular control over your network, these tools are invaluable. You can test them easily on a VM using VMware, VirtualBox, or my personal favourite, Proxmox, or deploy them on dedicated, often fanless, hardware appliances.

Meet the Contenders: PFSense and OPNsense

Two names dominate the open-source firewall scene today: PFSense and OPNsense. You’ve likely heard of PFSense; it’s been a stalwart in this space for decades. OPNsense (or OpenSense, as some say) is a newer, but rapidly growing, alternative.

Interestingly, both projects share common ancestry, tracing their roots back to the venerable M0n0wall project. PFSense emerged as an effort to expand M0n0wall’s capabilities. Later, around 2015, as M0n0wall’s development ceased, OPNsense was forked, partly from PFSense code, by developers aiming for a more modern approach and a fully open-source path, perhaps diverging from the direction PFSense was taking with its commercial backing.

PFSense: The Established Powerhouse

PFSense, particularly in its Community Edition (CE), offers a vast array of features:

• Core Networking: Stateful Packet Inspection, robust NAT capabilities, advanced routing.
• VPN Support: Includes OpenVPN, IPsec, and WireGuard (WireGuard often requires installing a package).
• Security Features: Intrusion Detection Systems (IDS/IPS via packages like Suricata or Snort), Captive Portal, Anti-Lockout rules.
• Management: A comprehensive web GUI for configuration, monitoring, and High Availability (HA) setups using CARP.
• Extensibility: A package manager allows adding functionality like PFBlockerNG (IP/DNS blocking), Zabbix agents/proxies, and more.

PFSense is backed by the company NetGate, which offers commercial support and hardware appliances running PFSense Plus. While based on the community version, PFSense Plus is positioned as a separate product with potentially faster updates, additional features (especially related to performance and hardware offloading), and professional support. This is a key distinction: PFSense CE updates might sometimes lag behind the Plus version.

The user interface, while powerful, is often described as more traditional or “retro” compared to OPNsense. It’s built on a FreeBSD base (currently FreeBSD 14 for the latest CE at the time of recording).

OPNsense: The Modern, Fully Open Challenger

OPNsense aims for a similar feature set but with a strong emphasis on usability, security, and a truly open-source model:

• Core Networking: All the essentials like Stateful Packet Inspection, NAT, Routing, VLAN support, GeoIP blocking.
• VPN Support: Also features OpenVPN, IPsec, and WireGuard integrated.
• Security Enhancements: Notably includes built-in Two-Factor Authentication (2FA) support, which is a great plus for securing the firewall itself. It also has strong reporting and traffic visualization tools (Insight).
• Management: Features a modern, clean, and arguably more intuitive web GUI. High Availability is also supported.
• Extensibility: Offers plugins for various services, including Zabbix agents/proxies, Let’s Encrypt certificate management, etc.

The biggest philosophical difference lies in its licensing and development model. OPNsense is fully open source under a BSD license. While there’s a “Business Edition” offering professional support (from Deciso B.V., the company heavily involved), the software itself is identical to the free version. There are no feature differences or separate “plus” tiers. Updates tend to be more frequent, often released on a fixed schedule (e.g., twice a year for major releases, with patches in between).

It’s also based on FreeBSD (currently FreeBSD 13.x at the time of recording, though this changes).

Key Differences Summarized

• Licensing Model: PFSense has CE (free) and Plus (commercial, tied to NetGate hardware or subscription). OPNsense is fully open source, with optional paid support that doesn’t unlock extra features.
• User Interface: OPNsense generally offers a more modern and potentially user-friendly GUI. PFSense has a more traditional, albeit very functional, interface.
• Development & Updates: OPNsense follows a more predictable release schedule and is community-driven (with corporate backing for infrastructure/support). PFSense CE updates can sometimes lag behind the commercial Plus version driven by NetGate.
• Out-of-the-Box Features: OPNsense includes things like 2FA and enhanced reporting built-in. PFSense might require packages for some equivalent functionalities (like WireGuard initially).
• Commercial Backing: PFSense is directly backed and developed by NetGate. OPNsense has backing from Deciso for support/infrastructure but development is more community-focused.

Which One Should You Choose?

Both are excellent, mature, and highly capable solutions. The choice often comes down to your priorities:

• Choose OPNsense if: You prioritize a modern UI, a predictable release cycle, built-in features like 2FA, and a strictly open-source philosophy without tiered versions. It’s often recommended for newcomers due to its interface.
• Choose PFSense (CE or Plus) if: You’re comfortable with its traditional interface, need features potentially exclusive to or better optimized in the Plus version (especially with NetGate hardware), or prefer the ecosystem and support structure provided by NetGate. For business deployments where guaranteed support and optimized hardware/software integration are key, buying a NetGate appliance with PFSense Plus is a very compelling option – you get the hardware, software license, and support in one package, often at a reasonable price point.

Personally, while I love the open-source nature of OPNsense, I also understand the need for companies like NetGate to fund development. If I were deploying for a client needing robust support, the NetGate appliance route with PFSense Plus offers significant peace of mind.

Wrapping Up

Both PFSense and OPNsense empower you to build incredibly powerful network security and management solutions using open-source software. They run on standard hardware or VMs, offer extensive features, and have active communities.

I encourage you to download both and try them out in a virtual environment to see which interface and workflow you prefer. They are fantastic tools for learning and for securing your networks.

Automate Smarter, Not Harder: Exploring N8n for AI-Powered Workflows

Good morning everyone! Dimitri Bellini here, back on Quadrata, my channel where we dive into the fascinating world of open source and IT. As I always say, I hope you find these topics as exciting as I do!

This week, we’re venturing back into the realm of artificial intelligence, but with a twist. We’ll be looking at an incredibly interesting, user-friendly, and – you guessed it – open-source tool called N8n (pronounced “N-eight-N”). While we’ve explored similar solutions before, N8n stands out with its vibrant community and powerful capabilities, especially its recent AI enhancements.

What is N8n and Why Should You Care?

At its core, N8n is a Workflow Automation Tool. It wasn’t born solely for AI; its primary goal is to help you automate sequences of tasks, connecting different applications and services together. Think of it as a visual way to build bridges between the tools you use every day.

Why opt for a tool like N8n instead of just writing scripts in Python or another language? The key advantage lies in maintainability and clarity. While scripts work, revisiting them months later often requires deciphering complex code. N8n uses a graphical user interface (GUI) with logical blocks. This visual approach makes workflows much easier to understand, debug, and modify, even long after you’ve created them. For me, especially for complex or evolving processes, this visual clarity is a huge plus.

The best part? You can install it right on your own hardware or servers, keeping your data and processes in-house.

Key Functionalities of N8n

N8n packs a punch when it comes to features:

• Visual Workflow Builder: Create complex automation sequences graphically using a web-based GUI. Drag, drop, and connect nodes to define your logic.
• Extensive Integrations: It boasts a vast library of pre-built integrations for countless applications and services (think Google Suite, Microsoft tools, databases, communication platforms, and much more).
• Customizable Nodes: If a pre-built integration doesn’t exist, you can create custom nodes, for example, to execute your own Python code within a workflow.
• AI Agent Integration: This is where it gets really exciting for us! N8n now includes dedicated modules (built using Langchain) to seamlessly integrate AI models, including self-hosted ones like those managed by Ollama.
• Data Manipulation: N8n isn’t just about triggering actions. It allows you to transform, filter, merge, split, and enrich data as it flows through your workflow, enabling sophisticated data processing.
• Strong Community & Templates: Starting from scratch can be daunting. N8n has a fantastic community that shares workflow templates. These are invaluable for learning and getting started quickly.

Getting Started: Installation with Docker

My preferred method for running N8n, especially for testing and home use, is using Docker and Docker Compose. It’s clean, contained, and easy to manage. While you *can* install it using npm, Docker keeps things tidy.

1. Use Docker Compose: I started with the official Docker Compose setup provided on the N8n GitHub repository. This typically includes N8n itself and a Postgres database for backend storage (though SQLite is built-in for simpler setups).
2. Configure Environment: Modify the .env file to set up database credentials and any other necessary parameters.
3. Launch: Run docker-compose up -d to start the containers.
4. Access: You should then be able to access the N8n web interface, usually at http://localhost:5678. You’ll need to create an initial user account.
5. Connect AI (Optional but Recommended): Have your Ollama instance running if you plan to use local Large Language Models (LLMs).

N8n in Action: Some Examples

Let’s look at a few examples I demonstrated in the video to give you a feel for how N8n works:

Example 1: The AI Calculator

This was a simple workflow designed to show the basic AI Agent block.

• It takes a mathematical question (e.g., “4 plus 5”).
• Uses the AI Agent node configured with an Ollama model (like Mistral) and Postgres for memory (to remember conversation context).
• The “tool” in this case is a simple calculator function.
• The AI understands the request, uses the tool to get the result (9), and then formulates a natural language response (“The answer is 9”).
• The execution log is fantastic here, showing step-by-step how the input flows through chat memory, the LLM, the tool, and back to the LLM for the final output.

Example 2: AI Web Agent with SERP API

This workflow demonstrated fetching external data and using AI to process it:

• It used the SERP API tool (requiring an API key) to perform a web search (e.g., “latest news about Zabbix”).
• The search results were passed to the first AI Agent (using Ollama) for initial processing/summarization.
• Crucially, I showed how to pass the output of one node as input to the next using N8n’s expression syntax ({{ $json.output }} or similar).
• A second AI Agent node was added with a specific prompt: “You are a very good AI agent specialized in blog writing.” This agent took the summarized web content and structured it into a blog post format.

Example 3: Simple Web Scraper

This showed basic web scraping without external APIs:

• Used the built-in HTTP Request node to fetch content from specific web pages.
• Applied filtering and data manipulation nodes to limit the number of pages and extract relevant text content (cleaning HTML).
• Passed the cleaned text to Ollama for summarization.
• The visual execution flow clearly showed each step turning green as it completed successfully.

I also briefly mentioned a much more complex potential workflow involving document processing (PDFs, text files), using Quadrant as a vector database, and Mistral for creating embeddings to build a Retrieval-Augmented Generation (RAG) system – showcasing the scalability of N8n.

Conclusion: Your Automation Powerhouse

N8n is a remarkably powerful and flexible tool for anyone looking to automate tasks, whether simple or complex. Its visual approach makes automation accessible, while its deep integration capabilities, including first-class support for AI models via tools like Ollama, open up a world of possibilities.

Being open-source and self-hostable gives you complete control over your workflows and data. Whether you’re automating IT processes, integrating marketing tools, processing data, or experimenting with AI, N8n provides a robust platform to build upon.

Simplify Your Web Services: An Introduction to Nginx Proxy Manager

Good morning everyone, Dimitri Bellini here from Quadrata! While my channel often dives deep into the world of Zabbix and open-source monitoring, today I want to shift gears slightly. I’ve realized that some foundational concepts, while powerful, aren’t always common knowledge, and sharing them can be incredibly helpful.

This week, we’re exploring Nginx Proxy Manager. It’s not a revolutionary concept in itself, but it’s a tool that significantly simplifies managing access to your web services, especially when dealing with HTTPS and multiple applications behind a single IP address.

What Exactly is Nginx Proxy Manager?

At its core, Nginx Proxy Manager is a reverse proxy built on top of the popular Nginx web server. But what makes it special? It packages several essential functionalities into one easy-to-manage solution, accessible via a clean web interface.

Here are its main characteristics:

• Reverse Proxy Functionality: It acts as an intermediary, allowing you to securely expose multiple internal web services (like your Zabbix frontend, internal wikis, etc.) to the internet using potentially just one public IP address. Instead of exposing your services directly, the proxy handles incoming requests and forwards them appropriately.
• Free SSL Certificates with Let’s Encrypt: It seamlessly integrates with Let’s Encrypt, enabling you to obtain and, crucially, automatically renew free, trusted SSL/TLS certificates for your domains. This makes setting up HTTPS incredibly straightforward.
• User-Friendly Web Interface: This is a huge plus! While configuring Nginx via text files is powerful (Infrastructure as Code!), it can be complex and time-consuming, especially if you don’t do it often. The web UI simplifies creating proxy hosts, managing certificates, and viewing logs, making it accessible even if you’re not an Nginx expert. Remembering complex configurations months later becomes much easier!
• Docker-Based: It runs as a Docker container, bundling all dependencies (Nginx, Certbot for Let’s Encrypt, the web UI) together. This makes installation, updates, and management very convenient.

Understanding Reverse Proxies (and why they’re not Forward Proxies)

It’s important to distinguish a reverse proxy from the traditional “forward” proxy many of us remember from the early internet days. A forward proxy sits between users on a network and the *external* internet, often used for caching or filtering outbound requests.

A reverse proxy does the opposite. It sits in front of your *internal* web servers and manages incoming requests from the *external* internet. When someone types zbx1.yourdomain.com, the request hits the reverse proxy first. The proxy then looks at the requested domain and forwards the traffic to the correct internal server (e.g., the machine hosting your Zabbix web GUI).

This is essential if you have only one public IP but want to host multiple websites or services using standard HTTPS (port 443).

The Crucial Role of DNS and Let’s Encrypt

DNS: Directing Traffic

How does a user’s browser know where to find your reverse proxy? Through DNS! You need to configure your public DNS records (usually on your domain registrar’s platform or DNS provider) so that the domain names you want to expose (e.g., zbx1.yourdomain.com, wiki.yourdomain.com) point to the public IP address of your Nginx Proxy Manager server. This is typically done using:

• A Record: Points a domain directly to an IPv4 address.
• CNAME Record: Points a domain to another domain name (often more flexible). For example, zbx1.yourdomain.com could be a CNAME pointing to proxy.yourdomain.com, which then has an A record pointing to your public IP.

Without correct DNS setup, requests will never reach your proxy.

Let’s Encrypt: Free and Automated SSL

Let’s Encrypt is a non-profit Certificate Authority that provides free, domain-validated SSL/TLS certificates. Before Let’s Encrypt, obtaining trusted certificates often involved significant cost and manual processes. Let’s Encrypt has democratized HTTPS, making it easy and free for everyone.

The main “catch” is that their certificates have a shorter validity period (e.g., 90 days). This is where Nginx Proxy Manager shines – it handles the initial domain validation (“challenge”) and the periodic, automatic renewal process, ensuring your sites remain secure without manual intervention.

Getting Started: Installation via Docker Compose

Installing Nginx Proxy Manager is straightforward using Docker Compose. Here’s a basic docker-compose.yml file similar to the one I use:

version: '3.8'

services:

  app:

    image: 'jc21/nginx-proxy-manager:latest'

    restart: unless-stopped

    ports:

      # Public HTTP Port for Let's Encrypt challenges

      - '80:8080' 

      # Public HTTPS Port

      - '443:443' 

      # Admin Web UI Port (access this in your browser)

      - '81:81' 

    volumes:

      - ./data:/data

      - ./letsencrypt:/etc/letsencrypt

    # For production, consider using a proper database like MySQL/MariaDB instead of SQLite

    # environment:

    #   DB_MYSQL_HOST: "db"

    #   DB_MYSQL_PORT: 3306

    #   DB_MYSQL_USER: "npm"

    #   DB_MYSQL_PASSWORD: "your_password"

    #   DB_MYSQL_NAME: "npm"

    # depends_on:

    #   - db



# Uncomment this section if using MySQL/MariaDB

#  db:

#    image: 'jc21/mariadb-aria:latest'

#    restart: unless-stopped

#    environment:

#      MYSQL_ROOT_PASSWORD: 'your_root_password'

#      MYSQL_DATABASE: 'npm'

#      MYSQL_USER: 'npm'

#      MYSQL_PASSWORD: 'your_password'

#    volumes:

#      - ./data/mysql:/var/lib/mysql

    

Key Ports Explained:

• 80:8080: Maps external port 80 to the container’s port 8080. Port 80 is needed externally for Let’s Encrypt HTTP-01 challenges.
• 443:443: Maps external port 443 (standard HTTPS) to the container’s port 443. This is where your proxied traffic will arrive.
• 81:81: Maps external port 81 to the container’s port 81. You’ll access the Nginx Proxy Manager admin interface via http://your_server_ip:81.

Volumes:

• ./data:/data: Stores configuration data (using SQLite by default).
• ./letsencrypt:/etc/letsencrypt: Stores your SSL certificates.

To start it, simply navigate to the directory containing your docker-compose.yml file and run:

docker-compose up -d

Note: For environments with many sites, the official documentation recommends using MySQL or MariaDB instead of the default SQLite for better performance.

Configuring a Proxy Host: A Quick Walkthrough

Once the container is running, access the web UI (http://your_server_ip:81). The default login credentials are usually admin@example.com / changeme (you’ll be prompted to change these immediately).

From the dashboard, you’ll see options like Proxy Hosts, Redirection Hosts, Streams (for TCP/UDP forwarding), and 404 Hosts.

To expose an internal service (like Zabbix):

1. Go to Hosts -> Proxy Hosts.
2. Click Add Proxy Host.
3. Details Tab:

   • Domain Names: Enter the public domain name(s) you configured in DNS (e.g., zbx1.quadrata.it).
   • Scheme: Select the protocol your *internal* service uses (usually http for Zabbix web UI).
   • Forward Hostname / IP: Enter the internal IP address of your Zabbix server (e.g., 192.168.1.100).
   • Forward Port: Enter the internal port your service listens on (e.g., 80 for Zabbix web UI).
   • Enable options like Block Common Exploits and Websockets Support if needed.

4. SSL Tab:

   • Select Request a new SSL Certificate from the dropdown.
   • Enable Force SSL (redirects HTTP to HTTPS).
   • Enable HTTP/2 Support.
   • Enter your email address (for Let’s Encrypt notifications).
   • Agree to the Let’s Encrypt Terms of Service.

5. Click Save.

Nginx Proxy Manager will now attempt to obtain the certificate from Let’s Encrypt using the domain you provided. If successful, the entry will show as green/online. You should now be able to access your internal Zabbix interface securely via https://zbx1.quadrata.it!

There’s also an Advanced tab where you can add custom Nginx configuration snippets for more complex scenarios, which is incredibly useful.

Wrapping Up

Nginx Proxy Manager is a fantastic tool that bundles complex functionalities like reverse proxying and SSL certificate management into an easy-to-use package. It lowers the barrier to entry for securely exposing web services and makes ongoing management much simpler, especially with its automated certificate renewals and clear web interface.

Whether you’re managing home lab services, small business applications, or just experimenting, I highly recommend giving it a try. It saves time, enhances security, and simplifies your infrastructure.

Building a Bulletproof PostgreSQL Cluster: My Go-To High Availability Setup

Good morning everyone! Dimitri Bellini here, back on Quadrata, my channel dedicated to the open-source world and the IT topics I love – and hopefully, you do too!

Thanks for tuning in each week. If you haven’t already, please hit that subscribe button and give this video a thumbs up – it really helps!

Today, we’re diving into a crucial topic for anyone running important applications, especially (but not only!) those using Zabbix: database resilience and performance. Databases are often the heart of our applications, but they can also be the source of major headaches – slow queries, crashes, data loss. Ensuring your database is robust and performs well is fundamental.

Why PostgreSQL and This Specific Architecture?

A few years back, we made a strategic decision to shift from MySQL to PostgreSQL. Why? Several reasons:

• The community and development activity around Postgres seemed much more vibrant.
• It felt like a more “serious,” robust database, even if maybe a bit more complex to configure initially compared to MySQL’s out-of-the-box readiness.
• For applications like Zabbix, which heavily utilize the database, especially in complex setups, having a reliable and performant backend is non-negotiable. Avoiding database disasters and recovery nightmares is paramount!

The architecture I’m showcasing today isn’t just for Zabbix; it’s a solid foundation for many applications needing high availability. We have clients using this exact setup for various purposes.

The Core Components

The solution we’ve settled on combines several powerful open-source tools:

• PostgreSQL: The core relational database.
• Patroni: A fantastic template for creating a High Availability (HA) PostgreSQL cluster. It manages the Postgres instances and orchestrates failover.
• etcd: A distributed, reliable key-value store. Patroni uses etcd for coordination and sharing state information between cluster nodes, ensuring consensus.
• PGBackrest: A reliable, feature-rich backup and restore solution specifically designed for PostgreSQL.
• HAProxy (Optional but Recommended): A load balancer to direct application traffic to the current primary node seamlessly.

How It Fits Together

Imagine a setup like this:

• Multiple PostgreSQL Nodes: Typically, at least two nodes running PostgreSQL instances.
• Patroni Control: Patroni runs on these nodes, monitoring the health of Postgres and managing roles (leader/replica).
• etcd Cluster: An etcd cluster (minimum 3 nodes for quorum – one can even be the backup server) stores the cluster state. Patroni instances consult etcd to know the current leader and overall cluster health.
• PGBackrest Node: Often one of the etcd nodes also serves as the PGBackrest repository server, storing backups and Write-Ahead Logs (WALs) for point-in-time recovery. Backups can be stored locally or, even better, pushed to an S3-compatible object store.
• Load Balancer: HAProxy (or similar) sits in front, checking an HTTP endpoint provided by Patroni on each node to determine which one is the current leader (primary) and directs all write traffic there.

This creates an active-standby (or active-passive) cluster. Your application connects to a single endpoint (the balancer), completely unaware of which physical node is currently active. HAProxy handles the redirection automatically during a switchover or failover.

Key Advantages of This Approach

• True High Availability: Provides a really bulletproof active-standby solution.
• Easy Balancer Integration: Uses simple HTTP checks, avoiding the complexities of virtual IPs (VIPs) and Layer 2 network requirements often seen in traditional clustering (like Corosync/Pacemaker), making it great for modern Layer 3 or cloud environments.
• “Simple” Configuration (Relatively!): Once you grasp the concepts, configuration is largely centralized in a single YAML file per node (patroni.yml).
• Highly Resilient & Automated: Handles node failures, switchovers, and even node reintegration automatically.
• Powerful Backup & Recovery: PGBackrest makes backups and, crucially, Point-in-Time Recovery (PITR) straightforward (again, “straightforward” for those familiar with database recovery!).
• 100% Open Source: No licensing costs or vendor lock-in. Test it, deploy it freely.
• Enterprise Ready & Supportable: These are mature projects. For production environments needing formal support, companies like Cybertec PostgreSQL (no affiliation, just an example we partner with) offer commercial support for this stack. We at Quadrata can also assist with first-level support and implementation.

In my opinion, this architecture brings PostgreSQL very close to the robustness you might expect from expensive proprietary solutions like Oracle RAC, but using entirely open-source components.

Let’s See It In Action: A Practical Demo

Talk is cheap, right? Let’s walk through some common management tasks and failure scenarios. In my lab, I have three minimal VMs (2 vCPU, 4GB RAM, 50GB disk): two for PostgreSQL/Patroni (node1, node2) and one for PGBackrest/etcd (backup-node). Remember, 3 nodes is the minimum for a reliable etcd quorum.

1. Checking Cluster Status

The primary command is patroni ctl. Let’s see the cluster members:

$ patronictl -c /etc/patroni/patroni.yml list

+ Cluster: my_cluster (73...) ---+----+-----------+

| Member | Host    | Role   | State   | TL | Lag in MB |

+--------+---------+--------+---------+----+-----------+

| node1  | 10.0.0.1| Leader | running | 10 |           |

| node2  | 10.0.0.2| Replica| running | 10 |         0 |

+--------+---------+--------+---------+----+-----------+

Here, node1 is the current Leader (primary), and node2 is a Replica, perfectly in sync (Lag 0 MB) on the same timeline (TL 10).

2. Performing a Manual Switchover

Need to do maintenance on the primary? Let’s gracefully switch roles:

$ patronictl -c /etc/patroni/patroni.yml switchover

Current cluster leader is node1

Available candidates for switchover:

1. node2

Select candidate from list [1]: 1

When should the switchover take place (e.g. 2023-10-27T10:00:00+00:00) [now]: now

Are you sure you want to switchover cluster 'my_cluster', leader 'node1' to member 'node2'? [y/N]: y

Successfully switched over to "node2"

... (Check status again) ...

+ Cluster: my_cluster (73...) ---+----+-----------+

| Member | Host    | Role   | State   | TL | Lag in MB |

+--------+---------+--------+---------+----+-----------+

| node1  | 10.0.0.1| Replica| running | 11 |         0 |

| node2  | 10.0.0.2| Leader | running | 11 |           |

+--------+---------+--------+---------+----+-----------+

Patroni handled demoting the old leader, promoting the replica, and ensuring the old leader started following the new one. Notice the timeline (TL) incremented.

3. Simulating a Primary Node Failure

What if the primary node just dies? Let’s stop Patroni on node2 (the current leader):

# systemctl stop patroni (on node2)

Now, check the status from node1:

$ patronictl -c /etc/patroni/patroni.yml list

+ Cluster: my_cluster (73...) ---+----+-----------+

| Member | Host    | Role   | State   | TL | Lag in MB |

+--------+---------+--------+---------+----+-----------+

| node1  | 10.0.0.1| Leader | running | 12 |           |

| node2  | 10.0.0.2|        | stopped |    |   unknown |

+--------+---------+--------+---------+----+-----------+

Patroni automatically detected the failure and promoted node1 to Leader. When node2 comes back online (systemctl start patroni), Patroni will automatically reintegrate it as a replica.

4. Recovering a Destroyed Node

What if a node is completely lost? Data disk corrupted, VM deleted? Let’s simulate this on node2 (assuming node1 is currently the leader):

# systemctl stop patroni (on node2)

# rm -rf /var/lib/patroni/data  # Or wherever your PG data directory is

# systemctl start patroni (on node2)

Watching the Patroni logs on node2, you’ll see it detects it has no data and initiates a `pg_basebackup` (or uses PGBackrest if configured) from the current leader (node1) to rebuild itself from scratch. Checking patroni ctl list shows its state transitioning through `creating replica` to `running` as a replica again, all automatically!

5. Point-in-Time Recovery (PITR) – The Real Lifesaver!

This is why I made the video! Recently, a bad deployment caused data corruption. We needed to restore to a state just *before* the incident. Here’s how PGBackrest and Patroni help.

Scenario: I accidentally deleted all records from a critical table.

psql> SELECT COUNT(*) FROM my_table; -- Shows 1000 rows

psql> DELETE FROM my_table;

psql> SELECT COUNT(*) FROM my_table; -- Shows 0 rows! Disaster!

Recovery Steps:

1. STOP PATRONI EVERYWHERE: This is critical. We need to prevent Patroni from interfering while we manipulate the database state manually.

   # systemctl stop patroni (on ALL nodes: node1, node2)

2. Identify Target Time/Backup: Use PGBackrest to find the backup and approximate time *before* the data loss.

   $ pgbackrest --stanza=my_stanza info 
   
   ... (Find the latest FULL backup timestamp, e.g., '2023-10-27 11:30:00') ...

3. Perform Restore on the (Ex-)Leader Node: Go to the node that *was* the leader (let’s say node1). Run the restore command, specifying the target time. The `–delta` option is efficient as it only restores changed files.

   $ pgbackrest --stanza=my_stanza --delta --type=time --target="2023-10-27 11:30:00" --target-action=pause restore

   (Note: `–target-action=pause` or `promote` might be needed depending on your exact recovery goal. For simplicity here, let’s assume we want to stop recovery at that point). Check PGBackrest docs for specifics. The video used a slightly different target specification based on the backup label.)

   Correction based on Video: The video demonstrated restoring to the end time of a specific full backup. A more typical PITR might use `–type=time` and a specific timestamp like `YYYY-MM-DD HH:MM:SS`. Let’s assume we used the backup label as shown in the video’s logic:

   $ pgbackrest --stanza=my_stanza --delta --set=20231027-xxxxxxF --type=default --target-action=promote restore

   (Replace `20231027-xxxxxxF` with your actual backup label. Using `–target-action=promote` tells Postgres to finish recovery and become promotable immediately after reaching the target.)

4. Start Postgres Manually (on the restored node): Start the database *without* Patroni first.

   # pg_ctl -D /var/lib/patroni/data start

   PostgreSQL will perform recovery using the restored files and WAL archives up to the specified target. Because we used `–target-action=promote`, it should finish recovery and be ready. If we had used `pause`, we would need `pg_ctl promote`.

5. Verify Data: Connect via `psql` and check if your data is back!

   psql> SELECT COUNT(*) FROM my_table; -- Should show 1000 rows again!

6. Restart Patroni: Now that the database is in the desired state, start Patroni on the restored node first, then on the other nodes.

   # systemctl start patroni (on node1)
   
   # systemctl start patroni (on node2)

   Patroni on `node1` will see it’s a valid database, assert leadership in etcd. Patroni on `node2` will detect it’s diverged (or has no data if we wiped it) and automatically re-sync from the now-restored leader (`node1`).

As you saw, we recovered from a potential disaster relatively quickly because the architecture and tools are designed for this.

Final Thoughts

Setting up this entire stack isn’t trivial – it requires understanding each component. That’s why I didn’t do a full step-by-step configuration in the video (it would be too long!). But I hope showing you *how it works* and its capabilities demonstrates *why* we chose this architecture.

It provides automation, resilience, and recovery options that are crucial for critical systems. Having an organized setup like this, combined with good documentation (please, write down your procedures!), turns stressful recovery scenarios into manageable tasks.

That’s all for this episode. A big greeting from me, Dimitri, and see you in the next one. Bye everyone!