Category: Technology

Grafana Alloy, parsing syslog RFC5424

2025/01/15 No Comments

Reading time: 31 – 51 minutes

Parsing syslog fields in a Grafana Alloy configuration file (config.alloy):


  stage.regex {
      expression = "^<(?P<priority>\\d|\\d{2}|1[1-8]\\d|19[01])>(?P<version>\\d{1,2})\\s(?P<timestamp>-|(?P<fullyear>[12]\\d{3})-(?P<month>0\\d|1[0-2])-(?P<mday>[0-2]\\d|3[01])T(?P<hour>[01]\\d|2[0-3]):(?P<minute>[0-5]\\d):(?P<second>[0-5]\\d|60)(?:\\.(?P<secfrac>\\d{1,6}))?(?P<numoffset>Z|[+-]\\d{2}:\\d{2}))\\s(?P<hostname>\\S{1,255})\\s(?P<appname>\\S{1,48})\\s(?P<procid>\\S{1,128})\\s(?P<msgid>\\S{1,32})\\s(?P<structureddata>-|\\[(?:[^\\[\\]]|\\\\.)*\\]) (?:\\s(?P<msg>.+))?$"
  }

  stage.labels  {
    values = {
      application = "appname",
      pid = "procid",
      msgid = "msgid",
      structureddata = "structureddata",
    }
  }

If Grafana Alloy logs are also parsed using this endpoint, it’s good to add next:

  stage.regex {
    expression = "^<(?P<priority>\\d|\\d{2}|1[1-8]\\d|19[01])>(?P<version>\\d{1,2})\\s(?P<timestamp>-|(?P<fullyear>[12]\\d{3})-(?P<month>0\\d|1[0-2])-(?P<mday>[0-2]\\d|3[01])T(?P<hour>[01]\\d|2[0-3]):(?P<minute>[0-5]\\d):(?P<second>[0-5]\\d|60)(?:\\.(?P<secfrac>\\d{1,6}))?(?P<numoffset>Z|[+-]\\d{2}:\\d{2}))\\s(?P<hostname>\\S{1,255})\\s(?P<appname>\\S{1,48})\\s(?P<procid>\\S{1,128})\\s(?P<msg>.*)$"
  }

  stage.output  {
    source = "msg"
  }

  stage.replace {
    expression = "(ts=\\S+\\s+level=\\S+\\s+)"
    source = "msg"
    replace = ""
  }

  stage.output {
    source = "msg"
  }

Accessing Zerotier’s REST API

2025/01/14 No Comments

Reading time: 6 – 10 minutes

Zerotier offers a powerful REST API that allows for seamless integration and management of your network. By default, the API is accessible on TCP port 9993. To securely interact with this API, an authentication token is required.

The authentication token is stored in the following file:

/var/lib/zerotier-one/authtoken.secret

To check if the Zerotier service is running correctly, you can use the curl command with the necessary authentication header. Here’s how to do it:

curl -Lv -H "X-ZT1-Auth: $(cat /var/lib/zerotier-one/authtoken.secret)" http://localhost:9993/status 2>&1|less

Breaking Down the Command:

curl -Lv
- -L: Follows any redirects the server might issue.
- -v: Enables verbose mode, providing detailed information about the request and response.
-H "X-ZT1-Auth: $(cat /var/lib/zerotier-one/authtoken.secret)"
- Adds a custom header X-ZT1-Auth with the value of your authentication token. This is essential for authorized access to the API.
http://localhost:9993/status
- The endpoint to check the current status of the Zerotier service.
2>&1 | less
- Redirects both standard output and standard error to less for easy reading and navigation of the output.

I hope you found this guide helpful in navigating Zerotier’s REST API.

Comparing Python Executable Packaging Tools: PEX, PyOxidizer, and PyInstaller

2024/10/25 No Comments

Reading time: 9 – 15 minutes

Packaging Python applications into standalone executables can simplify deployment and distribution, especially when dealing with users who may not have Python installed or when aiming for a seamless installation experience. Three prominent tools in this space are PEX, PyOxidizer, and PyInstaller. In this post, we’ll explore each of these tools, highlighting their features, how they work, and their pros and cons to help you decide which one suits your needs.

PEX (Python EXecutable)
PyOxidizer
PyInstaller
Comparison
Conclusion

PEX (Python EXecutable)

Website: docs.pex-tool.org
GitHub: github.com/pex-tool/pex

Overview

PEX stands for Python EXecutable. It creates self-contained executable Python environments that are runnable on other machines without requiring a Python interpreter or additional dependencies.

Features

Self-contained Executables: Packages all dependencies into a single file.
Virtual Environment Management: Manages dependencies in an isolated environment.
Support for Multiple Python Versions: Can target different Python versions.
Reproducible Builds: Ensures consistent builds across different environments.

How It Works

PEX files are ZIP files with a special header that makes them executable. When you run a PEX file, it sets up an isolated environment and executes your application within it. Dependencies are resolved and bundled at build time, ensuring that the executable has everything it needs to run.

Pros and Cons

Pros:

Ease of Use: Straightforward command-line interface.
Isolation: Avoids conflicts with system-installed packages.
Flexible Configuration: Supports complex dependency management.

Cons:

Platform Dependency: The generated PEX file is platform-specific. (OS and Python version)
Size Overhead: Can result in large executable files due to included dependencies.

PyOxidizer

Website: gregoryszorc.com
GitHub: github.com/indygreg/PyOxidizer

Overview

PyOxidizer is a tool that produces distributable binaries from Python applications. It embeds the Python interpreter and your application into a single executable file.

Features

Single Executable Output: Creates a single binary without external dependencies.
Embedded Python Interpreter: Bundles a Rust-based Python interpreter.
Cross-Compilation: Supports building executables for different platforms.
Performance Optimization: Optimizes startup time and reduces runtime overhead.

How It Works

PyOxidizer uses Rust to compile your Python application into a binary. It embeds the Python interpreter and compiles your Python code into bytecode, which is then included in the binary. This approach results in a single executable that can be distributed without requiring a separate Python installation.

Pros and Cons

Pros:

No Runtime Dependencies: Users don’t need Python installed.
Cross-Platform Support: Can build executables for Windows, macOS, and Linux.
Optimized Performance: Faster startup times compared to other tools.

Cons:

Complex Configuration: Requires understanding of Rust and PyOxidizer’s configuration.
Relatively New Tool: May have less community support and fewer resources.

PyInstaller

Website: pyinstaller.org
GitHub: github.com/pyinstaller/pyinstaller

Overview

PyInstaller bundles a Python application and all its dependencies into a single package, which can be a directory or a standalone executable.

Features

Multi-Platform Support: Works on Windows, macOS, and Linux.
Customizable Builds: Allows inclusion or exclusion of files and dependencies.
Support for Various Libraries: Handles complex dependencies like NumPy, PyQt, etc.
One-Folder and One-File Modes: Choose between a directory of files or a single executable.

How It Works

PyInstaller analyzes your Python script to discover every other module and library your script needs to run. It then collects copies of all those files—including the active Python interpreter—and packs them into a single executable or a folder.

Pros and Cons

Pros:

Ease of Use: Simple command-line usage.
Wide Compatibility: Supports many third-party packages.
Flexible Output Options: Choose between single-file or directory output.

Cons:

Executable Size: Can produce large files.
Hidden Imports: May miss some dependencies, requiring manual specification.

Comparison

Feature	PEX	PyOxidizer	PyInstaller
Single Executable	Yes (but requires Python)	Yes	Yes
No Python Required	No	Yes	Yes
Cross-Platform	Yes (build on target OS)	Yes (cross-compilation)	Yes (build on target OS)
Ease of Use	Moderate	Complex	Easy
Executable Size	Large	Smaller	Large
Configuration	Flexible	Requires Rust knowledge	Simple
Community Support	Active	Growing	Extensive
GitHub Activity	Actively maintained	Unmaintained	Actively maintained

Conclusion

Choosing the right tool depends on your specific needs and the assurance of ongoing support:

Use PEX if you need a self-contained environment for systems where Python is available. Its active maintenance ensures that you can rely on timely updates and community support.
Use PyOxidizer if you prefer a single executable without runtime dependencies and are comfortable with Rust. Its growing GitHub activity signifies a promising future and dedicated maintenance.
Use PyInstaller if you value simplicity and extensive community support. Its active maintenance status means you can expect regular updates and a wealth of community resources.

References:

Dynaconf: A Comprehensive Guide to Configuration Management in Python

2023/11/01 No Comments

Reading time: 35 – 58 minutes

Dynaconf is a powerful configuration management library for Python that simplifies the process of managing and accessing application settings. It offers a wide range of features, including support for multiple configuration sources, environment variables, and dynamic reloading.

Main features:

Multiple configuration sources: Dynaconf can read settings from various sources, including environment variables, files, and even Vault servers. This flexibility allows you to centralize your configuration management and adapt to different deployment environments.
Environment variable support: Dynaconf seamlessly integrates with environment variables, making it easy to inject configuration values from your system or containerized environments.
Dynamic reloading: Dynaconf can dynamically reload configuration changes without restarting your application, ensuring that your application always reflects the latest settings.

Supported configuration file formats:

Dynaconf supports a variety of configuration file formats, including:

TOML (recommended)
YAML
JSON
INI
Python files

Installation:

Dynaconf can be installed using either pip or poetry:

pip install dynaconf

# or

poetry add dynaconf

How to initialize Dynaconf

To initialize Dynaconf in your project, run the following command in your project’s root directory:

dynaconf init -f toml

This command will create the following files:

config.py: This file imports the Dynaconf settings object.
settings.toml: This file contains your application settings.
.secrets.toml: This file contains your sensitive data, such as passwords and tokens.

config.py

The config.py file is a Python file that imports the Dynaconf settings object. This file is required for Dynaconf to work.

Here is an example of a config.py file:

import dynaconf

settings = dynaconf.settings()

settings.toml

The settings.toml file contains your application settings. This file is optional, but it is the recommended way to store your settings.

Here is an example of a settings.toml file:

[default]
debug = false
host = "localhost"
port = 5000

secrets.toml

The .secrets.toml file contains your sensitive data, such as passwords and tokens. This file is optional, but it is recommended to store your sensitive data in a separate file from your application settings.

Here is an example of a .secrets.toml file:

[default]
database_password = "my_database_password"
api_token = "my_api_token"

Importing and using the Dynaconf settings object

Once you have initialized Dynaconf, you can import the settings object into your Python code. For example:

from config import settings

You can then access your application settings using the settings object. For example:

setting_value = settings.get('KEY')

Dynaconf also supports accessing settings using attributes. For example:

setting_value = settings.KEY

Importing environment variables in Docker-Compose:

When using Dynaconf with Docker-Compose, the best strategy for importing environment variables is to define them in your docker-compose.yml file and then access them using Dynaconf’s envvar_prefix setting.

version: "3.8"

services:
  app:
    build: .
    environment:
      - APP_DEBUG=true
      - APP_HOST=0.0.0.0
      - APP_PORT=8000

In your Python code, you can access these environment variables using Dynaconf:

import dynaconf

settings = dynaconf.settings(envvar_prefix="APP")

debug = settings.get("DEBUG")
host = settings.get("HOST")
port = settings.get("PORT")

Conclusion:

Dynaconf is a versatile and powerful configuration management library for Python that simplifies the process of managing and accessing application settings. Its comprehensive feature set and ease of use make it an ideal choice for a wide range of Python projects.

Ubuntu traceroute alternatives

2019/12/20 No Comments

Reading time: 4 – 7 minutes

# pre-installed on Ubuntu 18:
tracepath -n IP/HOST
mtr -n IP/HOST

oriolrius.cat

Category: Technology

Grafana Alloy, parsing syslog RFC5424

Accessing Zerotier’s REST API

Breaking Down the Command:

Comparing Python Executable Packaging Tools: PEX, PyOxidizer, and PyInstaller

Table of Contents

PEX (Python EXecutable)

Overview

Features

How It Works

Pros and Cons

PyOxidizer

Overview

Features

How It Works

Pros and Cons

PyInstaller

Overview

Features

How It Works

Pros and Cons

Comparison

Conclusion

Dynaconf: A Comprehensive Guide to Configuration Management in Python

Ubuntu traceroute alternatives