In a Dockerized environment, one often encounters the need to monitor network traffic. However, one might not always wish to install sniffing tools within the container itself. By diving into the network namespace of the container, one can employ the host’s network packages such as tcpdump, tcpflow, and others, to achieve this without augmenting the container’s environment.
Step 1: Dive into the Container’s Network Namespace
Fetch the SandboxKey, which denotes the container’s network namespace:
Having entered the namespace, you can now utilize the host’s packages.
Using tcpdump:
tcpdump -i <INTERFACE_NAME>-w <OUTPUT_FILE.pcap>
Replace <INTERFACE_NAME> as per requirement (typically eth0 for Docker containers). For tcpdump, <OUTPUT_FILE.pcap> is the desired capture file. For tcpflow, <OUTPUT_DIRECTORY> is where the captured streams will be saved.
Conclusion
By navigating into a Docker container’s network namespace, you can readily use the network tools installed on the host system. This strategy circumvents the need to pollute the container with additional packages, upholding the principle of container immutability.
If you’re using Docker, you might have noticed that over time, logs can accumulate and take up a significant amount of space on your system. This can be a concern, especially if you’re running containers that generate a lot of log data.
To help you avoid this issue, I’m sharing a quick configuration tweak for Docker. By adjusting the daemon.json file, you can limit the size and number of log files Docker retains.
“log-driver”: “json-file”: This ensures Docker uses the default json-file logging driver, which writes log messages in JSON format.
“log-opts”: {…}: This section contains the logging options.
“max-size”: “10m”: Limits the maximum size of each log file to 10MB.
“max-file”: “1”: Restricts Docker to retain only one log file.
By implementing this configuration, you ensure that Docker only keeps a single log file with a maximum size of 10MB. Once the log reaches this size, Docker will rotate it, ensuring that old logs don’t eat up your storage.
To apply this configuration, simply add the above JSON to your daemon.json file, typically located at /etc/docker/daemon.json on Linux systems. After making the change, restart the Docker service.
I hope this tip helps you manage your Docker logs more efficiently. Happy containerizing! ?
HTTPie is not only an intuitively designed tool, but it also offers user-friendly methods to send HTTP requests directly from the command line. For developers looking for a more elegant and visual approach than traditional tools like curl or wget, HTTPie comes as a refreshing solution.
Installing HTTPie Without System Packages
Sometimes, relying on system packages isn’t an option due to various constraints or the desire to always fetch the latest version directly. Here are three alternative methods to get the latest version of HTTPie:
python3 -c "from urllib.request import urlopen; from json import loads; open('http', 'wb').write(urlopen([asset['browser_download_url'] for asset in loads(urlopen('https://api.github.com/repos/httpie/cli/releases/latest').read().decode())['assets'] if asset['name'] == 'http'][0]).read())"
These methods ensure you’re directly fetching the binary from the latest GitHub release, bypassing any potential system package cache limitations.
Exploring HTTPie’s Features with Examples
To truly appreciate the capabilities of HTTPie, one should explore its rich array of features. The official HTTPie Examples page showcases a variety of use cases. From simple GET requests to more complex POST requests with data, headers, and authentication, the examples provided make it evident why HTTPie stands out.
For instance, performing a simple GET request is as easy as:
http https://httpie.io
Or, if you want to post data:
http POST httpie.io/post Hello=World
Dive deeper into the examples to discover how HTTPie can simplify your HTTP querying experience.
Conclusion
HTTPie offers a refreshing approach to HTTP interactions, bringing clarity and simplicity to the command line. With flexible installation methods and an array of powerful features, it’s an indispensable tool for developers aiming for efficiency. Give HTTPie a try, and it might just become your go-to for all HTTP-related tasks!
Logging is a fundamental practice in software development for tracking and debugging applications. While Python’s built-in logging module gets the job done, Structlog takes logging to a new level by providing enhanced flexibility and customization options. In this guide, we’ll explore the basics of using Structlog for powerful and tailored logging in Python.
Installation
Start by installing Structlog using the following command:
pip install structlog
Basic Logging Setup
Structlog simplifies the process of setting up logging and offers versatile customization features. Here’s a basic example of how you can configure Structlog for logging:
import structlogimport loggingimport oslevel = os.environ.get("LOG_LEVEL","INFO").upper()LOG_LEVEL =getattr(logging, level)structlog.configure(wrapper_class=structlog.make_filtering_bound_logger(LOG_LEVEL))logger = structlog.get_logger()logger.debug("Database connection established")logger.info("Processing data from the API")logger.warning("Resource usage is nearing capacity")logger.error("Failed to save the file. Please check permissions")logger.critical("System has encountered a critical failure. Shutting down")
Output Example:
2023-08-28T10:00:00Z[INFO] Processing data from the API2023-08-28T10:00:00Z[WARNING] Resource usage is nearing capacity2023-08-28T10:00:00Z[ERROR] Failed to save the file. Please check permissions2023-08-28T10:00:00Z[CRITICAL] System has encountered a critical failure. Shutting down
Asynchronous Logging with asyncio
Structlog seamlessly supports asynchronous operations, making it compatible with asyncio:
Logging exceptions and their tracebacks is straightforward with Structlog:
import structlogtry: result =1/0exceptZeroDivisionError: logger.exception("Cannot divide one by zero!")
Traceback Output Example:
{"event":"Cannot divide one by zero!","level":"error","timestamp":"2023-07-31T07:00:31.526266Z","exception":[{"exc_type":"ZeroDivisionError","exc_value":"division by zero","syntax_error": null,"is_cause": false,"frames":[{"filename":"/home/stanley/structlog_demo/app.py","lineno":16,"name":"<module>","line":"","locals":{"__name__":"__main__","__doc__":"None","__package__":"None","__loader__":"<_frozen_importlib_external.SourceFileLoader object at 0xffffaa2f3410>","__spec__":"None","__annotations__":"{}","__builtins__":"<module 'builtins' (built-in)>","__file__":"/home/stanley/structlog_demo/app.py","__cached__":"None","structlog":"\"<module 'structlog' from '/home/stanley/structlog_demo/venv/lib/python3.11/site-\"+32","logger":"'<BoundLoggerLazyProxy(logger=None, wrapper_class=None, processors=None, context_'+55"}}]}]}
Conclusion
Logging plays a vital role in maintaining and troubleshooting Python applications. Structlog empowers developers with its versatile capabilities and powerful features for effective logging. By referring to the examples provided in this guide, you’ll be well-equipped to implement Structlog in your projects. For more detailed exploration, consult the official Structlog documentation.
Networking issues can be a real headache, especially when dealing with containerized applications. Whether it’s latency, routing problems, DNS resolution, firewall issues, or incomplete ARPs, network problems can significantly degrade application performance. Fortunately, there’s a powerful tool that can help you troubleshoot and resolve these issues: netshoot.
What is Netshoot?
Netshoot is a Docker container equipped with a comprehensive set of networking troubleshooting tools. It’s designed to help you diagnose and fix Docker and Kubernetes networking issues. With a proper understanding of how Docker and Kubernetes networking works and the right tools, you can troubleshoot and resolve these networking issues more effectively.
Understanding Network Namespaces
Before diving into the usage of netshoot, it’s essential to understand a key concept: Network Namespaces. Network namespaces provide isolation of the system resources associated with networking. Docker uses network and other types of namespaces (pid,mount,user, etc.) to create an isolated environment for each container. Everything from interfaces, routes, and IPs is completely isolated within the network namespace of the container.
The cool thing about namespaces is that you can switch between them. You can enter a different container’s network namespace, perform some troubleshooting on its network stack with tools that aren’t even installed on that container. Additionally, netshoot can be used to troubleshoot the host itself by using the host’s network namespace. This allows you to perform any troubleshooting without installing any new packages directly on the host or your application’s package.
Using Netshoot with Docker
Container’s Network Namespace
If you’re having networking issues with your application’s container, you can launch netshoot with that container’s network namespace like this:
$ sudo docker run -it --net container:<container_name> nicolaka/netshoot
Host’s Network Namespace
If you think the networking issue is on the host itself, you can launch netshoot with that host’s network namespace:
$ sudo docker run -it --net host nicolaka/netshoot
Network’s Network Namespace
If you want to troubleshoot a Docker network, you can enter the network’s namespace using nsenter. This is explained in the nsenter section below.
Using Netshoot with Docker Compose
You can easily deploy netshoot using Docker Compose using something like this:
Netshoot includes a wide range of powerful tools for network troubleshooting. Here’s a list of the included packages along with a brief description of each:
apache2-utils: Utilities for web server benchmarking and server status monitoring.
bash: A popular Unix shell.
bind-tools: Tools for querying DNS servers.
bird: Internet routing daemon.
bridge-utils: Utilities for configuring the Linux Ethernet bridge.
busybox-extras: Provides several stripped-down Unix tools in a single executable.
conntrack-tools: Tools for managing connection tracking records.
curl: Tool for transferring data with URL syntax.
dhcping: Tool to send DHCP requests to DHCP servers.
drill: Tool similar to dig.
ethtool: Tool for displaying and changing NIC settings.
file: Tool to determine the type of a file.
fping: Tool to ping multiple hosts.
grpcurl: Command-line tool for interacting with gRPC servers.
iftop: Displays bandwidth usage on an interface.
iperf: Tool for measuring TCP and UDP bandwidth performance.
iperf3: A newer version of iperf.
iproute2: Collection of utilities for controlling TCP/IP networking.
ipset: Tool to manage IP sets.
iptables: User-space utility program for configuring the IP packet filter rules.
iptraf-ng: Network monitoring tool.
iputils: Set of small useful utilities for Linux networking.
ipvsadm: Utility to administer the IP Virtual Server services.
jq: Lightweight and flexible command-line JSON processor.
libc6-compat: Compatibility libraries for glibc.
liboping: C library to generate ICMP echo requests.
ltrace: A library call tracer.
mtr: Network diagnostic tool.
net-snmp-tools: Set of SNMP management tools.
netcat-openbsd: Networking tool known as the “Swiss army knife” of networking.
nftables: Successor to iptables.
ngrep: Network packet analyzer.
nmap: Network exploration tool and security scanner.
nmap-nping: Packet generation and response analysis tool.
nmap-scripts: Scripts for nmap.
openssl: Toolkit for the Transport Layer Security (TLS) and Secure Sockets Layer (SSL) protocols.
py3-pip: Package installer for Python.
py3-setuptools: Python Distutils Enhancements.
scapy: Packet manipulation tool.
socat: Relay for bidirectional data transfer.
speedtest-cli: Command-line interface for testing internet bandwidth.
openssh: OpenSSH client and server.
strace: System call tracer.
tcpdump: Packet analyzer.
tcptraceroute: Traceroute implementation using TCP packets.
tshark: Network protocol analyzer.
util-linux: Miscellaneous system utilities.
vim: Highly configurable text editor.
git: Distributed version control system.
zsh: Unix shell.
websocat: Simple WebSocket client.
swaks: Swiss Army Knife for SMTP.
perl-crypt-ssleay: Perl module for OpenSSL.
perl-net-ssleay: Perl module for using OpenSSL.
With this extensive set of tools, netshoot is a powerful ally in diagnosing and resolving network issues in your Docker and Kubernetes environments. Whether you’re dealing with latency, routing problems, DNS resolution, firewall issues, or incomplete ARPs, netshoot has the tools you need to troubleshoot and fix these issues.
If you’re interested in trying out netshoot for yourself, you can find the project on GitHub at https://github.com/nicolaka/netshoot. It’s a powerful tool that can help you troubleshoot and resolve network issues in your Docker and Kubernetes environments.
In this blog post, we’ll explore how to use Docker and the lightweight HTTP server, Darkhttpd, to serve static files. This setup is particularly useful when you need a simple web server for sharing files or hosting a static website. We’ll also discuss how to use a reverse proxy like Traefik to route external traffic to the Darkhttpd service.
Docker Compose Configuration
Below is the docker-compose.yml file that defines the Darkhttpd service:
The image field specifies the Docker image to use for the service.
The container_name field sets the name of the container.
The restart field configures the restart policy for the container.
The volumes field defines the volume mounts for the service.
The entrypoint field overrides the default entrypoint of the image.
The networks field specifies the networks that the service is connected to.
Setting Up the Service
Create a directory named site in the same directory as the docker-compose.yml file. Place the static files you want to serve in this directory.
Replace your_network, your_ipv4_address, and your_network_name in the docker-compose.yml file with the appropriate values for your setup.
Run the following command to start the Darkhttpd service:
docker-compose up -d
Access the static files by navigating to the IP address specified in the docker-compose.yml file.
Using a Reverse Proxy
To route external traffic to the Darkhttpd service, you can use a reverse proxy like Traefik. Configure the reverse proxy to forward requests to the IP address specified in the docker-compose.yml file.
Conclusion
Using Docker and Darkhttpd to serve static files is a simple and efficient solution for sharing files or hosting a static website. By adding a reverse proxy, you can easily route external traffic to the Darkhttpd service. This setup is ideal for scenarios where you need a lightweight web server without the overhead of a full-fledged web server like Apache or Nginx.
In today’s interconnected world, maintaining the security of your server infrastructure is paramount. One critical point of vulnerability is the SSH (Secure Shell) service, which allows remote administration of servers. Despite using a non-default port, many administrators still find their servers bombarded with brute-force and denial-of-service attacks. To address this challenge, I’ve developed a solution called StealthSSHAccess.
The Problem
Attackers often employ brute force attacks to gain unauthorized access to servers via SSH. Even if you’ve changed the default SSH port, determined attackers can still discover the new port and target it. These attacks can lead to service disruption, unauthorized data access, and potential breaches of sensitive information.
The Solution: StealthSSHAccess
StealthSSHAccess is an innovative approach to managing remote SSH access while mitigating the risks associated with brute-force attacks. Let’s delve into how it works and why it’s an effective solution:
Dynamic Access Control
StealthSSHAccess takes a dynamic and personalized approach to SSH access control. It operates as a smart gateway between potential attackers and your SSH service. Here’s a simplified breakdown of how it functions:
Monitoring for Intent: Instead of directly exposing the SSH port, StealthSSHAccess monitors a non-SSH TCP port for connection attempts. Attackers, unaware of this, can’t target the SSH port directly.
Capture and Response: When an attempt is made on the monitored port, StealthSSHAccess captures the IP address of the requester. This initial connection attempt fails, serving as a signal of intent to access SSH.
Secure Access Window: Based on this signal, StealthSSHAccess temporarily opens the SSH port exclusively for the captured IP address. This allows for a secure connection from that specific source.
Time-Bound Access: Access is granted for a predetermined duration. If SSH access isn’t established within this timeframe, the port is automatically closed for that specific IP. This tightens the window of exposure and bolsters security.
Automatic Closure: If the port remains unused during the allowed time, StealthSSHAccess automatically revokes access and closes the port. A continuous monitoring mechanism controls this process.
Benefits and Features
1. Enhanced Security: By hiding the SSH port from attackers, StealthSSHAccess reduces the attack surface and minimizes exposure to potential threats.
2. Selective Accessibility: With StealthSSHAccess, you control who gains access by simply attempting a connection to a specific port. This provides an additional layer of security.
3. Minimal Configuration: Implementing StealthSSHAccess is easy thanks to its Docker-based deployment. This means you can integrate it seamlessly into your existing system.
4. Persistence Across Restarts: StealthSSHAccess ensures continuity by persisting IP timer information across service interruptions or restarts. This keeps the system aware of pending access requests.
Getting Started with StealthSSHAccess
To deploy StealthSSHAccess, follow these steps:
Requirements: Ensure you have Docker and Docker Compose installed.
Configuration: Set up environment variables using the provided .env file. Customize parameters like LOGLEVEL, IFACE, PORT_TO_MONITOR, and more to match your environment.
Building and Running: Build the images using docker-compose build, and then launch the services with docker-compose up -d.
Data Persistence: IP timer data is stored in the ./data directory, so make sure it’s writable by the Docker user.
Security Note: Be aware that these services run with privileged access due to their interaction with the system’s network configuration. Understand the security implications before deployment.
Conclusion
In the ongoing battle against cybersecurity threats, StealthSSHAccess stands as a beacon of innovative protection for your servers. By intelligently managing SSH access and responding dynamically to legitimate requests, this solution offers heightened security without sacrificing convenience. Whether you’re an administrator or a security-conscious user, consider integrating StealthSSHAccess into your infrastructure to safeguard your servers from the persistent threats of the digital landscape.
To explore the project, access the source code, and learn more about its implementation, visit the StealthSSHAccess GitHub repository. Remember, security is a journey, and with StealthSSHAccess, you’re taking a proactive step toward a more resilient and secure server environment.
In macOS, managing network traffic is essential for a robust and secure IT infrastructure. The operating system allows users to have a granular level of control over Domain Name System (DNS) settings, improving network functionality. Today’s post will walk through setting up per-domain DNS configuration on macOS using resolver configuration files.
Adding a DNS Rule Per Domain
MacOS has a powerful feature that allows you to specify DNS servers for individual domains. This is accomplished by creating resolver configuration files in the /etc/resolver/ directory. Each file in this directory corresponds to a domain and specifies the DNS servers to be used for that domain.
Let’s add a DNS rule for the “ymbihq.local” domain:
Open Terminal.
Use the sudo command to create a new file in the /etc/resolver/ directory with the same name as your domain:
This command will create a new resolver file named ymbihq.local and adds a line specifying 10.0.0.1 as the nameserver. As a result, all DNS queries for “ymbihq.local” will be resolved by the DNS server at the IP address 10.0.0.1.
Reviewing DNS Rules
To verify that the resolver configuration file was created successfully and to review its content, use the cat command:
cat /etc/resolver/ymbihq.local
This will output the contents of the ymbihq.local file, which should look like this:
# Sample output:nameserver 10.0.0.1
This confirms that the DNS server for the “ymbihq.local” domain has been set to 10.0.0.1.
Removing a DNS Rule
If you need to remove a DNS rule, you can simply delete the corresponding resolver configuration file. Use the rm command for this:
sudo rm /etc/resolver/ymbihq.local
After running this command, your macOS system will no longer have a custom DNS server set for the “ymbihq.local” domain, and it will default to using your standard DNS servers.
By creating and managing resolver configuration files, you can precisely control your DNS settings on a per-domain basis. This powerful feature of macOS allows you to optimize your network to fit your specific needs.
The subtitle could be something like: Mastering DNS Client NRPT Rules with PowerShell
In today’s post, we will be looking at a compact, but powerful, the chunk of PowerShell code that allows us to interact with DNS Client Name Resolution Policy Table (NRPT) rules on a Windows machine. The commands in this code allow us to add, review, and remove rules, giving us control over the direction of our DNS traffic.
Adding a DNS Client NRPT Rule
Let’s take a look at the first command:
# add a Windows rule for ymbihq.local domainAdd-DnsClientNrptRule -Namespace ".ymbihq.local"-NameServers "10.0.0.1"
This command uses the Add-DnsClientNrptRule cmdlet to add a new rule for the “.ymbihq.local” namespace. The -Namespace parameter specifies the domain name for the rule, and the -NameServers parameter specifies the IP address of the DNS server that should be used for queries within this namespace.
In this instance, we’re setting a rule for any DNS queries under the “.ymbihq.local” domain to be resolved by the DNS server at the IP address 10.0.0.1. This can be especially useful in an enterprise environment where you have custom internal domains to be resolved by specific DNS servers.
Reviewing DNS Client NRPT Rules
After adding a rule, it’s essential to verify it. We can do this using the Get-DnsClientNrptRule command:
# get the list of rules<br><code>Get-DnsClientNrptRule
This command lists all the NRPT rules currently set on the machine. It will output the unique identifiers, names, namespaces, and other details for each rule. Here’s a sample output:
From this output, you can see various properties of the rule we’ve just added for the “.ymbihq.local” namespace, such as its unique identifier (Name) and the nameserver it’s associated with (NameServers).
Removing a DNS Client NRPT Rule
The final part of this block of code is dedicated to rule removal:
# remove the ruleRemove-DnsClientNrptRule -Name "{A7CCF814-7492-4019-9FB1-27F61327AE93}"
Here, we use the Remove-DnsClientNrptRule cmdlet with the -Name parameter followed by the unique identifier of the rule we wish to remove. After running this command, PowerShell will prompt you for confirmation before deleting the rule.
The process looks like this:
ConfirmRemoving NRPT rule for namespace .ymbihq.local with DAEnable: Disabled, DnsSecValidationRequired: Disabled, NameEncoding: Disable NameServers:10.0.0.1 Do you want to continue?[Y] Yes [N] No
Aprofito aquesta entrada per comentar una nova efemèride s’ha produït avui mateix. Ja que després de 46 anys menys dos dies he visitat de nou l’hospital on vaig néixer. Aquest cop per fer-me unes radiografies a les cervicals. Qui ho havia de dir?
