DevOps on Kang's Blog

Troubleshooting RAID Hard Drive Recognition Issues

Tue, 30 Dec 2025 00:00:00 +0000

Problem Background

During server maintenance, I encountered a challenging issue: after inserting a new 1TB SAS hard drive into the server, the lsblk command couldn’t detect this new drive. After investigation, I found that although the hard drive had been recognized by the RAID card, it was in an Unconfigured(good) state and hadn’t been configured as a virtual disk, making it inaccessible to the operating system.

This article documents the complete process of troubleshooting and resolving this issue using the MegaCli tool.

Installing MegaCli Tool

MegaCli is a command-line tool for managing LSI/Broadcom RAID cards. Here are the installation steps for Ubuntu/Debian systems:

Download MegaCli Package

`1`	`wget https://docs.broadcom.com/docs-and-downloads/raid-controllers/raid-controllers-common-files/8-07-14_MegaCLI.zip`

Extract the Package

`1`	`unzip 8-07-14_MegaCLI.zip`

Convert RPM Package to DEB Package

Since the downloaded package is in RPM format, use the alien tool to convert it to DEB format:

1
2

cd Linux
sudo alien MegaCli-8.07.14-1.noarch.rpm

Install the DEB Package

`1`	`sudo dpkg -i megacli_8.07.14-2_all.deb`

Fix Dependency Issues (If Needed)

If you encounter a missing libncurses.so.5 error, create a symbolic link:

`1`	`sudo ln -s /usr/lib/x86_64-linux-gnu/libncurses.so.6 /usr/lib/x86_64-linux-gnu/libncurses.so.5`

Verify Installation

`1`	`/opt/MegaRAID/MegaCli/MegaCli64 -v`

Checking RAID Status

Use the following command to view detailed information about all physical hard drives:

`1`	`/opt/MegaRAID/MegaCli/MegaCli64 -PDList -aALL`

Through MegaCli’s output, you can clearly see the current status of all hard drives on the server.

Investigation Conclusion

The server has a total of 7 physical hard drives, and the newly inserted drive is located at Slot 6, with a current status of Unconfigured(good) (unconfigured but in good condition). It hasn’t been configured as a Virtual Drive yet, so the Linux operating system (lsblk) cannot recognize it.

Detailed Hard Drive Distribution

For better understanding, the 7 hard drives are categorized into 4 groups by purpose:

System Drive (corresponds to `/dev/sda`)

Location: Slot 5
Model: Intel 120GB SSD
Status: Online
Configuration: Single-disk RAID 0 (or passthrough mode)
Purpose: System boot drive

Data Drive Array (corresponds to `/dev/sdb`, capacity 2.7TB)

Location: Slot 0, 1, 2, 3
Model: 4 Seagate 1TB SAS drives
Status: Online
Configuration: RAID 5 array (3 data blocks + 1 parity = 3TB usable capacity)

Data Drive (corresponds to `/dev/sdc`, capacity 931GB)

Location: Slot 7
Model: Seagate 1TB SAS drive
Status: Online
Configuration: Single-disk RAID 0

🔍 Key Finding: Newly Inserted Drive (Slot 6)

Location: Slot 6
Model: Toshiba 1TB SAS drive
Status: Unconfigured(good) (unconfigured but in good condition)
Foreign State: None (no foreign configuration)
Other Error Count: 9 (some historical error counts, not affecting current recognition)
Current Situation: The drive is physically connected properly and recognized by the RAID card, but since it hasn’t been added to any RAID group or created as a virtual disk, the RAID card won’t present it to the operating system

Problem Analysis: Why Wasn’t It Auto-Recognized?

Reason One: Foreign State is None

The output shows Foreign State: None, indicating that the RAID card hasn’t detected any recognizable old RAID configuration information on this disk (or it has been cleared).

If it was previously Linux software RAID (mdadm): Hardware RAID cards cannot recognize software RAID metadata and will treat it as an empty disk
If it was previously hardware RAID: The RAID metadata may be incompatible or has been cleared

Reason Two: Manual Virtual Disk Creation Required

On RAID cards like Dell PERC, physical disks must be configured as Virtual Drives (VD) before the operating system can access them.

Solution: Configure the Drive Online

Since the drive status is Unconfigured(good), it needs to be configured as a single-disk RAID 0 for the operating system to recognize it.

Create Single-Disk RAID 0

Execute the following command to configure the Slot 6 drive as a virtual disk:

# Configure Slot 6 drive as single-disk RAID 0
# [32:6] represents Enclosure 32, Slot 6
# -a0 represents adapter 0
sudo /opt/MegaRAID/MegaCli/MegaCli64 -CfgLdAdd -r0 [32:6] -a0

Expected Result

After successful command execution, the terminal will display:

`1`	`Adapter 0: Created VD 3`

Run the lsblk command again, and you should see the newly added /dev/sdd device.

⚠️ Data Safety Notice

Important: The -CfgLdAdd command creates RAID structure and rewrites disk header metadata, which may affect access to original data.

Data Recovery Options for Different Scenarios

Original Drive Configuration	Foreign State	Data Recovery Possibility
Linux Software RAID (mdadm)	None	After configuration, try using `mdadm --assemble --scan` to recover data
Hardware RAID	None	RAID card no longer recognizes old configuration, can only mount as new disk, recovery depends on partition table integrity
Brand New Drive	None	No data loss risk, can use directly

Refresh System Device List

After configuration, if the system still hasn’t automatically recognized the new device, you can manually refresh the SCSI bus:

# Scan all SCSI host controllers to make the system re-recognize drives
for host in /sys/class/scsi_host/host*/scan; do 
    echo "- - -" > "$host"
done

After execution, use lsblk or fdisk -l to verify if the new hard drive is visible.

Summary

Key steps in this troubleshooting process:

✅ Install MegaCli tool for RAID card management
✅ Use -PDList command to view all physical drive statuses
✅ Identify the problem: New drive is in Unconfigured(good) status
✅ Use -CfgLdAdd command to create single-disk RAID 0
✅ Refresh system device list and verify drive is online

Through this investigation, I gained a deep understanding of how hardware RAID cards work: physical drives must first be configured as virtual disks before the operating system can access them. In daily operations, when encountering hard drive recognition issues, you should first check the RAID card level configuration status rather than troubleshooting only at the operating system level.

Mounting WebDAV on Linux

Tue, 11 Nov 2025 00:00:00 +0000

Mounting WebDAV on Linux

This document describes how to mount WebDAV shares on Linux systems using davfs2, with Ubuntu 24.04 as an example.

Installing davfs2

Run the following command in the terminal to install the davfs2 package:

1
2

sudo apt-get update
sudo apt-get install davfs2

During installation, you may be prompted whether to allow non-root users to mount WebDAV. You can use the arrow keys to switch to the “Yes” option.

Creating a Mount Point Directory

Create a directory to serve as the mount point:

`1`	`sudo mkdir /mnt/webdav`

Configuring davfs2

Edit the davfs2.conf file to configure davfs2. Open the configuration file:

`1`	`sudo nano /etc/davfs2/davfs2.conf`

Find the use_locks configuration option in the file and ensure its value is set to 0. This disables file locking, as some WebDAV servers do not support locking.

`1`	`use_locks 0`

Save and close the file (press Ctrl+X, then press Y, and finally press Enter).

Configuring the davfs2 secrets File

Create a secrets file to store the username and password for the WebDAV server. Run the following command in the terminal:

`1`	`sudo nano /etc/davfs2/secrets`

Add a line similar to the following, replacing it with your WebDAV server’s username and password:

`1`	`http://your-webdav-url username password`

Replace the content in the above line with your actual information:

http://your-webdav-url - Your WebDAV server address
username - Your username
password - Your password

Save and close the file.

Setting File Permissions

To ensure the password in the secrets file is secure, set the file permissions:

`1`	`sudo chmod 600 /etc/davfs2/secrets`

Use the mount command to mount the WebDAV share to the previously created mount point:

`1`	`sudo mount -t davfs http://your-webdav-url /mnt/webdav`

Automatic Mounting on Boot (Optional)

If you need to mount automatically on boot, you can edit the /etc/fstab file:

`1`	`sudo nano /etc/fstab`

Add the following line:

`1`	`http://your-webdav-url /mnt/webdav davfs user,noauto 0 0`

When you need to unmount, run:

`1`	`sudo umount /mnt/webdav`

Installing SMB Service on Ubuntu 22.04

Fri, 04 Jul 2025 00:00:00 +0000

Introduction

SMB (Server Message Block) is a network file sharing protocol widely used for sharing files and printers between different operating systems. Installing and configuring SMB service on Ubuntu 22.04 makes it convenient to share files between Linux and Windows systems. This article will provide detailed instructions on how to install, configure, and use SMB service on Ubuntu 22.04.

Installing SMB Service

Update System Packages

First, ensure system packages are up to date:

1
2

sudo apt update
sudo apt upgrade -y

Install Samba

Samba is the open-source software package that implements the SMB protocol on Linux systems:

`1`	`sudo apt install samba samba-common-bin -y`

Verify Installation

Check the Samba service status:

1
2

sudo systemctl status smbd
sudo systemctl status nmbd

Configuring SMB Service

Backup Default Configuration File

Before modifying the configuration file, backup the original file:

`1`	`sudo cp /etc/samba/smb.conf /etc/samba/smb.conf.backup`

Edit Configuration File

Use a text editor to edit the SMB configuration file:

`1`	`sudo nano /etc/samba/smb.conf`

Basic Configuration Example

Add the following content to the configuration file:

[global]
   workgroup = WORKGROUP
   server string = Ubuntu SMB Server
   log file = /var/log/samba/log.%m
   max log size = 1000
   logging = file
   panic action = /usr/share/samba/panic-action %d
   server role = standalone server
   obey pam restrictions = yes
   unix password sync = yes
   passwd program = /usr/bin/passwd %u
   passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
   pam password change = yes
   map to guest = bad password
   usershare allow guests = yes

[shared]
   comment = Shared Folder
   path = /srv/samba/shared
   browseable = yes
   read only = no
   guest ok = yes
   create mask = 0755
   directory mask = 0755

Create Shared Directory

Create the directory for sharing:

1
2
3

sudo mkdir -p /srv/samba/shared
sudo chown nobody:nogroup /srv/samba/shared
sudo chmod 777 /srv/samba/shared

Verify Configuration File

Use the testparm command to verify configuration file syntax:

`1`	`sudo testparm`

User Management

Create SMB User

If user authentication is needed, first create a system user:

`1`	`sudo adduser smbuser`

Then add the user to the SMB database:

`1`	`sudo smbpasswd -a smbuser`

Enable/Disable User

Enable SMB user:

`1`	`sudo smbpasswd -e smbuser`

Disable SMB user:

`1`	`sudo smbpasswd -d smbuser`

Delete User

Delete SMB user:

`1`	`sudo smbpasswd -x smbuser`

Starting and Managing Services

Start Services

Start SMB services:

1
2

sudo systemctl start smbd
sudo systemctl start nmbd

Enable Auto-start

Set services to start automatically on boot:

1
2

sudo systemctl enable smbd
sudo systemctl enable nmbd

Restart Services

Restart SMB services:

1
2

sudo systemctl restart smbd
sudo systemctl restart nmbd

Check Service Status

Check service running status:

1
2

sudo systemctl status smbd
sudo systemctl status nmbd

Firewall Configuration

If firewall is enabled, open SMB related ports:

`1`	`sudo ufw allow 'Samba'`

Or manually open ports:

sudo ufw allow 139/tcp
sudo ufw allow 445/tcp
sudo ufw allow 137/udp
sudo ufw allow 138/udp

Testing Connection

Local Testing

Test SMB shares locally:

`1`	`smbclient -L localhost`

Mount SMB share on Linux client:

1
2

sudo mkdir /mnt/smbshare
sudo mount -t cifs //SERVER_IP/shared /mnt/smbshare -o username=smbuser

Windows Client

In Windows File Explorer address bar, enter:

`1`	`\\SERVER_IP\shared`

Advanced Configuration

Configure home directory sharing for each user:

[homes]
   comment = User Home Directories
   browseable = no
   read only = no
   create mask = 0700
   directory mask = 0700
   valid users = %S

Configure read-only share:

[readonly]
   comment = Read-only Share
   path = /srv/samba/readonly
   browseable = yes
   read only = yes
   guest ok = yes

Security Settings

Configuration for enhanced security:

[global]
   # Disable SMB1 protocol
   server min protocol = SMB2
   
   # Enable encryption
   server signing = mandatory
   
   # Restrict access by IP
   hosts allow = 192.168.1.0/24 127.0.0.1
   hosts deny = ALL

Common Issues

Permission Issues

If encountering permission issues, check the following settings:

File system permissions
SELinux settings (if enabled)
SMB user permissions

Connection Issues

If unable to connect, check:

Firewall settings
Network connectivity
Service status

Performance Optimization

Configuration for optimizing SMB performance:

[global]
   socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=131072 SO_SNDBUF=131072
   read raw = yes
   write raw = yes
   max xmit = 65535
   dead time = 15
   getwd cache = yes

Monitoring and Logging

View Logs

SMB log file locations:

1
2

sudo tail -f /var/log/samba/log.smbd
sudo tail -f /var/log/samba/log.nmbd

Connection Status

View current connections:

`1`	`sudo smbstatus`

List all shares:

`1`	`sudo smbclient -L localhost`

Summary

This article provides a complete guide to install and configure SMB service on Ubuntu 22.04. SMB service offers a convenient solution for cross-platform file sharing, suitable for both home networks and enterprise environments.

Key points:

Regularly backup configuration files
Set appropriate user permissions
Configure proper firewall rules
Monitor service status and logs
Adjust performance parameters as needed

A properly configured SMB service will provide stable and reliable file sharing functionality for network environments.

BIOS and UEFI: The Boot Journey from History to Modern Era

Wed, 23 Apr 2025 00:00:00 +0000

In the long history of computer development, boot firmware plays a crucial role. From the early BIOS to today’s powerful UEFI, it has continuously evolved to meet the growing performance, security, and management needs of hardware.

Origin: The Golden Age of BIOS

In the 1980s, IBM PC introduced BIOS (Basic Input/Output System), which was etched into the motherboard ROM and took on three major responsibilities for computer startup:

Power-On Self-Test (POST): Detecting whether CPU, memory, graphics cards, keyboards, and other hardware are available.
MBR Loading and Hardware Interrupt Interface: Reading the primary bootloader from the first sector of the hard disk (Master Boot Record, 512 bytes), and providing unified hardware access capabilities through interrupt calls such as INT 13h/INT 10h.

Although the BIOS architecture was simple and highly compatible, it encountered bottlenecks in the following aspects:

Disk Capacity Limitations: MBR supports a maximum of 2 TB and up to 4 primary partitions.
Difficult Firmware Updates: Driver code is hardcoded in ROM, making it inflexible for extension or patching.
Lack of Graphics and Network Capabilities: Only providing the most basic text interface and simple PXE network microcode loading.

Disruption: The Emergence of UEFI

Entering the 21st century, Intel initiated the UEFI (Unified Extensible Firmware Interface) alliance, aiming to create a more flexible, extensible, and secure boot environment.

GPT Partition Support: Breaking through TB-level storage limitations, capable of managing hundreds of partitions.
Modular Drivers: Extending hardware support through loadable .efi drivers without flashing ROM.
Modern Graphical Interface: Supporting GUI, mouse, and multiple languages for more user-friendly interactions.
Security and Remote Management:
- Verifying signatures of boot programs and operating systems at each stage to prevent tampering and rootkits (Secure Boot).
- Built-in HTTP/FTP clients and UEFI Shell for remote updates or script execution.

Brief UEFI Boot Process

Parallel Firmware Initialization: Multi-threaded loading of hardware drivers and firmware components.
Scanning ESP: Reading the EFI System Partition (FAT32 format), looking for .efi files under the \EFI\ path.
Running Boot Manager: Executing EFI applications according to the boot option order stored in NVRAM.
Loading Kernel: EFI applications (such as bootx64.efi) boot the operating system kernel and transfer control to it.

Comparative Summary

Feature	BIOS + MBR	UEFI + GPT
Disk Support	≤ 2 TB; max 4 partitions	Theoretically ≥ 9.4 ZB; max 128 partitions
Drivers & Extensions	ROM fixed, difficult to update	Dynamic loading of .efi modules, easy to upgrade
Boot Interface	Text or minimal graphics	Rich GUI, mouse, multi-language support
Secure Boot	Not supported	Supports signature verification, anti-tampering
Network Functionality	Only simple PXE boot	Supports HTTP/FTP, remote management

Real-World Choices and Migration

Legacy Devices & Embedded Systems: BIOS+MBR remains effective due to small size and strong compatibility.
Modern PCs & Servers: UEFI+GPT is standard, improving boot speed, security, and large disk support.
Dual/Multi-Boot Systems: Recommended to standardize on UEFI mode, using BCD for Windows and shim+GRUB EFI for Linux to reduce boot conflicts.

Migration Key Points:

BIOS → UEFI: Back up data, use Windows mbr2gpt or Linux gdisk to convert to GPT, create an ESP ≥100 MB, switch firmware mode, and rebuild the boot.
UEFI → BIOS: Convert GPT to MBR (note the loss of GPT information), rewrite traditional MBR boot code, and adjust firmware to Legacy mode.

Common Boot Issues and Troubleshooting Techniques

Unable to Recognize ESP
- Check partition type: ESP must be set to EF00 (GPT) or FAT32 (MBR).
- Verify files: Confirm that \EFI\BOOT\BOOTX64.EFI (or corresponding platform) exists.
Secure Boot Errors
- Disable or configure: Enter firmware settings, temporarily disable Secure Boot, or import the correct public keys (PK/KEK).
Missing Boot Options
- Use efibootmgr (Linux) or bcdedit (Windows) to recreate boot entries.
Unable to Boot After Switching Disk Mode
- Check controller mode: After switching IDE/RAID/AHCI, kernel modules and driver signatures need to be updated synchronously.

Advanced Customization and Extensions

UEFI Shell Automation: Write .nsh scripts to automatically mount partitions, perform self-tests, or remotely download firmware updates.
Driver Injection: Place vendor .efi drivers in ESP to extend native support for NVMe, RAID controllers, etc.
Variable Management: Use dmpstore (Shell) or efivar tools to read and write firmware variables, enabling automated startup or log collection.
Open Firmware Alternatives: Such as TianoCore (OVMF), Coreboot, can be customized for size and functionality as needed.

Kubernetes Package Management: The Ultimate Helm Guide

Sat, 18 Jan 2025 00:00:00 +0000

Introduction

Helm is a powerful tool for managing Kubernetes applications. Through Helm Charts, you can define, install, and upgrade even the most complex Kubernetes applications.

Helm Charts are easy to create, version control, share, and publish. This means you can use Helm more efficiently, avoiding the tedious process of manually copying and pasting configuration files.

Helm has graduated from CNCF (Cloud Native Computing Foundation) and is continuously maintained by the Helm community, ensuring its stability and ongoing development.

Managing Complexity

Helm Charts can describe even the most complex applications, providing reproducible installation workflows and serving as the single source of truth for application configuration to ensure consistency and reliability.
Easy Updates

With Helm, you can effortlessly manage application updates through in-place upgrades and custom hooks, minimizing disruptions caused by updates.
Simple Sharing

Helm Charts are easy to version control, share, and host on public or private repositories, facilitating team collaboration and application distribution.
Rollbacks

If issues arise during updates, the helm rollback command allows you to easily revert to a previous release version, ensuring stable application operation.

Helm’s Workflow

The workflow of Helm can be understood as follows:

Install Charts: Helm installs Charts into a Kubernetes cluster, creating a new Release each time.
Manage Releases: Each Release is independently managed, allowing the same Chart to be deployed multiple times to meet different requirements.
Find Charts: Charts can be found and obtained through Helm’s Chart Repository, allowing for new Charts to be deployed.

Overall Diagram Representation:

+---------------------+          +---------------------+
|    Chart Repository | <------> |        Helm         |
| (Store all Charts)    |          | (Install and Manage Charts)   |
+---------------------+          +---------+-----------+
                                         |
                                         v
                               +--------------------+
                               | Kubernetes Cluster |
                               | ------------------ |
                               | Release A          |
                               | Release B          |
                               | ...                |
                               +--------------------+

Three Core Concepts

Chart

Chart is a Helm package. It contains all the resource definitions required to run an application, tool, or service in a Kubernetes cluster. It can be compared to the Kubernetes version of Homebrew recipes, dpkg, or RPM files in Apt or Yum.

Diagram Representation:

+-----------------+
| Chart           |
| --------------- |
| Deployment.yaml |
| Service.yaml    |
| ConfigMap.yaml  |
| ...             |
+-----------------+

Helm will deploy the Chart in the following order:

Namespace —> NetworkPolicy —> ResourceQuota —> LimitRange —> PodSecurityPolicy —> PodDisruptionBudget —> ServiceAccount —> Secret —> SecretList —> ConfigMap —> StorageClass —> PersistentVolume —> PersistentVolumeClaim —> CustomResourceDefinition —> ClusterRole —> ClusterRoleList —> ClusterRoleBinding —> ClusterRoleBindingList —> Role —> RoleList —> RoleBinding —> RoleBindingList —> Service —> DaemonSet —> Pod —> ReplicationController —> ReplicaSet —> Deployment —> HorizontalPodAutoscaler —> StatefulSet —> Job —> CronJob —> Ingress —> APIService

Repository

Repository is a collection of Charts. It can be compared to the Apt or Yum repository.

Diagram Representation:

+---------------------+
| Chart Repository    |
| ------------------- |
| Chart A     Chart B |
| Chart C     Chart D |
| ...                 |
+---------------------+

Release

Release is a running instance of a Chart. A Chart can be installed multiple times in a Kubernetes cluster, each with its own Release. For example, if you have a MySQL Chart, you can install it twice in the same cluster, each with its own Release.

Diagram Representation:

+------------------+       +------------------+
| Release A 	   |       | Release B        |
|------------------|       |------------------|
| MySQL Chart      |       | MySQL Chart      |
| Release Name: A  |       | Release Name: B  |
| ...              |       | ...              |
+------------------+       +------------------+

Installation

Each release of Helm provides binary releases for a variety of OSes. These binary versions can be manually downloaded and installed.

Download your desired version
Unpack it (tar -zxvf helm-v3.0.0-linux-amd64.tar.gz)
Find the helm binary in the unpacked directory, and move it to its desired destination (mv linux-amd64/helm /usr/local/bin/helm)

Create Project Templates

Create a Helm project template using the command:

`1`	`helm create mychart`

This generates the following files and directory structure:

.
├── charts                          # Directory for dependency Charts
├── Chart.yaml                      # Metadata describing the Chart
├── templates                       # Directory for template files
│   ├── deployment.yaml             # Deployment template
│   ├── _helpers.tpl                # Helper templates
│   ├── hpa.yaml                    # Horizontal Pod Autoscaler template
│   ├── ingress.yaml                # Ingress template
│   ├── NOTES.txt                   # Installation notes
│   ├── serviceaccount.yaml         # Service Account template
│   ├── service.yaml                # Service template
│   └── tests                       # Test templates
│       └── test-connection.yaml    # Connection test template
└── values.yaml                     # Default values overriding template parameters

Summary

Helm simplifies the deployment and management of applications, tools, and services in Kubernetes clusters through three core concepts:

Charts: Define applications
Repositories: Enable sharing and distribution of Charts
Releases: Manage running instances of these applications in clusters

The charts provide application definitions, repositories facilitate sharing, and releases handle the lifecycle of deployed instances.

Complete Guide to Blocking Foreign IP Access to Servers

Wed, 01 Jan 2025 00:00:00 +0000

Why Block Foreign IP Addresses

Servers on public networks face severe security threats:

Numerous scanners on the internet continuously probe servers 24/7, attempting to gain unauthorized access and control
Server log analysis reveals that most attacks originate from foreign servers in countries like the Netherlands, United States, Singapore, and Japan
Whether using cloud servers or IDC-hosted servers, exposing service ports increases vulnerability whenever public services are offered

Solution Overview

For services primarily targeting domestic users, blocking foreign IP access can significantly enhance security.

Technical Foundation

Iptables: Linux firewall tool used to filter and block requests
Ipset module: Iptables extension that efficiently handles large IP address ranges
IPdeny: Provides regularly updated global IP address allocation data

Implementation Approach

Collect and organize domestic IP ranges into Ipset
Configure Iptables to use Ipset for checking source IPs
Allow domestic IP access while blocking foreign IP connections

Complete Implementation Steps

This guide is based on CentOS 7.6; commands may vary across different Linux distributions

Install Required Tools

1
2

# If ipset is not already installed
yum install -y ipset

Create IP Address Set

Download Domestic IP Ranges

`1`	`wget http://www.ipdeny.com/ipblocks/data/countries/cn.zone`

Convert to Ipset Commands

1
2

for i in `cat cn.zone`; do echo "ipset add china $i" >>ipset_result.sh; done
chmod +x ipset_result.sh

Create and Populate Ipset Collection

# Create the china set
ipset create china hash:net hashsize 10000 maxelem 1000000

# Add private network IP ranges
echo "ipset add china 10.0.0.0/8" >> ipset_result.sh
echo "ipset add china 172.0.0.0/8" >> ipset_result.sh
echo "ipset add china 192.0.0.0/8" >> ipset_result.sh

# Execute script to add IP ranges
bash ipset_result.sh

Verify IP Collection

1
2

ipset list china
ipset list china | wc -l  # Should contain approximately 8000+ entries

Configure Iptables Rules

# Clear existing rules (if necessary)
iptables -F
iptables -X

# Create basic rules
cat > /etc/sysconfig/iptables << EOF
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
# Add rules for other required ports below
# Example: -A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A INPUT -m set ! --match-set china src -j DROP
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
COMMIT
EOF

# Apply rules
iptables-restore < /etc/sysconfig/iptables

Ensure Configuration Persistence

To prevent configuration loss after server restart, set up persistence:

Persist Ipset Data

# Save Ipset data
ipset save china > /etc/ipset.conf

# Configure loading at startup
chmod +x /etc/rc.d/rc.local
echo "ipset restore < /etc/ipset.conf" >> /etc/rc.d/rc.local

Persist Iptables Rules

1
2

# Configure loading at startup
echo "/usr/sbin/iptables-restore < /etc/sysconfig/iptables" >> /etc/rc.d/rc.local

Automate IP Range Updates

To ensure IP ranges stay current, set up periodic updates:

# Create weekly update script
cat > /usr/local/bin/update_cn_ip.sh << EOF
#!/bin/bash
wget -O /tmp/cn.zone http://www.ipdeny.com/ipblocks/data/countries/cn.zone
ipset flush china
for ip in \$(cat /tmp/cn.zone); do ipset add china \$ip; done
# Add private network IP ranges
ipset add china 10.0.0.0/8
ipset add china 172.0.0.0/8
ipset add china 192.0.0.0/8
# Update persistence file
ipset save china > /etc/ipset.conf
EOF

chmod +x /usr/local/bin/update_cn_ip.sh

# Add weekly scheduled task
echo "0 0 * * 1 /usr/local/bin/update_cn_ip.sh" > /etc/cron.d/update_cn_ip

Verification and Troubleshooting

Testing Configuration

# Check Ipset collection
ipset list china

# Check Iptables rules
iptables -L -n

# Test domestic IP access (should be allowed)
# Test foreign IP access (should be blocked)

Common Issues and Solutions

Unable to SSH connect: Ensure SSH port rules are added before blocking rules
Local network access restricted: Verify private IP ranges are added to the china set
Configuration not persisting: Check rc.local file permissions and script content

Conclusion

By blocking foreign IP access, we can significantly reduce the risk of server attacks, particularly suitable for services primarily targeting domestic users. Note that this method may affect legitimate access from overseas users - adjust according to your specific business requirements.

Guide to Configuring TCP Service Forwarding with Ingress-Nginx in Kubernetes

Fri, 20 Dec 2024 00:00:00 +0000

This guide explains how to configure an Ingress-Nginx controller in a Kubernetes cluster to support TCP service forwarding, using MySQL service as an example.

Prerequisites

Kubernetes cluster installed
Ingress-Nginx controller deployed
Existing TCP service to expose (MySQL in this example)

Configuration Steps

Configure TCP Service Mapping

Create a tcp-services.yaml file to define TCP port mappings:

apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: ingress-nginx
data:
  # Syntax: <external_port>: "<namespace>/<service_name>:<target_port>"
  3306: "default/mysql-primary:3306"

Apply configuration:

`1`	`kubectl create -f tcp-services.yaml`

If the ConfigMap already exists, edit directly:

1
2

kubectl edit configmap tcp-services -n ingress-nginx
# Add new mapping under data: <external_port>: "<namespace>/<service_name>:<port>"

Update Ingress Controller Configuration

Edit the Ingress-Nginx Controller Deployment:

`1`	`kubectl edit deployment ingress-nginx-controller -n ingress-nginx`

Add the following configuration to the controller’s args section:

1
2
3

args:
  # ... existing parameters ...
  - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services

Configuring Ingress Controller Service

Edit the Ingress-Nginx Controller Service:

`1`	`kubectl edit service ingress-nginx-controller -n ingress-nginx`

Add TCP service port configuration in the ports section:

ports:
  # ... Other existing port configurations ...
  - name: mysql-primary
    port: 3306
    protocol: TCP
    targetPort: 3306

Verification

After completing configurations, verify with following methods:

Check ConfigMap creation:

`1`	`kubectl get configmap tcp-services -n ingress-nginx -o yaml`

Confirm Ingress Controller status:

`1`	`kubectl get pods -n ingress-nginx`

Verify port exposure:

`1`	`kubectl get svc ingress-nginx-controller -n ingress-nginx`

Notes

Ensure target services (e.g., MySQL) are properly running in the cluster
Check for port conflicts
Additional security group/firewall rule configurations may be required for cloud providers

Automatic Domain Resolution and Certificate Management for Kubernetes Ingress

Tue, 17 Dec 2024 00:00:00 +0000

Deployment

Please ensure that the following components are deployed:

cert-manager: Installation - cert-manager Documentation

`1`	`kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.15.3/cert-manager.yaml`

cert-manager-webhook-dnspod: imroc/cert-manager-webhook-dnspod: cert-manager webhook resolver for DNSPod (github.com)

`1`	`kubectl apply -f https://raw.githubusercontent.com/imroc/cert-manager-webhook-dnspod/master/bundle.yaml`

Modify the deployment file ingress-dnspod-solver.yaml as follows:

ingress-dnspod-solver.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: ingress-dnspod-solver
  namespace: cert-manager
  labels:
    app: ingress-dnspod-solver

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ingress-dnspod-solver
rules:
  - apiGroups: [ "networking.k8s.io" ]
    resources: [ "ingresses" ]
    verbs: [ "get", "list", "watch" ]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ingress-dnspod-solver
  labels:
    app: ingress-dnspod-solver
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-dnspod-solver
subjects:
  - apiGroup: ""
    kind: ServiceAccount
    name: ingress-dnspod-solver
    namespace: cert-manager

---
apiVersion: v1
kind: Secret
metadata:
  name: ingress-dnspod-solver
  namespace: cert-manager
type: Opaque
stringData:
  TENCENT_SECRET_KEY: "<secret_key>"  # Tencent Cloud API Secret Key

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: ingress-dnspod-solver
  namespace: cert-manager
data:
  DOMAIN: "example.com"                # Domain name
  POLICY: "retain"                     # Monitoring handling policy: retain (keep records on resource update), update (update records on resource update)
  TENCENT_SECRET_ID: "<secret_id>"  # Tencent Cloud API Secret ID
  RECORD_VALUE: "<record_value>"          # DNS record value

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ingress-dnspod-solver
  namespace: cert-manager
  labels:
    app: ingress-dnspod-solver
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-dnspod-solver
  template:
    metadata:
      labels:
        app: ingress-dnspod-solver
    spec:
      serviceAccountName: ingress-dnspod-solver
      containers:
        - name: ingress-dnspod-solver
          image: harbor.example.com/devops/ingress-dnspod-solver:latest
          imagePullPolicy: IfNotPresent
          env:
            - name: DOMAIN
              valueFrom:
                configMapKeyRef:
                  name: ingress-dnspod-solver
                  key: DOMAIN
            - name: POLICY
              valueFrom:
                configMapKeyRef:
                  name: ingress-dnspod-solver
                  key: POLICY
            - name: TENCENT_SECRET_ID
              valueFrom:
                configMapKeyRef:
                  name: ingress-dnspod-solver
                  key: TENCENT_SECRET_ID
            - name: TENCENT_SECRET_KEY
              valueFrom:
                secretKeyRef:
                  name: ingress-dnspod-solver
                  key: TENCENT_SECRET_KEY
            - name: RECORD_VALUE
              valueFrom:
                configMapKeyRef:
                  name: ingress-dnspod-solver
                  key: RECORD_VALUE

`1`	`kubectl apply -f ingress-dnspod-solver.yaml`

Source Code Analysis and Build

The source code is written in go and uses go mod to manage dependencies.

main.go

package main

import (
	"fmt"
	"github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common"
	"github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common/profile"
	dnspod "github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/dnspod/v20210323"
	"go.uber.org/zap"
	networkingv1 "k8s.io/api/networking/v1"
	"k8s.io/client-go/informers"
	"k8s.io/client-go/kubernetes"
	"k8s.io/client-go/rest"
	"k8s.io/client-go/tools/cache"
	"k8s.io/client-go/tools/clientcmd"
	"os"
)

var (
	logger       *zap.Logger
	version      string
	policy       string
	domain       string
	secretId     string
	secretKey    string
	recordValue  string
	dnsPodClient *dnspod.Client
	clientSet    *kubernetes.Clientset
)

func initLogger() {
	logger, _ = zap.NewDevelopment()
	defer func(logger *zap.Logger) {
		err := logger.Sync()
		if err != nil {
			fmt.Println("Logger sync error: ", err)
		}
	}(logger)
}

func initCheck() {
	version = os.Getenv("VERSION")
	if version == "" {
		version = "2021-03-23"
	}
	policy = os.Getenv("POLICY")
	if policy == "" {
		policy = "retain"
	}
	domain = os.Getenv("DOMAIN")
	if domain == "" {
		logger.Error("Please set the environment variable `DOMAIN`")
		os.Exit(1)
	}
	secretId = os.Getenv("TENCENT_SECRET_ID")
	secretKey = os.Getenv("TENCENT_SECRET_KEY")
	if secretId == "" || secretKey == "" {
		logger.Error("Please set the environment variables `TENCENT_SECRET_ID` and `TENCENT_SECRET_KEY`")
		os.Exit(1)
	}
	recordValue = os.Getenv("RECORD_VALUE")
	if recordValue == "" {
		logger.Error("Please set the environment variable `RECORD_VALUE`")
		os.Exit(1)
	}

	logger.Info("---------------------------------")
	logger.Info("Version" + ": " + version)
	logger.Info("Policy" + ": " + policy)
	logger.Info("Domain" + ": " + domain)
	logger.Info("SecretId" + ": " + secretId)
	logger.Info("SecretKey: *****")
	logger.Info("RecordValue" + ": " + recordValue)
	logger.Info("---------------------------------")
}

func getRecordDict() map[string]uint64 {
	recordDict := make(map[string]uint64)
	request := dnspod.NewDescribeRecordListRequest()
	request.Domain = common.StringPtr(domain)
	request.RecordType = common.StringPtr("A")
	request.Offset = common.Uint64Ptr(0)
	request.Limit = common.Uint64Ptr(10)

	// Response corresponds to the DescribeRecordListResponse instance
	response, err := dnsPodClient.DescribeRecordList(request)
	if err != nil {
		logger.Panic(err.Error())
	}
	for i := 0; i < len(response.Response.RecordList); i++ {
		recordDict[*response.Response.RecordList[i].Name] = *response.Response.RecordList[i].RecordId
	}
	domainCount := *response.Response.RecordCountInfo.ListCount
	domainTotal := *response.Response.RecordCountInfo.TotalCount
	for domainCount < domainTotal {
		request.Offset = common.Uint64Ptr(domainCount)
		response, err = dnsPodClient.DescribeRecordList(request)
		if err != nil {
			logger.Panic(err.Error())
		}
		for i := 0; i < len(response.Response.RecordList); i++ {
			recordDict[*response.Response.RecordList[i].Name] = *response.Response.RecordList[i].RecordId
		}
		domainCount += *response.Response.RecordCountInfo.ListCount
	}

	logger.Info("RecordDict: ", zap.Any("RecordDict", recordDict))
	return recordDict
}

func createRecord(subDomain string) {
	request := dnspod.NewCreateRecordRequest()
	request.Domain = common.StringPtr(domain)
	request.SubDomain = common.StringPtr(subDomain)
	request.RecordType = common.StringPtr("A")
	request.RecordLine = common.StringPtr("默认")
	request.Value = common.StringPtr(recordValue)

	_, err := dnsPodClient.CreateRecord(request)
	if err != nil {
		logger.Panic(err.Error())
	}
}

func updateRecord(recordId uint64, subDomain string) {
	request := dnspod.NewModifyRecordRequest()
	request.Domain = common.StringPtr(domain)
	request.RecordId = common.Uint64Ptr(recordId)
	request.SubDomain = common.StringPtr(subDomain)
	request.RecordType = common.StringPtr("A")
	request.RecordLine = common.StringPtr("默认")
	request.Value = common.StringPtr(recordValue)

	_, err := dnsPodClient.ModifyRecord(request)
	if err != nil {
		logger.Panic(err.Error())
	}
}

func deleteRecord(recordId uint64) {
	request := dnspod.NewDeleteRecordRequest()
	request.Domain = common.StringPtr(domain)
	request.RecordId = common.Uint64Ptr(recordId)

	_, err := dnsPodClient.DeleteRecord(request)
	if err != nil {
		logger.Panic(err.Error())
	}
}

func addHandler(obj interface{}) {
	logger.Info("Detected Ingress Add event")
	recordDict := getRecordDict()
	addIngress := obj.(*networkingv1.Ingress)
	for i := 0; i < len(addIngress.Spec.Rules); i++ {
		customDomain := addIngress.Spec.Rules[i].Host
		// Check if customDomain ends with domain
		if customDomain[len(customDomain)-len(domain):] != domain {
			continue
		}
		subDomain := customDomain[:len(customDomain)-len(domain)-1]
		if _, ok := recordDict[subDomain]; !ok {
			createRecord(subDomain)
		} else {
			if policy == "update" {
				updateRecord(recordDict[subDomain], subDomain)
			}
		}
	}
}

func updateHandler(oldObj, newObj interface{}) {
	logger.Info("Detected Ingress Update event")
	recordDict := getRecordDict()
	newIngress := newObj.(*networkingv1.Ingress)
	for i := 0; i < len(newIngress.Spec.Rules); i++ {
		customDomain := newIngress.Spec.Rules[i].Host
		// Check if customDomain ends with domain
		if customDomain[len(customDomain)-len(domain):] != domain {
			continue
		}
		subDomain := customDomain[:len(customDomain)-len(domain)-1]
		if _, ok := recordDict[subDomain]; !ok {
			createRecord(subDomain)
		} else {
			updateRecord(recordDict[subDomain], subDomain)
		}
	}
}

func deleteHandler(obj interface{}) {
	logger.Info("Detected Ingress Delete event")
	recordDict := getRecordDict()
	deleteIngress := obj.(*networkingv1.Ingress)
	for i := 0; i < len(deleteIngress.Spec.Rules); i++ {
		customDomain := deleteIngress.Spec.Rules[i].Host
		// Check if customDomain ends with domain
		if customDomain[len(customDomain)-len(domain):] != domain {
			continue
		}
		subDomain := customDomain[:len(customDomain)-len(domain)-1]
		if _, ok := recordDict[subDomain]; ok {
			deleteRecord(recordDict[subDomain])
		}
	}
}

func k8sInformer() {
	// Create shared Informer factory
	factory := informers.NewSharedInformerFactory(clientSet, 0)

	// Get Ingress watcher Informer
	ingressInformer := factory.Networking().V1().Ingresses().Informer()

	// Add event handlers for Ingress resource changes
	_, err := ingressInformer.AddEventHandler(cache.ResourceEventHandlerFuncs{
		AddFunc:    addHandler,
		UpdateFunc: updateHandler,
		DeleteFunc: deleteHandler,
	})
	if err != nil {
		return
	}

	// Start informer and wait for cache sync
	stopCh := make(chan struct{})
	defer close(stopCh)
	factory.Start(stopCh)
	factory.WaitForCacheSync(stopCh)

	// Block to keep listening for events
	<-stopCh
}

func main() {
	// Initialization
	initLogger()
	initCheck()

	// Create DNSPod client
	credential := common.NewCredential(secretId, secretKey)
	cpf := profile.NewClientProfile()
	cpf.HttpProfile.Endpoint = "dnspod.tencentcloudapi.com"
	dnsPodClient, _ = dnspod.NewClient(credential, "", cpf)

	// Create Kubernetes client
	var config *rest.Config
	var err error
	// Check if running inside cluster
	if os.Getenv("KUBERNETES_SERVICE_HOST") != "" {
		config, err = rest.InClusterConfig()
		if err != nil {
			logger.Panic(err.Error())
		}
	} else {
		if _, err = os.Stat("./.kube/config"); os.IsNotExist(err) {
			logger.Panic("Please set the kubeconfig file")
		}
		config, err = clientcmd.BuildConfigFromFlags("", "./.kube/config")
	}
	clientSet, err = kubernetes.NewForConfig(config)
	if err != nil {
		logger.Panic(err.Error())
	}
	k8sInformer()
}

Dockerfile

FROM golang:alpine AS builder
WORKDIR /app
COPY . .
RUN go mod download
RUN go build -o main .

FROM alpine:latest
WORKDIR /app
COPY --from=builder /app/main .
CMD ["./main"]

Build image

`1`	`docker build -t harbor.example.com/devops/ingress-dnspod-solver:latest .`

Push image

`1`	`docker push harbor.example.com/devops/ingress-dnspod-solver:latest`

Kubernetes Ingress-NGINX Service Certificate Configuration

Mon, 16 Dec 2024 00:00:00 +0000

Using Ingress | Kubernetes as the application traffic proxy for K8s, and configuring certificates

Service Deployment

Since the K8S cluster used is not provided by a cloud provider, deploying metallb is required when using LoadBalancer-type services for ingress-nginx. MetalLB is a load balancer implementation for bare-metal Kubernetes clusters, using standard routing protocols.

Deploy MetalLB

# Preview configuration changes (returns non-zero if changes detected)  
kubectl get configmap kube-proxy -n kube-system -o yaml | \  
sed -e "s/strictARP: false/strictARP: true/" | \  
kubectl diff -f - -n kube-system  

# Apply ARP configuration changes (returns non-zero only on errors)  
kubectl get configmap kube-proxy -n kube-system -o yaml | \  
sed -e "s/strictARP: false/strictARP: true/" | \  
kubectl apply -f - -n kube-system

1
2

# Install MetalLB components  
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.14.8/config/manifests/metallb-native.yaml

Create ip-pool.yaml to define the IP address pool:

apiVersion: metallb.io/v1beta1  
kind: IPAddressPool  
metadata:  
  name: default  
  namespace: metallb-system  
spec:  
  addresses:  
  - 172.16.0.90/32  # Manual IP assignment required for bare-metal clusters  
  autoAssign: true  
---  
apiVersion: metallb.io/v1beta1  
kind: L2Advertisement  
metadata:  
  name: default  
  namespace: metallb-system  
spec:  
  ipAddressPools:  
  - default

Deploy ingress-nginx

`1`	`kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.11.2/deploy/static/provider/cloud/deploy.yaml`

Configuring Certificates

Here we introduce 3 certificate configuration methods: manually generating self-signed certificates, manually obtaining trusted certificates, and using cert-manager to acquire certificates.

Manual Self-Signed Certificate

You can use OpenSSL or other tools to apply for certificates. The tls.crt certificate file and tls.key private key file, the base64 encoded content of the certificate and private key, can customize the relevant information of the certificate: country, province, city, company, department, domain name, validity period, etc.

# Generate private key
openssl genrsa -out tls.key 2048 # Generate private key file tls.key

# Generate certificate request
openssl req -new -key tls.key -out tls.csr -subj "/C=CN/ST=<state>/L=<city>/O=<company>/OU=<department>/CN=<domain>" # Generate certificate request file tls.csr

# Generate certificate
openssl x509 -req -in tls.csr -signkey tls.key -out tls.crt -days 3650 # Generate certificate file tls.crt

# View certificate information
openssl x509 -in tls.crt -text -noout

test-ingress.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: test-ingress


---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: service1
  namespace: test-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: service1
  template:
    metadata:
      labels:
        app: service1
    spec:
      containers:
      - name: service1
        image: nginx:alpine


---
apiVersion: v1
kind: Service
metadata:
  name: service1
  namespace: test-ingress
spec:
  selector:
    app: service1
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80


# letsencrypt certificate (manually created)
---
apiVersion: v1
kind: Secret
metadata:
  name: example-com
  namespace: test-ingress
data:
  tls.crt: <base64 encoded cert>
  tls.key: <base64 encoded key>
type: kubernetes.io/tls


---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: test-ingress
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - test.example.com
      secretName: example-com
  rules:
    - host: test.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: service1
                port:
                  number: 80

`1`	`kubectl apply -f test-ingress.yaml`

Manual Trusted Certificate Application

To apply for a certificate, you can use Let’s Encrypt or other Certificate Authorities (CAs).
The TLS certificate file (tls.crt) and private key file (tls.key) must be Base64-encoded. The default validity period is 90 days.

test-ingress.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: test-ingress

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: service1
  namespace: test-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: service1
  template:
    metadata:
      labels:
        app: service1
    spec:
      containers:
      - name: service1
        image: nginx:alpine

---
apiVersion: v1
kind: Service
metadata:
  name: service1
  namespace: test-ingress
spec:
  selector:
    app: service1
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80

# letsencrypt certificate (manually created)
---
apiVersion: v1
kind: Secret
metadata:
  name: example-com
  namespace: test-ingress
data:
  tls.crt: <base64 encoded cert>
  tls.key: <base64 encoded key>
type: kubernetes.io/tls

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: test-ingress
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - test.example.com
      secretName: example-com
  rules:
    - host: test.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: service1
                port:
                  number: 80

`1`	`kubectl apply -f test-ingress.yaml`

Using cert-manager to Obtain Certificates

cert-manager is a Kubernetes certificate management controller that utilizes CustomResourceDefinitions (CRDs), offering features such as certificate application, issuance, renewal, and deletion. Below are three methods for obtaining certificates using cert-manager: SelfSigned Issuer-type certificates, ACME Issuer-type certificates, and CA Issuer-type certificates.

Installing cert-manager

`1`	`kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.15.3/cert-manager.yaml`

By default, cert-manager will be installed in the cert-manager namespace. Although it can be deployed in a different namespace, this requires modifications to the deployment manifests.

After installation, verify the deployment by checking the pods in the cert-manager namespace:

$ kubectl get pods --namespace cert-manager

NAME                                       READY   STATUS    RESTARTS   AGE
cert-manager-cainjector-5fd6444f95-kmbmd   1/1     Running   0          60m
cert-manager-d894bbbd4-lrwp5               1/1     Running   0          60m
cert-manager-webhook-869674f96f-ljffr      1/1     Running   0          60m

Configuring SelfSigned Issuer Type Certificates

test-ingress.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: test-ingress


---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: service1
  namespace: test-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: service1
  template:
    metadata:
      labels:
        app: service1
    spec:
      containers:
      - name: service1
        image: nginx:alpine


---
apiVersion: v1
kind: Service
metadata:
  name: service1
  namespace: test-ingress
spec:
  selector:
    app: service1
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80


# SelfSigned Issuer: 100 years, automatically created
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-issuer
  namespace: test-ingress
spec:
  selfSigned: {}

# SelfSigned Issuer: 100 years, automatically created
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-com
  namespace: test-ingress
spec:
  secretName: example-com
  duration: 876000h  # 100 years
  renewBefore: 720h  # The certificate will be renewed 30 days before expiration
  issuerRef:
    name: selfsigned-issuer
    kind: ClusterIssuer
  commonName: test.example.com
  subject:
    organizations:
      - '*** Technology Co., Ltd.'
    organizationalUnits:
      - '*** Technology Co., Ltd. Operations Department'
  isCA: true
  privateKey:
    algorithm: RSA
    encoding: PKCS1
    size: 2048


---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: test-ingress
  annotations:
    cert-manager.io/cluster-issuer: selfsigned-issuer
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - test.example.com
      secretName: example-com
  rules:
    - host: test.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: service1
                port:
                  number: 80

`1`	`kubectl apply -f test-ingress.yaml`

ACME (Automated Certificate Management Environment) is a protocol used for automated certificate issuance and renewal. The ACME protocol is standardized by the IETF, with the most common current implementation being Let’s Encrypt. A key feature of the ACME protocol is that it allows Certificate Authorities (CAs) to verify the identity of certificate requesters without requiring human intervention. Another critical feature of the ACME protocol is its ability to enable automatic certificate renewal by CAs without manual involvement. The Issuer type represents an individual account registered with an ACME certificate authority server. When creating a new ACME Issuer, the Certificate Manager will generate a private key used for identification on the ACME server. By default, certificates issued by public ACME servers are typically trusted by client computers. This means that, for example, websites secured by ACME certificates issued for specific URLs will be automatically trusted by most web browsers. ACME certificates are generally free.

Configuring ACME Issuer with HTTP01 Certificate Type

The HTTP01 challenge is completed by exposing a computed secret key at a publicly accessible HTTP URL endpoint. This URL will use the domain name for which the certificate is being requested. Once the ACME server can retrieve this secret key from the URL via the internet, it verifies your ownership of the domain.

When creating an HTTP01 challenge, cert-manager will automatically configure your cluster ingress to route traffic accessing this URL to a small web server that presents the secret key. In simpler terms, cert-manager automatically creates an ingress route to a lightweight web server responsible for displaying the verification key, thereby proving domain ownership.

Note:

Fake certificate (1-year validity, auto-created): kubernetes ingress controller fake cert
Example file: test-ingress.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: test-ingress

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: service1
  namespace: test-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: service1
  template:
    metadata:
      labels:
        app: service1
    spec:
      containers:
      - name: service1
        image: nginx:alpine

---
apiVersion: v1
kind: Service
metadata:
  name: service1
  namespace: test-ingress
spec:
  selector:
    app: service1
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80

# ACME certificate, automatically created
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: sencrypt-prod
  namespace: test-ingress
spec:
  acme:
    email: CoderKang@hotmail.com
    privateKeySecretRef:
      name: sencrypt-prod
    server: https://acme-v02.api.letsencrypt.org/directory
    solvers:
      - http01:
          ingress:
            class: nginx

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: test-ingress
  annotations:
    cert-manager.io/cluster-issuer: sencrypt-prod
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - test.example.com
      secretName: example-com
  rules:
    - host: test.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: service1
                port:
                  number: 80

`1`	`kubectl apply -f test-ingress.yaml`

Configuring ACME Issuer dns01-type Certificates

The DNS01 challenge is completed by providing a cryptographic key that exists in a DNS TXT record. Once this TXT record propagates across the internet, the ACME server can retrieve the key via DNS queries and verify that the client requesting the certificate is the domain owner.
If proper permissions are granted, cert-manager will automatically add this TXT record to your specified DNS provider.

However, since Tencent Cloud DNS is used here, and cert-manager does not natively support Tencent Cloud DNS, a custom resolver (cert-manager webhook) is required.

DNS01 - cert-manager Documentation

`1`	`kubectl apply -f https://raw.githubusercontent.com/imroc/cert-manager-webhook-dnspod/master/bundle.yaml`

test-ingress.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: test-ingress


---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: service1
  namespace: test-ingress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: service1
  template:
    metadata:
      labels:
        app: service1
    spec:
      containers:
      - name: service1
        image: nginx:alpine


---
apiVersion: v1
kind: Service
metadata:
  name: service1
  namespace: test-ingress
spec:
  selector:
    app: service1
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80


# letsencrypt automatically created, valid for 90 days
# Configure the SecretId and SecretKey of the Tencent Cloud DNS provider
# https://console.dnspod.cn/account/token/apikey
---
apiVersion: v1
stringData:
  secret-key: <tencent cloud secret key>  # Tencent Cloud SecretKey
kind: Secret
metadata:
  name: dnspod-secret
  namespace: cert-manager
type: Opaque

---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: sencrypt-prod
  namespace: test-ingress
spec:
  acme:
    email: CoderKang@hotmail.com
    preferredChain: ""
    privateKeySecretRef:
      name: dnspod-letsencrypt
    server: https://acme-v02.api.letsencrypt.org/directory
    solvers:
      - dns01:
          webhook:
            config:
              secretId: <tencent cloud secret id>  # Tencent Cloud SecretId
              secretKeyRef:
                key: secret-key
                name: dnspod-secret
              ttl: 600
            groupName: acme.imroc.cc
            solverName: dnspod

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-com
  namespace: test-ingress
spec:
  dnsNames:
    - test.example.com
  issuerRef:
    kind: ClusterIssuer
    name: sencrypt-prod
  secretName: example-com

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: test-ingress
  annotations:
    cert-manager.io/cluster-issuer: sencrypt-prod
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - test.example.com
      secretName: example-com
  rules:
    - host: test.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: service1
                port:
                  number: 80

`1`	`kubectl apply -f test-ingress.yaml`

Deploying JumpServer on Kubernetes with DingTalk Alert Integration

Tue, 19 Nov 2024 00:00:00 +0000

Ingress Configuration

Requires setting up ingress (see 13-K8s ingress-nginx Service Certificate) and enabling TCP forwarding (see 23-K8s ingress TCP Forwarding).

JumpServer Deployment

“Installation”

`1`	`helm install jms-k8s ./jumpserver-v4.6.0 -n jumpserver --create-namespace -f values.yaml`

“Upgrade”

`1`	`helm upgrade jms-k8s ./jumpserver-v4.6.0 -n jumpserver --create-namespace -f values.yaml`

“Uninstallation”

`1`	`helm -n jumpserver delete jms-k8s`

Core Component Image

Modified core component to support DingTalk alerts.

docker pull docker.1ms.run/jumpserver/core:v4.6.0-ce

cd core
docker build -t jumpserver/core:v4.6.1-ce .

“Dockerfile”

1
2
3

    FROM docker.1ms.run/jumpserver/core:v4.6.0-ce

    COPY ./notifications.py /opt/jumpserver/apps/notifications/notifications.py

“notifications.py”

import textwrap
import time
import traceback
from itertools import chain

from celery import shared_task
from django.utils.translation import gettext_lazy as _
from html2text import HTML2Text

from common.utils import lazyproperty
from common.utils.timezone import local_now
from notifications.backends import BACKEND
from settings.utils import get_login_title
from terminal.const import RiskLevelChoices
from users.models import User
from .models import SystemMsgSubscription, UserMsgSubscription

__all__ = ('SystemMessage', 'UserMessage', 'system_msgs', 'Message')

system_msgs = []
user_msgs = []


class MessageType(type):
    def __new__(cls, name, bases, attrs: dict):
        clz = type.__new__(cls, name, bases, attrs)

        if 'message_type_label' in attrs \
                and 'category' in attrs \
                and 'category_label' in attrs:
            message_type = clz.get_message_type()

            msg = {
                'message_type': message_type,
                'message_type_label': attrs['message_type_label'],
                'category': attrs['category'],
                'category_label': attrs['category_label'],
            }
            if issubclass(clz, SystemMessage):
                system_msgs.append(msg)
            elif issubclass(clz, UserMessage):
                user_msgs.append(msg)

        return clz


@shared_task(
    verbose_name=_('Publish the station message'),
    description=_(
        """This task needs to be executed for sending internal messages for system alerts, 
        work orders, and other notifications"""
    )
)
def publish_task(receive_user_ids, backends_msg_mapper):
    Message.send_msg(receive_user_ids, backends_msg_mapper)


def send_dingtalk_task(message):
    # Send to custom webhook
    import hmac, hashlib, base64, urllib.parse
    timestamp = str(round(time.time() * 1000))
    # JumpServer credentials
    access_token = '<access token>'
    secret = '<secret>'

    secret_enc = secret.encode('utf-8')
    string_to_sign = f'{timestamp}\n{secret}'
    string_to_sign_enc = string_to_sign.encode('utf-8')
    hmac_code = hmac.new(secret_enc, string_to_sign_enc, digestmod=hashlib.sha256).digest()
    sign = urllib.parse.quote_plus(base64.b64encode(hmac_code))
    form_data = {
        "msgtype": "markdown",
        "markdown": {
            "title": "JumpServer Monitoring Alert",
            "text": message
        },
        "at": {
            "atMobiles": [
                "<mobile>"
            ],
            "isAtAll": False
        }
    }
    import requests
    try:
        res = requests.post(
            url=f'https://oapi.dingtalk.com/robot/send?access_token={access_token}&timestamp={timestamp}&sign={sign}',
            json=form_data)
    except Exception as e:
        with open('/opt/error.log', 'a+') as f:
            f.write(str(e))


class Message(metaclass=MessageType):
    """
    What's encapsulated here?
        Templates for different messages with a unified sending interface
        - publish: Implementation relates to message subscription table structure
        - send_msg
    """
    message_type_label: str
    category: str
    category_label: str
    text_msg_ignore_links = True
    command = None

    @classmethod
    def get_message_type(cls):
        return cls.__name__

    def publish_async(self):
        self.publish(is_async=True)

    @classmethod
    def gen_test_msg(cls):
        raise NotImplementedError

    def publish(self, is_async=False):
        raise NotImplementedError

    def get_backend_msg_mapper(self, backends):
        backends = set(backends)
        backends.add(BACKEND.SITE_MSG)  # Site message is mandatory
        backends_msg_mapper = {}
        for backend in backends:
            backend = BACKEND(backend)
            if not backend.is_enable:
                continue
            get_msg_method = getattr(self, f'get_{backend}_msg', self.get_common_msg)
            msg = get_msg_method()
            backends_msg_mapper[backend] = msg
        return backends_msg_mapper

    @staticmethod
    def send_msg(receive_user_ids, backends_msg_mapper):
        for backend, msg in backends_msg_mapper.items():
            try:
                backend = BACKEND(backend)
                client = backend.client()
                users = User.objects.filter(id__in=receive_user_ids).all()
                client.send_msg(users, **msg)
            except NotImplementedError:
                continue
            except:
                traceback.print_exc()

    @classmethod
    def send_test_msg(cls, ding=True, wecom=False):
        msg = cls.gen_test_msg()
        if not msg:
            return

        from users.models import User
        users = User.objects.filter(username='admin')
        backends = []
        if ding:
            backends.append(BACKEND.DINGTALK)
        if wecom:
            backends.append(BACKEND.WECOM)
        msg.send_msg(users, backends)

    @staticmethod
    def get_common_msg() -> dict:
        return {'subject': '', 'message': ''}

    def get_html_msg(self) -> dict:
        return self.get_common_msg()

    @staticmethod
    def html_to_markdown(html_msg):
        h = HTML2Text()
        h.body_width = 0
        content = html_msg['message']
        html_msg['message'] = h.handle(content)
        return html_msg

    def get_markdown_msg(self) -> dict:
        return self.html_to_markdown(self.get_html_msg())

    def get_text_msg(self) -> dict:
        h = HTML2Text()
        h.body_width = 90
        msg = self.get_html_msg()
        content = msg['message']
        h.ignore_links = self.text_msg_ignore_links
        msg['message'] = h.handle(content)
        return msg

    @lazyproperty
    def common_msg(self) -> dict:
        return self.get_common_msg()

    @lazyproperty
    def text_msg(self) -> dict:
        msg = self.get_text_msg()
        return msg

    @lazyproperty
    def markdown_msg(self):
        return self.get_markdown_msg()

    @lazyproperty
    def get_jumpserver_dingtalk_msg(self) -> str:
        date_str = time.strftime("%Y-%m-%d %H:%M:%S", time.localtime())
        msg = f"""## <font color="#FF0000">[JumpServer Monitoring Alert](https://jumpserver.example.com/)</font>🔥
### <font color="#FF0000">Alert Status</font>: {RiskLevelChoices.get_label(self.command['risk_level'])}
### <font color="#FF0000">Target Host</font>: {self.command['asset']}
### <font color="#FF0000">Target User</font>: {self.command['user']}
### <font color="#FF0000">Executed Command</font>: `{self.command['input']}`
### <font color="#FF0000">Alert Details</font>: User {self.command['user']} executed a risky command `{self.command['input']}` on {self.command['asset']}, please handle promptly!
### <font color="#FF0000">Trigger Time</font>: {date_str}"""
        return msg

    @lazyproperty
    def html_msg(self) -> dict:
        msg = self.get_html_msg()
        return msg

    @lazyproperty
    def html_msg_with_sign(self):
        msg = self.get_html_msg()
        msg['message'] = textwrap.dedent("""
        {}
        <small>
        <br />
        —
        <br />
        {}
        </small>
        """).format(msg['message'], self.signature)
        return msg

    @lazyproperty
    def text_msg_with_sign(self):
        msg = self.get_text_msg()
        msg['message'] = textwrap.dedent("""
        {}
        —
        {}
        """).format(msg['message'], self.signature)
        return msg

    @lazyproperty
    def signature(self):
        return get_login_title()

    # --------------------------------------------------------------
    # Support different messaging formats
    def get_dingtalk_msg(self) -> dict:
        # DingTalk restricts identical messages within a day, add timestamp suffix
        message = self.markdown_msg['message']
        time = local_now().strftime('%Y-%m-%d %H:%M:%S')
        suffix = '\n{}: {}'.format(_('Time'), time)

        return {
            'subject': self.markdown_msg['subject'],
            'message': message + suffix
        }

    def get_wecom_msg(self) -> dict:
        return self.markdown_msg

    def get_feishu_msg(self) -> dict:
        return self.markdown_msg

    def get_lark_msg(self) -> dict:
        return self.markdown_msg

    def get_email_msg(self) -> dict:
        return self.html_msg_with_sign

    def get_site_msg_msg(self) -> dict:
        return self.html_msg

    def get_slack_msg(self) -> dict:
        return self.markdown_msg

    def get_sms_msg(self) -> dict:
        return self.text_msg_with_sign

    @classmethod
    def get_all_sub_messages(cls):
        def get_subclasses(cls):
            """Returns all subclasses of argument, cls"""
            if issubclass(cls, type):
                subclasses = cls.__subclasses__(cls)
            else:
                subclasses = cls.__subclasses__()
            for subclass in subclasses:
                subclasses.extend(get_subclasses(subclass))
            return subclasses

        messages_cls = get_subclasses(cls)
        return messages_cls

    @classmethod
    def test_all_messages(cls, ding=True, wecom=False):
        messages_cls = cls.get_all_sub_messages()

        for _cls in messages_cls:
            try:
                _cls.send_test_msg(ding=ding, wecom=wecom)
            except NotImplementedError:
                continue


class SystemMessage(Message):
    def publish(self, is_async=False):
        subscription = SystemMsgSubscription.objects.get(
            message_type=self.get_message_type()
        )

        # Only send through enabled backends
        receive_backends = subscription.receive_backends
        receive_backends = BACKEND.filter_enable_backends(receive_backends)

        users = [
            *subscription.users.all(),
            *chain(*[g.users.all() for g in subscription.groups.all()])
        ]

        receive_user_ids = [u.id for u in users]
        backends_msg_mapper = self.get_backend_msg_mapper(receive_backends)
        if is_async:
            send_dingtalk_task(self.get_jumpserver_dingtalk_msg)
            # publish_task.delay(receive_user_ids, backends_msg_mapper)
        else:
            self.send_msg(receive_user_ids, backends_msg_mapper)

    @classmethod
    def post_insert_to_db(cls, subscription: SystemMsgSubscription):
        pass

    @classmethod
    def gen_test_msg(cls):
        raise NotImplementedError


class UserMessage(Message):
    user: User

    def __init__(self, user):
        self.user = user

    def publish(self, is_async=False):
        """
        Send messages through user-configured channels
        """
        sub = UserMsgSubscription.objects.get(user=self.user)
        backends_msg_mapper = self.get_backend_msg_mapper(sub.receive_backends)
        receive_user_ids = [self.user.id]
        if is_async:
            send_dingtalk_task(self.get_jumpserver_dingtalk_msg)
            # publish_task.delay(receive_user_ids, backends_msg_mapper)
        else:
            self.send_msg(receive_user_ids, backends_msg_mapper)

    @classmethod
    def get_test_user(cls):
        from users.models import User
        return User.objects.all().first()

    @classmethod
    def gen_test_msg(cls):
        raise NotImplementedError

Translated text here

values.yaml

    replicaCount: 1
    
    image:
      repository: nginx
      pullPolicy: IfNotPresent
      tag: ""
    
    imagePullSecrets: []
    nameOverride: ""
    fullnameOverride: ""
    
    serviceAccount:
      create: true
      annotations: {}
      name: ""
    
    podAnnotations: {}
    
    podSecurityContext: {}
    
    securityContext: {}
    
    service:
      type: ClusterIP
      port: 80
    
    ingress:
      enabled: false
    
    resources: {}
    
    autoscaling:
      enabled: false
    
    nodeSelector: {}
    
    tolerations: []
    
    affinity: {}

Modify the Image Version of the Jumpserver Core Module

To modify the image version of the Jumpserver core module, follow these steps:

# Step 1: Edit the core module deployment
kubectl -n jumpserver edit deployments.apps jms-k8s-jumpserver-jms-core

# Step 2: Locate the image field in the container configuration
# Find the containers section and modify the image version to:
# - name: jms-core
#   image: jumpserver/core:v4.6.1-ce

Docker Standalone IP Configuration Guide - Macvlan and Pipework Implementation

Thu, 11 Jul 2024 00:00:00 +0000

Macvlan

📝 Note

Network drivers overview | Docker Docs

Macvlan network driver | Docker Docs

Certain applications, particularly legacy software or programs that monitor network traffic, require direct connections to the physical network. In such scenarios, you can utilize the macvlan network driver to assign each container’s virtual network interface a unique MAC address, making them appear as physical network interfaces directly attached to the physical network. This configuration requires specifying a physical interface on the Docker host for macvlan usage, along with defining the network’s subnet and gateway. You may also employ different physical network interfaces to isolate your macvlan networks.

cat <<EOF > docker_network_macvlan.yaml
name: docker_network_macvlan
services:
  docker_network_macvlan:
    image: busybox
    container_name: docker_network_macvlan
    networks:
      macvlan_net:
        ipv4_address: 192.168.142.234  # Configure static IP
    privileged: true
    cap_add:
      - NET_ADMIN  # Add network privileges
    command: sleep infinity
    ports:
      - "17000:17000"

networks:
  macvlan_net:
    driver: macvlan  # Use macvlan network type
    # Macvlan networks allow containers to have MAC addresses, making them appear as physical devices on the network.
    # The Docker daemon can route traffic through container's MAC address.
    # Macvlan is often the best choice when dealing with legacy applications that expect direct physical network connection.
    driver_opts:
      parent: eno1  # Specify network interface
    ipam:
      config:
        - subnet: 192.168.142.0/24    # Subnet
          ip_range: 192.168.142.0/24  # IP range
          gateway: 192.168.142.1      # Gateway
EOF

`1`	`docker-compose -f docker_network_macvlan.yaml up -d`

📌 Important

The container currently cannot communicate with the host machine (using eno1 NIC) or other containers.

Pipework

jpetazzo/pipework: Software-Defined Networking tools for LXC (LinuX Containers) (github.com)

`1`	`sudo docker run -itd --name test ubuntu /bin/bash`

`1`	`sudo docker exec test ip addr show`

The host machine’s network is 172.16.0.100. Configure the network for container “test” and connect it to bridge br0, where the address after @ represents the gateway:

1
2

sudo pipework br0 test 172.16.0.156/24@172.16.0.1
# ip addr add 172.16.0.254/24 dev br0

sudo ip addr add 172.16.0.100/24 dev br0; \
    sudo ip addr del 172.16.0.100/24 dev enp1s0; \
    sudo brctl addif br0 enp1s0; \
    sudo ip route del default; \
    sudo ip route add default via 172.16.0.1 dev br0

Understanding Pod Termination Signals in Kubernetes

Wed, 27 Mar 2024 00:00:00 +0000

Kubernetes Pod Deletion Lifecycle

Deleting a container in Kubernetes involves the following steps and concepts. Specifically, container lifecycle management is handled by Pods and controllers (such as Deployments, ReplicaSets, etc.). You can delete a Pod or container using the kubectl delete command in Kubernetes, and the exact behavior depends on the resource type involved.

Process of Deleting a Container

Assume you execute the following command to delete a Pod (the container is one or more processes within the Pod):

`1`	`kubectl delete pod <pod-name>`

When this command is executed, the following process occurs:

Pod Scheduling Management: Controllers in Kubernetes (e.g., Deployment or ReplicaSet) monitor the number of Pod replicas and ensure the running Pod count matches the desired state. If you delete a Pod and the replica count is set to greater than 1 (e.g., a Deployment with 3 replicas), the controller will automatically create a new Pod to replace the deleted one.

If the replica count is set to 1 and there are no additional controllers managing the Pod, the Pod will not be recreated after deletion unless the replica count is manually adjusted.
Container Termination: When you delete a Pod, Kubernetes sends a termination signal (SIGTERM) to the containers running in the Pod. This gives the containers a grace period (default: 30 seconds) to perform cleanup operations and release resources.
Forced Termination: If a container does not stop within 30 seconds, it is forcibly terminated (via SIGKILL).
Pod Deletion: Once the containers terminate, the Pod itself is removed from the cluster, and associated resources (e.g., volumes) are cleaned up or reattached, depending on the volume type.
Resource Cleanup: The deleted container releases its allocated resources (CPU, memory, network connections, etc.). If the container uses mounted volumes (e.g., Persistent Volumes), their cleanup behavior is determined by the ReclaimPolicy (e.g., Retain or Delete).

Container Lifecycle Management

In Kubernetes, a container’s lifecycle is generally controlled by a Pod, while the Pod lifecycle itself is a process consisting of creation, execution, and termination phases.

Pod Lifecycle

Pending: The Pod is scheduled onto a node, but its containers have not yet started running. This is typically due to pending container image pulls or incomplete resource scheduling.
Running: The containers are actively running, and the Pod is executing.
Succeeded: All containers have terminated successfully, marking the end of the Pod’s lifecycle.
Failed: At least one container exited abnormally (e.g., by returning status code 1 or another non-zero exit status) and cannot be restarted, causing the Pod to enter the Failed state.
Unknown: Unable to retrieve the Pod’s status, usually due to node failures or network issues.

Container Lifecycle

A container’s lifecycle is managed by its Pod, but you can customize lifecycle behaviors in Pod configurations using lifecycle hooks, including:

PreStop: Invoked before container termination to perform cleanup tasks, such as closing network connections or saving temporary data.
PostStart: Invoked after container startup to execute initialization tasks.

Lifecycle Example

Pod Creation:
- The Pod is created, and the containers begin initializing.
- The containers start pulling images and booting up.
- During startup, the PostStart hook (if configured) is executed.
Pod Execution:
- Containers actively provide services during runtime.
- Resources (e.g., containers, volumes) remain active.
Pod Deletion:
- When deleting the Pod, Kubernetes sends a SIGTERM signal to the containers to request graceful termination.
- If a container fails to exit within the defined grace period, Kubernetes forces termination via SIGKILL.
- Post-deletion, Pod-associated resources (e.g., network, volumes) are cleaned up, and the containers/Pod enter a terminated state.
Container Exit:
- When a container exits, its exit status code is recorded. A status code of 0 indicates success; otherwise, it is marked as a failure.
- Based on the Pod’s restart policy (e.g., Always), Kubernetes may restart the container after termination.

Restart Policy and Replica Count

Always (Default): If a container crashes or is deleted, Kubernetes will attempt to restart it.
OnFailure: Containers are only restarted if they exit abnormally (return a non-zero exit code).
Never: Containers will not restart automatically and will remain permanently stopped after exiting.

Summary

When you delete a container, Kubernetes manages the Pod and container lifecycle through the following steps:

Graceful Container Shutdown: Sends a SIGTERM signal to allow graceful termination (default 30 seconds).
Forced Termination: If the container does not exit within the specified time, a SIGKILL signal is sent to forcefully terminate it.
Pod Deletion: After the container stops, the Pod resource is cleaned up.
Controller Behavior: If the Pod is managed by a Deployment or ReplicaSet, the controller ensures the Pod count matches the desired state, creating new Pods as needed to replace deleted ones.

If the replica count is 1, deleting the Pod will not automatically create a new Pod unless the replica count is manually adjusted or resources are recreated.

Graceful Termination vs. Forced Termination of Containers

In Kubernetes, container termination has two main phases: Graceful Termination and Forced Termination. These phases differ in how Kubernetes handles the exit process and whether the container has an opportunity to perform cleanup operations.

Container Graceful Termination

When you delete a Pod or container, Kubernetes will first attempt to terminate the container gracefully, which is known as Graceful Termination.

Sending SIGTERM Signal: When Kubernetes requests container termination, it sends a SIGTERM (termination signal). The container can catch this signal and begin normal shutdown operations.

Upon receiving SIGTERM, the container can perform actions such as:
- Closing network connections
- Cleaning up occupied resources (e.g., file handles, temporary data)
- Executing custom shutdown logic (e.g., database commits, logging)
Grace Period: After receiving SIGTERM, the container is granted a “Grace Period” to complete cleanup tasks. By default, this period is 30 seconds, which can be adjusted through the terminationGracePeriodSeconds field in the Pod manifest.
Graceful Exit: If the container exits normally within the grace period (completes cleanup and exits), Kubernetes marks the container status as Succeeded or Terminated. The Pod will then handle subsequent actions according to termination policies (e.g., deletion or rescheduling).

Forced Termination of Containers

If containers fail to exit normally within the grace period, Kubernetes initiates forced termination. This occurs when containers do not complete cleanup operations or do not respond to termination requests, prompting Kubernetes to take stricter measures to terminate them.

Sending SIGKILL Signal: If a container does not terminate gracefully within the time specified by terminationGracePeriodSeconds, Kubernetes sends a SIGKILL (force-kill signal) to stop the container immediately. Unlike SIGTERM, the SIGKILL signal cannot be captured or handled by the container, preventing any cleanup operations from being executed.
Immediate Termination: Upon receiving SIGKILL, the container is halted abruptly, and all running processes are killed. This means processes have no chance to release resources (e.g., file handles, temporary storage) and may leave behind uncleaned states.
Inability to Clean Up Resources: Forcibly terminated containers cannot perform any cleanup operations, potentially resulting in unreleased resources such as temporary files, memory, database connections, etc.

Differences Between Graceful Termination and Forced Termination of Containers

Feature	Graceful Termination	Forced Termination
Signal	SIGTERM	SIGKILL
Cleanup Time	Allows cleanup (Default: 30 seconds, configurable)	No cleanup time, immediate kill
Termination Process	Container can catch SIGTERM for cleanup	Container cannot catch SIGKILL
Resource Release	Resources (file handles, DB connections) released	Resources may leak
Container State	Exits normally (Succeeded/Terminated)	Forcibly stopped (Terminated)
Trigger Scenario	Normal shutdown (e.g., deletion request)	Unresponsive or grace period expired

Configuring Container Termination Behavior in Pod Specifications

To control container termination behavior, configure the terminationGracePeriodSeconds field in the Pod specification. Default: 30 seconds (configurable).

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  terminationGracePeriodSeconds: 60  # Set grace period to 60 seconds
  containers:
    - name: mycontainer
      image: myimage

Container Lifecycle Hooks

Kubernetes also provides lifecycle hooks that allow you to insert custom behaviors during a container’s startup and shutdown processes, particularly executing additional operations when the container terminates:

PreStop: Executes before container termination. You can use this hook to perform cleanup tasks, send notifications, or other operations. For example, terminating external service connections or closing database links.

Example:

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  containers:
    - name: mycontainer
      image: myimage
      lifecycle:
        preStop:
          exec:
            command: ["sh", "-c", "echo 'Container is stopping' > /tmp/shutdown.log"]

Summary

Graceful Container Termination: Kubernetes sends a SIGTERM signal to request the container to exit gracefully. The container has an opportunity to perform cleanup operations and release resources, typically completing within the grace period.
Forced Container Termination: If the container fails to exit normally within the grace period, Kubernetes sends a SIGKILL signal to forcibly terminate the container. The container cannot perform any cleanup and is immediately stopped, freeing up resources.

By appropriately configuring a Pod’s grace period and lifecycle hooks, you can control container termination behaviors, ensuring minimized resource leaks and service disruptions during container exits.

SIGTERM and SIGKILL

SIGTERM and SIGKILL are two commonly used Unix/Linux signals. Both are used to send termination requests to processes, but they differ in their purpose, behavior, and consequences. Below is a detailed explanation of these two signals:

SIGTERM (Signal Terminate)

Signal Number: 15
Action: SIGTERM is a signal requesting a process to terminate. It notifies the process to exit gracefully, allowing time to complete cleanup operations, save state, and release resources. This is the default signal for terminating processes.
Sending Method: You can send SIGTERM using the kill command or programmatically via kill(pid, SIGTERM).

Example:

`1`	`kill <pid> # Sends SIGTERM by default`

Characteristics:
- SIGTERM enables graceful exit and can be caught and handled by the process.
- Upon receiving SIGTERM, a process may perform cleanup tasks such as closing database connections, saving temporary data, or releasing file handles.
- If the process handles SIGTERM, it can execute cleanup logic before terminating.
- If the process does not respond to SIGTERM, the operating system will wait by default for a period (usually 30 seconds) before sending SIGKILL.
Handling Behavior:
- A process may choose to catch and handle SIGTERM, for example, by registering a signal handler function.
- If the process ignores SIGTERM and fails to exit within the timeout, SIGKILL will be enforced.
Use Cases:
- Used when allowing cleanup operations, e.g., closing network connections, saving data, or logging shutdown events.
- In Kubernetes or Docker, SIGTERM is sent first when deleting a container to request a graceful shutdown.

SIGKILL (Signal Kill) - Forced Termination Signal

Number: 9
Function: SIGKILL is a signal that forces immediate termination of a process. It instructs the operating system to stop the target process without allowing any cleanup operations. This signal cannot be caught, ignored, or handled by the process.

Example:

`1`	`kill -9 <pid> # Send SIGKILL signal`

Characteristics:
- SIGKILL is a mandatory termination signal; the targeted process cannot perform any cleanup operations.
- The operating system terminates the process immediately and releases its occupied resources, regardless of the process’s state.
- Processes cannot intercept or ignore SIGKILL; it removes the process from memory instantly.
- Upon sending SIGKILL, the process loses the opportunity to close files, free memory, write logs, or perform other essential cleanup tasks.
Use Cases:
- Use SIGKILL to forcibly terminate a process when it does not respond to SIGTERM or is hung (e.g., frozen or stuck in an infinite loop).
- In operating systems, SIGKILL is employed to terminate unresponsive or zombie processes, ensuring complete cleanup.

Comparison: SIGTERM vs. SIGKILL

Characteristic	SIGTERM (15)	SIGKILL (9)
Function	Requests graceful termination, allowing the process to perform cleanup tasks	Forces immediate process termination; cannot be captured or handled
Capturable/Handled	Can be captured by the process; supports custom termination logic	Cannot be captured or processed by the process
Graceful Exit	Permits resource cleanup (e.g., closing files, saving data)	Instantly terminates the process without cleanup
Ignorable	Can be ignored by the process (if no handler is registered)	Cannot be ignored; enforced termination
Default Behavior	System typically waits for process exit (default: 30 seconds)	Immediately kills the process without delay
Use Cases	Safe process shutdown requiring resource release or state preservation	Terminating unresponsive processes or overriding failed SIGTERM
OS Behavior	OS waits for the process to exit within a timeout	OS forcibly terminates the process and releases all resources

SIGTERM and SIGKILL in Kubernetes

In Kubernetes, when you delete a Pod or container, Kubernetes first sends a SIGTERM signal to the container, requesting its main process to gracefully exit. The process within the container has a specific period (30 seconds by default) to respond to SIGTERM and perform cleanup operations. If the container does not exit normally during this grace period, Kubernetes sends SIGKILL to forcibly terminate the container. This process aims to ensure graceful shutdown where possible, while using forceful termination if unresponsive.

SIGTERM Signal

Sent to: The main process within the container.
Behavior: When the container runs, there is a main process (typically the process defined by CMD or ENTRYPOINT at container startup). When Kubernetes requests to stop the container, it sends a SIGTERM signal to the container’s main process. SIGTERM is a graceful termination signal, meaning the process should handle it by performing cleanup tasks and exiting properly.
Behavior inside the container: The main process has the opportunity to capture the SIGTERM signal and execute cleanup operations within the allotted time, such as:
- Closing database connections.
- Saving log files or persistent data.
- Cleaning up temporary files or caches.
The container may capture the signal and execute custom termination logic (e.g., via the lifecycle hook PreStop) upon receiving SIGTERM.
Grace Period: Kubernetes provides a default of 30 seconds (configurable) for the container’s main process to shut down gracefully. If the process does not exit within this timeframe, Kubernetes will send a SIGKILL signal to forcibly terminate the container.

SIGKILL Signal

Target: Processes within the container (main process or any other processes)
Behavior: SIGKILL is an uncatchable and unignorable signal. It directly terminates processes in the container without allowing any cleanup operations. When a container fails to exit gracefully during the grace period after receiving SIGTERM, Kubernetes sends SIGKILL to forcefully terminate container processes.
- Process Behavior: Upon receiving SIGKILL, container processes terminate immediately without performing any cleanup. All file handles, network connections, database transactions may be abruptly interrupted, potentially leaving resources not properly released.
Container Termination: With processes forcibly terminated, the container ends its lifecycle. The Pod controller (e.g., Deployment or ReplicaSet) will then reschedule new containers based on replica counts.

Relationship Between Containers and Processes

In Kubernetes, a container essentially serves as a runtime environment encapsulating one or multiple processes (typically a single main process). Discussions about container startup, shutdown, or restart fundamentally refer to managing the lifecycle of processes within the container.
When Kubernetes sends SIGTERM or SIGKILL, these signals are directed to processes within the container rather than the container itself. The container itself has no executable code or processes - it acts merely as a runtime environment, while the containerized processes are the actual entities handling these signals.

How Processes in Containers Respond to SIGTERM and SIGKILL

Response to SIGTERM: Processes within a container can capture the SIGTERM signal. The container’s main process may implement code logic to handle this signal for cleanup operations or resource release.

For example, if the container runs an HTTP server process, it might stop accepting new requests and gracefully complete ongoing requests upon receiving SIGTERM, then exit.

Example code (Python):

  import signal
  import time
  
  def graceful_shutdown(signum, frame):
      print("Received SIGTERM, shutting down gracefully...")
      # Perform cleanup operations, such as closing database connections, flushing caches, etc.
      time.sleep(2)  # Simulate cleanup tasks
      exit(0)
  
  signal.signal(signal.SIGTERM, graceful_shutdown)
  
  print("Running... Press CTRL+C to exit.")
  while True:
      time.sleep(1)

Response to SIGKILL: Processes in containers cannot capture or handle the SIGKILL signal. The SIGKILL signal terminates the process directly, causing the process to exit immediately without any opportunity for cleanup.

When SIGKILL is received, the process is terminated abruptly by the system, leaving no time to save state or release resources. Recovery from this termination is not possible.

Summary

SIGTERM: A graceful termination request allowing processes to clean up resources and exit properly. If the process does not terminate within the specified time limit, the operating system sends SIGKILL.
SIGKILL: A forced termination signal that cannot be intercepted or handled by the process. The process is immediately terminated without any cleanup.

These two signals are commonly used in Unix/Linux system management. Understanding their roles and behaviors helps in effectively managing process lifecycles, particularly in containerized environments (e.g., Kubernetes) and automated operations.

SIGTERM and SIGKILL signals in Kubernetes are ultimately sent to the processes running inside the container, not directly to the container itself. A container is a runtime environment containing one or more processes. Therefore, these signals target the actual processes within the container.

SIGTERM is sent to processes inside the container to allow graceful exit and cleanup. If processes fail to terminate within the grace period (default 30 seconds), Kubernetes follows up with SIGKILL.
SIGKILL (sent to container processes) cannot be caught or ignored. Processes are forcibly terminated immediately, bypassing cleanup routines.

In summary, containers manage internal processes as runtime environments. The lifecycle management of a container depends on how its internal processes respond to these termination signals.

Log Collector Configuration Guide for Logstash and Filebeat

Sat, 09 Mar 2024 00:00:00 +0000

Log Format

`1`	`2024-01-29 16:11:11.189 \|INFO \| 1.1.1.1\|2345 \| com.smart.service.receive.impl.ReceiveServiceImpl:903 \| Capability>Total 04 steps \| 6df2f14fca4b40f6be89b9ef19382c42adasfasf`

Logstash Configuration Example

docker deployment

[root@master logstash]# cat docker-compose.yaml

version: '3'
services:
  logstash:
    image: docker.elastic.co/logstash/logstash:8.12.0
    container_name: logstash
    volumes:
      - ./conf/logstash.yml:/usr/share/logstash/config/logstash.yml
      - ./conf/conf.d:/usr/share/logstash/config/conf.d/
      - ./logs:/opt
    ports:
      - 5044:5044

config files

logstash.yml

1
2
3

http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.hosts: [ "http://192.168.142.106:9200" ]
path.config: /usr/share/logstash/config/conf.d/*.conf

input {
    file {
       type => "info_log"
       path => "/opt/kaikai.log"
       discover_interval => 10 # listen interval
       start_position => "end"
       # sincedb_path => "/usr/share/logstash/sincedb_kaikai"
       #start_position => "beginning"
       codec => multiline {
           pattern => "^%{TIMESTAMP_ISO8601}"
           negate => true
           what => "previous"
       }
    }

    file {
      type => "error_log"
      path => "/opt/error.log"
      discover_interval => 10
      start_position => "beginning"
      codec => multiline {
          pattern => "^%{TIMESTAMP_ISO8601}"
          negate => true
          what => "previous"
      }
    }
}

filter {

    grok {
      match => { "[log][file][path]" => "/(?<logfilename>[^/]+)\.log$" }  # get file name logfilename
    }
    grok {
        match => { "message" => "%{DATA:time}\|%{DATA:level}\|%{DATA:ip}\|%{DATA:pid}\|%{DATA:source}\|%{GREEDYDATA:content}"}
    }
    if "_grokparsefailure" in [tags] {
        mutate {
            add_field => { "content" => "%{message}" }
            add_field => { "level" => "ERROR" }
        }
    }

}

output {
    stdout { codec => rubydebug }
    elasticsearch {
        hosts => ["192.168.142.106:9200"]
        index => "%{logfilename}-%{+YYYY-MM-dd}"   # index by file name
    }

}

k8s deployment

logstash.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: log-file-config
data:
  logstash.yml: |
    http.host: "0.0.0.0"
    xpack.monitoring.elasticsearch.hosts: [ "http://192.168.142.106:9200" ]
    #xpack.monitoring.elasticsearch.hosts: [ "http://192.168.142.106:9200" ]
    path.config: /usr/share/logstash/config/conf.d/*.conf
  collect.conf: |
    input {
        beats {
            port => 5044
        }
    }

    filter {

        grok {
            match => { "[log][file][path]" => ["/(?<logfilename>[^/]+)\.log$"] }
        }
        grok {
            match => { "message" => "%{DATA:time}\|%{DATA:level}\|%{DATA:ip}\|%{DATA:pid}\|%{DATA:source}\|%{GREEDYDATA:content}" }
        }
        if "_grokparsefailure" in [tags] {
            mutate {
                add_field => { "content" => "%{message}" }
                add_field => { "level" => "ERROR" }
            }
        }


    }

    output {
        stdout { codec => rubydebug }
        elasticsearch {
            hosts => ["192.168.142.106:9200"]
            index => "%{logfilename}-%{+YYYY-MM-dd}"
        }


    }
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: logstash
  labels:
    app: logstash
spec:
  replicas: 4
  selector:
    matchLabels:
      app: logstash
  template:
    metadata:
      labels:
        app: logstash
      annotations:
        appName: logstash
        appType: java
    spec:
      containers:

        - name: logstash-logging
          image: registry.cn-beijing.aliyuncs.com/kaikai136/logstash:8.12.0
          volumeMounts:
          - name: logstash-config
            mountPath: /usr/share/logstash/config/logstash.yml
            subPath: logstash.yml
          - name: logstash-config
            mountPath: /usr/share/logstash/config/conf.d/collect.conf
            subPath: collect.conf

      volumes:
        - name: logstash-config
          configMap:
            name: log-file-config
            items:
              - key: logstash.yml
                path: logstash.yml
              - key: collect.conf
                path: collect.conf

      imagePullSecrets:
        - name: my-harbor


---
apiVersion: v1
kind: Service
metadata:
  name: logstash-svc
  labels:
    app: logstash-svc
spec:
  ports:
  - port: 5044
    targetPort: 5044
    protocol: TCP
    name: http
    nodePort: 32467
  type: NodePort
  selector:
    app: logstash

filebeat

collector test

filebeat.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
data:
  filebeat.yml: |
    filebeat.inputs:
      - type: log
        enabled: true
        paths:
          - /logs/*_info.log
        scan_frequency: 1s  # set scan frequency to 1 second
        harvester_buffer_size: 32768  # increase harvester buffer size
        backoff_factor: 2
        ignore_older: 24h  # ignore files older than 24 hours
        close_inactive: 5m  # close harvester inactive for 5 minutes
        clean_inactive: 72h  # clean inactive harvester after 72 hours
        close_removed: true  # close harvester when file is removed
        clean_removed: true  # clean removed harvester
        close_eof: true  # close harvester when file reaches EOF
        multiline.pattern: '^[0-9]{4}'  # match multiline logs
        multiline.negate: true
        multiline.match: after
        var.convert_timezone: true  # convert timezone
        encoding: UTF-8  # set encoding
        fields:
          wisentIp: 0.0.0.0  # add custom field
          log_type: info_log
      - type: log
        enabled: true
        paths:
          - /logs/*_error.log
        scan_frequency: 1s  # set scan frequency to 1 second
        harvester_buffer_size: 32768  # increase harvester buffer size
        backoff_factor: 2
        ignore_older: 24h  # ignore files older than 24 hours
        close_inactive: 5m  # close harvester inactive for 5 minutes
        clean_inactive: 72h  # clean inactive harvester after 72 hours
        close_removed: true  # close harvester when file is removed
        clean_removed: true  # clean removed harvester
        close_eof: true  # close harvester when file reaches EOF
        multiline.pattern: '^[0-9]{4}'  # match multiline logs
        multiline.negate: true
        multiline.match: after
        var.convert_timezone: true  # convert timezone
        encoding: UTF-8  # set encoding
        fields:
          wisentIp: 0.0.0.0  # add custom field
          log_type: error_log



    queue.mem:
      events: 4096  # memory queue size
      flush.min_events: 2048  # minimum flush events
      flush.timeout: 1s  # flush timeout

    #queue.disk:
    #  max_size: 1024mb  # maximum disk usage
    #  segment_size: 10mb  # size of each segment
    #  max_retries: 3  # maximum retries
    logging.level: debug
    filebeat.shutdown_timeout: 30s  # ensure enough time to process current events when shutting down Filebeat
    throttle: 5s  # set the time Filebeat waits before being throttled


    logging.level: info  # set logging level to info for detailed run information
    logging.to_files: true
    logging.files:
      path:  /usr/share/filebeat/logs
      name: filebeat
      keepfiles: 7
      permissions: 0644

    output.logstash:
      hosts: ["logstash-svc.default.svc.cluster.local:5044"]
---

kind: Deployment
apiVersion: apps/v1
metadata:
  name: filebeat
  labels:
    app: filebeat
spec:
  replicas: 1
  selector:
    matchLabels:
      app: filebeat
  template:
    metadata:
      labels:
        app: filebeat
      annotations:
        appName: filebeat
        appType: java
    spec:
      containers:

        - name: filebeat-logging
          image: registry.cn-beijing.aliyuncs.com/kaikai136/filebeat:8.12.0
          volumeMounts:
          - name: filebeat-config
            mountPath: /usr/share/filebeat/filebeat.yml
            subPath: filebeat.yml
          - name: myhostpath
            mountPath: /logs

      volumes:
        - name: filebeat-config
          configMap:
            name: filebeat-config
            items:
              - key: filebeat.yml
                path: filebeat.yml
        - name: myhostpath
          hostPath:
            path: /opt/kaikai/file-logstash/filebeat_log
            type: DirectoryOrCreate


      imagePullSecrets:
        - name: my-harbor

Comprehensive Guide to Linux Firewall Tool iptables: Configuration and Usage

Tue, 03 Oct 2023 00:00:00 +0000

In Linux system security maintenance, the firewall is an important tool for protecting servers from unauthorized access. iptables, as a Linux kernel firewall tool, provides powerful and flexible network traffic control mechanisms. This article will delve into the core concepts, configuration methods, and common implementation scenarios of iptables, helping system administrators build a more secure server environment.

What is iptables?

iptables is a firewall tool in the Linux kernel, directly interacting with the netfilter module, responsible for filtering, modifying, and forwarding network data packets. As a complete firewall framework, iptables provides fine-grained network traffic control capabilities, allowing complex network access policies to be formulated based on multiple conditions (such as source IP address, destination port, protocol type, etc.).

Core Components of iptables

The structure of iptables is based on the concepts of “tables” and “chains”:

Tables：Organize rules with specific functionalities
- filter：Default table, used for packet filtering
- nat：Used for network address translation
- mangle：Used for special packet modifications
- raw：Used for configuring exempted connection tracking
- security：Used for enforcing access control network rules
Chains：Each table contains multiple chains, defining when rules are applied
- INPUT：Processes incoming packets
- OUTPUT：Processes outgoing packets
- FORWARD：Processes forwarded packets
- PREROUTING：Pre-routing processing
- POSTROUTING：Post-routing processing

Basic Operations of iptables

Viewing Current Rules

# View all rules in the filter table
sudo iptables -L -v

# View all rules in the nat table
sudo iptables -t nat -L -v

Adding Rules

# Allow SSH connections (22 port)
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# Allow established connections
sudo iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow local loopback interface
sudo iptables -A INPUT -i lo -j ACCEPT

Deleting Rules

# Delete the first rule
sudo iptables -D INPUT 1

# Delete a specific rule
sudo iptables -D INPUT -p tcp --dport 80 -j ACCEPT

Setting Default Policies

# Set default INPUT policy to reject
sudo iptables -P INPUT DROP

# Set default OUTPUT policy to accept
sudo iptables -P OUTPUT ACCEPT

Common Implementation Scenarios and Configuration Examples

Basic Server Protection Configuration

A basic server protection configuration example that allows common services and defaults to rejecting other connections:

# Clear existing rules
sudo iptables -F
sudo iptables -X

# Set default policies
sudo iptables -P INPUT DROP
sudo iptables -P FORWARD DROP
sudo iptables -P OUTPUT ACCEPT

# Allow local loopback
sudo iptables -A INPUT -i lo -j ACCEPT

# Allow established connections
sudo iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow SSH (22 port)
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT

# Allow HTTP/HTTPS (80/443 ports)
sudo iptables -A INPUT -p tcp --dport 80 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 443 -j ACCEPT

# Allow ICMP (ping)
sudo iptables -A INPUT -p icmp --icmp-type echo-request -j ACCEPT

Port Forwarding Configuration

Forward external 80 port requests to internal 8080 port:

`1`	`sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080`

Limiting Connection Rate

Simple configuration to prevent DoS attacks:

`1`	`sudo iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update --seconds 60 --hitcount 4 -j DROP`

Blocking Specific IP Addresses

1
2

# Block a specific IP
sudo iptables -A INPUT -s 192.168.1.0/24 -j DROP

Rule Persistence

iptables rules are lost after system restart, so persistence configuration is needed:

Debian/Ubuntu System

# Install iptables-persistent
sudo apt-get install iptables-persistent

# Save current rules
sudo netfilter-persistent save

CentOS/RHEL System

1
2

# Save current rules
sudo iptables-save > /etc/sysconfig/iptables

Common Issues and Troubleshooting

Rule Ordering：iptables matches rules in order, stopping at the first match. Therefore, rule ordering is critical.
Locking Risk：Adding DROP rules requires caution, as incorrect configuration can lead to inability to remotely connect to the server. It is recommended to test new rules with physical access when possible.
Performance Considerations：Too many rules can affect network performance, so it is recommended to regularly clean up unnecessary rules.
Logging and Monitoring：Use the LOG target to record rejected connections for troubleshooting:

`1`	`sudo iptables -A INPUT -j LOG --log-prefix "iptables denied: " --log-level 7`

Advanced Features

Network Address Translation (NAT)

Configure simple NAT to share an Internet connection:

# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward

# Allow forwarding
sudo iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT

Traffic Control and QoS

Use iptables’s mangle table to mark traffic, combined with tc (Traffic Control) for QoS:

`1`	`sudo iptables -t mangle -A PREROUTING -p tcp --dport 22 -j MARK --set-mark 1`

Conclusion

iptables is a powerful firewall tool in the Linux kernel, mastering its use is crucial for server security. By properly configuring iptables rules, you can effectively control network traffic, protect servers from unauthorized access, and implement advanced features such as network address translation.

For more complex scenarios, consider using higher-level tools like ufw (Uncomplicated Firewall) or firewalld, which still use iptables but provide a more user-friendly interface.

Regardless of the tool used, understanding the core concepts and working principles of iptables is essential for building a secure Linux network environment.

Consul-based Auto Discovery Mechanism for Prometheus Monitoring

Tue, 26 Sep 2023 00:00:00 +0000

Prometheus Comprehensive Monitoring Platform - Consul-Based Auto Discovery

Background

Consul Documentation | Consul | HashiCorp Developer

The Prometheus configuration file prometheus-config.yaml contains numerous scraping rules, which are mostly manually managed by DevOps teams. When new nodes or components are added, manual modification of this configuration and a hot reload of Prometheus are required. Is it possible to dynamically monitor microservices? Prometheus provides multiple dynamic service discovery mechanisms, and this guide uses Consul as an example.

Consul-Based Auto Discovery

Consul is a distributed key-value database and a service registry component. Other services can register with Consul, including Prometheus. By leveraging Consul’s service discovery, we can avoid manually specifying large numbers of targets in Prometheus.

The workflow for Prometheus’ Consul-based service discovery is as follows:

Register or deregister services (monitoring targets) in Consul.
Prometheus continuously monitors Consul. When changes to services meeting the criteria are detected, Prometheus updates its monitoring targets accordingly.

Service Discovery Mechanisms Supported by Prometheus

Prometheus configuration for data sources falls into two main categories: static configuration and dynamic discovery. Commonly used mechanisms include:

1) static_configs:       # Static service discovery  
2) file_sd_configs:      # File-based service discovery  
3) dns_sd_configs:       # DNS-based service discovery  
4) kubernetes_sd_configs: # Kubernetes service discovery  
5) consul_sd_configs:    # Consul service discovery

In Kubernetes monitoring scenarios, frequently updated resources like Pods and Services best demonstrate the advantages of Prometheus’ auto-discovery capabilities.

Working Principles

Prometheus queries the configuration information stored in Consul’s KV storage through the Consul API, then extracts service metadata from it;
Prometheus uses this information to construct target service URLs and adds them to the service discovery target list;
When services are deregistered or become unavailable, Prometheus will automatically remove them from the target list.

Containerized Consul Cluster

For testing and validation purposes only. Not suitable for production use! Production environments must undergo comprehensive cluster-based deployment validation with service process guardianship and monitoring.

Create a single-node Consul cluster:

# docker run -id -expose=[8300,8301,8302,8500,8600] --restart always -p 18300:8300 -p 18301:8301 -p 18302:8302 -p 18500:8500 -p 18600:8600 --name server1 -e 'CONSUL_LOCAL_CONFIG={"skip_leave_on_interrupt": true}' consul agent -server -bootstrap-expect=1 -node=server1 -bind=0.0.0.0 -client=0.0.0.0 -ui -datacenter dc1

Parameter Explanations:

    -expose: Exposes Consul's required ports: 8300, 8301, 8302, 8500, 8600
    --restart: always ensures automatic container restart upon failure
    -p: Establishes host-container port mappings
    --name: Container name
    -e: Environment variable for Consul configuration
    consul: Refers to the Consul image name, not the command
    agent: Command executed in the container. Parameter details:
    -server: Designates the node as a server type
    -bootstrap-expect: Specifies the number of server nodes required to trigger leader election (1 for single-node)
    -node: Node name
    -bind: Internal cluster communication address (default 0.0.0.0)
    -client: Client interface address (default 127.0.0.1)
    -ui: Enables Consul web UI
    -datacenter: Data center name

Validation test:

Access the web UI at, for example: http://192.10.192.109:18500/

`1`	`# curl localhost:18500`

Register Host to Consul

Example: Register node-exporter on a virtual machine to consul.

## Format
$ curl -X PUT -d '{"id": "'${host_name}'","name": "node-exporter","address": "'${host_addr}'","port":9100,"tags": ["dam"],"checks": [{"http": "http://'${host_addr}':9100/","interval": "5s"}]}' http://192.10.192.109:18500/v1/agent/service/register

## Example
$ curl -X PUT -d '{"id": "sh-middler2","name": "node-exporter","address": "192.10.192.134","port":9100,"tags": ["middleware"],"checks": [{"http": "http://192.10.192.134:9100/metrics","interval": "3s"}]}' http://192.10.192.109:18500/v1/agent/service/register

Parameter Explanation

id : Registration ID (must be unique in consul)
name : Service name
address: Binding IP for auto-registration
port: Binding port for auto-registration
tags: Registration tags (multiple allowed)
checks : Health checks
http:  Data source for check
interval: Check interval
http://192.10.192.109:18500/v1/agent/service/register : Consul registration API endpoint

Delete:

## Format
$ curl -X PUT http://192.10.192.109:18500/v1/agent/service/deregister/${id}

## Example
$ curl -X PUT http://192.10.192.109:18500/v1/agent/service/deregister/sh-middler2

Configuring Prometheus with Consul for Automatic Service Discovery

Modify Prometheus’ ConfigMap configuration file: prometheus-config.yaml

    - job_name: consul
      honor_labels: true
      metrics_path: /metrics
      scheme: http
      consul_sd_configs:    # Configuration based on Consul service discovery
        - server: 192.10.192.109:18500    # Consul listening address
          services: []                 # Match all services in Consul
      relabel_configs:             # Relabel configuration settings
      - source_labels: ['__meta_consul_tags']    # Assign __meta_consul_tags value to product
        target_label: 'servername'
      - source_labels: ['__meta_consul_dc']   # Assign __meta_consul_dc value to idc
        target_label: 'idc'
      - source_labels: ['__meta_consul_service']   
        regex: "consul"  # Match services named "consul"
        action: drop       # Drop/remove matching entries

Reload Prometheus using the above method. Open the Prometheus Target page to see the defined mysql-exporter job:

`1`	`curl -XPOST http://prometheus.kubernets.cn/-/reload`

Summary

Dynamic Service Discovery and Monitoring: By integrating with Consul, Prometheus dynamically maintains its target list, ensuring prompt discovery and monitoring as new services are deployed.
Scalability: Automated service discovery simplifies infrastructure scaling while preserving reliable monitoring availability and performance.
Seamless Integration: As the service registry, Consul enables Prometheus to integrate seamlessly with other tools in the Consul ecosystem, delivering a comprehensive solution for service infrastructure monitoring and management.
Self-Healing Capability: Automatic service discovery allows Prometheus to detect infrastructure changes in real time, continuously updating its target list to ensure uninterrupted monitoring data and high performance.

Guide to Automatically Configure SSL Certificates with ACME on DNSPod

Mon, 18 Sep 2023 00:00:00 +0000

Deploy acme.sh with Docker

docker-compose.yml, replace the actual values of DNSPod API ID and Key.

services:
  acme-sh:
    image: neilpang/acme.sh:latest
    container_name: acme
    restart: always
    command: daemon
    environment:
      - TZ=Asia/Shanghai
      - DP_Id=******            # DNSPod API ID
      - DP_Key=**************** # DNSPod API Key
    volumes:
      - ./certs/:/acme.sh/

Configure acme.sh

1
2
3

docker compose up -d
docker compose exec -it acme sh
acme.sh --set-default-ca --server letsencrypt

Apply for a certificate

Generate a certificate, replace www.example.com with the actual domain name. The certificate will be mounted to the ./certs/ directory.

`1`	`acme.sh --issue --dns dns_dp -d www.example.com`

Complete Guide to FRP Reverse Proxy Deployment

Sun, 09 Jul 2023 00:00:00 +0000

What is frp?

frp is a high-performance reverse proxy application specialized in intranet penetration. It supports multiple protocols including TCP, UDP, HTTP, HTTPS, and features P2P communication capabilities. Using frp, you can securely and conveniently expose internal network services to the public network through relay nodes with public IP addresses.

Why choose frp?

By deploying an frp server on a node with a public IP, you can easily penetrate internal services to the public network while enjoying these professional features:

Multi-protocol Support: Client-server communication supports TCP, QUIC, KCP, Websocket, and other protocols.
TCP Connection Stream Multiplexing: Carry multiple requests over a single connection to reduce connection setup time and lower request latency.
Load Balancing between proxy groups.
Port Multiplexing: Multiple services can be exposed through the same server port.
P2P Communication: Traffic bypasses server relay, maximizing bandwidth utilization.
Client Plugins: Provides natively supported client plugins for static file viewing, HTTPS/HTTP protocol conversion, HTTP/SOCKS5 proxies, and more.
Server Plugin System: Highly extensible plugin system for custom functional expansion.
User-Friendly UI: Server and client interfaces for simplified configuration and monitoring.

frps

FRPS is the server part of frp, which needs a public IP and opens port 12345.

docker-compose.yaml

  services:
    frps:
      image: fatedier/frps:v0.61.0
      container_name: frps
      restart: always
      working_dir: /etc/frps/
      command: ["-c", "./config.toml"]
      volumes:
        - /etc/localtime:/etc/localtime:ro
        - ./:/etc/frps/
      network_mode: host

config.toml

  bindAddr = "0.0.0.0"
  bindPort = 12345
  
  vhostHTTPPort = 12380
  vhostHTTPSPort = 12343
  
  webServer.addr = "0.0.0.0"
  webServer.port = 12375
  webServer.user = "admin"
  webServer.password = "******"
  
  enablePrometheus = true
  
  log.to = "./logs/frps.log"
  log.level = "debug"
  log.maxDays = 7
  log.disablePrintColor = false
  
  auth.method = "token"
  auth.token = "******"
  
  custom404Page = "./404.html"

404.html

  <!DOCTYPE html>
  <html lang="en">
  <head>
      <meta charset="UTF-8">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>Not Found</title>
      <style>
          body {
              font-family: Arial, sans-serif;
              text-align: center;
              margin-top: 100px;
          }
          h1 {
              font-size: 50px;
              margin-bottom: 10px;
          }
          p {
              font-size: 20px;
              margin-top: 0;
          }
      </style>
  </head>
  <body>
  
      <h1>404</h1>
      <p>Page not found</p>
      
  </body>
  </html>

frpc

docker deployment

docker-compose.yaml

  services:
    net-tunnel:
      image: "harbor.example.com/devops/kube-net-tunnel:standalone"
      container_name: "net-tunnel"
      restart: "always"
      network_mode: "bridge"
      working_dir: /etc/net-tunnel/
      volumes:
        - "/etc/localtime:/etc/localtime:ro"
        - "./:/etc/net-tunnel/"
      command: ["-c", "/etc/net-tunnel/config.toml"]

config.toml

  user = "test"
  
  serverAddr = "*.*.*.*"
  serverPort = 12345
  
  auth.method = "token"
  auth.token = "******"
  
  log.to = "./logs/frpc.log"
  log.level = "debug"
  log.maxDays = 3
  log.disablePrintColor = false
  
  webServer.addr = "127.0.0.1"
  webServer.port = 12346
  webServer.user = "admin"
  webServer.password = "******"
  
  [[proxies]]
  name = "admin_ui"
  type = "tcp"
  localIP = "127.0.0.1"
  localPort = 12346
  remotePort = 12440
  
  [[proxies]]
  name = "ssh"
  type = "tcp"
  localIP = "172.17.0.1" # Windows or MacOS: "host.docker.internal"
  localPort = 22
  remotePort = 12441

K8s deployment

kube-net-tunnel.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: kube-net-tunnel-config
  namespace: kube-system
data:
  config.toml: |
    user="test"
    serverAddr = "*.*.*.*"
    serverPort = 12345

    auth.method = "token"
    auth.token = "******"

    log.to = "console"
    log.level = "debug"
    log.maxDays = 3
    log.disablePrintColor = false

    webServer.addr = "127.0.0.1"
    webServer.port = 7400
    webServer.user = "admin"
    webServer.password = "******"

    [[proxies]]
    name = "admin"
    type = "tcp"
    localIP = "127.0.0.1"
    localPort = 7400
    remotePort = 12431

    [[proxies]]
    name = "ssh"
    type = "tcp"
    localIP = "10.19.31.7"
    localPort = 22
    remotePort = 12432


---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kube-net-tunnel
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kube-net-tunnel
  template:
    metadata:
      labels:
        app: kube-net-tunnel
    spec:
      containers:
        - name: kube-net-tunnel
          image: harbor.example.com/devops/kube-net-tunnel:standalone
          imagePullPolicy: IfNotPresent
          args: ["-c", "/etc/kube-net-tunnel/config.toml"]
          volumeMounts:
            - name: config-volume
              mountPath: /etc/kube-net-tunnel/config.toml
              subPath: config.toml
      volumes:
        - name: config-volume
          configMap:
            defaultMode: 0755
            name: kube-net-tunnel-config
            items:
              - key: config.toml
                path: config.toml

Comprehensive Guide to FastDFS Cluster Deployment

Tue, 04 Jul 2023 00:00:00 +0000

Reference

https://www.cnblogs.com/lincappu/p/14926757.html

FastDFS

The author of FastDFS, Yu Qing, described it on GitHub as follows: “FastDFS is an open source high performance distributed file system. It’s major functions include: file storing, file syncing and file accessing (file uploading and file downloading), and it can resolve the high capacity and load balancing problem. FastDFS should meet the requirement of the website whose service based on files such as photo sharing site and video sharing site.” This means FastDFS is an open-source high-performance distributed file system. Its core functionalities include file storage, file synchronization, and file access (uploading and downloading), addressing challenges of large-scale storage and load balancing. It is designed to meet the needs of file-service-oriented websites like photo or video sharing platforms.

FastDFS has two roles: Tracker and Storage. The Tracker is responsible for scheduling file access requests and load balancing. Storage manages file operations, including storage, synchronization, and providing file access interfaces. It also handles metadata, represented as key-value attribute pairs associated with files. Both Tracker and Storage nodes can be composed of one or multiple servers. Servers can be added or removed without disrupting services, though at least one server in each cluster must remain operational. Notably, all servers in a Tracker cluster operate in a peer-to-peer (P2P) manner, allowing dynamic scaling based on server workload.

Additionally, the official documentation elaborates on the storage architecture. To support massive capacity, the Storage nodes adopt a volume (or group) organizing approach. The storage system consists of one or more independent volumes, where the total system capacity equals the sum of all volumes. Each volume can be hosted by one or multiple Storage servers. All servers within the same volume store identical files, serving purposes of redundant backup and load balancing. Adding servers to a volume…

When adding a new server, the system automatically performs file synchronization. Once synchronization is complete, the system automatically switches the new server online to provide services. When storage space is insufficient or near depletion, volumes can be dynamically expanded. Simply add one or more servers and configure them as new volumes to increase the storage system’s capacity. We will not delve too deeply into the concepts of volumes or groups here, as there will be detailed explanations in the subsequent installation and deployment.

In FastDFS, the file identifier consists of two parts: the volume name and the file name.

Environment Specifications

Operating System: CentOS Linux release 7.2.1511
System Disk: 274GB
Mounted Disks: 3.7TB * 12
CPU: 32 cores (Intel® Xeon®)
Memory: 8GB

Architecture Design

Workflow:

The client sends a request to the Tracker.
The Tracker retrieves metadata from Storage nodes and returns it to the client.
The client uses the metadata to directly request files from Storage nodes.

Key Design Principles

The core system contains two roles: Tracker Server and Storage Server.
All Tracker servers are peer-to-peer (P2P) with no Master-Slave relationships.
Storage servers are organized into groups; files are fully replicated within the same group.
Storage servers across different groups do not communicate. Synchronization occurs only within the same group.
Storage servers proactively report status to Trackers. Each Tracker maintains complete Storage server status records.
When the Trunk feature is enabled, Trackers coordinate with Storages to elect a Trunk-Server.

Cluster Deployment

Table 1 Software List and Versions

Name	Description	Link
CentOS	7.x (Installation OS)
libfastcommon	Utility function package for FastDFS	libfastcommon V1.0.39
FastDFS	FastDFS Main Program	FastDFS V5.11
fastdfs-nginx-module	FastDFS-Nginx integration module (resolves intra-group sync delay)	fastdfs-nginx-module V1.22
nginx	nginx 1.12.2 (Latest version via YUM for CentOS 7)	nginx 1.17.4

Table 2 Server IPs, Service Allocation, and Port Planning

Name	IP Address	Application Service	Port
Machine A
	10.58.10.136	tracker	22122
	10.58.10.136	storage-group1	23000
	10.58.10.136	storage-group2	33000
	10.58.10.136	libfastcommon	-
	10.58.10.136	nginx	8888
	10.58.10.136	fastdfs-nginx-module	-
Machine B
	10.58.10.137	tracker	22122

| | 10.58.10.137 | storage-group1 | 23000 | | | 10.58.10.137 | storage-group3 | 43000 | | | 10.58.10.137 | libfastcommon | - | | | 10.58.10.137 | nginx | 8888 | | | 10.58.10.137 | fastdfs-nginx-module | - | |Machine C| | | | | | 10.58.10.138 | tracker | 22122 | | | 10.58.10.138 | storage-group2 | 33000 | | | 10.58.10.138 | storage-group23 | 43000 | | | 10.58.10.138 | libfastcommon | - | | | 10.58.10.138 | nginx | 8888 | | | 10.58.10.138 | fastdfs-nginx-module | - |

Prerequisites before installation:

Ensure to grant read and write permissions to the storage directories (logs, data, PID files, etc.) that will be used.
All configuration files in the following sections contain comments marked with “#” for explanations. Be sure to remove these comment lines that start with “#” when implementing the configurations.

Initialize Environment

1
2

# Install build environment
yum groups install Development Tools perl redhat-rpm-config.noarch gd-devel perl-devel perl-ExtUtils-Embed pcre-devel openssl openssl-devel gcc-c++ autoconf automake zlib-devel libxml2 libxml2-dev libxslt-devel GeoIP GeoIP-devel GeoIP-data gperftools-y

Install libfastcommon

Execute the following operations on machines A, B, and C respectively:

tar -zxvf libfastcommon-1.0.39.tar.gz
cd libfastcommon-1.0.39/
./make.sh
./make.sh install

libfastcommon is installed to /usr/lib64/libfastcommon.so. Note the difference between new and old versions:

New versions will automatically create a symlink for libfastcommon.so to the /usr/local/lib directory.
Old versions require manual symlink creation:

1
2

ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so

If libfdfsclient.so exists, also add it to /usr/local/lib:

1
2

ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so
ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so

!!! Note It’s recommended to manually verify successful symlink creation using:

`1`	`ls \| grep libfastcommon`

Check in both `/usr/lib/` and `/usr/local/lib` directories.

Install Tracker

Perform the following operations on machines A, B, and C respectively

mkdir -p /data/fastdfs/tracker
tar -zxvf fastdfs-5.11.tar.gz
cd fastdfs-5.11/
./make.sh
./make.sh install
# Prepare configuration files
cp /etc/fdfs/tracker.conf.sample /etc/fdfs/tracker.conf # For tracker node

Modify tracker configuration file:

vim /etc/fdfs/tracker.conf
# Required modifications:
max_connections=1024 # default 256, maximum number of connections
port=22122 # tracker server port (default 22122, usually unchanged)
base_path=/data/fastdfs/tracker # root directory for storing logs and data

Add tracker.service to enable service management with systemctl (start/restart/stop) and auto-start on boot:

# Edit service file
vim /usr/lib/systemd/system/fastdfs-tracker.service
[Unit]
Description=The FastDFS File server
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
ExecStart=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf start
ExecStop=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf stop
ExecRestart=/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart

[Install]
WantedBy=multi-user.target

After saving the /usr/lib/systemd/system/fastdfs-tracker.service file and exiting vim, execute the following commands to start the FastDFS Tracker service:

1
2
3

$ systemctl daemon-reload  
$ systemctl enable fastdfs-tracker.service  
$ systemctl start fastdfs-tracker.service

After the tracker service starts, use the following command to verify if the port is properly opened:

`1`	`netstat -tulnp \| grep 22122 # Check if the service is running and the port is open`

Install Storage

Skip the extraction step if already performed during tracker installation.

$ tar -zxvf fastdfs-5.11.tar.gz  
$ cd fastdfs-5.11/  
$ ./make.sh  
$ ./make.sh install

Machine A (group1/group2)

Copy storage configuration files under fastdfs-5.11 directory (make two copies)

sudo mkdir -p /data/fastdfs/storage/group1
sudo mkdir -p /data/fastdfs/storage/group2
sudo cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage-group1.conf #storage node group1
sudo cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage-group2.conf #storage node group2
sudo cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf #client config for testing

According to architecture design, modify the three files sequentially:

Modify group1 configuration file: /etc/fdfs/storage-group1.conf

$ sudo vim /etc/fdfs/storage-group2.conf
# The following content needs to be modified:
group_name=group2
port=33000 # Storage service port (default 23000, modified to 33000)
base_path=/data/fastdfs/storage/group2 # Root directory for data and log file storage
store_path_count=6
store_path0=/data07/fastdfs # First storage directory for group2
store_path1=/data08/fastdfs # Second storage directory for group2
store_path2=/data09/fastdfs # Third storage directory for group2
store_path3=/data10/fastdfs # Fourth storage directory for group2
store_path4=/data11/fastdfs # Fifth storage directory for group2
store_path5=/data12/fastdfs # Sixth storage directory for group2
tracker_server=10.58.10.136:22122 # Tracker server IP and port
tracker_server=10.58.10.137:22122 # Tracker server IP and port
tracker_server=10.58.10.138:22122 # Tracker server IP and port
http.server_port=8888 # HTTP access port for files (default 8888, modify as needed to match Nginx configuration)

Modify client configuration file: /etc/fdfs/client.conf

$ sudo vim /etc/fdfs/client.conf
# The following content needs to be modified:
base_path=/data/fastdfs/client
tracker_server=10.58.10.136:22122 # Tracker server IP and port
tracker_server=10.58.10.137:22122 # Tracker server IP and port
tracker_server=10.58.10.138:22122 # Tracker server IP and port

After setting up the Storage service, we start the two services for Storage:

1
2
3

$ systemctl daemon-reload
$ systemctl enable fastdfs-storage-group1.service
$ systemctl start fastdfs-storage-group1.service

Possible issues during startup:

The service might fail to start due to permission issues or configuration errors.
Check service status using:
systemctl status fastdfs-storage-group1.service
Analyze logs (located at /data/fastdfs/storage/group1/logs/) to troubleshoot.
Refer to the “Troubleshooting” section for common pitfalls.

Verify service activation:

`1`	`netstat -tulnp # Check if services are running and ports are open (23000, 33000)`

Check FastDFS cluster status after successful startup:

1
2

# View cluster status
$ fdfs_monitor /etc/fdfs/storage-group1.conf list

??? note “Info”
[Cluster status details will be shown here]

The console printed the following information, indicating success: [2018-11-06 00:00:00] DEBUG - base_path=/data/fastdfs/storage/group1, connect_timeout=30, network_timeout=60, tracker_server_count=2, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0 server_count=3, server_index=0 tracker server is 10.58.10.136:22122,10.58.10.137:22122,10.58.10.138:22122 group count: 2 Group 1: …

Test file upload via client

`1`	`$ fdfs_upload_file /etc/fdfs/client.conf test.txt`

Machine B (group1/group3)

The configuration process is similar to Machine A, with the following modifications:

Create directories and copy configuration files

$ sudo mkdir -p /data/fastdfs/storage/group1  
$ sudo mkdir -p /data/fastdfs/storage/group3  
$ sudo cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage-group1.conf  # Storage node configuration for group1  
$ sudo cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage-group3.conf  # Storage node configuration for group3  
$ sudo cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf          # Client configuration for testing

Modify the configuration file for group3 (/etc/fdfs/storage-group3.conf). The configuration for group1 remains consistent with Machine A.

$ sudo vim /etc/fdfs/storage-group3.conf
# Required modifications are as follows:
group_name=group3
port=43000 # Storage service port (default: 23000, modified to 43000)
base_path=/data/fastdfs/storage/group3 # Root directory for data and log files
store_path_count=6
store_path0=/data07/fastdfs # First storage directory for group3
store_path1=/data08/fastdfs # Second storage directory for group3
store_path2=/data09/fastdfs # Third storage directory for group3
store_path3=/data10/fastdfs # Fourth storage directory for group3
store_path4=/data11/fastdfs # Fifth storage directory for group3
store_path5=/data12/fastdfs # Sixth storage directory for group3
tracker_server=10.58.10.136:22122 # Tracker server IP and port
tracker_server=10.58.10.137:22122 # Tracker server IP and port
tracker_server=10.58.10.138:22122 # Tracker server IP and port
http.server_port=8888 # HTTP file access port (default: 8888, keep consistent with Nginx)

The client configuration remains consistent with Machine A, no duplication needed here.
Create a service for starting group3. The configuration for fastdfs-storage-group1.service is identical to Machine A - simply copy it.
Execute the startup script until both fastdfs-storage-group1.service and fastdfs-storage-group3.service are running.

Configuration for Machine C (Group2/Group3)

The configuration process is similar to Machine A, with the following modifications:

Create directories and copy configuration files:

$ sudo mkdir -p /data/fastdfs/storage/group2
$ sudo mkdir -p /data/fastdfs/storage/group3
$ sudo cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage-group2.conf  # Storage node for group2
$ sudo cp /etc/fdfs/storage.conf.sample /etc/fdfs/storage-group3.conf  # Storage node for group3
$ sudo cp /etc/fdfs/client.conf.sample /etc/fdfs/client.conf  # Client configuration file (for testing)

Modify the configuration file for Group2 (/etc/fdfs/storage-group2.conf). The configuration content for Group3 should remain consistent with Machine B’s settings.

Configuration file for FastDFS Storage group2

$ sudo vim /etc/fdfs/storage-group2.conf

Modified configurations:

group_name=group2 port=33000 # Storage service port (default 23000, modified to 33000) base_path=/data/fastdfs/storage/group2 # Root directory for data and log storage store_path_count=6 store_path0=/data01/fastdfs # First storage directory for group2 store_path1=/data02/fastdfs # Second storage directory for group2 store_path2=/data03/fastdfs # Third storage directory for group2 store_path3=/data04/fastdfs # Fourth storage directory for group2 store_path4=/data05/fastdfs # Fifth storage directory for group2 store_path5=/data06/fastdfs # Sixth storage directory for group2 tracker_server=10.58.10.136:22122 # Tracker server IP and port tracker_server=10.58.10.137:22122 # Tracker server IP and port tracker_server=10.58.10.138:22122 # Tracker server IP and port http.server_port=8888 # HTTP file access port (default 8888, should match nginx configuration)

The client configuration file is consistent with Machine A, no repetition needed here.
Create startup services for group2 and group3 (existing templates can be copied directly).
Execute startup scripts until both fastdfs-storage-group2.service and fastdfs-storage-group3.services are running.

Installing Nginx and the FastDFS Nginx Module

Step 1: In the FastDFS directory, copy the http.conf and mime.types files to the /etc/fdfs directory to enable Nginx access to the Storage service.

1
2
3

# Execute on all three machines
$ cp ./conf/http.conf /etc/fdfs/ # For Nginx access
$ cp ./conf/mime.types /etc/fdfs/ # For Nginx access

Step 2: Install the FastDFS Nginx module:

1
2
3

# Execute on all three machines
$ tar -zxvf V1.20.tar.gz
$ cp fastdfs-nginx-module-1.20/src/mod_fastdfs.conf /etc/fdfs/mod_fastdfs.conf

Step 3: Modify the configuration file in fastdfs-nginx-module-1.20/src/config. Locate the ngx_module_incs and CORE_INCS entries and modify them as follows:

1
2

ngx_module_incs="/usr/include/fastdfs /usr/include/fastcommon/"
CORE_INCS="$CORE_INCS /usr/include/fastdfs /usr/include/fastcommon/"

If not modified, the following error will occur during Nginx compilation:
/usr/include/fastdfs/fdfs_define.h:15:27: fatal error: common_define.h: No such file or directory

Step 4: Then extract and install the Nginx service:

$ tar -zxvf nginx-1.12.2.tar.gz $ cd nginx-1.12.2 $ ./configure –prefix=/usr/share/nginx –sbin-path=/usr/sbin/nginx –modules-path=/usr/lib64/nginx/modules –conf-path=/etc/nginx/nginx.conf –error-log-path=/var/log/nginx/error.log –http-log-path=/var/log/nginx/access.log –http-client-body-temp-path=/var/lib/nginx/tmp/client_body –http-proxy-temp-path=/var/lib/nginx/tmp/proxy –http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi –http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi –http-scgi-temp-path=/var/lib/nginx/tmp/scgi –pid-path=/run/nginx.pid –lock-path=/run/lock/subsys/nginx –user=nginx –group=nginx –with-file-aio –with-ipv6 –with-http_auth_request_module –with-http_ssl_module –with-http_v2_module –with-http_realip_module –with-http_addition_module –with-http_xslt_module=dynamic –with-http_image_filter_module=dynamic –with-http_geoip_module=dynamic –with-http_sub_module –with-http_dav_module –with-http_flv_module –with-http_mp4_module –with-http_gunzip_module –with-http_gzip_static_module –with-http_random_index_module –with-http_secure_link_module –with-http_degradation_module –with-http_slice_module –with-http_stub_status_module –with-http_perl_module=dynamic –with-mail=dynamic –with-mail_ssl_module –with-pcre –with-pcre-jit –with-stream=dynamic –with-stream_ssl_module –with-google_perftools_module –with-debug –with-cc-opt=’-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong –param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic’ –with-ld-opt=’-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E’ –add-module=${YOUR_PATH}/fastdfs-nginx-module-1.20/src $ make $ make install

Note: Replace ${YOUR_PATH} in the above with the parent directory of fastdfs-nginx-module-1.20 to ensure the path is correct.

Modify configuration file /etc/fdfs/mod_fastdfs.conf on Machine A:

connect_timeout=2
network_timeout=30
base_path=/data/fastdfs/ngx_mod
load_fdfs_parameters_from_tracker=true
storage_sync_file_max_delay = 86400
use_storage_id = false
storage_ids_filename = storage_ids.conf
tracker_server=10.58.10.136:22122 # Tracker server IP and port
tracker_server=10.58.10.137:22122 # Tracker server IP and port
tracker_server=10.58.10.138:22122 # Tracker server IP and port
group_name=group1/group2 # Global setting
url_have_group_name = true
log_level=info
log_filename=
response_mode=proxy
if_alias_prefix=
flv_support = true
flv_extension = flv
group_count = 2

[group1]
group_name=group1 # Group-specific
storage_server_port=23000
store_path_count=6
store_path0=/data01/fastdfs
store_path1=/data02/fastdfs
store_path2=/data03/fastdfs
store_path3=/data04/fastdfs
store_path4=/data05/fastdfs
store_path5=/data06/fastdfs

[group2]
group_name=group2
storage_server_port=33000
store_path_count=6
store_path0=/data07/fastdfs
store_path1=/data08/fastdfs
store_path2=/data09/fastdfs
store_path3=/data10/fastdfs
store_path4=/data11/fastdfs
store_path5=/data12/fastdfs

user nginx; worker_processes on; worker_rlimit_nofile 65535;

error_log /var/log/nginx/error.log; pid /run/nginx.pid;

include /usr/share/nginx/modules/*.conf;

events { worker_connections 65535; use epoll; accept_mutex off; }

http { log_format main ‘$remote_addr - $remote_user [$time_local] “$request” ’ ‘$status $body_bytes_sent “$http_referer” ’ ‘"$http_user_agent" “$http_x_forwarded_for”’;

access_log  /var/log/nginx/access.log  main;

sendfile            on;
tcp_nopush          on;
tcp_nodelay         on;
keepalive_timeout   65;
types_hash_max_size 2048;
gzip                on;

server_names_hash_bucket_size 128;
client_header_buffer_size 32k;
larger_client_header_buffers 4 32k;
client_max_body_size 300m;

proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Connection '';
proxy_set_header REMOTE-HOST $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 90;
proxy_send_timeout   90;
proxy_read_timeout   90;
proxy_buffer_size 16k;
proxy_buffers 8 64k;
proxy_busy_buffers_size 128k;
proxy_temp_file_write_size 128k;

proxy_cache_path /data/fastdfs/cache/nginx/proxy_cache levels=1:2
keys_zone=http-cache:200m max_size=1g inactive=30d;
proxy_temp_path /data/fastdfs/cache/nginx/proxy_cache/temp;

include             /etc/nginx/mime.types;
default_type        application/octet-stream;

include /etc/nginx/conf.d/*.conf;

server {
    listen       80 default_server;
    listen       [::]:80 default_server;
    server_name  _;
    root         /usr/share/nginx/html;

    # Load default server block configuration
    #include /etc/nginx/default.d/*.conf;

    location ~ ^/ok(\..*)?$ {
        return 200 "OK";
    }

    location /nginx {
        stub_status on;
    }

    location /healthcheck {
        check_status on;
    }

    location ^~ /group1/ {
        proxy_next_upstream http_502 http_504 error timeout invalid_header;
        proxy_cache http-cache;
        proxy_cache_valid 200 304 12h;
        proxy_cache_key $uri$is_args$args;

        add_header 'Access-Control-Allow-Origin' $http_origin;
        add_header 'Access-Control-Allow-Credentials' 'true';
        add_header "Access-Control-Allow-Methods" "GET, POST, HEAD, PUT, DELETE, OPTIONS, PATCH";
        add_header "Access-Control-Allow-Headers" "Origin, No-Cache, Authorization, X-Requested-With, If-Modified-Since, Pragma, Last-Modified, Cache-Control, Expires, Content-Type";
        if ($request_method = 'OPTIONS') {
            return 200 'OK';
        }

        proxy_pass http://fdfs_group1;
        expires 30d;
    }
    location ^~ /group2/ {
        proxy_next_upstream http_502 http_504 error timeout invalid_header;
        proxy_cache http-cache;
        proxy_cache_valid 200 304 12h;
        proxy_cache_key $uri$is_args$args;

        add_header 'Access-Control-Allow-Origin' $http_origin;
        add_header 'Access-Control-Allow-Credentials' 'true';
        add_header "Access-Control-Allow-Methods" "GET, POST, HEAD, PUT, DELETE, OPTIONS, PATCH";
        add_header "Access-Control-Allow-Headers" "Origin, No-Cache, Authorization, X-Requested-With, If-Modified-Since, Pragma, Last-Modified, Cache-Control, Expires, Content-Type";
        if ($request_method = 'OPTIONS') {
            return 200 'OK';
        }

        proxy_pass http://fdfs_group2;
        expires 30d;
    }
    location ^~ /group3/ {
        proxy_next_upstream http_502 http_504 error timeout invalid_header;
        proxy_cache http-cache;
        proxy_cache_valid 200 304 12h;
        proxy_cache_key $uri$is_args$args;

        add_header 'Access-Control-Allow-Origin' $http_origin;
        add_header 'Access-Control-Allow-Credentials' 'true';
        add_header "Access-Control-Allow-Methods" "GET, POST, HEAD, PUT, DELETE, OPTIONS, PATCH";
        add_header "Access-Control-Allow-Headers" "Origin, No-Cache, Authorization, X-Requested-With, If-Modified-Since, Pragma, Last-Modified, Cache-Control, Expires, Content-Type";
        if ($request_method = 'OPTIONS') {
            return 200 'OK';
        }

        proxy_pass http://fdfs_group3;
        expires 30d;
    }

    location ~/purge(/.*) {
        allow 127.0.0.1;
        allow 192.168.1.0/24;
        allow 10.58.1.0/24;
        deny all;
        proxy_cache_purge http-cache $1$is_args$args;
    }
}

server {
    listen 8888;
    server_name localhost;
    
    location /ok.htm {
        return 200 "OK";
    }

    location ~/group[0-9]/ {
        ngx_fastdfs_module;
    }

    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
        root html;
    }
}

upstream fdfs_group1 {
    server 10.58.10.136:8888 max_fails=0;
    server 10.58.10.137:8888 max_fails=0;

    keepalive 10240;
    check interval=2000 rise=2 fall=3 timeout=1000 type=http default_down=false;
    check_http_send "GET /ok.htm HTTP/1.0\r\nConnection:keep-alive\r\n\r\n";
    check_keepalive_requests 100;
}

upstream fdfs_group2 {
    server 10.58.10.136:8888 max_fails=0;
    server 10.58.10.138:8888 max_fails=0;

    keepalive 10240;
    check interval=2000 rise=2 fall=3 timeout=1000 type=http default_down=false;
    check_http_send "GET /ok.htm HTTP/1.0\r\nConnection:keep-alive\r\n\r\n";
    check_keepalive_requests 100;
}

upstream fdfs_group3 {
    server 10.58.10.137:8888 max_fails=0;
    server 10.58.10.138:8888 max_fails=0;

    keepalive 10240;
    check interval=2000 rise=2 fall=3 timeout=1000 type=http default_down=false;
    check_http_send "GET /ok.htm HTTP/1.0\r\nConnection:keep-alive\r\n\r\n";
    check_keepalive_requests 100;
}

}

Start the nginx service:

sudo nginx -c /etc/nginx/nginx.conf

Access http://localhost/ok.htm to verify if it returns a 200 status code with “OK” content. If startup fails, check the nginx logs located at /var/log/nginx/error.log. This is the default error log path for nginx in CentOS. If the error_log configuration has been modified, check the log file specified in the error_log directive.

The configurations for servers B and C are essentially identical, with completely consistent nginx configuration files. The only required modification is in the /etc/fdfs/mod_fastdfs.conf file: adjust the group_name according to the corresponding storage node configuration. Note that the [groupX] identifiers in square brackets must follow sequential increments starting from 1.

Issue Summary

When encountering issues, first, we troubleshoot service startup failures through console log messages; second, investigate via service logs. In most cases, logs are the most effective troubleshooting tool, bar none.

Logs for tracker or storage services are stored under the logs directory within the path specified by the base_path configuration in their respective service configuration files.
For nginx logs, if a custom log path is configured, check the specified directory. By default on CentOS, logs are typically located under /var/log/nginx/.
Additionally, in this environment, we must also investigate logs for the fastdfs-nginx-module extension. Its logs reside in the directory specified by the base_path configuration in /etc/fdfs/mod_fastdfs.conf.

Port Configuration for Multiple Groups in a Cluster

Each group requires a dedicated storage service. Thus, port numbers for storage groups on the same host must not conflict. For example:
- Group1: Port 23000
- Group2: Port 33000
- Group3: Port 43000
Groups with the same name cannot exist on the same machine because the tracker synchronizes storage services within the same group, which requires identical port numbers. Therefore, groups with the same name across different hosts must use the same storage port configuration.

Storage Path Configuration for Multiple Groups in a Cluster

On the same host, storage paths for different groups must be configured separately and should not overlap.
Across different hosts, storage paths for groups with the same name may differ, but the number of storage nodes and disk capacity should remain consistent.

Configuring Multiple Storage Services on the Same Node

In a 3-node cluster, two groups were deployed on the same machine, with each group maintaining a replica on another node. When starting the services using systemctl start fastdfs-storage-groupx.service (where groupx represents group1, group2, or group3), one group repeatedly failed to start, displaying:

1
2
3

Loaded: loaded (/usr/lib/systemd/system/fastdfs-storage-group1.service; enabled; vendor preset: disabled)  
...  
Active: inactive (dead) ➔ Active: exited

Resolution: A systemctl daemon-reload is required before starting each storage service.
Example: To start both fastdfs-storage-group1.service and fastdfs-storage-group2.service on Machine A:

Start fastdfs-storage-group1.service:

1
2

   systemctl daemon-reload  
   systemctl start fastdfs-storage-group1.service

Start fastdfs-storage-group2.service:

1
2

   systemctl daemon-reload  
   systemctl start fastdfs-storage-group2.service

FastDFS Storage Service Configuration
On all storage machines, when modifying /etc/fdfs/mod_fastdfs.conf, adjust the group_name value according to the respective group of each storage. Pay special attention to two configuration items. Below is an example using group2 and group3:

Global group_name
The group_name in the global configuration must be separated by “/”, and its value should match the group_name in local configurations. For example:

`1`	`group_name=group2/group3`

Steps to Configure and Start Services

Execute the following commands for group1:

1
2
3

   sudo systemctl daemon-reload
   sudo systemctl enable fastdfs-storage-group1.service
   sudo systemctl start fastdfs-storage-group1.service

Check the status of fastdfs-storage-group1.service:

`1`	`systemctl status fastdfs-storage-group1.service`

If the status shows active (running), proceed to start fastdfs-storage-group2.service. If not, check logs to troubleshoot:

1
2
3

   sudo systemctl daemon-reload
   sudo systemctl enable fastdfs-storage-group2.service
   sudo systemctl start fastdfs-storage-group3.service  # Note: Verify group name consistency (group2 vs group3)

Monitor the status of fastdfs-storage-group2.service until it becomes active.

Notes

You may encounter a warning like Unknown lvalue 'ExecRestart' in section 'Service'. A known workaround (yum install systemd-*) might not resolve this issue. Contributions to fix this are welcome.

FastDFS Nginx Module Configuration
When modifying /etc/fdfs/mod_fastdfs.conf on storage machines:

Ensure group_name values are correctly set for each storage group (e.g., group2 and group3).
Follow the global/local group_name format as described above.

<group_entries> </group_entries>

Resolving apt Update GPG Signature Issues in Docker

Mon, 05 Jun 2023 00:00:00 +0000

Cause

After investigation, it was identified that Ubuntu 21.10 and Fedora 35 began using glibc 2.34 and higher versions. In glibc 2.34, a new system call named clone3 was introduced. Normally, Docker intercepts all system calls in containers and determines how to handle them. If Docker lacks specific policies for a particular system call, its default policy returns a “Permission Denied” error to the container. However, when glibc receives this error, it does NOT fallback to alternative methods. It would only attempt fallback procedures if the response indicated “This system call is unavailable.”

Solutions

Solution 1

Add the following parameter when running the container to bypass Docker’s system call restrictions:

`1`	`--security-opt seccomp=unconfined`

Important caveats:

This compromises container security.
This parameter cannot be used during image builds. Refer to Solution 2 for alternatives.

Solution 2

Upgrade Docker to version 20.10.8 or higher (>20.10.8).

Production environment considerations:

Upgrading Docker versions in production may be challenging.
When building images, avoid using Ubuntu 21.10, Fedora 35, or newer as base images, and verify if derived images are affected.
Most official images are based on Debian - confirm Debian-based image compatibility to ensure they aren’t impacted by this issue.

Solution 3

Upgrade runc.

https://github.com/opencontainers/runc/releases/

Check the runc version before upgrading using docker version:

1. Select version 1.0.0-rc95 and download runc.amd64

2. Upload the file to the server, rename it, and grant permissions

`1`	`mv runc.amd64 runc && chmod +x runc`

3. Back up the existing runc

1
2

which runc
mv

4. Stop Docker

`1`	`systemctl stop docker`

5. Replace with the new runc version

`1`	`cp runc /usr/bin/runc`

6. Start Docker

`1`	`systemctl start docker`

7. Verify if runc was upgraded successfully

`1`	`docker version`

Guide to Setting Up Confluence with Docker

Wed, 22 Mar 2023 00:00:00 +0000

Required File Downloads:

1
2

wget https://minio-console.coderkang.top/files/atlassian-agent.jar
wget https://minio-console.coderkang.top/files/mysql-connector-j-8.3.0.jar

docker-compose.yaml

version: '3.8'
name: atlassian
services:
  confluence:
    image: atlassian/confluence-server:8.5.2
    container_name: atlassian-confluence
    restart: always
    environment:
      - TZ=Asia/Shanghai
      - JVM_MINIMUM_MEMORY=4096m
      - JVM_MAXIMUM_MEMORY=8192m
      - ATL_DB_TYPE=mysql
      - ATL_JDBC_URL=jdbc:mysql://db:3306/confluence?sessionVariables=transaction_isolation='READ-COMMITTED'
      - ATL_JDBC_USER=root
      - ATL_JDBC_PASSWORD=******
      - JAVA_OPTS='-javaagent:/opt/atlassian-agent.jar'
    ports:
      - 38090:8090
      - 38091:8091
    volumes:
      - ./application-data/:/var/atlassian/application-data/confluence/
      # - ./setenv.sh:/opt/atlassian/confluence/bin/setenv.sh
      - ./atlassian-agent.jar:/opt/atlassian-agent.jar
      - ./mysql-connector-j-8.3.0.jar:/opt/atlassian/confluence/confluence/WEB-INF/lib/mysql-connector-j-8.3.0.jar
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8090"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s
    depends_on:
      db:
        condition: service_healthy

  db:
    image: mysql:latest
    container_name: atlassian-db
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: ******
      MYSQL_DATABASE: confluence
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_bin
    volumes:
      - ./application-db/data/:/var/lib/mysql/
      - ./application-db/conf.d/:/etc/mysql/conf.d/
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "root", "-p******"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s

1
2

docker compose up -d
# The startup process may take some time as it needs to wait for the MySQL service to fully initialize

Access the webpage to obtain the Server ID.

docker compose exec -it confluence java -jar /opt/atlassian-agent.jar -m <Email> -n <Username> -o <Organization> -p conf -s '<Server ID>'

# Example:
# docker compose exec -it confluence java -jar /opt/atlassian-agent.jar -m CoderKang@hotmail.com -n CoderKang -o NiKo -p conf -s 'BHE8-N86V-SW29-TDDO'

Click “Next” after entering the information and wait…

📌 Important

If encountering issues during installation, after stopping the container:

Delete the application-db and application-data directories

Rebuild the container

Kubernetes Lifecycle and Probes Deep Dive

Sat, 18 Feb 2023 00:00:00 +0000

Pod Lifecycle

📝 Note

Pods follow a predefined lifecycle, starting from the Pending phase. If at least one primary container starts successfully, the Pod transitions to the Running phase. Subsequently, depending on whether any container in the Pod exits with a failure status, it enters either the Succeeded or Failed phase.

When a Pod is deleted, some kubectl commands may display its status as Terminating. This Terminating state is not one of the official Pod phases.

Phase	Description
`Pending`	The Pod has been accepted by the Kubernetes system, but one or more containers have not been created or started. This phase includes time spent waiting for scheduling and downloading container images.
`Running`	The Pod is bound to a node, and all containers have been created. At least one container is still running, starting, or restarting.
`Succeeded`	All containers in the Pod have terminated successfully and will not be restarted.
`Failed`	All containers in the Pod have terminated, and at least one container exited due to failure (e.g., exited with a non-zero status or was terminated by the system without automatic restart configured).
`Unknown`	The Pod status cannot be retrieved, typically due to communication failures with the node hosting the Pod.

Container States

Kubernetes monitors the state of each container within a Pod, similar to how it tracks the Pod lifecycle.

Once the scheduler assigns a Pod to a node, the kubelet initiates container creation for the Pod through the container runtime. A container can be in one of three states: Waiting, Running, or Terminated. To inspect the state of containers in a Pod, use kubectl describe pod <pod-name>. The output includes the status of each container within the Pod.

Each state has specific implications:

`Waiting`

If a container is neither in the Running nor Terminated state, it is Waiting. A container in the Waiting state is still performing operations required to start successfully, such as pulling a container image from a registry or applying ConfigMap/Secret data to the container.

`Running`

The Running state indicates that the container is actively executing and functioning without issues.

`Terminated`

A container in the Terminated state has completed execution, either normally or due to a failure.

Pod Failure Scenarios

A Pod may encounter various exceptions during its lifecycle. Based on whether its containers are running, these failure scenarios can be broadly categorized into two groups:

Exceptions during container creation: These occur while the Pod is being scheduled or its containers are being created. The Pod remains stuck in the Pending phase.
Exceptions during container execution: These occur while containers are running. The Pod’s stage varies depending on the specific scenario.

4. Container Probes

📝 Note

Probes are a mechanism used by kubelet to periodically check the status of containers. To perform a check, kubelet can execute code inside the container or make a network request.

Liveness Probe

What is a Liveness Probe?

A liveness probe determines whether a container is running. If the probe fails, the kubelet kills the container, and the container is subjected to the restart policy.

If a container’s liveness probe fails multiple times, the kubelet will restart the container. If the liveness probe fails, the kubelet will kill the container, and the container will be subjected to the restart policy. If a container does not provide a liveness probe, the default status is Success.

The liveness probe does not wait for the readiness probe to succeed. If you want to wait for the liveness probe to execute before the readiness probe, you can define initialDelaySeconds, or use the startup probe.

When to Use a Liveness Probe?

If a container’s process can crash on its own, you may not need a liveness probe; kubelet will automatically restart the container based on the restartPolicy.

If you want the container to be killed and restarted when the probe fails, specify a liveness probe and set the restartPolicy to “Always” or “OnFailure”.

Readiness Probe

What is a Readiness Probe?

A readiness probe determines when a container is ready to accept traffic. This probe is useful when waiting for an application to perform time-consuming initial tasks, such as establishing network connections, loading files, and warming up caches.

If the readiness probe returns a failure status, Kubernetes removes the Pod from the endpoints of all associated Services.

The readiness probe continues to run throughout the container’s lifecycle.

When to Use a Readiness Probe?

Specify a readiness probe if you want to start sending traffic to a Pod only after the probe succeeds. In this case, the readiness probe might be the same as the liveness probe. However, the presence of a readiness probe in the specification ensures that the Pod does not receive any data during the startup phase and only starts receiving traffic after the probe succeeds.

You can also define a readiness probe if you want the container to enter a maintenance state independently. This probe should check a readiness-specific endpoint, which differs from the liveness probe.

If your application has strict dependencies on backend services, implement both liveness and readiness probes. After the liveness probe confirms the application is healthy, the readiness probe can perform additional checks to verify the availability of required backend services. This helps avoid directing traffic to Pods that might return errors.

For containers requiring large data loading, configuration file processing, or migrations during startup, use a startup probe. However, if you need to distinguish between a failed application and one still initializing, a readiness probe may be more appropriate.

Startup Probe (`startupProbe`)

What is a Startup Probe?

A startup probe checks whether an application within a container has started. It is designed for containers with slow startup times to prevent kubelet from terminating them prematurely before they begin running.

If configured, this probe disables liveness and readiness checks until the startup probe succeeds.

The startup probe executes only during the container’s initialization phase, unlike the readiness probe, which runs periodically.

When to Use a Startup Probe?

The startup probe is useful for Pods that contain containers requiring extended time to become ready. Instead of configuring a long liveness probe interval, you can set up a separate configuration option to probe the container during startup. This allows exceeding the duration permitted by the liveness check interval by a significant margin.

If a container’s startup time typically exceeds the total value of initialDelaySeconds + failureThreshold × periodSeconds, a startup probe should be configured to check the same endpoint used by the liveness probe. The default value for periodSeconds is 10 seconds. Set its failureThreshold high enough to ensure sufficient startup time for the container while retaining the default values for the liveness probe. This configuration helps mitigate deadlock scenarios.

Probe Execution

Probe	Target	Action	Effect	Runtime
Liveness	Container	Restart	Restart container	Entire container lifecycle
Readiness	Endpoint	Remove	Remove from service endpoints (no traffic)	Entire container lifecycle
Startup	Container	Restart	Restart container	Executed once after container starts Running

Probe Usage

There are four different methods to use probes to check containers. Each probe must be precisely defined as one of these four mechanisms:

Probe Type	Description
`exec`	Executes a specified command inside the container. The diagnosis is considered successful if the command exits with a status code of 0.
`tcpSocket`	Performs a TCP check on the specified port of the container’s IP address. The diagnosis is successful if the port is open. If the remote system (container) closes the connection immediately after opening it, this is still considered healthy.
`httpGet`	Sends an HTTP `GET` request to the specified port and path on the container’s IP address. The diagnosis is successful if the response status code is between 200 and 399 (inclusive).
`grpc`	Uses gRPC to perform a remote procedure call. The target should implement the gRPC health check. If the response status is “SERVING”, the diagnosis is considered successful.

Probe Parameters	Description
`initialDelaySeconds`	Number of seconds to wait after the container starts before initiating startup, liveness, and readiness probes. If a startup probe is defined, the delays for liveness and readiness probes will begin only after the startup probe succeeds. If `periodSeconds` is greater than `initialDelaySeconds`, `initialDelaySeconds` is ignored. The default is 0 seconds, and the minimum value is 0.
`periodSeconds`	Interval (in seconds) at which probes are executed. The default is 10 seconds. The minimum value is 1.

`timeoutSeconds`	The number of seconds to wait after a probe times out. Default is 1 second. Minimum value is 1.
`successThreshold`	Minimum consecutive successes required for a probe to be considered successful after failure. Default is 1. For liveness and startup probes, this value must be 1. Minimum value is 1.
`failureThreshold`	After `failureThreshold` consecutive failures, Kubernetes considers the overall check failed: container status becomes unready/unhealthy/inactive. Default is 3, minimum is 1. For liveness or startup probes: If ≥ `failureThreshold` probes fail, Kubernetes triggers container restart (following `terminationGracePeriodSeconds`). For readiness probes: kubelet continues executing failed probes but marks Pod’s `Ready` condition as false.
`terminationGracePeriodSeconds`	Configures the grace period for kubelet to wait between triggering container termination and forcing runtime stop. Default inherits Pod-level value (30s if unset). Minimum is 1. Added in Kubernetes v1.25, only effective for startup and liveness probes.

Example

apiVersion: v1
kind: Namespace
metadata:
  name: k8s-test


---
apiVersion: v1
kind: ConfigMap
metadata:
  name: probe-demo
  namespace: k8s-test
data:
  default.conf: |
    server {
        listen       80;
        server_name  localhost;
        keepalive_timeout  0;  # Disable Keep-Alive

        location / {
            root   /usr/share/nginx/html;
            index  index.html index.htm;
        }
    }


---
apiVersion: v1
kind: Service
metadata:
  name: probe-demo
  namespace: k8s-test
spec:
  selector:
    app: probe-demo
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
      nodePort: 31080
  type: NodePort


---
apiVersion: v1
kind: Pod
metadata:
  name: probe-demo
  namespace: k8s-test
  labels:
    app: probe-demo
spec:
  restartPolicy: OnFailure  # Pod restart policy: OnFailure (restart only on non-zero exit code), Always (restart on any exit), Never (no restart)
  containers:
    - name: probe-demo
      image: 192.168.142.99:7891/devops/nginx:latest
      # command: ["/bin/sh", "-c", "sleep 10"]  # Simulate Pod Succeeded status
      # command: ["/bin/sh", "-c", "sleep infinity"]  # For startup probe testing
      command: ["/bin/sh", "-c", "set -ex && nohup nginx && touch /tmp/healthy && sleep 30 && rm -f /tmp/healthy && sleep 600"]
      # startupProbe:
      #   exec:
      #     command:
      #       - cat
      #       - /tmp/healthy
      #   initialDelaySeconds: 5  # Wait 5s after container starts
      #   periodSeconds: 5       # Probe every 5s
      #   timeoutSeconds: 1      # Probe timeout
      #   successThreshold: 1    # Minimum consecutive successes to consider probe successful
      #   failureThreshold: 3    # Consecutive failures needed to mark probe failed
      # livenessProbe:
      #   exec:
      #     command:
      #       - cat
      #       - /tmp/healthy
      #   initialDelaySeconds: 5
      #   periodSeconds: 5
      #   timeoutSeconds: 1
      #   failureThreshold: 3
      readinessProbe:
        exec:
          command:
            - cat
            - /tmp/healthy
        initialDelaySeconds: 5
        periodSeconds: 5
        timeoutSeconds: 1
        failureThreshold: 3
      
      volumeMounts:
        - name: config-volume
          mountPath: /etc/nginx/conf.d/default.conf
          subPath: default.conf

  volumes:
    - name: config-volume
      configMap:
        name: probe-demo
        items:
          - key: default.conf
            path: default.conf

Practical Demonstration

Process Analysis

Dockerized OpenVPN Client with Proxy Integration

Tue, 14 Jun 2022 00:00:00 +0000

To resolve conflicts between OpenVPN and proxy software

wfg/docker-openvpn-client: OpenVPN client with killswitch and proxy servers; built on Alpine (github.com)

Build Files

Dockerfile

FROM alpine:3.17

RUN apk add --no-cache \
    bash \
    bind-tools \
    iptables \
    ip6tables \
    openvpn

COPY . /usr/local/bin

ENV KILL_SWITCH=on

ENTRYPOINT [ "entry.sh" ]

killswitch

#!/usr/bin/env bash

set -o errexit
set -o nounset
set -o pipefail

# Block all traffic not going through tun0 interface
iptables --insert OUTPUT \
    ! --out-interface tun0 \
    --match addrtype ! --dst-type LOCAL \
    ! --destination "$(ip -4 -oneline addr show dev eth0 | awk 'NR == 1 { print $4 }')" \
    --jump REJECT

# Create static routes for allowed subnets
default_gateway=$(ip -4 route | awk '$1 == "default" { print $3 }')
for subnet in ${1//,/ }; do
    ip route add "$subnet" via "$default_gateway"
    iptables --insert OUTPUT --destination "$subnet" --jump ACCEPT
done

# Whitelist OpenVPN server addresses
global_port=$(awk '$1 == "port" { print $2 }' "${config:?"config file not found by kill switch"}")
global_protocol=$(awk '$1 == "proto" { print $2 }' "${config:?"config file not found by kill switch"}")
remotes=$(awk '$1 == "remote" { print $2, $3, $4 }' "${config:?"config file not found by kill switch"}")
ip_regex='^(([1-9]?[0-9]|1[0-9][0-9]|2([0-4][0-9]|5[0-5]))\.){3}([1-9]?[0-9]|1[0-9][0-9]|2([0-4][0-9]|5[0-5]))$'
while IFS= read -r line; do
    # Process each remote entry
    IFS=" " read -ra remote <<< "${line%%\#*}"
    address=${remote[0]}
    port=${remote[1]:-${global_port:-1194}}
    protocol=${remote[2]:-${global_protocol:-udp}}

    # Allow IP addresses and handle domain resolution
    if [[ $address =~ $ip_regex ]]; then
        iptables --insert OUTPUT --destination "$address" --protocol "$protocol" --destination-port "$port" --jump ACCEPT
    else
        for ip in $(dig -4 +short "$address"); do
            iptables --insert OUTPUT --destination "$ip" --protocol "$protocol" --destination-port "$port" --jump ACCEPT
            echo "$ip $address" >> /etc/hosts
        done
    fi
done <<< "$remotes"

entry.sh

#!/usr/bin/env bash

set -o errexit
set -o nounset
set -o pipefail

cleanup() {
    kill TERM "$openvpn_pid"
    exit 0
}

is_enabled() {
    [[ ${1,,} =~ ^(true|t|yes|y|1|on|enable|enabled)$ ]]
}

# Either a specific file name or a pattern.
if [[ $CONFIG_FILE ]]; then
    config_file=$(find /config -name "$CONFIG_FILE" 2> /dev/null | sort | shuf -n 1)
else
    config_file=$(find /config -name '*.conf' -o -name '*.ovpn' 2> /dev/null | sort | shuf -n 1)
fi

if [[ -z $config_file ]]; then
    echo "no openvpn configuration file found" >&2
    exit 1
fi

echo "using openvpn configuration file: $config_file"

openvpn_args=(
    "--config" "$config_file"
    "--cd" "/config"
)

if is_enabled "$KILL_SWITCH"; then
    openvpn_args+=("--route-up" "/usr/local/bin/killswitch.sh $ALLOWED_SUBNETS")
fi

# Docker secret that contains the credentials for accessing the VPN.
if [[ $AUTH_SECRET ]]; then
    openvpn_args+=("--auth-user-pass" "/run/secrets/$AUTH_SECRET")
fi

openvpn "${openvpn_args[@]}" &
openvpn_pid=$!

trap cleanup TERM

wait $openvpn_pid

Build and Start

docker-compose.yaml

Public image: ghcr.io/wfg/openvpn-client
Build image:

`1`	`docker compose build`

Start:

`1`	`docker compose up -d`

services:
  openvpn-client:
    image: openvpn-client:latest  # ghcr.io/wfg/openvpn-client
    build:
      context: ./
      dockerfile: Dockerfile
    container_name: openvpn-client
    cap_add:
      - NET_ADMIN
    environment:
      # - HTTP_PROXY=on
      - SOCKS_PROXY=on
    devices:
      - /dev/net/tun:/dev/net/tun
    volumes:
      - ./local:/config
      - ./local:/data/vpn
    ports:
      # - 8080:8080  # HTTP_PROXY port
      - 1080:1080  # SOCKS_PROXY port
    restart: unless-stopped

Using with Proxy Software

Utilizing the Rule Configuration routing policy in Clash software

proxies:
	- { name: OpenVPN, server: 127.0.0.1, port: 1080, type: socks5 }

rules:
	- IP-CIDR,192.168.142.0/24,OpenVPN
	- IP-CIDR,10.10.10.0/24,OpenVPN
	- IP-CIDR,172.16.0.0/24,OpenVPN

One More Thing

In addition to OpenVPN, other campus VPN clients like EasyConnect (used for free CNKI access) can be deployed in Docker containers. Expose HTTP/SOCKS proxies and configure routing rules to achieve elegant cross-network environment access.

1
2

    - { name: 'CNKI', type: socks5, server: 10.10.10.45, port: 57080}
    - 'DOMAIN-SUFFIX,cnki.net,CNKI'

Complete Guide to Deploying Kubernetes 1.23.5 Cluster on CentOS 7

Tue, 31 May 2022 00:00:00 +0000

Environment Preparation

Disable Firewall and Mail Service

# Check firewall status
firewall-cmd --state
# Temporarily disable firewall
systemctl stop firewalld.service
# Disable firewall auto-start
systemctl disable firewalld.service

systemctl stop postfix.service
systemctl disable postfix.service

Disable SELinux

# Check SELinux status
getenforce
# Temporarily disable SELinux
setenforce 0
# Permanently disable SELinux
sed -i 's/^ *SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

Disable Swap

# Temporarily disable swap
swapoff -a
# Permanently disable swap
sed -i.bak '/swap/s/^/#/' /etc/fstab
# Verify
free -g

Tuning Kernel Parameters and Modules

After system installation, appropriately adjusting kernel parameters based on practical application scenarios can help establish a more efficient and stable system environment. This includes:

Optimizing resource allocation parameters (e.g., modifying the maximum number of open file descriptors to improve high-concurrency service capabilities)
Adjusting network parameters using sysctl (e.g., modifying ip_local_port_range, nf_conntrack_max, and socket buffer sizes)
Managing kernel modules: Loading or disabling unnecessary modules (e.g., disabling unused hardware drivers to reduce potential vulnerabilities while enabling specific acceleration modules for performance enhancement)

Typical operation examples:

# Modify network port range
sysctl -w net.ipv4.ip_local_port_range="1024 65000"

# Adjust connection tracking table size
echo 1048576 > /proc/sys/net/netfilter/nf_conntrack_max

# Dynamically load zRAM module
modprobe zram

Key principle: Make targeted adjustments according to actual service requirements, using monitoring data as guidance while maintaining parameter modification records.

Enable ipvs

If ipvs is not enabled, iptables will be used for packet forwarding (less efficient). Recommended to enable ipvs for better performance.

cat <<EOF> /etc/sysconfig/modules/ipvs.modules
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOF

# Load modules
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4

# Install ipset package
yum install ipset -y
 
# Install management tool ipvsadm
yum install ipvsadm -y

Synchronize server time

`1`	`yum install chrony -y`

# Master node configuration
vim /etc/chrony.conf
server ntp.aliyun.com iburst

allow 192.168.142.0/24

# Slave node
vim /etc/chrony.conf
server controller iburst

1
2

systemctl restart chronyd.service
systemctl enable chronyd.service

Modify Hostname and Hosts (Optional)

hostnamectl set-hostname master

hostnamectl set-hostname node1

hostnamectl set-hostname node2


cat >> /etc/hosts << EOF
192.168.231.3 master
192.168.231.4 node1
EOF
echo -e "\033[32m [Host Resolution] ==> OK \033[0m"

System Log Optimization

1
2

mkdir /var/log/journal
mkdir /etc/systemd/journald.conf.d

cat > /etc/systemd/journald.conf.d/99-prophet.conf << EOF
[journal]
# Persist logs to disk
Storage=persistent
# Compress historical logs
Compress=yes

# Set sync interval to disk as 5 minutes
SyncIntervalSec=5m
# Control rate limiting for log messages
RateLimitInterval=30s
RateLimitBurst=1000

# Maximum disk usage 10G
SystemMaxUse=10G

# Control runtime log size
RuntimeMaxUse=500M

# Max individual log file size 200M
SystemMaxFileSize=200M

# Log retention period (2 weeks)
MaxRetentionSec=2week

# Disable syslog forwarding
ForwardToSyslog=no

# Enable sealing mode when size limits reached
SystemMaxSealing=yes
RuntimeMaxSealing=yes

# Enable watchdog timer (restart service after 1h inactivity)
RuntimeWatchdogSec=1h

# Enable automatic log file optimization
SystemMaxFilesTidy=yes
RuntimeMaxFilesTidy=yes

# Max age for individual log files
RuntimeMaxFileSec=7days

# Auto-select best storage mode
StorageAuto=yes

EOF

1
2

systemctl restart systemd-journald
echo -e "\033[32m [Log Optimization] ==> OK \033[0m"

Docker Installation

set -e
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo yum install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

sudo systemctl start docker
sudo systemctl enable docker

vim /etc/docker/daemon.json

{
    "exec-opts": ["native.cgroupdriver=systemd"],
    "insecure-registries": [
        "192.168.142.99"
    ]
}

1
2

sudo systemctl restart docker
docker login 192.168.142.99

Kubernetes Installation

Installing kubelet, kubeadm, and kubectl

Adding Kubernetes Repository

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
        http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

Install kubeadm, kubelet, and kubectl

# Check version, the latest version is 1.23.5-0
yum list kubeadm --showduplicates | sort -r

yum install -y kubelet-1.23.5-0 kubectl-1.23.5-0 kubeadm-1.23.5-0

kubeadm version
# kubeadm version: &version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:57:37Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"linux/amd64"}

Modify kubelet configuration

1
2

# Modify configuration file /etc/sysconfig/kubelet (this file does not exist by default and needs to be created)
KUBELET_EXTRA_ARGS=--root-dir=/var/lib/kubelet

Start kubelet service and enable it to start on boot

1
2

systemctl start kubelet
systemctl enable kubelet

Initialize Kubernetes Cluster

Initialization via Configuration File

cat <<EOF> kubeadm.yaml 
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 192.168.4.27 # Internal IP of the apiserver node
bindPort: 6443
nodeRegistration:
criSocket: /run/containerd/containerd.sock  # Switch to containerd
imagePullPolicy: IfNotPresent
name: master
taints:
- effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
type: CoreDNS # DNS type (CoreDNS)
etcd:
local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers # Modified image repository for accessibility
kind: ClusterConfiguration
kubernetesVersion: 1.23.5 # Kubernetes version
networking:
dnsDomain: cluster.local
podSubnet: 10.244.0.0/16  
serviceSubnet: 10.96.0.0/12
scheduler: {}
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs  # kube-proxy mode
EOF

`1`	`kubeadm init --config kubeadm.yaml`

📌 Important

kubeadm-installed certificates have a default validity period of one year

kube-proxy operates in iptables mode by default. This can be modified via kubectl edit configmap kube-proxy -n kube-system

Imperative Initialization

kubeadm init \
    --control-plane-endpoint k8svip:8443 \
    --kubernetes-version=v1.23.5 \
    --service-cidr=172.96.0.0/12 \
    --pod-network-cidr=172.244.0.0/16 \
    --image-repository registry.aliyuncs.com/google_containers \
    --upload-certs

kubeadm join k8svip:8443 --token i8zsn5.dakiqfxexdxn7wdt \
    --discovery-token-ca-cert-hash sha256:61ed5a0941ecf47078dac91c4389bc8abb9c761149e869a40f9c3da859b39dba \
    --control-plane --certificate-key fc133d520c12052c9391e075c3aa6dda456599b70b1335aba2c3e0680e75af6e

kubeadm init --kubernetes-version v1.23.5 \
    --apiserver-advertise-address 172.16.0.185 \
    --image-repository registry.aliyuncs.com/google_containers \
    --service-cidr 172.96.0.0/12 \
    --pod-network-cidr 172.244.0.0/16 \
    --upload-certs \
    --v=5

Install Calico Network Plugin (Execute on master node)

1
2
3

# Download manifest
curl https://docs.projectcalico.org/manifests/calico.yaml -o calico.yaml
kubectl apply -f calico.yaml

Installing Auto-Completion Tools

yum install -y bash-completion
source /usr/share/bash-completion/bash_completion
source <(kubectl completion bash)
echo "source <(kubectl completion bash)" >> ~/.bashrc

Deployment Verification Cluster

cat << EOF > nginx-ds.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy-game
  namespace: default
spec:
  replicas: 8
  selector:
    matchLabels:
      app: game
      release: stabel
  template:
    metadata:
      labels:
        app: game
        release: stabel
        env: test
    spec:
      imagePullSecrets:
        - name: kkregcred
      containers:
      - name: game
        image: registry.cn-beijing.aliyuncs.com/kaikai136/docker-2048:v1
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: game-svc
  namespace: default
spec:
  type: NodePort
  selector:
    app: game
    release: stabel
  ports:
  - name: http
    port: 80
    targetPort: 80
    nodePort: 32000
    protocol: TCP
EOF

kubectl apply -f nginx-ds.yaml

kubectl create deployment nginx –image=nginx kubectl expose deployment nginx –port=80 –type=NodePort kubectl get pod,svc

Testing Calico Network

`1`	`kubectl run busybox --image docker.io/library/busybox:1.28 --image-pull-policy=IfNotPresent --restart=Never --rm -it busybox -- sh`

/ # ping www.baidu.com
PING www.baidu.com (180.101.50.242): 56 data bytes
64 bytes from 180.101.50.242: seq=0 ttl=51 time=7.880 ms
64 bytes from 180.101.50.242: seq=1 ttl=51 time=7.247 ms
^C
# The network accessibility indicates Calico network plugin is properly installed
--- www.baidu.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 7.247/7.563/7.880 ms
/ # nslookup kubernetes.default.svc.cluster.local
Server:    172.96.0.10
Address 1: 172.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kubernetes.default.svc.cluster.local
Address 1: 172.96.0.1 kubernetes.default.svc.cluster.local

# 172.96.0.10 corresponds to the clusterIP of coreDNS, verifying coreDNS configuration.
# Internal Service name resolution is handled by coreDNS.

# Note:
# Must use busybox version 1.28 as specified - latest versions may fail to resolve DNS and IP addresses with nslookup

Installing Additional Tools

Auto Completion Tools

yum install -y bash-completion
source /usr/share/bash-completion/bash_completion
source <(kubectl completion bash)
echo "source <(kubectl completion bash)" >> ~/.bashrc

Docker Deployment Guide for ELK

Sun, 22 May 2022 00:00:00 +0000

📝 Note

deviantony/docker-elk: The Elastic stack (ELK) powered by Docker and Compose. (github.com)

1
2

git clone https://github.com/deviantony/docker-elk
cd docker-elk

`1`	`vim .env`

1
2

docker compose up setup
docker compose up

1
2

docker compose down -v setup
docker compose down -v

CentOS 7 System Optimization and Deployment Guide

Wed, 23 Jun 2021 00:00:00 +0000

CentOS7 Kernel Upgrade

Download kernel source:

`1`	`rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm`

Install the latest kernel version:

`1`	`yum --enablerepo=elrepo-kernel install -y kernel-lt`

Check entries:

`1`	`cat /boot/grub2/grub.cfg \| grep menuentry`

Set default boot kernel:

`1`	`grub2-set-default "CentOS Linux (4.4.221-1.el7.elrepo.x86_64) 7 (Core)"`

Disable Firewall

1
2

systemctl stop firewalld  
systemctl disable firewalld

Install Common Tools

`1`	`yum install -y conntrack ntpdate ntp ipvsadm ipset jq iptables curl sysstat libseccomp wget vim git net-tools dos2unix lsof tcpdump lrzsz telnet bash-completion.noarch conntrack-tools`

Linux completion:

`1`	`yum install libvirt-bash-completion bash-completion gedit-plugin-bracketcompletion gedit-plugin-wordcompletion libguestfs-bash-completion -y`

Configure SELinux

1
2

setenforce 0  
sed -i '/^SELINUX=/ s/enforcing/disabled/' /etc/selinux/config

Update History and Shell Timeout Settings

Edit /etc/profile:

1
2

export HISTSIZE=100  
export TMOUT=300

Disable swap partition

swapoff -a
# To permanently disable swap partition, comment out the swap line in the following file
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
echo "vm.swappiness = 0">> /etc/sysctl.conf
sysctl -p

Disable mail service

1
2

systemctl stop postfix.service
systemctl disable postfix.service

Log optimization

mkdir /var/log/journal
mkdir /etc/systemd/journald.conf.d
cat > /etc/systemd/journald.conf.d/99-prophet.conf << EOF
[journal]
# Persistent storage to disk
Storage=persistent
# Compress historical logs
Compress=yes
 
SyncIntervalSec=5m
RateLimitInterval=30s
RateLimitBurst=1000
# Maximum disk space 10G
SystemMaxUse=10G
# Single log file maximum size 200M
SystemMaxFileSize=200M
# Log retention time 2 weeks
MaxRetentionSec=2week
# Do not forward logs to syslog
ForwardToSyslog=no
 
EOF
systemctl restart systemd-journald

Load ipvs modules

cat > /etc/sysconfig/modules/ipvs.modules <<EOF
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF
 
chmod 755 /etc/sysconfig/modules/ipvs.modules
bash /etc/sysconfig/modules/ipvs.modules

File Optimization

echo ‘* - nofile 65535 ’ »/etc/security/limits.conf echo ‘vm.max_map_count=262144 ’ »/etc/security/limits.conf

sysctl vm.overcommit_memory=1

tail -1 /etc/security/limits.conf sysctl -p

Kernel Optimization

cat >>/etc/sysctl.conf<<EOF
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 30

net.ipv4.tcp_keepalive_time = 1200
net.ipv4.ip_local_port_range = 4000 65000
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_max_tw_buckets = 36000

net.ipv4.route.gc_timeout = 100
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_synack_retries = 1
net.core.somaxconn = 262144
net.core.netdev_max_backlog = 262144
net.ipv4.tcp_max_orphans = 16384

net.ipv4.tcp_mem = 94500000 915000000 927000000
EOF

sysctl -p

Explanation of Network Parameters

net.ipv4.tcp_syncookies = 1: Enables SYN Cookies. When the SYN backlog overflows, cookies are used to handle connections, mitigating minor SYN flooding attacks. Default: 0 (disabled).
net.ipv4.tcp_tw_reuse = 1: Allows reusing TIME-WAIT sockets for new TCP connections. Default: 0 (disabled).
net.ipv4.tcp_tw_recycle = 1: Enables fast recycling of TIME-WAIT sockets. Default: 0 (disabled).
net.ipv4.tcp_fin_timeout = 30: Defines the time (in seconds) a connection remains in FIN-WAIT-2 state if closed locally.
net.ipv4.tcp_keepalive_time = 1200: Sets the frequency (in seconds) for TCP keepalive probes. Default: 7200 (2 hours), modified to 1200 (20 minutes).
net.ipv4.ip_local_port_range = 1024 65000: Specifies the port range for outgoing connections. Default: 32768-61000, expanded to 1024-65000.
net.ipv4.tcp_max_syn_backlog = 8192: Sets the maximum length of the SYN queue to accommodate more pending connections. Default: 1024.
net.ipv4.tcp_max_tw_buckets = 5000: Limits the maximum number of TIME-WAIT sockets. Exceeding this threshold triggers immediate cleanup. Default: 180000, adjusted for servers like Apache/Nginx to reduce TIME-WAIT sockets. Squid may require additional tuning.

SSH Service Optimization

# Edit SSH configuration
vim /etc/ssh/sshd_config

# Disable GSSAPI authentication
GSSAPIAuthentication no

# Disable DNS resolution checks
UseDNS no  # (Remove the '#' to uncomment; default is disabled)

# Restart SSH service
systemctl restart sshd

1. CPU Core Count, Model, and Clock Speed

`1`	`cat /proc/cpuinfo \| grep name \| cut -f2 -d: \| uniq -c`

2. Testing Disk I/O Performance

1). hdparm Command

The hdparm command provides a CLI interface for reading and setting parameters of IDE/SCSI hard drives. Note: This command only tests disk read speed.

[root@server-68.2.stage.polex.io var ]$ hdparm -Tt /dev/polex_pv/varvol

/dev/polex_pv/varvol:
 Timing cached reads:    MB in  2.00 seconds = 7803.05 MB/sec
 Timing buffered disk reads:  MB in  3.01 seconds = 374.90 MB/sec

[Additional translation of subsequent sections would continue here following the same pattern]

2). The dd Command

The Linux dd command is used to read, convert, and output data. dd can read data from standard input or files, transform it according to specified formats, and then output it to files, devices, or standard output.

We can use the copy function of the dd command to test the IO performance of a disk. Note that dd provides only a rough measurement of disk IO performance and is not highly accurate.

[root@server-68.2.stage.polex.io var ]$ time dd if=/dev/zero of=test.file bs=1G count= oflag=direct
+ records in
+ records out
 bytes (2.1 GB) copied, 13.5487 s,  MB/s
 
real    0m13.556s
user    0m0.000s
sys    0m0.888s

??? note “Parameter Explanation” As shown, the disk write speed for this partition is 159 MB/s. Key parameters include:

- `/dev/zero`: A pseudo-device that generates empty character streams; no IO is incurred.
- `if`: Specifies the input file for `dd` to read from.
- `of`: Specifies the output file for `dd` to write to.
- `bs`: Defines the block size for each write operation.
- `count`: Sets the number of blocks to write.
- `oflag=direct`: Required for IO testing, ensures direct writes to disk (bypassing cache).

3). FIO Testing Disk IO Performance

The fio command is specifically used to test IOPS and is more accurate than the dd command. The fio command has many parameters. Here are some examples for reference:

`1`	`yum install fio`

# Random read:  
fio -filename=/dev/sda1 -direct=1 -iodepth 1 -thread -rw=randread -ioengine=psync -bs=4k -size=60G -numjobs=64 -runtime=10 -group_reporting -name=file  
# Sequential read:  
fio -filename=/dev/sda1 -direct=1 -iodepth 1 -thread -rw=read -ioengine=psync -bs=4k -size=60G -numjobs=64 -runtime=10 -group_reporting -name=file  
# Random write:  
fio -filename=/dev/sda1 -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=4k -size=60G -numjobs=64 -runtime=10 -group_reporting -name=file  
# Sequential write:  
fio -filename=/dev/sda1 -direct=1 -iodepth 1 -thread -rw=write -ioengine=psync -bs=4k -size=60G -numjobs=64 -runtime=10 -group_reporting -name=file  
# Mixed random read/write:  
fio -filename=/dev/sda1 -direct=1 -iodepth 1 -thread -rw=randrw -rwmixread=30 -ioengine=psync -bs=4k -size=60G -numjobs=64 -runtime=10 -group_reporting -name=file -ioscheduler=noop

In the results, bw=1532.2KB/s, iops=383 indicates the measured IOPS.

??? note “Parameter Explanation”

filename=/dev/sda1: Test file name, typically selecting the data directory of the disk to be tested
direct=1: Bypasses system buffers during testing for more authentic results
rw=randwrite: Tests random write I/O
rw=randrw: Tests mixed random read/write I/O
rw=randread: Tests random read I/O
bs=4k: Block size per I/O operation is 4KB
bsrange=512-2048: Specifies data block size range
size=60g: Test file size set to 60GB with 4KB I/O operations
numjobs=64: Test runs with 64 concurrent threads
runtime=10: Test duration limited to 10 seconds
ioengine=psync: I/O engine uses psync mode
rwmixwrite=30: 30% write ratio in mixed read/write mode
group_reporting: Aggregates results per-process
Additional parameters:
- lockmem=1g: Limits memory usage to 1GB for testing
- zero_buffers: Initialize buffers with zeros
- nrfiles=8: Number of files generated per process

4). iostat Command

First use iostat to check if disk I/O has high read/write loads
If %util approaches 100%, it indicates too many I/O requests and the I/O system is saturated. The disk may be a bottleneck. Generally, if %util exceeds 70%, the I/O pressure is significant with considerable read wait time. Then check other parameters.

1
2
3

yum install sysstat

iostat -x 1 10

??? note “Explanation”

    rrqm/s: Number of read operations merged per second. Calculated as delta(rmerge)/s
    wrqm/s: Number of write operations merged per second. Calculated as delta(wmerge)/s
    r/s: Read I/O operations completed per second. Calculated as delta(rio)/s
    w/s: Write I/O operations completed per second. Calculated as delta(wio)/s
    rsec/s: Sectors read per second. Calculated as delta(rsect)/s
    wsec/s: Sectors written per second. Calculated as delta(wsect)/s
    rKB/s: Kilobytes read per second. Half of rsec/s (since sector size is 512 bytes)
    
    wKB/s: Kilobytes written per second. Half of wsec/s
    avgrq-sz: Average data size per I/O operation (sectors). Calculated as delta(rsect+wsect)/delta(rio+wio)
    avgqu-sz: Average I/O queue length. Calculated as delta(aveq)/s/1000 (since aveq is in milliseconds)
    await: Average wait time per I/O operation (milliseconds). Calculated as delta(ruse+wuse)/delta(rio+wio)
    svctm: Average service time per I/O operation (milliseconds). Calculated as delta(use)/delta(rio+wio)
    %util: Percentage of time with I/O operations active, or time when I/O queue was non-empty

5). iotop Command

A tool to identify processes with high I/O usage. Simply execute the iotop command:

`1`	`yum install iotop -y`

3. sar Command

The sar -u 1 1 command checks CPU utilization, sampling the data once every 1 second for 1 iteration.
The sar command is an essential tool for analyzing system bottlenecks, used to monitor performance metrics including CPU, memory, disk, and network.

[root@server-68.2.stage.polex.io var ]$ sar -d -p Linux 3.10.0-693.5.2.el7.x86_64 (server-) // x86_64 ( CPU)

:: PM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util :: PM sda 1.00 0.00 3.00 3.00 0.01 9.00 9.00 0.90 :: PM sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 :: PM polex_pv-rootvol 1.00 0.00 3.00 3.00 0.01 9.00 9.00 0.90 :: PM polex_pv-varvol 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 :: PM polex_pv-homevol 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

:: PM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util :: PM sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 :: PM sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 :: PM polex_pv-rootvol 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 :: PM polex_pv-varvol 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 :: PM polex_pv-homevol 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

Average: DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util Average: sda 0.50 0.00 1.50 3.00 0.00 9.00 9.00 0.45 Average: sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: polex_pv-rootvol 0.50 0.00 1.50 3.00 0.00 9.00 9.00 0.45 Average: polex_pv-varvol 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: polex_pv-homevol 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

In the command, the “-d” parameter represents viewing disk performance, the “-p” parameter displays dev devices by names like sda, sdb, etc., “1” indicates sampling values every 1 second, and “2” specifies collecting data a total of 2 times.

??? note “Parameter Explanation”
await: The average waiting time per device I/O operation (in milliseconds).

**svctm**: The average service time per device I/O operation (in milliseconds).  

**%util**: The percentage of time spent on I/O operations each second.  

For disk I/O performance, the following criteria generally apply:  

- Normally, **svctm** should be smaller than **await**. The value of **svctm** depends on disk performance, but CPU and memory load can also affect it. Excessive I/O requests may indirectly increase the **svctm** value.  

- The **await** value is typically influenced by **svctm**, the I/O queue length, and the I/O request pattern. If **svctm** is close to **await**, it indicates minimal I/O waiting, implying excellent disk performance. If **await** is significantly higher than **svctm**, it suggests a long I/O queue wait, which slows down applications. This can often be resolved by using a faster disk.  

- **%util** is another critical metric. If **%util** approaches 100%, the disk is handling too many I/O requests and operating at full capacity, indicating a potential bottleneck. Prolonged high utilization will degrade system performance. Solutions include optimizing programs or upgrading to a faster/higher-capacity disk.

4. vmstat Command

1
2
3

[root@server-68.2.stage.polex.io var ]$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st

In the output, the bi and bo values reflect current disk performance:

bi: Blocks received per second from block devices. Block devices include all disks and other block devices on the system. The default block size is 1024 bytes.
bo: Blocks sent per second to block devices. For example, reading files increases bo.
Generally, both bi and bo should be close to 0. Consistently high values indicate excessive I/O activity, requiring system adjustments.

5. uptime Command

uptime

The output displays:

Current system time
System uptime (duration since last reboot)
Number of logged-in users
Load averages for the last 1 minute, 5 minutes, and 15 minutes.

If the load average values consistently exceed the number of CPUs in the system, it indicates high CPU load, which may degrade performance.

1) netstat Command

`1`	`netstat -an \|grep tcp # View all active TCP connection details`

2). Socket Statistics Command

Previously using the netstat command was found to be inefficient on busy servers, sometimes consuming over 90% of CPU.

The Socket Statistics (ss) command, however, operates at a lower level using the tcp_diag module in the TCP protocol stack for statistical analysis, making it faster and more efficient.

Common ss Commands:

ss -t: Displays all current TCP connections.

??? note “Details”
- -t: Show TCP connection information only
- -a: Display all connection information
- -u: Show UDP connection information only

While nearly all Linux systems include `netstat` by default, `ss` may not be pre-installed (CentOS includes it by default).  

The `ss` command is part of the `iproute` toolkit, a suite of tools for managing TCP/UDP/IP networks with IPv4/IPv6 support.  

If the `ss` command is missing, install the toolkit with:

`1`	`yum install iproute iproute-doc`

7. Disk I/O, Throughput, and Storage IOPS

Disk I/O, Throughput, and Storage IOPS Performance Metrics

Cloud server disk storage performance metrics include Disk I/O, IOPS, and Throughput. Below is a detailed explanation of these terms and their relationships:

Storage IOPS (Input/Output Operations Per Second): The number of read/write operations a disk can perform per second.
Disk I/O: Refers to input (writing data to disk) and output (reading data from disk). The data volume per I/O request is measured in KiB (e.g., 4KiB, 256KiB, 1024KiB).
Throughput: The total data transfer rate per second, combining read and write operations.

Formula: Relationship Between IOPS, I/O Size, and Throughput

The relationship is defined as:
Throughput = IOPS × I/O Size

In other words, higher IOPS and larger I/O sizes result in greater throughput. While higher IOPS and throughput values are generally desirable, they are constrained by hardware limits.

For further details on disk I/O performance for cloud servers, refer to Alibaba Cloud’s documentation on ECS storage performance at ecs6.com.

Common Linux Monitoring Commands

free
df
top / htop
uptime
iftop
iostat
iotop
vmstat
netstat
nethogs (shows bandwidth used by each process)

Compiling Shell Scripts to Executable Binaries with SHC

Mon, 12 Oct 2020 00:00:00 +0000

Introduction

Official repository address: https://github.com/neurobin/shc

A generic shell script compiler. Shc takes a script specified in the command line and generates C source code. The generated source code is then compiled and linked to produce a stripped binary executable file.

The compiled binary still depends on the shell specified in the shebang line (i.e., #!/bin/sh) of the original shell script. Therefore, shc does not create fully independent binaries.

Shc itself is not a compiler like cc, but rather encodes and encrypts shell scripts while generating C source code with additional expiration functionality. It then uses the system compiler to build a stripped binary that behaves exactly like the original script. When executed, the compiled binary decrypts and runs the code using the shell -c option.

Installation

1
2
3

yum install epel-release
yum -y install gcc gcc-c++ libstdc++-devel
yum -y install shc

[root@template mnt]# shc -v
shc parse(-f): No source file specified

shc Usage: shc [-e date] [-m addr] [-i iopt] [-x cmnd] [-l lopt] [-o outfile] [-rvDSUHCABh] -f script

Testing

1
2

shc -v -rf HelloWorld.sh
./HelloWorld.x

Complete Guide to Installing Kubernetes 1.23.5 Cluster on Ubuntu 18.04

Sun, 31 May 2020 00:00:00 +0000

Installing Kubernetes

Install kubelet, kubeadm, kubectl

Add Kubernetes repository

  sudo apt-get install -y ca-certificates curl software-properties-common apt-transport-https curl
  curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -
  
  sudo tee /etc/apt/sources.list.d/kubernetes.list <<EOF
  deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
  EOF
  
  sudo apt-get update

Install kubeadm, kubelet, kubectl

`1`	`apt install -y kubelet=1.23.5-00 kubectl=1.23.5-00 kubeadm=1.23.5-00`

Start kubelet service and enable it to start on boot

1
2

  systemctl start kubelet
  systemctl enable kubelet

Initialize Kubernetes cluster

kubeadm init --kubernetes-version v1.23.5 \
--apiserver-advertise-address 192.168.142.63 \
--image-repository registry.aliyuncs.com/google_containers \
--service-cidr 172.95.0.0/12 \
--pod-network-cidr 172.245.0.0/16 \
--upload-certs \
--v=5

Install network plugin

calico-ipv4.yaml

DevOps on Kang's Blog

Troubleshooting RAID Hard Drive Recognition Issues

Problem Background

Installing MegaCli Tool

Download MegaCli Package

Extract the Package

Convert RPM Package to DEB Package

Install the DEB Package

Fix Dependency Issues (If Needed)

Verify Installation

Checking RAID Status

Investigation Conclusion

Detailed Hard Drive Distribution

System Drive (corresponds to /dev/sda)

Data Drive Array (corresponds to /dev/sdb, capacity 2.7TB)

Data Drive (corresponds to /dev/sdc, capacity 931GB)

🔍 Key Finding: Newly Inserted Drive (Slot 6)

Problem Analysis: Why Wasn’t It Auto-Recognized?

Reason One: Foreign State is None

Reason Two: Manual Virtual Disk Creation Required

Solution: Configure the Drive Online

Create Single-Disk RAID 0

Expected Result

⚠️ Data Safety Notice

Data Recovery Options for Different Scenarios

Refresh System Device List

Summary

Mounting WebDAV on Linux

Mounting WebDAV on Linux

Installing davfs2

Creating a Mount Point Directory

Configuring davfs2

Configuring the davfs2 secrets File

Setting File Permissions

Mounting the WebDAV Share

Automatic Mounting on Boot (Optional)

Unmounting the WebDAV Share

Installing SMB Service on Ubuntu 22.04

Introduction

Installing SMB Service

Update System Packages

Install Samba

Verify Installation

Configuring SMB Service

Backup Default Configuration File

Edit Configuration File

Basic Configuration Example

Create Shared Directory

Verify Configuration File

User Management

Create SMB User

Enable/Disable User

Delete User

Starting and Managing Services

Start Services

Enable Auto-start

Restart Services

Check Service Status

Firewall Configuration

Testing Connection

Local Testing

Mount Share

Windows Client

Advanced Configuration

Home Directory Sharing

Read-only Share

Security Settings

Common Issues

Permission Issues

Connection Issues

Performance Optimization

Monitoring and Logging

View Logs

Connection Status

Share List

Summary

BIOS and UEFI: The Boot Journey from History to Modern Era

Origin: The Golden Age of BIOS

Disruption: The Emergence of UEFI

Brief UEFI Boot Process

System Drive (corresponds to `/dev/sda`)

Data Drive Array (corresponds to `/dev/sdb`, capacity 2.7TB)

Data Drive (corresponds to `/dev/sdc`, capacity 931GB)