跳到內容

較新版本的指令

Site CLI 的指令介面會在不同版本之間增長。有七個 execcli 指令存在於比本機群所執行版本更新的版本中,因此閱讀本章節其餘部分的操作人員將不會找到這些指令——而閱讀 F5 本身較新文件的人則可能預期它們存在。

此介面是超集而非重新排列:較新版本有 89execcli 指令,本機群有 82 個,新增了這七個且無任何移除。

以下描述來自設備本身的完成選單。

指令設備的描述
collect-database-stats收集資料庫統計資料
iptables-lv顯示 iptables -L -v -n
marker-exists-crio檢查 crio 重新啟動標記是否存在
marker-exists-kubelet檢查 kubelet 重新啟動標記是否存在
marker-exists-NetworkManager檢查 NetworkManager 重新啟動標記是否存在
systemctl-restart-NetworkManager重新啟動 NetworkManager 服務
systemctl-start-crio-prune啟動 crio 映像清除服務

整體而言,大多數指令描述的是一個工作流程而非七個無關的工具:三個指令用於詢問某個服務是否已被標記為需要重新啟動,另外兩個則負責執行重新啟動或清除。標記檢查是一組操作中較安全的那一半,而另一半則具有破壞性。

七個指令中有四個僅執行讀取,三個會變更節點——其中一個並非您所料想的那個,這將在下方專屬章節中說明。

回報 crio 的重新啟動標記檔案是否存在。

Terminal window
execcli marker-exists-crio
Capture policy: never execute through the API
Terminal window
execcli marker-exists-kubelet
Capture policy: never execute through the API
Terminal window
execcli marker-exists-NetworkManager
Capture policy: never execute through the API

三個指令均回應 missing,表示該服務目前沒有待處理的重新啟動。

執行 iptables -L -v -n 並原封不動地回傳結果,因此它呈現的是節點過濾表的封包與位元組計數視圖,而非 F5 的抽象化呈現。

Terminal window
execcli iptables-lv
Chain INPUT (policy ACCEPT 2582K packets, 7365M bytes)
pkts bytes target prot opt in out source destination
53804 4167K KUBE-PROXY-FIREWALL all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW /* kubernetes load balancer firewall */
2389K 5628M KUBE-NODEPORTS all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes health check service ports */
53804 4167K KUBE-EXTERNAL-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW /* kubernetes externally-visible service portals */
2582K 7365M KUBE-FIREWALL all -- * * 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 KUBE-PROXY-FIREWALL all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW /* kubernetes load balancer firewall */
0 0 KUBE-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */
0 0 KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW /* kubernetes service portals */
0 0 KUBE-EXTERNAL-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW /* kubernetes externally-visible service portals */
0 0 DOCKER-USER all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 DOCKER-FORWARD all -- * * 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 1964K packets, 1248M bytes)
pkts bytes target prot opt in out source destination
54822 5298K KUBE-PROXY-FIREWALL all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW /* kubernetes load balancer firewall */
54822 5298K KUBE-SERVICES all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate NEW /* kubernetes service portals */
1964K 1248M KUBE-FIREWALL all -- * * 0.0.0.0/0 0.0.0.0/0
Chain DOCKER (1 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- !docker0 docker0 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-BRIDGE (1 references)
pkts bytes target prot opt in out source destination
0 0 DOCKER all -- * docker0 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-CT (1 references)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- * docker0 0.0.0.0/0 0.0.0.0/0 ctstate RELATED,ESTABLISHED
Chain DOCKER-FORWARD (1 references)
pkts bytes target prot opt in out source destination
0 0 DOCKER-CT all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 DOCKER-INTERNAL all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 DOCKER-BRIDGE all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 DROP all -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT all -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
Chain DOCKER-INTERNAL (1 references)
pkts bytes target prot opt in out source destination
Chain DOCKER-USER (1 references)
pkts bytes target prot opt in out source destination
Chain KUBE-EXTERNAL-SERVICES (2 references)
pkts bytes target prot opt in out source destination
Chain KUBE-FIREWALL (2 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * !127.0.0.0/8 127.0.0.0/8 /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT
Chain KUBE-FORWARD (1 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 ctstate INVALID
0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding rules */ mark match 0x4000/0x4000
0 0 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 /* kubernetes forwarding conntrack rule */ ctstate RELATED,ESTABLISHED
... [capture trimmed: first 60 of 79 lines]

其中大部分內容屬於 Kubernetes:KUBE- 鏈是 kube-proxy 的服務、nodeport 和防火牆規則,而 DOCKER-USER 屬於容器執行時期。第一行值得注意而非略過——

# Warning: iptables-legacy tables present, use iptables-legacy to see them

——因為它表示此視圖是不完整的。舊版表格中的規則不會顯示於此,而 Site CLI 也未提供任何能讀取它們的指令。

collect-database-stats 儘管名稱如此,實際上會寫入節點

Section titled “collect-database-stats 儘管名稱如此,實際上會寫入節點”

這個指令是個陷阱,而陷阱就在名稱本身。「收集資料庫統計資料」以及設備本身的描述,讀起來都像是觀察性操作。指令一開始確實如此——開啟檔案限制、etcd 持有的描述符和連線數量、一份 lsof 清單,以及 etcd 檔案系統的 df——然後它會執行一個長達十五秒的 fio 隨機寫入基準測試

Checking if fio disk performance test can run safely...
Running fio disk performance test (15 seconds)...
etcd-test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B ...
etcd-test: Laying out IO file (1 file / 250MiB)

它建立了一個 250 MiB 的檔案,並在下方的執行中,於一個擁有 3.5 GiB 可用空間的檔案系統上寫入了 10.6 MiB 的隨機 I/O。它確實會預先檢查是否可以安全執行,因此並非魯莽之舉——但它並非讀取操作。

Terminal window
execcli collect-database-stats
Current ulimit for open files:
1024
Open FDs for etcd process:
75
Open connections count (via ss):
67
List open connections with lsof (if available):
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
etcd 1928 etcd-member cwd DIR 8,2 253 58407916 /
etcd 1928 etcd-member rtd DIR 8,2 253 58407916 /
etcd 1928 etcd-member txt REG 8,2 38599880 572772 /var/opt/bin/etcd
etcd 1928 etcd-member mem-W REG 8,2 16801792 34215070 /var/lib/etcd/member/snap/db
etcd 1928 etcd-member mem REG 8,2 2549360 52803074 /usr/lib64/libc.so.6
etcd 1928 etcd-member mem REG 8,2 70520 56912400 /usr/lib64/libresolv.so.2
etcd 1928 etcd-member mem REG 8,2 938792 54504007 /usr/lib64/ld-linux-x86-64.so.2
etcd 1928 etcd-member 0r CHR 1,3 0t0 4 /dev/null
etcd 1928 etcd-member 1u unix 0xffff892ac0fb0500 0t0 31273 type=STREAM (CONNECTED)
etcd 1928 etcd-member 2u unix 0xffff892ac0fb0500 0t0 31273 type=STREAM (CONNECTED)
etcd 1928 etcd-member 3u IPv6 19211 0t0 TCP *:23801 (LISTEN)
etcd 1928 etcd-member 4u a_inode 0,15 0 2078 [eventpoll:3,5,6,11,13,14,15,17,18,19,29,36,42,44,45,46,47,48,50,52,55,56,57,62,63,65,66,67,68,69,73,74...]
etcd 1928 etcd-member 5u a_inode 0,15 0 2078 [eventfd:19]
etcd 1928 etcd-member 6u IPv6 1005 0t0 TCP *:23791 (LISTEN)
etcd 1928 etcd-member 7uW REG 8,2 16801792 34215070 /var/lib/etcd/member/snap/db
etcd 1928 etcd-member 8u REG 8,2 64000000 50887796 /var/lib/etcd/member/wal/0000000000000000-0000000000000000.wal
etcd 1928 etcd-member 9r DIR 8,2 64 50887794 /var/lib/etcd/member/wal
etcd 1928 etcd-member 10w REG 8,2 64000000 50887804 /var/lib/etcd/member/wal/0.tmp
etcd 1928 etcd-member 11u IPv6 34289 0t0 TCP ce0179-exp-01:23791->ce0179-exp-01:49464 (ESTABLISHED)
Etcd data directory disk usage:
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 29G 25G 3.5G 88% /var
Etcd data directory location and usage:
total 4.0K
drwxr-xr-x 3 etcd-member etcd-member 20 Jul 29 01:27 .
drwxr-xr-x. 45 root root 4.0K Jul 29 01:41 ..
drwx------ 4 etcd-member etcd-member 29 Jul 29 01:27 member
Disk space for etcd:
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 29G 25G 3.5G 88% /var
Checking if fio disk performance test can run safely...
Running fio disk performance test (15 seconds)...
etcd-test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=sync, iodepth=1
fio-3.35
Starting 1 process
etcd-test: Laying out IO file (1 file / 250MiB)
Jobs: 1 (f=1)
Jobs: 1 (f=1)
Jobs: 1 (f=1): [w(1)][20.0%][w=720KiB/s][w=180 IOPS][eta 00m:12s]
Jobs: 1 (f=1): [w(1)][26.7%][w=724KiB/s][w=181 IOPS][eta 00m:11s]
Jobs: 1 (f=1): [w(1)][33.3%][w=732KiB/s][w=183 IOPS][eta 00m:10s]
Jobs: 1 (f=1): [w(1)][40.0%][w=744KiB/s][w=186 IOPS][eta 00m:09s]
Jobs: 1 (f=1): [w(1)][46.7%][w=740KiB/s][w=185 IOPS][eta 00m:08s]
Jobs: 1 (f=1): [w(1)][53.3%][w=716KiB/s][w=179 IOPS][eta 00m:07s]
Jobs: 1 (f=1): [w(1)][60.0%][w=720KiB/s][w=180 IOPS][eta 00m:06s]
Jobs: 1 (f=1): [w(1)][66.7%][w=728KiB/s][w=182 IOPS][eta 00m:05s]
Jobs: 1 (f=1): [w(1)][73.3%][w=772KiB/s][w=193 IOPS][eta 00m:04s]
Jobs: 1 (f=1): [w(1)][80.0%][w=752KiB/s][w=188 IOPS][eta 00m:03s]
Jobs: 1 (f=1): [w(1)][86.7%][w=752KiB/s][w=188 IOPS][eta 00m:02s]
Jobs: 1 (f=1): [w(1)][93.3%][w=728KiB/s][w=182 IOPS][eta 00m:01s]
Jobs: 1 (f=1): [w(1)][100.0%][w=728KiB/s][w=182 IOPS][eta 00m:00s]
etcd-test: (groupid=0, jobs=1): err= 0: pid=60246: Wed Jul 29 02:46:28 2026
write: IOPS=180, BW=722KiB/s (739kB/s)(10.6MiB/15002msec); 0 zone resets
clat (usec): min=8, max=270, avg=21.24, stdev=11.09
lat (usec): min=8, max=270, avg=21.68, stdev=11.14
clat percentiles (usec):
| 1.00th=[ 12], 5.00th=[ 16], 10.00th=[ 18], 20.00th=[ 19],
| 30.00th=[ 19], 40.00th=[ 20], 50.00th=[ 20], 60.00th=[ 20],
| 70.00th=[ 21], 80.00th=[ 22], 90.00th=[ 24], 95.00th=[ 40],
| 99.00th=[ 47], 99.50th=[ 60], 99.90th=[ 212], 99.95th=[ 212],
| 99.99th=[ 269]
bw ( KiB/s): min= 320, max= 792, per=100.00%, avg=723.03, stdev=80.72, samples=29
iops : min= 80, max= 198, avg=180.76, stdev=20.18, samples=29
lat (usec) : 10=0.33%, 20=64.34%, 50=34.59%, 100=0.41%, 250=0.30%
lat (usec) : 500=0.04%
fsync/fdatasync/sync_file_range:
sync (msec): min=4, max=206, avg= 5.51, stdev= 4.38
sync percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 5], 10.00th=[ 5], 20.00th=[ 5],
| 30.00th=[ 5], 40.00th=[ 5], 50.00th=[ 6], 60.00th=[ 6],
| 70.00th=[ 6], 80.00th=[ 6], 90.00th=[ 6], 95.00th=[ 8],
| 99.00th=[ 17], 99.50th=[ 18], 99.90th=[ 23], 99.95th=[ 57],
| 99.99th=[ 207]
cpu : usr=0.17%, sys=1.41%, ctx=8134, majf=0, minf=10
IO depths : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,2706,0,0 short=2706,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=722KiB/s (739kB/s), 722KiB/s-722KiB/s (739kB/s-739kB/s), io=10.6MiB (11.1MB), run=15002-15002msec
Disk stats (read/write):
sda: ios=0/5758, merge=0/5, ticks=0/23665, in_queue=30845, util=94.00%
NOTE: Pay attention to the 99th percentile latency (clat percentiles)
Healthy etcd disks should have p99 < 10ms
Systemctl status etcd-member:
● etcd-member.service - etcd server for kubernetes
Loaded: loaded (/etc/systemd/system/etcd-member.service; enabled; preset: disabled)
Active: active (running) since Wed 2026-07-29 01:27:40 UTC; 1h 18min ago
Docs: https://github.com/coreos/etcd
Main PID: 1927 (etcd-member-wra)
Tasks: 17 (limit: 204894)
Memory: 121.7M (peak: 133.5M)
CPU: 1min 9.639s
CGroup: /system.slice/etcd-member.service
├─ 1927 /bin/bash /opt/bin/etcd-member-wrapper.sh
├─ 1928 /opt/bin/etcd
└─60311 sleep 10
Jul 29 02:32:45 ce0179-exp-01 etcd-member-wrapper.sh[1928]: {"level":"info","ts":"2026-07-29T02:32:45.049479Z","caller":"mvcc/index.go:194","msg":"compact tree index","revision":8048}
Jul 29 02:32:45 ce0179-exp-01 etcd-member-wrapper.sh[1928]: {"level":"info","ts":"2026-07-29T02:32:45.074183Z","caller":"mvcc/kvstore_compaction.go:70","msg":"finished scheduled compaction","compact-revision":8048,"took":"23.934135ms","hash":1757010509,"current-db-size-bytes":11337728,"current-db-size":"11 MB","current-db-size-in-use-bytes":4833280,"current-db-size-in-use":"4.8 MB"}
Jul 29 02:32:45 ce0179-exp-01 etcd-member-wrapper.sh[1928]: {"level":"info","ts":"2026-07-29T02:32:45.074238Z","caller":"mvcc/hash.go:157","msg":"storing new hash","hash":1757010509,"revision":8048,"compact-revision":7750}
Jul 29 02:37:45 ce0179-exp-01 etcd-member-wrapper.sh[1928]: {"level":"info","ts":"2026-07-29T02:37:45.063095Z","caller":"mvcc/index.go:194","msg":"compact tree index","revision":8465}
Jul 29 02:37:45 ce0179-exp-01 etcd-member-wrapper.sh[1928]: {"level":"info","ts":"2026-07-29T02:37:45.133419Z","caller":"mvcc/kvstore_compaction.go:70","msg":"finished scheduled compaction","compact-revision":8465,"took":"69.493344ms","hash":942406775,"current-db-size-bytes":11337728,"current-db-size":"11 MB","current-db-size-in-use-bytes":4464640,"current-db-size-in-use":"4.5 MB"}
Jul 29 02:37:45 ce0179-exp-01 etcd-member-wrapper.sh[1928]: {"level":"info","ts":"2026-07-29T02:37:45.133480Z","caller":"mvcc/hash.go:157","msg":"storing new hash","hash":942406775,"revision":8465,"compact-revision":8048}
Jul 29 02:42:45 ce0179-exp-01 etcd-member-wrapper.sh[1928]: {"level":"info","ts":"2026-07-29T02:42:45.076782Z","caller":"mvcc/index.go:194","msg":"compact tree index","revision":8769}
Jul 29 02:42:45 ce0179-exp-01 etcd-member-wrapper.sh[1928]: {"level":"info","ts":"2026-07-29T02:42:45.106096Z","caller":"mvcc/kvstore_compaction.go:70","msg":"finished scheduled compaction","compact-revision":8769,"took":"28.595158ms","hash":696184126,"current-db-size-bytes":11337728,"current-db-size":"11 MB","current-db-size-in-use-bytes":4640768,"current-db-size-in-use":"4.6 MB"}
Jul 29 02:42:45 ce0179-exp-01 etcd-member-wrapper.sh[1928]: {"level":"info","ts":"2026-07-29T02:42:45.106156Z","caller":"mvcc/hash.go:157","msg":"storing new hash","hash":696184126,"revision":8769,"compact-revision":8465}
Jul 29 02:46:13 ce0179-exp-01 etcd-member-wrapper.sh[1928]: {"level":"info","ts":"2026-07-29T02:46:13.681521Z","caller":"traceutil/trace.go:172","msg":"trace[669642079] transaction","detail":"{read_only:false; response_revision:9395; number_of_response:1; }","duration":"126.62976ms","start":"2026-07-29T02:46:13.554875Z","end":"2026-07-29T02:46:13.681505Z","steps":["trace[669642079] 'process raft request' (duration: 126.508857ms)"],"step_count":1}

解讀方式:開啟檔案的 ulimit 與描述符計數是支援工程師在懷疑 etcd 耗盡限制時所要求的配對資訊,而 clat 百分位數則是基準測試的重點。磁碟統計列中的 util=94.00% 表示裝置在測試期間的忙碌程度——這不是檔案系統使用率的數字,兩者很容易混淆。

systemctl-restart-NetworkManagersystemctl-start-crio-prune 僅以名稱記錄,沒有擷取的輸出。

Terminal window
execcli systemctl-restart-NetworkManager
execcli systemctl-start-crio-prune

重新啟動 NetworkManager 會中斷節點的網路連線,而清除服務則會刪除容器映像。兩者都會變更節點,因此均未執行——即使是在可拋棄的節點上也未執行。