Customer Edge diagnostics

A Customer Edge (CE) is an F5 Distributed Cloud node deployed in your cloud or local environment. Start with the SMSv2 deployment guide for the verified Terraform lifecycle and AWS/Azure procedures. The access examples below describe the dated Azure deployment built from the volterraedgeservices/volterra-node marketplace image.

Each node runs the Argo data plane, the Vega control plane, an Envoy proxy, a Kubernetes stack, and vpm (the agent that handles node registration and process management).

When troubleshooting a CE, the Site CLI is the primary diagnostic tool. Selecting the correct access path is essential, as choosing the wrong path produces no output and can easily be mistaken for an unresponsive node.

Four access paths, and they are not interchangeable

Debug APIFor a node that is ONLINE. Scriptable, with a 43-command catalog and an enforced capture allowlist.

Site ConsoleF5 built-in per-node troubleshooting UI over an Azure Bastion tunnel. Requires no SSH key or jump host.

SSHThe complete on-box Site CLI. Requires a public key injected at first boot and network access within the VNet.

Serial consoleFor an unregistered node. The only remaining path when the node has no network connectivity.

The debug API communicates with the node through the F5 Distributed Cloud control plane. Consequently, the API responds only after the node successfully registers and transitions to ONLINE. If a node fails to register (for example, due to a cloud-init failure, an expired token, or an unreachable route to register.ves.volterra.io), the debug API cannot serve diagnostics.

Site Console provides F5’s built-in troubleshooting user interface rather than individual command execution. It requires minimal prerequisites—no SSH keys, jump hosts, or public IP addresses on the node—relying instead on Azure Role-Based Access Control (RBAC). It remains accessible whether or not the site has registered. However, it requires an Azure Bastion host, which is controlled by enable_bastion and disabled by default.

SSH provides access to the appliance’s complete command surface (exceeding the 43 commands exposed via the debug API). SSH access has two key requirements: sshd listens exclusively on the node’s internal Site Local Inside (SLI) address (requiring connectivity within the virtual network (VNet)), and SSH public keys must be injected via cloud-init during initial boot. Because vpm ignores the admin user’s ssh_key field on the site object after creation, enabling SSH on existing nodes requires node recreation.

In summary:

• Debug API: Use when the site is ONLINE and the standard debug catalog is sufficient.
• Site Console: Use for graphical troubleshooting or when a node has network reachability but failed registration.
• SSH: Use when full on-box command access is required from within the VNet.
• Serial console: Use as the last resort when the node has no network connectivity.

Check the site state first — and check you are asking the right tenant

One request identifies the current site state:

curl -sS -H "Authorization: APIToken $XCSH_API_TOKEN" \
  "$XCSH_API_URL/api/config/namespaces/system/sites/<site>" \
  | jq -r '.spec.site_state'

ONLINE indicates that the debug API is operational. Any other state requires the Site Console or serial console.

If a site does not exist, verify the target tenant before assuming a registration failure. The expected tenant is defined by var.expected_xc_tenant and can be inspected via cd terraform && terraform output -raw xc_tenant; its console is hosted at https://<XC_TENANT>.console.ves.volterra.io. A credential minted for a different tenant returns an HTTP 401 Unauthorized error, which can be easily misinterpreted as an expired token.

Safety rules

These rules apply to all commands across the diagnostics suite:

• Read-only by default. The command surface is split into two privilege tiers. The Exec tier either mutates node configuration or modifies state markers. The documentation and capture harnesses never execute Exec commands.
• Never assume safety from a command name. For example, ip-link-set resembles a query command but brings an interface down. Similarly, systemctl-restart-crio restarts the container runtime under an active data plane.
• Prefer the narrowest command that answers the question. health and diagnosis summarize node state efficiently; flow-l dumps all active flows, whereas flow-l-match targets a single connection.
• Customer Edges serve live traffic. In shared demonstration environments, always assume another user is actively presenting through the site under investigation.

Documented surfaces

This documentation covers every command reachable through the debug API on the software build deployed in this tenant, with output captured directly from live nodes:

• The command reference catalogs all commands by category, privilege tier, and transport.
• The workflows combine individual commands into operational troubleshooting sequences.
• The on-box reference catalogs the six top-level commands and 89 execcli subcommands exposed by the build. scripts/sitecli_ssh_harvest.py drives the interactive completion menu over SSH and captures the appliance’s built-in descriptions. Execution is default-denied, executing only an explicit allowlist.

In addition to command references:

• The interface model explains the three network interface cards (NICs) on each node: the role of the Site Local Outside (SLO) interface, the constraints of the Secure Mesh Site v2 management network, and why the mgmt NIC is not an out-of-band interface.
• The high-availability guide contrasts multi-node clustering with independent sites joined by Border Gateway Protocol (BGP) and Equal-Cost Multi-Path (ECMP), detailing route withdrawal, path limits, flow affinity, and session persistence.