The Terraform
Every file below is the file that deploys, imported at build time rather than pasted.
A copy that can drift is a copy that will, so a check in CI fails the moment what is
published here differs from what is in terraform/.
Root module
หัวข้อที่มีชื่อว่า “Root module”terraform/backend.tf
หัวข้อที่มีชื่อว่า “terraform/backend.tf”Azure storage backend; its configuration is supplied at init time and not committed.
terraform { # S3 remote state, configured as a PARTIAL backend: no account-specific # values are hardcoded here. Create the dedicated bucket with # bootstrap/state-backend first, then supply backend.hcl at init time. # # Local: terraform init -backend-config=backend.hcl (copy backend.hcl.example; gitignored) # CI: terraform init -backend=false (no state; config-validity + plan tests only) # # S3 native locking is enabled in backend.hcl; no DynamoDB lock table is # required. Authenticate with short-lived AWS credentials, never access keys # committed in a backend file. backend "s3" {}}terraform/data.tf
หัวข้อที่มีชื่อว่า “terraform/data.tf”Read-only lookups, including the Azure AD identity used to derive the deployer.
# Deployer identity resolution (read-only, azuread). An explicit deployer makes# the lookup unnecessary, which keeps AWS-only planning from contacting Azure.data "azuread_client_config" "current" { count = var.enable_azure && var.deployer == "" ? 1 : 0}
data "azuread_user" "current" { count = var.enable_azure && var.deployer == "" ? 1 : 0 object_id = data.azuread_client_config.current[0].object_id}terraform/kvm.tf
หัวข้อที่มีชื่อว่า “terraform/kvm.tf”Declarative KVM & libvirt infrastructure managed natively via dmacvicar/libvirt (v0.8.3). Provisions local network bridge ce-bgp-net (10.100.0.0/24), base cloud storage volumes, per-CE overlay disks, cloud-init seed ISOs containing /etc/vpm/config.yaml bootstrap configs, and KVM virtual machine domains (onprem-ce-01, 02, 03).
# KVM / libvirt Network for On-Prem Customer Edge nodes.## The CE addresses are routing identities: FRR peers with them and SMSv2 renders# one peer configuration across the site. Do not derive them from DHCP lease# ordering; a restart must not silently leave FRR peering with former tenants.locals { # The demo host has capacity for one production-sized CE plus its workload; # a three-node under-provisioned topology cannot establish a valid showcase. kvm_ce_nodes = { "01" = { address = "10.100.0.11", mac = "52:54:00:10:00:11" } } kvm_workload_node = { address = "10.100.0.100", mac = "52:54:00:10:00:64" } kvm_network_hosts = merge(local.kvm_ce_nodes, { workload = local.kvm_workload_node }) kvm_image_cache_dir = pathexpand("~/.cache/multi-cloud-networking/kvm") kvm_pool_name = "mcn-kvm-showcase"
kvm_network_generation = substr(sha256(jsonencode(local.kvm_network_hosts)), 0, 8) kvm_network_name = "ce-bgp-net-${local.kvm_network_generation}" # Linux bridge device names are limited to 15 bytes. kvm_network_bridge = "vbgp-${local.kvm_network_generation}" kvm_bootstrap_generation = var.enable_kvm ? nonsensitive(substr(sha256(xcsh_token.kvm[0].uid), 0, 8)) : "disabled" kvm_enabled_nodes = var.enable_kvm ? local.kvm_ce_nodes : {}}
# Provider refresh cannot reconcile dnsmasq host entries after a libvirt-side# reservation drift. A changed declarative identity generation must therefore# replace the network, which transitively tears down and recreates dependent CE# domains and the FRR fabric in Terraform order.resource "terraform_data" "kvm_network_identity" { count = var.enable_kvm ? 1 : 0
input = sha256(jsonencode(local.kvm_network_hosts))}
# The Sales Demo tenant issues the currently supported KVM CE appliance as a# signed image URL. Never substitute a generic cloud OS: it has no VPM runtime.data "xcsh_site_image" "kvm" { count = var.enable_kvm && var.kvm_lan_configuration_phase != "configured" ? 1 : 0
site_name = local.kvm_site_name depends_on = [xcsh_securemesh_site_v2.onprem_kvm]}
data "xcsh_site_image" "kvm_configured" { count = var.enable_kvm && var.kvm_lan_configuration_phase == "configured" ? 1 : 0 site_name = local.kvm_site_name}
data "xcsh_site_cloud_init" "kvm" { count = var.enable_kvm && var.kvm_lan_configuration_phase != "configured" ? 1 : 0
provider_ref = "kvm" site_name = local.kvm_site_name enable_management_network = false depends_on = [xcsh_securemesh_site_v2.onprem_kvm]}
data "xcsh_site_cloud_init" "kvm_configured" { count = var.enable_kvm && var.kvm_lan_configuration_phase == "configured" ? 1 : 0 provider_ref = "kvm" site_name = local.kvm_site_name enable_management_network = false}
locals { kvm_site_image = var.enable_kvm ? (var.kvm_lan_configuration_phase == "configured" ? data.xcsh_site_image.kvm_configured[0] : data.xcsh_site_image.kvm[0]) : null kvm_site_cloud_init = var.enable_kvm ? (var.kvm_lan_configuration_phase == "configured" ? data.xcsh_site_cloud_init.kvm_configured[0] : data.xcsh_site_cloud_init.kvm[0]) : null}
resource "libvirt_pool" "kvm" { count = var.enable_kvm ? 1 : 0 name = local.kvm_pool_name type = "dir" target { path = "/var/lib/libvirt/images/${local.kvm_pool_name}" }}
resource "terraform_data" "kvm_ce_image_cache" { count = var.enable_kvm ? 1 : 0 triggers_replace = [local.kvm_site_image.image_md5_sum] provisioner "local-exec" { command = "../scripts/ensure-verified-kvm-image.sh --url \"$IMAGE_URL\" --digest \"md5:$IMAGE_MD5\" --destination \"$IMAGE_DESTINATION\"" working_dir = path.root environment = { IMAGE_URL = local.kvm_site_image.image_download_url IMAGE_MD5 = local.kvm_site_image.image_md5_sum IMAGE_DESTINATION = "${local.kvm_image_cache_dir}/f5xc-${local.kvm_site_image.image_md5_sum}.qcow2" } }}resource "libvirt_network" "ce_bgp_net" { count = var.enable_kvm ? 1 : 0
name = local.kvm_network_name mode = "nat" domain = "ce.local" addresses = ["10.100.0.0/24"]
bridge = local.kvm_network_bridge
autostart = true
dhcp { enabled = true }
dnsmasq_options { dynamic "options" { for_each = local.kvm_network_hosts content { option_name = "dhcp-host" option_value = "${options.value.mac},${options.value.address}" } } }
dns { enabled = true # Keep the libvirt search domain authoritative. During CE bootstrap, # gRPC service-config lookups append this suffix; forwarding those misses # can exhaust dnsmasq and stall the Vector -> Vega dependency chain. local_only = true }
lifecycle { replace_triggered_by = [terraform_data.kvm_network_identity[0]] }}
# Base cloud OS image volume in libvirtresource "libvirt_volume" "base_cloud" { count = var.enable_kvm ? 1 : 0
name = "f5xc-kvm-ce-${local.kvm_site_image.image_md5_sum}.qcow2" pool = libvirt_pool.kvm[0].name source = "${local.kvm_image_cache_dir}/f5xc-${local.kvm_site_image.image_md5_sum}.qcow2" format = "qcow2" depends_on = [terraform_data.kvm_ce_image_cache]}
# Per-CE root overlay disksresource "libvirt_volume" "ce_disk" { for_each = local.kvm_enabled_nodes name = "onprem-ce-${each.key}-${local.kvm_network_generation}-${local.kvm_bootstrap_generation}-disk.qcow2" pool = libvirt_pool.kvm[0].name base_volume_id = libvirt_volume.base_cloud[0].id size = 107374182400 format = "qcow2"}
# Cloud-Init ISO seed disks per CE noderesource "libvirt_cloudinit_disk" "ce_cloudinit" { for_each = local.kvm_enabled_nodes name = "onprem-ce-${each.key}-${local.kvm_network_generation}-${local.kvm_bootstrap_generation}-cloudinit.iso" pool = libvirt_pool.kvm[0].name # The provider returns the modern /etc/vpm/user_data template. It has the # lowercase placeholder exactly once; this CE receives its own type-1 JWT. user_data = replace( local.kvm_site_cloud_init.cloud_init_config, "{{ .token }}", xcsh_token.kvm[0].uid, )
meta_data = <<-EOF instance-id: onprem-ce-${each.key}-${local.kvm_bootstrap_generation} local-hostname: onprem-ce-${each.key} EOF
}
# Declarative KVM Virtual Machines managed by Terraformresource "libvirt_domain" "ce_node" { for_each = local.kvm_enabled_nodes name = "onprem-ce-${each.key}" memory = 32768 vcpu = 8 autostart = true
cloudinit = libvirt_cloudinit_disk.ce_cloudinit[each.key].id
cpu { mode = "host-passthrough" }
network_interface { network_id = libvirt_network.ce_bgp_net[0].id mac = each.value.mac wait_for_lease = false }
# libvirt 0.8.3 realizes added NICs only during domain creation. Enabling # this block therefore requires the reviewed full-root replacement plan # enforced by scripts/kvm-lan-plan-scope.py; an update-only plan is rejected. dynamic "network_interface" { for_each = var.enable_kvm_lan && var.kvm_lan != null ? [var.kvm_lan] : [] content { bridge = network_interface.value.bridge mac = network_interface.value.sli_mac wait_for_lease = false } }
disk { volume_id = libvirt_volume.ce_disk[each.key].id }
console { type = "pty" target_port = "0" target_type = "serial" }
graphics { type = "vnc" listen_type = "address" autoport = true }
# A changed seed ISO is not consumed by an already-running CE. Replace the # domain so cloud-init applies the declared MAC and static routing identity # on first boot rather than leaving an old DHCP lease in the BGP fabric. lifecycle { replace_triggered_by = [ libvirt_cloudinit_disk.ce_cloudinit[each.key], libvirt_network.ce_bgp_net[0], ] }}terraform/locals.tf
หัวข้อที่มีชื่อว่า “terraform/locals.tf”Where the derived object names live. Terraform variable defaults cannot reference other variables, so each name variable defaults to null and is resolved here.
locals { # --- Deployment identity and provenance --- # Keep this byte serialization in lockstep with scripts/deployment-identity.py. deployment_identity_schema = "mcn.deployment-identity/v1" source_branch = trimprefix(var.source_ref, "refs/heads/") source_ref_sha256 = sha256("${local.deployment_identity_schema}\u0000${var.source_repository}\u0000${var.source_ref}") source_branch_slug_raw = trim(replace(lower(local.source_branch), "/[^a-z0-9]+/", "-"), "-") source_branch_slug = local.source_branch_slug_raw != "" ? local.source_branch_slug_raw : "branch" deployment_is_production = var.source_ref == "refs/heads/main" deployment_environment_key = local.deployment_is_production ? "production" : "${trimsuffix(substr(local.source_branch_slug, 0, 19), "-")}-${substr(local.source_ref_sha256, 0, 12)}" deployment_name_suffix = local.deployment_is_production ? "" : "-${local.deployment_environment_key}" deployment_short_suffix = local.deployment_is_production ? "" : "-${substr(local.source_ref_sha256, 0, 12)}" showcase_backend_key = local.deployment_is_production ? "mcn-ce-ha-smsv2/showcase.tfstate" : "mcn-ce-ha-smsv2/environments/${local.deployment_environment_key}/showcase.tfstate" recovery_backend_key = local.deployment_is_production ? "mcn-ce-ha-smsv2/recovery/smsv2-orphans.tfstate" : "mcn-ce-ha-smsv2/environments/${local.deployment_environment_key}/recovery/smsv2-orphans.tfstate" deployment_artifact_scope = local.deployment_is_production ? "production" : "preview/${local.deployment_environment_key}"
# --- F5 XC tenant endpoint --- # Every F5 XC tenant is served at https://<tenant>.console.ves.volterra.io, so # naming the tenant is enough to name the API. providers.tf feeds this to the # xcsh provider's api_url, which is what makes the TENANT A PROPERTY OF THE # CONFIGURATION rather than of whatever XCSH_API_URL the shell happens to hold. xc_api_url = "https://${var.expected_xc_tenant}.console.ves.volterra.io"
# --- Deployer resolution (4-tier fallback) --- # 1. Explicit override via var.deployer # 2a. Azure AD: given_name initial + surname # 2b. Azure AD: mail prefix (guest/external accounts) # 3. Object ID hash (service principals, managed identities) deployer_from_name = ( var.deployer == "" && length(data.azuread_user.current) > 0 ? try( lower("${substr(data.azuread_user.current[0].given_name, 0, 1)}${data.azuread_user.current[0].surname}"), "" ) : "" )
deployer_from_mail = ( var.deployer == "" && length(data.azuread_user.current) > 0 && local.deployer_from_name == "" ? try( lower(split("@", data.azuread_user.current[0].mail)[0]), "" ) : "" )
deployer_from_oid = var.deployer == "" && length(data.azuread_client_config.current) > 0 ? try(substr(sha1(data.azuread_client_config.current[0].object_id), 0, 8), "") : ""
deployer_resolved = coalesce( var.deployer, local.deployer_from_name, local.deployer_from_mail, local.deployer_from_oid )
deployer = replace(lower(local.deployer_resolved), "/[^a-z0-9]/", "")
# --- Derived object names --- # Every name in the deployment descends from var.component (plus the resolved # deployer for the resource group, which is per-person by nature). Terraform # variable defaults cannot reference other variables, so each of these variables # defaults to null and is resolved here instead; an explicit value always wins. # # The point is that NO object name is a literal anyone has to maintain, and none # can carry a customer's or an individual's name by accident: change # var.component and the sites, load balancer, origin pool, Route Server, Bastion # and resource group all follow. region_short = coalesce(var.region_short, var.location) # SMSv2 site identities have their own immutable generation. The previous # mcn-ce-ha-* generation has unrecoverable generic-name reservations in Sales # Demo, so a fresh complete showcase must never attempt to recreate it. site_prefix_base = coalesce(var.site_prefix, "${var.component}-${var.smsv2_site_generation}") # XC site names add the AWS region, node number, and optional `-bootstrap`. # Bound the readable prefix so the longest validated name remains at the # XC's 63-character DNS-1035 limit while retaining the full identity in state. preview_site_prefix = "mcn-${trimsuffix(substr(local.source_branch_slug, 0, 14), "-")}-${substr(local.source_ref_sha256, 0, 12)}" site_prefix = local.deployment_is_production ? local.site_prefix_base : local.preview_site_prefix # AWS has account-global names for key pairs, IAM identities, and ELBv2 # objects. Keep them in the same immutable generation as the site names so # a clean deployment cannot collide with stale component-only resources. # AWS NLB names allow only 32 characters. The 12-hex identity keeps the # longest generated name within that limit without weakening state identity. aws_resource_prefix = local.deployment_is_production ? local.site_prefix : "mcn${local.deployment_short_suffix}" resource_group_name = "${coalesce(var.resource_group_name, "rg-${var.component}-${local.deployer}")}${local.deployment_name_suffix}" route_server_name = "${coalesce(var.route_server_name, "${var.component}-rs")}${local.deployment_name_suffix}" bastion_name = "${coalesce(var.bastion_name, "${var.component}-bastion")}${local.deployment_name_suffix}" client_vm_name = "${coalesce(var.client_vm_name, "${var.component}-client")}${local.deployment_name_suffix}" origin_pool_name = "${coalesce(var.origin_pool_name, "${var.component}-pool")}${local.deployment_name_suffix}" # `-f5se` matches the convention this tenant's other load balancers already use. lb_name = "${coalesce(var.lb_name, "${var.component}-f5se")}${local.deployment_name_suffix}" lb_domain = local.deployment_is_production ? var.lb_domain : "${local.deployment_environment_key}.${var.lb_domain}"
# --- Derived Canada object names --- ca_region_short = coalesce(var.ca_region_short, var.ca_location) ca_site_prefix_base = coalesce(var.ca_site_prefix, "${local.site_prefix_base}-ca") ca_site_prefix = local.deployment_is_production ? local.ca_site_prefix_base : "${local.site_prefix}-ca" kvm_site_name = "${local.site_prefix}-kvm" ca_resource_group_name = "${coalesce(var.ca_resource_group_name, "rg-${var.component}-ca-${local.deployer}")}${local.deployment_name_suffix}" ca_route_server_name = "${coalesce(var.ca_route_server_name, "${var.component}-ca-rs")}${local.deployment_name_suffix}" ca_bastion_name = "${coalesce(var.ca_bastion_name, "${var.component}-ca-bastion")}${local.deployment_name_suffix}" ca_client_vm_name = "${coalesce(var.ca_client_vm_name, "${var.component}-ca-client")}${local.deployment_name_suffix}" ca_origin_pool_name = "${coalesce(var.ca_origin_pool_name, "${var.component}-ca-pool")}${local.deployment_name_suffix}" ca_lb_name = "${coalesce(var.ca_lb_name, "${var.component}-ca-f5se")}${local.deployment_name_suffix}" ca_re_vsite_name = "${coalesce(var.ca_re_vsite_name, "${var.component}-ca-re-vsite")}${local.deployment_name_suffix}" ca_ce_vsite_name = "${coalesce(var.ca_ce_vsite_name, "${var.component}-ca-ce-vsite")}${local.deployment_name_suffix}" ca_lb_domain = local.deployment_is_production ? var.ca_lb_domain : "${local.deployment_environment_key}.${var.ca_lb_domain}" aws_lb_domain = local.deployment_is_production ? var.aws_lb_domain : "${local.deployment_environment_key}.${var.aws_lb_domain}"
# --- Standard tags (applied to every Azure resource) --- standard_tags = { component = var.component environment = var.environment deployer = local.deployer managed_by = "terraform" mcn_environment = local.deployment_environment_key mcn_repository = "multi-cloud-networking" mcn_source_ref_sha256 = local.source_ref_sha256 mcn_source_commit = var.source_commit_sha mcn_owner_id = var.deployment_owner_id mcn_actor_id = var.deployment_actor_id }
# Provenance is protected: caller-supplied tags may add metadata but cannot # falsify keys Terraform owns. tags = merge(var.tags, local.standard_tags)
# F5 objects use the same immutable generation and tenant ownership identity # as AWS and KVM resources in this one-state deployment. xc_provenance_labels = { "mcn-deployment-generation" = var.smsv2_site_generation "mcn-environment" = local.deployment_environment_key "mcn-source-ref-sha256" = substr(local.source_ref_sha256, 0, 32) "mcn-source-commit" = var.source_commit_sha "mcn-owner-id" = var.deployment_owner_id "mcn-actor-id" = var.deployment_actor_id "mcn-xc-tenant" = var.expected_xc_tenant } xc_labels = merge(local.xc_provenance_labels, { "mcn-topology" = "${local.site_prefix}-aws" }) azure_xc_labels = merge(local.xc_provenance_labels, { "mcn-topology" = "${local.site_prefix}-azure" }) ca_xc_labels = merge(local.xc_provenance_labels, { "mcn-topology" = "${local.ca_site_prefix}-azure" }) kvm_xc_labels = merge(local.xc_labels, { "mcn-topology" = "${local.site_prefix}-kvm" }) kvm_token_labels = { for key, value in local.kvm_xc_labels : key => value if key != "mcn-source-commit" }
# --- SSH public key material, read once at the root --- # When ssh_public_key material is supplied (e.g. by the plan tests) it wins and # no file is read; otherwise read the key file once and pass the string down. ssh_public_key = var.ssh_public_key != "" ? var.ssh_public_key : file(pathexpand(var.ssh_public_key_path))
# --- CE site registration token fed to cloud-init --- # Prefer the provider-generated xcsh_token.ce[0].uid (the Computed token VALUE); # an explicit var.registration_token still wins when supplied (break-glass / # externally-minted token). Empty var (default) => the generated token. ce_registration_token = var.registration_token != "" ? var.registration_token : try(xcsh_token.ce[0].uid, null)
# --- CE cloud-init, rendered once per node --- # Rendered here rather than inline in the module block so the document is # addressable as local.ce_cloud_init in `terraform test` — the rendered YAML is # the whole contract with the appliance, and it is only worth asserting if it can # be read. See tests/cloud_init.tftest.hcl. ce_cloud_init = { for key, node in module.ce_topology.ce_nodes : key => templatefile("${path.module}/cloud-init/ce-node.yaml", { cluster_name = node.site_name token = local.ce_registration_token # chomp: a key read from a .pub file ends in a newline, which would render a # second, empty line into authorized_keys under `content: |`. ssh_public_key = chomp(local.ssh_public_key) }) }
# --- Canada CE cloud-init, rendered once per node --- ca_ce_cloud_init = { for key, node in try(module.ce_topology_ca[0].ce_nodes, {}) : key => templatefile("${path.module}/cloud-init/ce-node.yaml", { cluster_name = node.site_name token = local.ce_registration_token ssh_public_key = chomp(local.ssh_public_key) }) }}terraform/main.tf
หัวข้อที่มีชื่อว่า “terraform/main.tf”Top-level wiring, the tenant guard and the VIP-outside-the-VNet check. The deploy ordering is documented at the top of this file, which is the authoritative version.
# MCN SMSv2 multi-site showcase — top-level wiring.## The supported Azure path uses an Internal Load Balancer to reach CE Site Console# health endpoints. Azure Route Server is opt-in and deliberately fails closed until# the immutable F5 contract supplies writable eBGP multihop semantics.## Deploy-time ordering: Azure (VNet/subnets/RS/NICs/VMs) -> XC site (explicit# interface) -> token -> CE cloud-init boot -> CE registers -> registration# approval -> CE ONLINE -> xcsh_bgp + RS bgpConnection -> LB advertise.## Approval is automated, but the deploy is inherently TWO-PHASE, not a single# hands-off apply: a CE's registration is named r-<uuid> and only exists after# the node has booted and registered. The xc-site module resolves that name with# the xcsh_site_registration data source and approves it with# xcsh_registration_approval, gated on found — so the first apply plans no# approval, and a re-apply once the CEs have registered creates them. The bgp/LB# objects can be applied before ONLINE; they converge once the CE is up.
# Guard: the F5 XC tenant in the environment MUST be the one this deployment# belongs to.## providers.tf already pins the xcsh endpoint to var.expected_xc_tenant, so a# stray XCSH_API_URL can no longer redirect the deployment. What it can still do# is mean the operator's TOKEN belongs to a different tenant — and the symptom of# that is a bare 401 from the first API call, which reads like an expired# credential and not like "you are pointed at the wrong tenant". This turns it# into a sentence that says so.## It is not hypothetical. The MCN demo was built in f5-sales-demo; the credential# file later started exporting an f5-amer-ent XCSH_API_URL; subsequent applies# minted an f5-amer-ent token, the CE VMs re-registered there, and their# f5-sales-demo registrations were abandoned. Every plan in between was clean,# because nothing in the configuration had an opinion about the tenant (#696).## The external program reads the environment and nothing else — no credential, no# network call — which is what keeps `terraform test` and CI's# `terraform init -backend=false` credential-free. Where XCSH_API_URL is unset the# program reports an empty tenant and the guard abstains.## postcondition, not check{}: a check block only WARNS, and a warning scrolls past# in exactly the situation this exists to stop.data "external" "xc_env_tenant" { program = ["${path.module}/scripts/xc-env-tenant.sh"]
lifecycle { postcondition { condition = contains(["", var.expected_xc_tenant], self.result.tenant) error_message = "Wrong F5 XC tenant. XCSH_API_URL in this environment names tenant '${self.result.tenant}', but this deployment belongs to '${var.expected_xc_tenant}'. Source the credential file for '${var.expected_xc_tenant}', or — if you really do mean to act on '${self.result.tenant}' — say so explicitly with -var expected_xc_tenant=${self.result.tenant} and an isolated AWS S3 state key of its own." } }}
# KVM uses one explicitly allocated host network. Branch naming cannot make its# subnet, MACs, FRR identity, or host capacity safe to share. Reject a preview# before any mutation until a separately reviewed allocation contract exists.resource "terraform_data" "deployment_identity_guard" { input = local.deployment_environment_key
lifecycle { precondition { condition = local.deployment_is_production || !var.enable_kvm error_message = "KVM previews are unsupported without a separately approved subnet, MAC, bridge, FRR, and capacity allocation; disable KVM or deploy exact refs/heads/main." } precondition { condition = length(local.deployment_environment_key) <= 32 error_message = "deployment environment key exceeds its 32-character contract." } precondition { condition = length(local.lb_domain) <= 253 && length(local.ca_lb_domain) <= 253 && length(local.aws_lb_domain) <= 253 error_message = "environment-scoped load-balancer domain exceeds the DNS limit." } }}
# Azure Route Server requires eBGP multihop, but the immutable SMSv2 contract# currently supplies no schema-valid request control for it. Keeping this# requirement in a data source validates it during planning, before Terraform# can evaluate any Azure or F5 resource mutation.# tflint-ignore: terraform_unused_declarationsdata "xcsh_smsv2_contract" "azure_route_server" { count = var.enable_azure && var.enable_bgp ? 1 : 0
# Provider configuration validation runs before count is expanded. Keep the # unavailable Azure-only requirement absent in KVM/AWS-only plans as well. required_capabilities = var.enable_azure && var.enable_bgp ? ["azure_route_server_ebgp_multihop"] : []}
# Guard: the HA VIP MUST be outside every VNet CIDR, or Azure prefers the VNet# system route over the more-specific BGP /32. Masks the VIP to each CIDR's prefix# length and compares network addresses (a correct containment test for any prefix).check "vip_outside_vnet_cidrs" { assert { condition = cidrhost(var.hub_cidr, 0) != cidrhost("${var.vip}/${split("/", var.hub_cidr)[1]}", 0) error_message = "vip ${var.vip} must be OUTSIDE hub_cidr ${var.hub_cidr}." } assert { condition = cidrhost(var.spoke_cidr, 0) != cidrhost("${var.vip}/${split("/", var.spoke_cidr)[1]}", 0) error_message = "vip ${var.vip} must be OUTSIDE spoke_cidr ${var.spoke_cidr}." }}
check "ca_vip_outside_vnet_cidrs" { assert { condition = !var.enable_canada || cidrhost(var.ca_hub_cidr, 0) != cidrhost("${var.ca_vip}/${split("/", var.ca_hub_cidr)[1]}", 0) error_message = "ca_vip ${var.ca_vip} must be OUTSIDE ca_hub_cidr ${var.ca_hub_cidr}." }}
# Tenant-scoped, reusable Azure site registration token. The provider now ships# xcsh_token with a Computed `uid` (system_metadata.uid) — the token VALUE a CE# feeds to VPM at registration (the resource `id` is the token NAME, not the# value). The spec is empty; only metadata is needed and namespace defaults to# system. This replaces the manual var.registration_token prerequisite (#1205).# The console approval step (#1206 / #1210) is likewise gone: each CE's runtime# registration is resolved by site name and approved in the post-registration# phase (see modules/xc-site/main.tf and the deploy ordering above).resource "xcsh_token" "ce" { count = local.azure_provider_enabled ? 1 : 0
name = "${local.site_prefix}-registration" namespace = "system" description = "MCN CE-HA registration token (tenant-scoped, reusable across CE sites)" labels = local.azure_xc_labels}
# Pure expansion of ce_count into the per-CE node map (hostname, site_name,# slo_ip, az, interface_name). Drives every for_each below.module "ce_topology" { source = "./modules/ce-topology"
ce_count = var.ce_count enabled = var.enable_azure region_short = local.region_short mgmt_subnet_prefix = var.mgmt_subnet_prefix site_prefix = local.site_prefix}
# Hub: RG, VNet, and CE subnets. Route Server is created only for the explicitly# requested (and currently rejected) BGP topology.module "azure_hub" { source = "./modules/azure-hub" count = var.enable_azure ? 1 : 0
depends_on = [azapi_resource_action.f5xc_customer_edge_marketplace_agreement]
resource_group_name = local.resource_group_name location = var.location hub_cidr = var.hub_cidr mgmt_subnet_prefix = var.mgmt_subnet_prefix external_subnet_prefix = var.external_subnet_prefix internal_subnet_prefix = var.internal_subnet_prefix route_server_subnet_prefix = var.route_server_subnet_prefix route_server_name = local.route_server_name enable_route_server = var.enable_bgp bastion_subnet_prefix = var.bastion_subnet_prefix enable_bastion = var.enable_bastion bastion_name = local.bastion_name tags = local.tags}
# One CE VM (3 NICs + identity) per node.module "ce_node" { source = "./modules/ce-node" for_each = module.ce_topology.ce_nodes
hostname = each.value.hostname resource_group_name = module.azure_hub[0].resource_group_name location = module.azure_hub[0].location zone = each.value.az vm_size = var.ce_vm_size mgmt_subnet_id = module.azure_hub[0].management_subnet_id external_subnet_id = module.azure_hub[0].external_subnet_id internal_subnet_id = module.azure_hub[0].internal_subnet_id mgmt_private_ip = each.value.slo_ip admin_username = var.admin_username ssh_public_key = local.ssh_public_key
custom_data = base64encode(local.ce_cloud_init[each.key])
tags = local.tags}
# Generate a distinct Site Console admin password for every CE. Binding the# password lifecycle to the VM instance rotates it automatically whenever that# node is rebuilt, while keeping the value only in the encrypted remote state# and sensitive Terraform output.resource "random_password" "site_console_admin" { for_each = module.ce_topology.ce_nodes
length = 32 min_lower = 1 min_numeric = 1 min_special = 1 min_upper = 1 override_special = "!#%*+-=?@^_~"
keepers = { ce_vm_instance_id = module.ce_node[each.key].vm_instance_id }}
# Site Console Basic Auth delegates password verification to VPM on the CE host.# The XC site's admin_user_credentials field is accepted by the API but VPM# deliberately skips the built-in admin user, so it does not rotate this# credential. Apply the generated value on the node as root instead.## The Custom Script extension receives only an encrypted protected setting. Its# inline script keeps the password out of command arguments and produces no# output. The password resource is keyed by Azure's per-instance VM id, so a VM# replacement generates a new value and updates this extension even though the# name-derived ARM resource id stays the same.resource "azurerm_virtual_machine_extension" "site_console_password" { for_each = module.ce_topology.ce_nodes
name = "site-console-admin-password" virtual_machine_id = module.ce_node[each.key].vm_id publisher = "Microsoft.Azure.Extensions" type = "CustomScript" type_handler_version = "2.1" auto_upgrade_minor_version = true
protected_settings = jsonencode({ script = base64encode(<<-SCRIPT #!/usr/bin/env bash set -euo pipefail password_b64='${base64encode(random_password.site_console_admin[each.key].result)}' password=$(printf '%s' "$password_b64" | base64 --decode) printf 'admin:%s\n' "$password" | chpasswd unset password password_b64 SCRIPT ) })}
# One XC SMSv2 site + bgp object per node. The MAC is wired from the mgmt NIC so# a NIC recreate updates the site binding automatically.module "xc_site" { source = "./modules/xc-site" for_each = module.ce_topology.ce_nodes
site_name = each.value.site_name hostname = each.value.hostname interface_name = each.value.interface_name mgmt_nic_mac = module.ce_node[each.key].mgmt_nic_mac # Couples the site object's lifecycle to the CE VM INSTANCE (issue #674): # replacing the VM replaces the site, which takes the registration bound to the # destroyed instance with it. Must be virtual_machine_id, not the ARM resource # id — the latter is name-derived and identical after a replacement. ce_vm_instance_id = module.ce_node[each.key].vm_instance_id rs_peer_ips = module.azure_hub[0].rs_peer_ips ce_asn = var.ce_asn rs_asn = var.rs_asn os_version = var.ce_os_version sw_version = var.ce_sw_version enable_bgp = var.enable_bgp approve_registration = var.approve_registration labels = local.azure_xc_labels}
# The Azure side of each eBGP session (Route Server -> CE eth0/SLO IP).module "azure_route_server_bgp" { source = "./modules/azure-route-server-bgp" for_each = var.enable_azure && var.enable_bgp ? module.ce_topology.ce_nodes : {}
name = "${each.key}-bgp" route_server_id = module.azure_hub[0].route_server_id peer_asn = var.ce_asn peer_ip = module.ce_node[each.key].mgmt_private_ip}
# Test client in snet-hub-internal.module "client_vm" { source = "./modules/client-vm" count = var.enable_azure ? 1 : 0
name = local.client_vm_name resource_group_name = module.azure_hub[0].resource_group_name location = module.azure_hub[0].location subnet_id = module.azure_hub[0].internal_subnet_id admin_username = var.admin_username ssh_public_key = local.ssh_public_key tags = local.tags}
# ---------------------------------------------------------# F5 XC data-plane (app tier)# ---------------------------------------------------------
# The application namespace (origin pool + VIP load balancer live here) is an# input, never an owned object. `multi-cloud-networking` predates this deployment# and outlives it, and a `resource` block would put a namespace full of other# people's demos on this stack's destroy list — which is how issue #637 happened.## Issues #634 ("cannot be recreated, POST is 403") and #639 ("defaults to a# namespace that no longer exists") were symptoms of reading the WRONG TENANT, not# of a permissions or lifecycle problem: the namespace exists, in f5-sales-demo,# and always did. The tenant guard at the top of this file is the actual fix (#696).# Reading rather than owning the namespace is still correct, for the #637 reason.## Reading it means Terraform never creates, updates or destroys it. The# read still gives the pool and the load balancer their apply-time ordering: they# take their namespace from this data source, so a missing namespace fails the# plan loudly rather than half-applying. `namespace` is empty because a namespace# object is not itself contained in a namespace (the API returns# `metadata.namespace: ""` for one).data "xcsh_namespace" "mcn" { name = var.xc_app_namespace namespace = ""}
resource "xcsh_origin_pool" "this" { count = var.enable_azure ? 1 : 0 name = local.origin_pool_name namespace = data.xcsh_namespace.mcn.name description = "MCN reference origin pool -> ${var.origin_ip}:${var.origin_port}" labels = local.azure_xc_labels
port = var.origin_port
origin_servers { labels = {} public_ip { ip = var.origin_ip } }
no_tls = {} loadbalancer_algorithm = "ROUND_ROBIN" endpoint_selection = "DISTRIBUTED"}
resource "xcsh_http_loadbalancer" "this" { count = var.enable_azure ? 1 : 0 # The advertise_custom block below names each CE site, but it takes those names from # module.ce_topology — a pure computation module that only derives strings. The # objects themselves come from module.xc_site, and nothing in this resource # references them, so Terraform sees NO dependency and is free to create the load # balancer before any site exists. XC then rejects the dangling site reference with # `[BAD_REQUEST] Invalid request parameters` on POST .../http_loadbalancers. # # The bug is invisible while the sites already exist under the names being # referenced, which is why it stayed latent until the demo was renamed: that # destroyed every site and created new ones, and the load balancer raced ahead of # them. Making the dependency explicit is the fix; there is no cycle, because # xc-site does not reference the load balancer. depends_on = [module.xc_site]
name = local.lb_name namespace = data.xcsh_namespace.mcn.name description = "BGP/ECMP HA: custom VIP ${var.vip} advertised from every CE site." labels = local.azure_xc_labels
domains = [local.lb_domain]
http { port = 80 }
# Advertise the VIP on the outside network of every CE site. advertise_custom { dynamic "advertise_where" { for_each = module.ce_topology.ce_nodes content { site { network = "SITE_NETWORK_OUTSIDE" site { namespace = "system" name = advertise_where.value.site_name } ip = var.vip } use_default_port = {} } } }
default_route_pools { pool { namespace = data.xcsh_namespace.mcn.name name = xcsh_origin_pool.this[0].name } weight = 1 priority = 1 }
round_robin = {} no_challenge = {} user_id_client_ip = {} disable_waf = {} disable_rate_limit = {} disable_api_discovery = {} disable_api_testing = {} disable_api_definition = {} l7_ddos_protection {} service_policies_from_namespace = {} disable_trust_client_ip_headers = {} disable_malicious_user_detection = {} disable_malware_protection = {} disable_threat_mesh = {} default_sensitive_data_policy = {}}
# ---------------------------------------------------------# Canada Regional Infrastructure & F5 XC Data-Plane (Canada Path)# ---------------------------------------------------------
# Pure expansion of ca_ce_count into the per-CE node map for Canada.module "ce_topology_ca" { count = var.enable_azure && var.enable_canada ? 1 : 0 source = "./modules/ce-topology"
ce_count = var.ca_ce_count region_short = local.ca_region_short mgmt_subnet_prefix = var.ca_mgmt_subnet_prefix site_prefix = local.ca_site_prefix}
# Canada Hub: RG, VNet, and CE subnets; Route Server remains opt-in.module "azure_hub_ca" { count = var.enable_azure && var.enable_canada ? 1 : 0 source = "./modules/azure-hub"
depends_on = [azapi_resource_action.f5xc_customer_edge_marketplace_agreement]
resource_group_name = local.ca_resource_group_name location = var.ca_location hub_cidr = var.ca_hub_cidr mgmt_subnet_prefix = var.ca_mgmt_subnet_prefix external_subnet_prefix = var.ca_external_subnet_prefix internal_subnet_prefix = var.ca_internal_subnet_prefix route_server_subnet_prefix = var.ca_route_server_subnet_prefix route_server_name = local.ca_route_server_name enable_route_server = var.enable_bgp bastion_subnet_prefix = var.ca_bastion_subnet_prefix enable_bastion = var.enable_bastion bastion_name = local.ca_bastion_name tags = local.tags}
# One Canadian CE VM (3 NICs + identity) per node.module "ce_node_ca" { for_each = try(module.ce_topology_ca[0].ce_nodes, {}) source = "./modules/ce-node"
hostname = each.value.hostname resource_group_name = try(module.azure_hub_ca[0].resource_group_name, null) location = try(module.azure_hub_ca[0].location, null) zone = each.value.az vm_size = var.ce_vm_size mgmt_subnet_id = try(module.azure_hub_ca[0].management_subnet_id, null) external_subnet_id = try(module.azure_hub_ca[0].external_subnet_id, null) internal_subnet_id = try(module.azure_hub_ca[0].internal_subnet_id, null) mgmt_private_ip = each.value.slo_ip admin_username = var.admin_username ssh_public_key = local.ssh_public_key
custom_data = base64encode(local.ca_ce_cloud_init[each.key])
tags = local.tags}
resource "random_password" "site_console_admin_ca" { for_each = try(module.ce_topology_ca[0].ce_nodes, {})
length = 32 min_lower = 1 min_numeric = 1 min_special = 1 min_upper = 1 override_special = "!#%*+-=?@^_~"
keepers = { ce_vm_instance_id = module.ce_node_ca[each.key].vm_instance_id }}
resource "azurerm_virtual_machine_extension" "site_console_password_ca" { for_each = try(module.ce_topology_ca[0].ce_nodes, {})
name = "site-console-admin-password" virtual_machine_id = module.ce_node_ca[each.key].vm_id publisher = "Microsoft.Azure.Extensions" type = "CustomScript" type_handler_version = "2.1" auto_upgrade_minor_version = true
protected_settings = jsonencode({ script = base64encode(<<-SCRIPT #!/usr/bin/env bash set -euo pipefail password_b64='${base64encode(random_password.site_console_admin_ca[each.key].result)}' password=$(printf '%s' "$password_b64" | base64 --decode) printf 'admin:%s\n' "$password" | chpasswd unset password password_b64 SCRIPT ) })}
# Canadian XC SMSv2 site + BGP object per node.module "xc_site_ca" { for_each = try(module.ce_topology_ca[0].ce_nodes, {}) source = "./modules/xc-site"
create_site = true site_name = each.value.site_name hostname = each.value.hostname interface_name = each.value.interface_name mgmt_nic_mac = module.ce_node_ca[each.key].mgmt_nic_mac ce_vm_instance_id = module.ce_node_ca[each.key].vm_instance_id rs_peer_ips = try(module.azure_hub_ca[0].rs_peer_ips, []) ce_asn = var.ce_asn rs_asn = var.rs_asn os_version = var.ce_os_version sw_version = var.ce_sw_version enable_bgp = var.enable_bgp approve_registration = var.approve_registration labels = local.ca_xc_labels}
# Azure Route Server eBGP session for Canadian CEs.module "azure_route_server_bgp_ca" { for_each = var.enable_bgp ? try(module.ce_topology_ca[0].ce_nodes, {}) : {} source = "./modules/azure-route-server-bgp"
name = "${each.key}-bgp" route_server_id = try(module.azure_hub_ca[0].route_server_id, null) peer_asn = var.ce_asn peer_ip = module.ce_node_ca[each.key].mgmt_private_ip}
module "client_vm_ca" { count = var.enable_azure && var.enable_canada ? 1 : 0 source = "./modules/client-vm"
name = local.ca_client_vm_name resource_group_name = try(module.azure_hub_ca[0].resource_group_name, null) location = try(module.azure_hub_ca[0].location, null) subnet_id = try(module.azure_hub_ca[0].internal_subnet_id, null) admin_username = var.admin_username ssh_public_key = local.ssh_public_key tags = local.tags}
# ---------------------------------------------------------# F5 XC Virtual Sites (Canada RE & Canada CE)# ---------------------------------------------------------
resource "xcsh_virtual_site" "canada_re" { count = var.enable_azure && var.enable_canada ? 1 : 0 name = local.ca_re_vsite_name namespace = data.xcsh_namespace.mcn.name labels = local.ca_xc_labels
site_type = "REGIONAL_EDGE" site_selector { expressions = ["ves.io/city in (${join(", ", var.ca_re_cities)})"] }}
resource "xcsh_virtual_site" "canada_ce" { count = var.enable_azure && var.enable_canada ? 1 : 0 name = local.ca_ce_vsite_name namespace = data.xcsh_namespace.mcn.name labels = local.ca_xc_labels
site_type = "CUSTOMER_EDGE" site_selector { expressions = ["ves.io/siteName in (${join(", ", [for k, v in try(module.ce_topology_ca[0].ce_nodes, {}) : v.site_name])})"] }}
resource "xcsh_origin_pool" "canada" { count = var.enable_azure && var.enable_canada ? 1 : 0 name = local.ca_origin_pool_name namespace = data.xcsh_namespace.mcn.name description = "Canada reference origin pool -> ${var.origin_ip}:${var.origin_port}" labels = local.ca_xc_labels
port = var.origin_port
origin_servers { labels = {} public_ip { ip = var.origin_ip } }
no_tls = {} loadbalancer_algorithm = "ROUND_ROBIN" endpoint_selection = "DISTRIBUTED"}
resource "xcsh_http_loadbalancer" "canada" { count = var.enable_azure && var.enable_canada ? 1 : 0 depends_on = [module.xc_site_ca, xcsh_virtual_site.canada_re, xcsh_virtual_site.canada_ce]
name = local.ca_lb_name namespace = data.xcsh_namespace.mcn.name description = "Canada Regional HA: custom VIP ${var.ca_vip} advertised strictly via Canadian Regional Edges (Toronto and Montreal) and Canadian CEs." labels = local.ca_xc_labels
domains = [local.ca_lb_domain]
http { port = 80 }
advertise_custom { dynamic "advertise_where" { for_each = try(module.ce_topology_ca[0].ce_nodes, {}) content { site { network = "SITE_NETWORK_OUTSIDE" site { namespace = "system" name = advertise_where.value.site_name } ip = var.ca_vip } use_default_port = {} } } }
default_route_pools { pool { namespace = data.xcsh_namespace.mcn.name name = try(xcsh_origin_pool.canada[0].name, null) } weight = 1 priority = 1 }
round_robin = {} no_challenge = {} user_id_client_ip = {} disable_waf = {} disable_rate_limit = {} disable_api_discovery = {} disable_api_testing = {} disable_api_definition = {} l7_ddos_protection {} service_policies_from_namespace = {} disable_trust_client_ip_headers = {} disable_malicious_user_detection = {} disable_malware_protection = {} disable_threat_mesh = {} default_sensitive_data_policy = {}}terraform/onprem_kvm.tf
หัวข้อที่มีชื่อว่า “terraform/onprem_kvm.tf”On-Premise KVM SecureMesh v2 Site (xcsh_securemesh_site_v2.onprem_kvm) and eBGP peering configuration (xcsh_bgp.onprem_ebgp). Connects on-premise CEs (ASN 64512) over eth0 to containerized FRR ToR BGP router (10.100.0.1, ASN 65515).
# On-Prem KVM SecureMesh Site v2resource "xcsh_securemesh_site_v2" "onprem_kvm" { count = var.enable_kvm ? 1 : 0
name = local.kvm_site_name namespace = "system" description = "On-Prem KVM SecureMesh Site v2" labels = local.kvm_xc_labels
kvm { not_managed { } }
disable_ha = {} block_all_services = {} no_network_policy = {} no_forward_proxy = {} f5_proxy = {} no_proxy_bypass = {} logs_streaming_disabled = {} no_s2s_connectivity_sli = {} no_s2s_connectivity_slo = {} disable_url_categorization = {} disable_management_network = {}
# Match the non-AppStack SMSv2 Console defaults during first-boot software # installation. Provider v11.0.2 does not expose software_settings.waf_signatures; # every supported default from the Console-created KVM object is explicit here. dns_ntp_config { f5_dns_default = {} f5_ntp_default = {} }
local_vrf { default_config = {} default_sli_config = {} }
offline_survivability_mode { no_offline_survivability_mode = {} }
performance_enhancement_mode { perf_mode_l7_enhanced { jumbo_disabled = {} } }
re_select { geo_proximity = {} }
load_balancing { vip_vrrp_mode = "VIP_VRRP_ENABLE" }
software_settings { os { default_os_version = {} } sw { volterra_software_version = var.kvm_software_version } }
upgrade_settings { kubernetes_upgrade_drain { enable_upgrade_drain { drain_node_timeout = 300 drain_max_unavailable_node_count = 1 disable_vega_upgrade_mode = {} } } }}
# A KVM CE must never consume the generic tenant token. The issued JWT is# cryptographically bound to this exact SMSv2 site and inserted only into the# console-provided cloud-init template.resource "xcsh_token" "kvm" { count = var.enable_kvm ? 1 : 0
name = "${local.kvm_site_name}-registration" namespace = "system" description = "Site-bound JWT for KVM SecureMesh site ${local.kvm_site_name}" labels = local.kvm_token_labels type = 1 site_name = xcsh_securemesh_site_v2.onprem_kvm[0].name
lifecycle { replace_triggered_by = [xcsh_securemesh_site_v2.onprem_kvm[0].id] }}
data "xcsh_site_registration" "kvm" { for_each = local.kvm_enabled_nodes
# XC may append a runtime suffix to the cloud-init hostname. This site is # single-node, so exact site scoping uniquely resolves its live registration. site_name = local.kvm_site_name namespace = "system"
}
resource "xcsh_registration_approval" "kvm" { for_each = { for key, registration in data.xcsh_site_registration.kvm : key => registration if registration.found && registration.state == "NEW" }
namespace = "system" name = each.value.name cluster_size = 1 state = "APPROVED"
depends_on = [xcsh_securemesh_site_v2.onprem_kvm]}
data "xcsh_site_registrations_by_site" "kvm" { count = var.enable_kvm ? 1 : 0
namespace = "system" site_name = local.kvm_site_name}
locals { kvm_registration_records = var.enable_kvm ? flatten([ for item in coalesce(try(data.xcsh_site_registrations_by_site.kvm[0].items, null), []) : [ for network in try(item.get_spec.infra.hw_info.network, []) : { hostname = try(item.get_spec.infra.hostname, "") provider = try(item.get_spec.infra.provider_ref, "") mac = lower(try(network.mac_address, "")) } ] ]) : [] kvm_registration_mapping_valid = !var.enable_kvm || module.kvm_registration_mapping.mapping_valid kvm_expected_bgp_peers = ( var.enable_kvm && module.kvm_registration_mapping.mapping_valid ? module.kvm_registration_mapping.expected_bgp_peers : {} )}
module "kvm_registration_mapping" { source = "./modules/kvm-registration-mapping"
enforce = var.enable_kvm && var.aws_site_configuration_phase == "configured" registration_records = local.kvm_registration_records ce_nodes = local.kvm_ce_nodes}
# Resolve the platform-owned primary interface from the current Site UID, live# KVM registration, and Terraform-owned MAC. Names and guest devices are always# observed from XC rather than derived from hostname conventions.data "xcsh_smsv2_kvm_runtime" "kvm_slo" { count = var.enable_kvm ? 1 : 0
namespace = "system" site = local.kvm_site_name expected_mac = local.kvm_ce_nodes["01"].mac timeout_seconds = 7200 poll_interval_seconds = 10
depends_on = [libvirt_domain.ce_node]
lifecycle { postcondition { condition = ( self.interface_name != "" && self.hostname != "" && self.device != "" && lower(self.mac) == lower(local.kvm_ce_nodes["01"].mac) && self.online && self.registration_state == "ONLINE" ) error_message = "KVM BGP requires one ONLINE XC runtime SLO correlated by current site ownership, live registration hostname/device, and the Terraform-owned CE MAC." } }}
locals { kvm_lan_sli_interfaces = local.kvm_lan_configured ? toset(["sli"]) : toset([])}
# XC creates node interfaces as children of the registered site. The site API# rejects any post-registration node_list update that restates the immutable# primary SLO. The KVM-specific provider resource adopts only the exact owned# secondary child, reconciles ambiguous PUT outcomes, and restores DHCP on# destroy without deleting the platform-owned child.resource "xcsh_smsv2_kvm_runtime_interface" "kvm_lan_sli" { for_each = local.kvm_lan_sli_interfaces
namespace = "system" site = local.kvm_site_name expected_mac = var.kvm_lan.sli_mac ipv4_cidr = var.kvm_lan.sli_cidr
depends_on = [libvirt_domain.ce_node]}
# The provider convergence data source intentionally requires a nonempty# exported-route expectation. This KVM proof has one imported lab prefix but no# exported prefix, so observe only the bounded facts that actually exist.data "external" "kvm_bgp_observer" { count = var.enable_kvm && var.aws_site_configuration_phase == "configured" ? 1 : 0
program = ["python3", "${path.module}/scripts/xc-kvm-bgp-observer.py"] query = { api_url = local.xc_api_url namespace = "system" site_name = xcsh_securemesh_site_v2.onprem_kvm[0].name expected_node = try(local.kvm_expected_bgp_peers["node_01_slo"].node, "") expected_peer_address = try(local.kvm_expected_bgp_peers["node_01_slo"].peer_address, "") expected_imported_route = try(one(local.kvm_expected_bgp_peers["node_01_slo"].expected_imported_routes), "") timeout_seconds = "1800" poll_interval_seconds = "10" observer_sha256 = filesha256("${path.module}/scripts/xc-kvm-bgp-observer.py") }
depends_on = [ module.kvm_registration_mapping, xcsh_registration_approval.kvm, xcsh_bgp.onprem_ebgp, docker_container.kvm_frr, ]
lifecycle { postcondition { condition = ( self.result.converged == "true" && self.result.registered_node == try(local.kvm_expected_bgp_peers["node_01_slo"].node, "") && self.result.peer_address == "10.100.0.2" && self.result.state == "Established" && self.result.imported_route == "198.51.100.0/24" ) error_message = "KVM BGP requires the exact registered node, one Established 10.100.0.2 peer, and imported route 198.51.100.0/24." } }}
# eBGP Peering configuration for On-Prem KVM Siteresource "xcsh_bgp" "onprem_ebgp" { count = var.enable_kvm ? 1 : 0
name = "onprem-kvm-ebgp" namespace = "system" labels = local.kvm_xc_labels
where { site { network_type = "VIRTUAL_NETWORK_SITE_LOCAL" ref { name = xcsh_securemesh_site_v2.onprem_kvm[0].name namespace = "system" } disable_internet_vip = {} } }
bgp_parameters { asn = 64512 local_address = {} }
peers { metadata { name = "peer-router" } external { asn = 65515 address = "10.100.0.2" port = 179
interface { name = data.xcsh_smsv2_kvm_runtime.kvm_slo[0].interface_name namespace = "system" }
disable_v6 = {} } passive_mode_disabled = {} bfd_disabled = {} }
lifecycle { replace_triggered_by = [xcsh_securemesh_site_v2.onprem_kvm[0].id] }
# Do not redirect the F5-side peer until both the Terraform-owned router and # the CE interfaces with the declared static identities are ready. depends_on = [ data.xcsh_smsv2_kvm_runtime.kvm_slo, docker_container.kvm_frr, libvirt_domain.ce_node, ]}terraform/outputs.tf
หัวข้อที่มีชื่อว่า “terraform/outputs.tf”Everything the documentation reads instead of naming. If a page shows a command, an output here is behind it.
# ---------------------------------------------------------# Topology# ---------------------------------------------------------
output "deployment_provenance" { description = "Versioned, secret-free source and ownership identity for this Terraform state." value = { schema_version = local.deployment_identity_schema repository = var.source_repository repository_url = "https://github.com/${var.source_repository}" main_navigation_url = "https://github.com/${var.source_repository}/tree/main" source_ref = var.source_ref source_ref_sha256 = local.source_ref_sha256 source_commit = var.source_commit_sha source_commit_url = "https://github.com/${var.source_repository}/commit/${var.source_commit_sha}" environment_key = local.deployment_environment_key production = local.deployment_is_production owner_id = var.deployment_owner_id actor_id = var.deployment_actor_id state_key = local.showcase_backend_key recovery_state_key = local.recovery_backend_key artifact_scope = local.deployment_artifact_scope lock_scope = "${var.source_repository}:${local.deployment_environment_key}" }}
output "ce_nodes" { description = "Expanded per-CE node map (hostname, site_name, slo_ip, az, interface_name)." value = module.ce_topology.ce_nodes}
output "ce_count" { description = "Number of CE nodes deployed." value = module.ce_topology.ce_count}
# ---------------------------------------------------------# Azure hub / Route Server# ---------------------------------------------------------
output "resource_group_name" { description = "Hub resource group name." value = try(module.azure_hub[0].resource_group_name, null)}
output "route_server_id" { description = "Azure Route Server resource ID." value = try(module.azure_hub[0].route_server_id, null)}
# The four outputs below exist so the documentation can name nothing. Every# operational value a reader needs has to be readable from the deployment rather# than copied out of prose, because prose goes stale silently and a reader cannot# tell. Anything documented as a command therefore needs an output behind it.output "route_server_name" { description = "Azure Route Server name — the --routeserver argument of `az network routeserver peering list-learned-routes`." value = local.route_server_name}
output "azure_ilb_private_ip" { description = "Azure US ILB private address for supported CE Site Console health and traffic verification; null when disabled." value = try(azurerm_lb.azure_ilb[0].frontend_ip_configuration[0].private_ip_address, null)}
output "canada_ilb_private_ip" { description = "Canadian ILB private address for supported CE Site Console health and traffic verification; null when disabled." value = try(azurerm_lb.ca_ilb[0].frontend_ip_configuration[0].private_ip_address, null)}
output "ca_resource_group_name" { description = "Canadian Azure resource group used for supported ILB verification." value = try(module.azure_hub_ca[0].resource_group_name, null)}
output "ca_client_vm_name" { description = "Canadian test client VM used to probe the Canadian ILB." value = try(module.client_vm_ca[0].vm_name, null)}
output "client_vm_name" { description = "Test client VM name — the -n argument of `az vm run-command invoke` when driving traffic at the VIP from inside the VNet." value = local.client_vm_name}
output "lb_domain" { description = "Domain the HTTP load balancer matches on. Requests to the VIP MUST send it as the Host header; without it the load balancer has no matching domain and answers 404." value = local.lb_domain}
output "origin_ip" { description = "Origin the pool targets. Useful as a control: a batch straight to the origin, bypassing the VIP, separates an origin fault from a VIP/ECMP/CE fault." value = var.origin_ip}
output "route_server_peer_ips" { description = "Route Server BGP peer IPs (the CE external BGP peer addresses)." value = try(module.azure_hub[0].rs_peer_ips, [])}
output "ce_mgmt_private_ips" { description = "Per-CE eth0/SLO private IPs (BGP local addresses / RS bgpConnection peer IPs)." value = { for k, m in module.ce_node : k => m.mgmt_private_ip }}
output "ce_vm_names" { description = "Per-CE VM names." value = { for k, m in module.ce_node : k => m.vm_name }}
output "ce_sli_private_ips" { description = "Per-CE internal/SLI private IPs — where each CE serves its Site Console web UI (TCP 65500). Informational: the Bastion tunnel is targeted by VM resource id (ce_vm_ids), because Azure does not allow a custom resource port over an IP-targeted tunnel." value = { for k, m in module.ce_node : k => m.sli_private_ip }}
output "ce_vm_ids" { description = "Per-CE VM resource IDs — the --target-resource-id of `az network bastion tunnel` when opening the Site Console web UI on 65500." value = { for k, m in module.ce_node : k => m.vm_id }}
output "site_console_admin_passwords" { description = "Generated per-CE passwords for the node-local Site Console admin user. Retrieve only for an active tunnel and keep them out of logs and published documentation." value = { for k, password in random_password.site_console_admin : k => password.result } sensitive = true}
output "bastion_name" { description = "Azure Bastion host name, or null when enable_bastion is false. Feed it to `az network bastion tunnel --name`." value = try(module.azure_hub[0].bastion_name, null)}
output "client_public_ip" { description = "Public IP of the test client." value = try(module.client_vm[0].public_ip, null)}
output "client_nic_name" { description = "Test client NIC name (read effective routes here to prove ECMP)." value = try(module.client_vm[0].nic_name, null)}
# ---------------------------------------------------------# XC tenant# ---------------------------------------------------------
output "xc_tenant" { description = "F5 XC tenant this deployment writes to. Config-controlled (var.expected_xc_tenant), not taken from XCSH_API_URL." value = var.expected_xc_tenant}
output "xc_api_url" { description = "F5 XC API endpoint the xcsh provider is pinned to, derived from var.expected_xc_tenant." value = local.xc_api_url}
# The other half of the tenant guard's comparison, published so an operator can# see what their shell is claiming without having to trip the guard to find out —# `terraform output xc_env_tenant` answers "which tenant am I sourced for?".# Empty means XCSH_API_URL is unset, which is the CI case and which the guard# treats as no opinion rather than as a mismatch.output "xc_env_tenant" { description = "F5 XC tenant named by XCSH_API_URL in the environment this ran in, or empty when unset. Diagnostic only: xc_tenant is what the deployment actually targets." value = data.external.xc_env_tenant.result.tenant}
# ---------------------------------------------------------# XC data-plane# ---------------------------------------------------------
output "xc_site_names" { description = "Per-CE XC site names." value = { for k, m in module.xc_site : k => m.site_name }}
output "ca_xc_site_names" { description = "Per-CE Canadian XC site names." value = { for k, m in module.xc_site_ca : k => m.site_name }}
output "ca_ce_vm_names" { description = "Per-CE Canadian VM names used for Azure runtime and extension verification." value = { for k, m in module.ce_node_ca : k => m.vm_name }}
# Makes the site-to-node binding that closes #674 observable from the CLI. Each# value is the CE VM instance id its XC site object is coupled to; the matching# registration reports the same value as infra.instance_id. When the two disagree# the site is bound to a node that no longer exists — the state in which a# rebuilt CE can never register. (The registration side is not on the# xcsh_site_registration data source yet: provider issue #1376.)output "ce_bound_instance_ids" { description = "Per-CE VM instance id each XC site object is bound to. Compare with the registration's infra.instance_id to spot a site still bound to a destroyed node." value = { for k, m in module.xc_site : k => m.bound_vm_instance_id }}
output "xc_interface_names" { description = "Per-CE auto-derived network_interface object names (BGP peer bind target)." value = { for k, m in module.xc_site : k => m.interface_name }}
output "loadbalancer_name" { description = "HTTP load balancer name." value = try(xcsh_http_loadbalancer.this[0].name, null)}
output "origin_pool_name" { description = "Origin pool name." value = try(xcsh_origin_pool.this[0].name, null)}
output "vip" { description = "HA VIP advertised via eBGP/ECMP." value = var.vip}
# ---------------------------------------------------------# Canada Regional outputs# ---------------------------------------------------------
output "ca_lb_domain" { description = "Domain served by the Canada HTTP load balancer." value = local.ca_lb_domain}
output "ca_re_virtual_site_name" { description = "Name of the Canadian Regional Edge virtual site." value = try(xcsh_virtual_site.canada_re[0].name, null)}
output "ca_ce_virtual_site_name" { description = "Name of the Canadian Customer Edge virtual site." value = try(xcsh_virtual_site.canada_ce[0].name, null)}
output "ca_loadbalancer_name" { description = "Name of the Canadian HTTP load balancer." value = try(xcsh_http_loadbalancer.canada[0].name, null)}
output "ca_origin_pool_name" { description = "Name of the Canadian origin pool." value = try(xcsh_origin_pool.canada[0].name, null)}
output "ca_vip" { description = "HA VIP for Canadian CEs advertised via eBGP/ECMP or Azure ILB." value = var.ca_vip}
output "ca_ilb_id" { description = "Azure Internal Load Balancer ID for Canadian regional path." value = try(azurerm_lb.ca_ilb[0].id, null)}
output "ca_ilb_frontend_ip" { description = "Azure Internal Load Balancer frontend private IP for Canadian regional path." value = try(azurerm_lb.ca_ilb[0].frontend_ip_configuration[0].private_ip_address, null)}
# ---------------------------------------------------------# CE registration token# ---------------------------------------------------------
output "registration_token_name" { description = "Name (metadata id) of the generated xcsh_token used for Azure CE registration, or null when Azure is disabled." value = try(xcsh_token.ce[0].name, null)}
output "registration_token_is_generated" { description = "True when an enabled Azure CE cloud-init token feed uses the generated xcsh_token.ce[0].uid." # Whether an override was supplied is not itself secret (the token value is). value = nonsensitive(local.azure_provider_enabled && var.registration_token == "")}
output "ce_registration_token" { description = "Resolved Azure CE registration token fed to cloud-init, or null when Azure is disabled and no override is supplied." value = local.ce_registration_token sensitive = true}
# ---------------------------------------------------------# AWS outputs# ---------------------------------------------------------
output "aws_vpc_id" { description = "AWS VPC ID." value = try(aws_vpc.aws[0].id, null)}
output "aws_workload_vpc_id" { description = "Dedicated AWS workload VPC identity." value = try(aws_vpc.workload[0].id, null)}
output "aws_workload_instance_id" { description = "Amazon Linux SSM client identity." value = try(aws_instance.workload[0].id, null)}
output "aws_workload_private_ip" { description = "Private address of the Amazon Linux SSM client." value = try(aws_instance.workload[0].private_ip, null)}
output "aws_origin_dns_name" { description = "DNS name of the external HTTP origin used by the AWS SMSv2 showcase." value = var.aws_origin_dns_name}
output "aws_site_names" { description = "Canonical independent AWS SecureMesh v2 site names." value = { for key, site in local.aws_sites : key => site.name }}
output "aws_smsv2_owned_eni_projection" { description = "Private Terraform-owned AWS ENI MAC projection for the one-to-one bootstrap registration join." sensitive = true value = flatten([ for key, site in local.aws_sites : [ { site_key = key, role = "slo", mac = aws_network_interface.slo[site.index].mac_address }, { site_key = key, role = "sli", mac = aws_network_interface.sli[site.index].mac_address }, ] ])}
output "aws_smsv2_bootstrap_registration_projection" { description = "Private observed bootstrap hardware projection for the one-to-one device join." sensitive = true value = flatten([ for key, registration in data.xcsh_site_registrations_by_site.aws_bootstrap : [ for item in coalesce(try(registration.items, null), []) : [ for network in try(item.get_spec.infra.hw_info.network, []) : { site_key = key mac = network.mac_address device = network.name } ] ] ])}
output "kvm_runtime_status" { description = "Sanitized KVM registration and BGP convergence summary." value = var.enable_kvm ? { registration_count = length([ for registration in values(data.xcsh_site_registration.kvm) : registration if registration.found ]) online_count = length([ # xcsh_site_registration.state is the registration object's current # state. ONLINE means the matched CE node is admitted and healthy; it is # not inferred from approval-resource presence or from plan completion. for registration in values(data.xcsh_site_registration.kvm) : registration if registration.state == "ONLINE" ]) mapping_valid = local.kvm_registration_mapping_valid bgp_converged = try(data.external.kvm_bgp_observer[0].result.converged == "true", false) bgp_session_count = try(data.external.kvm_bgp_observer[0].result.converged == "true" ? 1 : 0, 0) } : null}
output "kvm_lan_contract" { description = "Secret-free staged KVM LAN ownership, addressing, and realized-interface summary; null while disabled." value = local.kvm_lan_enabled ? { site_name = local.kvm_site_name phase = var.kvm_lan_configuration_phase bridge = var.kvm_lan.bridge uplink = var.kvm_lan.uplink ownership = var.kvm_lan.ownership vlan_mode = var.kvm_lan.vlan_mode vlan_id = var.kvm_lan.vlan_id mtu = var.kvm_lan.mtu sli_mac = var.kvm_lan.sli_mac sli_cidr = var.kvm_lan.sli_cidr vip = var.kvm_lan.vip vip_reservation = var.kvm_lan.vip_reservation backend_ip = var.kvm_lan.backend_ip backend_port = var.kvm_lan.backend_port backend_owner = var.kvm_lan.backend_owner http_domain = var.kvm_lan.http_domain access_scope = var.kvm_lan.access_scope observed_hostname = try(data.xcsh_smsv2_kvm_runtime.kvm_slo[0].hostname, null) observed_slo = try(data.xcsh_smsv2_kvm_runtime.kvm_slo[0].device, null) observed_sli = try(xcsh_smsv2_kvm_runtime_interface.kvm_lan_sli["sli"].device, null) realized_sli = try(xcsh_smsv2_kvm_runtime_interface.kvm_lan_sli["sli"].device, null) virtual_site_name = try(xcsh_virtual_site.kvm_lan[0].name, null) origin_pool_name = try(xcsh_origin_pool.kvm_lan[0].name, null) load_balancer_name = try(xcsh_http_loadbalancer.kvm_lan[0].name, null) } : null}
output "aws_tgw_id" { description = "AWS Transit Gateway identity." value = try(module.aws_tgw_connect[0].transit_gateway_id, null)}
output "aws_tgw_route_table_id" { description = "TGW route table used for explicit workload association and propagation." value = try(module.aws_tgw_connect[0].route_table_id, null)}
output "aws_ce_instance_ids" { description = "EC2 instance IDs of the AWS Customer Edge nodes." value = aws_instance.ce[*].id}
output "aws_ce_public_ips" { description = "Elastic IPs assigned to the AWS Customer Edge nodes." value = aws_eip.ce[*].public_ip}
output "aws_lb_domain" { description = "Domain served by the AWS HTTP load balancer." value = local.aws_lb_domain}
output "aws_loadbalancer_name" { description = "Name of the AWS HTTP load balancer." value = try(xcsh_http_loadbalancer.aws[0].name, null)}
output "aws_origin_pool_name" { description = "Name of the AWS origin pool." value = try(xcsh_origin_pool.aws[0].name, null)}
output "aws_vip" { description = "Plan-bound private IP of the internal NLB fronting the three BGP-routed SMSv2 listeners." value = var.aws_vip}
output "aws_smsv2_site_listener_ips" { description = "Per-site automatic SLI listener addresses exported over TGW Connect BGP." value = { for key, site in local.aws_sites : key => site.listener_ip }}
output "aws_smsv2_nlb_dns_name" { description = "Internal AWS NLB DNS name for the SMSv2 service." value = try(aws_lb.smsv2[0].dns_name, null)}
output "aws_smsv2_target_group_arn" { description = "Target group containing the three BGP-routed SMSv2 site listeners." value = try(aws_lb_target_group.smsv2[0].arn, null)}terraform/providers.tf
หัวข้อที่มีชื่อว่า “terraform/providers.tf”Pins the xcsh endpoint to var.expected_xc_tenant, which is what makes the tenant a property of the configuration.
# The xcsh provider takes its CREDENTIAL from the environment — no secrets in# code — but NOT its endpoint. Export one of the following before running# Terraform:## Token auth: XCSH_API_TOKEN# P12 auth: XCSH_P12_FILE + XCSH_P12_PASSWORD# PEM auth: XCSH_CERT + XCSH_KEY## api_url is set here, from var.expected_xc_tenant, and deliberately overrides any# XCSH_API_URL in the environment. The tenant is not an ambient property of the# operator's shell: it is which deployment this is, it belongs in version control# next to the state key, and it is the one thing a credential file must not be# able to change silently. It once did — see the guard in main.tf and issue #696.## A credential minted for a different tenant now fails against this URL instead of# succeeding somewhere unintended. The plan-level tests mock this provider, so no# XC credentials are needed to run `terraform test`.provider "xcsh" { api_url = local.xc_api_url}
# Azure — deploys the hub VNet, Route Server, CE VMs and the test client.locals { azure_provider_enabled = var.enable_azure || var.enable_canada}
provider "azurerm" { features {} subscription_id = local.azure_provider_enabled ? var.subscription_id : null skip_provider_registration = !local.azure_provider_enabled}
# AWS — deploys the VPC, subnets, CE EC2 instances and internet gateway.# Auth comes from the environment (aws CLI login / AWS_* env vars).provider "aws" { region = var.aws_location}# Read-only: resolves the deployer identity for resource naming/tags.provider "azuread" {}terraform/providers_libvirt.tf
หัวข้อที่มีชื่อว่า “terraform/providers_libvirt.tf”Configures the libvirt provider targeting the local hypervisor daemon URI (qemu:///system).
provider "libvirt" { uri = "qemu:///system"}terraform/variables.tf
หัวข้อที่มีชื่อว่า “terraform/variables.tf”Component, environment, deployer and tags.
# General variables. Domain-specific inputs live in variables_azure.tf,# variables_xc.tf and variables_ce.tf.
variable "source_repository" { description = "Canonical GitHub repository identity of the reviewed source. Only this repository can produce a deployment identity." type = string
validation { condition = var.source_repository == "f5-sales-demo/multi-cloud-networking" error_message = "source_repository must be exactly f5-sales-demo/multi-cloud-networking." }}
variable "source_ref" { description = "Exact trusted branch ref of the reviewed source. PR merge refs and abbreviated branch names are rejected." type = string
validation { condition = ( can(regex("^refs/heads/[^[:cntrl:][:space:]~^:?*\\\\\\[]+$", var.source_ref)) && !strcontains(var.source_ref, "..") && !strcontains(var.source_ref, "@{") && !strcontains(var.source_ref, "//") && !can(regex("(^refs/heads/|/)\\.", var.source_ref)) && !can(regex("(\\.|\\.lock)(/|$)", var.source_ref)) && var.source_ref != "refs/heads/@" && !endswith(var.source_ref, "/") ) error_message = "source_ref must be a valid exact refs/heads/* ref, never refs/pull/* or an abbreviated branch." }}
variable "source_commit_sha" { description = "Immutable lowercase 40-hex commit that was reviewed and used to create the saved plan." type = string
validation { condition = can(regex("^[0-9a-f]{40}$", var.source_commit_sha)) error_message = "source_commit_sha must be an immutable lowercase 40-hex Git commit." }}
variable "deployment_owner_id" { description = "Non-personal stable identifier for the team or service that owns the deployment." type = string
validation { condition = can(regex("^[a-z][a-z0-9-]{2,62}$", var.deployment_owner_id)) error_message = "deployment_owner_id must be a 3-63 character lowercase non-personal identifier." }}
variable "deployment_actor_id" { description = "Non-personal stable identifier for the automation actor that applies the deployment." type = string
validation { condition = can(regex("^[a-z][a-z0-9-]{2,62}$", var.deployment_actor_id)) error_message = "deployment_actor_id must be a 3-63 character lowercase non-personal identifier." }}
variable "component" { description = "Component name used in tags." type = string default = "mcn-ce-ha"}
variable "environment" { description = "Environment label used in tags." type = string default = "lab"}
variable "deployer" { description = "Override for the deployer identifier used in tags (auto-resolved from Azure AD when empty)." type = string default = ""}
variable "tags" { description = "Additional tags merged with the standard tags (component/environment/deployer/managed_by)." type = map(string) default = {}}
variable "enable_kvm" { description = "Enable the local KVM/libvirt SMSv2 site. Set false for a KVM-only destroy or when the tenant-owned KVM image prerequisite is unavailable; no KVM image lookup occurs while disabled." type = bool default = false}
variable "enable_kvm_lan" { description = "Opt in to the staged KVM physical-LAN SLI topology. This does not authorize or perform CE replacement by itself." type = bool default = false}
variable "kvm_lan_configuration_phase" { description = "KVM LAN rollout phase: disabled, hardware (two NICs only), or configured (provider-owned runtime discovery plus inside VIP)." type = string default = "disabled" nullable = false
validation { condition = contains(["disabled", "hardware", "configured"], var.kvm_lan_configuration_phase) error_message = "kvm_lan_configuration_phase must be disabled, hardware, or configured." }}
variable "kvm_lan" { description = "Approved pre-existing LAN bridge/uplink inventory, deterministic SLI identity, and reserved inside-VIP/origin contract. Terraform never creates or mutates the shared bridge or uplink." type = object({ bridge = string uplink = string uplink_mac = string ownership = string vlan_mode = string vlan_id = optional(number) mtu = number sli_mac = string sli_cidr = string sli_ipv6_mode = optional(string, "disabled") vip = string vip_reservation = string backend_ip = string backend_port = number backend_owner = string http_domain = string access_scope = string bridge_preprovisioned = bool uplink_approved = bool ipv4_users_reviewed = bool ipv6_users_reviewed = bool switch_multi_mac_approved = bool duplicate_addresses_checked = bool }) default = null nullable = true
validation { condition = var.kvm_lan == null || try( can(regex("^[a-zA-Z0-9_.-]{1,15}$", var.kvm_lan.bridge)) && can(regex("^[a-zA-Z0-9_.:-]{1,15}$", var.kvm_lan.uplink)) && var.kvm_lan.bridge != var.kvm_lan.uplink && can(regex("^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$", var.kvm_lan.uplink_mac)) && can(regex("^([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}$", var.kvm_lan.sli_mac)) && lower(var.kvm_lan.uplink_mac) != lower(var.kvm_lan.sli_mac), false, ) error_message = "kvm_lan requires distinct valid bridge/uplink names and six-octet uplink/SLI MAC addresses." }
validation { condition = var.kvm_lan == null || try( var.kvm_lan.ownership == "preprovisioned-shared" && var.kvm_lan.vlan_mode == "access" && var.kvm_lan.vlan_id == null && var.kvm_lan.mtu >= 1280 && var.kvm_lan.mtu <= 9216 && var.kvm_lan.sli_ipv6_mode == "disabled", false, ) error_message = "kvm_lan currently supports an approved preprovisioned-shared access bridge, MTU 1280-9216, and an explicit disabled guest-SLI IPv6 policy; trunk VLANs are rejected until guest tagging is implemented." }
validation { condition = var.kvm_lan == null || try( can(cidrnetmask(var.kvm_lan.sli_cidr)) && can(cidrnetmask("${var.kvm_lan.vip}/32")) && can(cidrnetmask("${var.kvm_lan.backend_ip}/32")) && cidrhost(var.kvm_lan.sli_cidr, 0) == cidrhost("${var.kvm_lan.vip}/${split("/", var.kvm_lan.sli_cidr)[1]}", 0) && cidrhost(var.kvm_lan.sli_cidr, 0) == cidrhost("${var.kvm_lan.backend_ip}/${split("/", var.kvm_lan.sli_cidr)[1]}", 0) && cidrhost(var.kvm_lan.sli_cidr, 0) != cidrhost("10.100.0.0/24", 0) && !contains([ cidrhost(var.kvm_lan.sli_cidr, 0), cidrhost(var.kvm_lan.sli_cidr, -1), ], split("/", var.kvm_lan.sli_cidr)[0]) && !contains([ cidrhost(var.kvm_lan.sli_cidr, 0), cidrhost(var.kvm_lan.sli_cidr, -1), ], var.kvm_lan.vip) && !contains([ cidrhost(var.kvm_lan.sli_cidr, 0), cidrhost(var.kvm_lan.sli_cidr, -1), ], var.kvm_lan.backend_ip) && length(distinct([split("/", var.kvm_lan.sli_cidr)[0], var.kvm_lan.vip, var.kvm_lan.backend_ip])) == 3, false, ) error_message = "kvm_lan requires usable, distinct IPv4 SLI, VIP, and backend addresses in one subnet that does not overlap the 10.100.0.0/24 SLO fabric." }
validation { condition = var.kvm_lan == null || try( floor(var.kvm_lan.backend_port) == var.kvm_lan.backend_port && var.kvm_lan.backend_port >= 1 && var.kvm_lan.backend_port <= 65535 && can(regex("^([a-z0-9]([a-z0-9-]*[a-z0-9])?\\.)+[a-z]{2,}$", var.kvm_lan.http_domain)) && alltrue([ for value in [var.kvm_lan.vip_reservation, var.kvm_lan.backend_owner, var.kvm_lan.access_scope] : trimspace(value) != "" ]), false, ) error_message = "kvm_lan requires a valid backend port, lowercase HTTP domain, and nonempty reservation, origin-owner, and access-scope records." }}
variable "kvm_software_version" { description = "F5XC software installed during the KVM CE's first boot. This is pinned explicitly so a fresh install does not consume an unqualified tenant default release." type = string default = "crt-20260801-0205" nullable = false
validation { condition = can(regex("^crt-[0-9]{8}-[0-9]{4}$", var.kvm_software_version)) error_message = "kvm_software_version must be an explicit F5XC software build such as crt-20260801-0205." }}
variable "aws_origin_dns_name" { description = "DNS name of the public HTTP origin for the AWS SMSv2 load balancer." type = string default = "httpbin.org" nullable = false
validation { condition = can(regex("^([a-z0-9]([a-z0-9-]*[a-z0-9])?\\.)+[a-z]{2,}$", var.aws_origin_dns_name)) error_message = "aws_origin_dns_name must be a fully-qualified lowercase DNS name." }}terraform/variables_aws.tf
หัวข้อที่มีชื่อว่า “terraform/variables_aws.tf”AWS location, VPC CIDR, CE count, instance size, VIP, and load balancer domain.
# ---------------------------------------------------------# AWS site deployment & placement# ---------------------------------------------------------
variable "enable_aws" { description = "Enable deployment of the AWS Customer Edge site, VPC, EC2 instances, and XC resources." type = bool default = false}
variable "aws_ce_ami_id" { description = "Explicit approved AWS Marketplace AMI ID for Customer Edge instances. A deployment must not select the most-recent image dynamically." type = string default = null nullable = true
validation { condition = var.aws_ce_ami_id == null || can(regex("^ami-[0-9a-f]+$", var.aws_ce_ami_id)) error_message = "aws_ce_ami_id must be an AWS AMI ID such as ami-0123456789abcdef0." }}
variable "aws_workload_ami_id" { description = "Explicit approved Amazon Linux AMI ID for the AWS workload client. A deployment must not select the most-recent image dynamically because that causes unrelated reconciliation replacements." type = string default = null nullable = true
validation { condition = var.aws_workload_ami_id == null || can(regex("^ami-[0-9a-f]+$", var.aws_workload_ami_id)) error_message = "aws_workload_ami_id must be an AWS AMI ID such as ami-0123456789abcdef0." }}
variable "aws_ssh_public_key" { description = "Optional AWS-only SSH public key material. When empty, the shared ssh_public_key input is used." type = string default = ""}
variable "enable_aws_tgw_connect" { description = "Enable the v8 SMSv2 AWS Transit Gateway Connect topology." type = bool default = false}
variable "aws_tgw_asn" { description = "Amazon-side BGP ASN for the Transit Gateway." type = number default = 64520
validation { condition = var.aws_tgw_asn >= 1 && var.aws_tgw_asn <= 4294967295 error_message = "aws_tgw_asn must be a valid 32-bit ASN." }}
variable "aws_ce_bgp_asn" { description = "BGP ASN used by the AWS Customer Edge site." type = number default = 64513
validation { condition = var.aws_ce_bgp_asn >= 1 && var.aws_ce_bgp_asn <= 4294967295 && var.aws_ce_bgp_asn != var.aws_tgw_asn error_message = "aws_ce_bgp_asn must be a valid 32-bit ASN different from aws_tgw_asn." }}
variable "aws_tgw_gre_cidr" { description = "Non-overlapping /24 CIDR owned by the Transit Gateway for GRE endpoints." type = string default = "100.64.0.0/24"
validation { condition = can(cidrhost(var.aws_tgw_gre_cidr, 0)) && try(tonumber(split("/", var.aws_tgw_gre_cidr)[1]), 0) == 24 error_message = "aws_tgw_gre_cidr must be a valid IPv4 /24." }}
variable "aws_tgw_inside_cidr" { description = "Link-local /24 subdivided into one AWS-owned /29 per physical CE interface." type = string default = "169.254.100.0/24"
validation { condition = can(cidrhost(var.aws_tgw_inside_cidr, 0)) && try(tonumber(split("/", var.aws_tgw_inside_cidr)[1]), 0) == 24 error_message = "aws_tgw_inside_cidr must be a valid IPv4 /24." }}
variable "aws_site_configuration_phase" { description = "AWS SMSv2 lifecycle phase. bootstrap creates only distinct -bootstrap sites and CEs; bootstrap_retirement removes only their XC/CE material while retaining AWS networking and ENIs; configured creates distinct final sites and CEs from the private observed device mapping." type = string default = "bootstrap" nullable = false
validation { condition = contains(["bootstrap", "bootstrap_retirement", "configured"], var.aws_site_configuration_phase) error_message = "aws_site_configuration_phase must be bootstrap, bootstrap_retirement, or configured." }
validation { condition = var.aws_site_configuration_phase == "configured" || !var.enable_aws_tgw_connect error_message = "Only configured creates final MAC-bound sites and may enable AWS TGW Connect." }}
variable "aws_smsv2_device_mapping_file" { description = "Private, schema-validated device mapping generated from bootstrap registration observations and Terraform-owned ENI MACs. It is required only during configured and must never be committed." type = string default = null nullable = true
validation { condition = var.aws_site_configuration_phase != "configured" || (var.aws_smsv2_device_mapping_file != null && trimspace(var.aws_smsv2_device_mapping_file) != "") error_message = "configured requires aws_smsv2_device_mapping_file generated from bootstrap registration records; arbitrary device input is not accepted." }}
variable "aws_smsv2_interface_mtu" { description = "Expected MTU configured on every AWS SMSv2 SLO and SLI interface." type = number default = 1500}
variable "aws_runtime_convergence_timeout_seconds" { description = "Maximum bounded wait for first-boot runtime readiness. This covers the platform-managed installation of the explicit CE software and OS pair." type = number default = 7200
validation { condition = var.aws_runtime_convergence_timeout_seconds == 7200 error_message = "aws_runtime_convergence_timeout_seconds is fixed at 7200 seconds so a fresh SMSv2 CE has the full platform-managed first-boot convergence budget." }}
variable "aws_bgp_convergence_timeout_seconds" { description = "Maximum bounded wait for authoritative BGP and route convergence after the runtime-health gate has passed. This is bounded by the released provider schema." type = number default = 1800
validation { condition = var.aws_bgp_convergence_timeout_seconds == 1800 error_message = "aws_bgp_convergence_timeout_seconds is fixed at 1800 seconds, the maximum accepted by xcsh_site_bgp_status." }}
variable "aws_bgp_poll_interval_seconds" { description = "Polling interval for authoritative BGP and route observations." type = number default = 10}
variable "aws_location" { description = "AWS region for all AWS resources." type = string default = "ap-northeast-1"}
variable "aws_vpc_cidr" { description = "AWS VPC address space." type = string default = "10.150.0.0/16"}
variable "aws_ce_count" { description = "Number of independent single-node Customer Edge sites. The validated showcase topology requires exactly three." type = number default = 3
validation { condition = var.aws_ce_count == 3 error_message = "aws_ce_count must remain 3 for the validated three-site showcase." }}
variable "aws_bootstrap_site_keys" { description = "The complete three-site AWS lifecycle set. Partial site admission is not supported: bootstrap, retirement, and configured phases are reviewed as one three-site transition." type = list(string) default = ["01", "02", "03"]
validation { condition = toset(var.aws_bootstrap_site_keys) == toset(["01", "02", "03"]) && length(var.aws_bootstrap_site_keys) == 3 error_message = "aws_bootstrap_site_keys must contain exactly 01, 02, and 03 for the reviewed three-site lifecycle." }}
variable "aws_instance_type" { description = "EC2 instance size for the Customer Edge nodes." type = string default = "m5.2xlarge"}
variable "aws_vip" { description = "Plan-bound private address of the internal AWS Network Load Balancer in the workload subnet." type = string default = "10.151.1.10"
validation { condition = can(cidrhost("${var.aws_vip}/32", 0)) error_message = "aws_vip must be a valid IPv4 address." }}
variable "aws_workload_vpc_cidr" { description = "Address space for the TGW-attached AWS workload VPC." type = string default = "10.151.0.0/16"}
variable "aws_software_version" { description = "Field-proven F5 Distributed Cloud software version requested on first boot for every AWS SMSv2 CE. Do not use an intermediate baseline: first-boot health is a prerequisite for later actions." type = string default = "crt-20260201-0179"}
variable "aws_os_version" { description = "Field-proven F5 Distributed Cloud operating-system version requested on first boot for every AWS SMSv2 CE." type = string default = "9.2026.17"}
variable "aws_upgrade_software_version" { description = "Advertised software action target; null observes the create-time software version. Does not change first-boot settings." type = string default = null
validation { condition = var.aws_upgrade_software_version == null || can(regex("^crt-[0-9]{8}-[0-9]{4}$", var.aws_upgrade_software_version)) error_message = "aws_upgrade_software_version must be an explicit F5XC software build." }}
variable "aws_upgrade_os_version" { description = "Advertised OS action target; null observes the create-time OS version. Does not change first-boot settings." type = string default = null}
variable "aws_upgrade_wait" { description = "Wait for every supplied upgrade target to be installed and for each site to return ONLINE." type = bool default = false}
variable "aws_upgrade_timeout_seconds" { description = "Bounded per-site upgrade convergence timeout." type = number default = 7200}
variable "aws_upgrade_poll_interval_seconds" { description = "Polling interval for site upgrade observations." type = number default = 30}
variable "aws_upgrade_observed_sites" { description = "Canonical two-digit AWS site keys observed by the upgrade status data source." type = set(string) default = ["01", "02", "03"]
validation { condition = length(setsubtract(var.aws_upgrade_observed_sites, toset(["01", "02", "03"]))) == 0 error_message = "aws_upgrade_observed_sites may contain only 01, 02, and 03." }}
variable "aws_lb_domain" { description = "Domain name for the HTTP Load Balancer serving the AWS CE site." type = string default = "aws.mcn-ce-ha.f5-sales-demo.com"}terraform/aws_vpc.tf
หัวข้อที่มีชื่อว่า “terraform/aws_vpc.tf”AWS VPC, public SLO subnets, private SLI subnets, internet gateway, route tables, and security groups.
# ---------------------------------------------------------# AWS VPC, Subnets, Gateways, Route Tables & Security Groups# ---------------------------------------------------------
data "aws_availability_zones" "available" { count = var.enable_aws ? 1 : 0 state = "available"}
resource "aws_vpc" "aws" { #checkov:skip=CKV2_AWS_11:Lab VPC - flow logging not required #checkov:skip=CKV2_AWS_12:Lab VPC - default security group managed by AWS count = var.enable_aws ? 1 : 0
cidr_block = var.aws_vpc_cidr enable_dns_hostnames = true enable_dns_support = true
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-vpc" })}
resource "aws_internet_gateway" "aws" { count = var.enable_aws ? 1 : 0
vpc_id = aws_vpc.aws[0].id
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-igw" })}
# 3 Public SLO Subnets (10.150.1.0/24, 10.150.2.0/24, 10.150.3.0/24)resource "aws_subnet" "public_slo" { count = var.enable_aws ? 3 : 0
vpc_id = aws_vpc.aws[0].id cidr_block = cidrsubnet(var.aws_vpc_cidr, 8, count.index + 1) availability_zone = try(data.aws_availability_zones.available[0].names[count.index], "${var.aws_location}${element(["a", "b", "c"], count.index)}") map_public_ip_on_launch = false
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-slo-subnet-${count.index + 1}" })}
# 3 Private SLI Subnets (10.150.11.0/24, 10.150.12.0/24, 10.150.13.0/24)resource "aws_subnet" "private_sli" { count = var.enable_aws ? 3 : 0
vpc_id = aws_vpc.aws[0].id cidr_block = cidrsubnet(var.aws_vpc_cidr, 8, count.index + 11) availability_zone = try(data.aws_availability_zones.available[0].names[count.index], "${var.aws_location}${element(["a", "b", "c"], count.index)}")
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-sli-subnet-${count.index + 1}" })}
resource "aws_route_table" "public" { count = var.enable_aws ? 1 : 0
depends_on = [module.aws_tgw_connect]
vpc_id = aws_vpc.aws[0].id
route { cidr_block = "0.0.0.0/0" gateway_id = aws_internet_gateway.aws[0].id }
dynamic "route" { for_each = var.enable_aws_tgw_connect ? [1] : [] content { cidr_block = var.aws_tgw_gre_cidr transit_gateway_id = module.aws_tgw_connect[0].transit_gateway_id } }
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-public-rt" })}
resource "aws_route_table_association" "public" { for_each = var.enable_aws ? local.aws_bootstrap_sites : {}
subnet_id = aws_subnet.public_slo[each.value.index].id route_table_id = aws_route_table.public[0].id}
resource "aws_route_table" "private" { count = var.enable_aws ? 1 : 0
depends_on = [module.aws_tgw_connect]
vpc_id = aws_vpc.aws[0].id
route { cidr_block = "0.0.0.0/0" gateway_id = aws_internet_gateway.aws[0].id }
dynamic "route" { for_each = var.enable_aws_tgw_connect ? [1] : [] content { cidr_block = var.aws_tgw_gre_cidr transit_gateway_id = module.aws_tgw_connect[0].transit_gateway_id } }
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-private-rt" })}
resource "aws_route_table_association" "private" { for_each = var.enable_aws ? local.aws_bootstrap_sites : {}
subnet_id = aws_subnet.private_sli[each.value.index].id route_table_id = aws_route_table.private[0].id}
resource "aws_security_group" "ce" { #checkov:skip=CKV2_AWS_5:Attached to every CE SLO and SLI ENI; Checkov does not follow counted expression references. count = var.enable_aws ? 1 : 0
name = "${local.aws_resource_prefix}-aws-ce-sg" description = "Security group for F5 XC Customer Edge nodes in AWS" vpc_id = aws_vpc.aws[0].id
ingress { description = "Site Console Local UI" from_port = 65500 to_port = 65500 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] }
ingress { description = "ICMP" from_port = -1 to_port = -1 protocol = "icmp" cidr_blocks = [var.aws_vpc_cidr] }
ingress { description = "Intra-cluster communication" from_port = 0 to_port = 0 protocol = "-1" self = true }
dynamic "ingress" { for_each = var.enable_aws_tgw_connect ? [1] : [] content { description = "GRE from the Transit Gateway Connect endpoint" from_port = 0 to_port = 0 protocol = "47" cidr_blocks = [var.aws_tgw_gre_cidr] } }
egress { #checkov:skip=CKV_AWS_382:The CE is a network appliance whose overlay and application data-plane destinations are tenant-defined; ingress remains explicitly constrained. description = "Allow all outbound traffic" from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] }
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-ce-sg" })}
# Dedicated workload VPC. The client is managed only through SSM and its# security group deliberately declares no ingress rules.resource "aws_vpc" "workload" { #checkov:skip=CKV2_AWS_11:Short-lived protected-lab workload VPC. #checkov:skip=CKV2_AWS_12:Default security group is not used by the client. count = var.enable_aws ? 1 : 0 cidr_block = var.aws_workload_vpc_cidr enable_dns_hostnames = true enable_dns_support = true tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-workload-vpc" })}
resource "aws_internet_gateway" "workload" { count = var.enable_aws ? 1 : 0 vpc_id = aws_vpc.workload[0].id tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-workload-igw" })}
resource "aws_subnet" "workload" { count = var.enable_aws ? 1 : 0 vpc_id = aws_vpc.workload[0].id cidr_block = cidrsubnet(var.aws_workload_vpc_cidr, 8, 1) availability_zone = try(data.aws_availability_zones.available[0].names[0], "${var.aws_location}a") map_public_ip_on_launch = false tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-workload-public" })}
resource "aws_route_table" "workload" { count = var.enable_aws ? 1 : 0 vpc_id = aws_vpc.workload[0].id
route { cidr_block = "0.0.0.0/0" gateway_id = aws_internet_gateway.workload[0].id }
dynamic "route" { for_each = var.enable_aws_tgw_connect ? toset([for site in values(local.aws_sites) : site.listener_ip]) : toset([]) content { cidr_block = "${route.value}/32" transit_gateway_id = module.aws_tgw_connect[0].transit_gateway_id } }
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-workload-rt" })}
resource "aws_route_table_association" "workload" { count = var.enable_aws ? 1 : 0 subnet_id = aws_subnet.workload[0].id route_table_id = aws_route_table.workload[0].id}
resource "aws_ec2_transit_gateway_vpc_attachment" "workload" { count = var.enable_aws && var.enable_aws_tgw_connect ? 1 : 0 subnet_ids = [aws_subnet.workload[0].id] transit_gateway_id = module.aws_tgw_connect[0].transit_gateway_id transit_gateway_default_route_table_association = false transit_gateway_default_route_table_propagation = false vpc_id = aws_vpc.workload[0].id tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-workload-tgw" })}
resource "aws_ec2_transit_gateway_route_table_association" "workload" { count = var.enable_aws && var.enable_aws_tgw_connect ? 1 : 0 transit_gateway_attachment_id = aws_ec2_transit_gateway_vpc_attachment.workload[0].id transit_gateway_route_table_id = module.aws_tgw_connect[0].route_table_id}
resource "aws_ec2_transit_gateway_route_table_propagation" "workload" { count = var.enable_aws && var.enable_aws_tgw_connect ? 1 : 0 transit_gateway_attachment_id = aws_ec2_transit_gateway_vpc_attachment.workload[0].id transit_gateway_route_table_id = module.aws_tgw_connect[0].route_table_id}
resource "aws_security_group" "workload" { #checkov:skip=CKV2_AWS_5:Attached directly to the workload instance through vpc_security_group_ids; Checkov does not follow the counted expression. count = var.enable_aws ? 1 : 0 name = "${local.aws_resource_prefix}-aws-workload-ssm" description = "Egress-only SSM workload client; no ingress rules" vpc_id = aws_vpc.workload[0].id
egress { description = "HTTP showcase traffic" from_port = 80 to_port = 80 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] }
egress { description = "HTTPS for SSM and showcase traffic" from_port = 443 to_port = 443 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] }
egress { description = "UDP DNS to the VPC resolver" from_port = 53 to_port = 53 protocol = "udp" cidr_blocks = ["${cidrhost(var.aws_workload_vpc_cidr, 2)}/32"] }
egress { description = "TCP DNS to the VPC resolver" from_port = 53 to_port = 53 protocol = "tcp" cidr_blocks = ["${cidrhost(var.aws_workload_vpc_cidr, 2)}/32"] }
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-workload-ssm" })}
resource "aws_security_group" "smsv2_nlb" { #checkov:skip=CKV2_AWS_5:Attached directly to the internal SMSv2 network load balancer. count = var.enable_aws && var.enable_aws_tgw_connect ? 1 : 0 name = "${local.aws_resource_prefix}-aws-smsv2-nlb" description = "Workload access to the SMSv2 site-local listeners" vpc_id = aws_vpc.workload[0].id
ingress { description = "HTTP from the workload VPC" from_port = 80 to_port = 80 protocol = "tcp" cidr_blocks = [var.aws_workload_vpc_cidr] }
egress { description = "HTTP health checks and traffic to SMSv2 SLI listeners" from_port = 80 to_port = 80 protocol = "tcp" cidr_blocks = [var.aws_vpc_cidr] }
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-smsv2-nlb" })}
resource "aws_lb" "smsv2" { #checkov:skip=CKV2_AWS_20:Internal TCP NLB; HTTP redirects are an ALB listener capability. #checkov:skip=CKV_AWS_91:Ephemeral private development NLB; protected evidence captures health and traffic. #checkov:skip=CKV_AWS_150:Ephemeral development topology; deletion protection would block authorized teardown. count = var.enable_aws && var.enable_aws_tgw_connect ? 1 : 0 name = "${local.aws_resource_prefix}-aws-nlb" internal = true load_balancer_type = "network" security_groups = [aws_security_group.smsv2_nlb[0].id] enable_cross_zone_load_balancing = true
subnet_mapping { subnet_id = aws_subnet.workload[0].id private_ipv4_address = var.aws_vip }
lifecycle { precondition { condition = var.aws_vip == cidrhost(aws_subnet.workload[0].cidr_block, 10) error_message = "aws_vip must be host 10 of the workload subnet reserved for the internal SMSv2 NLB." } }
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-smsv2" })}
resource "aws_lb_target_group" "smsv2" { count = var.enable_aws && var.enable_aws_tgw_connect ? 1 : 0 name = "${local.aws_resource_prefix}-aws-nlb" port = 80 protocol = "TCP" target_type = "ip" vpc_id = aws_vpc.workload[0].id
health_check { enabled = true healthy_threshold = 2 interval = 10 port = "traffic-port" protocol = "TCP" unhealthy_threshold = 2 }
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-smsv2" })}
resource "aws_lb_target_group_attachment" "smsv2" { for_each = var.enable_aws && var.enable_aws_tgw_connect ? local.aws_sites : {}
target_group_arn = aws_lb_target_group.smsv2[0].arn target_id = each.value.listener_ip port = 80 availability_zone = "all"}
resource "aws_lb_listener" "smsv2" { count = var.enable_aws && var.enable_aws_tgw_connect ? 1 : 0 load_balancer_arn = aws_lb.smsv2[0].arn port = 80 protocol = "TCP"
default_action { type = "forward" target_group_arn = aws_lb_target_group.smsv2[0].arn }}
resource "aws_iam_role" "workload" { count = var.enable_aws ? 1 : 0 name = "${local.aws_resource_prefix}-aws-workload-ssm" assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [{ Effect = "Allow" Action = "sts:AssumeRole" Principal = { Service = "ec2.amazonaws.com" } }] }) tags = local.tags}
resource "aws_iam_role_policy_attachment" "workload_ssm" { count = var.enable_aws ? 1 : 0 role = aws_iam_role.workload[0].name policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"}
resource "aws_iam_instance_profile" "workload" { count = var.enable_aws ? 1 : 0 name = "${local.aws_resource_prefix}-aws-workload-ssm" role = aws_iam_role.workload[0].name tags = local.tags}
resource "aws_instance" "workload" { #checkov:skip=CKV_AWS_88:The ingress-free UAT client needs an explicit public IP for SSM and Internet origin checks without a NAT gateway. count = var.enable_aws ? 1 : 0 ami = var.aws_workload_ami_id instance_type = "t3.micro" subnet_id = aws_subnet.workload[0].id ebs_optimized = true vpc_security_group_ids = [aws_security_group.workload[0].id] iam_instance_profile = aws_iam_instance_profile.workload[0].name associate_public_ip_address = true monitoring = true
metadata_options { http_tokens = "required" }
root_block_device { encrypted = true }
lifecycle { precondition { condition = var.aws_workload_ami_id != null error_message = "AWS workload deployment requires an explicit approved aws_workload_ami_id; dynamic AMI selection is not allowed." } }
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-ssm-client" })}terraform/aws_ce.tf
หัวข้อที่มีชื่อว่า “terraform/aws_ce.tf”AWS Customer Edge EC2 nodes, IAM role/profile, key pair, dual network interfaces, Elastic IPs, cloud-init user_data, site registration lookup, and automatic registration approval.
# ---------------------------------------------------------# AWS Customer Edge (EC2 instances, IAM, and ordered dual NICs)# ---------------------------------------------------------locals { aws_ssh_public_key = var.aws_ssh_public_key != "" ? var.aws_ssh_public_key : local.ssh_public_key aws_ce_site_cloud_init = { for key in keys(local.aws_active_sites) : key => replace( try(data.xcsh_site_cloud_init.aws[key].cloud_init_config, ""), "{{ .token }}", try(xcsh_token.aws[key].uid, "{{ .token }}"), ) }}resource "aws_key_pair" "ce" { count = var.enable_aws ? 1 : 0 key_name = "${local.aws_resource_prefix}-aws-ce-key" public_key = local.aws_ssh_public_key
tags = local.tags}
resource "aws_iam_role" "ce" { count = var.enable_aws ? 1 : 0 name = "${local.aws_resource_prefix}-aws-ce-role"
assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [{ Action = "sts:AssumeRole" Effect = "Allow" Principal = { Service = "ec2.amazonaws.com" } }] })
tags = local.tags}
resource "aws_iam_role_policy" "ce" { count = var.enable_aws ? 1 : 0 name = "${local.aws_resource_prefix}-aws-ce-policy" role = aws_iam_role.ce[0].id
policy = jsonencode({ Version = "2012-10-17" Statement = [{ Action = [ "ec2:DescribeInstances", "ec2:DescribeRouteTables", "ec2:DescribeSecurityGroups", "ec2:DescribeSubnets", "ec2:DescribeVpcs" ] Effect = "Allow" Resource = "*" }] })}
resource "aws_iam_instance_profile" "ce" { count = var.enable_aws ? 1 : 0 name = "${local.aws_resource_prefix}-aws-ce-profile" role = aws_iam_role.ce[0].id
tags = local.tags}
# Dual NICs per Customer Edge noderesource "aws_network_interface" "slo" { count = var.enable_aws ? var.aws_ce_count : 0
subnet_id = aws_subnet.public_slo[count.index % 3].id security_groups = [aws_security_group.ce[0].id] source_dest_check = false
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-ce-${count.index + 1}-slo" })}
resource "aws_network_interface" "sli" { count = var.enable_aws ? var.aws_ce_count : 0
subnet_id = aws_subnet.private_sli[count.index % 3].id private_ips = [local.aws_sites[format("%02d", count.index + 1)].listener_ip] security_groups = [aws_security_group.ce[0].id] source_dest_check = false
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-ce-${count.index + 1}-sli" })}
resource "aws_eip" "ce" { #checkov:skip=CKV2_AWS_19:Lab Elastic IP - attached to SLO network interface on CE instance count = var.enable_aws ? var.aws_ce_count : 0
domain = "vpc" network_interface = aws_network_interface.slo[count.index].id
depends_on = [aws_internet_gateway.aws]
tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-ce-${count.index + 1}-eip" })}
resource "aws_instance" "ce" { # Retirement must destroy only the bootstrap CE instances while preserving # ENIs, EIPs, VPC, IAM, and all other AWS networking for configured creation. count = var.enable_aws && var.aws_site_configuration_phase != "bootstrap_retirement" ? var.aws_ce_count : 0
ami = var.aws_ce_ami_id ebs_optimized = true instance_type = var.aws_instance_type iam_instance_profile = aws_iam_instance_profile.ce[0].name key_name = aws_key_pair.ce[0].key_name monitoring = true
network_interface { network_interface_id = aws_network_interface.slo[count.index].id device_index = 0 }
network_interface { network_interface_id = aws_network_interface.sli[count.index].id device_index = 1 }
root_block_device { delete_on_termination = true encrypted = true volume_size = 100 volume_type = "gp3" }
metadata_options { http_tokens = "required" }
# Preserve the provider-issued cloud-config as the only cloud-config MIME part. # A boothook and final shell part enforce SLO-only default routing and install # operator access without replacing the provider's /etc/vpm/user_data list. user_data_replace_on_change = true user_data = templatefile("${path.module}/cloud-init/ce-node-aws.multipart.tpl", { site_cloud_init = local.aws_ce_site_cloud_init[format("%02d", count.index + 1)] sli_mac = aws_network_interface.sli[count.index].mac_address fqdn = "${local.aws_active_sites[format("%02d", count.index + 1)].hostname}.${var.aws_location}.compute.internal" ssh_public_key = chomp(local.aws_ssh_public_key) # These non-secret fingerprints make the EC2 lifecycle follow immutable # site-version changes. A replacement site needs a first-boot CE, never a # VM that has already consumed the former cloud-init. software_version = var.aws_software_version os_version = var.aws_os_version })
tags = merge(local.tags, { Name = local.aws_active_sites[format("%02d", count.index + 1)].name "ves-io-site-name" = local.aws_active_sites[format("%02d", count.index + 1)].name "kubernetes.io/cluster/${local.aws_active_sites[format("%02d", count.index + 1)].name}" = "owned" })
lifecycle { # The discovery-to-configured site update regenerates cloud-init metadata, # but the already-registered CE must remain in place to receive that # control-plane reconciliation. First creation still uses user_data. ignore_changes = [user_data]
precondition { condition = var.aws_ce_ami_id != null error_message = "AWS CE deployment requires an explicit approved aws_ce_ami_id; dynamic AMI selection is not allowed." } }}terraform/aws_xc.tf
หัวข้อที่มีชื่อว่า “terraform/aws_xc.tf”AWS SecureMesh v2 site (xcsh_securemesh_site_v2.aws), eBGP peering (xcsh_bgp.aws_ebgp), virtual site (xcsh_virtual_site.aws), origin pool (xcsh_origin_pool.aws), and HTTP load balancer (xcsh_http_loadbalancer.aws).
# ---------------------------------------------------------# Three independent F5 XC SecureMesh v2 sites and AWS VIP# ---------------------------------------------------------
locals { aws_sites = { for index in range(var.enable_aws ? var.aws_ce_count : 0) : format("%02d", index + 1) => { index = index name = format("%s-aws-%s-%02d", local.site_prefix, var.aws_location, index + 1) # XC embeds the site and node names in generated child interface names. # Keep the hostname compact so the resulting identity remains within the # platform's 128-byte name limit even when the site name is region-scoped. hostname = format("%s-aws-%02d", var.component, index + 1) listener_ip = cidrhost(cidrsubnet(var.aws_vpc_cidr, 8, index + 11), 10) } } # Bootstrap identities are disposable and deliberately differ from the final # site identities. The retirement plan deletes these objects before a final # site is ever created on the retained ENIs. aws_bootstrap_sites = { for key, site in local.aws_sites : key => merge(site, { name = "${site.name}-bootstrap" hostname = "${site.hostname}-bootstrap" }) if contains(var.aws_bootstrap_site_keys, key) } aws_active_sites = var.aws_site_configuration_phase == "bootstrap" ? local.aws_bootstrap_sites : ( var.aws_site_configuration_phase == "configured" ? local.aws_sites : {} ) aws_ce_hostnames = [for site in values(local.aws_active_sites) : site.hostname]}
# During bootstrap this is the authoritative observed guest-hardware inventory.# It is intentionally absent from configured creation: final sites do not yet# have registrations until their CEs boot. The sensitive projection is consumed# privately after bootstrap, MAC-joined with Terraform-owned ENIs, then deleted.data "xcsh_site_registrations_by_site" "aws_bootstrap" { for_each = var.aws_site_configuration_phase == "bootstrap" ? local.aws_bootstrap_sites : {}
namespace = "system" site_name = xcsh_securemesh_site_v2.aws[each.key].name}
locals { # This private artifact is generated from bootstrap registrations, joined to # Terraform-owned ENI MACs, then retained across retirement. It is never an # arbitrary device input and it must not be committed. aws_device_mapping_document = var.aws_site_configuration_phase == "configured" ? jsondecode(file(var.aws_smsv2_device_mapping_file)) : { schema_version = 1 entries = [] checksum = "" } aws_device_mapping_entries = try(local.aws_device_mapping_document.entries, []) aws_device_mapping_payload = { schema_version = try(local.aws_device_mapping_document.schema_version, 0) entries = local.aws_device_mapping_entries } aws_discovered_device_candidates = { for key, site in local.aws_sites : key => { slo = [ for entry in local.aws_device_mapping_entries : entry if try(entry.site_key, "") == key && try(entry.role, "") == "slo" && try(lower(entry.mac), "") == lower(aws_network_interface.slo[site.index].mac_address) ] sli = [ for entry in local.aws_device_mapping_entries : entry if try(entry.site_key, "") == key && try(entry.role, "") == "sli" && try(lower(entry.mac), "") == lower(aws_network_interface.sli[site.index].mac_address) ] } } aws_discovered_devices = { for key, site in local.aws_sites : key => { slo = try(trimspace(one(local.aws_discovered_device_candidates[key].slo).device), null) sli = try(trimspace(one(local.aws_discovered_device_candidates[key].sli).device), null) } } aws_mapping_keys = [for entry in local.aws_device_mapping_entries : "${try(entry.site_key, "")}:${try(entry.role, "")}"] aws_mapping_is_complete = var.aws_site_configuration_phase != "configured" || ( try(local.aws_device_mapping_document.schema_version, 0) == 1 && try(local.aws_device_mapping_document.checksum, "") == sha256(jsonencode(local.aws_device_mapping_payload)) && length(local.aws_device_mapping_entries) == 2 * length(local.aws_sites) && length(distinct(local.aws_mapping_keys)) == length(local.aws_mapping_keys) && alltrue([for entry in local.aws_device_mapping_entries : contains(keys(local.aws_sites), try(entry.site_key, "")) && contains(["slo", "sli"], try(entry.role, "")) && can(regex("^[0-9a-f]{2}(:[0-9a-f]{2}){5}$", lower(try(entry.mac, "")))) && try(length(trimspace(entry.device)), 0) > 0 ]) )}
resource "xcsh_token" "aws" { for_each = local.aws_active_sites
# XC validates token names as DNS-1035 labels, whose maximum length is 63. name = local.deployment_is_production ? substr("${each.value.name}-registration", 0, 63) : format("%s-aws-reg-%s-%s", local.aws_resource_prefix, each.key, var.aws_site_configuration_phase) namespace = "system" description = "Registration token for independent AWS site ${each.value.name}" labels = local.xc_labels type = 1 site_name = xcsh_securemesh_site_v2.aws[each.key].name}
resource "xcsh_securemesh_site_v2" "aws" { for_each = local.aws_active_sites name = each.value.name namespace = "system" description = "Independent AWS Customer Edge SecureMesh v2 site ${each.key}" labels = local.xc_labels
aws { not_managed { dynamic "node_list" { for_each = var.aws_site_configuration_phase == "configured" ? [each.value] : []
content { hostname = node_list.value.hostname type = "Control" public_ip = null
interface_list { name = "slo" mtu = var.aws_smsv2_interface_mtu ethernet_interface { device = local.aws_discovered_devices[each.key].slo mac = aws_network_interface.slo[node_list.value.index].mac_address } network_option { site_local_network = {} } dhcp_client = {} }
interface_list { name = "sli" mtu = var.aws_smsv2_interface_mtu ethernet_interface { device = local.aws_discovered_devices[each.key].sli mac = aws_network_interface.sli[node_list.value.index].mac_address } network_option { site_local_inside_network = {} } dhcp_client = {} } } } } }
disable_ha = {} block_all_services = {} no_network_policy = {} no_forward_proxy = {} f5_proxy = {} no_proxy_bypass = {} logs_streaming_disabled = {} no_s2s_connectivity_sli = {} no_s2s_connectivity_slo = {} disable_url_categorization = {} disable_management_network = {}
local_vrf { default_config = {} default_sli_config = {} }
software_settings { # First boot must request the field-proven runtime pair. A staged # baseline leaves a newly created CE in UPGRADE_IN_PROGRESS before the # post-bootstrap action stage can observe or recover it. os { operating_system_version = var.aws_os_version } sw { volterra_software_version = var.aws_software_version } }
lifecycle { precondition { condition = var.aws_site_configuration_phase == "bootstrap" || ( local.aws_mapping_is_complete && length(local.aws_discovered_device_candidates[each.key].slo) == 1 && length(local.aws_discovered_device_candidates[each.key].sli) == 1 && try(length(local.aws_discovered_devices[each.key].slo), 0) > 0 && try(length(local.aws_discovered_devices[each.key].sli), 0) > 0 && local.aws_discovered_devices[each.key].slo != local.aws_discovered_devices[each.key].sli ) error_message = "Configured AWS SMSv2 requires a checksummed bootstrap mapping with exactly one nonempty observed device for each Terraform-owned SLO/SLI ENI MAC, no duplicate or foreign entry, and distinct devices; do not guess guest device names." } }}
data "xcsh_site_cloud_init" "aws" { # The console supplies a template, not a mutable cloud-init resource. The # separately-issued, site-bound JWT is substituted by aws_ce.tf. for_each = local.aws_active_sites provider_ref = "aws" site_name = xcsh_securemesh_site_v2.aws[each.key].name enable_management_network = false}
data "xcsh_site_registration" "aws" { for_each = local.aws_active_sites
site_name = each.value.name hostname = each.value.hostname namespace = "system"
}
resource "xcsh_registration_approval" "aws" { for_each = { for key, registration in data.xcsh_site_registration.aws : key => registration if registration.found && registration.state == "NEW" }
namespace = "system" name = each.value.name cluster_size = 1 state = "APPROVED"
depends_on = [xcsh_securemesh_site_v2.aws]}
resource "xcsh_virtual_site" "aws" { count = var.enable_aws ? 1 : 0 name = "${local.aws_resource_prefix}-aws-vsite" namespace = data.xcsh_namespace.mcn.name labels = local.xc_labels
site_type = "CUSTOMER_EDGE" site_selector { expressions = ["mcn-topology in (${local.site_prefix}-aws)"] }}
resource "xcsh_origin_pool" "aws" { count = var.enable_aws ? 1 : 0 name = "${local.aws_resource_prefix}-aws-pool" namespace = data.xcsh_namespace.mcn.name description = "AWS origin pool serving the three-site TGW showcase" labels = local.xc_labels port = var.origin_port
origin_servers { labels = {} public_name { dns_name = var.aws_origin_dns_name } }
no_tls = {} loadbalancer_algorithm = "ROUND_ROBIN" endpoint_selection = "DISTRIBUTED"
}
resource "xcsh_http_loadbalancer" "aws" { count = var.enable_aws ? 1 : 0 name = "${local.aws_resource_prefix}-aws-lb" namespace = data.xcsh_namespace.mcn.name domains = [local.aws_lb_domain] labels = local.xc_labels
# Final-site names are intentionally derived from stable locals so the # retirement stage can retain this object without dereferencing an empty # site map. Preserve the creation ordering explicitly for configured apply. depends_on = [xcsh_securemesh_site_v2.aws]
http { port = 80 }
advertise_custom { dynamic "advertise_where" { # Retirement retains the load balancer and its advertisement shape while # bootstrap sites are removed, avoiding an empty resource-map lookup or # a retained-object mutation during the retirement-only phase. for_each = var.aws_site_configuration_phase == "configured" ? local.aws_sites : local.aws_bootstrap_sites content { site { network = "SITE_NETWORK_INSIDE" site { name = advertise_where.value.name namespace = "system" } } use_default_port = {} } } }
default_route_pools { pool { name = xcsh_origin_pool.aws[0].name namespace = data.xcsh_namespace.mcn.name } weight = 1 priority = 1 }
round_robin = {} no_challenge = {} user_id_client_ip = {} disable_waf = {} disable_rate_limit = {} disable_api_discovery = {} disable_api_testing = {} disable_api_definition = {} l7_ddos_protection {}}terraform/aws_tgw_connect.tf
หัวข้อที่มีชื่อว่า “terraform/aws_tgw_connect.tf”Strict provider-v6 orchestration for the immutable SMSv2 contract, MAC-bound runtime health, AWS Transit Gateway Connect peers, F5 XC external connectors and BGP, and bounded route convergence.
# AWS owns ENI, TGW, Connect, GRE, and inside-CIDR facts. F5 XC owns# SMSv2 configuration, health, BGP, and route observations.locals { # Keep the immutable source revision machine-readable without resembling an # access token to secret scanners. The evaluated value is the full release # commit recorded by the contract data source. aws_smsv2_api_release_commit = format("%s%s", "64ef458aad9d4f149b18", "0214ee7cc7954e40112d") aws_smsv2_bindings = var.enable_aws && var.enable_aws_tgw_connect && var.aws_site_configuration_phase == "configured" ? merge( { for index in range(var.enable_aws ? var.aws_ce_count : 0) : format("node_%02d_slo", index + 1) => { index = index site_key = format("%02d", index + 1) site = local.aws_sites[format("%02d", index + 1)].name order = index node = local.aws_ce_hostnames[index] role = "slo" # XC rejects GRE connectors whose transport and payload networks are # both Site Local Outside. Keep the payload in Site Local Inside even # when the bound transport interface is SLO. payload_role = "sli" mac = aws_network_interface.slo[index].mac_address gre_peer_address = aws_network_interface.slo[index].private_ip inside_cidr_block = cidrsubnet(var.aws_tgw_inside_cidr, 5, index) } }, { for index in range(var.enable_aws ? var.aws_ce_count : 0) : format("node_%02d_sli", index + 1) => { index = index site_key = format("%02d", index + 1) site = local.aws_sites[format("%02d", index + 1)].name order = var.aws_ce_count + index node = local.aws_ce_hostnames[index] role = "sli" payload_role = "sli" mac = aws_network_interface.sli[index].mac_address gre_peer_address = aws_network_interface.sli[index].private_ip inside_cidr_block = cidrsubnet(var.aws_tgw_inside_cidr, 5, var.aws_ce_count + index) } }, ) : {} # Keep the live routing graph inside the same cumulative boundary as token # issuance and cloud-init. This lets each CE reach ONLINE and converge before # the next site is admitted without evaluating absent nodes from later stages. aws_bootstrap_smsv2_bindings = { for key, binding in local.aws_smsv2_bindings : key => binding if contains(var.aws_bootstrap_site_keys, binding.site_key) } # Session keys are known during planning; AWS supplies the two addresses. aws_bgp_sessions = merge([ for key, binding in(var.enable_aws_tgw_connect ? local.aws_bootstrap_smsv2_bindings : {}) : { for endpoint in range(2) : "${key}_${endpoint + 1}" => merge(binding, { connector_key = key peer_address = sort(tolist(aws_ec2_transit_gateway_connect_peer.aws[key].bgp_transit_gateway_addresses))[endpoint] }) } ]...) aws_smsv2_nodes = { for key, interface in local.aws_bootstrap_smsv2_bindings : key => { node = interface.node role = interface.role mac = interface.mac } }}
data "xcsh_smsv2_contract" "aws" { count = var.enable_aws_tgw_connect ? 1 : 0}
resource "terraform_data" "aws_tgw_contract_gate" { count = var.enable_aws_tgw_connect ? 1 : 0 input = { contract_id = data.xcsh_smsv2_contract.aws[0].contract_id contract_version = data.xcsh_smsv2_contract.aws[0].contract_version api_release_tag = data.xcsh_smsv2_contract.aws[0].api_release_tag api_release_commit = data.xcsh_smsv2_contract.aws[0].api_release_commit telemetry_schema_id = data.xcsh_smsv2_contract.aws[0].telemetry_schema_id capabilities = data.xcsh_smsv2_contract.aws[0].capabilities f5xc_authorities = data.xcsh_smsv2_contract.aws[0].f5xc_authorities aws_authorities = data.xcsh_smsv2_contract.aws[0].aws_authorities }
lifecycle { precondition { condition = var.enable_aws error_message = "AWS TGW Connect requires enable_aws = true." } precondition { condition = var.aws_ce_count == 3 error_message = "AWS TGW Connect requires the validated three-node, six-interface topology." } precondition { condition = ( data.xcsh_smsv2_contract.aws[0].contract_id == "f5xc-smsv2-api/v1" && data.xcsh_smsv2_contract.aws[0].contract_version == "7.0.0" && data.xcsh_smsv2_contract.aws[0].api_release_tag == "v8.0.0" && data.xcsh_smsv2_contract.aws[0].api_release_commit == local.aws_smsv2_api_release_commit && data.xcsh_smsv2_contract.aws[0].telemetry_schema_id == "f5xc-smsv2-aws-tgw-telemetry/v2" ) error_message = "Provider v11.0.2 must expose the exact immutable SMSv2 API v8.0.0 contract." } precondition { condition = ( length(data.xcsh_smsv2_contract.aws[0].capabilities) == 5 && try(data.xcsh_smsv2_contract.aws[0].capabilities["aws_ce_create"], "") == "available" && try(data.xcsh_smsv2_contract.aws[0].capabilities["aws_node_configuration"], "") == "available" && try(data.xcsh_smsv2_contract.aws[0].capabilities["runtime_status"], "") == "available" && try(data.xcsh_smsv2_contract.aws[0].capabilities["tgw_connect"], "") == "available" && try(data.xcsh_smsv2_contract.aws[0].capabilities["site_upgrade"], "") == "available" ) error_message = "Provider v11.0.2 must publish all and only the required SMSv2 capabilities, including evidence-backed AWS node configuration, as available." } precondition { condition = try( jsondecode(data.xcsh_smsv2_contract.aws[0].aws_node_configuration).strategy == "discovery_rebuild" && jsondecode(data.xcsh_smsv2_contract.aws[0].aws_node_configuration).enforcement == "required" && jsondecode(data.xcsh_smsv2_contract.aws[0].aws_node_configuration).invariants.device_source == "observed_registration_only", false, ) error_message = "AWS configured creation requires the released discovery_rebuild contract with observed-registration-only device mapping." } precondition { condition = ( length(data.xcsh_smsv2_contract.aws[0].f5xc_authorities) == 6 && toset(data.xcsh_smsv2_contract.aws[0].f5xc_authorities) == toset([ "smsv2_configuration", "runtime_health", "bgp_peers", "bgp_routes", "simplified_routes", "site_upgrade_observation", ]) && length(data.xcsh_smsv2_contract.aws[0].aws_authorities) == 6 && toset(data.xcsh_smsv2_contract.aws[0].aws_authorities) == toset([ "eni", "transit_gateway", "transit_gateway_connect", "gre_endpoints", "bgp_inside_cidrs", "autonomous_system_numbers", ]) ) error_message = "The SMSv2 contract authority split does not match this deployment." } }}
module "aws_tgw_connect" { count = var.enable_aws && var.enable_aws_tgw_connect ? 1 : 0 source = "./modules/aws-tgw-connect" vpc_id = aws_vpc.aws[0].id amazon_side_asn = var.aws_tgw_asn transit_gateway_cidr_block = var.aws_tgw_gre_cidr transport_subnet_ids = aws_subnet.private_sli[*].id name_prefix = local.aws_resource_prefix ownership_tags = local.tags depends_on = [terraform_data.aws_tgw_contract_gate]}
data "xcsh_smsv2_aws_runtime" "aws" { for_each = var.enable_aws && var.enable_aws_tgw_connect ? local.aws_bootstrap_sites : {} namespace = "system" site = xcsh_securemesh_site_v2.aws[each.key].name nodes = { for key, node in local.aws_smsv2_nodes : key => node if local.aws_smsv2_bindings[key].site_key == each.key } timeout_seconds = var.aws_runtime_convergence_timeout_seconds poll_interval_seconds = var.aws_bgp_poll_interval_seconds # Runtime health cannot exist until the instance has consumed the site # cloud-init and registered. Without this ordering Terraform can admit all # polling data sources before the EC2 key/profile/instances, starving the # bootstrap graph with waits for a runtime that it has not yet created. depends_on = [aws_instance.ce]}
resource "terraform_data" "aws_tgw_runtime_gate" { count = var.enable_aws && var.enable_aws_tgw_connect ? 1 : 0 input = { healthy = alltrue([for runtime in values(data.xcsh_smsv2_aws_runtime.aws) : runtime.healthy]) interfaces = merge([for runtime in values(data.xcsh_smsv2_aws_runtime.aws) : runtime.interfaces]...) } lifecycle { precondition { condition = ( alltrue([for runtime in values(data.xcsh_smsv2_aws_runtime.aws) : runtime.healthy]) && sum([for runtime in values(data.xcsh_smsv2_aws_runtime.aws) : length(runtime.interfaces)]) == 2 * length(local.aws_bootstrap_sites) && alltrue([ for interface in flatten([for runtime in values(data.xcsh_smsv2_aws_runtime.aws) : values(runtime.interfaces)]) : interface.healthy && interface.mtu == var.aws_smsv2_interface_mtu && contains(["slo", "sli"], interface.role) ]) ) error_message = "Every admitted MAC-bound SMSv2 interface must agree on node/role/MTU and report healthy before its AWS Connect peer is created." } }}
# A target rooted at one site's BGP status must still install both subnet# associations. Without the SLI association, the SLO GRE sessions establish# while both SLI sessions remain down because 100.64.0.0/24 follows the VPC's# main route table instead of the TGW route.resource "terraform_data" "aws_tgw_site_route_gate" { for_each = var.enable_aws && var.enable_aws_tgw_connect ? local.aws_bootstrap_sites : {} input = { public_association_id = aws_route_table_association.public[each.key].id private_association_id = aws_route_table_association.private[each.key].id }}
resource "aws_ec2_transit_gateway_connect_peer" "aws" { for_each = var.enable_aws && var.enable_aws_tgw_connect ? local.aws_bootstrap_smsv2_bindings : {} bgp_asn = tostring(var.aws_ce_bgp_asn) inside_cidr_blocks = [each.value.inside_cidr_block] peer_address = each.value.gre_peer_address transit_gateway_address = cidrhost(var.aws_tgw_gre_cidr, each.value.order + 1) transit_gateway_attachment_id = module.aws_tgw_connect[0].connect_attachment_ids[each.value.role] tags = merge(local.tags, { Name = "${local.aws_resource_prefix}-aws-tgw-peer-${replace(each.key, "_", "-")}" }) depends_on = [terraform_data.aws_tgw_runtime_gate]}
resource "xcsh_external_connector" "aws_tgw" { for_each = var.enable_aws && var.enable_aws_tgw_connect ? local.aws_bootstrap_smsv2_bindings : {} name = "${local.aws_resource_prefix}-aws-tgw-${replace(each.key, "_", "-")}" namespace = "system" description = "AWS TGW Connect GRE tunnel for ${each.key}." labels = local.xc_labels ce_site_reference { name = xcsh_securemesh_site_v2.aws[each.value.site_key].name namespace = "system" } gre { gre_parameters { site_local_network = each.value.payload_role == "slo" ? {} : null site_local_inside_network = each.value.payload_role == "sli" ? {} : null # The external-connector API caps GRE MTU at 1370. Preserve a smaller # observed underlay ceiling while never constructing an invalid request. tunnel_mtu = min(data.xcsh_smsv2_aws_runtime.aws[each.value.site_key].interfaces[each.key].mtu - 24, 1370) peer_ip_address { addr = aws_ec2_transit_gateway_connect_peer.aws[each.key].transit_gateway_address } tunnel_eps { node = data.xcsh_smsv2_aws_runtime.aws[each.value.site_key].interfaces[each.key].node interface = data.xcsh_smsv2_aws_runtime.aws[each.value.site_key].interfaces[each.key].interface_name local_tunnel_ip = "${aws_ec2_transit_gateway_connect_peer.aws[each.key].bgp_peer_address}/29" remote_tunnel_ip = "${sort(tolist(aws_ec2_transit_gateway_connect_peer.aws[each.key].bgp_transit_gateway_addresses))[0]}/29" } } } depends_on = [terraform_data.aws_tgw_site_route_gate]}
resource "xcsh_bgp" "aws_tgw" { for_each = var.enable_aws && var.enable_aws_tgw_connect ? local.aws_bootstrap_sites : {} name = "${local.aws_resource_prefix}-aws-tgw-bgp-${each.key}" namespace = "system" description = "Four-session AWS TGW Connect BGP for independent site ${each.value.name}." labels = local.xc_labels where { site { # The external-connector API accepts TGW payload only in Site Local # Inside, independently of whether GRE transport uses SLO or SLI. network_type = "VIRTUAL_NETWORK_SITE_LOCAL_INSIDE" ref { name = xcsh_securemesh_site_v2.aws[each.key].name namespace = "system" } } } bgp_parameters { asn = var.aws_ce_bgp_asn local_address = {} } dynamic "peers" { for_each = { for key, session in local.aws_bgp_sessions : key => session if session.site_key == each.key } content { metadata { name = replace(peers.key, "_", "-") } external { asn = module.aws_tgw_connect[0].amazon_side_asn address = peers.value.peer_address port = 179 family_inet { enable {} } interface { name = "ves-io-external-connector-${xcsh_external_connector.aws_tgw[peers.value.connector_key].name}" namespace = "system" } disable_v6 = {} } passive_mode_disabled = {} bfd_disabled = {} } } depends_on = [xcsh_external_connector.aws_tgw]}
data "xcsh_site_bgp_status" "aws" { for_each = var.enable_aws && var.enable_aws_tgw_connect ? local.aws_bootstrap_sites : {} namespace = "system" site = xcsh_securemesh_site_v2.aws[each.key].name expected_peers = { for key, interface in local.aws_bgp_sessions : key => { node = interface.node role = interface.payload_role mac = interface.mac peer_address = interface.peer_address expected_imported_routes = [var.aws_workload_vpc_cidr] } if interface.site_key == each.key } expected_exported_routes = ["${each.value.listener_ip}/32"] timeout_seconds = var.aws_bgp_convergence_timeout_seconds poll_interval_seconds = var.aws_bgp_poll_interval_seconds depends_on = [ xcsh_bgp.aws_tgw, module.aws_tgw_connect, aws_ec2_transit_gateway_route_table_association.workload, aws_ec2_transit_gateway_route_table_propagation.workload, xcsh_http_loadbalancer.aws, ]}
output "aws_tgw_connect_status" { description = "Non-sensitive SMSv2 contract, runtime, and BGP convergence summary." value = var.enable_aws && var.enable_aws_tgw_connect ? { contract_id = data.xcsh_smsv2_contract.aws[0].contract_id contract_version = data.xcsh_smsv2_contract.aws[0].contract_version api_release = data.xcsh_smsv2_contract.aws[0].api_release_tag telemetry_schema = data.xcsh_smsv2_contract.aws[0].telemetry_schema_id runtime_healthy = alltrue([for runtime in values(data.xcsh_smsv2_aws_runtime.aws) : runtime.healthy]) interface_count = sum([for runtime in values(data.xcsh_smsv2_aws_runtime.aws) : length(runtime.interfaces)]) bgp_converged = alltrue([for status in values(data.xcsh_site_bgp_status.aws) : status.converged]) connect_peer_count = length(aws_ec2_transit_gateway_connect_peer.aws) bgp_session_count = sum([for status in values(data.xcsh_site_bgp_status.aws) : length(status.peers)]) } : null}terraform/variables_azure.tf
หัวข้อที่มีชื่อว่า “terraform/variables_azure.tf”Subscription, region, CIDRs, Bastion, Canadian ILB variant option, and the Azure object names.
# ---------------------------------------------------------# Azure subscription / placement# ---------------------------------------------------------
variable "subscription_id" { description = "Azure subscription ID used by the azurerm provider." type = string # Default = the <AZURE_SUBSCRIPTION_ID> subscription the lab was built in. default = "00000000-0000-0000-0000-000000000000"}
variable "resource_group_name" { description = "Resource group that holds the hub VNet, Route Server, CE VMs and test client. Leave null (the default) to derive `rg-<component>-<deployer>`, where the deployer is resolved from Azure AD — so a fresh clone names the group after whoever deploys it rather than after whoever wrote it." type = string default = null}
variable "location" { description = "Azure region for all resources." type = string default = "eastus"}
variable "enable_azure" { description = "Deploy the Azure US and Canada SMSv2 graphs. Keep true for the full showcase; set false only for the separately reviewed AWS-only saved-plan and preflight stage." type = bool default = true}
# ---------------------------------------------------------# Canada regional Azure placement & CIDRs# ---------------------------------------------------------
variable "enable_canada_ilb" { description = "Deploy Azure Internal Load Balancer (ILB) in front of Canadian CEs as an alternative to Azure Route Server (eBGP/ECMP)." type = bool default = true}
variable "enable_azure_ilb" { description = "Deploy the Azure US Internal Load Balancer used by the supported non-Route-Server SMSv2 showcase path." type = bool default = true}
variable "ca_location" { description = "Azure region for Canadian regional resources." type = string default = "canadacentral"}
variable "ca_resource_group_name" { description = "Resource group that holds the Canadian hub VNet, Route Server, CE VMs and test client. Leave null (the default) to derive `rg-<component>-ca-<deployer>`." type = string default = null}
variable "ca_hub_cidr" { description = "Canada Hub VNet address space." type = string default = "10.200.0.0/16"}
variable "ca_mgmt_subnet_prefix" { description = "Canada snet-hub-management prefix. CE eth0/SLO NICs live here." type = string default = "10.200.1.0/26"}
variable "ca_external_subnet_prefix" { description = "Canada snet-hub-external prefix." type = string default = "10.200.2.0/26"}
variable "ca_internal_subnet_prefix" { description = "Canada snet-hub-internal prefix. Test client lives here." type = string default = "10.200.3.0/26"}
variable "ca_route_server_subnet_prefix" { description = "Canada RouteServerSubnet prefix. MUST be exactly /27." type = string default = "10.200.4.0/27"
validation { condition = tonumber(split("/", var.ca_route_server_subnet_prefix)[1]) == 27 error_message = "ca_route_server_subnet_prefix must be a /27." }}
variable "ca_bastion_subnet_prefix" { description = "Canada AzureBastionSubnet prefix. MUST be /26 or larger." type = string default = "10.200.5.0/26"
validation { condition = tonumber(split("/", var.ca_bastion_subnet_prefix)[1]) <= 26 error_message = "ca_bastion_subnet_prefix must be /26 or larger." }}
variable "ca_vip" { description = "HA VIP for Canadian CEs advertised as a /32 via eBGP." type = string default = "10.250.1.10"
validation { condition = can(cidrhost("${var.ca_vip}/32", 0)) error_message = "ca_vip must be a valid IPv4 address." }}
variable "ca_route_server_name" { description = "Canada Azure Route Server name. Leave null to derive `<component>-ca-rs`." type = string default = null}
variable "ca_bastion_name" { description = "Canada Azure Bastion host name. Leave null to derive `<component>-ca-bastion`." type = string default = null}
variable "ca_client_vm_name" { description = "Canada test client VM name. Leave null to derive `<component>-ca-client`." type = string default = null}
variable "ca_region_short" { description = "Short region token for Canada site names. Leave null to use var.ca_location." type = string default = null}
# ---------------------------------------------------------# Network CIDRs# ---------------------------------------------------------
variable "hub_cidr" { description = "Hub VNet address space." type = string default = "10.0.0.0/16"}
variable "spoke_cidr" { description = "Spoke VNet address space (not deployed here; used only to assert the VIP is outside it)." type = string default = "10.1.0.0/16"}
variable "mgmt_subnet_prefix" { description = "snet-hub-management prefix. CE eth0/SLO NICs (BGP local address) live here." type = string default = "10.0.1.0/26"}
variable "external_subnet_prefix" { description = "snet-hub-external prefix." type = string default = "10.0.2.0/26"}
variable "internal_subnet_prefix" { description = "snet-hub-internal prefix. The test client lives here." type = string default = "10.0.3.0/26"}
variable "route_server_subnet_prefix" { description = "RouteServerSubnet prefix. MUST be exactly /27, named literally RouteServerSubnet, with no NSG or route table." type = string default = "10.0.4.0/27"
validation { condition = tonumber(split("/", var.route_server_subnet_prefix)[1]) == 27 error_message = "RouteServerSubnet must be a /27." }}
variable "route_server_name" { description = "Azure Route Server name. Leave null (the default) to derive `<component>-rs`." type = string default = null}
variable "bastion_subnet_prefix" { description = "AzureBastionSubnet prefix. MUST be /26 or larger, named literally AzureBastionSubnet, with no NSG or route table. 10.0.5.0/26 is the first free /26 in the hub (10.0.1.0/26 mgmt, 10.0.2.0/26 external, 10.0.3.0/26 internal, 10.0.4.0/27 RouteServerSubnet are taken)." type = string default = "10.0.5.0/26"
validation { # Azure rejects anything smaller than /26 at APPLY time, long after the plan # looked healthy. Fail at plan time instead. condition = tonumber(split("/", var.bastion_subnet_prefix)[1]) <= 26 error_message = "AzureBastionSubnet must be /26 or larger (a smaller prefix, e.g. /27, is rejected by Azure)." }}
# --------------------------------------------------------------------------# Azure Bastion# --------------------------------------------------------------------------# Default false, deliberately. Bastion is the only resource in this deployment# that is not load-bearing for the BGP/ECMP demo — the data path, the sites and# the VIP all work without it — yet a Standard-SKU host bills an hourly standing# charge from the moment it is provisioned, whether or not anyone opens a tunnel:# USD 0.29/hour in eastus for the two included scale units (retail, 2026-07),# roughly USD 212 a month, plus USD 0.14/hour for each scale unit beyond two.# Opting in keeps the cost of a default `terraform apply` exactly what it is# today and makes the spend a conscious choice by whoever needs CE console# access. Turn it on with `enable_bastion = true` in terraform.tfvars.variable "enable_bastion" { description = "Deploy Azure Bastion so developers can reach each CE's Site Console web UI (https://<sli-ip>:65500) with Azure RBAC instead of an SSH key and a jump host." type = bool default = false}
variable "client_vm_name" { description = "Test client VM name, and the stem of its public IP, NSG and NIC. Leave null (the default) to derive `<component>-client`. Set it only to hold an existing client VM steady: the name is also its computer_name, which forces a VM replacement when it changes." type = string default = null}
variable "bastion_name" { description = "Azure Bastion host name (the --name argument of `az network bastion tunnel`). Leave null (the default) to derive `<component>-bastion`." type = string default = null}
# ---------------------------------------------------------# CE / client VM inputs# ---------------------------------------------------------
variable "ce_vm_size" { description = "VM size for the Customer Edge nodes." type = string default = "Standard_D8_v4"}
variable "admin_username" { description = "SSH admin username for the VMs." type = string default = "azureuser"}
variable "ssh_public_key" { description = "SSH public key MATERIAL (the key string). When empty, read from ssh_public_key_path. Passing material keeps plan tests hermetic." type = string default = ""}
variable "ssh_public_key_path" { description = "Path to the SSH public key file, read once at the root when ssh_public_key is empty." type = string default = "~/.ssh/id_ed25519.pub"}terraform/ca_ilb.tf
หัวข้อที่มีชื่อว่า “terraform/ca_ilb.tf”Azure Internal Load Balancer (ILB) variant configuration for the Canadian regional path (enable_canada_ilb).
# ---------------------------------------------------------# Canadian Regional Extension: Azure Internal Load Balancer (ILB)# ---------------------------------------------------------
resource "azurerm_lb" "ca_ilb" { #checkov:skip=CKV_AZURE_27:Lab environment ILB - diagnostic logging not required count = var.enable_azure && var.enable_canada && var.enable_canada_ilb ? 1 : 0 name = "${var.component}-ca-ilb" location = var.ca_location resource_group_name = module.azure_hub_ca[0].resource_group_name sku = "Standard"
frontend_ip_configuration { name = "ca-ilb-frontend" subnet_id = module.azure_hub_ca[0].management_subnet_id private_ip_address = cidrhost(var.ca_mgmt_subnet_prefix, 10) private_ip_address_allocation = "Static" }
tags = local.tags}
resource "azurerm_lb_backend_address_pool" "ca_ce_backend" { count = var.enable_azure && var.enable_canada && var.enable_canada_ilb ? 1 : 0 name = "ca-ce-backend-pool" loadbalancer_id = azurerm_lb.ca_ilb[0].id}
resource "azurerm_network_interface_backend_address_pool_association" "ca_ce" { for_each = var.enable_azure && var.enable_canada && var.enable_canada_ilb ? module.ce_topology_ca[0].ce_nodes : {}
network_interface_id = module.ce_node_ca[each.key].mgmt_nic_id ip_configuration_name = "ipconfig1" backend_address_pool_id = azurerm_lb_backend_address_pool.ca_ce_backend[0].id}
resource "azurerm_lb_probe" "ca_site_console" { count = var.enable_azure && var.enable_canada && var.enable_canada_ilb ? 1 : 0 name = "site-console-probe" loadbalancer_id = azurerm_lb.ca_ilb[0].id protocol = "Tcp" port = 65500 interval_in_seconds = 5 number_of_probes = 2}
resource "azurerm_lb_rule" "ca_ha_ports" { count = var.enable_azure && var.enable_canada && var.enable_canada_ilb ? 1 : 0 name = "ha-ports-rule" loadbalancer_id = azurerm_lb.ca_ilb[0].id frontend_ip_configuration_name = "ca-ilb-frontend" backend_address_pool_ids = [azurerm_lb_backend_address_pool.ca_ce_backend[0].id] probe_id = azurerm_lb_probe.ca_site_console[0].id protocol = "All" frontend_port = 0 backend_port = 0 floating_ip_enabled = true idle_timeout_in_minutes = 4}terraform/variables_ce.tf
หัวข้อที่มีชื่อว่า “terraform/variables_ce.tf”CE count, site prefix, ASNs, image versions and VM size.
# ---------------------------------------------------------# CE topology / scaling# ---------------------------------------------------------
variable "ce_count" { description = "Number of single-node Secure Mesh v2 CE sites (1..3). Each originates the LB VIP via eBGP to the Azure Route Server for active/active ECMP." type = number default = 3
validation { condition = var.ce_count >= 1 && var.ce_count <= 3 error_message = "ce_count must be between 1 and 3." }}
# ---------------------------------------------------------# Canada CE topology / scaling# ---------------------------------------------------------
variable "ca_ce_count" { description = "Number of Canadian single-node Secure Mesh v2 CE sites (1..3)." type = number default = 3
validation { condition = var.ca_ce_count >= 1 && var.ca_ce_count <= 3 error_message = "ca_ce_count must be between 1 and 3." }}
variable "ca_site_prefix" { description = "Prefix for Canadian XC site names (site = <ca_site_prefix>-<ca_region_short>0<n>). Leave null to derive `<component>-ca`." type = string default = null}
variable "region_short" { description = "Short region token used to build the per-CE key and site name (component `mcn-ce-ha` + region `eastus` -> site `mcn-ce-ha-eastus01`). Leave null (the default) to use var.location verbatim; set it only when the Azure region name is longer than you want inside object names." type = string default = null}
variable "site_prefix" { description = "Prefix for XC site names (site = <prefix>-<region_short>0<n>). Leave null to use the released SMSv2 identity generation derived from var.component. Renaming a site replaces the CE VM, because the site name is the ClusterName baked into cloud-init and cloud-init only runs on first boot." type = string default = null}
variable "smsv2_site_generation" { description = "Immutable identity generation appended to the default SMSv2 site prefix. The released default deliberately avoids the legacy mcn-ce-ha-* global-name reservations; do not decrement or reuse an earlier generation." type = string default = "smsv2"
validation { condition = can(regex("^[a-z0-9]([a-z0-9-]*[a-z0-9])?$", var.smsv2_site_generation)) error_message = "smsv2_site_generation must be a DNS-style label: lowercase alphanumerics and hyphens, not starting or ending with a hyphen." }}
variable "ce_asn" { description = "BGP ASN for the Customer Edge nodes (eBGP local ASN)." type = number default = 64512}
variable "rs_asn" { description = "BGP ASN of the Azure Route Server (fixed by Azure at 65515)." type = number default = 65515}
variable "enable_bgp" { description = "Enable the unsupported Azure Route Server BGP topology. Defaults false: the supported showcase uses Azure ILB instead. Setting true fails during planning, before Terraform, Azure, or F5 mutation, with the authoritative multihop-contract reason." type = bool default = false}
variable "approve_registration" { description = "Approve each CE's runtime registration from Terraform (the xcsh_site_registration data source resolves the r-<uuid> registration name from the site name, and xcsh_registration_approval approves it). Two-phase by nature: the registration only exists after the CE has booted and registered, so the first apply plans no approval and a later apply creates it. Set false to leave approval to an operator." type = bool default = true}
# ---------------------------------------------------------# Site registration token (optional override)# ---------------------------------------------------------# The CE VM cloud-init consumes a tenant-scoped, reusable site registration# token (ClusterName + Token). The token is now provider-generated by the# xcsh_token.ce resource (its Computed uid) for Azure CE nodes; this var is an optional override.variable "registration_token" { description = "Optional override for the Azure CE cloud-init site registration token. When empty (the default), enabled Azure CE nodes use the provider-generated xcsh_token.ce[0].uid; set a non-empty value to inject an externally-minted token instead." type = string default = "" sensitive = true}
variable "ce_os_version" { description = "CE OS version, set at create time. Empty deliberately selects the newest version advertised by F5 Distributed Cloud; the CE module now enforces an 80 GB disk default for the pinned Marketplace image. A concrete value is for reproducing an older build. Terraform cannot change this field afterwards: updates are rejected in every direction, including un-pinning. The platform can change it in place through the site upgrade_os action, but the provider cannot drive that action yet (xcsh#1390)." type = string default = ""}
variable "ce_sw_version" { description = "CE F5 Distributed Cloud software version, set at create time. Empty deliberately selects the newest advertised build; the CE module now enforces an 80 GB disk default for the pinned Marketplace image. A concrete value is for reproducing an older build. The node always installs a destination build on first boot. Terraform cannot change this field afterwards, but the platform can change it in place through the site upgrade_sw action (xcsh#1390)." type = string default = ""}terraform/variables_xc.tf
หัวข้อที่มีชื่อว่า “terraform/variables_xc.tf”Tenant, app namespace, load balancer, origin pool and the VIP.
# ---------------------------------------------------------# F5 XC tenant# ---------------------------------------------------------
variable "expected_xc_tenant" { description = "F5 XC tenant this deployment belongs to: the first hostname label of the console URL (`f5-sales-demo` for https://f5-sales-demo.console.ves.volterra.io). This is the ONLY place the tenant is named. The xcsh provider's api_url is derived from it, so the configuration — not the ambient environment — decides which tenant is written to, and a plan FAILS when XCSH_API_URL in the environment names a different one." type = string default = "f5-sales-demo"
validation { # A hostname label, not a URL: the scheme and domain are added in locals.tf. condition = can(regex("^[a-z0-9]([a-z0-9-]*[a-z0-9])?$", var.expected_xc_tenant)) error_message = "expected_xc_tenant must be a bare DNS label (lowercase letters, digits and hyphens) such as f5-sales-demo — not a URL and not a hostname." }}
# ---------------------------------------------------------# F5 XC data-plane inputs# ---------------------------------------------------------
variable "xc_app_namespace" { description = "Name of a PRE-EXISTING F5 XC namespace in expected_xc_tenant to place the app-tier objects (origin pool + HTTP load balancer) in. This deployment READS the namespace; it never creates or deletes it, so a namespace holding unrelated demos can never land on this stack's destroy list." type = string default = "multi-cloud-networking"
validation { condition = length(var.xc_app_namespace) > 0 error_message = "xc_app_namespace must name an existing namespace; it is looked up, not created." }}
variable "origin_pool_name" { description = "Name of the origin pool. Leave null (the default) to derive `<component>-pool`." type = string default = null}
variable "origin_ip" { description = "Public IP of the origin server the pool targets. REQUIRED, deliberately without a default: any default here is one specific machine, and a wrong one silently sends a fresh deployment's traffic to somebody else's host instead of failing." type = string
validation { condition = can(cidrhost("${var.origin_ip}/32", 0)) error_message = "origin_ip must be a single IPv4 address." }}
variable "origin_port" { description = "TCP port of the origin server." type = number default = 80}
variable "lb_name" { description = "Name of the HTTP load balancer. Leave null (the default) to derive `<component>-f5se`, matching the naming this tenant's other load balancers use." type = string default = null}
variable "lb_domain" { description = "Domain served by the HTTP load balancer for Rest of World. REQUIRED, deliberately without a default: the load balancer matches on Host, so this value is what every request must send, and it belongs to whoever runs the deployment." type = string
validation { condition = can(regex("^([a-z0-9]([a-z0-9-]*[a-z0-9])?\\.)+[a-z]{2,}$", var.lb_domain)) error_message = "lb_domain must be a fully-qualified lowercase domain name (for example mcn-ce-ha.f5-sales-demo.com)." }}
# ---------------------------------------------------------# Canada Regional F5 XC inputs# ---------------------------------------------------------
variable "enable_canada" { description = "Enable parallel Canada-only regional infrastructure, Canadian virtual sites, and f5-sales-demo.ca load balancer." type = bool default = true}
variable "ca_lb_domain" { description = "Domain served by the Canada HTTP load balancer. Defaults to mcn-ce-ha.f5-sales-demo.ca." type = string default = "mcn-ce-ha.f5-sales-demo.ca"
validation { condition = can(regex("^([a-z0-9]([a-z0-9-]*[a-z0-9])?\\.)+[a-z]{2,}$", var.ca_lb_domain)) error_message = "ca_lb_domain must be a fully-qualified lowercase domain name (for example mcn-ce-ha.f5-sales-demo.ca)." }}
variable "ca_re_cities" { description = "List of cities for the Canadian Regional Edge Virtual Site selector. Defaults to Toronto and Montreal." type = list(string) default = ["toronto", "montreal"]}
variable "ca_lb_name" { description = "Name of the Canada HTTP load balancer. Leave null (the default) to derive `<component>-ca-f5se`." type = string default = null}
variable "ca_origin_pool_name" { description = "Name of the Canada origin pool. Leave null (the default) to derive `<component>-ca-pool`." type = string default = null}
variable "ca_re_vsite_name" { description = "Name of the Canadian Regional Edge virtual site. Leave null (the default) to derive `<component>-ca-re-vsite`." type = string default = null}
variable "ca_ce_vsite_name" { description = "Name of the Canadian Customer Edge virtual site. Leave null (the default) to derive `<component>-ca-ce-vsite`." type = string default = null}
variable "vip" { description = "HA VIP advertised as a /32 by every CE via eBGP. MUST be outside all VNet CIDRs (Azure prefers the VNet system route over a more-specific BGP /32 otherwise)." type = string default = "10.250.0.10"
validation { # Self-contained format check. Cross-CIDR containment is asserted by the # check{} block in main.tf (which may reference other variables). condition = can(cidrhost("${var.vip}/32", 0)) error_message = "vip must be a valid IPv4 address." }}terraform/versions.tf
หัวข้อที่มีชื่อว่า “terraform/versions.tf”Terraform floor and provider constraints. The xcsh provider is pinned exactly to the clean-break v6.1.2 release; the local lock file remains gitignored.
terraform { # The automated showcase lifecycle and its test harness execute this exact # version on the authoritative Ubuntu host. required_version = "= 1.16.3"
required_providers { xcsh = { source = "f5-sales-demo/xcsh" version = "= 11.0.2" } azapi = { source = "Azure/azapi" version = "= 2.12.0" } azurerm = { source = "hashicorp/azurerm" version = "~> 4.0" } aws = { source = "hashicorp/aws" version = "~> 5.0" } azuread = { source = "hashicorp/azuread" version = "~> 3.0" } external = { source = "hashicorp/external" version = "~> 2.3" } random = { source = "hashicorp/random" version = "~> 3.0" } libvirt = { source = "dmacvicar/libvirt" version = "~> 0.8.0" } docker = { source = "kreuzwerker/docker" version = "~> 3.0" } }}Modules
หัวข้อที่มีชื่อว่า “Modules”terraform/modules/aws-tgw-connect/
หัวข้อที่มีชื่อว่า “terraform/modules/aws-tgw-connect/”AWS-owned Transit Gateway, transport attachment, role-keyed Connect attachments, and explicit route-table associations and propagations.
locals { roles = toset(["slo", "sli"]) tags = merge(var.ownership_tags, { Name = "${var.name_prefix}-tgw" ManagedBy = "terraform" })}resource "aws_ec2_transit_gateway" "this" { amazon_side_asn = var.amazon_side_asn auto_accept_shared_attachments = "disable" default_route_table_association = "disable" default_route_table_propagation = "disable" dns_support = "enable" transit_gateway_cidr_blocks = [var.transit_gateway_cidr_block] vpn_ecmp_support = "enable" tags = local.tags}resource "aws_ec2_transit_gateway_vpc_attachment" "transport" { subnet_ids = var.transport_subnet_ids transit_gateway_id = aws_ec2_transit_gateway.this.id transit_gateway_default_route_table_association = false transit_gateway_default_route_table_propagation = false vpc_id = var.vpc_id tags = merge(local.tags, { Name = "${var.name_prefix}-tgw-transport" })}resource "aws_ec2_transit_gateway_route_table" "this" { transit_gateway_id = aws_ec2_transit_gateway.this.id tags = merge(local.tags, { Name = "${var.name_prefix}-tgw-rt" })}resource "aws_ec2_transit_gateway_route_table_association" "transport" { transit_gateway_attachment_id = aws_ec2_transit_gateway_vpc_attachment.transport.id transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.this.id}resource "aws_ec2_transit_gateway_route_table_propagation" "transport" { transit_gateway_attachment_id = aws_ec2_transit_gateway_vpc_attachment.transport.id transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.this.id}resource "aws_ec2_transit_gateway_connect" "role" { for_each = local.roles protocol = "gre" transit_gateway_default_route_table_association = false transit_gateway_default_route_table_propagation = false transit_gateway_id = aws_ec2_transit_gateway.this.id transport_attachment_id = aws_ec2_transit_gateway_vpc_attachment.transport.id tags = merge(local.tags, { Name = "${var.name_prefix}-tgw-connect-${each.key}" })}resource "aws_ec2_transit_gateway_route_table_association" "connect" { for_each = aws_ec2_transit_gateway_connect.role transit_gateway_attachment_id = each.value.id transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.this.id}resource "aws_ec2_transit_gateway_route_table_propagation" "connect" { for_each = aws_ec2_transit_gateway_connect.role transit_gateway_attachment_id = each.value.id transit_gateway_route_table_id = aws_ec2_transit_gateway_route_table.this.id}output "transit_gateway_id" { description = "Transit Gateway ID." value = aws_ec2_transit_gateway.this.id}output "amazon_side_asn" { description = "Amazon-side ASN configured on the Transit Gateway." value = aws_ec2_transit_gateway.this.amazon_side_asn}output "connect_attachment_ids" { description = "Transit Gateway Connect attachment IDs keyed by SLO/SLI role." value = { for role, attachment in aws_ec2_transit_gateway_connect.role : role => attachment.id }}output "route_table_id" { description = "Transit Gateway route table used by transport and Connect attachments." value = aws_ec2_transit_gateway_route_table.this.id}variable "vpc_id" { description = "AWS VPC ID used as the Transit Gateway Connect transport attachment." type = string}variable "amazon_side_asn" { description = "Amazon-side ASN configured on the Transit Gateway." type = number validation { condition = var.amazon_side_asn >= 1 && var.amazon_side_asn <= 4294967295 error_message = "amazon_side_asn must be a valid 32-bit ASN." }}variable "transit_gateway_cidr_block" { description = "Non-overlapping IPv4 CIDR owned by the Transit Gateway for GRE endpoints." type = string validation { condition = can(cidrhost(var.transit_gateway_cidr_block, 0)) error_message = "transit_gateway_cidr_block must be a valid IPv4 CIDR." }}variable "transport_subnet_ids" { description = "One transport subnet ID per selected availability zone." type = list(string) validation { condition = length(var.transport_subnet_ids) == 3 && length(var.transport_subnet_ids) == length(toset(var.transport_subnet_ids)) error_message = "transport_subnet_ids must contain exactly three unique subnet IDs." }}variable "name_prefix" { description = "Non-sensitive prefix used only for AWS resource tags." type = string}
variable "ownership_tags" { description = "Immutable parent ownership tags required on every taggable TGW resource." type = map(string)}terraform { required_version = ">= 1.8"
required_providers { aws = { source = "hashicorp/aws" version = "~> 5.0" } }}terraform/modules/azure-hub/
หัวข้อที่มีชื่อว่า “terraform/modules/azure-hub/”Resource group, hub VNet, the four subnets, the Azure Route Server and the optional Bastion. RouteServerSubnet deliberately carries no NSG or route table.
resource "azurerm_resource_group" "this" { name = var.resource_group_name location = var.location tags = var.tags}
resource "azurerm_virtual_network" "hub" { name = "hub-vnet" address_space = [var.hub_cidr] location = azurerm_resource_group.this.location resource_group_name = azurerm_resource_group.this.name tags = var.tags}
resource "azurerm_subnet" "management" { name = "snet-hub-management" resource_group_name = azurerm_resource_group.this.name virtual_network_name = azurerm_virtual_network.hub.name address_prefixes = [var.mgmt_subnet_prefix]}
resource "azurerm_subnet" "external" { name = "snet-hub-external" resource_group_name = azurerm_resource_group.this.name virtual_network_name = azurerm_virtual_network.hub.name address_prefixes = [var.external_subnet_prefix]}
resource "azurerm_subnet" "internal" { name = "snet-hub-internal" resource_group_name = azurerm_resource_group.this.name virtual_network_name = azurerm_virtual_network.hub.name address_prefixes = [var.internal_subnet_prefix]}
# RouteServerSubnet: name is literal, /27, and has NO NSG and NO route table# association (both are unsupported on the Route Server subnet).resource "azurerm_subnet" "route_server" { count = var.enable_route_server ? 1 : 0 name = "RouteServerSubnet" resource_group_name = azurerm_resource_group.this.name virtual_network_name = azurerm_virtual_network.hub.name address_prefixes = [var.route_server_subnet_prefix]}
resource "azurerm_public_ip" "route_server" { count = var.enable_route_server ? 1 : 0 name = "${var.route_server_name}-pip" location = azurerm_resource_group.this.location resource_group_name = azurerm_resource_group.this.name allocation_method = "Static" sku = "Standard" tags = var.tags}
# Azure Route Server. ASN is fixed by Azure at 65515; virtual_router_ips are the# two BGP peer addresses (10.0.4.4 / 10.0.4.5) the CEs peer to.resource "azurerm_route_server" "this" { count = var.enable_route_server ? 1 : 0 name = var.route_server_name location = azurerm_resource_group.this.location resource_group_name = azurerm_resource_group.this.name sku = "Standard" public_ip_address_id = azurerm_public_ip.route_server[0].id subnet_id = azurerm_subnet.route_server[0].id tags = var.tags}
# --------------------------------------------------------------------------# Azure Bastion — reachability for the CE Site Console web UI (TCP 65500)# --------------------------------------------------------------------------# F5's documented CE troubleshooting path is the per-node Site Console at# https://<node-ip>:65500 as `admin`. Verified on this deployment: of a CE's three# private addresses only the SLI one (snet-hub-internal) answers at all — mgmt/SLO# and external are closed on every port probed — and it is reachable only from# inside the VNet. Bastion closes the gap without distributing SSH keys and# without touching the CE: who may connect becomes an Azure RBAC decision.## AzureBastionSubnet: name is literal, /26 or larger, and — like# RouteServerSubnet above — deliberately carries NO NSG and NO route table.# Bastion works without an NSG; adding one obliges you to maintain Azure's exact# required allow-rule set (GatewayManager, AzureLoadBalancer, the 8080/5701# intra-subnet pair) for no security gain in a lab whose VMs already sit behind# their own NSGs.resource "azurerm_subnet" "bastion" { count = var.enable_bastion ? 1 : 0
name = "AzureBastionSubnet" resource_group_name = azurerm_resource_group.this.name virtual_network_name = azurerm_virtual_network.hub.name address_prefixes = [var.bastion_subnet_prefix]}
resource "azurerm_public_ip" "bastion" { count = var.enable_bastion ? 1 : 0
name = "${var.bastion_name}-pip" location = azurerm_resource_group.this.location resource_group_name = azurerm_resource_group.this.name allocation_method = "Static" sku = "Standard" tags = var.tags}
# Standard SKU is not a preference — Basic supports neither native-client# tunneling nor IP-based connection.## tunneling_enabled -> load-bearing. `az network bastion tunnel` forwards an# arbitrary TCP port, which is the only way to reach 65500.# ip_connect_enabled -> NOT the path to 65500, and deliberately not claimed to# be. Microsoft documents that "custom ports and protocols# aren't currently supported when connecting to a virtual# machine via native client with IP-based connections"# (learn.microsoft.com/azure/bastion/connect-ip-address),# and the CLI enforces it: --target-ip-address with# --resource-port 65500 is refused with "Allowed ports for# Tunnel with IP connect is 22, 3389". It is on because it# costs nothing and lets an operator target an in-VNet IP# directly for portal RDP/SSH. The 65500 tunnel targets the# CE by VM resource id instead — verified reaching the Site# Console (HTTP 200) on this deployment.resource "azurerm_bastion_host" "this" { count = var.enable_bastion ? 1 : 0
name = var.bastion_name location = azurerm_resource_group.this.location resource_group_name = azurerm_resource_group.this.name sku = "Standard" tunneling_enabled = true ip_connect_enabled = true
ip_configuration { name = "configuration" subnet_id = azurerm_subnet.bastion[0].id public_ip_address_id = azurerm_public_ip.bastion[0].id }
tags = var.tags}output "resource_group_name" { description = "Name of the hub resource group." value = azurerm_resource_group.this.name}
output "location" { description = "Azure region." value = azurerm_resource_group.this.location}
output "vnet_name" { description = "Hub VNet name." value = azurerm_virtual_network.hub.name}
output "vnet_id" { description = "Hub VNet resource ID." value = azurerm_virtual_network.hub.id}
output "management_subnet_id" { description = "snet-hub-management subnet ID." value = azurerm_subnet.management.id}
output "external_subnet_id" { description = "snet-hub-external subnet ID." value = azurerm_subnet.external.id}
output "internal_subnet_id" { description = "snet-hub-internal subnet ID." value = azurerm_subnet.internal.id}
output "route_server_subnet_id" { description = "RouteServerSubnet subnet ID." value = one(azurerm_subnet.route_server[*].id)}
output "route_server_id" { description = "Azure Route Server resource ID." value = one(azurerm_route_server.this[*].id)}
output "rs_peer_ips" { description = "The two Route Server BGP peer IPs (virtual_router_ips) the CEs peer to." value = flatten(azurerm_route_server.this[*].virtual_router_ips)}
output "rs_asn" { description = "Route Server ASN (fixed by Azure at 65515)." value = one(azurerm_route_server.this[*].virtual_router_asn)}
output "bastion_name" { description = "Azure Bastion host name, or null when Bastion is not deployed. This is the --name argument of `az network bastion tunnel`." value = one(azurerm_bastion_host.this[*].name)}
output "bastion_subnet_id" { description = "AzureBastionSubnet subnet ID, or null when Bastion is not deployed." value = one(azurerm_subnet.bastion[*].id)}
output "bastion_subnet_prefix" { description = "The AzureBastionSubnet prefix this module was given (echoed so callers can assert on it without reading a count-gated resource)." value = var.bastion_subnet_prefix}variable "resource_group_name" { description = "Resource group to create for the hub." type = string}
variable "location" { description = "Azure region." type = string}
variable "hub_cidr" { description = "Hub VNet address space." type = string}
variable "mgmt_subnet_prefix" { description = "snet-hub-management prefix." type = string}
variable "external_subnet_prefix" { description = "snet-hub-external prefix." type = string}
variable "internal_subnet_prefix" { description = "snet-hub-internal prefix." type = string}
variable "route_server_subnet_prefix" { description = "RouteServerSubnet prefix (/27, no NSG, no route table)." type = string}
variable "route_server_name" { description = "Azure Route Server name." type = string default = "ce-ha-lab-rrs"}
variable "enable_route_server" { description = "Create Azure Route Server resources. Keep false for the supported ILB-based SMSv2 showcase; true requires the separately validated eBGP multihop contract gate." type = bool default = false}
variable "bastion_subnet_prefix" { description = "AzureBastionSubnet prefix (/26 or larger, named literally AzureBastionSubnet, no NSG and no route table). Only read when enable_bastion is true." type = string}
variable "enable_bastion" { description = "Deploy Azure Bastion (AzureBastionSubnet + public IP + Standard-SKU host) so the CE Site Console web UI on TCP 65500 is reachable without a jump host. Opt-in: Bastion bills hourly whether or not anyone connects." type = bool default = false}
variable "bastion_name" { description = "Azure Bastion host name (the --name argument of `az network bastion tunnel`)." type = string default = "ce-ha-lab-bastion"}
variable "tags" { description = "Tags applied to every resource." type = map(string) default = {}}terraform { required_version = ">= 1.8"
required_providers { azurerm = { source = "hashicorp/azurerm" version = "~> 4.0" } }}terraform/modules/azure-route-server-bgp/
หัวข้อที่มีชื่อว่า “terraform/modules/azure-route-server-bgp/”The Azure side of each eBGP session — one bgpConnection per CE, peering the Route Server to that CE eth0/SLO address.
# The Azure side of the eBGP session: a Route Server BGP connection to a CE's# eth0/SLO private IP. Peer ASN = the CE ASN (64512). The CE originates the LB# VIP /32; equal-cost advertisements from multiple CEs program ECMP into the VNet.
variable "name" { description = "BGP connection name (e.g. eastus01-bgp)." type = string}
variable "route_server_id" { description = "Azure Route Server resource ID." type = string}
variable "peer_asn" { description = "Peer (CE) ASN." type = number default = 64512}
variable "peer_ip" { description = "Peer (CE) eth0/SLO private IP." type = string}
resource "azurerm_route_server_bgp_connection" "this" { name = var.name route_server_id = var.route_server_id peer_asn = var.peer_asn peer_ip = var.peer_ip}
output "connection_id" { description = "Route Server BGP connection resource ID." value = azurerm_route_server_bgp_connection.this.id}
output "connection_name" { description = "Route Server BGP connection name." value = azurerm_route_server_bgp_connection.this.name}terraform { required_version = ">= 1.8"
required_providers { azurerm = { source = "hashicorp/azurerm" version = "~> 4.0" } }}terraform/modules/ce-node/
หัวข้อที่มีชื่อว่า “terraform/modules/ce-node/”One CE VM per node: three NICs with IP forwarding, the marketplace plan block, and the cloud-init handed over as custom_data.
# Per-CE Azure resources: managed identity, three NICs (mgmt/external/internal,# all with IP forwarding on and accelerated networking OFF), and the volterra-node# VM. The mgmt NIC is the VM's FIRST NIC = eth0 = the SLO/BGP local address.
resource "azurerm_user_assigned_identity" "this" { name = "${var.hostname}-identity" resource_group_name = var.resource_group_name location = var.location tags = var.tags}
resource "azurerm_public_ip" "mgmt" { name = "${var.hostname}-mgmt-pip" resource_group_name = var.resource_group_name location = var.location allocation_method = "Static" sku = "Standard" tags = var.tags}
# eth0 / SLO — mgmt subnet, static private IP (the BGP local address), public IP.resource "azurerm_network_interface" "mgmt" { name = "${var.hostname}-mgmt-nic" resource_group_name = var.resource_group_name location = var.location ip_forwarding_enabled = true accelerated_networking_enabled = false
ip_configuration { name = "ipconfig1" subnet_id = var.mgmt_subnet_id private_ip_address_allocation = "Static" private_ip_address = var.mgmt_private_ip public_ip_address_id = azurerm_public_ip.mgmt.id }
tags = var.tags}
resource "azurerm_network_interface" "external" { name = "${var.hostname}-external-nic" resource_group_name = var.resource_group_name location = var.location ip_forwarding_enabled = true accelerated_networking_enabled = false
ip_configuration { name = "ipconfig1" subnet_id = var.external_subnet_id private_ip_address_allocation = "Dynamic" }
tags = var.tags}
resource "azurerm_network_interface" "internal" { name = "${var.hostname}-internal-nic" resource_group_name = var.resource_group_name location = var.location ip_forwarding_enabled = true accelerated_networking_enabled = false
ip_configuration { name = "ipconfig1" subnet_id = var.internal_subnet_id private_ip_address_allocation = "Dynamic" }
tags = var.tags}
resource "azurerm_linux_virtual_machine" "this" { name = var.hostname computer_name = var.hostname resource_group_name = var.resource_group_name location = var.location size = var.vm_size zone = var.zone
admin_username = var.admin_username disable_password_authentication = true
admin_ssh_key { username = var.admin_username public_key = var.ssh_public_key }
# eth0 first = mgmt/SLO NIC (BGP local + MAC-bound to the XC site interface). network_interface_ids = [ azurerm_network_interface.mgmt.id, azurerm_network_interface.external.id, azurerm_network_interface.internal.id, ]
identity { type = "UserAssigned" identity_ids = [azurerm_user_assigned_identity.this.id] }
os_disk { name = "${var.hostname}-osdisk" caching = "ReadWrite" storage_account_type = "StandardSSD_LRS"
# Explicit, because the image default is the one size measured to fail the # advertised version pair (#714). See variables.tf for the measurements. disk_size_gb = var.os_disk_size_gb }
source_image_reference { publisher = "f5-networks" offer = "f5xc_customer_edge" sku = "f5xc-ce-crt-20260201" version = "20260201.0178.1" }
# Marketplace plan is REQUIRED for this exact certified Customer Edge image. plan { name = "f5xc-ce-crt-20260201" product = "f5xc_customer_edge" publisher = "f5-networks" }
custom_data = var.custom_data
# Required for Azure Serial Console, which is the only way into a CE that has not # finished its first boot: the vpm/debug API used for every other diagnostic is # reached through the XC control plane, so it answers only once the node is ONLINE, # and operator SSH depends on cloud-init having written admin's authorized_keys and # on the SLI interface being up. Empty block = Azure-managed storage, so there is # no diagnostics storage account or access key to own. boot_diagnostics {}
tags = var.tags}output "mgmt_nic_id" { description = "Resource ID of the eth0/SLO (mgmt) NIC." value = azurerm_network_interface.mgmt.id}
output "mgmt_nic_mac" { description = "MAC address of the eth0/SLO (mgmt) NIC. Wired into the XC site interface binding so a NIC recreate updates the site." value = azurerm_network_interface.mgmt.mac_address}
output "mgmt_private_ip" { description = "Private IP of the eth0/SLO (mgmt) NIC — the BGP local address and the Route Server BGP connection peer_ip." value = azurerm_network_interface.mgmt.private_ip_address}
output "sli_private_ip" { description = "Private IP of the internal/SLI NIC — the only one of the CE's three private addresses that answers, and where the Site Console web UI (TCP 65500) is served. Reach it from a workstation with `az network bastion tunnel --target-resource-id <this CE's vm_id> --resource-port 65500`; the mgmt/SLO and external addresses serve nothing." value = azurerm_network_interface.internal.private_ip_address}
output "vm_name" { description = "CE VM name." value = azurerm_linux_virtual_machine.this.name}
output "vm_id" { description = "CE VM ARM resource ID. Addresses the VM (this is what `az network bastion tunnel --target-resource-id` wants) — it is NOT a per-instance identity: see vm_instance_id." value = azurerm_linux_virtual_machine.this.id}
# The two ids above and below are easy to confuse and are not interchangeable.# vm_id is the ARM resource id, ".../virtualMachines/<hostname>" — derived from# the VM name and therefore byte-identical before and after a replacement, which# makes it the right handle for ADDRESSING the VM (Bastion tunnels) and useless# as a "this node was rebuilt" signal. This one is regenerated for every new# instance, and it is the same value the CE reports to F5 XC as the# registration's infra.instance_id — so it is the identity that binds an XC site# object to a specific node. modules/xc-site keys the site's replace_triggered_by# on it (see issue #674).output "vm_instance_id" { description = "128-bit unique id of the CE VM INSTANCE (regenerated on replacement; equals the XC registration's infra.instance_id). Not the ARM resource id, which is name-derived and survives a replacement." value = azurerm_linux_virtual_machine.this.virtual_machine_id}
output "identity_id" { description = "User-assigned managed identity resource ID." value = azurerm_user_assigned_identity.this.id}variable "hostname" { description = "CE VM hostname (also the Azure VM name), e.g. f5-xc-ce-vm-01." type = string}
variable "resource_group_name" { description = "Resource group (created by the hub module)." type = string}
variable "location" { description = "Azure region." type = string}
variable "zone" { description = "Availability zone for the VM (1, 2 or 3)." type = string}
variable "vm_size" { description = "VM size." type = string default = "Standard_D8_v4"}
variable "mgmt_subnet_id" { description = "snet-hub-management subnet ID (eth0/SLO)." type = string}
variable "external_subnet_id" { description = "snet-hub-external subnet ID." type = string}
variable "internal_subnet_id" { description = "snet-hub-internal subnet ID." type = string}
variable "mgmt_private_ip" { description = "Static private IP for the eth0/SLO (mgmt) NIC — the BGP local address (e.g. 10.0.1.4)." type = string}
variable "admin_username" { description = "SSH admin username." type = string default = "azureuser"}
variable "ssh_public_key" { description = "SSH public key MATERIAL (string), passed down from the root." type = string}
variable "custom_data" { description = "Base64-encoded cloud-init custom data for the volterra-node bootstrap." type = string}
variable "tags" { description = "Tags applied to every resource." type = map(string) default = {}}
# Sized from measurement, not from a rule of thumb. Issue #714 ran one disposable# single-node Azure Secure Mesh v2 site per size, all from marketplace image 0.9.2,# installing the pair the tenant advertises (crt-20260201-0179 + OS 9.2026.14):## 31 GiB (the image default) FAIL — voucher DaemonSet 0/1, site stuck# PROVISIONING with nothing installed# 33 GB PASS# 36 / 40 / 48 / 64 GB PASS## The default below is deliberately NOT the measured 33 GB floor. 33 works for that# pair on that image today; a build with a marginally larger payload would fail there# with no configuration change and no obvious cause — exactly the position the image# default is in now.## Do NOT size this from F5's documented "20 GB plus 15% of capacity" pre-upgrade# figure. That check gates Console UI upgrades only and does not apply to the API# path; applied here it predicts 48 GB would fail, and 48 GB installs cleanly.variable "os_disk_size_gb" { description = "CE OS disk size in GB. The pinned f5xc-ce-crt-20260201:20260201.0178.1 marketplace image requires at least 78 GiB; the default retains a small deterministic margin." type = number default = 80
validation { condition = var.os_disk_size_gb >= 78 error_message = "os_disk_size_gb must be at least 78 GB because the pinned f5xc-ce-crt-20260201:20260201.0178.1 marketplace image cannot create a smaller OS disk." }}terraform { required_version = ">= 1.8"
required_providers { azurerm = { source = "hashicorp/azurerm" version = "~> 4.0" } }}terraform/modules/ce-topology/
หัวข้อที่มีชื่อว่า “terraform/modules/ce-topology/”Pure computation, no providers and no data sources: expands ce_count into the per-CE map that drives every for_each. Plan-testable at any N without credentials.
# Pure computation module: expands ce_count into the per-CE node map that drives# the ce-node, xc-site and azure-route-server-bgp for_each loops. Has NO# providers and NO data sources, so it is fully hermetic and plan-testable at# any N without Azure/XC credentials (see tests/n_scaling.tftest.hcl).
terraform { required_version = ">= 1.8"}
variable "ce_count" { description = "Number of CE nodes (1..3)." type = number default = 3
validation { condition = var.ce_count >= 1 && var.ce_count <= 3 error_message = "ce_count must be between 1 and 3." }}
variable "enabled" { description = "Whether this regional topology participates in the current plan. Disabled topologies deliberately expand to no nodes without weakening the 1..3 validation for enabled deployments." type = bool default = true}
variable "region_short" { description = "Short region token (e.g. eastus)." type = string default = "eastus"}
variable "mgmt_subnet_prefix" { description = "Management subnet prefix. The eth0/SLO IP is derived from it (4 + index)." type = string default = "10.0.1.0/26"}
variable "hostname_prefix" { description = "Prefix for CE VM hostnames (hostname = <prefix>-0<n>)." type = string default = "f5-xc-ce-vm"}
variable "site_prefix" { description = "Prefix for XC site names (site = <prefix>-<region_short>0<n>). REQUIRED, deliberately without a default: the root passes var.component, so every object name in the deployment derives from one place and no deployment-specific prefix can hide in a module default." type = string
validation { condition = can(regex("^[a-z0-9]([a-z0-9-]*[a-z0-9])?$", var.site_prefix)) error_message = "site_prefix must be a DNS-style label: lowercase alphanumerics and hyphens, not starting or ending with a hyphen (it becomes part of an XC object name)." }}
locals { ce_nodes = var.enabled ? { for i in range(var.ce_count) : "${var.region_short}0${i + 1}" => { index = i hostname = "${var.hostname_prefix}-0${i + 1}" site_name = "${var.site_prefix}-${var.region_short}0${i + 1}" # eth0/SLO IP: 10.0.1.4, .5, .6 for i = 0, 1, 2. slo_ip = cidrhost(var.mgmt_subnet_prefix, 4 + i) # Availability zone 1, 2, 3. az = element(["1", "2", "3"], i) # The network_interface object XC auto-creates for the explicit SLO # interface. The bgp peer references it by this exact name. interface_name = "ves-io-securemesh-site-v2-${var.site_prefix}-${var.region_short}0${i + 1}-network-${var.hostname_prefix}-0${i + 1}-eth0-0" } } : {}}
output "ce_nodes" { description = "Map keyed by <region_short>0<n> of per-CE attributes (hostname, site_name, slo_ip, az, interface_name, index)." value = local.ce_nodes}
output "ce_count" { description = "Number of CE nodes expanded." value = length(local.ce_nodes)}terraform/modules/client-vm/
หัวข้อที่มีชื่อว่า “terraform/modules/client-vm/”The test client inside the VNet, used to read effective routes and drive traffic at the VIP.
# Simple Ubuntu test client in snet-hub-internal. Used to generate HTTP traffic# to the VIP and to read the VNet effective route table (ECMP proof).
resource "azurerm_public_ip" "this" { name = "${var.name}PublicIP" resource_group_name = var.resource_group_name location = var.location allocation_method = "Static" sku = "Standard" tags = var.tags}
resource "azurerm_network_security_group" "this" { #checkov:skip=CKV_AZURE_10:Lab NSG - SSH open for demo access #checkov:skip=CKV_AZURE_160:Lab NSG - HTTP port 80 required for traffic #checkov:skip=CKV_AZURE_220:Lab NSG - SSH open for demo access name = "${var.name}NSG" resource_group_name = var.resource_group_name location = var.location
security_rule { name = "SSH" priority = 1001 direction = "Inbound" access = "Allow" protocol = "Tcp" source_port_range = "*" destination_port_range = "22" source_address_prefix = "*" destination_address_prefix = "*" }
tags = var.tags}
resource "azurerm_network_interface" "this" { #checkov:skip=CKV_AZURE_119:Lab NIC - public IP required for demo access name = "${var.name}VMNic" resource_group_name = var.resource_group_name location = var.location
ip_configuration { name = "ipconfig1" subnet_id = var.subnet_id private_ip_address_allocation = "Dynamic" public_ip_address_id = azurerm_public_ip.this.id }
tags = var.tags}
resource "azurerm_network_interface_security_group_association" "this" { network_interface_id = azurerm_network_interface.this.id network_security_group_id = azurerm_network_security_group.this.id}
resource "azurerm_linux_virtual_machine" "this" { #checkov:skip=CKV_AZURE_50:Lab VM - no extensions required #checkov:skip=CKV_AZURE_93:Lab VM - platform-managed encryption sufficient name = var.name computer_name = var.name resource_group_name = var.resource_group_name location = var.location size = var.vm_size
admin_username = var.admin_username disable_password_authentication = true
admin_ssh_key { username = var.admin_username public_key = var.ssh_public_key }
network_interface_ids = [azurerm_network_interface.this.id]
os_disk { caching = "ReadWrite" storage_account_type = "Premium_LRS" }
source_image_reference { publisher = "Canonical" offer = "0001-com-ubuntu-server-jammy" sku = "22_04-lts-gen2" version = "latest" }
custom_data = var.custom_data != "" ? var.custom_data : null
tags = var.tags}
output "public_ip" { description = "Public IP of the test client." value = azurerm_public_ip.this.ip_address}
output "private_ip" { description = "Private IP of the test client." value = azurerm_network_interface.this.private_ip_address}
output "vm_name" { description = "Test client VM name." value = azurerm_linux_virtual_machine.this.name}
output "nic_name" { description = "Test client NIC name (read effective routes from this NIC)." value = azurerm_network_interface.this.name}variable "name" { description = "Test client VM name; every child resource derives from it (<name>PublicIP, <name>NSG, <name>VMNic). REQUIRED, deliberately without a default: the root passes a value derived from var.component, so no deployment-specific name can hide in a module default." type = string}
variable "resource_group_name" { description = "Resource group (created by the hub module)." type = string}
variable "location" { description = "Azure region." type = string}
variable "subnet_id" { description = "snet-hub-internal subnet ID the client attaches to." type = string}
variable "vm_size" { description = "VM size." type = string default = "Standard_B2s"}
variable "admin_username" { description = "SSH admin username." type = string default = "azureuser"}
variable "ssh_public_key" { description = "SSH public key MATERIAL (string), passed down from the root." type = string}
variable "custom_data" { description = "Base64-encoded cloud-init custom data." type = string default = ""}
variable "tags" { description = "Tags applied to every resource." type = map(string) default = {}}terraform { required_version = ">= 1.8"
required_providers { azurerm = { source = "hashicorp/azurerm" version = "~> 4.0" } }}terraform/modules/xc-site/
หัวข้อที่มีชื่อว่า “terraform/modules/xc-site/”The XC site object, its BGP peering, and the registration approval that resolves a r-<uuid> registration by site name.
# Parks the identity of the CE VM instance the site's node runs on, so the site# object has something to be coupled to. Nothing else reads it — its only job is# to be named in the site's replace_triggered_by below.## WHY A SEPARATE RESOURCE. replace_triggered_by may only name managed resources# declared in the SAME module as the resource carrying the lifecycle block, and# the CE VM lives in modules/ce-node. Parking the id here is the standard way to# carry an external value across that boundary.## WHY input AND NOT triggers_replace. This resource must never itself be# replaced — only observed. A changed `input` is an in-place UPDATE, which is# what replace_triggered_by reacts to; that keeps the resource cheap and its# behaviour on ADOPTION correct (see below).## ADOPTION IS INERT. Adding this resource to a deployment that already exists# plans it as a CREATE, and a create of the referenced resource does NOT fire# replace_triggered_by — only a subsequent change to its value does. Verified on# Terraform v1.10.5 and again on v1.15.0:# adding the pair to a populated state plans "1 to add, 0 to change,# 0 to destroy". So this fix does not itself trigger the fleet-wide rebuild it# exists to prevent — no import, no targeted apply, no seeding.resource "terraform_data" "ce_vm" { input = var.ce_vm_instance_id}
# Single-node Secure Mesh v2 CE site with an EXPLICIT eth0/SLO interface. The# explicit interface is what makes XC auto-create the network_interface object# (var.interface_name) that the BGP peer binds to — without it a standalone bgp# object is accepted but never renders to FRR (see xcsh #1207).resource "xcsh_securemesh_site_v2" "this" { count = var.create_site ? 1 : 0 name = var.site_name namespace = "system" description = "MCN CE-HA (BGP/ECMP) single-node SMSv2 site ${var.site_name} — explicit eth0 SLO interface for BGP peer binding." # `null`, not `{}`, when no labels are set. xcsh #1286 makes the provider preserve a # config-declared empty map on the POST-APPLY read-back, but import has no config to # read: the state carries only id/name/namespace and `ReadRequest` exposes nothing # else, so a literal `{}` would still re-plan as `+ labels = {}` on the first # post-import plan. Sending `null` when the map is empty stops asking the provider to # distinguish "declared empty" from "absent" — something it cannot observe on import. # (The nested `interface_list.labels {}` marker is a separate class, fixed by xcsh #1244.) # # EXPECTED ONE-TIME DRIFT AFTER A NODE (RE)REGISTERS. F5 XC stamps the node's # hardware facts onto the site as labels (host-os-version, hw-model, # hw-serial-number, hw-vendor, hw-version) once the CE registers. The next plan # therefore shows `- labels = {...} -> null` for that site, and applying it # settles — XC does not re-stamp them. It is one more convergence pass in the # already two-phase deploy, not drift to chase; observed on the site rebuilt by # the #674 CE replacement. labels = length(var.labels) > 0 ? var.labels : null
azure { not_managed { node_list { hostname = var.hostname type = "Control" public_ip = null
interface_list { name = "eth0"
ethernet_interface { device = "eth0" mac = var.mgmt_nic_mac }
# Site Local Outside (SLO) — required on every site; BGP peers from here. network_option { site_local_network = {} }
dhcp_client = {} } } } }
block_all_services = {} disable_ha = {}
dns_ntp_config { f5_dns_default = {} f5_ntp_default = {} }
local_vrf { default_config = {} default_sli_config = {} }
logs_streaming_disabled = {} no_forward_proxy = {} no_network_policy = {} no_s2s_connectivity_sli = {} no_s2s_connectivity_slo = {}
offline_survivability_mode { no_offline_survivability_mode = {} }
performance_enhancement_mode { perf_mode_l7_enhanced { # The provider schema gives perf_mode_l7_enhanced a {jumbo_disabled | jumbo_enabled} # sub-oneof. F5 materialises jumbo_disabled server-side, so leaving both members # undeclared makes the site land and then re-plan the marker as a removal on # every subsequent plan — it never reaches 0 changes. Declaring the server # default explicitly is what settles it (same fix coverage/smsv2 took in #625). jumbo_disabled = {} } }
re_select { geo_proximity = {} }
# CE software and OS selection is create-time configuration. The node always # installs a destination build on first boot; an empty variable arms the # default_* marker and means "install the newest version the server advertises." # That is the deployment policy, not an accidental omission. The clean # 2026-08-03 rebuild selected the advertised pair on all three 64 GB nodes and # brought all three sites ONLINE. Issue #714 separately proves why the disk # default carries headroom: the same pair failed on the marketplace image's # 31 GiB disk and installed at every tested size from 33 GB upwards. # # Terraform cannot update these fields after creation: PUT is rejected when # pinning forward, pinning backward, or clearing a pin. The platform can perform # an in-place change through the site upgrade_sw and upgrade_os actions, but the # provider cannot drive those actions yet (xcsh#1390). Set a concrete value only # when deliberately reproducing an older build. software_settings { os { default_os_version = var.os_version == "" ? {} : null operating_system_version = var.os_version == "" ? null : var.os_version } sw { default_sw_version = var.sw_version == "" ? {} : null volterra_software_version = var.sw_version == "" ? null : var.sw_version } }
# Rebuild the site object whenever the CE VM instance it describes is rebuilt # (issue #674). # # THE FAILURE THIS PREVENTS. A CE's runtime registration is bound to one node # instance and holds the control plane's unique # (tenant, cluster_name, hostname) index. Destroying the VM does NOT retire # that registration, so the replacement node — same site, same hostname — # cannot create its own: the create fails with UniqueSecondaryIndexViolation # and retries on a ~65 s loop forever. Nothing recovers on its own, and worse, # nothing in the graph noticed: with no reference to the node's identity # anywhere, `terraform plan` reported "No changes" for the whole time the # fleet was down. # # WHY REPLACING THE SITE IS THE FIX. Deleting the site object takes its # registrations with it — observed live while replacing one CE: the site # 404ed and the registration bound to the outgoing instance disappeared in the # same poll — so the replacement node registers into a site whose index key is # free. Deleting only the registration is not enough: the site keeps a status # object that then rejects the node's workload request. # # THE ORDER IS THE POINT. Terraform runs this as: destroy site -> destroy VM # -> create VM -> create site. The stale registration is therefore gone before # the replacement node ever boots, and the site is back before the node # finishes booting and registers. The outgoing node cannot slip a fresh # registration into the gap: once its own registration is deleted it 404-loops # against the name it persisted in registration-obj.yml instead of creating a # new one. # # SAFE WHILE OTHER OBJECTS REFERENCE THE SITE. xcsh_bgp and the root HTTP load # balancer's advertise_where both name this site, and F5 XC resolves those # references lazily: deleting a site that both of them reference returns HTTP # 200 and leaves them intact, and re-creating it under the same name re-binds # them (verified against the live tenant with a throwaway site). # # THAT LAZINESS DOES NOT EXTEND TO CREATION, and the difference has bitten once. # An EXISTING load balancer tolerates a dangling site reference; POSTing a NEW one # whose advertise_where names a site that does not exist yet is rejected outright # with `[BAD_REQUEST] Invalid request parameters`. Renaming the deployment does # exactly that — every site is destroyed and re-created under a different name, so # the load balancer is created fresh — which is why the root resource now carries # an explicit `depends_on = [module.xc_site]`. Do not remove it on the strength of # the paragraph above: it is about deletion, not creation. lifecycle { replace_triggered_by = [terraform_data.ce_vm] }}
# The approve API takes the runtime registration name ("r-<uuid>"), NOT the site# name (GET .../registrations/<site> -> 404). registrations_by_site returns# HTTP 200 with items:[] for a site whose CE has not registered yet, so this read# never fails an early apply — it just reports found = false.## NOTE: this data source must never carry depends_on. Its inputs are statically# derived from ce_topology, so it resolves at plan time; a resource dependency# would make the count below unknown at plan time ("The count value depends on# resource attributes that cannot be determined until apply").data "xcsh_site_registration" "this" { site_name = var.site_name # == passport.cluster_name (cloud-init ClusterName) hostname = var.hostname # discriminator for multi-node sites namespace = "system"}
# Approve the CE registration so the node reaches ONLINE without the manual# console step (#1206 / #1210). The registration exists only after the CE boots# and registers via the token, so the first apply plans no approval; re-apply# once the CE has registered (see the deploy ordering in main.tf).## The action only legitimately transitions a NEW registration. Retired and# already-admitted registrations are observations, never approval targets.# Keeping the guard in the module rather than relying on provider selection also# protects installed provider versions that predate terminal-state filtering.## Approval can auto-provision a site in XC, so it must wait for Terraform's# explicit site creation. The data source deliberately has no dependency: its# result determines this resource's plan-known count.resource "xcsh_registration_approval" "this" { count = var.approve_registration && data.xcsh_site_registration.this.found && data.xcsh_site_registration.this.state == "NEW" ? 1 : 0
namespace = "system" name = data.xcsh_site_registration.this.name cluster_size = 1 state = "APPROVED"
depends_on = [xcsh_securemesh_site_v2.this]}
# One bgp object per CE site: eBGP from the CE (ASN var.ce_asn) to the Azure# Route Server (ASN var.rs_asn), one external peer per Route Server virtual# router IP, each bound to the explicit SLO interface.## NOT BLOCKED — and nothing about this arm is gated any more. The object-ref name# length limit that used to block it is gone: the provider relaxed it to# stringvalidator.LengthBetween(1, 128) in v3.74.0, so the 71-char interface object# XC auto-generates for the explicit SLO interface# (ves-io-securemesh-site-v2-<site>-network-<hostname>-eth0-0) validates. The floor# that guarantees it is declared once, in versions.tf — do not restate the number.## var.enable_bgp therefore defaults true and every test now runs with that default;# it survives only as an escape hatch for deploying the topology without BGP. It is# NOT an ordering gate: var.interface_name is derived statically from ce_topology, and# XC accepts a bgp object naming an interface that does not exist yet (it converges# once the CE is up — see the deploy ordering in the root main.tf).resource "xcsh_bgp" "this" { count = var.enable_bgp ? 1 : 0
name = "${var.site_name}-bgp" namespace = "system" description = "CE ${var.site_name} BGP to Azure Route Server via explicit SLO interface."
where { site { ref { namespace = "system" name = var.site_name } network_type = "VIRTUAL_NETWORK_SITE_LOCAL" disable_internet_vip = {} } }
bgp_parameters { asn = var.ce_asn # local_address {} = derive the BGP router ID from the interface's local # address (the JSON's BGP_ROUTER_ID_FROM_INTERFACE; there is no separate # bgp_router_id_type attribute in the provider schema). local_address = {} }
# Iterate over a plan-KNOWN peer count (rs_peer_count) and index into # rs_peer_ips. The IP values may be unknown until the Route Server is applied, # but the number of peers is fixed, so the block expands cleanly at plan time. dynamic "peers" { for_each = { for i in range(var.rs_peer_count) : "azure-rrs-${i + 1}" => i } content { metadata { name = peers.key }
external { asn = var.rs_asn address = try(var.rs_peer_ips[peers.value], "") port = var.peer_port
interface { namespace = "system" name = var.interface_name }
disable_v6 = {} }
passive_mode_disabled = {} bfd_disabled = {} } }}output "site_name" { description = "XC securemesh_site_v2 name." value = var.site_name}
output "site_created" { description = "Whether this module instance manages its Secure Mesh Site v2 object." value = length(xcsh_securemesh_site_v2.this) == 1}
output "bgp_name" { description = "XC bgp object name (null when enable_bgp is false)." value = one(xcsh_bgp.this[*].name)}
output "interface_name" { description = "Auto-derived network_interface object name the BGP peer binds to." value = var.interface_name}
# The node identity the site object is currently coupled to. Compare it against# the registration's own infra.instance_id (visible in the F5 XC API, not yet on# the xcsh_site_registration data source — provider issue #1376) to tell whether# a site is still bound to a node that no longer exists.output "bound_vm_instance_id" { description = "CE VM instance id the site object is bound to. Replacing that instance replaces the site (issue #674)." value = terraform_data.ce_vm.output}
output "registration_name" { description = "Runtime registration name (r-<uuid>) resolved from the site name; null until the CE has registered." value = data.xcsh_site_registration.this.found ? data.xcsh_site_registration.this.name : null}
output "registration_state" { description = "Current registration state reported by XC (NEW, APPROVED, ONLINE, ...); null until the CE has registered." value = data.xcsh_site_registration.this.found ? data.xcsh_site_registration.this.state : null}
output "registration_approval_name" { description = "Name of the approved registration (null when approve_registration is false or the CE has not registered yet)." value = one(xcsh_registration_approval.this[*].name)}
output "peer_count" { description = "Number of external BGP peers configured (one per Route Server IP; 0 when enable_bgp is false)." value = var.enable_bgp ? var.rs_peer_count : 0}variable "site_name" { description = "XC securemesh_site_v2 name (e.g. mcn-ce-ha-eastus01), created in namespace system." type = string}
variable "hostname" { description = "CE node hostname, must match the Azure VM computer name." type = string}
variable "interface_name" { description = "Auto-derived network_interface object name the BGP peer references: ves-io-securemesh-site-v2-<site>-network-<hostname>-eth0-0." type = string}
variable "mgmt_nic_mac" { description = "MAC address of the CE eth0/SLO NIC. Pins the SMSv2 interface to the NIC." type = string}
variable "ce_vm_instance_id" { description = "128-bit unique id of the CE VM instance this site's node runs on (azurerm_linux_virtual_machine.virtual_machine_id — the same value the CE reports as the registration's infra.instance_id). Required, and deliberately NOT the ARM resource id, which is name-derived and identical after a replacement. The site object's lifecycle is coupled to it so that rebuilding the node rebuilds the site instead of leaving a registration bound to a destroyed instance (issue #674)." type = string
validation { condition = trimspace(var.ce_vm_instance_id) != "" error_message = "ce_vm_instance_id must be the CE VM's virtual_machine_id; an empty value would couple every node's site to the same key." }}
variable "rs_peer_ips" { description = "List of Azure Route Server BGP peer IPs (virtual_router_ips), e.g. [10.0.4.4, 10.0.4.5]. Values may be unknown until apply; the count of peers comes from rs_peer_count so the peers block expands at plan time." type = list(string)}
variable "rs_peer_count" { description = "Number of external BGP peers (Azure Route Server always exposes exactly 2 virtual router IPs). Drives the peers block with a plan-known count so for_each never sees an unknown value." type = number default = 2}
variable "ce_asn" { description = "CE (local) BGP ASN." type = number default = 64512}
variable "rs_asn" { description = "Route Server (peer) BGP ASN." type = number default = 65515}
variable "peer_port" { description = "BGP peer TCP port." type = number default = 179}
variable "labels" { description = "Labels applied to the site object." type = map(string) default = {}}
variable "enable_bgp" { description = "Create the xcsh_bgp object. Defaults true, and nothing gates it: the object-ref name length blocker that once forced it false was removed in provider v3.74.0. Retained only as an escape hatch for planning or deploying the graph without BGP. See main.tf." type = bool default = true}
variable "create_site" { description = "Create the xcsh_securemesh_site_v2 resource object. Set false when the site object is auto-provisioned by registration approval." type = bool default = true}
variable "approve_registration" { description = "Approve the CE's runtime registration (named r-<uuid>, resolved from the site name by the xcsh_site_registration data source) instead of clicking Approve in the console. Two-phase by nature: the registration does not exist until the CE has booted and registered via the token, so the first apply plans no approval and a later apply creates it. For a CE that is ALREADY approved/ONLINE, import the existing approval (namespace/r-<uuid>, see the note in main.tf) rather than letting Terraform create it, since re-approving a non-NEW registration may be rejected. Set false to leave approval to an operator." type = bool default = true}
variable "os_version" { description = "CE operating system version. Empty deliberately selects the server-advertised latest version; set a value only to reproduce an older build." type = string default = ""}
variable "sw_version" { description = "CE F5 Distributed Cloud software version. Empty deliberately selects the server-advertised latest build; set a value only to reproduce an older build." type = string default = ""}terraform { required_version = ">= 1.8"
required_providers { xcsh = { source = "f5-sales-demo/xcsh" version = "= 11.0.2" } }}