Development Guide

This guide describes the development workflow for the F5 XC API Enrichment Pipeline.

Quick Start

First-Time Setup

# Clone the repository
git clone https://github.com/f5-sales-demo/api-specs-enriched.git
cd api-specs-enriched

# Create virtual environment and install dependencies
make install

# Install pre-commit hooks
make pre-commit-install

# Download specs and run pipeline
make build

# Preview documentation locally
make serve

Daily Development

# Quick rebuild (uses cached specs)
make rebuild

# Run all quality checks
make pre-commit-run

# Preview documentation
make serve

Architecture Overview

Two-Folder Design

┌─────────────────────┐     ┌─────────────────────────────┐
│   specs/original/   │────▶│   docs/specifications/api/  │
│   (READ-ONLY)       │     │   (GENERATED)               │
│   - Downloaded      │     │   - Domain specs            │
│   - Gitignored      │     │   - Master spec             │
│   - Release pinned  │     │   - Release artifacts       │
└─────────────────────┘     └─────────────────────────────┘

Pipeline Flow

Download (release) → Enrich → Normalize → Canonical merge → Lint → Serve
     │              │          │         │       │
     ▼              ▼          ▼         ▼       ▼
 Source specs   Branding    Fix refs   Domain docs  Spectral
               + Grammar    + Types    + Master     rules

Directory Structure

Directory	Purpose	Git Status
`specs/original/`	F5 source specs	Gitignored
`specs/discovered/`	API discovery output	Tracked (openapi.json, session.json)
`docs/specifications/api/`	Generated domain specs	Tracked release input
`scripts/`	Python pipeline scripts	Tracked
`config/`	Pipeline configuration	Tracked
`reports/`	Generated reports	Gitignored

Workflow Patterns

1. Discovery Workflow

Prerequisites:

VPN connection to F5 XC environment
Valid API credentials

# Set credentials
export XCSH_API_URL="https://your-tenant.console.ves.volterra.io/api"
export XCSH_API_TOKEN="your-api-token"

# Run discovery
make discover

# View results
jq '.statistics' specs/discovered/session.json

# Commit for CI/CD
make push-discovery

Discovery captures:

Actual required/optional fields
Enum values from live responses
Default values
Pattern validations
Response examples

2. Release Workflow

Releases are automated:

Daily schedule (6 AM UTC) or push to main triggers workflow
The latest stable upstream release is resolved and its asset digest verified
Pipeline processes and enriches specs
The committed build version is calculated before artifact generation
A release commit and tag are created for that exact generated tree
GitHub Release created with changelog
Documentation deployed to GitHub Pages

Version Bump Rules:

Condition	Bump Type	Example
`[major]` in commit	Major	1.0.0 → 2.0.0
`BREAKING CHANGE` in commit	Major	1.0.0 → 2.0.0
New domain spec added	Minor	1.0.0 → 1.1.0
Any other change	Patch	1.0.0 → 1.0.1

3. Development Workflow

# Start from the fetched protected branch
git fetch --prune
git switch --no-track -c feature/123-my-change origin/main

# Make changes to config or scripts

# Test locally
make pipeline
make lint

# Commit (pre-commit hooks run automatically)
git add config/example.yaml scripts/example.py tests/test_example.py
git diff --cached --check
git commit -m "feat: add new enrichment rule"

# Push and create PR
git push -u origin feature/my-change
gh pr create

Discovery Deep Dive

What is Discovery?

Discovery explores the live F5 XC API to find:

Undocumented constraints: Required fields not marked in OpenAPI
Actual enum values: Real values seen in production
Default behaviors: Server-applied values when fields are omitted
Response patterns: Actual data shapes

Discovery Configuration

discovery:
  exploration:
    namespaces:
      - "system"
      - "shared"
    methods:
      - "GET"
      - "OPTIONS"
    max_endpoints_per_run: 500

  schema_inference:
    sample_size: 3
    detect_patterns: true
    detect_constraints: true

Discovery Extensions

Discovery adds x-discovered-* extensions to specs:

{
  "properties": {
    "name": {
      "type": "string",
      "x-discovered-required": true,
      "x-discovered-pattern": "^[a-z][a-z0-9-]*$",
      "x-discovered-examples": ["my-app", "prod-lb"]
    }
  }
}

Constraint Analysis Report

make constraint-report

Release Process

Release Package Contents

f5xc-api-specs-v1.0.14.zip
├── openapi.json       # Master combined spec
├── openapi.yaml       # YAML format
├── domains/           # Individual domain specs
│   ├── api_security.json
│   ├── load_balancer.json
│   └── ...
├── index.json         # Metadata
├── CHANGELOG.md       # Release notes
└── README.md          # Usage instructions

Manual Workflow Trigger

gh workflow run sync-and-enrich.yml
gh run watch

Configuration Guide

Enrichment Configuration

enrichment:
  branding:
    replacements:
      "Volterra": "F5 Distributed Cloud"
      "VES": "F5 XC"

  acronyms:
    API: "Application Programming Interface"
    DNS: "Domain Name System"

  grammar:
    enabled: true
    language_tool: true

Normalization Configuration

normalization:
  orphan_refs:
    fix: true
    remove_if_unresolvable: true

  empty_operations:
    remove: true

  type_standardization:
    enabled: true

Spectral linter rules

extends:
  - spectral:oas

rules:
  operation-operationId: error
  operation-tags: warn

OpenAPI Extensions

The enrichment pipeline uses vendor extensions to embed validation and default value metadata.

Extension Categories

Extension	Meaning	Implication
`x-f5xc-server-default`	The F5 XC API server applies this value when the field is omitted	Field is optional; omitting it produces the documented default behavior
`x-f5xc-recommended-value`	The F5 XC web console pre-populates this value for new resources	Field has no server default but this value represents typical configuration
`x-f5xc-recommended-oneof-variant`	The F5 XC console preselects this OneOf variant	Identifies the typical choice when multiple mutually exclusive options exist
`x-f5xc-conflicts-with`	Lists other properties that cannot be used together with this field	Property is part of a OneOf group; only one of the conflicting properties can be specified

Required Field Extensions

Extension	Source	Purpose
`x-ves-required: "true"`	F5 XC original spec	Field requires non-zero/non-empty value
`x-f5xc-required-for`	Enrichment pipeline	Context-specific required status

x-f5xc-required-for contexts:

create: Required when creating the resource
update: Required when updating the resource
minimum_config: Required for minimum viable configuration

Default Value and OneOf Extensions

Extension	Purpose
`x-f5xc-server-default: true`	Marks server-applied defaults
`x-f5xc-recommended-value`	F5 XC console pre-populated value
`x-f5xc-recommended-oneof-variant`	Recommended OneOf variant
`x-f5xc-conflicts-with`	Mutual exclusivity with other OneOf properties

x-f5xc-server-default

Type: boolean

When true, indicates the accompanying default value is enforced by the F5 XC API server. Fields with this extension can be safely omitted from API requests — the server applies the default automatically.

use_http2:
  type: boolean
  default: false
  x-f5xc-server-default: true

x-f5xc-recommended-value

Type: any (matches field type)

Specifies a value that the F5 XC web console uses as a pre-populated default. This value is not server-enforced but represents the typical starting configuration for new resources created via the console.

timeout:
  type: integer
  x-f5xc-recommended-value: 3

x-f5xc-recommended-oneof-variant

Type: object (map of group name to variant name)

For schemas with mutually exclusive field groups, identifies which variant is the default or most common choice. The key is the OneOf group name and the value is the recommended variant field name.

healthcheckCreateSpecType:
  type: object
  x-f5xc-recommended-oneof-variant:
    health_check: "http_health_check"

x-f5xc-conflicts-with

Type: array of strings

Added: v3.2.0 (Issue #494)

Declares mutual exclusivity relationships between OneOf group members. Auto-derived from x-ves-oneof-field-* extensions. This enables downstream tools (Terraform, CLI, MCP) to validate conflicts at schema level rather than runtime.

host_header:
  type: string
  x-f5xc-conflicts-with: ["use_origin_server_name"]

use_origin_server_name:
  type: object
  x-f5xc-conflicts-with: ["host_header"]

Use cases:

Terraform providers can generate validation rules
CLI tools can warn about conflicting field combinations
AI assistants can generate correct OneOf configurations
IDE extensions can provide conflict-aware autocompletion

Source: Auto-derived from F5 native x-ves-oneof-field-* extensions during pipeline enrichment.

Validation Rule Implications

The enricher inspects x-ves-validation-rules to infer required status:

Rule	Implication
`ves.io.schema.rules.message.required: "true"`	Field is required
`ves.io.schema.rules.uint32.gte: N`	If N >= 1 and no server default, field is required
`ves.io.schema.rules.repeated.min_items: N`	If N >= 1, array requires at least N items
`ves.io.schema.rules.string.min_bytes: N`	If N >= 1, string requires at least N bytes

Resources with Server-Applied Defaults

Some resources accept empty specs because the server applies defaults:

Resource	Server-Applied Defaults
`app_firewall`	`monitoring: \{\}`, `default_detection_settings: \{\}`
`rate_limiter`	`limits: []`, `user_identification: []`
`api_definition`	`swagger_specs: []`, default `api_groups`
`healthcheck`	`jitter: 0`, `jitter_percent: 0`, nested http_health_check defaults

For these resources, x-f5xc-required-for.create may be false even when x-ves-required is true.

Default Value Patterns

Pattern	Type	Example
`\{\}`	Empty object (choice selection)	`monitoring: \{\}`
`[]`	Empty array	`expected_status_codes: []`
`0`	Numeric	`jitter: 0`
`""`	String	`expected_response: ""`
`false`	Boolean	`use_http2: false`

Adding Discovered Defaults

Create resource in F5 XC with minimal spec via API
Read back the resource to see server-applied values
Document defaults in config/discovered_defaults.yaml
Run pipeline: make pipeline
Verify with: jq '.components.schemas | to_entries[] | select(.key | contains("resource_name"))'

Troubleshooting

Pre-commit Takes Too Long

The pipeline runs on every commit to ensure consistency. For work-in-progress commits:

make pre-commit-run
git commit -m "fix: describe the reviewed change"

Discovery Fails

# Check VPN
ping your-tenant.console.ves.volterra.io

# Check presence without printing either value
test -n "${XCSH_API_TOKEN:-}" && echo "XCSH_API_TOKEN is set"
test -n "${XCSH_API_URL:-}" && echo "XCSH_API_URL is set"

Linting errors on generated specs

# Check lint report
cat reports/lint-report.json | jq '.errors'

Fix issues in enrichment/normalization config, not the output files.

Release and Build Identity

Downloads are pinned by the immutable six-field .github_release receipt: version, tag_name, published_at, asset_name, asset_size, and asset_sha256. The asset name and SHA-256 identify the downloaded upstream bytes; the committed release version identifies the generated build. Neither identity is derived from an ETag or from a mutable branch.

# The release workflow calculates VERSION, then stamps that exact identity.
python -m scripts.stamp_release_version docs/specifications/api --version "$VERSION"

Missing Specs After Clone

Generated specs are gitignored:

make build  # Downloads and generates everything

Large File Blocked

Add exclusion to .pre-commit-config.yaml:

- id: check-added-large-files
  exclude: ^path/to/large/file\.json$

Quick Reference

Make Targets

Target	Description
`make build`	Full build (download + pipeline)
`make rebuild`	Quick rebuild (skip download)
`make download`	Download the pinned stable release
`make download-force`	Force download
`make pipeline`	Run enrichment pipeline
`make lint`	Spectral linting
`make serve`	Local documentation server
`make discover`	API discovery (VPN required)
`make push-discovery`	Commit discovery data
`make clean`	Remove generated files
`make pre-commit-run`	Run all quality checks

Environment Variables

Variable	Purpose
`XCSH_API_URL`	F5 XC tenant API URL
`XCSH_API_TOKEN`	API authentication token

Key Files

File	Purpose
`.github_release`	Immutable six-field upstream release receipt
`CHANGELOG.md`	Auto-generated changelog
`config/enrichment.yaml`	Enrichment rules
`config/normalization.yaml`	Normalization rules
`config/discovery.yaml`	Discovery settings
`config/spectral.yaml`	Linter rules
`scripts/utils/version_calculator.py`	Tag-based version calculation

Note: Release generation supplies and commits the exact build version; the upstream receipt independently identifies the source archive.