HTTP access logs

Access logs show HTTP status codes, response classes, and methods for requests in a fixed window. A response class groups status codes such as 4xx for client errors and 5xx for server errors. Counts cover the logs returned by the query; logging, sampling, and ingestion can differ from API activity counters.

Run the bounded request sequence

This optional procedure produces known status codes on the authorized current demo. Skip it when reading existing telemetry.

Record a baseline with Load balancer metrics. This sequence sends 30 requests: 20 health requests, five explicit 404 responses, and five explicit 503 responses. Each request has a ten-second timeout and a unique marker. The origin stays online. A status mismatch stops the sequence.

Send the bounded sequence and stop on any status mismatch.

export XCSH_RUN_MARKER="statistics-outcomes-$(date -u +%Y%m%dT%H%M%SZ)"
export XCSH_BURST_START="$(date -u +%s)"
for XCSH_SEQUENCE in $(seq 1 30); do
  if [ "$XCSH_SEQUENCE" -le 20 ]; then
    XCSH_PATH=/health
    XCSH_EXPECTED=200
  elif [ "$XCSH_SEQUENCE" -le 25 ]; then
    XCSH_PATH=/httpbin/status/404
    XCSH_EXPECTED=404
  else
    XCSH_PATH=/httpbin/status/503
    XCSH_EXPECTED=503
  fi
  XCSH_STATUS="$(curl --silent --show-error --max-time 10 --output /dev/null \
    --write-out '%{http_code}' --get \
    --data-urlencode "statistics_probe=$XCSH_RUN_MARKER-$XCSH_SEQUENCE" \
    "https://<XCSH_DOMAINNAME>$XCSH_PATH")"
  printf 'sequence=%s expected=%s status=%s\n' "$XCSH_SEQUENCE" "$XCSH_EXPECTED" "$XCSH_STATUS"
  test "$XCSH_STATUS" = "$XCSH_EXPECTED"
done
export XCSH_BURST_END="$(date -u +%s)"

After telemetry rollup, refresh the query window and compare complete log counts, API activity, and load balancer metrics. Allow up to ten minutes for a new five-minute metric bucket.

Measured request results

The sequence ran from 2026-10-06T03:09:40Z to 2026-10-06T03:10:01Z. All 30 statuses matched: 20 × 200, five × 404, and five × 503. The timer continued to send traffic, so window-wide counts include background requests.

Count requests by status code

Use XCSH_NAMESPACE, the discovered XCSH_VIRTUAL_HOST, and the fixed two-hour XCSH_START_TIME–XCSH_END_TIME window. This query counts one page of at most 100 matching logs.

Build a virtual-host query capped at 100 records.

XCSH_LOG_QUERY="$(jq -nr --arg vh "<XCSH_VIRTUAL_HOST>" \
  '"{vh_name=" + ($vh | tojson) + "}"')"
XCSH_REQUEST="$(jq -nc --arg ns "<XCSH_NAMESPACE>" --arg start "$XCSH_START_TIME" \
  --arg end "$XCSH_END_TIME" --arg query "$XCSH_LOG_QUERY" '{
    namespace: $ns, start_time: $start, end_time: $end,
    limit: 100, sort: "DESCENDING", query: $query
  }')"

Fetch a page and retain only outcome fields.

XCSH_LOG_PAGE="$(curl --silent --show-error --fail-with-body --max-time 60 \
  --request POST \
  --header "Authorization: APIToken <XCSH_API_TOKEN>" \
  --header 'Content-Type: application/json' \
  --data-binary "$XCSH_REQUEST" \
  "<XCSH_API_URL>/api/data/namespaces/<XCSH_NAMESPACE>/access_logs")"
XCSH_TOTAL_HITS="$(jq -er '.total_hits | tonumber' <<<"$XCSH_LOG_PAGE")"
XCSH_LOG_ROWS="$(jq -ec --arg ns "<XCSH_NAMESPACE>" --arg vh "<XCSH_VIRTUAL_HOST>" '
  [.logs[]? | (if type == "string" then fromjson else . end)
    | select(.namespace == $ns and .vh_name == $vh)
    | {rsp_code, rsp_code_class, method}]
' <<<"$XCSH_LOG_PAGE")"
XCSH_COUNT_SCOPE=first_page
XCSH_PAGES=1
jq -er --arg scope "$XCSH_COUNT_SCOPE" \
  --arg start "$XCSH_START_TIME" --arg end "$XCSH_END_TIME" \
  --argjson total "$XCSH_TOTAL_HITS" --argjson pages "$XCSH_PAGES" '
  {scope: $scope, start_time: $start, end_time: $end,
   total_hits: $total, pages: $pages, counted: length,
   response_codes: (sort_by(.rsp_code) | group_by(.rsp_code)
     | map({code: .[0].rsp_code, count: length})),
   response_classes: (sort_by(.rsp_code_class) | group_by(.rsp_code_class)
     | map({class: .[0].rsp_code_class, count: length})),
   methods: (sort_by(.method) | group_by(.method)
     | map({method: .[0].method, count: length}))}
' <<<"$XCSH_LOG_ROWS"

Measured first-page response

Captured 2026-10-06T03:13:43Z; HTTP 200 for all pages.

{
  "scope": "first_page",
  "start_time": "1791249220",
  "end_time": "1791256420",
  "total_hits": 430,
  "pages": 1,
  "counted": 100,
  "response_codes": [
    {
      "code": "200",
      "count": 90
    },
    {
      "code": "404",
      "count": 5
    },
    {
      "code": "503",
      "count": 5
    }
  ],
  "response_classes": [
    {
      "class": "2xx",
      "count": 90
    },
    {
      "class": "4xx",
      "count": 5
    },
    {
      "class": "5xx",
      "count": 5
    }
  ],
  "methods": [
    {
      "method": "GET",
      "count": 100
    }
  ]
}

total_hits is the API’s matching-log count; counted and the grouped counts cover only this page. When total_hits exceeds counted, these are incomplete counts for the window.

Count all matching logs

Use the same XCSH_NAMESPACE, XCSH_VIRTUAL_HOST, XCSH_LOG_QUERY, and fixed two-hour window as the first-page query. Reissue it with scroll: true. Retrieve every page promptly: each cursor is valid for two minutes. The loop caps the export at 100 pages and stops on a scope mismatch or a count mismatch rather than publishing an incomplete total. Increase that bound deliberately for a larger authorized export.

Start a fresh scroll snapshot.

XCSH_REQUEST="$(jq -nc --arg ns "<XCSH_NAMESPACE>" --arg start "$XCSH_START_TIME" \
  --arg end "$XCSH_END_TIME" --arg query "$XCSH_LOG_QUERY" '{
    namespace: $ns, start_time: $start, end_time: $end,
    limit: 100, sort: "DESCENDING", scroll: true, query: $query
  }')"
XCSH_LOG_PAGE="$(curl --silent --show-error --fail-with-body --max-time 60 \
  --request POST \
  --header "Authorization: APIToken <XCSH_API_TOKEN>" \
  --header 'Content-Type: application/json' \
  --data-binary "$XCSH_REQUEST" \
  "<XCSH_API_URL>/api/data/namespaces/<XCSH_NAMESPACE>/access_logs")"
XCSH_TOTAL_HITS="$(jq -er '.total_hits | tonumber' <<<"$XCSH_LOG_PAGE")"
XCSH_LOG_ROWS='[]'
XCSH_PAGES=0

Collect only response codes, classes, and methods from every page.

while :; do
  XCSH_PAGES="$((XCSH_PAGES + 1))"
  test "$XCSH_PAGES" -le 100
  XCSH_PAGE_ROWS="$(jq -ec --arg ns "<XCSH_NAMESPACE>" \
    --arg vh "<XCSH_VIRTUAL_HOST>" '
    [.logs[]? | (if type == "string" then fromjson else . end)
      | if .namespace == $ns and .vh_name == $vh
        then {rsp_code, rsp_code_class, method}
        else error("Log outside selected scope") end]
  ' <<<"$XCSH_LOG_PAGE")"
  test "$(jq -r 'length' <<<"$XCSH_PAGE_ROWS")" -gt 0 || break
  XCSH_LOG_ROWS="$(jq -nc --argjson rows "$XCSH_LOG_ROWS" \
    --argjson page "$XCSH_PAGE_ROWS" '$rows + $page')"
  XCSH_SCROLL_ID="$(jq -r '.scroll_id // empty' <<<"$XCSH_LOG_PAGE")"
  test -n "$XCSH_SCROLL_ID" || break
  XCSH_REQUEST="$(jq -nc --arg ns "<XCSH_NAMESPACE>" --arg cursor "$XCSH_SCROLL_ID" \
    '{namespace: $ns, scroll_id: $cursor}')"
  XCSH_LOG_PAGE="$(curl --silent --show-error --fail-with-body --max-time 60 \
    --request POST \
    --header "Authorization: APIToken <XCSH_API_TOKEN>" \
    --header 'Content-Type: application/json' \
    --data-binary "$XCSH_REQUEST" \
    "<XCSH_API_URL>/api/data/namespaces/<XCSH_NAMESPACE>/access_logs/scroll")"
done
test "$(jq -r 'length' <<<"$XCSH_LOG_ROWS")" -eq "$XCSH_TOTAL_HITS"
XCSH_COUNT_SCOPE=complete_window
jq -er --arg scope "$XCSH_COUNT_SCOPE" \
  --arg start "$XCSH_START_TIME" --arg end "$XCSH_END_TIME" \
  --argjson total "$XCSH_TOTAL_HITS" --argjson pages "$XCSH_PAGES" '
  {scope: $scope, start_time: $start, end_time: $end,
   total_hits: $total, pages: $pages, counted: length,
   response_codes: (sort_by(.rsp_code) | group_by(.rsp_code)
     | map({code: .[0].rsp_code, count: length})),
   response_classes: (sort_by(.rsp_code_class) | group_by(.rsp_code_class)
     | map({class: .[0].rsp_code_class, count: length})),
   methods: (sort_by(.method) | group_by(.method)
     | map({method: .[0].method, count: length}))}
' <<<"$XCSH_LOG_ROWS"

Measured complete-window response

Captured 2026-10-06T03:13:47Z; HTTP 200 for all pages.

{
  "scope": "complete_window",
  "start_time": "1791249220",
  "end_time": "1791256420",
  "total_hits": 430,
  "pages": 6,
  "counted": 430,
  "response_codes": [
    {
      "code": "200",
      "count": 413
    },
    {
      "code": "404",
      "count": 12
    },
    {
      "code": "503",
      "count": 5
    }
  ],
  "response_classes": [
    {
      "class": "2xx",
      "count": 413
    },
    {
      "class": "4xx",
      "count": 12
    },
    {
      "class": "5xx",
      "count": 5
    }
  ],
  "methods": [
    {
      "method": "GET",
      "count": 430
    }
  ]
}

The API can return a nonempty cursor on an empty final page, so the loop also stops on an empty page. A successful equality check verifies that all matching logs in this snapshot were counted. It does not guarantee all real requests were logged or that late telemetry had arrived. Requery after ingestion delay when verifying a new sequence.

The log snapshot at 2026-10-06T03:13:40Z contained 430 logs across five populated pages and an empty final page. All returned methods were GET.

Find failed requests

For a later query, change the matcher to response codes in the 4xx or 5xx classes. To filter a method, add an exact method="GET" matcher. Rebuild the request and run the pagination procedure for complete filtered counts.

XCSH_LOG_QUERY="$(jq -nr --arg vh "<XCSH_VIRTUAL_HOST>" \
  '"{vh_name=" + ($vh | tojson) + ",rsp_code=~\"[45].*\",method=\"GET\"}"')"

The bounded request sequence produces explicit 404 and 503 responses without taking the origin offline. An HTTP status does not identify WAF action: the demo rejection page can return 200, so confirm blocks with security events.