11. Log StreamingΒΆ

Configuration Definition: Subscriber
Configuration Application: Property
Default Behavior: Off/Disabled
Limitations: Log Streaming is a chargeable service and can only be configured if purchased. Up to 20 log endpoints can be defined.

The Log Streaming service collects access logs from all CTL CDN edge nodes, filters them per SubscriberID, formats to the desired log format and POSTs, via HTTP, to a customer-provided endpoint. Logs are batched up into a JSON-formatted payload and POSTed to the customer-provided endpoint in near real-time (sub-5 minutes). It should be noted that customers with large traffic volumes can create a significant volume of logs and must provide a log endpoint capable of ingesting at a high rate. Failure to provide such an endpoint may result in logs being dropped as the CTL log processing system only queues logs for a short period of time if the endpoint is unavailable or unable to keep up. For more details on the log streaming service, see https://www.centurylink.com/business/help/cdn/cdn-log-streaming.html.

Note that Logs cannot be filtered to a subset (ie only stream me the errors) of the requests at this time.

A Log endpoint is defined as a top level JSON object as shown in the example below:

"logEndpoints": [
  {
    "id": 1,
    "name": "LogStreamingEndpoint",
    "protocol": "HTTPS",
    "validateCert": true,
    "absPath": "/logs",
    "logLineFormat": "JSON_Array",
    "port": 443,
    "connections": 0,
    "headers": [
      {
        "name": "ctl-cdn",
        "value": "log_streaming"
      }
    ],
    "host": "www.example.com",
    "maxBytesPerMessage": 524288,
    "maxPostInterval": 2,
  }
]

Attributes of a logEndpoint definition are:

  • id: unique static identifier for a logEndpoint definition. Used to identify which logEndpoint is to be used. Integer, Required.
  • name: human-readable name given for the logEndpoint definition, unique within a single subscriber. Required.
  • Protocol: The protocol used to deliver the logs, valid options are HTTP or HTTPS
  • ** validateCert:** Boolean. If set to true, makes sure the presented certificate CN matches the hostname. If set to false, does not check the presented certificates CN.
  • absPath The path to where the logs will be placed on the remote server.
  • logLineFormat: JSON_Array or JSON_Newline
  • port: The port the remote server is listening on.
  • connections:
  • Headers: A name/value array of headers to be added to the HTTP/HTTPS requests.
  • host: The hostname of the remote server, must resolve to a routable IP address.
  • maxBytesPerMessage The maximum number of bytes (log data) that will be sent in a given POST.
  • maxPostInterval: Maximum time the system will hold the logs till they are sent to the remote server, if max bytes is not reached. In seconds.

Once a Log Endpoint is defined, its ID can be associated with each ReportingID you wish to receive logs for, as shown in the example below:

"reportingIds": [
  {
    "logEndpoints": [
      1
    ],
    "id": 24152,
    "name": "ReportingID 24152"
  }
]

Changed in version 19.1: Updated to allow definition of multiple log endpoints.