11. Log StreamingΒΆ
The Log Streaming service collects access logs from all CTL CDN edge nodes, filters them per SubscriberID, formats to the desired log format and POSTs, via HTTP, to a customer-provided endpoint. Logs are batched up into a JSON-formatted payload and POSTed to the customer-provided endpoint in near real-time (sub-5 minutes). It should be noted that customers with large traffic volumes can create a significant volume of logs and must provide a log endpoint capable of ingesting at a high rate. Failure to provide such an endpoint may result in logs being dropped as the CTL log processing system only queues logs for a short period of time if the endpoint is unavailable or unable to keep up. For more details on the log streaming service, see https://www.centurylink.com/business/help/cdn/cdn-log-streaming.html.
Note that Logs cannot be filtered to a subset (ie only stream me the errors) of the requests at this time.
A Log endpoint is defined as a top level JSON object as shown in the example below:
"logEndpoints": [ { "id": 1, "name": "LogStreamingEndpoint", "protocol": "HTTPS", "validateCert": true, "absPath": "/logs", "logLineFormat": "JSON_Array", "port": 443, "connections": 0, "headers": [ { "name": "ctl-cdn", "value": "log_streaming" } ], "host": "www.example.com", "maxBytesPerMessage": 524288, "maxPostInterval": 2, } ]
Attributes of a logEndpoint definition are:
- id: unique static identifier for a logEndpoint definition. Used to identify which logEndpoint is to be used. Integer, Required.
- name: human-readable name given for the logEndpoint definition, unique within a single subscriber. Required.
- Protocol: The protocol used to deliver the logs, valid options are HTTP or HTTPS
- ** validateCert:** Boolean. If set to true, makes sure the presented certificate CN matches the hostname. If set to false, does not check the presented certificates CN.
- absPath The path to where the logs will be placed on the remote server.
- logLineFormat: JSON_Array or JSON_Newline
- port: The port the remote server is listening on.
- connections:
- Headers: A name/value array of headers to be added to the HTTP/HTTPS requests.
- host: The hostname of the remote server, must resolve to a routable IP address.
- maxBytesPerMessage The maximum number of bytes (log data) that will be sent in a given POST.
- maxPostInterval: Maximum time the system will hold the logs till they are sent to the remote server, if max bytes is not reached. In seconds.
Once a Log Endpoint is defined, its ID can be associated with each ReportingID you wish to receive logs for, as shown in the example below:
"reportingIds": [ { "logEndpoints": [ 1 ], "id": 24152, "name": "ReportingID 24152" } ]
Changed in version 19.1: Updated to allow definition of multiple log endpoints.