Different systems have standardized on different logging formats over time. Fastly believes logging should be as customizable as possible, working with whichever infrastructure you already have in place. This guide details some of the more complicated custom logging strings (e.g., JSON, Key/Value, CSV, and URL-encoded) you can use to implement the logging formats mentioned in the Apache logging module.
IMPORTANT: Be sure to take into account security, privacy, and compliance requirements when making configuration and endpoint decisions for the data you intend to include in streamed logs.
1
| %h %l %u %t "%r" %>s %b
|
This is the default for many of our logging providers.
1
| %v %h %l %u %t "%r" %>s %b
|
1
| %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i"
|
You’ll have to create a regular Syslog logging object pointing at your endpoint. For example, if Fastly didn't have a Loggly logging object then this would mean setting hostname
to logs-01.loggly.com
, port
to 6514
, use_tls
to true
, and the message format
field to blank
. Then, in the format
field, you would put the following:
1
| <134>1 %{"%Y-%m-%dT%T%z"}t %{server.datacenter}V <log name> - - [<token>@<PEN> tag="fastly" tag="other tag" <tags>] <regular format string>
|
The various fields you need to replace are:
- log name - this can be whatever you want but we recommend using the same name as you’ve used for the logging object in Fastly.
- token - the private token for your RFC5424 endpoint (if sending to Loggly this is your Customer Token).
- PEN - this is a Private Enterprise Number. For example the Loggly PEN (Private Enterprise Number) is 41058. If you want to send to another provider then you can look up their PEN on the IANA registry and use that.
- tags - these can be any key/value pairs you want. In the example we have two: fastly and other tag
- regular format string - this is regular Fastly logging directives, put whatever you want here (for example: the Common Log Format mentioned above).
Structured data
The examples below demonstrate different representations of the same variables and variable types:
Name |
VCL Value |
Type |
Description |
Protocol |
req.protocol |
string |
The HTTP protocol version. |
Epoch Seconds |
time.start.sec |
number |
The time at the start of the request in seconds. |
Start Time |
begin:%Y-%m-%dT%H:%M:%S%z |
time |
The time at the start of the request in ISO 8601. format |
User Agent |
req.http.User-Agent |
escaped string |
The User-Agent request header. |
Is IPv6 |
req.is_ipv6 |
boolean |
Whether the request was made over IPv6 or not. |
ID |
deadbeef |
literal string |
A generic ID. |
Some String |
dwayne "the rock" johnson |
escaped literal string |
A string with quotation marks in it. |
Version |
1.1 |
literal number |
A generic version number. |
JSON
1
2
3
4
5
6
7
8
9
10
| {
"protocol" : "%H",
"epoch_seconds" : %{time.start.sec}V,
"time_start" : "%{begin:%Y-%m-%dT%H:%M:%S%z}t",
"user_agent" : "%{User-Agent}i",
"is_ipv6" : %{if(req.is_ipv6, "true", "false")}V,
"some_string":"%{json.escape(\{"dwayne "the rock" johnson"\})}V",
"id" : "deadbeef",
"version" : 1.1
}
|
CSV
1
| %H, %{time.start.sec}V, %{begin:%Y-%m-%dT%H:%M:%S%z}t, %{regsub(req.http.User-Agent, \{"""\}, \{"""""\})}V, %{if(req.is_ipv6, "true", "false")}V, deadbeef, %{regsub(\{"dwayne "the rock" johnson"\}, \{"""\}, \{"""""\})}V, 1.1
|
Key/Value
1
| protocol:%H, epoch_seconds:%{time.start.sec}V, time_start:%{begin:%Y-%m-%dT%H:%M:%S%z}t, user_agent:%{User-Agent}i, is_ipv6:%{if(req.is_ipv6, "true", "false")}V, id:deadbeef, some_string:%{json.escape(\{"dwayne "the rock" johnson"\})}V, version:1.1
|
URL Encoded
1
| protocol=%H&epoch_seconds=%{time.start.sec}V&time_start=%{begin:%Y-%m-%dT%H:%M:%S%z}t&user_agent=%{urlencode(req.http.User-Agent)}i&is_ipv6=%{if(req.is_ipv6, "true", "false")}V&some_string=%{urlencode(\{"dwayne "the rock" johnson"\})}V&id=deadbeef&version=1.1
|