Articles in this section

See more

Log streaming: Honeycomb

  • Updated

Fastly's Real-Time Log Streaming feature can send logs in JSON format to Honeycomb. Honeycomb is a tool that allows developers to explore the operations of complex systems, microservices, and databases.

Prerequisites

Before adding Honeycomb as a logging endpoint for Fastly services, you'll need to perform the following steps:

  • Sign up for a Honeycomb account if you don't already have one.
  • Obtain the Write Key for your team on the Honeycomb Account page.
  • Choose a Dataset name. If you plan to collect data from multiple environments (like production, development, staging), Honeycomb recommends creating a Dataset for each environment and naming your Datasets accordingly (e.g., prod.queries, dev.queries, and staging.queries). If a Dataset doesn't exist, Honeycomb will create one automatically.

Adding Honeycomb as a logging endpoint

  1. Review the information in our Setting Up Remote Log Streaming guide.

  2. Click the Honeycomb Create endpoint button. The Create a Honeycomb endpoint page appears.

    the create a Honeycomb endpoint

  3. Fill out the Create a Honeycomb endpoint fields as follows:
    • In the Name field, type a human-readable name for the endpoint.
    • In the Log format field, enter the data to send to Honeycomb. See the example format section for details.
    • In the Write Key field, type the write key for your Honeycomb team. This is available on the Honeycomb Account page.
    • In the Dataset field, type the name of the Honeycomb Dataset (e.g., myDataset).

  4. Click the Advanced options link of the Create a Honeycomb endpoint page. The Advanced options appear.

    the advanced options on the create a new Honeycomb endpoint page

  5. In the Placement area, select where the logging call should be placed in the generated VCL. Valid values are Format Version Default, None, and waf_debug (waf_debug_log). Selecting None creates a logging object that can only be used in custom VCL. See our guide on WAF logging for more information about waf_debug_log.
  6. Click the Create button to create the new logging endpoint.
  7. Click the Activate button to deploy your configuration changes.

Example format

Data sent to Honeycomb must be serialized as a JSON object. Here's an example format string for sending data to Honeycomb:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

{
  "time":"%{begin:%Y-%m-%dT%H:%M:%SZ}t",
  "data":  {
    "service_id":"%{req.service_id}V",
    "time_elapsed":%D,
    "request":"%m",
    "host":"%{Fastly-Orig-Host}i",
    "url":"%{cstr_escape(req.url)}V",
    "protocol":"%H",
    "is_ipv6":%{if(req.is_ipv6, "true", "false")}V,
    "is_tls":%{if(req.is_ssl, "true", "false")}V,
    "is_h2":%{if(fastly_info.is_h2, "true", "false")}V,
    "client_ip":"%h",
    "geo_city":"%{client.geo.city.utf8}V",
    "geo_country_code":"%{client.geo.country_code}V",
    "server_datacenter":"%{server.datacenter}V",
    "request_referer":"%{Referer}i",
    "request_user_agent":"%{User-Agent}i",
    "request_accept_content":"%{Accept}i",
    "request_accept_language":"%{Accept-Language}i",
    "request_accept_charset":"%{Accept-Charset}i",
    "cache_status":"%{regsub(fastly_info.state, "^(HIT-(SYNTH)|(HITPASS|HIT|MISS|PASS|ERROR|PIPE)).*", "\\2\\3") }V",
    "status":"%s",
    "content_type":"%{Content-Type}o",
    "req_header_size":%{req.header_bytes_read}V,
    "req_body_size":%{req.body_bytes_read}V,
    "resp_header_size":%{resp.header_bytes_written}V,
    "resp_body_size":%{resp.body_bytes_written}V,
  }
}

Related articles