Custom log format to additional logging endpoint

Comments

5 comments

  • joeshaw

    Ok, after some experimentation I figured it out on my own. The community post I linked to above is very helpful, once you understand a few things.

    First, create a new logging destination. Set it up exactly how you want it except for the log format, which won't matter. Then add a condition on that log destination (click the gear, then Conditions). In order to prevent double log lines, you need to add a condition that prevents this handler from ever running. Something like !req.url in the "Apply if" field is good. Now if you look at your generated VCL you'll see something like:

    if( !req.url ) { #syslog Dest_Three log {"syslog 6wMdDggKIUzAZy47sjjhW9 Dest Three :: "} req.http.Fastly-Client-IP {" "} {""-""} {" "} {""-""} {" "} now {" "} req.request {" "} req.url {" "} resp.status; }

    The if check there ensures that this handler will not run. Take note of the first part of the log line (the {syslog 6wMdDggKIUzAZy47sjjhW9 Dest Three :: "} part) -- you'll need to copy that part into your custom VCL to ensure that Fastly delivers the logs to the right place.

    Now, onto the custom VCL. vcl_log is just another subroutine that gets run. I'm not exactly sure where in the flow it lives because I can't find much documentation on it, but it gets run somewhere in the Varnish machinery. Fastly's boilerplate VCL does not define it, which is good, because you don't end up overloading any of their code.

    In it, log however you want, something like:

    sub vcl_log { log {"syslog 6wMdDggKIUzAZy47sjjhW9 Dest Three ::"} {" timestamp="} time.start.sec {" host="} req.http.Host {" status="} resp.status {" bytes="} resp.http.content-length {" client_ip="} req.http.Fastly-Client-IP {" method="} req.request {" url=""} req.url {"""} {" content_type=""} resp.http.Content-Type {"""} if(geoip.city,{" geoip_city=""} geoip.city {"""},{""}) if(geoip.region,{" geoip_region=""} geoip.region {"""},{""}) if(geoip.country_code,{" geoip_country="} geoip.country_code,{""}) if(geoip.continent_code,{" geoip_continent="} geoip.continent_code,{""}) {" fastly_region="} server.region {" fastly_datacenter="} server.datacenter {" fastly_node="} server.identity {" fastly_state="} fastly_info.state {" duration_ms="} time.elapsed.msec; }

    And then upload that custom VCL and you're done!

    One last thing from that community post that threw me off: it warns you about adding the #FASTLY deliver macro to the vcl_deliver handler. Ignore that. Your custom VCL is not touching the vcl_deliver handler, so you don't need to worry about that.

  • devforce

    Thanks a lot for publishing this info. It really helps to make sense of the original Fastly post on this subject. Really helpful.

  • ryanartecona

    This was exactly what I was just trying to do, and it worked like a charm. Thanks!

  • joeshaw

    Worth pointing out that custom VCL might not be necessary anymore with the improved log formatting options available now. See https://docs.fastly.com/guides/streaming-logs/custom-log-formats and https://docs.fastly.com/guides/streaming-logs/changing-log-line-formats for more info.

  • teamouting

    This site include blogs are different and interesting.i visit in this site for the first time.i really interesting to read this blog.i got a clear information of this topic.thanku so much for including this article.team outing in and around Bangalore

Please sign in to leave a comment.