Googlebot always seems to bypass the cache

Comments

2 comments

  • Thomas Davis

    And here is a browser request that is working

    access-control-allow-headers:X-Requested-With access-control-allow-origin:* age:1395 cache-control:no-cache, no-store, must-revalidate cf-ray:2968ed9eca981d68-MEL content-encoding:gzip content-type:text/html; charset=utf-8 date:Wed, 20 Apr 2016 13:26:28 GMT expires:0 fastly-debug-digest:a7b7ff5fbbfb6b4d6a4135e2b64da745fe75f1277a6a5d57bde4f418ce574c3b pragma:no-cache server:cloudflare-nginx status:200 status:200 OK via:1.1 varnish via:1.1 vegur via:1.1 varnish x-cache:MISS, HIT x-cache-hits:0, 2 x-powered-by:Phusion Passenger 5.0.25 x-served-by:cache-jfk1036-JFK, cache-mel6521-MEL x-timer:S1461158788.924244,VS0,VE0

  • Justin

    [quote="thomasdavis, post:1, topic:677"] Set-Cookie: __cfduid=d4277cb882bba4e2c64c73047786f4b4f1461158237; expires=Thu, [/quote]

    Hi Thomas

    The set-cookie header prevents us from caching the response. This is to prevent user-specific content being cached and served among different users.

    It looks like it only gets set when the request didn't come in with a corresponding cookie header (the Googlebot request), while the one from your browser does have one.

    If you need to, you can remove the set-cookie header from the response:

    https://docs.fastly.com/guides/basic-configuration/removing-headers-from-backend-response

Please sign in to leave a comment.