Stale content on error with shielding



  • Peter Wohlers

    Hi Ray-

    I was actually investigating a similar issue over the weekend and have filed an issue about how having an origin shield configured kind of complicates this interaction. With soft-purge and SWR configured, you should expect the first response post-purge to be a stale response while it refreshes in the background, and you see that when origin shield is not involved.

    I do eventually see the fresh object getting delivered but it takes more requests than I would expect it to. e.g. i just tested this and it seems like there were ten or so requests before everything was consistent.

    I think what is happening is that in a POP, there are primary and secondary storage locations for any given object where they get written to disk. Busy objects will get cached locally on the rest of the nodes in the POP in RAM. I think what's happening is that the SWR response is getting returned from each of the cache nodes where the objects are resident in RAM, so you get more stale-while-revalidate responses while the cache nodes refresh asynchronously.

    I can update this when I get confirmation.

    Comment actions Permalink
  • Ray Diciaccio

    Hi Peter,

    Thanks for the response. That's really interesting behavior you're seeing. If you take a look at the TTLs of the objects being served up from the edge nodes, what do you see? Are they getting set as they normally would for content delivered from your shield?

    In my case we're setting large TTLs on our objects, and it appears that even when the shield serves up that initial stale object, the edge nodes set the full TTL on the response. So it looks to me like they're fully caching the object on the edge. I'll see if I get similar behavior to you after a bunch of requests.

    Comment actions Permalink
  • Peter Wohlers

    Hi rayd -

    sorry for the delay. It turns out that when one has Stale-While-Revalidate and shielding configured and does a soft-purge, the edge nodes, when they receive a new object from the shield (a SWR response), it doesn't recognize that object as a SWR response.

    The way to get around this is by detecting if the request is shield request and disabling SWR for those requests.

    This can be done in custom VCL above the #FASTLY recv macro

    sub vcl_recv {
      /* if shielding is enabled, the below code is required */
      if (req.http.Fastly-FF) {
        set req.max_stale_while_revalidate = 0s;
    #FASTLY recv

    hope this helps.

    Comment actions Permalink

Please sign in to leave a comment.