Varnish is excellent at caching, managing, and delivering small files but historically the handling of large files has been challenging. By default, Fastly limits cached file sizes to 2GB. Using streaming miss increases that limit for files up to 5GB. Fastly’s also allows you to enable support for files larger than the 5GB caching limit using VCL snippets. Regardless of which feature you use, there are a few failure modes you may encounter when working with large files, especially those larger than 5GB in size.
Maximum object size limits
If the response from the origin has a Content-Length header field that exceeds the maximum object size, Fastly will immediately generate a 503 response to the client unless you tell Fastly to specifically pass those objects directly to the user from the origin If no Content-Length header field is returned, Fastly will start to fetch the response body. If, while fetching the response body, we determine that the object exceeds maximum object size, we will generate a status 503 response to the client (again, unless you specifically tell Fastly to do otherwise as previously mentioned).
If no Content-Length header field is present and Streaming Miss is in effect, Fastly will stream the content back to the client. However, if while streaming the response body Fastly determines that the object exceeds the maximum cacheable object size, it will terminate the client connection abruptly. The client will detect a protocol violation, because it will see its connection close without a properly terminating 0-length chunk.
Origin read failures
If reading the response body from the origin fails or times out, the problem will be reported differently depending on whether or not you’ve enabled Streaming Miss to act when the object is fetched. Without Streaming Miss, a 503 response will be generated. With Streaming Miss, however, it is already too late to send an error response since the header will already have been sent. In this case, Fastly will abruptly terminate the client connection and the client will detect a protocol violation. If the response was chunked, the client will see its connection close without a properly terminating 0-length chunk. If Content-Length was known, the client will see the connection close before the number of bytes given.