Nothing has changed: 304
check if remote files have changed…
What the heck is (the HTTP Status-Code) 304?
I believe many don´t know because it´s nothing you see in the browser, except you are constantly watching the network tab in the dev tools 😏
The explanation can be found here.
“The HTTP
304 Not Modifiedclient redirection response code indicates that there is no need to retransmit the requested resources. It is an implicit redirection to a cached resource.”
What is this good for? Let’s consider the scenario where you need to download a file whenever it changes. Let’s say your boss told you that the “change” should be processed as soon as possible, so you are forced to check every minute, and let’s pretend the other Host cannot trigger you to let you know that there is a change. So it’s up to you to find out.
And if that is not enough, the filesize is 5GB 🙈
So a clever mind should immediately notice that the worst concept would be downloading 5GB every 60 seconds, building a hash (or whatever you use) to check if something has changed — right? 🤦♂️
Thankfully, the HTTP Protocol has something for us
to solve this problem 💪
What I am going to explain only works, if the Webserver is sending a specific Header, you can test this very easily by using curl:
>>>
curl -I https://something.com/averylargefile.csv
<<<<
HTTP/2 200
date: Thu, 09 Nov 2023 19:52:20 GMT
server: Apache
strict-transport-security: max-age=31104000
last-modified: Thu, 09 Nov 2023 19:45:01 GMT <----- NICE !
etag: "9c869-609bd708a103c" <---------------------- NICE !
accept-ranges: bytes
content-length: 641129
cache-control: max-age=0
expires: Thu, 09 Nov 2023 19:52:20 GMT
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
content-type: text/csvSuch a header is great because we can make use of
“last-modified” and “etag”.
Without downloading the file we can ask the Webserver if the file has changed there for we make use of the corresponding headers:
“If-Modified-Since”
and
“If-None-Match”
curl -I \
-H "If-Modified-Since: Thu, 09 Nov 2023 19:45:01 GMT" \
-H "If-None-Match: \"9c869-609bd708a103c\"" \
https://something.com/averylargefile.csvIf the file has NOT changed, we will get back:
HTTP/2 304
date: Thu, 09 Nov 2023 19:57:15 GMT
server: Apache
etag: "9c869-609bd708a103c"
expires: Thu, 09 Nov 2023 19:57:15 GMT
cache-control: max-age=0If the file HAS CHANGED, we will get back:
HTTP/2 200
date: Thu, 09 Nov 2023 20:22:04 GMT
server: Apache
strict-transport-security: max-age=31104000
last-modified: Thu, 09 Nov 2023 20:15:01 GMT
etag: "9bb62-609bddbce5d18"
accept-ranges: bytes
content-length: 637794
cache-control: max-age=0
expires: Thu, 09 Nov 2023 20:22:04 GMT
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
content-type: text/csvSimple, isn’t it ? 😉
If “etag” does not exist, you can still use “If-Modified-Since”, but it's getting tricky if that header doesn’t exist as well. You can check the “Content-Length” but that is a very buggy comparison and you should only use it when you are 100% sure that every change is “a bigger one”. If a “1” becomes a “2” the content length won’t change 😉
In case you are in control of a web server where 3rd parties periodically download your files to process them, check your headers,
and check if the others are using “304” — it will save both sides tons of traffic and resources ☝️
Btw… did you know that some web server support modules/scripts and in case you need 100% sure that a file has changed you can send the MD5-Hash of the file in the header. This method costs resources as well, because of generating the HASH, but it’s independent of “last-modified”, “etag”, “content-length” and everything else 😏
I wouldn’t recommend that solution, but sometimes there is no other/better way. I just wanted to let you know. 🤓
