Google has added a brand new part to the crawl price range administration assist doc. The brand new part is for the If-Modified-Since request header. The part is titled “Specify content material adjustments with HTTP standing codes.”

Google wrote, “Google typically helps the If-Modified-Since and If-None-Match HTTP request headers for crawling. Google’s crawlers do not ship the headers with all crawl makes an attempt; it is determined by the use case of the request (for instance, AdsBot is extra prone to set the If-Modified-Since and If-None-Match HTTP request headers).”

“If our crawlers ship the If-Modified-Since header, the header’s worth is the date and time the content material was final crawled. Primarily based on that worth, the server could select to return a 304 (Not Modified) HTTP standing code with no response physique, wherein case Google will reuse the content material model it crawled the final time. If the content material is newer than the date specified by the crawler within the If-Modified-Since header, the server can return a 200 (OK) HTTP standing code with the response physique.”

“Independently of the request headers, you’ll be able to ship a 304 (Not Modified) HTTP standing code and no response physique for any Googlebot request if the content material hasn’t modified since Googlebot final visited the URL. This may save your server processing time and sources, which can not directly enhance crawl effectivity.”

I’m not certain if I’ve any commentary on this, exterior of the truth that this new part was added to this web page.

Discussion board dialogue at Twitter.