Google has additionally up to date the crawl stats report assist documentation final week. Google made plenty of assist documentation modifications prior to now week together with product wealthy outcomes, Googlebot doc, job posting assist doc, transferring the HTTP standing codes doc and in addition this crawl stats report assist doc.

You possibly can see this crawl stats assist doc right here and the modifications had been summed up very properly by Brodie Clark on Twitter who stated:

• Notice about robots.txt 404ing (can crawl any URL)
• Utilizing a 12hr – 30 day timeframe for fetching
• Particulars about homepage utilization

He posted the doc earlier than and after modifications as effectively:

Brodie wrote, “Heads-up: Google simply added some fascinating clarifications to their Crawl Stats Report doc.” Good catch Brodie!

Discussion board dialogue at Twitter.