Super Web Mart - Discount Coupons & Deals - Discount Promos, Coupon Codes, Discount deals and web site reviews
 

Author Topic: Whats the largest page size that Googles spider will crawl?  (Read 517 times)

0 Members and 1 Guest are viewing this topic.

Offline Randy

  • Jr. Member
  • **
  • Posts: 98
    • View Profile
Whats the largest page size that Googles spider will crawl?
« on: April 06, 2018, 05:34:50 AM »
Hello folks,

Whats the largest page size that Googles spider will crawl?

Webmaster Forum

Whats the largest page size that Googles spider will crawl?
« on: April 06, 2018, 05:34:50 AM »

Offline Orlova

  • Jr. Member
  • **
  • Posts: 33
    • View Profile
Re: Whats the largest page size that Googles spider will crawl?
« Reply #1 on: August 01, 2019, 10:28:55 PM »
Google crawls websites of any size. However, very large sites tend to not be crawled all at once. It's kind of like eating a large meal. You have some now, and save some for later. Google indexes parts of large sites during it's routine spidering process. Then they'll return later on to index the rest.

Offline JustinC

  • Jr. Member
  • **
  • Posts: 30
    • View Profile
Re: Whats the largest page size that Googles spider will crawl?
« Reply #2 on: August 01, 2020, 05:39:03 AM »
According to Google's Documentation, files which are larger than 30MB will not be crawled. The crawlers will index up to 2.5MB of an HTML file.
This is it.
I believe that Google will increase the file size in coming few years.

Webmaster Forum

Re: Whats the largest page size that Googles spider will crawl?
« Reply #2 on: August 01, 2020, 05:39:03 AM »