PDA

View Full Version : Google freshbot...


jebster
08-03-2003, 03:36 AM
Does it have a different UserAgent then googles deep crawler? Cause I got a hit on August 03rd, 2003 07:56.01 AM(Not sure what timezone that is, not mine seeing how it is only 2:00am here...) by "Googlebot/2.1 (+http://www.googlebot.com/bot.html)" with the IP "64.68.82.168" and was wondering if this meant google is doing its monthly deep crawl or if it is just the freshbot?!

bobat
08-03-2003, 06:57 AM
i think google uses its freshbot once a month, and deep crawl every three months.

creative666
08-03-2003, 10:30 AM
Nearly bobat, the Google deepbot is what is used to collect the data for the monthly crawl. The Google freshbot is used nearly all the time to collect any new pages or sites that are added in between the big monthly update.

jebster
08-03-2003, 10:50 AM
So that was the freshbot? Just installed a new stats system that actually logs when search engines spider my site(Unlike sitemeter which I did use) so its all kinda new to me! :D

bobat
08-06-2003, 06:07 PM
is it still active or has it stopped now?

creative666
08-07-2003, 02:53 AM
The freshbot has been acting alot more like the deepcrawler recently, but it is still out there... I had a fresh date on 3 of my sites over the weekend!