Logs and Google. - geek/talk: Signature-free discourse for serious web publishers

Nintendo · 02-03-2003, 06:12 PM

Using logs, is there some way to tell if Google is doing a fresh couple day update, or doing a DeepCrawl that will show up when it dances?

iisbum · 02-03-2003, 07:38 PM

You should be able to look for the User Agent String in your logs to see how many pages googlebot requested.

I have a script on a low traffic page that emails me when the User Agent String matches googlebot's. Google visits most of my sites about once a day.

Nintendo · 02-03-2003, 08:28 PM

Every time the Google bot comes, it shows up like this.

64.68.82.18 - - [01/Feb/2003:13:19:31 -0800]
"GET /books/greatest_speeches_john_f.shtml
HTTP/1.0" 200 2592 "-" "Googlebot/2.1
(+http://www.googlebot.com/bot.html)"

Is there a way to tell if it's looking at your site to do a Fresh update, or doing a deep crawl? I've hurd that if the IP address startes with 64 then it's just doing a fresh update, but that's the only number that I ever see it start with.

BTW, welcome to geek/talk, you learn a lot here.

iisbum · 02-03-2003, 08:47 PM

I saw this over at webmasterworld:

freshBot:
64.68.82.*
deepcrawler:
216.239.46.*

Is Googlebot Deep Crawling?

Thanks for the welcome, I've been lurking for a long time, learn't lots already!

02-03-2003, 06:12 PM	#1
Nintendo Join Date: Feb 2001 Location: Planet Zeekois Posts: 1,289	Logs and Google. Using logs, is there some way to tell if Google is doing a fresh couple day update, or doing a DeepCrawl that will show up when it dances? __________________ .

02-03-2003, 08:28 PM	#3
Nintendo Join Date: Feb 2001 Location: Planet Zeekois Posts: 1,289	Every time the Google bot comes, it shows up like this. 64.68.82.18 - - [01/Feb/2003:13:19:31 -0800] "GET /books/greatest_speeches_john_f.shtml HTTP/1.0" 200 2592 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)" Is there a way to tell if it's looking at your site to do a Fresh update, or doing a deep crawl? I've hurd that if the IP address startes with 64 then it's just doing a fresh update, but that's the only number that I ever see it start with. BTW, welcome to geek/talk, you learn a lot here. __________________ .

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Google Adsense	fcmedianet	Making Money with CPC and/or CPM Programs	7	02-24-2007 06:04 AM
Google AdSense Account Disabled	Sphosting	Making Money with CPC and/or CPM Programs	19	08-29-2005 01:43 PM
Dealing with internation traffic	Xisto	Making Money with Regional Affiliate Programs and Networks	5	04-18-2005 08:46 AM
Google Adwords Discrepency	thegauntlet	Marketing Your Website Online	6	12-21-2004 06:37 PM
Good day with google	Ted S	Search Engine Marketing & Optimization	2	12-30-2001 01:18 PM

02-03-2003, 07:38 PM	#2
iisbum Registered User Join Date: Aug 2002 Posts: 60	You should be able to look for the User Agent String in your logs to see how many pages googlebot requested. I have a script on a low traffic page that emails me when the User Agent String matches googlebot's. Google visits most of my sites about once a day.

02-03-2003, 08:47 PM	#4
iisbum Registered User Join Date: Aug 2002 Posts: 60	I saw this over at webmasterworld: freshBot: 64.68.82.* deepcrawler: 216.239.46.* Is Googlebot Deep Crawling? Thanks for the welcome, I've been lurking for a long time, learn't lots already!