To What Depth Will Search Engine Spiders Go? - geek/talk: Signature-free discourse for serious web publishers

donds87 · 04-10-2003, 06:18 PM

Actually, the spiders will go to any depth of a site. However, they
don't place as much emphasis on pages within the site that take more than
a couple of clicks to get to from the main page.

http://www.searchengineguide.com/wha.../0410_jw1.html

Steve_S · 04-10-2003, 07:33 PM

She is a very well known writer and I like her stuff very much, so naturally I also take issue with her since websites are not just about SEO.

Your menu structure should always be global in nature (every page/every link/or a a logical grouping) and I'm not just referring to the home page link. This is all about usability. This normally requires 2 "grids"

It's still a darn good article and perhaps unfair of me <gasp> to pick a little hole in it so please excuse me

BUT money making/sucessfull sites need a reasonable balance between usability and SEO and G also rewards you for this

She sites a 4 level deep example of:

yoursite.com/dir1/dir2/dir3/dir4/deep-page.htm

Which is fine and dandy. No problem. With good usability its not rare to get top 3 ranking or top 10 with:

yoursite.com/dir1/dir2/ducky_wucky.html

And you?

hobbnet · 04-12-2003, 04:14 PM

I'm not taking the time to read the article but as far as GoogleBot goes PR also affects how deep Google will spider your website's pages. Of course if you have a shallow site it won't matter what your PR is.

hobbnet · 04-12-2003, 04:19 PM

Ahh, and after reading Steve's post I wanted to add this:

Typically, I'd have to say having pages in sub folders is a bad idea for optimization. PR goes down (typically) when googlebot has to drill down sub folders to get to the content. Take notice next time you go to a site with a lot of subfolders...and subfolders of those subfolders. You will most definitely see that as you go into a new subfolder PR will drop by one for every level you go down.

So for a PR 7 site www.example_domain.com

you would get:

PR 6: www.example_domain.com/animals
PR 5: www.example_domain.com/animals/mammals
PR 4: www.example_domain.com/animals/mammals/gorillas

This is generally true, there will always be exceptions.

So for this reason, when I design sites now I have all my pages in the root directory.

04-10-2003, 06:18 PM	#1
donds87 Registered User Join Date: Mar 2003 Posts: 27	To What Depth Will Search Engine Spiders Go? Actually, the spiders will go to any depth of a site. However, they don't place as much emphasis on pages within the site that take more than a couple of clicks to get to from the main page. http://www.searchengineguide.com/wha.../0410_jw1.html

04-10-2003, 07:33 PM	#2
Steve_S Join Date: Aug 1998 Location: Las Vegas, Nevada, USA Posts: 5,223	She is a very well known writer and I like her stuff very much, so naturally I also take issue with her since websites are not just about SEO. Your menu structure should always be global in nature (every page/every link/or a a logical grouping) and I'm not just referring to the home page link. This is all about usability. This normally requires 2 "grids" It's still a darn good article and perhaps unfair of me <gasp> to pick a little hole in it so please excuse me BUT money making/sucessfull sites need a reasonable balance between usability and SEO and G also rewards you for this She sites a 4 level deep example of: yoursite.com/dir1/dir2/dir3/dir4/deep-page.htm Which is fine and dandy. No problem. With good usability its not rare to get top 3 ranking or top 10 with: yoursite.com/dir1/dir2/ducky_wucky.html And you? Last edited by Steve_S; 04-10-2003 at 07:40 PM.

04-12-2003, 04:19 PM	#4
hobbnet Registered User Join Date: Sep 1999 Location: San Francisco, CA Posts: 627	Ahh, and after reading Steve's post I wanted to add this: Typically, I'd have to say having pages in sub folders is a bad idea for optimization. PR goes down (typically) when googlebot has to drill down sub folders to get to the content. Take notice next time you go to a site with a lot of subfolders...and subfolders of those subfolders. You will most definitely see that as you go into a new subfolder PR will drop by one for every level you go down. So for a PR 7 site www.example_domain.com you would get: PR 6: www.example_domain.com/animals PR 5: www.example_domain.com/animals/mammals PR 4: www.example_domain.com/animals/mammals/gorillas This is generally true, there will always be exceptions. So for this reason, when I design sites now I have all my pages in the root directory. Last edited by hobbnet; 04-12-2003 at 04:20 PM.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Post subject: Increasing Your Search Engine Ranking	montyauto	Search Engine Marketing & Optimization	2	07-16-2008 11:00 AM
www.Googer.net -- PPC Search Engine Website for sale	coosee	Website For Sale - Purchase - Barter - Trade	4	07-11-2008 01:30 AM
Monitoring Search Engine Positions	montyauto	Search Engine Marketing & Optimization	4	05-29-2008 02:12 AM
Page Rank search engine	Larwee	Search Engine Marketing & Optimization	0	12-31-2005 09:43 PM
Question on new search engine	krink1	Marketing Your Website Online	2	10-25-2001 04:23 PM

04-12-2003, 04:14 PM	#3
hobbnet Registered User Join Date: Sep 1999 Location: San Francisco, CA Posts: 627	I'm not taking the time to read the article but as far as GoogleBot goes PR also affects how deep Google will spider your website's pages. Of course if you have a shallow site it won't matter what your PR is.