Go Back  HTML Forums - Free Webmaster Forums and Help Forums > PROMOTION & COMMERCE > Search Engine Tactics & Optimization
Reload this Page prevent spider grabbing URLs....
User Name:
Password:
 

Reply
Thread Tools   Display Modes
  View First Unread
 
Old 05-06-2008, 06:53 AM
  #1
nommer
Aspirant (Level 2)
 
nommer's Avatar
 
Join Date: Oct 2007
Posts: 18
iTrader: (0)
nommer is an unknown quantity at this point
prevent spider grabbing URLs....
Is there any way to prevent a spider from grabbing URLs that you want to keep off search engines?

thanks......
_________________
indoor fountain florida refinance
nommer is offline   Add to del.icio.us Add to del.icio.us    Can you digg it?Can you digg it? Reply With Quote
Old 05-06-2008, 07:18 AM
  #2
Horus_Kol
Mod of the Underlay
 
Horus_Kol's Avatar
 
Join Date: Jun 2002
Location: At a desk, hooked up and ready to rock
Posts: 15,505
iTrader: (0)
Horus_Kol has a spectacular aura aboutHorus_Kol has a spectacular aura aboutHorus_Kol has a spectacular aura about
for, at least, the reputable ones - robots.txt or the nofollow attribute...

or, more reliable, - don't link to the page anywhere else... can't follow what isn't there...

or, put the content behind a password protected area...
__________________


So sue me for being pedantic... I can give you a bullet-list of reasons why you can't
Horus_Kol is offline   Add to del.icio.us Add to del.icio.us    Can you digg it?Can you digg it? Reply With Quote
Old 05-07-2008, 11:56 PM
  #3
Jason
1.21 Gigawatts
 
Jason's Avatar
 
Join Date: Nov 1999
Location: Hill Valley
Posts: 4,620
iTrader: (0)
Jason will become famous soon enoughJason will become famous soon enough
If there is a specific spider that is causing annoyance, you could also always block it using iptables.
__________________


BoxedArt.com Web Templates
Your complete web developer resource
Jason is offline   Add to del.icio.us Add to del.icio.us    Can you digg it?Can you digg it? Reply With Quote
Old 05-12-2008, 06:21 PM
  #4
VIROKILL
Champion (Level 13)
 
VIROKILL's Avatar
 
Join Date: Mar 2006
Posts: 226
iTrader: (0)
VIROKILL is an unknown quantity at this point
i agree with robots.txt . just dissalow those pages that you dont want on search engines
__________________
dont click this
VIROKILL is offline   Add to del.icio.us Add to del.icio.us    Can you digg it?Can you digg it? Reply With Quote
Reply
« Previous Thread | Next Thread »


 
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
  
  
 
 



 
  POSTING RULES
 
 
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Thread Tools
Display Modes
Linear Mode Linear Mode

Forum Jump

 

All times are GMT -5. The time now is 04:17 PM.
   
Network Sites: Web Tools and Resources | Web Templates | Free Webmaster Resources | Free JavaScripts | Free Clip Art | Clipart Resources |
Turnkey Website Templates | Free Fonts | HTML Help | Stock Photos | Quality Hosting | Learn Perl and CGI | Top Web Hosts | CMS Templates

Partner Sites: Free Web Templates | Webmaster Resources | Javascripts | Website Reviews | DHTML Scripts | Create a free forum | Web Hosting Reviews |Free Ecommerce Web Hosting

Mascot team created by Drawshop.com

Powered by vBulletin® Version 3.6.7
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.

Server Monitoring by ENIACmonitor 0.01
HTMLforums.com © Big Resources, Inc. Web Design by BoxedArt.com
vRewrite 1.5 beta SEOed URLs completed by Tech Help Forum and Chalo Na.