Preparing ASP pages for spidering

Results 1 to 2 of 2

Thread: Preparing ASP pages for spidering

  1. #1
    Join Date
    Dec 1969

    Default Preparing ASP pages for spidering

    I have just spent several months building a nonprofit web site,, and on Friday we went live. (hooray!) <BR><BR>Today, I began the process of registering the site in search engines and discovered something that is very disappointing. Forgive me if this is common knowledge. Many spiders don&#039;t index .asp pages. <BR><BR>As an example, I got the following message from AltaVista: <BR><BR>"Note that AltaVista does not accept .asp pages, as the spider <BR>will not be able to crawl through and index the site." <BR><BR>Pretty disappointing, considering that 99% of the pages in my site are .asp. <BR><BR>I have just spent about an hour combing through the FAQs and posts at 4Guys, CNET, ZDNet, and found surprisingly little written about this issue. I certainly did not find an "magic bullet" for dealing with the situation. <BR><BR>Is there a magic bullet? <BR><BR>I apologize if this is a very newbie question and I&#039;m just missing it. <BR><BR>BTW. I have tried to think of a couple of work around solutions. For example: creating "index.htm" pages in my root and all main (i.e., section) directories which refreshes immediately to the corresponding "index.asp" pages. This might work for some spiders. But I got a message back from one free, online "meta tag analyzer" indicating that &#060;refresh&#062; tags are "not spider friendly"--one can understand why--no matter what time duration is set. <BR><BR>An alternative solution might be to create .htm "splash pages," but what a pain in the behind for my web visitors, huh? I don&#039;t think I want to require them always to click TWICE, instead of just once, every time they try to link to one of my .asp pages from a search engine. <BR><BR>Anyway... <BR><BR>Any ideas or references surely would be appreciated.

  2. #2
    SPG Guest

    Default 2 Options

    1: If you control the server, rename all of your .asp pages to .htm and tell IIS that it should compile *.htm as *.asp. That will catch the less observant of the spiders.<BR><BR>2: Instead of using &#060;meta&#062; tags, use &#060;script&#062; to redirect. I wouldn&#039;t expect that most spiders compile JavaScript to help them decide to *not* index a page, so it would probably work. (Of course, add a alternate link for those with JavaScript disabled.)<BR><BR>HiH<BR>SPG

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts