Crawl a site
Web Server forum
Back To The Forum Home!Search!Private Messaging System

Web Server Talk Web Server Talk > Web Servers reviews > IIS server support > IIS ASP > Crawl a site




  Last Thread   Next Thread Next
  Show Printable Version Email this Page Subscribe to this Thread      Post New Thread    Post A Reply      

    Crawl a site  
Dave


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
06-26-04 03:18 PM

Is it possible to crawl a site using ASP & XML HTTP?

I know you can hit one link, but how can you go through each link in a
page and validate that it returns 200?





[ Post a follow-up to this message ]



    Re: Crawl a site  
Eric Gibson


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
06-28-04 01:55 PM

Dave wrote:
> Is it possible to crawl a site using ASP & XML HTTP?
>
> I know you can hit one link, but how can you go through each link in a
> page and validate that it returns 200?

Using PerlScript and the Robot module.

http://search.cpan.org/~awrigley/WW...ib/WWW/Robot.pm

Eric







[ Post a follow-up to this message ]



    Sponsored Links  




 





   All times are GMT. The time now is 08:45 AM.      Post New Thread    Post A Reply      
  Last Thread   Next Thread Next


Most Popular forums 

Forum Jump:
Rate This Thread:

Forum Rules:
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is OFF
vB code is ON
Smilies are ON
[IMG] code is OFF
 
Medical and Health forum | Computer Games Reviews | Graphics design forum

Back To The Top
Home | Usercp | Faq | Register