Results 1 to 2 of 2
  1. #1
    Join Date
    Apr 2005
    Posts
    194

    * Visit Pages Auto with Time

    Hi,

    I have arround 1,000 URL and want it will visit once daily with some timer interval.

    Actualy when the page visit it will come in cache of my news applicaiton. If I visit pages manual it will be ok but daily i can not visit arround 1,000 pages manualy.

    I have some of pages like below which coming from ohter website.

    For example:
    My Domain is: abc.com
    Page want to visit from: xyz.com

    Now I have link like below:
    http://www.xyz.com/news.php?1
    http://www.xyz.com/news.php?2
    http://www.xyz.com/news.php?3
    http://www.xyz.com/news.php?4
    http://www.xyz.com/news.php?5


    Any Idea how to do this using any small script or corn job etc..?


    Thanks,
    Raza

  2. #2
    If you have cURL enabled you can use this, if not change it so it uses php's fopen or fread!

    PHP Code:
    <?
    function curl($url){
    $ch curl_init();
    curl_setopt($chCURLOPT_URL$url);
    curl_setopt($chCURLOPT_HEADER1);
    curl_setopt($chCURLOPT_RETURNTRANSFER1);
    curl_setopt($chCURLOPT_USERAGENT"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0");
    $all str_replace(array("\n","\r","\t","\r\n"), "",unhtmlentities(curl_exec($ch)));
    curl_close($ch);
    return 
    $all;
    }

    function 
    unhtmlentities($string){
    $trans_tbl get_html_translation_table(HTML_ENTITIES);
    $trans_tbl array_flip($trans_tbl);
    return 
    strtr($string$trans_tbl);
    }

    for (
    $i=1$i1000$i++) {
    $html=curl("http://xyz.com/page.php?$i");
    echo 
    "Page $i has been crawled\r\n" 
    }
    ?>
    <<Please see rules for signature setup>>

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •