Results 1 to 39 of 39

Thread: wget google

  1. #1
    Join Date
    Apr 2005
    Posts
    694

    wget google

    Anybody here know how to use wget within linux to get this link from google?

    PHP Code:
    http://vp.video.google.com/videodownload?version=0&secureurl=tAAAAMM7Pe6fxq34m-dckO2jE0ctIb3TsxAaQpTAQwfLQ-7B3oZS4CxvKAEtGG_17cKnSyqYfNqD8Ui6r-zB6-70dJInlUEHH-tFd299eiDCC70xBionQvW9m-eILudzhw6bL1y7zpr-fHKL2UoeamkrW7kSG622hBYjUP9pZJEiSyxok237-ffF2uvgR_HSKFf79rCdrhPzqgx8x6iW-rsiire_OwQR14BMLpDK6IpsRJ7Tf3vg_uSh2OpG3jS5AY-rVQ&sigh=Id4VMzKKDSqHvTJ0X7Zjoi5UHNU&begin=0&len=388160&docid=5147852388396123647 
    We can download this directly from iexplorer, but not through:
    PHP Code:
    wget http://vp.video.google.com/videodownload?version=0&secureurl=tAAAAMM7Pe6fxq34m-dckO2jE0ctIb3TsxAaQpTAQwfLQ-7B3oZS4CxvKAEtGG_17cKnSyqYfNqD8Ui6r-zB6-70dJInlUEHH-tFd299eiDCC70xBionQvW9m-eILudzhw6bL1y7zpr-fHKL2UoeamkrW7kSG622hBYjUP9pZJEiSyxok237-ffF2uvgR_HSKFf79rCdrhPzqgx8x6iW-rsiire_OwQR14BMLpDK6IpsRJ7Tf3vg_uSh2OpG3jS5AY-rVQ&sigh=Id4VMzKKDSqHvTJ0X7Zjoi5UHNU&begin=0&len=388160&docid=5147852388396123647 
    in linux...

    Please help if you can...

  2. #2
    Join Date
    Apr 2006
    Location
    NJ, USA
    Posts
    258
    What distro are you running?


  3. #3
    Join Date
    Apr 2005
    Posts
    694
    just Centos 4.3...

  4. #4
    Join Date
    Oct 2004
    Location
    San Francisco, CA
    Posts
    2,454
    Probably better here, since you're using wget and not a programming language persay.
    Tyler Cole
    Eeek, a Blog

  5. #5
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,290
    try this instead

    wget "http://vp.video.google.com/videodownload?version=0&secureurl=tAAAAMM7Pe6fxq34m-dckO2jE0ctIb3TsxAaQpTAQwfLQ-7B3oZS4CxvKAEtGG_17cKnSyqYfNqD8Ui6r-zB6-70dJInlUEHH-tFd299eiDCC70xBionQvW9m-eILudzhw6bL1y7zpr-fHKL2UoeamkrW7kSG622hBYjUP9pZJEiSyxok237-ffF2uvgR_HSKFf79rCdrhPzqgx8x6iW-rsiire_OwQR14BMLpDK6IpsRJ7Tf3vg_uSh2OpG3jS5AY-rVQ&sigh=Id4VMzKKDSqHvTJ0X7Zjoi5UHNU&begin=0&len=388160&docid=5147852388396123647" -O file
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  6. #6
    Join Date
    Apr 2005
    Posts
    694
    Thanks! that works perfect!

  7. #7
    Join Date
    Apr 2005
    Posts
    694
    how would we put that into a php code:

    wget "\"http......\"";

    is that so?

  8. #8
    Join Date
    Nov 2004
    Location
    Australia
    Posts
    1,683
    Quote Originally Posted by ti_nhatrang
    how would we put that into a php code:

    wget "\"http......\"";

    is that so?
    Yes, exactly.

    Something like:

    PHP Code:
    system('wget -O file "http://blahblah.com/url?dsds=3434&hop=7" '); 
    I put the -O file at the start, makes it more obvious. Bear in mind if that crazy URL is time or session based, it will stop working in a while; if that happens, you may have to look at some refinements to make it keep working.

  9. #9
    Join Date
    Apr 2005
    Posts
    694
    I've tried

    PHP Code:
    <?

    exec
    ('wget -P/home/domix/public_html/video http://vp.video.google.com/videodownload?version=0&secureurl=swAAAFxHENxoED_R-3UsXxv5YNc7RQhgjXIn5-LgTF2bOgd2_QIq86fGuHTqXkiw-f2vNa0P0cK_YCRW5sHHgq23NT5Bi2QKjIHzvD5u88RhJu0PEdSdXJhf1p-gWh_qjd1Rb4G02dFuXayqx5J848Vwol70EkxNNdM6hxMnhOd6fTJnoQO0xiBkLok-kFlwWZP2hDddsGr9m8F8rS-nDLz2zmu5xgNNO62_ua9fWH2xI-lEPU2AFp8vtGHruhctqJEf6g&sigh=Z1MX2wPv7qA8Vs07wAZ311L4Xgg&begin=0&len=69866&docid=4526819805867391097 -O $filename.mp4');

    ?>
    And it didn't seem to work... please help?

  10. #10
    Join Date
    Nov 2004
    Location
    Australia
    Posts
    1,683
    No problems, I'll put on my tinfoil telepathy hat and work out what the error message was so I can solve it for you ...

  11. #11
    Join Date
    Apr 2005
    Posts
    694
    Thanks! I appreciate it! I eve try putting the -O infront of the link itself and still doesn't work... Thanks for trying, I will wait for your reply.

  12. #12
    Join Date
    Jul 2006
    Location
    Switzerland
    Posts
    96
    I think you will have to wait for a while then, biranoz is trying to tell you that you need to explain what kind of error messages you are receiving, instead of just stating that i does not work.

    Have you tried putting your URL in quotes? like

    Code:
    exec('wget -P "/home/domix/public_html/video http://vp.video.google.com/videodownload?version=0&secureurl=swAAAFxHENxoED_R-3UsXxv5YNc7RQhgjXIn5-LgTF2bOgd2_QIq86fGuHTqXkiw-f2vNa0P0cK_YCRW5sHHgq23NT5Bi2QKjIHzvD5u88RhJu0PEdSdXJhf1p-gWh_qjd1Rb4G02dFuXayqx5J848Vwol70EkxNNdM6hxMnhOd6fTJnoQO0xiBkLok-kFlwWZP2hDddsGr9m8F8rS-nDLz2zmu5xgNNO62_ua9fWH2xI-lEPU2AFp8vtGHruhctqJEf6g&sigh=Z1MX2wPv7qA8Vs07wAZ311L4Xgg&begin=0&len=69866&docid=4526819805867391097" -O $filename.mp4');
    I am not that familiar with php, but if it has same behaviour as bash the & sign will give you problems without the quotes.

  13. #13
    Join Date
    Nov 2004
    Location
    Australia
    Posts
    1,683
    Yes, I hadn't noticed the lack of quotes in the example - that would probably
    destroy any chance of it working in one hit! (but I'll leave you wondering whether
    I actually own a tinfoil telepathy hat... )

  14. #14
    Join Date
    Apr 2005
    Posts
    694
    Wolle, I've tried yours and still didn't download via php, however, if I do any of these via SSH, then it works...

    I do not get any errors, it just simply does not download.

  15. #15
    Join Date
    May 2004
    Location
    Singapore
    Posts
    370
    You should escape the URL using PHP's escapeshellarg() function.

    Code:
    exec('wget -P /home/domix/public_html/video '.escapeshellarg('http://vp.video.google.com/videodownload?version=0&secureurl=swAAAFxHENxoED_R-3UsXxv5YNc7RQhgjXIn5-LgTF2bOgd2_QIq86fGuHTqXkiw-f2vNa0P0cK_YCRW5sHHgq23NT5Bi2QKjIHzvD5u88RhJu0PEdSdXJhf1p-gWh_qjd1Rb4G02dFuXayqx5J848Vwol70EkxNNdM6hxMnhOd6fTJnoQO0xiBkLok-kFlwWZP2hDddsGr9m8F8rS-nDLz2zmu5xgNNO62_ua9fWH2xI-lEPU2AFp8vtGHruhctqJEf6g&sigh=Z1MX2wPv7qA8Vs07wAZ311L4Xgg&begin=0&len=69866&docid=4526819805867391097').' -O $filename.mp4');
    Make sure $filename has a valid value.

  16. #16
    Join Date
    Apr 2005
    Posts
    694
    that's right, I even remove the $filename and just try it movie.mp4 and it still doesn't download... Have you tried it on your own server? Does it work? If it does, then I guess it's me internally then rather than the script itself...

  17. #17
    Join Date
    May 2004
    Location
    Singapore
    Posts
    370
    If the same command works in shell but not in PHP, then you may want to check that the apache's user has permission to run wget, and apache can find wget (use full path instead).

  18. #18
    Join Date
    Apr 2005
    Posts
    694
    I can download say from url: wget http://www.yahoo.com/file.zip fine... just that the long crazy url doesn't work... also, if I use tinyurl.com to shorten the url, then it works fine as well with any the quotations marks etc... I'm just hoping there's another solution rathern than going through tinyurl.com....

  19. #19
    Join Date
    May 2004
    Location
    Singapore
    Posts
    370
    There is no reason wget will fail with long URLs.

    Have you tried the code I suggested? I tried it on my own server and it works.

  20. #20
    Join Date
    Apr 2005
    Posts
    694
    I certainly have, infact this is my exact php code:

    PHP Code:
    <?

    exec
    ('wget -P /home/domix/public_html '.escapeshellarg('http://vp.video.google.com/videodownload?version=0&secureurl=swAAAFxHENxoED_R-3UsXxv5YNc7RQhgjXIn5-LgTF2bOgd2_QIq86fGuHTqXkiw-f2vNa0P0cK_YCRW5sHHgq23NT5Bi2QKjIHzvD5u88RhJu0PEdSdXJhf1p-gWh_qjd1Rb4G02dFuXayqx5J848Vwol70EkxNNdM6hxMnhOd6fTJnoQO0xiBkLok-kFlwWZP2hDddsGr9m8F8rS-nDLz2zmu5xgNNO62_ua9fWH2xI-lEPU2AFp8vtGHruhctqJEf6g&sigh=Z1MX2wPv7qA8Vs07wAZ311L4Xgg&begin=0&len=69866&docid=4526819805867391097').' -O filename.mp4');


    ?>
    no go...

  21. #21
    Join Date
    Apr 2005
    Posts
    694
    can anybody else help me with this? Thanks in advance!

  22. #22
    Join Date
    Nov 2004
    Location
    Australia
    Posts
    1,683
    Try my version, with the double quotes inside the single quotes, does that work?

    It's almost certainly a quoting problem biting you here.

  23. #23
    Join Date
    Apr 2005
    Posts
    694
    I definately have for the past 2 days, and it still does not download... Have you tried it yourself? Are you getting any errors or what kind of success are you getting from that particular link?

  24. #24
    Join Date
    Apr 2005
    Posts
    694
    I mean, I obviously do not need to use wget if you have another solution, anything to download the file is fine...

  25. #25
    Join Date
    Nov 2004
    Location
    Australia
    Posts
    1,683
    I think the problem is the ridiculous URL. Maybe you could use tinyurl.com to solve that problem.

    ... ah ... tested and works beautifully.

    PHP Code:
    system("wget -O file.mp4 http://tinyurl.com/z66ls"); 
    In any case, it turns out that's not the actual problem - the problem is that you put a -P option in there which breaks the -O option. Perhaps be a little more careful next time around, don't just go sticking extra options in without understanding what they do.

  26. #26
    Join Date
    Apr 2005
    Posts
    694
    the P option is for me to declare which folder I want the file to be downloaded into... is there a way around this?

  27. #27
    Join Date
    Apr 2005
    Posts
    694
    I used your code without the P option and it still doesn't download...

  28. #28
    Join Date
    Nov 2004
    Location
    Australia
    Posts
    1,683
    It works perfectly here, there's something else wrong. The most likely, is wget in your path and executable?

    Try [PHP]passthru("wget -O file.mp4 http://tinyurl.com/z66ls 1>&2"); [/PHPwhich is equivalent to system but will make error messages go through to your browser.

    You've misunderstood the P option, you don't need it, it conflicts with the -O option which is already
    determining where the output goes.

  29. #29
    Join Date
    Apr 2005
    Posts
    694
    I do not think anything is wrong with my wget, I can download fine with tinyurl, but not the long url from google....

  30. #30
    Join Date
    Apr 2005
    Posts
    694
    this is what I have now per your recommendation:

    PHP Code:
    <?

    passthru
    ("wget -O file.mp4 http://vp.video.google.com/videodownload?version=0&secureurl=swAAAFxHENxoED_R-3UsXxv5YNc7RQhgjXIn5-LgTF2bOgd2_QIq86fGuHTqXkiw-f2vNa0P0cK_YCRW5sHHgq23NT5Bi2QKjIHzvD5u88RhJu0PEdSdXJhf1p-gWh_qjd1Rb4G02dFuXayqx5J848Vwol70EkxNNdM6hxMnhOd6fTJnoQO0xiBkLok-kFlwWZP2hDddsGr9m8F8rS-nDLz2zmu5xgNNO62_ua9fWH2xI-lEPU2AFp8vtGHruhctqJEf6g&sigh=Z1MX2wPv7qA8Vs07wAZ311L4Xgg&begin=0&len=69866&docid=4526819805867391097 1>&2");

    ?>
    let me know if that's even working for you on your server while executing from a php file...

  31. #31
    Join Date
    Nov 2004
    Location
    Australia
    Posts
    1,683
    I couldn't get the long URL to work, and suspect it's something to do with quoting, whatever, since the short URL works, I'm giving up on the long one.

    If you try out various means of quoting you may be able to solve the problem. For instance, replace wget with an executable shell script that does "echo $*" and check that it displays the correct URL (do a character-by-character comparison against the working one). Then try varying the quoting until the shell script is echoing the same URL you are feeding it, then replace the shell script with wget and it will all start working. Or you could just fiddle with quotes until you get it to work!

    The escapechell() code above might work if you try it without the -P in the mix.

  32. #32
    Join Date
    Apr 2005
    Posts
    694
    I understand, I take every replies here constructively and tried it in many ways, with/without the -P etc. Still doesn't like it for some odd reason... I'm hoping for a better solution rather than having to use tinyurl...

    this is what I'm onto next which also does not work...

    PHP Code:
    <?
    $url 
    "http://vp.video.google.com/videodownload?version=0&secureurl=swAAAFxHENxoED_R-3UsXxv5YNc7RQhgjXIn5-LgTF2bOgd2_QIq86fGuHTqXkiw-f2vNa0P0cK_YCRW5sHHgq23NT5Bi2QKjIHzvD5u88RhJu0PEdSdXJhf1p-gWh_qjd1Rb4G02dFuXayqx5J848Vwol70EkxNNdM6hxMnhOd6fTJnoQO0xiBkLok-kFlwWZP2hDddsGr9m8F8rS-nDLz2zmu5xgNNO62_ua9fWH2xI-lEPU2AFp8vtGHruhctqJEf6g&sigh=Z1MX2wPv7qA8Vs07wAZ311L4Xgg&begin=0&len=69866&docid=4526819805867391097";

    exec("/usr/bin/wget -P/home/domix/public_html/testxml/a $url -O file.mp4");

    ?>

  33. #33
    Join Date
    Apr 2005
    Posts
    694
    if anyone got another solution or idea of grabing files via php instead of wget, please share it with us/me... thanks!

  34. #34
    You could use:
    http://uk.php.net/manual/en/function...t-contents.php

    and: http://uk.php.net/manual/en/function.fwrite.php

    Then write the contents of the file to a new file using fwrite.

  35. #35
    Join Date
    Apr 2005
    Posts
    694
    Thanks! But can you show me some examples using the links I've given above?

  36. #36
    Join Date
    Nov 2004
    Location
    Australia
    Posts
    1,683
    Quote Originally Posted by ti_nhatrang
    PHP Code:
    <?
    $url 
    "http://vp.video.google.com/videodownload?version=0&secureurl=swAAAFxHENxoED_R-3UsXxv5YNc7RQhgjXIn5-LgTF2bOgd2_QIq86fGuHTqXkiw-f2vNa0P0cK_YCRW5sHHgq23NT5Bi2QKjIHzvD5u88RhJu0PEdSdXJhf1p-gWh_qjd1Rb4G02dFuXayqx5J848Vwol70EkxNNdM6hxMnhOd6fTJnoQO0xiBkLok-kFlwWZP2hDddsGr9m8F8rS-nDLz2zmu5xgNNO62_ua9fWH2xI-lEPU2AFp8vtGHruhctqJEf6g&sigh=Z1MX2wPv7qA8Vs07wAZ311L4Xgg&begin=0&len=69866&docid=4526819805867391097";

    exec("/usr/bin/wget -P/home/domix/public_html/testxml/a $url -O file.mp4");

    ?>
    As mentioned, you don't need the -P if you have the -O. The two options conflict, and the -P may actually prevent it working altogether.

    I did show you some working code!!!

    Did you try the example with escapeshell()?

    You could also try:

    PHP Code:
    exec("/usr/bin/wget '$url' -O file.mp4"); 

  37. #37
    Join Date
    Nov 2004
    Location
    Australia
    Posts
    1,683
    This code works, definitely, both attempts downloaded a 1720981 byte file ...
    PHP Code:
    <?php

    error_reporting
    (E_ALL);

    echo 
    "<pre>\n";

    $filename "/tmp/fileoutaa";
    system("rm -f ${filename}*");

    echo 
    "downloading  attempt 1 ...\n";
    flush();

    $url 'http://vp.video.google.com/videodownload?version=0&secureurl=swAAAFxHENxoED_R-3UsXxv5YNc7RQhgjXIn5-LgTF2bOgd2_QIq86fGuHTqXkiw-f2vNa0P0cK_YCRW5sHHgq23NT5Bi2QKjIHzvD5u88RhJu0PEdSdXJhf1p-gWh_qjd1Rb4G02dFuXayqx5J848Vwol70EkxNNdM6hxMnhOd6fTJnoQO0xiBkLok-kFlwWZP2hDddsGr9m8F8rS-nDLz2zmu5xgNNO62_ua9fWH2xI-lEPU2AFp8vtGHruhctqJEf6g&sigh=Z1MX2wPv7qA8Vs07wAZ311L4Xgg&begin=0&len=69866&docid=4526819805867391097';

    passthru("wget  -O ${filename}-1.mp4 '$url' 2>&1");

    echo 
    "\n\n\n";

    echo 
    "downloading  attempt 2 ...\n";

    passthru("wget -O ${filename}-2.mp4 'http://vp.video.google.com/videodownload?version=0&secureurl=swAAAFxHENxoED_R-3UsXxv5YNc7RQhgjXIn5-LgTF2bOgd2_QIq86fGuHTqXkiw-f2vNa0P0cK_YCRW5sHHgq23NT5Bi2QKjIHzvD5u88RhJu0PEdSdXJhf1p-gWh_qjd1Rb4G02dFuXayqx5J848Vwol70EkxNNdM6hxMnhOd6fTJnoQO0xiBkLok-kFlwWZP2hDddsGr9m8F8rS-nDLz2zmu5xgNNO62_ua9fWH2xI-lEPU2AFp8vtGHruhctqJEf6g&sigh=Z1MX2wPv7qA8Vs07wAZ311L4Xgg&begin=0&len=69866&docid=4526819805867391097' 2>&1");

    flush();

    echo 
    "\n\ndone downloading ...\n";


    passthru("ls -l {$filename}* 2>&1");

    ?>
    The use of the quotes is critical. Single quotes inside double quotes lose their meaning for PHP, but they protect the URL from expansion by the shell
    that would then break the URL so it didn't work. In particular, & and ? are special characters for the shell.

    You might have to use a full path for wget on your server to make it work, but it works on mine!

  38. #38
    Join Date
    Apr 2005
    Posts
    694
    Yes it does! You ARE AMAZING! seems like it downloading twice, can we not do this or do we have to include it in there just so the first time it didn't get downloaded?

  39. #39
    Join Date
    Nov 2004
    Location
    Australia
    Posts
    1,683
    Thanks, I'm not sure about amazing, but I do stun myself in the mirror from now and then.

    It's downloading twice because two wget's are being called. I'd kill off the second one, with the long URL, and keep the one with the $url in it.

    Cheers,

    Brian

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •