Hello,

I am currently developing a site for a store. The problem is that they have 20,000 products and the manufacter's website shows all of them on an asp page.

They expect you to click on each one and do a save as for it. 2 pictures per product (Thumbnail and full) This would be 40,000 clicks and would be a pain in the but to do!

This is an ASP page with dynamic and password protected links so getright and flashget and all those do not work correctly.

Any ideas?

Thanks

Contact the manufacturer and request access or a disk? If it's helping to sell their products, they might allow this..

Already have, they won't since they say the images are in some kind of asp database along with thoudands of other files. They are really not helpful at all :-(

Is there some kind of script maybe a php or cgi that will go there and download it onto my server? A program that will run locally on my own machine and do this?

Anyone?

Thanks,

If you know the url for the images and if the images are in sequential orders (meaning that the name of the images have number sequence to them to show the order like image1, image2 etc..) then you can use a for loop to grab all the images using PHP easily. Check out the file functions in PHP. I hope I was any help.

Hey,

I just tried it and I am getting a permission denied. I think it is because the remote site needs a username and password at the main page, and then it sets a cookie for the remaining browsing session. Is there any way i can get my php script to parse the form (fill in the u and p) get the cookie and then start copying?

Do you knwo what functions would work?

Thanks,

Brian

Copy the complete cookie off your PC into a PHP variable. Before you request an image, put:
header("Cookie: $cookiedata\r\n");

This wont work exactly, as I dont really have time to elaborate...but it gives you an idea of what you need to do

Yea, basically you need to fake the cookie. Problem I am thinking is knowing what kind of cookie they save. Example....they might be using session_id's which I am not sure how to fake. Or they could be saving the username/pass in the cookie and are just checking it against the database.

Again, I am not really sure. Search Google for some help.

Just thought of this....


Is it a browser authentication window? Meaning does it popup and look like the login for a cpanel server?

If so, do this http://USERNAMEHERE:PASSHERE@domain.com/directory/here

Then run that through your GetRight or local program and see if it works.

Hello,

I am not sure what cpanel looks like (Plesk fan here) but anyway it is not http authorize, meaning that the login screen is a form that you click submit, then it looks like some type of cookie is getting passed to the computer.

Thanks,

Brian

Hmmmm....The problem is knowing if its using SID's or just checking against the DB with the Cookie.

Check the cookie, see if its storing a username or an ID...then again, they might be using the username as the SID which is possible too :(

Heres the cookie:
SITESERVER
ID=70a037fa67ecb7ff05de126b95df33a7
smcorp.com/
1536
642859008
31887777
1114504608
29543552
*

None of that "junk" is my username or password, so I guess it is a session id?

Would it be possible to log on with my pc copy the session id, and then have the script use the ID from our web server to copy the image?

I ahve the php script set up all perfect it's just the cookies that are screwing it up!

Thanks,

Brian

:( From what I know, you can't fake a session ID. They are pretty much random things that change when you exit and start a new one.


Logout and login to see what I am talking about. Now compare the text in this cookie to the text in the cookie that you showed me.

I think you might be able to do this though:

If you login to the site...then run your local image extractor program, your session Id's should match and there shouldn't be an authentication problem.

Give it a shot.

sessions sort of...dissolve off, so login, then immediatly fire off the php script using that. it should work as the sessions is still 'alive'

Hello,

If I want to do this live on the webserver since it is a php script and i don't have any server stuff on my localhost....can I use lynx to sign in and browse and then run the php script?

Thanks,

Brian

Originally posted by awsolutions
Hello,

If I want to do this live on the webserver since it is a php script and i don't have any server stuff on my localhost....can I use lynx to sign in and browse and then run the php script?

Thanks,

Brian

Yes, try it.