I need to insert about 30,000,000 rows data to the table from old table (DISTINCT data required).

It got error after I ran it about 5 minutes.
How can I fix it ?
Do I need to change some setting in mysql on the dedicate server?

Thanks



// here are the code I am running.

$check_repeat_keyword = mysql_query ("SELECT DISTINCT keyword FROM $get_table GROUP BY keyword ") or die("insert data failed : " . mysql_error());

while ($do = mysql_fetch_array($check_repeat_keyword, MYSQL_ASSOC))

{

$input_keyword = $do["keyword"];

mysql_query ("INSERT INTO $insert_table VALUES ('$input_keyword')") or die("insert data failed : " . mysql_error());



} // end while

Whats the error you are getting ?

you are probably hitting your php script timeout.... also that is a horribly inefficent way to do that.

There is a max execution time in your php.ini. Increase it from 30 seconds (by default) to something longer like 5 minutes.

The code you're using is not optimized and what you're trying to do can be done in the following query:
CREATE TABLE $insert_table FROM SELECT DISTINCT keyword FROM $get_table GROUP BY keyword

Originally posted by fraudgate
There is a max execution time in your php.ini. Increase it from 30 seconds (by default) to something longer like 5 minutes.

The code you're using is not optimized and what you're trying to do can be done in the following query:
CREATE TABLE $insert_table FROM SELECT DISTINCT keyword FROM $get_table GROUP BY keyword


I did change it to higher value (7200 currently).

But the problem is still exist. Can anyone help me please?

Thanks for your idea for optimization of my code.
:0

Originally posted by jason216
I did change it to higher value (7200 currently).

But the problem is still exist. Can anyone help me please?

Thanks for your idea for optimization of my code.
:0


use his SQL, you are complicating these things WAY too much you are making a mess out of it.

I tried to use his way but its failed too.

It's actually
CREATE TABLE $insert_table SELECT DISTINCT keyword FROM $get_table GROUP BY keyword

(He had an extra FROM in there.) When something fails, post the actual error message. Even if it looks like gibberish to you, people with more experience with the application probably understand it.

Originally posted by hiryuu
It's actually
CREATE TABLE $insert_table SELECT DISTINCT keyword FROM $get_table GROUP BY keyword

(He had an extra FROM in there.) When something fails, post the actual error message. Even if it looks like gibberish to you, people with more experience with the application probably understand it.

There is no error message when I run and fail.

it just like a page can't be loaded.

It took around 300 seconds and failed , I think I incease some variable of exec. second in the setting of MYSQL

but I don't know what it is, can anyone help me please?