Hello

I need to regularly send about 10,000 personalized emails of about 20 KB each.

I have already wrote all the code to compile the messages, but now I am not sure whether the PHP mail() function can handle all the emails.

I'm afraid the following might happen:
1. The web page sending the email stops half way (timeout error)
2. The local SMTP server takes up all resources of the server (denial of service to other clients)

Does anybody know if the PHP mail() function will work? And if not, any suggestions on what are the best practices on sending bulk email?

Thank you

OK, I think I got the answer. I believe this problem to be very common – especially for websites of large companies, so I will post the basic idea here.

Basically what you do is:

1. Create a script that saves all your already compiled emails in a database, for example:

CREATE TABLE mail_queue (
id bigint(20) NOT NULL default '0',
sender varchar(50) NOT NULL default '',
recipient text NOT NULL,
headers text NOT NULL,
body longtext NOT NULL,
PRIMARY KEY (id),
KEY id (id),
);

2. Create another manage_queue.php file that

a) Sends one email from the table (if any records)
b) And then delete that record

3. Create a crontab that execute the script in the background every second.

That’s it!

Remember to bill your client for the fancy system!


The method supplied by php.net on http://pear.php.net/manual/en/package.mail.mail-queue.mail-queue.tutorial.php works on pretty much the same way. That solution is probably much more complete. I just don’t think I want to go through the trouble of figuring out how to install it.

Some notes:
- All your websites can write to the mail_queue database!
- You can send more than one email in a script run and then execute the script in larger intervals. I guess smaller is better for load balancing.
- Remember NOT to save the manage_queue.php in a web accessible folder!
- Beware if your intervals are too small, you might get overlapping. I’m not sure what will happen then, but to be safe I will set a status variable to active/idle before & after the script executed to prevent problems. (Then the script simply checks the variable before executing.)
- You will probably also need to manage or delete problematic email addresses from the queue, so that it doesn’t get stuck at one address.

ps. Anybody with better solutions, please feel free to post!

I've completed the code and copied here if someone can find it useful.

The table:


CREATE TABLE mail_queue (
id bigint(20) NOT NULL default '0',
sender varchar(50) NOT NULL default '',
recipient text NOT NULL,
subject text NOT NULL,
body longtext NOT NULL,
PRIMARY KEY (id)
);


bulkmail_client.php (The client file that you can give to other client developers):

<?php
function bulkmail_OpenDB () {
$database="****";
$host="****";
$db_user="****"; //this DB user should only have SQL INSERT rights, so that multiple clients can use the same DB
$db_pass="****";
global $linkID;
global $bulkmail_linkID;
$bulkmail_linkID = mysql_connect($host, $db_user, $db_pass);
mysql_select_db($database, $bulkmail_linkID);
}

function queueNewMail($sender,$recipient,$subject,$body){
global $bulkmail_linkID;
mysql_query("insert into mail_queue values('','".$sender."','".$recipient."','".$subject."','".strToHex($body)."');", $bulkmail_linkID);
}

function bulkmail_CloseDB () {
global $bulkmail_linkID;
mysql_close($bulkmail_linkID);
}

//I had too much problems with special characters in the html $body, so this function take care of any thinkable special character. (Remember to decode it before sending!)
function strToHex($strString){ // encode any string to a combination of A..Z, 0..9
$newString="";
for ($i = 0; $i < strlen($strString); $i++) {
if (ord($strString[$i])>=16)
$newString.=dechex(ord($strString[$i]));
else
$newString.="0".dechex(ord($strString[$i]));
}
return $newString;
}
?>


and here are the admin script that is suggested to execute every 10 minutes: process_bulkmail_queue.php



<?php

$n_mail=20; // sent it every 10minutes

OpenDB();
$result = sql("SELECT * FROM mail_queue LIMIT ".$n_mail);
while ($row = mysql_fetch_array($result)){
if (validEmail($row["recipient"]) && validEmail($row["sender"]))
SendEmail($row["recipient"],$row["subject"],hexToString($row["body"]),$row["sender"]);
sql("DELETE FROM mail_queue WHERE id=".$row["id"]);
}
CloseDB();


function sql ($command) {
global $linkID;
return mysql_query($command, $linkID);
}

function OpenDB () {
$database="****";
$host="****";
$db_user="root"; //DB root user is fine
$db_pass="****";
global $linkID;
$linkID = mysql_connect($host, $db_user, $db_pass);
mysql_select_db($database, $linkID);
}
function CloseDB () {
global $linkID;
mysql_close($linkID);
}

function SendEmail($to_address, $subject, $message_body, $from){
$message_body='
<html>
<head>
<title>'.$subject.'</title>
</head>
<body>
'.$message_body.'
</body>
</html>
';
$headers = 'MIME-Version: 1.0' . "\n";
$headers .= 'Content-type: text/html; charset=iso-8859-1' . "\n";
$headers .= 'From: '.$from.'' . "\n";

if (mail($to_address,$subject,$message_body,$headers) ){
return true;
}else{
return false;
}
}

function validEmail($email){
if(eregi("^[_a-z0-9-]+(\.[_a-z0-9-]+)*@[a-z0-9-]+(\.[a-z0-9-]+)*(\.[a-z]{2,3})$", $email)) {
return true;
}
else {
return false;
}
}

function hexToString($hexString){
$newString="";
for ($i = 0; $i < strlen($hexString); $i+=2) $newString.=chr(hexdec(substr($hexString,$i,2)));
return $newString;
}

?>




I've tested all the code except for the SendEmail(), but it worked the last time I used it (about a month ago).

Hope this helps someone...