Web Hosting Talk







View Full Version : IMAP & pack rats... How much is too much?


WebsterD
09-07-2005, 01:23 PM
I have a customer who runs a business with 15+ employees. Email is an important part of the business. They're hosted on a Virtuozo VPS in a RHL/Apache/DirectAdmin environment. Squirrelmail is the webmail of choice when needed. Most people use Outlook Express or one of the Mozilla clients. A few people are using IMAP, most use POP.

The boss uses IMAP. He travels a lot and uses his account from home and office, so he benefits from IMAP. But he doesn't do any maintenance. He can't bring himself to delete anything. There are messages going back 20 months, some of it organized in various folders. I'm sure his inbox is no less crowded. There's even undeleted spam.

He complains about performance, stalling, folder discrepancies, etc., and he seems to suffer most in Squirrelmail. He's stubborn about letting go of old messages, or any other suggestions for that matter. He knows how to move something from the IMAP inbox to a local folder but moving from the server to the local computer is too messy in the volume required to make a difference.

I tell him his account is over-weight. I tell him as volume gets larger and his IMAP folders get more complex, things slow down and at some point something may fail. Other users get along easier, especially POP users. We've been going back and forth over this for months.

I'm interested in opinions on what is acceptable. For starters, what's a recommended limit for IMAP storage? What could I do better?

jt2377
09-07-2005, 02:48 PM
the storage is limit to what you set in the email server. tell him to use Outlook and simply back it all up then deleted them. heck, there is even backup software for Outlook. it ain't hard if you search around.

i'm supprised that his company didn't simply use Exchange. it got way more backup option (3rd party software)

WebsterD
09-07-2005, 03:10 PM
I don't disagree about the advantages of an Exchange server if it can be justified. I think it's over-kill in our case.

There are plenty of options to backup and clean up, the user just doesn't do it.

What I'm trying to get at is what kind of thresholds I should be looking for in a common hosting environment. Like mailbox size versus performance. Maybe my customer isn't out of line at all and just needs something tougher.

error404
09-07-2005, 03:44 PM
I've never had performance issues with IMAP, using Maildirs anyway. I'm not sure what the DirectAdmin default configuration is, but if it's using mbox format, that will easily explain the slowdown. Maildir is way, way more scalable though (esp. on a modern filesystem like Reiser), and can handle thousands of messages per folder with very little performance issue.

20 months of mail is not an unreasonable amount, and keeping it in IMAP probably isn't a bad idea either. I have about 250MB in my various IMAP folders. It's a good central location, requires very little extra work to keep organized, and doesn't require you dig out some special backup software and track down your backup files when you need to access some old correspondence. It's especially useful for people such as your boss that may want ubiquitous access to their archives.

Clients (that is, mail clients) will balk at huge folders, but that's just a matter of keeping an archive organized. If his organizational skills are really that bad, just tell him (or even script) to move everything in his inbox to a dated folder at the end of the month or something. That should keep it to a usable size for most clients. Though, I've used SquirrelMail with folders with 1000s of messages with perfectly acceptable speed. I'd guess the issue is that your mail system is using the older and slower mbox mail format to store the mail. If it's viable, migrate to Maildir and you should see a major difference. Otherwise, about the best you can do is archive locally. I'd recommend keeping mbox files under 25MB of raw mail.

WebsterD
09-07-2005, 05:16 PM
I think you have identified the problem. By mbox versus Maildirs I relate that to UW versus Courier IMAP. Most DirectAdmin servers use UW, and I keep hearing that this daemon is lame compared to Courier IMAP. I'm beginning to see it. My problem user's mailbox is approaching 100 mb, it started acting up at 50 mb.

Your reply is appreciated!

Amish_Geek
09-07-2005, 08:21 PM
I use IMAP with thunderbird and squirrelmail. My inbox is sitting at about 300mb in size.

I use courrier IMAP, and I have no problems whatsoever.

Jay Suds
09-07-2005, 09:16 PM
100MB of storage for 20 months of email is nothing ... sounds to me more like a configuration / server performance issue than improper mail usage. While obviously not a direct comparison, on our Windows server we have customers who have 800-1200MB of mail storage for one account, and don't run into any performance issues.

JTY
09-07-2005, 09:26 PM
See if you can convert his mbox folders, into mbx. The mbx format is supported by UW-IMAP, and it's much faster than mbox.

WebsterD
09-07-2005, 11:56 PM
All very interesting. Especially mbx format, I'll check that out. I need to do something fast. I thought of an external mail service but don't want extra expenses.

Which IMAP is typically installed on cPanel servers?

Kiamori
09-08-2005, 01:15 AM
Just make sure your plans are properly structured for high usage email accounts. Not sure what type of client you have but PR Firms use the most email I have ever seen but they also pay for it.

My IMAP folder is at "1663.067 MB" right now and I have a few clients that have much larger IMAP folders on another server without any issues.

Exchange servers have issues unless properly maintained, cost a lot to maintain/license properly for antispam/virus, etc.

I suggest getting an IMAP with a collaboration suite that shares contacts, calendar, tasks and can sync mail folders if you don't like IMAP. It is highly profitable and still much cheaper then running an exchange server.

Check out some software like: http://ipswitch.com/products/collaboration/index.asp

bqinternet
09-08-2005, 06:49 PM
We have several IMAP accounts with several gigabytes of files in each, on a server running Cyrus IMAPd, and we haven't seen any slowdowns. I would investigate your server configuration, especially the format that the emails are being stored in.

AH-Tina
09-09-2005, 05:13 AM
I have just over 5000 emails stored via IMAP for one of my email accounts, in 26 folders. Although I regularly delete email I no longer need, I have some going back to 1999. :)

--Tina

Steven
09-11-2005, 10:19 AM
I have close to 40 thousand emails in one of my imap accounts on the Maildir format. Works great

WebsterD
09-11-2005, 12:29 PM
UW-IMAP might have shined in the early days but it's obvious to me now that it has out-lived its usefulness.

I came across scripts for converting mbox to maildir, or for adding maildir support to UW-IMAP, but I would be more comfortable switching to a server with Courier or Qmail as the standard mail systems.

An easy alternative would've been to move this customer from a DirectAdmin server to cPanel... easier compared to replacing the mail daemon entirely. But cPanel also uses UW-IMAP. Later versions of cPanel have a script that will convert mbox to maildir format, which adds to my confusion. Does this mean that UW-IMAP supports maildir storage? Or is this a cPanel modification only?

It's probably time for me to post my questions in a different category.

Steven
09-12-2005, 09:10 PM
when you convert to maildir on cpanel it installs courier imap

brianoz
09-12-2005, 10:19 PM
Dovecot is also a robust IMAP server and should be simpler to install. I've had it running in a shared environment for months now with no problems.

ArtieFishill
09-14-2005, 05:06 PM
Tell him to get a Gmail acct...lol.

KDAWebServices
09-14-2005, 05:44 PM
Outlook, Outlook Express don't work all that well with large IMAP folders in my experience. I did actually split my IMAP box a while back, as it had got to 2GB in size, so I moved it all out in to another account (Still IMAP) and started afresh with a new IMAP account, which is just approaching 2GB.