Results 1 to 17 of 17
-
02-20-2012, 04:28 AM #1Newbie
- Join Date
- Feb 2012
- Posts
- 11
Recommended size of log partition in Linux
Hi
On a busy LAMP-server, where all logs from Apache are stored under /var/log (but mysql is not logging transactions), what size would you recommend the /var/log partition to be? I want to keep logs for several years (say 10 years), but compress them using logrotate and gzip. And I want to keep all logs, not just apache but also syslog, authlog, daemon etc. I also have logs from other software like snort, mod_security and other.
Any ideas? is 10gb overkill?
-
02-20-2012, 04:43 AM #2Disabled
- Join Date
- Nov 2010
- Posts
- 2,564
Why do you like to keep all logs for that long? If you really like to, probably you need 1TB or more depends how many logs your website generate every day.
-
02-20-2012, 05:08 AM #3Newbie
- Join Date
- Feb 2012
- Posts
- 11
Personally I wouldn't want to keep them for long but my client says it's legel requirements.
1TB sounds a little bit excessive though. Sure it would mean I would not have to worry about disk space, but I was just wondering if anyone had any experience with typical log partition sizes.
-
02-20-2012, 05:10 AM #4Moving the Web Forward
- Join Date
- Jun 2010
- Location
- Modesto California
- Posts
- 6,858
Looking for an awesome VPS Offer? CLICK HERE
"Knowing is not enough, we must apply. Willing is not enough, we must do." – Bruce Lee
-
02-20-2012, 05:14 AM #5Newbie
- Join Date
- Feb 2012
- Posts
- 11
-
02-20-2012, 05:16 AM #6Always Ask...Don't Pretend!
- Join Date
- Aug 2010
- Location
- CPU
- Posts
- 2,187
If your customer tells you that this is legal requirement then tell him that money is also a requirement to keep this logs. Make sure to bill him for 2TB at least you have enough buffer and income for keeping the logs and make it clear that the cost is incremental for certain threshold.
█ Ask for Server IP & Nameservers IP to check if your reseller provider truly provides 100% white-label.
-
02-20-2012, 05:26 AM #7Newbie
- Join Date
- Feb 2012
- Posts
- 11
Thanks!
But I am really more interested in finding out how much the /var/log usage would grow in one day on a very busy server. If anyone out there can share some numbers it would be great.
-
02-20-2012, 07:03 AM #8Web Hosting Evangelist
- Join Date
- Dec 2009
- Location
- Milan, IT
- Posts
- 478
What mean "busy" for you? How many hits you you have a present? Your logs are likely to be larger if there are a lot of requestes per pages...
To give you an example, I have 350-400MB of compressed logs each day for a very busy site.... :-)Prometeus.com & iperweb.com Hosting Solutions since 1997
We are LIR (RIPE NCC member) and do BGP with ASN 34971 Multiple gigabit/s network capacity
iwStack.com IaaS Cloud services
-
02-20-2012, 07:36 AM #9New Member
- Join Date
- Feb 2012
- Posts
- 2
Disk partitioning is the creation of separate divisions of a hard disk drive using partition editors such as fdisk. Once a disk is divided into several partitions, directories and files of different categories may be stored in different partitions.Many new Linux sys admin (or Windows admin) create only two partitions / (root) and swap for entire hard drive.
-
02-20-2012, 08:03 AM #10Newbie
- Join Date
- Feb 2012
- Posts
- 11
It's a new domain so it's not yet launched. When I say busy I mean more than a million hits per day. (and I know that is nothing compared to sites that generate over a billion hits a day)
To give you an example, I have 350-400MB of compressed logs each day for a very busy site.... :-)
-
02-20-2012, 08:15 AM #11Web Hosting Evangelist
- Join Date
- Dec 2009
- Location
- Milan, IT
- Posts
- 478
Ok looking at some site with that numbers I see that 1.5 million of rows are 25M compressed...
Prometeus.com & iperweb.com Hosting Solutions since 1997
We are LIR (RIPE NCC member) and do BGP with ASN 34971 Multiple gigabit/s network capacity
iwStack.com IaaS Cloud services
-
02-20-2012, 08:18 AM #12Newbie
- Join Date
- Feb 2012
- Posts
- 11
-
02-20-2012, 08:22 AM #13New Member
- Join Date
- Feb 2012
- Posts
- 1
Minimum 1 TB
-
02-20-2012, 08:40 AM #14Junior Guru Wannabe
- Join Date
- Mar 2011
- Posts
- 86
You should consider predicting the size of the logs based on the growth of your current logs. One of the framework that we host can log 10GB of logs per week for having 1k visitors per day. So to keep all the logs it would require 1TB of disk space per year.
♥ Managed1.com ♥ - Affordable budget Fully Managed Hosting Services for individuals and business clients. Website Hosting. Domain Name management. High Availability Cloud servers - 99.9% Uptime, Self-Healing Failover, Load Balancing and Mirrored Backups.
♥ Whisperdale.net ♥ - Internet Service Provider since 2007!
-
02-20-2012, 08:47 AM #15Newbie
- Join Date
- Feb 2012
- Posts
- 11
Thanks, but counting the number of visitors doesn't say much about the log size. Apart from the problem of defining a visitor (based on IP? cookies?), there is the problem that one visitor alone could in theory cause the entire 10GB log data if he/she was very active on the website.
-
02-23-2012, 04:23 AM #16Newbie
- Join Date
- Feb 2012
- Posts
- 11
FYI, a related website that I host generated 600K hits yesterday, and 200GB bandwidth. Apache log is 142MB uncompressed, standard customlog.
-
02-23-2012, 05:37 AM #17New Member
- Join Date
- Feb 2012
- Posts
- 1
Similar Threads
-
Swap size recommended for redhat linux
By linuxlinux in forum Hosting Security and TechnologyReplies: 10Last Post: 12-18-2004, 03:57 PM -
Best Partition Size
By asc2000 in forum Dedicated ServerReplies: 12Last Post: 01-29-2004, 10:29 AM -
linux command to check partition size and or disk size
By MarcD in forum Hosting Security and TechnologyReplies: 7Last Post: 08-01-2003, 12:40 PM -
Changing partition size?
By spiderman in forum Hosting Security and TechnologyReplies: 2Last Post: 05-03-2002, 09:36 PM -
Adjusting Swap Partition Size on Linux / RAQ3 Systems...
By Groo in forum Dedicated ServerReplies: 0Last Post: 03-02-2001, 10:17 PM