Web Hosting Talk : Web Hosting Main Forums : Hosting Security and Technology : which php cache engine should i use ??

[jmansoor]

I have a very huge dynamic and heavy traffic website with more then 700,000,000 pages all of them generated dynamically.

I'm looking to optimize the website using cache engine and want to cache the whole 700,000,000 pages. i found in the forum that mmcache is the best cache engine for php, but the problem is that, it store all the cached pages in a single directory. storing 700,000,000 files in a single directory will create lot of problems for the server.

is there any caching engine out there that can store the cached pages in multiple sub directories based on directory names? ie /tmp/cahce/1/2/3/123234324.something instead of /tmp/cache/123234324.something

[Captian_Spike]

MMCache is open source, you could probally modify it to store files differently. You could probally do it your self or if you rather make a post in the offers forum and get someone else to do it for a small fee.

[jmansoor]

Thanks u2mike!

i can make changes myself if you can guide me little bit.

i have already installed mmcache on the server.
do i need to get the mmcache source code, modify it and recompile in order to work or can i use the already complied script and modify it to do the job ?

Thanks for the help.

[LoganNZ]

jmansoor.

Can i ask what website do you run?

That is some BIG website, with a lot of dynamic pages.

Send the URl via PM if you like, I am very interested

Don't worry i'm not a botnet kiddie....

[jmansoor]

hi LganNZ, The website belongs to one of our client and i can not declare the name as per our privacy policy.

sorry for that.

[LoganNZ]

No problem.

[almahdi]

I have always used jpcache, it works in all the php envoirments, I also modified it to give a basic protection from dos, and bandwidth stealing, if a user refreshes few times, the user will be blocked for an adjustable time, this way we reduce the server load, and bandwidth, since jpcache uses gzip compression to send the data.

jpcache + mmcache = incredible performance.

If you are a good programmer, you do what ever you need, you can modify jpcache and mmcahe to do you job..

[jmansoor]

Thanks Almahdi for the tip. my main problem with jpcache is it caches only the whole page, and can not cache a part of page.

am i right ?

[almahdi]

Hmm, I cannot say that you are not...

but you can implement your own cache system, based on jpcache, or modify jpcache to do your task, just exactly as we do..

If you just explain your problem a bit clearer, we all going to be glad to help you.

Something else, is that, what do you mean by many files in one directory causes problems in the server?? We've been using mmcache for a while on shared hosting servers, we never faced a single problem..

[jmansoor]

Almahdi, as per my information and experience, if you put lot of files, let say more then 10,000 files in one directory, it will give hard time to OS & CPU to access those files. it's advisable to seperate files in different directories so that OS can easily access it.

Actually i want to cache each page of my website in 3 parts. header, body and footer. the header and footer will change according to the reseller account, but body will be same for each reseller but will contain unique content for each page.

i will see how easy it will be to modify jpcache to start cache different parts of the same page instead of the whole..

Thanks alot for the help.

[almahdi]

as per my information and experience, if you put lot of files, let say more then 10,000 files in one directory, it will give hard time to OS & CPU to access those files. it's advisable to seperate files in different directories so that OS can easily access it.

We've been running shared hosting servers, and all run with mmcache for caching with no problem at all.

Anyway, you can modify jpcache to do your job easily.. only if you know what you are doing...

[dollar]

The type of cache engine you are looking for does not exist as far as I know. I am not an expert on MMCache, but I don't think it would cache every single page, only the ones that are accessed often.

[jmansoor]

almost all the pages of my sites are visited .. so mmcache will cache all of them..

[dollar]

Assuming each page is visited in one month.

700,000,000 pages in a month
700,000,000 / 30 = 23,333,333 Page Views A Day
23,333,333 / 24 = 972,222 Page Views an Hour
972,222 / 60 = 16,203 Page Views A Minute
16,203 / 60 = 270 Page Views A Second

Even at that rate that would mean each page is only being accessed one time, so i really don't think all the pages are visited that often, but it's your website and you know it much better than me.

That aside, if you wanted to cache all 700,000,000 pages do you know what type of space that would require?

700,000,000 Pages @ 8Kb a piece = 5,600,000,000 Kilobytes
5,600,000,000 / 1024 = 5,468,750 Megabytes
5,468,750 / 1024 = 5,340 Gigabytes
5,340 / 1024 = 5.2154 Terabytes of storage alone...

[jmansoor]

justadollarhostin! thanks for the clarification. i agreed with you now. i can not cache all the pages.

i think i have to write a code to see which page is getting more views, and cache only those pages.