Results 1 to 12 of 12
  1. #1
    Join Date
    Nov 2004
    Posts
    109

    Number of Files in a Directory

    I have a webserver with CentOS, I was wondering if there's a limit to the number of files in a directory?

  2. #2
    I have never done it this way, but can't you just setup quotas and put a limit on the inodes?

  3. #3
    Join Date
    Nov 2004
    Posts
    109
    No, actually, I'm going to be creating a lot of files in a single directory, I just want to make sure this isn't going to do bad things

  4. #4
    Join Date
    Mar 2003
    Location
    Canada
    Posts
    8,910
    Do you mean how many files can be in any directory, or how -large- the files can total in one directory?
    Patrick William | RACK911 Labs | Software Security Auditing
    400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com

    www.HostingSecList.com - Security notices for the hosting community.

  5. #5
    Join Date
    Mar 2003
    Location
    Canada
    Posts
    8,910
    Quote Originally Posted by Pryach
    No, actually, I'm going to be creating a lot of files in a single directory, I just want to make sure this isn't going to do bad things
    In that case, there really shouldn't be a problem...
    Patrick William | RACK911 Labs | Software Security Auditing
    400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com

    www.HostingSecList.com - Security notices for the hosting community.

  6. #6
    Join Date
    Nov 2004
    Posts
    109
    Do you mean how many files can be in any directory, or how -large- the files can total in one directory?
    I meant "how many." The files that are created are very small in size, but there are a lot of them.

  7. #7
    Join Date
    Jun 2003
    Posts
    961
    how many are "lot"? and which filesystem your using to host them?

  8. #8
    Join Date
    Nov 2004
    Posts
    109
    It will probably get into the thousands.

    I'm basically querying another website for information, and while I have permission from this site to do that, they've asked that I cache the information so that if it's queried again, I look at my cached files instead of querying their site again.

    Initially I was going to put the information into a database, but I worried about MySQL crashing if too may users tried to access it.

    How do you look at the filesystem in CentOS?

  9. #9
    Join Date
    Apr 2003
    Location
    Los Angeles, CA
    Posts
    800
    CentOS uses ext3 by default which probably is fine. You can check with "mount":

    # mount
    /dev/hda3 on / type ext3 (rw)
    ...

    The limit is unlikely under 32K files because one of my mailboxes (in maildir format) has ~24,000 messages in it (about 500 MB, one file per message). The mailbox opens in under 1 seconds via IMAP (ls -l needs less than a second as well), so performance is OK. Nevertheless, if you can, I would suggest splitting it into sub-directories...
    Pings <1 ms, Unlimited Transfer, Lowest Price: http://localhost/

  10. #10
    Join Date
    Jun 2003
    Posts
    961
    the total number of files/dirs on ext2/3(?) is limited by the number of inodes
    you could check this number with "dumpe2fs /dev/hdXY" or similar
    for big amount of small files i would recommend reiserfs or xfs

    why would your mysql db crash?

  11. #11
    Join Date
    Jan 2003
    Posts
    1,715
    I'd probably go with ReiserFS or XFS for many files in one directory, although newer ext3 installs can use a tree format for directories, too. The largest I've dealt with was 200k files, but I'm sure you can go larger.
    Game Servers are the next hot market!
    Slim margins, heavy support, fickle customers, and moronic suppliers!
    Start your own today!

  12. #12
    Join Date
    Sep 2004
    Location
    Flint, Michigan
    Posts
    5,765
    I think you have things backwards Going to a file based system instead of a database system is almost like going back in time. I would not be worried about mysql crashing over a few thousand rows (I would not be worried about a million rows). Design your database properly and I am sure you will see great speed benefits compared to caching on disk.
    Mike from Zoodia.com
    Professional web design and development services.
    In need of a fresh hosting design? See what premade designs we have in stock!
    Web design tips, tricks, and more at MichaelPruitt.com

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •