[NTLUG:Discuss] Maximum files per directory

[Vaidya, Harshal (Cognizant)]

Hey Richard,

  Files can be created as long as inodes can be allocated from the
filesystem. The inodes are allocated from the superblock.  It dosen't
matter whether the files are created in a single directory or different
directories.

As Steve Baker mentioned there are performance implications if you have
an excessively large number of files in a directory. This is because of
the way files references are stored in the directory. A directory, as
you may be aware, is a simple table,column representation of 
filename vs names vs inode numbers. 

When ever, a file is opened, the directory which it's contained in is
searched for the inode number for that file. The inode is then accessed
from the superblock. If there are a huge number of files in the same
directory, this search for the inode number takes a long time,
consequently, resulting in slow throughput by the OS.

Regards,
Harshal Vaidya.




   

-----Original Message-----
From: Steve Baker [mailto:sjbaker1 at airmail.net] 
Sent: Sunday, May 18, 2003 3:22 AM
To: rwolfe at rwolfe.com; NTLUG Discussion List
Subject: Re: [NTLUG:Discuss] Maximum files per directory


Richard Wolfe wrote:
> Is there a limit to the number of files I can put in a single 
> directory? And even if there's no limit, are there practical reasons 
> why I wouldn't want to exceed a certain number of files? Is 10,000 ok?

> How about 100,000? I'm running 2.4.18 (RedHat 8.0), and the filesystem

> is ext3, if any of that makes a difference.

A lot of things start to slow down quite a bit when you have a lot of
files in one directory.  We had a system that put 6,400 files in one
directory and the time taken to open a file went WAY up.

I'd strongly advise you to find a way to split it up into at least a
couple of levels of heirarchy.

---------------------------- Steve Baker -------------------------
HomeEmail: <sjbaker1 at airmail.net>    WorkEmail: <sjbaker at link.com>
HomePage : http://www.sjbaker.org
Projects : http://plib.sf.net    http://tuxaqfh.sf.net
            http://tuxkart.sf.net http://prettypoly.sf.net -----BEGIN
GEEK CODE BLOCK----- GCS d-- s:+ a+ C++++$ UL+++$ P--- L++++$ E--- W+++
N o+ K? w--- !O M- V-- PS++ PE- Y-- PGP-- t+ 5 X R+++ tv b++ DI++ D G+
e++ h--(-) r+++ y++++ -----END GEEK CODE BLOCK-----


_______________________________________________
https://ntlug.org/mailman/listinfo/discuss
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: InterScan_Disclaimer.txt
Url: http://ntlug.org/pipermail/discuss/attachments/20030519/35bc16eb/InterScan_Disclaimer.txt