Tech Off Post

Single Post Permalink

View Thread: SQL Server 2008 CTP, FILESTREAM & NTFS limitations
  • User profile image
    figuerres

    Well I have known for a long time that folders that have a large number of files can lead to perf issues on all file and folder operations.

    I first saw this on Sun / Solaris around 1997
    An ISP I worked for had users email on a disk array and saw massive slowdowns as the number of users went up.

    also there were / perhaps still are IIS plugins for servers that want to host lots of free web sites.

    the general answer is to create a tree in place of a flat directory.

    example: 

    Mailboxes/
    a/
    b/
    c/
    ...
    z/

    then in each of the folders a..z create folders a..z and in each of them repeate.

    so for example if my pop3 user name was "Denny"
    the file would be found in the folder/path
    Mailboxes/d/e/n/denny/*

    with the result that each folder has a limited number of files and no one folder has a lot of files.
    assuming of course that names are "spread out"
    sometimes a hash is used to create folder names and map real names to the hash.

    I would take the 900000 plus files and make a tree.

    I suspect you could see the same access problem doing a dir-search or a set of file open functions in a program.