| Folkert Rienstra 2004-07-19, 5:45 pm |
|
"Marc de Vries" <marcdevries@geen.spam.zonnet.nl> wrote in message news:qi7nf0piehsg6g4nebgdp15encc38ng9s6@
4ax.com
> On Mon, 19 Jul 2004 09:28:50 +0200, "Benno..." <0@0.invalid> wrote:
>
>
> More spindles also gives better performnance with those very small
> files.
Nope, only on busy servers that do alot of them simultaniously.
> The array controller then has the option to read multiple files
> simultaneously from different spindles.
Therefor still reads at full stripe width transfer rates.
>
> Especially with small files you should set the stripe size to the
> maximum that the controller supports. So it already has the optimum
> setting.
Not if this is not a "busy" server. The bigger the stripe size the more
small files that sit on a single disk and transfer at single disk speeds.
If that's not compensated by the shear number of them that a part of
is read simultaniously all the time then you loose.
> (The idea behind that is that the small files are stored on as few disks
> as possible, which will increase the chance that you can read several files
> simultaneously)
So you actually make them slower, to read more of them simultaniously.
On a not so busy server you are insuring that the small files will transfer
even slower compared to doing nothing. Nice one.
When you leave it as it is that you thought was best, at least you don't
make the ones that fill a stripe width slower, and, when they are less
than that, smaller files automatically fill up the gap when the server is
busy and has many outstanding IO.
>
> But whatever you do, you might increase performance, but you will
> never ever get large transferrates with small files.
Right, now for yourself to let that sink in.
> The reason for that is the following: The time it takes to search for
> that file on the disk and open it is very large in comparison to the
> time it takes to transfer it.
>
> The same might also happen on the network. I'm not sure what overhead
> you get on small files in the network. But you might want to take a
> look at what happens when you access those small files directly on the
> server, compared to what happens when you access them over the
> network.
>
> Are you doing a lot of writing to the array controller? You have to
> accept that writes will never be fast. Extra cache will not fix that.
Not on a busy server, no. And not if the write speed is not disk related.
It will if it is and the cache can catch up in less busier periods acting
as a buffer.
> And it
What "it"?
> might slow the the reads down a bit in such a way that on
> average the user experience is slower.
> I don't have personal experience with roaming profiles, but I'd guess
> they require more read than write capacity. More cache in the array
> controller might help. if you have the option to add more.
>
> You could experiment with the cluster size, but I don't think that
> that will help. The cause of the slow performance is the relatively
> large seektime when accessing small files.
> And that doesn't change when the clustersize is smaller.
Actually it does when already small files fragment because of it.
>
> I'm afraid that I can't think of much to improve the situation.
> Basically the applications shouldn't create so many extremely small
> files, because that will always hurt performance.
Unless they sit on a dedicated drive that is not mechanical in nature.
Solid State Disk.
> (Your backup software is probably not too happy about it either)
That obviously depends on the type of backup.
>
> Marc
|