|
Home > Archive > Data Storage > September 2006 > Designing A File Server With Best Price Performance?
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Designing A File Server With Best Price Performance?
|
|
|
| We are migrating shared files off of a Windows domain controller to a
discrete file server, and I'm thinking through what would be the best design
for that new server. Given that 64 bit operating systems are now here,
I'm thinking we do not need to spend much on fast drives, but we should
instead invest in the lowest cost 64 bit server (a file server won't
bottleneck on CPU so even a 1 GHz machine would be fast enough) and install
gigabit ethernet and lots of memory. I'm sure that the most requested
files would fit into a memory cache that is under 10 GB in size. A
computer with 12 GB of memory, 64 bit Windows 2003 server web edition, and
gigabit ethernet should provide the best possible speed for the case where
there is not much write activity, but lots of read activity on less than 10
GB of data.
Assume less than 20 users, heavy read activity, very low write activity.
Disk with all file shares would be under 100 GB, but under 10 GB of that
represents 95% of the activity.
Is my design correct if I want to maximize performance for this small
network?
--
Will
| |
| robertwessel2@yahoo.com 2006-08-27, 7:14 am |
|
Will wrote:
> We are migrating shared files off of a Windows domain controller to a
> discrete file server, and I'm thinking through what would be the best design
> for that new server. Given that 64 bit operating systems are now here,
> I'm thinking we do not need to spend much on fast drives, but we should
> instead invest in the lowest cost 64 bit server (a file server won't
> bottleneck on CPU so even a 1 GHz machine would be fast enough) and install
> gigabit ethernet and lots of memory. I'm sure that the most requested
> files would fit into a memory cache that is under 10 GB in size. A
> computer with 12 GB of memory, 64 bit Windows 2003 server web edition, and
> gigabit ethernet should provide the best possible speed for the case where
> there is not much write activity, but lots of read activity on less than 10
> GB of data.
>
> Assume less than 20 users, heavy read activity, very low write activity.
> Disk with all file shares would be under 100 GB, but under 10 GB of that
> represents 95% of the activity.
>
> Is my design correct if I want to maximize performance for this small
> network?
If you've correctly characterized your workload, it's not an
unreasonable plan, except I didn't think MS has done a 64 bit WS2003
Web Server Edition yet, and second, the Web Server Edition is *not*
licensed for local file serving.
64-bit WS2003 Standard Edition is probably what you're looking for.
Nonetheless, reasonably fast drives will help at initial loads and when
hitting files that don't cache. But I'd probably be more concerned
about drive reliability than performance. A mirrored pair of server
class SATA drives of the requisite capacity (but not the fastest
available) are probably a much better idea than cheaping out and
putting a couple of desktop drives in there.
Second on a server, and *especially* with that much RAM, I'd
absolutely insist on ECC'd memory.
| |
| Jon Metzger 2006-08-28, 1:18 pm |
| Will wrote:
> We are migrating shared files off of a Windows domain controller to a
> discrete file server, and I'm thinking through what would be the best design
> for that new server. Given that 64 bit operating systems are now here,
> I'm thinking we do not need to spend much on fast drives, but we should
> instead invest in the lowest cost 64 bit server (a file server won't
> bottleneck on CPU so even a 1 GHz machine would be fast enough) and install
> gigabit ethernet and lots of memory. I'm sure that the most requested
> files would fit into a memory cache that is under 10 GB in size. A
> computer with 12 GB of memory, 64 bit Windows 2003 server web edition, and
> gigabit ethernet should provide the best possible speed for the case where
> there is not much write activity, but lots of read activity on less than 10
> GB of data.
>
> Assume less than 20 users, heavy read activity, very low write activity.
> Disk with all file shares would be under 100 GB, but under 10 GB of that
> represents 95% of the activity.
>
> Is my design correct if I want to maximize performance for this small
> network?
>
10GB of RAM sounds like overkill. File servers of the size you're
talking about are generally well underutilized. I've seen VMWare
servers with 4GB of RAM running 10 Windows file servers serving more
users per server than you are looking for. If money is no object, but
all means go for it. If you're looking to size your server to your
workload you could probably save yourself several thousand dollars by
going with a lower end box with less RAM.
Also, you don't mention what type of data you'll be working with, but
typically file servers are very tolerant of disk latency. If an end
user opening a file sees a 50ms delay instead of a 4ms delay, do they
really notice the difference? If you're working with large sound or
video files and that's a concern, you could go with 15K rpm SCSI disks.
If not, you're probably going to be okay with lower cost and higher
capacity SATA disks.
| |
|
| "Jon Metzger" <blagh@___gmail.com> wrote in message
news:ecut56$6j0$1@lenny.tc.umn.edu...
> 10GB of RAM sounds like overkill. File servers of the size you're
> talking about are generally well underutilized. I've seen VMWare
> servers with 4GB of RAM running 10 Windows file servers serving more
> users per server than you are looking for. If money is no object, but
> all means go for it. If you're looking to size your server to your
> workload you could probably save yourself several thousand dollars by
> going with a lower end box with less RAM.
Point well taken. What tools are available for a Windows environment to
help us profile what amount of the file system is actually being read over
the course of a week?
--
Will
| |
|
| <robertwessel2@yahoo.com> wrote in message
news:1156674534.173513.147570@i42g2000cwa.googlegroups.com...
> If you've correctly characterized your workload, it's not an
> unreasonable plan, except I didn't think MS has done a 64 bit WS2003
> Web Server Edition yet, and second, the Web Server Edition is *not*
> licensed for local file serving.
It looks like you are correct. What a pity. Not very good marketing on
Microsoft's part. I would probably try to get more life out of a Windows
2000 box than force an expensive software upgrade.
--
Will
| |
| Jon Metzger 2006-08-29, 1:18 pm |
| Will wrote:
> "Jon Metzger" <blagh@___gmail.com> wrote in message
> news:ecut56$6j0$1@lenny.tc.umn.edu...
>
> Point well taken. What tools are available for a Windows environment to
> help us profile what amount of the file system is actually being read over
> the course of a week?
>
Check out Perfmon, it's built in to Windows. It may be a little tricky
to set up, but all the information you'll need you should be able to
find there.
| |
|
| "Jon Metzger" <blagh@___gmail.com> wrote in message
news:ed1bl1$lk0$1@lenny.tc.umn.edu...
> Check out Perfmon, it's built in to Windows. It may be a little tricky
> to set up, but all the information you'll need you should be able to
> find there.
I don't think Perfmon would track anything more than low level performance
characteristics like number of reads and writes on a logical or physical
volume.
How is that going to help me determine that 95% of the reads are on xx GB of
the file system?
--
Will
| |
| Jon Metzger 2006-08-30, 1:14 pm |
| Will wrote:
> "Jon Metzger" <blagh@___gmail.com> wrote in message
> news:ed1bl1$lk0$1@lenny.tc.umn.edu...
>
> I don't think Perfmon would track anything more than low level performance
> characteristics like number of reads and writes on a logical or physical
> volume.
>
> How is that going to help me determine that 95% of the reads are on xx GB of
> the file system?
>
Sorry, I misread your original question...PerfMon probably won't get you
that level of detail. I'm guessing you're wanting to know this
statistic in order to size your RAM so that all of this "most read" data
will remain in filesystem cache. I think you may be over-engineering
your solution. I've seen people insist upon 15K rpm SCSI disks for
fileservers that have since been migrated to ATA without anyone knowing
the difference.
With that said, having hard data to plan with is always better than
guessing. I'd watch PerfMon for reads and writes per second and KB
transferred and purchase disks which can handle that load plus some
headroom for growth. The filesystem cache is all well and good, but
it's a good idea to plan for your disk to handle the load without it.
Then any performance gain you get from cache is a bonus. I think that's
the safest approach to your problem, and it should be less expensive
than your original plan to boot.
| |
| Moojit 2006-08-31, 7:20 pm |
| I think you may be focusing on the wrong areas. DISK performance and NIC
performance should be the priority, less on 64bit. If you get a NIC that
has it's own TOE, 64 bit and processor become less important. For the
DISKS, a RAID controller that supports WRITE BACK cache in addition to READ
cache will be optimal.
Da Moojit
"Will" <westes-usc@noemail.nospam> wrote in message
news:6pKdnWwmUPdKJ23ZnZ2dnUVZ_s6dnZ2d@gi
ganews.com...
> We are migrating shared files off of a Windows domain controller to a
> discrete file server, and I'm thinking through what would be the best
> design
> for that new server. Given that 64 bit operating systems are now here,
> I'm thinking we do not need to spend much on fast drives, but we should
> instead invest in the lowest cost 64 bit server (a file server won't
> bottleneck on CPU so even a 1 GHz machine would be fast enough) and
> install
> gigabit ethernet and lots of memory. I'm sure that the most requested
> files would fit into a memory cache that is under 10 GB in size. A
> computer with 12 GB of memory, 64 bit Windows 2003 server web edition, and
> gigabit ethernet should provide the best possible speed for the case where
> there is not much write activity, but lots of read activity on less than
> 10
> GB of data.
>
> Assume less than 20 users, heavy read activity, very low write activity.
> Disk with all file shares would be under 100 GB, but under 10 GB of that
> represents 95% of the activity.
>
> Is my design correct if I want to maximize performance for this small
> network?
>
> --
> Will
>
>
| |
|
| "Moojit" <moojit@hotmail.com> wrote in message
news:yDJJg.9121$dl.4187@tornado.texas.rr.com...
> I think you may be focusing on the wrong areas. DISK performance and NIC
> performance should be the priority, less on 64bit. If you get a NIC that
> has it's own TOE, 64 bit and processor become less important. For the
> DISKS, a RAID controller that supports WRITE BACK cache in addition to
READ
> cache will be optimal.
64 bit is required to get support for more than 4 GB of memory. It's the
extra memory I want, to cache the most commonly used data for fast reads,
not the extra processing capabilities of 64 bit CPUs.
The application does not involve heavy write activity, so I don't think we
would benefit much from battery backed write cache.
--
Will
> "Will" <westes-usc@noemail.nospam> wrote in message
> news:6pKdnWwmUPdKJ23ZnZ2dnUVZ_s6dnZ2d@gi
ganews.com...
here,[vbcol=seagreen]
requested[vbcol=seagreen]
and[vbcol=seagreen]
where[vbcol=seagreen]
>
>
| |
| Bill Todd 2006-09-01, 1:18 am |
| Will wrote:
> "Moojit" <moojit@hotmail.com> wrote in message
> news:yDJJg.9121$dl.4187@tornado.texas.rr.com...
> READ
>
> 64 bit is required to get support for more than 4 GB of memory.
Actually, it's not: Intel's IA32 processors (at least in their server
lines) have supported up to 64 GB of RAM since 1996.
I don't think that standard 32-bit Windows file systems can use the
additional RAM for caching, however: you might check to see whether
some file system on Linux or *BSD does if you're interested.
- bill
| |
|
| "Bill Todd" <billtodd@metrocast.net> wrote in message
news:dvmdnUQAqMVpOWrZnZ2dnUVZ_rGdnZ2d@me
trocastcablevision.com...
>
> Actually, it's not: Intel's IA32 processors (at least in their server
> lines) have supported up to 64 GB of RAM since 1996.
Unfortunately, since it is a Windows based file server, what would matter
would be the additional limitations of the Microsoft OS and not the
hardware. For 32-bit Windows Standard Edition server, the maximum memory
utilization appears to be 4 GB. The 64-bit Windows Standard Edition jumps
that to 32 GB. I need about 12 GB, therefore it's 64 bit or nothing.
> I don't think that standard 32-bit Windows file systems can use the
> additional RAM for caching, however: you might check to see whether
> some file system on Linux or *BSD does if you're interested.
I was planning to use Veritas Storage Foundation which has a feature Vcache
built in that lets you specify exact amounts of cache for each logical
volume. I guess I should not take for granted that it will not max out
before 12 GB. If I can't cache the file system, then the whole approach
I'm taking is pointless.
--
Will
| |
| Curtis Preston 2006-09-01, 1:18 am |
| As long as you're sticking with Windows as your OS, I would definitely agree
with the previous comment about investing in a TCP Offload Engine (TOE)
card. There are a few of them out there, but the one that's been around the
longest is Alacritech.
I'm not sure I agree with your comment that you can't get CPU bound with
file sharing, since I've seen plenty of really high-end file servers that
cost hundreds of thousands of dollars get CPU-bound. But if I add "for
twenty users" to the end of your sentence, then I'd agree with it. I'm
guessing that's what you meant. Just wanted to clarify the point for anyone
else who might be reading along.
On 8/31/06, Will <westes-usc@noemail.nospam> wrote:
>
> "Bill Todd" <billtodd@metrocast.net> wrote in message
> news:dvmdnUQAqMVpOWrZnZ2dnUVZ_rGdnZ2d@me
trocastcablevision.com...
>
> Unfortunately, since it is a Windows based file server, what would matter
> would be the additional limitations of the Microsoft OS and not the
> hardware. For 32-bit Windows Standard Edition server, the maximum memory
> utilization appears to be 4 GB. The 64-bit Windows Standard Edition
> jumps
> that to 32 GB. I need about 12 GB, therefore it's 64 bit or nothing.
>
>
>
>
> I was planning to use Veritas Storage Foundation which has a feature
> Vcache
> built in that lets you specify exact amounts of cache for each logical
> volume. I guess I should not take for granted that it will not max out
> before 12 GB. If I can't cache the file system, then the whole approach
> I'm taking is pointless.
>
> --
> Will
>
>
>
| |
|
| "Curtis Preston" <backupmeister@gmail.com> wrote in message
news:mailman.0.1157089107.29532.test2_wcurtispreston.com@wcurtispreston.com...
> As long as you're sticking with Windows as your OS, I would definitely
agree
> with the previous comment about investing in a TCP Offload Engine (TOE)
> card. There are a few of them out there, but the one that's been around
the
> longest is Alacritech.
This works as the ethernet card, or does it supplement the ethernet card
already in the system? All I can remember about TCP offloading is that
when I enabled that feature on the old Compaq NC6134 fiber optic ethernet
cards, every TCP packet was showing checksum errors in a sniffer. So much
for automation.  I ended up turning off TCP offloading and it seemed to
work without errors after that.
> I'm not sure I agree with your comment that you can't get CPU bound with
> file sharing, since I've seen plenty of really high-end file servers that
> cost hundreds of thousands of dollars get CPU-bound. But if I add "for
> twenty users" to the end of your sentence, then I'd agree with it. I'm
> guessing that's what you meant. Just wanted to clarify the point for
anyone
> else who might be reading along.
Just curious were those CPU bound servers Solaris boxes that were spawning
processes for each connected user? If the I/O is asynchronous and you
limit the number of threads and processes that are being context-switched,
you should bottleneck on I/O (file, network, or both) before you hit any CPU
limits. If you give the CPU hundreds of spawning processes, then of
course you are going to overwhelm it at some point just in context switches.
Fortunately, Windows server uses asynchronous file I/O.
--
Will
| |
| Curtis Preston 2006-09-01, 7:19 am |
| A TOE card replaces the onboard NIC. I think you'll find things have
been approved.
with[vbcol=seagreen]
that[vbcol=seagreen]
"for[vbcol=seagreen]
I'm[vbcol=seagreen]
anyone[vbcol=seagreen]
[vbcol=seagreen]
>Just curious were those CPU bound servers Solaris boxes that were
spawning
>processes for each connected user? If the I/O is asynchronous and
you
>limit the number of threads and processes that are being
context-switched,
>you should bottleneck on I/O (file, network, or both) before you hit
any >CPU limits. If you give the CPU hundreds of spawning processes,
then of
>course you are going to overwhelm it at some point just in context
switches.
>Fortunately, Windows server uses asynchronous file I/O.
Be careful, you could start a religious war with your comments. ;)
But as long as you brought it up... ;)
These were NAS filers that were specifically designed for file sharing
and nothing else, not Solaris boxes. Throw a few thousand users at
anything and it's bound to run out of room. Actually, I'd say that most
of the CPU in these cases is eaten up with making GbE frames, hence my
comment on TOEs.
As to OSs, I'd pit a Linux box running Samba against the exact same
hardware running Windows any day of the week. Add to that comment that
I don't know a single high-end NAS vendor that uses Windows as its base
OS. (Maybe because you don't have to reboot it all the time to install
the latest security patch.)
| |
| Hans Jørgen Jakobsen 2006-09-01, 1:16 pm |
| On Thu, 31 Aug 2006 23:52:28 -0700, Will wrote:
> "Curtis Preston" <backupmeister@gmail.com> wrote in message
> news:mailman.0.1157089107.29532.test2_wcurtispreston.com@wcurtispreston.com...
> agree
> the
>
> This works as the ethernet card, or does it supplement the ethernet card
> already in the system? All I can remember about TCP offloading is that
> when I enabled that feature on the old Compaq NC6134 fiber optic ethernet
> cards, every TCP packet was showing checksum errors in a sniffer. So much
> for automation. I ended up turning off TCP offloading and it seemed to
> work without errors after that.
>
Was the sniffing done on the sending machine?
On a machine with TOE you will see errors on outgoing packets because the
checksum will first be calculated when the packet hits the card. And the
sniffer (tcpdump) takes a copy before.
/hjj
| |
|
| "Hans Jørgen Jakobsen" <hjj@wheel.dk> wrote in message
news:slrnefgc2s.2pcm.hjj@freesbee.wheel.dk...
> Was the sniffing done on the sending machine?
> On a machine with TOE you will see errors on outgoing packets because the
> checksum will first be calculated when the packet hits the card. And the
> sniffer (tcpdump) takes a copy before.
The errors were on receiving, but regarding the sending why would the
sniffer see a different packet than the card does? The sniffer is on the
machine that has the TOE card, and it uses the TOE card to see the packet.
No doubt I don't understand the concepts here well enough.
--
Will
| |
| Moojit 2006-09-02, 1:14 am |
|
"Will" <westes-usc@noemail.nospam> wrote in message
news:iKGdnV0dcPk2EmrZnZ2dnUVZ_tqdnZ2d@gi
ganews.com...
> "Moojit" <moojit@hotmail.com> wrote in message
> news:yDJJg.9121$dl.4187@tornado.texas.rr.com...
> READ
>
> 64 bit is required to get support for more than 4 GB of memory. It's the
> extra memory I want, to cache the most commonly used data for fast reads,
> not the extra processing capabilities of 64 bit CPUs.
Actually, this is not true. A couple things to consider ...
1. You can enable the extended address memory support on a IA32 processor
by using the /PAE switch in the boot.ini file (processor switches to a 36
bit address bus vs. 32 bit address bus). This will get you over the 4GB
boundary.
2. The Windows kernel allocates 2GB of memory maximum per user mode process
(most of this vitualized memory resides in paging file).
3. Applications can be customized to go beyond this 2GB boundary using
special Win32 API calls. Enterprise class database applications such as
Oracle and SQL take advantage of this. But in general, most applications
do not.
Da Moojit
>
> The application does not involve heavy write activity, so I don't think we
> would benefit much from battery backed write cache.
>
> --
> Will
>
>
> here,
> requested
> and
> where
>
>
| |
|
| "Moojit" <moojit@hotmail.com> wrote in message
news:ds6Kg.11159$dl.10650@tornado.texas.rr.com...
the[vbcol=seagreen]
reads,[vbcol=seagreen]
>
> Actually, this is not true. A couple things to consider ...
>
> 1. You can enable the extended address memory support on a IA32 processor
> by using the /PAE switch in the boot.ini file (processor switches to a 36
> bit address bus vs. 32 bit address bus). This will get you over the 4GB
> boundary.
> 2. The Windows kernel allocates 2GB of memory maximum per user mode
process
> (most of this vitualized memory resides in paging file).
> 3. Applications can be customized to go beyond this 2GB boundary using
> special Win32 API calls. Enterprise class database applications such as
> Oracle and SQL take advantage of this. But in general, most applications
> do not.
So let's distinguish empirical from theoretical.
Empirically, if I want to budget for Windows Server 2003 Standard Edition,
and my application is a file server using the built in Windows protocols,
then I need a 64 bit edition in order to get more than 4 GB.
Empirically, Microsoft did not build the 32 bit versions of Standard Edition
to use more than 4 GB for file sharing. They could have. They did not.
--
Will
|
|
|
|
|