Data Storage - IOPS calculation for SAN Design

Author

IOPS calculation for SAN Design

hw@hwi-networksecurity.de

2007-10-06, 1:14 pm

Hi, we want to establish a SAN in near future and therefore we
calculate the technical basics at the moment. First of all we are
interested in disk performance, particularly with regard to iops and
throughput. The capacity planer from VMware delivers the average
values of PhysicalDisk\trans, PhysicalDisk\Bytes/s per server and
overall. According to these results, we have only a few iops (peak
load 162) with 20 servers. In this analysis I have no declaration of
the peak value per server, so I tried to find it out with the
Microsoft perfmon. Here I get some strange values. I logged the disk
performance with perfmon during one day. Perfmon shows a peak value of
PhysicalDisk\trans/s of over 2000 (iops) at a server witch two SATA
disks (Raid-1). The technical datasheet of these disks contain a
typical value of 85 iops. Can this be true? How can I get realistic
peak values of really existing iops to make a realistic design?

Thanks for help,

Holger

moojit

2007-10-06, 7:13 pm

DataMover for Linux and Windows will provide this data for you. The
application performs an automated performance test across 14 different
tranfer sizes and records throughput and IOPS for each transfer size along
with standard deviation values for each of these data points.

Contact the Moojit for a 30 day evaluation license.

www.moojit.net

<hw@hwi-networksecurity.de> wrote in message
news:1191675785.873476.283950@57g2000hsv.googlegroups.com...
> Hi, we want to establish a SAN in near future and therefore we
> calculate the technical basics at the moment. First of all we are
> interested in disk performance, particularly with regard to iops and
> throughput. The capacity planer from VMware delivers the average
> values of PhysicalDisk\trans, PhysicalDisk\Bytes/s per server and
> overall. According to these results, we have only a few iops (peak
> load 162) with 20 servers. In this analysis I have no declaration of
> the peak value per server, so I tried to find it out with the
> Microsoft perfmon. Here I get some strange values. I logged the disk
> performance with perfmon during one day. Perfmon shows a peak value of
> PhysicalDisk\trans/s of over 2000 (iops) at a server witch two SATA
> disks (Raid-1). The technical datasheet of these disks contain a
> typical value of 85 iops. Can this be true? How can I get realistic
> peak values of really existing iops to make a realistic design?
>
> Thanks for help,
>
> Holger
>

Faeandar

2007-10-09, 1:14 am

On Sat, 06 Oct 2007 06:03:05 -0700, hw@hwi-networksecurity.de wrote:

>Hi, we want to establish a SAN in near future and therefore we
>calculate the technical basics at the moment. First of all we are
>interested in disk performance, particularly with regard to iops and
>throughput. The capacity planer from VMware delivers the average
>values of PhysicalDisk\trans, PhysicalDisk\Bytes/s per server and
>overall. According to these results, we have only a few iops (peak
>load 162) with 20 servers. In this analysis I have no declaration of
>the peak value per server, so I tried to find it out with the
>Microsoft perfmon. Here I get some strange values. I logged the disk
>performance with perfmon during one day. Perfmon shows a peak value of
>PhysicalDisk\trans/s of over 2000 (iops) at a server witch two SATA
>disks (Raid-1). The technical datasheet of these disks contain a
>typical value of 85 iops. Can this be true? How can I get realistic
>peak values of really existing iops to make a realistic design?
>
>Thanks for help,
>
>Holger

Your stats are showing cache ops, not disk. If you use hardware raid
the controller will have some amount of cache or even if you are using
a single drive the OS will do write caching to memory unless you
specifically mount the file system with forcedirectio (unix only, not
sure what Windows uses).

The thing about cache is that, no matter how much you have, you can
only delay the inevitable if you have alot of IO. For instance, if
you bottleneck on IO with 10G of cache after 30 seconds, adding
another 10G of cache will only delay the bottleneck for 1 minute or
so. If you are IOP intensive then you need to look at disk IOPS and
cache will just be gravy.

Disk IOPS are generally 80 for SATA and 120 for FC. Use those numbers
to determine your requirements and you won't get bottlenecked.

Something to keep in mind too, IOPS and throughput are not directly
related. If you have 10 SATA drives you can get ~800 IOPS total, but
if your data is large sequential you could get 200MB/sec of
throughput.

~F

alexup

2007-10-19, 1:15 am

On 6 oct, 08:03, h...@hwi-networksecurity.de wrote:
> Hi, we want to establish a SAN in near future and therefore we
> calculate the technical basics at the moment. First of all we are
> interested in disk performance, particularly with regard to iops and
> throughput. The capacity planer from VMware delivers the average
> values of PhysicalDisk\trans, PhysicalDisk\Bytes/s per server and
> overall. According to these results, we have only a few iops (peak
> load 162) with 20 servers. In this analysis I have no declaration of
> the peak value per server, so I tried to find it out with the
> Microsoft perfmon. Here I get some strange values. I logged the disk
> performance with perfmon during one day. Perfmon shows a peak value of
> PhysicalDisk\trans/s of over 2000 (iops) at a server witch two SATA
> disks (Raid-1). The technical datasheet of these disks contain a
> typical value of 85 iops. Can this be true? How can I get realistic
> peak values of really existing iops to make a realistic design?
>
> Thanks for help,
>
> Holger

Hi Holger,

First, and it's a rule, create your sizing at 70% of you peak, that's
mean, if you have 2000 IOPS in your peak IO you must to configure your
storage thinking in 2860 IOPS because it's a rule of the thumb, so,
how many disks you need, depends, First you must consider, a physical
disk can operate in average between 130 and 150 IOPS per disk, so, if
you have 2860 IOPS you must to install 22 HDD aprox. and then you can
obtain this 2860 IOPS. and depends of RAID level that you're using is
the USABLE space that you can obtain, so here's another variable, cost
per transaction. becuase if you are using RAID 5 you obtain more
USABLE space than RAID 1+0.

I hope that's information is useful for your.

regards.

victor.engle@gmail.com

2007-10-21, 1:13 pm

Don't under estimate the value of cache. I believe if you have an
intelligent raid controller using cache effectively you're backend
disks won't need to meet the peak iops you measured in many
situations.

For example, are the 2000 iops a mixture of reads and writes. Read
cache hits would require no physical disk operation. I have seen a
hitachi 9980 serving storage for a multi terabyte transaction
processing database maintain a very good cache hit rate.

Also, when a raid controller has write cache it can stage writes to
physical media efficiently so that 2000 iops from the host's view
would result in something potentially much smaller on the array's
backend disks.

Regards,
Vic

On Oct 18, 9:58 pm, alexup <ale...@mexcrafts.com> wrote:
> On 6 oct, 08:03, h...@hwi-networksecurity.de wrote:
>
>
>
>
>
>
> Hi Holger,
>
> First, and it's a rule, create your sizing at 70% of you peak, that's
> mean, if you have 2000 IOPS in your peak IO you must to configure your
> storage thinking in 2860 IOPS because it's a rule of the thumb, so,
> how many disks you need, depends, First you must consider, a physical
> disk can operate in average between 130 and 150 IOPS per disk, so, if
> you have 2860 IOPS you must to install 22 HDD aprox. and then you can
> obtain this 2860 IOPS. and depends of RAID level that you're using is
> the USABLE space that you can obtain, so here's another variable, cost
> per transaction. becuase if you are using RAID 5 you obtain more
> USABLE space than RAID 1+0.
>
> I hope that's information is useful for your.
>
> regards.