Hi,

I am trying to find out optimum amount of RAM for a Linux based application
server.

Details:
* RH Advanced Server 3.x (kernel 2.4.21, but with 2.5 and 2.6 features
added).
* Dual Xeon 2.8GHz 512 L2 Cache
* xGB RAM
* 300 GB RAID-5.  2GB CACHE on controller. (external SAN box via 2 GB fiber)
* Ext3 file system
* Cobol-based application (using ISAM database). Written in AcuCobol.
* About 150 concurrent users via telnet & ssh.
* Will also be Samba server for about 50 users (mostly light usage).

2GB of RAM meets and exceeds the requirements for the application (with 150
users), but I am considering 4GB of RAM instead.

My question is:  With 4GB instead (2GB more of RAM available for file system
caching), am I likely to get a noticable performance increase?  How about
added risk of dataloss due to more data not yet written to disk at the time
of a system crash?

Any advice on this would be appreciated.

Thanks in advance,

Lars

[ Post a follow-up to this message ]

Lars Oberg wrote:
> Hi,
>
> I am trying to find out optimum amount of RAM for a Linux based
> application server.
>
> Details: * RH Advanced Server 3.x (kernel 2.4.21, but with 2.5 and
> 2.6 features added). * Dual Xeon 2.8GHz 512 L2 Cache * xGB RAM * 300
> GB RAID-5.  2GB CACHE on controller. (external SAN box via 2 GB
> fiber) * Ext3 file system * Cobol-based application (using ISAM
> database). Written in AcuCobol. * About 150 concurrent users via
> telnet & ssh. * Will also be Samba server for about 50 users (mostly
> light usage).
>
> 2GB of RAM meets and exceeds the requirements for the application
> (with 150 users), but I am considering 4GB of RAM instead.
>
> My question is:  With 4GB instead (2GB more of RAM available for file
> system caching), am I likely to get a noticable performance increase?
> How about added risk of dataloss due to more data not yet written to
> disk at the time of a system crash?
>
> Any advice on this would be appreciated.
>
If you are using a dbms, it should be running the hard drives raw, so it
will not consider a transaction comitted until it gets into the disk
drive's cache at the soonest, and that should be pretty safe since the
dbms is doing the buffering, not the OS. And it will have written the
transaction to the journal file before it even writes it to the database
disk, so it can restore if a crash happens at an inopportune time. It
seems to me that Linux tries to flush the buffers at least every 5
seconds, so that might be a clue (if I am correct) as to about how much
data you are likely to lose.

If you are not using a dbms, then you are less certain, that's all. I do
not think you can be certain (even by calling sync() after every
write(), but that would be crazy) ever. Reducing the amount of real RAM
will never help much because it might do some paging instead of forcing
buffers out.

The more you try to outsmart an OS, the more it will outsmart you.
[one of those old programming axioms]

--
.~.  Jean-David Beyer           Registered Linux User 85642.
/V\                             Registered Machine    73926.
/( )\ Shrewsbury, New Jersey     http://counter.li.org
^^-^^ 5:40pm up 46 days, 5:01, 2 users, load average: 2.25, 2.20, 2.12

[ Post a follow-up to this message ]

"Jean-David Beyer" <jdbeyer@exit109.com> wrote in message
news:4037E048.1010400@exit109.com...
>
> If you are not using a dbms, then you are less certain, that's all. I do
> not think you can be certain (even by calling sync() after every
> write(), but that would be crazy) ever. Reducing the amount of real RAM
> will never help much because it might do some paging instead of forcing
> buffers out.
>
> The more you try to outsmart an OS, the more it will outsmart you.
> [one of those old programming axioms]
>

Thanks for the advice.

The system is not using a dbms.  It is a Cobol app reading & writing ISAM
database files via its "Vision File System" (which I believe is a virtual
file system that lies between the application and the OS file system
[ext3]).

The application is mostly Point-Of-Sale, inventory and accounting related
(over telnet & ssh sessions).

I know it is really hard to answer this question, but is it likely (under
the above circumstances) to give a noticable performance boost to double the
RAM (from 2GB to 4GB) in the server?

Thanks,

Lars

[ Post a follow-up to this message ]

"Lars Oberg" <n-o-sp-am-lars_oberg@hotmail.com> wrote in message
news:TlXZb.28714$Rn2.23896@newssvr25.news.prodigy.com...
>
> The application is mostly Point-Of-Sale, inventory and accounting related
> (over telnet & ssh sessions).
>
> I know it is really hard to answer this question, but is it likely (under
> the above circumstances) to give a noticable performance boost to double
the
> RAM (from 2GB to 4GB) in the server?

You can get a pretty good idea by observing the machine under
a typical load.  Run 'top' and look at the free memory and swap
usage sections.   If you are using a substantial amount of swap
space adding RAM will help greatly.   If you have a large amount
of 'free' memory now, adding more RAM won't help.  If neither
value is large, more RAM may help but it probably won't be
dramatic.

---
Les Mikesell
lesmikesell@comcast.net

[ Post a follow-up to this message ]

Les Mikesell wrote:
> "Lars Oberg" <n-o-sp-am-lars_oberg@hotmail.com> wrote in message
> news:TlXZb.28714$Rn2.23896@newssvr25.news.prodigy.com...
> 
>
> the
> 
>
>
> You can get a pretty good idea by observing the machine under a
> typical load.  Run 'top' and look at the free memory and swap usage
> sections.   If you are using a substantial amount of swap space
> adding RAM will help greatly.   If you have a large amount of 'free'
> memory now, adding more RAM won't help.  If neither value is large,
> more RAM may help but it probably won't be dramatic.
>
I find it more fun to run xosview and watch the paging, io rates, etc.
If you find it is doing more than a page or two per second, you should
start to consider getting more memory. If you are compute-limited, you
should consider more or faster CPUs. If you are IO limited, get faster
peripherals. If none of above and too slow, see if you are seeking too
much (my current machine does this).

If you seek too much, see if you can get more hard drives. That is what
I am doing with my new machine (just finished, but not yet smoke tested
even). I have 4 small 10,000rpm Ultra/320 SCSI hard drives on there. I
could easily have gotten all that onto a single hard drive, but what I
wanted was more spindles so that no one drive would have to seek as
much. If I lay out the data correctly, that is. I can tell my present
system is seek-limited (though not memory thrashing) because the job
runs too slowly but is no where near consuming all the CPU time, but it
is not transferring to disk anywhere near its maximum measured transfer
rate either. One job takes about 9 hours, runs the CPUs around 5% each,
transfers only about 1 megabyte/sec to disk even though the disks are
considerably faster (this is on my current, not my new faster, machine):

/dev/sda:
geometry     = 8759/64/32, sectors = 17938986, start = 0
Timing buffer-cache reads:   128 MB in  1.04 seconds =123.08 MB/sec
Timing buffered disk reads:  64 MB in  2.61 seconds = 24.52 MB/sec

/dev/sdb:
geometry     = 8759/64/32, sectors = 17938986, start = 0
Timing buffer-cache reads:   128 MB in  1.17 seconds =109.40 MB/sec
Timing buffered disk reads:  64 MB in  2.60 seconds = 24.62 MB/sec
valinux:root[/home/jdbeyer]#

So if it is not waiting for CPU time (two 550MHz Pentium IIIs), not
thrashing memory (512MBytes), not transferring data to or from the
hardrives very fast (4% of capacity), what is it waiting for? Easy, just
listen to the machine. It seeks so fast and so often (when running that
job) that you cannot hear the individual clicks the drive makes when it
seeks.


If you do not have xosview and do not want to bother getting it
(requires X Window System), vmstat will get you the same kind of data,
but the presentation is not as fun to watch.



--
.~.  Jean-David Beyer           Registered Linux User 85642.
/V\                             Registered Machine    73926.
/( )\ Shrewsbury, New Jersey     http://counter.li.org
^^-^^ 6:15pm up 47 days, 5:36, 4 users, load average: 2.20, 2.49, 3.14

[ Post a follow-up to this message ]

Thanks guys for the advice!

/ Lars

"Jean-David Beyer" <jdbeyer@exit109.com> wrote in message
news:40393C6D.30506@exit109.com...
> Les Mikesell wrote: 
> I find it more fun to run xosview and watch the paging, io rates, etc.
> If you find it is doing more than a page or two per second, you should
> start to consider getting more memory. If you are compute-limited, you
> should consider more or faster CPUs. If you are IO limited, get faster
> peripherals. If none of above and too slow, see if you are seeking too
> much (my current machine does this).
>
> If you seek too much, see if you can get more hard drives. That is what
> I am doing with my new machine (just finished, but not yet smoke tested
> even). I have 4 small 10,000rpm Ultra/320 SCSI hard drives on there. I
> could easily have gotten all that onto a single hard drive, but what I
> wanted was more spindles so that no one drive would have to seek as
> much. If I lay out the data correctly, that is. I can tell my present
> system is seek-limited (though not memory thrashing) because the job
> runs too slowly but is no where near consuming all the CPU time, but it
> is not transferring to disk anywhere near its maximum measured transfer
> rate either. One job takes about 9 hours, runs the CPUs around 5% each,
> transfers only about 1 megabyte/sec to disk even though the disks are
> considerably faster (this is on my current, not my new faster, machine):
>
> /dev/sda:
>   geometry     = 8759/64/32, sectors = 17938986, start = 0
>   Timing buffer-cache reads:   128 MB in  1.04 seconds =123.08 MB/sec
>   Timing buffered disk reads:  64 MB in  2.61 seconds = 24.52 MB/sec
>
> /dev/sdb:
>   geometry     = 8759/64/32, sectors = 17938986, start = 0
>   Timing buffer-cache reads:   128 MB in  1.17 seconds =109.40 MB/sec
>   Timing buffered disk reads:  64 MB in  2.60 seconds = 24.62 MB/sec
> valinux:root[/home/jdbeyer]#
>
> So if it is not waiting for CPU time (two 550MHz Pentium IIIs), not
> thrashing memory (512MBytes), not transferring data to or from the
> hardrives very fast (4% of capacity), what is it waiting for? Easy, just
> listen to the machine. It seeks so fast and so often (when running that
> job) that you cannot hear the individual clicks the drive makes when it
> seeks.
>
>
> If you do not have xosview and do not want to bother getting it
> (requires X Window System), vmstat will get you the same kind of data,
> but the presentation is not as fun to watch.
>
>
>
> --
>    .~.  Jean-David Beyer           Registered Linux User 85642.
>    /V\                             Registered Machine    73926.
>   /( )\ Shrewsbury, New Jersey     http://counter.li.org
>   ^^-^^ 6:15pm up 47 days, 5:36, 4 users, load average: 2.20, 2.49, 3.14
>

[ Post a follow-up to this message ]

On Sun, 22 Feb 2004 19:30:35 +0000, Les Mikesell wrote:

> "Lars Oberg" <n-o-sp-am-lars_oberg@hotmail.com> wrote in message
> news:TlXZb.28714$Rn2.23896@newssvr25.news.prodigy.com... 

memory needed by applications + sum size of frequently accessed files + a
little extra.

If you have several GiB of data for the application (eg images of every
product in the database could make it grow this big) then I'd go for the
memory.

If the database is 100MiB and unlikly to grow a lot then I'd leave it as
it is.


--
NPV

"the large print giveth, and the small print taketh away"
Tom Waits - Step right up

[ Post a follow-up to this message ]