|
Home > Archive > Data Storage > May 2006 > huge box of ram?
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
|
|
| Paul Rubin 2006-04-27, 7:13 pm |
| Is there such a thing as a relatively inexpensive box (up to a few K$,
say) that can hold a huge amount of ram, like 200-1000 GB? It doesn't
have to be battery backed or anything like that, but it should have
ECC or at least parity. The application is a high traffic database
cache.
| |
| Al Dykes 2006-04-27, 7:13 pm |
| In article <7x8xpqokbq.fsf_-_@ruckus.brouhaha.com>,
Paul Rubin <http://phr.cx@NOSPAM.invalid> wrote:
>Is there such a thing as a relatively inexpensive box (up to a few K$,
>say) that can hold a huge amount of ram, like 200-1000 GB? It doesn't
>have to be battery backed or anything like that, but it should have
>ECC or at least parity. The application is a high traffic database
>cache.
Google "solid state disk" and you might want to add "-CF" to avoid all
the people that want to sell you CF cards.
I'd figure 100 bucks/GB as a WAG for price. How much money do you have?
--
a d y k e s @ p a n i x . c o m
Don't blame me. I voted for Gore.
| |
| Paul Rubin 2006-04-27, 7:14 pm |
| adykes@panix.com (Al Dykes) writes:
> I'd figure 100 bucks/GB as a WAG for price. How much money do you have?
That amount of bucks/GB is reasonable. A $50K expenditure for this
application might not be out of line, if it resulted in getting 400GB
of ram ($10K for the box and 40k for the ram). All the "solid state
disk" applicances on that scale I know of cost a heck of a lot more
than that.
| |
| Paul Rubin 2006-04-28, 7:12 am |
| Zak <jute@zak.invalid> writes:
> I'd think $50K is way too low. You are talking a specialty product
> (what interface does it use...).
Just about anything--PCIe, Infiniband, heck, even USB2 or Firewire.
> Also the commecial life is short - it will have to move along with
> available DRAMs.
Is that a big deal? What is so high tech about this? It's just a
bunch of dimm sockets on a board, right? Giga-byte makes a PCI
ramdisk with four dimm sockets for $50, but it can only take 1GB
dimms. I just want something like that but a lot bigger, maybe
multiple boards, and able to use 2gb or 4gb dimms. Let's say a 1U
rack box containing a board with 32 or 64 dimm sockets. A few of
those with 2gb dimms (around $140/GB right now with ECC) and I'm
there.
> Can't you use a bunch of PC's? Even then you have limited choice
> because of the ECC/parity requirement. Though that could be done in
> software of course (modulo the small chance the checking software is
> going to be hit).
What PC's exist that can take more than 16GB? I'd have to use dozens
of them to get 500 GB of ram.
I also notice that flash memory in the form of SD cards is now around
$25/GB in 4GB cards:
http://www.zipzoomfly.com/jsp/Produ...ctCode=82502-19
I wonder if there's some way of sticking 128 or so of those in a box.
Thanks.
| |
|
| Paul Rubin wrote:
>
> Just about anything--PCIe, Infiniband, heck, even USB2 or Firewire.
For you that's okay, for others maybe not. That divides up the market.
>
> Is that a big deal? What is so high tech about this?
It means the manufacturer can sell it for a few years at most, then has
to change the design and so on. Doesn't make things cheap.
> It's just a
> bunch of dimm sockets on a board, right? Giga-byte makes a PCI
> ramdisk with four dimm sockets for $50, but it can only take 1GB
> dimms. I just want something like that but a lot bigger, maybe
> multiple boards, and able to use 2gb or 4gb dimms. Let's say a 1U
> rack box containing a board with 32 or 64 dimm sockets. A few of
> those with 2gb dimms (around $140/GB right now with ECC) and I'm
> there.
Where do you buy cheap 2 or 4 GB dimms? Not a commodity product either.
And it is not just sockets - it is drivers too. Running everything at
lower speed may help. But again this needs research.
>
> What PC's exist that can take more than 16GB? I'd have to use dozens
> of them to get 500 GB of ram.
Yes. And PC's are cheap compared to a custom design for a very narrow
market.
> I also notice that flash memory in the form of SD cards is now around
> $25/GB in 4GB cards:
>
> http://www.zipzoomfly.com/jsp/Produ...ctCode=82502-19
>
> I wonder if there's some way of sticking 128 or so of those in a box.
There is (look up USB card reader). Then go and look at the speed and
durability. And look at your ECC requirement again. It is cheap, yes. But...
Thomas
| |
| Al Dykes 2006-04-28, 1:12 pm |
| In article <44521782$0$6423$2e0edba0@news.tweakdsl.nl>,
Zak <jute@zak.invalid> wrote:
>Paul Rubin wrote:
>
>
>For you that's okay, for others maybe not. That divides up the market.
>
>
>It means the manufacturer can sell it for a few years at most, then has
>to change the design and so on. Doesn't make things cheap.
I don't see anyone buying a big RAMdisk without ECC and internal UPS
and it's not going to have a consumer plug on it; GBe and better these
days.
I'll bet that a 10k RPM 200GB SATA disk ($200) would give faster
throughput than 200GB (at least $20,000) of ram at the end of a USB2
cable.
--
a d y k e s @ p a n i x . c o m
Don't blame me. I voted for Gore.
| |
| Paul Rubin 2006-04-28, 1:12 pm |
| Zak <jute@zak.invalid> writes:
>
> It means the manufacturer can sell it for a few years at most, then
> has to change the design and so on.
But that's true of just about any technology product. This thing
should have no more complexity than a microcomputer memory board of
the CP/M era, which (unpopulated) cost a few hundred bucks, and those
things were made in fairly small quantity.
Or even simpler, is there such a thing as a PCI or AGP board with a
bunch of dimm slots? I only know of the Giga-byte one, which has just
4 slots and 1gb/slot maximum. Using a full length board should
accomodate 8 slots, and accomodating 2gb dimms would move the density
up to 16gb per board. So with 4 of those boards in a 1U PC, that's
64gb per rack slot, which is getting somewhere. Except for the 4GB
limit that board is almost exactly what I want, and it's just $50.
> Where do you buy cheap 2 or 4 GB dimms? Not a commodity product
> either. And it is not just sockets - it is drivers too. Running
> everything at lower speed may help. But again this needs research.
2GB is here at $120/GB:
http://www.zipzoomfly.com/jsp/Produ...oductCode=86040
4GB is still out of reach, but that will improve, and any boards being
designed today should be to accomodate them.
> Yes. And PC's are cheap compared to a custom design for a very narrow
> market.
Do you really think it's that narrow? Fancy enterprise servers always
have large amounts of ram, and often do tasks that could be handled by
PC's with similar amounts of ram. I'd think lots of people would want
this.
>
> There is (look up USB card reader). Then go and look at the speed and
> durability. And look at your ECC requirement again. It is cheap, yes.
Hmm, is there such a thing as a USB2 hub with 128 ports? The ECC
could be handled with a RAID-like configuration.
Thanks.
| |
| Paul Rubin 2006-04-28, 1:12 pm |
| adykes@panix.com (Al Dykes) writes:
> I'll bet that a 10k RPM 200GB SATA disk ($200) would give faster
> throughput than 200GB (at least $20,000) of ram at the end of a USB2
> cable.
The problem is seek latency. There are a lot of small requests with a
a completely random access pattern.
| |
| Paul Rubin 2006-04-28, 1:12 pm |
| Paul Rubin <http://phr.cx@NOSPAM.invalid> writes:
> Using a full length [PCI] board should
> accomodate 8 slots, and accomodating 2gb dimms would move the density
> up to 16gb per board. So with 4 of those boards in a 1U PC, that's
> 64gb per rack slot, which is getting somewhere.
Bah, 1U cabinets usually can't accomodate multiple PCI cards, so it would
have to be 2U, still ok.
| |
| Ralph Becker-Szendy 2006-04-28, 1:12 pm |
| I have a Toyota Corolla which I use for grocery store runs. I'm
redoing my yard, and need to haul 10 tons of gravel home. Can I just
add a few extra wheels on the side of the Corolla? Bicycle wheels are
pretty cheap at the Salvation Army store, I can just buy two dozen of
them. Since suspensions and axles are too expensive, I'll just
duct-tape them to the side of the Corolla. But I need the reliability
of a high-end hauler, so I'll carry a spraycan of fix-a-flat with me.
Sorry Paul, but you are being completely unrealistic in your request.
You need to buy a Mack truck. They are available at truck dealers
near you. They are very very good at hauling gravel: efficient to
operate, very reliable, easy to use (once you've had the required
training). But not cheap.
In article <7xbqum5d7l.fsf@ruckus.brouhaha.com>,
Paul Rubin <http://phr.cx@NOSPAM.invalid> wrote:
>Zak <jute@zak.invalid> writes:
>
>Just about anything--PCIe, Infiniband, heck, even USB2 or Firewire.
There is no market demand for very large RAM disk systems with
consumer (can you say "toy") interfaces, which have consumer speeds
and reliability. People who need 500GB of RAM won't use a connection
that's designed for digital cameras; they will use something like
Fibre Channel or Infiniband.
>Is that a big deal? What is so high tech about this? It's just a
>bunch of dimm sockets on a board, right?
That's the easy part. The hard part is the ECC, the memory
controller, the memory bus to interface controller (typically a
high-end CPU in and of itself), and the redundancy of the interface
controller. You are talking about a big RAM-disk. The easiest way to
acquire this is to buy a disk array with a lot of cache RAM.
Give me $50M, and I'll found a startup that will manufacture such
large RAM-disks with Infiniband interfaces. You have to commit to buy
$200M worth of them per year from me though - because there seems to
be little other demand for them.
> Giga-byte makes a PCI
>ramdisk with four dimm sockets for $50, but it can only take 1GB
>dimms. I just want something like that but a lot bigger, maybe
>multiple boards, and able to use 2gb or 4gb dimms. Let's say a 1U
>rack box containing a board with 32 or 64 dimm sockets. A few of
>those with 2gb dimms (around $140/GB right now with ECC) and I'm
>there.
Find a motherboard with a two dozen PCI slots (which is nearly an
electrical impossibility, you will have multiple PCI busses at this
point, which is no longer a commodity PC). If you want to use the RAM
as adressable core, you need a CPU with a huge address space, which is
not available on PCI slots. This just can't be done with commodity
hardware.
>What PC's exist that can take more than 16GB? I'd have to use dozens
>of them to get 500 GB of ram.
Go to HPs website, and look for whatever the Superdome is called now.
Look for an Itanium-based machine with 64 or 128 CPUs. I bet that
thing can take 500 GB of RAM. From a 50000' level, this thing is a PC
(it even boots Windows if I remember right) with quite a few CPUs and
a lot of memory, but with a single system image. I'm sure you can get
equivalent offerings from IBM, Sun, and all the other usual suspects,
but those might be a little less PC-like, and more like Unix machines.
This is easily accomplished - but a little expensive.
>I also notice that flash memory in the form of SD cards is now around
>$25/GB in 4GB cards:
>
> http://www.zipzoomfly.com/jsp/Produ...ctCode=82502-19
>
>I wonder if there's some way of sticking 128 or so of those in a box.
What's the speed of CF or SD? You might wish for a spinning harddisk
instead. Also, Flash memory (like SD and CF) can only be overwritten
so many times (the estimates range from 10K to 10M times); not
suitable for use like RAM. And the reliability of such a device is
likely to be horrible.
There is a good reason we refer to these things as CRAP:
<C>ommodity-based, built from <R>eadily <A>vailable <P>arts.
I'm very sorry, you are trying to buy a Mack truck on a Corolla
budget. Your best bet is the commercially available RAM disks (which
are frightfully expensive, because they serve a boutique-market).
Also carefully consider the following: if you can organize your
workload a little better to reduce seeks, and spread your workload
over a lot of spindles (disk drives), and you use very fast but
low-capacity disk drives, then disk drives might be your best bet.
For example, used 9, 18 or 36GB SCSI or FC disks, with 15K RPM (nice
Seagates), can be had used for not very much money today. Those have
very very good seek times, much better than typical SATA disks. If
you can stripe your workload over a hundred drives, and short-stroke
them (only the first few GB of each drive are used), you might be in
business.
Good luck!
--
The address in the header is invalid for obvious reasons. Please
reconstruct the address from the information below (look for _).
Ralph Becker-Szendy _firstname_@lr_dot_los-gatos_dot_ca.us
| |
|
| Paul Rubin wrote:
> But that's true of just about any technology product. This thing
> should have no more complexity than a microcomputer memory board of
> the CP/M era, which (unpopulated) cost a few hundred bucks, and those
> things were made in fairly small quantity.
Except that stuff like DDR ram is harder to use than the static RAM of
the time, even before you add timing and the need for multi layer boards.
And people will want this with a fibre channel interface, if it is multi
gigabytes. And those are complex beatst with a lot of software and
interop testing.
> Or even simpler, is there such a thing as a PCI or AGP board with a
> bunch of dimm slots? I only know of the Giga-byte one, which has just
> 4 slots and 1gb/slot maximum. Using a full length board should
> accomodate 8 slots, and accomodating 2gb dimms would move the density
> up to 16gb per board. So with 4 of those boards in a 1U PC, that's
> 64gb per rack slot, which is getting somewhere. Except for the 4GB
> limit that board is almost exactly what I want, and it's just $50.
There are PCI bridges and extenders that should allow a nearly unlimited
number of slots. But then again, these are expensiev as they are
non-standard.
>
>
> 2GB is here at $120/GB:
>
> http://www.zipzoomfly.com/jsp/Produ...oductCode=86040
Hmm, that is good news. But it is registered, which means it will not
work in your consumer socket.
>
> Do you really think it's that narrow? Fancy enterprise servers always
> have large amounts of ram, and often do tasks that could be handled by
> PC's with similar amounts of ram. I'd think lots of people would want
> this.
If it was easy, there would be PCs with a gazillion DIMM slots. There
are none.
> Hmm, is there such a thing as a USB2 hub with 128 ports? The ECC
> could be handled with a RAID-like configuration.
Cascading. But flash wears faster than mechanical drives....
Thomas
| |
| Paul Rubin 2006-04-28, 7:12 pm |
| lr@idiom.com (Ralph Becker-Szendy) writes:
>
> That's the easy part. The hard part is the ECC, the memory
> controller, the memory bus to interface controller (typically a
> high-end CPU in and of itself), and the redundancy of the interface
> controller. You are talking about a big RAM-disk. The easiest way to
> acquire this is to buy a disk array with a lot of cache RAM.
Well, fine, what about the same thing without the disks?
> Give me $50M, and I'll found a startup that will manufacture such
> large RAM-disks with Infiniband interfaces. You have to commit to buy
> $200M worth of them per year from me though - because there seems to
> be little other demand for them.
Are you joking? This thing doesn't need any ASIC's. It's all off the
shelf.
> Find a motherboard with a two dozen PCI slots (which is nearly an
> electrical impossibility, you will have multiple PCI busses at this
> point, which is no longer a commodity PC).
32 sockets (as mentioned above) would take four PCI cards with 8
sockets per card. Nothing like dozens of PCI slots. Is there some
obstacle to an 8-dimm version of the Giga-byte card, that handles
2gb dimms? The existing card takes 4 dimms with 1gb each.
> If you want to use the RAM as adressable core, you need a CPU with a
> huge address space, which is not available on PCI slots. This just
> can't be done with commodity hardware.
64-bit PCI doesn't handle 64-bit addresses? Hmm.
>
> Go to HPs website, and look for whatever the Superdome is called now.
> Look for an Itanium-based machine with 64 or 128 CPUs. I bet that
> thing can take 500 GB of RAM.
Hmm, the 128-way version can take 1TB of memory. But why does that
ram capacity only exist on monstrous processors like that (the 32-way
version was $600-700K in Nov 2001, which by Moore's Law would put the
128-way version at around $300K today)? That's what I don't
understand. I can put almost infinite amounts of commodity disk space
on a consumer PC. I don't understand why there's not comparable
ramdisks.
> What's the speed of CF or SD? You might wish for a spinning harddisk
> instead. Also, Flash memory (like SD and CF) can only be overwritten
> so many times (the estimates range from 10K to 10M times); not
> suitable for use like RAM. And the reliability of such a device is
> likely to be horrible.
The write wear may be an issue though most of the traffic is read.
The application takes many thousands of small, random-access requests
per second, making disks not such a great solution.
> For example, used 9, 18 or 36GB SCSI or FC disks, with 15K RPM (nice
> Seagates), can be had used for not very much money today. Those have
> very very good seek times, much better than typical SATA disks. If
> you can stripe your workload over a hundred drives, and short-stroke
> them (only the first few GB of each drive are used), you might be in
> business.
This is maybe worth thinking about, just for the added parallelism.
| |
|
|
| Paul Rubin 2006-04-28, 7:12 pm |
| Zak <jute@zak.invalid> writes:
> There are PCI bridges and extenders that should allow a nearly
> unlimited number of slots. But then again, these are expensiev as they
> are non-standard.
Well, I don't think I'm asking for that, I'm simply observing that I
can get a consumer PCI card with four dimm slots that hold 1GB each (4
GB total) and asking if there's some obstacle to making a similar card
with 8 dimm slots that hold 2GB each (16GB total). I'd hope that just
means using a physically bigger card (full length PCI), and updating
the on-board FPGA to handle the extra couple of address lines. Then I
could have 64GB in a commodity PC with four PCI slots, which would
satisfy me. Is there some big problem with that?
>
> Hmm, that is good news. But it is registered, which means it will not
> work in your consumer socket.
Hmm, ok, I'm not sure what the difference is. How about this? It's
$100/GB and doesn't say registered (but it lacks ECC):
http://www.zipzoomfly.com/jsp/Produ...ctCode=83433-31
Anyway, if registered memory needs another kind of socket, that
doesn't sound like a big deal.
> If it was easy, there would be PCs with a gazillion DIMM slots. There
> are none.
Hmm, ok. But it's hard to find PC's with even 8 dimm slots.
| |
| Faeandar 2006-04-28, 7:12 pm |
| On 27 Apr 2006 13:56:45 -0700, Paul Rubin
<http://phr.cx@NOSPAM.invalid> wrote:
>adykes@panix.com (Al Dykes) writes:
>
>That amount of bucks/GB is reasonable. A $50K expenditure for this
>application might not be out of line, if it resulted in getting 400GB
>of ram ($10K for the box and 40k for the ram). All the "solid state
>disk" applicances on that scale I know of cost a heck of a lot more
>than that.
I've never heard of a "relatively inexpensive" server that can hold up
to 1TB or ram. I'd be very interested in this.
The best I've seen is RamSAN from Texas Memory Systems and if they are
relatively inexpesive by your standards then knock yourself out.
even getting a Sun or x86 box to hold 96GB of ram is a decent
expenditure; about $50k just for the box and the memory can run you
past $100k for one host.
~F
| |
| Paul Rubin 2006-04-28, 7:12 pm |
| "Globe Treader" <kiranghag@gmail.com> writes:
> check this out...
> http://www.superssd.com/products/tera-ramsan/
Thanks, yes, that box starts at around $60K in a minimal
configuration. Looks like a great product but way too high end for
the application I'm thinking of.
| |
| Paul Rubin 2006-04-28, 7:12 pm |
| Faeandar <mr_castalot@yahoo.com> writes:
> even getting a Sun or x86 box to hold 96GB of ram is a decent
> expenditure; about $50k just for the box and the memory can run you
> past $100k for one host.
Yes, this is what's baffling me. 96GB of ram is about $10K worth.
Why should 96GB of ram in a box cost 10x as much as the ram itself
costs? It was never like that back in the day. The current situation
seems like a big scam.
| |
| Rob Turk 2006-04-28, 7:12 pm |
| "Paul Rubin" <http://phr.cx@NOSPAM.invalid> wrote in message
news:7x8xpqokbq.fsf_-_@ruckus.brouhaha.com...
> Is there such a thing as a relatively inexpensive box (up to a few K$,
> say) that can hold a huge amount of ram, like 200-1000 GB? It doesn't
> have to be battery backed or anything like that, but it should have
> ECC or at least parity. The application is a high traffic database
> cache.
Can you be a bit more specific? What will the front-end be? Where will the
'real' database live? SAN? NAS? iSCSI?
Maybe you can get away with a couple of standard 16GB RAM servers and a
network load balancer?
Rob
| |
| Paul Rubin 2006-04-29, 1:13 am |
| "Rob Turk" <_wipe_me_r.turk@chello.nl> writes:
>
> Can you be a bit more specific? What will the front-end be? Where will the
> 'real' database live? SAN? NAS? iSCSI?
It's is a system that's already running and overloaded; I'm just
trying to think up approaches to speeding it up. The front end is an
LVS load balancer pointing to a bunch of Squid caches on PC's equipped
with 74GB disks. The "real" database is currently on a 1.5TB RAID 10
cluster consisting of eight 400GB SATA desktop drives. I'm not sure
what RAID controller is being used.
> Maybe you can get away with a couple of standard 16GB RAM servers and a
> network load balancer?
I think that's nowhere near enough; every request to the database is
already missing a cache of at least 74GB and maybe a lot more.
Anyway, I'm persuaded by the folks here that the huge RAM cache scheme
isn't easy to accomplish right now. It's probably way too expensive
for the organization anyway, even for just raw dimms.
| |
| Faeandar 2006-05-01, 1:15 am |
| On 28 Apr 2006 13:31:54 -0700, Paul Rubin
<http://phr.cx@NOSPAM.invalid> wrote:
>Faeandar <mr_castalot@yahoo.com> writes:
>
>Yes, this is what's baffling me. 96GB of ram is about $10K worth.
>Why should 96GB of ram in a box cost 10x as much as the ram itself
>costs? It was never like that back in the day. The current situation
>seems like a big scam.
Hmm, a couple things come to mind regarding this.
1) is it $10k for 2gb or better density memory? I think that density
is a bit more costly.
2) a machine that can address all that memory is not something you
pick up at the local computer store.
3) it may still be a scam but I suspect not as big as you make it
sound.
~F
| |
| Ed Wilts 2006-05-01, 7:12 pm |
| Paul Rubin wrote:
> It's is a system that's already running and overloaded; I'm just
> trying to think up approaches to speeding it up. The front end is an
> LVS load balancer pointing to a bunch of Squid caches on PC's equipped
> with 74GB disks. The "real" database is currently on a 1.5TB RAID 10
> cluster consisting of eight 400GB SATA desktop drives. I'm not sure
> what RAID controller is being used.
Now we're getting somewhere. Replace that disk subsystem with a bunch
of mirrored 36GB 15K SCSI drives on separate controllers. Yes, you'll
need a lot of them but you'll dramatically improve your throughput.
Large SATA drives aren't designed for high-access database activities.
They're far better suited to large digital image collections that
aren't accessed that much.
> Anyway, I'm persuaded by the folks here that the huge RAM cache scheme
> isn't easy to accomplish right now. It's probably way too expensive
> for the organization anyway, even for just raw dimms.
A huge RAM cache for a database almost always means that the database
is not well designed. DBAs will always tell you to get faster disk
and more memory rather than just indexing it properly and optimizing
queries.
.../Ed
| |
| Jesper Monsted 2006-05-02, 7:13 am |
| Paul Rubin wrote:
> lr@idiom.com (Ralph Becker-Szendy) writes:
> 64-bit PCI doesn't handle 64-bit addresses? Hmm.
Nope. That's a 64 bit data path and doesn't change addressing. I don't
even think that changes with PCIe.
> Hmm, the 128-way version can take 1TB of memory. But why does that
> ram capacity only exist on monstrous processors like that (the 32-way
> version was $600-700K in Nov 2001, which by Moore's Law would put the
> 128-way version at around $300K today)? That's what I don't
> understand. I can put almost infinite amounts of commodity disk space
> on a consumer PC. I don't understand why there's not comparable
> ramdisks.
Because nobody buys them. Engineering a motherboard with one CPU socket
and 64 RAM sockets would just not be interesting to more than a few
people and couldn't possibly be profitable to the designer.
If i were you, i'd go buy a HDS Tagmastor/9980V (or any other box of
that class) with a couple of drives and 256 GB of cache, then set it up
with "sticky cache" to export one 256 GB volume that'll always be in
cache. It'll come with fibre channel interfaces. It'll probably cost you
a small fortune, though.
--
Jesper Monsted
BCFP, BCSD, MCP (That's the McData kind)
| |
| Maxim S. Shatskih 2006-05-02, 7:13 am |
| > > 64-bit PCI doesn't handle 64-bit addresses? Hmm.
>
> Nope. That's a 64 bit data path and doesn't change addressing. I don't
> even think that changes with PCIe.
Even 32bit PCI devices can often handle 64bit addressing _as busmasters_, using
the Dual Address Cycle feature.
Dunno about the slave mode though.
--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com
| |
| Al Dykes 2006-05-02, 1:14 pm |
| In article <44571286$0$47020$edfadb0f@news.tele.dk>,
Jesper Monsted <newsspam@monsted.dk> wrote:
>Paul Rubin wrote:
>
>
>Nope. That's a 64 bit data path and doesn't change addressing. I don't
>even think that changes with PCIe.
>
>
>Because nobody buys them. Engineering a motherboard with one CPU socket
>and 64 RAM sockets would just not be interesting to more than a few
>people and couldn't possibly be profitable to the designer.
>
>If i were you, i'd go buy a HDS Tagmastor/9980V (or any other box of
>that class) with a couple of drives and 256 GB of cache, then set it up
>with "sticky cache" to export one 256 GB volume that'll always be in
>cache. It'll come with fibre channel interfaces. It'll probably cost you
>a small fortune, though.
>
Based on the OP's description of the business problem he's trying to
solve, IMO he needs to step back and look at the entire application
and do real bottleneck analysis and then consider everything. Until he
has some hard numbers on subsystem performance and had experts look at
them, he's just throwing expensive darts in the dark.
--
a d y k e s @ p a n i x . c o m
Don't blame me. I voted for Gore. Proudly then, even more so in hindsight.
| |
| Thor Lancelot Simon 2006-05-02, 1:14 pm |
| In article <44571286$0$47020$edfadb0f@news.tele.dk>,
Jesper Monsted <newsspam@monsted.dk> wrote:
>Paul Rubin wrote:
>
>
>Nope. That's a 64 bit data path and doesn't change addressing. I don't
>even think that changes with PCIe.
False on both counts. Even 32-bit PCI cards that support DAC (Dual
Address Cycle) can DMA to memory above 4G physical; and even the last
versions of the plain-old-PCI standard (never mind PCI-X or PCI-E)
require that.
--
Thor Lancelot Simon tls@rek.tjls.com
"We cannot usually in social life pursue a single value or a single moral
aim, untroubled by the need to compromise with others." - H.L.A. Hart
| |
| Paul Rubin 2006-05-04, 7:12 pm |
| adykes@panix.com (Al Dykes) writes:
> Based on the OP's description of the business problem he's trying to
> solve, IMO he needs to step back and look at the entire application
> and do real bottleneck analysis and then consider everything. Until he
> has some hard numbers on subsystem performance and had experts look at
> them, he's just throwing expensive darts in the dark.
Nah, tossing around ideas and posting newsgroup messages doesn't cost
anything. The darts are only expensive if actual money is spent.
Anyway, it looks like there was a bug in the cache coordination
software in this system which was causing more cache misses (and thus
more server load) than there should have been. It's fixed now, let's
see how much it helps.
Thanks for all responses.
| |
| soldack@gmail.com 2006-05-05, 1:13 pm |
| You are going to do nearly as well in performance and much better in
price if you pick up a disk array that has lots of cache and a fast
connection (4Gb FC or InfiniBand). You also might do better just to
add more RAM to your database servers. Unisys and SGI make linux boxes
that take a lot of CPUs and lots of RAM. Finally, there are lots of
clustered file systems which would let you use a bunch of cheaper boxes
with lots of RAM and SATA disks for storage. Connect them with
InfiniBand and you get your low latency.
-Ack
|
|
|
|
|