|
Home > Archive > WebSphere Edge Server > March 2004 > Edge V5 on Linux
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
|
|
| Fredrik Gundersen 2004-02-11, 7:33 pm |
| Hi
Have anyone the same problem as me???
My problem is that after some time one of my clusters stop working
When i look in ndamin it all see ok , the advisor is showning that the
backend servers are up and the cluster address is bound to the eth0 card.
The backend servers are Domino HTTP servers.
And its the same cluster that fail all the time.
My setup:
2 Linux RedHat EA 2.1 servers with edge 5.0 in HA
5 Edge clusters ( but only one fail )
Anyone have any idee???
Regards
Fredrik
| |
| Harley Stenzel 2004-02-12, 2:34 am |
| Fredrik Gundersen wrote:
> My setup:
> 2 Linux RedHat EA 2.1 servers with edge 5.0 in HA
> 5 Edge clusters ( but only one fail )
>
> Anyone have any idee???
This sounds to me like the Linux arp problem.
Briefly, the problem with Linux is 2-fold:
1. Linux will respond to arp requests on any interface for any address
configured on any interface, including loopback. So, Linux will respond
to an arp request that comes in on eth0 with an address configured on lo:1.
2. Linux will select any address configured on any interface as the
source IP address for broadcast arp requests. This generates a
"gratitious arp", which causes every host which receives that arp
request to update their arp cache with the mac:IP mapping specified in
the arp request. This can cause a malconfigured server to effectively
take over all incoming traffic by updating the router's arp cache.
There are several solutions, any one of which will solve the problem.
These solutions must be applied on the back-end server, and are not
needed on the load balancer unless colocated servers are configured.
1. Use the "hidden" patch, available at http://www.ssi.bg/~ja/#hidden
.. Note that SLES8/UL1 include this patch as of SP3. Also note that
this will require a kernel rebuild.
2. Use the "noarp" kernel module, available at
http://www.masarlabs.com/noarp/ . Note that this does not require a
kernel rebuild, but it does require development tools and a configured
kernel source tree.
3. Use iptables the iptables rule:
iptables -t nat -A PREROUTING -d $CLUSTERADDRESS -j REDIRECT
to redirect all traffic that comes in on the cluster address to the
local address. Note that this imposes additional overhead on the
servers; my informal tests indicated ~6.4%.
> Regards
> Fredrik
--Harley
| |
| Fredrik Gundersen 2004-02-16, 12:33 am |
| Hi
The backend servers are Windows 2000 and the Edge servers are running Linux
Redhat EA 2.1.
Does this still apply??
Is there some way to test this??
Fredrik
"Harley Stenzel" <hstenzel@us.ibm.com> wrote in message
news:c0g8c3$6gpq$1@news.boulder.ibm.com...
> Fredrik Gundersen wrote:
>
> This sounds to me like the Linux arp problem.
>
> Briefly, the problem with Linux is 2-fold:
>
> 1. Linux will respond to arp requests on any interface for any address
> configured on any interface, including loopback. So, Linux will respond
> to an arp request that comes in on eth0 with an address configured on
lo:1.
>
> 2. Linux will select any address configured on any interface as the
> source IP address for broadcast arp requests. This generates a
> "gratitious arp", which causes every host which receives that arp
> request to update their arp cache with the mac:IP mapping specified in
> the arp request. This can cause a malconfigured server to effectively
> take over all incoming traffic by updating the router's arp cache.
>
> There are several solutions, any one of which will solve the problem.
> These solutions must be applied on the back-end server, and are not
> needed on the load balancer unless colocated servers are configured.
>
> 1. Use the "hidden" patch, available at http://www.ssi.bg/~ja/#hidden
> . Note that SLES8/UL1 include this patch as of SP3. Also note that
> this will require a kernel rebuild.
>
> 2. Use the "noarp" kernel module, available at
> http://www.masarlabs.com/noarp/ . Note that this does not require a
> kernel rebuild, but it does require development tools and a configured
> kernel source tree.
>
> 3. Use iptables the iptables rule:
> iptables -t nat -A PREROUTING -d $CLUSTERADDRESS -j REDIRECT
> to redirect all traffic that comes in on the cluster address to the
> local address. Note that this imposes additional overhead on the
> servers; my informal tests indicated ~6.4%.
>
>
> --Harley
>
| |
| Harley Stenzel 2004-02-16, 2:37 am |
| Fredrik Gundersen wrote:
> Hi
> The backend servers are Windows 2000 and the Edge servers are running Linux
> Redhat EA 2.1.
The advisors won't tell you if your Windows servers have their loopback
addresses configured properly. (I'm assuming that you're using MAC
forwarding, not NAT or CBR). I'd start there. Making sure that the
extra route gets deleted and that each of your clusters has an alias on
the loopback adapter. There's a good checklist in the manual that I
referr to.
Also, make sure that the route gets deleted on reboot.
> Does this still apply??
It mostly dosn't apply. You just have to ensure that your active &
backup dispatchers (if you're using HA) don't have the cluster addresses
configured on any of the interfaces, including loopback, at the same time.
> Is there some way to test this??
Yes, but it's quite involved and I don't think that it's the problem
you're having. I don't want to further confuse the issue.
Let me know if you really want to know, I'll write up the procedure.
--Harley
> Fredrik
>
> "Harley Stenzel" <hstenzel@us.ibm.com> wrote in message
> news:c0g8c3$6gpq$1@news.boulder.ibm.com...
>
>
> lo:1.
>
>
>
>
| |
| Kamal 2004-03-11, 10:36 am |
| Hi Harley,
I tried the iptables method in Redhat Linux 8.0 (Kernel 2.4.18) to
solve arp-response problem. But looks like iptable method is not
working. What I did
1. In Linux box set the iptable rule as follow
# iptables -t nat -A PREROUTING -d 192.168.199.93 -j REDIRECT
where 192.168.199.93 is CLUSTER address and defined as follow
# ifconfig lo:1 192.168.199.93 netmask 255.255.255.255
The actual NIC address is "192.168.199.80" defined on eth0.
Now if I do ping from another box defined in same subnet (window box),
I can easily ping the cluster address(I checked arp was empty before
doing any ping.
Using tcpdump on eth0 in linux box, clearly show that linux reply the
arp .
I also set on linux box
# echo 1 /proc/sys/net/ipv4/ip_forward
I verified that iptables is functioning properly. If i set the rule
"-j DROP" , I can see packet being dropped in tcpdump(no reply of
ping).
Any idea what step I am missing.
Regards
Kamal
Harley Stenzel <hstenzel@us.ibm.com> wrote in message news:<c0qne7$518c$1@news.boulder.ibm.com>...[color=darkred]
> Fredrik Gundersen wrote:
>
> The advisors won't tell you if your Windows servers have their loopback
> addresses configured properly. (I'm assuming that you're using MAC
> forwarding, not NAT or CBR). I'd start there. Making sure that the
> extra route gets deleted and that each of your clusters has an alias on
> the loopback adapter. There's a good checklist in the manual that I
> referr to.
>
> Also, make sure that the route gets deleted on reboot.
>
>
> It mostly dosn't apply. You just have to ensure that your active &
> backup dispatchers (if you're using HA) don't have the cluster addresses
> configured on any of the interfaces, including loopback, at the same time.
>
>
> Yes, but it's quite involved and I don't think that it's the problem
> you're having. I don't want to further confuse the issue.
>
> Let me know if you really want to know, I'll write up the procedure.
>
> --Harley
>
| |
| Harley Stenzel 2004-03-11, 11:35 am |
| Kamal wrote:
> Hi Harley,
>
> I tried the iptables method in Redhat Linux 8.0 (Kernel 2.4.18) to
> solve arp-response problem. But looks like iptable method is not
> working. What I did
> 1. In Linux box set the iptable rule as follow
> # iptables -t nat -A PREROUTING -d 192.168.199.93 -j REDIRECT
> where 192.168.199.93 is CLUSTER address and defined as follow
So far, so good.
> # ifconfig lo:1 192.168.199.93 netmask 255.255.255.255
> The actual NIC address is "192.168.199.80" defined on eth0.
You *must* *not* do this. If using the iptables work-around for the arp
problem, the cluster address must not be configured anywhere on the box.
That's the workaround. iptables rewrites the destination IP address
to the address of the local box.
Similarly, the go* scripts for HA must never alias loopback cluster
addresses for HA+colocation, but the iptables rule should be used instead.
If you're using bind-specific servers, then you must use a different
flavor of the iptables rule that permits you to specify the destination
address. Something like:
iptables -t nat -A PREROUTING -d cluster -j DNAT --to-destination
localaddress
> Any idea what step I am missing.
>
> Regards
> Kamal
--Harley
hstenzel@us.ibm.com
|
|
|
|
|