|
Home > Archive > Voice over IP Cisco > June 2006 > Weird behaviour after a reboot
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Weird behaviour after a reboot
|
|
| Nathan Reeves 2006-06-27, 1:11 am |
| Saw an issues with our CCM yesterday which I'd be interested to hear if
anyone else has seen similar.
Had a bugcheck reboot on the SUB early Sunday morning. Everything failed
back to the SUB once it came back up, but most of the AC Pilot Points were
not working. Was seeing CPU spikes on both the SUB and PUB. Rebooted the
PUB and things went back to normal on that server. Was still seeing CPU
spikes on the SUB (mostly caused by CTIManager.exe and CCM.exe).
Tried to get the AC Pilot Points working during the day with no luck.
Completed a SUB reboot in the evening and when the server came back up, none
of the devices would fail back. Saw errors in the event log to do with
CTIManager, cisco Extended Functions and RIS Data Collector all terminating
unexpectedly. Completed a second reboot and this time things came up
normally, and devices failed back to the SUB.
I'm trying to work out why I would see services fail like this on one
restart, yet work normally on the next restart. Our only recent change has
been the installation of 4.1.3 Sr3a from 4.1.3 Sr2. Anyone ever seen this
kind of issue with restarts causing CCM to not come up cleanly?
Thanks
Nathan
| |
| Ted Nugent 2006-06-27, 1:11 am |
| I've seen similar issues in the past with 4.1(3) at
multiple customer sites, both with OS patches and CM
patches. I was actually able to reproduce it in the
lab on both 4.13sr1 and sr2. When the problem occured
I was able to see in DBLHelper that replication was
actually broken at the time. For whatever reason
simply rebooting the PUB resolved the problem in all
cases. We investigated this with TAC and got no
solution except "well if that works then after patches
reboot the PUB" outsourced TAC at its best and low
BINGO. At any rate we added this to our upgrade
procedure and it appears to be the key.
Patch PUB (reboot)
Patch SUBs and other server per the upgrade guides
(reboot)
and lastly reboot the PUB again.
--- Nathan Reeves <nathan.a.reeves@gmail.com> wrote:
> Saw an issues with our CCM yesterday which I'd be
> interested to hear if
> anyone else has seen similar.
>
> Had a bugcheck reboot on the SUB early Sunday
> morning. Everything failed
> back to the SUB once it came back up, but most of
> the AC Pilot Points were
> not working. Was seeing CPU spikes on both the SUB
> and PUB. Rebooted the
> PUB and things went back to normal on that server.
> Was still seeing CPU
> spikes on the SUB (mostly caused by CTIManager.exe
> and CCM.exe).
>
> Tried to get the AC Pilot Points working during the
> day with no luck.
> Completed a SUB reboot in the evening and when the
> server came back up, none
> of the devices would fail back. Saw errors in the
> event log to do with
> CTIManager, cisco Extended Functions and RIS Data
> Collector all terminating
> unexpectedly. Completed a second reboot and this
> time things came up
> normally, and devices failed back to the SUB.
>
> I'm trying to work out why I would see services fail
> like this on one
> restart, yet work normally on the next restart. Our
> only recent change has
> been the installation of 4.1.3 Sr3a from 4.1.3 Sr2.
> Anyone ever seen this
> kind of issue with restarts causing CCM to not come
> up cleanly?
>
> Thanks
>
> Nathan
> cisco-voip mailing list
> cisco-voip@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-voip
>
________________________________________
__________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
| |
| Lelio Fulgenzi 2006-06-27, 1:11 am |
| We've got into the habbit of a cluster reboot after any patches. Takes a
while, but I'm sure it's saved us from calls to the TAC which say 'reboot
please'.
--------------------------------------------------------------------------------
Lelio Fulgenzi, B.A.
Network Analyst (CCS) * university of Guelph * Guelph, Ontario N1G 2W1
(519) 824-4120 x56354 (519) 767-1060 FAX (JNHN)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
"I can eat fifty eggs." "Nobody can eat fifty eggs."
----- Original Message -----
From: "Ted Nugent" <tednugent69@yahoo.com>
To: "Nathan Reeves" <nathan.a.reeves@gmail.com>;
<cisco-voip@puck.nether.net>
Sent: Monday, June 26, 2006 10:18 PM
Subject: Re: [cisco-voip] Weird behaviour after a reboot
> I've seen similar issues in the past with 4.1(3) at
> multiple customer sites, both with OS patches and CM
> patches. I was actually able to reproduce it in the
> lab on both 4.13sr1 and sr2. When the problem occured
> I was able to see in DBLHelper that replication was
> actually broken at the time. For whatever reason
> simply rebooting the PUB resolved the problem in all
> cases. We investigated this with TAC and got no
> solution except "well if that works then after patches
> reboot the PUB" outsourced TAC at its best and low
> BINGO. At any rate we added this to our upgrade
> procedure and it appears to be the key.
> Patch PUB (reboot)
> Patch SUBs and other server per the upgrade guides
> (reboot)
> and lastly reboot the PUB again.
>
>
>
>
> --- Nathan Reeves <nathan.a.reeves@gmail.com> wrote:
>
>
>
> ________________________________________
__________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> ________________________________________
_______
> cisco-voip mailing list
> cisco-voip@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-voip
>
| |
| Simon, Bill 2006-06-27, 1:11 am |
| I hate this kind of stuff. It's "the Microsoft solution." Something not
working? "Reboot."
I'll accept that--what choice do I have?--but I'd like to know why.
-----Original Message-----
From: Lelio Fulgenzi
To: Ted Nugent; Nathan Reeves; cisco-voip@puck.nether.net
Sent: 6/26/2006 10:21 PM
Subject: Re: [cisco-voip] Weird behaviour after a reboot
We've got into the habbit of a cluster reboot after any patches. Takes a
while, but I'm sure it's saved us from calls to the TAC which say
'reboot
please'.
------------------------------------------------------------------------
--------
Lelio Fulgenzi, B.A.
Network Analyst (CCS) * university of Guelph * Guelph, Ontario N1G 2W1
(519) 824-4120 x56354 (519) 767-1060 FAX (JNHN)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
"I can eat fifty eggs." "Nobody can eat fifty eggs."
----- Original Message -----
From: "Ted Nugent" <tednugent69@yahoo.com>
To: "Nathan Reeves" <nathan.a.reeves@gmail.com>;
<cisco-voip@puck.nether.net>
Sent: Monday, June 26, 2006 10:18 PM
Subject: Re: [cisco-voip] Weird behaviour after a reboot
> I've seen similar issues in the past with 4.1(3) at
> multiple customer sites, both with OS patches and CM
> patches. I was actually able to reproduce it in the
> lab on both 4.13sr1 and sr2. When the problem occured
> I was able to see in DBLHelper that replication was
> actually broken at the time. For whatever reason
> simply rebooting the PUB resolved the problem in all
> cases. We investigated this with TAC and got no
> solution except "well if that works then after patches
> reboot the PUB" outsourced TAC at its best and low
> BINGO. At any rate we added this to our upgrade
> procedure and it appears to be the key.
> Patch PUB (reboot)
> Patch SUBs and other server per the upgrade guides
> (reboot)
> and lastly reboot the PUB again.
>
>
>
>
> --- Nathan Reeves <nathan.a.reeves@gmail.com> wrote:
>
>
>
> ________________________________________
__________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> ________________________________________
_______
> cisco-voip mailing list
> cisco-voip@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-voip
>
________________________________________
_______
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip
| |
| Voll, Scott 2006-06-27, 1:11 pm |
| What OS? 4.2sr7 had a bug where CCM caused 100% CPU utilization. SR7a
fixed it.
scott
________________________________
From: cisco-voip-bounces@puck.nether.net
[mailto:cisco-voip-bounces@puck.nether.net] On Behalf Of Nathan Reeves
Sent: Monday, June 26, 2006 7:04 PM
To: cisco-voip@puck.nether.net
Subject: [cisco-voip] Weird behaviour after a reboot
Saw an issues with our CCM yesterday which I'd be interested to hear if
anyone else has seen similar.
Had a bugcheck reboot on the SUB early Sunday morning. Everything
failed back to the SUB once it came back up, but most of the AC Pilot
Points were not working. Was seeing CPU spikes on both the SUB and PUB.
Rebooted the PUB and things went back to normal on that server. Was
still seeing CPU spikes on the SUB (mostly caused by CTIManager.exe and
CCM.exe).
Tried to get the AC Pilot Points working during the day with no luck.
Completed a SUB reboot in the evening and when the server came back up,
none of the devices would fail back. Saw errors in the event log to do
with CTIManager, cisco Extended Functions and RIS Data Collector all
terminating unexpectedly. Completed a second reboot and this time
things came up normally, and devices failed back to the SUB.
I'm trying to work out why I would see services fail like this on one
restart, yet work normally on the next restart. Our only recent change
has been the installation of 4.1.3 Sr3a from 4.1.3 Sr2. Anyone ever
seen this kind of issue with restarts causing CCM to not come up
cleanly?
Thanks
Nathan
| |
| Wes Sisk 2006-06-27, 1:11 pm |
| 1. bugcheck on the sub - what OS version? CSCsd53918,
2. rebooted sub, services did not come back - what os version?
CSCsd18255
3. CTIManager spiking the CPU - CSCsc26241 and the followup
CSCsd94511 which is not yet resolved. Both of these would interfere
with your AC Pilot Points. Both would additionally be triggered by a
CTIManager with existing device handle going out of service (server
BSOD or reboot)
/Wes
On Jun 26, 2006, at 10:03 PM, Nathan Reeves wrote:
Saw an issues with our CCM yesterday which I'd be interested to hear
if anyone else has seen similar.
Had a bugcheck reboot on the SUB early Sunday morning. Everything
failed back to the SUB once it came back up, but most of the AC Pilot
Points were not working. Was seeing CPU spikes on both the SUB and
PUB. Rebooted the PUB and things went back to normal on that
server. Was still seeing CPU spikes on the SUB (mostly caused by
CTIManager.exe and CCM.exe).
Tried to get the AC Pilot Points working during the day with no
luck. Completed a SUB reboot in the evening and when the server came
back up, none of the devices would fail back. Saw errors in the
event log to do with CTIManager, cisco Extended Functions and RIS
Data Collector all terminating unexpectedly. Completed a second
reboot and this time things came up normally, and devices failed back
to the SUB.
I'm trying to work out why I would see services fail like this on one
restart, yet work normally on the next restart. Our only recent
change has been the installation of 4.1.3 Sr3a from 4.1.3 Sr2.
Anyone ever seen this kind of issue with restarts causing CCM to not
come up cleanly?
Thanks
Nathan
________________________________________
_______
cisco-voip mailing list
cisco-voip@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-voip
| |
| Nathan Reeves 2006-06-28, 1:11 am |
| Thanks for the pointers there. Sounds pretty much like what we saw. For
reference we're running 2000.2.7sr8.
The services in Q were CTIManager, RIS Data Collector and the cisco Extended
Functions. CCM and others started okay, but the three mentioned services
all terminated upon startup.
Time for me to start searching the Bug reports a bit more often.
Much appreciated.
On 6/28/06, Wes Sisk <wsisk@cisco.com> wrote:
>
> 1. bugcheck on the sub - what OS version? CSCsd53918,
> 2. rebooted sub, services did not come back - what os version?
> CSCsd18255
> 3. CTIManager spiking the CPU - CSCsc26241 and the followup
> CSCsd94511 which is not yet resolved. Both of these would interfere
> with your AC Pilot Points. Both would additionally be triggered by a
> CTIManager with existing device handle going out of service (server
> BSOD or reboot)
>
> /Wes
>
>
>
|
|
|
|
|