|
Home > Archive > Unix Programming > January 2004 > Time to live on messages
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Time to live on messages
|
|
| Sony Antony 2004-01-23, 4:57 pm |
| We have a distributed application spread over a bunch of Solaris
servers.
The interaction of different processes on different machines is using
messages ( Message oriented middleware ).
We need to implement a time to live algorithm.
Meaning that the messages will be thrown away, if it has lived more
than a pre decided number of seconds, starting from its time of
creation.
The problem is that the time in different machines are different ( not
synchronized with each other ), and NTP ( Network time protocol )
cannot be used due to the company's policies.
Is there any other way to acheive this with the above restrictions (
Any new filed can be added to the message )
Thanks
--sony
| |
| Anthony Mandic 2004-01-23, 4:57 pm |
| Sony Antony <sonyantony@hotmail.com> wrote:quote:
> We have a distributed application spread over a bunch of Solaris
> servers.
>
> The interaction of different processes on different machines is using
> messages ( Message oriented middleware ).
> We need to implement a time to live algorithm.
> Meaning that the messages will be thrown away, if it has lived more
> than a pre decided number of seconds, starting from its time of
> creation.
>
> The problem is that the time in different machines are different ( not
> synchronized with each other ), and NTP ( Network time protocol )
> cannot be used due to the company's policies.
Contact Scott Adams.
-am © 2003
| |
| Anthony Mandic 2004-01-23, 4:57 pm |
| Sony Antony <sonyantony@hotmail.com> wrote:quote:
> We have a distributed application spread over a bunch of Solaris
> servers.
>
> The interaction of different processes on different machines is using
> messages ( Message oriented middleware ).
> We need to implement a time to live algorithm.
> Meaning that the messages will be thrown away, if it has lived more
> than a pre decided number of seconds, starting from its time of
> creation.
>
> The problem is that the time in different machines are different ( not
> synchronized with each other ), and NTP ( Network time protocol )
> cannot be used due to the company's policies.
Contact Scott Adams.
-am © 2003
| |
| Michael Vilain 2004-01-23, 4:57 pm |
| In article <3eb007f1.0311201739.ee4b1a6@posting.google.com>,
sonyantony@hotmail.com (Sony Antony) wrote:
quote:
> We have a distributed application spread over a bunch of Solaris
> servers.
>
> The interaction of different processes on different machines is using
> messages ( Message oriented middleware ).
> We need to implement a time to live algorithm.
> Meaning that the messages will be thrown away, if it has lived more
> than a pre decided number of seconds, starting from its time of
> creation.
>
> The problem is that the time in different machines are different ( not
> synchronized with each other ), and NTP ( Network time protocol )
> cannot be used due to the company's policies.
>
> Is there any other way to acheive this with the above restrictions (
> Any new filed can be added to the message )
You can still run ntp and sync to a single system or router. Then all
the distributed systems will have the same time. It won't be accurate
to actual wall clock time, but all the systems will be the same or close
enough. Or is this _still_ against company policy?
If you install your own GPS ground station, then you won't need to go
out to the internet to sync to the actual wall clock time. One contract
site I was at used this approach with the main campus router as the
central time server.
Unless you can synchronize the time on your systems, I don't see any way
to accomplish your timing requirements. Alternatly, you can move the
application from all the distributed systems to a single system.
What's the specific objection to running ntp at this company? Is it a
nebulous fear of 'insecurity'?
--
DeeDee, don't press that button! DeeDee! NO! Dee...
| |
| Michael Vilain 2004-01-23, 4:57 pm |
| In article <3eb007f1.0311201739.ee4b1a6@posting.google.com>,
sonyantony@hotmail.com (Sony Antony) wrote:
quote:
> We have a distributed application spread over a bunch of Solaris
> servers.
>
> The interaction of different processes on different machines is using
> messages ( Message oriented middleware ).
> We need to implement a time to live algorithm.
> Meaning that the messages will be thrown away, if it has lived more
> than a pre decided number of seconds, starting from its time of
> creation.
>
> The problem is that the time in different machines are different ( not
> synchronized with each other ), and NTP ( Network time protocol )
> cannot be used due to the company's policies.
>
> Is there any other way to acheive this with the above restrictions (
> Any new filed can be added to the message )
You can still run ntp and sync to a single system or router. Then all
the distributed systems will have the same time. It won't be accurate
to actual wall clock time, but all the systems will be the same or close
enough. Or is this _still_ against company policy?
If you install your own GPS ground station, then you won't need to go
out to the internet to sync to the actual wall clock time. One contract
site I was at used this approach with the main campus router as the
central time server.
Unless you can synchronize the time on your systems, I don't see any way
to accomplish your timing requirements. Alternatly, you can move the
application from all the distributed systems to a single system.
What's the specific objection to running ntp at this company? Is it a
nebulous fear of 'insecurity'?
--
DeeDee, don't press that button! DeeDee! NO! Dee...
| |
| Morris Dovey 2004-01-23, 4:58 pm |
| Sony Antony wrote:quote:
> We have a distributed application spread over a bunch of Solaris
> servers.
>
> The interaction of different processes on different machines is using
> messages ( Message oriented middleware ).
> We need to implement a time to live algorithm.
> Meaning that the messages will be thrown away, if it has lived more
> than a pre decided number of seconds, starting from its time of
> creation.
>
> The problem is that the time in different machines are different ( not
> synchronized with each other ), and NTP ( Network time protocol )
> cannot be used due to the company's policies.
>
> Is there any other way to acheive this with the above restrictions (
> Any new filed can be added to the message )
Sony...
It's not a pretty solution, but you might consider adding two fields:
[1] Time of message creation/arrival on the current server, and
[2] Accumulated message age (updated when message sent). If/when
this age would be set to a value exceeding the threshold value,
do whatever your system requires to cleanly discard the message.
It'd probably be a good idea to have a control mechanism to
enable/disable logging of dropped messages for initial and future
regression testing.
Obviously, it would really help if you have middleware sources or
a really customer-oriented middleware vendor. :-)
--
Morris Dovey
West Des Moines, Iowa USA
C links at http://www.iedu.com/c
Read my lips: The apple doesn't fall far from the tree.
| |
| Morris Dovey 2004-01-23, 4:58 pm |
| Sony Antony wrote:quote:
> We have a distributed application spread over a bunch of Solaris
> servers.
>
> The interaction of different processes on different machines is using
> messages ( Message oriented middleware ).
> We need to implement a time to live algorithm.
> Meaning that the messages will be thrown away, if it has lived more
> than a pre decided number of seconds, starting from its time of
> creation.
>
> The problem is that the time in different machines are different ( not
> synchronized with each other ), and NTP ( Network time protocol )
> cannot be used due to the company's policies.
>
> Is there any other way to acheive this with the above restrictions (
> Any new filed can be added to the message )
Sony...
It's not a pretty solution, but you might consider adding two fields:
[1] Time of message creation/arrival on the current server, and
[2] Accumulated message age (updated when message sent). If/when
this age would be set to a value exceeding the threshold value,
do whatever your system requires to cleanly discard the message.
It'd probably be a good idea to have a control mechanism to
enable/disable logging of dropped messages for initial and future
regression testing.
Obviously, it would really help if you have middleware sources or
a really customer-oriented middleware vendor. :-)
--
Morris Dovey
West Des Moines, Iowa USA
C links at http://www.iedu.com/c
Read my lips: The apple doesn't fall far from the tree.
| |
| Sony Antony 2004-01-23, 4:58 pm |
| Morris Dovey <mrdovey@iedu.com> wrote in message news:<v5nvb.511$ru1.2758@news.uswest.net>...quote:
> Sony Antony wrote:
>
> Sony...
>
> It's not a pretty solution, but you might consider adding two fields:
>
> [1] Time of message creation/arrival on the current server, and
> [2] Accumulated message age (updated when message sent). If/when
> this age would be set to a value exceeding the threshold value,
> do whatever your system requires to cleanly discard the message.
> It'd probably be a good idea to have a control mechanism to
> enable/disable logging of dropped messages for initial and future
> regression testing.
>
> Obviously, it would really help if you have middleware sources or
> a really customer-oriented middleware vendor. :-)
Thanks a lot Morris:
As you pointed out we dont have control over the middleware software.
So getting the time of arrival is almost impossible.
I have an algorithm that will work, if there is some way I can get the
remote machine's time ( and there by the time difference )
Is there any way in solaris to get the remote machine's time ( date )
without running NTP
--sony
| |
| Sony Antony 2004-01-23, 4:58 pm |
| Morris Dovey <mrdovey@iedu.com> wrote in message news:<v5nvb.511$ru1.2758@news.uswest.net>...quote:
> Sony Antony wrote:
>
> Sony...
>
> It's not a pretty solution, but you might consider adding two fields:
>
> [1] Time of message creation/arrival on the current server, and
> [2] Accumulated message age (updated when message sent). If/when
> this age would be set to a value exceeding the threshold value,
> do whatever your system requires to cleanly discard the message.
> It'd probably be a good idea to have a control mechanism to
> enable/disable logging of dropped messages for initial and future
> regression testing.
>
> Obviously, it would really help if you have middleware sources or
> a really customer-oriented middleware vendor. :-)
Thanks a lot Morris:
As you pointed out we dont have control over the middleware software.
So getting the time of arrival is almost impossible.
I have an algorithm that will work, if there is some way I can get the
remote machine's time ( and there by the time difference )
Is there any way in solaris to get the remote machine's time ( date )
without running NTP
--sony
| |
| Darren Dunham 2004-01-23, 4:58 pm |
| In comp.unix.solaris Sony Antony <sonyantony@hotmail.com> wrote:quote:
> I have an algorithm that will work, if there is some way I can get the
> remote machine's time ( and there by the time difference )
> Is there any way in solaris to get the remote machine's time ( date )
> without running NTP
I don't understand what the actual restriction is.
Is there something that says "thou shalt not use NTP", or "time
synchronization is expressly forbidden", or is it more complex relating
to what machines may communicate with other machines, or a general
restriction on UDP packets...
I don't want to come up with a suggestion that will still fall under the
administrative restrictions...
Perhaps porting NTP and calling it "bob's time protocol"?
--
Darren Dunham ddunham@taos.com
Unix System Administrator Taos - The SysAdmin Company
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >
| |
| Darren Dunham 2004-01-23, 4:58 pm |
| In comp.unix.solaris Sony Antony <sonyantony@hotmail.com> wrote:quote:
> I have an algorithm that will work, if there is some way I can get the
> remote machine's time ( and there by the time difference )
> Is there any way in solaris to get the remote machine's time ( date )
> without running NTP
I don't understand what the actual restriction is.
Is there something that says "thou shalt not use NTP", or "time
synchronization is expressly forbidden", or is it more complex relating
to what machines may communicate with other machines, or a general
restriction on UDP packets...
I don't want to come up with a suggestion that will still fall under the
administrative restrictions...
Perhaps porting NTP and calling it "bob's time protocol"?
--
Darren Dunham ddunham@taos.com
Unix System Administrator Taos - The SysAdmin Company
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >
| |
| Morris Dovey 2004-01-23, 4:58 pm |
| Sony Antony wrote:quote:
> Morris Dovey <mrdovey@iedu.com> wrote in message news:<v5nvb.511$ru1.2758@news.uswest.net>...
>
[QUOTE][color=darkred]
> As you pointed out we dont have control over the middleware software.
> So getting the time of arrival is almost impossible.
>
> I have an algorithm that will work, if there is some way I can get the
> remote machine's time ( and there by the time difference )
> Is there any way in solaris to get the remote machine's time ( date )
> without running NTP
Sony...
I'm not really the person to answer this one. Its been a while
since I dealt with this kind of problem. (Talarian SmartSockets
then and the customer had enough clout to be treated especially
well :-)
Check the docs for your cloud - I would really expect there to be
some callback-type hooks in the middleware that could be used to
touch incoming and outgoing messages. Even if they don't appear
in the docs, it would be worth talking to the vendor (How could
they ever do problem analysis without such a facility?)
Otherwise, if your messages are aging because of throughput
difficulties, I may be able to offer some specific suggestions to
deal with the problem from the performance enhancement angle (I
hacked one stock exchange's middleware interface to produce a 76x
throughput boost [to their considerable delight]) - but short of
the obvious suggestion that you use a simple time server app, I
think the best help might come from the Solaris crowd.
--
Morris Dovey
West Des Moines, Iowa USA
C links at http://www.iedu.com/c
Read my lips: The apple doesn't fall far from the tree.
| |
| Morris Dovey 2004-01-23, 4:58 pm |
| Sony Antony wrote:quote:
> Morris Dovey <mrdovey@iedu.com> wrote in message news:<v5nvb.511$ru1.2758@news.uswest.net>...
>
[QUOTE][color=darkred]
> As you pointed out we dont have control over the middleware software.
> So getting the time of arrival is almost impossible.
>
> I have an algorithm that will work, if there is some way I can get the
> remote machine's time ( and there by the time difference )
> Is there any way in solaris to get the remote machine's time ( date )
> without running NTP
Sony...
I'm not really the person to answer this one. Its been a while
since I dealt with this kind of problem. (Talarian SmartSockets
then and the customer had enough clout to be treated especially
well :-)
Check the docs for your cloud - I would really expect there to be
some callback-type hooks in the middleware that could be used to
touch incoming and outgoing messages. Even if they don't appear
in the docs, it would be worth talking to the vendor (How could
they ever do problem analysis without such a facility?)
Otherwise, if your messages are aging because of throughput
difficulties, I may be able to offer some specific suggestions to
deal with the problem from the performance enhancement angle (I
hacked one stock exchange's middleware interface to produce a 76x
throughput boost [to their considerable delight]) - but short of
the obvious suggestion that you use a simple time server app, I
think the best help might come from the Solaris crowd.
--
Morris Dovey
West Des Moines, Iowa USA
C links at http://www.iedu.com/c
Read my lips: The apple doesn't fall far from the tree.
| |
| Morris Dovey 2004-01-23, 4:58 pm |
| Darren Dunham wrote:
quote:
> In comp.unix.solaris Sony Antony <sonyantony@hotmail.com> wrote:
>
>
>
> I don't understand what the actual restriction is.
>
> Is there something that says "thou shalt not use NTP", or "time
> synchronization is expressly forbidden", or is it more complex relating
> to what machines may communicate with other machines, or a general
> restriction on UDP packets...
>
> I don't want to come up with a suggestion that will still fall under the
> administrative restrictions...
>
> Perhaps porting NTP and calling it "bob's time protocol"?
Darren...
Frequently point of using a middleware layer is to allow nodes to
come and go as they will (without the rest of the network knowing
or caring). Perhaps most common in publisher/subscriber networks.
If a time server is designated, and that server goes away
(detaches from the cloud), then things might get messy.
Time messages could be sent through the middleware; but during
those peak periods when messages are aging most (becoming stale
in transit) you would not want to further degrade communication
performance with time traffic.
I'll take a wild guess that there are also non-Solaris (perhaps
even non-POSIX) boxen attached to the cloud for which there is no
NTP support; and the strategy chosen needs to include them.
--
Morris Dovey
West Des Moines, Iowa USA
C links at http://www.iedu.com/c
Read my lips: The apple doesn't fall far from the tree.
| |
| Morris Dovey 2004-01-23, 4:58 pm |
| Darren Dunham wrote:
quote:
> In comp.unix.solaris Sony Antony <sonyantony@hotmail.com> wrote:
>
>
>
> I don't understand what the actual restriction is.
>
> Is there something that says "thou shalt not use NTP", or "time
> synchronization is expressly forbidden", or is it more complex relating
> to what machines may communicate with other machines, or a general
> restriction on UDP packets...
>
> I don't want to come up with a suggestion that will still fall under the
> administrative restrictions...
>
> Perhaps porting NTP and calling it "bob's time protocol"?
Darren...
Frequently point of using a middleware layer is to allow nodes to
come and go as they will (without the rest of the network knowing
or caring). Perhaps most common in publisher/subscriber networks.
If a time server is designated, and that server goes away
(detaches from the cloud), then things might get messy.
Time messages could be sent through the middleware; but during
those peak periods when messages are aging most (becoming stale
in transit) you would not want to further degrade communication
performance with time traffic.
I'll take a wild guess that there are also non-Solaris (perhaps
even non-POSIX) boxen attached to the cloud for which there is no
NTP support; and the strategy chosen needs to include them.
--
Morris Dovey
West Des Moines, Iowa USA
C links at http://www.iedu.com/c
Read my lips: The apple doesn't fall far from the tree.
| |
| Darren Dunham 2004-01-23, 4:58 pm |
| In comp.unix.solaris Morris Dovey <mrdovey@iedu.com> wrote:quote:
> Darren...
quote:
> Frequently point of using a middleware layer is to allow nodes to
> come and go as they will (without the rest of the network knowing
> or caring). Perhaps most common in publisher/subscriber networks.
> If a time server is designated, and that server goes away
> (detaches from the cloud), then things might get messy.
A) NTP can discipline the local clock, so even if it cannot synchronize
externally, it can still let the clock run very close to correct for a
long time.
B) NTP can use several different servers. It is recommended that you
have 3 servers for redundancy.
I think of NTP and DNS as being configured similarly. There should be a
"right" answer, but the data can be served from redundant machines.
I'm no middleware guy, but is DNS handled any differently in such a
setup?
quote:
> Time messages could be sent through the middleware; but during
> those peak periods when messages are aging most (becoming stale
> in transit) you would not want to further degrade communication
> performance with time traffic.
I'm not sure what you mean by "messages sent through the middleware". I
was assuming the OS would send messages directly to a configured server,
rather like DNS.
In a stable NTP time network, you have one pair of packets sent between
a client to a server every 1024 seconds. I doubt even at normal startup
rates (one packet pair every 64 seconds between client server pairs)
that the traffic would even be noticed.
quote:
> I'll take a wild guess that there are also non-Solaris (perhaps
> even non-POSIX) boxen attached to the cloud for which there is no
> NTP support; and the strategy chosen needs to include them.
That's an even better reason to use NTP. What architectures were you
thinking of? My impression is that NTP is widely available.
--
Darren Dunham ddunham@taos.com
Unix System Administrator Taos - The SysAdmin Company
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >
| |
| Darren Dunham 2004-01-23, 4:58 pm |
| In comp.unix.solaris Morris Dovey <mrdovey@iedu.com> wrote:quote:
> Darren...
quote:
> Frequently point of using a middleware layer is to allow nodes to
> come and go as they will (without the rest of the network knowing
> or caring). Perhaps most common in publisher/subscriber networks.
> If a time server is designated, and that server goes away
> (detaches from the cloud), then things might get messy.
A) NTP can discipline the local clock, so even if it cannot synchronize
externally, it can still let the clock run very close to correct for a
long time.
B) NTP can use several different servers. It is recommended that you
have 3 servers for redundancy.
I think of NTP and DNS as being configured similarly. There should be a
"right" answer, but the data can be served from redundant machines.
I'm no middleware guy, but is DNS handled any differently in such a
setup?
quote:
> Time messages could be sent through the middleware; but during
> those peak periods when messages are aging most (becoming stale
> in transit) you would not want to further degrade communication
> performance with time traffic.
I'm not sure what you mean by "messages sent through the middleware". I
was assuming the OS would send messages directly to a configured server,
rather like DNS.
In a stable NTP time network, you have one pair of packets sent between
a client to a server every 1024 seconds. I doubt even at normal startup
rates (one packet pair every 64 seconds between client server pairs)
that the traffic would even be noticed.
quote:
> I'll take a wild guess that there are also non-Solaris (perhaps
> even non-POSIX) boxen attached to the cloud for which there is no
> NTP support; and the strategy chosen needs to include them.
That's an even better reason to use NTP. What architectures were you
thinking of? My impression is that NTP is widely available.
--
Darren Dunham ddunham@taos.com
Unix System Administrator Taos - The SysAdmin Company
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >
| |
| Maurizio Loreti 2004-01-23, 4:59 pm |
| Morris Dovey <mrdovey@iedu.com> writes:
quote:
> Time messages could be sent through the middleware;
The best you can do IMHO is to implement an internal synchronization
between the LAN hosts. There are *hundredths* of trusted hosts
outside the LAN that you can access for time. For the code, I have an
example one - open http://wwcdf.pd.infn.it/MLO/ and get
synctime.tar.gz ; the comments include a pointer to the list of the
above hosts.
--
Maurizio Loreti http://www.pd.infn.it/~loreti/mlo.html
Dept. of Physics, Univ. of Padova, Italy ROT13: ybergv@cq.vasa.vg
| |
| Maurizio Loreti 2004-01-23, 4:59 pm |
| Morris Dovey <mrdovey@iedu.com> writes:
quote:
> Time messages could be sent through the middleware;
The best you can do IMHO is to implement an internal synchronization
between the LAN hosts. There are *hundredths* of trusted hosts
outside the LAN that you can access for time. For the code, I have an
example one - open http://wwcdf.pd.infn.it/MLO/ and get
synctime.tar.gz ; the comments include a pointer to the list of the
above hosts.
--
Maurizio Loreti http://www.pd.infn.it/~loreti/mlo.html
Dept. of Physics, Univ. of Padova, Italy ROT13: ybergv@cq.vasa.vg
| |
| Mark Rafn 2004-01-23, 4:59 pm |
| >> In comp.unix.solaris Sony Antony <sonyantony@hotmail.com> wrote:quote:
[QUOTE][color=darkred]
>Darren Dunham wrote:
Morris Dovey <mrdovey@iedu.com> wrote:[QUOTE][color=darkred]
>Frequently point of using a middleware layer is to allow nodes to
>come and go as they will (without the rest of the network knowing
>or caring). Perhaps most common in publisher/subscriber networks.
>If a time server is designated, and that server goes away
>(detaches from the cloud), then things might get messy.
No messier than anything else. NTP can discover and/or broadcast,
synchronizing among a group of peers that adjusts membership as nodes come
and go. Or it can be a core network service, provided by infrastructure
servers that are not part of the middleware node management, in the same
way that DNS and DHCP probably is. The latter approach is what I'd normally
recommend.
quote:
>Time messages could be sent through the middleware; but during
>those peak periods when messages are aging most (becoming stale
>in transit) you would not want to further degrade communication
>performance with time traffic.
NTP is pretty lightweight, trafficwise. Middleware messages may not be, and
more importantly often don't track latency (the whole point of this excercise),
which makes them pretty useless for time synchronization.
quote:
>I'll take a wild guess that there are also non-Solaris (perhaps
>even non-POSIX) boxen attached to the cloud for which there is no
>NTP support; and the strategy chosen needs to include them.
Very unlikely. There's good NTP support for every major platform, and it's a
published spec so even wacky embedded systems could support it if time
synchronization was important to them.
The original problem has two main ways it can be solved:
1) Synchronize the time on the machines. NTP makes this painless, but any
other method could be used if NTP is unusable for some reason. Heck, buy
radio or atomic clocks for each node if you decide that's the way to go.
2) Instead of an expiration date in the TTL field which gets compared against
the (possibly wrong) local clock, allow each latency-inducing processor/router
to decrement the "time remaining to live" field before sending it on, and drop
it if it gets to 0. This is less accurate (because there will be latencies
that don't decrement the remaining life), but doesn't require clocks to be
precise.
--
Mark Rafn dagon@dagon.net <http://www.dagon.net/>
| |
| Mark Rafn 2004-01-23, 4:59 pm |
| >> In comp.unix.solaris Sony Antony <sonyantony@hotmail.com> wrote:quote:
[QUOTE][color=darkred]
>Darren Dunham wrote:
Morris Dovey <mrdovey@iedu.com> wrote:[QUOTE][color=darkred]
>Frequently point of using a middleware layer is to allow nodes to
>come and go as they will (without the rest of the network knowing
>or caring). Perhaps most common in publisher/subscriber networks.
>If a time server is designated, and that server goes away
>(detaches from the cloud), then things might get messy.
No messier than anything else. NTP can discover and/or broadcast,
synchronizing among a group of peers that adjusts membership as nodes come
and go. Or it can be a core network service, provided by infrastructure
servers that are not part of the middleware node management, in the same
way that DNS and DHCP probably is. The latter approach is what I'd normally
recommend.
quote:
>Time messages could be sent through the middleware; but during
>those peak periods when messages are aging most (becoming stale
>in transit) you would not want to further degrade communication
>performance with time traffic.
NTP is pretty lightweight, trafficwise. Middleware messages may not be, and
more importantly often don't track latency (the whole point of this excercise),
which makes them pretty useless for time synchronization.
quote:
>I'll take a wild guess that there are also non-Solaris (perhaps
>even non-POSIX) boxen attached to the cloud for which there is no
>NTP support; and the strategy chosen needs to include them.
Very unlikely. There's good NTP support for every major platform, and it's a
published spec so even wacky embedded systems could support it if time
synchronization was important to them.
The original problem has two main ways it can be solved:
1) Synchronize the time on the machines. NTP makes this painless, but any
other method could be used if NTP is unusable for some reason. Heck, buy
radio or atomic clocks for each node if you decide that's the way to go.
2) Instead of an expiration date in the TTL field which gets compared against
the (possibly wrong) local clock, allow each latency-inducing processor/router
to decrement the "time remaining to live" field before sending it on, and drop
it if it gets to 0. This is less accurate (because there will be latencies
that don't decrement the remaining life), but doesn't require clocks to be
precise.
--
Mark Rafn dagon@dagon.net <http://www.dagon.net/>
| |
| phil-news-nospam@ipal.net 2004-01-23, 4:59 pm |
| In comp.unix.programmer Sony Antony <sonyantony@hotmail.com> wrote:
| The problem is that the time in different machines are different ( not
| synchronized with each other ), and NTP ( Network time protocol )
| cannot be used due to the company's policies.
We really need to know and understand what it is that caused management
to have such a regressive screwed up policy. This knowledge would help
us to counter any sources of FUD (fear, uncertainty, doubt) that may be
spreading around. It would also help to know where the management got
this misinformation from in the first place.
I can speculate on a number of things. One might be that someone said
NTP was insecure and a hacker can cause clocks to go crazy. But any
manager that buys into that clearly doesn't realize all that needs to
be done is isolate your NTP network through proper configuration. Many
other protocols you might be using on the net are less secure than NTP.
Of course none are prefectly secure. There is no perfect security. But
proper procedures and proper contingencies can make failures in security
and other failures have no impact on your operations.
Apparently management has a greater negative impact than any hacker ever
could.
--
-----------------------------------------------------------------------------
| Phil Howard KA9WGN | http://linuxhomepage.com/ http://ham.org/ |
| (first name) at ipal.net | http://phil.ipal.org/ http://ka9wgn.ham.org/ |
-----------------------------------------------------------------------------
| |
| phil-news-nospam@ipal.net 2004-01-23, 4:59 pm |
| In comp.unix.programmer Sony Antony <sonyantony@hotmail.com> wrote:
| The problem is that the time in different machines are different ( not
| synchronized with each other ), and NTP ( Network time protocol )
| cannot be used due to the company's policies.
We really need to know and understand what it is that caused management
to have such a regressive screwed up policy. This knowledge would help
us to counter any sources of FUD (fear, uncertainty, doubt) that may be
spreading around. It would also help to know where the management got
this misinformation from in the first place.
I can speculate on a number of things. One might be that someone said
NTP was insecure and a hacker can cause clocks to go crazy. But any
manager that buys into that clearly doesn't realize all that needs to
be done is isolate your NTP network through proper configuration. Many
other protocols you might be using on the net are less secure than NTP.
Of course none are prefectly secure. There is no perfect security. But
proper procedures and proper contingencies can make failures in security
and other failures have no impact on your operations.
Apparently management has a greater negative impact than any hacker ever
could.
--
-----------------------------------------------------------------------------
| Phil Howard KA9WGN | http://linuxhomepage.com/ http://ham.org/ |
| (first name) at ipal.net | http://phil.ipal.org/ http://ka9wgn.ham.org/ |
-----------------------------------------------------------------------------
| |
| Morris Dovey 2004-01-23, 4:59 pm |
| Mark Rafn wrote:
quote:
>
<snip>
[QUOTE][color=darkred]
>
> Very unlikely. There's good NTP support for every major
> platform, and it's a published spec so even wacky embedded
> systems could support it if time synchronization was important
> to them.
The trading engines at PHLX that I worked on were Stratus/VOS
boxes. I'm not sure about NTP, but there were other even more
basic protocols normally taken for granted that Stratus didn't
support and required third party drivers. I wouldn't be too
surprised to hear that NTP falls into this catagory.
quote:
> The original problem has two main ways it can be solved: 1)
> Synchronize the time on the machines. NTP makes this
> painless, but any other method could be used if NTP is
> unusable for some reason. Heck, buy radio or atomic clocks
> for each node if you decide that's the way to go.
I have nothing against NTP but the management policy would seem
to preclude this option. It's difficult to assess the validity of
the policy without more info - and a determination that the
policy rationale is faulty doesn't mean that a policy change will
follow.
Given that, the clocks are a good idea. I suspect that if the
target interval is slightly stretched, it won't be a serious problem.
quote:
> 2) Instead of an expiration date in the TTL field which gets
> compared against the (possibly wrong) local clock, allow each
> latency-inducing processor/router to decrement the "time
> remaining to live" field before sending it on, and drop it if
> it gets to 0. This is less accurate (because there will be
> latencies that don't decrement the remaining life), but
> doesn't require clocks to be precise.
Personally, I like this idea even better (but not as well as I
liked my own which, although similar, allowed dynamic
control/tuning of message life. :-)
But all of these methods require that the middleware allow
"touching" the message. Sony says: "No touching allowed," so the
only meaningful alternative is to increase throughput until the
problem goes away. Sometimes this can be accomplished in software
(I boosted throughput 76X at PHLX with a software-only solution);
and sometimes it's necessary to bite the bullet and order another
truckload of Ultras and bring in another T3 or two.
--
Morris Dovey
West Des Moines, Iowa USA
C links at http://www.iedu.com/c
Read my lips: The apple doesn't fall far from the tree.
| |
| Morris Dovey 2004-01-23, 4:59 pm |
| Mark Rafn wrote:
quote:
>
<snip>
[QUOTE][color=darkred]
>
> Very unlikely. There's good NTP support for every major
> platform, and it's a published spec so even wacky embedded
> systems could support it if time synchronization was important
> to them.
The trading engines at PHLX that I worked on were Stratus/VOS
boxes. I'm not sure about NTP, but there were other even more
basic protocols normally taken for granted that Stratus didn't
support and required third party drivers. I wouldn't be too
surprised to hear that NTP falls into this catagory.
quote:
> The original problem has two main ways it can be solved: 1)
> Synchronize the time on the machines. NTP makes this
> painless, but any other method could be used if NTP is
> unusable for some reason. Heck, buy radio or atomic clocks
> for each node if you decide that's the way to go.
I have nothing against NTP but the management policy would seem
to preclude this option. It's difficult to assess the validity of
the policy without more info - and a determination that the
policy rationale is faulty doesn't mean that a policy change will
follow.
Given that, the clocks are a good idea. I suspect that if the
target interval is slightly stretched, it won't be a serious problem.
quote:
> 2) Instead of an expiration date in the TTL field which gets
> compared against the (possibly wrong) local clock, allow each
> latency-inducing processor/router to decrement the "time
> remaining to live" field before sending it on, and drop it if
> it gets to 0. This is less accurate (because there will be
> latencies that don't decrement the remaining life), but
> doesn't require clocks to be precise.
Personally, I like this idea even better (but not as well as I
liked my own which, although similar, allowed dynamic
control/tuning of message life. :-)
But all of these methods require that the middleware allow
"touching" the message. Sony says: "No touching allowed," so the
only meaningful alternative is to increase throughput until the
problem goes away. Sometimes this can be accomplished in software
(I boosted throughput 76X at PHLX with a software-only solution);
and sometimes it's necessary to bite the bullet and order another
truckload of Ultras and bring in another T3 or two.
--
Morris Dovey
West Des Moines, Iowa USA
C links at http://www.iedu.com/c
Read my lips: The apple doesn't fall far from the tree.
| |
| Noel R. Nihill 2004-01-23, 4:59 pm |
|
"Sony Antony" <sonyantony@hotmail.com> wrote in message
news:3eb007f1.0311201739.ee4b1a6@posting.google.com...quote:
> We have a distributed application spread over a bunch of Solaris
> servers.
>
> The interaction of different processes on different machines is using
> messages ( Message oriented middleware ).
> We need to implement a time to live algorithm.
> Meaning that the messages will be thrown away, if it has lived more
> than a pre decided number of seconds, starting from its time of
> creation.
>
> The problem is that the time in different machines are different ( not
> synchronized with each other ), and NTP ( Network time protocol )
> cannot be used due to the company's policies.
These policies are broken. What is the problem with NTP?
quote:
> Is there any other way to acheive this with the above restrictions (
> Any new filed can be added to the message )
In order to figure out how old a message is, you need to know two things.
One, what time was the message assembled at, and two, which host
assembled it. Assuming these two pieces of data are contained in each
message, one may get the time on the remote server in the absence of NTP
by using either:
'REMOTE_TIME=`rdate host`'
or
'REMOTE_TIME=`telnet <hostname>13 2>/dev/null | tail -1`
Now you can calculate the age of the message.
Hint: Try using time-converting code from an open-source
MTA to assist this calculation.
quote:
> Thanks
> --sony
--
Noel R. Nihill
UNIX® platform development
Motorola NSS
I *could* be arguing in my spare time
| |
| Noel R. Nihill 2004-01-23, 4:59 pm |
|
"Sony Antony" <sonyantony@hotmail.com> wrote in message
news:3eb007f1.0311201739.ee4b1a6@posting.google.com...quote:
> We have a distributed application spread over a bunch of Solaris
> servers.
>
> The interaction of different processes on different machines is using
> messages ( Message oriented middleware ).
> We need to implement a time to live algorithm.
> Meaning that the messages will be thrown away, if it has lived more
> than a pre decided number of seconds, starting from its time of
> creation.
>
> The problem is that the time in different machines are different ( not
> synchronized with each other ), and NTP ( Network time protocol )
> cannot be used due to the company's policies.
These policies are broken. What is the problem with NTP?
quote:
> Is there any other way to acheive this with the above restrictions (
> Any new filed can be added to the message )
In order to figure out how old a message is, you need to know two things.
One, what time was the message assembled at, and two, which host
assembled it. Assuming these two pieces of data are contained in each
message, one may get the time on the remote server in the absence of NTP
by using either:
'REMOTE_TIME=`rdate host`'
or
'REMOTE_TIME=`telnet <hostname>13 2>/dev/null | tail -1`
Now you can calculate the age of the message.
Hint: Try using time-converting code from an open-source
MTA to assist this calculation.
quote:
> Thanks
> --sony
--
Noel R. Nihill
UNIX® platform development
Motorola NSS
I *could* be arguing in my spare time
|
|
|
|
|