|
Home > Archive > Unix Programming > March 2005 > Monitering socket buffers
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Monitering socket buffers
|
|
|
| Hi all,
I have a strange problem. I have a network program. Sender appends some
header information at the begining of data it needs to send. Header
information includes the total size of the message that it is supposed to
receive in that datagram. Receiver first reads the header information to
find out the total length of the message receive. Then once it figures out
the total length of the message from the header, it reads the remaining
data.
This works fine with TCP. With UDP I am having problem. Receiver can first
read the header information and decodes. Then it tries to read the rest of
the message. When it tries to read the rest of the message it gets the
message along with the header information.But the sender actually read the
infomation in the previous recvform() call. So, it shouldn't be reading it
again. But I am getting the same header information twice. Both sender and
receiver are running on local machine so I there is little of chance of
message getting garbled. I used ethereal to see what data is send by the
sender and it looks correct.
I want to know what data is exactly in the internal socket buffers and what
I am exactly is being copied on to my data buffer from the socket buffers.I
am running this on Fedora core 2. Any pointers in fixing the problem are
greatly appreciated. Thank you.
--MSR
| |
| Barry Margolin 2005-03-25, 2:48 am |
| In article <VbadnbaYFKymwd7fRVn-1Q@comcast.com>,
"MSR" <msreddy123@comcast.net> wrote:
> Hi all,
> I have a strange problem. I have a network program. Sender appends some
> header information at the begining of data it needs to send. Header
> information includes the total size of the message that it is supposed to
> receive in that datagram. Receiver first reads the header information to
> find out the total length of the message receive. Then once it figures out
> the total length of the message from the header, it reads the remaining
> data.
> This works fine with TCP. With UDP I am having problem. Receiver can first
> read the header information and decodes. Then it tries to read the rest of
> the message. When it tries to read the rest of the message it gets the
> message along with the header information.But the sender actually read the
> infomation in the previous recvform() call. So, it shouldn't be reading it
> again. But I am getting the same header information twice. Both sender and
> receiver are running on local machine so I there is little of chance of
> message getting garbled. I used ethereal to see what data is send by the
> sender and it looks correct.
> I want to know what data is exactly in the internal socket buffers and what
> I am exactly is being copied on to my data buffer from the socket buffers.I
> am running this on Fedora core 2. Any pointers in fixing the problem are
> greatly appreciated. Thank you.
>
> --MSR
UDP is not a stream -- recv() returns whole datagrams. Implementations
differ in how they handle the case where the caller's buffer is not big
enough to hold the entire datagram; some will simply discard the excess,
so you're actually lucky that you're getting it at all.
What you should do is use the MSG_PEEK option in the first recv() call.
This reads the datagram, but ensures that it won't be removed from the
socket buffer. Then you can call recv() again, knowing that the buffer
will be large enough. This will return the header, but you can easily
skip over it.
--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
| |
| David Schwartz 2005-03-25, 2:48 am |
|
"Barry Margolin" <barmar@alum.mit.edu> wrote in message
news:barmar-AACBF7.22102524032005@comcast.dca.giganews.com...
> What you should do is use the MSG_PEEK option in the first recv() call.
> This reads the datagram, but ensures that it won't be removed from the
> socket buffer. Then you can call recv() again, knowing that the buffer
> will be large enough. This will return the header, but you can easily
> skip over it.
In most cases, it's better to just receive the entire thing once into a
64Kb buffer (or one large enough to hold the largest datagram supported by
your protocol). If you're not keeping the data around for very long, the
extra memory size will be paid back by the reduction of a system call. If
you're keeping the data around for a long time, copy it into an
appropriately sized buffer. Although the copy will cost more than the system
call, you rarely keep much data around for a very long time, so you usually
won't be doing this often enough to make a difference.
It's a rare case where the peek followed by a receive is more efficient.
DS
| |
| Malte Starostik 2005-03-25, 6:03 pm |
| David Schwartz schrieb:
> "Barry Margolin" <barmar@alum.mit.edu> wrote in message
> news:barmar-AACBF7.22102524032005@comcast.dca.giganews.com...
>
>
>
>
> In most cases, it's better to just receive the entire thing once into a
> 64Kb buffer (or one large enough to hold the largest datagram supported by
> your protocol). If you're not keeping the data around for very long, the
> extra memory size will be paid back by the reduction of a system call. If
> you're keeping the data around for a long time, copy it into an
> appropriately sized buffer. Although the copy will cost more than the system
> call, you rarely keep much data around for a very long time, so you usually
> won't be doing this often enough to make a difference.
>
> It's a rare case where the peek followed by a receive is more efficient.
While it introduces a second system call as well, you could also use a
FIONREAD ioctl() and allocate a buffer large enough to hold as many
bytes as indicated by its result.
Cheers,
Malte
| |
| David Schwartz 2005-03-25, 8:50 pm |
|
"Malte Starostik" <malte@starostik.de> wrote in message
news:4244784d$1@olaf.komtel.net...
> While it introduces a second system call as well, you could also use a
> FIONREAD ioctl() and allocate a buffer large enough to hold as many
> bytes as indicated by its result.
Unfortunately, FIONREAD is not guaranteed to provide useful information.
I have yet to see a platform or standard that guaranteed that the result of
FIONREAD was still useful later on when you call 'recv' or 'recvfrom'.
Nothing stops the datagram from being discarded and your later call to
'recv' or 'recvfrom' from getting a different datagram of a different size.
DS
| |
| Malte Starostik 2005-03-26, 2:48 am |
| David Schwartz schrieb:
> "Malte Starostik" <malte@starostik.de> wrote in message
> news:4244784d$1@olaf.komtel.net...
>
>
>
>
> Unfortunately, FIONREAD is not guaranteed to provide useful information.
> I have yet to see a platform or standard that guaranteed that the result of
> FIONREAD was still useful later on when you call 'recv' or 'recvfrom'.
> Nothing stops the datagram from being discarded and your later call to
> 'recv' or 'recvfrom' from getting a different datagram of a different size.
Thanks for that info, I imaged that more data could have arrived in the
mean time, which wouldn't be that bad with a datagram socket, only
result in a larger than necessary buffer. But this way it's indeed a
bad idea. Out of curiosity: when would a datagram be discarded after it
had already been received by the kernel?
Thanks,
Malte
| |
| David Schwartz 2005-03-26, 2:48 am |
|
"Malte Starostik" <malte@starostik.de> wrote in message
news:4244d544$1@olaf.komtel.net...
[vbcol=seagreen]
> Thanks for that info, I imaged that more data could have arrived in the
> mean time, which wouldn't be that bad with a datagram socket, only
> result in a larger than necessary buffer. But this way it's indeed a
> bad idea. Out of curiosity: when would a datagram be discarded after it
> had already been received by the kernel?
Memory shortage would be the most obvious reason. A decision to prefer
newer datagrams over older ones would be another. (One can argue this is
superior to keeping older ones and dropping newer ones, but you can also
argue the reverse. This is as moot as the chicken and the egg).
Implementation oddities could do it as well, for example, datagrams may be
pegged internally to an interface or route, and the loss of the interface or
route could cause the loss of the datagram.
DS
|
|
|
|
|