|
Home > Archive > Unix Programming > February 2007 > best practice to send structured data over a socket
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
best practice to send structured data over a socket
|
|
| Oliver Kowalke 2007-02-26, 7:16 pm |
| Hello,
what is the best practice to send structured data over the socket.
If my application protocol contains following header:
struct header
{
char name[3];
unsigned short version;
unsigned long data_size;
};
how should I pass the data to the socket?
Should I create 3 buffers for each header element or should I create one big
buffer containing all data (offset + length and memcpy for each element
have to be used).
regards,Oliver
| |
| Giorgos Keramidas 2007-02-26, 7:16 pm |
| On Mon, 26 Feb 2007 20:33:05 +0100,
Oliver Kowalke <oliver.kowalke@gmx.de> wrote:
> Hello,
> what is the best practice to send structured data over the socket.
> If my application protocol contains following header:
>
> struct header
> {
> char name[3];
> unsigned short version;
> unsigned long data_size;
> };
>
> how should I pass the data to the socket?
>
> Should I create 3 buffers for each header element or should I create
> one big buffer containing all data (offset + length and memcpy for
> each element have to be used).
There is a full-fledged and *standard* way to do this, see i.e.:
R. Srinivasan. "XDR: External Data Representation Standard".
IETF (1995). <http://www.ietf.org/rfc/rfc1832.txt>
You can certainly re-implement XDR in one of its countless variations,
and there are even "modern" XML-based things like SOAP, but I tend to
prefer XDR whenever possible.
- Giorgos
| |
| Oliver Kowalke 2007-02-26, 7:16 pm |
| Giorgos Keramidas wrote:
> There is a full-fledged and *standard* way to do this, see i.e.:
>
> _ R. Srinivasan. "XDR: External Data Representation Standard".
> _ _ _ IETF (1995). _<http://www.ietf.org/rfc/rfc1832.txt>
How is it solved by DHCP, DNS, FTP etc.? I believe they don't use XDR.
> You can certainly re-implement XDR in one of its countless variations,
> and there are even "modern" XML-based things like SOAP, but I tend to
Please no XML - it iss was of space (many additional data/tags) and time
(lot
of parsing).
Oliver
| |
| Giorgos Keramidas 2007-02-26, 7:16 pm |
| On 2007-02-26 21:04, Oliver Kowalke <oliver.kowalke@gmx.de> wrote:
>Giorgos Keramidas wrote:
>
> How is it solved by DHCP, DNS, FTP etc.? I believe they don't use XDR.
FTP is a text-based protocol (at least the 'control channel' part).
The rest use a pretty rigidly defined network data representation, which
is specified to the *byte* *level* by the relevant standard. In these
cases, you have two options:
a) instruct the compiler to "pack" structures, but then you risk
memory alignment problems on some platforms
b) use memcpy() to copy the data from your own struct to the outgoing
network buffers, or from the incoming network buffers
| |
| Ian Collins 2007-02-26, 7:16 pm |
| Giorgos Keramidas wrote:
> On 2007-02-26 21:04, Oliver Kowalke <oliver.kowalke@gmx.de> wrote:
>
>
>
> FTP is a text-based protocol (at least the 'control channel' part).
>
> The rest use a pretty rigidly defined network data representation, which
> is specified to the *byte* *level* by the relevant standard. In these
> cases, you have two options:
>
> a) instruct the compiler to "pack" structures, but then you risk
> memory alignment problems on some platforms
>
> b) use memcpy() to copy the data from your own struct to the outgoing
> network buffers, or from the incoming network buffers
>
Which still leaves you exposed to endian issues.
If you are going to serialise data, you have to set a standard and stick
to it. Consider some of these issues:
1) do you want binary or text data on the wire?
2) if binary, which byte order?
3) how are you going to delimit objects?
4) how are you going to escape the delimiter?
Before long for all but the most trivial case (one object per packet,
binary, same architecture at both ends), you will end up reinventing an
existing standard.
--
Ian Collins.
| |
| Giorgos Keramidas 2007-02-26, 7:16 pm |
| On Tue, 27 Feb 2007 09:54:10 +1300,
Ian Collins <ian-news@hotmail.com> wrote:
> If you are going to serialise data, you have to set a standard and
> stick to it. Consider some of these issues:
>
> 1) do you want binary or text data on the wire?
>
> 2) if binary, which byte order?
>
> 3) how are you going to delimit objects?
>
> 4) how are you going to escape the delimiter?
>
> Before long for all but the most trivial case (one object per packet,
> binary, same architecture at both ends), you will end up reinventing
> an existing standard.
Well said 
| |
| Alex Colvin 2007-02-26, 7:16 pm |
| >> If you are going to serialise data, you have to set a standard and
[vbcol=seagreen]
>Well said
true enough. If you re-invent XML or XPL-RPC, shoot yourself!
--
mac the naïf
| |
| Rainer Weikusat 2007-02-27, 7:21 am |
| Ian Collins <ian-news@hotmail.com> writes:
> Giorgos Keramidas wrote:
[...]
[vbcol=seagreen]
Even worse, depending on the compiler, you 'risk' using byte accesses
and composition logic for perfectly well aligned machine words (eg gcc
3.3 on ARM). If you want to send binary data over a network, define a
host-independent representation (for an example, check the DHCP or DNS
RFCs) and use routines that convert from internal to external
representation and back. The source code for these routines can be
host-independent, too.
[vbcol=seagreen]
> Which still leaves you exposed to endian issues.
>
> If you are going to serialise data, you have to set a standard and stick
> to it. Consider some of these issues:
>
> 1) do you want binary or text data on the wire?
>
> 2) if binary, which byte order?
>
> 3) how are you going to delimit objects?
>
> 4) how are you going to escape the delimiter?
The usual 'simple, default case' is that sender and receiver both now
the format of the messages they are going to exchange and transmitting
meta-information about the format itself is therefore unnecessary.
> Before long for all but the most trivial case (one object per packet,
> binary, same architecture at both ends), you will end up reinventing an
> existing standard.
There isn't an 'existing standard' for binary messages without encoded
metainformation, except insofar that traditional internet protocols
don't encode metainformation for ease of implementation and
processing, while 'newer' internet protocols, for instance, frame
syslog text strings into some custom XML DTD framed into ... which, for all
practical purposes (except that SuSE Linux will certainly start to use
this at some point in time) is mostly an exercise in intellectual
masturbation.
| |
|
| On Feb 27, 4:47 am, Rainer Weikusat <rainer.weiku...@sncag.com> wrote:
> Ian Collins <ian-n...@hotmail.com> writes:
>
>
>
>
> [...]
>
>
>
> Even worse, depending on the compiler, you 'risk' using byte accesses
> and composition logic for perfectly well aligned machine words (eg gcc
> 3.3 on ARM). If you want to send binary data over a network, define a
> host-independent representation (for an example, check the DHCP or DNS
> RFCs) and use routines that convert from internal to external
> representation and back. The source code for these routines can be
> host-independent, too.
>
>
>
>
>
>
>
>
> The usual 'simple, default case' is that sender and receiver both now
> the format of the messages they are going to exchange and transmitting
> meta-information about the format itself is therefore unnecessary.
>
>
> There isn't an 'existing standard' for binary messages without encoded
> metainformation,
ASN.1? http://asn1.elibel.tm.fr/introduction/
> except insofar that traditional internet protocols
> don't encode metainformation for ease of implementation and
> processing, while 'newer' internet protocols, for instance, frame
> syslog text strings into some custom XML DTD framed into ... which, for all
> practical purposes (except that SuSE Linux will certainly start to use
> this at some point in time) is mostly an exercise in intellectual
> masturbation.
|
|
|
|
|