Unix Programming - portability of integral types in C++ on UNIX

Author

portability of integral types in C++ on UNIX

Benjamin Rutt

2004-01-25, 4:35 am

I'd like to have the following explicit integral types available on my
system, for portability reasons (we're sending binary data between
UNIX machines and to files and therefore need to use explicit types in
these cases):

uint8_t
int8_t
uint16_t
int16_t
uint32_t
int32_t
uint64_t
int64_t

Note, also, that the choice to use binary vs. ascii has been made; our
users are going to show up with binary files, and we need to read this
data and transmit the data to other machines, so using ascii isn't an
option.

AFAIK the best way to do this is to #include <inttypes.h>, which
contains typedefs for all of these types. But, who knows, our project
may be built on a system that does not provide inttypes.h (there's no
way to know for sure, is there?). And we are using a makefile
generator (cmake) that can check for the presence of a header file,
and take an action if it doesn't exist, and take a different action if
it does. So I figured we'd use the following little program
(createExplicitSizes.cpp) to generate a suitable .h file, say
e.g. generatedtypes.h that does include these typedefs.

Then, regardless of the presense of inttypes.h, we'd generate a file
called getintegraltypes.h which we include in our project. In the
case that inttypes.h existed, getintegraltypes.h would contain:

#include <inttypes.h>

and in the case that inttypes.h didn't exist, getintegraltypes.h would
contain:

#include "generatedtypes.h"

Finally, our project would just include "getintegraltypes.h", which
would now serve as a point of indirection and would either include
inttypes.h or the header our program below generated.

Is this a good approach for portability? Comments appreciated.
Thanks,

Benjamin Rutt

/* createExplicitSizes.cpp: create typedefs for explicit integral types */

#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#define SIGNED 1
#define UNSIGNED 0

char * getMatchingType(int bytes, int signed_type)
{
if (signed_type) {
if (sizeof(signed char) == bytes)
return "signed char";
else if (sizeof(signed short) == bytes)
return "signed short";
else if (sizeof(signed int) == bytes)
return "signed int";
else if (sizeof(signed long) == bytes)
return "signed long";
else if (sizeof(signed long long) == bytes)
return "signed long long";
else {
exit(EXIT_FAILURE);
}
}
else {
if (sizeof(unsigned char) == bytes)
return "unsigned char";
else if (sizeof(unsigned short) == bytes)
return "unsigned short";
else if (sizeof(unsigned int) == bytes)
return "unsigned int";
else if (sizeof(unsigned long) == bytes)
return "unsigned long";
else if (sizeof(unsigned long long) == bytes)
return "unsigned long long";
else {
exit(EXIT_FAILURE);
}
}
}

int main(int argc, char * argv[])
{
int loop;
FILE * f;

if (argc != 2) {
fprintf(stderr, "usage: %s <filename.h>\n", argv[0]);
fprintf(stderr, " OR\n");
fprintf(stderr, "usage: %s -\n", argv[0]);
exit(EXIT_FAILURE);
}

if (strcmp(argv[1], "-") == 0) {
f = stdout;
}
else {
f = fopen(argv[1], "w");
if (!f) {
fprintf(stderr, "Could not open file %s for writing!\n", argv[1]);
}
}

fprintf(f, "typedef %20s int8_t;\n", getMatchingType(1, SIGNED));
fprintf(f, "typedef %20s uint8_t;\n", getMatchingType(1, UNSIGNED));
fprintf(f, "typedef %20s int16_t;\n", getMatchingType(2, SIGNED));
fprintf(f, "typedef %20s uint16_t;\n", getMatchingType(2, UNSIGNED));
fprintf(f, "typedef %20s int32_t;\n", getMatchingType(4, SIGNED));
fprintf(f, "typedef %20s uint32_t;\n", getMatchingType(4, UNSIGNED));
fprintf(f, "typedef %20s int64_t;\n", getMatchingType(8, SIGNED));
fprintf(f, "typedef %20s uint64_t;\n", getMatchingType(8, UNSIGNED));

if (f != stdout) {
fclose(f);
}

return 0;
}

Jeff Schwab

2004-01-25, 5:36 am

Benjamin Rutt wrote:

quote:

> I'd like to have the following explicit integral types available on my
> system, for portability reasons (we're sending binary data between
> UNIX machines and to files and therefore need to use explicit types in
> these cases):
>
> uint8_t
> int8_t
> uint16_t
> int16_t
> uint32_t
> int32_t
> uint64_t
> int64_t
>
> Note, also, that the choice to use binary vs. ascii has been made; our
> users are going to show up with binary files, and we need to read this
> data and transmit the data to other machines, so using ascii isn't an
> option.
>
> AFAIK the best way to do this is to #include <inttypes.h>, which
> contains typedefs for all of these types. But, who knows, our project
> may be built on a system that does not provide inttypes.h (there's no
> way to know for sure, is there?). And we are using a makefile
> generator (cmake) that can check for the presence of a header file,
> and take an action if it doesn't exist, and take a different action if
> it does. So I figured we'd use the following little program
> (createExplicitSizes.cpp) to generate a suitable .h file, say
> e.g. generatedtypes.h that does include these typedefs.

Could you ship boost stdint with your project?

Jerry Feldman

2004-01-25, 6:26 am

On Sun, 25 Jan 2004 13:14:59 -0500
Jeff Schwab <jeffplus@comcast.net> wrote:
[QUOTE][color=darkred]
> Benjamin Rutt wrote:
In addition to Jeff's comment, all machines should be the same endian
value. Alpha and Intel machines are little endian and HP PA-RISC and Sun
Sparc are big endian machines.

If you need to be portable and still want to ship binaries, you need to
make sure you agree on what endian you are going to send numbers (ints,
longs, doubles, floats, etc). Then, the receiving machine should then
convert from the network value to the host value.

The IP protocols supply the following functions for integrals:
ntohs Network to host short (16 bit)
htons Host to network short (16 bit)
ntohl Network to host long (32 bit)
htonl Host to network long (32 bit)
There are no standard functions for 64 bit longs.

--
Jerry Feldman <gaf-nospam-at-blu.org>
Boston Linux and Unix user group
http://www.blu.org PGP key id:C5061EA9
PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9

Benjamin Rutt

2004-01-25, 9:35 am

Jeff Schwab <jeffplus@comcast.net> writes:

quote:

> Could you ship boost stdint with your project?

I'll discuss with my group. We will be releasing our software under
the GPL. As long as the boost licence doesn't prohibit us using it,
probably we could.

But here's a practical question: how do you use just the boost stdint
library, and not all of it? At a glance, I can't see how to install
it or part of it.

And finally, it looks like cstdint.hpp looks for some
platform-specific #defines (e.g. __FreeBSD__). This is exactly the
kind of thing I'm trying to get away from. I think I'd rather take
the approach of just generating the sizes automatically using my
little program, then you know they are 100% correct.
--
Benjamin

Benjamin Rutt

2004-01-25, 9:35 am

Jerry Feldman <gaf-noSPAM@blu.org> writes:

quote:

> In addition to Jeff's comment, all machines should be the same endian
> value. Alpha and Intel machines are little endian and HP PA-RISC and Sun
> Sparc are big endian machines.
>
> If you need to be portable and still want to ship binaries, you need to
> make sure you agree on what endian you are going to send numbers (ints,
> longs, doubles, floats, etc). Then, the receiving machine should then
> convert from the network value to the host value.
>
> The IP protocols supply the following functions for integrals:
> ntohs Network to host short (16 bit)
> htons Host to network short (16 bit)
> ntohl Network to host long (32 bit)
> htonl Host to network long (32 bit)
> There are no standard functions for 64 bit longs.

Thanks for the pointer, we're using our own endian conversion routines
as it turns out. To answer your question about binaries, we're only
going to ship sources actually. And instead of agreeing to use the
same endianness everywhere, we're going to mark every packet (using
our own definition of a packet) that goes across the network with the
endianness of the creator of the packet, and then have the receiver of
the packet do endian-swapping if necessary. This way, we only need to
do swapping conversions when absolutely necessary.
--
Benjamin

those who know me have no need of my name

2004-01-25, 10:34 am

in comp.unix.programmer i read:

quote:

>I'd like to have the following explicit integral types available on my
>system, for portability reasons (we're sending binary data between
>UNIX machines and to files and therefore need to use explicit types in
>these cases):
>
>uint8_t
>int8_t
>uint16_t
>int16_t
>uint32_t
>int32_t
>uint64_t
>int64_t

these types aren't available in standard c++, though you might expect to
see them in the next revision as more of c99 is incorporated. in c99 they
are optional, depending on whether one of the standard or extended types
are these particular sizes.

quote:

>I figured we'd use the following little program
>(createExplicitSizes.cpp) to generate a suitable .h file, say
>e.g. generatedtypes.h that does include these typedefs.

this is the most likely to work well with all commonly available c and c++
implementations. in general your program has problems, but i doubt you
will be working with systems that have user visible parity or padding
within integral types, or which uses something other than twos-complement.
also, be aware that sizeof(int) can equal 1 yet still have 16 value bits,
on such a system CHAR_BIT would be 16 -- again i doubt this will matter to
you, since this is only common on embedded platforms and posix/sus demands
a CHAR_BIT of 8.

--
a signature

Benjamin Rutt

2004-01-25, 10:50 am

those who know me have no need of my name <not-a-real-address@usa.net> writes:

quote:

> this is the most likely to work well with all commonly available c
> and c++ implementations. in general your program has problems, but
> i doubt you will be working with systems that have user visible
> parity or padding within integral types, or which uses something
> other than twos-complement. also, be aware that sizeof(int) can
> equal 1 yet still have 16 value bits, on such a system CHAR_BIT
> would be 16 -- again i doubt this will matter to you, since this is
> only common on embedded platforms and posix/sus demands a CHAR_BIT
> of 8.

We're targeting linux on x86, linux on ia64, and solaris 8 on sparc,
so I these platforms are not exactly unusual systems. Are there any
other superior approaches to programatically computing explicit size
types other than what I've done here? Thanks for your input.
--
Benjamin

Walt Fles

2004-01-26, 5:35 am

AT&T's nmake has a ".JOINT" rule:

gram.c gram.h : .gram.y .JOINT

and is intelligent enough to know not to run twice, and does not need a
phony or dummy target

Benjamin Rutt wrote:

quote:

> I'd like to have the following explicit integral types available on my
> system, for portability reasons (we're sending binary data between
> UNIX machines and to files and therefore need to use explicit types in
> these cases):
>
> uint8_t
> int8_t
> uint16_t
> int16_t
> uint32_t
> int32_t
> uint64_t
> int64_t
>
> Note, also, that the choice to use binary vs. ascii has been made; our
> users are going to show up with binary files, and we need to read this
> data and transmit the data to other machines, so using ascii isn't an
> option.
>
> AFAIK the best way to do this is to #include <inttypes.h>, which
> contains typedefs for all of these types. But, who knows, our project
> may be built on a system that does not provide inttypes.h (there's no
> way to know for sure, is there?). And we are using a makefile
> generator (cmake) that can check for the presence of a header file,
> and take an action if it doesn't exist, and take a different action if
> it does. So I figured we'd use the following little program
> (createExplicitSizes.cpp) to generate a suitable .h file, say
> e.g. generatedtypes.h that does include these typedefs.
>
> Then, regardless of the presense of inttypes.h, we'd generate a file
> called getintegraltypes.h which we include in our project. In the
> case that inttypes.h existed, getintegraltypes.h would contain:
>
> #include <inttypes.h>
>
> and in the case that inttypes.h didn't exist, getintegraltypes.h would
> contain:
>
> #include "generatedtypes.h"
>
> Finally, our project would just include "getintegraltypes.h", which
> would now serve as a point of indirection and would either include
> inttypes.h or the header our program below generated.
>
> Is this a good approach for portability? Comments appreciated.
> Thanks,
>
> Benjamin Rutt
>
> /* createExplicitSizes.cpp: create typedefs for explicit integral types */
>
> #include <stdlib.h>
> #include <stdio.h>
> #include <string.h>
>
> #define SIGNED 1
> #define UNSIGNED 0
>
> char * getMatchingType(int bytes, int signed_type)
> {
> if (signed_type) {
> if (sizeof(signed char) == bytes)
> return "signed char";
> else if (sizeof(signed short) == bytes)
> return "signed short";
> else if (sizeof(signed int) == bytes)
> return "signed int";
> else if (sizeof(signed long) == bytes)
> return "signed long";
> else if (sizeof(signed long long) == bytes)
> return "signed long long";
> else {
> exit(EXIT_FAILURE);
> }
> }
> else {
> if (sizeof(unsigned char) == bytes)
> return "unsigned char";
> else if (sizeof(unsigned short) == bytes)
> return "unsigned short";
> else if (sizeof(unsigned int) == bytes)
> return "unsigned int";
> else if (sizeof(unsigned long) == bytes)
> return "unsigned long";
> else if (sizeof(unsigned long long) == bytes)
> return "unsigned long long";
> else {
> exit(EXIT_FAILURE);
> }
> }
> }
>
> int main(int argc, char * argv[])
> {
> int loop;
> FILE * f;
>
> if (argc != 2) {
> fprintf(stderr, "usage: %s <filename.h>\n", argv[0]);
> fprintf(stderr, " OR\n");
> fprintf(stderr, "usage: %s -\n", argv[0]);
> exit(EXIT_FAILURE);
> }
>
> if (strcmp(argv[1], "-") == 0) {
> f = stdout;
> }
> else {
> f = fopen(argv[1], "w");
> if (!f) {
> fprintf(stderr, "Could not open file %s for writing!\n", argv[1]);
> }
> }
>
> fprintf(f, "typedef %20s int8_t;\n", getMatchingType(1, SIGNED));
> fprintf(f, "typedef %20s uint8_t;\n", getMatchingType(1, UNSIGNED));
> fprintf(f, "typedef %20s int16_t;\n", getMatchingType(2, SIGNED));
> fprintf(f, "typedef %20s uint16_t;\n", getMatchingType(2, UNSIGNED));
> fprintf(f, "typedef %20s int32_t;\n", getMatchingType(4, SIGNED));
> fprintf(f, "typedef %20s uint32_t;\n", getMatchingType(4, UNSIGNED));
> fprintf(f, "typedef %20s int64_t;\n", getMatchingType(8, SIGNED));
> fprintf(f, "typedef %20s uint64_t;\n", getMatchingType(8, UNSIGNED));
>
> if (f != stdout) {
> fclose(f);
> }
>
> return 0;
> }

Benjamin Rutt

2004-01-26, 5:35 am

Walt Fles <wefles@ameritech.net> writes:

quote:

> AT&T's nmake has a ".JOINT" rule:
>
> gram.c gram.h : .gram.y .JOINT
>
> and is intelligent enough to know not to run twice, and does not need a
> phony or dummy target

I'm pretty sure this isn't related to my original question.
--
Benjamin