I've got templates that figure out how big an int is, but I've been 
stumped by the rules for static expressions when trying to figure out 
the endianness at compile time. Don't want a run-time check (want to use 
it as a template specialization selector) or a system file (ports 
poorly) and I want to have my own source discover the right endianness 
at compile time. How? Thanks.

Ivan

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Ivan Godard" <igodard.at.pacbell@net.invalid> wrote in message 
news:9Uo4f.16888$6e1.16250@newssvr14.news.prodigy.com...
> I've got templates that figure out how big an int is, but I've been
> stumped by the rules for static expressions when trying to figure out
> the endianness at compile time. Don't want a run-time check (want to use
> it as a template specialization selector) or a system file (ports
> poorly) and I want to have my own source discover the right endianness
> at compile time. How? Thanks.
>
> Ivan


I don't think the standard mentions endianness anywhere and I don't think any 
standard conforming compile-time or run-time tests are possible. Any such test 
would invoke either undefined or unspecified behavior, because they rely on 
reinterpreting types. Theoretically endianness can be anything, not just little 
or big we are used to. There isn't even requirement that every memory allocation 
must produce the same word layout, though it would be strange if it were not. 
But as long as the program cannot without invoking undefined behavior detect 
what endianness is, compiler is free to generate any layout in memory. 
Endianness is an extra-standard knowledge that may be supplied by 
platform/compiler vendor.

-- Gene
----------------------------------------------------------------
There is no greatness where there is no simplicity, goodness and truth. ~ Leo 
Tolstoy 


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gene Bushuyev wrote:

> There isn't even requirement that every memory allocation
> must produce the same word layout, though it would be strange if it were not.

Not only strange. There is a requirement that PODs are memcpy'able and
with different layouts at each allocation that would be more than tricky
to get.


-- 
Maciej Sobczak : http://www.msobczak.com/
Programming    : http://www.msobczak.com/prog/

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Ivan Godard wrote:
> I've got templates that figure out how big an int is, but I've been
> stumped by the rules for static expressions when trying to figure out
> the endianness at compile time. Don't want a run-time check (want to use
> it as a template specialization selector) or a system file (ports
> poorly) and I want to have my own source discover the right endianness
> at compile time. How? Thanks.

It's possible before or after compile time, not while.

It is not possible at compile time because learning the endiannes
requires memory access. Even if a compiler could read memory at compile
time for you, you might be cross-compiling for another platform so that
the results obtained so would not make any sense.

Use <endian.h> Linux header, <boost/detail/endian.hpp>, or pass the
endianness as a macro definition to your compiler.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Ivan Godard wrote:
> I've got templates that figure out how big an int is, but I've been
> stumped by the rules for static expressions when trying to figure out
> the endianness at compile time. Don't want a run-time check (want to use
> it as a template specialization selector) or a system file (ports
> poorly) and I want to have my own source discover the right endianness
> at compile time. How? Thanks.

Although it is not guaranteed by the standard, you might find
multicharacter literals useful.  The value of 'abcd' is implementation
defined, but in practice

  int i = 'abcd';
  int j;
  memcpy(&j, "abcd", 4);
  assert(i == j);

will work on systems with 32-bit integers and 8-bit characters.  So the
conventional test for big-endian would be something like

  'abcd' == ('a'<<24)|('b'<<16)|('c'<<8)|('d');

This is ICE, so the test can be made at compile-time.  I would back it
up with run-time tests, but I don't think you'll find many systems
where the run-time and compile-time tests disagree.

HTH


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <9Uo4f.16888$6e1.16250@newssvr14.news.prodigy.com>, Ivan 
Godard <igodard.at.pacbell@net.invalid> writes
>I've got templates that figure out how big an int is, but I've been
>stumped by the rules for static expressions when trying to figure out
>the endianness at compile time. Don't want a run-time check (want to use
>it as a template specialization selector) or a system file (ports
>poorly) and I want to have my own source discover the right endianness
>at compile time. How? Thanks.
>

I think that you can find approaches to this on the net.  But that some 
systems can change their endianness at run-time.  See e.g.:

http://www.codeguru.com/Cpp/data/data-misc/values/article.php/c10243/

-- 
Alec Ross


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

wade@stoner.com wrote:
> Ivan Godard wrote:
> > I've got templates that figure out how big an int is, but I've been
> > stumped by the rules for static expressions when trying to figure out
> > the endianness at compile time. Don't want a run-time check (want to use
> > it as a template specialization selector) or a system file (ports
> > poorly) and I want to have my own source discover the right endianness
> > at compile time. How? Thanks.
>
> Although it is not guaranteed by the standard, you might find
> multicharacter literals useful.  The value of 'abcd' is implementation
> defined, but in practice
>
>   int i = 'abcd';
>   int j;
>   memcpy(&j, "abcd", 4);
>   assert(i == j);
>
> will work on systems with 32-bit integers and 8-bit characters.  So the
> conventional test for big-endian would be something like
>
>   'abcd' == ('a'<<24)|('b'<<16)|('c'<<8)|('d');

IMO, it only test the ediannes of the platform your compiler was
compiled for. If for instance you do cross compile, it won't help you
much.

Anyway, since one can not be 100% sure with this test, I would get the
target endianness from a header or a macro.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

wade@stoner.com wrote:
> Although it is not guaranteed by the standard, you might find
> multicharacter literals useful.  The value of 'abcd' is implementation
> defined

In my book, this means "there exists a conforming implementation such
that the value of every multicharacter literal is zero". It is thus not
possible to reliably detect endianness using multicharacter literals.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Maciej Sobczak" <no.spam@no.spam.com> wrote in message
news:dj021l$b25$1@sunnews.cern.ch...
> Gene Bushuyev wrote:
>
>> There isn't even requirement that every memory allocation
>> must produce the same word layout, though it would be strange if it were not.
>
> Not only strange. There is a requirement that PODs are memcpy'able and
> with different layouts at each allocation that would be more than tricky
> to get.


The fact that PODs must be memcpy-able doesn't preclude such theoretical
possibility. In fact some machines are bi-endian, - they can change endianness.
Whether they can do it for each memory allocation I don't know, but it's not
impossible.

-- Gene
----------------------------------------------------------------
There is no greatness where there is no simplicity, goodness and truth. ~ Leo
Tolstoy


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Yuri Khan wrote:

>> wade@stoner.com wrote:
>
>>> > Although it is not guaranteed by the standard, you might find
>>> > multicharacter literals useful.  The value of 'abcd' is 
implementation
>>> > defined
>
>>
>> In my book, this means "there exists a conforming implementation such
>> that the value of every multicharacter literal is zero". It is thus not
>> possible to reliably detect endianness using multicharacter literals.


It is a tool.  A fireman's axe may not be able to break down every
door, but until something better comes along, he'll keep the axe on his
truck, and occasionally use it even when he isn't sure it will work.

If I'm porting to a new machine, and this test says the big machine is
big-endian, I can spend my remaining time dealing with other issues.
If my test doesn't reveal the endianness (and it is something I care
about), it quickly tells me what one of my porting issues will be.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Maxim Yegorushkin wrote:


>>> >   'abcd' == ('a'<<24)|('b'<<16)|('c'<<8)|('d');
>
>>
>> IMO, it only test the ediannes of the platform your compiler was
>> compiled for. If for instance you do cross compile, it won't help you
>> much.


It is ICE.  There may be systems where sizeof(int) can be changed at
run time, but a conforming C++ implementation still needs to handle

   char foo[sizeof(int)];

and it must also handle

   char bar[('abcd' & 255u) + 1];

In both cases it would seem that the runtime 'change' would be
difficult to implement once any user code had executed.


>> Anyway, since one can not be 100% sure with this test, I would get the
>> target endianness from a header or a macro.


A header is where this code would go:

   #if __IMPLEMENTATION_X
     const endian my_endian = GOOFY_ENDIAN;
   #elif __IMPLEMENTATION_Y || __IMPLEMENTATION_Z
     const endian my_endian = BIG_ENDIAN;
   #else
     const endian my_endian = ... some multicharacter literal test ...
   #endif
   compile_time_assert(IsValidEndian(my_endian));

When porting to a new system, you may have hundreds of system
dependencies to handle.  Some system-dependent settings can always be
detected automatically.  Some settings (such as this one) can often be
detected automatically, and can reliably tell if the automatic
detection worked.  Others will require some intelligent assistance.  If
you do a lot of porting, you'll want to automate what you can.  Of
course there are other approaches to doing the automation (program A's
config.h is produced by running program B) so that the results of
runtime tests are available at compile time.  There are valid reasons
for using either (or both) approaches.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gene Bushuyev wrote:
> "Ivan Godard" <igodard.at.pacbell@net.invalid> wrote in message
> news:9Uo4f.16888$6e1.16250@newssvr14.news.prodigy.com...
> > I've got templates that figure out how big an int is, but
> > I've been stumped by the rules for static expressions when
> > trying to figure out the endianness at compile time. Don't
> > want a run-time check (want to use it as a template
> > specialization selector) or a system file (ports poorly) and
> > I want to have my own source discover the right endianness
> > at compile time. How? Thanks.

> I don't think the standard mentions endianness anywhere and I
> don't think any standard conforming compile-time or run-time
> tests are possible.  Any such test would invoke either
> undefined or unspecified behavior, because they rely on
> reinterpreting types.

But that's precisely the goal: to do something specific to the
machine.  If the test doesn't invoke something beyond what the
standard guarantees, it can't work.

> Theoretically endianness can be anything, not just little or
> big we are used to.

More than theoretically -- I've worked with at least three
different byte orders for long.  Also (and again more than just
theoretically), endianness is only the tip of the iceberg.
There exist machines with 9 bit bytes, and there exist machines
which use one's complement.

> There isn't even requirement that every memory allocation must
> produce the same word layout, though it would be strange if it
> were not.  But as long as the program cannot without invoking
> undefined behavior detect what endianness is, compiler is free
> to generate any layout in memory.  Endianness is an
> extra-standard knowledge that may be supplied by
> platform/compiler vendor.

I think that there are some restrictions.  If I write a value to
an int in one allocation block, then memcpy it to an int in
another allocation block, the two int's are required to compare
equal.  And memcpy copies bytewise.

Still, I agree that the question only makes sense if we restrict
ourselves to a limited subset of machines.  And even then...
it's very, very rare to need to know the byte order.  The only
times I've needed it is for truly low-level programming:
implementing frexp() and ldexp() in the C library, for example
(although today you could use long long to do the work).

--
James Kanze                                           GABI Software
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Maxim Yegorushkin wrote:
> wade@stoner.com wrote:
> > Ivan Godard wrote:
> > > I've got templates that figure out how big an int is, but
> > > I've been stumped by the rules for static expressions when
> > > trying to figure out the endianness at compile time. Don't
> > > want a run-time check (want to use it as a template
> > > specialization selector) or a system file (ports poorly)
> > > and I want to have my own source discover the right
> > > endianness at compile time. How? Thanks.

> > Although it is not guaranteed by the standard, you might
> > find multicharacter literals useful.  The value of 'abcd' is
> > implementation defined, but in practice

> >   int i = 'abcd';
> >   int j;
> >   memcpy(&j, "abcd", 4);
> >   assert(i == j);

> > will work on systems with 32-bit integers and 8-bit
> > characters.  So the conventional test for big-endian would
> > be something like

> >   'abcd' == ('a'<<24)|('b'<<16)|('c'<<8)|('d');

> IMO, it only test the ediannes of the platform your compiler
> was compiled for. If for instance you do cross compile, it
> won't help you much.

More correctly, it only tests how the compiler writer has
decided to implement multibyte character literals.  There's
certainly no guarantee that the program offered above will work.

> Anyway, since one can not be 100% sure with this test, I would
> get the target endianness from a header or a macro.

In practice, you're going to have some system dependencies in
there anyway, so you might as well add this one.

--
James Kanze                                           GABI Software
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

wade@stoner.com wrote:
> Maxim Yegorushkin wrote:
>
>
> >>> >   'abcd' == ('a'<<24)|('b'<<16)|('c'<<8)|('d');
> >
> >>
> >> IMO, it only test the ediannes of the platform your compiler was
> >> compiled for. If for instance you do cross compile, it won't help you
> >> much.
>
>
> It is ICE.  There may be systems where sizeof(int) can be changed at
> run time, but a conforming C++ implementation still needs to handle
>
>    char foo[sizeof(int)];
>
> and it must also handle
>
>    char bar[('abcd' & 255u) + 1];
>
> In both cases it would seem that the runtime 'change' would be
> difficult to implement once any user code had executed.

I'm sorry, I don't follow how the above relates to endianess.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

<wade@stoner.com> wrote:
<snip>
> If I'm porting to a new machine, and this test says the big machine is
> big-endian, I can spend my remaining time dealing with other issues.
> If my test doesn't reveal the endianness (and it is something I care
> about), it quickly tells me what one of my porting issues will be.

If you're porting to a new machine, surely you look at the
documentation its architecture and for the implementation-defined
behaviour of the particular implementation you are using to compile
for it?

If you want non-programmers to be able to build your program without
knowing such information, then you should probably be using a system
such as the GNU autotools.

Ben.

-- 
Ben Hutchings
Having problems with C++ templates?  Your questions may be answered by
<http://womble.decadentplace.org.uk/c++/template-faq.html>.

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

wade@stoner.com wrote:
> Yuri Khan wrote:

> >> wade@stoner.com wrote:

> >>> > Although it is not guaranteed by the standard, you might
> >>> > find multicharacter literals useful.  The value of
> >>> > 'abcd' is implementation defined

> >> In my book, this means "there exists a conforming
> >> implementation such that the value of every multicharacter
> >> literal is zero". It is thus not possible to reliably
> >> detect endianness using multicharacter literals.

> It is a tool.  A fireman's axe may not be able to break down
> every door, but until something better comes along, he'll keep
> the axe on his truck, and occasionally use it even when he
> isn't sure it will work.

> If I'm porting to a new machine, and this test says the big
> machine is big-endian, I can spend my remaining time dealing
> with other issues.  If my test doesn't reveal the endianness
> (and it is something I care about), it quickly tells me what
> one of my porting issues will be.

The problem is that your test tells you absolutely nothing about
endianness.  It tells you a little, perhaps, about the compiler
implementation of multicharacter literals (information which you
should be able to get from the compiler documentation).

--
James Kanze                                           GABI Software
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]