Hi, I need your help.

----------
struct SvrList{
    unsigned int uNum;
    GameSvr  svr[0];            //line A
};
---------

Once I declared a struct like this to store server list info.
It's supposed to be used like this.

----------
SvrList* pList = (SvrList*)malloc(sizeof(
SvrList) + svrNum*sizeof(GameSvr));
pList->uNum, pList->svr[0], pList->svr[1].... blabla..
---------

The vs2005 compiler gives me the "nonstandard extension used : zero-
sized array in struct/union" warning though.

I may keep my eye closed to the warning since everything looks fine.
But I don't know whether anything bad may happen to me some day due to
running environment changes(porting to different platforms, or any
other conditions).

One work-around way is to declare a one-sized array as struct member,
but I didn't see any substantial changes except no warning in this
case.

Any body give some suggestions?

El 18/08/2010 13:30, thomas escribi�:
> Hi, I need your help.
>
> ----------
> struct SvrList{
>      unsigned int uNum;
>      GameSvr  svr[0];            //line A
> };
> ---------
>
> Once I declared a struct like this to store server list info.
> It's supposed to be used like this.
>
> ----------
> SvrList* pList = (SvrList*)malloc(sizeof(
> SvrList) + svrNum*sizeof(GameSvr));
> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> ---------
>
> The vs2005 compiler gives me the "nonstandard extension used : zero-
> sized array in struct/union" warning though.
>
> I may keep my eye closed to the warning since everything looks fine.
> But I don't know whether anything bad may happen to me some day due to
> running environment changes(porting to different platforms, or any
> other conditions).
>
> One work-around way is to declare a one-sized array as struct member,
> but I didn't see any substantial changes except no warning in this
> case.
>
> Any body give some suggestions?

You can change to

struct SvrList{
      unsigned int uNum;
      GameSvr  *svr;            //line A
};

and still use array indexes.

On Aug 18, 7:33=A0pm, Juli=E1n Rodr=EDguez Bajo <jrodrig...@amper.es> wrote=
:
> El 18/08/2010 13:30, thomas escribi=F3:
>
>
>
> > Hi, I need your help.
>
> > ----------
> > struct SvrList{
> > =A0 =A0 =A0unsigned int uNum;
> > =A0 =A0 =A0GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> > };
> > ---------
>
> > Once I declared a struct like this to store server list info.
> > It's supposed to be used like this.
>
> > ----------
> > SvrList* pList =3D (SvrList*)malloc(sizeof(
> > SvrList) + svrNum*sizeof(GameSvr));
> > pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> > ---------
>
> > The vs2005 compiler gives me the "nonstandard extension used : zero-
> > sized array in struct/union" warning though.
>
> > I may keep my eye closed to the warning since everything looks fine.
> > But I don't know whether anything bad may happen to me some day due to
> > running environment changes(porting to different platforms, or any
> > other conditions).
>
> > One work-around way is to declare a one-sized array as struct member,
> > but I didn't see any substantial changes except no warning in this
> > case.
>
> > Any body give some suggestions?
>
> You can change to
>
> struct SvrList{
> =A0 =A0 =A0 unsigned int uNum;
> =A0 =A0 =A0 GameSvr =A0*svr; =A0 =A0 =A0 =A0 =A0 =A0//line A
>
> };
>
> and still use array indexes.

Thanks, that works.
But I will call "new" two times: one to allocate memory for struct
SvrList, one for "*svr".

I really want to know whether there's any side effect if I use zero-
sized member.

thomas wrote:
> Hi, I need your help.
> 
> ----------
> struct SvrList{
>     unsigned int uNum;
>     GameSvr  svr[0];            //line A
> };
> ---------
> 
> Once I declared a struct like this to store server list info.
> It's supposed to be used like this.
> 
> ----------
> SvrList* pList = (SvrList*)malloc(sizeof(
> SvrList) + svrNum*sizeof(GameSvr));
> pList->uNum, pList->svr[0], pList->svr[1].... blabla..

I wouldn't call this fine. Even
pList->svr[0]
is accessing the element that is out of array's bounds, and that is UB.
How come your program is not crashing, or at least going crazy? Maybe 
you are just unlucky to have a bug hidden.

> ---------
> 
> The vs2005 compiler gives me the "nonstandard extension used : zero-
> sized array in struct/union" warning though.
> 
> I may keep my eye closed to the warning since everything looks fine.
> But I don't know whether anything bad may happen to me some day due to
> running environment changes(porting to different platforms, or any
> other conditions).
> 
> One work-around way is to declare a one-sized array as struct member,
> but I didn't see any substantial changes except no warning in this
> case.
> 
> Any body give some suggestions?

Vladimir Jovic wrote:
> thomas wrote:
>> Hi, I need your help.
>>
>> ----------
>> struct SvrList{
>>     unsigned int uNum;
>>     GameSvr  svr[0];            //line A
>> };
>> ---------
>>
>> Once I declared a struct like this to store server list info.
>> It's supposed to be used like this.
>>
>> ----------
>> SvrList* pList = (SvrList*)malloc(sizeof(
>> SvrList) + svrNum*sizeof(GameSvr));
>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> 
> I wouldn't call this fine. Even
> pList->svr[0]
> is accessing the element that is out of array's bounds, and that is UB.
> How come your program is not crashing, or at least going crazy? Maybe 
> you are just unlucky to have a bug hidden.

It's an old C programmers hack. I've come across this idiom in lot's old 
C code, particularly driver and os code. Microsoft Win32 is rife with it.

Jeff

"thomas" <freshthomas@gmail.com> wrote in message
news:7931b381-68ca-4728-9803-64388b7dc319@f6g2000yqa.googlegroups.com=20
> Hi, I need your help.
>=20
> ----------
> struct SvrList{
>    unsigned int uNum;
>    GameSvr  svr[0];            //line A
> };
> ---------
>=20
> Once I declared a struct like this to store server list info.
> It's supposed to be used like this.
>=20
> ----------
> SvrList* pList =3D (SvrList*)malloc(sizeof(
> SvrList) + svrNum*sizeof(GameSvr));
> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> ---------
>=20
> The vs2005 compiler gives me the "nonstandard extension used : zero-
> sized array in struct/union" warning though.
>=20
> I may keep my eye closed to the warning since everything looks fine.
> But I don't know whether anything bad may happen to me some day due to
> running environment changes(porting to different platforms, or any
> other conditions).
>=20
> One work-around way is to declare a one-sized array as struct member,
> but I didn't see any substantial changes except no warning in this
> case.
>=20
> Any body give some suggestions?

You can use std::vector. A vector can have 0 or more elements.
It also removes the need for malloc and free.

Vladimir Jovic <vladaspams@gmail.com> wrote:
> thomas wrote:
>> Hi, I need your help.
>> 
>> ----------
>> struct SvrList{
>>     unsigned int uNum;
>>     GameSvr  svr[0];            //line A
>> };
>> ---------
>> 
>> Once I declared a struct like this to store server list info.
>> It's supposed to be used like this.
>> 
>> ----------
>> SvrList* pList = (SvrList*)malloc(sizeof(
>> SvrList) + svrNum*sizeof(GameSvr));
>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> 
> I wouldn't call this fine. Even
> pList->svr[0]
> is accessing the element that is out of array's bounds, and that is UB.
> How come your program is not crashing, or at least going crazy?

  pList->svr[0] is accessing memory allocated by the malloc() call,
hence it can't crash (well, at least if 'GameSvr' is a POD type).

Fred Zwarts <F.Zwarts@kvi.nl> wrote:
> You can use std::vector. A vector can have 0 or more elements.
> It also removes the need for malloc and free.

  And also makes the struct something like 10 times slower to allocate
and deallocate, as well as consuming more memory.

Juha Nieminen wrote:
> Vladimir Jovic <vladaspams@gmail.com> wrote:
>> thomas wrote:
>>> Hi, I need your help.
>>>
>>> ----------
>>> struct SvrList{
>>>     unsigned int uNum;
>>>     GameSvr  svr[0];            //line A
>>> };
>>> ---------
>>>
>>> Once I declared a struct like this to store server list info.
>>> It's supposed to be used like this.
>>>
>>> ----------
>>> SvrList* pList = (SvrList*)malloc(sizeof(
>>> SvrList) + svrNum*sizeof(GameSvr));
>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
>> I wouldn't call this fine. Even
>> pList->svr[0]
>> is accessing the element that is out of array's bounds, and that is UB.
>> How come your program is not crashing, or at least going crazy?
> 
>   pList->svr[0] is accessing memory allocated by the malloc() call,
> hence it can't crash (well, at least if 'GameSvr' is a POD type).

If the array's size is zero, how can you access even one element?
The example similar to the original example, except this one compiles :

#include <iostream>
using namespace std;
struct A
{
     int a;
     int b[0];
};
int main()
{
     A *p = new A;

     p->a = 5;
     p->b[0] = 6;
     p->b[1] = 7;
     p->b[2] = 8;

     std::cout << "p->a="<<p->a << std::endl
               << "p->b[0]="<<p->b[0] << std::endl
               << "p->b[1]="<<p->b[1] << std::endl
               << "p->b[2]="<<p->b[2] << std::endl;

     delete( p );
}



And the output :

../a.out
p->a=5 

p->b[0]=6 

p->b[1]=7 

p->b[2]=8
*** glibc detected *** ./a.out: free(): invalid next size (fast): 
0x087b8008 ***
======= Backtrace: =========
/lib/libc.so.6[0x260874]
/lib/libc.so.6(cfree+0x96)[0x2628d6]
/usr/lib/libstdc++.so.6(_ZdlPv+0x21)[0x1ba461]
../a.out(__gxx_personality_v0+0x24a)[0x8048886]
/lib/libc.so.6(__libc_start_main+0xe6)[0x2095d6]
../a.out(__gxx_personality_v0+0x35)[0x8048671]
======= Memory map: ========
00101000-001e8000 r-xp 00000000 fd:07 442834 
/usr/lib/libstdc++.so.6.0.10
001e8000-001ec000 r--p 000e6000 fd:07 442834 
/usr/lib/libstdc++.so.6.0.10
001ec000-001ee000 rw-p 000ea000 fd:07 442834 
/usr/lib/libstdc++.so.6.0.10
001ee000-001f3000 rw-p 001ee000 00:00 0
001f3000-00356000 r-xp 00000000 fd:00 58475      /lib/libc-2.8.so
00356000-00358000 r--p 00163000 fd:00 58475      /lib/libc-2.8.so
00358000-00359000 rw-p 00165000 fd:00 58475      /lib/libc-2.8.so
00359000-0035c000 rw-p 00359000 00:00 0
00821000-0083d000 r-xp 00000000 fd:00 58474      /lib/ld-2.8.so
0083d000-0083e000 r--p 0001c000 fd:00 58474      /lib/ld-2.8.so
0083e000-0083f000 rw-p 0001d000 fd:00 58474      /lib/ld-2.8.so
00848000-00849000 r-xp 00848000 00:00 0          [vdso]
009ac000-009d3000 r-xp 00000000 fd:00 58481      /lib/libm-2.8.so
009d3000-009d4000 r--p 00026000 fd:00 58481      /lib/libm-2.8.so
009d4000-009d5000 rw-p 00027000 fd:00 58481      /lib/libm-2.8.so
00db3000-00dc0000 r-xp 00000000 fd:00 58490 
/lib/libgcc_s-4.3.0-20080428.so.1
00dc0000-00dc1000 rw-p 0000c000 fd:00 58490 
/lib/libgcc_s-4.3.0-20080428.so.1
08048000-08049000 r-xp 00000000 fd:09 4555963 
/sandbox/vladimir/data_create/a.out
08049000-0804a000 rw-p 00000000 fd:09 4555963 
/sandbox/vladimir/data_create/a.out
087b8000-087d9000 rw-p 087b8000 00:00 0          [heap]
b7f00000-b7f21000 rw-p b7f00000 00:00 0
b7f21000-b8000000 ---p b7f21000 00:00 0
b80a5000-b80a8000 rw-p b80a5000 00:00 0
b80c7000-b80c8000 rw-p b80c7000 00:00 0
bfe96000-bfeab000 rw-p bffeb000 00:00 0          [stack]
Aborted

Juha Nieminen wrote:
> Fred Zwarts <F.Zwarts@kvi.nl> wrote:
>> You can use std::vector. A vector can have 0 or more elements.
>> It also removes the need for malloc and free.
>
>  And also makes the struct something like 10 times slower to
> allocate and deallocate, as well as consuming more memory.

Yes, but you get rid of the SvrList (which keeps size and buffer) if 
you let the std::vector do the job. You don't have to dynamically 
allocate the vector itself.


Bo Persson

Vladimir Jovic <vladaspams@gmail.com> wrote in news:i4gueb$vqv$1
@news.albasani.net:

> 
> If the array's size is zero, how can you access even one element?

What's your point?  Why would you do that?  If b were a vector, you would 
get similar behavior.  That is, it is just as undefined if you don't 
bother to check the size of the vector before assigning data. This is a 
idiom used by device drivers to return information requiring a minimum 
number of allocations and a minimum number of frees.  It is always used 
with malloc and the size of the array is always embedded in the struct 
somewhere so that you don't go out of bounds.  So, to fix up your code 
below.

> The example similar to the original example, except this one compiles :
> 
> #include <iostream>
> using namespace std;
> struct A
> {
>      int a;
       int b_len;
>      int b[0];
> };

A * AFactory(int sz)
{
   A * p = (A *)malloc(sizeof(A) + sz * sizeof(b[0]));
   p->b_len = sz;
   return p
}

> int main()
> {
>      A *p = new A;
       A *p = AFactory(0);

> 
>      p->a = 5;

    	 // We know we didn't allocate space so we wouldn't bother with 
this, but...
       if (p->b_len >= 3)
{
>      p->b[0] = 6;
>      p->b[1] = 7;
>      p->b[2] = 8;
}
> 
>      std::cout << "p->a="<<p->a << std::endl;
       for (int ix = 0; ix < p->b_len; ++ix)
    	    	std::cout << "p->b[" << ix << "] = " << p->b[ix] << std::endl;
> 
    	  free(p);  // you could also have function for this, but 
deallocation doesn't require additional info.

> }
> 
> 

I would never advocate the above as a C++ application pattern, but if you 
are messing in drivers and kernel, then every clock cycle counts and 
multiple heap hits can kill performance.

This is one of the places I would really like to see some sort of VLA 
available (at least if it allocated memory in this fashion).  The nice 
thing is that it is one heap allocation and free.  Rolling your own as 
above is a bit of a pain.  Fortunately, I haven't had to do stuff like 
that in years.

joe

thomas wrote:
> Hi, I need your help.
> 
> ----------
> struct SvrList{
>     unsigned int uNum;
>     GameSvr  svr[0];            //line A
> };
> ---------
> 
> Once I declared a struct like this to store server list info.
> It's supposed to be used like this.
> 
> ----------
> SvrList* pList = (SvrList*)malloc(sizeof(
> SvrList) + svrNum*sizeof(GameSvr));
> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> ---------
> 
> The vs2005 compiler gives me the "nonstandard extension used : zero-
> sized array in struct/union" warning though.
> ...
> One work-around way is to declare a one-sized array as struct member,
> but I didn't see any substantial changes except no warning in this
> case.

Well, declaring a 1-sized array is the way it is normally done. The 
memory allocation is supposed to be done as follows

    SvrList* pList = (SvrList*) malloc(offsetof(SvrList, svr) +
      svrNum * sizeof(GameSvr));

As you can notice, you can declare the array with any non-zero size in 
this case, since the above total size calculation approach is not 
affected by the declared array size.

-- 
Best regards,
Andrey Tarasevich

Joe Greer wrote:
> Vladimir Jovic <vladaspams@gmail.com> wrote in news:i4gueb$vqv$1
> @news.albasani.net:
> 
>> If the array's size is zero, how can you access even one element?
> 
> What's your point?  Why would you do that?  If b were a vector, you would 
> get similar behavior.  That is, it is just as undefined if you don't 
> bother to check the size of the vector before assigning data. This is a 

Will next assertion pass or fail :
	int a[0];
	assert( sizeof( a ) == 0 );
?
What bothers me is how can you use an array of size zero.

> idiom used by device drivers to return information requiring a minimum 
> number of allocations and a minimum number of frees.  It is always used 
> with malloc and the size of the array is always embedded in the struct 
> somewhere so that you don't go out of bounds.  So, to fix up your code 
> below.
> 
>> The example similar to the original example, except this one compiles :
>>
>> #include <iostream>
>> using namespace std;
>> struct A
>> {
>>      int a;
>        int b_len;
>>      int b[0];
>> };
> 
> A * AFactory(int sz)
> {
>    A * p = (A *)malloc(sizeof(A) + sz * sizeof(b[0]));
>    p->b_len = sz;
>    return p
> }

This looks like a hack. Why declare an array of size 0? Why not use a 
pointer (like someone suggested)? Is it another (now obsolete and 
depreciated) way of declaring pointers?

On Aug 18, 1:36 pm, Jeff Flinn <TriumphSprint2...@hotmail.com> wrote:
> Vladimir Jovic wrote:
> > thomas wrote:
> >> Hi, I need your help.

> >> ----------
> >> struct SvrList{
> >>     unsigned int uNum;
> >>     GameSvr  svr[0];            //line A
> >> };
> >> ---------

> >> Once I declared a struct like this to store server list info.
> >> It's supposed to be used like this.

> >> ----------
> >> SvrList* pList = (SvrList*)malloc(sizeof(
> >> SvrList) + svrNum*sizeof(GameSvr));
> >> pList->uNum, pList->svr[0], pList->svr[1].... blabla..

> > I wouldn't call this fine. Even
> > pList->svr[0]
> > is accessing the element that is out of array's bounds, and
> > that is UB.  How come your program is not crashing, or at
> > least going crazy? Maybe you are just unlucky to have a bug
> > hidden.

> It's an old C programmers hack. I've come across this idiom in
> lot's old C code, particularly driver and os code. Microsoft
> Win32 is rife with it.

Except that it's not legal C, either.  The C language was
carefully specified so that a bounds checking implementation
would be legal, and the above will fail in such cases.

In C++, of course, the declaration he's looking for is
std::vector<SvrList>, without any manual new.

--
James Kanze

On Aug 19, 11:50=A0am, Vladimir Jovic <vladasp...@gmail.com> wrote:
> Joe Greer wrote:
> > Vladimir Jovic <vladasp...@gmail.com> wrote in news:i4gueb$vqv$1
> > @news.albasani.net:
>
> >> If the array's size is zero, how can you access even one element?
>
> > What's your point? =A0Why would you do that? =A0If b were a vector, you=
 would
> > get similar behavior. =A0That is, it is just as undefined if you don't
> > bother to check the size of the vector before assigning data. This is a
>
> Will next assertion pass or fail :
> =A0 =A0 =A0 =A0 int a[0];
> =A0 =A0 =A0 =A0 assert( sizeof( a ) =3D=3D 0 );
> ?

That snippet should not compile, because it is not allowed to define
objects with size 0.

> What bothers me is how can you use an array of size zero.

You can't, except as a non-standard predecessor of the 'flexible array
member' of C99.
Some compilers will allow a zero-sized array as the last member of a
structure as a way to let the structure contain an array of
unspecified size. It is the full responsibility of the programmer to
ensure enough space gets allocated or the number of array members that
is actually needed.

<snip>
>
> > A * AFactory(int sz)
> > {
> > =A0 =A0A * p =3D (A *)malloc(sizeof(A) + sz * sizeof(b[0]));
> > =A0 =A0p->b_len =3D sz;
> > =A0 =A0return p
> > }
>
> This looks like a hack. Why declare an array of size 0? Why not use a
> pointer (like someone suggested)? Is it another (now obsolete and
> depreciated) way of declaring pointers?

It is a hack, and it is commonly known as the 'struct hack'.
The reasons for not using a pointer can be very diverse.
In the (C-)code that I am currently working with, this hack is used in
the definition of the messages that are passed between different parts
of the system.
As the system consists of multiple (embedded) processors, it is a
requirement that a message is always fully contained within a single
contiguous buffer, so that it can be easily transported to a different
processor. This also rules out the use of internal pointers within the
buffer.

Bart v Ingen Schenau

On Aug 18, 3:30 pm, Juha Nieminen <nos...@thanks.invalid> wrote:
> Vladimir Jovic <vladasp...@gmail.com> wrote:
> > thomas wrote:
> >> Hi, I need your help.

> >> ----------
> >> struct SvrList{
> >>     unsigned int uNum;
> >>     GameSvr  svr[0];            //line A
> >> };
> >> ---------

> >> Once I declared a struct like this to store server list info.
> >> It's supposed to be used like this.

> >> ----------
> >> SvrList* pList = (SvrList*)malloc(sizeof(
> >> SvrList) + svrNum*sizeof(GameSvr));
> >> pList->uNum, pList->svr[0], pList->svr[1].... blabla..

> > I wouldn't call this fine. Even
> > pList->svr[0]
> > is accessing the element that is out of array's bounds, and
> > that is UB.  How come your program is not crashing, or at
> > least going crazy?

> pList->svr[0] is accessing memory allocated by the malloc()
> call, hence it can't crash (well, at least if 'GameSvr' is
> a POD type).

pList->svr[0] is accessing beyond the end of the array, hence,
the code is undefined behavior, both in C and in C++.  The
C language has been carefully specified to allow bounds checking
implementations---there has been (and maybe still is) one.

In C99, flexible array members were introduced to support this
idiom.  In C++, std::vector does the job a lot cleaner.

--
James Kanze

James Kanze wrote:
> On Aug 18, 1:36 pm, Jeff Flinn <TriumphSprint2...@hotmail.com> wrote:
>> Vladimir Jovic wrote:
>>> thomas wrote:
>>>> Hi, I need your help.
> 
>>>> ----------
>>>> struct SvrList{
>>>>     unsigned int uNum;
>>>>     GameSvr  svr[0];            //line A
>>>> };
>>>> ---------
> 
>>>> Once I declared a struct like this to store server list info.
>>>> It's supposed to be used like this.
> 
>>>> ----------
>>>> SvrList* pList = (SvrList*)malloc(sizeof(
>>>> SvrList) + svrNum*sizeof(GameSvr));
>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> 
>>> I wouldn't call this fine. Even
>>> pList->svr[0]
>>> is accessing the element that is out of array's bounds, and
>>> that is UB.  How come your program is not crashing, or at
>>> least going crazy? Maybe you are just unlucky to have a bug
>>> hidden.
> 
>> It's an old C programmers hack. I've come across this idiom in
>> lot's old C code, particularly driver and os code. Microsoft
>> Win32 is rife with it.
> 
> Except that it's not legal C, either.  The C language was
> carefully specified so that a bounds checking implementation
> would be legal, and the above will fail in such cases.

Hence the term 'hack'. ;-)

> In C++, of course, the declaration he's looking for is
> std::vector<SvrList>, without any manual new.

Exactly, unless the op is dealing with some of that legacy Hacked code.

Jeff

On Aug 18, 1:30=A0pm, thomas <freshtho...@gmail.com> wrote:
> Hi, I need your help.
>
> ----------
> struct SvrList{
> =A0 =A0 unsigned int uNum;
> =A0 =A0 GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A};
>
> ---------
>
> Once I declared a struct like this to store server list info.
> It's supposed to be used like this.
>
> ----------
> SvrList* pList =3D (SvrList*)malloc(sizeof(
> SvrList) + svrNum*sizeof(GameSvr));
> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> ---------
>
> The vs2005 compiler gives me the "nonstandard extension used : zero-
> sized array in struct/union" warning though.
>
> I may keep my eye closed to the warning since everything looks fine.
> But I don't know whether anything bad may happen to me some day due to
> running environment changes(porting to different platforms, or any
> other conditions).
>
> One work-around way is to declare a one-sized array as struct member,
> but I didn't see any substantial changes except no warning in this
> case.
>
> Any body give some suggestions?

If you ever plan to change actual number of elements in your object,
just drop this crap and use a vector. If not, ponder this (I am
presuming that your uNum is number of elements you have there):

template<typename T>
// hbvla: Heap Based Variable Length Array
class hbvla : public boost::noncopyable // Can't assign, nor copy-
construct.
{
public:
  void operator delete(void* p)
  {
    delete reinterpret_cast<char*>(p);
  }
  void operator delete(void* p, size_t /*cElements*/)
  { // Compiler calls this in case of exception in ctor
    // that gets called through overloaded operator new (see ^^^).
    // (note matched size_t cElements argument in two cases)
    delete reinterpret_cast<char*>(p);
  }

  static hbvla* create(size_t cElements) { return new (cElements)
hbvla(cElements); }

  ~hbvla()
  {
    destroy_elements();
  }

  // Trivial helper methods.
  size_t size() const { return _size; }
  T& operator[](size_t i) { assert(i < size()); return _elements[i]; }
  const T& operator[](size_t i) const { assert(i < size()); return
_elements[i]; }
  T* begin() { return _elements; }
  const T* begin() const { return _elements; }
  T* end() { return _elements+size(); }
  const T* end() const { return _elements+size(); }

private:
  size_t _size;
  T _elements[1]; // Standard does not allow 0.
  // 1 is OK, but I could have put anything due to particular design.

  // ^^^
  void* operator new(size_t objectSize, size_t cElements)
  { // Only this operator new shall be used.

    // Handle overflow.
    const size_t MaxElements =3D (SIZE_MAX-objectSize)/cElements;
    if (cElements > MaxElements)
      throw std::bad_alloc();

    return new char[objectSize + cElements*sizeof(T)];
  }

  /*
  Using "standard" operators new with hbvla is does not work, so hide
them
  (and don't implement them).

  The [] form is not really needed in this case because default ctor
is hidden,
  but I am leaving it here for informative purposes.

  operator delete[] does not work either.
  */
  void* operator new[](size_t objectSize);
  void* operator new(size_t objectSize);
  void operator delete[](void*);

  typedef hbvla<T> this_type; // helper typedef.

  // ctor: we must empty-construct our _elements from 1 to
(cElements-1).
  // Empty ctor is dangerous. But since we have a ctor, we don't need
to write and hide anything.
  hbvla(size_t cElements) : _size(0)
  {
    if (!cElements)
      return; // Strange but possible.

    // Construct elements from 1 to (cElements-1).
    // Use ScopeGuard to destruct partially-constructed array of
elements.
    _size =3D 1;
    ScopeGuard elementsGuard =3D MakeObjGuard(*this,
&this_type::destroy_elements);
    while (_size<cElements)
    {
      T* p =3D &_elements[_size];
      new (p) T;
      _size++;
    }
    elementsGuard.Dismiss();
  }

  void destroy_elements()
  { // destroy elements that we create "by hand"
    // (C++ runtime destroys _element[0]).
    std::for_each(begin()+1, end(), &this_type::destroy);
  }

  static void destroy(const T& element)
  {
    element.~T();
  }
};

Goran.

On 2010-08-19 07:15:29 -0400, James Kanze said:

> On Aug 18, 1:36 pm, Jeff Flinn <TriumphSprint2...@hotmail.com> wrote:
>> Vladimir Jovic wrote:
>>> thomas wrote:
>>>> Hi, I need your help.
> 
>>>> ----------
>>>> struct SvrList{
>>>> unsigned int uNum;
>>>> GameSvr  svr[0];            //line A
>>>> };
>>>> ---------
> 
>>>> Once I declared a struct like this to store server list info.
>>>> It's supposed to be used like this.
> 
>>>> ----------
>>>> SvrList* pList = (SvrList*)malloc(sizeof(
>>>> SvrList) + svrNum*sizeof(GameSvr));
>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> 
>>> I wouldn't call this fine. Even
>>> pList->svr[0]
>>> is accessing the element that is out of array's bounds, and
>>> that is UB.  How come your program is not crashing, or at
>>> least going crazy? Maybe you are just unlucky to have a bug
>>> hidden.
> 
>> It's an old C programmers hack. I've come across this idiom in
>> lot's old C code, particularly driver and os code. Microsoft
>> Win32 is rife with it.
> 
> Except that it's not legal C, either.

Which is why it's referred to above as a "hack". Quite a common one, too.

-- 
  Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com) Author of "The 
Standard C++ Library Extensions: a Tutorial and Reference 
(www.petebecker.com/tr1book)

On Wed, 2010-08-18, thomas wrote:
> Hi, I need your help.
>
> ----------
> struct SvrList{
>     unsigned int uNum;
>     GameSvr  svr[0];            //line A
> };
....

> The vs2005 compiler gives me the "nonstandard extension used : zero-
> sized array in struct/union" warning though.
>
> I may keep my eye closed to the warning since everything looks fine.
> But I don't know whether anything bad may happen to me some day due to
> running environment changes(porting to different platforms, or any
> other conditions).
>
> One work-around way is to declare a one-sized array as struct member,
> but I didn't see any substantial changes except no warning in this
> case.

That's a substantial change, IMHO.  Take gcc for example. There you
have (last time I checked) no way to disable that warning without
disabling C++98 conformance. That can be *really* annoying, when you
have seen the warning daily for a year or two ...

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .  .
\X/     snipabacken.se>   O  o   .

Vladimir Jovic <vladaspams@gmail.com> wrote in
news:i4iuof$s5j$1@news.albasani.net: 

> 
> This looks like a hack. Why declare an array of size 0? Why not use a 
> pointer (like someone suggested)? Is it another (now obsolete and 
> depreciated) way of declaring pointers?

Indeed, it is a hack, but a useful one.  Usually the array is declared of 
size 1 and not 0, but that doesn't make it any prettier.  You can't use a 
pointer because that would require 2 allocations and doesn't pass through 
the network near as well.  This is a mechanism for creating a struct with 
some dynamically sized data using only 1 buffer.  Memory allocation is not 
in anyway cheap (not only to you have the usual overhead, but there is also 
a mutex in there somewhere and possibly a kernel transistion) and in some 
cases, one needs to have only one.  This sort of thing can be the 
difference between passing your data in hours instead of days for some 
kinds of applications.  You certainly would only use this technique if 
profiling indicated that it was required, but it is useful to understand 
it.  

joe

James Kanze <james.kanze@gmail.com> wrote in news:19c5cd18-a44b-4b5f-b8f8-
11e1c956e585@x25g2000yqj.googlegroups.com:

> 
> In C99, flexible array members were introduced to support this
> idiom.  In C++, std::vector does the job a lot cleaner.
> 
> --
> James Kanze
> 

Not really.  This hack is used to reduce memory allocations.  Using a 
vector doesn't really do that.

joe

On Aug 18, 4:30=A0am, thomas <freshtho...@gmail.com> wrote:
> Hi, I need your help.
>
> ----------
> struct SvrList{
> =A0 =A0 unsigned int uNum;
> =A0 =A0 GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A};
>
> ---------
>
> Once I declared a struct like this to store server list info.
> It's supposed to be used like this.
>
> ----------
> SvrList* pList =3D (SvrList*)malloc(sizeof(
> SvrList) + svrNum*sizeof(GameSvr));
> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> ---------
>
> The vs2005 compiler gives me the "nonstandard extension used : zero-
> sized array in struct/union" warning though.
>
> I may keep my eye closed to the warning since everything looks fine.
> But I don't know whether anything bad may happen to me some day due to
> running environment changes(porting to different platforms, or any
> other conditions).
>
> One work-around way is to declare a one-sized array as struct member,
> but I didn't see any substantial changes except no warning in this
> case.
>
> Any body give some suggestions?

It is a useful nonstandard feature that gives you a warning. If you
allocate your struct big enough to hold your data, nothing bad will
happen. The extension is non-standard. If you use a pointer as has
been suggested elsewhere, you have to do two allocations.

On the other hand, this is very useful technique to point into a an
already allocated variable array of the GameSvr structs with the
number of them preceding.

Pete Becker wrote:
>>>>> Hi, I need your help.
>>>>> ----------
>>>>> struct SvrList{
>>>>> unsigned int uNum;
>>>>> GameSvr  svr[0];            //line A
>>>>> };
>>>>> ---------
>>>>> Once I declared a struct like this to store server list info.
>>>>> It's supposed to be used like this.
>>>>> ----------
>>>>> SvrList* pList = (SvrList*)malloc(sizeof(
>>>>> SvrList) + svrNum*sizeof(GameSvr));
>>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
>>>> I wouldn't call this fine. Even
>>>> pList->svr[0]
>>>> is accessing the element that is out of array's bounds, and
>>>> that is UB.  How come your program is not crashing, or at
>>>> least going crazy? Maybe you are just unlucky to have a bug
>>>> hidden.
>>> It's an old C programmers hack. I've come across this idiom in
>>> lot's old C code, particularly driver and os code. Microsoft
>>> Win32 is rife with it.
>> Except that it's not legal C, either.
> 
> Which is why it's referred to above as a "hack". Quite a common one, too.

Usually we use the term "hack" when the code relies on a specific 
manifestation of undefined (or unspecified) behavior, but otherwise is 
well-formed.

In this case the code is ill-formed, since 0-size array declaration is 
illegal in C++ (as well as in C). In other words, arguing about the "out 
of bounds" access here doesn't make much sense, since the code is 
formally non-compilable.

Declaring the array with size 1 formally turns that code into a "hack" 
by making it well-formed.

Of course, if one uses the `offsetof` method for calculating the memory 
block size

     SvrList* pList = (SvrList*) malloc(offsetof(SvrList, svr) +
       svrNum * sizeof(GameSvr));

one can declare the array with arbitrarily large size, thus potentially 
completely eliminating the chances of "out of bounds" access.

I wonder if it still a "hack" in that latter case. I.e. is the behavior 
defined if we allocate less raw memory than `sizeof` of the object for 
an object with a trivial constructor/destructor (assuming we never 
access the parts of the object that fall beyond the boundary of the 
allocated memory block)?

-- 
Best regards,
Andrey Tarasevich

On 2010-08-19 14:01:15 -0400, Andrey Tarasevich said:

> Pete Becker wrote:
>>>>>> Hi, I need your help.
>>>>>> ----------
>>>>>> struct SvrList{
>>>>>> unsigned int uNum;
>>>>>> GameSvr  svr[0];            //line A
>>>>>> };
>>>>>> ---------
>>>>>> Once I declared a struct like this to store server list info.
>>>>>> It's supposed to be used like this.
>>>>>> ----------
>>>>>> SvrList* pList = (SvrList*)malloc(sizeof(
>>>>>> SvrList) + svrNum*sizeof(GameSvr));
>>>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
>>>>> I wouldn't call this fine. Even
>>>>> pList->svr[0]
>>>>> is accessing the element that is out of array's bounds, and
>>>>> that is UB.  How come your program is not crashing, or at
>>>>> least going crazy? Maybe you are just unlucky to have a bug
>>>>> hidden.
>>>> It's an old C programmers hack. I've come across this idiom in
>>>> lot's old C code, particularly driver and os code. Microsoft
>>>> Win32 is rife with it.
>>> Except that it's not legal C, either.
>> 
>> Which is why it's referred to above as a "hack". Quite a common one, too.
> 
> Usually we use the term "hack" when the code relies on a specific 
> manifestation of undefined (or unspecified) behavior, but otherwise is 
> well-formed.

I wasn't aware that "we" do that. I don't think I'm going to redefine 
my vocabulary to match yours.

-- 
  Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com) Author of "The 
Standard C++ Library Extensions: a Tutorial and Reference 
(www.petebecker.com/tr1book)

Vladimir Jovic <vladaspams@gmail.com> wrote:
> If the array's size is zero, how can you access even one element?

  Simple: By indexing.

> The example similar to the original example

  "Similar" is not the same as "identical". You have a radical difference:
You are not allocating any memory for the array.

  Perhaps you missed the point that in the original example memory is being
"overallocated" with malloc() and the array indexing is done inside that
allocated memory.

  If you have a memory allocated with malloc(), you can do whatever you
want within that memory block. That's like the very definition of memory
allocation.

On Aug 19, 7:15=A0am, James Kanze <james.ka...@gmail.com> wrote:
> On Aug 18, 1:36 pm, Jeff Flinn <TriumphSprint2...@hotmail.com> wrote:
> > Vladimir Jovic wrote:
> > > thomas wrote:
> > >> Hi, I need your help.
> > >> ----------
> > >> struct SvrList{
> > >> =A0 =A0 unsigned int uNum;
> > >> =A0 =A0 GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> > >> };
> > >> ---------
> > >> Once I declared a struct like this to store server list info.
> > >> It's supposed to be used like this.
> > >> ----------
> > >> SvrList* pList =3D (SvrList*)malloc(sizeof(
> > >> SvrList) + svrNum*sizeof(GameSvr));
> > >> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> > > I wouldn't call this fine. Even
> > > pList->svr[0]
> > > is accessing the element that is out of array's bounds, and
> > > that is UB. =A0How come your program is not crashing, or at
> > > least going crazy? Maybe you are just unlucky to have a bug
> > > hidden.
> > It's an old C programmers hack. I've come across this idiom in
> > lot's old C code, particularly driver and os code. Microsoft
> > Win32 is rife with it.
>
> Except that it's not legal C, either. =A0The C language was
> carefully specified so that a bounds checking implementation
> would be legal, and the above will fail in such cases.
>
> In C++, of course, the declaration he's looking for is
> std::vector<SvrList>, without any manual new.

You mean

   std::vector<GameSvr>

right?

KHD

Vladimir Jovic <vladaspams@gmail.com> wrote:
> What bothers me is how can you use an array of size zero.

  You see, there's this operator which uses the [ and ] symbols...

> This looks like a hack. Why declare an array of size 0?

  What size would you want to use, given that the size is determined
dynamically at runtime?

> Why not use a pointer (like someone suggested)?

  Because indexing a pointer would access the memory pointed by it.
Where would you make the pointer point to?

James Kanze <james.kanze@gmail.com> wrote:
> In C++, of course, the declaration he's looking for is
> std::vector<SvrList>, without any manual new.

  Except that it doesn't do the same thing.

  It's a bit like the difference between a two-dimensional array and an
array of pointers (all of which point to separately allocated arrays of
their own). Accessing them might use the same syntax, but they are not
the same thing.

On 08/20/10 12:50 AM, Pete Becker wrote:
> On 2010-08-19 07:15:29 -0400, James Kanze said:
>
>> On Aug 18, 1:36 pm, Jeff Flinn <TriumphSprint2...@hotmail.com> wrote:
>>> Vladimir Jovic wrote:
>>
>>>> I wouldn't call this fine. Even
>>>> pList->svr[0]
>>>> is accessing the element that is out of array's bounds, and
>>>> that is UB. How come your program is not crashing, or at
>>>> least going crazy? Maybe you are just unlucky to have a bug
>>>> hidden.
>>
>>> It's an old C programmers hack. I've come across this idiom in
>>> lot's old C code, particularly driver and os code. Microsoft
>>> Win32 is rife with it.
>>
>> Except that it's not legal C, either.
>
> Which is why it's referred to above as a "hack". Quite a common one, too.

So common it was standardised (in a slightly different form) in C99 
(6.7.2.1 p16).  "struct hack" even appears in the standard's index!

-- 
Ian Collins

On 18 Aug., 13:30, thomas <freshtho...@gmail.com> wrote:
> Hi, I need your help.
>
> ----------
> struct SvrList{
> =A0 =A0 unsigned int uNum;
> =A0 =A0 GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A};
>
> ---------


> The vs2005 compiler gives me the "nonstandard extension used :zero-
> sized array in struct/union" warning though.

> Any body give some suggestions?

1)

If this is a compiler-specific warning, you could suppress it compiler-
specific:

#ifdef _MSCVER
#pragma warning(push)
#pragma warning(disable:4200)
// define your sruct
#pragma warning(pop)
#endif

2)
What about a placement new?

Juha Nieminen wrote:
> Vladimir Jovic <vladaspams@gmail.com> wrote:
>> What bothers me is how can you use an array of size zero.
> 
>   You see, there's this operator which uses the [ and ] symbols...
> 
>> This looks like a hack. Why declare an array of size 0?
> 
>   What size would you want to use, given that the size is determined
> dynamically at runtime?
> 
>> Why not use a pointer (like someone suggested)?
> 
>   Because indexing a pointer would access the memory pointed by it.
> Where would you make the pointer point to?

What is wrong with next code?


struct SvrList{
     unsigned int uNum;
     GameSvr  *svr;
};

....
SvrList list;
list.uNum = 5;
list.svr = (GameSvr*) malloc( list.uNum * sizeof( GameSvr ) );
....


Since someone pointed out that the original code is used for 
performances reasons, how is this worse then the code in the first post?

On Aug 19, 7:15=A0pm, James Kanze <james.ka...@gmail.com> wrote:
> On Aug 18, 1:36 pm, Jeff Flinn <TriumphSprint2...@hotmail.com> wrote:
>
>
>
> > Vladimir Jovic wrote:
> > > thomas wrote:
> > >> Hi, I need your help.
> > >> ----------
> > >> struct SvrList{
> > >> =A0 =A0 unsigned int uNum;
> > >> =A0 =A0 GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> > >> };
> > >> ---------
> > >> Once I declared a struct like this to store server list info.
> > >> It's supposed to be used like this.
> > >> ----------
> > >> SvrList* pList =3D (SvrList*)malloc(sizeof(
> > >> SvrList) + svrNum*sizeof(GameSvr));
> > >> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> > > I wouldn't call this fine. Even
> > > pList->svr[0]
> > > is accessing the element that is out of array's bounds, and
> > > that is UB. =A0How come your program is not crashing, or at
> > > least going crazy? Maybe you are just unlucky to have a bug
> > > hidden.
> > It's an old C programmers hack. I've come across this idiom in
> > lot's old C code, particularly driver and os code. Microsoft
> > Win32 is rife with it.
>
> Except that it's not legal C, either. =A0The C language was
> carefully specified so that a bounds checking implementation
> would be legal, and the above will fail in such cases.
>
> In C++, of course, the declaration he's looking for is
> std::vector<SvrList>, without any manual new.
>
> --
> James Kanze

But I hate vectors. Different distributors may have different
implementations. When I pass vectors between dlls, things unexpected
may happen.

On Aug 20, 2:01=A0am, Andrey Tarasevich <andreytarasev...@hotmail.com>
wrote:
> Pete Becker wrote:
> >>>>> Hi, I need your help.
> >>>>> ----------
> >>>>> struct SvrList{
> >>>>> unsigned int uNum;
> >>>>> GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> >>>>> };
> >>>>> ---------
> >>>>> Once I declared a struct like this to store server list info.
> >>>>> It's supposed to be used like this.
> >>>>> ----------
> >>>>> SvrList* pList =3D (SvrList*)malloc(sizeof(
> >>>>> SvrList) + svrNum*sizeof(GameSvr));
> >>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> >>>> I wouldn't call this fine. Even
> >>>> pList->svr[0]
> >>>> is accessing the element that is out of array's bounds, and
> >>>> that is UB. =A0How come your program is not crashing, or at
> >>>> least going crazy? Maybe you are just unlucky to have a bug
> >>>> hidden.
> >>> It's an old C programmers hack. I've come across this idiom in
> >>> lot's old C code, particularly driver and os code. Microsoft
> >>> Win32 is rife with it.
> >> Except that it's not legal C, either.
>
> > Which is why it's referred to above as a "hack". Quite a common one, to=
o.
>
> Usually we use the term "hack" when the code relies on a specific
> manifestation of undefined (or unspecified) behavior, but otherwise is
> well-formed.
>
> In this case the code is ill-formed, since 0-size array declaration is
> illegal in C++ (as well as in C). In other words, arguing about the "out
> of bounds" access here doesn't make much sense, since the code is
> formally non-compilable.

Wait.. I don't think it's illegal in C++. At least I will definitely
object making it illegal by the standard community.
It can be dangerous but it can also do good. It depends on whether we
are using it correctly.
Will you guys getting crazy if we do things like this

-------code-----
struct A{
    int num;
    int p[0];
};
A *pA =3D (A*)malloc(sizeof(A)+sizeof(int)*10);
printf("%d\n", &pA->p[10] - &pA->p[0]);           //accessing out of
bounds.
------code---

>
> Declaring the array with size 1 formally turns that code into a "hack"
> by making it well-formed.
>
> Of course, if one uses the `offsetof` method for calculating the memory
> block size
>
> =A0 =A0 =A0SvrList* pList =3D (SvrList*) malloc(offsetof(SvrList, svr) +
> =A0 =A0 =A0 =A0svrNum * sizeof(GameSvr));
>
> one can declare the array with arbitrarily large size, thus potentially
> completely eliminating the chances of "out of bounds" access.
>
> I wonder if it still a "hack" in that latter case. I.e. is the behavior
> defined if we allocate less raw memory than `sizeof` of the object for
> an object with a trivial constructor/destructor (assuming we never
> access the parts of the object that fall beyond the boundary of the
> allocated memory block)?
>
> --
> Best regards,
> Andrey Tarasevich

Juha Nieminen wrote:
> Vladimir Jovic <vladaspams@gmail.com> wrote:
>> If the array's size is zero, how can you access even one element?
> 
>   Simple: By indexing.
> 
>> The example similar to the original example
> 
>   "Similar" is not the same as "identical". You have a radical difference:
> You are not allocating any memory for the array.
> 
>   Perhaps you missed the point that in the original example memory is being
> "overallocated" with malloc() and the array indexing is done inside that
> allocated memory.
> 
>   If you have a memory allocated with malloc(), you can do whatever you
> want within that memory block. That's like the very definition of memory
> allocation.

Yes, I must admit I missed that when I posted that code, but later I 
realised what was done. The question is still valid : why was it 
declared like array of zero size, and not like pointer?

Your point with operator[] doesn't stand (else thread), as you can use 
it for pointers as well.

On Aug 20, 4:23=A0pm, Vladimir Jovic <vladasp...@gmail.com> wrote:
> Juha Nieminen wrote:
> > Vladimir Jovic <vladasp...@gmail.com> wrote:
> >> If the array's size is zero, how can you access even one element?
>
> > =A0 Simple: By indexing.
>
> >> The example similar to the original example
>
> > =A0 "Similar" is not the same as "identical". You have a radical differ=
ence:
> > You are not allocating any memory for the array.
>
> > =A0 Perhaps you missed the point that in the original example memory is=
 being
> > "overallocated" with malloc() and the array indexing is done inside tha=
t
> > allocated memory.
>
> > =A0 If you have a memory allocated with malloc(), you can do whatever y=
ou
> > want within that memory block. That's like the very definition of memor=
y
> > allocation.
>
> Yes, I must admit I missed that when I posted that code, but later I
> realised what was done. The question is still valid : why was it
> declared like array of zero size, and not like pointer?
>
> Your point with operator[] doesn't stand (else thread), as you can use
> it for pointers as well.

Different styles. It definitely works if defining it as a pointer.
But I like the other way, why not?

On 20 aug, 11:19, thomas <freshtho...@gmail.com> wrote:
> On Aug 20, 2:01=A0am, Andrey Tarasevich <andreytarasev...@hotmail.com>
> wrote:
>
>
>
> > Pete Becker wrote:
> > >>>>> Hi, I need your help.
> > >>>>> ----------
> > >>>>> struct SvrList{
> > >>>>> unsigned int uNum;
> > >>>>> GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> > >>>>> };
> > >>>>> ---------
> > >>>>> Once I declared a struct like this to store server list info.
> > >>>>> It's supposed to be used like this.
> > >>>>> ----------
> > >>>>> SvrList* pList =3D (SvrList*)malloc(sizeof(
> > >>>>> SvrList) + svrNum*sizeof(GameSvr));
> > >>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> > >>>> I wouldn't call this fine. Even
> > >>>> pList->svr[0]
> > >>>> is accessing the element that is out of array's bounds, and
> > >>>> that is UB. =A0How come your program is not crashing, or at
> > >>>> least going crazy? Maybe you are just unlucky to have a bug
> > >>>> hidden.
> > >>> It's an old C programmers hack. I've come across this idiom in
> > >>> lot's old C code, particularly driver and os code. Microsoft
> > >>> Win32 is rife with it.
> > >> Except that it's not legal C, either.
>
> > > Which is why it's referred to above as a "hack". Quite a common one, =
too.
>
> > Usually we use the term "hack" when the code relies on a specific
> > manifestation of undefined (or unspecified) behavior, but otherwise is
> > well-formed.
>
> > In this case the code is ill-formed, since 0-size array declaration is
> > illegal in C++ (as well as in C). In other words, arguing about the "ou=
t
> > of bounds" access here doesn't make much sense, since the code is
> > formally non-compilable.
>
> Wait.. I don't think it's illegal in C++. At least I will definitely
> object making it illegal by the standard community.
> It can be dangerous but it can also do good. It depends on whether we
> are using it correctly.
> Will you guys getting crazy if we do things like this
>
> -------code-----
> struct A{
> =A0 =A0 int num;
> =A0 =A0 int p[0];};
>
> A *pA =3D (A*)malloc(sizeof(A)+sizeof(int)*10);
> printf("%d\n", &pA->p[10] - &pA->p[0]); =A0 =A0 =A0 =A0 =A0 //accessing o=
ut of
> bounds.
> ------code---

Crazy? Why? C++ compilers should not simply compile it by language
rules that does not allow zero-sized arrays. If a compiler compiles it
then it is non-standard extension.

I have seen similar C code, but that used "int p[1];". If it was
compiled with C++ compiler and run-time bounds-checking on then it did
fault when accessing anything but p[0].

In C++ you should use vector<int> instead, it is about as fast.

On Aug 20, 4:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
> On 20 aug, 11:19, thomas <freshtho...@gmail.com> wrote:
>
>
>
> > On Aug 20, 2:01=A0am, Andrey Tarasevich <andreytarasev...@hotmail.com>
> > wrote:
>
> > > Pete Becker wrote:
> > > >>>>> Hi, I need your help.
> > > >>>>> ----------
> > > >>>>> struct SvrList{
> > > >>>>> unsigned int uNum;
> > > >>>>> GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> > > >>>>> };
> > > >>>>> ---------
> > > >>>>> Once I declared a struct like this to store server list info.
> > > >>>>> It's supposed to be used like this.
> > > >>>>> ----------
> > > >>>>> SvrList* pList =3D (SvrList*)malloc(sizeof(
> > > >>>>> SvrList) + svrNum*sizeof(GameSvr));
> > > >>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> > > >>>> I wouldn't call this fine. Even
> > > >>>> pList->svr[0]
> > > >>>> is accessing the element that is out of array's bounds, and
> > > >>>> that is UB. =A0How come your program is not crashing, or at
> > > >>>> least going crazy? Maybe you are just unlucky to have a bug
> > > >>>> hidden.
> > > >>> It's an old C programmers hack. I've come across this idiom in
> > > >>> lot's old C code, particularly driver and os code. Microsoft
> > > >>> Win32 is rife with it.
> > > >> Except that it's not legal C, either.
>
> > > > Which is why it's referred to above as a "hack". Quite a common one=
, too.
>
> > > Usually we use the term "hack" when the code relies on a specific
> > > manifestation of undefined (or unspecified) behavior, but otherwise i=
s
> > > well-formed.
>
> > > In this case the code is ill-formed, since 0-size array declaration i=
s
> > > illegal in C++ (as well as in C). In other words, arguing about the "=
out
> > > of bounds" access here doesn't make much sense, since the code is
> > > formally non-compilable.
>
> > Wait.. I don't think it's illegal in C++. At least I will definitely
> > object making it illegal by the standard community.
> > It can be dangerous but it can also do good. It depends on whether we
> > are using it correctly.
> > Will you guys getting crazy if we do things like this
>
> > -------code-----
> > struct A{
> > =A0 =A0 int num;
> > =A0 =A0 int p[0];};
>
> > A *pA =3D (A*)malloc(sizeof(A)+sizeof(int)*10);
> > printf("%d\n", &pA->p[10] - &pA->p[0]); =A0 =A0 =A0 =A0 =A0 //accessing=
 out of
> > bounds.
> > ------code---
>
> Crazy? Why? C++ compilers should not simply compile it by language
> rules that does not allow zero-sized arrays. If a compiler compiles it
> then it is non-standard extension.
>
> I have seen similar C code, but that used "int p[1];". If it was
> compiled with C++ compiler and run-time bounds-checking on then it did
> fault when accessing anything but p[0].
>
> In C++ you should use vector<int> instead, it is about as fast.

I hate vectors, again. C++ is not portable as binary code.
Passing vectors as arguments out of a dll in windows may crash the
system when accessing it.

On 20 aug, 11:03, Vladimir Jovic <vladasp...@gmail.com> wrote:
> Juha Nieminen wrote:
> > Vladimir Jovic <vladasp...@gmail.com> wrote:
> >> What bothers me is how can you use an array of size zero.
>
> > =A0 You see, there's this operator which uses the [ and ] symbols...
>
> >> This looks like a hack. Why declare an array of size 0?
>
> > =A0 What size would you want to use, given that the size is determined
> > dynamically at runtime?
>
> >> Why not use a pointer (like someone suggested)?
>
> > =A0 Because indexing a pointer would access the memory pointed by it.
> > Where would you make the pointer point to?
>
> What is wrong with next code?
>
> struct SvrList{
> =A0 =A0 =A0unsigned int uNum;
> =A0 =A0 =A0GameSvr =A0*svr;
>
> };
>
> ...
> SvrList list;
> list.uNum =3D 5;
> list.svr =3D (GameSvr*) malloc( list.uNum * sizeof( GameSvr ) );
> ...
>
> Since someone pointed out that the original code is used for
> performances reasons, how is this worse then the code in the first post?

Because you create SvrList with automatic storage type and so pretend
its costless. Actual code equal to original should allocate SvrList
itself too dynamically together with place for GameSvr it contains.
That makes two mallocs and malloc may be one of the slowest things on
some crappy platforms.

On 2010-08-20 10:19, thomas wrote:
>> In this case the code is ill-formed, since 0-size array declaration is
>> illegal in C++ (as well as in C). In other words, arguing about the "out
>> of bounds" access here doesn't make much sense, since the code is
>> formally non-compilable.
>
> Wait.. I don't think it's illegal in C++. At least I will definitely
> object making it illegal by the standard community.

None of the C or C++ standards allow an array size of 0.

C99 has flexible arrays (which do basically the same thing), but they 
are not included in C++03. Unfortunately, it doesn't look like C99 
flexible arrays are part of C++0x either. The somewhat good news is that 
GCC, Visual C++ 2010 (and presumably Intel C++) do support them.

(GCC and Visual C++ support 0-sized arrays as extension.)

thomas wrote:
> On Aug 20, 2:01 am, Andrey Tarasevich <andreytarasev...@hotmail.com>
> wrote:
>> Pete Becker wrote:
>>>>>>> Hi, I need your help.
>>>>>>> ----------
>>>>>>> struct SvrList{
>>>>>>> unsigned int uNum;
>>>>>>> GameSvr  svr[0];            //line A
>>>>>>> };
>>>>>>> ---------
>>>>>>> Once I declared a struct like this to store server list info.
>>>>>>> It's supposed to be used like this.
>>>>>>> ----------
>>>>>>> SvrList* pList = (SvrList*)malloc(sizeof(
>>>>>>> SvrList) + svrNum*sizeof(GameSvr));
>>>>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
>>>>>> I wouldn't call this fine. Even
>>>>>> pList->svr[0]
>>>>>> is accessing the element that is out of array's bounds, and
>>>>>> that is UB.  How come your program is not crashing, or at
>>>>>> least going crazy? Maybe you are just unlucky to have a bug
>>>>>> hidden.
>>>>> It's an old C programmers hack. I've come across this idiom in
>>>>> lot's old C code, particularly driver and os code. Microsoft
>>>>> Win32 is rife with it.
>>>> Except that it's not legal C, either.
>>> Which is why it's referred to above as a "hack". Quite a common one, too.
>> Usually we use the term "hack" when the code relies on a specific
>> manifestation of undefined (or unspecified) behavior, but otherwise is
>> well-formed.
>>
>> In this case the code is ill-formed, since 0-size array declaration is
>> illegal in C++ (as well as in C). In other words, arguing about the "out
>> of bounds" access here doesn't make much sense, since the code is
>> formally non-compilable.
> 
> Wait.. I don't think it's illegal in C++. At least I will definitely
> object making it illegal by the standard community.

In the c++ standard, see 8.3.4.1 , this part :

.... If the constant-expression (5.19) is present, it shall be an 
integral constant expression and its value shall be greater than zero...

therefore it is illegal c++ code.

> It can be dangerous but it can also do good. It depends on whether we
> are using it correctly.

Can't use it correctly. It is illegal, therefore undefined behaviour.

> Will you guys getting crazy if we do things like this
> 
> -------code-----
> struct A{
>     int num;
>     int p[0];
> };
> A *pA = (A*)malloc(sizeof(A)+sizeof(int)*10);
> printf("%d\n", &pA->p[10] - &pA->p[0]);           //accessing out of
> bounds.
> ------code---

No, but compiler will go crazy (thanks gcc for -Werror flag :) )

> But I hate vectors. Different distributors may have different
> implementations. When I pass vectors between dlls, things unexpected
> may happen.

C++ language knows nothing of DLLs. Your first error there is trying
to expose C++ - specific (and not only that, but potentially
particular compiler and standard library version version specific)
interface out of your module.

This is such a massive no-no that it's not even funny. You can only do
that if all modules will be compiled with same compiler (version,
too), and if interface uses standard library, if modules will use same
standard library version and even compilation options. For example,
with STL shipped with VS, compiling a DLL exposing standard containers
in it's interface with/without SECURE_STL breaks low-level binary
compatibility for any container.

It's not a question of liking at all. This sort of interoperability
works only under very specific circumstances (e.g. if you control, or
at least mandate, the build process and the deployment process
__entirely__).

Goran.

thomas <freshthomas@gmail.com>, on 20/08/2010 01:06:51, wrote:

> On Aug 19, 7:15 pm, James Kanze<james.ka...@gmail.com>  wrote:
>> On Aug 18, 1:36 pm, Jeff Flinn<TriumphSprint2...@hotmail.com>  wrote:
>>
>>
>>
>>> Vladimir Jovic wrote:
>>>> thomas wrote:
>>>>> Hi, I need your help.
>>>>> ----------
>>>>> struct SvrList{
>>>>>      unsigned int uNum;
>>>>>      GameSvr  svr[0];            //line A
>>>>> };
>>>>> ---------
>>>>> Once I declared a struct like this to store server list info.
>>>>> It's supposed to be used like this.
>>>>> ----------
>>>>> SvrList* pList = (SvrList*)malloc(sizeof(
>>>>> SvrList) + svrNum*sizeof(GameSvr));
>>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
>>>> I wouldn't call this fine. Even
>>>> pList->svr[0]
>>>> is accessing the element that is out of array's bounds, and
>>>> that is UB.  How come your program is not crashing, or at
>>>> least going crazy? Maybe you are just unlucky to have a bug
>>>> hidden.
>>> It's an old C programmers hack. I've come across this idiom in
>>> lot's old C code, particularly driver and os code. Microsoft
>>> Win32 is rife with it.
>>
>> Except that it's not legal C, either.  The C language was
>> carefully specified so that a bounds checking implementation
>> would be legal, and the above will fail in such cases.
>>
>> In C++, of course, the declaration he's looking for is
>> std::vector<SvrList>, without any manual new.
>>
>> --
>> James Kanze
>
> But I hate vectors. Different distributors may have different
> implementations. When I pass vectors between dlls, things unexpected
> may happen.

You could define interfaces using "C" linkage and send out vectors as 
arrays by simply passing the address of the first element and the size 
of the vector, just to make the communication layer happy and compatible.

-- 
  FSC - http://userscripts.org/scripts/show/59948
  http://fscode.altervista.org - http://sardinias.com

On Aug 20, 4:55=A0pm, Goran Pusic <gor...@cse-semaphore.com> wrote:
> > But I hate vectors. Different distributors may have different
> > implementations. When I pass vectors between dlls, things unexpected
> > may happen.
>
> C++ language knows nothing of DLLs. Your first error there is trying
> to expose C++ - specific (and not only that, but potentially
> particular compiler and standard library version version specific)
> interface out of your module.
>
> This is such a massive no-no that it's not even funny. You can only do
> that if all modules will be compiled with same compiler (version,
> too), and if interface uses standard library, if modules will use same
> standard library version and even compilation options. For example,
> with STL shipped with VS, compiling a DLL exposing standard containers
> in it's interface with/without SECURE_STL breaks low-level binary
> compatibility for any container.
>
> It's not a question of liking at all. This sort of interoperability
> works only under very specific circumstances (e.g. if you control, or
> at least mandate, the build process and the deployment process
> __entirely__).
>
> Goran.

I like your words.
In my understanding, your points are:
1. using STL containers in module interface may cause interoperability
problems.
2. my zero-sized array is illegal in C++, so may also cause
interoperability problems.

Both should be avoided.
So the only way is to using pointers or one-sized array.

On Aug 20, 5:01=A0pm, "Francesco S. Carta" <entul...@gmail.com> wrote:
> thomas <freshtho...@gmail.com>, on 20/08/2010 01:06:51, wrote:
>
>
>
> > On Aug 19, 7:15 pm, James Kanze<james.ka...@gmail.com> =A0wrote:
> >> On Aug 18, 1:36 pm, Jeff Flinn<TriumphSprint2...@hotmail.com> =A0wrote=
:
>
> >>> Vladimir Jovic wrote:
> >>>> thomas wrote:
> >>>>> Hi, I need your help.
> >>>>> ----------
> >>>>> struct SvrList{
> >>>>> =A0 =A0 =A0unsigned int uNum;
> >>>>> =A0 =A0 =A0GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> >>>>> };
> >>>>> ---------
> >>>>> Once I declared a struct like this to store server list info.
> >>>>> It's supposed to be used like this.
> >>>>> ----------
> >>>>> SvrList* pList =3D (SvrList*)malloc(sizeof(
> >>>>> SvrList) + svrNum*sizeof(GameSvr));
> >>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> >>>> I wouldn't call this fine. Even
> >>>> pList->svr[0]
> >>>> is accessing the element that is out of array's bounds, and
> >>>> that is UB. =A0How come your program is not crashing, or at
> >>>> least going crazy? Maybe you are just unlucky to have a bug
> >>>> hidden.
> >>> It's an old C programmers hack. I've come across this idiom in
> >>> lot's old C code, particularly driver and os code. Microsoft
> >>> Win32 is rife with it.
>
> >> Except that it's not legal C, either. =A0The C language was
> >> carefully specified so that a bounds checking implementation
> >> would be legal, and the above will fail in such cases.
>
> >> In C++, of course, the declaration he's looking for is
> >> std::vector<SvrList>, without any manual new.
>
> >> --
> >> James Kanze
>
> > But I hate vectors. Different distributors may have different
> > implementations. When I pass vectors between dlls, things unexpected
> > may happen.
>
> You could define interfaces using "C" linkage and send out vectors as
> arrays by simply passing the address of the first element and the size
> of the vector, just to make the communication layer happy and compatible.
>
> --
> =A0 FSC -http://userscripts.org/scripts/show/59948
> =A0http://fscode.altervista.org-http://sardinias.com

gotcha!

On 20 aug, 11:45, thomas <freshtho...@gmail.com> wrote:
> On Aug 20, 4:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
>
>
> > On 20 aug, 11:19, thomas <freshtho...@gmail.com> wrote:
>
> > > On Aug 20, 2:01=A0am, Andrey Tarasevich <andreytarasev...@hotmail.com=
>
> > > wrote:
>
> > > > Pete Becker wrote:
> > > > >>>>> Hi, I need your help.
> > > > >>>>> ----------
> > > > >>>>> struct SvrList{
> > > > >>>>> unsigned int uNum;
> > > > >>>>> GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> > > > >>>>> };
> > > > >>>>> ---------
> > > > >>>>> Once I declared a struct like this to store server list info.
> > > > >>>>> It's supposed to be used like this.
> > > > >>>>> ----------
> > > > >>>>> SvrList* pList =3D (SvrList*)malloc(sizeof(
> > > > >>>>> SvrList) + svrNum*sizeof(GameSvr));
> > > > >>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> > > > >>>> I wouldn't call this fine. Even
> > > > >>>> pList->svr[0]
> > > > >>>> is accessing the element that is out of array's bounds, and
> > > > >>>> that is UB. =A0How come your program is not crashing, or at
> > > > >>>> least going crazy? Maybe you are just unlucky to have a bug
> > > > >>>> hidden.
> > > > >>> It's an old C programmers hack. I've come across this idiom in
> > > > >>> lot's old C code, particularly driver and os code. Microsoft
> > > > >>> Win32 is rife with it.
> > > > >> Except that it's not legal C, either.
>
> > > > > Which is why it's referred to above as a "hack". Quite a common o=
ne, too.
>
> > > > Usually we use the term "hack" when the code relies on a specific
> > > > manifestation of undefined (or unspecified) behavior, but otherwise=
 is
> > > > well-formed.
>
> > > > In this case the code is ill-formed, since 0-size array declaration=
 is
> > > > illegal in C++ (as well as in C). In other words, arguing about the=
 "out
> > > > of bounds" access here doesn't make much sense, since the code is
> > > > formally non-compilable.
>
> > > Wait.. I don't think it's illegal in C++. At least I will definitely
> > > object making it illegal by the standard community.
> > > It can be dangerous but it can also do good. It depends on whether we
> > > are using it correctly.
> > > Will you guys getting crazy if we do things like this
>
> > > -------code-----
> > > struct A{
> > > =A0 =A0 int num;
> > > =A0 =A0 int p[0];};
>
> > > A *pA =3D (A*)malloc(sizeof(A)+sizeof(int)*10);
> > > printf("%d\n", &pA->p[10] - &pA->p[0]); =A0 =A0 =A0 =A0 =A0 //accessi=
ng out of
> > > bounds.
> > > ------code---
>
> > Crazy? Why? C++ compilers should not simply compile it by language
> > rules that does not allow zero-sized arrays. If a compiler compiles it
> > then it is non-standard extension.
>
> > I have seen similar C code, but that used "int p[1];". If it was
> > compiled with C++ compiler and run-time bounds-checking on then it did
> > fault when accessing anything but p[0].
>
> > In C++ you should use vector<int> instead, it is about as fast.
>
> I hate vectors, again. C++ is not portable as binary code.
> Passing vectors as arguments out of a dll in windows may crash the
> system when accessing it.

Assuming binary compatibility without effort is wrong anyway. Are you
seriously claiming that such originally-posted hacks let you to
achieve binary compatibility between modules compiled with different C
compilers and under different compilation options?

On Aug 20, 5:12=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
> On 20 aug, 11:45, thomas <freshtho...@gmail.com> wrote:
>
>
>
> > On Aug 20, 4:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > On 20 aug, 11:19, thomas <freshtho...@gmail.com> wrote:
>
> > > > On Aug 20, 2:01=A0am, Andrey Tarasevich <andreytarasev...@hotmail.c=
om>
> > > > wrote:
>
> > > > > Pete Becker wrote:
> > > > > >>>>> Hi, I need your help.
> > > > > >>>>> ----------
> > > > > >>>>> struct SvrList{
> > > > > >>>>> unsigned int uNum;
> > > > > >>>>> GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> > > > > >>>>> };
> > > > > >>>>> ---------
> > > > > >>>>> Once I declared a struct like this to store server list inf=
o.
> > > > > >>>>> It's supposed to be used like this.
> > > > > >>>>> ----------
> > > > > >>>>> SvrList* pList =3D (SvrList*)malloc(sizeof(
> > > > > >>>>> SvrList) + svrNum*sizeof(GameSvr));
> > > > > >>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> > > > > >>>> I wouldn't call this fine. Even
> > > > > >>>> pList->svr[0]
> > > > > >>>> is accessing the element that is out of array's bounds, and
> > > > > >>>> that is UB. =A0How come your program is not crashing, or at
> > > > > >>>> least going crazy? Maybe you are just unlucky to have a bug
> > > > > >>>> hidden.
> > > > > >>> It's an old C programmers hack. I've come across this idiom i=
n
> > > > > >>> lot's old C code, particularly driver and os code. Microsoft
> > > > > >>> Win32 is rife with it.
> > > > > >> Except that it's not legal C, either.
>
> > > > > > Which is why it's referred to above as a "hack". Quite a common=
 one, too.
>
> > > > > Usually we use the term "hack" when the code relies on a specific
> > > > > manifestation of undefined (or unspecified) behavior, but otherwi=
se is
> > > > > well-formed.
>
> > > > > In this case the code is ill-formed, since 0-size array declarati=
on is
> > > > > illegal in C++ (as well as in C). In other words, arguing about t=
he "out
> > > > > of bounds" access here doesn't make much sense, since the code is
> > > > > formally non-compilable.
>
> > > > Wait.. I don't think it's illegal in C++. At least I will definitel=
y
> > > > object making it illegal by the standard community.
> > > > It can be dangerous but it can also do good. It depends on whether =
we
> > > > are using it correctly.
> > > > Will you guys getting crazy if we do things like this
>
> > > > -------code-----
> > > > struct A{
> > > > =A0 =A0 int num;
> > > > =A0 =A0 int p[0];};
>
> > > > A *pA =3D (A*)malloc(sizeof(A)+sizeof(int)*10);
> > > > printf("%d\n", &pA->p[10] - &pA->p[0]); =A0 =A0 =A0 =A0 =A0 //acces=
sing out of
> > > > bounds.
> > > > ------code---
>
> > > Crazy? Why? C++ compilers should not simply compile it by language
> > > rules that does not allow zero-sized arrays. If a compiler compiles i=
t
> > > then it is non-standard extension.
>
> > > I have seen similar C code, but that used "int p[1];". If it was
> > > compiled with C++ compiler and run-time bounds-checking on then it di=
d
> > > fault when accessing anything but p[0].
>
> > > In C++ you should use vector<int> instead, it is about as fast.
>
> > I hate vectors, again. C++ is not portable as binary code.
> > Passing vectors as arguments out of a dll in windows may crash the
> > system when accessing it.
>
> Assuming binary compatibility without effort is wrong anyway. Are you
> seriously claiming that such originally-posted hacks let you to
> achieve binary compatibility between modules compiled with different C
> compilers and under different compilation options?

nope.. I mean using vectors in module interface may cause
interoperability problems.
My hack can solve the problem in case using the same padding policy
for modules involved.

thomas <freshthomas@gmail.com>, on 20/08/2010 02:12:23, wrote:
> On Aug 20, 5:01 pm, "Francesco S. Carta"<entul...@gmail.com>  wrote:
>> thomas<freshtho...@gmail.com>, on 20/08/2010 01:06:51, wrote:

<snip>

>>> But I hate vectors. Different distributors may have different
>>> implementations. When I pass vectors between dlls, things unexpected
>>> may happen.
>>
>> You could define interfaces using "C" linkage and send out vectors as
>> arrays by simply passing the address of the first element and the size
>> of the vector, just to make the communication layer happy and compatible.

<snip sigs please>

> gotcha!

What?

-- 
  FSC - http://userscripts.org/scripts/show/59948
  http://fscode.altervista.org - http://sardinias.com

On 20 aug, 12:20, thomas <freshtho...@gmail.com> wrote:
> On Aug 20, 5:12=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
>
>
> > On 20 aug, 11:45, thomas <freshtho...@gmail.com> wrote:
>
> > > On Aug 20, 4:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > > On 20 aug, 11:19, thomas <freshtho...@gmail.com> wrote:
>
> > > > > On Aug 20, 2:01=A0am, Andrey Tarasevich <andreytarasev...@hotmail=
..com>
> > > > > wrote:
>
> > > > > > Pete Becker wrote:
> > > > > > >>>>> Hi, I need your help.
> > > > > > >>>>> ----------
> > > > > > >>>>> struct SvrList{
> > > > > > >>>>> unsigned int uNum;
> > > > > > >>>>> GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> > > > > > >>>>> };
> > > > > > >>>>> ---------
> > > > > > >>>>> Once I declared a struct like this to store server list i=
nfo.
> > > > > > >>>>> It's supposed to be used like this.
> > > > > > >>>>> ----------
> > > > > > >>>>> SvrList* pList =3D (SvrList*)malloc(sizeof(
> > > > > > >>>>> SvrList) + svrNum*sizeof(GameSvr));
> > > > > > >>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> > > > > > >>>> I wouldn't call this fine. Even
> > > > > > >>>> pList->svr[0]
> > > > > > >>>> is accessing the element that is out of array's bounds, an=
d
> > > > > > >>>> that is UB. =A0How come your program is not crashing, or a=
t
> > > > > > >>>> least going crazy? Maybe you are just unlucky to have a bu=
g
> > > > > > >>>> hidden.
> > > > > > >>> It's an old C programmers hack. I've come across this idiom=
 in
> > > > > > >>> lot's old C code, particularly driver and os code. Microsof=
t
> > > > > > >>> Win32 is rife with it.
> > > > > > >> Except that it's not legal C, either.
>
> > > > > > > Which is why it's referred to above as a "hack". Quite a comm=
on one, too.
>
> > > > > > Usually we use the term "hack" when the code relies on a specif=
ic
> > > > > > manifestation of undefined (or unspecified) behavior, but other=
wise is
> > > > > > well-formed.
>
> > > > > > In this case the code is ill-formed, since 0-size array declara=
tion is
> > > > > > illegal in C++ (as well as in C). In other words, arguing about=
 the "out
> > > > > > of bounds" access here doesn't make much sense, since the code =
is
> > > > > > formally non-compilable.
>
> > > > > Wait.. I don't think it's illegal in C++. At least I will definit=
ely
> > > > > object making it illegal by the standard community.
> > > > > It can be dangerous but it can also do good. It depends on whethe=
r we
> > > > > are using it correctly.
> > > > > Will you guys getting crazy if we do things like this
>
> > > > > -------code-----
> > > > > struct A{
> > > > > =A0 =A0 int num;
> > > > > =A0 =A0 int p[0];};
>
> > > > > A *pA =3D (A*)malloc(sizeof(A)+sizeof(int)*10);
> > > > > printf("%d\n", &pA->p[10] - &pA->p[0]); =A0 =A0 =A0 =A0 =A0 //acc=
essing out of
> > > > > bounds.
> > > > > ------code---
>
> > > > Crazy? Why? C++ compilers should not simply compile it by language
> > > > rules that does not allow zero-sized arrays. If a compiler compiles=
 it
> > > > then it is non-standard extension.
>
> > > > I have seen similar C code, but that used "int p[1];". If it was
> > > > compiled with C++ compiler and run-time bounds-checking on then it =
did
> > > > fault when accessing anything but p[0].
>
> > > > In C++ you should use vector<int> instead, it is about as fast.
>
> > > I hate vectors, again. C++ is not portable as binary code.
> > > Passing vectors as arguments out of a dll in windows may crash the
> > > system when accessing it.
>
> > Assuming binary compatibility without effort is wrong anyway. Are you
> > seriously claiming that such originally-posted hacks let you to
> > achieve binary compatibility between modules compiled with different C
> > compilers and under different compilation options?
>
> nope.. I mean using vectors in module interface may cause
> interoperability problems.
> My hack can solve the problem in case using the same padding policy
> for modules involved.

If same options and same compiler are used then these interoperability
problems you refer to are likely because you add elements to vector or
attempt to destroy it outside of that windows dll context. Your hack
does not even let to add more elements to it (it is like
vector<GameSvr> const) and you may face same problems with C if you
free() it outside of that dll.

* tni, on 20.08.2010 10:52:
> On 2010-08-20 10:19, thomas wrote:
>>> In this case the code is ill-formed, since 0-size array declaration is
>>> illegal in C++ (as well as in C). In other words, arguing about the "out
>>> of bounds" access here doesn't make much sense, since the code is
>>> formally non-compilable.
>>
>> Wait.. I don't think it's illegal in C++. At least I will definitely
>> object making it illegal by the standard community.
>
> None of the C or C++ standards allow an array size of 0.
>
> C99 has flexible arrays (which do basically the same thing), but they
> are not included in C++03. Unfortunately, it doesn't look like C99
> flexible arrays are part of C++0x either. The somewhat good news is that
> GCC, Visual C++ 2010 (and presumably Intel C++) do support them.

It's somewhat bad news.

C99 VLAs are, uh, monstrosities.

I think the reason why C++ did not gain useful variable length array support 
(type safe alternative to alloca) was the sheer badness of C99 VLAs...

They don't play well with anything else.



> (GCC and Visual C++ support 0-sized arrays as extension.)

Well, at least g++ does. It tricks people.


Cheers,

- Alf

-- 
blog at <url: http://alfps.wordpress.com>

On Aug 20, 5:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
> On 20 aug, 12:20, thomas <freshtho...@gmail.com> wrote:
>
>
>
> > On Aug 20, 5:12=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > On 20 aug, 11:45, thomas <freshtho...@gmail.com> wrote:
>
> > > > On Aug 20, 4:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > > > On 20 aug, 11:19, thomas <freshtho...@gmail.com> wrote:
>
> > > > > > On Aug 20, 2:01=A0am, Andrey Tarasevich <andreytarasev...@hotma=
il.com>
> > > > > > wrote:
>
> > > > > > > Pete Becker wrote:
> > > > > > > >>>>> Hi, I need your help.
> > > > > > > >>>>> ----------
> > > > > > > >>>>> struct SvrList{
> > > > > > > >>>>> unsigned int uNum;
> > > > > > > >>>>> GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> > > > > > > >>>>> };
> > > > > > > >>>>> ---------
> > > > > > > >>>>> Once I declared a struct like this to store server list=
 info.
> > > > > > > >>>>> It's supposed to be used like this.
> > > > > > > >>>>> ----------
> > > > > > > >>>>> SvrList* pList =3D (SvrList*)malloc(sizeof(
> > > > > > > >>>>> SvrList) + svrNum*sizeof(GameSvr));
> > > > > > > >>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> > > > > > > >>>> I wouldn't call this fine. Even
> > > > > > > >>>> pList->svr[0]
> > > > > > > >>>> is accessing the element that is out of array's bounds, =
and
> > > > > > > >>>> that is UB. =A0How come your program is not crashing, or=
 at
> > > > > > > >>>> least going crazy? Maybe you are just unlucky to have a =
bug
> > > > > > > >>>> hidden.
> > > > > > > >>> It's an old C programmers hack. I've come across this idi=
om in
> > > > > > > >>> lot's old C code, particularly driver and os code. Micros=
oft
> > > > > > > >>> Win32 is rife with it.
> > > > > > > >> Except that it's not legal C, either.
>
> > > > > > > > Which is why it's referred to above as a "hack". Quite a co=
mmon one, too.
>
> > > > > > > Usually we use the term "hack" when the code relies on a spec=
ific
> > > > > > > manifestation of undefined (or unspecified) behavior, but oth=
erwise is
> > > > > > > well-formed.
>
> > > > > > > In this case the code is ill-formed, since 0-size array decla=
ration is
> > > > > > > illegal in C++ (as well as in C). In other words, arguing abo=
ut the "out
> > > > > > > of bounds" access here doesn't make much sense, since the cod=
e is
> > > > > > > formally non-compilable.
>
> > > > > > Wait.. I don't think it's illegal in C++. At least I will defin=
itely
> > > > > > object making it illegal by the standard community.
> > > > > > It can be dangerous but it can also do good. It depends on whet=
her we
> > > > > > are using it correctly.
> > > > > > Will you guys getting crazy if we do things like this
>
> > > > > > -------code-----
> > > > > > struct A{
> > > > > > =A0 =A0 int num;
> > > > > > =A0 =A0 int p[0];};
>
> > > > > > A *pA =3D (A*)malloc(sizeof(A)+sizeof(int)*10);
> > > > > > printf("%d\n", &pA->p[10] - &pA->p[0]); =A0 =A0 =A0 =A0 =A0 //a=
ccessing out of
> > > > > > bounds.
> > > > > > ------code---
>
> > > > > Crazy? Why? C++ compilers should not simply compile it by languag=
e
> > > > > rules that does not allow zero-sized arrays. If a compiler compil=
es it
> > > > > then it is non-standard extension.
>
> > > > > I have seen similar C code, but that used "int p[1];". If it was
> > > > > compiled with C++ compiler and run-time bounds-checking on then i=
t did
> > > > > fault when accessing anything but p[0].
>
> > > > > In C++ you should use vector<int> instead, it is about as fast.
>
> > > > I hate vectors, again. C++ is not portable as binary code.
> > > > Passing vectors as arguments out of a dll in windows may crash the
> > > > system when accessing it.
>
> > > Assuming binary compatibility without effort is wrong anyway. Are you
> > > seriously claiming that such originally-posted hacks let you to
> > > achieve binary compatibility between modules compiled with different =
C
> > > compilers and under different compilation options?
>
> > nope.. I mean using vectors in module interface may cause
> > interoperability problems.
> > My hack can solve the problem in case using the same padding policy
> > for modules involved.
>
> If same options and same compiler are used then these interoperability
> problems you refer to are likely because you add elements to vector or
> attempt to destroy it outside of that windows dll context. Your hack
> does not even let to add more elements to it (it is like
> vector<GameSvr> const) and you may face same problems with C if you
> free() it outside of that dll.

Right, but how about passing out the address of the pointer?
The point is adding more elements may make it necessary to move
existing elements elsewhere, so we may need to modify the pointer in
dll.
Actually passing pointers between functions may also cause this kind
of problems, nothing special to communication between DLLs.

�� Tiib <ootiib@hot.ee> wrote:
> In C++ you should use vector<int> instead, it is about as fast.

  In which system, exactly?

  Unfortunately 'new' (and 'delete') is a rather heavy operation with
most C/C++ standard memory allocators, and if you use std::vector in
the place of the struct hack, you will be doing two 'news' instead of
one malloc().

  If that's everything you are doing, then there's no practical difference.
However, if you are instantiating a million objects of the struct type,
then it starts making a big difference.

  Additionally, using std::vector there will increase memory fragmentation,
making things even worse.

Vladimir Jovic <vladaspams@gmail.com> wrote:
> What is wrong with next code?
> 
> 
> struct SvrList{
>     unsigned int uNum;
>     GameSvr  *svr;
> };
> 
> ...
> SvrList list;
> list.uNum = 5;
> list.svr = (GameSvr*) malloc( list.uNum * sizeof( GameSvr ) );
> ...
> 
> 
> Since someone pointed out that the original code is used for 
> performances reasons, how is this worse then the code in the first post?

  You have bound 'list' into the scope where it's being instantiated.
When it goes out of scope, the object dies. If that's indeed what you
are trying to do, then there's little difference.

  However, if the 'list' object above must survive the scope where it's
created (and possibly shared by several objects), then you will have to
do *two* malloc() calls (or two 'new' calls or whatever). Most malloc/new
implementations in most systems are quite heavy, plus making two allocations
increases memory fragmentation.

Vladimir Jovic <vladaspams@gmail.com> wrote:
> Yes, I must admit I missed that when I posted that code, but later I 
> realised what was done. The question is still valid : why was it 
> declared like array of zero size, and not like pointer?

  Because if you allocate dynamically an instance of the struct, and the
struct has an array as its last element, you can "overallocate" memory
for the struct and use that extra memory as that array. Thus you need
only one allocation for the dynamically-sized object.

  However, if you used a pointer instead of an array, you would need two
allocations: One for the object itself and another for the array. Memory
allocation is unfortunately a relatively heavy operation in most systems.
Moreover, two allocations will increase memory fragmentation more than one.

> Your point with operator[] doesn't stand (else thread), as you can use 
> it for pointers as well.

  It was sarcasm. The question was "how can you index a zero-sized array?"
I answered "with operator[]".

On Aug 20, 11:10=A0am, thomas <freshtho...@gmail.com> wrote:
> On Aug 20, 4:55=A0pm, Goran Pusic <gor...@cse-semaphore.com> wrote:
>
>
>
> > > But I hate vectors. Different distributors may have different
> > > implementations. When I pass vectors between dlls, things unexpected
> > > may happen.
>
> > C++ language knows nothing of DLLs. Your first error there is trying
> > to expose C++ - specific (and not only that, but potentially
> > particular compiler and standard library version version specific)
> > interface out of your module.
>
> > This is such a massive no-no that it's not even funny. You can only do
> > that if all modules will be compiled with same compiler (version,
> > too), and if interface uses standard library, if modules will use same
> > standard library version and even compilation options. For example,
> > with STL shipped with VS, compiling a DLL exposing standard containers
> > in it's interface with/without SECURE_STL breaks low-level binary
> > compatibility for any container.
>
> > It's not a question of liking at all. This sort of interoperability
> > works only under very specific circumstances (e.g. if you control, or
> > at least mandate, the build process and the deployment process
> > __entirely__).
>
> > Goran.
>
> I like your words.
> In my understanding, your points are:
> 1. using STL containers in module interface may cause interoperability
> problems.
> 2. my zero-sized array is illegal in C++, so may also cause
> interoperability problems.
>
> Both should be avoided.
> So the only way is to using pointers or one-sized array.

Well, as far as executable modules interoperability is concerned,
frankly, all bets are off anyhow. Even if you use plain C interface.
You take one compiler/compilation options to build your module, and
another compiler/compilation options to build your caller, and you can
bork it up horribly. E.g. there's nothing to guarantee alignment
issues between compilers/compilation options, nor is there anything to
guarantee that malloc/free between the two are compatible.

Plain C interface is, in theory, just as useless for executable
modules. C language knows nothing of executable modules, so anything
that works, does so through concerted effort off parties involved
(obey alignment rules, use agreed upon alloc/dealloc function, obey
calling conventions etc). There is only this impression that "plain C"
interface "works" and C++ one does not, simply because C++ interface
gets much more complex in no time, and hence there's much more to
break, and faster.

And in reality, it's __all__ about binary interfaces, and neither C
nor C++ make any __provisions__, let alone guarantees. Toolchains do
make provisions, and one (version of a) toolchain gives you a
guarantee, but that's all.

That's why, IMHO, interoperability technologies (e.g. COM) are
paramount. They __mandate__ interoperability (through a spec), that's
their purpose. A toolchain/library-specific implementation of the
interoperability tech has to obey the spec, and only like that can you
inter operate well. When you use e.g. plain C interface, you only get
implicit, kinda-sorta, full of hidden and unspoken presumptions,
interoperability. On the flip side, that mostly works ;-).

Goran.

On 20 aug, 12:59, thomas <freshtho...@gmail.com> wrote:
> On Aug 20, 5:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
>
>
> > On 20 aug, 12:20, thomas <freshtho...@gmail.com> wrote:
>
> > > On Aug 20, 5:12=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > > On 20 aug, 11:45, thomas <freshtho...@gmail.com> wrote:
>
> > > > > On Aug 20, 4:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > > > > On 20 aug, 11:19, thomas <freshtho...@gmail.com> wrote:
>
> > > > > > > On Aug 20, 2:01=A0am, Andrey Tarasevich <andreytarasev...@hot=
mail.com>
> > > > > > > wrote:
>
> > > > > > > > Pete Becker wrote:
> > > > > > > > >>>>> Hi, I need your help.
> > > > > > > > >>>>> ----------
> > > > > > > > >>>>> struct SvrList{
> > > > > > > > >>>>> unsigned int uNum;
> > > > > > > > >>>>> GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> > > > > > > > >>>>> };
> > > > > > > > >>>>> ---------
> > > > > > > > >>>>> Once I declared a struct like this to store server li=
st info.
> > > > > > > > >>>>> It's supposed to be used like this.
> > > > > > > > >>>>> ----------
> > > > > > > > >>>>> SvrList* pList =3D (SvrList*)malloc(sizeof(
> > > > > > > > >>>>> SvrList) + svrNum*sizeof(GameSvr));
> > > > > > > > >>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla.=
..
> > > > > > > > >>>> I wouldn't call this fine. Even
> > > > > > > > >>>> pList->svr[0]
> > > > > > > > >>>> is accessing the element that is out of array's bounds=
, and
> > > > > > > > >>>> that is UB. =A0How come your program is not crashing, =
or at
> > > > > > > > >>>> least going crazy? Maybe you are just unlucky to have =
a bug
> > > > > > > > >>>> hidden.
> > > > > > > > >>> It's an old C programmers hack. I've come across this i=
diom in
> > > > > > > > >>> lot's old C code, particularly driver and os code. Micr=
osoft
> > > > > > > > >>> Win32 is rife with it.
> > > > > > > > >> Except that it's not legal C, either.
>
> > > > > > > > > Which is why it's referred to above as a "hack". Quite a =
common one, too.
>
> > > > > > > > Usually we use the term "hack" when the code relies on a sp=
ecific
> > > > > > > > manifestation of undefined (or unspecified) behavior, but o=
therwise is
> > > > > > > > well-formed.
>
> > > > > > > > In this case the code is ill-formed, since 0-size array dec=
laration is
> > > > > > > > illegal in C++ (as well as in C). In other words, arguing a=
bout the "out
> > > > > > > > of bounds" access here doesn't make much sense, since the c=
ode is
> > > > > > > > formally non-compilable.
>
> > > > > > > Wait.. I don't think it's illegal in C++. At least I will def=
initely
> > > > > > > object making it illegal by the standard community.
> > > > > > > It can be dangerous but it can also do good. It depends on wh=
ether we
> > > > > > > are using it correctly.
> > > > > > > Will you guys getting crazy if we do things like this
>
> > > > > > > -------code-----
> > > > > > > struct A{
> > > > > > > =A0 =A0 int num;
> > > > > > > =A0 =A0 int p[0];};
>
> > > > > > > A *pA =3D (A*)malloc(sizeof(A)+sizeof(int)*10);
> > > > > > > printf("%d\n", &pA->p[10] - &pA->p[0]); =A0 =A0 =A0 =A0 =A0 /=
/accessing out of
> > > > > > > bounds.
> > > > > > > ------code---
>
> > > > > > Crazy? Why? C++ compilers should not simply compile it by langu=
age
> > > > > > rules that does not allow zero-sized arrays. If a compiler comp=
iles it
> > > > > > then it is non-standard extension.
>
> > > > > > I have seen similar C code, but that used "int p[1];". If it wa=
s
> > > > > > compiled with C++ compiler and run-time bounds-checking on then=
 it did
> > > > > > fault when accessing anything but p[0].
>
> > > > > > In C++ you should use vector<int> instead, it is about as fast.
>
> > > > > I hate vectors, again. C++ is not portable as binary code.
> > > > > Passing vectors as arguments out of a dll in windows may crash th=
e
> > > > > system when accessing it.
>
> > > > Assuming binary compatibility without effort is wrong anyway. Are y=
ou
> > > > seriously claiming that such originally-posted hacks let you to
> > > > achieve binary compatibility between modules compiled with differen=
t C
> > > > compilers and under different compilation options?
>
> > > nope.. I mean using vectors in module interface may cause
> > > interoperability problems.
> > > My hack can solve the problem in case using the same padding policy
> > > for modules involved.
>
> > If same options and same compiler are used then these interoperability
> > problems you refer to are likely because you add elements to vector or
> > attempt to destroy it outside of that windows dll context. Your hack
> > does not even let to add more elements to it (it is like
> > vector<GameSvr> const) and you may face same problems with C if you
> > free() it outside of that dll.
>
> Right, but how about passing out the address of the pointer?
> The point is adding more elements may make it necessary to move
> existing elements elsewhere, so we may need to modify the pointer in
> dll.
> Actually passing pointers between functions may also cause this kind
> of problems, nothing special to communication between DLLs.

All problems i have seen people facing there are still about different
c runtimes or memory/resource contexts between DLL and application. So
it is not about passing pointers if you can not free() the thing it
points at since it was malloc()'ed by different memory management.

On Aug 20, 12:34=A0pm, Juha Nieminen <nos...@thanks.invalid> wrote:
> =D6=F6 Tiib <oot...@hot.ee> wrote:
> > In C++ you should use vector<int> instead, it is about as fast.
>
> =A0 In which system, exactly?
>
> =A0 Unfortunately 'new' (and 'delete') is a rather heavy operation with
> most C/C++ standard memory allocators, and if you use std::vector in
> the place of the struct hack, you will be doing two 'news' instead of
> one malloc().

Why? IMO, new and delete are, on the success path, practically
equivalent on many common implementations. Difference is one if on
malloc failure, and new is inlined.

I say this claim of yours is poorly founded.

> =A0 Additionally, using std::vector there will increase memory fragmentat=
ion,
> making things even worse.

In this case, you can use vector::reserve, so not really.

And even locality of reference is not a concern, because allocators
mostly do a good job of allocating blocks close in space if allocation
is close in time. E.g.

std::vector* p =3D new vector;
p->reserve(X);

is in practice quite OK wrt locality.

I am speaking about locality because on current hardware, and in a
running system, it easily makes more difference than allocation. (I am
speaking abut a case where there's not too create-use-destroy cycles,
of course; and IME(xperience), best course of action when there is a
lot of that, are algorithmic changes; OPs hack comes in dead last,
when other options are explored).

Goran.

On Aug 20, 6:54=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
> On 20 aug, 12:59, thomas <freshtho...@gmail.com> wrote:
>
>
>
> > On Aug 20, 5:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > On 20 aug, 12:20, thomas <freshtho...@gmail.com> wrote:
>
> > > > On Aug 20, 5:12=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > > > On 20 aug, 11:45, thomas <freshtho...@gmail.com> wrote:
>
> > > > > > On Aug 20, 4:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > > > > > On 20 aug, 11:19, thomas <freshtho...@gmail.com> wrote:
>
> > > > > > > > On Aug 20, 2:01=A0am, Andrey Tarasevich <andreytarasev...@h=
otmail.com>
> > > > > > > > wrote:
>
> > > > > > > > > Pete Becker wrote:
> > > > > > > > > >>>>> Hi, I need your help.
> > > > > > > > > >>>>> ----------
> > > > > > > > > >>>>> struct SvrList{
> > > > > > > > > >>>>> unsigned int uNum;
> > > > > > > > > >>>>> GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line A
> > > > > > > > > >>>>> };
> > > > > > > > > >>>>> ---------
> > > > > > > > > >>>>> Once I declared a struct like this to store server =
list info.
> > > > > > > > > >>>>> It's supposed to be used like this.
> > > > > > > > > >>>>> ----------
> > > > > > > > > >>>>> SvrList* pList =3D (SvrList*)malloc(sizeof(
> > > > > > > > > >>>>> SvrList) + svrNum*sizeof(GameSvr));
> > > > > > > > > >>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabl=
a..
> > > > > > > > > >>>> I wouldn't call this fine. Even
> > > > > > > > > >>>> pList->svr[0]
> > > > > > > > > >>>> is accessing the element that is out of array's boun=
ds, and
> > > > > > > > > >>>> that is UB. =A0How come your program is not crashing=
, or at
> > > > > > > > > >>>> least going crazy? Maybe you are just unlucky to hav=
e a bug
> > > > > > > > > >>>> hidden.
> > > > > > > > > >>> It's an old C programmers hack. I've come across this=
 idiom in
> > > > > > > > > >>> lot's old C code, particularly driver and os code. Mi=
crosoft
> > > > > > > > > >>> Win32 is rife with it.
> > > > > > > > > >> Except that it's not legal C, either.
>
> > > > > > > > > > Which is why it's referred to above as a "hack". Quite =
a common one, too.
>
> > > > > > > > > Usually we use the term "hack" when the code relies on a =
specific
> > > > > > > > > manifestation of undefined (or unspecified) behavior, but=
 otherwise is
> > > > > > > > > well-formed.
>
> > > > > > > > > In this case the code is ill-formed, since 0-size array d=
eclaration is
> > > > > > > > > illegal in C++ (as well as in C). In other words, arguing=
 about the "out
> > > > > > > > > of bounds" access here doesn't make much sense, since the=
 code is
> > > > > > > > > formally non-compilable.
>
> > > > > > > > Wait.. I don't think it's illegal in C++. At least I will d=
efinitely
> > > > > > > > object making it illegal by the standard community.
> > > > > > > > It can be dangerous but it can also do good. It depends on =
whether we
> > > > > > > > are using it correctly.
> > > > > > > > Will you guys getting crazy if we do things like this
>
> > > > > > > > -------code-----
> > > > > > > > struct A{
> > > > > > > > =A0 =A0 int num;
> > > > > > > > =A0 =A0 int p[0];};
>
> > > > > > > > A *pA =3D (A*)malloc(sizeof(A)+sizeof(int)*10);
> > > > > > > > printf("%d\n", &pA->p[10] - &pA->p[0]); =A0 =A0 =A0 =A0 =A0=
 //accessing out of
> > > > > > > > bounds.
> > > > > > > > ------code---
>
> > > > > > > Crazy? Why? C++ compilers should not simply compile it by lan=
guage
> > > > > > > rules that does not allow zero-sized arrays. If a compiler co=
mpiles it
> > > > > > > then it is non-standard extension.
>
> > > > > > > I have seen similar C code, but that used "int p[1];". If it =
was
> > > > > > > compiled with C++ compiler and run-time bounds-checking on th=
en it did
> > > > > > > fault when accessing anything but p[0].
>
> > > > > > > In C++ you should use vector<int> instead, it is about as fas=
t.
>
> > > > > > I hate vectors, again. C++ is not portable as binary code.
> > > > > > Passing vectors as arguments out of a dll in windows may crash =
the
> > > > > > system when accessing it.
>
> > > > > Assuming binary compatibility without effort is wrong anyway. Are=
 you
> > > > > seriously claiming that such originally-posted hacks let you to
> > > > > achieve binary compatibility between modules compiled with differ=
ent C
> > > > > compilers and under different compilation options?
>
> > > > nope.. I mean using vectors in module interface may cause
> > > > interoperability problems.
> > > > My hack can solve the problem in case using the same padding policy
> > > > for modules involved.
>
> > > If same options and same compiler are used then these interoperabilit=
y
> > > problems you refer to are likely because you add elements to vector o=
r
> > > attempt to destroy it outside of that windows dll context. Your hack
> > > does not even let to add more elements to it (it is like
> > > vector<GameSvr> const) and you may face same problems with C if you
> > > free() it outside of that dll.
>
> > Right, but how about passing out the address of the pointer?
> > The point is adding more elements may make it necessary to move
> > existing elements elsewhere, so we may need to modify the pointer in
> > dll.
> > Actually passing pointers between functions may also cause this kind
> > of problems, nothing special to communication between DLLs.
>
> All problems i have seen people facing there are still about different
> c runtimes or memory/resource contexts between DLL and application. So
> it is not about passing pointers if you can not free() the thing it
> points at since it was malloc()'ed by different memory management.

Really? DLLs are loaded to the current context and will be in the same
address space. How come the deallocation issue?

On Fri, 2010-08-20, thomas wrote:
> On Aug 19, 7:15�pm, James Kanze <james.ka...@gmail.com> wrote:
>> On Aug 18, 1:36 pm, Jeff Flinn <TriumphSprint2...@hotmail.com> wrote:
>>
>> > Vladimir Jovic wrote:
>> > > thomas wrote:
>> > >> Hi, I need your help.
>> > >> ----------
>> > >> struct SvrList{
>> > >> � � unsigned int uNum;
>> > >> � � GameSvr �svr[0]; � � � � � �//line A
>> > >> };
>> > >> ---------
>> > >> Once I declared a struct like this to store server list info.
>> > >> It's supposed to be used like this.
>> > >> ----------
>> > >> SvrList* pList = (SvrList*)malloc(sizeof(
>> > >> SvrList) + svrNum*sizeof(GameSvr));
>> > >> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
>> > > I wouldn't call this fine. Even
>> > > pList->svr[0]
>> > > is accessing the element that is out of array's bounds, and
>> > > that is UB. �How come your program is not crashing, or at
>> > > least going crazy? Maybe you are just unlucky to have a bug
>> > > hidden.
>> > It's an old C programmers hack. I've come across this idiom in
>> > lot's old C code, particularly driver and os code. Microsoft
>> > Win32 is rife with it.
>>
>> Except that it's not legal C, either. �The C language was
>> carefully specified so that a bounds checking implementation
>> would be legal, and the above will fail in such cases.
>>
>> In C++, of course, the declaration he's looking for is
>> std::vector<SvrList>, without any manual new.

> But I hate vectors. Different distributors may have different
> implementations. When I pass vectors between dlls, things unexpected
> may happen.

You may hate vectors, but you should hate the Foo foo[0] hack even more.
Those get hard to manage, and are hard to get rid of once they have
dug into the code base.

You even have to use malloc() in the code above, so your GameSvrs are
poisoned by your choice of container too -- goodbye to things like
constructors and destructors ...

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .  .
\X/     snipabacken.se>   O  o   .

On 20 aug, 13:34, Juha Nieminen <nos...@thanks.invalid> wrote:
> =D6=F6 Tiib <oot...@hot.ee> wrote:
> > In C++ you should use vector<int> instead, it is about as fast.
>
> =A0 In which system, exactly?
>
> =A0 Unfortunately 'new' (and 'delete') is a rather heavy operation with
> most C/C++ standard memory allocators, and if you use std::vector in
> the place of the struct hack, you will be doing two 'news' instead of
> one malloc().
>
> =A0 If that's everything you are doing, then there's no practical differe=
nce.
> However, if you are instantiating a million objects of the struct type,
> then it starts making a big difference.
>
> =A0 Additionally, using std::vector there will increase memory fragmentat=
ion,
> making things even worse.

Ok points taken. If the "about as fast" is not fast enough and you do
not need dynamic array then use boost::array<>, it is vector with
removed overhead of dynamic size. Also it is present in technical
reports IIRC, so eventually comes to C++ as well.

* thomas, on 20.08.2010 13:02:
>
> Really? DLLs are loaded to the current context and will be in the same
> address space. How come the deallocation issue?

Formally because C++ has no support for dynamic libraries...

In practice, with a Windows DLL, because a DLL can use a different runtime 
library implementation (or just instance) than the main program or another DLL 
in the same process.


Cheers & hth.,

- Alf

-- 
blog at <url: http://alfps.wordpress.com>

On 20 aug, 14:02, thomas <freshtho...@gmail.com> wrote:
> On Aug 20, 6:54=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
>
>
> > On 20 aug, 12:59, thomas <freshtho...@gmail.com> wrote:
>
> > > On Aug 20, 5:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > > On 20 aug, 12:20, thomas <freshtho...@gmail.com> wrote:
>
> > > > > On Aug 20, 5:12=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > > > > On 20 aug, 11:45, thomas <freshtho...@gmail.com> wrote:
>
> > > > > > > On Aug 20, 4:39=A0pm, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> > > > > > > > On 20 aug, 11:19, thomas <freshtho...@gmail.com> wrote:
>
> > > > > > > > > On Aug 20, 2:01=A0am, Andrey Tarasevich <andreytarasev...=
@hotmail.com>
> > > > > > > > > wrote:
>
> > > > > > > > > > Pete Becker wrote:
> > > > > > > > > > >>>>> Hi, I need your help.
> > > > > > > > > > >>>>> ----------
> > > > > > > > > > >>>>> struct SvrList{
> > > > > > > > > > >>>>> unsigned int uNum;
> > > > > > > > > > >>>>> GameSvr =A0svr[0]; =A0 =A0 =A0 =A0 =A0 =A0//line =
A
> > > > > > > > > > >>>>> };
> > > > > > > > > > >>>>> ---------
> > > > > > > > > > >>>>> Once I declared a struct like this to store serve=
r list info.
> > > > > > > > > > >>>>> It's supposed to be used like this.
> > > > > > > > > > >>>>> ----------
> > > > > > > > > > >>>>> SvrList* pList =3D (SvrList*)malloc(sizeof(
> > > > > > > > > > >>>>> SvrList) + svrNum*sizeof(GameSvr));
> > > > > > > > > > >>>>> pList->uNum, pList->svr[0], pList->svr[1].... bla=
bla..
> > > > > > > > > > >>>> I wouldn't call this fine. Even
> > > > > > > > > > >>>> pList->svr[0]
> > > > > > > > > > >>>> is accessing the element that is out of array's bo=
unds, and
> > > > > > > > > > >>>> that is UB. =A0How come your program is not crashi=
ng, or at
> > > > > > > > > > >>>> least going crazy? Maybe you are just unlucky to h=
ave a bug
> > > > > > > > > > >>>> hidden.
> > > > > > > > > > >>> It's an old C programmers hack. I've come across th=
is idiom in
> > > > > > > > > > >>> lot's old C code, particularly driver and os code. =
Microsoft
> > > > > > > > > > >>> Win32 is rife with it.
> > > > > > > > > > >> Except that it's not legal C, either.
>
> > > > > > > > > > > Which is why it's referred to above as a "hack". Quit=
e a common one, too.
>
> > > > > > > > > > Usually we use the term "hack" when the code relies on =
a specific
> > > > > > > > > > manifestation of undefined (or unspecified) behavior, b=
ut otherwise is
> > > > > > > > > > well-formed.
>
> > > > > > > > > > In this case the code is ill-formed, since 0-size array=
 declaration is
> > > > > > > > > > illegal in C++ (as well as in C). In other words, argui=
ng about the "out
> > > > > > > > > > of bounds" access here doesn't make much sense, since t=
he code is
> > > > > > > > > > formally non-compilable.
>
> > > > > > > > > Wait.. I don't think it's illegal in C++. At least I will=
 definitely
> > > > > > > > > object making it illegal by the standard community.
> > > > > > > > > It can be dangerous but it can also do good. It depends o=
n whether we
> > > > > > > > > are using it correctly.
> > > > > > > > > Will you guys getting crazy if we do things like this
>
> > > > > > > > > -------code-----
> > > > > > > > > struct A{
> > > > > > > > > =A0 =A0 int num;
> > > > > > > > > =A0 =A0 int p[0];};
>
> > > > > > > > > A *pA =3D (A*)malloc(sizeof(A)+sizeof(int)*10);
> > > > > > > > > printf("%d\n", &pA->p[10] - &pA->p[0]); =A0 =A0 =A0 =A0 =
=A0 //accessing out of
> > > > > > > > > bounds.
> > > > > > > > > ------code---
>
> > > > > > > > Crazy? Why? C++ compilers should not simply compile it by l=
anguage
> > > > > > > > rules that does not allow zero-sized arrays. If a compiler =
compiles it
> > > > > > > > then it is non-standard extension.
>
> > > > > > > > I have seen similar C code, but that used "int p[1];". If i=
t was
> > > > > > > > compiled with C++ compiler and run-time bounds-checking on =
then it did
> > > > > > > > fault when accessing anything but p[0].
>
> > > > > > > > In C++ you should use vector<int> instead, it is about as f=
ast.
>
> > > > > > > I hate vectors, again. C++ is not portable as binary code.
> > > > > > > Passing vectors as arguments out of a dll in windows may cras=
h the
> > > > > > > system when accessing it.
>
> > > > > > Assuming binary compatibility without effort is wrong anyway. A=
re you
> > > > > > seriously claiming that such originally-posted hacks let you to
> > > > > > achieve binary compatibility between modules compiled with diff=
erent C
> > > > > > compilers and under different compilation options?
>
> > > > > nope.. I mean using vectors in module interface may cause
> > > > > interoperability problems.
> > > > > My hack can solve the problem in case using the same padding poli=
cy
> > > > > for modules involved.
>
> > > > If same options and same compiler are used then these interoperabil=
ity
> > > > problems you refer to are likely because you add elements to vector=
 or
> > > > attempt to destroy it outside of that windows dll context. Your hac=
k
> > > > does not even let to add more elements to it (it is like
> > > > vector<GameSvr> const) and you may face same problems with C if you
> > > > free() it outside of that dll.
>
> > > Right, but how about passing out the address of the pointer?
> > > The point is adding more elements may make it necessary to move
> > > existing elements elsewhere, so we may need to modify the pointer in
> > > dll.
> > > Actually passing pointers between functions may also cause this kind
> > > of problems, nothing special to communication between DLLs.
>
> > All problems i have seen people facing there are still about different
> > c runtimes or memory/resource contexts between DLL and application. So
> > it is not about passing pointers if you can not free() the thing it
> > points at since it was malloc()'ed by different memory management.
>
> Really? DLLs are loaded to the current context and will be in the same
> address space. How come the deallocation issue?

Static variables in DLL are allocated in its own memory context. Such
static variables can be writable. One good way to get rid of writable
static variables in DLLs is to publicly execute the authors and
destroy the carriers of such DLLs or source code. I only am in process
to gain the necessary authority.

On 20 aug, 14:15, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> Static variables in DLL are allocated in its own memory context.

No i was wrong here, sorry. I did look into my papers and it was not
static variable causing it but several c runtimes were linked
statically.

thomas wrote:
> Passing vectors as arguments out of a dll in windows may crash the
> system when accessing it.

Priceless! And people still pay good money for a crap called windows :)

"Juha Nieminen" <nospam@thanks.invalid>

>  In which system, exactly?
>
>  Unfortunately 'new' (and 'delete') is a rather heavy operation with
> most C/C++ standard memory allocators, and if you use std::vector in
> the place of the struct hack, you will be doing two 'news' instead of
> one malloc().

You keep stating that, but why on earth would vector do two allocations? Or 
use anything but placement new at all, that for in compiles to nothing...

>  If that's everything you are doing, then there's no practical difference.
> However, if you are instantiating a million objects of the struct type,
> then it starts making a big difference.

Again, where does that difference come from?   Please come up with a fair 
comparision of the candidates, that do the same thing on the same types, let 
the payload struct be POD and your operation a single resize() from the 
empty state...  Unless certainly you can show different operations the 
'hack' can show.

�� Tiib <ootiib@hot.ee> wrote in news:73971626-b0d7-4d6e-843f-
8830ecd62d7e@f42g2000yqn.googlegroups.com:

> On 20 aug, 14:15, �� Tiib <oot...@hot.ee> wrote:
>>
>> Static variables in DLL are allocated in its own memory context.
> 
> No i was wrong here, sorry. I did look into my papers and it was not
> static variable causing it but several c runtimes were linked
> statically.
> 

Yes, but it is still easy to get DLL interface working, just pass the 
data pointer always together with the pointer to the deallocation 
function (wrapped into a nice C++ class of course).

One can attempt to do the same with a custom allocator for std::vector, 
but this is much harder to get working as std::vector implementation is 
not under user control and there is no guarantee that std::vectors of 
different implementations (e.g. with _SECURE_SCL macro defined or not) 
are binary compatible (they aren't). 

Cheers
Paavo

On Aug 20, 2:47 am, "Alf P. Steinbach /Usenet" <alf.p.steinbach
+use...@gmail.com> wrote:
> * tni, on 20.08.2010 10:52:
>
> > On 2010-08-20 10:19, thomas wrote:
> >>> In this case the code is ill-formed, since 0-size array declaration is
> >>> illegal in C++ (as well as in C). In other words, arguing about the "out
> >>> of bounds" access here doesn't make much sense, since the code is
> >>> formally non-compilable.
>
> >> Wait.. I don't think it's illegal in C++. At least I will definitely
> >> object making it illegal by the standard community.
>
> > None of the C or C++ standards allow an array size of 0.
>
> > C99 has flexible arrays (which do basically the same thing), but they
> > are not included in C++03. Unfortunately, it doesn't look like C99
> > flexible arrays are part of C++0x either. The somewhat good news is that
> > GCC, Visual C++ 2010 (and presumably Intel C++) do support them.
>
> It's somewhat bad news.
>
> C99 VLAs are, uh, monstrosities.

Agreed, but flexible arrays and VLAs (variable-length arrays) are two
different things.  Flexible arrays are just a standardized version of
the struct hack.  The OP's struct, using flexible arrays, would look
like this:

struct SvrList {
    unsigned int uNum;
    GameSvr svr[];
};

On Aug 20, 9:19 am, thomas <freshtho...@gmail.com> wrote:
> On Aug 20, 2:01 am, Andrey Tarasevich <andreytarasev...@hotmail.com>
> wrote:
> > Pete Becker wrote:
> > >>>>> Hi, I need your help.
> > >>>>> ----------
> > >>>>> struct SvrList{
> > >>>>> unsigned int uNum;
> > >>>>> GameSvr  svr[0];            //line A
> > >>>>> };
> > >>>>> ---------
> > >>>>> Once I declared a struct like this to store server list info.
> > >>>>> It's supposed to be used like this.
> > >>>>> ----------
> > >>>>> SvrList* pList = (SvrList*)malloc(sizeof(
> > >>>>> SvrList) + svrNum*sizeof(GameSvr));
> > >>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
> > >>>> I wouldn't call this fine. Even
> > >>>> pList->svr[0]
> > >>>> is accessing the element that is out of array's bounds, and
> > >>>> that is UB.  How come your program is not crashing, or at
> > >>>> least going crazy? Maybe you are just unlucky to have a bug
> > >>>> hidden.
> > >>> It's an old C programmers hack. I've come across this idiom in
> > >>> lot's old C code, particularly driver and os code. Microsoft
> > >>> Win32 is rife with it.
> > >> Except that it's not legal C, either.

> > > Which is why it's referred to above as a "hack". Quite
> > > a common one, too.

> > Usually we use the term "hack" when the code relies on a specific
> > manifestation of undefined (or unspecified) behavior, but otherwise is
> > well-formed.

> > In this case the code is ill-formed, since 0-size array declaration is
> > illegal in C++ (as well as in C). In other words, arguing about the "out
> > of bounds" access here doesn't make much sense, since the code is
> > formally non-compilable.

> Wait.. I don't think it's illegal in C++. At least I will definitely
> object making it illegal by the standard community.

You're about twenty years too late.  It was banned by C90.

--
James Kanze

On 20 aug, 17:51, Paavo Helde <myfirstn...@osa.pri.ee> wrote:
> =D6=F6 Tiib <oot...@hot.ee> wrote in news:73971626-b0d7-4d6e-843f-
> 8830ecd62...@f42g2000yqn.googlegroups.com:
>
> > On 20 aug, 14:15, =D6=F6 Tiib <oot...@hot.ee> wrote:
>
> >> Static variables in DLL are allocated in its own memory context.
>
> > No i was wrong here, sorry. I did look into my papers and it was not
> > static variable causing it but several c runtimes were linked
> > statically.
>
> Yes, but it is still easy to get DLL interface working, just pass the
> data pointer always together with the pointer to the deallocation
> function (wrapped into a nice C++ class of course).
>
> One can attempt to do the same with a custom allocator for std::vector,
> but this is much harder to get working as std::vector implementation is
> not under user control and there is no guarantee that std::vectors of
> different implementations (e.g. with _SECURE_SCL macro defined or not)
> are binary compatible (they aren't).

Evil things like that _SECURE_SCL defined or not can screw up
statically linked library as well, so one should always compile
everything in a product with same options.

Goran Pusic <goranp@cse-semaphore.com> wrote:
> On Aug 20, 12:34�pm, Juha Nieminen <nos...@thanks.invalid> wrote:
>> �� Tiib <oot...@hot.ee> wrote:
>> > In C++ you should use vector<int> instead, it is about as fast.
>>
>> � In which system, exactly?
>>
>> � Unfortunately 'new' (and 'delete') is a rather heavy operation with
>> most C/C++ standard memory allocators, and if you use std::vector in
>> the place of the struct hack, you will be doing two 'news' instead of
>> one malloc().
> 
> Why? IMO, new and delete are, on the success path, practically
> equivalent on many common implementations. Difference is one if on
> malloc failure, and new is inlined.

  You are basing your claims on your personal *opinion*? Rather than,
you know, actually testing it in practice?

> I say this claim of yours is poorly founded.

  It's quite well founded. For example, take this short piece of code:

    int main()
    {
        std::set<int> someSet;
        for(int i = 0; i < 10000000; ++i) someSet.insert(i);
    }

  Compiling that with "g++ -O3 -march=native" on my Pentium4 system takes
about 10 seconds. So, where do you think all that time is spent? Perhaps
rearranging the binary tree each time a new element is inserted? That sounds
like something which would take its time.

  Nope. The majority of that time is spent *allocating memory*. If I
change the above code to use a very fast memory allocator, such as
http://warp.povusers.org/FSBAllocator/ so that the declaration of 'someSet'
becomes:

    std::set<int, std::less<int>, FSBAllocator<int> > someSet;

then the running time drops down to about 2.5 seconds.

  From the 10 seconds running time of the original program above over
7.5 seconds is spent solely on memory allocation and deallocation.
That's quite a *lot*. (There are many reasons why this is so, one of the
major ones being that the default allocator is thread-safe, which the
FSBAllocator above isn't.)

  'new' and 'delete' are significantly heavy operations.

>> � Additionally, using std::vector there will increase memory fragmentation,
>> making things even worse.
> 
> In this case, you can use vector::reserve, so not really.

  If you are allocating a million instances of the struct, each such
instance having an std::vector object inside, reserve() would do nothing
to alleviate the memory fragmentation.

> And even locality of reference is not a concern, because allocators
> mostly do a good job of allocating blocks close in space if allocation
> is close in time. E.g.
> 
> std::vector* p = new vector;
> p->reserve(X);
> 
> is in practice quite OK wrt locality.

  Not if the memory is heavily fragmented, which is one major problem here.

�� Tiib <ootiib@hot.ee> wrote:
> Ok points taken. If the "about as fast" is not fast enough and you do
> not need dynamic array then use boost::array<>, it is vector with
> removed overhead of dynamic size. Also it is present in technical
> reports IIRC, so eventually comes to C++ as well.

  Exactly how would boost::array make this any faster? There's no
difference. The issue is not reallocation.

Balog Pal <pasa@lib.hu> wrote:
> "Juha Nieminen" <nospam@thanks.invalid>
> 
>>  In which system, exactly?
>>
>>  Unfortunately 'new' (and 'delete') is a rather heavy operation with
>> most C/C++ standard memory allocators, and if you use std::vector in
>> the place of the struct hack, you will be doing two 'news' instead of
>> one malloc().
> 
> You keep stating that, but why on earth would vector do two allocations? Or 
> use anything but placement new at all, that for in compiles to nothing...

  How many times does this have to be explained?

  If you need to instantiate the struct dynamically, there will be only
one malloc() call if the struct hack is used. However, if the struct hack
is not used, but std::vector or a raw array pointer is used instead, you
will need two allocations: One for the struct and another for the array
inside the struct.

  What's so hard to understand in that?

>>  If that's everything you are doing, then there's no practical difference.
>> However, if you are instantiating a million objects of the struct type,
>> then it starts making a big difference.
> 
> Again, where does that difference come from?

  The difference comes from the amount of allocations needed.

  If you need to instantiate the struct dynamically one million times,
and the struct hack is used, then one million malloc() calls will suffice.
However, if the struct hack is not used, but std::vector or a raw array
pointer is used instead, you will need two million allocations.

  'new' and 'delete' (or malloc() and free()) are quite heavy operations,
so this can have a significant impact in performance. It also increases
memory fragmentation.

>   Please come up with a fair 
> comparision of the candidates, that do the same thing on the same types, let 
> the payload struct be POD and your operation a single resize() from the 
> empty state...  Unless certainly you can show different operations the 
> 'hack' can show.

  I don't even understand that paragraph.

Juha Nieminen <nospam@thanks.invalid> wrote:
>  Compiling that with "g++ -O3 -march=native" on my Pentium4 system takes
> about 10 seconds.

  Of course I meant:

  Compiling that with "g++ -O3 -march=native" *and running it* on my
Pentium4 system takes about 10 seconds.

On 08/21/10 06:50 PM, Juha Nieminen wrote:
> Goran Pusic<goranp@cse-semaphore.com>  wrote:
>> On Aug 20, 12:34 pm, Juha Nieminen<nos...@thanks.invalid>  wrote:
>>> �� Tiib<oot...@hot.ee>  wrote:
>>>> In C++ you should use vector<int>  instead, it is about as fast.
>>>
>>>    In which system, exactly?
>>>
>>>    Unfortunately 'new' (and 'delete') is a rather heavy operation with
>>> most C/C++ standard memory allocators, and if you use std::vector in
>>> the place of the struct hack, you will be doing two 'news' instead of
>>> one malloc().
>>
>> Why? IMO, new and delete are, on the success path, practically
>> equivalent on many common implementations. Difference is one if on
>> malloc failure, and new is inlined.
>
>    You are basing your claims on your personal *opinion*? Rather than,
> you know, actually testing it in practice?
>
>> I say this claim of yours is poorly founded.
>
>    It's quite well founded. For example, take this short piece of code:
>
>      int main()
>      {
>          std::set<int>  someSet;
>          for(int i = 0; i<  10000000; ++i) someSet.insert(i);
>      }

You are cheating.  std::set<int> != std::vector<int> and is in no way an 
equivalent to a dynamically allocated array.

Try your test with a vector, before and after reserving space (which is 
the more realistic comparison).

-- 
Ian Collins

"Juha Nieminen" <nospam@thanks.invalid>

> Balog Pal <pasa@lib.hu> wrote:
>> "Juha Nieminen" <nospam@thanks.invalid>
>>
>>>  In which system, exactly?
>>>
>>>  Unfortunately 'new' (and 'delete') is a rather heavy operation with
>>> most C/C++ standard memory allocators, and if you use std::vector in
>>> the place of the struct hack, you will be doing two 'news' instead of
>>> one malloc().
>>
>> You keep stating that, but why on earth would vector do two allocations? 
>> Or
>> use anything but placement new at all, that for in compiles to nothing...
>
>  How many times does this have to be explained?
>
>  If you need to instantiate the struct dynamically, there will be only
> one malloc() call if the struct hack is used. However, if the struct hack
> is not used, but std::vector or a raw array pointer is used instead, you
> will need two allocations: One for the struct and another for the array
> inside the struct.
>
>  What's so hard to understand in that?

Nothimg, it is just hard to believe that you actually meant to drag that 
gremlin in an expert group and hope to get away with it.
We certainly do NOT do stuff like
new vector<int>
in real life, but have the vector as direct member, or it passed in, or 
returned -- such code would hardly pass any review.     So your "if" 
situation only works on "paper" not reality.

>>   Please come up with a fair
>> comparision of the candidates, that do the same thing on the same types, 
>> let
>> the payload struct be POD and your operation a single resize() from the
>> empty state...  Unless certainly you can show different operations the
>> 'hack' can show.
>
>  I don't even understand that paragraph.

Okay, let's just drop it. 

"Juha Nieminen" <nospam@thanks.invalid>

>>> > In C++ you should use vector<int> instead, it is about as fast.
>>>
>>> In which system, exactly?
>>>
>>> Unfortunately 'new' (and 'delete') is a rather heavy operation with
>>> most C/C++ standard memory allocators, and if you use std::vector in
>>> the place of the struct hack, you will be doing two 'news' instead of
>>> one malloc().
>>
>> Why? IMO, new and delete are, on the success path, practically
>> equivalent on many common implementations. Difference is one if on
>> malloc failure, and new is inlined.
>
>  You are basing your claims on your personal *opinion*? Rather than,
> you know, actually testing it in practice?
>
>> I say this claim of yours is poorly founded.
>
>  It's quite well founded. For example, take this short piece of code:
>
>    int main()
>    {
>        std::set<int> someSet;
>        for(int i = 0; i < 10000000; ++i) someSet.insert(i);
>    }
>
>  Compiling that with "g++ -O3 -march=native" on my Pentium4 system takes
> about 10 seconds. So, where do you think all that time is spent? Perhaps
> rearranging the binary tree each time a new element is inserted? That
> sounds
> like something which would take its time.
>
>  Nope. The majority of that time is spent *allocating memory*.

Hard to decide that your article is a sour attemt at cheating or just basic
trolling.   We discuss performance of vector<int> then you insert a snippet
with set<int> like it had any relevance.  :-(((

Different collections have different characteristics wrt performance, memory
footprint, etc, we ALL know that.  Now please go back to the original claim
instead of drifting in all directions.

....
>  If you are allocating a million instances of the struct, each such
> instance having an std::vector object inside, reserve() would do nothing
> to alleviate the memory fragmentation.


And if you allocate a million instances of the struct hack, it will take a
while too.   now do you suggest using the struct hack in the *payload*?

On 21 aug, 09:51, Juha Nieminen <nos...@thanks.invalid> wrote:
> =D6=F6 Tiib <oot...@hot.ee> wrote:
> > Ok points taken. If the "about as fast" is not fast enough and you do
> > not need dynamic array then use boost::array<>, it is vector with
> > removed overhead of dynamic size. Also it is present in technical
> > reports IIRC, so eventually comes to C++ as well.
>
> =A0 Exactly how would boost::array make this any faster? There's no
> difference. The issue is not reallocation.

What is the issue, then? At some point i start to forget the issue
when all go "blah-blah" and no code is posted. Original 'hack' was
something like that:

 // -------
 struct SvrList
 {
     unsigned int uNum;
     GameSvr svr[];
 };

 SvrList* pList =3D (SvrList*)malloc( sizeof SvrList
                                  + svrNum * sizeof GameSvr );
 int len =3D pList->uNum;
 GameSvr a =3D pList->svr[0];
 pList->svr[1] =3D a;
 // -------

It is simply pile of bad things. Lot better (no pointers, no arrays,
hacks if any in well-tested templates) with boost would look something
like that:

 // -------
 typedef boost::array<GameSvr, svrNum> SvrList;
 typedef boost::shared_ptr<SvrList> SvrListPtr;
 SvrListPtr pList =3D boost::make_shared<SvrList>();

 int len =3D pList->size();
 GameSvr a =3D pList->at(0);
 pList->at(1) =3D a;
 // -------

Are you claiming that it is lot less efficient than the original?
Ok ... one additional indirection from shared_ptr and that is all
basically, you won't notice it in profiler. However all hacking
headache has gone away.

On Aug 21, 7:50 am, Juha Nieminen <nos...@thanks.invalid> wrote:
> Goran Pusic <gor...@cse-semaphore.com> wrote:

    [...]
> Nope. The majority of that time is spent *allocating memory*.
> If I change the above code to use a very fast memory
> allocator, such ashttp://warp.povusers.org/FSBAllocator/so
> that the declaration of 'someSet' becomes:

>     std::set<int, std::less<int>, FSBAllocator<int> > someSet;

> then the running time drops down to about 2.5 seconds.

> From the 10 seconds running time of the original program above over
> 7.5 seconds is spent solely on memory allocation and deallocation.

Maybe.  Alternatively, it's spent because one allocator results
in better locality than the other, with less page faults.

> That's quite a *lot*. (There are many reasons why this is so,
> one of the major ones being that the default allocator is
> thread-safe, which the FSBAllocator above isn't.)

> 'new' and 'delete' are significantly heavy operations.

Typically, yes, although some implementations (including VC++!)
do optimize them for "small" allocations---replacing the
allocator for the int's in a shared_ptr, for example, rarely
gains much, since the implementation already picks up the
"small" allocation, and uses a specialized strategy for it
(which isn't that heavy).

    [...]
> > And even locality of reference is not a concern, because allocators
> > mostly do a good job of allocating blocks close in space if allocation
> > is close in time. E.g.

I'm not sure where he got this from.  In my experience,
allocators do a good job in allocating similarly sized blocks
close together, but when the sizes are radically different, it
varies considerably.

--
James Kanze

Goran Pusic wrote:

> I am speaking about locality because on current hardware, and in a
> running system, it easily makes more difference than allocation. (I am
> speaking abut a case where there's not too create-use-destroy cycles,
> of course; and IME(xperience), best course of action when there is a
> lot of that, are algorithmic changes; OPs hack comes in dead last,
> when other options are explored).
>

"OP's hack"? The technique is very well-known from C and C99 standardized 
it (C99 uses empty brackets rather than a array dimension of  0 or 1). Do 
a web search on "struct hack" for all the details. In addition, you can 
search on "C99", and "flexible array member". And please note that the 
C99 "VLA" (Variable-Length Array)  is entirely different that "flexible 
array member". Why C++ isn't following suit on this one escapes me. I use 
it in C++ using an array dimension of 1, because a struct with a 
zero-length array can't be derived from (for no good reason as far as I 
can tell, as of course the programmer wouldn't be adding data members to 
a class derived from another with flexible array as the last member) and 
I forget what the result of trying to use empty brackets is, but you can 
try it and see. 

Juha Nieminen wrote:
> Balog Pal <pasa@lib.hu> wrote:
>> "Juha Nieminen" <nospam@thanks.invalid>
>>
>>>  In which system, exactly?
>>>
>>>  Unfortunately 'new' (and 'delete') is a rather heavy operation with
>>> most C/C++ standard memory allocators, and if you use std::vector in
>>> the place of the struct hack, you will be doing two 'news' instead
>>> of
>>> one malloc().
>>
>> You keep stating that, but why on earth would vector do two
>> allocations? Or use anything but placement new at all, that for in
>> compiles to nothing...
>
>  How many times does this have to be explained?
>
>  If you need to instantiate the struct dynamically, there will be only
> one malloc() call if the struct hack is used. However, if the struct
> hack is not used, but std::vector or a raw array pointer is used
> instead, you will need two allocations: One for the struct and
> another for the array inside the struct.
>
>  What's so hard to understand in that?
>
>>>  If that's everything you are doing, then there's no practical
>>> difference. However, if you are instantiating a million objects of
>>> the struct type, then it starts making a big difference.
>>
>> Again, where does that difference come from?
>
>  The difference comes from the amount of allocations needed.
>
>  If you need to instantiate the struct dynamically one million times,
> and the struct hack is used, then one million malloc() calls will
> suffice. However, if the struct hack is not used, but std::vector or
> a raw array pointer is used instead, you will need two million
> allocations.
>
>  'new' and 'delete' (or malloc() and free()) are quite heavy
> operations, so this can have a significant impact in performance. It
> also increases memory fragmentation.
>
>>   Please come up with a fair
>> comparision of the candidates, that do the same thing on the same
>> types, let the payload struct be POD and your operation a single
>> resize() from the empty state...  Unless certainly you can show
>> different operations the 'hack' can show.
>
>  I don't even understand that paragraph.

I don't think performance is the issue at all with this apples-to-oranges 
comparison of std::vector and the struct hack: it's contiguity of the 
var-len array with the other members of the struct that is the reason for 
the struct hack's usefulness and popularity. std::vector is a very 
specialized container whereas the struct hack is just a down-at-the-metal 
technique used in building higher-level abstractions. Note that the OP 
had the var-len array as a part of another struct. This isn't about 
"array containers". 

tni wrote:
> On 2010-08-20 10:19, thomas wrote:
>>> In this case the code is ill-formed, since 0-size array declaration
>>> is illegal in C++ (as well as in C). In other words, arguing about
>>> the "out of bounds" access here doesn't make much sense, since the
>>> code is formally non-compilable.
>>
>> Wait.. I don't think it's illegal in C++. At least I will definitely
>> object making it illegal by the standard community.
>
> None of the C or C++ standards allow an array size of 0.
>
> C99 has flexible arrays (which do basically the same thing), but they
> are not included in C++03. Unfortunately, it doesn't look like C99
> flexible arrays are part of C++0x either. The somewhat good news is
> that GCC, Visual C++ 2010 (and presumably Intel C++) do support them.
>
> (GCC and Visual C++ support 0-sized arrays as extension.)

Try to derive something from a class that has a zero-length or 
undefined-length array as the last member using VC++. 

Vladimir Jovic wrote:
> thomas wrote:
>> On Aug 20, 2:01 am, Andrey Tarasevich <andreytarasev...@hotmail.com>
>> wrote:
>>> Pete Becker wrote:
>>>>>>>> Hi, I need your help.
>>>>>>>> ----------
>>>>>>>> struct SvrList{
>>>>>>>> unsigned int uNum;
>>>>>>>> GameSvr  svr[0];            //line A
>>>>>>>> };
>>>>>>>> ---------
>>>>>>>> Once I declared a struct like this to store server list info.
>>>>>>>> It's supposed to be used like this.
>>>>>>>> ----------
>>>>>>>> SvrList* pList = (SvrList*)malloc(sizeof(
>>>>>>>> SvrList) + svrNum*sizeof(GameSvr));
>>>>>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
>>>>>>> I wouldn't call this fine. Even
>>>>>>> pList->svr[0]
>>>>>>> is accessing the element that is out of array's bounds, and
>>>>>>> that is UB.  How come your program is not crashing, or at
>>>>>>> least going crazy? Maybe you are just unlucky to have a bug
>>>>>>> hidden.
>>>>>> It's an old C programmers hack. I've come across this idiom in
>>>>>> lot's old C code, particularly driver and os code. Microsoft
>>>>>> Win32 is rife with it.
>>>>> Except that it's not legal C, either.
>>>> Which is why it's referred to above as a "hack". Quite a common
>>>> one, too.
>>> Usually we use the term "hack" when the code relies on a specific
>>> manifestation of undefined (or unspecified) behavior, but otherwise
>>> is well-formed.
>>>
>>> In this case the code is ill-formed, since 0-size array declaration
>>> is illegal in C++ (as well as in C). In other words, arguing about
>>> the "out of bounds" access here doesn't make much sense, since the
>>> code is formally non-compilable.
>>
>> Wait.. I don't think it's illegal in C++. At least I will definitely
>> object making it illegal by the standard community.
>
> In the c++ standard, see 8.3.4.1 , this part :
>
> ... If the constant-expression (5.19) is present, it shall be an
> integral constant expression and its value shall be greater than
> zero...
> therefore it is illegal c++ code.
>
>> It can be dangerous but it can also do good. It depends on whether we
>> are using it correctly.
>
> Can't use it correctly. It is illegal, therefore undefined behaviour.

Undefined by the standard, but DEFINED by all implementations. The "it's 
undefined behavior" ranting gets quite annoying. Try compiling something 
with the standard instead of an implementation sometime and see how far 
you get!

Ian Collins <ian-news@hotmail.com> wrote:
> You are cheating.  std::set<int> != std::vector<int> and is in no way an 
> equivalent to a dynamically allocated array.
> 
> Try your test with a vector, before and after reserving space (which is 
> the more realistic comparison).

  How many times does this have to be repeated?

  The so-called struct hack is a technique for allocating a variable-length
struct. It's struct which has an array as the last element, and the size
of this array is determined at runtime by "overallocating" memory using
malloc(). When this last element is then indexed, this will access that
allocated memory.

  So using the struct hack you would have:

    struct MyStruct
    {
        int size; // or whatever
        int array[0]; // or int array[1] if the compiler demands it
    };

  Then you allocate such structs with

    malloc(sizeof(MyStruct) + amount_of_elements * sizeof(int));

  Now you have a dynamically allocated instantiation of the struct, where
the size of the 'array' element is decided at runtime.

  The other option is to do it like:

    struct MyStruct
    {
        std::vector<int> array;

        MyStruct(int size): array(size) {}
    };

  Then you allocate the struct like:

    new MyStruct(amount_of_elements);

  (It has to be allocated dynamically if the struct instantiation needs to
survive the scope where it was created.)

  In the latter case there will be *two* allocations: One for the struct
and another for the std::vector. In the former case there will be only
one allocation.

  std::vector::reserve has nothing to do with this.

Balog Pal <pasa@lib.hu> wrote:
> Hard to decide that your article is a sour attemt at cheating or just basic
> trolling.   We discuss performance of vector<int> then you insert a snippet
> with set<int> like it had any relevance.  :-(((

  No, we were discussing the efficiency of memory allocation.

  If you are instantiating a million std::vector instances (initialized
with a non-zero size), there will be a million allocations.

> Different collections have different characteristics wrt performance, memory
> footprint, etc, we ALL know that.  Now please go back to the original claim
> instead of drifting in all directions.

  The discussion was about the efficiency of memory allocation. 'new' and
'delete' are heavy operations.

>>  If you are allocating a million instances of the struct, each such
>> instance having an std::vector object inside, reserve() would do nothing
>> to alleviate the memory fragmentation.
> 
> And if you allocate a million instances of the struct hack, it will take a
> while too.   now do you suggest using the struct hack in the *payload*?

  If you allocate a million instances of the struct using the struct hack,
you will be making one million memory allocations.

  If you allocate a million instances of the struct without the struct hack
(ie. it has a std::vector as an element instead), you will be making *two*
million memory allocations.

  What is so hard to understand in that?

�� Tiib <ootiib@hot.ee> wrote:
>> � Exactly how would boost::array make this any faster? There's no
>> difference. The issue is not reallocation.
> 
> What is the issue, then?

  How many times does this have to be repeated?

  The issue is that with a member vector (or boost::array or whatever you
want to use), there will be *two* memory allocations (one for the struct
itself and another for the member array), while with the struct hack
there will be only *one* memory allocation.

  Why is that so hard to understand?

  If your argument is that it doesn't matter efficiency-wise, then you
are simply wrong if the amount of struct instantiations is significantly
large because memory allocation is a heavy operation in most systems.

Balog Pal <pasa@lib.hu> wrote:
>>  How many times does this have to be explained?
>>
>>  If you need to instantiate the struct dynamically, there will be only
>> one malloc() call if the struct hack is used. However, if the struct hack
>> is not used, but std::vector or a raw array pointer is used instead, you
>> will need two allocations: One for the struct and another for the array
>> inside the struct.
>>
>>  What's so hard to understand in that?
> 
> Nothimg, it is just hard to believe that you actually meant to drag that 
> gremlin in an expert group and hope to get away with it.
> We certainly do NOT do stuff like
> new vector<int>
> in real life, but have the vector as direct member, or it passed in, or 
> returned -- such code would hardly pass any review.     So your "if" 
> situation only works on "paper" not reality.

  How many times does this have to be explained? I just can't think how
I would explain it more clearly.

  Ok, let me try once more. Assume you have this:

    struct MyStruct
    {
        // Some other member variables here.
        int size;
        int array[0];
    };

  Then you need to allocate an instance of that struct dynamically (because
it has to survive the scope where it is being created). You do it like:

    malloc(sizeof(MyStruct) + elements_amount * sizeof(int));

  Please count the number of allocations being made. It's 1. Follow me so
far?

  Ok, now consider the alternative:

    struct MyStruct
    {
        // Some other member variables here.
        std::vector<int> array;

        MyStruct(int size): array(size) {}
    };

  Then when you need to allocate an instance of that struct dynamically,
you do it like:

    new MyStruct(elements_amount);

  Please count the number of allocations being made. It's 2 (one for the
struct itself, and one performed by std::vector).

  Understand now?

joe <jc1996@att.net> wrote:
> I don't think performance is the issue at all with this apples-to-oranges 
> comparison of std::vector and the struct hack:

  Exactly how is it an "apples-to-oranges comparison"? The two techniques
are achieving the same thing: A struct instantiation which contains an
array whose size is decided at runtime (rather than at compile time).
The difference between them is that the struct hack achieves this with
one single allocation per struct instance, while the "cleaner" method
requires two allocations per struct instance (one for the struct itself
and another for the vector).

  Performance will not be an issue if you need to allocate just a few
instances of the struct. However, it can become an issue if you need to
allocate millions of instances.

> it's contiguity of the 
> var-len array with the other members of the struct that is the reason for 
> the struct hack's usefulness and popularity.

  That may be so, but you can't dismiss the efficiency benefit in cases
where huge amounts of instantiations need to be done.

> std::vector is a very specialized container

  Why are people here so obsessed about the specifics of std::vector
here? It's only being used here as a substitute for a raw array. In
terms of efficiency it doesn't make any difference compared to one.
It's just easier to use.

> Note that the OP 
> had the var-len array as a part of another struct. This isn't about 
> "array containers". 

  There seems to be some kind of failure at communication here.

  I am not talking about the efficiency of allocating an array. I am talking
*precisely* about the efficiency of allocating a struct which has an array
as the last member, comparing the struct hack to a more regular solution
(which has a pointer or a std::vector as the last struct element).

On 08/22/10 04:44 PM, Juha Nieminen wrote:
> Ian Collins<ian-news@hotmail.com>  wrote:
>> You are cheating.  std::set<int>  != std::vector<int>  and is in no way an
>> equivalent to a dynamically allocated array.
>>
>> Try your test with a vector, before and after reserving space (which is
>> the more realistic comparison).
>
>    How many times does this have to be repeated?

What, that you cheated?

>    The so-called struct hack is a technique for allocating a variable-length
> struct. It's struct which has an array as the last element, and the size
> of this array is determined at runtime by "overallocating" memory using
> malloc(). When this last element is then indexed, this will access that
> allocated memory.

I discovered that 30 odd years ago, no news there.

>    So using the struct hack you would have:
>
>      struct MyStruct
>      {
>          int size; // or whatever
>          int array[0]; // or int array[1] if the compiler demands it
>      };

or int array[] if you have a conforming C compiler.

>    Then you allocate such structs with
>
>      malloc(sizeof(MyStruct) + amount_of_elements * sizeof(int));

In the C world, yes.

>    Now you have a dynamically allocated instantiation of the struct, where
> the size of the 'array' element is decided at runtime.

The struct hack is a C idiom which is seldom used in C++.  In C++ we 
have other techniques.

>    The other option is to do it like:
>
>      struct MyStruct
>      {
>          std::vector<int>  array;
>
>          MyStruct(int size): array(size) {}
>      };
>
>    Then you allocate the struct like:
>
>      new MyStruct(amount_of_elements);
>
>    (It has to be allocated dynamically if the struct instantiation needs to
> survive the scope where it was created.)
>
>    In the latter case there will be *two* allocations: One for the struct
> and another for the std::vector. In the former case there will be only
> one allocation.

Big deal.

>    std::vector::reserve has nothing to do with this.

neither has std::set.

-- 
Ian Collins

Ian Collins <ian-news@hotmail.com> wrote:
> On 08/22/10 04:44 PM, Juha Nieminen wrote:
>> Ian Collins<ian-news@hotmail.com>  wrote:
>>> You are cheating.  std::set<int>  != std::vector<int>  and is in no way an
>>> equivalent to a dynamically allocated array.
>>>
>>> Try your test with a vector, before and after reserving space (which is
>>> the more realistic comparison).
>>
>>    How many times does this have to be repeated?
> 
> What, that you cheated?

  Cheated exactly how? I was demonstrating the speed of memory allocations.
Allocating a million std::set nodes and allocating a million instances of
(non-zero-sized) std::vector instances both cause a million memory
allocations. The point was to demonstrate the speed of memory allocation,
not the speed of std::vector vs. std::set. I could just as well have used
std::list, std::map or outright raw 'new' calls for the same purpose.

  In other words, I was demonstrating the significance of the amount of
times that 'new' and 'delete' are executed.

>>    Now you have a dynamically allocated instantiation of the struct, where
>> the size of the 'array' element is decided at runtime.
> 
> The struct hack is a C idiom which is seldom used in C++.  In C++ we 
> have other techniques.

  And this has what to do with my original point, namely that 'new' and
'delete' are heavy operations?

>>    In the latter case there will be *two* allocations: One for the struct
>> and another for the std::vector. In the former case there will be only
>> one allocation.
> 
> Big deal.

  So you *do* understand my point, finally? In other words, that with the
struct hack the total amount of memory allocations will be halved compared
to the cleaner solution (using a dynamically allocated array as the member
of the struct).

  If you are making a huge amount of such allocations, it *will* be a big
deal because 'new' and 'delete' (or malloc() and free() if you like) are
quite heavy operations. That's what my example with the std::set was
demonstrating.

>>    std::vector::reserve has nothing to do with this.
> 
> neither has std::set.

  The point was to demonstrate how heavy 'new' and 'delete' are.

Juha Nieminen <nospam@thanks.invalid> wrote:
> The point was to demonstrate the speed of memory allocation,
> not the speed of std::vector vs. std::set. I could just as well have used
> std::list, std::map or outright raw 'new' calls for the same purpose.

  Btw, to delve more into why I used std::set in the example:

  The point was to demonstrate that 'new' and 'delete' are heavy
operations. But how heavy? It's clear that if you do nothing else in
a program other than N allocations or 2*N allocations, the latter will
take about twice the time compared to the former. However, that tells
little about how heavy the allocation is compared to other operations
that the program might be doing. If 'new' and 'delete' take one clock
cycle then it's rather irrelevant.

  std::set does quite a lot of things when elements are inserted. It's
a balanced binary tree, and every time an element is inserted, especially
if there is already a big amount of elements, it will perform quite many
operations, mainly to re-balance the tree. It will traverse the tree and
update pointers and flags, etc. (It does this to about O(log n) elements
of the tree, but we are still talking about dozens of operations needed
to re-balance the tree.)

  So inserting an element to a std::set is a relatively heavy operation
because of all the operations it needs to do internally to re-balance the
tree. Thus one would easily think that when inserting 10 million elements
to a std::set, the vast majority of the time is being spent on these
operations.

  However, rather surprisingly, over 75% of the time is actually being
used by 'new'. In other words, 'new' is at least three times slower than
re-balancing a binary tree with millions of elements in it.

  This goes to demonstrate how heavy 'new' is, and why it may be a good
idea to minimize how many times it's called in a program, if possible.

  If you want to directly measure the speed difference between the struct
hack allocation and the more regular solution, doing that is rather easy
with a program like:

#if(1)
//--------------------------------------------------------------
#include <cstdlib>

struct MyStruct
{
    int someData;
    int size;
    int array[0];
};

int main()
{
    for(int i = 0; i < 100000000; ++i)
    {
        const int arraySize = 10 + i % 10;
        MyStruct* p = (MyStruct*) std::malloc
            (sizeof(MyStruct) + arraySize * sizeof(int));
        std::free(p);
    }
}
//--------------------------------------------------------------
#else
//--------------------------------------------------------------
#include <vector>

struct MyStruct
{
    int someData;
    std::vector<int> array;

    MyStruct(int size): array(size) {}
};

int main()
{
    for(int i = 0; i < 100000000; ++i)
    {
        const int arraySize = 10 + i % 10;
        MyStruct* p = new MyStruct(arraySize);
        delete p;
    }
}
//--------------------------------------------------------------
#endif

  For example in my computer the first version of the program, which uses
the struct hack, takes 10 seconds to run, while the latter version takes
20 seconds to run.

  Of course given the attitude that some people seem to have formed in
this thread, I'm pretty sure someone will come up with some reason why
I "cheated" there as well (most probably by ignoring the original premise
that the struct needs to be allocated dynamically in cases where it has
to outlive the scope where it's being created).

On 22 aug, 07:55, Juha Nieminen <nos...@thanks.invalid> wrote:
> =D6=F6 Tiib <oot...@hot.ee> wrote:
> >> =A0 Exactly how would boost::array make this any faster? There's no
> >> difference. The issue is not reallocation.
>
> > What is the issue, then?
>
> =A0 How many times does this have to be repeated?
>
> =A0 The issue is that with a member vector (or boost::array or whatever y=
ou
> want to use), there will be *two* memory allocations (one for the struct
> itself and another for the member array), while with the struct hack
> there will be only *one* memory allocation.

No. boost::array does usually allocate together with struct that
contains it. It is like any usual non dynamic array. Since i used
boost::make_shared<> in my example it did exactly one allocation
(allocating room both for svrlist and for shared_ptr pointing at it at
once).

> =A0 Why is that so hard to understand?
>
> =A0 If your argument is that it doesn't matter efficiency-wise, then you
> are simply wrong if the amount of struct instantiations is significantly
> large because memory allocation is a heavy operation in most systems.

No. My argument is that there is always a way to write it in C++. That
struct hack is there since C provides you no much other ways. If the
already existing things do not suit you then fine, but that hack is
ugly don't you see?

�� Tiib <ootiib@hot.ee> wrote:
> No. boost::array does usually allocate together with struct that
> contains it. It is like any usual non dynamic array. Since i used
> boost::make_shared<> in my example it did exactly one allocation
> (allocating room both for svrlist and for shared_ptr pointing at it at
> once).

  There's a failure at communication here. You are creating an array of
struct objects. That's not the issue here. The issue is the array which
is inside the struct, as its last member, which size is determined at
runtime (rather than at compile time).

> No. My argument is that there is always a way to write it in C++.

  There's always a way to write *what* in C++?

> That
> struct hack is there since C provides you no much other ways.

  Of course C provides you other ways. You could do this in C:

struct MyStruct
{
    int size;
    int* array; // instead of int array[0];
};

  However, now you need to allocate the array separately. Thus you end
up with two allocations: One for the struct instantiation itself, and
another for the array inside it. The struct hack avoids the latter
allocation by making the array be in the same memory block as the
struct instantiation itself.

  In C++ you can use std::vector instead of int*, but that changes nothing
(it only makes it easier and safer to use, but it still causes the same
amount of memory allocations to be performed).

> If the
> already existing things do not suit you then fine, but that hack is
> ugly don't you see?

  I am not questioning the ugliness, safety or anything else. Of course
using std::vector instead of the struct hack (or a raw array with int*)
is easier and safer. The only thing I'm arguing is allocation efficiency.

Juha Nieminen wrote:
>
>  Of course C provides you other ways. You could do this in C:
>
> struct MyStruct
> {
>    int size;
>    int* array; // instead of int array[0];
> };
>
>  However, now you need to allocate the array separately. Thus you
> end
> up with two allocations: One for the struct instantiation itself,
> and another for the array inside it. The struct hack avoids the
> latter allocation by making the array be in the same memory block
> as the
> struct instantiation itself.
>
>  In C++ you can use std::vector instead of int*, but that changes
> nothing (it only makes it easier and safer to use, but it still
> causes the same amount of memory allocations to be performed).

No, in C++ you would use a std::vector in place of the MyStruct, not 
in place of the buffer pointer. That is what people have been trying 
to tell you here.

C and C++ are two different languages, and you do things differently.


Bo Persson
 

Bo Persson <bop@gmb.dk> wrote:
> No, in C++ you would use a std::vector in place of the MyStruct, not 
> in place of the buffer pointer. That is what people have been trying 
> to tell you here.

  Care to show me how it would be done?

  The idea in the original post, in other words, the idea with the so-called
"struct hack", is that you have an object which has an array as member, the
size of this array being determined at runtime. In other words, what you
would normally write like this:

    struct MyStruct
    {
        int someData, someOtherData;
        std::vector<int> anArray;
    };

  That's a struct which has an array as member. Its minor "problem" is
that the array is allocated separately from the struct itself. The struct
just contains a pointer to that array (which is what std::vector really is,
internally). This usually doesn't matter much. However, if you need to
instantiate MyStruct dynamically, and you need to do a lot, then you are
effectively doubling the amount of allocations being performed (each
allocation of MyStruct also causes an additional allocation performed
by std::vector).

  The C struct hack is a low-level optimization of this, where the array
is allocated alongside the struct itself. Due to how C (and C++) works,
this is technically possible and feasible. Thus you write the above as:

    struct MyStruct
    {
        int someData, someOtherData;
        int anArray[0];
    };

and then when allocating an instance of MyStruct, you "overallocate" memory
for it. The additional memory is used as the memory needed by the 'anArray'
member, which can then be accessed normally with operator[].

  This way only one allocation is needed, even though the size of 'anArray'
is determined at runtime.

  Now, would you kindly show me how std::vector<MyStruct> achieves this
same result?

  I think you are confused about what the struct hack is all about. Please
try to understand what is it that the struct hack is doing. I have tried
to explain it as well as I can.

Juha Nieminen wrote:

> Balog Pal <pasa@lib.hu> wrote:
>> Hard to decide that your article is a sour attemt at cheating or just
>> basic
>> trolling.   We discuss performance of vector<int> then you insert a
>> snippet
>> with set<int> like it had any relevance.  :-(((
> 
>   No, we were discussing the efficiency of memory allocation.
> 
>   If you are instantiating a million std::vector instances (initialized
> with a non-zero size), there will be a million allocations.
> 
>> Different collections have different characteristics wrt performance,
>> memory
>> footprint, etc, we ALL know that.  Now please go back to the original
>> claim instead of drifting in all directions.
> 
>   The discussion was about the efficiency of memory allocation. 'new' and
> 'delete' are heavy operations.
> 
>>>  If you are allocating a million instances of the struct, each such
>>> instance having an std::vector object inside, reserve() would do nothing
>>> to alleviate the memory fragmentation.
>> 
>> And if you allocate a million instances of the struct hack, it will take
>> a
>> while too.   now do you suggest using the struct hack in the *payload*?
> 
>   If you allocate a million instances of the struct using the struct hack,
> you will be making one million memory allocations.
> 
>   If you allocate a million instances of the struct without the struct
>   hack
> (ie. it has a std::vector as an element instead), you will be making *two*
> million memory allocations.
> 
>   What is so hard to understand in that?

Nothing. However, _why_ would you use a vector _member_ in the struct? The 
natural thing is to use a vector _instead_ of the struct. After all, the 
struct as it stands just serves the same purpose as a vector. It has no 
other members than the length and the array of elements.

So, instead of:

struct SvrList{
    unsigned int uNum;
    GameSvr  svr[0];            
};
....
SvrList* pList = 
  (SvrList*)malloc(sizeof( SvrList) + svrNum*sizeof(GameSvr));
for ( int i = 0; i < svrNum; ++i ) {
  pList->svr[i] = GameSvr();
}

you would do:

  std::vector< GameSvr > svr_seq ( svrNum );


If you use std::vector instead of the struct, you have just as many dynamic 
memory allocations. There still are technical differences: (a) the size 
variable could be stored in a different place, (b) vector also manages a 
capacity, (c) std::vector initializes the elements, and (d) you can only use 
it if (d) GameSvr satisfies the CopyConstructible and Assignable concept. 
However, the number of allocations would be the same.


Best

Kai-Uwe Bux

Kai-Uwe Bux <jkherciueh@gmx.net> wrote:
> Nothing. However, _why_ would you use a vector _member_ in the struct? The 
> natural thing is to use a vector _instead_ of the struct. After all, the 
> struct as it stands just serves the same purpose as a vector. It has no 
> other members than the length and the array of elements.

  The struct in the examples is simplified for the sake of being an example.
The struct can have (and often has) other members besides the size of the
array.

  The struct hack is not necessarily about making a struct which acts as an
array which contains the size of the array. The struct hack is about a struct
which, besides any other members it may have, also has an array as its
last element, the size of which is determined at runtime.

  But yes: If a C program is using the struct hack solely for the same
purpose as you would normally use std::vector (in other words, the struct
only contains the array size plus the array itself), then std::vector
would definitely be the better choice in the equivalent C++ program.

  However, as said, the struct hack can be used for more than that.

On 08/22/10 05:35 PM, Juha Nieminen wrote:
> Ian Collins<ian-news@hotmail.com>  wrote:
>> On 08/22/10 04:44 PM, Juha Nieminen wrote:
>
>>>     In the latter case there will be *two* allocations: One for the struct
>>> and another for the std::vector. In the former case there will be only
>>> one allocation.
>>
>> Big deal.
>
>    So you *do* understand my point, finally? In other words, that with the
> struct hack the total amount of memory allocations will be halved compared
> to the cleaner solution (using a dynamically allocated array as the member
> of the struct).
>
>    If you are making a huge amount of such allocations, it *will* be a big
> deal because 'new' and 'delete' (or malloc() and free() if you like) are
> quite heavy operations. That's what my example with the std::set was
> demonstrating.

If you are allocating a large number of vectors *and* those allocations 
are a performance bottleneck, you would use a custom allocator (possibly 
by utilising std::vector's  allocator template parameter) so new is no 
longer be a heavy operation.

-- 
Ian Collins

Juha Nieminen wrote:

> Kai-Uwe Bux <jkherciueh@gmx.net> wrote:
>> Nothing. However, _why_ would you use a vector _member_ in the struct?
>> The natural thing is to use a vector _instead_ of the struct. After all,
>> the struct as it stands just serves the same purpose as a vector. It has
>> no other members than the length and the array of elements.
> 
>   The struct in the examples is simplified for the sake of being an
>   example.
> The struct can have (and often has) other members besides the size of the
> array.
>
>   The struct hack is not necessarily about making a struct which acts as
>   an
> array which contains the size of the array. The struct hack is about a
> struct which, besides any other members it may have, also has an array as
> its last element, the size of which is determined at runtime.
> 
>   But yes: If a C program is using the struct hack solely for the same
> purpose as you would normally use std::vector (in other words, the struct
> only contains the array size plus the array itself), then std::vector
> would definitely be the better choice in the equivalent C++ program.
> 
>   However, as said, the struct hack can be used for more than that.

Now, you made me curious. Could you present an example? I would try to 
rewrite that without performance penalty in a "hack free" manner. The 
reasong is: I have a hard time imagining a use for the struct hack in C++ 
such that doing the same thing in a more idiomatic way comes with a 
performance hit. On the other hand, I know that sometimes my imagination is 
just lacking.


Best

Kai-Uwe Bux

Ian Collins <ian-news@hotmail.com> wrote:
> If you are allocating a large number of vectors *and* those allocations 
> are a performance bottleneck, you would use a custom allocator (possibly 
> by utilising std::vector's  allocator template parameter) so new is no 
> longer be a heavy operation.

  That would indeed be a cleaner C++ solution to the problem. Of course
creating such an allocator is quite complicated (and quite hard to make
it any more efficient than the default allocator if you need it to be
thread-safe).

  Of course the best solution would be to see if the program could be
designed in such way that you don't need so many individual allocations
in the first place... (Naturally this is not always possible, but it's
always a good option to consider.)

Kai-Uwe Bux <jkherciueh@gmx.net> wrote:
> Now, you made me curious. Could you present an example? I would try to 
> rewrite that without performance penalty in a "hack free" manner. The 
> reasong is: I have a hard time imagining a use for the struct hack in C++ 
> such that doing the same thing in a more idiomatic way comes with a 
> performance hit. On the other hand, I know that sometimes my imagination is 
> just lacking.

  Ok, off the top of my head (iow. I haven't checked if there's a much
better way of doing this equally efficiently):

  An array (which size is determined at runtime) which has the size
of the array, a reference count (so that the array can have shared
ownership) and a lock data structure so that the reference counting
can be done in a thread-safe manner.

On 2010-08-22 10:53, Kai-Uwe Bux wrote:
> Juha Nieminen wrote:
>
>>    But yes: If a C program is using the struct hack solely for the same
>> purpose as you would normally use std::vector (in other words, the struct
>> only contains the array size plus the array itself), then std::vector
>> would definitely be the better choice in the equivalent C++ program.
>>
>>    However, as said, the struct hack can be used for more than that.
>
> Now, you made me curious. Could you present an example? I would try to
> rewrite that without performance penalty in a "hack free" manner. The
> reasong is: I have a hard time imagining a use for the struct hack in C++
> such that doing the same thing in a more idiomatic way comes with a
> performance hit. On the other hand, I know that sometimes my imagination is
> just lacking.

How about this:

     #include <boost/intrusive/bs_set_hook.hpp>
     #include <boost/intrusive/treap.hpp>

     struct Message : public boost::intrusive::bs_set_base_hook<> {
         uint16_t priority;
         uint16_t length;
         uint8_t payload[];
     };

     // Message comparison for treap is based on payload
     boost::intrusive::treap<Message> mtreap;

Let's assume, you have to use a treap-like data structure, but you are 
free to distribute the data any way you like. "bs_set_base_hook<>" is 
basically a set of 3 pointers (to other messages). Whenever you access a 
message, you typically use priority, length, payload and a random 
pointer from "bs_set_base_hook<>".

The messages are small, lets say on average 16 bytes. Let's assume on a 
32-bit platform, sizeof(Message) will be 16 without payload and pointers 
are 4 bytes each, so messages will typically fit into a single cache line.

Let's assume for the purposes of this example, access to the messages is 
random and memory usage is more than 100x the CPU cache size.

Given these constraints, the only choice you have, is to locate all the 
data for a message in one place. You could of course use a vector with 
something like the small string optimization (instead of the flexible 
array). That would result in higher memory usage and more complex code 
but a much cleaner design/interface and re-usability.

\\

That being said, I'll happily admit that you can usually hide/eliminate 
the extra indirection 'proper' C++ requires and that the struct hack 
should be used very, very rarely.

On 22 aug, 10:02, Juha Nieminen <nos...@thanks.invalid> wrote:
> =D6=F6 Tiib <oot...@hot.ee> wrote:
> > No. boost::array does usually allocate together with struct that
> > contains it. It is like any usual non dynamic array. Since i used
> > boost::make_shared<> in my example it did exactly one allocation
> > (allocating room both for svrlist and for shared_ptr pointing at it at
> > once).
>
> =A0 There's a failure at communication here. You are creating an array of
> struct objects. That's not the issue here. The issue is the array which
> is inside the struct, as its last member, which size is determined at
> runtime (rather than at compile time).

You may allocate sufficient memory for several things (some of what
may be arrays with run-time decided sice) at once in C++ as well. You
have to manage it and use placement new to put all things into same
storage.

There may be some issues with alignment and padding, but some rules
help with it:
1) Structs can be used as members of arrays.
2) Arrays can be used as members of structs.
3) Arrays do not have padding between their members.
4) There is no padding before the first member of a structure, but
there may be padding between structure elements and at the end.

When you create things with placement new then you have to wrap
creation and destruction of it anyway to avoid bugs ... so i do not
see problems there too.


> > If the
> > already existing things do not suit you then fine, but that hack is
> > ugly don't you see?
>
> =A0 I am not questioning the ugliness, safety or anything else. Of course
> using std::vector instead of the struct hack (or a raw array with int*)
> is easier and safer. The only thing I'm arguing is allocation efficiency.

POD structure members are arranged in memory in the order they are
defined, C++ relaxes that requirement for classes in situations.
Perhaps best is to have pointers to such (allocated at once) arrays
anyway.

On Aug 21, 8:50=A0am, Juha Nieminen <nos...@thanks.invalid> wrote:
> > Why? IMO, new and delete are, on the success path, practically
> > equivalent on many common implementations. Difference is one if on
> > malloc failure, and new is inlined.
>
> =A0 You are basing your claims on your personal *opinion*? Rather than,
> you know, actually testing it in practice?
>
> > I say this claim of yours is poorly founded.
>
> =A0 It's quite well founded. For example, take this short piece of code:
>
> =A0 =A0 int main()
> =A0 =A0 {
> =A0 =A0 =A0 =A0 std::set<int> someSet;
> =A0 =A0 =A0 =A0 for(int i =3D 0; i < 10000000; ++i) someSet.insert(i);
> =A0 =A0 }
>

Oh, come on! set is not a vector. That's a MAJOR flaw in your example
and example is therefore completely of the mark.

> =A0 'new' and 'delete' are significantly heavy operations.
>
> >> =A0 Additionally, using std::vector there will increase memory fragmen=
tation,
> >> making things even worse.
>
> > In this case, you can use vector::reserve, so not really.
>
> =A0 If you are allocating a million instances of the struct, each such
> instance having an std::vector object inside, reserve() would do nothing
> to alleviate the memory fragmentation.

What!? Reserve would cause exactly 0 memory fragmentation, at least
until code starts reallocating or freeing these vectors, at which
point, there would be, fragmentation-wise, little, if any, difference
between a vector and discussed struct hack.

> > And even locality of reference is not a concern, because allocators
> > mostly do a good job of allocating blocks close in space if allocation
> > is close in time. E.g.
>
> > std::vector* p =3D new vector;
> > p->reserve(X);
>
> > is in practice quite OK wrt locality.
>
> =A0 Not if the memory is heavily fragmented, which is one major problem h=
ere.

You're still to prove how memory is fragmented. Until you reach
reserved size in a vector, there's no fragmentation.

All you have is one allocation more with a vector. That's easily
swamped by the rest of the code, especially if you have millions of
elements in it.

You need to have _a lot_ of vectors, all with a _small_ number of
elements in it for your complaint to be relevant. And for that,
there's no need to use contortions until one can measurein running
code, that performance hit is indeed relevant. You are trying to do it
backwards, and especially because programmers are proven time and time
over to be poor judges of performance problems.

Goran.

On Aug 21, 11:05=A0pm, "joe" <jc1...@att.net> wrote:
> Goran Pusic wrote:
> > I am speaking about locality because on current hardware, and in a
> > running system, it easily makes more difference than allocation. (I am
> > speaking abut a case where there's not too create-use-destroy cycles,
> > of course; and IME(xperience), best course of action when there is a
> > lot of that, are algorithmic changes; OPs hack comes in dead last,
> > when other options are explored).
>
> "OP's hack"? The technique is very well-known from C and C99 standardized
> it (C99 uses empty brackets rather than a array dimension of =A00 or 1). =
Do
> a web search on "struct hack" for all the details. In addition, you can
> search on "C99", and "flexible array member".

I shall do no such thing. Did you take a look at hbvla class from my
first post in this thread? I can implement this thing, in C++, with
the best here. And I certainly did it better than a random C hack like
OP's.

And I still know it's a hack that has very limited usefulness in real-
world code, because performance aspect is a massive case of
prematureoptimizationitis.

Then, hack has peformance worse than vector as soon as of elements
needs to change, as you can't do it at all without a complete
reallocation. Then, it's a question of standard-compliance.

There's no need to be condescending.

Goran.

Goran <goran.pusic@gmail.com> wrote:
>> > I say this claim of yours is poorly founded.
>>
>> � It's quite well founded. For example, take this short piece of code:
>>
>> � � int main()
>> � � {
>> � � � � std::set<int> someSet;
>> � � � � for(int i = 0; i < 10000000; ++i) someSet.insert(i);
>> � � }
>>
> 
> Oh, come on! set is not a vector. That's a MAJOR flaw in your example
> and example is therefore completely of the mark.

  The example is demonstrating the speed of 'new' and 'delete' compared
to other relatively complex operations (in this case inserting an element
into a balanced binary tree). In this case 'new' is over 3 times slower
than all the rebalancing needed in the element insertion, which is quite
a lot. That's the point in the example.

  Why are you nitpicking on the choice of data container when the whole
point was to demonstrate the speed of memory allocation?

>> >> � Additionally, using std::vector there will increase memory fragmentation,
>> >> making things even worse.
>>
>> > In this case, you can use vector::reserve, so not really.
>>
>> � If you are allocating a million instances of the struct, each such
>> instance having an std::vector object inside, reserve() would do nothing
>> to alleviate the memory fragmentation.
> 
> What!? Reserve would cause exactly 0 memory fragmentation, at least
> until code starts reallocating or freeing these vectors, at which
> point, there would be, fragmentation-wise, little, if any, difference
> between a vector and discussed struct hack.

  True. reserve() in itself causes 0 memory fragmentation in this case.
However, I didn't say that reserve() causes memory fragmentation. I said
that reserve() does not alleviate the memory fragmentation caused by
having a std::vector as a member of the struct. You should really try to
read what I'm writing.

  The std::vector is used simply as a subsitute of a raw array in this
case. Reallocation is not the issue here.

>> > And even locality of reference is not a concern, because allocators
>> > mostly do a good job of allocating blocks close in space if allocation
>> > is close in time. E.g.
>>
>> > std::vector* p = new vector;
>> > p->reserve(X);
>>
>> > is in practice quite OK wrt locality.
>>
>> � Not if the memory is heavily fragmented, which is one major problem here.
> 
> You're still to prove how memory is fragmented. Until you reach
> reserved size in a vector, there's no fragmentation.

  I'm still to prove how memory is fragmented? Hello? Do you even know
what memory fragmentation means, and how it happens during the execution
of a typical program?

  Making two memory allocations worsens memory fragmentation with higher
probability than making only one. reserve() has nothing to do with that.

> All you have is one allocation more with a vector. That's easily
> swamped by the rest of the code, especially if you have millions of
> elements in it.

  The problem is not one vector having a million elements. The problem
is a million vectors, each having a few elements. That's what the struct
hack is optimizing (well, one of the things).

> You need to have _a lot_ of vectors, all with a _small_ number of
> elements in it for your complaint to be relevant.

  And that's exactly what's happening if you have an array as the member
of a struct, and then you instantiate that struct millions of times. Which
was my original point.

> And for that,
> there's no need to use contortions until one can measurein running
> code, that performance hit is indeed relevant. You are trying to do it
> backwards, and especially because programmers are proven time and time
> over to be poor judges of performance problems.

  I'm trying to do it backwards? I'm not trying to do anything. I'm not
even advocating the use of the struct hack. All I'm saying is that one
of the possible reasons to use the struct hack is that it lessens the
amount of memory allocations (making the program potentially faster)
besides lessening memory fragmentation.

  There are many reasons to avoid such a low-level hack, and I have never
denied that.

Goran <goran.pusic@gmail.com> wrote:
>> "OP's hack"? The technique is very well-known from C and C99 standardized
>> it (C99 uses empty brackets rather than a array dimension of �0 or 1). Do
>> a web search on "struct hack" for all the details. In addition, you can
>> search on "C99", and "flexible array member".
> 
> I shall do no such thing.

  Yeah, way to go. Don't even try to understand what the struct hack
technique is all about, and then scream loudly how you know these things
better than anybody else. That's the way to discuss and to learn things.

> There's no need to be condescending.

  Look who's talking.

�� Tiib <ootiib@hot.ee> wrote:
> On 22 aug, 10:02, Juha Nieminen <nos...@thanks.invalid> wrote:
>> �� Tiib <oot...@hot.ee> wrote:
>> > No. boost::array does usually allocate together with struct that
>> > contains it. It is like any usual non dynamic array. Since i used
>> > boost::make_shared<> in my example it did exactly one allocation
>> > (allocating room both for svrlist and for shared_ptr pointing at it at
>> > once).
>>
>> � There's a failure at communication here. You are creating an array of
>> struct objects. That's not the issue here. The issue is the array which
>> is inside the struct, as its last member, which size is determined at
>> runtime (rather than at compile time).
> 
> You may allocate sufficient memory for several things (some of what
> may be arrays with run-time decided sice) at once in C++ as well. You
> have to manage it and use placement new to put all things into same
> storage.

  Well, that is, basically, what the struct hack is all about. The only
difference is that in C you don't use placement new to initialize the
object (and, technically speaking, you don't need to use placement new
in C++ either, if the struct is fully a POD type, in which case it will
work just as in C).

On 22 aug, 21:55, Juha Nieminen <nos...@thanks.invalid> wrote:
> =D6=F6 Tiib <oot...@hot.ee> wrote:
> > On 22 aug, 10:02, Juha Nieminen <nos...@thanks.invalid> wrote:
> >> =D6=F6 Tiib <oot...@hot.ee> wrote:
> >> > No. boost::array does usually allocate together with struct that
> >> > contains it. It is like any usual non dynamic array. Since i used
> >> > boost::make_shared<> in my example it did exactly one allocation
> >> > (allocating room both for svrlist and for shared_ptr pointing at it =
at
> >> > once).
>
> >> =A0 There's a failure at communication here. You are creating an array=
 of
> >> struct objects. That's not the issue here. The issue is the array whic=
h
> >> is inside the struct, as its last member, which size is determined at
> >> runtime (rather than at compile time).
>
> > You may allocate sufficient memory for several things (some of what
> > may be arrays with run-time decided sice) at once in C++ as well. You
> > have to manage it and use placement new to put all things into same
> > storage.
>
> =A0 Well, that is, basically, what the struct hack is all about. The only
> difference is that in C you don't use placement new to initialize the
> object (and, technically speaking, you don't need to use placement new
> in C++ either, if the struct is fully a POD type, in which case it will
> work just as in C).

We discuss performance optimization here. Sure C++ works somewhere
under surface technically as close to metal as C. For platform-
specific performance optimizations one has sometimes to hack even
deeper. What i am objecting against is building and exposing such
structs in some C++ interface what OP was all about.

On Aug 22, 8:52=A0pm, Juha Nieminen <nos...@thanks.invalid> wrote:
> Goran <goran.pu...@gmail.com> wrote:
> >> "OP's hack"? The technique is very well-known from C and C99 standardi=
zed
> >> it (C99 uses empty brackets rather than a array dimension of =A00 or 1=
). Do
> >> a web search on "struct hack" for all the details. In addition, you ca=
n
> >> search on "C99", and "flexible array member".
>
> > I shall do no such thing.
>
> =A0 Yeah, way to go. Don't even try to understand what the struct hack
> technique is all about, and then scream loudly how you know these things
> better than anybody else. That's the way to discuss and to learn things.

Please read my very first post in this thread and __then__ tell me I
don't understand what struct hack technique is all about.

But flames aside, I think we ultimately disagree only about one thing:
how often will one benefit from employing it.

My contention is: in practice, almost never. In practice, other
processing will swamp the cost of that one allocation. And if actual
number of elements needs to change (and in what I write, that's almost
always), vector beats struct hack, because struck hack has horrible
reallocation performance.

Goran.

Goran Pusic <goranp@cse-semaphore.com> wrote:
> But flames aside, I think we ultimately disagree only about one thing:
> how often will one benefit from employing it.

  Who is "we"? At least I am not claiming that the struct hack should
be used frequently, if at all. In fact, if I had such a situation in some
project of mine where using the struct hack would potentially bring
efficiency benefits, I would nevertheless try to think if the program
could be redesigned in such way that the benefit is achievable without
the struct hack.

thomas wrote:
>>
>> In this case the code is ill-formed, since 0-size array declaration is
>> illegal in C++ (as well as in C). In other words, arguing about the "out
>> of bounds" access here doesn't make much sense, since the code is
>> formally non-compilable.
> 
> Wait.. I don't think it's illegal in C++. At least I will definitely
> object making it illegal by the standard community.

It is formally illegal in both C and C++, as is explicitly stated in 
both standards.

> Will you guys getting crazy if we do things like this
> 
> -------code-----
> struct A{
>     int num;
>     int p[0];
> };
> A *pA = (A*)malloc(sizeof(A)+sizeof(int)*10);
> printf("%d\n", &pA->p[10] - &pA->p[0]);           //accessing out of
> bounds.
> ------code---

Just don't do things like this. If you want to use the "struct hack", 
either declare the array with size 1 (as shown in my post) or maybe even 
declare it with some huge size. Use `offsetof` instead of `sizeof` to 
calculate the size for `malloc`.

Once you think about it, you should realize that the habit of declaring 
an array with size 0 in "struct hack" originates from one and only one 
source: the desire to use `sizeof(A)` under `malloc` with no extra 
corrections (referring to your example). This, in turn, based on the 
simple fact that not too many programmers know about `offsetof` and its 
applications.

Of course, in C99 you should use the size-less declaration, which was 
introduced specifically for "struct hack" and specifically because 
zero-size array declaration is illegal.

-- 
Best regards,
Andrey Tarasevich

"Andrey Tarasevich" <andreytarasevich@hotmail.com> wrote in message
news:i4uc8m$hrs$1@news.eternal-september.org=20
> thomas wrote:
>>>=20
>>> In this case the code is ill-formed, since 0-size array declaration
>>> is illegal in C++ (as well as in C). In other words, arguing about
>>> the "out of bounds" access here doesn't make much sense, since the
>>> code is formally non-compilable.
>>=20
>> Wait.. I don't think it's illegal in C++. At least I will definitely
>> object making it illegal by the standard community.
>=20
> It is formally illegal in both C and C++, as is explicitly stated in
> both standards.
>=20
>> Will you guys getting crazy if we do things like this
>>=20
>> -------code-----
>> struct A{
>>     int num;
>>     int p[0];
>> };
>> A *pA =3D (A*)malloc(sizeof(A)+sizeof(int)*10);
>> printf("%d\n", &pA->p[10] - &pA->p[0]);           //accessing out of
>> bounds.
>> ------code---
>=20
> Just don't do things like this. If you want to use the "struct hack",
> either declare the array with size 1 (as shown in my post) or maybe
> even declare it with some huge size. Use `offsetof` instead of
> `sizeof` to calculate the size for `malloc`.

The problem with 'offsetof' in C++ is that it must be a compile time =
constant.
In the example above this is not a problem, but in general one sometimes
needs a size calculated at run time.

joe wrote:
> Vladimir Jovic wrote:
>> thomas wrote:
>>> On Aug 20, 2:01 am, Andrey Tarasevich <andreytarasev...@hotmail.com>
>>> wrote:
>>>> Pete Becker wrote:
>>>>>>>>> Hi, I need your help.
>>>>>>>>> ----------
>>>>>>>>> struct SvrList{
>>>>>>>>> unsigned int uNum;
>>>>>>>>> GameSvr  svr[0];            //line A
>>>>>>>>> };
>>>>>>>>> ---------
>>>>>>>>> Once I declared a struct like this to store server list info.
>>>>>>>>> It's supposed to be used like this.
>>>>>>>>> ----------
>>>>>>>>> SvrList* pList = (SvrList*)malloc(sizeof(
>>>>>>>>> SvrList) + svrNum*sizeof(GameSvr));
>>>>>>>>> pList->uNum, pList->svr[0], pList->svr[1].... blabla..
>>>>>>>> I wouldn't call this fine. Even
>>>>>>>> pList->svr[0]
>>>>>>>> is accessing the element that is out of array's bounds, and
>>>>>>>> that is UB.  How come your program is not crashing, or at
>>>>>>>> least going crazy? Maybe you are just unlucky to have a bug
>>>>>>>> hidden.
>>>>>>> It's an old C programmers hack. I've come across this idiom in
>>>>>>> lot's old C code, particularly driver and os code. Microsoft
>>>>>>> Win32 is rife with it.
>>>>>> Except that it's not legal C, either.
>>>>> Which is why it's referred to above as a "hack". Quite a common
>>>>> one, too.
>>>> Usually we use the term "hack" when the code relies on a specific
>>>> manifestation of undefined (or unspecified) behavior, but otherwise
>>>> is well-formed.
>>>>
>>>> In this case the code is ill-formed, since 0-size array declaration
>>>> is illegal in C++ (as well as in C). In other words, arguing about
>>>> the "out of bounds" access here doesn't make much sense, since the
>>>> code is formally non-compilable.
>>> Wait.. I don't think it's illegal in C++. At least I will definitely
>>> object making it illegal by the standard community.
>> In the c++ standard, see 8.3.4.1 , this part :
>>
>> ... If the constant-expression (5.19) is present, it shall be an
>> integral constant expression and its value shall be greater than
>> zero...
>> therefore it is illegal c++ code.
>>
>>> It can be dangerous but it can also do good. It depends on whether we
>>> are using it correctly.
>> Can't use it correctly. It is illegal, therefore undefined behaviour.
> 
> Undefined by the standard, but DEFINED by all implementations. The "it's 
> undefined behavior" ranting gets quite annoying. Try compiling something 
> with the standard instead of an implementation sometime and see how far 
> you get!

"not defined by the standard" means the implementation can do whatever 
it wants. In my opinion, the code should be compiled without warnings.
Following the standard, it is much easier to change compiler and even 
the OS and target platform.

I might be wrong, but if we take this definition of rant :
http://en.wikipedia.org/wiki/Rant
then what I wrote is not a rant :)

On Aug 22, 10:53=A0am, Kai-Uwe Bux <jkherci...@gmx.net> wrote:
>
> Now, you made me curious. Could you present an example? I would try to
> rewrite that without performance penalty in a "hack free" manner. The
> reasong is: I have a hard time imagining a use for the struct hack in C++
> such that doing the same thing in a more idiomatic way comes with a
> performance hit. On the other hand, I know that sometimes my imagination =
is
> just lacking.
>
> Best
>
> Kai-Uwe Bux

Here is an example where the struct-hack is extensively used in a real
production environment.

The system in question uses a message-passing mechanism to communicate
with both internal and external entities.
The basic message structure looks like this:

typedef struct
{
    unsigned char  media;
    unsigned char  receiver_dev;
    unsigned char  sender_dev;
    unsigned char  function;
    unsigned char  len[2];
    unsigned char  receiver_obj;
    unsigned char  sender_obj;
    unsigned char  data[1];
} MESSAGE_T;

The payload of the message, which is contained in the data array, is
typically between 4 and 16 bytes, but can be as much as 4000 bytes.
As said, these messages can be sent both to internal destinations
(sender_dev =3D=3D received_dev) and to external devices using a variety
of (serial) communication mechanisms.

Bart v Ingen Schenau

On Aug 24, 11:35=A0am, Bart van Ingen Schenau <b...@ingen.ddns.info>
wrote:
> On Aug 22, 10:53=A0am, Kai-Uwe Bux <jkherci...@gmx.net> wrote:
>
>
>
> > Now, you made me curious. Could you present an example? I would try to
> > rewrite that without performance penalty in a "hack free" manner. The
> > reasong is: I have a hard time imagining a use for the struct hack in C=
++
> > such that doing the same thing in a more idiomatic way comes with a
> > performance hit. On the other hand, I know that sometimes my imaginatio=
n is
> > just lacking.
>
> > Best
>
> > Kai-Uwe Bux
>
> Here is an example where the struct-hack is extensively used in a real
> production environment.
>
> The system in question uses a message-passing mechanism to communicate
> with both internal and external entities.
> The basic message structure looks like this:
>
> typedef struct
> {
> =A0 =A0 unsigned char =A0media;
> =A0 =A0 unsigned char =A0receiver_dev;
> =A0 =A0 unsigned char =A0sender_dev;
> =A0 =A0 unsigned char =A0function;
> =A0 =A0 unsigned char =A0len[2];
> =A0 =A0 unsigned char =A0receiver_obj;
> =A0 =A0 unsigned char =A0sender_obj;
> =A0 =A0 unsigned char =A0data[1];
>
> } MESSAGE_T;
>
> The payload of the message, which is contained in the data array, is
> typically between 4 and 16 bytes, but can be as much as 4000 bytes.
> As said, these messages can be sent both to internal destinations
> (sender_dev =3D=3D received_dev) and to external devices using a variety
> of (serial) communication mechanisms.

Sent over serial? In C++ i would have "std::vector<uint8_t> message_"
buried into some Message class and done.

Juha Nieminen wrote:
> joe <jc1996@att.net> wrote:
>> I don't think performance is the issue at all with this
>> apples-to-oranges comparison of std::vector and the struct hack:
>
>  Exactly how is it an "apples-to-oranges comparison"? The two
> techniques are achieving the same thing: A struct instantiation which
> contains an array whose size is decided at runtime (rather than at
> compile time).
> The difference between them is that the struct hack achieves this with
> one single allocation per struct instance, while the "cleaner" method
> requires two allocations per struct instance (one for the struct
> itself and another for the vector).

Apples and oranges. One is contiguous and one is not. You're "turning it 
inside out": performance comes as a side effect of the contiguity and the 
contiguity is the important part of the technique, not the side effect.

>
>  Performance will not be an issue if you need to allocate just a few
> instances of the struct. However, it can become an issue if you need
> to allocate millions of instances.
>
>> it's contiguity of the
>> var-len array with the other members of the struct that is the
>> reason for the struct hack's usefulness and popularity.
>
>  That may be so, but you can't dismiss the efficiency benefit in cases
> where huge amounts of instantiations need to be done.
>
>> std::vector is a very specialized container
>
>  Why are people here so obsessed about the specifics of std::vector
> here? It's only being used here as a substitute for a raw array. In
> terms of efficiency it doesn't make any difference compared to one.
> It's just easier to use.
>
>> Note that the OP
>> had the var-len array as a part of another struct. This isn't about
>> "array containers".
>
>  There seems to be some kind of failure at communication here.
>
>  I am not talking about the efficiency of allocating an array. I am
> talking *precisely* about the efficiency of allocating a struct which
> has an array as the last member, comparing the struct hack to a more
> regular solution (which has a pointer or a std::vector as the last
> struct element).

I was just pointing out that I don't think that performance is hardly the 
reason that the struct hack came to be. As such, I find the discussion 
about performance tangential to the OP. Performance is, of course, the 
proverbial low-hanging-fruit of discussion in programming groups. (I 
think that's what prompted me to ask in another post what is more 
worthwhile to know, "Big O" science or Assembly language. Obviously, I 
think the latter). 

Juha Nieminen wrote:

      struct Apple
>    {
>        // Some other member variables here.
>        int size;
>        int array[0];
>    };


      struct Orange
>    {
>        // Some other member variables here.
>        std::vector<int> array;
>
>        MyStruct(int size): array(size) {}
>    };

That the performance is different is a non-issue at best or a clouding of 
the purpose of the struct hack worst. I'd like to see C++ support the 
technique better than using a one-element array, which of course is 
syntactically hackish.

Ian Collins wrote:

> The struct hack is a C idiom which is seldom used in C++.

It's definitely a missing piece in C++, but at least the hack still 
works.

> In C++ we have other techniques.

Show me.

Juha Nieminen wrote:

> The struct hack is
> about a struct which, besides any other members it may have, also has
> an array as its
> last element, the size of which is determined at runtime.
>

Which, of course is incorrect. At least it is lax or unknowing by only 
giving one example usage (using dynamic mem with the struct hack) and 
presenting it as a definition of the struct hack. Again though, 
performance is hardly the point of the struct hack and std:vector is not 
a substitute for the struct hack. C was right in standardizing it and C++ 
would do good to follow suit. While there are more issues to address in 
developing an "alternative" to the struct hack in C++, they are not 
unsurmountable I think.

Kai-Uwe Bux wrote:
> Juha Nieminen wrote:
>
>> Kai-Uwe Bux <jkherciueh@gmx.net> wrote:
>>> Nothing. However, _why_ would you use a vector _member_ in the
>>> struct? The natural thing is to use a vector _instead_ of the
>>> struct. After all, the struct as it stands just serves the same
>>> purpose as a vector. It has no other members than the length and
>>> the array of elements.
>>
>>   The struct in the examples is simplified for the sake of being an
>>   example.
>> The struct can have (and often has) other members besides the size
>> of the array.
>>
>>   The struct hack is not necessarily about making a struct which
>>   acts as an
>> array which contains the size of the array. The struct hack is about
>> a struct which, besides any other members it may have, also has an
>> array as its last element, the size of which is determined at
>> runtime.
>>
>>   But yes: If a C program is using the struct hack solely for the
>> same purpose as you would normally use std::vector (in other words,
>> the struct only contains the array size plus the array itself), then
>> std::vector would definitely be the better choice in the equivalent
>> C++ program.
>>
>>   However, as said, the struct hack can be used for more than that.
>
> Now, you made me curious. Could you present an example? I would try to
> rewrite that without performance penalty in a "hack free" manner. The
> reasong is: I have a hard time imagining a use for the struct hack in
> C++ such that doing the same thing in a more idiomatic way comes with
> a performance hit. On the other hand, I know that sometimes my
> imagination is just lacking.

There is no alternative to the struct hack in C++. One has to use the 
hack if one needs/wants the characteristics of it because C++ has not 
formalized or accepted any form of the technique. A hole in the C++ spec 
(or worse) if you ask me. I'm fine with using the one-element array 
version of the technique, but I do feel like that is unnecessarily (for 
lack of formal C++ recognition and support of the technique) hackish (but 
not in comparison to how I use/abuse the preprocessor!).

tni wrote:
> On 2010-08-22 10:53, Kai-Uwe Bux wrote:
>> Juha Nieminen wrote:
>>
>>>    But yes: If a C program is using the struct hack solely for the
>>> same purpose as you would normally use std::vector (in other words,
>>> the struct only contains the array size plus the array itself),
>>> then std::vector would definitely be the better choice in the
>>> equivalent C++ program. However, as said, the struct hack can be used 
>>> for more than that.
>>
>> Now, you made me curious. Could you present an example? I would try
>> to rewrite that without performance penalty in a "hack free" manner.
>> The reasong is: I have a hard time imagining a use for the struct
>> hack in C++ such that doing the same thing in a more idiomatic way
>> comes with a performance hit. On the other hand, I know that
>> sometimes my imagination is just lacking.
>
> How about this:
>
>     #include <boost/intrusive/bs_set_hook.hpp>
>     #include <boost/intrusive/treap.hpp>
>
>     struct Message : public boost::intrusive::bs_set_base_hook<> {
>         uint16_t priority;
>         uint16_t length;
>         uint8_t payload[];
>     };
>
>     // Message comparison for treap is based on payload
>     boost::intrusive::treap<Message> mtreap;
>
> Let's assume, you have to use a treap-like data structure, but you are
> free to distribute the data any way you like. "bs_set_base_hook<>" is
> basically a set of 3 pointers (to other messages). Whenever you
> access a message, you typically use priority, length, payload and a
> random pointer from "bs_set_base_hook<>".
>
> The messages are small, lets say on average 16 bytes. Let's assume on
> a 32-bit platform, sizeof(Message) will be 16 without payload and
> pointers are 4 bytes each, so messages will typically fit into a
> single cache line.
> Let's assume for the purposes of this example, access to the messages
> is random and memory usage is more than 100x the CPU cache size.
>
> Given these constraints, the only choice you have, is to locate all
> the data for a message in one place. You could of course use a vector
> with something like the small string optimization (instead of the
> flexible array). That would result in higher memory usage and more
> complex code but a much cleaner design/interface and re-usability.
>
> \\
>
> That being said, I'll happily admit that you can usually
> hide/eliminate the extra indirection 'proper' C++ requires and that
> the struct hack should be used very, very rarely.

Those selective placements though are very, very important. Perhaps 
having come from a C originally I still have an appreciation for 
programming close to the metal whereas that concept escapes those that 
started out with the decidely higher-level C++ (or buy into it, lock, 
stock and barrel). 

Goran Pusic wrote:
> On Aug 22, 8:52 pm, Juha Nieminen <nos...@thanks.invalid> wrote:
>> Goran <goran.pu...@gmail.com> wrote:
>>>> "OP's hack"? The technique is very well-known from C and C99
>>>> standardized it (C99 uses empty brackets rather than a array
>>>> dimension of 0 or 1). Do a web search on "struct hack" for all the
>>>> details. In addition, you can search on "C99", and "flexible array
>>>> member".
>>
>>> I shall do no such thing.
>>
>> Yeah, way to go. Don't even try to understand what the struct hack
>> technique is all about, and then scream loudly how you know these
>> things better than anybody else. That's the way to discuss and to
>> learn things.
>
> Please read my very first post in this thread and __then__ tell me I
> don't understand what struct hack technique is all about.
>
> But flames aside, I think we ultimately disagree only about one thing:
> how often will one benefit from employing it.
>
> My contention is: in practice, almost never. In practice, other
> processing will swamp the cost of that one allocation. And if actual
> number of elements needs to change (and in what I write, that's almost
> always), vector beats struct hack, because struck hack has horrible
> reallocation performance.
>

Performance never was the issue. 

Goran wrote:
> On Aug 21, 11:05 pm, "joe" <jc1...@att.net> wrote:
>> Goran Pusic wrote:
>>> I am speaking about locality because on current hardware, and in a
>>> running system, it easily makes more difference than allocation. (I
>>> am speaking abut a case where there's not too create-use-destroy
>>> cycles, of course; and IME(xperience), best course of action when
>>> there is a lot of that, are algorithmic changes; OPs hack comes in
>>> dead last, when other options are explored).
>>
>> "OP's hack"? The technique is very well-known from C and C99
>> standardized it (C99 uses empty brackets rather than a array
>> dimension of 0 or 1). Do a web search on "struct hack" for all the
>> details. In addition, you can search on "C99", and "flexible array
>> member".
>
> I shall do no such thing. Did you take a look at hbvla class from my
> first post in this thread? I can implement this thing, in C++, with
> the best here. And I certainly did it better than a random C hack like
> OP's.
>
> And I still know it's a hack that has very limited usefulness in real-
> world code, because performance aspect is a massive case of
> prematureoptimizationitis.

Performance is not the issue. You are missing the whole point behind the 
struct hack. You and Juha.

>
> Then, hack has peformance worse than vector as soon as of elements
> needs to change, as you can't do it at all without a complete
> reallocation. Then, it's a question of standard-compliance.
>
> There's no need to be condescending.
>
> Goran. 

Juha Nieminen wrote:
> Goran Pusic <goranp@cse-semaphore.com> wrote:
>> But flames aside, I think we ultimately disagree only about one
>> thing: how often will one benefit from employing it.
>
>  Who is "we"? At least I am not claiming that the struct hack should
> be used frequently, if at all. In fact, if I had such a situation in
> some project of mine where using the struct hack would potentially
> bring efficiency benefits, I would nevertheless try to think if the
> program could be redesigned in such way that the benefit is
> achievable without the struct hack.

Before you can use a tool, you must understand what its purpose is. The 
struct hack is not a performance optimization. You have categorized it 
incorrectly. 

�� Tiib wrote:
> On 22 aug, 07:55, Juha Nieminen <nos...@thanks.invalid> wrote:
>> �� Tiib <oot...@hot.ee> wrote:
>>>> Exactly how would boost::array make this any faster? There's no
>>>> difference. The issue is not reallocation.
>>
>>> What is the issue, then?
>>
>> How many times does this have to be repeated?
>>
>> The issue is that with a member vector (or boost::array or whatever
>> you want to use), there will be *two* memory allocations (one for
>> the struct itself and another for the member array), while with the
>> struct hack there will be only *one* memory allocation.
>
> No. boost::array does usually allocate together with struct that
> contains it. It is like any usual non dynamic array. Since i used
> boost::make_shared<> in my example it did exactly one allocation
> (allocating room both for svrlist and for shared_ptr pointing at it at
> once).
>
>> Why is that so hard to understand?
>>
>> If your argument is that it doesn't matter efficiency-wise, then you
>> are simply wrong if the amount of struct instantiations is
>> significantly large because memory allocation is a heavy operation
>> in most systems.
>
> No. My argument is that there is always a way to write it in C++.

The only way to get struct hack characteristics in C++ is to use the 
struct hack (some version of it). C++ provides no alternative nor does it 
recognize the technique. It is deficient in that regard.

> That struct hack is there since C provides you no much other ways.

C99 standardizes the technique and therefor provides the only way, while 
C++ has a hole in the spec for the capabilities of the struct hack.

> If the
> already existing things do not suit you then fine, but that hack is
> ugly don't you see?

It doesn't matter how ugly it is, if it is the correct tool for the job. 
In C++, when you need the struct hack, you can either build the right 
thing, using the struct hack, even though it is ugly, or you can build 
the wrong thing that is pretty using some inappropriate C++ 
construct/abstraction. Correctness vs. cosmetics. 

Juha Nieminen wrote:
> �� Tiib <ootiib@hot.ee> wrote:
>> No. boost::array does usually allocate together with struct that
>> contains it. It is like any usual non dynamic array. Since i used
>> boost::make_shared<> in my example it did exactly one allocation
>> (allocating room both for svrlist and for shared_ptr pointing at it
>> at once).
>
>  There's a failure at communication here. You are creating an array of
> struct objects. That's not the issue here. The issue is the array
> which is inside the struct, as its last member, which size is
> determined at runtime (rather than at compile time).
>
>> No. My argument is that there is always a way to write it in C++.
>
>  There's always a way to write *what* in C++?
>
>> That
>> struct hack is there since C provides you no much other ways.
>
>  Of course C provides you other ways. You could do this in C:
>
> struct MyStruct
> {
>    int size;
>    int* array; // instead of int array[0];
> };

That's not contiguous. Entirely different animal. You may be able to 
replace the above with the struct hack, but you can't replace the struct 
hack with the above, for it would be fundamentatlly incorrect to do so.

Bo Persson wrote:
> Juha Nieminen wrote:
>>
>>  Of course C provides you other ways. You could do this in C:
>>
>> struct MyStruct
>> {
>>    int size;
>>    int* array; // instead of int array[0];
>> };
>>
>>  However, now you need to allocate the array separately. Thus you
>> end
>> up with two allocations: One for the struct instantiation itself,
>> and another for the array inside it. The struct hack avoids the
>> latter allocation by making the array be in the same memory block
>> as the
>> struct instantiation itself.
>>
>>  In C++ you can use std::vector instead of int*, but that changes
>> nothing (it only makes it easier and safer to use, but it still
>> causes the same amount of memory allocations to be performed).
>
> No, in C++ you would use a std::vector in place of the MyStruct, not
> in place of the buffer pointer. That is what people have been trying
> to tell you here.

You are missing the fact that the above array-looking struct is just an 
example and that the struct could and probably will have other members 
and represent some application-specific thing rather than simply an 
array. Someone posted an example of a Message struct that should clear 
things up a bit for you.

>
> C and C++ are two different languages, and you do things differently.

Except for the struct hack. Well, in C99 its no longer a hack, but in C++ 
one has no alternative but to use the traditional hack.

Juha Nieminen wrote:
> �� Tiib <ootiib@hot.ee> wrote:
>> On 22 aug, 10:02, Juha Nieminen <nos...@thanks.invalid> wrote:
>>> �� Tiib <oot...@hot.ee> wrote:
>>>> No. boost::array does usually allocate together with struct that
>>>> contains it. It is like any usual non dynamic array. Since i used
>>>> boost::make_shared<> in my example it did exactly one allocation
>>>> (allocating room both for svrlist and for shared_ptr pointing at
>>>> it at once).
>>>
>>> There's a failure at communication here. You are creating an array
>>> of struct objects. That's not the issue here. The issue is the
>>> array which is inside the struct, as its last member, which size is
>>> determined at runtime (rather than at compile time).
>>
>> You may allocate sufficient memory for several things (some of what
>> may be arrays with run-time decided sice) at once in C++ as well. You
>> have to manage it and use placement new to put all things into same
>> storage.
>
>  Well, that is, basically, what the struct hack is all about.

No it isn't. Dynamic memory allocation is orthogonal to the concepts of 
struct hack.

On Aug 25, 10:27=A0am, "joe" <jc1...@att.net> wrote:
> [...]
> It doesn't matter how ugly it is, if it is the correct tool for the job.
> In C++, when you need the struct hack, you can either build the right
> thing, using the struct hack, even though it is ugly, or you can build
> the wrong thing that is pretty using some inappropriate C++
> construct/abstraction. Correctness vs. cosmetics.

But when does one need the struct hack? If you need a variable-sized
member but don't want to pay the cost for double indirection, the
idiomatic C++ answer can *still* be std::vector: after all, most
operations are applied on ranges (iterator pairs), not containers. So
you pass iterators by value anyway, not containers by reference (which
is where you may pay an additional indirection depending on QoI). If
you're working with an API that expects containers, it's (IMO) a bad
design on the part of the designers: the concern of container classes
is to manage the lifetimes of their elements, passing container
instances or references to them around is not that as innocent as
working with views (i.e. iterators in C++) to their elements.

In fact, even in C99, if you have a function of the form
ret_t some_func(hack_t *inst);

it could be written in the form:
ret_t some_func(elt_t *data, size_t count);

After all you can only have one flexible member of known type.
Of course you can then argue that the new function expects *any* sort
of element range, where perhaps you wanted a module-like interface,
where different hack_t objects not only had different flexible member
length (i.e. the count argument), but different interpretation of it.
A classical use of C99 flexible members for encapsulation and/or
opaque types.

Then use interfaces in C++. The "problem" of double indirection is
still likely to be here depending on QoI, but really, when you use the
struct hack/flexible members in C for encapsulation/data hiding (as
opposed to a dynamic size only) you're really implementing some sort
of dynamic dispatch and so you *have* to pay some amount of
indirection; in C++ might as well let the implementation deal with it
with the explicifit language facilities instead of succumbing to
premature optimization. The implementation is (with all due respect)
more likely to cache or otherwise deal with the double indirection.

If you're wondering I do not find the struct hack/flexible member
'ugly'. I make use of it when writing C99 and am very glad that it was
made part of the standard.

On Aug 25, 10:06=A0am, lucdanton <lucdan...@free.fr> wrote:
> On Aug 25, 10:27=A0am, "joe" <jc1...@att.net> wrote:
>
> > [...]
> > It doesn't matter how ugly it is, if it is the correct tool for the job=
..
> > In C++, when you need the struct hack, you can either build the right
> > thing, using the struct hack, even though it is ugly, or you can build
> > the wrong thing that is pretty using some inappropriate C++
> > construct/abstraction. Correctness vs. cosmetics.
>
> But when does one need the struct hack? If you need a variable-sized
> member but don't want to pay the cost for double indirection,

The "cost of double indirection" is not the same as "the performance
hit of any extra malloc() and and an extra dereference".  Double
indirection means non-contiguous. Sometimes you need a simple,
*contiguous* structure of variable size.  In particular, inter-process/
inter-language operations are often greatly simplified by using such
structures.

If you can't see any applications where contiguous data structures are
important, you will never be convinced that std::vector isn't the
answer.  If you can, you will.

On Aug 25, 12:19=A0pm, gwowen <gwo...@gmail.com> wrote:
> [...]
> In particular, inter-process/
> inter-language operations are often greatly simplified by using such
> structures.
>
> If you can't see any applications where contiguous data structures are
> important, you will never be convinced that std::vector isn't the
> answer. =A0If you can, you will.

Those are nice examples, thanks. For those specific domains I'd
probably write a class with the variable aspect of it tucked away in a
container (idiomatic C++), and write serializing and deserializing
operations that pack the representation on a contiguous strip. Well,
assuming I have the need for the class on the C++ side, just the
serialization operations with containers arguments if only an
interface is needed.

If I don't have the luxury of preparing a new representation each time
I serialize, then I could emulate the struct hack by writing a class
that only pass around the contiguous strip (typically a
std::vector<unsigned char>) with an interface if that's important.
Obviously more painful than C99 (or C90 with extensions) since this
will probably require all that offsetof magic; especially tedious if
the types on the strip are not trivial (but then that comes from C++,
not C). By this point I'm convinced that flexible members would be
neat in C++, even if it required trivialness like (restricted) unions
do (well, POD-ness) :) although to be honest I wouldn't give it much
priority (still higher than VLA's though!).

joe <jc1996@att.net> wrote:
> There is no alternative to the struct hack in C++.

  However, C++ offers you tools to abstract away the struct hack (ie.
behind easier-to-use public interfaces of classes and/or via using a
custom memory allocator), making the usage easier, cleaner and safer.

joe <jc1996@att.net> wrote:
> Before you can use a tool, you must understand what its purpose is. The 
> struct hack is not a performance optimization. You have categorized it 
> incorrectly. 

  Well, let's say that the only advantage I can think of with struct hacks
is efficiency (they consume less memory and are faster to allocate than
the "normal" solution of allocating the struct and the array separately).

  I can't think of any significant advantages. If memory usage efficiency
is not an issue, why would I use the struct hack for anything? There are
much cleaner, easier and safer ways of achieving the same result in cases
where efficiency is not an issue.

"Juha Nieminen" <nospam@thanks.invalid> wrote in message 
news:4c74ffeb$0$12259$7b1e8fa0@news.nbl.fi...
> joe <jc1996@att.net> wrote:
>
>  Well, let's say that the only advantage I can think of with struct hacks
> is efficiency (they consume less memory and are faster to allocate than
> the "normal" solution of allocating the struct and the array separately).
>
>  I can't think of any significant advantages. If memory usage efficiency
> is not an issue, why would I use the struct hack for anything?
Marshalling can be an issue. Think of passing the struct from one process to 
another using shared memory. When you have the array in a stl vector, you 
have additional costs. The same applies to inter process communication using 
any socket layer like TCP.

Helge

Helge Kruse wrote:
> "Juha Nieminen" <nospam@thanks.invalid> wrote in message 
> news:4c74ffeb$0$12259$7b1e8fa0@news.nbl.fi...
>> joe <jc1996@att.net> wrote:
>>
>>  Well, let's say that the only advantage I can think of with struct hacks
>> is efficiency (they consume less memory and are faster to allocate than
>> the "normal" solution of allocating the struct and the array separately).
>>
>>  I can't think of any significant advantages. If memory usage efficiency
>> is not an issue, why would I use the struct hack for anything?
> Marshalling can be an issue. Think of passing the struct from one process to 
> another using shared memory. When you have the array in a stl vector, you 
> have additional costs. The same applies to inter process communication using 
> any socket layer like TCP.

But in that case, why use hacks of this type? For IPC, you can not pass 
pointers. There are other mechanisms, therefore there are no needs for 
such hacks.

"Vladimir Jovic" <vladaspams@gmail.com> wrote in message 
news:i52vos$kd8$1@news.albasani.net...
> Helge Kruse wrote:
>> "Juha Nieminen" <nospam@thanks.invalid> wrote in message 
>> news:4c74ffeb$0$12259$7b1e8fa0@news.nbl.fi...
>>> joe <jc1996@att.net> wrote:
>>>
>>>  Well, let's say that the only advantage I can think of with struct 
>>> hacks
>>> is efficiency (they consume less memory and are faster to allocate than
>>> the "normal" solution of allocating the struct and the array 
>>> separately).
>>>
>>>  I can't think of any significant advantages. If memory usage efficiency
>>> is not an issue, why would I use the struct hack for anything?
>> Marshalling can be an issue. Think of passing the struct from one process 
>> to another using shared memory. When you have the array in a stl vector, 
>> you have additional costs. The same applies to inter process 
>> communication using any socket layer like TCP.
>
> But in that case, why use hacks of this type? For IPC, you can not pass 
> pointers. There are other mechanisms, therefore there are no needs for 
> such hacks.

What pointers? The OP wrote this:

struct SvrList{
    unsigned int uNum;
    GameSvr  svr[0];            //line A
};

On Aug 25, 12:35=A0pm, Juha Nieminen <nos...@thanks.invalid> wrote:

> I can't think of any significant advantages.

Sometime contiguity is needed, or very, very helpful, to the extent
that is a significant.  The fact that you do not believe this has no
bearing on the fact that it is true.

Helge Kruse wrote:
> "Vladimir Jovic" <vladaspams@gmail.com> wrote in message 
> news:i52vos$kd8$1@news.albasani.net...
>> Helge Kruse wrote:
>>> "Juha Nieminen" <nospam@thanks.invalid> wrote in message 
>>> news:4c74ffeb$0$12259$7b1e8fa0@news.nbl.fi...
>>>> joe <jc1996@att.net> wrote:
>>>>
>>>>  Well, let's say that the only advantage I can think of with struct 
>>>> hacks
>>>> is efficiency (they consume less memory and are faster to allocate than
>>>> the "normal" solution of allocating the struct and the array 
>>>> separately).
>>>>
>>>>  I can't think of any significant advantages. If memory usage efficiency
>>>> is not an issue, why would I use the struct hack for anything?
>>> Marshalling can be an issue. Think of passing the struct from one process 
>>> to another using shared memory. When you have the array in a stl vector, 
>>> you have additional costs. The same applies to inter process 
>>> communication using any socket layer like TCP.
>> But in that case, why use hacks of this type? For IPC, you can not pass 
>> pointers. There are other mechanisms, therefore there are no needs for 
>> such hacks.
> 
> What pointers? The OP wrote this:
> 
> struct SvrList{
>     unsigned int uNum;
>     GameSvr  svr[0];            //line A
> };
> 
> 
> 

You switched to interprocess communication. For that, this hack should 
not be used. For IPC (at least on linux, don't know how it is done on 
other platforms), it is done this way:
http://linux.die.net/man/7/mq_overview
http://linux.die.net/man/2/mmap
http://linux.die.net/man/2/shmget

On Aug 25, 2:59=A0pm, Vladimir Jovic <vladasp...@gmail.com> wrote:
>
> You switched to interprocess communication. For that, this hack should
> not be used. For IPC (at least on linux, don't know how it is done on
> other platforms), it is done this way:http://linux.die.net/man/7/mq_overv=
iew
> http://linux.die.net/man/2/mmap
> http://linux.die.net/man/2/shmget

ATWAV

"Vladimir Jovic" <vladaspams@gmail.com> wrote in message 
news:i537l0$a5$1@news.albasani.net...
> Helge Kruse wrote:
>> "Vladimir Jovic" <vladaspams@gmail.com> wrote in message 
>> news:i52vos$kd8$1@news.albasani.net...
>>> Helge Kruse wrote:
>>>> "Juha Nieminen" <nospam@thanks.invalid> wrote in message 
>>>> news:4c74ffeb$0$12259$7b1e8fa0@news.nbl.fi...
>>>>> joe <jc1996@att.net> wrote:
>>>>>
>>>>>  Well, let's say that the only advantage I can think of with struct 
>>>>> hacks
>>>>> is efficiency (they consume less memory and are faster to allocate 
>>>>> than
>>>>> the "normal" solution of allocating the struct and the array 
>>>>> separately).
>>>>>
>>>>>  I can't think of any significant advantages. If memory usage 
>>>>> efficiency
>>>>> is not an issue, why would I use the struct hack for anything?
>>>> Marshalling can be an issue. Think of passing the struct from one 
>>>> process to another using shared memory. When you have the array in a 
>>>> stl vector, you have additional costs. The same applies to inter 
>>>> process communication using any socket layer like TCP.
>>> But in that case, why use hacks of this type? For IPC, you can not pass 
>>> pointers. There are other mechanisms, therefore there are no needs for 
>>> such hacks.
>>
>> What pointers? The OP wrote this:
>>
>> struct SvrList{
>>     unsigned int uNum;
>>     GameSvr  svr[0];            //line A
>> };
>>
>>
>>
>
> You switched to interprocess communication. For that, this hack should not 
> be used. For IPC (at least on linux, don't know how it is done on other 
> platforms), it is done this way:
> http://linux.die.net/man/7/mq_overview
> http://linux.die.net/man/2/mmap
> http://linux.die.net/man/2/shmget

Thanks for these (os-dependent) examples. These functions like msgsnd expect 
a pointer to _one_ object. That fits perfect to the struct show above. The 
GameSvr array is embedded in the SrvList object. This avoids pointers.
>>> ... For IPC, you can not pass pointers.  ...

When you have a pointer to an addtional chunk of data, like std::vector 
implementations use, instead of an embedded object you have to build an 
message object, compose a memory layout and than you are ready to pass it to 
msgsend. Finally you drop the temporary object.

gwowen wrote:
> On Aug 25, 10:06 am, lucdanton <lucdan...@free.fr> wrote:
>> On Aug 25, 10:27 am, "joe" <jc1...@att.net> wrote:
>>
>>> [...]
>>> It doesn't matter how ugly it is, if it is the correct tool for the
>>> job. In C++, when you need the struct hack, you can either build
>>> the right thing, using the struct hack, even though it is ugly, or
>>> you can build the wrong thing that is pretty using some
>>> inappropriate C++ construct/abstraction. Correctness vs. cosmetics.
>>
>> But when does one need the struct hack? If you need a variable-sized
>> member but don't want to pay the cost for double indirection,
>

>  Double
> indirection means non-contiguous. Sometimes you need a simple,
> *contiguous* structure of variable size.  In particular,
> inter-process/ inter-language operations are often greatly simplified
> by using such structures.


It's amazing how many times that can be said and no one hears it.

>
> If you can't see any applications where contiguous data structures are
> important, you will never be convinced that std::vector isn't the
> answer.

IOW, "can't see the forrest for the trees"?

> If you can, you will. 

lucdanton wrote:
> On Aug 25, 12:19 pm, gwowen <gwo...@gmail.com> wrote:
>> [...]
>> In particular, inter-process/
>> inter-language operations are often greatly simplified by using such
>> structures.
>>
>> If you can't see any applications where contiguous data structures
>> are important, you will never be convinced that std::vector isn't the
>> answer. If you can, you will.
>
> Those are nice examples, thanks. For those specific domains I'd
> probably write a class with the variable aspect of it tucked away in a
> container (idiomatic C++), and write serializing and deserializing
> operations that pack the representation on a contiguous strip.

That, is what C++ "forces" you to do because it lacks formalization of 
the struct hack. I'm not so sure that lack of struct hack standardization 
in C++ or avoidance of standardizing it isn't because of a fundamental 
incorrrectness of C++'s design.

Juha Nieminen wrote:
> joe <jc1996@att.net> wrote:
>> There is no alternative to the struct hack in C++.
>
>  However, C++ offers you tools to abstract away the struct hack (ie.
> behind easier-to-use public interfaces of classes and/or via using a
> custom memory allocator), making the usage easier, cleaner and safer.

Wrong. That is something different. Apples and oranges again, and 
actually, they're not even both fruits! 

Juha Nieminen wrote:
> joe <jc1996@att.net> wrote:
>> Before you can use a tool, you must understand what its purpose is.
>> The struct hack is not a performance optimization. You have
>> categorized it incorrectly.
>
>  Well, let's say that the only advantage I can think of with struct
> hacks is efficiency (they consume less memory and are faster to
> allocate than the "normal" solution of allocating the struct and the
> array separately).

It's obvious that you do think that.

>
>  I can't think of any significant advantages. If memory usage
> efficiency is not an issue, why would I use the struct hack for
> anything?

Go back and reread the other posts where it has already been explained.

> There are much cleaner, easier and safer ways of achieving
> the same result in cases where efficiency is not an issue.

No, there really isn't. It isn't an issue of the endpoint, it's an issue 
of the means by which to achieve it, for the topic is of a programming 
technique, and of facilitation of programming. It is not an issue of "oh 
well, it can be done in assembly language too!". That you personally 
don't care to have the tool in your toolbox is fine and dandy. I, OTOH, 
hold it as one of my favorite tools (not that it gets a lot of use, but 
that I find it very important to have when I need it). 

joe <jc1996@att.net> wrote:
> Juha Nieminen wrote:
>> joe <jc1996@att.net> wrote:
>>> There is no alternative to the struct hack in C++.
>>
>>  However, C++ offers you tools to abstract away the struct hack (ie.
>> behind easier-to-use public interfaces of classes and/or via using a
>> custom memory allocator), making the usage easier, cleaner and safer.
> 
> Wrong.

  Wrong? C++ does not offer tools to abstract the struct hack and make
it easier and safer to use?

  We'll have to disagree, then.

On Aug 27, 11:58=A0am, Juha Nieminen <nos...@thanks.invalid> wrote:
> Wrong? C++ does not offer tools to abstract the struct hack and make
> it easier and safer to use?
>
> =A0 We'll have to disagree, then.

The struct hack gives me a storage region of run-time-variable size,
contiguous with a place to store that size itself.  Which C++
container gives you that?  How would you abstract that requirement.

(Clue: Given such a container, what would sizeof(Container) evaluate
to - remember this needs to be a compile time constant?).

Helge Kruse wrote:
> "Vladimir Jovic" <vladaspams@gmail.com> wrote in message 
> news:i537l0$a5$1@news.albasani.net...
>> Helge Kruse wrote:
>>> "Vladimir Jovic" <vladaspams@gmail.com> wrote in message 
>>> news:i52vos$kd8$1@news.albasani.net...
>>>> Helge Kruse wrote:
>>>>> "Juha Nieminen" <nospam@thanks.invalid> wrote in message 
>>>>> news:4c74ffeb$0$12259$7b1e8fa0@news.nbl.fi...
>>>>>> joe <jc1996@att.net> wrote:
>>>>>>
>>>>>>  Well, let's say that the only advantage I can think of with struct 
>>>>>> hacks
>>>>>> is efficiency (they consume less memory and are faster to allocate 
>>>>>> than
>>>>>> the "normal" solution of allocating the struct and the array 
>>>>>> separately).
>>>>>>
>>>>>>  I can't think of any significant advantages. If memory usage 
>>>>>> efficiency
>>>>>> is not an issue, why would I use the struct hack for anything?
>>>>> Marshalling can be an issue. Think of passing the struct from one 
>>>>> process to another using shared memory. When you have the array in a 
>>>>> stl vector, you have additional costs. The same applies to inter 
>>>>> process communication using any socket layer like TCP.
>>>> But in that case, why use hacks of this type? For IPC, you can not pass 
>>>> pointers. There are other mechanisms, therefore there are no needs for 
>>>> such hacks.
>>> What pointers? The OP wrote this:
>>>
>>> struct SvrList{
>>>     unsigned int uNum;
>>>     GameSvr  svr[0];            //line A
>>> };
>>>
>>>
>>>
>> You switched to interprocess communication. For that, this hack should not 
>> be used. For IPC (at least on linux, don't know how it is done on other 
>> platforms), it is done this way:
>> http://linux.die.net/man/7/mq_overview
>> http://linux.die.net/man/2/mmap
>> http://linux.die.net/man/2/shmget
> 
> Thanks for these (os-dependent) examples. These functions like msgsnd expect 
> a pointer to _one_ object. That fits perfect to the struct show above. The 
> GameSvr array is embedded in the SrvList object. This avoids pointers.

Ok, I see. By using the hack, you will send variable size array of 
objects. Not bad.


>>>> ... For IPC, you can not pass pointers.  ...
> 
> When you have a pointer to an addtional chunk of data, like std::vector 
> implementations use, instead of an embedded object you have to build an 
> message object, compose a memory layout and than you are ready to pass it to 
> msgsend. Finally you drop the temporary object.
> 
> 

For IPC you can not even use vectors. At least I do not see how.

gwowen <gwowen@gmail.com> wrote:
> On Aug 27, 11:58�am, Juha Nieminen <nos...@thanks.invalid> wrote:
>> Wrong? C++ does not offer tools to abstract the struct hack and make
>> it easier and safer to use?
>>
>> � We'll have to disagree, then.
> 
> The struct hack gives me a storage region of run-time-variable size,
> contiguous with a place to store that size itself.  Which C++
> container gives you that?  How would you abstract that requirement.

  "C++ offers you tools" does not mean "C++ offers you a standard container".

  C++ offers tools for you to build your own abstract containers which
internally use the struct hack, but from the outside are easier and safer
to use than using the raw struct hack directly.

  Such tools include classes (with public and private sections),
constructors, copy constructors, assignment operator overloading,
destructors, the RAII mechanism, and templates.

  In other words, instead of using a raw struct pointer directly (which
points to a memory block which has been "overallocated" for the purposes
of the struct hack), you use an abstract object which resembles a smart
pointer and which hides and automatizes the low-level dirty details inside,
making the usage of the struct hack easier and safer.

On Aug 27, 12:16=A0pm, Juha Nieminen <nos...@thanks.invalid> wrote:

> C++ offers tools for you to build your own abstract containers which
> internally use the struct hack, but from the outside are easier and safer
> to use than using the raw struct hack directly.

Really?  I challenge you write one suitable for use in IPC.  Remember,
if your abstract container contains a pointer, its not going to work.

* gwowen, on 27.08.2010 13:20:
> On Aug 27, 12:16 pm, Juha Nieminen<nos...@thanks.invalid>  wrote:
>
>> C++ offers tools for you to build your own abstract containers which
>> internally use the struct hack, but from the outside are easier and safer
>> to use than using the raw struct hack directly.
>
> Really?  I challenge you write one suitable for use in IPC.  Remember,
> if your abstract container contains a pointer, its not going to work.

Please post code that shows the raw struct hack in action without using a pointer.



Cheers & hth.,

- Alf

-- 
blog at <url: http://alfps.wordpress.com>

gwowen <gwowen@gmail.com> wrote:
> Really?  I challenge you write one suitable for use in IPC.  Remember,
> if your abstract container contains a pointer, its not going to work.

  What do you think malloc() returns?

  That's right: It returns a raw pointer. This pointer points to a
memory block which you have to manage manually. You need to make sure
it doesn't leak, you don't free it more than once, you don't use it
after it has been freed, and you don't access the memory block out of
boundaries. Additionally, there's also the small chance of making a
mistake with the parameter to malloc() as well (in other words, you
accidentally allocate too little, or too much much memory, eg. because
of a typo or a miscalculation).

  That's where C++ offers you tools to both make it easier and safer.

  Think about char* vs. std::string. Both are used for basically the
same thing, but the latter is much harder to leak, to delete twice, or
to access after it has been deleted (and in theory the latter could have
additional checks to make sure you don't index out of boundaries, which
is the case with some compilers in debug mode).

On 08/27/10 11:20 PM, gwowen wrote:
> On Aug 27, 12:16 pm, Juha Nieminen<nos...@thanks.invalid>  wrote:
>
>> C++ offers tools for you to build your own abstract containers which
>> internally use the struct hack, but from the outside are easier and safer
>> to use than using the raw struct hack directly.
>
> Really?  I challenge you write one suitable for use in IPC.  Remember,
> if your abstract container contains a pointer, its not going to work.

I write a lot of IPC library code and I've never resorted to to the 
struct hack in C++.  Show us an example where you have had to use it and 
I'm sure you'll get plenty of counterexamples.

-- 
Ian Collins

Juha Nieminen wrote:
> gwowen <gwowen@gmail.com> wrote:
>> On Aug 27, 11:58 am, Juha Nieminen <nos...@thanks.invalid> wrote:
>>> Wrong? C++ does not offer tools to abstract the struct hack and make
>>> it easier and safer to use?
>>>
>>> We'll have to disagree, then.
>>
>> The struct hack gives me a storage region of run-time-variable size,
>> contiguous with a place to store that size itself.  Which C++
>> container gives you that?  How would you abstract that requirement.
>
>  "C++ offers you tools" does not mean "C++ offers you a standard
> container".
>
>  C++ offers tools for you to build your own abstract containers which
> internally use the struct hack, but from the outside are easier and
> safer to use than using the raw struct hack directly.
>
>  Such tools include classes (with public and private sections),
> constructors, copy constructors, assignment operator overloading,
> destructors, the RAII mechanism, and templates.
>
>  In other words, instead of using a raw struct pointer directly (which
> points to a memory block which has been "overallocated" for the
> purposes
> of the struct hack), you use an abstract object which resembles a
> smart pointer and which hides and automatizes the low-level dirty
> details inside, making the usage of the struct hack easier and safer.

OK. I thought you were saying originally above that C++ offered an 
alternative to the struct hack (because you have been on the track that 
something with a std::vector in it was the "same thing"). The "same ol" 
still applies though: because C++ does not recognize the struct hack as a 
viable technique (like C99 does), you're still (and I do, do this) 
wrapping a hack that may be compiler-specific or perhaps not guaranteed 
to work. And also, it's hackish because in certain forms you loose the 
ability to derive from such a wrapper, etc, and is therefore a 
dialect-style of C++ programming. 

Alf P. Steinbach /Usenet wrote:
> * gwowen, on 27.08.2010 13:20:
>> On Aug 27, 12:16 pm, Juha Nieminen<nos...@thanks.invalid>  wrote:
>>
>>> C++ offers tools for you to build your own abstract containers which
>>> internally use the struct hack, but from the outside are easier and
>>> safer to use than using the raw struct hack directly.
>>
>> Really?  I challenge you write one suitable for use in IPC. Remember, 
>> if your abstract container contains a pointer, its not
>> going to work.
>
> Please post code that shows the raw struct hack in action without
> using a pointer.

struct msg
{
    uint32 msg_id;
    uint32 data_cap; // capacity
    uint32 data_len;  // length
    char data[1];

    msg(uint32 id, uint32 cap, uint32 len, const char* dat)
   {
      msg_id = id;
      data_cap = cap;
      data_len = 0;
      strncpy(data, dat, len);
   }
};

void sendmsg(msg const& m);

const uint32 MSG_ID_HELLO = 1;

int main()
{
   unsigned char buff[256];
   msg* m =
       new((void*)buff) msg(MSG_ID_HELLO, 244, 13, "Hello world!");
   sendmsg(m);
   return 0;
} 

joe wrote:
> Alf P. Steinbach /Usenet wrote:
>> * gwowen, on 27.08.2010 13:20:
>>> On Aug 27, 12:16 pm, Juha Nieminen<nos...@thanks.invalid>  wrote:
>>>
>>>> C++ offers tools for you to build your own abstract containers
>>>> which internally use the struct hack, but from the outside are
>>>> easier and safer to use than using the raw struct hack directly.
>>>
>>> Really?  I challenge you write one suitable for use in IPC.
>>> Remember, if your abstract container contains a pointer, its not
>>> going to work.
>>
>> Please post code that shows the raw struct hack in action without
>> using a pointer.
>
> struct msg
> {
>    uint32 msg_id;
>    uint32 data_cap; // capacity
>    uint32 data_len;  // length
>    char data[1];
>
>    msg(uint32 id, uint32 cap, uint32 len, const char* dat)
>   {
>      msg_id = id;
>      data_cap = cap;
>      data_len = 0;
>      strncpy(data, dat, len);
>   }
> };
>
> void sendmsg(msg const& m);
>
> const uint32 MSG_ID_HELLO = 1;
>
> int main()
> {
>   unsigned char buff[256];
>   msg* m =
>       new((void*)buff) msg(MSG_ID_HELLO, 244, 13, "Hello world!");
>   sendmsg(m);
>   return 0;
> }

data_len = 0 in the constructor of msg should have been: data_len = len. 
(I initially wrote the constructor as a simple initializing one without 
the ID, data, and length args). 

On Aug 27, 1:56=A0pm, "joe" <jc1...@att.net> wrote:
> Juha Nieminen wrote:
> > gwowen <gwo...@gmail.com> wrote:
> >> On Aug 27, 11:58 am, Juha Nieminen <nos...@thanks.invalid> wrote:
> >>> Wrong? C++ does not offer tools to abstract the struct hack and make
> >>> it easier and safer to use?
>
> >>> We'll have to disagree, then.
>
> >> The struct hack gives me a storage region of run-time-variable size,
> >> contiguous with a place to store that size itself. =A0Which C++
> >> container gives you that? =A0How would you abstract that requirement.
>
> > =A0"C++ offers you tools" does not mean "C++ offers you a standard
> > container".
>
> > =A0C++ offers tools for you to build your own abstract containers which
> > internally use the struct hack, but from the outside are easier and
> > safer to use than using the raw struct hack directly.
>
> > =A0Such tools include classes (with public and private sections),
> > constructors, copy constructors, assignment operator overloading,
> > destructors, the RAII mechanism, and templates.
>
> > =A0In other words, instead of using a raw struct pointer directly (whic=
h
> > points to a memory block which has been "overallocated" for the
> > purposes
> > of the struct hack), you use an abstract object which resembles a
> > smart pointer and which hides and automatizes the low-level dirty
> > details inside, making the usage of the struct hack easier and safer.
>
> OK. I thought you were saying originally above that C++ offered an
> alternative to the struct hack (because you have been on the track that
> something with a std::vector in it was the "same thing"). The "same ol"
> still applies though: because C++ does not recognize the struct hack as a
> viable technique (like C99 does), you're still (and I do, do this)
> wrapping a hack that may be compiler-specific or perhaps not guaranteed
> to work. And also, it's hackish because in certain forms you loose the
> ability to derive from such a wrapper, etc, and is therefore a
> dialect-style of C++ programming.

I don't think you could ever derive from a struct hack struct in C++.
The data trails off past the end of the struct hack struct, and if you
derive from the struct and add new data members, then those data
members will exist in the "off the end" hack part from the base class.
This makes no sense to me offhand.

There is a way to do the equivalent of the struct hack in a fully
conforming way in C90 and C++03. It requires some slightly different
notation for use, but it has the same performance and layout. What
follows is some mostly untested code of the idea for C++03. I'm not
entirely sure it's correct, and while it does use some creative
casting hackery, I think that it is perfectly conforming code which is
guaranteed by standard to do what is intended, and it should work on
all reasonable implementations as well.

#ifndef SAFE_STRUCT_HACK_HPP_INCLUDE
#define SAFE_STRUCT_HACK_HPP_INCLUDE

#include <cstdlib>
using std::size_t; //because I'm lazy for this sample

// Public Interface

template <typename prefix_t, typename element_t>
struct safe_struct_hack_t; //Type not defined. It's an opaque type.

template <typename prefix_t, typename element_t>
inline
safe_struct_hack_t<prefix_t, element_t>*
    safe_struct_hack_allocate(
        size_t num_elements);

template <typename prefix_t, typename element_t>
inline
void
    deallocate(
        safe_struct_hack_t<prefix_t, element_t> const* );

template <typename prefix_t, typename element_t>
inline
prefix_t*
    get_prefix(
        safe_struct_hack_t<prefix_t, element_t>* );
template <typename prefix_t, typename element_t>
inline
prefix_t const*
    get_prefix(
        safe_struct_hack_t<prefix_t, element_t> const* );

template <typename prefix_t, typename element_t>
inline
size_t
    get_num_elements(
        safe_struct_hack_t<prefix_t, element_t>const * );

template <typename prefix_t, typename element_t>
inline
element_t*
    get_element(
        safe_struct_hack_t<prefix_t, element_t>* , size_t index);
template <typename prefix_t, typename element_t>
inline
element_t const*
    get_element(
        safe_struct_hack_t<prefix_t, element_t> const* , size_t
index);


//Private Implementation follows.
//Do not use or depend on implementation details!

//Overly conservative, but it should work portably
//for alignment. Right?
template <size_t x, size_t y>
struct roundup_x_to_nearest_multiple_of_y
{   enum { remainder =3D (x % y) };
    enum { result =3D (remainder =3D=3D 0) ? (x) : (x + (y - remainder)) };
};
template <typename prefix_t, typename element_t>
struct safe_struct_hack_offset_to_num_elements
{   enum { x =3D roundup_x_to_nearest_multiple_of_y<sizeof(prefix_t),
sizeof(size_t)>::result };
};
template <typename prefix_t, typename element_t>
struct safe_struct_hack_offset_to_first_element
{   enum { x =3D
        roundup_x_to_nearest_multiple_of_y<
            safe_struct_hack_offset_to_num_elements<prefix_t,
element_t>::x + sizeof(size_t),
            sizeof(element_t)
            >::result
            };
};

template <typename prefix_t, typename element_t>
inline
safe_struct_hack_t<prefix_t, element_t>*
    safe_struct_hack_allocate(
        size_t num_elements)
{   size_t const allocated_size =3D
            safe_struct_hack_offset_to_first_element<prefix_t,
element_t>::x
            + num_elements * sizeof(element_t);
    char* c =3D new char[allocated_size];
    safe_struct_hack_t<prefix_t, element_t>* const x =3D
            reinterpret_cast<safe_struct_hack_t<prefix_t,
element_t>*>(c);
    try
    {   new (c) prefix_t;
        try
        {   new (c + safe_struct_hack_offset_to_num_elements<prefix_t,
element_t>::x) size_t(num_elements);
            size_t i=3D0;
            try
            {   for ( ; i < num_elements; ++i)
                    new (get_element(x, i)) element_t();
            } catch (...)
            {   for (;;)
                {   if (i =3D=3D 0)
                        break;
                    --i;
                    get_element(x, i) -> ~element_t();
                }
                throw;
            }
        } catch (...)
        {   reinterpret_cast<prefix_t*>(c) -> ~prefix_t();
            throw;
        }
    } catch (...)
    {   delete[] c;
        throw;
    }
    return x;
}

template <typename prefix_t, typename element_t>
inline
void
    deallocate(
        safe_struct_hack_t<prefix_t, element_t> const* x)
{   reinterpret_cast<prefix_t const*>(x) -> ~prefix_t();
    size_t const num_elements =3D get_num_elements(x);
    for (size_t i=3D0; i<num_elements; ++i)
        get_element(x, i) -> ~element_t();
    delete[] reinterpret_cast<char const*>(x);
}

template <typename prefix_t, typename element_t>
inline
prefix_t*
    get_prefix(
        safe_struct_hack_t<prefix_t, element_t>* x)
{   return reinterpret_cast<prefix_t*>(x);
}
template <typename prefix_t, typename element_t>
inline
prefix_t const*
    get_prefix(
        safe_struct_hack_t<prefix_t, element_t> const* x)
{   return reinterpret_cast<prefix_t const*>(x);
}

template <typename prefix_t, typename element_t>
inline
size_t
    get_num_elements(
        safe_struct_hack_t<prefix_t, element_t>const * x)
{   char const* c =3D
            reinterpret_cast<char const*>(x)
            + safe_struct_hack_offset_to_num_elements<prefix_t,
element_t>::x;
    return * reinterpret_cast<size_t const*>(c);
}

template <typename prefix_t, typename element_t>
inline
element_t*
    get_element(
        safe_struct_hack_t<prefix_t, element_t>* x, size_t index)
{   char* ptr_to_first_ele =3D
            reinterpret_cast<char*>(x);
            + safe_struct_hack_offset_to_first_element<prefix_t,
element_t>::x;
    return reinterpret_cast<element_t*>(ptr_to_first_ele) + index;
}
template <typename prefix_t, typename element_t>
inline
element_t const*
    get_element(
        safe_struct_hack_t<prefix_t, element_t> const* x, size_t
index)
{   char const* ptr_to_first_ele =3D
            reinterpret_cast<char const*>(x);
            + safe_struct_hack_offset_to_first_element<prefix_t,
element_t>::x;
    return reinterpret_cast<element_t const*>(ptr_to_first_ele) +
index;
}

#endif

Joshua Maurice wrote:
> On Aug 27, 1:56 pm, "joe" <jc1...@att.net> wrote:
>> Juha Nieminen wrote:
>>> gwowen <gwo...@gmail.com> wrote:
>>>> On Aug 27, 11:58 am, Juha Nieminen <nos...@thanks.invalid> wrote:
>>>>> Wrong? C++ does not offer tools to abstract the struct hack and
>>>>> make it easier and safer to use?
>>
>>>>> We'll have to disagree, then.
>>
>>>> The struct hack gives me a storage region of run-time-variable
>>>> size, contiguous with a place to store that size itself. Which C++
>>>> container gives you that? How would you abstract that requirement.
>>
>>> "C++ offers you tools" does not mean "C++ offers you a standard
>>> container".
>>
>>> C++ offers tools for you to build your own abstract containers which
>>> internally use the struct hack, but from the outside are easier and
>>> safer to use than using the raw struct hack directly.
>>
>>> Such tools include classes (with public and private sections),
>>> constructors, copy constructors, assignment operator overloading,
>>> destructors, the RAII mechanism, and templates.
>>
>>> In other words, instead of using a raw struct pointer directly
>>> (which points to a memory block which has been "overallocated" for
>>> the purposes
>>> of the struct hack), you use an abstract object which resembles a
>>> smart pointer and which hides and automatizes the low-level dirty
>>> details inside, making the usage of the struct hack easier and
>>> safer.
>>
>> OK. I thought you were saying originally above that C++ offered an
>> alternative to the struct hack (because you have been on the track
>> that something with a std::vector in it was the "same thing"). The
>> "same ol" still applies though: because C++ does not recognize the
>> struct hack as a viable technique (like C99 does), you're still (and
>> I do, do this) wrapping a hack that may be compiler-specific or
>> perhaps not guaranteed to work. And also, it's hackish because in
>> certain forms you loose the ability to derive from such a wrapper,
>> etc, and is therefore a dialect-style of C++ programming.
>
> I don't think you could ever derive from a struct hack struct in C++.

That depends on what you consider the struct hack. I use an array with a 
dimension of 1 and all is fine.

> The data trails off past the end of the struct hack struct, and if you
> derive from the struct and add new data members, then those data
> members will exist in the "off the end" hack part from the base class.
> This makes no sense to me offhand.

It is assumed that a developer smart enough to know the technique of the 
struct hack will not be adding data members in derived classes! Added 
behavior, OK. Adding data members is a no no. Also assumed, since structs 
based on the struct hack are close-to-the-metal, is that virtual 
functions won't be used. These things are close to being primitives after 
all.

[I'll have to digest your template code later]

On 08/28/10 09:30 AM, joe wrote:
> Alf P. Steinbach /Usenet wrote:
>> * gwowen, on 27.08.2010 13:20:
>>> On Aug 27, 12:16 pm, Juha Nieminen<nos...@thanks.invalid>   wrote:
>>>
>>>> C++ offers tools for you to build your own abstract containers which
>>>> internally use the struct hack, but from the outside are easier and
>>>> safer to use than using the raw struct hack directly.
>>>
>>> Really?  I challenge you write one suitable for use in IPC. Remember,
>>> if your abstract container contains a pointer, its not
>>> going to work.
>>
>> Please post code that shows the raw struct hack in action without
>> using a pointer.
>
> struct msg
> {
>      uint32 msg_id;
>      uint32 data_cap; // capacity
>      uint32 data_len;  // length
>      char data[1];
>
>      msg(uint32 id, uint32 cap, uint32 len, const char* dat)
>     {
>        msg_id = id;
>        data_cap = cap;
>        data_len = 0;
>        strncpy(data, dat, len);
>     }
> };
>
> void sendmsg(msg const&  m);

The obvious design fix is to pass the header and a pointer to the data 
to sendmsg and save the unnecessary copy!

-- 
Ian Collins

Ian Collins wrote:
> On 08/28/10 09:30 AM, joe wrote:
>> Alf P. Steinbach /Usenet wrote:
>>> * gwowen, on 27.08.2010 13:20:
>>>> On Aug 27, 12:16 pm, Juha Nieminen<nos...@thanks.invalid>   wrote:
>>>>
>>>>> C++ offers tools for you to build your own abstract containers
>>>>> which internally use the struct hack, but from the outside are
>>>>> easier and safer to use than using the raw struct hack directly.
>>>>
>>>> Really?  I challenge you write one suitable for use in IPC.
>>>> Remember, if your abstract container contains a pointer, its not
>>>> going to work.
>>>
>>> Please post code that shows the raw struct hack in action without
>>> using a pointer.
>>
>> struct msg
>> {
>>      uint32 msg_id;
>>      uint32 data_cap; // capacity
>>      uint32 data_len;  // length
>>      char data[1];
>>
>>      msg(uint32 id, uint32 cap, uint32 len, const char* dat)
>>     {
>>        msg_id = id;
>>        data_cap = cap;
>>        data_len = 0;
>>        strncpy(data, dat, len);
>>     }
>> };
>>
>> void sendmsg(msg const&  m);
>
> The obvious design fix is to pass the header and a pointer to the data
> to sendmsg and save the unnecessary copy!

It was just an example of how the struct hack works, not an actual usage. 

* Ian Collins, on 28.08.2010 01:02:
> On 08/28/10 09:30 AM, joe wrote:
>> Alf P. Steinbach /Usenet wrote:
>>> * gwowen, on 27.08.2010 13:20:
>>>> On Aug 27, 12:16 pm, Juha Nieminen<nos...@thanks.invalid>   wrote:
>>>>
>>>>> C++ offers tools for you to build your own abstract containers which
>>>>> internally use the struct hack, but from the outside are easier and
>>>>> safer to use than using the raw struct hack directly.
>>>>
>>>> Really?  I challenge you write one suitable for use in IPC. Remember,
>>>> if your abstract container contains a pointer, its not
>>>> going to work.
>>>
>>> Please post code that shows the raw struct hack in action without
>>> using a pointer.
>>
>> struct msg
>> {
>>      uint32 msg_id;
>>      uint32 data_cap; // capacity
>>      uint32 data_len;  // length
>>      char data[1];
>>
>>      msg(uint32 id, uint32 cap, uint32 len, const char* dat)
>>     {
>>        msg_id = id;
>>        data_cap = cap;
>>        data_len = 0;
>>        strncpy(data, dat, len);
>>     }
>> };
>>
>> void sendmsg(msg const&  m);
>
> The obvious design fix is to pass the header and a pointer to the data to
> sendmsg and save the unnecessary copy!

Hm, somehow Joe's reply didn't show up on my server.

So, I don't know what the code above is meant to illustrate (no context).

However, my question was rhetorical: there's no way to allocate an arbitrary 
(known at run time) sized struct without using dynamic allocation.


Cheers,

- Alf

-- 
blog at <url: http://alfps.wordpress.com>

joe <jc1996@att.net> wrote:
> Alf P. Steinbach /Usenet wrote:
>> Please post code that shows the raw struct hack in action without
>> using a pointer.
> 
> int main()
> {
>   unsigned char buff[256];
>   msg* m =
>       new((void*)buff) msg(MSG_ID_HELLO, 244, 13, "Hello world!");
>   sendmsg(m);
>   return 0;
> } 

  What do you think that 'm' is if not a pointer?

  And your code perfectly exemplifies why that pointer should be abstracted
away in C++. You are leaking it.

Juha Nieminen <nospam@thanks.invalid> wrote in
news:4c792d06$0$32119$7b1e8fa0@news.nbl.fi: 

> joe <jc1996@att.net> wrote:
>> Alf P. Steinbach /Usenet wrote:
>>> Please post code that shows the raw struct hack in action without
>>> using a pointer.
>> 
>> int main()
>> {
>>   unsigned char buff[256];
>>   msg* m =
>>       new((void*)buff) msg(MSG_ID_HELLO, 244, 13, "Hello world!");
>>   sendmsg(m);
>>   return 0;
>> } 
> 
>   What do you think that 'm' is if not a pointer?
> 
>   And your code perfectly exemplifies why that pointer should be
>   abstracted 
> away in C++. You are leaking it.

Sorry, but there is no leak (cannot be as there is no dynamic memory 
allocation). And one can easily get rid of the m pointer, it is not 
needed here, one could just sendmsg(buff).

OTOH, this code exhibits UB because buff is not guaranteed to be aligned 
properly for msg. But this can be fixed.

Cheers
Paavo

On 28 aug, 19:15, Paavo Helde <myfirstn...@osa.pri.ee> wrote:
> Juha Nieminen <nos...@thanks.invalid> wrote innews:4c792d06$0$32119$7b1e8=
fa0@news.nbl.fi:
>
>
>
> > joe <jc1...@att.net> wrote:
> >> Alf P. Steinbach /Usenet wrote:
> >>> Please post code that shows the raw struct hack in action without
> >>> using a pointer.
>
> >> int main()
> >> {
> >> =A0 unsigned char buff[256];
> >> =A0 msg* m =3D
> >> =A0 =A0 =A0 new((void*)buff) msg(MSG_ID_HELLO, 244, 13, "Hello world!"=
);
> >> =A0 sendmsg(m);
> >> =A0 return 0;
> >> }
>
> > =A0 What do you think that 'm' is if not a pointer?
>
> > =A0 And your code perfectly exemplifies why that pointer should be
> > =A0 abstracted
> > away in C++. You are leaking it.
>
> Sorry, but there is no leak (cannot be as there is no dynamic memory
> allocation). And one can easily get rid of the m pointer, it is not
> needed here, one could just sendmsg(buff).
>
> OTOH, this code exhibits UB because buff is not guaranteed to be aligned
> properly for msg. But this can be fixed.

It can be fixed yes. Like they say here that the whole point of struct
hack is to have something that has run-time decided size. If it is
created into storage with compile-time constant size (256 bytes)
then ... what was the point again?

Alf P. Steinbach /Usenet wrote:
> * Ian Collins, on 28.08.2010 01:02:
>> On 08/28/10 09:30 AM, joe wrote:
>>> Alf P. Steinbach /Usenet wrote:
>>>> * gwowen, on 27.08.2010 13:20:
>>>>> On Aug 27, 12:16 pm, Juha Nieminen<nos...@thanks.invalid>   wrote:
>>>>>
>>>>>> C++ offers tools for you to build your own abstract containers
>>>>>> which internally use the struct hack, but from the outside are
>>>>>> easier and safer to use than using the raw struct hack directly.
>>>>>
>>>>> Really?  I challenge you write one suitable for use in IPC.
>>>>> Remember, if your abstract container contains a pointer, its not
>>>>> going to work.
>>>>
>>>> Please post code that shows the raw struct hack in action without
>>>> using a pointer.
>>>
>>> struct msg
>>> {
>>>      uint32 msg_id;
>>>      uint32 data_cap; // capacity
>>>      uint32 data_len;  // length
>>>      char data[1];
>>>
>>>      msg(uint32 id, uint32 cap, uint32 len, const char* dat)
>>>     {
>>>        msg_id = id;
>>>        data_cap = cap;
>>>        data_len = 0;
>>>        strncpy(data, dat, len);
>>>     }
>>> };
>>>
>>> void sendmsg(msg const&  m);
>>
>> The obvious design fix is to pass the header and a pointer to the
>> data to sendmsg and save the unnecessary copy!
>
> Hm, somehow Joe's reply didn't show up on my server.
>
> So, I don't know what the code above is meant to illustrate (no
> context).

It was in response to your solicitation (as shown above) to post code 
that shows the struct hack in action without using a pointer. At least 
that's how I groked what you were looking for.

>
> However, my question was rhetorical: there's no way to allocate an
> arbitrary (known at run time) sized struct without using dynamic
> allocation.

Well now you are talking about VLAs and that is different from "flexible 
array member". VLA implementations do the allocation implicitly so just 
saying "dynamic allocation" is not enough. There is explicit (by the 
programmer) dynamic allocation, implicit (by the compiler) dynamic 
allocation and those can be either on the stack (alloca) or on the heap.

The struct hack is, of course, not about VLA but rather "flexible array 
member". Note that if, in my example, the dimension of data is changed to 
some other number like 10, you have a fixed-size msg but it's 
structurally as much a msg still (and can be used as one after a cast) as 
the variable-length one:

struct msg10
{
   uint32 msg_id;
   uint32 data_cap;
   uint32 data_len;
   char data[10];
};

Juha Nieminen wrote:
> joe <jc1996@att.net> wrote:
>> Alf P. Steinbach /Usenet wrote:
>>> Please post code that shows the raw struct hack in action without
>>> using a pointer.
>>
>> int main()
>> {
>>   unsigned char buff[256];
>>   msg* m =
>>       new((void*)buff) msg(MSG_ID_HELLO, 244, 13, "Hello world!");
>>   sendmsg(m);
>>   return 0;
>> }
>
>  What do you think that 'm' is if not a pointer?
>
>  And your code perfectly exemplifies why that pointer should be
> abstracted away in C++. You are leaking it.

Nothing is leaking. The "new" is placement new. There are no pointers in 
the struct msg as there would be if some other C++-like composition were 
used. 

�� Tiib wrote:
> On 28 aug, 19:15, Paavo Helde <myfirstn...@osa.pri.ee> wrote:
>> Juha Nieminen <nos...@thanks.invalid> wrote
>> innews:4c792d06$0$32119$7b1e8fa0@news.nbl.fi:
>>
>>
>>
>>> joe <jc1...@att.net> wrote:
>>>> Alf P. Steinbach /Usenet wrote:
>>>>> Please post code that shows the raw struct hack in action without
>>>>> using a pointer.
>>
>>>> int main()
>>>> {
>>>> unsigned char buff[256];
>>>> msg* m =
>>>> new((void*)buff) msg(MSG_ID_HELLO, 244, 13, "Hello world!");
>>>> sendmsg(m);
>>>> return 0;
>>>> }
>>
>>> What do you think that 'm' is if not a pointer?
>>
>>> And your code perfectly exemplifies why that pointer should be
>>> abstracted
>>> away in C++. You are leaking it.
>>
>> Sorry, but there is no leak (cannot be as there is no dynamic memory
>> allocation). And one can easily get rid of the m pointer, it is not
>> needed here, one could just sendmsg(buff).
>>
>> OTOH, this code exhibits UB because buff is not guaranteed to be
>> aligned
>> properly for msg. But this can be fixed.
>
> It can be fixed yes. Like they say here that the whole point of struct
> hack is to have something that has run-time decided size.

Well I didn't say that, maybe someone else did. The point is to get 
something contiguous that can vary in length, not necessarily just at 
runtime.

> If it is
> created into storage with compile-time constant size (256 bytes)
> then ... what was the point again? 

�� Tiib <ootiib@hot.ee> wrote in
news:b841541b-82f8-448e-9b45-abd12d56e06a@q2g2000yqq.googlegroups.com: 

> On 28 aug, 19:15, Paavo Helde <myfirstn...@osa.pri.ee> wrote:
>> Juha Nieminen <nos...@thanks.invalid> wrote
>> innews:4c792d06$0$32119$7b1e8 
> fa0@news.nbl.fi:
>>
>>
>>
>> > joe <jc1...@att.net> wrote:
>> >> Alf P. Steinbach /Usenet wrote:
>> >>> Please post code that shows the raw struct hack in action without
>> >>> using a pointer.
>>
>> >> int main()
>> >> {
>> >> � unsigned char buff[256];
>> >> � msg* m 
>> >> � � � new((void*)buff) msg(MSG_ID_HELLO, 244, 13, "Hello world!"
> );
>> >> � sendmsg(m);
>> >> � return 0;
>> >> }
>>
>> > � What do you think that 'm' is if not a pointer?
>>
>> > � And your code perfectly exemplifies why that pointer should be
>> > � abstracted
>> > away in C++. You are leaking it.
>>
>> Sorry, but there is no leak (cannot be as there is no dynamic memory
>> allocation). And one can easily get rid of the m pointer, it is not
>> needed here, one could just sendmsg(buff).
>>
>> OTOH, this code exhibits UB because buff is not guaranteed to be
>> aligned properly for msg. But this can be fixed.
> 
> It can be fixed yes. Like they say here that the whole point of struct
> hack is to have something that has run-time decided size. If it is
> created into storage with compile-time constant size (256 bytes)
> then ... what was the point again?

Yes, if the maximum buffer size is known at compile time in every use 
point, then it would be easy to provide something like:

template<int max_buff_size> 
struct msg {
    	// ...
    	unsigned char buff[max_buff_size];
};

msg<256> m1(MSG_ID_HELLO, 244, 13, "Hello world!");
sendmsg(&m1, m1.actual_size());
 
This will avoid UB both from alignment and out-of-bounds errors related 
to the struct hack, while allowing for different size buffers at 
different call sites. Note that the actual serialized size may depend on 
the actual content of each message and can well be less than sizeof(msg
<256>).

Cheers
Paavo