f



Optimizing structure memory allocation

How is the memory allocated for structures? I need to optimize the
memory usage and bit fields are not doing the trick.

Any details about the memory allocation for the structures would be a
great help.

PS - I already have asked this in gcc group. Please refrain from
directing me towards other groups.
0
rahulsinner (161)
5/27/2008 12:25:47 PM
comp.lang.c 30438 articles. 2 followers. spinoza1111 (3246) is leader. Post Follow

14 Replies
296 Views

Similar Articles

[PageSpeed] 44

rahul said:

> How is the memory allocated for structures?

"A structure type describes a sequentially allocated set of member 
objects", says the Standard.

Later, it adds: "If the objects pointed to are members of the same 
aggregate object, pointers to structure members declared later compare 
higher than pointers to members declared earlier in the structure".

Finally, "There may also be unnamed padding at the end of a structure or 
union, as necessary to achieve the appropriate alignment were the 
structure or union to be a member of an array."

<snip>

> PS - I already have asked this in gcc group. Please refrain from
> directing me towards other groups.

Gladly, if you are happy to accept that the answers you get here will be 
related to what the language guarantees, rather than which particular 
choice an implementation might make where the language offers such a 
choice.

-- 
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999
0
rjh (10790)
5/27/2008 12:44:42 PM
In article <859569b1-e7e0-4711-969e-8960bc5df728@i36g2000prf.googlegroups.com>,
rahul  <rahulsinner@gmail.com> wrote:

>How is the memory allocated for structures? I need to optimize the
>memory usage and bit fields are not doing the trick.
>
>Any details about the memory allocation for the structures would be a
>great help.

You'll probably get better help if you ask a more specific question.
Show us what you want to put in the structure.

The members of a struct are stored in the order you specify.  Some
members may have types that require a particular alignment, typically
equal to their size.  For example, while chars can go anywhere you may
find that if ints are 4 bytes long then they are always placed on
4-byte boundaries.  This is common even if the processor allows
arbitrary alignment, because it's usually faster to access
suitably-aligned data.

Suppose shorts are 2 bytes and ints are 4, with corresponding
alignment requirements, and you want to store 2 chars, a short, and an
int in your struct.  These could fit in 8 bytes, and will if you order
them correctly, for example

  struct foo {
    int a;
    short b;
    char c, d;
  };

but if you do

  struct foo2 {
    char c;
    int a;
    char d;
    short b;
  };

it will take 12 bytes, because the 3 bytes after c and the byte after d
are wasted.

If you use bitfields, then you have to consider similar issues at the
bit level.  Adjacent bitfields that fit within a "unit" (probably an
int) will be packed together, so don't spread them out amongst other
members of the struct, and try to order them so that they don't go
over too many int boundaries.

-- Richard
-- 
In the selection of the two characters immediately succeeding the numeral 9,
consideration shall be given to their replacement by the graphics 10 and 11 to
facilitate the adoption of the code in the sterling monetary area. (X3.4-1963)
0
richard91 (3692)
5/27/2008 12:53:47 PM
rahul wrote:
> 
> How is the memory allocated for structures? I need to optimize
> the memory usage and bit fields are not doing the trick.
> 
> Any details about the memory allocation for the structures would
> be a great help.
> 
> PS - I already have asked this in gcc group. Please refrain from
> directing me towards other groups.

If you are using gcc, then a gcc group is appropriate.  This one is
not.  A compiler system can use any method of memory allocation it
pleases, so long as it works.

-- 
 [mail]: Chuck F (cbfalconer at maineline dot net) 
 [page]: <http://cbfalconer.home.att.net>
            Try the download section.

** Posted from http://www.teranews.com **
0
cbfalconer (19194)
5/27/2008 6:41:32 PM
On 27 May 2008 at 12:25, rahul wrote:
> How is the memory allocated for structures? I need to optimize the
> memory usage and bit fields are not doing the trick.
>
> PS - I already have asked this in gcc group. Please refrain from
> directing me towards other groups.

You might want to look at gcc's packed attribute, which "specifies that
a variable or structure field should have the smallest possible
alignment - one byte for a variable, and one bit for a field, unless you
specify a larger value with the aligned attribute."

You can either pack specific fields, e.g.

struct foo {
  char a;
  int b __attribute__((packed));
};

or whole structs at once, e.g.

struct bar {
  int a;
  char b;
  int c;
  char d;
  int e;
} __attribute__((packed));

0
nospam59 (11087)
5/27/2008 7:18:08 PM
Antoninus Twink <nospam@nospam.invalid> writes:
> On 27 May 2008 at 12:25, rahul wrote:
>> How is the memory allocated for structures? I need to optimize the
>> memory usage and bit fields are not doing the trick.
>>
>> PS - I already have asked this in gcc group. Please refrain from
>> directing me towards other groups.
>
> You might want to look at gcc's packed attribute,
[...]

which is, of course, off-topic in comp.lang.c.

rahul, you asked us not to direct you to other groups.  Why?
Questions about gcc-specific features are appropriate in gnu.gcc.help;
they are not appropriate in comp.lang.c.

Or you can consult the extensive documentation that comes with gcc.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
0
kst-u (21963)
5/27/2008 7:33:34 PM
In article <g1h08r$1c9f$1@pc-news.cogsci.ed.ac.uk>,
Richard Tobin <richard@cogsci.ed.ac.uk> wrote:
>In article <859569b1-e7e0-4711-969e-8960bc5df728@i36g2000prf.googlegroups.com>,
>rahul  <rahulsinner@gmail.com> wrote:

>>How is the memory allocated for structures? I need to optimize the
>>memory usage and bit fields are not doing the trick.

>The members of a struct are stored in the order you specify.

Expanding slightly on Richard's answer for emphasis:

C *requires* that structure members be placed in the order given,
and in increasing address order in memory. Storing the members in
the order coded is not just a matter of convention: it is part
of the language definition. As is the fact that alignment restrictions
exist and are adhered to by the compiler by adding unnamed padding
between structure members if needed (or advised) by the target machine.
-- 
  "All is vanity."                                   -- Ecclesiastes
0
roberson2 (8606)
5/27/2008 7:44:51 PM
roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson) writes:
> In article <g1h08r$1c9f$1@pc-news.cogsci.ed.ac.uk>,
> Richard Tobin <richard@cogsci.ed.ac.uk> wrote:
>>In article <859569b1-e7e0-4711-969e-8960bc5df728@i36g2000prf.googlegroups.com>,
>>rahul  <rahulsinner@gmail.com> wrote:
>
>>>How is the memory allocated for structures? I need to optimize the
>>>memory usage and bit fields are not doing the trick.
>
>>The members of a struct are stored in the order you specify.
>
> Expanding slightly on Richard's answer for emphasis:
>
> C *requires* that structure members be placed in the order given,
> and in increasing address order in memory. Storing the members in
> the order coded is not just a matter of convention: it is part
> of the language definition. As is the fact that alignment restrictions
> exist and are adhered to by the compiler by adding unnamed padding
> between structure members if needed (or advised) by the target machine.

Right, but it's perfectly legal for the alignment restriction to be
"any alignment is ok".  It's also legal (but silly) for the compiler
to insert padding whether it's required for alignment or not.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
0
kst-u (21963)
5/27/2008 8:02:33 PM
On Tue, 27 May 2008 13:02:33 -0700, Keith Thompson wrote:
> Right, but it's perfectly legal for the alignment restriction to be "any
> alignment is ok".  It's also legal (but silly) for the compiler to
> insert padding whether it's required for alignment or not.

It's not silly if older systems required a particular alignment, newer 
systems don't, but the compiler still inserts padding to maintain binary 
compatibility with those older systems. It's good that implementors are 
given a lot of freedom in choosing what works best for their systems.
0
truedfx (1926)
5/27/2008 8:35:12 PM
Keith Thompson wrote:
> roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson) writes:
>> 
>> C *requires* that structure members be placed in the order given,
>> and in increasing address order in memory. Storing the members in
>> the order coded is not just a matter of convention: it is part
>> of the language definition. As is the fact that alignment restrictions
>> exist and are adhered to by the compiler by adding unnamed padding
>> between structure members if needed (or advised) by the target machine.
> 
> Right, but it's perfectly legal for the alignment restriction to be
> "any alignment is ok".  It's also legal (but silly) for the compiler
> to insert padding whether it's required for alignment or not.

     The ambiguity lies in "requirement."  On one popular
platform, for example, a `double' can be accessed at any
address divisible by four, but can be accessed more quickly
if the address is also divisible by eight.  Should the
alignment "requirement" be taken as four or as eight?

-- 
Eric.Sosman@sun.com
0
Eric.Sosman (4552)
5/27/2008 9:03:59 PM
Eric Sosman <Eric.Sosman@sun.com> writes:
> Keith Thompson wrote:
>> roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson) writes:
>>> C *requires* that structure members be placed in the order given,
>>> and in increasing address order in memory. Storing the members in
>>> the order coded is not just a matter of convention: it is part
>>> of the language definition. As is the fact that alignment restrictions
>>> exist and are adhered to by the compiler by adding unnamed padding
>>> between structure members if needed (or advised) by the target machine.
>> Right, but it's perfectly legal for the alignment restriction to be
>> "any alignment is ok".  It's also legal (but silly) for the compiler
>> to insert padding whether it's required for alignment or not.
>
>     The ambiguity lies in "requirement."  On one popular
> platform, for example, a `double' can be accessed at any
> address divisible by four, but can be accessed more quickly
> if the address is also divisible by eight.  Should the
> alignment "requirement" be taken as four or as eight?

Could be either.  My point is that the compiler is allowed to insert,
say, 16 bytes of padding, even if it doesn't help performance at all.

It's just an example of the kind of assumption that shouldn't be made
in portable code.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
0
kst-u (21963)
5/27/2008 9:14:36 PM
In article <ln8wxvqwjn.fsf@nuthaus.mib.org>,
Keith Thompson  <kst-u@mib.org> wrote:

>Could be either.  My point is that the compiler is allowed to insert,
>say, 16 bytes of padding, even if it doesn't help performance at all.
>
>It's just an example of the kind of assumption that shouldn't be made
>in portable code.

I disagree (though obviously only up to a point).  It's perfectly
legal for a compiler to insert pointless alignment, but it's also
reasonable to assume that it won't.  For example, given any sensible
alignment scheme you will get good packing by ordering members by
decreasing size (int then short then char for example).  A portable
program can quite reasonably order structure members on this
assumption, and if it turns out to be false then the blame should be
assigned to the implementation, not the programmer.

We aren't obliged to ignore common sense in our programs, even if
conforming compilers can defeat it.

-- Richard
-- 
In the selection of the two characters immediately succeeding the numeral 9,
consideration shall be given to their replacement by the graphics 10 and 11 to
facilitate the adoption of the code in the sterling monetary area. (X3.4-1963)
0
richard91 (3692)
5/27/2008 9:30:18 PM
Keith Thompson wrote:
> Eric Sosman <Eric.Sosman@sun.com> writes:
>> Keith Thompson wrote:
>>> [...] It's also legal (but silly) for the compiler
>>> to insert padding whether it's required for alignment or not.
>>     The ambiguity lies in "requirement."  On one popular
>> platform, for example, a `double' can be accessed at any
>> address divisible by four, but can be accessed more quickly
>> if the address is also divisible by eight.  Should the
>> alignment "requirement" be taken as four or as eight?
> 
> Could be either.  My point is that the compiler is allowed to insert,
> say, 16 bytes of padding, even if it doesn't help performance at all.

     ... and my point was about the word "silly:" Until you
can nail down "requirement" you can't assess the silliness
of the padding scheme.

-- 
Eric Sosman
esosman@ieee-dot-org.invalid
0
esosman2 (3096)
5/28/2008 12:22:59 PM
Eric Sosman <esosman@ieee-dot-org.invalid> writes:
> Keith Thompson wrote:
>> Eric Sosman <Eric.Sosman@sun.com> writes:
>>> Keith Thompson wrote:
>>>> [...] It's also legal (but silly) for the compiler
>>>> to insert padding whether it's required for alignment or not.
>>>     The ambiguity lies in "requirement."  On one popular
>>> platform, for example, a `double' can be accessed at any
>>> address divisible by four, but can be accessed more quickly
>>> if the address is also divisible by eight.  Should the
>>> alignment "requirement" be taken as four or as eight?
>> Could be either.  My point is that the compiler is allowed to insert,
>> say, 16 bytes of padding, even if it doesn't help performance at all.
>
>     ... and my point was about the word "silly:" Until you
> can nail down "requirement" you can't assess the silliness
> of the padding scheme.

Ok, good point.

To re-state my point more precisely, a compiler is allowed to insert
padding between struct members even if doing so has no benefit
whatsoever.  (I don't know of any compilers that actually do so.)

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
0
kst-u (21963)
5/28/2008 4:13:48 PM
CBFalconer <cbfalconer@yahoo.com> writes:
> rahul wrote:
>> How is the memory allocated for structures? I need to optimize
>> the memory usage and bit fields are not doing the trick.
>> 
>> Any details about the memory allocation for the structures would
>> be a great help.
>> 
>> PS - I already have asked this in gcc group. Please refrain from
>> directing me towards other groups.
>
> If you are using gcc, then a gcc group is appropriate.  This one is
> not.  A compiler system can use any method of memory allocation it
> pleases, so long as it works.

But the C standard imposes a number of requirements on how memory is
allocated for structures (as we already discussed in this thread).

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"
0
kst-u (21963)
5/30/2008 2:11:19 AM
Reply: