I am wondering whether I can get gcc or as (or any other compiler or
assembler) to compile a C program in ILP32 model, but to use amd64
instructions that will work even in the 64-bit/long mode mode. The
default operand size when in long mode is 32 bits. The default address
size is 64 bits but, at least in theory, the address size can be
overridden using an address size override prefix.

To make my point clearer here the steps

Step 1  write t.c

void xcopy(char *s, char *d, int n)
{
        while (n) {
                *d++ = *s++;
                n--;
        }
        return;
}

Step 2 Compile t.c into assembly code in ILP32 mode

gcc -m32 -O -S t.c

Step 3 Modify t.s as mentioned below

        .file   "t.c"
        .text
        .p2align 4,,15
..globl xcopy
        .type   xcopy, @function
xcopy:
        .code64              <====== manually added
        pushl   %ebp
        movl    %esp, %ebp
        pushl   %ebx
        movl    8(%ebp), %ebx
        movl    12(%ebp), %ecx
        movl    16(%ebp), %edx
        testl   %edx, %edx
        je      .L6
        .p2align 4,,7
..L4:
        movzbl  (%ebx), %eax
        incl    %ebx
        movb    %al, (%ecx)
        incl    %ecx
        decl    %edx
        jne     .L4
..L6:
        popl    %ebx
        leave
        ret
        .size   xcopy, .-xcopy
        .ident  "GCC: (GNU) 3.4.6 [FreeBSD] 20060305"


Step 4 compile t.s in the 64 bit mode
[root@pi_mpx ~/pk]# gcc -m64 t.s
t.s: Assembler messages:
t.s:7: Error: suffix or operands invalid for `push'
t.s:9: Error: suffix or operands invalid for `push'
t.s:10: Error: `8(%ebp)' is not a valid 64 bit base/index expression
t.s:11: Error: `12(%ebp)' is not a valid 64 bit base/index expression
t.s:12: Error: `16(%ebp)' is not a valid 64 bit base/index expression
t.s:17: Error: `(%ebx)' is not a valid 64 bit base/index expression
t.s:19: Error: `(%ecx)' is not a valid 64 bit base/index expression
t.s:24: Error: suffix or operands invalid for `pop'

Obviously it didn't work. But if there was a way to tell gcc to select
instructions carefully such that they will be valid both in
compatibility and long mode ... then I would have had a ILP32 program
running in 64 bit mode.

Can this be made to work?

Regards,
Prakash

On Dec 4, 1:27�am, pkhemani  <spamt...@crayne.org> wrote:
> I am wondering whether I can get gcc or as (or any other compiler or
> assembler) to compile a C program in ILP32 model, but to use amd64
> instructions that will work even in the 64-bit/long mode mode. The
> default operand size when in long mode is 32 bits. The default address
> size is 64 bits but, at least in theory, the address size can be
> overridden using an address size override prefix.
>
> To make my point clearer here the steps
>
> Step 1 �write t.c
>
> void xcopy(char *s, char *d, int n)
> {
> � � � � while (n) {
> � � � � � � � � *d++ = *s++;
> � � � � � � � � n--;
> � � � � }
> � � � � return;
>
> }
>
> Step 2 Compile t.c into assembly code in ILP32 mode
>
> gcc -m32 -O -S t.c
>
> Step 3 Modify t.s as mentioned below
>
> � � � � .file � "t.c"
> � � � � .text
> � � � � .p2align 4,,15
> .globl xcopy
> � � � � .type � xcopy, @function
> xcopy:
> � � � � .code64 � � � � � � �<====== manually added
> � � � � pushl � %ebp
> � � � � movl � �%esp, %ebp
> � � � � pushl � %ebx
> � � � � movl � �8(%ebp), %ebx
> � � � � movl � �12(%ebp), %ecx
> � � � � movl � �16(%ebp), %edx
> � � � � testl � %edx, %edx
> � � � � je � � �.L6
> � � � � .p2align 4,,7
> .L4:
> � � � � movzbl �(%ebx), %eax
> � � � � incl � �%ebx
> � � � � movb � �%al, (%ecx)
> � � � � incl � �%ecx
> � � � � decl � �%edx
> � � � � jne � � .L4
> .L6:
> � � � � popl � �%ebx
> � � � � leave
> � � � � ret
> � � � � .size � xcopy, .-xcopy
> � � � � .ident �"GCC: (GNU) 3.4.6 [FreeBSD] 20060305"
>
> Step 4 compile t.s in the 64 bit mode
> [root@pi_mpx ~/pk]# gcc -m64 t.s
> t.s: Assembler messages:
> t.s:7: Error: suffix or operands invalid for `push'
> t.s:9: Error: suffix or operands invalid for `push'
> t.s:10: Error: `8(%ebp)' is not a valid 64 bit base/index expression
> t.s:11: Error: `12(%ebp)' is not a valid 64 bit base/index expression
> t.s:12: Error: `16(%ebp)' is not a valid 64 bit base/index expression
> t.s:17: Error: `(%ebx)' is not a valid 64 bit base/index expression
> t.s:19: Error: `(%ecx)' is not a valid 64 bit base/index expression
> t.s:24: Error: suffix or operands invalid for `pop'
>
> Obviously it didn't work. But if there was a way to tell gcc to select
> instructions carefully such that they will be valid both in
> compatibility and long mode ... then I would have had a ILP32 program
> running in 64 bit mode.
>
> Can this be made to work?
>
> Regards,
> Prakash

Can you elaborate on what exactly you're trying to achieve?

A few points...

0. In 64-bit assembler you can't do "movb %al, (%ecx)" since there's
no such addressing mode, at least formally. It has to be "movb %al,
(%rcx)" with the appropriate value of rcx. This can be prefixed with
the address override prefix, though.
1. Some instructions in 64-bit mode have 64-bit default operand (e.g.
stack-related instructions) and there's no way to override them to 32
bits (16 possible, though). See the AMD documentation for details.
You'll need to do something about the PUSH/POP instructions (replace
them with something that functions as PUSH/POP with 32-bit operands).
2. You can use 32-bit pointers in 64-bit mode if and only if all the
code and data (including stack) of the application is below 4G virtual
address (you just truncate/ignore the top 32 bits of the address) OR
is within the same 4G (that is, the top 32 bits of all addresses are
the same; here you modify the code in such a way that these top 32
bits are a known constant part of all pointers and the low 32 bits are
variable, kind of like the default segments in a .COM program allowing
it to access 64K just like that).
3. Some instructions are unavailable in 64-bit code. See the docs.
4. One address encoding isn't available in 64-bit mode (due to RIP-
relative addressing).
5. Normally, the same binary can't run in 32-bit and 64-bit
environments and produce the same results. You need to restrict the
instruction set used and replace some instructions by others to be
able to do that in some way.
6. Expect issues with the code, tools and operating environment.

Alex

In comp.lang.asm.x86 pkhemani <spamtrap@crayne.org> wrote:
> I am wondering whether I can get gcc or as (or any other compiler or
> assembler) to compile a C program in ILP32 model, but to use amd64
> instructions that will work even in the 64-bit/long mode mode. The
> default operand size when in long mode is 32 bits. The default address
> size is 64 bits but, at least in theory, the address size can be
> overridden using an address size override prefix.
> 
> To make my point clearer here the steps
> 
> Step 1  write t.c
> 
> Step 2 Compile t.c into assembly code in ILP32 mode
> 
> gcc -m32 -O -S t.c
> 
> Step 3 Modify t.s as mentioned below
> 
> Step 4 compile t.s in the 64 bit mode

> t.s:7: Error: suffix or operands invalid for `push'
> 
> Obviously it didn't work. But if there was a way to tell gcc to select
> instructions carefully such that they will be valid both in
> compatibility and long mode ... then I would have had a ILP32 program
> running in 64 bit mode.
> 
> Can this be made to work?

It is not clear what you want to do.  Do you want to have the
same binary working both in 32 and 64 bit mode?  Then you
will have problems with system calls and program header (they
differ between 32 and 64-bit mode).  If you want just generate
program which uses 32-bit model and tolerate small amount of
64-bit code than you can write apropriate asm by had.  Note
that you need to replace pop/push by mov instruction.  If
you insist on 32-bit return adresses you have to replace
call/ret by jumps.  So it is technically possible.

If you want re-use gcc to generate such code than I think
you need to port gcc to new model -- AFAIK such thing was
done for MIPS (n32 model) but there are no plans to do it
for AMD64.  Simply slapping .code 64 before 32-bit instructions
is not going to work and changes look to deep to do them
by postprocessing assembly output.

Also to make you code usable you probably need to port C
library and possibly some other libraries -- looks like
substantial work. 

Coming back to your question, it is not clear why do you
want ILP32 code running in 64-bit mode.  Normally you
can run both 32-bit and 64-bit code on the same processor,
even in single address space.  If you want to call
32-bit code from 64-bit or vice versa you will have bigger
problems than mode switching -- you need a set of routines
to convert parameters.  If you think that ILP32 code
in 64-bit mode will give you better performance than you
probably want compiler port to use extra registers.
Lacking compiler port it is probably easier to re-write
program to use 32-bit data types, compile it in 64-bit
mode make sure all data is in first 4G of memory.

If you want ILP 32 for compatibility reasons but do not
care much about performance than you may try adapt Qemu
so that Qemu runs in 64-bit mode and executes 32-bit
binary of your program.  Alternatively you may adapt
some simple C compiler (porting non-optimizing compiler
is easier than optimizing one like gcc).

-- 
                              Waldek Hebisch
hebisch@math.uni.wroc.pl 

Thank you all for your replies. Here is some background.

I have ILP32 code that doesn't use the C library and doesn't make any
system calls.

Parts of this code needs larger address space. It is difficult to port
the entire program to 64 bits. But we can port parts of it to 64 bit,
massage the x86 32-bit ABI into amd64 ABI, and make far-jumps between
32 bit and 64 bit code.

The issue is that far-jumps are expensive - 20 times or more
expensive. (I am not too sure whether all far jumps are expensive or
only the ones that switch between compatibility and long mode are
expensive.) SYSCALL/SYSRET don't work in 32 bit mode on Intel CPUs
(they do in AMD). I am currently experimenting with SYSENTER/SYSEXIT
but from what I have read it will not perform as well. (SYSENTER will
also change the privelege level ... but if it is fast enough then I
can work around that).

So this is where my question comes in. What if I could always stay in
long mode, execute my ILP32 code in long mode and make near-jumps to
64-bit code when I have to?

>From what I have read so far, theoretically it is possible but there
isn't much incentive for the tools to support such a thing.

Thanks,
Prakash




On Dec 3, 5:15�pm, "Alexei A. Frounze"  <spamt...@crayne.org> wrote:
> On Dec 4, 1:27�am, pkhemani �<spamt...@crayne.org> wrote:
>
>
>
> > I am wondering whether I can get gcc or as (or any other compiler or
> > assembler) to compile a C program in ILP32 model, but to use amd64
> > instructions that will work even in the 64-bit/long mode mode. The
> > default operand size when in long mode is 32 bits. The default address
> > size is 64 bits but, at least in theory, the address size can be
> > overridden using an address size override prefix.
>
> > To make my point clearer here the steps
>
> > Step 1 �write t.c
>
> > void xcopy(char *s, char *d, int n)
> > {
> > � � � � while (n) {
> > � � � � � � � � *d++ = *s++;
> > � � � � � � � � n--;
> > � � � � }
> > � � � � return;
>
> > }
>
> > Step 2 Compile t.c into assembly code in ILP32 mode
>
> > gcc -m32 -O -S t.c
>
> > Step 3 Modify t.s as mentioned below
>
> > � � � � .file � "t.c"
> > � � � � .text
> > � � � � .p2align 4,,15
> > .globl xcopy
> > � � � � .type � xcopy, @function
> > xcopy:
> > � � � � .code64 � � � � � � �<====== manually added
> > � � � � pushl � %ebp
> > � � � � movl � �%esp, %ebp
> > � � � � pushl � %ebx
> > � � � � movl � �8(%ebp), %ebx
> > � � � � movl � �12(%ebp), %ecx
> > � � � � movl � �16(%ebp), %edx
> > � � � � testl � %edx, %edx
> > � � � � je � � �.L6
> > � � � � .p2align 4,,7
> > .L4:
> > � � � � movzbl �(%ebx), %eax
> > � � � � incl � �%ebx
> > � � � � movb � �%al, (%ecx)
> > � � � � incl � �%ecx
> > � � � � decl � �%edx
> > � � � � jne � � .L4
> > .L6:
> > � � � � popl � �%ebx
> > � � � � leave
> > � � � � ret
> > � � � � .size � xcopy, .-xcopy
> > � � � � .ident �"GCC: (GNU) 3.4.6 [FreeBSD] 20060305"
>
> > Step 4 compile t.s in the 64 bit mode
> > [root@pi_mpx ~/pk]# gcc -m64 t.s
> > t.s: Assembler messages:
> > t.s:7: Error: suffix or operands invalid for `push'
> > t.s:9: Error: suffix or operands invalid for `push'
> > t.s:10: Error: `8(%ebp)' is not a valid 64 bit base/index expression
> > t.s:11: Error: `12(%ebp)' is not a valid 64 bit base/index expression
> > t.s:12: Error: `16(%ebp)' is not a valid 64 bit base/index expression
> > t.s:17: Error: `(%ebx)' is not a valid 64 bit base/index expression
> > t.s:19: Error: `(%ecx)' is not a valid 64 bit base/index expression
> > t.s:24: Error: suffix or operands invalid for `pop'
>
> > Obviously it didn't work. But if there was a way to tell gcc to select
> > instructions carefully such that they will be valid both in
> > compatibility and long mode ... then I would have had a ILP32 program
> > running in 64 bit mode.
>
> > Can this be made to work?
>
> > Regards,
> > Prakash
>
> Can you elaborate on what exactly you're trying to achieve?
>
> A few points...
>
> 0. In 64-bit assembler you can't do "movb %al, (%ecx)" since there's
> no such addressing mode, at least formally. It has to be "movb %al,
> (%rcx)" with the appropriate value of rcx. This can be prefixed with
> the address override prefix, though.
> 1. Some instructions in 64-bit mode have 64-bit default operand (e.g.
> stack-related instructions) and there's no way to override them to 32
> bits (16 possible, though). See the AMD documentation for details.
> You'll need to do something about the PUSH/POP instructions (replace
> them with something that functions as PUSH/POP with 32-bit operands).
> 2. You can use 32-bit pointers in 64-bit mode if and only if all the
> code and data (including stack) of the application is below 4G virtual
> address (you just truncate/ignore the top 32 bits of the address) OR
> is within the same 4G (that is, the top 32 bits of all addresses are
> the same; here you modify the code in such a way that these top 32
> bits are a known constant part of all pointers and the low 32 bits are
> variable, kind of like the default segments in a .COM program allowing
> it to access 64K just like that).
> 3. Some instructions are unavailable in 64-bit code. See the docs.
> 4. One address encoding isn't available in 64-bit mode (due to RIP-
> relative addressing).
> 5. Normally, the same binary can't run in 32-bit and 64-bit
> environments and produce the same results. You need to restrict the
> instruction set used and replace some instructions by others to be
> able to do that in some way.
> 6. Expect issues with the code, tools and operating environment.
>
> Alex

Alexei A. Frounze wrote:
> 0. In 64-bit assembler you can't do "movb %al, (%ecx)" since there's
> no such addressing mode, at least formally. It has to be "movb %al,
> (%rcx)" with the appropriate value of rcx. This can be prefixed with
> the address override prefix, though.

It's perfectly formal, but gas doesn't understand it.  It's a gas bug.
gas only recently learned to deal with address size prefixes for 16 and
32 bit mode, so it's not all that surprising.

> 4. One address encoding isn't available in 64-bit mode (due to RIP-
> relative addressing).

Shouldn't matter, though, since there is an equivalent encoding.

> 5. Normally, the same binary can't run in 32-bit and 64-bit
> environments and produce the same results. You need to restrict the
> instruction set used and replace some instructions by others to be
> able to do that in some way.

If you want to run a 32-bit binary directly, just run it in
compatibility mode.  If what you need is a polyglot *binary* (one that
can be run in either mode), the easiest way is probably to simply detect
your mode and then jump to either 32- or 64-bit code.

Here is a stub which will run in either mode and do just that:

     1                                          bits 32
     2
     3 00000000 31C0                            xor eax,eax
     4 00000002 40                              inc eax
     5 00000003 90                              nop
     6 00000004 0F8401000000                    jz near code64
     7
     8                                  code32:
	[...]
    11                                  code64:

In 64-bit mode, this turns into:

00000000  31C0              xor eax,eax
00000002  4090              nop
00000004  0F8401000000      jz dword 0xb

	-hpa

pkhemani wrote:
> Parts of this code needs larger address space. It is difficult to port
> the entire program to 64 bits. But we can port parts of it to 64 bit,
> massage the x86 32-bit ABI into amd64 ABI, and make far-jumps between
> 32 bit and 64 bit code.
> 
> The issue is that far-jumps are expensive - 20 times or more
> expensive. 

I don't think the concept of a far jump exists anymore in 64-bit mode. 
That is if the concept of a far jump is a "segment:offset" type of jump. 
All jumps in 64-bit mode should be near. It ignores the segment 
overrides in 64-bit mode.

	Yousuf Khan

On Fri, 05 Dec 2008 11:37:36 -0500
Yousuf Khan  <spamtrap@crayne.org> wrote:

> I don't think the concept of a far jump exists anymore in 64-bit
> mode. That is if the concept of a far jump is a "segment:offset" type
> of jump. 

Far jumps in both 32 and 64 bit protected modes are "selector:offset".
Segment override prefixes are not involved.

-- 
Chuck 
http://www.pacificsites.com/~ccrayne/charles.html

Yousuf Khan wrote:
> pkhemani wrote:
>> Parts of this code needs larger address space. It is difficult to port
>> the entire program to 64 bits. But we can port parts of it to 64 bit,
>> massage the x86 32-bit ABI into amd64 ABI, and make far-jumps between
>> 32 bit and 64 bit code.
>>
>> The issue is that far-jumps are expensive - 20 times or more
>> expensive. 
> 
> I don't think the concept of a far jump exists anymore in 64-bit mode.
> That is if the concept of a far jump is a "segment:offset" type of jump.
> All jumps in 64-bit mode should be near. It ignores the segment
> overrides in 64-bit mode.
> 

No, far jumps still exist in 64-bit mode.  Code segment transfers still
matter in 64-bit mode, since they are the mechanism to switch between
kernel and user mode as well as between 64-bit mode and compatibility mode.

	-hpa

On Dec 3 2008, 4:27 pm, pkhemani  <spamt...@crayne.org> wrote:
> I am wondering whether I can get gcc or as (or any other compiler or
> assembler) to compile a C program in ILP32 model, but to use amd64
> instructions that will work even in the 64-bit/long mode mode. The
> default operand size when in long mode is 32 bits. The default address
> size is 64 bits but, at least in theory, the address size can be
> overridden using an address size override prefix.

I think the responders have forgotten that the larger address
space is not the only benefit of 64-bit mode.
If one hasn't more than 4GB of memory, the larger address space is a
detriment:
Pointers take twice as much memory as needed.
If one uses a lot of pointers, that could be too much.
It would be nice to have 32-bit pointers and keep the other benefits.

> To make my point clearer here the steps
>
> Step 1  write t.c
>
> void xcopy(char *s, char *d, int n)
> {
>         while (n) {
>                 *d++ = *s++;
>                 n--;
>         }
>         return;
>
> }

In C++:

templace<class Target> class ShortPtr {
public:
    typedef Target *Ptr;  // 64 bits

    ShortPtr(Ptr p)
        : datum(static_cast<unsigned>(reinterpret_cast<unsigned long>
(p))) {}
    operator Ptr() const
            { return reinterpret_cast<Ptr>(static_cast<unsigned long>
(datum)); }
private:
    unsigned datum;  // 32 bits
} // ShortPtr

// This probably wreaks hell with optimization.
// 'Twould be nice if the compiler did it natively.

// Did I get my casts right?

void xcopy(ShortPtr<char>s, ShortPtr<char>d, int n)
{
// as before
....
} // xcopy

Can an x-64 do a 32- to 64-bit load?
A 64- to 32-bit store?
IIRC gcc has an attribute or compiler option
that affects the size of long doubles.
Is there a reason something similar couldn't or
shouldn't be done with pointers on 64-bit machines?

On Thu, 1 Jan 2009 18:15:01 -0800 (PST)
skeeve  <spamtrap@crayne.org> wrote:

> Is there a reason something similar couldn't or
> shouldn't be done with pointers on 64-bit machines?

The problem with attempting to use 32-bit pointers in 64-bit mode is
that the OS is likely to load parts of your program and/or data at
virtual addresses which cannot be accessed with such pointers.

The solution is to use a small number of 64-bit base registers along
with 32-bit displacements, e.g. [RBP+32] to address a stack variable.
New in 64-bit mode is the ability to address data with a 32-bit offset
from RIP.

-- 
Chuck 
http://www.pacificsites.com/~ccrayne/charles.html

Charles Crayne wrote:
> 
> The problem with attempting to use 32-bit pointers in 64-bit mode is
> that the OS is likely to load parts of your program and/or data at
> virtual addresses which cannot be accessed with such pointers.
> 

That is usually at least partially within the control of the program.
You're still introducing a whole new ABI, which all your compilers,
libraries, etc. have to support.  Nothing really all that different from
the way 32 vs 64-bit binaries are already handled, but it has to be done
right.

	-hpa