What is the largest page size
supported by Solaris on SPARC?

On my system (20 CPU Ultra-3)
with 20 GB memory installed,
getpagesizes() returns 8 kB,
64 kB, 512 kB and 4 MB as
supported pages sizes. Is
4 MB really the limit, or
can the system be set up
for larger pages sizes?
And if so, how?

Any pointers and/or advice
would be greatly appreciated.



dk 

Dan Koren <dankoren@yahoo.com> wrote:
> What is the largest page size
> supported by Solaris on SPARC?

> On my system (20 CPU Ultra-3)
> with 20 GB memory installed,
> getpagesizes() returns 8 kB,
> 64 kB, 512 kB and 4 MB as
> supported pages sizes. Is
> 4 MB really the limit, or
> can the system be set up
> for larger pages sizes?
> And if so, how?

According to "Solaris Internals", it's basically related to the hardware
support from the MMU.  That book is old and only goes through the US-II,
but mentions that 4M is in fact the largest page supported by the MMU on
a sun4u architecture.

Do you think that having a larger page size available would give you a
significant performance boost?  Are you seeing a lot of TLB misses or
something?

-- 
Darren Dunham                                           ddunham@taos.com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

"Darren Dunham" <ddunham@redwood.taos.com> wrote in message 
news:%_U4e.19364$zl.9462@newssvr13.news.prodigy.com...
> Dan Koren <dankoren@yahoo.com> wrote:
>> What is the largest page size
>> supported by Solaris on SPARC?
>
>> On my system (20 CPU Ultra-3)
>> with 20 GB memory installed,
>> getpagesizes() returns 8 kB,
>> 64 kB, 512 kB and 4 MB as
>> supported pages sizes. Is
>> 4 MB really the limit, or
>> can the system be set up
>> for larger pages sizes?
>> And if so, how?
>
> According to "Solaris Internals", it's basically related to the hardware
> support from the MMU.  That book is old and only goes through the US-II,
> but mentions that 4M is in fact the largest page supported by the MMU on
> a sun4u architecture.
>
> Do you think that having a larger page size available would give you a
> significant performance boost?  Are you seeing a lot of TLB misses or
> something?
>


Thanks for replying.

We need to allocate a very large chunk of
shared memory (say from 4 GB to 64 GB) and
we'd like it to use the fewest TLB entries
possible -- preferably one!



dk 

Certainly the studio 10 cc man page suggest that larger page sizes
may be used...

 >   -xpagesize=n
 >         (SPARC) Set the preferred page size for the stack and
 >         the heap.
 >
 >         The n value must be one of the following:
 >          8K 64K 512K 4M 32M 256M 2G 16G or default.
 >
 >         You must specify a valid page size for the Solaris
 >         operating environment on the target platform, as
 >         returned by getpagesize(3C).  If you do not specify a
 >         valid pagesize, the request is silently ignored at
 >         run-time. The Solaris operating system offers no
 >         guarantee that the page size request will be honored.
 >
 >         You can use pmap(1) or meminfo(2) to determine page
 >         size of the target platform.
 >
 >         The -xpagesize option has no effect unless you use it
 >         at compile time and at link time. For a complete list
 >         of compiler options that must be specified at both com-
 >         pile time and at link time, see the C User's Guide.>
 >         Note that this feature is not available on the Solaris
 >         8 operating system. A program compiled with this option
 >         will not link on the Solaris 8 operating system.
 >
 >         If you specify -xpagesize=default, the operating
 >         environment sets the page size.

But you might want to read the following "blueprint" help you
determine 1) whether they are available on your processor, and
2) whether they will improve the performance of your app/system.

	http://www.sun.com/blueprints/0204/817-5489.pdf

Bigger isn't always better.

- Bob



Dan Koren wrote:
> What is the largest page size
> supported by Solaris on SPARC?
> 
> On my system (20 CPU Ultra-3)
> with 20 GB memory installed,
> getpagesizes() returns 8 kB,
> 64 kB, 512 kB and 4 MB as
> supported pages sizes. Is
> 4 MB really the limit, or
> can the system be set up
> for larger pages sizes?
> And if so, how?
> 
> Any pointers and/or advice
> would be greatly appreciated.
> 
> dk 
> 
> 

Darren Dunham wrote:
> 
> Dan Koren <dankoren@yahoo.com> wrote:
> > What is the largest page size
> > supported by Solaris on SPARC?
> 
> > On my system (20 CPU Ultra-3)
> > with 20 GB memory installed,
> > getpagesizes() returns 8 kB,
> > 64 kB, 512 kB and 4 MB as
> > supported pages sizes. Is
> > 4 MB really the limit, or
> > can the system be set up
> > for larger pages sizes?
> > And if so, how?
> 
> According to "Solaris Internals", it's basically related to the hardware
> support from the MMU.  That book is old and only goes through the US-II,
> but mentions that 4M is in fact the largest page supported by the MMU on
> a sun4u architecture.

AFAIK the UltraSPARC supports all page sizes from 8K up to 16GB in 8x
steps, e.g. 8K, 64K, 512K, 4M, 32M, 256M and 16GB.

----

Bye,
Roland

-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 7950090
 (;O/ \/ \O;)

Thanks for the pointers. I am familiar
with the tools and the "blueprint" you
mentioned.

Note however that my question was about
setting page sizes for shared memory
segments -- not heaps or stacks.

In this case, however, there can be no
question that bigger is better. The
system I am trying to bring up is one
dedicated to a single application which
is essentially a huge cache. It needs to
allocate as much memory as possible at
(or shortly after) boot time, lock it
in core, and use it with the fewest
possible TLB and cache misses. Just
as simple as that.



dk


"Robert Lawhead" <news0000.5.unixguru@spamgourmet.com> wrote in message 
news:OoKdnUJBeeMDq8nfRVn-pQ@speakeasy.net...
> Certainly the studio 10 cc man page suggest that larger page sizes
> may be used...
>
> >   -xpagesize=n
> >         (SPARC) Set the preferred page size for the stack and
> >         the heap.
> >
> >         The n value must be one of the following:
> >          8K 64K 512K 4M 32M 256M 2G 16G or default.
> >
> >         You must specify a valid page size for the Solaris
> >         operating environment on the target platform, as
> >         returned by getpagesize(3C).  If you do not specify a
> >         valid pagesize, the request is silently ignored at
> >         run-time. The Solaris operating system offers no
> >         guarantee that the page size request will be honored.
> >
> >         You can use pmap(1) or meminfo(2) to determine page
> >         size of the target platform.
> >
> >         The -xpagesize option has no effect unless you use it
> >         at compile time and at link time. For a complete list
> >         of compiler options that must be specified at both com-
> >         pile time and at link time, see the C User's Guide.>
> >         Note that this feature is not available on the Solaris
> >         8 operating system. A program compiled with this option
> >         will not link on the Solaris 8 operating system.
> >
> >         If you specify -xpagesize=default, the operating
> >         environment sets the page size.
>
> But you might want to read the following "blueprint" help you
> determine 1) whether they are available on your processor, and
> 2) whether they will improve the performance of your app/system.
>
> http://www.sun.com/blueprints/0204/817-5489.pdf
>
> Bigger isn't always better.
>
> - Bob
>
>
>
> Dan Koren wrote:
>> What is the largest page size
>> supported by Solaris on SPARC?
>>
>> On my system (20 CPU Ultra-3)
>> with 20 GB memory installed,
>> getpagesizes() returns 8 kB,
>> 64 kB, 512 kB and 4 MB as
>> supported pages sizes. Is
>> 4 MB really the limit, or
>> can the system be set up
>> for larger pages sizes?
>> And if so, how?
>>
>> Any pointers and/or advice
>> would be greatly appreciated.
>>
>> dk 

Roland Mainz wrote:
> > Dan Koren <dankoren@yahoo.com> wrote:
> > > What is the largest page size
> > > supported by Solaris on SPARC?
> >
> > > On my system (20 CPU Ultra-3)
> > > with 20 GB memory installed,
> > > getpagesizes() returns 8 kB,
> > > 64 kB, 512 kB and 4 MB as
> > > supported pages sizes. Is
> > > 4 MB really the limit, or
> > > can the system be set up
> > > for larger pages sizes?
> > > And if so, how?
> >
> > According to "Solaris Internals", it's basically related to the hardware
> > support from the MMU.  That book is old and only goes through the US-II,
> > but mentions that 4M is in fact the largest page supported by the MMU on
> > a sun4u architecture.
> 
> AFAIK the UltraSPARC supports all page sizes from 8K up to 16GB in 8x
> steps, e.g. 8K, 64K, 512K, 4M, 32M, 256M and 16GB.

Mhhh... I am counting seven page sizes here (which I've taken from the
Sun Workshop/Forte documentation, see -xpagesize= option) ... what
happened to the eight one ? Is that 128GB or some
magic/unsupported/reserved value ?

----

Bye,
Roland

-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 7950090
 (;O/ \/ \O;)

Respectfully Dan,
	Your post (quoted in my reply), says nothing about either
shared memory or the specialized nature of the environment in
which your app is running!

- Bob

Dan Koren wrote:
> Thanks for the pointers. I am familiar
> with the tools and the "blueprint" you
> mentioned.
> 
> Note however that my question was about
> setting page sizes for shared memory
> segments -- not heaps or stacks.
> 
> In this case, however, there can be no
> question that bigger is better. The
> system I am trying to bring up is one
> dedicated to a single application which
> is essentially a huge cache. It needs to
> allocate as much memory as possible at
> (or shortly after) boot time, lock it
> in core, and use it with the fewest
> possible TLB and cache misses. Just
> as simple as that.
> 

Your original post...

>>>What is the largest page size
>>>supported by Solaris on SPARC?
>>>
>>>On my system (20 CPU Ultra-3)
>>>with 20 GB memory installed,
>>>getpagesizes() returns 8 kB,
>>>64 kB, 512 kB and 4 MB as
>>>supported pages sizes. Is
>>>4 MB really the limit, or
>>>can the system be set up
>>>for larger pages sizes?
>>>And if so, how?
>>>
>>>Any pointers and/or advice
>>>would be greatly appreciated.
>>>
>>>dk 
> 
> 
> 

Of course not. Mea culpa for the
ommission. I appologize if that
threw off some people, though it
might not be enirely unreasonable
to imagine such things can be read
between the lines ;-)

I do appreciate all the help so
graciously offered. Many thanks
to all who replied to my questions
about TLBs, MMUs, page sizes and
shared memory.

I can barely wait for Solaris to
go open source so I can modify
the kernel to my taste... ;-)



dk


"Robert Lawhead" <news0000.5.unixguru@spamgourmet.com> wrote in message 
news:WZudnaNr3IoYEMnfRVn-iA@speakeasy.net...
> Respectfully Dan,
> Your post (quoted in my reply), says nothing about either
> shared memory or the specialized nature of the environment in
> which your app is running!
>
> - Bob
>
> Dan Koren wrote:
>> Thanks for the pointers. I am familiar
>> with the tools and the "blueprint" you
>> mentioned.
>>
>> Note however that my question was about
>> setting page sizes for shared memory
>> segments -- not heaps or stacks.
>>
>> In this case, however, there can be no
>> question that bigger is better. The
>> system I am trying to bring up is one
>> dedicated to a single application which
>> is essentially a huge cache. It needs to
>> allocate as much memory as possible at
>> (or shortly after) boot time, lock it
>> in core, and use it with the fewest
>> possible TLB and cache misses. Just
>> as simple as that.
>>
>
> Your original post...
>
>>>>What is the largest page size
>>>>supported by Solaris on SPARC?
>>>>
>>>>On my system (20 CPU Ultra-3)
>>>>with 20 GB memory installed,
>>>>getpagesizes() returns 8 kB,
>>>>64 kB, 512 kB and 4 MB as
>>>>supported pages sizes. Is
>>>>4 MB really the limit, or
>>>>can the system be set up
>>>>for larger pages sizes?
>>>>And if so, how?
>>>>
>>>>Any pointers and/or advice
>>>>would be greatly appreciated.
>>>>
>>>>dk
>>
>> 

Future releases of Solaris 10 (eg updates) will provide:

* Automatic large page size support; eg system will automatically
use larger pages as it deems suitable for the application's heap
and stack for the current hardware.

* Large page size support for program and shared library text.

* Support for pages larger than 4M as hardware having such
capabilities becomes available...

- Bart

<barts@smaalders.net> wrote in message 
news:1112851011.867249.207000@o13g2000cwo.googlegroups.com...
> Future releases of Solaris 10 (eg updates) will provide:
>
> * Automatic large page size support; eg system will automatically
> use larger pages as it deems suitable for the application's heap
> and stack for the current hardware.
>
> * Large page size support for program and shared library text.
>
> * Support for pages larger than 4M as hardware having such
> capabilities becomes available...
>


Thanks.

Please note however that my
needs are very modest. All I
want is the ability to grab
the largest possible shared
memory segment shortly after
boot. I can take it from
there...

It would also be nice to be
able to find out on the fly
the attributes of the cache
and the TLB so the memory
allocator can configure
itself in optimal fashion.



dk 

<barts@smaalders.net> wrote in message 
news:1112851011.867249.207000@o13g2000cwo.googlegroups.com...
> Future releases of Solaris 10 (eg updates) will provide:
>
> * Automatic large page size support; eg system will automatically
> use larger pages as it deems suitable for the application's heap
> and stack for the current hardware.
>
> * Large page size support for program and shared library text.
>
> * Support for pages larger than 4M as hardware having such
> capabilities becomes available...
>


How about adding support for boot time memory
reservations for specific applications?

Say, a system config file where one could enter
something like:

application-name  reserved-memory-size  alignment

Doesn't seem too hard -- or is it?



dk 

barts@smaalders.net wrote:
> Future releases of Solaris 10 (eg updates) will provide:
> 
> * Automatic large page size support; eg system will automatically
> use larger pages as it deems suitable for the application's heap
> and stack for the current hardware.
> 
> * Large page size support for program and shared library text.
> 
> * Support for pages larger than 4M as hardware having such
> capabilities becomes available...

And (already there for some time) trapstat(1M) utility to measure
how much time is spent on TLBs for each page size.
Dan probably already knows about it but could be usefull for others.

-- 
                                                Robert Milkowski
                                                rmilkowskiDSFDSF@wp-sa.pl

Dan Koren wrote:
> <barts@smaalders.net> wrote in message 
> news:1112851011.867249.207000@o13g2000cwo.googlegroups.com...
> 
>>Future releases of Solaris 10 (eg updates) will provide:
>>
>>* Automatic large page size support; eg system will automatically
>>use larger pages as it deems suitable for the application's heap
>>and stack for the current hardware.
>>
>>* Large page size support for program and shared library text.
>>
>>* Support for pages larger than 4M as hardware having such
>>capabilities becomes available...
>>
> 
> 
> 
> How about adding support for boot time memory
> reservations for specific applications?
> 
> Say, a system config file where one could enter
> something like:
> 
> application-name  reserved-memory-size  alignment
> 
> Doesn't seem too hard -- or is it?

Depends. What about non-contiguous physical memory. Or support of
dynamic reconfiguration (adding/removing physical memory without reboot).

It is possible to do it, but you have to be carefull not to break other
features of the system.

I've done it in Solaris 8 :-)
My solution was an own memory segment driver loaded via "forceload". But
due to copyright reasons I can't give you the source.
And without access to the Solaris sources I think it is impossible.

Juergen

-- 
Juergen Gross                Telephone: +49 (0) 89 636 47950
Fujitsu Siemens Computers    Telefax:   +49 (0) 89 636 40638
Otto-Hahn-Ring 6             e-mail:    juergen.gross@fujitsu-siemens.com
81739 Muenchen, Germany      Internet:  http://www.fujitsu-siemens.com

"J�rgen Gro�" <juergen.gross@fujitsu-siemens.com> wrote in message 
news:d32r5d$cma$1@nntp.fujitsu-siemens.com...
> Dan Koren wrote:
>> <barts@smaalders.net> wrote in message 
>> news:1112851011.867249.207000@o13g2000cwo.googlegroups.com...
>>
>>>Future releases of Solaris 10 (eg updates) will provide:
>>>
>>>* Automatic large page size support; eg system will automatically
>>>use larger pages as it deems suitable for the application's heap
>>>and stack for the current hardware.
>>>
>>>* Large page size support for program and shared library text.
>>>
>>>* Support for pages larger than 4M as hardware having such
>>>capabilities becomes available...
>>
>> How about adding support for boot time memory
>> reservations for specific applications?
>>
>> Say, a system config file where one could enter
>> something like:
>>
>> application-name  reserved-memory-size  alignment
>>
>> Doesn't seem too hard -- or is it?
>
> Depends. What about non-contiguous physical memory.


Bummer in general. However, not an issue at all
for the kind of stuff I'm doing. Every system we
use is maxed out to the limit.


> Or support of dynamic reconfiguration (adding/
> removing physical memory without reboot).


Not an issue in my case.


> It is possible to do it, but you have to be carefull not to break other 
> features of the system.


No other features are relevant. It is a huge,
dedicated cache. Everything else is secondary
-- or even less important.


> I've done it in Solaris 8 :-)


Congrats.


> My solution was an own memory segment driver loaded via "forceload". But 
> due to copyright reasons I can't give you the source.



Not a problem, I know how to do this myself. I
would however prefer to get it from Sun... ;-)


> And without access to the Solaris sources I think it is impossible.


Pretty soon we should all have access to Solaris
source ;-)



dk 

Roland Mainz <roland.mainz@nrubsig.org> wrote:
>> AFAIK the UltraSPARC supports all page sizes from 8K up to 16GB in 8x
>> steps, e.g. 8K, 64K, 512K, 4M, 32M, 256M and 16GB.
> Mhhh... I am counting seven page sizes here (which I've taken from the
> Sun Workshop/Forte documentation, see -xpagesize= option) ... what
> happened to the eight one ? Is that 128GB or some
> magic/unsupported/reserved value ?

256M * 8 = 2G
2G   * 8 = 16G

Something might be missing in between here, given the factor of 8 you
mentioned.

Thanks for this info, anyway, very interesting. Can I only set this when
compiling or can this be a runtime setting, too?

lg, Bernd

"Dan" == Dan Koren <dankoren@yahoo.com> writes:

Dan> What is the largest page size supported by Solaris on SPARC?

Dan> On my system (20 CPU Ultra-3) with 20 GB memory installed,
Dan> getpagesizes() returns 8 kB, 64 kB, 512 kB and 4 MB as supported
Dan> pages sizes. Is 4 MB really the limit, or can the system be set
Dan> up for larger pages sizes?  And if so, how?

On UltraSPARC systems up through UltraSPARC IV, 4MB is the largest.  For
US-IV+ and Niagara we also have 32MB and 256MB.  Some future
processors in planning are also talking about 2GB and 16GB, and I've
heard rumors of 128GB on a processor even further out.

As always, use getpagesizes() to determine what's available.  Then you
can use MPSS functions like memcntl(), mpss.so.1 or ppgsz to select
page sizes for heap, stack and anonymous memory.  We're talking about
having the kernel select page sizes for programs in the future, using
the intimate details of the page sizes available and DTLB
characteristics to choose.
-- 
Dave Marquardt
Sun Microsystems, Inc.
Austin, TX
+1 512 401-1077

In article <4254e9cc@news.meer.net>,
	"Dan Koren" <dankoren@yahoo.com> writes:
> <barts@smaalders.net> wrote in message 
> news:1112851011.867249.207000@o13g2000cwo.googlegroups.com...
>> Future releases of Solaris 10 (eg updates) will provide:
>>
>> * Automatic large page size support; eg system will automatically
>> use larger pages as it deems suitable for the application's heap
>> and stack for the current hardware.
>>
>> * Large page size support for program and shared library text.
>>
>> * Support for pages larger than 4M as hardware having such
>> capabilities becomes available...
>>
> 
> 
> Thanks.
> 
> Please note however that my
> needs are very modest. All I
> want is the ability to grab
> the largest possible shared
> memory segment shortly after
> boot. I can take it from
> there...
> 
> It would also be nice to be
> able to find out on the fly
> the attributes of the cache
> and the TLB so the memory
> allocator can configure
> itself in optimal fashion.
> 
> 


Here's a portion of the prtconf -pv output on my
Sun Blade 100, running Solaris 9.


    Node 0xf0066918
        clock-frequency:  1debe980
        manufacturer#:  00000017
        implementation#:  00000013
        mask#:  00000014
        clock-divisors:  00000001.00000002.00000006
        ecache-size:  00040000
        name:  'SUNW,UltraSPARC-IIe'
        device_type:  'cpu'
        sparc-version:  00000009
        ecache-associativity:  00000004
        ecache-line-size:  00000040
        #dtlb-entries:  00000040
        dcache-associativity:  00000001
        dcache-line-size:  00000020
        dcache-size:  00004000
        #itlb-entries:  00000040
        icache-associativity:  00000002
        icache-line-size:  00000020
        icache-size:  00004000
        upa-portid:  00000000
        reg:  000001c0.00000000.00000000.00000008



Note the various cache related items.  I suppose if one knew how to
interpret them, they'd be quite informative.  For instance, the
dcache-line-size is 0x20; so maybe I'd decide I always wanted a minimum
of 16 byte alignment in my allocator.  (I have no idea if that really
makes sense, that's just an example).

In other words, at least on SPARC, it looks like there's plenty of
information.  How it might vary on different models, and how to interpret
it, that's the thing.   (One can get at the info in prtconf -pv
programmatically, although off the top of my head I couldn't tell you how.)


-- 
mailto:rlhamil@smart.net  http://www.smart.net/~rlhamil

Lasik/PRK theme music:
    "In the Hall of the Mountain King", from "Peer Gynt"

In article <xejy64yyofmo.fsf@hogwart.central.sun.com>,
	Dave Marquardt <Dave.Marquardt@Sun.COM> writes:
> "Dan" == Dan Koren <dankoren@yahoo.com> writes:
> 
> Dan> What is the largest page size supported by Solaris on SPARC?
> 
> Dan> On my system (20 CPU Ultra-3) with 20 GB memory installed,
> Dan> getpagesizes() returns 8 kB, 64 kB, 512 kB and 4 MB as supported
> Dan> pages sizes. Is 4 MB really the limit, or can the system be set
> Dan> up for larger pages sizes?  And if so, how?
> 
> On UltraSPARC systems up through UltraSPARC IV, 4MB is the largest.  For
> US-IV+ and Niagara we also have 32MB and 256MB.  Some future
> processors in planning are also talking about 2GB and 16GB, and I've
> heard rumors of 128GB on a processor even further out.
> 
> As always, use getpagesizes() to determine what's available.  Then you
> can use MPSS functions like memcntl(), mpss.so.1 or ppgsz to select
> page sizes for heap, stack and anonymous memory.  We're talking about
> having the kernel select page sizes for programs in the future, using
> the intimate details of the page sizes available and DTLB
> characteristics to choose.

Is there any way to cause SysV shared memory segments to use a particular
page size?  How about mmap()'ed regions?

-- 
mailto:rlhamil@smart.net  http://www.smart.net/~rlhamil

Lasik/PRK theme music:
    "In the Hall of the Mountain King", from "Peer Gynt"

Take a look at the memcntl(2) call; specifically the MC_HAT_ADVISE
option:

    NAME
     memcntl - memory management control

SYNOPSIS
     #include <sys/types.h>
     #include <sys/mman.h>

     int memcntl(caddr_t addr, size_t len, int cmd, caddr_t  arg,
     int attr, int mask);

DESCRIPTION
     The memcntl() function allows the calling process to apply a
     variety of control operations over the address space identi-
     fied by the  mappings  established  for  the  address  range
     [addr, addr + len).

     The addr argument must be a  multiple  of  the  pagesize  as
     returned by sysconf(3C). The scope of the control operations
     can be further defined with  additional  selection  criteria
     (in  the  form  of  attributes) according to the bit pattern
     contained in attr.

      .....

    MC_HAT_ADVISE   Advise system how a  region  of  user-mapped
                     memory will be accessed. The arg argument is
                     interpreted as a "struct memcntl_mha *". The
                     following  members  are  defined in a struct
                     memcntl_mha:

                     uint_t mha_cmd;
                     uint_t mha_flags;
                     size_t mha_pagesize;


                     The accepted values for mha_cmd are:

                     MHA_MAPSIZE_VA
                     MHA_MAPSIZE_STACK
                     MHA_MAPSIZE_BSSBRK


                     The mha_flags member is reserved for  future
                     use  and  must  always  be  set  to  0.  The
                     mha_pagesize member must be a valid size  as
                     obtained  from  getpagesizes(3C) or the con-
                     stant value 0 to allow the system to  choose
                     an  appropriate hardware address translation
                     mapping size.

                     MHA_MAPSIZE_VA sets the  preferred  hardware
                   address  translation  mapping  size  of  the
                     region of memory from addr to  addr  +  len.
                     Both  addr  and  len  must  be aligned to an
                     mha_pagesize boundary.  The  entire  virtual
                     address  region from addr to addr + len must
                     not have any holes. Permissions within  each
                     mha_pagesize-aligned  portion  of the region
                     must be consistent.  When a  size  of  0  is
                     specified, the system selects an appropriate
                     size based on the size and alignment of  the
                     memory  region, type of processor, and other
                     considerations.

                     MHA_MAPSIZE_STACK   sets    the    preferred
                     hardware address translation mapping size of
                     the process main thread stack  segment.  The
                     addr  and  len arguments must be NULL and 0,
                     respectively.

                     MHA_MAPSIZE_BSSBRK   sets   the    preferred
                     hardware address translation mapping size of
                     the process heap. The addr and len arguments
                     must  be  NULL  and 0, respectively. See the
                     NOTES section of the  ppgsz(1)  manual  page
                     for  additional  information on process heap
                     alignment.

                     The  attr  argument  must  be  0   for   all
                     MC_HAT_ADVISE operations.

"Dave Marquardt" <Dave.Marquardt@Sun.COM> wrote in message 
news:xejy64yyofmo.fsf@hogwart.Central.Sun.COM...
> "Dan" == Dan Koren <dankoren@yahoo.com> writes:
>
> Dan> What is the largest page size supported by Solaris on SPARC?
>
> Dan> On my system (20 CPU Ultra-3) with 20 GB memory installed,
> Dan> getpagesizes() returns 8 kB, 64 kB, 512 kB and 4 MB as supported
> Dan> pages sizes. Is 4 MB really the limit, or can the system be set
> Dan> up for larger pages sizes?  And if so, how?
>
> On UltraSPARC systems up through UltraSPARC IV, 4MB is the largest.  For
> US-IV+ and Niagara we also have 32MB and 256MB.  Some future
> processors in planning are also talking about 2GB and 16GB, and I've
> heard rumors of 128GB on a processor even further out.
>


Thanks for the info.

I would very much like a 4GB
page size to be supported.

Guess why.... ;-)



dk 

"Dan" == Dan Koren <dankoren@yahoo.com> writes:

Dan> "Dave Marquardt" <Dave.Marquardt@Sun.COM> wrote in message 
Dan> news:xejy64yyofmo.fsf@hogwart.Central.Sun.COM...
>> "Dan" == Dan Koren <dankoren@yahoo.com> writes:
>> 
Dan> What is the largest page size supported by Solaris on SPARC?
>> 
Dan> On my system (20 CPU Ultra-3) with 20 GB memory installed,
Dan> getpagesizes() returns 8 kB, 64 kB, 512 kB and 4 MB as supported
Dan> pages sizes. Is 4 MB really the limit, or can the system be set
Dan> up for larger pages sizes?  And if so, how?
>> 
>> On UltraSPARC systems up through UltraSPARC IV, 4MB is the largest.  For
>> US-IV+ and Niagara we also have 32MB and 256MB.  Some future
>> processors in planning are also talking about 2GB and 16GB, and I've
>> heard rumors of 128GB on a processor even further out.
>> 


Dan> Thanks for the info.

Dan> I would very much like a 4GB
Dan> page size to be supported.

Yeah, I bet you would.  But you'll notice that the page sizes increase by
a factor of 8, so you'll have to use 2 2GB pages.  Or waste 12 GB of a
16GB page.  Still a heck of a lot better than 1024 4MB pages or 524288
8K pages! :-)
-- 
Dave Marquardt
Sun Microsystems, Inc.
Austin, TX
+1 512 401-1077

Bernd Haug wrote:
> >> AFAIK the UltraSPARC supports all page sizes from 8K up to 16GB in 8x
> >> steps, e.g. 8K, 64K, 512K, 4M, 32M, 256M and 16GB.
> > Mhhh... I am counting seven page sizes here (which I've taken from the
> > Sun Workshop/Forte documentation, see -xpagesize= option) ... what
> > happened to the eight one ? Is that 128GB or some
> > magic/unsupported/reserved value ?
> 
> 256M * 8 = 2G
> 2G   * 8 = 16G
> 
> Something might be missing in between here, given the factor of 8 you
> mentioned.

Weired... either the Sun Workshop/Forte documentation is incomplete
(Seongbae Park from Sun may be able to answer that question) or the
hardware isn't consistent here (unlikely) ...

> Thanks for this info, anyway, very interesting. Can I only set this when
> compiling or can this be a runtime setting, too?

In Solaris >= 9 you can select the page sizes for heap and stack at both
compile time and run time... the quiz is whether the kernel really gives
you what you want - which depends on issues like supported page sizes
(by both kernel and MMU hardware) and availability (for example
fragmentation may prevent allocation of large page sizes (AFAIK the
Solaris kernel lacks a deamon which does memory defragmentation yet) and
alighment rules may hit you, too) ...

----

Bye,
Roland

-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 7950090
 (;O/ \/ \O;)

Roland Mainz <roland.mainz@nrubsig.org> wrote:
> Bernd Haug wrote:
>> >> AFAIK the UltraSPARC supports all page sizes from 8K up to 16GB in 8x
>> >> steps, e.g. 8K, 64K, 512K, 4M, 32M, 256M and 16GB.
>> > Mhhh... I am counting seven page sizes here (which I've taken from the
>> > Sun Workshop/Forte documentation, see -xpagesize= option) ... what
>> > happened to the eight one ? Is that 128GB or some
>> > magic/unsupported/reserved value ?
>> 
>> 256M * 8 = 2G
>> 2G   * 8 = 16G
>> 
>> Something might be missing in between here, given the factor of 8 you
>> mentioned.

The compiler accepts 8K,64K,512K,4M,32M,256M,2G,16G.

> Weired... either the Sun Workshop/Forte documentation is incomplete
> (Seongbae Park from Sun may be able to answer that question) or the
> hardware isn't consistent here (unlikely) ...

US1/2/3/4 supports only up to 4MB page size (8k,64k,512k,4M).
US4+ will add 32MB and 256MB page support.
IIRC, the latest (or upcoming?) Fujitsu SPARC64 also supports 256MB.

2GB and 16GB support will come at some point.
I don't know how soon we'll need 128GB page 
but my hunch is that it will be quite a while before
we'll need such a humongous page size 
(one could argue that this is just another evidence
that the paging is not ideal for managing 
multi-terabyte memory).
-- 
#pragma ident "Seongbae Park, compiler, http://blogs.sun.com/seongbae/"

Seongbae Park wrote:
> >> >> AFAIK the UltraSPARC supports all page sizes from 8K up to 16GB in 8x
> >> >> steps, e.g. 8K, 64K, 512K, 4M, 32M, 256M and 16GB.
> >> > Mhhh... I am counting seven page sizes here (which I've taken from the
> >> > Sun Workshop/Forte documentation, see -xpagesize= option) ... what
> >> > happened to the eight one ? Is that 128GB or some
> >> > magic/unsupported/reserved value ?
> >>
> >> 256M * 8 = 2G
> >> 2G   * 8 = 16G
> >>
> >> Something might be missing in between here, given the factor of 8 you
> >> mentioned.
> 
> The compiler accepts 8K,64K,512K,4M,32M,256M,2G,16G.

Is it possible that the documentation skipped 2GB by accident ?

> > Weired... either the Sun Workshop/Forte documentation is incomplete
> > (Seongbae Park from Sun may be able to answer that question) or the
> > hardware isn't consistent here (unlikely) ...
> 
> US1/2/3/4 supports only up to 4MB page size (8k,64k,512k,4M).
> US4+ will add 32MB and 256MB page support.
> IIRC, the latest (or upcoming?) Fujitsu SPARC64 also supports 256MB.
> 
> 2GB and 16GB support will come at some point.
> I don't know how soon we'll need 128GB page
> but my hunch is that it will be quite a while before
> we'll need such a humongous page size

Depends on how fast SPARC hardware can handle more memory... AFAIK the
SunFire15K could just have four 128GB pages (assuming the maximum memory
limit is still 512GB when a hypothetical US-IV++ gets 128GB page
support). I guess adding 128GB page support starts to make sense when a
system can have more than 16 of such pages (e.g.  on a system >= 2TB
main memory).

> (one could argue that this is just another evidence
> that the paging is not ideal for managing
> multi-terabyte memory).

In that case the complainers should make a proposal how else this should
be managed... :)

----

Bye,
Roland

-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 7950090
 (;O/ \/ \O;)

"Dave Marquardt" <Dave.Marquardt@Sun.COM> wrote in message 
news:xejyekdlmj0k.fsf@hogwart.Central.Sun.COM...
>>>
>>> On UltraSPARC systems up through UltraSPARC IV, 4MB is the largest.  For
>>> US-IV+ and Niagara we also have 32MB and 256MB.  Some future
>>> processors in planning are also talking about 2GB and 16GB, and I've
>>> heard rumors of 128GB on a processor even further out.
>
> Dan> Thanks for the info.
>
> Dan> I would very much like a 4GB
> Dan> page size to be supported.
>
> Yeah, I bet you would.  But you'll notice that the page sizes increase by
> a factor of 8, so you'll have to use 2 2GB pages.  Or waste 12 GB of a
> 16GB page.  Still a heck of a lot better than 1024 4MB pages or 524288
> 8K pages! :-)


I would actually prefer the page sizes to start at
16 kB (2^14) and go up by factors of 64 (2^6). This
leads to the following sequence: 16 KB, 1 MB, 64 MB,
4 GB, 256 GB, 8 TB.

Guess why this is better.... ;-)



dk

Dan Koren wrote:
> >>> On UltraSPARC systems up through UltraSPARC IV, 4MB is the largest.  For
> >>> US-IV+ and Niagara we also have 32MB and 256MB.  Some future
> >>> processors in planning are also talking about 2GB and 16GB, and I've
> >>> heard rumors of 128GB on a processor even further out.
> >
> > Dan> Thanks for the info.
> >
> > Dan> I would very much like a 4GB
> > Dan> page size to be supported.
> >
> > Yeah, I bet you would.  But you'll notice that the page sizes increase by
> > a factor of 8, so you'll have to use 2 2GB pages.  Or waste 12 GB of a
> > 16GB page.  Still a heck of a lot better than 1024 4MB pages or 524288
> > 8K pages! :-)
> 
> I would actually prefer the page sizes to start at
> 16 kB (2^14) and go up by factors of 64 (2^6). This
> leads to the following sequence: 16 KB, 1 MB, 64 MB,
> 4 GB, 256 GB, 8 TB.
> 
> Guess why this is better.... ;-)

I hope this is something like sarcasm...
.... such a non-finegrained pagesize scheme would be a nightmare to page
in/out from/to disk.

----

Bye,
Roland

-- 
  __ .  . __
 (o.\ \/ /.o) roland.mainz@nrubsig.org
  \__\/\/__/  MPEG specialist, C&&JAVA&&Sun&&Unix programmer
  /O /==\ O\  TEL +49 641 7950090
 (;O/ \/ \O;)

Dan Koren wrote:
> I would actually prefer the page sizes to start at
> 16 kB (2^14) and go up by factors of 64 (2^6). This
> leads to the following sequence: 16 KB, 1 MB, 64 MB,
> 4 GB, 256 GB, 8 TB.

Because you have a machine with 4 GB of physical memory
and you want to have one TLB entry instead of two?  It
hardly seems like a big hardship to have two to me.

Of course, it could be for some other reason I don't get.
(That happens sometimes...)

   - Logan

Roland Mainz wrote:
> Roland Mainz wrote:
> 
>>>Dan Koren <dankoren@yahoo.com> wrote:
>>>
>>>>What is the largest page size
>>>>supported by Solaris on SPARC?
>>>
>>>>On my system (20 CPU Ultra-3)
>>>>with 20 GB memory installed,
>>>>getpagesizes() returns 8 kB,
>>>>64 kB, 512 kB and 4 MB as
>>>>supported pages sizes. Is
>>>>4 MB really the limit, or
>>>>can the system be set up
>>>>for larger pages sizes?
>>>>And if so, how?
>>>
>>>According to "Solaris Internals", it's basically related to the hardware
>>>support from the MMU.  That book is old and only goes through the US-II,
>>>but mentions that 4M is in fact the largest page supported by the MMU on
>>>a sun4u architecture.
>>
>>AFAIK the UltraSPARC supports all page sizes from 8K up to 16GB in 8x
>>steps, e.g. 8K, 64K, 512K, 4M, 32M, 256M and 16GB.
> 
> 
> Mhhh... I am counting seven page sizes here (which I've taken from the
> Sun Workshop/Forte documentation, see -xpagesize= option) ... what
> happened to the eight one ? Is that 128GB or some
> magic/unsupported/reserved value ?
> 
> ----
> 
> Bye,
> Roland
> 
I think the 2GB increment was missing between 256M and 16G.

Dan,

Unless I miss something obvious, can't you simply make sure you use
intimate shared memory when you create the shared memory segment
(setting flag SHM_SHARE_MMU)? ISM has a side effect of locking memory
in core and you don't need to worry about paging and physical-virtual
memory address translation. In addition, "Solaris Internals" (p.203)
says ISM causes 4M pages to be used instead of the default. I think 4M
happened to be the largest back then. It's possible ISM just causes the
largest possible pages (`pagesize -a`) to be used.

BTW, do you mind telling us what the app is? If it's Oracle, ISM is
already enabled unless you disable it with the undocumented parameter
_use_ism.

Yong Huang