On Solaris 10, resource_controls(5) man page says:

     zone.max-swap

         Total amount of swap that can be  consumed  by  user  process
         address space mappings and tmpfs mounts for this zone.


Is there a way to set an equivalent limit on a group of processes? I have
some forking monsters (PHP, Perl, Python) which have a tendency to eat all
available memory (RAM + swap) if there are enough user requests, so I'd
like to limit memory usage.

Those processes can be run under some kind of process manager (like
FastCGI), but all process managers that I'm aware of limit the number of
processes, which is somewhat related to the amount of free memory, but
isn't exactly what I'm looking for.

Setting up a zone for each forking monster just to limit swap usage seems
like an overkill. It also eats IP addresses and adds complexity (in
startup scripts, if nothing else), so I'd like to avoid it if I can.

Dtrace solution might be good enough if there's no direct support in the
kernel. In fact, it might be preferrable even if there is a kernel
support.

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave@fly.srk.fer.hr

In article <slrnij0e76.om8.dave@fly.srk.fer.hr>,
Drazen Kacar  <dave@fly.srk.fer.hr> wrote:
>     zone.max-swap

>         Total amount of swap that can be  consumed  by  user  process
>         address space mappings and tmpfs mounts for this zone.

>Is there a way to set an equivalent limit on a group of processes? I have
>some forking monsters (PHP, Perl, Python) which have a tendency to eat all
>available memory (RAM + swap) if there are enough user requests, so I'd
>like to limit memory usage.

It's not exactly what you asked for, but we use a daemon which monitors each 
users' memory usage and kills their processes when they use more than a 
configured limit.  We use this on login and web servers to prevent individual 
users from using all our memory, but if you could run all your (Fast)CGI 
processes under the same uid, it should work for that as well.

You would probably have to modify it slightly to only kill certain users' 
processes; by default it kills all users who aren't explicitly exempted.

	- river.

In article <slrnij0e76.om8.dave@fly.srk.fer.hr>,
Drazen Kacar  <dave@fly.srk.fer.hr> wrote:
>Setting up a zone for each forking monster just to limit swap usage seems
>like an overkill. It also eats IP addresses and adds complexity (in
>startup scripts, if nothing else), so I'd like to avoid it if I can.

Can you move Solaris 11 Express?
zones with VNICs with private IP addresses and behind proxy front-end
solves my problem of address exhaustion.

zones simplify my web hosting environment with multiple
applications running mod_perl, mod_jk, mod_passenger, mod_php
mod_etc_etc_etc.
Ruby developers can go off the rails and the Perl guys
are none the wiser.

John
groenveld@acm.org

> =A0 =A0 =A0zone.max-swap
>
> =A0 =A0 =A0 =A0 =A0Total amount of swap that can be =A0consumed =A0by =A0=
user =A0process
> =A0 =A0 =A0 =A0 =A0address space mappings and tmpfs mounts for this zone.
>
> Is there a way to set an equivalent limit on a group of processes? I have
> some forking monsters (PHP, Perl, Python) which have a tendency to eat al=
l
> available memory (RAM + swap) if there are enough user requests, so I'd
> like to limit memory usage.
>
> Those processes can be run under some kind of process manager (like
> FastCGI), but all process managers that I'm aware of limit the number of
> processes, which is somewhat related to the amount of free memory, but
> isn't exactly what I'm looking for.

First thing that came to mind was using projects, but the output of
"rctladm" shows no max-swap equivalent for projects...  Maybe
something with "ulimit"?

John D Groenveld wrote:
>  In article <slrnij0e76.om8.dave@fly.srk.fer.hr>,
>  Drazen Kacar  <dave@fly.srk.fer.hr> wrote:
> >Setting up a zone for each forking monster just to limit swap usage seems
> >like an overkill. It also eats IP addresses and adds complexity (in
> >startup scripts, if nothing else), so I'd like to avoid it if I can.
> 
>  Can you move Solaris 11 Express?

No. :-)

>  zones with VNICs with private IP addresses and behind proxy front-end
>  solves my problem of address exhaustion.
> 
>  zones simplify my web hosting environment with multiple
>  applications running mod_perl, mod_jk, mod_passenger, mod_php
>  mod_etc_etc_etc.

I don't have a web hosting environment. My forking monsters are a part of
diagnostics environment, so the number of processes tends to grow when
something somewhere doesn't work and then other programs and humans want
to run diagnostic tools. So all of mod_xxx processes essentialy go under
the same resource limits.

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave@fly.srk.fer.hr

River Tarnell wrote:
>  In article <slrnij0e76.om8.dave@fly.srk.fer.hr>,
>  Drazen Kacar  <dave@fly.srk.fer.hr> wrote:
> >     zone.max-swap
> 
> >         Total amount of swap that can be  consumed  by  user  process
> >         address space mappings and tmpfs mounts for this zone.
> 
> >Is there a way to set an equivalent limit on a group of processes? I have
> >some forking monsters (PHP, Perl, Python) which have a tendency to eat all
> >available memory (RAM + swap) if there are enough user requests, so I'd
> >like to limit memory usage.
> 
>  It's not exactly what you asked for, but we use a daemon which monitors
>  each users' memory usage and kills their processes when they use more
>  than a configured limit.

Close enough, if the source is available.

>  We use this on login and web servers to prevent individual 
>  users from using all our memory, but if you could run all your (Fast)CGI 
>  processes under the same uid, it should work for that as well.

They are under the same UID, which is also used by some processes that
shouldn't be affected. But I can rearrange all that if necessary.

>  You would probably have to modify it slightly to only kill certain users' 
>  processes; by default it kills all users who aren't explicitly exempted.

Well, I'd like to modify it with soft and hard limits. On soft limit it
should contact FastCGI process manager and tell it not to fork more
processes until further notice.

Upon reaching hard limit it can kill some processes, but I hope that
would not happen.

Is the source available?

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave@fly.srk.fer.hr

In article <slrnij14gq.6jq.dave@fly.srk.fer.hr>,
Drazen Kacar  <dave@fly.srk.fer.hr> wrote:
>River Tarnell wrote:
>>  It's not exactly what you asked for, but we use a daemon which monitors
>>  each users' memory usage and kills their processes when they use more
>>  than a configured limit.

>Close enough, if the source is available.

The source is in subversion:

% svn co https://svn.toolserver.org/svnroot/toolserver/trunk/slayerd

I can put up a tar file if you don't have an svn client.

Build:
% gmake
% gmake install PREFIX=/opt/slayerd

You'll need a built version of Boost (not just headers) as well, and a version 
of GNU install (or install it by hand).  I'm afraid it's somewhat messy; I was 
in a hurry to write it since Linux's "helpful" OOM behaviour (kill random 
processes) was causing us rather a lot of hassle.  We've been running it in 
production for 3-4 years without problems, though.

>>  You would probably have to modify it slightly to only kill certain users' 
>>  processes; by default it kills all users who aren't explicitly exempted.

>Well, I'd like to modify it with soft and hard limits. On soft limit it
>should contact FastCGI process manager and tell it not to fork more
>processes until further notice.

That should not be difficult; see check_user() in slayerd.cc.

	- river.

River Tarnell wrote:
>  In article <slrnij14gq.6jq.dave@fly.srk.fer.hr>,
>  Drazen Kacar  <dave@fly.srk.fer.hr> wrote:
> >River Tarnell wrote:
> >>  It's not exactly what you asked for, but we use a daemon which monitors
> >>  each users' memory usage and kills their processes when they use more
> >>  than a configured limit.
> 
> >Close enough, if the source is available.
> 
>  The source is in subversion:
> 
>  % svn co https://svn.toolserver.org/svnroot/toolserver/trunk/slayerd
> 
>  I can put up a tar file if you don't have an svn client.

Got it, thanks.

>  You'll need a built version of Boost (not just headers) as well, and a
>  version of GNU install (or install it by hand).  I'm afraid it's
>  somewhat messy; I was in a hurry to write it since Linux's "helpful"
>  OOM behaviour (kill random processes) was causing us rather a lot of
>  hassle.

I hear that there's an all new, all improved OOM killer nowadays. I
normally turn it off, so I don't know if it's any better. :-)

> >Well, I'd like to modify it with soft and hard limits. On soft limit it
> >should contact FastCGI process manager and tell it not to fork more
> >processes until further notice.
> 
>  That should not be difficult; see check_user() in slayerd.cc.

Well, yesterday I played with dtrace a bit and it seems better suited for
what I have in mind. Since all my memory hogs allocate on heap, I could
catch all brk() calls from interesting processes and do a bit of
accounting. 

It's weird that dtrace doesn't make contract id available in psinfo_t, but
parent PID should be good enough for my limited purposes.

And there doesn't seem to be a normal way for another process to ask for
something from dtrace script, but that can be worked around. Although
there really should be, so if powers that be are reading, pretty please...
:-)

-- 
 .-.   .-.    Yes, I am an agent of Satan, but my duties are largely
(_  \ /  _)   ceremonial.
     |
     |        dave@fly.srk.fer.hr