Hi all,

I am having a strange issue with a fork call. After some runs, I
cannot call fork from my program. This is my code:

  pid_t pID=-1;
  while (pID<0)
    {
      cout << "trying to fork ... " << endl;
      pID = fork ();
      if (pID < 0)              // failed to fork
        {
          cerr << "errno: " << strerror(errno) << endl;
          cerr << "Failed to fork" << endl;
          sleep(5);
        }
    }

After a number of runs that varies between 20 and 30, I have the
following error output:

trying to fork ...
errno: Cannot allocate memory
Failed to fork
trying to fork ...
errno: Cannot allocate memory
Failed to fork

And so on. But a look to free gives enough memory:

$ free -m
                     total       used       free     shared
buffers     cached
Mem:          1011        656        355          0         30
353
-/+ buffers/cache:        271        739
Swap:         1176          0       1176

I have about 355 MB of physical and more than 1GB of swap, which
should be enough.
I also had a look to the number of processes running:
$ ps aux | wc -l
111

And the system limitations:
$ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
max nice                        (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 8183
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
max rt priority                 (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 8183
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited


So I have no idea where the problem is ... any help please?

Thanks

On Oct 28, 2:22 pm, bel...@gmail.com wrote:
> Hi all,
>
> I am having a strange issue with a fork call. After some runs, I
> cannot call fork from my program. This is my code:
>
>   pid_t pID=-1;
>   while (pID<0)
>     {
>       cout << "trying to fork ... " << endl;
>       pID = fork ();
>       if (pID < 0)              // failed to fork
>         {
>           cerr << "errno: " << strerror(errno) << endl;
>           cerr << "Failed to fork" << endl;
>           sleep(5);
>         }
>     }
>
> After a number of runs that varies between 20 and 30, I have the
> following error output:
>
> trying to fork ...
> errno: Cannot allocate memory
> Failed to fork
> trying to fork ...
> errno: Cannot allocate memory
> Failed to fork

"Cannot allocate memory" is presumably ENOMEM.  It's kind of a
catchall.  Some systems will fail with ENOMEM if some other resource
is exhausted; and calling fork() duplicates a lot of resources.  File
descriptors, memory mappings, etc.

What system is this on?  Someone might know in what specific
situations it returns ENOMEM.  Also, does the process do anything
noteworthy that might consume a lot of some resource?

> What system is this on?  Someone might know in what specific
> situations it returns ENOMEM.  Also, does the process do anything
> noteworthy that might consume a lot of some resource?
I see no reason for consuming a lot of resources. After doing the
fork, the child launches a external program using execlp(). The father
monitors this program, and after a timeout it kills it:

 kill( pID, SIGKILL);  // Kill child process
 int childExitStatus;
 waitpid (pID, &childExitStatus, 0);

And then just repeat.

On Oct 29, 2:22 am, bel...@gmail.com wrote:
> Hi all,
>
> I am having a strange issue with a fork call. After some runs, I
> cannot call fork from my program. This is my code:
>
>   pid_t pID=-1;
>   while (pID<0)
>     {
>       cout << "trying to fork ... " << endl;
>       pID = fork ();
>       if (pID < 0)              // failed to fork
>         {
>           cerr << "errno: " << strerror(errno) << endl;
>           cerr << "Failed to fork" << endl;
>           sleep(5);
>         }
>     }
>
> After a number of runs that varies between 20 and 30, I have the
> following error output:
>
> trying to fork ...
> errno: Cannot allocate memory
> Failed to fork
> trying to fork ...
> errno: Cannot allocate memory
> Failed to fork
>
> And so on. But a look to free gives enough memory:
>
> $ free -m
>                      total       used       free     shared
> buffers     cached
> Mem:          1011        656        355          0         30
> 353
> -/+ buffers/cache:        271        739
> Swap:         1176          0       1176
>
> I have about 355 MB of physical and more than 1GB of swap, which
> should be enough.
> I also had a look to the number of processes running:
> $ ps aux | wc -l
> 111
>
> And the system limitations:
> $ ulimit -a
> core file size          (blocks, -c) 0
> data seg size           (kbytes, -d) unlimited
> max nice                        (-e) 0
> file size               (blocks, -f) unlimited
> pending signals                 (-i) 8183
> max locked memory       (kbytes, -l) 32
> max memory size         (kbytes, -m) unlimited
> open files                      (-n) 1024
> pipe size            (512 bytes, -p) 8
> POSIX message queues     (bytes, -q) 819200
> max rt priority                 (-r) 0
> stack size              (kbytes, -s) 8192
> cpu time               (seconds, -t) unlimited
> max user processes              (-u) 8183
> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
>
> So I have no idea where the problem is ... any help please?
>
> Thanks

I understand that you are running your program as a non-root user?
If that assumption is right, then does the number of child processes
that the parent process can spawn change if you are logged in as root
(or if the parent process is running on behalf of root)?
If the answer to that is yes, then as is mentioned in other comments
to this messae, you are probably reaching some resource limit while
running the parent process as non-root.
Try changing open files limit etc.
HTH.

Finally it resulted to be a memory leak ... but it was funny because
neither the "free" command or "top" did not report excesive memory
usage.

Searching in ulimit, the only limitation that I may be breaking is the
stack size, which is limited to 8192 kB.

On 29 oct, 08:16, Amol Vaikar <amol.vai...@gmail.com> wrote:
> On Oct 29, 2:22 am, bel...@gmail.com wrote:
>
>
>
> > Hi all,
>
> > I am having a strange issue with a fork call. After some runs, I
> > cannot call fork from my program. This is my code:
>
> >   pid_t pID=-1;
> >   while (pID<0)
> >     {
> >       cout << "trying to fork ... " << endl;
> >       pID = fork ();
> >       if (pID < 0)              // failed to fork
> >         {
> >           cerr << "errno: " << strerror(errno) << endl;
> >           cerr << "Failed to fork" << endl;
> >           sleep(5);
> >         }
> >     }
>
> > After a number of runs that varies between 20 and 30, I have the
> > following error output:
>
> > trying to fork ...
> > errno: Cannot allocate memory
> > Failed to fork
> > trying to fork ...
> > errno: Cannot allocate memory
> > Failed to fork
>
> > And so on. But a look to free gives enough memory:
>
> > $ free -m
> >                      total       used       free     shared
> > buffers     cached
> > Mem:          1011        656        355          0         30
> > 353
> > -/+ buffers/cache:        271        739
> > Swap:         1176          0       1176
>
> > I have about 355 MB of physical and more than 1GB of swap, which
> > should be enough.
> > I also had a look to the number of processes running:
> > $ ps aux | wc -l
> > 111
>
> > And the system limitations:
> > $ ulimit -a
> > core file size          (blocks, -c) 0
> > data seg size           (kbytes, -d) unlimited
> > max nice                        (-e) 0
> > file size               (blocks, -f) unlimited
> > pending signals                 (-i) 8183
> > max locked memory       (kbytes, -l) 32
> > max memory size         (kbytes, -m) unlimited
> > open files                      (-n) 1024
> > pipe size            (512 bytes, -p) 8
> > POSIX message queues     (bytes, -q) 819200
> > max rt priority                 (-r) 0
> > stack size              (kbytes, -s) 8192
> > cpu time               (seconds, -t) unlimited
> > max user processes              (-u) 8183
> > virtual memory          (kbytes, -v) unlimited
> > file locks                      (-x) unlimited
>
> > So I have no idea where the problem is ... any help please?
>
> > Thanks
>
> I understand that you are running your program as a non-root user?
> If that assumption is right, then does the number of child processes
> that the parent process can spawn change if you are logged in as root
> (or if the parent process is running on behalf of root)?
> If the answer to that is yes, then as is mentioned in other comments
> to this messae, you are probably reaching some resource limit while
> running the parent process as non-root.
> Try changing open files limit etc.
> HTH.

On Oct 31, 8:01 am, bel...@gmail.com wrote:
> Finally it resulted to be a memory leak ... but it was funny because
> neither the "free" command or "top" did not report excesive memory
> usage.
>
> Searching in ulimit, the only limitation that I may be breaking is the
> stack size, which is limited to 8192 kB.
>
> On 29 oct, 08:16, Amol Vaikar <amol.vai...@gmail.com> wrote:
>
> > On Oct 29, 2:22 am, bel...@gmail.com wrote:
>
> > > Hi all,
>
> > > I am having a strange issue with a fork call. After some runs, I
> > > cannot call fork from my program. This is my code:
>
> > >   pid_t pID=-1;
> > >   while (pID<0)
> > >     {
> > >       cout << "trying to fork ... " << endl;
> > >       pID = fork ();
> > >       if (pID < 0)              // failed to fork
> > >         {
> > >           cerr << "errno: " << strerror(errno) << endl;
> > >           cerr << "Failed to fork" << endl;
> > >           sleep(5);
> > >         }
> > >     }
>
> > > After a number of runs that varies between 20 and 30, I have the
> > > following error output:
>
> > > trying to fork ...
> > > errno: Cannot allocate memory
> > > Failed to fork
> > > trying to fork ...
> > > errno: Cannot allocate memory
> > > Failed to fork
>
> > > And so on. But a look to free gives enough memory:
>
> > > $ free -m
> > >                      total       used       free     shared
> > > buffers     cached
> > > Mem:          1011        656        355          0         30
> > > 353
> > > -/+ buffers/cache:        271        739
> > > Swap:         1176          0       1176
>
> > > I have about 355 MB of physical and more than 1GB of swap, which
> > > should be enough.
> > > I also had a look to the number of processes running:
> > > $ ps aux | wc -l
> > > 111
>
> > > And the system limitations:
> > > $ ulimit -a
> > > core file size          (blocks, -c) 0
> > > data seg size           (kbytes, -d) unlimited
> > > max nice                        (-e) 0
> > > file size               (blocks, -f) unlimited
> > > pending signals                 (-i) 8183
> > > max locked memory       (kbytes, -l) 32
> > > max memory size         (kbytes, -m) unlimited
> > > open files                      (-n) 1024
> > > pipe size            (512 bytes, -p) 8
> > > POSIX message queues     (bytes, -q) 819200
> > > max rt priority                 (-r) 0
> > > stack size              (kbytes, -s) 8192
> > > cpu time               (seconds, -t) unlimited
> > > max user processes              (-u) 8183
> > > virtual memory          (kbytes, -v) unlimited
> > > file locks                      (-x) unlimited
>
> > > So I have no idea where the problem is ... any help please?
>
> > > Thanks
>
> > I understand that you are running your program as a non-root user?
> > If that assumption is right, then does the number of child processes
> > that the parent process can spawn change if you are logged in as root
> > (or if the parent process is running on behalf of root)?
> > If the answer to that is yes, then as is mentioned in other comments
> > to this messae, you are probably reaching some resource limit while
> > running the parent process as non-root.
> > Try changing open files limit etc.
> > HTH.

A memory leak where? Concerning the fork() function or elsewhere in
your program?
It is a sad fact that malloc will almost never return NULL as a sign
there's no memory left,
nor will C++'s new operator, but instead they usually just block the
whole system, slow
it down or kill the process. Or is it just from my experience?

On Oct 28, 5:22 pm, bel...@gmail.com wrote:
> Hi all,
>
> I am having a strange issue with a fork call. After some runs, I
> cannot call fork from my program. This is my code:
>
>   pid_t pID=-1;
>   while (pID<0)
>     {
>       cout << "trying to fork ... " << endl;
>       pID = fork ();
>       if (pID < 0)              // failed to fork
>         {
>           cerr << "errno: " << strerror(errno) << endl;
>           cerr << "Failed to fork" << endl;
>           sleep(5);
>         }
>     }
>
> After a number of runs that varies between 20 and 30, I have the
> following error output:
>
> trying to fork ...
> errno: Cannot allocate memory
> Failed to fork
> trying to fork ...
> errno: Cannot allocate memory
> Failed to fork
>
> And so on. But a look to free gives enough memory:

You've come close to reinventing the fork bomb, where you take up all
the available process table entries with your forking program, and
leave no entries for any other new processes. Fortunately, you have
some system limits (see below) that prevent you from taking /all/ the
entries.

[snip irrelevant free stats]

> And the system limitations:
> $ ulimit -a
[snip]
> max user processes              (-u) 8183

OK, so there cannot be any more than 8183 processes running with your
UID. That's the limit to your fork bomb.

> virtual memory          (kbytes, -v) unlimited
> file locks                      (-x) unlimited
>
> So I have no idea where the problem is ... any help please?


Take a look at the fork(2) documentation ("man 2 fork"), and read the
ERRORS section. For Linux, fork(2) says

ERRORS
       EAGAIN fork() cannot allocate sufficient memory

       EAGAIN It was not possible to create a new process

       ENOMEM fork() failed to allocate the necessary kernel
structures because memory is tight.

So, if you are running on Linux, you ran into a kernel structure out-
of-memory condition before you ran into the ulimit process limit
condition.

To remedy the situation, either get more real memory for your kernel
to use, or stop forking so many child processes.

HTH
--
Lew

Lew Pitcher wrote:
> On Oct 28, 5:22 pm, bel...@gmail.com wrote:
>> Hi all,
>>
>> I am having a strange issue with a fork call. After some runs, I
>> cannot call fork from my program. This is my code:
>>
>>   pid_t pID=-1;
>>   while (pID<0)
>>     {
>>       cout << "trying to fork ... " << endl;
>>       pID = fork ();
>>       if (pID < 0)              // failed to fork
>>         {
>>           cerr << "errno: " << strerror(errno) << endl;
>>           cerr << "Failed to fork" << endl;
>>           sleep(5);
>>         }
>>     }

> You've come close to reinventing the fork bomb

I thought so too until I read carefully.  The condition on the
while() loop is only true if the fork() fails.  So this loop
will only result in one *successful* call to fork().  It's not
a fork() bomb; it's fork() with error handling and a retry when
it fails (and until it succeeds).  Interestingly, when it does
succeed, both parent and child break out of the loop because
the pid is nonnegative for both.

Were I to write it, I would probably write something like this
instead:

     pid_t pid;
     while (1) {
         pid = fork();
         if (pid != -1) {
             break;
         }

         printf("fork failed; will try again\n");
         sleep(5);
     }

I believe that's equivalent, but to me it's easier to read.

   - Logan

Fork problems are always weird.

-- 
The most powerful Usenet tool you have ever heard of.
NewsMaestro v. 4.0.6 - Dictionary Update/Expert Mode has been released.

* Significant improvement in symbol substitution mechanism
  for verb tense and plurals.
* Expert mode.
* Miscellaneous improvements and bug fixes.
* Templates generator improvements.
* Multi-job support.

Note: In some previous releases some class files were missing.
As a result, the program would not run.
Sorry for the inconvenience.

Web page:
http://newsmaestro.sourceforge.net/

Download page:
http://newsmaestro.sourceforge.net/Download_Information.htm

Send any feedback, ideas, suggestions, test results to
newsmaestroinfo \at/ mail.ru.

Your personal info will not be released and your privacy
will be honored.

On 2007-10-31, belion@gmail.com <belion@gmail.com> wrote:
> Finally it resulted to be a memory leak ... but it was funny because
> neither the "free" command or "top" did not report excesive memory
> usage.
[...]

Were you looking at the right place? In "top" the VIRT column was supposed
to give you increasing size of your virtual memory. In "free" you should
have seen the increased amount of used swap space.


-- 
Minds, like parachutes, function best when open