Consider the following scenario : your programme written in an AMM
language needs to allocate a new object (or objects) but there isn't
enough memory available. The language does garbage collection but there
still isn't enough memory and the operating system refuses to give any
more memory. What I would like to know is how existing languages with
AMM handle such a situation.

The other question is what would you consider the Right Way to handle
it ? The obvious answer is that the language generates an exception.
Generating an exception may also require allocating a new object which
would be problematic if you're out of memory to begin with. But
the language could at the beginning of execution of each programme
allocate some emergency memory which will only be used to signal
exceptions for out of memory conditions (OOMC).

For an exception to be most useful it needs to contain the information
of which piece of code caused the exception and for OOMCs this is a bit
problematic.  Imagine for example that the code contains an expression
like b*c + d*e .  If the language can handle arbitrarily large numbers
then any of the operations can cause an OOMC if the runtime environment
needs to allocate extra memory to hold the result. For a compiled
language it would be a bit difficult to add enough additional
information to the executable so that it can be determined at runtime
which operation caused the OOMC i.e. whether it was b*c or d*e or the
addition. I'm guessing that it would also make the executable more
slow.

So perhaps the best practical solution is that for OOMCs the exception
does not contain the information of which piece of code caused the
condition or does not pinpoint the code very closely ; for example it
could say that it happened as a result of executing such and such
function. If an OOMC does not get trapped then the runtime environment
will simply print an OOMC message , possibly stating how much memory it
tried to allocate but could not , and then terminate the programme. But
the programmer would have the option of trapping the condition using
the general mechanism the language provides for such a thing.

What do you think ?

-- 
Some people say there is a God. Some say there is no God. Personally ,
I feel the truth lies probably somewhere between these two extremes.
  J.B. Morton

Spiros Bousbouras <spibou@gmail.com> writes:

> The other question is what would you consider the Right Way to handle
> it ? The obvious answer is that the language generates an exception.

Yes.

> Generating an exception may also require allocating a new object which
> would be problematic if you're out of memory to begin with.  But
> the language could at the beginning of execution of each programme
> allocate some emergency memory which will only be used to signal
> exceptions for out of memory conditions (OOMC).

Yes.


> For an exception to be most useful it needs to contain the information
> of which piece of code caused the exception and for OOMCs this is a bit
> problematic.  

Not at all.  That information is already in the memory.

Perhaps you're thinking "C++", where people keep copying data all the
time, because they don't have a garbage collector that would allow them
to keep track of what data is used and what is not.

But when you have a garbage collector, you can easily just keep
references to any data anywhere.  This also make your exeception objects
small, since they only need a few reference slots.



> So perhaps the best practical solution is that for OOMCs the exception
> does not contain the information of which piece of code caused the
> condition or does not pinpoint the code very closely ; for example it
> could say that it happened as a result of executing such and such
> function. If an OOMC does not get trapped then the runtime environment
> will simply print an OOMC message , possibly stating how much memory it
> tried to allocate but could not , and then terminate the programme. But
> the programmer would have the option of trapping the condition using
> the general mechanism the language provides for such a thing.

There's no point in not providing the information you have, since it
doesn't cost anything to provide it.


> What do you think ?

You could consider the conditions (exceptions) of Common Lisp.  You can
handle them either with or without unwinding the call stack.  If you
handle them  without unwinding the call stack, you know exactly where
the condition occured, and you can even continue execution after the
problem is cured.  (It would be rather useless to try to handle out of
memory conditions if you couldn't resume after curation).

Notice also that there would be no point in all this, if the application
didn't have the knowledge and the means to free some more memory.

This is because in some cases the application is the only one to know
what memory can be freed under heavy memory requirements that the
garbage collectors may rely on it and signal a condition when it's out
of memory as a way to ask the application to free it.

It is of course a requirement for such a condition handler to do its
work without allocating any more memory.



http://www.gigamonkeys.com/book/beyond-exception-handling-conditions-and-restarts.html
http://www.nhplace.com/kent/Papers/Condition-Handling-2001.html

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

On Sat, 09 Oct 2010 00:03:19 +0200
pjb@informatimago.com (Pascal J. Bourguignon) wrote:
> Spiros Bousbouras <spibou@gmail.com> writes:
> 
> > The other question is what would you consider the Right Way to handle
> > it ? The obvious answer is that the language generates an exception.
> 
> Yes.
> 
> > Generating an exception may also require allocating a new object which
> > would be problematic if you're out of memory to begin with.  But
> > the language could at the beginning of execution of each programme
> > allocate some emergency memory which will only be used to signal
> > exceptions for out of memory conditions (OOMC).
> 
> Yes.
> 
> 
> > For an exception to be most useful it needs to contain the information
> > of which piece of code caused the exception and for OOMCs this is a bit
> > problematic.  
> 
> Not at all.  That information is already in the memory.

Executables have a copy of the source code in memory ? That's news to
me.

> Perhaps you're thinking "C++", where people keep copying data all the
> time, because they don't have a garbage collector that would allow them
> to keep track of what data is used and what is not.

I gave a specific example involving the expression b*c + d*e which you
snipped. Imagine the equivalent code in any language you care.

[...]

> > What do you think ?
> 
> You could consider the conditions (exceptions) of Common Lisp.

What got me thinking on this issue was actually Common Lisp. Does the
standard mention any OOMC ?

> You can
> handle them either with or without unwinding the call stack.  If you
> handle them  without unwinding the call stack, you know exactly where
> the condition occured, and you can even continue execution after the
> problem is cured.  (It would be rather useless to try to handle out of
> memory conditions if you couldn't resume after curation).

Wrapping every form which has the potential of allocating memory inside
an exception handler isn't practical. I don't think that the Common
Lisp standard specifies which operations might allocate memory so every
single Common Lisp form is a candidate.

> Notice also that there would be no point in all this, if the application
> didn't have the knowledge and the means to free some more memory.

Yes there would be. If for example the application tried to create a
large array and it failed with an OOMC then the application wouldn't
need to free anything because the array wasn't created at all.

But let's say that it is more important for the application to create a
new object than preserve an already existing one. How does one free
memory in a language with AMM ? The only way I can think of is to
manually keep track of all the references to an object and then assign
to all those references some indifferent value. The only practical way
I can think of to achieve this would be to anticipate that some large
object may need to be freed and make sure that there is only one
(global) variable referencing the object and that all the code that
needs to access the object would use this variable. A bit tedious if
the variable has a long descriptive name but workable I guess.

But there is a more serious difficulty : assume that the global
variable var holds a reference to some large object. Function foo needs
to assign some temporary value to var so it copies var to var2 ,
assigns a new value to var and at some later point calls function bar.
During the execution of bar an OOMC happens. But bar doesn't know about
var2 so it cannot free the object. You can return to foo but then the
stack has unwound which might not be desirable. So you have a problem.

> This is because in some cases the application is the only one to know
> what memory can be freed under heavy memory requirements that the
> garbage collectors may rely on it and signal a condition when it's out
> of memory as a way to ask the application to free it.

It wouldn't be the garbage collector which signals the condition but
the code which tries to create a new object. I certainly wouldn't want
to have an OOMC signalled unless I tried to create a new object and
there isn't enough memory to create it.

> It is of course a requirement for such a condition handler to do its
> work without allocating any more memory.

That may not be possible but it could use the emergency memory pool. Of
course then you would need a way to tell the translator that a particular
piece of code is supposed to have access to the emergency memory pool.

> http://www.gigamonkeys.com/book/beyond-exception-handling-conditions-and-restarts.html
> http://www.nhplace.com/kent/Papers/Condition-Handling-2001.html

Neither of these says anything about OOMCs.

-- 
Judicial punishment is revenge abstracted.

On 10/ 9/10 11:03 AM, Pascal J. Bourguignon wrote:
> Spiros Bousbouras<spibou@gmail.com>  writes:
>
>> The other question is what would you consider the Right Way to handle
>> it ? The obvious answer is that the language generates an exception.
>
> Yes.
>
>> Generating an exception may also require allocating a new object which
>> would be problematic if you're out of memory to begin with.  But
>> the language could at the beginning of execution of each programme
>> allocate some emergency memory which will only be used to signal
>> exceptions for out of memory conditions (OOMC).
>
> Yes.
>
>
>> For an exception to be most useful it needs to contain the information
>> of which piece of code caused the exception and for OOMCs this is a bit
>> problematic.
>
> Not at all.  That information is already in the memory.
>
> Perhaps you're thinking "C++", where people keep copying data all the
> time, because they don't have a garbage collector that would allow them
> to keep track of what data is used and what is not.

Eh?  If we don't care exactly when dynamic memory is freed, we pass 
round smart pointers to it.

> But when you have a garbage collector, you can easily just keep
> references to any data anywhere.  This also make your exeception objects
> small, since they only need a few reference slots.

Ditto smart pointers.  Note in idiomatic C++, exceptions are caught by 
const reference, so the object isn't copied.

-- 
Ian Collins

Spiros Bousbouras <spibou@gmail.com> writes:

> On Sat, 09 Oct 2010 00:03:19 +0200
> pjb@informatimago.com (Pascal J. Bourguignon) wrote:
>> Spiros Bousbouras <spibou@gmail.com> writes:
>> 
>> > The other question is what would you consider the Right Way to handle
>> > it ? The obvious answer is that the language generates an exception.
>> 
>> Yes.
>> 
>> > Generating an exception may also require allocating a new object which
>> > would be problematic if you're out of memory to begin with.  But
>> > the language could at the beginning of execution of each programme
>> > allocate some emergency memory which will only be used to signal
>> > exceptions for out of memory conditions (OOMC).
>> 
>> Yes.
>> 
>> 
>> > For an exception to be most useful it needs to contain the information
>> > of which piece of code caused the exception and for OOMCs this is a bit
>> > problematic.  
>> 
>> Not at all.  That information is already in the memory.
>
> Executables have a copy of the source code in memory ? That's news to
> me.

Well, in lisp we tend to keep the sources around.  Not always, but more
often than with other programming languages.


>> Perhaps you're thinking "C++", where people keep copying data all the
>> time, because they don't have a garbage collector that would allow them
>> to keep track of what data is used and what is not.
>
> I gave a specific example involving the expression b*c + d*e which you
> snipped. Imagine the equivalent code in any language you care.
>
> [...]
>
>> > What do you think ?
>> 
>> You could consider the conditions (exceptions) of Common Lisp.
>
> What got me thinking on this issue was actually Common Lisp. Does the
> standard mention any OOMC ?

It doesn't even specify a garbage collector!


But all implementations have one (competition and customer pressure).
Re. OOMC, have a look at: 
http://blo.udoidio.info/2008/10/out-of-memory-sad-case.html
http://groups.google.com/group/comp.lang.lisp/browse_frm/thread/34bdfdde528aa416/62e920ad391f2678?hl=en&q=%22out+of+memory%22+condition+implementation+group:comp.lang.lisp&pli=1
It gives the situation two years ago.  Things have improved.

>> You can
>> handle them either with or without unwinding the call stack.  If you
>> handle them  without unwinding the call stack, you know exactly where
>> the condition occured, and you can even continue execution after the
>> problem is cured.  (It would be rather useless to try to handle out of
>> memory conditions if you couldn't resume after curation).
>
> Wrapping every form which has the potential of allocating memory inside
> an exception handler isn't practical. I don't think that the Common
> Lisp standard specifies which operations might allocate memory so every
> single Common Lisp form is a candidate.

With HANDLER-BIND, you would actually set your handler only at the point
of the dynamic countour of resource management.  

For example, suppose you set up a cache of web page in an web spider
application.  You only need to set up the handler once for all in the
main, when you've initialized the cache.

Something like:

(defmacro with-page-cache (cache-var &body body)
  `(let ((,cache-var (make-cache)))
     (unwind-protect
        (handler-bind
           ((storage-condition 
              (lambda (err)
                (let ((required-size (compute-required-size-from-condition err)))
                  (cache-free-space ,cache-var required-size)))))
          (progn ,@body))
      (cache-free ,cache-var))))


(defun main ()
  (with-page-cache pcache
    (walk-the-web)))



>> Notice also that there would be no point in all this, if the application
>> didn't have the knowledge and the means to free some more memory.
>
> Yes there would be. If for example the application tried to create a
> large array and it failed with an OOMC then the application wouldn't
> need to free anything because the array wasn't created at all.

Of course.


> But let's say that it is more important for the application to create a
> new object than preserve an already existing one. How does one free
> memory in a language with AMM ? The only way I can think of is to
> manually keep track of all the references to an object and then assign
> to all those references some indifferent value. The only practical way
> I can think of to achieve this would be to anticipate that some large
> object may need to be freed and make sure that there is only one
> (global) variable referencing the object and that all the code that
> needs to access the object would use this variable. A bit tedious if
> the variable has a long descriptive name but workable I guess.

Of course, it's easier in the case of a cache, where unused objects have
only a single reference to them.  But an application could also walk the
graph of its objects and find unessential stuff to free.   There are
often a lot of cached data that can be thrown out and recomputed.  Just
ask each object to get slim.


> But there is a more serious difficulty : assume that the global
> variable var holds a reference to some large object. Function foo needs
> to assign some temporary value to var so it copies var to var2 ,
> assigns a new value to var and at some later point calls function bar.

Objects are not large.  They're small.  And they contain references to
other objects.  Perhaps a lot of them indeed.

Sure, if you manage memory yourself, and you want to keep your sanity,
you try to allocate big chunks.  But with a garbage collector, it's
easier and even often more efficient to allocate a lot of small
objects.  In the extreme, just CONS cells containing just two
references.


> During the execution of bar an OOMC happens. But bar doesn't know about
> var2 so it cannot free the object. You can return to foo but then the
> stack has unwound which might not be desirable. So you have a problem.

That's why you'd use HANDLER-BIND instead of HANDLER-CASE.


>> This is because in some cases the application is the only one to know
>> what memory can be freed under heavy memory requirements that the
>> garbage collectors may rely on it and signal a condition when it's out
>> of memory as a way to ask the application to free it.
>
> It wouldn't be the garbage collector which signals the condition but
> the code which tries to create a new object. 

Not at all.  The object creation function would call the allocation
function which is part of the memory management module, which includes
the garbage collector.   Actually, with some garbage collection
algorithms, the garbage collection occurs effectively in the allocation
function.  See for example http://home.pipeline.com/~hbaker1/RealTimeGC.html


> I certainly wouldn't want
> to have an OOMC signalled unless I tried to create a new object and
> there isn't enough memory to create it.

Of course.  And who knows there's not enough memory?  Only the garbage
collector can know that, once it has collected all the garbage, and it
sees there's not enough space.

Actually, generationnal garbage collectors will then try to collect even
more garbage (in older generations).  There may be more than two
generations.  Before giving up and signaling the OOM condition.



>> It is of course a requirement for such a condition handler to do its
>> work without allocating any more memory.
>
> That may not be possible but it could use the emergency memory pool. Of
> course then you would need a way to tell the translator that a particular
> piece of code is supposed to have access to the emergency memory pool.

Right.  Things could be set up to have the condition handler (or at
least the condition handler handling some specific conditions) use this
emergency memory pool.


>> http://www.gigamonkeys.com/book/beyond-exception-handling-conditions-and-restarts.html
>> http://www.nhplace.com/kent/Papers/Condition-Handling-2001.html
>
> Neither of these says anything about OOMCs.

But they explain the difference between HANDLER-CASE and HANDLER-BIND.
Few languages have the equivalent of HANDLER-BIND.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

Ian Collins <ian-news@hotmail.com> writes:

> On 10/ 9/10 11:03 AM, Pascal J. Bourguignon wrote:
>> Spiros Bousbouras<spibou@gmail.com>  writes:
>>
>>> For an exception to be most useful it needs to contain the information
>>> of which piece of code caused the exception and for OOMCs this is a bit
>>> problematic.
>>
>> Not at all.  That information is already in the memory.
>>
>> Perhaps you're thinking "C++", where people keep copying data all the
>> time, because they don't have a garbage collector that would allow them
>> to keep track of what data is used and what is not.
>
> Eh?  If we don't care exactly when dynamic memory is freed, we pass
> round smart pointers to it.

Smart pointers cannot deal with circular structures.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

On 10/ 9/10 02:37 PM, Pascal J. Bourguignon wrote:
> Ian Collins<ian-news@hotmail.com>  writes:
>
>> On 10/ 9/10 11:03 AM, Pascal J. Bourguignon wrote:
>>> Spiros Bousbouras<spibou@gmail.com>   writes:
>>>
>>>> For an exception to be most useful it needs to contain the information
>>>> of which piece of code caused the exception and for OOMCs this is a bit
>>>> problematic.
>>>
>>> Not at all.  That information is already in the memory.
>>>
>>> Perhaps you're thinking "C++", where people keep copying data all the
>>> time, because they don't have a garbage collector that would allow them
>>> to keep track of what data is used and what is not.
>>
>> Eh?  If we don't care exactly when dynamic memory is freed, we pass
>> round smart pointers to it.
>
> Smart pointers cannot deal with circular structures.

Nor do I, so that isn't a problem!

Circular structures aren't that common.

-- 
Ian Collins

Ian Collins <ian-news@hotmail.com> writes:

> On 10/ 9/10 02:37 PM, Pascal J. Bourguignon wrote:
>> Ian Collins<ian-news@hotmail.com>  writes:
>>
>>> On 10/ 9/10 11:03 AM, Pascal J. Bourguignon wrote:
>>>> Spiros Bousbouras<spibou@gmail.com>   writes:
>>>>
>>>>> For an exception to be most useful it needs to contain the information
>>>>> of which piece of code caused the exception and for OOMCs this is a bit
>>>>> problematic.
>>>>
>>>> Not at all.  That information is already in the memory.
>>>>
>>>> Perhaps you're thinking "C++", where people keep copying data all the
>>>> time, because they don't have a garbage collector that would allow them
>>>> to keep track of what data is used and what is not.
>>>
>>> Eh?  If we don't care exactly when dynamic memory is freed, we pass
>>> round smart pointers to it.
>>
>> Smart pointers cannot deal with circular structures.
>
> Nor do I, so that isn't a problem!
>
> Circular structures aren't that common.

They are not common when you have smart pointers.   When you have a
garbage collector, you use them much more often.  Instead of having to
deal with undirectional relationships between objects, there's no cost
in implementing bidirectional relationships (just one back reference),
and then navigating the object graphs is much easier.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

"Spiros Bousbouras" <spibou@gmail.com> wrote in message news:uLKro.18876$7B6.6976@newsfe07.ams2...
| Consider the following scenario : your programme written in an AMM
| language needs to allocate a new object (or objects) but there isn't
| enough memory available. The language does garbage collection but there
| still isn't enough memory and the operating system refuses to give any
| more memory. What I would like to know is how existing languages with
| AMM handle such a situation.

In PL/I, the run-time raises an exception.
It it possible to intercept this exception and to continue executing
the program, possibly with a smaller request for memory.

| The other question is what would you consider the Right Way to handle
| it ? The obvious answer is that the language generates an exception.
| Generating an exception may also require allocating a new object which
| would be problematic if you're out of memory to begin with. But
| the language could at the beginning of execution of each programme
| allocate some emergency memory which will only be used to signal
| exceptions for out of memory conditions (OOMC).

I would expect the error-handling code (written by the programmer)
to handle the exception and to print some error message,
and also to print values of variables, etc.
(as well as, of course, printing the statement number of the
offending statement).

| For an exception to be most useful it needs to contain the information
| of which piece of code caused the exception and for OOMCs this is a bit
| problematic.

In PL/I, the number of the statement that failed to obtain memory
can be printed.

|  Imagine for example that the code contains an expression
| like b*c + d*e .  If the language can handle arbitrarily large numbers
| then any of the operations can cause an OOMC if the runtime environment
| needs to allocate extra memory to hold the result. For a compiled
| language it would be a bit difficult to add enough additional
| information to the executable so that it can be determined at runtime
| which operation caused the OOMC i.e. whether it was b*c or d*e or the
| addition. I'm guessing that it would also make the executable more
| slow.

| So perhaps the best practical solution is that for OOMCs the exception
| does not contain the information of which piece of code caused the
| condition or does not pinpoint the code very closely ;

In PL/I, the location in the object program is normally available,
and can be printed.
However, more useful for that class of error is the statement number
of the offending statement, which can be printed.

| for example it
| could say that it happened as a result of executing such and such
| function. If an OOMC does not get trapped then the runtime environment
| will simply print an OOMC message , possibly stating how much memory it
| tried to allocate but could not , and then terminate the programme. But
| the programmer would have the option of trapping the condition using
| the general mechanism the language provides for such a thing. 

On 10/10/10 12:47 AM, Pascal J. Bourguignon wrote:
> Ian Collins<ian-news@hotmail.com>  writes:
>
>> On 10/ 9/10 02:37 PM, Pascal J. Bourguignon wrote:
>>> Ian Collins<ian-news@hotmail.com>   writes:
>>>
>>>> On 10/ 9/10 11:03 AM, Pascal J. Bourguignon wrote:
>>>>> Spiros Bousbouras<spibou@gmail.com>    writes:
>>>>>
>>>>>> For an exception to be most useful it needs to contain the information
>>>>>> of which piece of code caused the exception and for OOMCs this is a bit
>>>>>> problematic.
>>>>>
>>>>> Not at all.  That information is already in the memory.
>>>>>
>>>>> Perhaps you're thinking "C++", where people keep copying data all the
>>>>> time, because they don't have a garbage collector that would allow them
>>>>> to keep track of what data is used and what is not.
>>>>
>>>> Eh?  If we don't care exactly when dynamic memory is freed, we pass
>>>> round smart pointers to it.
>>>
>>> Smart pointers cannot deal with circular structures.
>>
>> Nor do I, so that isn't a problem!
>>
>> Circular structures aren't that common.
>
> They are not common when you have smart pointers.   When you have a
> garbage collector, you use them much more often.  Instead of having to
> deal with undirectional relationships between objects, there's no cost
> in implementing bidirectional relationships (just one back reference),
> and then navigating the object graphs is much easier.

While true, a lot depends on the problem being addressed.  At least in 
the domains I work in, unidirectional relationships are far more common 
than bidirectional.  Garbage collectors both have their strengths and 
weaknesses and I'm happy to use either, as the problem at hand demands. 
  It's nice to work with a language that offers the choice.

There is one very widely used garbage collector that also fails to deal 
with circular structures - the JavaScript engine in IE (at least as far 
as IE6, I assume it's now fixed)

-- 
Ian Collins

Ian Collins <ian-news@hotmail.com> writes:

> There is one very widely used garbage collector that also fails to
> deal with circular structures - the JavaScript engine in IE (at least
> as far as IE6, I assume it's now fixed)

Thus you may understand why I don't care too much of such languages,
when these problems have been solved for 50 years in Lisp (and only
improved upon ever after, instead of trying to catch up state of the
art...).

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

On 10/10/10 03:07 PM, Pascal J. Bourguignon wrote:
> Ian Collins<ian-news@hotmail.com>  writes:
>
>> There is one very widely used garbage collector that also fails to
>> deal with circular structures - the JavaScript engine in IE (at least
>> as far as IE6, I assume it's now fixed)
>
> Thus you may understand why I don't care too much of such languages,
> when these problems have been solved for 50 years in Lisp (and only
> improved upon ever after, instead of trying to catch up state of the
> art...).

When programming for the browser, JavaScript is the only option.  Couple 
that with a client daft enough to insist on IE and life stops being fun!

But seriously, the problem was (I hope the past tense is appropriate) in 
an implementation, not in the language.

-- 
Ian Collins

"Spiros Bousbouras" <spibou@gmail.com> wrote in message news:qrOro.4836$AM5.3325@newsfe16.ams2...
| On Sat, 09 Oct 2010 00:03:19 +0200
| pjb@informatimago.com (Pascal J. Bourguignon) wrote:
| > Spiros Bousbouras <spibou@gmail.com> writes:

| > > For an exception to be most useful it needs to contain the information
| > > of which piece of code caused the exception and for OOMCs this is a bit
| > > problematic.
| >
| > Not at all.  That information is already in the memory.
|
| Executables have a copy of the source code in memory ? That's news to
| me.

Not normally.  However, statement numbers can be stored with the
executable, and thus tell the user where the exception occurred. 

On Sat, 9 Oct 2010 23:07:02 +1100
"robin" <robin51@dodo.com.au> wrote:
> "Spiros Bousbouras" <spibou@gmail.com> wrote in message news:uLKro.18876$7B6.6976@newsfe07.ams2...
> | Consider the following scenario : your programme written in an AMM
> | language needs to allocate a new object (or objects) but there isn't
> | enough memory available. The language does garbage collection but there
> | still isn't enough memory and the operating system refuses to give any
> | more memory. What I would like to know is how existing languages with
> | AMM handle such a situation.
> 
> In PL/I, the run-time raises an exception.
> It it possible to intercept this exception and to continue executing
> the program, possibly with a smaller request for memory.
> 
> | The other question is what would you consider the Right Way to handle
> | it ? The obvious answer is that the language generates an exception.
> | Generating an exception may also require allocating a new object which
> | would be problematic if you're out of memory to begin with. But
> | the language could at the beginning of execution of each programme
> | allocate some emergency memory which will only be used to signal
> | exceptions for out of memory conditions (OOMC).
> 
> I would expect the error-handling code (written by the programmer)
> to handle the exception and to print some error message,
> and also to print values of variables, etc.
> (as well as, of course, printing the statement number of the
> offending statement).

Actually what I meant was what would you consider the Right Way for a
language to handle it i.e. what facilities should it provide for this
sort of thing ?  Since you mention PL/I , would you consider ideal the
way it handles it or can you think of improvements ?

> | For an exception to be most useful it needs to contain the information
> | of which piece of code caused the exception and for OOMCs this is a bit
> | problematic.
> 
> In PL/I, the number of the statement that failed to obtain memory
> can be printed.

I skimmed through the Wikipedia article on PL/I and it doesn't say what
the statement number is. The 2 short listings it has don't have
anything which looks like a statement number. In a line like
a = b*c + d*e;
which I think is legal PL/I , what are the statement numbers ?

> |  Imagine for example that the code contains an expression
> | like b*c + d*e .  If the language can handle arbitrarily large numbers
> | then any of the operations can cause an OOMC if the runtime environment
> | needs to allocate extra memory to hold the result. For a compiled
> | language it would be a bit difficult to add enough additional
> | information to the executable so that it can be determined at runtime
> | which operation caused the OOMC i.e. whether it was b*c or d*e or the
> | addition. I'm guessing that it would also make the executable more
> | slow.
> 
> | So perhaps the best practical solution is that for OOMCs the exception
> | does not contain the information of which piece of code caused the
> | condition or does not pinpoint the code very closely ;
> 
> In PL/I, the location in the object program is normally available,
> and can be printed.
> However, more useful for that class of error is the statement number
> of the offending statement, which can be printed.

In <4cb1be12$0$33792$c30e37c6@exi-reader.telstra.net> you say:
"However, statement numbers can be stored with the
executable, and thus tell the user where the exception occurred."
Does it make the executable slower if you choose to do this ? (I'm
assuming you're still talking about PL/I.)

On Sat, 09 Oct 2010 03:35:49 +0200
pjb@informatimago.com (Pascal J. Bourguignon) wrote:
> Spiros Bousbouras <spibou@gmail.com> writes:
> 
> > On Sat, 09 Oct 2010 00:03:19 +0200
> > pjb@informatimago.com (Pascal J. Bourguignon) wrote:
> >> Spiros Bousbouras <spibou@gmail.com> writes:
> >> 
> >> > The other question is what would you consider the Right Way to handle
> >> > it ? The obvious answer is that the language generates an exception.
> >> 
> >> Yes.
> >> 
> >> > Generating an exception may also require allocating a new object which
> >> > would be problematic if you're out of memory to begin with.  But
> >> > the language could at the beginning of execution of each programme
> >> > allocate some emergency memory which will only be used to signal
> >> > exceptions for out of memory conditions (OOMC).
> >> 
> >> Yes.
> >> 
> >> 
> >> > For an exception to be most useful it needs to contain the information
> >> > of which piece of code caused the exception and for OOMCs this is a bit
> >> > problematic.  
> >> 
> >> Not at all.  That information is already in the memory.
> >
> > Executables have a copy of the source code in memory ? That's news to
> > me.
> 
> Well, in lisp we tend to keep the sources around.  Not always, but more
> often than with other programming languages.

So the information is *not* in memory contrary to what you say above.

I would also appreciate it if you stop trying to turn this into a Lisp
vs other languages comparison.

[...]

> >> You can
> >> handle them either with or without unwinding the call stack.  If you
> >> handle them  without unwinding the call stack, you know exactly where
> >> the condition occured, and you can even continue execution after the
> >> problem is cured.  (It would be rather useless to try to handle out of
> >> memory conditions if you couldn't resume after curation).
> >
> > Wrapping every form which has the potential of allocating memory inside
> > an exception handler isn't practical. I don't think that the Common
> > Lisp standard specifies which operations might allocate memory so every
> > single Common Lisp form is a candidate.
> 
> With HANDLER-BIND, you would actually set your handler only at the point
> of the dynamic countour of resource management.  
> 
> For example, suppose you set up a cache of web page in an web spider
> application.  You only need to set up the handler once for all in the
> main, when you've initialized the cache.
> 
> Something like:
> 
> (defmacro with-page-cache (cache-var &body body)
>   `(let ((,cache-var (make-cache)))
>      (unwind-protect
>         (handler-bind
>            ((storage-condition 
>               (lambda (err)
>                 (let ((required-size (compute-required-size-from-condition err)))
>                   (cache-free-space ,cache-var required-size)))))
>           (progn ,@body))
>       (cache-free ,cache-var))))
> 
> 
> (defun main ()
>   (with-page-cache pcache
>     (walk-the-web)))
> 

Your example doesn't enlighten me. How about we do the one I mentioned ?
So we have the form  (+ (* b c) (* d e))
Each of the 3 forms has the potential of causing an OOMC. If it
happens we would , ideally , want to know the specific form which
caused it. Can we do it with just one HANDLER-BIND or would we have to
put one around each form ?

> >> Notice also that there would be no point in all this, if the application
> >> didn't have the knowledge and the means to free some more memory.
> >
> > Yes there would be. If for example the application tried to create a
> > large array and it failed with an OOMC then the application wouldn't
> > need to free anything because the array wasn't created at all.
> 
> Of course.
> 
> > But let's say that it is more important for the application to create a
> > new object than preserve an already existing one. How does one free
> > memory in a language with AMM ? The only way I can think of is to
> > manually keep track of all the references to an object and then assign
> > to all those references some indifferent value. The only practical way
> > I can think of to achieve this would be to anticipate that some large
> > object may need to be freed and make sure that there is only one
> > (global) variable referencing the object and that all the code that
> > needs to access the object would use this variable. A bit tedious if
> > the variable has a long descriptive name but workable I guess.
> 
> Of course, it's easier in the case of a cache, where unused objects have
> only a single reference to them.  But an application could also walk the
> graph of its objects and find unessential stuff to free.   There are
> often a lot of cached data that can be thrown out and recomputed.  Just
> ask each object to get slim.

I don't see how during runtime an application can locate all the
objects it has created never mind actually assess which are needed. Only
during writing the code the programmer knows that if at point A an
object cannot be created due to lack of memory , then it is better to
lose some previously created object (and which one) than not creating
the new one.

> > But there is a more serious difficulty : assume that the global
> > variable var holds a reference to some large object. Function foo needs
> > to assign some temporary value to var so it copies var to var2 ,
> > assigns a new value to var and at some later point calls function bar.
> 
> Objects are not large.  They're small.  And they contain references to
> other objects.  Perhaps a lot of them indeed.
> 
> Sure, if you manage memory yourself, and you want to keep your sanity,
> you try to allocate big chunks.  But with a garbage collector, it's
> easier and even often more efficient to allocate a lot of small
> objects.  In the extreme, just CONS cells containing just two
> references.

I don't agree but it's not the topic of the thread so I won't pursue
it.

> > During the execution of bar an OOMC happens. But bar doesn't know about
> > var2 so it cannot free the object. You can return to foo but then the
> > stack has unwound which might not be desirable. So you have a problem.
> 
> That's why you'd use HANDLER-BIND instead of HANDLER-CASE.

Ok , consider this:

(defvar *var* some-large-object)

(defun bar (...)
    (let ((a some-object))
    ...code...))

(defun foo (...)
    (let ((*var* other-object))
        ...code...
        (bar)
        ...more code...))

Let's say that when bar tries to allocate some-object an OOMC happens
and that it is more important to have some-object than
some-large-object .How do you set things up so that bar deallocates
some-large-object , creates some-object and continues its execution ?

[...]

> Few languages have the equivalent of HANDLER-BIND.

Here we go again with the Lisp vs other laguages comparisons.

-- 
Abstractions = page-faults
  Richard Gabriel in "Money Through Innovation Reconsidered"

Spiros Bousbouras wrote:
> On Sat, 9 Oct 2010 23:07:02 +1100
> "robin" <robin51@dodo.com.au> wrote:
>> "Spiros Bousbouras" <spibou@gmail.com> wrote in message news:uLKro.18876$7B6.6976@newsfe07.ams2...
>> | Consider the following scenario : your programme written in an AMM
>> | language needs to allocate a new object (or objects) but there isn't
>> | enough memory available. The language does garbage collection but there
>> | still isn't enough memory and the operating system refuses to give any
>> | more memory. What I would like to know is how existing languages with
>> | AMM handle such a situation.
>>
>> In PL/I, the run-time raises an exception.
>> It it possible to intercept this exception and to continue executing
>> the program, possibly with a smaller request for memory.
>>
>> | The other question is what would you consider the Right Way to handle
>> | it ? The obvious answer is that the language generates an exception.
>> | Generating an exception may also require allocating a new object which
>> | would be problematic if you're out of memory to begin with. But
>> | the language could at the beginning of execution of each programme
>> | allocate some emergency memory which will only be used to signal
>> | exceptions for out of memory conditions (OOMC).
>>
>> I would expect the error-handling code (written by the programmer)
>> to handle the exception and to print some error message,
>> and also to print values of variables, etc.
>> (as well as, of course, printing the statement number of the
>> offending statement).
> 
> Actually what I meant was what would you consider the Right Way for a
> language to handle it i.e. what facilities should it provide for this
> sort of thing ?  Since you mention PL/I , would you consider ideal the
> way it handles it or can you think of improvements ?

I think PL/I is close to the "right way", but of course I'm biased.  The 
programmer should have the chance of terminating the process or thread 
or of continuing, presumably after handling the error, possibly in a 
different section of code.

The problem with PL/I's handing (IMHO) is that the handler is 
established at run-time rather than lexically.

> 
>> | For an exception to be most useful it needs to contain the information
>> | of which piece of code caused the exception and for OOMCs this is a bit
>> | problematic.
>>
>> In PL/I, the number of the statement that failed to obtain memory
>> can be printed.
> 
> I skimmed through the Wikipedia article on PL/I and it doesn't say what
> the statement number is. The 2 short listings it has don't have
> anything which looks like a statement number. In a line like
> a = b*c + d*e;
> which I think is legal PL/I , what are the statement numbers ?

The statement number is a compiler-generated sequential number for each 
statement (not line) compiled.  Some compilers use the file/line number 
instead of the statement number.

On Fri, 08 Oct 2010 20:00:26 GMT
Spiros Bousbouras <spibou@gmail.com> wrote:
> Consider the following scenario : your programme written in an AMM
> language needs to allocate a new object (or objects) but there isn't
> enough memory available. The language does garbage collection but there
> still isn't enough memory and the operating system refuses to give any
> more memory. What I would like to know is how existing languages with
> AMM handle such a situation.

I'm surprised that there haven't been more replies. Surely there are
people here familiar with Java , Perl , Python , Ruby , etc. Or is this
group dying ? I hadn't been here in months and it seems to have a lot
less activity than what I remember although Google statistics don't
seem that down.

On Mon, 11 Oct 2010 17:16:04 -0400
Peter Flass <Peter_Flass@Yahoo.com> wrote:
> Spiros Bousbouras wrote:
> > On Sat, 9 Oct 2010 23:07:02 +1100
> > "robin" <robin51@dodo.com.au> wrote:
> >> "Spiros Bousbouras" <spibou@gmail.com> wrote in message news:uLKro.18876$7B6.6976@newsfe07.ams2...

> >> | For an exception to be most useful it needs to contain the information
> >> | of which piece of code caused the exception and for OOMCs this is a bit
> >> | problematic.
> >>
> >> In PL/I, the number of the statement that failed to obtain memory
> >> can be printed.
> > 
> > I skimmed through the Wikipedia article on PL/I and it doesn't say what
> > the statement number is. The 2 short listings it has don't have
> > anything which looks like a statement number. In a line like
> > a = b*c + d*e;
> > which I think is legal PL/I , what are the statement numbers ?
> 
> The statement number is a compiler-generated sequential number for each 
> statement (not line) compiled.  Some compilers use the file/line number 
> instead of the statement number.

And a PL/I development environment can give you from the statement
number the corresponding point in the source code , yes ?

Spiros Bousbouras wrote:
> On Mon, 11 Oct 2010 17:16:04 -0400
> Peter Flass <Peter_Flass@Yahoo.com> wrote:
>> Spiros Bousbouras wrote:
>>> On Sat, 9 Oct 2010 23:07:02 +1100
>>> "robin" <robin51@dodo.com.au> wrote:
>>>> "Spiros Bousbouras" <spibou@gmail.com> wrote in message news:uLKro.18876$7B6.6976@newsfe07.ams2...
> 
>>>> | For an exception to be most useful it needs to contain the information
>>>> | of which piece of code caused the exception and for OOMCs this is a bit
>>>> | problematic.
>>>>
>>>> In PL/I, the number of the statement that failed to obtain memory
>>>> can be printed.
>>> I skimmed through the Wikipedia article on PL/I and it doesn't say what
>>> the statement number is. The 2 short listings it has don't have
>>> anything which looks like a statement number. In a line like
>>> a = b*c + d*e;
>>> which I think is legal PL/I , what are the statement numbers ?
>> The statement number is a compiler-generated sequential number for each 
>> statement (not line) compiled.  Some compilers use the file/line number 
>> instead of the statement number.
> 
> And a PL/I development environment can give you from the statement
> number the corresponding point in the source code , yes ?

Yes.

"Spiros Bousbouras" <spibou@gmail.com> wrote in message
news:kdLso.39388$af7.11137@newsfe28.ams2...
> On Fri, 08 Oct 2010 20:00:26 GMT
> Spiros Bousbouras <spibou@gmail.com> wrote:
>> Consider the following scenario : your programme written in an AMM
>> language needs to allocate a new object (or objects) but there isn't
>> enough memory available. The language does garbage collection but there
>> still isn't enough memory and the operating system refuses to give any
>> more memory. What I would like to know is how existing languages with
>> AMM handle such a situation.
>
> I'm surprised that there haven't been more replies. Surely there are
> people here familiar with Java , Perl , Python , Ruby , etc. Or is this
> group dying ? I hadn't been here in months and it seems to have a lot
> less activity than what I remember although Google statistics don't
> seem that down.

Have you tried these languages to see how they handle such an error?

I think that if a machine is out of memory, then it's out of memory. If the
memory allocator is any good, then it will already have tried consolidating
memory to get any more, but that's just delaying tactics.

But might happen instead is that the system might get slow and unstable
before it gives up and admits there is no more (virtual) memory.

I've just tried a big calculation in Python, and it just says MemoryError,
and showing the line where it occurred (and the actual term can be
pinpointed more finely I suppose by splitting  a calculation over several
lines).

If an end-user was running the application, then the error report won't be
much consolation. If the out-of-error was due to some logic error (a
calculation gone out of control), then perhaps the report will be useful to
the programmer.

Where it is not desirable to have an application failing with such an error,
then it must simply be programmed more robustly.

But I'm not sure exceptions are the way to do this. Using lots of logic and
a set of 'safe' operations (which evaluate whether a pending operation will
succeed or not) may not work either; a big operation may succeed but might
have used up 99% of the memory, and the program then falls down on something
simple.

Perhaps the answer might be in partitioning an application into parts, where
if one part may fails, the others can recover (for example when the Python
program failed, it did not bring down the OS, or other applications.
Although in this case, they weren't dependent on the Python app).

-- 
Bartc 

Spiros Bousbouras <spibou@gmail.com> writes:

> On Sat, 09 Oct 2010 03:35:49 +0200
> pjb@informatimago.com (Pascal J. Bourguignon) wrote:
>> Spiros Bousbouras <spibou@gmail.com> writes:
>> 
>> > On Sat, 09 Oct 2010 00:03:19 +0200
>> > pjb@informatimago.com (Pascal J. Bourguignon) wrote:
>> >> Spiros Bousbouras <spibou@gmail.com> writes:
>> >> 
>> >> > The other question is what would you consider the Right Way to handle
>> >> > it ? The obvious answer is that the language generates an exception.
>> >> 
>> >> Yes.
>> >> 
>> >> > Generating an exception may also require allocating a new object which
>> >> > would be problematic if you're out of memory to begin with.  But
>> >> > the language could at the beginning of execution of each programme
>> >> > allocate some emergency memory which will only be used to signal
>> >> > exceptions for out of memory conditions (OOMC).
>> >> 
>> >> Yes.
>> >> 
>> >> 
>> >> > For an exception to be most useful it needs to contain the information
>> >> > of which piece of code caused the exception and for OOMCs this is a bit
>> >> > problematic.  
>> >> 
>> >> Not at all.  That information is already in the memory.
>> >
>> > Executables have a copy of the source code in memory ? That's news to
>> > me.
>> 
>> Well, in lisp we tend to keep the sources around.  Not always, but more
>> often than with other programming languages.
>
> So the information is *not* in memory contrary to what you say above.


CL-USER> (function-lambda-expression 'hash-tree-get)
(LAMBDA (TREE &REST KEYS) (DECLARE (SYSTEM::IN-DEFUN HASH-TREE-GET))
 (BLOCK HASH-TREE-GET
  (LABELS
   ((WALK (HASH KEYS)
     (COND ((NULL KEYS) HASH) ((NULL HASH) (HASH-TREE-DEFAULT TREE))
      (T (WALK (GETHASH (FIRST KEYS) HASH) (REST KEYS))))))
   (WALK (HASH-TREE-TABLE TREE) KEYS))))
#(NIL NIL NIL NIL
  ((DECLARATION ALSO-USE-PACKAGES XLIB::ARRAY-REGISTER XLIB::INDENTATION
    XLIB::ARGLIST XLIB::CLX-VALUES OPTIMIZE DECLARATION)))
HASH-TREE-GET
CL-USER> 


Looks like it was in memory.  It didn't stop to do slow I/O.


> I would also appreciate it if you stop trying to turn this into a Lisp
> vs other languages comparison.

Yes, I understand Lisp's shaking your assumptions...


>> > Wrapping every form which has the potential of allocating memory inside
>> > an exception handler isn't practical. I don't think that the Common
>> > Lisp standard specifies which operations might allocate memory so every
>> > single Common Lisp form is a candidate.
>> 
>> With HANDLER-BIND, you would actually set your handler only at the point
>> of the dynamic countour of resource management.  
>> 
>> For example, suppose you set up a cache of web page in an web spider
>> application.  You only need to set up the handler once for all in the
>> main, when you've initialized the cache.
>> 
>> Something like:
>> 
>> (defmacro with-page-cache (cache-var &body body)
>>   `(let ((,cache-var (make-cache)))
>>      (unwind-protect
>>         (handler-bind
>>            ((storage-condition 
>>               (lambda (err)
>>                 (let ((required-size (compute-required-size-from-condition err)))
>>                   (cache-free-space ,cache-var required-size)))))
>>           (progn ,@body))
>>       (cache-free ,cache-var))))
>> 
>> 
>> (defun main ()
>>   (with-page-cache pcache
>>     (walk-the-web)))
>> 
>
> Your example doesn't enlighten me.

In English, my example shows that you don't have to wrap every form
which has the potential of allocating memory inside an exception
handler.  Just shaking yet another of your assumptions, sorry.



> How about we do the one I mentioned ?
> So we have the form  (+ (* b c) (* d e))
> Each of the 3 forms has the potential of causing an OOMC. If it
> happens we would , ideally , want to know the specific form which
> caused it. Can we do it with just one HANDLER-BIND or would we have to
> put one around each form ?

It would depend on various things, such as the implementation, the level
of optimization and debugging requested when compiling.  But in anycase,
the information is in the memory and could be retrieved, either
inspecting the stack frames or the program counter.


For example, so you may see the control flow, and the data flow:


(define-condition oomc (error)
  ((var :initarg :var :accessor oomc-var))
  (:report (lambda (oomc stream)
             (format stream "~S trying to allocate ~S"
                     'oomc
                     (refname (oomc-var oomc))))))

(defmacro & (var)
  `(lambda (m &optional val)
     (case m
       (:name ',var)
       (:get  ,var)
       (:set  (setf ,var val)))))
(defun deref (loc) (funcall loc :get))
(defun (setf deref) (value loc) (funcall loc :set value))
(defun refname (loc) (funcall loc :name))


(handler-bind
    ((oomc (lambda (err)
             (princ err) (terpri)
             (princ "Let's free some space.") (terpri)
             (setf (deref (oomc-var err)) (random 10))
             (invoke-restart 'continue))))
  (let (b c d e)
    (print (+ (* (progn (cerror "retry allocating b after clean up" 'oomc :var (& b)) b)
                 (progn (cerror "retry allocating c after clean up" 'oomc :var (& c)) c))
              (* (progn (cerror "retry allocating d after clean up" 'oomc :var (& d)) d)
                 (progn (cerror "retry allocating e after clean up" 'oomc :var (& e)) e))))))

OOMC trying to allocate B
Let's free some space.
OOMC trying to allocate C
Let's free some space.
OOMC trying to allocate D
Let's free some space.
OOMC trying to allocate E
Let's free some space.

34 


With the PROGN forms inside the arithmetic expression, I simulate OOM
conditions, like an implementation could do, signaling a continuable
error, giving enough details explicitely as slots of the condition (but
the handler could also use introspection of the stack to detect what
occured and where).

The variables are initialized to NIL by the LET form.  The continuable
error condition is caught by the handler function installed by
handler-bind: that function (an anonymous function introduced by LAMBDA
in my example) is called at the point where the CERROR form is.  You may
imagine it's called inside CERROR.  In my example, the handler prints a
message, and initialize the problematic variable with a random integer.

The evaluation restarts then with the continue restart which go on
executing the program after the CERROR call, but now the variable has a
value, which is used in the arithmetic expression and gives eventually
a result.

The same mechanism can be used for real OOM conditions, as long as
precautions are taken with respect to the use of memory in the condition
handling and condition handlers.



> I don't see how during runtime an application can locate all the
> objects it has created never mind actually assess which are needed.

Yes, that's the problem.  You don't see.

However, garbage collectors, which are parts of Lisp applications
amongst others, have no problem in tracking all the objects created, and
in assessing which are needed.

Perhaps you could learn some about them?



> Only
> during writing the code the programmer knows that if at point A an
> object cannot be created due to lack of memory , then it is better to
> lose some previously created object (and which one) than not creating
> the new one.

Exactly!  That's why the memory manager just cannot decide what object
to delete.  It must have a hook to call up the application to do so.
This is what the condition handler is: application provided code to let
the memory manager further clean up the memory when it cannot find
enough space for the next allocation.



>> > During the execution of bar an OOMC happens. But bar doesn't know about
>> > var2 so it cannot free the object. You can return to foo but then the
>> > stack has unwound which might not be desirable. So you have a problem.
>> 
>> That's why you'd use HANDLER-BIND instead of HANDLER-CASE.
>
> Ok , consider this:
>
> (defvar *var* some-large-object)
>
> (defun bar (...)
>     (let ((a some-object))
>     ...code...))
>
> (defun foo (...)
>     (let ((*var* other-object))
>         ...code...
>         (bar)
>         ...more code...))
>
> Let's say that when bar tries to allocate some-object an OOMC happens
> and that it is more important to have some-object than
> some-large-object .How do you set things up so that bar deallocates
> some-large-object , creates some-object and continues its execution ?

Obviously, if you have a cache of disposable objects that you want to
dispose of, you don't hide it with a local dynamic binding like you do
in foo.

More over, what you want to dispose is probably not a monolithic object,
but some kind of structure (eg. a cache), where you throw away only a
disposable part, either because you can get the data from some secondary
storage, from the network or by recomputing it.


Any problem in computers can always be solved with one more indirection.
So you could do something like:


(setf *var* (make-proxy *var*))
(let ((disposable-proxy *var*))
  (handler-bind
    ((oomc (lambda (err)
              (when (proxy-object disposable-proxy)
                (setf (proxy-object disposable-proxy) nil) 
                (invoke-restart 'try-garbage-collection again)))))
   (foo)))




>> Few languages have the equivalent of HANDLER-BIND.
>
> Here we go again with the Lisp vs other laguages comparisons.

Not my fault.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

"Spiros Bousbouras" <spibou@gmail.com> wrote in message news:86Iso.24175$_x2.4217@newsfe30.ams2...
| On Sat, 9 Oct 2010 23:07:02 +1100
| "robin" <robin51@dodo.com.au> wrote:
| > "Spiros Bousbouras" <spibou@gmail.com> wrote in message news:uLKro.18876$7B6.6976@newsfe07.ams2...
| > | Consider the following scenario : your programme written in an AMM
| > | language needs to allocate a new object (or objects) but there isn't
| > | enough memory available. The language does garbage collection but there
| > | still isn't enough memory and the operating system refuses to give any
| > | more memory. What I would like to know is how existing languages with
| > | AMM handle such a situation.
| >
| > In PL/I, the run-time raises an exception.
| > It it possible to intercept this exception and to continue executing
| > the program, possibly with a smaller request for memory.
| >
| > | The other question is what would you consider the Right Way to handle
| > | it ? The obvious answer is that the language generates an exception.
| > | Generating an exception may also require allocating a new object which
| > | would be problematic if you're out of memory to begin with. But
| > | the language could at the beginning of execution of each programme
| > | allocate some emergency memory which will only be used to signal
| > | exceptions for out of memory conditions (OOMC).
| >
| > I would expect the error-handling code (written by the programmer)
| > to handle the exception and to print some error message,
| > and also to print values of variables, etc.
| > (as well as, of course, printing the statement number of the
| > offending statement).
|
| Actually what I meant was what would you consider the Right Way for a
| language to handle it i.e. what facilities should it provide for this
| sort of thing ?  Since you mention PL/I , would you consider ideal the
| way it handles it or can you think of improvements ?

I regard the PL/I design for handling this type of error
as being excellent.  PL/I provdes the means to detect the error,
and the PL/I programmer is able to write code to print error messages
and values of variables, and the statement number where the error occurred.
It also provides the means to recover from the error and to continue execution.

However, for this particular error (out of memory) it's possible
that there is not sufficient memory to run the error handler.  That would be
rare, but theoretically could happen.  With such large memories that
are currently available, such a problem would be rare.
Nevertheless, to avoid it, the run-time should always
keep in reserve a separate piece of memory strictly for the error handler.

   Keep in mind also that when a request for memory cannot be satisfied,
there is typically some memory still available (but less that what was
requested).  This ought still be enough to run the error handler.

| > | For an exception to be most useful it needs to contain the information
| > | of which piece of code caused the exception and for OOMCs this is a bit
| > | problematic.
| >
| > In PL/I, the number of the statement that failed to obtain memory
| > can be printed.
|
| I skimmed through the Wikipedia article on PL/I and it doesn't say what
| the statement number is. The 2 short listings it has don't have
| anything which looks like a statement number. In a line like
| a = b*c + d*e;
| which I think is legal PL/I , what are the statement numbers ?

A statement is a piece of code that is a unit, such as your assignment statement
a = b*c + d+e; .
In PL/I, it generally corresponds to that code falling between two semicolons.
Hence, a statement number is a number associated with each statement;
which is 1 for the first statement and increasing by 1 for each subsequent statement.

| > |  Imagine for example that the code contains an expression
| > | like b*c + d*e .  If the language can handle arbitrarily large numbers
| > | then any of the operations can cause an OOMC if the runtime environment
| > | needs to allocate extra memory to hold the result. For a compiled
| > | language it would be a bit difficult to add enough additional
| > | information to the executable so that it can be determined at runtime
| > | which operation caused the OOMC i.e. whether it was b*c or d*e or the
| > | addition. I'm guessing that it would also make the executable more
| > | slow.
| >
| > | So perhaps the best practical solution is that for OOMCs the exception
| > | does not contain the information of which piece of code caused the
| > | condition or does not pinpoint the code very closely ;
| >
| > In PL/I, the location in the object program is normally available,
| > and can be printed.
| > However, more useful for that class of error is the statement number
| > of the offending statement, which can be printed.
|
| In <4cb1be12$0$33792$c30e37c6@exi-reader.telstra.net> you say:
| "However, statement numbers can be stored with the
| executable, and thus tell the user where the exception occurred."
| Does it make the executable slower if you choose to do this ? (I'm
| assuming you're still talking about PL/I.)

It does make the executable slower, but trivially.  All that's required
is a store instruction that stores an integer (statement number)
at the start of the object code for each statement. 

"Spiros Bousbouras" <spibou@gmail.com> wrote in message news:ijLso.39415$af7.16557@newsfe28.ams2...
| On Mon, 11 Oct 2010 17:16:04 -0400
| Peter Flass <Peter_Flass@Yahoo.com> wrote:
| > Spiros Bousbouras wrote:
| > > On Sat, 9 Oct 2010 23:07:02 +1100
| > > "robin" <robin51@dodo.com.au> wrote:
| > >> "Spiros Bousbouras" <spibou@gmail.com> wrote in message news:uLKro.18876$7B6.6976@newsfe07.ams2...
|
| > >> | For an exception to be most useful it needs to contain the information
| > >> | of which piece of code caused the exception and for OOMCs this is a bit
| > >> | problematic.
| > >>
| > >> In PL/I, the number of the statement that failed to obtain memory
| > >> can be printed.
| > >
| > > I skimmed through the Wikipedia article on PL/I and it doesn't say what
| > > the statement number is. The 2 short listings it has don't have
| > > anything which looks like a statement number. In a line like
| > > a = b*c + d*e;
| > > which I think is legal PL/I , what are the statement numbers ?
| >
| > The statement number is a compiler-generated sequential number for each
| > statement (not line) compiled.  Some compilers use the file/line number
| > instead of the statement number.
|
| And a PL/I development environment can give you from the statement
| number the corresponding point in the source code , yes ?

You can get an error message such as
"occurred at statement 25".
Alternatively, "at line 34." 

"BartC" <bc@freeuk.com> wrote in message news:i902rf$b8u$1@news.eternal-september.org...
| "Spiros Bousbouras" <spibou@gmail.com> wrote in message
| news:kdLso.39388$af7.11137@newsfe28.ams2...
| > On Fri, 08 Oct 2010 20:00:26 GMT
| > Spiros Bousbouras <spibou@gmail.com> wrote:
| >> Consider the following scenario : your programme written in an AMM
| >> language needs to allocate a new object (or objects) but there isn't
| >> enough memory available. The language does garbage collection but there
| >> still isn't enough memory and the operating system refuses to give any
| >> more memory. What I would like to know is how existing languages with
| >> AMM handle such a situation.
| >
| > I'm surprised that there haven't been more replies. Surely there are
| > people here familiar with Java , Perl , Python , Ruby , etc. Or is this
| > group dying ? I hadn't been here in months and it seems to have a lot
| > less activity than what I remember although Google statistics don't
| > seem that down.
|
| Have you tried these languages to see how they handle such an error?
|
| I think that if a machine is out of memory, then it's out of memory. If the
| memory allocator is any good, then it will already have tried consolidating
| memory to get any more, but that's just delaying tactics.

Yes, byt keep in mind that when the system is out of memory,
it means that the current request cannot be met.  Let's say that
the program requested N bytes of memory.
Then the actual memory remaining unused is somewhere between 0 and N-1 bytes.
Thus, an error handler may well stilll run.

| But might happen instead is that the system might get slow and unstable
| before it gives up and admits there is no more (virtual) memory.
|
| I've just tried a big calculation in Python, and it just says MemoryError,
| and showing the line where it occurred (and the actual term can be
| pinpointed more finely I suppose by splitting  a calculation over several
| lines).
|
| If an end-user was running the application, then the error report won't be
| much consolation.

True, but at least he got an error message saying what the problem is.

| If the out-of-error was due to some logic error (a
| calculation gone out of control), then perhaps the report will be useful to
| the programmer.

Indeed.

| Where it is not desirable to have an application failing with such an error,
| then it must simply be programmed more robustly.

Murphy's Law still operates.

| But I'm not sure exceptions are the way to do this. Using lots of logic and
| a set of 'safe' operations (which evaluate whether a pending operation will
| succeed or not) may not work either; a big operation may succeed but might
| have used up 99% of the memory, and the program then falls down on something
| simple.
|
| Perhaps the answer might be in partitioning an application into parts, where
| if one part may fails, the others can recover (for example when the Python
| program failed, it did not bring down the OS, or other applications.
| Although in this case, they weren't dependent on the Python app). 

On 10/12/2010 03:17 AM, robin wrote:

> |
> | If an end-user was running the application, then the error report won't be
> | much consolation.
>
> True, but at least he got an error message saying what the problem is.

Oh, fun fun fun. I can hear my wife (back when she was on windows): 
"Hey, I got a Send - Don't send again, please fix my computer once and 
for all!"

;)



-- 

Zlatko

"robin" <robin51@dodo.com.au> wrote in message
news:4cb3b4be$0$55987$c30e37c6@exi-reader.telstra.net...
> "Spiros Bousbouras" <spibou@gmail.com> wrote in message
> news:86Iso.24175$_x2.4217@newsfe30.ams2...

> | In <4cb1be12$0$33792$c30e37c6@exi-reader.telstra.net> you say:
> | "However, statement numbers can be stored with the
> | executable, and thus tell the user where the exception occurred."
> | Does it make the executable slower if you choose to do this ? (I'm
> | assuming you're still talking about PL/I.)
>
> It does make the executable slower, but trivially.  All that's required
> is a store instruction that stores an integer (statement number)
> at the start of the object code for each statement.

That sounds like a significant slowdown to me. I used that approach in an
interpreter, and it cost at least 5%. Probably more it tight, compiled code 
where it also increases code size.

But it shouldn't be necessary: it should be easy enough to lookup the
address of the instruction where the error occurred, and match to a source
file/line number. The table required doesn't even need to be part of the
executable.

(This does require the call-stack to be reconstructed, as the actual error
may be remote from the application code. So some extra instructions may be
required for call/return.)

-- 
Bartc 

"robin" <robin51@dodo.com.au> wrote in message 
news:4cb3b72f$0$55991$c30e37c6@exi-reader.telstra.net...
> "BartC" <bc@freeuk.com> wrote in message 
> news:i902rf$b8u$1@news.eternal-september.org...
> | "Spiros Bousbouras" <spibou@gmail.com> wrote in message
> | news:kdLso.39388$af7.11137@newsfe28.ams2...
<snip>
> | I think that if a machine is out of memory, then it's out of memory. If 
> the
> | memory allocator is any good, then it will already have tried 
> consolidating
> | memory to get any more, but that's just delaying tactics.
>
> Yes, byt keep in mind that when the system is out of memory,
> it means that the current request cannot be met.  Let's say that
> the program requested N bytes of memory.
> Then the actual memory remaining unused is somewhere between 0 and N-1 
> bytes.
> Thus, an error handler may well stilll run.
>
One thing that I don't think I have seen mentioned in this thread is that 
the run-time environment (Language Environment in the case of PL/I on z/OS) 
may actually "reserve" some amount of storage for the condition handler to 
use - even in the case of an out-of-storage situation.  See (for example),

http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/CEEA31B0/1.2.57

and look for
  " reserve_size"

Obviously, whether this is useful or not will depend on both the application 
and the condition handler 

I'm not sure about the mainframe version of IBM's PL/I, but the
Windows version has never provided a statement number of any detected
exception. The early documentation (maybe the current) had compiler
options to instruct the compiler to include a statement table so that
one could be provided, but is was never implemented. The SNAP option
will cause procedure prologs and epilogs to keep track of the call
stack, providing more useful information when an exception occurs. You
will at least know the offset within the procedure where the exception
occurs. The programmers guide says that the SNAP option can
"significantly increase the size and reduce the performance of your
programs". Hooey! I have never been able to measure much difference. I
think this is language carried forward from the 1960s! Of course, if
you like itty-bitty procudures and lots of calls -- well performance
is not that much of an issue for you anyway.

I wrote to Peter Elderon in the 1990s begging for statement numbers
for exceptions. It wasn't going to happen, so I wrote a program that
post-processed the generated code listing (COD) and appended a
statement offset table to the end of the normal listing (LST). I
called the program COD2OFF, sent it to Peter and they have
incorporated into the deliverable ever since. Might be in the SAMPLES
or EXAMPLES folder -- something like that. I don't know if it's still
there, but it was when I retired at the end of 2007. They put their
"marks" on it -- must be something about not using contributed code
asis because the program does exactly the same thing as the one I
sent. It's just more sloppy code! Armed with this table, which was
what the old PL/I F compiler (I think) provided in the 1960s, you have
all you need in the listing to isolate the statement that caused the
error.

Back to that running-out-of-memory issue. I have a program that I run
on Windows that is nothing but an memory allocation loop, that
allocates 1M chunks. It has an ON STORAGE block that displays how much
memory could be allocated. I used this when running on different
versions of Windows to find out how much memory was actually
available, because, as you must know, it has varied, especially in the
past. It has never failed to display the amount. You could easily
write such a program to reduce the allocation size in the ON STORAGE
block to 1K, 512, 256, etc., keep going and see if PL/I could still
trap the exception and display the message.

John Rausch wrote:
> 
> Back to that running-out-of-memory issue. I have a program that I run
> on Windows that is nothing but an memory allocation loop, that
> allocates 1M chunks. It has an ON STORAGE block that displays how much
> memory could be allocated. I used this when running on different
> versions of Windows to find out how much memory was actually
> available, because, as you must know, it has varied, especially in the
> past. It has never failed to display the amount. You could easily
> write such a program to reduce the allocation size in the ON STORAGE
> block to 1K, 512, 256, etc., keep going and see if PL/I could still
> trap the exception and display the message.
> 

There is a difference between stack space and heap space.  I would 
assume the IBM compiler gets "allocated" data from the heap.  Stack 
space is usually saved registers for calls plus AUTO storage (again, 
YMMV).  To check the stack you want something like a never-ending 
recursive call to a procedure with a reasonable amount of AUTO storage, 
and count the number of calls before an exception is raised.  Does the 
IBM compiler raise STORAGE when it runs out of stack space?  Exception 
handling in this case is more problematic.

John Rausch wrote:
> 
> Back to that running-out-of-memory issue. I have a program that I run
> on Windows that is nothing but an memory allocation loop, that
> allocates 1M chunks. It has an ON STORAGE block that displays how much
> memory could be allocated. I used this when running on different
> versions of Windows to find out how much memory was actually
> available, because, as you must know, it has varied, especially in the
> past. It has never failed to display the amount. You could easily
> write such a program to reduce the allocation size in the ON STORAGE
> block to 1K, 512, 256, etc., keep going and see if PL/I could still
> trap the exception and display the message.
> 

There is a difference between stack space and heap space.  I would 
assume the IBM compiler gets "allocated" data from the heap.  Stack 
space is usually saved registers for calls plus AUTO storage (again, 
YMMV).  To check the stack you want something like a never-ending 
recursive call to a procedure with a reasonable amount of AUTO storage, 
and count the number of calls before an exception is raised.  Does the 
IBM compiler raise STORAGE when it runs out of stack space?  Exception 
handling in this case is more problematic.

In comp.lang.pl1 Spiros Bousbouras <spibou@gmail.com> wrote:
> On Sat, 9 Oct 2010 23:07:02 +1100
> "robin" <robin51@dodo.com.au> wrote:
>> "Spiros Bousbouras" <spibou@gmail.com> wrote in message news:uLKro.18876$7B6.6976@newsfe07.ams2...
>> | Consider the following scenario : your programme written in an AMM
>> | language needs to allocate a new object (or objects) but there isn't
>> | enough memory available. 
(snip)

>> In PL/I, the run-time raises an exception.
>> It it possible to intercept this exception and to continue executing
>> the program, possibly with a smaller request for memory.

In most cases, that is a fine model, but it is a problem with
memory allocation.  It usually takes some memory to process the
exception, which might also not be available, in which case it
also fails.

(snip, someone wrote)
 
>> In PL/I, the number of the statement that failed to obtain memory
>> can be printed.
 
> I skimmed through the Wikipedia article on PL/I and it doesn't say what
> the statement number is. The 2 short listings it has don't have
> anything which looks like a statement number. In a line like
> a = b*c + d*e;
> which I think is legal PL/I , what are the statement numbers ?

In the batch processing days, it was usual for the compiler
to generate a listing file with line numbers and statement numbers
as it compiled.  (It was optional, but the default was usually yes.)

The PL/I (F) compiler, and likely other IBM compilers, could also
generate code to keep track of the current statement at run time.
For PL/I (F) that is done with MVI instructions between each 
statment.  Even more, with the M91 option it generates BR 0
instructions, which are a special type of NO-OP.  Pipelined
processors are supposed to flush the pipeline before executing
the next instruction.  Otherwise, you get a message like:

ERROR OCCURRED NEAR STATEMENT NUMBER 123.

(snip)
>> In PL/I, the location in the object program is normally available,
>> and can be printed.
>> However, more useful for that class of error is the statement number
>> of the offending statement, which can be printed.
 
> In <4cb1be12$0$33792$c30e37c6@exi-reader.telstra.net> you say:
> "However, statement numbers can be stored with the
> executable, and thus tell the user where the exception occurred."
> Does it make the executable slower if you choose to do this ? (I'm
> assuming you're still talking about PL/I.)

Some systems keep a table off offsets, such that the statement
number can be computed from the offset.  Though with optimizers
that doesn't work so well.  Storing the number somewhere between
each statement is a common way.

-- glen

In comp.lang.pl1 Peter Flass <Peter_Flass@yahoo.com> wrote:
(snip)

> There is a difference between stack space and heap space.  I would 
> assume the IBM compiler gets "allocated" data from the heap.  

IBM S/360 and S/370 don't have a stack.  As the overhead for GETMAIN
is fairly high, the library does a GETMAIN for a reasonable size,
and parcels that out as needed, for smaller allocations such as
AUTOMATIC variables and save areas.  

> Stack 
> space is usually saved registers for calls plus AUTO storage (again, 
> YMMV).  To check the stack you want something like a never-ending 
> recursive call to a procedure with a reasonable amount of AUTO storage, 
> and count the number of calls before an exception is raised.  Does the 
> IBM compiler raise STORAGE when it runs out of stack space?  Exception 
> handling in this case is more problematic.

-- glen

glen herrmannsfeldt wrote:
> In comp.lang.pl1 Peter Flass <Peter_Flass@yahoo.com> wrote:
> (snip)
> 
>> There is a difference between stack space and heap space.  I would 
>> assume the IBM compiler gets "allocated" data from the heap.  
> 
> IBM S/360 and S/370 don't have a stack.  As the overhead for GETMAIN
> is fairly high, the library does a GETMAIN for a reasonable size,
> and parcels that out as needed, for smaller allocations such as
> AUTOMATIC variables and save areas.  
> 

Yes, but the OP was discussing windoze compiler (IIRC) so I replied to that.

"John Rausch" <john@johnrausch.com> wrote in message 
news:72764fb8-fe23-4f7e-89c9-5de82a320bb1@i21g2000yqg.googlegroups.com...

| Back to that running-out-of-memory issue. I have a program that I run
| on Windows that is nothing but an memory allocation loop, that
| allocates 1M chunks. It has an ON STORAGE block that displays how much
| memory could be allocated. I used this when running on different
| versions of Windows to find out how much memory was actually
| available, because, as you must know, it has varied, especially in the
| past. It has never failed to display the amount. You could easily
| write such a program to reduce the allocation size in the ON STORAGE
| block to 1K, 512, 256, etc., keep going and see if PL/I could still
| trap the exception and display the message.

Would the Windows and OS/2 versions use the stack for run-time
storage?  Would this be different from storage obtained via ALLOCATE, etc? 

"glen herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message news:i92f5d$u5e$2@speranza.aioe.org...
| In comp.lang.pl1 Peter Flass <Peter_Flass@yahoo.com> wrote:
| (snip)
|
| > There is a difference between stack space and heap space.  I would
| > assume the IBM compiler gets "allocated" data from the heap.
|
| IBM S/360 and S/370 don't have a stack.  As the overhead for GETMAIN
| is fairly high, the library does a GETMAIN for a reasonable size,
| and parcels that out as needed, for smaller allocations such as
| AUTOMATIC variables and save areas.

The current mainframe compilers use /z archicture.  S/360 is 1960s and is out-of-date. 

"glen herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message news:i92erc$u5e$1@speranza.aioe.org...
| In comp.lang.pl1 Spiros Bousbouras <spibou@gmail.com> wrote:
| > On Sat, 9 Oct 2010 23:07:02 +1100
| > "robin" <robin51@dodo.com.au> wrote:
| >> "Spiros Bousbouras" <spibou@gmail.com> wrote in message news:uLKro.18876$7B6.6976@newsfe07.ams2...
| >> | Consider the following scenario : your programme written in an AMM
| >> | language needs to allocate a new object (or objects) but there isn't
| >> | enough memory available.
| (snip)
|
| >> In PL/I, the run-time raises an exception.
| >> It it possible to intercept this exception and to continue executing
| >> the program, possibly with a smaller request for memory.
|
| In most cases, that is a fine model, but it is a problem with
| memory allocation.  It usually takes some memory to process the
| exception, which might also not be available, in which case it
| also fails.

On the PC, invoking the error-handler would use the stack.
I expect that this space is not used for general memory requests
such as would be initiated by ALLOCATE etc statements.
However, if the error handler is not in memory, it would have to be
brought in (DLL), in which case there would need to be sufficient general memory
for it to be loaded. 

In <ijLso.39415$af7.16557@newsfe28.ams2>, on 10/11/2010
   at 09:27 PM, Spiros Bousbouras <spibou@gmail.com> said:

>And a PL/I development environment can give you from the statement
>number the corresponding point in the source code , yes ?

That depends on the compiler options used.

-- 
Shmuel (Seymour J.) Metz, SysProg and JOAT  <http://patriot.net/~shmuel>

Unsolicited bulk E-mail subject to legal action.  I reserve the
right to publicly post or ridicule any abusive E-mail.  Reply to
domain Patriot dot net user shmuel+news to contact me.  Do not
reply to spamtrap@library.lspace.org

In <i91cin$dqh$1@news.eternal-september.org>, on 10/12/2010
   at 11:15 AM, "BartC" <bc@freeuk.com> said:

>(This does require the call-stack to be reconstructed,

Examined, yes, but not reconstructed; raising an exception does not
pop the stack.

>So some extra instructions may be
>required for call/return.)

At worst storing one address on the stack. The actual table should be
out of line.

-- 
Shmuel (Seymour J.) Metz, SysProg and JOAT  <http://patriot.net/~shmuel>

Unsolicited bulk E-mail subject to legal action.  I reserve the
right to publicly post or ridicule any abusive E-mail.  Reply to
domain Patriot dot net user shmuel+news to contact me.  Do not
reply to spamtrap@library.lspace.org

In <4cb3b4be$0$55987$c30e37c6@exi-reader.telstra.net>, on 10/12/2010
   at 12:07 PM, "robin" <robin51@dodo.com.au> said:

>All that's required is a store instruction that stores an integer
>(statement number) at the start of the object code for each 
>statement. 

It doesn't even require that, although that is the easiest
implementation.

-- 
Shmuel (Seymour J.) Metz, SysProg and JOAT  <http://patriot.net/~shmuel>

Unsolicited bulk E-mail subject to legal action.  I reserve the
right to publicly post or ridicule any abusive E-mail.  Reply to
domain Patriot dot net user shmuel+news to contact me.  Do not
reply to spamtrap@library.lspace.org

In <i92f5d$u5e$2@speranza.aioe.org>, on 10/12/2010
   at 08:07 PM, glen herrmannsfeldt <gah@ugcs.caltech.edu> said:

>IBM S/360 and S/370 don't have a stack. 

Of course it does; it's just not implemented as an array.

>As the overhead for GETMAIN
>is fairly high, the library does a GETMAIN for a reasonable size,
>and parcels that out as needed,

That's what the "Optimizing" compiler did, but the earlier F compiler
did a GETMAIN for each DSA.

-- 
Shmuel (Seymour J.) Metz, SysProg and JOAT  <http://patriot.net/~shmuel>

Unsolicited bulk E-mail subject to legal action.  I reserve the
right to publicly post or ridicule any abusive E-mail.  Reply to
domain Patriot dot net user shmuel+news to contact me.  Do not
reply to spamtrap@library.lspace.org

"Shmuel (Seymour J.) Metz" <spamtrap@library.lspace.org.invalid> wrote in message 
news:4cb88e19$1$fuzhry+tra$mr2ice@news.patriot.net...
| In <i92f5d$u5e$2@speranza.aioe.org>, on 10/12/2010
|   at 08:07 PM, glen herrmannsfeldt <gah@ugcs.caltech.edu> said:
|
| >IBM S/360 and S/370 don't have a stack.
|
| Of course it does; it's just not implemented as an array.

S/360 doesn't have a hardware stack.
In software, anything's possible.
And in PL/I, successive allocations of CONTROLLED arrays,
for example, are stacked. 

In comp.lang.pl1 robin <robin51@dodo.com.au> wrote:
(snip)

> And in PL/I, successive allocations of CONTROLLED arrays,
> for example, are stacked. 

Are you sure that they are stacked, and not heaped?

-- glen

LS0tLS1CRUdJTiBQR1AgU0lHTkVEIE1FU1NBR0UtLS0tLQ0KSGFzaDogU0hBMQ0KDQpPbiBTYXQs
IDE2IE9jdCAyMDEwIDA5OjA3OjQzICswMDAwIChVVEMpLCBnbGVuIGhlcnJtYW5uc2ZlbGR0IHdy
b3RlDQphYm91dCBSZTogSG93IGRvIGxhbmd1YWdlcyB3aXRoIGF1dG9tYXRpYyBtZW1vcnkgbWFu
YWdlbWVudCAoQU1NKQ0KaGFuZGxlIG91dCBvZiBtZW1vcnkgY29uZGl0aW9ucyA/Og0KDQo+SW4g
Y29tcC5sYW5nLnBsMSByb2JpbiA8cm9iaW41MUBkb2RvLmNvbS5hdT4gd3JvdGU6DQo+KHNuaXAp
DQo+DQo+PiBBbmQgaW4gUEwvSSwgc3VjY2Vzc2l2ZSBhbGxvY2F0aW9ucyBvZiBDT05UUk9MTEVE
IGFycmF5cywNCj4+IGZvciBleGFtcGxlLCBhcmUgc3RhY2tlZC4gDQo+DQo+QXJlIHlvdSBzdXJl
IHRoYXQgdGhleSBhcmUgc3RhY2tlZCwgYW5kIG5vdCBoZWFwZWQ/DQoNClRoZXkgYXJlIHN0YWNr
ZWQsIExJRk8gc3R5bGUuDQoNClNpbmNlIHdlJ3JlIHRhbGtpbmcgT1MvMzYwIG9ud2FyZHMsIGVh
Y2ggQ09OVFJPTExFRCB2YXJpYWJsZSBoYXMgaXRzDQpvd24gc3RhY2ssIGFuY2hvcmVkIGJ5IHBz
ZXVkby1yZWdpc3RlciAoYS5rLmEuIGV4dGVybmFsIGR1bW15IHNlY3Rpb24pLg0KVGhlIHBoeXNp
Y2FsIGFsbG9jYXRpb25zIGFyZSBvbiB0aGUgaGVhcCwgYnV0IHRoZXkgYXJlIG1hbmFnZWQgdGhy
b3VnaCBhDQpMSUZPIHN0YWNrIHNwZWNpZmljIHRvIHRoZSB2YXJpYWJsZS4NCg0KT24gbm9uLW1h
aW5mcmFtZSBhcmNoaXRlY3R1cmVzIHRoZSBtZWNoYW5pc20gbWlnaHQgYmUgZGlmZmVyZW50LCBi
dXQgaXQNCnN0aWxsIGhhcyB0byBiZWhhdmUgbGlrZSBhIExJRk8gc3RhY2suDQotIC0tIA0KUmVn
YXJkcywNCg0KRGF2ZSAgW1JMVSAjMzE0NDY1XQ0KKi0qLSotKi0qLSotKi0qLSotKi0qLSotKi0q
LSotKi0qLSotKi0qLSotKi0qLSotKi0qLSotKi0qLSotKi0qLSotKi0qLSoNCmR3bm9vbkBzcGFt
dHJhcC5udGx3b3JsZC5jb20gKERhdmlkIFcgTm9vbikNClJlbW92ZSBzcGFtIHRyYXAgdG8gcmVw
bHkgYnkgZS1tYWlsLg0KKi0qLSotKi0qLSotKi0qLSotKi0qLSotKi0qLSotKi0qLSotKi0qLSot
Ki0qLSotKi0qLSotKi0qLSotKi0qLSotKi0qLSoNCi0tLS0tQkVHSU4gUEdQIFNJR05BVFVSRS0t
LS0tDQpWZXJzaW9uOiBHbnVQRyB2Mi4wLjE2IChHTlUvTGludXgpDQoNCmlFWUVBUkVDQUFZRkFr
eTVrSjBBQ2drUTlNcWFVSlF3Mk1sYXp3Q2dtazkrK1dxVFAvNE1SR1V4RDhPKzhPVzANCmxXNEFu
MW9iaVI0S25kMXRtNnJWWWNVWTkwbTVYV1NrDQo9Rkt3WA0KLS0tLS1FTkQgUEdQIFNJR05BVFVS
RS0tLS0tDQo=

"glen herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message news:i9bq0v$h61$1@news.eternal-september.org...
| In comp.lang.pl1 robin <robin51@dodo.com.au> wrote:
| (snip)
|
| > And in PL/I, successive allocations of CONTROLLED arrays,
| > for example, are stacked.
|
| Are you sure that they are stacked, and not heaped?

From the point of view of the programmer,
they are stacked. 

"Shmuel (Seymour J.) Metz" <spamtrap@library.lspace.org.invalid> wrote in message 
news:4cb87782$2$fuzhry+tra$mr2ice@news.patriot.net...
| In <4cb3b4be$0$55987$c30e37c6@exi-reader.telstra.net>, on 10/12/2010
|   at 12:07 PM, "robin" <robin51@dodo.com.au> said:
|
| >All that's required is a store instruction that stores an integer
| >(statement number) at the start of the object code for each
| >statement.
|
| It doesn't even require that, although that is the easiest
| implementation.

I mentioned that because it is the easiest and simplest implementation,
namely, to emit one instruction to store the statemnet number somewhere.
Other schemes are more involved.
Even when code (being part of a source statement) is moved (e.g., out of a loop, etc),
that code can be preceded by an instruction to store the statement number. 

"BartC" <bc@freeuk.com> wrote in message news:i91cin$dqh$1@news.eternal-september.org...
| "robin" <robin51@dodo.com.au> wrote in message
| news:4cb3b4be$0$55987$c30e37c6@exi-reader.telstra.net...
| > "Spiros Bousbouras" <spibou@gmail.com> wrote in message
| > news:86Iso.24175$_x2.4217@newsfe30.ams2...
|
| > | In <4cb1be12$0$33792$c30e37c6@exi-reader.telstra.net> you say:
| > | "However, statement numbers can be stored with the
| > | executable, and thus tell the user where the exception occurred."
| > | Does it make the executable slower if you choose to do this ? (I'm
| > | assuming you're still talking about PL/I.)
| >
| > It does make the executable slower, but trivially.  All that's required
| > is a store instruction that stores an integer (statement number)
| > at the start of the object code for each statement.
|
| That sounds like a significant slowdown to me. I used that approach in an
| interpreter, and it cost at least 5%. Probably more it tight, compiled code
| where it also increases code size.

No doubt if all the instructions in a program are trivial sassignments
such as A = 1, B = 2, c = 7, etc,
you would notice some difference.
In general, the difference would not be measurable.

| But it shouldn't be necessary: it should be easy enough to lookup the
| address of the instruction where the error occurred, and match to a source
| file/line number. The table required doesn't even need to be part of the
| executable.

How many users carry around with them the assembly-langage
listing or the memory dump when an error occurs?
Plus the source?
It isn't actually convenient to do that.
And why should anyone be forced to do that,
when a computer can do it trivially for you? 

In comp.lang.pl1 robin <robin51@dodo.com.au> wrote:
(snip)

> I mentioned that because it is the easiest and simplest implementation,
> namely, to emit one instruction to store the statemnet number somewhere.
> Other schemes are more involved.
> Even when code (being part of a source statement) is moved 
> (e.g., out of a loop, etc), that code can be preceded by an 
> instruction to store the statement number. 

For PL/I (F), the only one I have looked in detail at the
generated code and the PLM, with the STMT option it generates
MVI to store the number.  With the M91 option, it generates
BR 0 instructions before each statement, including the MVI.
That flushes any processor pipelines to make sure that only 
instructions relating to that statement are executed while,
and that they match the indicated statement.  It also slows
down pipelined processors.

-- glen

"robin" <robin51@dodo.com.au> wrote in message 
news:4cc03769$0$78208$c30e37c6@exi-reader.telstra.net...
> "BartC" <bc@freeuk.com> wrote in message 
> news:i91cin$dqh$1@news.eternal-september.org...
> | "robin" <robin51@dodo.com.au> wrote in message
> | news:4cb3b4be$0$55987$c30e37c6@exi-reader.telstra.net...
> | > "Spiros Bousbouras" <spibou@gmail.com> wrote in message
> | > news:86Iso.24175$_x2.4217@newsfe30.ams2...
> |
> | > | In <4cb1be12$0$33792$c30e37c6@exi-reader.telstra.net> you say:
> | > | "However, statement numbers can be stored with the
> | > | executable, and thus tell the user where the exception occurred."
> | > | Does it make the executable slower if you choose to do this ? (I'm
> | > | assuming you're still talking about PL/I.)
> | >
> | > It does make the executable slower, but trivially.  All that's 
> required
> | > is a store instruction that stores an integer (statement number)
> | > at the start of the object code for each statement.
> |
> | That sounds like a significant slowdown to me. I used that approach in 
> an
> | interpreter, and it cost at least 5%. Probably more it tight, compiled 
> code
> | where it also increases code size.
>
> No doubt if all the instructions in a program are trivial sassignments
> such as A = 1, B = 2, c = 7, etc,
> you would notice some difference.
> In general, the difference would not be measurable.

I've just measured again on a real app (a compiler), and it is about 5%. 
Some of the runtime is interpreter overhead, so in compiled code, it could 
be more.

> | But it shouldn't be necessary: it should be easy enough to lookup the
> | address of the instruction where the error occurred, and match to a 
> source
> | file/line number. The table required doesn't even need to be part of the
> | executable.
>
> How many users carry around with them the assembly-langage
> listing or the memory dump when an error occurs?
> Plus the source?
> It isn't actually convenient to do that.
> And why should anyone be forced to do that,
> when a computer can do it trivially for you?

I didn't mean the lookup process should be done manually, but by the error 
handler.

The files and tables needed can be part of the executable, part of the 
installation, or accessed over the internet.

-- 
bartc 

"glen herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message news:i9pqqk$d26$1@news.eternal-september.org...
| In comp.lang.pl1 robin <robin51@dodo.com.au> wrote:
| (snip)
|
| > I mentioned that because it is the easiest and simplest implementation,
| > namely, to emit one instruction to store the statemnet number somewhere.
| > Other schemes are more involved.
| > Even when code (being part of a source statement) is moved
| > (e.g., out of a loop, etc), that code can be preceded by an
| > instruction to store the statement number.
|
| For PL/I (F), the only one I have looked in detail at the
| generated code and the PLM, with the STMT option it generates
| MVI to store the number.

MVI would store only 8 bits, allowing statement numbers
in the range 0 to 255. 

"BartC" <bc@freeuk.com> wrote in message news:i9q74u$8jb$1@news.eternal-september.org...
| "robin" <robin51@dodo.com.au> wrote in message
| news:4cc03769$0$78208$c30e37c6@exi-reader.telstra.net...
| > "BartC" <bc@freeuk.com> wrote in message
| > news:i91cin$dqh$1@news.eternal-september.org...
| > | "robin" <robin51@dodo.com.au> wrote in message
| > | news:4cb3b4be$0$55987$c30e37c6@exi-reader.telstra.net...
| > | > "Spiros Bousbouras" <spibou@gmail.com> wrote in message
| > | > news:86Iso.24175$_x2.4217@newsfe30.ams2...
| > |
| > | > | In <4cb1be12$0$33792$c30e37c6@exi-reader.telstra.net> you say:
| > | > | "However, statement numbers can be stored with the
| > | > | executable, and thus tell the user where the exception occurred."
| > | > | Does it make the executable slower if you choose to do this ? (I'm
| > | > | assuming you're still talking about PL/I.)
| > | >
| > | > It does make the executable slower, but trivially.  All that's
| > required
| > | > is a store instruction that stores an integer (statement number)
| > | > at the start of the object code for each statement.
| > |
| > | That sounds like a significant slowdown to me. I used that approach in
| > an
| > | interpreter, and it cost at least 5%. Probably more it tight, compiled
| > code
| > | where it also increases code size.
| >
| > No doubt if all the instructions in a program are trivial sassignments
| > such as A = 1, B = 2, c = 7, etc,
| > you would notice some difference.
| > In general, the difference would not be measurable.
|
| I've just measured again on a real app (a compiler), and it is about 5%.
| Some of the runtime is interpreter overhead, so in compiled code, it could
| be more.
|
| > | But it shouldn't be necessary: it should be easy enough to lookup the
| > | address of the instruction where the error occurred, and match to a
| > source
| > | file/line number. The table required doesn't even need to be part of the
| > | executable.
| >
| > How many users carry around with them the assembly-langage
| > listing or the memory dump when an error occurs?
| > Plus the source?
| > It isn't actually convenient to do that.
| > And why should anyone be forced to do that,
| > when a computer can do it trivially for you?
|
| I didn't mean the lookup process should be done manually, but by the error
| handler.
|
| The files and tables needed can be part of the executable, part of the
| installation, or accessed over the internet.

Part of the executable: even larger storage requirements than storing the
                                    statement number;
Part of the installation: How?  The installation has just the compiler;
Accessed over the internet: Really? That's even worse.  And where/how
                                     is the executable going to find it? 

In comp.lang.pl1 robin <robin51@dodo.com.au> wrote:
(snip, I wrote)

> | For PL/I (F), the only one I have looked in detail at the
> | generated code and the PLM, with the STMT option it generates
> | MVI to store the number.
 
> MVI would store only 8 bits, allowing statement numbers
> in the range 0 to 255. 

The compiler is smart.  It knows that from one statement to
the next, usually only the low bits change.  It also knows
where GOTO targets are.  Other bytes are updated only when
needed.

-- glen

"robin" <robin51@dodo.com.au> wrote in message 
news:4cc0e6fa$0$78207$c30e37c6@exi-reader.telstra.net...
> "BartC" <bc@freeuk.com> wrote in message 
> news:i9q74u$8jb$1@news.eternal-september.org...
> | "robin" <robin51@dodo.com.au> wrote in message
> | news:4cc03769$0$78208$c30e37c6@exi-reader.telstra.net...
> | > "BartC" <bc@freeuk.com> wrote in message
> | > news:i91cin$dqh$1@news.eternal-september.org...
> | > | "robin" <robin51@dodo.com.au> wrote in message

> | > | > It does make the executable slower, but trivially.  All that's
> | > required
> | > | > is a store instruction that stores an integer (statement number)
> | > | > at the start of the object code for each statement.

> | > | But it shouldn't be necessary: it should be easy enough to lookup 
> the
> | > | address of the instruction where the error occurred, and match to a
> | > source
> | > | file/line number. The table required doesn't even need to be part of 
> the
> | > | executable.
> | >
> | > How many users carry around with them the assembly-langage
> | > listing or the memory dump when an error occurs?
> | > Plus the source?
> | > It isn't actually convenient to do that.
> | > And why should anyone be forced to do that,
> | > when a computer can do it trivially for you?
> |
> | I didn't mean the lookup process should be done manually, but by the 
> error
> | handler.
> |
> | The files and tables needed can be part of the executable, part of the
> | installation, or accessed over the internet.
>
> Part of the executable: even larger storage requirements than storing the
>                                    statement number;

Having the statement number stored at the start of each block of code is 
little different from a table of entries storing statement number:address. 
The error handler needs to do a bit of work to convert the failure address 
to a statement number.

(I assume you mean having an /instruction/ setting up the statement number; 
this instruction needs to contain the address of a location, the statement 
number, /and/ the opcodes needed. And in a few places (eg. function calls) 
the 'last' statement number needs code to preserve it.)

Anyway, without any source code, how useful is a statement number to the 
user when a program goes wrong (and here I assume we're talking about an 
end-user, not a programmer.)

> Part of the installation: How?  The installation has just the compiler;

I'm talking about the installation of an application.

> Accessed over the internet: Really? That's even worse.  And where/how
>                                     is the executable going to find it?

By accessing a file via a URL? Again, I'm talking about a user running a 
program where he doesn't even know what language it was written in. Only 
that it's gone wrong...

-- 
Bartc 

BartC wrote:
> 
> By accessing a file via a URL? Again, I'm talking about a user running a 
> program where he doesn't even know what language it was written in. Only 
> that it's gone wrong...
> 

In this case the information is useless to the user.  The error would 
have to be reported in terms that would let him do something about the 
problem, if possible.

"glen herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message news:i9r1gu$59i$3@news.eternal-september.org...
| In comp.lang.pl1 robin <robin51@dodo.com.au> wrote:
| (snip, I wrote)
|
| > | For PL/I (F), the only one I have looked in detail at the
| > | generated code and the PLM, with the STMT option it generates
| > | MVI to store the number.
|
| > MVI would store only 8 bits, allowing statement numbers
| > in the range 0 to 255.
|
| The compiler is smart.  It knows that from one statement to
| the next, usually only the low bits change.  It also knows
| where GOTO targets are.  Other bytes are updated only when
| needed.

So, it actually used two MVI's, sometimes one. 

"BartC" <bc@freeuk.com> wrote in message news:i9rn8s$l9$1@news.eternal-september.org...
| "robin" <robin51@dodo.com.au> wrote in message
| news:4cc0e6fa$0$78207$c30e37c6@exi-reader.telstra.net...
| > "BartC" <bc@freeuk.com> wrote in message
| > news:i9q74u$8jb$1@news.eternal-september.org...
| > | "robin" <robin51@dodo.com.au> wrote in message
| > | news:4cc03769$0$78208$c30e37c6@exi-reader.telstra.net...
| > | > "BartC" <bc@freeuk.com> wrote in message
| > | > news:i91cin$dqh$1@news.eternal-september.org...
| > | > | "robin" <robin51@dodo.com.au> wrote in message
|
| > | > | > It does make the executable slower, but trivially.  All that's
| > | > required
| > | > | > is a store instruction that stores an integer (statement number)
| > | > | > at the start of the object code for each statement.
|
| > | > | But it shouldn't be necessary: it should be easy enough to lookup
| > the
| > | > | address of the instruction where the error occurred, and match to a
| > | > source
| > | > | file/line number. The table required doesn't even need to be part of
| > the
| > | > | executable.
| > | >
| > | > How many users carry around with them the assembly-langage
| > | > listing or the memory dump when an error occurs?
| > | > Plus the source?
| > | > It isn't actually convenient to do that.
| > | > And why should anyone be forced to do that,
| > | > when a computer can do it trivially for you?
| > |
| > | I didn't mean the lookup process should be done manually, but by the
| > error
| > | handler.
| > |
| > | The files and tables needed can be part of the executable, part of the
| > | installation, or accessed over the internet.
| >
| > Part of the executable: even larger storage requirements than storing the
| >                                    statement number;
|
| Having the statement number stored at the start of each block of code is
| little different from a table of entries storing statement number:address.
| The error handler needs to do a bit of work to convert the failure address
| to a statement number.

The error handler does no work other than converting the
statement number/line number to character form.
In the case of a table, it has to do a search of the tables.
Obviously, there is nmore work for this latter case, and
it is better for the error handler that the work is minimised
to avoid the risk of raising another error during the process.

| (I assume you mean having an /instruction/ setting up the statement number;
| this instruction needs to contain the address of a location, the statement
| number, /and/ the opcodes needed. And in a few places (eg. function calls)
| the 'last' statement number needs code to preserve it.)

Nothing is required for function calls.  (The code for the function
itself generates statement numbers.)

| Anyway, without any source code, how useful is a statement number to the
| user when a program goes wrong (and here I assume we're talking about an
| end-user, not a programmer.)

The end user can obtain the source code, or if not can refer
all details of the error (including the statement number, of course)
to the originator of the code so that the error can be corrected.

Of couse, if the error arose on account of fault with the data,
the problem may be resolved without needing the statement number. 

In <4cc4de16$0$500$c30e37c6@exi-reader.telstra.net>, on 10/25/2010
   at 12:32 PM, "robin" <robin51@dodo.com.au> said:

>The error handler does no work other than converting the statement
>number/line number to character form.
>In the case of a table, it has to do a search of the tables.
>Obviously, there is nmore work for this latter case, and it is better
>for the error handler that the work is minimised to avoid the risk of
>raising another error during the process.

Searching a table requires more CPU cycles in the error handler, but
not additional storage. The CPU time used would be modest compared to
that used by, e.g., SNAP.

-- 
Shmuel (Seymour J.) Metz, SysProg and JOAT  <http://patriot.net/~shmuel>

Unsolicited bulk E-mail subject to legal action.  I reserve the
right to publicly post or ridicule any abusive E-mail.  Reply to
domain Patriot dot net user shmuel+news to contact me.  Do not
reply to spamtrap@library.lspace.org

"Shmuel (Seymour J.) Metz" <spamtrap@library.lspace.org.invalid> wrote in message 
news:4cb6d740$28$fuzhry+tra$mr2ice@news.patriot.net...
| In <ijLso.39415$af7.16557@newsfe28.ams2>, on 10/11/2010
|   at 09:27 PM, Spiros Bousbouras <spibou@gmail.com> said:
|
| >And a PL/I development environment can give you from the statement
| >number the corresponding point in the source code , yes ?
|
| That depends on the compiler options used.

It doesn't depend on anything.

If you have the statement number,
that tells you where to look in the source program. 

In <4cc80f5d$0$502$c30e37c6@exi-reader.telstra.net>, on 10/27/2010
   at 10:39 PM, "robin" <robin51@dodo.com.au> said:

>It doesn't depend on anything.

Of course it does, David.

>If you have the statement number,
>that tells you where to look in the source program. 

Right answer to wrong question. With the proper compiler options you
don't need to manually look at the listing to go to the right line in
the editor. In some environments, it's even more automated, but not
with the wrong compiler options.

-- 
Shmuel (Seymour J.) Metz, SysProg and JOAT  <http://patriot.net/~shmuel>

Unsolicited bulk E-mail subject to legal action.  I reserve the
right to publicly post or ridicule any abusive E-mail.  Reply to
domain Patriot dot net user shmuel+news to contact me.  Do not
reply to spamtrap@library.lspace.org

"Shmuel (Seymour J.) Metz" <spamtrap@library.lspace.org.invalid> wrote in message 
news:4cc95945$4$fuzhry+tra$mr2ice@news.patriot.net...
| In <4cc80f5d$0$502$c30e37c6@exi-reader.telstra.net>, on 10/27/2010
|   at 10:39 PM, "robin" <robin51@dodo.com.au> said:
|
| >It doesn't depend on anything.
|
| Of course it does, David.

Are you just having another bad day, or are you like this all the time?

| >If you have the statement number,
| >that tells you where to look in the source program.
|
| Right answer to wrong question.

Still having a bad day?

| With the proper compiler options you
| don't need to manually look at the listing to go to the right line in
| the editor. In some environments, it's even more automated, but not
| with the wrong compiler options.

In <4ccaebca$0$499$c30e37c6@exi-reader.telstra.net>, on 10/30/2010
   at 02:44 AM, David Frank <robin51@dodo.com.au> said:

>Are you just having another bad day, 

No, I'm just replying to another article from an illiterate and
ignorant poster.

>Still having a bad day?

No, just noting that you seem to have a propensity for attempting to
pass off responses to statements that nobody wrote as relevant
responses to what they actually wrote. The issue in dispute is the
ability to get to the *source* line in error, not a line an the
*listing*.

-- 
Shmuel (Seymour J.) Metz, SysProg and JOAT  <http://patriot.net/~shmuel>

Unsolicited bulk E-mail subject to legal action.  I reserve the
right to publicly post or ridicule any abusive E-mail.  Reply to
domain Patriot dot net user shmuel+news to contact me.  Do not
reply to spamtrap@library.lspace.org