I'm learning Lisp and am reading Paul Graham's book on ANSI Common
Lisp and using LispWorks Personal edition for testing.  He gives a
simple example of creating a variable in lexical scope:

(let ((x 10))
   (defun foo () x))

then says "global variables established by calling setf at the
toplevel are implicitly special":

> (setf x 30)
30
> (foo)
30

The output of 30 seems totally expected, given what he said there.
However, confusingly, if I enter all three snippets into the top
level, I get 10 from the final call to (foo), implying a pretty
fundamental bug in LispWorks?  A change in the language since the book
was written?  I don't understand something crucial about this that
would make this behavior seem normal?  Somebody please help!

Furthermore, another little session that seems to imply that variables
created with setf at the top level are NOT implicitly special (but are
if you use defparameter):

CL-USER 1 > (setf x 30)
30

CL-USER 2 > (let ((x 10)) (defun foo () x))
FOO

CL-USER 3 > (foo)
10

CL-USER 4 > x
30

CL-USER 5 > (defparameter y 47)
Y

CL-USER 6 > (let ((y -5)) (defun goo () y))
GOO

CL-USER 7 > (goo)
47

CL-USER 8 > y
47

Dave Slayton wrote:

> I'm learning Lisp and am reading Paul Graham's book on ANSI Common
> Lisp and using LispWorks Personal edition for testing.  He gives a
> simple example of creating a variable in lexical scope:
> 
> (let ((x 10))
>    (defun foo () x))
> 
> then says "global variables established by calling setf at the
> toplevel are implicitly special":
> 
> > (setf x 30)
> 30
> > (foo)
> 30
> 
> The output of 30 seems totally expected, given what he said there.
> However, confusingly, if I enter all three snippets into the top
> level, I get 10 from the final call to (foo), implying a pretty
> fundamental bug in LispWorks?  A change in the language since the book
> was written?  I don't understand something crucial about this that
> would make this behavior seem normal?  Somebody please help!

* (setf x 30)

; in: LAMBDA NIL
;     (SETF X 30)
; ==>
;   (SETQ X 30)
;
; caught WARNING:
;   undefined variable: X
;
; compilation unit finished
;   Undefined variable:
;     X
;   caught 1 WARNING condition

30

Dave Slayton <lwoj.sjuwklim@gmail.com> writes:

> I'm learning Lisp and am reading Paul Graham's book on ANSI Common
> Lisp and using LispWorks Personal edition for testing.  He gives a
> simple example of creating a variable in lexical scope:
>
> (let ((x 10))
>    (defun foo () x))
>
> then says "global variables established by calling setf at the
> toplevel are implicitly special":

Actually, it depends on the implementation.  The ANSI Common Lisp
Standard doesn't specify what happens if you set a variable without
defining it.

So you have to check the documentation of your documentation, and it is
a faux pas to use that kind of form in a book or a tutorial, that is
intended to be read by people using other CL implementations.  Clearly,
it should not be titled "ANSI Common Lisp", but "The Lisp implemented by
this very specific implementation So and So".

>> (setf x 30)
> 30
>> (foo)
> 30
>
> The output of 30 seems totally expected, given what he said there.

That's what we mean when we say it's not specified.  Anything can
happen.


> However, confusingly, if I enter all three snippets into the top
> level, I get 10 from the final call to (foo), implying a pretty
> fundamental bug in LispWorks?

No, it's a fundamental bug in this Paul Graham's book.   Unfortunately,
it's a bug that is often found in books and tutorials about CL.  That's
why it has become a FAQ.


> A change in the language since the book was written?  I don't
> understand something crucial about this that would make this behavior
> seem normal?  Somebody please help!



If you define (let ((x 10)) (defun foo () x)) first, then the
implementation will compile the reference to x in foo as a lexical
reference, and (foo) will return 10.  Even if you later declare x
special, explicitely (with a defvar, defparameter, or special
declaimation or proclaimation) or implicitely (with a setq or setf,
assuming the implementation makes it special then).

But if you define or redefine (let ((x 10)) (defun foo () x)) AFTER
having declared x special, then the binding will be dynamic, and the
reference to x in foo is a dynamic reference.  Therefore (foo) will
return whatever value x happen to have WHEN you call it.   Since the
interpretation of a form like (let ((x 10)) (defun foo () x)) thus
depends on whether x has been declared special or not, to avoid this
confusion we adopt the convention of naming all the global special
variables with "earmuff", ie. with stars around: *x*.  

    (let ((x 10)) (defun foo () x))
    (foo) --> 10 ; a lexical reference to the x in the closure.
    (defvar *x* 42)
    (let ((x 10)) (defun foo () x))
    (foo) --> 10 ; still a lexical reference to the x in the closure.
    (let ((*x* 10)) (defun bar () *x*)) ; a dynamic binding, unused.
    (bar) --> 42 ; a dynamic reference to the global *x*.



If you want to use the implementation dependant behavior of setf with
undefined variables, and if your implementation then declare
automatically the variable special and global, then I would strongly
advise you to name those variable with the earmuff convention!

    (let ((y 10)) (defun foo () y))
    (foo) --> 10 ; a lexical reference to the y in the closure.
    (setf *y* 42)
    (let ((y 10)) (defun foo () y))
    (foo) --> 10 ; still a lexical reference to the y in the closure.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Dave Slayton <lwoj.sjuwklim@gmail.com> writes:

> Furthermore, another little session that seems to imply that variables
> created with setf at the top level are NOT implicitly special (but are
> if you use defparameter):

Whatever.  Anything can happen, it's not standardized.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

> then says "global variables established by calling setf at the
> toplevel are implicitly special":

He is wrong.

>> (setf x 30)
> 30
>> (foo)
> 30

> The output of 30 seems totally expected,

I would not expect this from a compiler, even if statement above was 
true (and it is not).

> level, I get 10 from the final call to (foo), implying a pretty
> fundamental bug in LispWorks?

In fact Lispwork's behaviour is very reasonable.

Thank you all very much for your responses.  I understand the
situation much much better now.

On 2012-06-09, Dave Slayton <lwoj.sjuwklim@gmail.com> wrote:
> Furthermore, another little session that seems to imply that variables
> created with setf at the top level are NOT implicitly special (but are
> if you use defparameter):

The behavior of assigning a never-before-seen variable is not defined by the
ANSI standard. By experimenting with this, you can only confirm what happens on
various implementations.

To see that this is the case, one possible interpretation pathway through the
spec is this:

when a symbol is used as a form, then if it is not a symbol macro, it is
necessarily a variable:

  3.1.2.1.1 Symbols as Forms

  If a form is a symbol that is not a symbol macro, then it is the name of a
  variable, and the value of that variable is returned. There are three kinds
  of variables: lexical variables, dynamic variables, and constant variables. A
  variable can store one object. The main operations on a variable are to
  read[1] and to write[1] its value.

  An error of type unbound-variable should be signaled if an unbound variable
  is referenced. 

Okay, so there is no such thing as a symbol which is used as a form, but is not
a variable.  If the symbol is not a symbol macro, and has no binding in any
environment, then it is an unbound variable, but still a variable.

So when we have:

  (setf never-before-seen 42)

we are in fact trying to assign to a variable.  The problem is that the we do
not know what kind of variable this should be once it acquires the value.

We might hope that it becomes a dynamic variable, but the spec does not give us that.

We have this section on dynamic variables:

  3.1.2.1.1.2 Dynamic Variables

  A variable is a dynamic variable if one of the following conditions hold:

  *  It is locally declared or globally proclaimed special.

  *  It occurs textually within a form that creates a dynamic binding for 
     variable of the same name, and the binding is not shadowed[2] by a form
     that creates a lexical binding of the same variable name. 

Neither of these two condition holds if the variable has never been seen before!
The same section, 3.1.2.1.1.2 also says:

  A dynamic variable is unbound unless and until explicitly assigned a value,
  except for those variables whose initial value is defined in this
  specification or by an implementation. 

So, there can be variables which are unbound, but are also known to be dynamic
at the same time.  This occurs in the case when the variable is globally
proclaimed special, but no value has been assigned. In this situation, the
symbol is a kind of latent dynamic variable: it is unbound, but it will give
rise to a dynamic binding when assigned.

No requirements are given about what happens if any other kind of variable is
assigned. Lexical variables and constants are never bound by assignment, only
dynamic variables are.

In other words, we cannot interpret what the behavior of:

  (setf never-before-seen 42)

ought to be.

By the way, what about using the symbol-value API to manipulate the value cell
of a symbol?

  (setf (symbol-value 'never-before-seen) 42)

Here, the behavior is also not defined. The spec says that the argument
is a symbol which must have a value (regardless of whether the value is being accessed or
replaced with a new value). It says that an error of type unbound-variable
"should" be signaled on a read, but not on a write.  When a program violates a
"must" requirement, and there is no requirement for an error signal, the
behavior is not defined.

Kaz Kylheku <kaz@kylheku.com> writes:

> On 2012-06-09, Dave Slayton <lwoj.sjuwklim@gmail.com> wrote:
>> Furthermore, another little session that seems to imply that variables
>> created with setf at the top level are NOT implicitly special (but are
>> if you use defparameter):
>
> The behavior of assigning a never-before-seen variable is not defined by the
> ANSI standard. By experimenting with this, you can only confirm what happens on
> various implementations.
>
> To see that this is the case, one possible interpretation pathway through the
> spec is this:
>
> when a symbol is used as a form, then if it is not a symbol macro, it is
> necessarily a variable:
>
>   3.1.2.1.1 Symbols as Forms
>
>   If a form is a symbol that is not a symbol macro, then it is the name of a
>   variable, and the value of that variable is returned. There are three kinds
>   of variables: lexical variables, dynamic variables, and constant variables. A
>   variable can store one object. The main operations on a variable are to
>   read[1] and to write[1] its value.
>
>   An error of type unbound-variable should be signaled if an unbound variable
>   is referenced. 
>
> Okay, so there is no such thing as a symbol which is used as a form, but is not
> a variable.  If the symbol is not a symbol macro, and has no binding in any
> environment, then it is an unbound variable, but still a variable.
>
> So when we have:
>
>   (setf never-before-seen 42)
>
> we are in fact trying to assign to a variable.  The problem is that the we do
> not know what kind of variable this should be once it acquires the value.
>
> We might hope that it becomes a dynamic variable, but the spec does not give us that.
>
> We have this section on dynamic variables:
>
>   3.1.2.1.1.2 Dynamic Variables
>
>   A variable is a dynamic variable if one of the following conditions hold:
>
>   *  It is locally declared or globally proclaimed special.
>
>   *  It occurs textually within a form that creates a dynamic binding for 
>      variable of the same name, and the binding is not shadowed[2] by a form
>      that creates a lexical binding of the same variable name. 
>
> Neither of these two condition holds if the variable has never been seen before!
> The same section, 3.1.2.1.1.2 also says:
>
>   A dynamic variable is unbound unless and until explicitly assigned a value,
>   except for those variables whose initial value is defined in this
>   specification or by an implementation. 
>
> So, there can be variables which are unbound, but are also known to be dynamic
> at the same time.  This occurs in the case when the variable is globally
> proclaimed special, but no value has been assigned. In this situation, the
> symbol is a kind of latent dynamic variable: it is unbound, but it will give
> rise to a dynamic binding when assigned.
>
> No requirements are given about what happens if any other kind of variable is
> assigned. Lexical variables and constants are never bound by assignment, only
> dynamic variables are.
>
> In other words, we cannot interpret what the behavior of:
>
>   (setf never-before-seen 42)
>
> ought to be.
>
> By the way, what about using the symbol-value API to manipulate the value cell
> of a symbol?
>
>   (setf (symbol-value 'never-before-seen) 42)
>
> Here, the behavior is also not defined. The spec says that the argument
> is a symbol which must have a value (regardless of whether the value is being accessed or
> replaced with a new value). It says that an error of type unbound-variable
> "should" be signaled on a read, but not on a write.  When a program violates a
> "must" requirement, and there is no requirement for an error signal, the
> behavior is not defined.

Well,

    (defvar x) == (declaim (special x))

and:

    (defvar x)  == (declaim (special x))
    (setf x 42) == (setf (symbol-value 'x) 42)

so we could specify that without a definition for x,

    (setf x 42) == (setf (symbol-value 'x) 42)



Then

    x --> 42

    (let ((x 0)) ; lexical binding
       x) --> 0

    (funcall (let ((x 0)) ; lexical binding
                (lambda () x)))  --> 0

    x --> 42

    (funcall (lambda () x)) --> undefined variable error.
    (funcall (lambda () (declare (special x)) x)) --> 42



But that would mean changing some implementations…
-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

On 2012-06-10, Pascal J. Bourguignon <pjb@informatimago.com> wrote:
> Kaz Kylheku <kaz@kylheku.com> writes:
>
>> On 2012-06-09, Dave Slayton <lwoj.sjuwklim@gmail.com> wrote:
>>> Furthermore, another little session that seems to imply that variables
>>> created with setf at the top level are NOT implicitly special (but are
>>> if you use defparameter):
>>
>> The behavior of assigning a never-before-seen variable is not defined by the
>> ANSI standard. By experimenting with this, you can only confirm what happens on
>> various implementations.
>>
>> To see that this is the case, one possible interpretation pathway through the
>> spec is this:
>>
>> when a symbol is used as a form, then if it is not a symbol macro, it is
>> necessarily a variable:
>>
>>   3.1.2.1.1 Symbols as Forms
>>
>>   If a form is a symbol that is not a symbol macro, then it is the name of a
>>   variable, and the value of that variable is returned. There are three kinds
>>   of variables: lexical variables, dynamic variables, and constant variables. A
>>   variable can store one object. The main operations on a variable are to
>>   read[1] and to write[1] its value.
>>
>>   An error of type unbound-variable should be signaled if an unbound variable
>>   is referenced. 
>>
>> Okay, so there is no such thing as a symbol which is used as a form, but is not
>> a variable.  If the symbol is not a symbol macro, and has no binding in any
>> environment, then it is an unbound variable, but still a variable.
>>
>> So when we have:
>>
>>   (setf never-before-seen 42)
>>
>> we are in fact trying to assign to a variable.  The problem is that the we do
>> not know what kind of variable this should be once it acquires the value.
>>
>> We might hope that it becomes a dynamic variable, but the spec does not give us that.
>>
>> We have this section on dynamic variables:
>>
>>   3.1.2.1.1.2 Dynamic Variables
>>
>>   A variable is a dynamic variable if one of the following conditions hold:
>>
>>   *  It is locally declared or globally proclaimed special.
>>
>>   *  It occurs textually within a form that creates a dynamic binding for 
>>      variable of the same name, and the binding is not shadowed[2] by a form
>>      that creates a lexical binding of the same variable name. 
>>
>> Neither of these two condition holds if the variable has never been seen before!
>> The same section, 3.1.2.1.1.2 also says:
>>
>>   A dynamic variable is unbound unless and until explicitly assigned a value,
>>   except for those variables whose initial value is defined in this
>>   specification or by an implementation. 
>>
>> So, there can be variables which are unbound, but are also known to be dynamic
>> at the same time.  This occurs in the case when the variable is globally
>> proclaimed special, but no value has been assigned. In this situation, the
>> symbol is a kind of latent dynamic variable: it is unbound, but it will give
>> rise to a dynamic binding when assigned.
>>
>> No requirements are given about what happens if any other kind of variable is
>> assigned. Lexical variables and constants are never bound by assignment, only
>> dynamic variables are.
>>
>> In other words, we cannot interpret what the behavior of:
>>
>>   (setf never-before-seen 42)
>>
>> ought to be.
>>
>> By the way, what about using the symbol-value API to manipulate the value cell
>> of a symbol?
>>
>>   (setf (symbol-value 'never-before-seen) 42)
>>
>> Here, the behavior is also not defined. The spec says that the argument
>> is a symbol which must have a value (regardless of whether the value is being accessed or
>> replaced with a new value). It says that an error of type unbound-variable
>> "should" be signaled on a read, but not on a write.  When a program violates a
>> "must" requirement, and there is no requirement for an error signal, the
>> behavior is not defined.
>
> Well,
>
>     (defvar x) == (declaim (special x))
>
> and:
>
>     (defvar x)  == (declaim (special x))
>     (setf x 42) == (setf (symbol-value 'x) 42)
>
> so we could specify that without a definition for x,
>
>     (setf x 42) == (setf (symbol-value 'x) 42)

If x doesn't exist, then basically we are equating two undefined behaviors. 

On one hand, it seems harmless. On the other, it's nonsensical because 
we're saying that there are no requirements for construct X, and there are no
requirements for construct Y. Oh, but the behavior of X and Y should be the
same (which is a requirement).

>
> Then
>
>     x --> 42
>
>     (let ((x 0)) ; lexical binding
>        x) --> 0
>
>     (funcall (let ((x 0)) ; lexical binding
>                 (lambda () x)))  --> 0
>
>     x --> 42
>
>     (funcall (lambda () x)) --> undefined variable error.
>     (funcall (lambda () (declare (special x)) x)) --> 42
>
> But that would mean changing some implementations…

I'm afraid I don't follow, and part of the reason is that you didn't use SETF.

If the form x nicely evaluates to 42 at the top level, why would it flake out
in the body of a lambda.

Kaz Kylheku <kaz@kylheku.com> writes:

> On 2012-06-10, Pascal J. Bourguignon <pjb@informatimago.com> wrote:
>> Well,
>>
>>     (defvar x) == (declaim (special x))
>>
>> and:
>>
>>     (defvar x)  == (declaim (special x))
>>     (setf x 42) == (setf (symbol-value 'x) 42)
>>
>> so we could specify that without a definition for x,
>>
>>     (setf x 42) == (setf (symbol-value 'x) 42)
>
> If x doesn't exist, then basically we are equating two undefined behaviors. 

No.  (setf (symbol-value 'x) 42)  is always well defined.

> On one hand, it seems harmless. On the other, it's nonsensical because 
> we're saying that there are no requirements for construct X, and there are no
> requirements for construct Y. Oh, but the behavior of X and Y should be the
> same (which is a requirement).
>
>>
>> Then
>>
>>     x --> 42
>>
>>     (let ((x 0)) ; lexical binding
>>        x) --> 0
>>
>>     (funcall (let ((x 0)) ; lexical binding
>>                 (lambda () x)))  --> 0
>>
>>     x --> 42
>>
>>     (funcall (lambda () x)) --> undefined variable error.
>>     (funcall (lambda () (declare (special x)) x)) --> 42
>>
>> But that would mean changing some implementations…
>
> I'm afraid I don't follow, and part of the reason is that you didn't use SETF.
>
> If the form x nicely evaluates to 42 at the top level, why would it flake out
> in the body of a lambda.

Because it has never been declared special, therefore it's a lexical
reference, and unfortunately, it's a free variable with no lexical
binding.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

In article 
<1c3590c5-b4bf-4da2-b0d6-8194746b7ba1@h41g2000yqm.googlegroups.com>,
 Dave Slayton <lwoj.sjuwklim@gmail.com> wrote:

> I'm learning Lisp and am reading Paul Graham's book on ANSI Common
> Lisp and using LispWorks Personal edition for testing.  He gives a
> simple example of creating a variable in lexical scope:
> 
> (let ((x 10))
>    (defun foo () x))
> 
> then says "global variables established by calling setf at the
> toplevel are implicitly special":
> 
> > (setf x 30)
> 30
> > (foo)
> 30
> 
> The output of 30 seems totally expected, given what he said there.
> However, confusingly, if I enter all three snippets into the top
> level, I get 10 from the final call to (foo), implying a pretty
> fundamental bug in LispWorks?  A change in the language since the book
> was written?  I don't understand something crucial about this that
> would make this behavior seem normal?  Somebody please help!

http://www.flownet.com/ron/specials.pdf
http://blog.rongarret.info/2009/08/global-variables-done-right.html

rg

On 2012-06-10, Pascal J. Bourguignon <pjb@informatimago.com> wrote:
> Kaz Kylheku <kaz@kylheku.com> writes:
>
>> On 2012-06-10, Pascal J. Bourguignon <pjb@informatimago.com> wrote:
>>> Well,
>>>
>>>     (defvar x) == (declaim (special x))
>>>
>>> and:
>>>
>>>     (defvar x)  == (declaim (special x))
>>>     (setf x 42) == (setf (symbol-value 'x) 42)
>>>
>>> so we could specify that without a definition for x,
>>>
>>>     (setf x 42) == (setf (symbol-value 'x) 42)
>>
>> If x doesn't exist, then basically we are equating two undefined behaviors. 
>
> No.  (setf (symbol-value 'x) 42)  is always well defined.

That's not what I'm reading in the CLHS. The specification for symbol-value
clearly states that the input symbol "must have a value", and that it "[s]hould
signal unbound-variable if symbol is unbound and an attempt is made to read its
value." Furthermore, "[N]o such error is signaled on an attempt to write its
value." Note how it doesn't simply say that writing is a well-defined
operation, only that it is undiagnosed, and that it is only an attempt.

If you want to regard that as a defect, I'm with you.

>>> Then
>>>
>>>     x --> 42
>>>
>>>     (let ((x 0)) ; lexical binding
>>>        x) --> 0
>>>
>>>     (funcall (let ((x 0)) ; lexical binding
>>>                 (lambda () x)))  --> 0
>>>
>>>     x --> 42
>>>
>>>     (funcall (lambda () x)) --> undefined variable error.
>>>     (funcall (lambda () (declare (special x)) x)) --> 42
>>>
>>> But that would mean changing some implementations…
>>
>> I'm afraid I don't follow, and part of the reason is that you didn't use SETF.
>>
>> If the form x nicely evaluates to 42 at the top level, why would it flake out
>> in the body of a lambda.
>
> Because it has never been declared special, therefore it's a lexical
> reference, and unfortunately, it's a free variable with no lexical
> binding.

Then why is it not a lexical reference at top level, where it resolves to 42?

It looks as if the existence of a lexical scope is shadowing the reference,
even though that lexical scope doesn't bind x.

-- 
If you ever need any coding done, I'm your goto man!

On Sunday, June 10, 2012 5:58:51 PM UTC+2, Kaz Kylheku wrote:
> On 2012-06-10, Pascal J. Bourguignon 
>  wrote:
> > Kaz Kylheku <kaz@kylheku.com> writes:
> >
> >> On 2012-06-10, Pascal J. Bourguignon 
>  wrote:
> >>> Well,
> >>>
> >>>     (defvar x) == (declaim (special x))
> >>>
> >>> and:
> >>>
> >>>     (defvar x)  == (declaim (special x))
> >>>     (setf x 42) == (setf (symbol-value 'x) 42)
> >>>
> >>> so we could specify that without a definition for x,
> >>>
> >>>     (setf x 42) == (setf (symbol-value 'x) 42)
> >>
> >> If x doesn't exist, then basically we are equating two undefined behaviors. 
> >
> > No.  (setf (symbol-value 'x) 42)  is always well defined.
> 
> That's not what I'm reading in the CLHS. The specification for symbol-value
> clearly states that the input symbol "must have a value", and that it "[s]hould
> signal unbound-variable if symbol is unbound and an attempt is made to read its
> value." Furthermore, "[N]o such error is signaled on an attempt to write its
> value." Note how it doesn't simply say that writing is a well-defined
> operation, only that it is undiagnosed, and that it is only an attempt.
> 
> If you want to regard that as a defect, I'm with you.
> 

I think that PJB is trying to make a statement about (SETF (SYMBOL-VALUE 'X) ...)
In this case, you are not "evaluating" the call to SYMBOL-VALUE, therefore you are not bound by the spec passage you mention.

Cheers
--
MA

On 10/06/2012 18:43, Marco Antoniotti wrote:
> On Sunday, June 10, 2012 5:58:51 PM UTC+2, Kaz Kylheku wrote:
>> On 2012-06-10, Pascal J. Bourguignon
>>   wrote:
>>> Kaz Kylheku<kaz@kylheku.com>  writes:
>>>
>>>> On 2012-06-10, Pascal J. Bourguignon
>>   wrote:
>>>>> Well,
>>>>>
>>>>>      (defvar x) == (declaim (special x))
>>>>>
>>>>> and:
>>>>>
>>>>>      (defvar x)  == (declaim (special x))
>>>>>      (setf x 42) == (setf (symbol-value 'x) 42)
>>>>>
>>>>> so we could specify that without a definition for x,
>>>>>
>>>>>      (setf x 42) == (setf (symbol-value 'x) 42)
>>>>
>>>> If x doesn't exist, then basically we are equating two undefined behaviors.
>>>
>>> No.  (setf (symbol-value 'x) 42)  is always well defined.
>>
>> That's not what I'm reading in the CLHS. The specification for symbol-value
>> clearly states that the input symbol "must have a value", and that it "[s]hould
>> signal unbound-variable if symbol is unbound and an attempt is made to read its
>> value." Furthermore, "[N]o such error is signaled on an attempt to write its
>> value." Note how it doesn't simply say that writing is a well-defined
>> operation, only that it is undiagnosed, and that it is only an attempt.
>>
>> If you want to regard that as a defect, I'm with you.
>>
>
> I think that PJB is trying to make a statement about (SETF (SYMBOL-VALUE 'X) ...)
> In this case, you are not "evaluating" the call to SYMBOL-VALUE, therefore you are not bound by the spec passage you mention.

Also compare to the entry for SET, which doesn't mention any error 
situation and specifies SET and (SETF SYMBOL-FUNCTION) to be equivalent.


Pascal

-- 
My website: http://p-cos.net
Common Lisp Document Repository: http://cdr.eurolisp.org
Closer to MOP & ContextL: http://common-lisp.net/project/closer/
The views expressed are my own, and not those of my employer.

Kaz Kylheku <kaz@kylheku.com> writes:

> On 2012-06-10, Pascal J. Bourguignon <pjb@informatimago.com> wrote:
>> Kaz Kylheku <kaz@kylheku.com> writes:
>>
>>> On 2012-06-10, Pascal J. Bourguignon <pjb@informatimago.com> wrote:
>>>> Well,
>>>>
>>>>     (defvar x) == (declaim (special x))
>>>>
>>>> and:
>>>>
>>>>     (defvar x)  == (declaim (special x))
>>>>     (setf x 42) == (setf (symbol-value 'x) 42)
>>>>
>>>> so we could specify that without a definition for x,
>>>>
>>>>     (setf x 42) == (setf (symbol-value 'x) 42)
>>>
>>> If x doesn't exist, then basically we are equating two undefined behaviors. 
>>
>> No.  (setf (symbol-value 'x) 42)  is always well defined.
>
> That's not what I'm reading in the CLHS. The specification for symbol-value
> clearly states that the input symbol "must have a value", and that it "[s]hould
> signal unbound-variable if symbol is unbound and an attempt is made to read its
> value." Furthermore, "[N]o such error is signaled on an attempt to write its
> value." Note how it doesn't simply say that writing is a well-defined
> operation, only that it is undiagnosed, and that it is only an attempt.
>
> If you want to regard that as a defect, I'm with you.

You're confusing the specification of the reader with that of the
writer.  Here I'm using the writer.

Just _read_ the CLHS!

    Should signal unbound-variable if symbol is unbound and an attempt
    is made to read its value. (No such error is signaled on an attempt
    to write its value.)


Notice that after calling (setf symbol-value), the symbol is bound, and
you can call symbol-value.


>>>> Then
>>>>
>>>>     x --> 42
>>>>
>>>>     (let ((x 0)) ; lexical binding
>>>>        x) --> 0
>>>>
>>>>     (funcall (let ((x 0)) ; lexical binding
>>>>                 (lambda () x)))  --> 0
>>>>
>>>>     x --> 42
>>>>
>>>>     (funcall (lambda () x)) --> undefined variable error.
>>>>     (funcall (lambda () (declare (special x)) x)) --> 42
>>>>
>>>> But that would mean changing some implementations…
>>>
>>> I'm afraid I don't follow, and part of the reason is that you didn't use SETF.
>>>
>>> If the form x nicely evaluates to 42 at the top level, why would it flake out
>>> in the body of a lambda.
>>
>> Because it has never been declared special, therefore it's a lexical
>> reference, and unfortunately, it's a free variable with no lexical
>> binding.
>
> Then why is it not a lexical reference at top level, where it resolves to 42?

Because there exists no toplevel lexical variables.


> It looks as if the existence of a lexical scope is shadowing the reference,
> even though that lexical scope doesn't bind x.

The lexical scope in question cannot shadow any toplevel variable X,
since there is none defined.


But perhaps it's better that those things be undefined, so that we just
don't use them.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

On 2012-06-10, Marco Antoniotti <marcoxa@gmail.com> wrote:
> On Sunday, June 10, 2012 5:58:51 PM UTC+2, Kaz Kylheku wrote:
>> On 2012-06-10, Pascal J. Bourguignon 
>>  wrote:
>> > Kaz Kylheku <kaz@kylheku.com> writes:
>> >
>> >> On 2012-06-10, Pascal J. Bourguignon 
>>  wrote:
>> >>> Well,
>> >>>
>> >>>     (defvar x) == (declaim (special x))
>> >>>
>> >>> and:
>> >>>
>> >>>     (defvar x)  == (declaim (special x))
>> >>>     (setf x 42) == (setf (symbol-value 'x) 42)
>> >>>
>> >>> so we could specify that without a definition for x,
>> >>>
>> >>>     (setf x 42) == (setf (symbol-value 'x) 42)
>> >>
>> >> If x doesn't exist, then basically we are equating two undefined behaviors. 
>> >
>> > No.  (setf (symbol-value 'x) 42)  is always well defined.
>> 
>> That's not what I'm reading in the CLHS. The specification for symbol-value
>> clearly states that the input symbol "must have a value", and that it "[s]hould
>> signal unbound-variable if symbol is unbound and an attempt is made to read its
>> value." Furthermore, "[N]o such error is signaled on an attempt to write its
>> value." Note how it doesn't simply say that writing is a well-defined
>> operation, only that it is undiagnosed, and that it is only an attempt.
>> 
>> If you want to regard that as a defect, I'm with you.
>> 
>
> I think that PJB is trying to make a statement about (SETF (SYMBOL-VALUE 'X) ...)
> In this case, you are not "evaluating" the call to SYMBOL-VALUE, therefore you are not bound by the spec passage you mention.

SYMBOL-VALUE and (SETF SYMBOL-VALUE) are described together, as an accessor.
There is no text there which restricts the applicability of the passage to just
the reader.

-- 
If you ever need any coding done, I'm your goto man!

On 2012-06-10, Pascal J. Bourguignon <pjb@informatimago.com> wrote:
> Kaz Kylheku <kaz@kylheku.com> writes:
>
>> On 2012-06-10, Pascal J. Bourguignon <pjb@informatimago.com> wrote:
>>> Kaz Kylheku <kaz@kylheku.com> writes:
>>>
>>>> On 2012-06-10, Pascal J. Bourguignon <pjb@informatimago.com> wrote:
>>>>> Well,
>>>>>
>>>>>     (defvar x) == (declaim (special x))
>>>>>
>>>>> and:
>>>>>
>>>>>     (defvar x)  == (declaim (special x))
>>>>>     (setf x 42) == (setf (symbol-value 'x) 42)
>>>>>
>>>>> so we could specify that without a definition for x,
>>>>>
>>>>>     (setf x 42) == (setf (symbol-value 'x) 42)
>>>>
>>>> If x doesn't exist, then basically we are equating two undefined behaviors. 
>>>
>>> No.  (setf (symbol-value 'x) 42)  is always well defined.
>>
>> That's not what I'm reading in the CLHS. The specification for symbol-value
>> clearly states that the input symbol "must have a value", and that it "[s]hould
>> signal unbound-variable if symbol is unbound and an attempt is made to read its
>> value." Furthermore, "[N]o such error is signaled on an attempt to write its
>> value." Note how it doesn't simply say that writing is a well-defined
>> operation, only that it is undiagnosed, and that it is only an attempt.
>>
>> If you want to regard that as a defect, I'm with you.
>
> You're confusing the specification of the reader with that of the
> writer.  Here I'm using the writer.
>
> Just _read_ the CLHS!

The reader and writer are not separately specified.

  Accessor SYMBOL-VALUE

  Syntax:

  symbol-value symbol => value

  (setf (symbol-value symbol) new-value)

  Arguments and Values:

  symbol---a symbol that must have a value.

  value, new-value---an object. 

Where does it say that the symbol must have a value when the
reader is being used, but not when the writer is being used?

Are we not looking at the same document?

>     Should signal unbound-variable if symbol is unbound and an attempt
>     is made to read its value. (No such error is signaled on an attempt
>     to write its value.)

Why would an operation which is expected to succeed unconditionally be
described as an "attempt"? Why is not written that if the symbol is unbound,
writing the value is okay and makes it bound?

If writing to a symbol was unconditionally okay, the above wording would be a
very, very poor way document that requirement.

> Notice that after calling (setf symbol-value), the symbol is bound, and
> you can call symbol-value.

I'm afraid this is a hack outside of the standard, just like SETF-ing an
undefined variable.

If the symbol is bound, and then later that symbol is evaluated as a form,
what kind of variable is it? dynamic or lexical?

  If a form is a symbol that is not a symbol macro, then it is the name of a
  variable, and the value of that variable is returned. There are three kinds of
  variables: lexical variables, dynamic variables, and constant variables. A
  variable can store one object. The main operations on a variable are to read[1]
  and to write[1] its value.

  An error of type unbound-variable should be signaled if an unbound variable is referenced. 
  [3.1.2.1.1 Symbols as Forms]

If you think it's dynamic, then how did it get that way? Neither of the two
conditions for making a dynamic variable have been met.

  3.1.2.1.1.2 Dynamic Variables

  A variable is a dynamic variable if one of the following conditions hold:

  * It is locally declared or globally proclaimed special.

  * It occurs textually within a form that creates a dynamic binding for a
    variable of the same name, and the binding is not shadowed[2] by a form
    that creates a lexical binding of the same variable name. 
  [...]
  A dynamic variable is unbound unless and until explicitly assigned a value,
  except for those variables whose initial value is defined in this
  specification or by an implementation. 

I think even this is not conforming:

  (defvar foo) ;; no initial value

  (setf (symbol-value 'foo) 42) ;; oops, "symbol must have value" requirement violated

The definition of symbol-value does not say that it's okay to set 
an unbound symbol if that symbol is a dynamic variable with no value.

According to the definition of a dynamic variable above, the only way an
unbound dynamic variable can acquire a value is by assignment,
and (setf symbol-value) is not an assignment.

>> Then why is it not a lexical reference at top level, where it resolves to 42?
>
> Because there exists no toplevel lexical variables.

That question was badly worded, oops.

>> It looks as if the existence of a lexical scope is shadowing the reference,
>> even though that lexical scope doesn't bind x.
>
> The lexical scope in question cannot shadow any toplevel variable X,
> since there is none defined.

Yes, but in some sense it captures the reference. Simply by being wrapped in
the lambda, X in your example does not resolve to the same thing as an
unwrapped X. The wrapped reference is in a lexical environment which doesn't
define X, but somehow that interferes with X resolving to its global value.

That set of evaluation rules is nonsensical. I don't see where you are coming
from with that, or where it going.

Kaz Kylheku <kaz@kylheku.com> writes:

> That set of evaluation rules is nonsensical. I don't see where you are coming
> from with that, or where it going.

They would give a meaning to a toplevel  (setf x 42)  when x is not
defined, that would not interfer with lexical bindings of X.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Kaz Kylheku <kaz@kylheku.com> writes:

> […]
>   Arguments and Values:
>
>   symbol---a symbol that must have a value.
> […]
> I think even this is not conforming:
>
>   (defvar foo) ;; no initial value
>
>   (setf (symbol-value 'foo) 42) ;; oops, "symbol must have value" requirement violated

That would be worth a clarifying ISSUE.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

On 2012-06-10, Pascal Costanza <pc@p-cos.net> wrote:
> Also compare to the entry for SET, which doesn't mention any error 
> situation and specifies SET and (SETF SYMBOL-FUNCTION) to be equivalent.

I'm afraid this doesn't help you.

Just because SET doesn't mention any error condition doesn't mean there isn't one.
Because it is equivalent to SETF SYMBOL-FUNCTION, it inherits all of the issues
of that equivalent form.

Furthermore, SET is defined as changing the contents of the "value cell". What
is a value cell?

  value cell n. Trad. (of a symbol) The place which holds the value, if any, of
  the dynamic variable named by that symbol, and which is accessed by
  symbol-value. See cell. 

A symbol which is unbound has no value cell, and cannot be accessed by symbol-value.
(Oh, and it is a dynamic variable which may have a value cell! If we intern a
symbol out of thin air, it is not a dynamic variable. Oops!)

Look, I have a lot of experience reading various standards. The thing about
standards is that ultimately you have to read what is written there, not what
you wish should be written there.

This whole business of dynamic-variables /is/ slightly over-wrought and a
little bit stinky.

I suspect that programs which store values in a symbol via symbol-function are
quite portable.  I would not think twice about doing it. It's just that the
behavior is not codified in the standard.

You know, who is to say that every single thing which as de-facto portable
before the standard was codified?

Quite definitely, it should be permissible to set the value of an unbound
dynamic variable using SET and SYMBOL-VALUE. If the variable name X resolves to
a dynamic variable in the environment of the given context, then (SET 'X ...)
should be equivalent to (SETF X ...).

I'm coming to the conclusion, though, that I do not disagree with (setf
(symbol-value 'never-seen-before) 42) being erroneous. It's a good idea to
diagnose this kind of thing. Diagnosing assignments to nonexistent variables
can find bugs in programs.

  (defvar misspelled)

  ;; error signaled here saves your ass:
  (setf misspeled 42)

If it saves your ass there, it likewise saves your ass here:

  (setf (symbol-value 'misspeled) 42)

Kaz Kylheku <kaz@kylheku.com> writes:

I assume you meant s/SYMBOL-FUNCTION/symbol-value/g


> On 2012-06-10, Pascal Costanza <pc@p-cos.net> wrote:
>> Also compare to the entry for SET, which doesn't mention any error 
>> situation and specifies SET and (SETF SYMBOL-FUNCTION) to be equivalent.
>
> I'm afraid this doesn't help you.
>
> Just because SET doesn't mention any error condition doesn't mean there isn't one.
> Because it is equivalent to SETF SYMBOL-FUNCTION, it inherits all of the issues
> of that equivalent form.
>
> Furthermore, SET is defined as changing the contents of the "value cell". What
> is a value cell?
>
>   value cell n. Trad. (of a symbol) The place which holds the value, if any, of
>   the dynamic variable named by that symbol, and which is accessed by
>   symbol-value. See cell. 
>
> A symbol which is unbound has no value cell, and cannot be accessed by symbol-value.
> (Oh, and it is a dynamic variable which may have a value cell! If we intern a
> symbol out of thin air, it is not a dynamic variable. Oops!)

No, you read it wrong again.

  "The place which holds the value, if any, of the dynamic variable
   named by that symbol, …"

means:

   There is a place.
   That place may hold a value or not.
   If it holds a value, it's the value of the dynamic variable named by
   that symbol.

The place always exists, that's why you can always evaluate:

   (setf (symbol-value sym) value)
         ^^^^^^^^^^^^^^^^^^
               |
               +--- this is a place!


The presence of absence of value in that place can be manipulated with
boundp, makunbound, and set/setq/setf of that place.

Furthermore, set/setq/setf of that place may change the value bound to
that place.



> Look, I have a lot of experience reading various standards. The thing about
> standards is that ultimately you have to read what is written there, not what
> you wish should be written there.
>
> This whole business of dynamic-variables /is/ slightly over-wrought and a
> little bit stinky.
>
> I suspect that programs which store values in a symbol via symbol-function are
> quite portable.  I would not think twice about doing it. It's just that the
> behavior is not codified in the standard.

Yes, perhaps it was so obvious, the comitee had difficulties formalize
it.


> You know, who is to say that every single thing which as de-facto portable
> before the standard was codified?

Well, that was the purpose of the CL standardization efford.


> Quite definitely, it should be permissible to set the value of an unbound
> dynamic variable using SET and SYMBOL-VALUE. If the variable name X resolves to
> a dynamic variable in the environment of the given context, then (SET 'X ...)
> should be equivalent to (SETF X ...).
>
> I'm coming to the conclusion, though, that I do not disagree with (setf
> (symbol-value 'never-seen-before) 42) being erroneous. It's a good idea to
> diagnose this kind of thing. Diagnosing assignments to nonexistent variables
> can find bugs in programs.
>
>   (defvar misspelled)
>
>   ;; error signaled here saves your ass:
>   (setf misspeled 42)
>
> If it saves your ass there, it likewise saves your ass here:
>
>   (setf (symbol-value 'misspeled) 42)

Only, what you have usually is (setf (symbol-value sym) 42) and here
there's no mispelling risk.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.