Dear,

Does someone know if there is any work done to generate an inverse of
any given lisp function (if possible).

Example:

> (defun f (x)  (+ x 1))
> (f 2)
3
> (funcall (inverse #'f) 3)
2
> (funcall (inverse #'f) (f 5))
5

When a lisp function has conditional branchs then the inverse function
will
have a code to test both paths (eg. choice and fail constructs).

This example is very simple and serves only to describe my needs; I
would like to use the inverse function
for complex lisp defined functions.

Thank you for your help

On 2010-02-13 17:27:43 +0000, Taoufik said:

> Does someone know if there is any work done to generate an inverse of
> any given lisp function (if possible).

Inverses only exist for functions which are 1-1 (is this an injection?  
I think it is).  So many functions do not have them.  Even for those 
that do, computing the inverse based on the definition is probably 
intractable in general.

Taoufik <taoufik@mazeboard.com> writes:

> Dear,
>
> Does someone know if there is any work done to generate an inverse of
> any given lisp function (if possible).
>
> Example:
>
>> (defun f (x)  (+ x 1))
>> (f 2)
> 3
>> (funcall (inverse #'f) 3)
> 2
>> (funcall (inverse #'f) (f 5))
> 5
>
> When a lisp function has conditional branchs then the inverse function
> will have a code to test both paths (eg. choice and fail constructs).
>
> This example is very simple and serves only to describe my needs; I
> would like to use the inverse function
> for complex lisp defined functions.
>
> Thank you for your help


What is (inverse (constantly 1)) ?


Otherwise, there's not a lot of work on this matter.  (Google returns
more article for:
   "inverse turing machine" 
than for:
   "inverse function" "lambda calculus"
)

Have a look at:
    http://historical.ncstrl.org/litesite-data/stan/CS-TR-73-390.pdf
    http://www.springerlink.com/index/jx1v04262u084382.pdf 


-- 
__Pascal Bourguignon__

Taoufik <taoufik@mazeboard.com> writes:

> Dear,
>
> Does someone know if there is any work done to generate an inverse of
> any given lisp function (if possible).
>
> Example:
>
>> (defun f (x)  (+ x 1))
>> (f 2)
> 3
>> (funcall (inverse #'f) 3)
> 2
>> (funcall (inverse #'f) (f 5))
> 5
>
> When a lisp function has conditional branchs then the inverse function
> will have a code to test both paths (eg. choice and fail constructs).
>
> This example is very simple and serves only to describe my needs; I
> would like to use the inverse function for complex lisp defined
> functions.

When I suggested you on the OpenMCL mailing list to go to comp.lang.lisp
with this question it was not that I thought you were answered
inappropriate there.  In fact, the same people answering you there would
have answered you probably here as well.  It was because I think that it
is worthwile and amusing to think also about such things and
comp.lang.lisp is a broader audience.

Nice answers have been given what you could do with such a function, Ron
Garret showing that you could decrypt RSA, Robert Goldman mentioned that
you would have solved the halting problem, Pascal Bourguignon showed
that the answer could easily be an infinite set, rather obviously it
could also solve every problem of number theory (e.g. for Fermat's last
problem set f(a,b,c,n)=c^n-a^n-b^n and ask for those (a,b,c,n) in
f^{-1}(0)=0 where
n>2.)

I liked Tobias Rittweiler's answer best.  Below is my variation of it
which satisfies the requirements you pose above (although it is probably
not what you want:-) with only slightly modified syntax.

Nicolas

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(defun memoize-one-argument-function (funsym)
  (let ((unmemoized (symbol-function funsym)))
    (symbol-macrolet ((table (get funsym :memoization-table)))
      (setf table ())
      (setf (symbol-function funsym)
            (lambda (x)
              (let ((entry (assoc x table)))
                (if entry
                    (cdr entry)
                    (cdar (pushnew (cons x (funcall unmemoized x))
                                   table)))))))))

(defun inverse (funsym)
  (lambda (y)
    (let ((entries (remove-if-not (lambda (entry)
                                    (eql (cdr entry) y))
                                  (get funsym :memoization-table))))
      (cond
        ((null entries) (error "Sorry - not yet defined"))
        ((= (length entries) 1) (caar entries))
        (t (mapcar #'car entries))))))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(defun f (x) (1+ x))

;; this line has to be inserted
(memoize-one-argument-function 'f)

;; and inverse has to be called with 'f instead of #'f
(f 2)
(funcall (inverse 'f) 3)
(funcall (inverse 'f) (f 5))

+ Taoufik <taoufik@mazeboard.com>:

> Does someone know if there is any work done to generate an inverse of
> any given lisp function (if possible).

As must be clear from the answers so far, it is impossible in general.
However, for specialized problem domains, such as linear algebra,
something likt it is not only possible but frequently desirable. For one
example, consider Knuth's metafont, which is a program to define fonts
for use in typesetting. It contains a fairly general facility for doing
linear algebra. You just give it linear equations involving a number of
variables, and once enough equations have been given to fix the value of
some variable, that variable can now be used freely. This facility is
amazingly useful when creating graphics, and it might be a fun as well
as useful exercise to implement such a library in lisp. I often miss
something like this when using other graphics packages. One thing you
wouldn't do, though, is to structure this around DEFUNs defining linear
algebra. The whole style is declarative more than imperative, and it
requires primitives to match that style. (BTW, there is a graphics
drawing program called metapost which is based on the metafont language
but which produces postscript output.)

-- 
* Harald Hanche-Olsen     <URL:http://www.math.ntnu.no/~hanche/>
- It is undesirable to believe a proposition
  when there is no ground whatsoever for supposing it is true.
  -- Bertrand Russell

On Feb 13, 12:27=A0pm, Taoufik <taou...@mazeboard.com> wrote:
> Dear,
>
> Does someone know if there is any work done to generate an inverse of
> any given lisp function (if possible).
>
> Example:
>
> > (defun f (x) =A0(+ x 1))
> > (f 2)
> 3
> > (funcall (inverse #'f) 3)
> 2
> > (funcall (inverse #'f) (f 5))
>
> 5
>
> When a lisp function has conditional branchs then the inverse function
> will
> have a code to test both paths (eg. choice and fail constructs).
>
> This example is very simple and serves only to describe my needs; I
> would like to use the inverse function
> for complex lisp defined functions.
>
> Thank you for your help

Easy! Just guess randomly until you get it right.

(defun invert (f guess)
  (lambda (x)
    (loop for y =3D (funcall guess)
        when (=3D x (funcall f y))
        return y)))

(defun my-guess () (random 1000))

> (funcall (invert (lambda (x) (+ 1 x)) #'my-guess) 100)
99

Kyle

A good (though not easy to use) library for solving linear equation systems
is LAPACK. It originally is coded in FORTRAN. I (still) use the Java port but
I even found a LISP port in the maxima source tree:

http://sage.math.washington.edu/home/wstein/tmp/maxima-5.16.3.p2/src/share/lapack/

Norbert

Harald Hanche-Olsen wrote:
> + Taoufik <taoufik@mazeboard.com>:
> 
>> Does someone know if there is any work done to generate an inverse of
>> any given lisp function (if possible).
> 
> As must be clear from the answers so far, it is impossible in general.
> However, for specialized problem domains, such as linear algebra,
> something likt it is not only possible but frequently desirable. For one
> example, consider Knuth's metafont, which is a program to define fonts
> for use in typesetting. It contains a fairly general facility for doing
> linear algebra. You just give it linear equations involving a number of
> variables, and once enough equations have been given to fix the value of
> some variable, that variable can now be used freely. This facility is
> amazingly useful when creating graphics, and it might be a fun as well
> as useful exercise to implement such a library in lisp. I often miss
> something like this when using other graphics packages. One thing you
> wouldn't do, though, is to structure this around DEFUNs defining linear
> algebra. The whole style is declarative more than imperative, and it
> requires primitives to match that style. (BTW, there is a graphics
> drawing program called metapost which is based on the metafont language
> but which produces postscript output.)
> 

I suppose that even your "(if possible)" restriction is untractable:

Assume a unary function #'uninvertible, where inversion is impossible, existed
and assume a predicate #'if-possible-p (where "if" stamds for "Inversion of
Function"). Then do

   (defun mean-function (x)
      (if (if-possible-p #'mean-function)
          (uninvertable x)
          (identity x) ) )

So when #'if-possible-p says that #'mean-function is invertable then it
is not (as it behaves identically to #'uninvertable) and vice versa (as then
it behaves like the invertible function #'identity. So #'if-possible-p at
least errs with #'mean-function.

With a similar argument, I guess, you can demonstrate the existance of such
an uninvertable function. Maybe even an uninvertable bijection exists.
I didn't yet care about that.

So, anyway, you must restrict yourself to an incomplete set of /explicitly
declared/ invertible functions, maybe registered in a hash table. You might
then also define a macro like

(definvertable fun (x)
    <body>
    <inverse-body> )

Note that 'invertability depends on arity and domain of the function.
A unary #'+  ( +: T -> T) is invertible, whereas a binary #'+
(or +:T^2 -> T ) is not.
Also #'not is invertible if applied to {nil, T} only.
#'list and #'cons are invertible, if you consider the argument list one
argument and are satisfied with #'equal results.

Then there may also be rules which tell what combinations of invertible
functions themselves yield invertible function. For example composition of
invertible functions is invertible. I strongly suppose that this set of rules
also will always be incomplete.


Norbert.

Taoufik wrote:
> Dear,
> 
> Does someone know if there is any work done to generate an inverse of
> any given lisp function (if possible).
> 
> Example:
> 
>> (defun f (x)  (+ x 1))
>> (f 2)
> 3
>> (funcall (inverse #'f) 3)
> 2
>> (funcall (inverse #'f) (f 5))
> 5
> 
> When a lisp function has conditional branchs then the inverse function
> will
> have a code to test both paths (eg. choice and fail constructs).
> 
> This example is very simple and serves only to describe my needs; I
> would like to use the inverse function
> for complex lisp defined functions.
> 
> Thank you for your help

I suppose that even your "(if possible)" restriction is untractable:

Assume a unary function #'uninvertible, where inversion is impossible, existed
and assume a predicate #'if-possible-p (where "if" stamds for "Inversion of
Function"). Then do

   (defun mean-function (x)
      (if (if-possible-p #'mean-function)
          (uninvertable x)
          (identity x) ) )

So when #'if-possible-p says that #'mean-function is invertable then it
is not (as it behaves identically to #'uninvertable) and vice versa (as then
it behaves like the invertible function #'identity. So #'if-possible-p at
least errs with #'mean-function.

With a similar argument, I guess, you can demonstrate the existance of such
an uninvertable function.

So, anyway, you must restrict yourself to an incomplete set of /explicitly
declared/ invertible functions, maybe registered in a hash table. You might
then also define a macro like

(definvertable fun (x)
    <body>
    <inverse-body> )

Note that 'invertability depends on arity and domain of the function.
A unary #'+  ( +: T -> T) is invertible, whereas a binary #'+
(or +:T^2 -> T ) is not.
Also #'not is invertible if applied to {nil, T} only.
#'list and #'cons are invertible, if you consider the argument list one
argument and are satisfied with #'equal results.

Then there may also be rules which tell what combinations of invertible
functions themselves yield invertible function. For example composition of
invertible functions is invertible. I strongly suppose that this set of rules
also will always be incomplete.


Norbert.

Taoufik wrote:
> Dear,
> 
> Does someone know if there is any work done to generate an inverse of
> any given lisp function (if possible).
> 
> Example:
> 
>> (defun f (x)  (+ x 1))
>> (f 2)
> 3
>> (funcall (inverse #'f) 3)
> 2
>> (funcall (inverse #'f) (f 5))
> 5
> 
> When a lisp function has conditional branchs then the inverse function
> will
> have a code to test both paths (eg. choice and fail constructs).
> 
> This example is very simple and serves only to describe my needs; I
> would like to use the inverse function
> for complex lisp defined functions.
> 
> Thank you for your help

On 13 Feb, 17:27, Taoufik <taou...@mazeboard.com> wrote:
> Dear,
>
> Does someone know if there is any work done to generate an inverse of
> any given lisp function (if possible).
>
> Example:
>
> > (defun f (x) =A0(+ x 1))
> > (f 2)
> 3
> > (funcall (inverse #'f) 3)
> 2
> > (funcall (inverse #'f) (f 5))
>
> 5
>
> When a lisp function has conditional branchs then the inverse function
> will
> have a code to test both paths (eg. choice and fail constructs).
>
> This example is very simple and serves only to describe my needs; I
> would like to use the inverse function
> for complex lisp defined functions.
>
> Thank you for your help

This kind of thing is much more tractable in Prolog where f(x) =3D y is
represented as a dyadic predicate f(X,Y). In theory one can run this
in either direction using either X or Y as the output variable.  It
would be possible to build a Lisp interpreter in Prolog which could be
'run backwards' as it were.  In the case of many-one functions
multiple values could be returned.

In practice though, there are three problems with doing this even in
Prolog.  The first is that most Prolog programs are not pure Horn
clauses but use the arithmetic engine and this is very directional.
So you have to limit yourself to pure Prolog and use the successor
notation for arithmetic.  Second, multiple values may in certain cases
be infinite.  Third, even in pure Prolog there are programs that
cannot be 'run backward'  without engendering an infinite regress.  an
example of this is the naive reverse program in Prolog.

Often this is a result of the incomplete search strategy of Prolog so
you have to build a loop detector into your Prolog which traps some of
these loops.  This is called a subsumption test. It's a nice exercise
to experiment with representing functional programming in logic
programming and will probably teach you a lot about the limits of
computing inverses.

Mark

On Sat, 13 Feb 2010 17:40:13 +0000, Tim Bradshaw <tfb@tfeb.org> wrote:

>On 2010-02-13 17:27:43 +0000, Taoufik said:
>
>> Does someone know if there is any work done to generate an inverse of
>> any given lisp function (if possible).
>
>Inverses only exist for functions which are 1-1 (is this an injection?  
>I think it is).  So many functions do not have them.  Even for those 
>that do, computing the inverse based on the definition is probably 
>intractable in general.

I have my own private library for doing this for simple arithmetic
expressions; I think I've used it about twice: once for the orginal
appication back in 1999, and then I found it useful for something else
in 2005. (Both spare-time "hobby programming" -- I doubt many of us
still work full time writing lisp code! :-/ ) But it's not production
quality code... I tend to write code which says "error - this bit not
implemented yet" and only implement the bits I actually need at the
time! (It's a good idea, if you think it might be useful later, to
design as if you intend to implement the whole thing, and put in lots of
checks for special cases, even if they can't occur in the current
application. Otherwise they'll bite you later, when you've forgotten.)

It was useful for writing transformations between coordinate systems in
a graphics application. It was only necessary to write the
transformation in one direction.

Incidentally, it doesn't produce an inverse *function* -- the scaling or
transformation was written as a macro, and the inverse was generated by
another macro, so I could pass in the name of the macro I wanted an
inverse for, and use macroexpand to look at it.

For example:
(defmacro scale (n)
   `(1+ (* (1- ,n) *sfactor*)))

then
    (setf y-prime (scale y))
is forwards, and
    (setf y (invert scale y-prime))
is backwards.

Anyway, all that's a comment on the original question. I'm following up
to Tim Bradshaw because, while it's true that there may not be a unique
inverse, that might not matter, in the sense that *any* inverse might be
suitable. For a very simple case, sqrt is "the" inverse of (expt n 2),
except that minus the square root is also an inverse. Yet the sqrt
function is still useful.

Once or twice I've attempted to write an inverse of (Conway's) life.
(The cellular automaton thingy.) It was inspired by a story in which
some guy sent secret messages as apparently random starting patterns in
life, but which after a few hundred generations evolved into words
(spelled out as patterns of dots, or live cells).

It would be really cool to come up with some minimal pattern of live
cells which, after a hundred or a thousand generations spelled out my
name. All you need to do is set up your name on the grid, and then run
life backwards for a hundred generations.

Of course, some patterns are "garden of Eden" patterns: there is no
possible previous generation. But usually there are many. (In
particular, you can sprinkle isolated live cells throughout any large
empty space when generating the previous generation: they'll all die
out.) So the interesting question is to find a minimal starting pattern.

And that's a hard problem.

Jonathan

-- 
Writers: Many are called, but few are chosen.

Mark Tarver wrote:
> This kind of thing is much more tractable in Prolog where f(x) = y is
> represented as a dyadic predicate f(X,Y). In theory one can run this
> in either direction using either X or Y as the output variable.  It
> would be possible to build a Lisp interpreter in Prolog which could be
> 'run backwards' as it were.  In the case of many-one functions
> multiple values could be returned.

In SWI-Prolog you can even enter X which answers 42. (And gives a nice
error message which informs you that the implementors are still working
on the question)

Anyway, you cannot overcome mathematical limitations by switching to
another language. But I suppose you didn't mean that either.

Norbert

On 2010-02-13 18:58:08 +0000, Pascal J. Bourguignon said:

> Otherwise, there's not a lot of work on this matter.  (Google returns
> more article for:
>    "inverse turing machine"
> than for:
>    "inverse function" "lambda calculus"
> )

There is a whole complexity here which I hadn't thought about at all.  
The laws of physics are reversible, so presumably any machine which 
obeys them also is reversible, if you think of it the right way.  Your 
example of CONSTANTLY is some kind of example of not thinking "the 
right way" about the problem because any non-invertible function throws 
information away, and that's not allowed in a reversible system.

There is a literature on reversible computation (which is quite closely 
related to the literature on quantum computing I think), but I haven't 
read anything on this in a long time.  The Wikipedia entry 
(http://en.wikipedia.org/wiki/Reversible_computing) looks quite good.

--tim

Tim Bradshaw <tfb@tfeb.org> writes:

> On 2010-02-13 18:58:08 +0000, Pascal J. Bourguignon said:
>
>> Otherwise, there's not a lot of work on this matter.  (Google returns
>> more article for:
>>    "inverse turing machine"
>> than for:
>>    "inverse function" "lambda calculus"
>> )
>
> There is a whole complexity here which I hadn't thought about at all.
> The laws of physics are reversible, so presumably any machine which
> obeys them also is reversible, if you think of it the right way.  Your
> example of CONSTANTLY is some kind of example of not thinking "the
> right way" about the problem because any non-invertible function
> throws information away, and that's not allowed in a reversible
> system.

The law of physics that are inversible are incomplete (ie they don't
apply to open systems).

The universe constantly throws information away too.  Or rather afar.
Therefore any system smaller than the universe loses information (even
the black holes, that should tell you something!).

You would have to consider the whole universe, from beginning to end, to
be able to enter eternity and use reversible laws.


Otherwise, we could use specific hardware, reversible computers, which
are computers that take care not to lose any bit of information.


On such a computer, constantly would be defined as:

(defun constantly (result)
   (lambda (&rest arguments) (values result arguments)))

and you wouldn't be allowed to lose values, so that you'd have to keep
all the arguments to your constant functions.



> There is a literature on reversible computation (which is quite
> closely related to the literature on quantum computing I think), but I
> haven't read anything on this in a long time.  The Wikipedia entry
> (http://en.wikipedia.org/wiki/Reversible_computing) looks quite good.

There's a nice paper about a garbage collector on such a reverse
computer.
http://www.pipeline.com/~hba-ker1/ReverseGC.html
See also:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.39.1005&rep=rep1&type=pdf
and
http://www.portal.acm.org/citation.cfm?id=1244404

The works about reversible hardware are often motivated by the energy
sparing they allow, and vice versa, it's by avoiding dissipating energy,
that is information far away in the universe that you can implement a
reversible system.


-- 
__Pascal Bourguignon__

Pascal J. Bourguignon <pjb@informatimago.com> wrote:
+---------------
| Tim Bradshaw <tfb@tfeb.org> writes:
| > There is a literature on reversible computation (which is quite
| > closely related to the literature on quantum computing I think), but I
| > haven't read anything on this in a long time.  The Wikipedia entry
| > (http://en.wikipedia.org/wiki/Reversible_computing) looks quite good.
| 
| There's a nice paper about a garbage collector on such a reverse
| computer.
| http://www.pipeline.com/~hba-ker1/ReverseGC.html
+---------------

Uh... That gets a 404; I think you meant:

  http://www.pipeline.com/~hbaker1/ReverseGC.html


-Rob

-----
Rob Warnock			<rpw3@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607

On 2010-02-19 17:26:32 +0000, Pascal J. Bourguignon said:

> The law of physics that are inversible are incomplete (ie they don't
> apply to open systems).

If I parse this correctly, this is mostly a tautology - if you're 
allowed to throw stuff into some non-considered bit of the system 
(which is what "open" means), then you can throw information away, 
because you can throw it there.

I don't mean this to say I disagree with you or I think your comment is 
wrong/stupid - I don't at all, I just want to make sure I understand 
what you're saying.

> 
> The universe constantly throws information away too.  Or rather afar.
> Therefore any system smaller than the universe loses information

Yes, that's right, or rather I think: any system which interacts with 
the universe may throw information away (typically as heat), because 
the universe provides this vast sink for information.

I think this all very related to measurement in QM.

>  (even
> the black holes, that should tell you something!).

The whole "no hair" thing is very interesting I think, especially when 
combined with Hawking radiation - because black holes *do* throw 
information away of course: you can throw completely arbitrary stuff 
into a black hole, and it will then give it back to you as 
black-body-spectrum radiation, having forgottoen everything but mass, 
charge and angular momentum. So much for reversibility...

I am no longer sure (if I ever was sure, it is now a long time since I 
had pretensions to be a physicist) if the loss of information is due to 
the event horizon or the singularity: I think that the event horizon 
provides some kind of partitioning, but the ultimate loss of 
information (as opposed to it merely sitting in some inaccessible bit 
of the state space) must be the singularity. And this makes me feel a 
bit better because singularities are so obviously bogus things in some 
way that there has to be some new physics in there which will avoid 
them.

Anyway, this is kind of off-topic now.

Tim Bradshaw <tfb@tfeb.org> writes:

> On 2010-02-19 17:26:32 +0000, Pascal J. Bourguignon said:
>
>> The law of physics that are inversible are incomplete (ie they don't
>> apply to open systems).
>
> If I parse this correctly, this is mostly a tautology - if you're
> allowed to throw stuff into some non-considered bit of the system
> (which is what "open" means), then you can throw information away,
> because you can throw it there.
>
> I don't mean this to say I disagree with you or I think your comment
> is wrong/stupid - I don't at all, I just want to make sure I
> understand what you're saying.

Yes, you understood correctly.

Too often people forget the pre-conditions of the theorems (or "laws").


>> The universe constantly throws information away too.  Or rather afar.
>> Therefore any system smaller than the universe loses information
>
> Yes, that's right, or rather I think: any system which interacts with
> the universe may throw information away (typically as heat), because
> the universe provides this vast sink for information.
>
> I think this all very related to measurement in QM.
>
>>  (even
>> the black holes, that should tell you something!).
>
> The whole "no hair" thing is very interesting I think, especially when
> combined with Hawking radiation - because black holes *do* throw
> information away of course: you can throw completely arbitrary stuff
> into a black hole, and it will then give it back to you as
> black-body-spectrum radiation, having forgottoen everything but mass,
> charge and angular momentum. So much for reversibility...
>
> I am no longer sure (if I ever was sure, it is now a long time since I
> had pretensions to be a physicist) if the loss of information is due
> to the event horizon or the singularity: I think that the event
> horizon provides some kind of partitioning, but the ultimate loss of
> information (as opposed to it merely sitting in some inaccessible bit
> of the state space) must be the singularity. And this makes me feel a
> bit better because singularities are so obviously bogus things in some
> way that there has to be some new physics in there which will avoid
> them.
>
> Anyway, this is kind of off-topic now.

Well, not if you believe the universe is simulation in a computer,
what's more, written in Lisp! :-)  
(Even if some believe it's hacked in perl).

-- 
__Pascal Bourguignon__

On 2010-02-22 11:37:01 +0000, Pascal J. Bourguignon said:

> Well, not if you believe the universe is simulation in a computer,
> what's more, written in Lisp! :-)

This depends on who you ask I think.
Mathemeticians probably think it's written in some pure functional language.
Theoretical physicists think it is written in Common Lisp (specifically 
not any other Lisp dialect).
Expermintal physicists think it is FORTRAN (specificlly FORTRAN 66)
Biologists think it is Perl using some statistics package
Chemists COBOL

--tim

Tim Bradshaw <tfb@tfeb.org> writes:

> Biologists think it is Perl using some statistics package

And I thought they would suggest APL.
-- 
  (espen)

On 2010-02-22 12:14:08 +0000, Espen Vestre said:

> And I thought they would suggest APL.

That's better: biologists APL and chemists Perl (or maybe the other way 
around, but I can never tell biologists and chemists apart: they both 
spend their time manfacturing illicit drugs in their garage don't 
they?).  Polititians COBOL, which explains a lot.

Don't forget corporate officials: Excel and Powerpoint.

Tim Bradshaw wrote:
> On 2010-02-22 11:37:01 +0000, Pascal J. Bourguignon said:
>
>> Well, not if you believe the universe is simulation in a computer,
>> what's more, written in Lisp! :-)
>
> This depends on who you ask I think.
> Mathemeticians probably think it's written in some pure functional
> language.
> Theoretical physicists think it is written in Common Lisp (specifically
> not any other Lisp dialect).
> Expermintal physicists think it is FORTRAN (specificlly FORTRAN 66)
> Biologists think it is Perl using some statistics package
> Chemists COBOL

On 2010-02-22 13:59:24 +0000, Norbert_Paul said:

> Don't forget corporate officials: Excel and Powerpoint.

Actually, of course, the secret truth is that the banking crisis is 
largely due to people using Excel as a database.

--tim

(It terrifies me that this might be true)

I have even been told of an applicant who wasn't hired as
software developper because he refused to accept that Excel
is a DBMS.

However, what you said might be a secret, but it is not a truth.
Using real men DBMS would have increased their efficiency in
running us into an even deeper crisis.

Norbert

Tim Bradshaw wrote:
> On 2010-02-22 13:59:24 +0000, Norbert_Paul said:
>
>> Don't forget corporate officials: Excel and Powerpoint.
>
> Actually, of course, the secret truth is that the banking crisis is
> largely due to people using Excel as a database.
>
> --tim
>
> (It terrifies me that this might be true)
>

In article <hlts3c$1om$1@news.eternal-september.org>,
 Tim Bradshaw <tfb@tfeb.org> wrote:

> On 2010-02-22 11:37:01 +0000, Pascal J. Bourguignon said:
> 
> > Well, not if you believe the universe is simulation in a computer,
> > what's more, written in Lisp! :-)
> 
> This depends on who you ask I think.
> Mathemeticians probably think it's written in some pure functional language.
> Theoretical physicists think it is written in Common Lisp (specifically 
> not any other Lisp dialect).
> Expermintal physicists think it is FORTRAN (specificlly FORTRAN 66)
> Biologists think it is Perl using some statistics package
> Chemists COBOL
> 
> --tim

The universe is written in QLisp (quantum Lisp), the fundamental 
construct of which is QLambda, which binds quantum wave functions and 
performs unitary transformations on them.  QCONS is also known as 
entanglement.  QCAR and QCDR are EPR "measurements".

I have the manual around here somewhere if anyone is interested.

;-)

rg

In article <hlu3vo$7tm$1@news.eternal-september.org>,
 Tim Bradshaw <tfb@tfeb.org> wrote:

> On 2010-02-22 13:59:24 +0000, Norbert_Paul said:
> 
> > Don't forget corporate officials: Excel and Powerpoint.
> 
> Actually, of course, the secret truth is that the banking crisis is 
> largely due to people using Excel as a database.
> 
> --tim
> 
> (It terrifies me that this might be true)

I have bad news for you.  I once worked as a consultant for someone who 
is a serious dude in the financial world.  Very famous.  Manages 
billions of dollars.  His primary model was an Excel spreadsheet.  Not 
only that, but it was the most horrible mess that you can imagine, an 
unspeakable monstrosity.  Dozens of separate sheets, many tens of 
thousands of cells, many of which led off into dead ends that never 
affected the final result.  And in the middle of it all were a set of 
arbitrary fudge factors that he tweaked according to his intuition, so 
that the end result was basically just a wild guess.

I really think that the vast majority of the people on Wall Street 
really are completely clueless.  The only question in my mind is whether 
they are *all* completely clueless, or if at the core there are a few 
guys who actually understand what's going on and keep things humming 
along.

rg

On 2010-02-22 17:52:50 +0000, Ron Garret said:

> I have bad news for you.  I once worked as a consultant for someone who
> is a serious dude in the financial world.  Very famous.  Manages
> billions of dollars.  His primary model was an Excel spreadsheet.  Not
> only that, but it was the most horrible mess that you can imagine, an
> unspeakable monstrosity.  Dozens of separate sheets, many tens of
> thousands of cells, many of which led off into dead ends that never
> affected the final result.  And in the middle of it all were a set of
> arbitrary fudge factors that he tweaked according to his intuition, so
> that the end result was basically just a wild guess.

This actually terrifies me less than it might.  He may have been a 
really rotten programmer (to the extent that he probably did not 
understand what he was doing was programming), but at least he was 
using a spreadsheet to do some kind of numerical modelling, which is 
what they're mostly for.  What just makes me feel ill are all the 
people using a spreadsheet (generally Excel of course) when they 
actually want a database, or a project planning tool, or a word 
processor or something.

> 
> I really think that the vast majority of the people on Wall Street
> really are completely clueless.  The only question in my mind is whether
> they are *all* completely clueless, or if at the core there are a few
> guys who actually understand what's going on and keep things humming
> along.

I agree with this.  My theory (about 3 years ago) was that, well, they 
used to suck up physics PhDs, and my experience of physicists (having 
been one for a short while) is that they are some of the best people I 
have ever known at really understanding how the things they're 
interested in work, and just not taking any bullshit at all[*].  So 
surely those people would have understood stuff?  But, in fact, I think 
that maybe if you are offered enough money to stop worrying about what 
your model means and just keep tweaking it until more money falls out, 
perhaps even physics people decide they don't need to really understand 
it: certainly it looks like they didn't.  (Or, perhaps they *did*, and 
they knew it would all fall to bits, but hopefully not until after 
their bonus.  Maybe that's even a reasonable, if selfish, strategy.)

--tim

[*] Having been a physicist can be very damaging in later life.  
Physicists (and maybe all hard scientists) tend to be very capable of 
*just saying when something is wrong* and equally *not minding when 
someone says some idea of yours is wrong* (because it's the *idea* 
that's wrong, not *you*).  This makes technical discussions go very 
quickly.   Then you try this with other tribes, and they suddenly go 
off in an enormous huff because they can't work this out and think that 
if you say their idea is wrong you are calling them stupid.  Instead 
you have to do all sorts of bullshit hint-dropping until they 
eventually realise that the idea is wrong themselves, and everything 
takes much, much longer.  Damn, I miss being a physicist more than 
anything, I think.  (There needs to be a reference to Erik here.)

In article <hlui10$qob$1@news.eternal-september.org>,
 Tim Bradshaw <tfb@tfeb.org> wrote:

> I agree with this.  My theory (about 3 years ago) was that, well, they 
> used to suck up physics PhDs, and my experience of physicists (having 
> been one for a short while) is that they are some of the best people I 
> have ever known at really understanding how the things they're 
> interested in work, and just not taking any bullshit at all[*].  So 
> surely those people would have understood stuff?  But, in fact, I think 
> that maybe if you are offered enough money to stop worrying about what 
> your model means and just keep tweaking it until more money falls out, 
> perhaps even physics people decide they don't need to really understand 
> it: certainly it looks like they didn't.  (Or, perhaps they *did*, and 
> they knew it would all fall to bits, but hopefully not until after 
> their bonus.  Maybe that's even a reasonable, if selfish, strategy.)

It's not only reasonable, it's *necessary*.  You can't succeed on Wall 
Street any other way.  The fundamental problem is that Wall Street's 
*customers* don't really understand what's going on, and that ignorance 
infects the system because ultimately enough people to the quality 
metric of who made the most money last year that nothing else matters.  
It's not much consolation when your competition is flying their private 
jets to their private islands to know that your strategy, if everyone 
had followed it, would not have led to collapse.  Because you can do the 
Right Thing and the collapse will happen anyway, because everyone *else* 
is an idiot.  So what's the point in being smart and prudent?

Damn, and this was shaping up to be such a good day.  I just got the 
last feature of the new version of lexicons to work.  Now look what 
you've gone and made me do!  Totally harshed my mellow.

> [*] Having been a physicist can be very damaging in later life.  
> Physicists (and maybe all hard scientists) tend to be very capable of 
> *just saying when something is wrong* and equally *not minding when 
> someone says some idea of yours is wrong* (because it's the *idea* 
> that's wrong, not *you*).  This makes technical discussions go very 
> quickly.   Then you try this with other tribes, and they suddenly go 
> off in an enormous huff because they can't work this out and think that 
> if you say their idea is wrong you are calling them stupid.

You mean like when someone suggests that there might be a better way to 
do namespacing than packages?

> Instead 
> you have to do all sorts of bullshit hint-dropping until they 
> eventually realise that the idea is wrong themselves, and everything 
> takes much, much longer.  Damn, I miss being a physicist more than 
> anything, I think.  (There needs to be a reference to Erik here.)

Eh, we've still got Kenny.

rg

On Mon, 22 Feb 2010 18:22:24 +0000, Tim Bradshaw wrote:

> On 2010-02-22 17:52:50 +0000, Ron Garret said:
> 
>> I really think that the vast majority of the people on Wall Street
>> really are completely clueless.  The only question in my mind is
>> whether they are *all* completely clueless, or if at the core there are
>> a few guys who actually understand what's going on and keep things
>> humming along.
> 
> I agree with this.  My theory (about 3 years ago) was that, well, they
> used to suck up physics PhDs, and my experience of physicists (having
> been one for a short while) is that they are some of the best people I
> have ever known at really understanding how the things they're
> interested in work, and just not taking any bullshit at all[*].  So
> surely those people would have understood stuff?  But, in fact, I think
> that maybe if you are offered enough money to stop worrying about what
> your model means and just keep tweaking it until more money falls out,
> perhaps even physics people decide they don't need to really understand
> it: certainly it looks like they didn't.  (Or, perhaps they *did*, and
> they knew it would all fall to bits, but hopefully not until after their
> bonus.  Maybe that's even a reasonable, if selfish, strategy.)

All empirical results so far indicate that markets process information
pretty efficiently.  That is to say, you can't beat markets
systematically (after adjusting for risk).  The qualifier, of course,
is "unless you have information no one else has", but most people don't,
and those who do are closely watched (eg insider trading regulations).

This is very hard to stomach for most people, who are looking for
fabulous extra returns.  Unfortunately, there are always people who
are willing to promise this, since they benefit from your investment
--- usually a fixed share of your assets, which can be quite low for a
plain vanilla index fund, and very large for actively managed funds
which promise higher returns (but fail to deliver systematically, when
adjusted for risk).  The market for investments is, in a certain
sense, like the market for laundry detergent or toothpaste: suppliers
make (amazing) claims which most buyers are not in the position to
evaluate or validate.

Physicists are employed in this industry because they have been
trained to deal with the math --- mostly continuous time differential
equations and statistics.  The rest is easy to pick up on a
superficial level, but lacking training in economics, what they are
doing is mostly mechanical.  The models they are using are not that
sophisticated, and since there is only so much information you can
extract from data using pure statistical methods (without economic
ingredients beyond no-arbitrage), you can't expect a lot from them.
For example, most pricing "models" calculate the "fundamentals" from
today's prices (basically inverting a bijection), which they use to
calculate other prices, but they don't really care if the
"fundamentals" are different from one minute to the next.

That said, Excel spreadsheets & such are certainly not to blame for
the current crisis, and neither are physicists without economic
training.  If you want to read about such things, look up Nicholas
Dunbar's Inventing Money: The story of Long-Term Capital Management
and the legends behind it, it is an entertaining book.

Tamas

In article <7ug3b0Ft9jU1@mid.individual.net>,
 Tamas K Papp <tkpapp@gmail.com> wrote:

> On Mon, 22 Feb 2010 18:22:24 +0000, Tim Bradshaw wrote:
> 
> > On 2010-02-22 17:52:50 +0000, Ron Garret said:
> > 
> >> I really think that the vast majority of the people on Wall Street
> >> really are completely clueless.  The only question in my mind is
> >> whether they are *all* completely clueless, or if at the core there are
> >> a few guys who actually understand what's going on and keep things
> >> humming along.
> > 
> > I agree with this.  My theory (about 3 years ago) was that, well, they
> > used to suck up physics PhDs, and my experience of physicists (having
> > been one for a short while) is that they are some of the best people I
> > have ever known at really understanding how the things they're
> > interested in work, and just not taking any bullshit at all[*].  So
> > surely those people would have understood stuff?  But, in fact, I think
> > that maybe if you are offered enough money to stop worrying about what
> > your model means and just keep tweaking it until more money falls out,
> > perhaps even physics people decide they don't need to really understand
> > it: certainly it looks like they didn't.  (Or, perhaps they *did*, and
> > they knew it would all fall to bits, but hopefully not until after their
> > bonus.  Maybe that's even a reasonable, if selfish, strategy.)
> 
> All empirical results so far indicate that markets process information
> pretty efficiently.  That is to say, you can't beat markets
> systematically (after adjusting for risk).

First, that depends on how you define risk.  Economic models invariably 
define risk in terms of volatility, but that breaks badly when you're 
playing a martingale.  Second, even if no one can beat the market that 
is not necessarily because information is being processed efficiently.  
You can't beat the house in Vegas either, but that's not because 
everyone is processing the available information efficiently.  Third, 
Goldman Sachs does seem to have an uncanny ability to beat the markets 
(though the possibility that they're doing so through some shady means 
cannot be ruled out).  And fourth, if you have the U.S. Congress in your 
hip pocket you don't have to give a tinker's damn about the market 
because you can just shake down the taxpayers whenever you need to 
restock the minibar on the G5 with '78 Montrachet and you're feeling a 
little short.

> The qualifier, of course,
> is "unless you have information no one else has", but most people don't,
> and those who do are closely watched (eg insider trading regulations).

Again, that depends.  If you're hooked in to the right communications 
channels you can easily have information no one else has without running 
afoul of the SEC -- for short periods of time.  But that can be enough.

rg

On Mon, 22 Feb 2010 11:38:25 -0800, Ron Garret wrote:

> In article <7ug3b0Ft9jU1@mid.individual.net>,
>  Tamas K Papp <tkpapp@gmail.com> wrote:
> 
>> All empirical results so far indicate that markets process information
>> pretty efficiently.  That is to say, you can't beat markets
>> systematically (after adjusting for risk).
> 
> First, that depends on how you define risk.  Economic models invariably
> define risk in terms of volatility, but that breaks badly when you're

Economists usually understand risk via utility (using the concept of the
stochastic discount factor).  Volatility is used more in finance (of
course macroeconomics and finance are converging on various
frontiers).

> cannot be ruled out).  And fourth, if you have the U.S. Congress in your
> hip pocket you don't have to give a tinker's damn about the market
> because you can just shake down the taxpayers whenever you need to

Agreed.  In fact, you don't even need to have them in your hip pocket,
making everyone believe that you are "too big to fail" is enough for a
bailout.  This will remain a challenge for regulators in the future.

>> The qualifier, of course,
>> is "unless you have information no one else has", but most people
>> don't, and those who do are closely watched (eg insider trading
>> regulations).
> 
> Again, that depends.  If you're hooked in to the right communications
> channels you can easily have information no one else has without running
> afoul of the SEC -- for short periods of time.  But that can be enough.

Certainly.  But such possibilities are not available for most ordinary
people, and thus play no role in their investment strategies.

I would love to continue this discussion, but it is quite off-topic,
so I will stop here.

Tamas

Tim Bradshaw  <tfb@tfeb.org> wrote:
+---------------
| Pascal J. Bourguignon said:
| > The law of physics that are inversible are incomplete (ie they don't
| > apply to open systems).
....
| The whole "no hair" thing is very interesting I think, especially when 
| combined with Hawking radiation - because black holes *do* throw 
| information away of course: you can throw completely arbitrary stuff 
| into a black hole, and it will then give it back to you as 
| black-body-spectrum radiation, having forgottoen everything but mass, 
| charge and angular momentum. So much for reversibility...
| 
| I am no longer sure (if I ever was sure, it is now a long time since I 
| had pretensions to be a physicist) if the loss of information is due to 
| the event horizon or the singularity: I think that the event horizon 
| provides some kind of partitioning, but the ultimate loss of 
| information (as opposed to it merely sitting in some inaccessible bit 
| of the state space) must be the singularity. And this makes me feel a 
| bit better because singularities are so obviously bogus things in some 
| way that there has to be some new physics in there which will avoid them.
+---------------

This is certainly still an active open issue which is being investigated
by a number of researchers, see:

    http://en.wikipedia.org/wiki/Black_hole_information_paradox

    http://arxiv.org/abs/hep-th/0507171

    http://backreaction.blogspot.com/2010/02/black-holes-and-information-loss.html


-Rob

-----
Rob Warnock			<rpw3@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607

Pascal J. Bourguignon <pjb@informatimago.com> wrote:
+---------------
| Tim Bradshaw <tfb@tfeb.org> writes:
| > Anyway, this is kind of off-topic now.
| 
| Well, not if you believe the universe is simulation in a computer,
| what's more, written in Lisp! :-)  
| (Even if some believe it's hacked in perl).
+---------------

Is that a thinly veiled reference to this Randall Munroe classic?  ;-}

    http://xkcd.com/224/


-Rob

-----
Rob Warnock			<rpw3@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607

On 2010-02-22 21:56:42 +0000, Tamas K Papp said:

> To a certain extent, I agree with you.  But the problem is very hard,
> and it requires thinking in terms of economic models (vs purely
> statistical ones), and general equilibrium (instead of partial
> equilibrium concepts).  I don't think that anyone without serious
> training in economics has a chance.  And moreover, progress is quite
> slow, at least on the scale of financial companies.  They don't have
> the incentive to employ people who do basic research, so I don't blame
> them.

I think that's what disapoints me.  My view is that someone who really 
is a physicist is someone who would look at some problem domain (not 
just a physics domain) and make a really good stab at understanding it 
properly, more-or-less whatever it was.  Not someone who would just 
cobble together some na�ve and obviously wrong model and be happy with 
that. (The same is true for engineers I think.)  Of course economics is 
not easy, but it's not harder than physics (well, perhaps it is, 
because we don't actually have economic theories that really work very 
well, and maybe the ones that do work will turn out to be harder than 
physics, but I'd guess not).

> 
> Nowadays, most universities have a graduate program in "financial
> engineering" or similar, and graduates from these programs are
> starting to displace physicists (some would argue that they have
> already done so in certain areas).  Nowadays, students who want to
> work as quants can go straight to these programs, so physicists can
> concentrate on physics, which is just as well.

I don't know if this is a good or a bad thing.  Obviously good that 
banks will no longer be nicking all the good physics PhDs.

In the previous post I had stated somewhere that "with the same argument
you can even show that uninvertible bijections exist." But "uninvertible
bijection" sounded so stupid to me, that a cancelled the message and
replaced it with a verion with that potential bull#### censored out.

But, in fact, you can diagonalize #'invert by constructing an univertible
bijection from the assumption such #'invert existed:

(defun docile-minus (x)
   "A docile version of unary negation: non-numbers are simply returned"
   (if (numberp x)
       (- x)
       (identity x) ) )

This function inverts itself (IOW is an involution).

Now assume #'invert existed. Then define

(defun mean-docile-minus (x)
    "I like that having both 'mean' and 'docile' in the function name."
    (if (eql (funcall (invert #'mean-docile-minus) x) x)
        (docile-minus x)
        (identity x) ) )

Now #'mean-docile-minus either behaves like #'identity or like
#'docile-minus (homework: this does not depend on the argument x).
In both cases it is an involution, hence (invert #'mean-docile-minus)
and #'mean-docile-minus are both the same functions (not necessarily
identical).

case 1: #'mean-docile-minus be same as identity
   evaluate (mean-docile-minus 1):
     Then
       (eql (funcall (invert #'mean-docile-minus) 1) 1)
     is equivalent to
       (eql (funcall (invert #'identity) 1) 1)
     is equivalent to
       (eql (funcall #'identity 1) 1)
     which is
       (eql 1 1)
     which is
       true.
     Hence (mean-docile-minus 1) returns -1
     This is a contradiction to the assumption.

case 2: #'mean-docile-minus be same as docile-minus
   evaluate (mean-docile-minus 1):
     Then
       (eql (funcall (invert #'mean-docile-minus) 1) 1)
     is equivalent to
       (eql (funcall (invert #'docile-minus) 1) 1)
     is equivalent to
       (eql (funcall #'docile-minus 1) 1)
     which is
       (eql -1 1)
     which is
       false.
     Hence (mean-docile-minus 1) returns 1
     This is also a contradiction to the assumption.

So #'mean-docile-minus in fact behaves mean to #'invert making it
err at that point.

Therefore whith a function #'invert you can construct an
uninvertible bijeciton which is a contradiction.


Norbert_Paul wrote:
> I suppose that even your "(if possible)" restriction is untractable:
> ...

On Tue, 23 Feb 2010 09:43:04 +0000, Tim Bradshaw wrote:

> On 2010-02-22 21:56:42 +0000, Tamas K Papp said:
> 
>> To a certain extent, I agree with you.  But the problem is very hard,
>> and it requires thinking in terms of economic models (vs purely
>> statistical ones), and general equilibrium (instead of partial
>> equilibrium concepts).  I don't think that anyone without serious
>> training in economics has a chance.  And moreover, progress is quite
>> slow, at least on the scale of financial companies.  They don't have
>> the incentive to employ people who do basic research, so I don't blame
>> them.
> 
> I think that's what disapoints me.  My view is that someone who really
> is a physicist is someone who would look at some problem domain (not
> just a physics domain) and make a really good stab at understanding it
> properly, more-or-less whatever it was.  Not someone who would just
> cobble together some naïve and obviously wrong model and be happy with

I think that you may have an idealized image about physicists.  I can
understand why, since many physicists are people of scientific
integrity.  But getting a degree in physics does not imply that a
person will behave as a good scientist for the rest of his/her life.

>> Nowadays, most universities have a graduate program in "financial
>> engineering" or similar, and graduates from these programs are starting
>> to displace physicists (some would argue that they have already done so
>> in certain areas).  Nowadays, students who want to work as quants can
>> go straight to these programs, so physicists can concentrate on
>> physics, which is just as well.
> 
> I don't know if this is a good or a bad thing.  Obviously good that
> banks will no longer be nicking all the good physics PhDs.

I don't think that banks ever got the good physicists --- the latter
_want_ to do physics, you can't just buy them off for some back-office
drudge work.  I think that banks mostly got people with a degree but
no burning interest in the field.  And actually, I don't think that
they would like to employ someone who would make a good scientist, it
is a totally different environment which encourages minor innovations
but discourages major ones.  If you have the mind of a scientist, the
work done by quants would appear to be extremely dull.

Tamas

On 2010-02-23 10:32:24 +0000, Tamas K Papp said:

> I think that you may have an idealized image about physicists.  I can
> understand why, since many physicists are people of scientific
> integrity.  But getting a degree in physics does not imply that a
> person will behave as a good scientist for the rest of his/her life.

I don't think so.  I don't really think it's to do with integrity, I 
think it's to do with curiosity.  Or may be I do think so, I'm not sure.

> And actually, I don't think that
> they would like to employ someone who would make a good scientist, it
> is a totally different environment which encourages minor innovations
> but discourages major ones.  If you have the mind of a scientist, the
> work done by quants would appear to be extremely dull.

That may well be a factor. Dull but well-paid I guess encourages you 
not to think too hard.

On Feb 23, 5:32=A0am, Tamas K Papp <tkp...@gmail.com> wrote:

> On Tue, 23 Feb 2010 09:43:04 +0000, Tim Bradshaw wrote:
[...]
> > I don't know if this is a good or a bad thing. =A0Obviously good that
> > banks will no longer be nicking all the good physics PhDs.

> I don't think that banks ever got the good physicists --- the latter
> _want_ to do physics, you can't just buy them off for some back-office
> drudge work. =A0I think that banks mostly got people with a degree but
> no burning interest in the field. =A0

Speaking as someone who gave up on a career in theoretical physics and
spent some time looking for a job as a quant, I think this is
essentially correct. I got my PhD and then went on to a post-doc which
I basically hated so I started looking for other jobs. The only part
of the post-doc I enjoyed was the designing and implementing the Monte
Carlo simulations we used (in CL! :)), but in retrospect, I think the
underlying research problem was just not that interesting. Maybe it
even qualified as back-office drudge work.

Once I made my decision to leave academia, well, I'm in the NYC area,
so there were quite a few finance jobs that looked like reasonable
fits. They paid well, but my primary motivation was, "Goddamn but I
don't want to do physics anymore."

I ended up in a totally different part of the private sector, but
based on my own experience and the experiences of friends who ended up
in finance, the banks were getting people with a burning *disinterest*
in the field.

Cheers,
Pillsy

On Feb 22, 3:22=A0pm, Tim Bradshaw <t...@tfeb.org> wrote:
> On 2010-02-22 13:59:24 +0000, Norbert_Paul said:
>
> > Don't forget corporate officials: Excel and Powerpoint.
>
> Actually, of course, the secret truth is that the banking crisis is
> largely due to people using Excel as a database.
>
> --tim
>
> (It terrifies me that this might be true)

We installed an ERP system for a bank and after a few months some
lucky programmer like me asked one of the managers how they like it.
He answered: Kid before you installed that thing we did everything in
Excel. Now we do it in Excel again but we have to enter the data in
your program afterwards, now and then.

Bobi

Slobodan Blazeski <slobodan.blazeski@gmail.com> writes:

> He answered: Kid before you installed that thing we did everything in
> Excel. Now we do it in Excel again but we have to enter the data in
> your program afterwards, now and then.

ROTFL!
-- 
  (espen)

On Feb 22, 8:13=A0pm, Tamas K Papp <tkp...@gmail.com> wrote:
> On Mon, 22 Feb 2010 18:22:24 +0000, Tim Bradshaw wrote:
> > On 2010-02-22 17:52:50 +0000, Ron Garret said:
>
> >> I really think that the vast majority of the people on Wall Street
> >> really are completely clueless. =A0The only question in my mind is
> >> whether they are *all* completely clueless, or if at the core there ar=
e
> >> a few guys who actually understand what's going on and keep things
> >> humming along.
>
> > I agree with this. =A0My theory (about 3 years ago) was that, well, the=
y
> > used to suck up physics PhDs, and my experience of physicists (having
> > been one for a short while) is that they are some of the best people I
> > have ever known at really understanding how the things they're
> > interested in work, and just not taking any bullshit at all[*]. =A0So
> > surely those people would have understood stuff? =A0But, in fact, I thi=
nk
> > that maybe if you are offered enough money to stop worrying about what
> > your model means and just keep tweaking it until more money falls out,
> > perhaps even physics people decide they don't need to really understand
> > it: certainly it looks like they didn't. =A0(Or, perhaps they *did*, an=
d
> > they knew it would all fall to bits, but hopefully not until after thei=
r
> > bonus. =A0Maybe that's even a reasonable, if selfish, strategy.)
>
> All empirical results so far indicate that markets process information
> pretty efficiently. =A0That is to say, you can't beat markets
> systematically (after adjusting for risk). =A0The qualifier, of course,
> is "unless you have information no one else has", but most people don't,
> and those who do are closely watched (eg insider trading regulations).
>
> This is very hard to stomach for most people, who are looking for
> fabulous extra returns. =A0Unfortunately, there are always people who
> are willing to promise this, since they benefit from your investment
> --- usually a fixed share of your assets, which can be quite low for a
> plain vanilla index fund, and very large for actively managed funds
> which promise higher returns (but fail to deliver systematically, when
> adjusted for risk). =A0The market for investments is, in a certain
> sense, like the market for laundry detergent or toothpaste: suppliers
> make (amazing) claims which most buyers are not in the position to
> evaluate or validate.
>
> Physicists are employed in this industry because they have been
> trained to deal with the math --- mostly continuous time differential
> equations and statistics. =A0The rest is easy to pick up on a
> superficial level, but lacking training in economics, what they are
> doing is mostly mechanical. =A0The models they are using are not that
> sophisticated, and since there is only so much information you can
> extract from data using pure statistical methods (without economic
> ingredients beyond no-arbitrage), you can't expect a lot from them.
> For example, most pricing "models" calculate the "fundamentals" from
> today's prices (basically inverting a bijection), which they use to
> calculate other prices, but they don't really care if the
> "fundamentals" are different from one minute to the next.
>
> That said, Excel spreadsheets & such are certainly not to blame for
> the current crisis, and neither are physicists without economic
> training. =A0If you want to read about such things, look up Nicholas
> Dunbar's Inventing Money: The story of Long-Term Capital Management
> and the legends behind it, it is an entertaining book.
>
> Tamas
The Demon of our own design is pretty good too.

Bobi

On Feb 23, 10:43=A0am, Tim Bradshaw <t...@tfeb.org> wrote:
> On 2010-02-22 21:56:42 +0000, Tamas K Papp said:
>
> > To a certain extent, I agree with you. =A0But the problem is very hard,
> > and it requires thinking in terms of economic models (vs purely
> > statistical ones), and general equilibrium (instead of partial
> > equilibrium concepts). =A0I don't think that anyone without serious
> > training in economics has a chance. =A0And moreover, progress is quite
> > slow, at least on the scale of financial companies. =A0They don't have
> > the incentive to employ people who do basic research, so I don't blame
> > them.
>
> I think that's what disapoints me. =A0My view is that someone who really
> is a physicist is someone who would look at some problem domain (not
> just a physics domain) and make a really good stab at understanding it
> properly, more-or-less whatever it was. =A0Not someone who would just
> cobble together some na=EFve and obviously wrong model and be happy with
> that. (The same is true for engineers I think.) =A0Of course economics is
> not easy, but it's not harder than physics (well, perhaps it is,
> because we don't actually have economic theories that really work very
> well, and maybe the ones that do work will turn out to be harder than
> physics, but I'd guess not).
>
>
>
> > Nowadays, most universities have a graduate program in "financial
> > engineering" or similar, and graduates from these programs are
> > starting to displace physicists (some would argue that they have
> > already done so in certain areas). =A0Nowadays, students who want to
> > work as quants can go straight to these programs, so physicists can
> > concentrate on physics, which is just as well.
>
> I don't know if this is a good or a bad thing. =A0Obviously good that
> banks will no longer be nicking all the good physics PhDs.
You mean all those who aren't in the string theory?
http://xkcd.com/397/

Bobi

Tim Bradshaw <tfb@tfeb.org> writes:

> On 2010-02-22 21:56:42 +0000, Tamas K Papp said:
>
>> To a certain extent, I agree with you.  But the problem is very hard,
>> and it requires thinking in terms of economic models (vs purely
>> statistical ones), and general equilibrium (instead of partial
>> equilibrium concepts).  I don't think that anyone without serious
>> training in economics has a chance.  And moreover, progress is quite
>> slow, at least on the scale of financial companies.  They don't have
>> the incentive to employ people who do basic research, so I don't blame
>> them.
>
> I think that's what disapoints me.  My view is that someone who really is a
> physicist is someone who would look at some problem domain (not just a physics
> domain) and make a really good stab at understanding it properly, more-or-less
> whatever it was.  Not someone who would just cobble together some naïve and
> obviously wrong model and be happy with that. (The same is true for engineers
> I think.)  Of course economics is not easy, but it's not harder than physics
> (well, perhaps it is, because we don't actually have economic theories that
> really work very well, and maybe the ones that do work will turn out to be
> harder than physics, but I'd guess not).
>

I'm not sure if that is naive or not, but expecting someone from one
field to also be able to obtain expertise in another field is often the
cause of many problems. Expertise in one area does not automatically
translate to expertise in other areas even if the individual would like
to believe so. While you wold expect a physisist to have a strong grasp
of research methodologies and scientific rigor, this is in itself not
sufficient. There are plenty of poor physisists out there as well as
poor engineers. Such an assumption also runs close to arrogance as it
implies that all the research, thought and theory development done by
economists is somehow trivial and easily understood by a "real
scientist". There are som ewho would argue that areas like economics are
harder because they don't have the same level of experimental
reproducibility that is common in the physical sciences and lack the
verifiable 'laws' that are generally accepted in areas such as physics. 

I would suggest many of the problems we have are due to too many people
with expertise in one area speaking with authority and supposed
expertise in areas for which they hav little real knowledge or
udnerstanding. I would also argue that possibly we need *more*
intelligent, trained people in areas like economics and financial
mangement, not less.

Tim






>

-- 
tcross (at) rapttech dot com dot au

Slobodan Blazeski wrote:
> On Feb 22, 3:22 pm, Tim Bradshaw <t...@tfeb.org> wrote:
>> On 2010-02-22 13:59:24 +0000, Norbert_Paul said:
>>
>>> Don't forget corporate officials: Excel and Powerpoint.
>> Actually, of course, the secret truth is that the banking crisis is
>> largely due to people using Excel as a database.
>>
>> --tim
>>
>> (It terrifies me that this might be true)
> 
> We installed an ERP system for a bank and after a few months some
> lucky programmer like me asked one of the managers how they like it.
> He answered: Kid before you installed that thing we did everything in
> Excel. Now we do it in Excel again but we have to enter the data in
> your program afterwards, now and then.
> 
> Bobi

Most Excellent Answer!

On Feb 24, 5:59=A0am, "Dimiter \"malkia\" Stanev" <mal...@mac.com>
wrote:
> Slobodan Blazeski wrote:
> > On Feb 22, 3:22 pm, Tim Bradshaw <t...@tfeb.org> wrote:
> >> On 2010-02-22 13:59:24 +0000, Norbert_Paul said:
>
> >>> Don't forget corporate officials: Excel and Powerpoint.
> >> Actually, of course, the secret truth is that the banking crisis is
> >> largely due to people using Excel as a database.
>
> >> --tim
>
> >> (It terrifies me that this might be true)
>
> > We installed an ERP system for a bank and after a few months some
> > lucky programmer like me asked one of the managers how they like it.
> > He answered: Kid before you installed that thing we did everything in
> > Excel. Now we do it in Excel again but we have to enter the data in
> > your program afterwards, now and then.
>
> > Bobi
>
> Most Excellent Answer!

Not if you had an account in that bank. I switched since but how I
know that the other one is any better.

Bobi