I know that coding for free is quite an achievement, and that it is
daring to ask free developers for something, but I thought it would be
nice for once to be on the asking side ;-)

I am setting up an automated process for downloading, building and
testing relevant, popular, or simply LARGE software made in Common
Lisp. The goal is to ensure that the code is not broken by changes in
ECL. It is a selfish goal, but probably exportable to other
implementations.

http://ecls.sourceforge.net/logs_lib.html

What I would like to ask CL developers is to begin shipping regression
tests with their libraries, and perhaps agree on an standard so that
one can programatically check whether a library builds or not. This
feature could be even included in ASDF, asdf-install, etc, just like
"make check" is now a de-facto standad in Autoconf.

The agreement can be very simple: a package name (cffi-test for cffi,
etc) and a function that is to be executed and returns the number of
tests run, number of failures, and perhaps a descriptive list of the
test to be pretty printed.

Do you think it makes sense?

Juanjo

On 2009-08-04 17:27:19 -0400, Juanjo 
<juanjose.garciaripoll@googlemail.com> said:

> Do you think it makes sense?

Yes, but it requires that common lisp library writers agree on a 
standard test protocol. I would be very happy if things like this:

(asdf:oos 'asdf:test-op :bordeaux-threads)

could be relied on to work with all asdf systems.

regards,

Ralph


-- 
Raffael Cavallaro

On Aug 5, 4:07 pm, Raffael Cavallaro
<raffaelcavall...@pas.espam.s.il.vous.plait.mac.com> wrote:
> On 2009-08-04 17:27:19 -0400, Juanjo
> <juanjose.garciarip...@googlemail.com> said:
>
> > Do you think it makes sense?
>
> Yes, but it requires that common lisp library writers agree on a
> standard test protocol. I would be very happy if things like this:
>
> (asdf:oos 'asdf:test-op :bordeaux-threads)
>
> could be relied on to work with all asdf systems.

In many cases they work. Some people have made statistics and about
20% of the libraries implement some kind of tests. The problem is that
they do not agree on the output of the test function, if it has any.
See some real life examples below with opposite conventions

> (asdf:oos 'asdf:test-op 'alexandria)
[...]
No tests failed.
NIL
> (asdf:oos 'asdf:test-op 'flexi-streams)
[...]
Some tests failed.
NIL
>