Hi all.

I'm a hobby C++ programmer who have reached the point where
strain of testing and debugging code are about to outweigh the
fun and usefulness of my projects.

As is said in most books (e.g. "Pragmatic programmer"), "code a
little, test a lot."

However, this might be very good advice, but how does one go
about actually achieving this? There have to be things that are
smart to do, and others that are not at all smart to do? Books
that address such problems? Those I have found, deal with C#
or Java, not C++.

Any opinions, tips and warnings are very welcome.

Rune


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> [snip]
>
> As is said in most books (e.g. "Pragmatic programmer"), "code a
> little, test a lot."
>
> However, this might be very good advice, but how does one go
> about actually achieving this? There have to be things that are
> smart to do, and others that are not at all smart to do? Books
> that address such problems? Those I have found, deal with C#
> or Java, not C++.
>

For unit testing, I've found the CppUnit framework to be effective for
my needs:
http://cppunit.sourceforge.net

Both Boost and Trolltech's QT come with unit testing frameworks, but
I've never used either of these:
http://www.boost.org/libs/test/doc/index.html
http://doc.trolltech.com/4.2/qtestlib-manual.html

Trolltech's product apparently has support for testing their GUI too.

For integration and systems testing, I tend to use a combination of
shell scripts and hacked-up unit tests.

- Niek Sanders
http://www.cis.rit.edu/~njs8030/


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Feb 12, 7:03 pm, "Rune Allnor" <all...@tele.ntnu.no> wrote:

> As is said in most books (e.g. "Pragmatic programmer"), "code a
> little, test a lot."

I would actually advise the opposite.
Testing is a long, tiring, boring, and imperfect process. You should
try to avoid it whenever you can.
For that, the best is to have as much compile-time checks as possible.
You should try to reach the point where if it compiles, then the code
is almost certainly correct.


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Rune Allnor" <allnor@tele.ntnu.no> wrote:

> Hi all.
> 
> I'm a hobby C++ programmer who have reached the point where
> strain of testing and debugging code are about to outweigh the
> fun and usefulness of my projects.
> 
> As is said in most books (e.g. "Pragmatic programmer"), "code a
> little, test a lot."
> 
> However, this might be very good advice, but how does one go
> about actually achieving this? There have to be things that are
> smart to do, and others that are not at all smart to do? Books
> that address such problems? Those I have found, deal with C#
> or Java, not C++.

It sounds like you are asking about coding styles that will help 
mitigate bugs rather than asking how to test code. Is that correct?

If so, then the advice you read about C# and Java generally also works 
with C++ though a few extra things need to be done in the C++ case. We 
have to also make sure that values are initialized with reasonable 
defaults and make sure that dynamically allocated memory is returned to 
the system.

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Niek Sanders" <njs8030@yahoo.com> writes:

> > [snip]
> >
> > As is said in most books (e.g. "Pragmatic programmer"), "code a
> > little, test a lot."
> >
> > However, this might be very good advice, but how does one go
> > about actually achieving this? There have to be things that are
> > smart to do, and others that are not at all smart to do? Books
> > that address such problems? Those I have found, deal with C#
> > or Java, not C++.
> >
> 
> For unit testing, I've found the CppUnit framework to be effective for
> my needs:
> http://cppunit.sourceforge.net
> 
> Both Boost and Trolltech's QT come with unit testing frameworks, but
> I've never used either of these:
> http://www.boost.org/libs/test/doc/index.html
> http://doc.trolltech.com/4.2/qtestlib-manual.html

 We've been using the boost framework for about a year and a half, and
 are quite happy with it.  It's not perfect, but it has a lot going
 for it.  I particularly like how terse it is:

  BOOST_AUTO_UNIT_TEST( test_name )
  {
     BOOST_CHECK( 1.0 == 1.0 );
     ...
  }

 The small amount of extra typing involved in writing unit tests is a
 _big_ plus around here.

 As far as guidelines for unit testing in general, the best I've found
 is the NASA FSW guide: <http://software.gsfc.nasa.gov/process.cfm>
 and poke around in their product development --> testing
 directories.  Good stuff.

----------------------------------------------------------------------
Dave Steffen, Ph.D.                Disobey this command!
Software Engineer IV                 - Douglas Hofstadter
Numerica Corporation
dg@steffen a@t numerica d@ot us  (remove @'s to email me)

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On 13 Feb, 20:54, "Daniel T." <danie...@earthlink.net> wrote:
> "Rune Allnor" <all...@tele.ntnu.no> wrote:
> > Hi all.
>
> > I'm a hobby C++ programmer who have reached the point where
> > strain of testing and debugging code are about to outweigh the
> > fun and usefulness of my projects.
>
> > As is said in most books (e.g. "Pragmatic programmer"), "code a
> > little, test a lot."
>
> > However, this might be very good advice, but how does one go
> > about actually achieving this? There have to be things that are
> > smart to do, and others that are not at all smart to do? Books
> > that address such problems? Those I have found, deal with C#
> > or Java, not C++.
>
> It sounds like you are asking about coding styles that will help
> mitigate bugs rather than asking how to test code. Is that correct?

Well, yes, at least partially.

I learned a lot from some thread here a few weeks ago, where the
claim was that C++ class and method names ought to reflect the
responsibilities of objects and actions/purposes of the methods,
basically providing a self-documenting C++ code. A very small
"trick" at first glance, but incredible effective when tried
in a project above a certain scale.

I am on a hobby budget (i.e. $ == 0) which means that I have to
manually do (or at least prepare) what test frameworks
automatically do in "real" projects.

> If so, then the advice you read about C# and Java generally also works
> with C++ though a few extra things need to be done in the C++ case. We
> have to also make sure that values are initialized with reasonable
> defaults and make sure that dynamically allocated memory is returned to
> the system.

These things are OK; I never learned to use Java or C++ so
I am not "spoiled" by their extra functionality...

I hoped to get some answers to how to design code in order to
facilitate writing a "test suite" for my code (does that make
sense?), and then some clues about how to actually design the
test suite itself. As far as I can tell, "test suites" are
computer programs that test other computer programs. This
seems very intriguing, but I can't find any clues about
how to write those test suites.

As for now, I am not very interested in GUIs, just core C++
functionality.

Rune


--
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Mathias Gaunard" <loufoque@gmail.com> wrote in
news:1171381727.412236.33140@p10g2000cwp.googlegroups.com:

> On Feb 12, 7:03 pm, "Rune Allnor" <all...@tele.ntnu.no> wrote:
>
>> As is said in most books (e.g. "Pragmatic programmer"), "code a
>> little, test a lot."
>
> I would actually advise the opposite.
> Testing is a long, tiring, boring, and imperfect process. You should
> try to avoid it whenever you can.

I completely agree.  To add to that, testing is less efficient at
defect removal than doing your own structured code and design reviews
(e.g., checklist driving).  Good design ahead of time reduces defects
further.

Testing can be improved by building it into design (e.g., sequence
diagrams).  A test plan improved the process by making the tests
repeatable.  Maybe not much fun for the hobbyist.

> For that, the best is to have as much compile-time checks as possible.
> You should try to reach the point where if it compiles, then the code
> is almost certainly correct.

These are one type of defensive programming approach.  Runtime and
compile-time  assertions can both identify problems quickly and make
code more self-documenting.

Static analysis has its limitations, though.  The compiler is unlikely
to pick up defects having to do with incorrect initalization and
off-by-one.  Structural design flaws are missed altogether.  So the
code might be correct, but not right.


--
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Rune Allnor" <allnor@tele.ntnu.no> wrote in
news:1171300025.458119.131820@a75g2000cwd.googlegroups.com:

> Hi all.
>
> I'm a hobby C++ programmer who have reached the point where
> strain of testing and debugging code are about to outweigh the
> fun and usefulness of my projects.

Unfortunately, that's life as a developer.  Coding ranks highest on
most people's list.  The new projects are always more interesting than
maintenance.  Designing before coding can reduce a lot of defects,
especially, if your design captures unit tests, by way of dynamic
modeling (e.g., UML sequence and collaboration diagrams).  If you wish
to spend less time fixing things and more time writing new features,
get the design and code right before you even hit the compiler.  Get
to where design is as much fun as the code, especially if you can
automate the boilerblate code through model generation.  Unfortuantely
there aren't a lot of free C++ tools to help you there.

> As is said in most books (e.g. "Pragmatic programmer"), "code a
> little, test a lot."
>
> However, this might be very good advice, but how does one go
> about actually achieving this?

One way to write less code is to reuse more code.  Use solid
frameworks like STL, Boost, or whatever fits your particular
environment.  Next, write reusable code.  The things you have to write
that are separate concerns from your main task (e.g., you're writing
an accounting program and you need a special accumulator) should be
independently testable because they're independent.  Finally,
encapsulate things that don't have visibility to your core logic
(e.g., threading issues should be hidden from business logic code).

> There have to be things that are
> smart to do, and others that are not at all smart to do? Books
> that address such problems? Those I have found, deal with C#
> or Java, not C++.

The general lessons of other languages apply well to C++ when it comes
to good engineering practices.

When you have to test, make testing more effective.  Plan the tests
(see above) so that you're assumptions are correct.

To make code easier to debug, incorporate a logging framework such as
log4cxx (modeled after log4j).  It's better when you can filter it.
In debuggers, it's easier to view the values of simple expressions, so
consider breaking compound expressions over separate statements, so
instead of the following:
   if (foo(a,b) && bar(c,d))
do
   bool success = foo(a,b);
   success &&= bar(c,d);
   if (success)
Maybe not the best example, but it makes it easier to step through the
code and see the results.

Assert() like crazy.  Use BOOST_STATIC_ASSERT.  Avoid catching fatal
exceptions in debug builds.

Make your code readable by an automated tool, such as Doxygen or
Doxys, so that when you come back to it, you can understand a function
without digging into the weeds of its implementation.

As far as smart things to do, read good style guides.  Check out
Meyers's Effective C++ is and Sutter and Alexandrescu's C++ Coding
Standards.  The bibliographies should lead you to more smart things.

--
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mathias Gaunard wrote:
> On Feb 12, 7:03 pm, "Rune Allnor" <all...@tele.ntnu.no> wrote:

> > As is said in most books (e.g. "Pragmatic programmer"), "code a
> > little, test a lot."

> I would actually advise the opposite.

Yeh, that's what customers are there for, to do the testing.
Regretfully, a lot of my suppliers seem to take the same
attitude.

> Testing is a long, tiring, boring, and imperfect process. You should
> try to avoid it whenever you can.

You need it, in order to validate your process.  Obviously, if
the tests find an error, there's something wrong in your process
upstream, which you'll want to address.  But in my experience,
if you skip on the testing, things will degenerate upstream, and
you'll end up with very bad code.

Note that a lot of books seem to neglect this point.  You don't
test to find errors in the code; you test to ensure that your
process is working.

> For that, the best is to have as much compile-time checks as
> possible.  You should try to reach the point where if it
> compiles, then the code is almost certainly correct.

That's certainly a laudable goal, but it isn't nearly 100%
possible.  What the compiler can't catch, of course, code review
should.  But without the tests to back things up, code review
(and the addition of compile time checks) tends to become
watered down; most developers seem to find it more fun to add
new features to their own code than to review other people's
code.

Unit tests are important for regression testing, as well.  Full
scale code reviews, while necessary for initial development and
major reworks, are a bit expensive for one line bug fixes---in
such cases, an extensive set of unit tests will ensure with a
fair degree of assurance that you didn't accidentally break
anything else.  (As a general rule, you never correct an error
in the actual code until you have a unit test which reveals it.
Otherwise, how do you know that you've really corrected it.)

Another important point of unit tests is that they exercise
exactly one thing at a time.  So if a unit test fails, you know
where to look in the sources.  Not as much as if a code reviewer
says that I forgot to initialize variable x in line y of file z,
but still, it's a lot more precise than if a customer calls and
says he's getting random core dumps twice a day.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orientie objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place Simard, 78210 St.-Cyr-l'Icole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mathias Gaunard wrote:
> On Feb 12, 7:03 pm, "Rune Allnor" <all...@tele.ntnu.no> wrote:
> 
>> As is said in most books (e.g. "Pragmatic programmer"), "code a
>> little, test a lot."
> 
> I would actually advise the opposite.
> Testing is a long, tiring, boring, and imperfect process. You should
> try to avoid it whenever you can.
> For that, the best is to have as much compile-time checks as possible.
> You should try to reach the point where if it compiles, then the code
> is almost certainly correct.

I feel that this can be read to imply testing is unnecessary.  However, 
I don't think you meant to imply that.  Rather if you can enforce a 
constraint by design, then there's no need to test.

To put it another way: "Test what you need to test".

IMHO, a function body requires at least the same amount of code again to 
test it properly.  This is due to conditionals, loops, etc which need to 
have each path tested.

If your testing requires that you test *all* path combinations, and not 
just all paths, then I think the amount of test code will n times larger 
than the source.



Regards,

Richard


-- 
Richard Corden

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On 13 Feb., 20:54, "Daniel T." <danie...@earthlink.net> wrote:
> If so, then the advice you read about C# and Java generally also works
> with C++ though a few extra things need to be done in the C++ case. We
> have to also make sure that values are initialized with reasonable
> defaults and make sure that dynamically allocated memory is returned to
> the system.

Actually the same two points apply to C# and Java as well. The fact
that
both languages define *predictable* initial values of class members
doesn't mean that these are correct ones, that is, that those values
would
auto*magically* fulfill the corresponding class constraints.
Another point is, that you cannot provide a userdefined default c'tor
for
C#'s ValueType's, so you have to ensure that such a "zero-
initialization"
creates an instance fulfilling its class constraints - otherwise you
don't
have to forget to call the proper init function or another userdefined
c'tor.
Proper resource management is also relevant for all GC languages: Non-
memory resources are 100% your responsibility and inproper class
design
can also lead to memory "garbadge" that is not freed because of cyclic
dependencies that are not manually resolved (similarily: Proper use
of WeakReferences).

Greetings from Bremen,

Daniel Kr�gler



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On 14 Feb, 09:01, "W. J. La Cholter" <with...@giganews.com> wrote:
> "Mathias Gaunard" <loufo...@gmail.com> wrote innews:1171381727.412236.33140@p10g2000cwp.googlegroups.com:
>
> > On Feb 12, 7:03 pm, "Rune Allnor" <all...@tele.ntnu.no> wrote:
>
> >> As is said in most books (e.g. "Pragmatic programmer"), "code a
> >> little, test a lot."
>
> > I would actually advise the opposite.
> > Testing is a long, tiring, boring, and imperfect process. You should
> > try to avoid it whenever you can.
>
> I completely agree.

At the risk of offending the moderator, I wasn't shure whether or not
this claim was a troll in Matias' post. "Avoid testing if you can"
doesn't go very well with what seems to be a persistent main matra
in present-day textbooks?

> To add to that, testing is less efficient at
> defect removal than doing your own structured code and design reviews
> (e.g., checklist driving).  Good design ahead of time reduces defects
> further.

I'm making the effort. Not always successful, but I'm making the
effort.

> Testing can be improved by building it into design (e.g., sequence
> diagrams).  A test plan improved the process by making the tests
> repeatable.  Maybe not much fun for the hobbyist.

No. But debugging stuff that are caused by interaction issues
between peripherals buried deep inside an application is even
less fun. When I need a file-access object, I all too often
hack up something which works there and then, and which inevitably
blows up a few weeks later when I need something similar, adjusts
the class I already have, which then breaks the original
application.

"Design the file access class" is the obvious answer, but I would
like to be able to come up with some sort of test so that I can
check the effects of my changes, and verify that the original class
still works.

I don't mind spending a couple of hours worth of effort on designing
tests when I first code up something, if that can avoid me spending
days of debugging and re-coding later.

Finding Doxygen was a huge leap forward for me; that was what caused
me to start documenting my code. Doxygen did all that grunt work for
me,
made the presentations, drew the overall pictures. While I always
knew
that documenting the code was a good thing, I never did it because I
also knew I'd have to do it all over when writing a report. Now that
doxygen takes all that drudgery off my hands and make those beautiful
web pages for me, I actually write the comments.

The same thing with design and tests: I don't design as often as I
know
I should, simply because there is so much drudgery involved. If I can
get ideas about exploiting a design for testing and verifiction of
run-time programs, I am more inclined to actually invest the drudgery
in the design: a drudgery that is needed anyway for debugging when I
*don't* design my code as I would like to.

I guess it's down to sheer psychology: I know what is "the right
thing"
to do, but I don't actually do it till I see tangible benefits of
doing it.

> > For that, the best is to have as much compile-time checks as possible.
> > You should try to reach the point where if it compiles, then the code
> > is almost certainly correct.
>
> These are one type of defensive programming approach.  Runtime and
> compile-time  assertions can both identify problems quickly and make
> code more self-documenting.

When I am finished, my code always compiles "cleanly." This is
partially
because some very bad experiences with portability issues many years
ago; partially because I know my code has enough other problems, so
handling compile-time issues gives a certain satisfaction about
having
done at least *something* right...

> Static analysis has its limitations, though.  The compiler is unlikely
> to pick up defects having to do with incorrect initalization and
> off-by-one.  Structural design flaws are missed altogether.  So the
> code might be correct, but not right.

These are the tests I am looking for: Make sure the code does
what it is supposed to do, first after initial design; later after
corrections and general maintenance.

Just to give some hint of my background and style of working: I am
trained as a "computer techie"/data analyst/signal processor and
have in the past designed some algorithms for my own use. At times
the question comes up to design prototype algorithms and include
these in production processing systems. Right now I am playing
(on my own time and budget) with ideas for alternatives to what is
standard operating procedure in my day job, hence a hobbyist
dabbling with somewhat demanding projects.

For my own work I have used matlab as reference, using examples in
textbooks and parameter reference tables in older texts as
verification
data for my programs: If my code reproduces the answers in the
textbooks,
the individual steps (functions/methods/arithemtics) work. If I can
reproduce tabulated reference material, the system works. Once the
matlab prototype works, I code in C++ to gain run-time performance
and
reduce code maintenance.

These days I am playing with prototypes for more elaborate non-maths
parts of data processing systems, meaning no more textbook examples
or tabulated reference material. Hence the need to get into more
elaborate testing issues.

Rune


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Rune Allnor wrote:

> I hoped to get some answers to how to design code in order to
> facilitate writing a "test suite" for my code (does that make
> sense?), and then some clues about how to actually design the
> test suite itself.

I'm not going to give you that many clues, but consider writing your
tests first. First write how you going to use your code, then write
the actual code. That way your code is written from a usage
perspective, and usually more user friendly, so to speak. It reads
better...

Also, use interfaces to interface from the code being tested to the
testing code. The testing code then just gets replaced by the real
code, and wholla...

Remember to invoke all exception cases from your testing code.

Kind Regards,

Werner Erasmus


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Rune Allnor" <allnor@tele.ntnu.no> wrote:
> "Daniel T." <danie...@earthlink.net> wrote:
> > "Rune Allnor" <all...@tele.ntnu.no> wrote:
> >
> > > I'm a hobby C++ programmer who have reached the point where
> > > strain of testing and debugging code are about to outweigh the
> > > fun and usefulness of my projects.
> > >
> > > As is said in most books (e.g. "Pragmatic programmer"), "code a
> > > little, test a lot."
> > >
> > > However, this might be very good advice, but how does one go
> > > about actually achieving this? There have to be things that are
> > > smart to do, and others that are not at all smart to do? Books
> > > that address such problems? Those I have found, deal with C# or
> > > Java, not C++.
> >
> > It sounds like you are asking about coding styles that will help
> > mitigate bugs rather than asking how to test code. Is that correct?
> 
> Well, yes, at least partially.
> 
> I learned a lot from some thread here a few weeks ago, where the
> claim was that C++ class and method names ought to reflect the
> responsibilities of objects and actions/purposes of the methods,
> basically providing a self-documenting C++ code. A very small
> "trick" at first glance, but incredible effective when tried in a
> project above a certain scale.

Some other ideas along the same lines may help then...

1) When you have a loop that does something to a bunch of objects, put
it in its own function.
2) When you have an 'if' statement with a complex predicate, create a
const function that returns bool, name the function after what the
predicate is supposed to test, and put the complex bit in it.
3) Count how many of the following you have in your function: 'if',
'while', 'for', '||', '&&', '?:'. If you have more than five of them in
the same function, then break up the function into separate parts. (To
learn more about this google "Cyclomatic Complexity".)

> I am on a hobby budget (i.e. $ == 0) which means that I have to
> manually do (or at least prepare) what test frameworks automatically
> do in "real" projects.

Professionals have to manually set-up that stuff too. We just get paid
to do it. :-)

> I hoped to get some answers to how to design code in order to
> facilitate writing a "test suite" for my code (does that make
> sense?), and then some clues about how to actually design the test
> suite itself.

So you have heard the mantra, "write code that is easy to test" and you
are wondering how to do that?

> As far as I can tell, "test suites" are computer
> programs that test other computer programs. This seems very
> intriguing, but I can't find any clues about how to write those test
> suites.

I sent you an email with a simple testing framework and example of use.

Good luck!

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On 14 Feb, 17:18, "Daniel T." <danie...@earthlink.net> wrote:
> "Rune Allnor" <all...@tele.ntnu.no> wrote:

-- useful hints snipped --

> > I hoped to get some answers to how to design code in order to
> > facilitate writing a "test suite" for my code (does that make
> > sense?), and then some clues about how to actually design the test
> > suite itself.
>
> So you have heard the mantra, "write code that is easy to test" and you
> are wondering how to do that?

Yes.

> > As far as I can tell, "test suites" are computer
> > programs that test other computer programs. This seems very
> > intriguing, but I can't find any clues about how to write those test
> > suites.
>
> I sent you an email with a simple testing framework and example of use.

It will probably bounce. The address in my posts has been obsolete
for a few years now. Is your email address valid? If so, I'll send
you a mail with a valid address.

> Good luck!

Thanks, and thanks for your help.

Rune


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Feb 14, 3:21 pm, "James Kanze" <james.ka...@gmail.com> wrote:

> You need it, in order to validate your process.  Obviously, if
> the tests find an error, there's something wrong in your process
> upstream, which you'll want to address.  But in my experience,
> if you skip on the testing, things will degenerate upstream, and
> you'll end up with very bad code.

> That's certainly a laudable goal, but it isn't nearly 100%
> possible.  What the compiler can't catch, of course, code review
> should.

There is difference between testing everytime when writing code and
making a fully fledged review at the end of the development cycle.

What the OP said, "code a little, test a lot" made me think it meant
you couldn't write 100 lines without spending a long time testing them.


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Rune Allnor" <allnor@tele.ntnu.no> wrote:
> "Daniel T." <danie...@earthlink.net> wrote:
> > "Rune Allnor" <all...@tele.ntnu.no> wrote:
>
> > > As far as I can tell, "test suites" are computer
> > > programs that test other computer programs. This seems very
> > > intriguing, but I can't find any clues about how to write those test
> > > suites.
> >
> > I sent you an email with a simple testing framework and example of use.
>
> It will probably bounce. The address in my posts has been obsolete
> for a few years now. Is your email address valid? If so, I'll send
> you a mail with a valid address.

Yes, my email address if valid, but my spam blocker will trash your mail
unless you put the word "sheltie" in as part of the subject.

Send your address to me and I'll re-send the framework and example. It's
something I use for hobby projects of my own and sometimes for small
professional bits.

--
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Rune Allnor" <allnor@tele.ntnu.no> writes:

> I'm a hobby C++ programmer who have reached the point where
> strain of testing and debugging code are about to outweigh the
> fun and usefulness of my projects.
>
> As is said in most books (e.g. "Pragmatic programmer"), "code a
> little, test a lot."
>
> However, this might be very good advice, but how does one go
> about actually achieving this? There have to be things that are
> smart to do, and others that are not at all smart to do? Books
> that address such problems? Those I have found, deal with C#
> or Java, not C++.

Lots of what is said about C# and Java testing applies equally to C++.
In
fact, I would go as far as to recommend "JUnit recipes" by J.B.
Rainsberger,
despite it being a Java-based book.

Here's a short list of things that I find help with testing in C++:

* Write short functions. The less a function does, the easier it is to
test.

* Isolate dependencies with abstract classes. Code that access
external
databases, network resources and files on disk tends to be slow to
test, and
is also subject to the availability of the external resource. Where
possible,
create an abstract class for the interface to such a resource, and
pass in a
dummy implementation in your test that provides exactly the behaviour
you
require for the test. This also helps with reducing coupling.

* Apply the Single Responsibility Principle. If a class or function
only has
one thing to do, it is easier to test. Separate large classes or
functions
into several smaller ones.

* Test through the public interface to a class. If the test needs
access to
private data or functions, consider making them public. If making them
public
on this class would be bad for some reason, consider extracting a
smaller
class on which they can be made public, and which can be used as a
building
block for the original class.

* Write the test first. Writing a test can help you think about the
design of
the class or function from the perspective of code that needs to use
it. Writing one small test, and then some code, and then another small
test,
and then some code can lead to better designed, easier to test code.

If you've got an existing codebase that you're finding difficult to
test, it
might be worth reading Michael Feathers' "Working Effectively with
Legacy
Code" --- this book contains lots of techniques for getting code under
test,
and making testing easier. It does contain some examples in C++.

Hope that helps.

Anthony
--
Anthony Williams
Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk
Registered in England, Company Number 5478976.
Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL



--
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mathias Gaunard wrote:
> On Feb 14, 3:21 pm, "James Kanze" <james.ka...@gmail.com> wrote:

> > You need it, in order to validate your process.  Obviously, if
> > the tests find an error, there's something wrong in your process
> > upstream, which you'll want to address.  But in my experience,
> > if you skip on the testing, things will degenerate upstream, and
> > you'll end up with very bad code.

> > That's certainly a laudable goal, but it isn't nearly 100%
> > possible.  What the compiler can't catch, of course, code review
> > should.

> There is difference between testing everytime when writing code and
> making a fully fledged review at the end of the development cycle.

I'm not sure what you mean by "testing everytime".  As I pointed
out, regression tests are important because regardless of the
principles involved, you're not going to do a full fledged code
review for every one line bug fix.  And of course, testing keeps
the reviewers (and everyone else) honest.

> What the OP said, "code a little, test a lot" made me think it meant
> you couldn't write 100 lines without spending a long time testing them.

It doesn't have to be a long time, but extensive unit tests are
useful.  And some units aren't much more than 100 lines
(although a 1000 lines would be more common).

Tests are an important part of the quality process, and cannot
be neglected.  That doesn't mean, however, that they are the
entire quality process.  You can't effectively test until you
know what to test, and that means e.g. hard requirement
specifications up front; for unit tests, that means a good,
solid design which clearly specifies exactly what each testable
unit is to do.  And tests will never replace good, independant
code review---their most important use, in fact, is detecting
errors in the review process.  (Note that unit tests are
normally part of the code review---insufficient unit tests can
mean that the code fails the review.)

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                    Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

werasm wrote:
> Rune Allnor wrote:

> > I hoped to get some answers to how to design code in order to
> > facilitate writing a "test suite" for my code (does that make
> > sense?), and then some clues about how to actually design the
> > test suite itself.

> I'm not going to give you that many clues, but consider writing your
> tests first.

Without adding to it, that's bad advice.  Consider writing the
requirements specifications first, then write the class and its
tests.  In any order you want---you should be delivering both
together, and the job isn't done until both are done.

At the class level, most of my documentation (detailed design
specification) is in the header file, so that's what I write
first.  I don't start writing a function or its tests until the
documentation for it is done.  After that, I generally write the
function before its tests, but that's a personal choice; the
order doesn't matter.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                    Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

anthony.ajw@gmail.com writes:

[...]

> Here's a short list of things that I find help with testing in C++:
[...]
> * Write short functions. The less a function does, the easier it is
> to test.
> 
> * Isolate dependencies with abstract classes.
[...]

> * Apply the Single Responsibility Principle.
[...]

 I agree completely... but so far, these are what I would consider to
 be "good design", regardless of unit testing.  Or put it another way,
 good designs are easily unit tested.

> * Test through the public interface to a class. If the test needs
> access to private data or functions, consider making them public. If
> making them public on this class would be bad for some reason,
> consider extracting a smaller class on which they can be made
> public, and which can be used as a building block for the original
> class.

 Here I strongly, completely, and absolutely disagree.  There are good
 reasons for data and methods to be private. Breaking encapsulation
 for the sake of unit tests is a cardinal sin in my book.

 We had a discussion a while back on the Boost users list about this
 issue, which can be called the "How do I unit test private methods"
 issue.  I'll summarize results as:


 A) Black Box Testing: Don't test private methods or members at all.
 Test only the public interface.  Your classes have a specified
 interface with specified behaviors.  Test that and don't worry about
 the rest.

 B) White Box Testing: poke and prod at the object's guts, e.g. private
 members and methods.  The question here is how to get access to the
 object's guts.

 Note that in some cases black box testing is appropriate, and in some
 cases white box testing is appropriate.  One isn't better than the
 other, they're for different kinds of testing.

 Now, if you're going to do white box testing, you need access to the
 object's guts, e.g. you need to "break" encapsulation in some way:

 B1)  Inheritance:  publicly derive a 'testClass' from the class in
 question.  The derived class either exposes guts of its parent for
 unit testing purposes, or is actually the unit test itself.

     pros: maintains encapsulation, doesn't raise hackles

     cons: mandates that testable classes make (at least some)
     internal stuff protected instead of private. May or may not play
     well with your unit testing framework.

 B2)  Friend testers:  give your class a friend, define the friend in
 your unit test, and give the friend class an interface that exposes
 the guts of your class.

     pros: minimal changes to source code (just give Class a friend
     testClass).

     cons: takes a bit of typing to set up, can sometimes be clunky.

 B3)  #define private public at the top of your unit tests.

     pros: spills the guts of every class you might ever want to get
     at, with no changes to source code whatsoever.

     cons: raises the hackles of good C++ programmers around the
     world; feels like a bletcherous hack, even after you've gotten
     used to it.


 Now, all three of these have their advocates and their detractors, in
 approximately equal numbers, from which I conclude that they're all
 acceptable.  An interesting issue for B3:  a discussion here (I
 think) a while back about "can #define private public ever change the
 behavior of a correct program" resulted in the answer "yes, but only
 if a class has overloaded member functions with different access
 specifiers, which is probably a bad design anyway".

:-)

 Personally, I've used B2 extensively with the Boost Test framework,
 and it's just dandy.  I've been using B3 recently, and despite the
 hackles it raises, I've found it to be even better.

 B4)  Redesign (or at least recode) the class to make private things
 public in order to unit test them.

    pros: makes unit testing easy, since the object's guts are already
    available.

    cons: breaks encapsulation completely, since the object's guts are
    also available _to all other code in the application_.

 IIRC this option was universally disliked; your endorsement is the
 only one I've come across.

 Personally, I can't imaging that breaking a fundamental tenet of OO
 design across your application is ever a good idea, even if it does
 make unit testing more convenient.  B1-B3 break encapsulation, but
 they either do it very selecively, or they do it _only_ within the
 unit test code itself.

----------------------------------------------------------------------
Dave Steffen, Ph.D.                Disobey this command!
Software Engineer IV                 - Douglas Hofstadter
Numerica Corporation
dg@steffen a@t numerica d@ot us  (remove @'s to email me)

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

James Kanze wrote:
> Without adding to it, that's bad advice.  Consider writing the
> requirements specifications first, then write the class and its
> tests.

What method do you (or others) recommend to unit-test private
functions (in C++) or file-scope (static) functions without
littering the actual code with stuff needed only for testing?

Testing only the public (or global scope) interface does not
really solve the complexity problem, or does it?

Kind regards
     Ingolf

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dave Steffen wrote:

>  B3)  #define private public at the top of your unit tests.
> 
>      pros: spills the guts of every class you might ever want to get
>      at, with no changes to source code whatsoever.
> 
>      cons: raises the hackles of good C++ programmers around the
>      world; feels like a bletcherous hack, even after you've gotten
>      used to it.

Alternative without the cons above:

class A
{
public:
     // ...

#ifndef UNIT_TEST_ENABLED
private:
#endif
     // ...
};

This is at least kosher with respect to redefining keywords.

>  Now, all three of these have their advocates and their detractors

I would add additional one for consideration: add public testMe or 
doSelfTest method:

class A
{
public:
     // ...

     void doSelfTest();

private:
     // all stuff encapsulated as usual
     // ...
};

Of course, doSelfTest might be defined in a separate .cpp file, so it 
can be excluded from production code by just not linking it. In a test 
environment, this function is defined to do whatever is has to do, 
called by the test driver and linked in on demand.

The fact that doSelfTest is by convention in a separate .cpp file makes 
it easy to have a whole set of such independent functions (just select 
them at link time), possibly provided with different testing frameworks 
or objectives in mind.


-- 
Maciej Sobczak : http://www.msobczak.com/
Programming    : http://www.msobczak.com/prog/

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

James Kanze wrote:
> werasm wrote:
> > Rune Allnor wrote:
>
> > > I hoped to get some answers to how to design code in order to
> > > facilitate writing a "test suite" for my code (does that make
> > > sense?), and then some clues about how to actually design the
> > > test suite itself.
>
> > I'm not going to give you that many clues, but consider writing your
> > tests first.
>
> Without adding to it, that's bad advice.  Consider writing the
> requirements specifications first, then write the class and its
> tests.  In any order you want---you should be delivering both
> together, and the job isn't done until both are done.

Well, to be honest - our process of deriving requirements (the
functional ones) involve a detailed Use Case analysis. From this we
create high level Sequence Diagrams involving the main Actors, or
Stakeholders. From these, we determine how our system will be tested
functionally, and priorities in terms of increments. We do have a
Requirement Specification to start of with, yes - but slogging through
the sequences and deriving tests from them help us think through the
requirements much better and help us get the increments right. This we
do before we touch code. After this we do a grammatical analysis from
which objects and interactions are derived. We identify patterns. We
have an architecture that seems to work for most (if not all) of our
systems. We identify domains, and interfaces between them. Often these
interfaces are derived as code is written. Sometimes, if you need to
implement a domain (for instance), writing the test case first (from
the use case/requirements) help one get insights wrt. the code (or
responsibilities) of the applicable domain(related classes, class or
function).

When we code, we endeavour to write our test first iaw. with the use
case description or sequence. Looking at an interface from the
client's (or code user, in this case's) perspective help one create a
much better interface eventually. There are two ways to write code,
really - there where no physical entity exist, create an abstraction
that will be implemted later. Or... write the physical entity (or
test) and this indicates what the abstraction needs to look like -
replace the test (who implements an abstraction) with the real code,
and the client (or user of the abstraction) is now tested. Coding
towards a test forces one to use abstractions (as the test cannot be
the implementation), and makes one's code modular automatically.

Makes sense to me.

> After that, I generally write the
> function before its tests, but that's a personal choice; the
> order doesn't matter.

I always used to do it that way, until someone told me the code reads
better if one writes the test first - for instance, writing the test
(calling the function) first would cause the function to be named from
a usage perspective. Obviously requirements need to be derived, else
there would be no test.

Regards,

Werner


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dave Steffen <dgsteffen@numerica.us> writes:

> anthony.ajw@gmail.com writes:
>
> [...]
>
>> Here's a short list of things that I find help with testing in C++:
> [...]
>> * Write short functions. The less a function does, the easier it is
>> to test.
>> 
>> * Isolate dependencies with abstract classes.
> [...]
>
>> * Apply the Single Responsibility Principle.
> [...]
>
>  I agree completely... but so far, these are what I would consider to
>  be "good design", regardless of unit testing.  Or put it another way,
>  good designs are easily unit tested.

Yes.

>> * Test through the public interface to a class. If the test needs
>> access to private data or functions, consider making them public. If
>> making them public on this class would be bad for some reason,
>> consider extracting a smaller class on which they can be made
>> public, and which can be used as a building block for the original
>> class.
>
>  Here I strongly, completely, and absolutely disagree.  There are good
>  reasons for data and methods to be private. Breaking encapsulation
>  for the sake of unit tests is a cardinal sin in my book.
>
>  We had a discussion a while back on the Boost users list about this
>  issue, which can be called the "How do I unit test private methods"
>  issue.  I'll summarize results as:
>
>  A) Black Box Testing

>  B) White Box Testing

>  Now, if you're going to do white box testing, you need access to the
>  object's guts, e.g. you need to "break" encapsulation in some way:
>
>  B1)  Inheritance:  publicly derive a 'testClass' from the class in
>  question.  The derived class either exposes guts of its parent for
>  unit testing purposes, or is actually the unit test itself.
>
>      pros: maintains encapsulation, doesn't raise hackles
>
>      cons: mandates that testable classes make (at least some)
>      internal stuff protected instead of private. May or may not play
>      well with your unit testing framework.

I don't think this is a good idea, precisely because you have to make things
protected. This breaks the encapsulation of the class --- protected is
"public
in some contexts", and it means that other code can also get at the "guts"
of
your class just by deriving from it.

>  B2)  Friend testers:  give your class a friend, define the friend in
>  your unit test, and give the friend class an interface that exposes
>  the guts of your class.

I've done this before, and I'd really rather not. It means you have to
modify
the class (i.e. add a friend declaration) every time you want to add a new
test class, which limits the extent to which you can readily break your
tests
into separate classes.

>  B3)  #define private public at the top of your unit tests.

This renders your program not-well-formed. I wouldn't go there.

>  B4)  Redesign (or at least recode) the class to make private things
>  public in order to unit test them.
>
>     pros: makes unit testing easy, since the object's guts are already
>     available.
>
>     cons: breaks encapsulation completely, since the object's guts are
>     also available _to all other code in the application_.

No it doesn't. It means that all the other code in the application can
create
a class with the same behaviour as the guts of your class, but they still
can't meddle with the guts of an object your main class. For example, if you
need a linked list as part of the implementation of your class, you might
write one (assuming you didn't have the STL) as part of the private details
of
the class. That makes it hard to test the list implementation, so extract
that
to its own class, and now you have a separate list implementation which you
can test, and which your class can make use of. It doesn't break
encapsulation
of your class, since external code still can't get at the internal list.

>  IIRC this option was universally disliked; your endorsement is the
>  only one I've come across.

>  Personally, I can't imaging that breaking a fundamental tenet of OO
>  design across your application is ever a good idea, even if it does
>  make unit testing more convenient.  B1-B3 break encapsulation, but
>  they either do it very selecively, or they do it _only_ within the
>  unit test code itself.

I think you are misunderstanding my suggestion. It is merely *an
application*
of good OO design --- ensure each class has a single responsibility. If the
class needs its private implementation details testing in a way that can't
be
done through the public interface, then it's probably doing too much, and
there is another class with its own Single Responsibility waiting to be
extracted (and which can therefore be tested in isolation), so the code in
the
original outer class can focus on the actual responsibility of that class.

Anthony
-- 
Anthony Williams
Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk
Registered in England, Company Number 5478976.
Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Ingolf Steinbach wrote:
> James Kanze wrote:
> > Without adding to it, that's bad advice.  Consider writing the
> > requirements specifications first, then write the class and its
> > tests.

> What method do you (or others) recommend to unit-test private
> functions (in C++) or file-scope (static) functions without
> littering the actual code with stuff needed only for testing?

I'm not sure I understand.  I test that a component meets its
external requirements specification.  Private functions aren't
part of that.

> Testing only the public (or global scope) interface does not
> really solve the complexity problem, or does it?

What complexity problem?  What problem are you trying to solve
by testing private functions?

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orientie objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place Simard, 78210 St.-Cyr-l'Icole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dave Steffen wrote:
      [...]
>  We had a discussion a while back on the Boost users list about this
>  issue, which can be called the "How do I unit test private methods"
>  issue.  I'll summarize results as:

>  A) Black Box Testing: Don't test private methods or members at all.
>  Test only the public interface.  Your classes have a specified
>  interface with specified behaviors.  Test that and don't worry about
>  the rest.

This is essential if the tests are to be used for regression
testing.  Anything which isn't part of the specified interface
may change from one version to the next.  If you test it, you'll
have problems when the test suite automatically runs.
(Generally speaking, checking in a component will cause the unit
tests to be automatically executed.  If they fail, for any
reason, the new version of the component will not be visible to
other users; depending on the system, the check-in itself will
fail.)

>  B) White Box Testing: poke and prod at the object's guts, e.g. private
>  members and methods.  The question here is how to get access to the
>  object's guts.

>  Note that in some cases black box testing is appropriate, and in some
>  cases white box testing is appropriate.  One isn't better than the
>  other, they're for different kinds of testing.

White box testing cannot be used for regression tests, which
practically means, for any formallized test procedure.  I've
never found them of much use anywhere, but it's possible that
others find them useful for debugging and such.  If that's the
case, access to the private parts isn't a problem; most
debuggers ignore access protection completely.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orientie objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place Simard, 78210 St.-Cyr-l'Icole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <1171546178.649654.103620@a34g2000cwb.googlegroups.com>, 
James Kanze <james.kanze@gmail.com> writes
>> What the OP said, "code a little, test a lot" made me think it meant
>> you couldn't write 100 lines without spending a long time testing them.


There is another way f viewing that injunction.

Write a little code
test it
add a little  more code
test it
add a little more code
test it
etc.
and as the code grows so do the tests because you want to make sure that 
your code retains its earlier behaviour.

The process of writing tests should intermingle with the process of 
writing application code. If you ever write some code and cannot think 
how to test it, think again because if the behaviour of the code is not 
testable the code is not reliable.

For novice code the test is often a matter of running the code but as 
soon as you start writing classes you should have a test harness that 
tests each member function (and I generally write a member function then 
test it before I progress to writing the next one.

-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' and "You Can Program in C++"
see http://www.spellen.org/youcandoit
For project ideas and contributions:
http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <lb6fa4-mk1.ln1@helios.steinba.ch>, Ingolf Steinbach 
<ylokm2y02@sneakemail.com> writes
>James Kanze wrote:
>> Without adding to it, that's bad advice.  Consider writing the
>> requirements specifications first, then write the class and its
>> tests.
>
>What method do you (or others) recommend to unit-test private
>functions (in C++) or file-scope (static) functions without
>littering the actual code with stuff needed only for testing?

This is just an idea for those who have to do their own testing. Include 
the following in your class definition:

#if TESTING
      friend void test_function();
#endif

Now test function has full access when TESTING is #edefed to something 
other than 0

-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' and "You Can Program in C++"
see http://www.spellen.org/youcandoit
For project ideas and contributions:
http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <1171519295.374058.321510@v45g2000cwv.googlegroups.com>, 
anthony.ajw@gmail.com writes
>Here's a short list of things that I find help with testing in C++:
>
>* Write short functions. The less a function does, the easier it is to
>test.
Well a function should only conceptually do one thing :-) Yes there are 
exceptions to that guideline but keeping to it makes writing test a 
sight easier.

-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' and "You Can Program in C++"
see http://www.spellen.org/youcandoit
For project ideas and contributions:
http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Fri, 16 Feb 2007 07:43:24 CST, Anthony Williams <anthony_w.geo@yahoo.com>
wrote:
>>  B2)  Friend testers:  give your class a friend, define the friend in
>>  your unit test, and give the friend class an interface that exposes
>>  the guts of your class.
>
>I've done this before, and I'd really rather not. It means you have to
>modify
>the class (i.e. add a friend declaration) every time you want to add a new
>test class, which limits the extent to which you can readily break your
>tests
>into separate classes.

I've used the Friend Tester behavior successfully in many projects. For any
class that has unit tests (which is essentially any class that has any kind
of
code in it), we reserve a name for the friend tester and declare it.

For example:

class MyClass
{
public:
    void doubleA();
    // ...

protected:
    // ...

private:
    int a;
    double b;
    // ...

    friend class MyClass_Test;
};

Leaving the declaration in the production code is completely harmless as
long
as it doesn't clash with any other class name (Note that the _Test suffix is
chosen because it violates the CamelCase naming convention of regular
classes,
eliminating the possibility of accidental clashes).

In the test code, the following class may be declared:


class MyClass_Test
{
public:
    static int& get_a(MyClass& m) { return m.a; }
    static int get_a(const MyClass& m) const { return m.a; }
    static double& get_b(MyClass& m) { return m.b; }
    static double get_b(const MyClass&m) const { return m.b; }
};

void doTest()
{
    MyClass classUnderTest;
    MyClass_Test::get_a(classUnderTest) = 4;
    classUnderTest.doubleA();
    VERIFY(MyClass_Test::get_a(classUnderTest) == 8);
}

This allows us to inject and verify changes to the private members of the
class. The syntax is kind of cumbersome, but that encourages testers to
conduct tests using the public interface as much as possible.


BTW, templated classes get templated friend classes as well:

template <typename A, typename B>
class MyClass
{
    // ...

    template <typename A, typename B>
    friend class MyClass_Test;
};

-dr

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"James Kanze" <james.kanze@gmail.com> writes:

> Dave Steffen wrote:
[...]

> >  A) Black Box Testing: Don't test private methods or members at
> >  all.  Test only the public interface.  Your classes have a
> >  specified interface with specified behaviors.  Test that and
> >  don't worry about the rest.
> 
> This is essential if the tests are to be used for regression
> testing.  Anything which isn't part of the specified interface may
> change from one version to the next.  If you test it, you'll have
> problems when the test suite automatically runs.

  Yup.

[...]

> >  B) White Box Testing: poke and prod at the object's guts,
> >  e.g. private members and methods.  The question here is how to
> >  get access to the object's guts.
> 
> >  Note that in some cases black box testing is appropriate, and in
> >  some cases white box testing is appropriate.  One isn't better
> >  than the other, they're for different kinds of testing.
> 
> White box testing cannot be used for regression tests, which
> practically means, for any formallized test procedure.  I've never
> found them of much use anywhere, but it's possible that others find
> them useful for debugging and such.  If that's the case, access to
> the private parts isn't a problem; most debuggers ignore access
> protection completely.

  Yup. Absolutely.

  I've found white box testing to be very useful when building a
  difficult or complicated class; you can test that the innards do
  what you think they do as you go along.  But as you mention above,
  such tests are less useful (or even harmful) later.

  The other place such testing is necessary is for fielded systems,
  e.g. software that's going on an aircraft or spacecraft.  As you
  mention, a lot of that testing is done with a debugger, in which
  case the whole "how do I unit test the guts" question is irrelevent.
  However, I suspect having white box unit tests will also be very
  useful.  Check back with me in about two years, when we'll be
  looking seriously at deploying the thing I'm building a prototype of
  now. I'm sure I'll have a more informed opinion then. :-)

----------------------------------------------------------------------
Dave Steffen, Ph.D.                Disobey this command!
Software Engineer IV                 - Douglas Hofstadter
Numerica Corporation
dg@steffen a@t numerica d@ot us  (remove @'s to email me)


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Maciej Sobczak <no.spam@no.spam.com> writes:

> Dave Steffen wrote:
> 
> >  B3)  #define private public at the top of your unit tests.
> > 
> >      pros: spills the guts of every class you might ever want to get
> >      at, with no changes to source code whatsoever.
> > 
> >      cons: raises the hackles of good C++ programmers around the
> >      world; feels like a bletcherous hack, even after you've gotten
> >      used to it.
> 
> Alternative without the cons above:
> 
> class A
> {
> public:
>      // ...
> 
> #ifndef UNIT_TEST_ENABLED
> private:
> #endif
>      // ...
> };
> 
> This is at least kosher with respect to redefining keywords.

  Yes, but it also means modifying your application code to support
  unit testing, which is not ideal.  Of course, redefining keywords
  isn't ideal either. :-)

> >  Now, all three of these have their advocates and their detractors
> 
> I would add additional one for consideration: add public testMe or 
> doSelfTest method:
> 
> class A
> {
> public:
>      // ...
> 
>      void doSelfTest();
> 
> private:
>      // all stuff encapsulated as usual
>      // ...
> };
> 
> Of course, doSelfTest might be defined in a separate .cpp file, so it 
> can be excluded from production code by just not linking it. In a test 
> environment, this function is defined to do whatever is has to do, 
> called by the test driver and linked in on demand.
> 
> The fact that doSelfTest is by convention in a separate .cpp file makes 
> it easy to have a whole set of such independent functions (just select 
> them at link time), possibly provided with different testing frameworks 
> or objectives in mind.

  OK, I'll buy that as another option.  However, you're still going to
  have your doSelfTest.cpp file compiled and linked in to your
  executable or I think your linker will complain.  Also, it may or
  may not play nicely with your unit test framework.  But other
  options have the same problems.  And this is rather more similar to
  the approaches taken in, say, Python and (sometimes) Java, where the
  unit tests are just part of the class.

----------------------------------------------------------------------
Dave Steffen, Ph.D.                Disobey this command!
Software Engineer IV                 - Douglas Hofstadter
Numerica Corporation
dg@steffen a@t numerica d@ot us  (remove @'s to email me)

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

{ bogus "proprietary information" disclaimer has been removed. -mod }

Anthony Williams <anthony_w.geo@yahoo.com> writes:

> Dave Steffen <dgsteffen@numerica.us> writes:
> 
> > anthony.ajw@gmail.com writes:
[...]

> >> * Test through the public interface to a class. If the test needs
> >> access to private data or functions, consider making them public. If
> >> making them public on this class would be bad for some reason,
> >> consider extracting a smaller class on which they can be made
> >> public, and which can be used as a building block for the original
> >> class.
> >
> >  Here I strongly, completely, and absolutely disagree.  There are good
> >  reasons for data and methods to be private. Breaking encapsulation
> >  for the sake of unit tests is a cardinal sin in my book.
> 
> I think you are misunderstanding my suggestion. It is merely *an
> application* of good OO design --- ensure each class has a single
> responsibility. If the class needs its private implementation
> details testing in a way that can't be done through the public
> interface, then it's probably doing too much, and there is another
> class with its own Single Responsibility waiting to be extracted
> (and which can therefore be tested in isolation), so the code in the
> original outer class can focus on the actual responsibility of that
> class.

  Ahh.  Yes, I did misunderstand.  Your point is don't test the
  internals; pull the internals out into a separate class and unit
  test that.  Yes, this is an excellent approach when applicable.
  It's not white box testing, but it's black box testing on smaller
  black boxes, which I don't think anyone will argue with. :-)

[... and reordered ...]

> >  Now, if you're going to do white box testing, you need access to the
> >  object's guts, e.g. you need to "break" encapsulation in some way:
> >
> >  B1)  Inheritance:  publicly derive a 'testClass' from the class in
> >  question.  The derived class either exposes guts of its parent for
> >  unit testing purposes, or is actually the unit test itself.
> >
> >      pros: maintains encapsulation, doesn't raise hackles
> >
> >      cons: mandates that testable classes make (at least some)
> >      internal stuff protected instead of private. May or may not play
> >      well with your unit testing framework.
> 
> I don't think this is a good idea, precisely because you have to
> make things protected. This breaks the encapsulation of the class
> --- protected is "public in some contexts", and it means that other
> code can also get at the "guts" of your class just by deriving from
> it.

  It breaks encapsulation, but only amongst family members.  :-)
  Agreed, this isn't ideal.  Neither are any of the other options.  Of
  the three possibilities I listed, almost nobody thinks all three are
  a good idea, but almost everybody thinks that at least one is
  acceptable. 

> >  B2)  Friend testers:  give your class a friend, define the friend in
> >  your unit test, and give the friend class an interface that exposes
> >  the guts of your class.
> 
> I've done this before, and I'd really rather not. It means you have
> to modify the class (i.e. add a friend declaration) every time you
> want to add a new test class, which limits the extent to which you
> can readily break your tests into separate classes.

  Again, yup, it's not ideal.  It has the advantage, though, of
  requiring _minimal_ changes to the class; one line, at the end,
  adding a friend tester class.

  You don't need different tester classes, though; you just need one
  that spills enough of the class' guts to let you poke around in
  them. :-)

> >  B3)  #define private public at the top of your unit tests.
> 
> This renders your program not-well-formed. I wouldn't go there.

  Well, wait a minute.  I'm aware of the prohibitions against
  redefining keywords (although I don't remember the details), but I
  suspect that is is fairly portable, safe, and appropriate _for unit
  tests_.  Of course, it's anathema anywhere else. :-) And as an
  earlier conversation established, it won't change the behavior of
  valid code unless you're doing things you probably shouldn't be
  doing.

  That is, I suspect it's a viable option from a _practical_ point of
  view.  I'd like to hear evidence or arguments to the contrary...

----------------------------------------------------------------------
Dave Steffen, Ph.D.       Fools ignore complexity.
Software Engineer IV      Pragmatists suffer it.
Numerica Corporation      Some can avoid it.
ph (970) 461-2000 x227    Geniuses remove it.
dgsteffen@numerica.us            -- Alan Perlis

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

{ bogus "proprietary information" disclaimer has been removed. -mod }

Ingolf Steinbach <ylokm2y02@sneakemail.com> writes:

> James Kanze wrote:

> > Without adding to it, that's bad advice.  Consider writing the
> > requirements specifications first, then write the class and its
> > tests.
> 
> What method do you (or others) recommend to unit-test private
> functions (in C++) or file-scope (static) functions without
> littering the actual code with stuff needed only for testing?

  Well, see my long post up-thread... this is exactly the question.
  Given that you want to unit test stuff that's encapsulated, how do
  you "break" that encapsulation to test it?

> Testing only the public (or global scope) interface does not really
> solve the complexity problem, or does it?

  Again, see up-thread for my comments and James Kanze's excellent
  response.

----------------------------------------------------------------------
Dave Steffen, Ph.D.       Fools ignore complexity.
Software Engineer IV      Pragmatists suffer it.
Numerica Corporation      Some can avoid it.
ph (970) 461-2000 x227    Geniuses remove it.
dgsteffen@numerica.us            -- Alan Perlis

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Feb 15, 4:52 pm, Ingolf Steinbach <ylokm2...@sneakemail.com> wrote:
> James Kanze wrote:
> > Without adding to it, that's bad advice.  Consider writing the
> > requirements specifications first, then write the class and its
> > tests.
>
> What method do you (or others) recommend to unit-test private
> functions (in C++) or file-scope (static) functions without
> littering the actual code with stuff needed only for testing?
>
> Testing only the public (or global scope) interface does not
> really solve the complexity problem, or does it?

In cases where I need to test private functions, I typically add one
or more public functions that perform various tests on the private
implementation and return boolean results. The test drivers can call
those functions and react to their results. Also, those testing
functions are usually const -- there's not really a reason for them to
change the state of the object you're testing. (And when I say
"usually," I mean that I've never made a non-const test function, but
I don't want to make a blanket statement).

Bob


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Francis Glassborow wrote:
> In article <1171546178.649654.103620@a34g2000cwb.googlegroups.com>,
> James Kanze <james.kanze@gmail.com> writes
> >> What the OP said, "code a little, test a lot" made me think it meant
> >> you couldn't write 100 lines without spending a long time testing them.

> There is another way f viewing that injunction.

> Write a little code
> test it
> add a little  more code
> test it
> add a little more code
> test it
> etc.
> and as the code grows so do the tests because you want to make sure that
> your code retains its earlier behaviour.

That's not a bad procedure, as long as it's understood what you
mean by it.  It doesn't mean, for example, neglecting design or
specification.

Often, a component will be small enough to qualify directly as a
"little code", and what you are suggesting really comes down to
unit test each component before attacking the next.  Other
times, components will be larger; it's quite reasonable to
implement only part of the component, then the unit tests for
that part.

> The process of writing tests should intermingle with the process of
> writing application code. If you ever write some code and cannot think
> how to test it, think again because if the behaviour of the code is not
> testable the code is not reliable.

On the other hand, there are things that you cannot easily test,
especially in low level code.  Or threading.  How do you test
something that depends on when the OS does a context switch?  If
you know of some way to effectively test thread safety, I'm all
ears, but to date, I'm unaware of anything reliable.

> For novice code the test is often a matter of running the code but as
> soon as you start writing classes you should have a test harness that
> tests each member function (and I generally write a member function then
> test it before I progress to writing the next one.

That's one way of doing it.  Although I think you often have to
group them: it's hard to test a setter without having written
the getter, and vice versa.

--
James Kanze (Gabi Software)            email: james.kanze@gmail.com
Conseils en informatique orient�e objet/
                    Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Fri, 16 Feb 2007 07:59:14 CST, Francis Glassborow wrote:

>I generally write a member function then test it before I progress to
>writing the next one.

Hmm, I had a look at my code base, and there are a lot of places where
that would have not been possible (as I was expecting): very often one
of the member functions require other member functions to work
(sometimes private member function, but even public ones). Would you
consider that a sign of (possibly) bad design or implementation?

Genny.

--
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Francis Glassborow wrote:
>> I generally write a member function then test it before I progress to
>> writing the next one.

Gennaro Prota <clcppm-poster@this.is.invalid> wrote:
> Hmm, I had a look at my code base, and there are a lot of places where
> that would have not been possible (as I was expecting): very often one
> of the member functions require other member functions to work
> (sometimes private member function, but even public ones). Would you
> consider that a sign of (possibly) bad design or implementation?

There are certainly lots of times where one public method calls another,
but the call graph is usually a tree (or at least, a DAG).  If that's the
case, just write (and test) the leaf methods first.  Of course, this
assumes you know enough about how the class will end up when you start
writing it to know what the call graph will be :-)

If you really do have two methods which depend on each other, just write
the two methods at the same time and test them at the same time.  The goal
is to produce working code, not to have religious debates about the purity
of your software development process.

Another possibility is to write one, stub out the other (so it returns a
constant, but semantically correct result) and start your testing there.
In essence, you've broken the call graph cycle by doing this, and reduced
the problem back to the simple case.  Once you've got the "first" method
completely tested, then flesh out the second, re-establishing the cycle in
the process.  You might even find that you want to repeat the
stub-one-and-test-the-other process in reverse before letting them talk to
each other.  If you're really lucky, in the process of doing this you'll
discover a way to refactor both of them to eliminate the cyclic dependency
:-)

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On 18 Feb, 16:00, Gennaro Prota <clcppm-pos...@this.is.invalid> wrote:
> On Fri, 16 Feb 2007 07:59:14 CST, Francis Glassborow wrote:
> >I generally write a member function then test it before I progress to
> >writing the next one.
>
> Hmm, I had a look at my code base, and there are a lot of places where
> that would have not been possible (as I was expecting): very often one
> of the member functions require other member functions to work
> (sometimes private member function, but even public ones). Would you
> consider that a sign of (possibly) bad design or implementation?

....and what about virtual functions? Does it make sense to specify
a test for a method that will not be implemented for a couple of
inheritance steps?

Rune


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <1171706042.378302.7240@j27g2000cwj.googlegroups.com>, James
Kanze <james.kanze@gmail.com> writes
>That's one way of doing it.  Although I think you often have to
>group them: it's hard to test a setter without having written
>the getter, and vice versa.
True, but if I write such pairs of functions I start questioning my
design (sometimes it is fine, but often it is a symptom of limited
understanding)

-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' and "You Can Program in C++"
see http://www.spellen.org/youcandoit
For project ideas and contributions: 
http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <4ucgt2t28evkdhoh71msn9rrrk2r50l34l@4ax.com>, Gennaro Prota
<clcppm-poster@this.is.invalid> writes
>On Fri, 16 Feb 2007 07:59:14 CST, Francis Glassborow wrote:
>
>>I generally write a member function then test it before I progress to
>>writing the next one.
>
>Hmm, I had a look at my code base, and there are a lot of places where
>that would have not been possible (as I was expecting): very often one
>of the member functions require other member functions to work
>(sometimes private member function, but even public ones). Would you
>consider that a sign of (possibly) bad design or implementation?
If that is happening a lot I have my doubts about the design.
If you have a cluster of functions that can only be tested together it
seems to me that there is far too much dependency.

Of course some functions depend on others, but you write those other
first. For example I normally start by writing a ctor. Until that is
done there is not much else that can be done. I often follow by writing
a void print(ostream &) const next. That function will need repetitive
attention as I continue refining the rest but it gives me a good tool
for examining the state of a class object (and it is also useful when I
want to implement an operator << (ostream &, type const &)


-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' and "You Can Program in C++"
see http://www.spellen.org/youcandoit
For project ideas and contributions: 
http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Francis Glassborow wrote:
> In article <1171706042.378302.7240@j27g2000cwj.googlegroups.com>, James
> Kanze <james.kanze@gmail.com> writes
> >That's one way of doing it.  Although I think you often have to
> >group them: it's hard to test a setter without having written
> >the getter, and vice versa.
> True, but if I write such pairs of functions I start questioning my
> design (sometimes it is fine, but often it is a symptom of limited
> understanding)

It depends on the application, but that particular pair was only
meant as an example.  The point is that many functions do
"group" in some way; that the results of one function depend on
other functions.  In fact, I'd also say that if any of the
functions in a class are independant, I'd start questioning the
design; it might be that I really should be using several
different classes.

-- 
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Francis Glassborow wrote:
> In article <4ucgt2t28evkdhoh71msn9rrrk2r50l34l@4ax.com>, Gennaro Prota
> <clcppm-poster@this.is.invalid> writes
> >On Fri, 16 Feb 2007 07:59:14 CST, Francis Glassborow wrote:

> >>I generally write a member function then test it before I progress to
> >>writing the next one.

> >Hmm, I had a look at my code base, and there are a lot of places where
> >that would have not been possible (as I was expecting): very often one
> >of the member functions require other member functions to work
> >(sometimes private member function, but even public ones). Would you
> >consider that a sign of (possibly) bad design or implementation?

> If that is happening a lot I have my doubts about the design.
> If you have a cluster of functions that can only be tested together it
> seems to me that there is far too much dependency.

I'd say almost the opposite.  If the functions don't depend on
one another in some way, what are they doing in the same class?

> Of course some functions depend on others, but you write those other
> first. For example I normally start by writing a ctor. Until that is
> done there is not much else that can be done.

And until you have some accessors, you cannot test the
constructor.  For most of my value classes, about all that's
left is assignment.

-- 
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Rune Allnor" <allnor@tele.ntnu.no> wrote:

> On 18 Feb, 16:00, Gennaro Prota <clcppm-pos...@this.is.invalid> wrote:
> > On Fri, 16 Feb 2007 07:59:14 CST, Francis Glassborow wrote:
> > >I generally write a member function then test it before I progress to
> > >writing the next one.
> >
> > Hmm, I had a look at my code base, and there are a lot of places where
> > that would have not been possible (as I was expecting): very often one
> > of the member functions require other member functions to work
> > (sometimes private member function, but even public ones). Would you
> > consider that a sign of (possibly) bad design or implementation?
> 
> ...and what about virtual functions? Does it make sense to specify
> a test for a method that will not be implemented for a couple of
> inheritance steps?

Certainly! If you are defining a pure virtual member-function, then you
are doing so in the process of creating a context for that function to
be called in (if that isn't the case, then I question the design.) You
want to test to make sure the context uses the virtual correctly.

This means deriving a test class from the abstraction that simply
records when and how the abstraction is used, passing an object of the
test abstraction into the context and then reading the results out of
the test abstraction.

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <roy-F68511.11180218022007@032-325-625.area1.spcsdns.net>,
 Roy Smith <roy@panix.com> wrote:

> Francis Glassborow wrote:
> >> I generally write a member function then test it before I progress to
> >> writing the next one.
> 
> Gennaro Prota <clcppm-poster@this.is.invalid> wrote:
> > Hmm, I had a look at my code base, and there are a lot of places where
> > that would have not been possible (as I was expecting): very often one
> > of the member functions require other member functions to work
> > (sometimes private member function, but even public ones). Would you
> > consider that a sign of (possibly) bad design or implementation?
> 
> There are certainly lots of times where one public method calls another,
> but the call graph is usually a tree (or at least, a DAG).  If that's the
> case, just write (and test) the leaf methods first.  Of course, this
> assumes you know enough about how the class will end up when you start
> writing it to know what the call graph will be :-)

I remember reading once in some old C programming book, "design from the
top down, implement from the bottom up." (I think the title was
something like "Graphics in C")

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"James Kanze" <james.kanze@gmail.com> wrote:
> I'd say almost the opposite.  If the functions don't depend on
> one another in some way, what are they doing in the same class?

They all operate on the same data?

> And until you have some accessors, you cannot test the
> constructor.  For most of my value classes, about all that's
> left is assignment.

As PC would say, "Not true!".  My first test case will typically be just to
see that I can construct an object.  It's the "smoke test" of a new class.
To get to the point where you can compile and run:

test_fooConstructor() {
    foo f;
}

you need to do all of (details will depend on your environment):

1) Create a .h file and declare the class in it

2) Create a .cpp file and define the constructor

3) Teach your build system (and your version control system) that both
files exist

4) Create a test case

5) Teach your build system (and your version control system) about the new
test case

6) Document what the class does (which might mean teaching your doc
generation system that the class exists)

Most of this is pretty trivial, but it's enough stuff that there's plenty
of room to do something wrong, even if it's only making a typo.  You've got
to do all this stuff anyway, so you might as well get it all out of the way
now.  You might even discover that it doesn't compile (or link, or run).
With so little code actually written, it should be trivial to figure out
what went wrong at this point!

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"James Kanze" <james.kanze@gmail.com> wrote in
news:1171447862.583583.126700@s48g2000cws.googlegroups.com:

> You need it, in order to validate your process.  Obviously, if
> the tests find an error, there's something wrong in your process
> upstream, which you'll want to address.  But in my experience,
> if you skip on the testing, things will degenerate upstream, and
> you'll end up with very bad code.
> 
> Note that a lot of books seem to neglect this point.  You don't
> test to find errors in the code; you test to ensure that your
> process is working.

Actually, you test to verify that your software meets the
requirements.  If it doesn't meet the requirements, then there are
errors in the code.  While unit testing, you test each case.  If
something fails, you fix it.  If after testing, you discover a high
defect rate, e.g., a metric of defects/KLOC, then there could be a
problem with the process or application of the process.  You use
metrics to determine if your process works.

Process validation is an entirely different meta-activity conducted
outside the scope of solving the problem at hand.  You use that to
figure out if you're spending too much or too little time on an
activity or whether you're even doing the activity right.  That should
be done during a post-mortem phase, perhaps immediately following unit
test.

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Feb 18, 5:23 pm, Francis Glassborow <fran...@robinton.demon.co.uk>
wrote:
> In article <1171706042.378302.7...@j27g2000cwj.googlegroups.com>, James
> Kanze <james.ka...@gmail.com> writes>That's one way of doing it.  Although I think you often have to
> >group them: it's hard to test a setter without having written
> >the getter, and vice versa.
>
> True, but if I write such pairs of functions I start questioning my
> design (sometimes it is fine, but often it is a symptom of limited
> understanding)

What about functions like std::vector's pop_back? How would one test
pop_back without first having items in the vector?  And how could one
have items in the vector without first having working (tested)
functions that put items there (e.g. push_back)?

--Michael Fawcett


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Rune Allnor" <allnor@tele.ntnu.no> wrote in
news:1171445038.719625.156480@q2g2000cwa.googlegroups.com:

> At the risk of offending the moderator, I wasn't shure whether or not
> this claim was a troll in Matias' post. "Avoid testing if you can"
> doesn't go very well with what seems to be a persistent main matra
> in present-day textbooks?

There's hyberpole to the statement, but there's a grain of truth.
Sure you need to do some testing, but it shouldn't be a substitute for
good design, careful coding, or early formal inspection.  Testing
should play a small role in quality from a developer's perspective.
Proper design, design reviews, and code reviews should take about 50%
of the total effort for solving a problem, with code taking roughly
30% and unit test another 10% (overhead is 10%).  These are from the
SEI Personal Software Process.

>From an organizational level, testing plays a much more important
role.  That's when things like regression testing and independent
verification and validation come into play.  As a hobbyist, you'll take
on more of that role just because you'll have to wear all the hats of
an engineering shop (like it or not).  The big difference is there's
less rigor because you're working on smaller projects with fewer
stakeholders.

The problem with a lot of gray-beard books is that they make sweeping
generalizations without offering data.  I smacked my head several
times while reading Rapid Development: half the time reading horror
stories, half the time wanting more statistics.  Without statistics,
it's programming not engineering.  The neat thing about gathering
the right metrics is that you can see whether more design time really
did result in fewer defects, whether reviews were effective, and
whether the code has a positive trend in defects discovered after unit
testing.


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Feb 16, 7:59 am, Francis Glassborow <fran...@robinton.demon.co.uk>
wrote:
> In article <1171546178.649654.103...@a34g2000cwb.googlegroups.com>,
> James Kanze <james.ka...@gmail.com> writes
>
> >> What the OP said, "code a little, test a lot" made me think it meant
> >> you couldn't write 100 lines without spending a long time testing them.
>
> There is another way f viewing that injunction.
>
> Write a little code
> test it
> add a little  more code
> test it
> add a little more code
> test it
> etc.

I prefer to:

write a little test
code until all tests pass
write another test
code until all tests pass
....

The art is in figuring out what to test next.


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Roy Smith wrote:
> "James Kanze" <james.kanze@gmail.com> wrote:
> > I'd say almost the opposite.  If the functions don't depend on
> > one another in some way, what are they doing in the same class?

> They all operate on the same data?

Maybe.  My point is just that totally unrelated functions don't
belong in the same class, and that related functions can't
necessarily be tested separatedly.

> > And until you have some accessors, you cannot test the
> > constructor.  For most of my value classes, about all that's
> > left is assignment.

> As PC would say, "Not true!".  My first test case will typically be just to
> see that I can construct an object.  It's the "smoke test" of a new class.
> To get to the point where you can compile and run:

> test_fooConstructor() {
>     foo f;
> }

> you need to do all of (details will depend on your environment):

> 1) Create a .h file and declare the class in it

> 2) Create a .cpp file and define the constructor

> 3) Teach your build system (and your version control system) that both
> files exist

> 4) Create a test case

> 5) Teach your build system (and your version control system) about the new
> test case

> 6) Document what the class does (which might mean teaching your doc
> generation system that the class exists)

That sounds a bit backwards.  Before I'll even write "class Toto
{};", I'll have documented the role and the responsibilities of
the class.  Once the documentation of the class is finished (the
entire .hh file, more or less), I'll attack the implementation.
Not before.

> Most of this is pretty trivial, but it's enough stuff that there's plenty
> of room to do something wrong, even if it's only making a typo.

Certainly.  On the other hand, is the chance of getting it wrong
high enough to justify special tests just for this.  (Or maybe I
just don't consider it "tests"; even if I don't test, I have to
go through all of the above, except for 4.)

> You've got
> to do all this stuff anyway, so you might as well get it all out of the way
> now.

I'd recommend writing the .hh and the .cc files before trying to
compile the code, too.  As well as introducing the files to the
build system.  Depending on how version control is used, it may
or may not be relevant---I can think of organizations where you
can't check code in until it passes code review, which means
that it must be complete (including tests); in other
organizations, everything is under version control, from the
moment you decide on the name of the file.  (I sort of prefer
the latter, but both can be made to work.)

I see no strong reason for or against writing the first test
case before finishing all of the .cc files, or leaving them to
the end.  It's a question of personal preference.  In the end,
you have a deliverable; that deliverable consists of all of the
sources and all of the test code, and possibly some
documentation (although in my experience, very little
documentation is necessary in addition to that which defines the
deliverable, and which, obviously, must be done before starting
on the deliverable).  The internal order you choose to use to
develope the deliverable really doesn't matter.  Use what works
for you.

> You might even discover that it doesn't compile (or link, or run).
> With so little code actually written, it should be trivial to figure out
> what went wrong at this point!

Generally, if the deliverable is small enough, it should be
trivial to figure out what went wrong whenever the unit test
reveals an error.  (This is one of the main reasons why I find
debuggers of limited use.  But again, used on a unit test, if it
works for you, why not?)

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

kevin  cline wrote:
> On Feb 16, 7:59 am, Francis Glassborow <fran...@robinton.demon.co.uk>
> wrote:
> > In article <1171546178.649654.103...@a34g2000cwb.googlegroups.com>,
> > James Kanze <james.ka...@gmail.com> writes

> > >> What the OP said, "code a little, test a lot" made me think it meant
> > >> you couldn't write 100 lines without spending a long time testing them.

> > There is another way f viewing that injunction.

> > Write a little code
> > test it
> > add a little  more code
> > test it
> > add a little more code
> > test it
> > etc.

> I prefer to:

> write a little test
> code until all tests pass
> write another test
> code until all tests pass
> ...

> The art is in figuring out what to test next.

Don't the requirement specifications tell you that?

The real art is figuring out how to write a test which will fail
if the code doesn't meet the requirement specifications.  Some
things are easy: after push_back() on an empty vector, size()
should return 1, etc.  Others are considerably more difficult:
threading issues come to mind.  (I, and a lot of other people,
I'm sure, would be very interested if you know how to write a
test case for libstdc++ bug no. 21334, for example.)

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Michael Fawcett wrote:
> On Feb 18, 5:23 pm, Francis Glassborow <fran...@robinton.demon.co.uk>
> wrote:
> > In article <1171706042.378302.7...@j27g2000cwj.googlegroups.com>, James
> > Kanze <james.ka...@gmail.com> writes>
> > >That's one way of doing it.  Although I think you often
> > >have to group them: it's hard to test a setter without
> > >having written the getter, and vice versa.

> > True, but if I write such pairs of functions I start questioning my
> > design (sometimes it is fine, but often it is a symptom of limited
> > understanding)

> What about functions like std::vector's pop_back? How would one test
> pop_back without first having items in the vector?  And how could one
> have items in the vector without first having working (tested)
> functions that put items there (e.g. push_back)?

I'm not sure that's a good example.  Obviously, the first
function you write (and test) is the constructor; std::vector
has constructors which fill the vector.  On the other hand, it's
pretty hard to test whether the constructor worked if you don't
have some accessors; either [] or begin()/end().  And like it or
not, you're going to (probably) test the destructor some as well
with the first tests.

Once you've got these basics (constructor, destructor and
accessors), vector actually lends itself very well to Francis'
approach; write a function, then test it.  If that's what you
feel most comfortable with.  (I tend to not do it this way, but
it's a personal choice, not something essential.)

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

W. J. La Cholter wrote:
> "James Kanze" <james.kanze@gmail.com> wrote in
> news:1171447862.583583.126700@s48g2000cws.googlegroups.com:

> > You need it, in order to validate your process.  Obviously, if
> > the tests find an error, there's something wrong in your process
> > upstream, which you'll want to address.  But in my experience,
> > if you skip on the testing, things will degenerate upstream, and
> > you'll end up with very bad code.

> > Note that a lot of books seem to neglect this point.  You don't
> > test to find errors in the code; you test to ensure that your
> > process is working.

> Actually, you test to verify that your software meets the
> requirements.

Yes and no.  If your process is working, your software will meet
the requirements, whether you test it or not.  And of course,
it's not unusual that some requirements aren't really testable.
True quality comes mostly from good design, careful coding and
rigorous review.  Testing really just confirms (or inconfirms)
that these are doing their job.

> If it doesn't meet the requirements, then there are
> errors in the code.

And if there are errors in the code, something upstream in the
process isn't working as well as it should.

> While unit testing, you test each case.  If
> something fails, you fix it.  If after testing, you discover a high
> defect rate, e.g., a metric of defects/KLOC, then there could be a
> problem with the process or application of the process.  You use
> metrics to determine if your process works.

To a degree.  The basic principle is based on metrics.  A
reasonable metric for defects/KLOC would be about 1 defect/100
KLOC, going into integration tests.  (For critical software, of
course, you expect less.)  For smaller projects (or
sub-projects), that means that even a single defect has passed
the limit; the metric has become anything more than 0 defects.

> Process validation is an entirely different meta-activity conducted
> outside the scope of solving the problem at hand.  You use that to
> figure out if you're spending too much or too little time on an
> activity or whether you're even doing the activity right.  That should
> be done during a post-mortem phase, perhaps immediately following unit
> test.

In theory, yes.  Process validation is certainly outside the
scope of solving the problem at hand.  But it's activity is
triggered and determined by measures from the project at hand.

I've found that at least on smaller projects, it's worth
checking after every error detected in tests, whether there was
something you could have done upstream to avoid it: better code
review, better specification of the requirements, etc.  Perhaps
the process hasn't been as good as it should be in the places
I've worked, but in almost every case, a defect detected in
testing was a symptom of something we should have done
differently upstream.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

W. J. La Cholter wrote:
> "Rune Allnor" <allnor@tele.ntnu.no> wrote in
> news:1171445038.719625.156480@q2g2000cwa.googlegroups.com:

> > At the risk of offending the moderator, I wasn't shure whether or not
> > this claim was a troll in Matias' post. "Avoid testing if you can"
> > doesn't go very well with what seems to be a persistent main matra
> > in present-day textbooks?

> There's hyberpole to the statement, but there's a grain of truth.
> Sure you need to do some testing, but it shouldn't be a substitute for
> good design, careful coding, or early formal inspection.

Testing is necessary, but not sufficient.  I think that's what
we've been saying.  (At least, that's what I've been saying:-).)
My own experience has been that if developers know that there is
no serious testing behind them to catch them out, they'll tend
to get a bit careless in the design, coding and inspection.  The
tests aren't there to catch errors in the code, but to keep the
developers honest in the other parts.  (As Dijkstra said,
testing can never show that code is correct, only that it is
incorrect.  My experience has been that if you neglect good
design, careful coding or formal inspection, the code will be
incorrect in ways that show up in testing.)

> Testing
> should play a small role in quality from a developer's perspective.
> Proper design, design reviews, and code reviews should take about 50%
> of the total effort for solving a problem, with code taking roughly
> 30% and unit test another 10% (overhead is 10%).  These are from the
> SEI Personal Software Process.

They sound about right.  Testing plays a small role.  But an
essential role.  In addition to the SEI site, I'd suggest
http://www.idinews.com/.  One of the best sites I know for the
larger issues of how to write good C++ (and Java) code.  A lot
there is just plain common sense, but plain common sense often
seems to be in short supply.

> From an organizational level, testing plays a much more important
> role.  That's when things like regression testing and independent
> verification and validation come into play.  As a hobbyist, you'll take
> on more of that role just because you'll have to wear all the hats of
> an engineering shop (like it or not).  The big difference is there's
> less rigor because you're working on smaller projects with fewer
> stakeholders.

Even for commercial projects, small projects require less
formalization of the process.  If there are only two people
working in software development in your shop, a lot of what has
to be written down in a larger shop can be handled by informal,
verbal agreements.  (But this shouldn't be carried too far.  The
fact that you have code reviews is fine even if it is only
implicitly understood.  Depending on the context, it might even
be acceptable to not write down the results of the review,
although I'm slightly sceptical.  But you definitly want the
requirements you're reviewing against in black and white.)

> The problem with a lot of gray-beard books is that they make sweeping
> generalizations without offering data.  I smacked my head several
> times while reading Rapid Development: half the time reading horror
> stories, half the time wanting more statistics.  Without statistics,
> it's programming not engineering.

Good programming is engineering.  Without any concrete
measurements, it's really just wishful thinking.

> The neat thing about gathering
> the right metrics is that you can see whether more design time really
> did result in fewer defects, whether reviews were effective, and
> whether the code has a positive trend in defects discovered after unit
> testing.

That's an important point.  If you can't measure it, you don't
know whether what you've changed is an improvement or not.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

James Kanze wrote:

>> There's hyberpole to the statement, but there's a grain of truth.
>> Sure you need to do some testing, but it shouldn't be a substitute for
>> good design, careful coding, or early formal inspection.
> 
> Testing is necessary, but not sufficient.

Which immediately brings a question that if there is "something else" 
that is sufficient, then maybe testing is not necessary? :-)

OK, OK - the "sufficient" part can be achieved by combining various 
techniques. No silver bullets, as usual.

> The
> tests aren't there to catch errors in the code, but to keep the
> developers honest in the other parts.

This is very reasonable, but also brings another question - if tests are 
supposed to be the "discipline" motivators for designers and developers, 
then should they write the tests by themselves?

If the developer writes both the test and the code (which is what is 
advocated by many), then it doesn't have this motivating effect, because 
developers will know the extent to which they can be "honest enough". In 
other words, by writing the test they will be able to decide how much 
honesty they will require from themselves later on.

With this in mind I'd argue that some additional benefits can be reached 
when developers do not know what the tests are (did I say - "black-box 
tests?" ;-) ) - then they will be as honest as possible. But this brings 
us back to the isolated QA teams in black suits and the "us and them" 
mindsets.

-- 
Maciej Sobczak : http://www.msobczak.com/
Programming    : http://www.msobczak.com/prog/

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Feb 20, 6:52 am, "James Kanze" <james.ka...@gmail.com> wrote:

> I see no strong reason for or against writing the first test
> case before finishing all of the .cc files, or leaving them to
> the end.

It is one way of testing the test.

The specification for FooTest will typically be something like "If Foo
works, output "Passed", otherwise ...

If every actual run of FooTest outputs "Passed", how confident can you
be that it would successfully detect failure?

In principle you could maintain "broken" versions of Foo, as a means
of testing FooTest.  In practice (at least here) the only time FooTest
reports failure is when it is first written (before the finished Foo
is checked in), or during regression testing (because we broke
something).

I've certainly seen plenty of 'tests' that didn't actually test the
right thing.

Having said that, I've got to admit that a large fraction of my tests
aren't completely written until after the thing being tested has been
written.


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"kevin  cline" <kevin.cline@gmail.com> wrote in
news:1171919431.183022.171430@v33g2000cwv.googlegroups.com: 

> I prefer to:
> 
> write a little test
> code until all tests pass
> write another test
> code until all tests pass
> ...
> 
> The art is in figuring out what to test next.

When do you like to design?

How much do you like to refactor?

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"James Kanze" <james.kanze@gmail.com> wrote in
news:1171968236.031001.216110@m58g2000cwm.googlegroups.com: 

> Michael Fawcett wrote:
>> On Feb 18, 5:23 pm, Francis Glassborow <fran...@robinton.demon.co.uk>
>> wrote:
>> > In article <1171706042.378302.7...@j27g2000cwj.googlegroups.com>,
>> > James Kanze <james.ka...@gmail.com> writes>
>> > >That's one way of doing it.  Although I think you often
>> > >have to group them: it's hard to test a setter without
>> > >having written the getter, and vice versa.
> 
>> > True, but if I write such pairs of functions I start questioning my
>> > design (sometimes it is fine, but often it is a symptom of limited
>> > understanding)
> 
>> What about functions like std::vector's pop_back? How would one test
>> pop_back without first having items in the vector?  And how could one
>> have items in the vector without first having working (tested)
>> functions that put items there (e.g. push_back)?
> 
> I'm not sure that's a good example.  Obviously, the first
> function you write (and test) is the constructor; std::vector
> has constructors which fill the vector.  On the other hand, it's
> pretty hard to test whether the constructor worked if you don't
> have some accessors; either [] or begin()/end().  And like it or
> not, you're going to (probably) test the destructor some as well
> with the first tests.
> 
> Once you've got these basics (constructor, destructor and
> accessors), vector actually lends itself very well to Francis'
> approach; write a function, then test it.  If that's what you
> feel most comfortable with.  (I tend to not do it this way, but
> it's a personal choice, not something essential.)

This kind of gets at a point I should have made earlier.  The approach
of write, test, write, test or code, compile, code, compile is
contrary to good design practice.  With fast modern compilers, it
feels tolerable to do fix, check, fix, check, etc., but it encourages
instant gratification of seeing it work, as opposed to confidence in
implementation and verification by inspection.  Using Personal
Software Process data, I could see that when I got into that mode of
development, I bogged down with constant context-switches.

It also suggests the design wasn't completely fleshed out before
implementation was begun, if the developer tests one thing before
implementing another.  If you've implemented function1 then tested
immediately, but not function2, you may have missed a good design
optimization.  Consider where function1 and function2 could both be
implemented better in terms of functionImp: rather than implementing
functionImp up-front, you need to reimplement function1 in terms of
this new functionImp.  And in terms of defect density, changed code is
the same as new code, whereas reused code is almost free.

Of course process optimization is a tricky thing.  You don't want to
get stuck in a full-on waterfall where everything is designed
up-front.  The Personal Software Process (2.1) is usually optimium for
300-500 LOC in most coding standards for procedural languages.  Beyond
that size, and there needs to be more coordination, planning, and
architecture.  That's not to say everyone's going to get swept up in a
tide of PSP (especially with the opposing trend of Extreme
Programming), but that it corresponds to an industry rule-of-thumb of
aiming for approximately 40 hour work week effort on a task.

I'm not as troubled by unit tests testing more than one interface or
requirement at a time.  A design element might meet more than one
requirement, so at first glance, it seems reasonable.  As long as it's
clear that something is exercising two elements and perhaps those two
are covered in converse order (e.g., to test error cases with pop() on
an empty container that detects errors--unchecked
std::vector::pop_back() being a bad example), it seems like a
reasonable unit test optimization.

On the other hand, if you need to be rigorous about regression tests,
then non-granular unit tests hurt because you're not testing
individual interfaces or features, and you can't reuse your unit tests
for regression testing.

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Maciej Sobczak wrote:
> James Kanze wrote:

> >> There's hyberpole to the statement, but there's a grain of truth.
> >> Sure you need to do some testing, but it shouldn't be a substitute for
> >> good design, careful coding, or early formal inspection.

> > Testing is necessary, but not sufficient.

> Which immediately brings a question that if there is "something else"
> that is sufficient, then maybe testing is not necessary? :-)

In theory, or in practice?  In theory, good design and careful
code review can result in code so good that the tests never
reveal any errors, and so shouldn't be necessary.  In practice,
how do you know that your design is that good and your code
reviews are that careful?  The fact that testing reveals no
errors can be an important indication that you are doing things
right.  (Of course, it can also be an important indication that
your test procedures aren't very good either:-).  If testing
reveals no errors, all you know is that you are at one of the
extremes; you'll need other indicators to tell you which.)

> OK, OK - the "sufficient" part can be achieved by combining various
> techniques. No silver bullets, as usual.

No absolute rules, either.  I once (about five years ago) did a
project with absolutely no tests; I only got my first complete
link a week before delivering it.  After deliverly, the most
serious error that was found in it was a spelling error in one
of the log messages.  It's not a procedure that I'd recommend,
but it happened to work that one time.  (There was no code
review or design review either, which makes me think that the
fact that it worked was close on to a miracle.)

> > The
> > tests aren't there to catch errors in the code, but to keep the
> > developers honest in the other parts.

> This is very reasonable, but also brings another question - if tests are
> supposed to be the "discipline" motivators for designers and developers,
> then should they write the tests by themselves?

Good question.  I've worked in organizations where tests were
the responsibility of the architecture team, who specified what
the class should do.  At the very least, of course, the test
suite must be reviewed with the rest of the code.

> If the developer writes both the test and the code (which is what is
> advocated by many), then it doesn't have this motivating effect, because
> developers will know the extent to which they can be "honest enough". In
> other words, by writing the test they will be able to decide how much
> honesty they will require from themselves later on.

Independantly of honesty, I've generally found that if I've
forgotten to handle a case in the code, it's because I've
forgotten the case, and won't have tested it either.

Review is good here, since it is often easier to recognize that
a test case has been left out, than it is to spot an error in
handling the case in the actual code.

When working alone (as in the project mentionned above), I try
to put code I've written aside, do something else, and then come
back to it, when my "preconceptions" are no longer fixed in my
head (hopefully).  I also find it useful to pretend I'm
explaining it in detail to a collegue, even if that collegue
doesn't exist.  Trying to explain it in terms someone not
familiar with the code would understand often helps.

> With this in mind I'd argue that some additional benefits can be reached
> when developers do not know what the tests are (did I say - "black-box
> tests?" ;-) ) - then they will be as honest as possible. But this brings
> us back to the isolated QA teams in black suits and the "us and them"
> mindsets.

I've heard of companies where the programmers weren't even
allowed to compile the code.  It does make them carefuler about
typing errors, if nothing else.

Other than that: probably the most important role of management
is preventing this "us and them" mindset.  The programmer should
get a positive feeling from code review---it's not personal
criticism, but a chance for him to learn something.  Or even to
show off a little.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                    Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

W. J. La Cholter wrote:
> "James Kanze" <james.kanze@gmail.com> wrote in
> news:1171968236.031001.216110@m58g2000cwm.googlegroups.com:

> > Michael Fawcett wrote:
> >> On Feb 18, 5:23 pm, Francis Glassborow <fran...@robinton.demon.co.uk>
> >> wrote:
> >> > In article <1171706042.378302.7...@j27g2000cwj.googlegroups.com>,
> >> > James Kanze <james.ka...@gmail.com> writes>
> >> > >That's one way of doing it.  Although I think you often
> >> > >have to group them: it's hard to test a setter without
> >> > >having written the getter, and vice versa.

> >> > True, but if I write such pairs of functions I start questioning my
> >> > design (sometimes it is fine, but often it is a symptom of limited
> >> > understanding)

> >> What about functions like std::vector's pop_back? How would one test
> >> pop_back without first having items in the vector?  And how could one
> >> have items in the vector without first having working (tested)
> >> functions that put items there (e.g. push_back)?

> > I'm not sure that's a good example.  Obviously, the first
> > function you write (and test) is the constructor; std::vector
> > has constructors which fill the vector.  On the other hand, it's
> > pretty hard to test whether the constructor worked if you don't
> > have some accessors; either [] or begin()/end().  And like it or
> > not, you're going to (probably) test the destructor some as well
> > with the first tests.

> > Once you've got these basics (constructor, destructor and
> > accessors), vector actually lends itself very well to Francis'
> > approach; write a function, then test it.  If that's what you
> > feel most comfortable with.  (I tend to not do it this way, but
> > it's a personal choice, not something essential.)

> This kind of gets at a point I should have made earlier.  The approach
> of write, test, write, test or code, compile, code, compile is
> contrary to good design practice.  With fast modern compilers, it
> feels tolerable to do fix, check, fix, check, etc., but it encourages
> instant gratification of seeing it work, as opposed to confidence in
> implementation and verification by inspection.

Quite.  IMHO, there is some justification in leaving this up to
the individual engineer (just as e.g. you'd let him choose his
editor).  As long as his final deliverable meets the specs (as
determined by code review, unit tests, etc.), he should have a
fair degree of freedom as to how he gets there.  If the instant
gratification gives him some additional personal satisfaction,
and doesn't have any negative effects on the process, why not?

The key, of course, is that it doesn't have any negative effects
on the process.  I rather suspect that the attitude being
forwarded by some, that you can't write code without first
having written tests for it, will have a very negative effect.
On the other hand, if A feels more at home writing all of the
code in one block, then all of the tests (that's me), where as B
prefers breaking it up into a couple of distinct blocks, writing
the tests for each block before attacking the coding of the
next, I think it acceptable to let each organize this to what he
feels best.

> Using Personal
> Software Process data, I could see that when I got into that mode of
> development, I bogged down with constant context-switches.

Obviously, if you're writing the test for each one or two line
function.  And it may just be you---and me.  Perhaps others
actually work better with constant context switches.  If you're
using a personal software process, of course, you'll experiment
a little with each, measure which is better for you, and use
it.  Again, personally (but in this case, I suspect that it's
fairly univeral), I like to feel productive.  Using a process
which allows me to know, concretely, that I'm productive
provides a great deal of personal satisfaction.

> It also suggests the design wasn't completely fleshed out before
> implementation was begun, if the developer tests one thing before
> implementing another.  If you've implemented function1 then tested
> immediately, but not function2, you may have missed a good design
> optimization.  Consider where function1 and function2 could both be
> implemented better in terms of functionImp: rather than implementing
> functionImp up-front, you need to reimplement function1 in terms of
> this new functionImp.  And in terms of defect density, changed code is
> the same as new code, whereas reused code is almost free.

Well, I'd say that you have to design first, and make these
decisions up front.  And that if two functions really can share
part of the implementation, then you probably shouldn't put them
in separate blocks.  But consider for example something like
std::basic_string (ignoring for the moment that an interface
like that would never occur with good design proceedures): do
you really think it would hurt the process if a programmer
preferred writing the mutator functions first, then writing the
tests for them, before attacking all of the find functions?

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                    Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

James Kanze wrote:

> If the instant gratification gives him some additional personal
> satisfaction, and doesn't have any negative effects on the process, 
> why not?
> 
> The key, of course, is that it doesn't have any negative effects
> on the process.  I rather suspect that the attitude being
> forwarded by some, that you can't write code without first
> having written tests for it, will have a very negative effect.

Do you suspect that because you think that whether it'll work well is 
an individual trait, or for other reasons?


Martin

-- 
Quidquid latine scriptum est, altum videtur.

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Feb 20, 7:49 pm, "W. J. La Cholter" <with...@giganews.com> wrote:
> "kevin  cline" <kevin.cl...@gmail.com> wrote
innews:1171919431.183022.171430@v33g2000cwv.googlegroups.com:
>
> > I prefer to:
>
> > write a little test
> > code until all tests pass
> > write another test
> > code until all tests pass
> > ...
>
> > The art is in figuring out what to test next.
>
> When do you like to design?

In between testing and coding.  I design a little at a time.
>
> How much do you like to refactor?

A fair bit.  Over the years I've seen generally poor results from
attempts to design a lot, code a lot, test a lot, but much better
results from test a little, design a little, code and refactor a
little.


--
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Feb 20, 6:52 am, "James Kanze" <james.ka...@gmail.com> wrote:
> kevin  cline wrote:
> > On Feb 16, 7:59 am, Francis Glassborow <fran...@robinton.demon.co.uk>
> > wrote:
> > > In article <1171546178.649654.103...@a34g2000cwb.googlegroups.com>,
> > > James Kanze <james.ka...@gmail.com> writes
> > > >> What the OP said, "code a little, test a lot" made me think it
meant
> > > >> you couldn't write 100 lines without spending a long time testing
them.
> > > There is another way f viewing that injunction.
> > > Write a little code
> > > test it
> > > add a little  more code
> > > test it
> > > add a little more code
> > > test it
> > > etc.
> > I prefer to:
> > write a little test
> > code until all tests pass
> > write another test
> > code until all tests pass
> > ...
> > The art is in figuring out what to test next.
>
> Don't the requirement specifications tell you that?

Usually not.  For example, if one were attempting to write an HTML
browser, it wouldn't be wise to start with a test to display an
complex web page.  Instead, I would first test the display of an empty
page.  Then I might test the display of simple text elements.
>
> The real art is figuring out how to write a test which will fail
> if the code doesn't meet the requirement specifications...
>
> (I, and a lot of other people,
> I'm sure, would be very interested if you know how to write a
> test case for libstdc++ bug no. 21334, for example.)

My approach for multi-threaded code is to attempt to write code which
is provably thread-safe.
Usually this means writing a rather small class, defining the valid
states for the class, and then proving that instances are always in a
valid state except when locked.


--
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Feb 22, 2:49 am, "kevin  cline" <kevin.cl...@gmail.com> wrote:
> On Feb 20, 6:52 am, "James Kanze" <james.ka...@gmail.com> wrote:

> > kevin  cline wrote:
> > > On Feb 16, 7:59 am, Francis Glassborow <fran...@robinton.demon.co.uk>
> > > wrote:
> > > > In article <1171546178.649654.103...@a34g2000cwb.googlegroups.com>,
> > > > James Kanze <james.ka...@gmail.com> writes
> > > > >> What the OP said, "code a little, test a lot" made me
> > > > >> think it meant you couldn't write 100 lines without
> > > > >> spending a long time testing them.
> > > > There is another way f viewing that injunction.
> > > > Write a little code
> > > > test it
> > > > add a little  more code
> > > > test it
> > > > add a little more code
> > > > test it
> > > > etc.
> > > I prefer to:
> > > write a little test
> > > code until all tests pass
> > > write another test
> > > code until all tests pass
> > > ...
> > > The art is in figuring out what to test next.

> > Don't the requirement specifications tell you that?

> Usually not.  For example, if one were attempting to write an HTML
> browser, it wouldn't be wise to start with a test to display an
> complex web page.  Instead, I would first test the display of an empty
> page.  Then I might test the display of simple text elements.

If I were attempting to write an HTML browser, I'd start with a
detailed description of what it should and should not support.
I'd then do some design, to break it up into smaller pieces; I
wouldn't just sit down and write it as a single component.  And
of course, each of the smaller pieces would have a requirements
specification concerning what that piece should do, and would be
designed, and---given the complexity of a browser---probably
broken up into still smaller pieces.  In the end, I would end up
with pieces which were small enough to work with.

Of course, in practice, one person probably wouldn't write a
browser alone.  It would be a team effort.  And the tests for
the complete browser would probably be developed by the
integration team, in parallel with development.  So the test
suite would be complete and ready as soon as you started getting
complete builds.

> > The real art is figuring out how to write a test which will fail
> > if the code doesn't meet the requirement specifications...

> > (I, and a lot of other people,
> > I'm sure, would be very interested if you know how to write a
> > test case for libstdc++ bug no. 21334, for example.)

> My approach for multi-threaded code is to attempt to write code which
> is provably thread-safe.

Exactly what I do.  And the justification of the proof is in
code review; unless the reviewers are convinced that my proof is
valid, the code doesn't pass review.

I've yet to find an effect means of testing thread-safety (but
I'd like to).

> Usually this means writing a rather small class, defining the valid
> states for the class, and then proving that instances are always in a
> valid state except when locked.

Which still doesn't address things like race conditions.
(Sometimes, too, it's non-trivial to define "valid state".  One
could argue that in the bug cited above, the instance of
basic_string is always in a valid state.  Just not the correct
valid state with regards to other things that are being done.
Of course, if the state is not the correct one, it's not really
valid, but my point is that whether a state is valid or not may
depend on context.)

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                    Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <1172054583.998007.66330@v45g2000cwv.googlegroups.com>, James
Kanze <james.kanze@gmail.com> writes
>When working alone (as in the project mentionned above), I try
>to put code I've written aside, do something else, and then come
>back to it, when my "preconceptions" are no longer fixed in my
>head (hopefully).  I also find it useful to pretend I'm
>explaining it in detail to a collegue, even if that collegue
>doesn't exist.  Trying to explain it in terms someone not
>familiar with the code would understand often helps.
One of the problems for some programmers is that they write tests that
they expect to work. Ordinary code should be written with a positive
hope that it functions correctly. However test code should be written
with the serious intent to break the code being tested. It needs an
attitude that many find difficult when the code they are writing tests
for is their own.

-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' and "You Can Program in C++"
see http://www.spellen.org/youcandoit
For project ideas and contributions: 
http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"James Kanze" <james.kanze@gmail.com> wrote in
news:1172055638.930084.12370@j27g2000cwj.googlegroups.com:

[snip]

> Quite.  IMHO, there is some justification in leaving this up to
> the individual engineer (just as e.g. you'd let him choose his
> editor).  As long as his final deliverable meets the specs (as
> determined by code review, unit tests, etc.), he should have a
> fair degree of freedom as to how he gets there.  If the instant
> gratification gives him some additional personal satisfaction,
> and doesn't have any negative effects on the process, why not?

Let the data show which approach is better, and let the developer make
his/her own interpretations.

This is one of the hardest things about the PSP training: you have to
get out of your comfort zone.  The process is set up so that you *can*
do implement-compile-implement-compile.  It's just you don't get
credit for implementing code if you're doing it in the compilation
phase.  The data show that that the implement-compile-implement
approach is less effective.  It wasn't just my class.  As I looked at
classes all over, the trends were identical across projects.

During the training, you have to suspend your disbelief and get
comfortable with time accounting to the minute.  The whole approach is
not for everyone.  But you get personal satisfaction seeing that a
minute spent in review eliminates two minutes spent in unit test and
that overall LOC/hour remains constant or improves slightly while
defects/KLOC decreases.  By keeping the lifecycle pretty short, you
get a deep satisfaction at the end of the week.

[snip]

> Well, I'd say that you have to design first, and make these
> decisions up front.  And that if two functions really can share
> part of the implementation, then you probably shouldn't put them
> in separate blocks.  But consider for example something like
> std::basic_string (ignoring for the moment that an interface
> like that would never occur with good design proceedures): do
> you really think it would hurt the process if a programmer
> preferred writing the mutator functions first, then writing the
> tests for them, before attacking all of the find functions?

I agree that it's worth tackling them separately for other reasons
that strike at the core of std::basic_string's problem: the thing is
too dang big to develop in one or two weeks.  To do all design, all
design inspection, all coding, etc. one waterfall would be a huge
mistake.  It far exceeds 1 KLOC (VC 8's implementation file "xstring"
exceeds 2 K physical LOC and it's pretty dense).  So, to follow any
good personal process (be it SEI PSP or anything else), a developer is
likely to break it out into multiple iterations, each having the
normal phases.

I implemented a scalable character buffer class (before I became aware
of Alexandrescu's yasli::vector) that was something like a vector.  I
totally underestimated the difficulty and should have broken it out
into multiple iterations.  The end result was a beast that took much
longer because I didn't have early first iteration defect removal hell
to warn me and enable proper replanning.

Long waterfalls are risk aggregators, and applied on a small scale,
are very instructive--as a hobbyist or individual, you can learn this
quick without having suffered through a death march or reading _The
Mythical Man Month_.

But to get back to the design of basic_string, overly complex classes
are bad because they are hard to develop up front, hard to maintain,
and inflexible.  The search algorithms work fine with random access
iterators passed to non-member functions.

And the original poster pointed out later on in the thread the
modularizing does make it much easier to use and test.

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Feb 22, 1:23 pm, "James Kanze" <james.ka...@gmail.com> wrote:
> If I were attempting to write an HTML browser, I'd start with a
> detailed description of what it should and should not support.
> I'd then do some design, to break it up into smaller pieces; I
> wouldn't just sit down and write it as a single component.  And
> of course, each of the smaller pieces would have a requirements
> specification concerning what that piece should do, and would be
> designed, and---given the complexity of a browser---probably
> broken up into still smaller pieces.  In the end, I would end up
> with pieces which were small enough to work with.
>
> Of course, in practice, one person probably wouldn't write a
> browser alone.

I did.  We needed to display some dynamic content for an embedded
application, and none of the available HTML browsers were
satisfactory.  So I wrote a minibrowser for a subset of HTML -- just
text, images, tables, some text formatting, and a few input controls,
and I wrote it in just the manner described. First there was an empty
window, and then there was filled text, and then tables of text, and
then images and then input elements.  As the code grew it was
gradually factored into multiple classes, and the end result was
readily understandable and maintainable.  No piece was ever allowed to
get very large.  As new element types were supported, classes were
created to implement them.  Interfaces were introduced to reduce
coupling and avoid cyclic dependencies.  The work proceeded smoothly
and at all times there was a working component that could have been
delivered if needed.


> > My approach for multi-threaded code is to attempt to write code which
> > is provably thread-safe.
>
> Exactly what I do.  And the justification of the proof is in
> code review; unless the reviewers are convinced that my proof is
> valid, the code doesn't pass review.
>
> I've yet to find an effect means of testing thread-safety (but
> I'd like to).

I can imagine a tool which would simulate two threads, trying all
possible execution sequences.  For a small class I think such
exhaustive testing might be possible.  Yet another reason to build
multi-threaded applications from small thread-safe components.


>
> > Usually this means writing a rather small class, defining the valid
> > states for the class, and then proving that instances are always in a
> > valid state except when locked.
>
> Which still doesn't address things like race conditions.

The statement was intended to address race conditions, but evidently
my meaning wasn't clear.  At this point I think only an example would
be sufficiently precise, but I don't have one handy.

> (Sometimes, too, it's non-trivial to define "valid state".  One
> could argue that in the bug cited above, the instance of
> basic_string is always in a valid state.

I think it's pretty clear that the string is in an invalid state when
the reference count is negative.



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <Xns98DF5FE8D516Ewrybredgmailcom@216.196.97.142>, W. J. La
Cholter <witheld@giganews.com> writes
>But to get back to the design of basic_string, overly complex classes
>are bad because they are hard to develop up front, hard to maintain,
>and inflexible.  The search algorithms work fine with random access
>iterators passed to non-member functions.
IMO basic_string is a good example f why design by committee is a poor
approach. Democracy is not beneficial to design as it leads to trading.
You can have your favourite feature as long as you will support mine :)


-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' and "You Can Program in C++"
see http://www.spellen.org/youcandoit
For project ideas and contributions: 
http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

kevin  cline wrote:

> On Feb 22, 1:23 pm, "James Kanze" <james.ka...@gmail.com> wrote:
>> > My approach for multi-threaded code is to attempt to write code
>> > which is provably thread-safe.
>>
>> Exactly what I do.  And the justification of the proof is in
>> code review; unless the reviewers are convinced that my proof is
>> valid, the code doesn't pass review.
>>
>> I've yet to find an effect means of testing thread-safety (but
>> I'd like to).
> 
> I can imagine a tool which would simulate two threads, trying all
> possible execution sequences.  For a small class I think such
> exhaustive testing might be possible.  Yet another reason to build
> multi-threaded applications from small thread-safe components.

That's called model checking, and it's not just for multi-threaded
programmes. The big problem is that the number of states increases
exponentially, which makes even small programmes very expensive to
check. If they're multi-threaded, even more so. A lot of research in
this field is about techniques for reducing the size of the state
space while still getting the correct results.

Here's an article on a model checker for C++:

P. Leven, T. Mehler, S. Edelkamp, S. Graf, L. Mounier - Directed error
detection in C++ with the assembly-level model checker StEAM
Abstract: http://cat.inist.fr/?aModele=afficheN&cpsidt=15758901
PDF: http://www.spinroot.com/spin/Workshops/ws04/033-spin2004.pdf

Any further discussion on this probably belongs in a different news
group though :).

Lourens


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"James Kanze" <james.kanze@gmail.com> writes:

> Roy Smith wrote:
>> As PC would say, "Not true!".  My first test case will typically be just to
>> see that I can construct an object.  It's the "smoke test" of a new class.
>> To get to the point where you can compile and run:
>
>> test_fooConstructor() {
>>     foo f;
>> }
>
>> you need to do all of (details will depend on your environment):
>
>> 1) Create a .h file and declare the class in it
>
>> 2) Create a .cpp file and define the constructor
>
>> 3) Teach your build system (and your version control system) that both
>> files exist
>
>> 4) Create a test case
>
>> 5) Teach your build system (and your version control system) about the new
>> test case
>
>> 6) Document what the class does (which might mean teaching your doc
>> generation system that the class exists)
>
> That sounds a bit backwards.  Before I'll even write "class Toto
> {};", I'll have documented the role and the responsibilities of
> the class.  Once the documentation of the class is finished (the
> entire .hh file, more or less), I'll attack the implementation.
> Not before.

I use the process of writing the test cases as a means to assist with
identifying the roles and responsibilities, and designing the interface.

I also find that writing a single, small, test and then just enough
implementation to make it pass helps me ensure that the code does what I
intended, and pinpoint problems if it doesn't. The tests also helps me
identify where I have unintentionally changed the behaviour when making a
later change --- acting as regression tests.

>> You've got
>> to do all this stuff anyway, so you might as well get it all out of the way
>> now.
>
> I'd recommend writing the .hh and the .cc files before trying to
> compile the code, too.  As well as introducing the files to the
> build system.  Depending on how version control is used, it may
> or may not be relevant---I can think of organizations where you
> can't check code in until it passes code review, which means
> that it must be complete (including tests); in other
> organizations, everything is under version control, from the
> moment you decide on the name of the file.  (I sort of prefer
> the latter, but both can be made to work.)

I always do the latter, even in environments where the former is 
required ---
I just use a local version control repository rather than the offical 
project
one.

I also write just enough of the .hh and .cc files to pass the current 
(small)
test case before compiling --- I like working in a small grained fashion, as
it is easier for me. I use to write whole swathes of code before 
compiling and
testing, without too much problem, but it is easier to do it in smaller
chunks.

> I see no strong reason for or against writing the first test
> case before finishing all of the .cc files, or leaving them to
> the end.  It's a question of personal preference.  In the end,
> you have a deliverable; that deliverable consists of all of the
> sources and all of the test code, and possibly some
> documentation (although in my experience, very little
> documentation is necessary in addition to that which defines the
> deliverable, and which, obviously, must be done before starting
> on the deliverable).  The internal order you choose to use to
> develope the deliverable really doesn't matter.  Use what works
> for you.

Totally agreed here.

>> You might even discover that it doesn't compile (or link, or run).
>> With so little code actually written, it should be trivial to figure out
>> what went wrong at this point!
>
> Generally, if the deliverable is small enough, it should be
> trivial to figure out what went wrong whenever the unit test
> reveals an error.  (This is one of the main reasons why I find
> debuggers of limited use.  But again, used on a unit test, if it
> works for you, why not?)

Yes. That's why I work in such small chunks these days --- rather than
designing, specifying and implementing a whole class at once, I work on 
a tiny
piece at a time, starting with a test for the desired behaviour.

Anthony
-- 
Anthony Williams
Just Software Solutions Ltd - http://www.justsoftwaresolutions.co.uk
Registered in England, Company Number 5478976.
Registered Office: 15 Carrallack Mews, St Just, Cornwall, TR19 7UL

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Rune Allnor wrote:

> I'm a hobby C++ programmer who have reached the point where
> strain of testing and debugging code are about to outweigh the
> fun and usefulness of my projects.
>
> As is said in most books (e.g. "Pragmatic programmer"), "code a
> little, test a lot."
>
> However, this might be very good advice, but how does one go
> about actually achieving this?

Take all the time you spend, after coding, in debugging.

Transfer it to time spent, before coding, writing test cases.

You will have lots of time left over.

There are those who call this "test-driven development". Write failing
tests, then write code to pass the tests. This implies you (generally) only
write new code if you can get a test to fail, first, because it's not there.

That forces all your code to be testable, and that forces it to strongly
decouple, and improves its design. All as a side-effect of coding very
rapidly, without bugs.

> There have to be things that are
> smart to do, and others that are not at all smart to do? Books
> that address such problems? Those I have found, deal with C#
> or Java, not C++.

Here's the best C++ unit test framework I have reviewed so far:

http://unittest-cpp.sourceforge.net/

And here is one of its tests:

TEST (PassingTestHasNoFailures)
{
     class PassingTest : public UnitTest::Test
     {
     public:
         PassingTest() : Test("passing") {}
         virtual void RunImpl(UnitTest::TestResults& testResults_) const
         {
             CHECK(true);
         }
     };

     UnitTest::TestResults results;
     PassingTest().Run(results);
     CHECK_EQUAL(0, results.GetFailureCount());
}

One thing that's remarkable about that test (compared to >cough< Brand 
X) is
it fits an entire test runner inside its own test case. That proves the 
test
runner class has no run-away dependencies. If these test cases are 
versioned
descendents of cases that forced that class to exist, then the target class
never had the opportunity to grow tangled.

Test-first is an excellent way to generate a fully vetted design.

-- 
   Phlip
   http://www.greencheese.us/ZeekLand <-- NOT a blog!!!


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> White box testing cannot be used for regression tests, which
> practically means, for any formallized test procedure.  I've
> never found them of much use anywhere, but it's possible that
> others find them useful for debugging and such.  If that's the
> case, access to the private parts isn't a problem; most
> debuggers ignore access protection completely.

Respectfully disagree.  White box testing is important because
particular implementation choices may suggest particular test cases.
And it's not unusual for white box testing to be compatible with
regression testing.  That may seem like a strange assertion, and it
comes down to definitions, so let me give mine.

White box philosophy means considering the implementation when
choosing test cases.  Implementation may or may not require checking
internal state at various points during operation (based on the
knowledge that failures will indicate that observable behaviours will
eventually fail too).  The philosophy is useful, as considering the
implementation may lead to additional test cases that wouldn't be
obviously useful from a black-box perspective, but are required for
the testing to be thorough.  If those tests don't require inspection
of private data, they won't break during regression testing, although
they may become practically redundant and inadequate with
implementation changes.

Examples:
- implementation uses a cache: increase the number of test cases to
check the behaviour continues to be appropriate when the cache is
full,
- implementation may stream doubles: check the values retain proper
precision,
- implementation uses specialisation for some types: explicitly check
those types as well as at least one other type
- implementation optimises certain cases with hardcoded results: check
those results as well as some unoptimised results

In all these cases, pure/blind black box testing may be completely
inadequate.


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Phlip wrote:

> Write failing
> tests, then write code to pass the tests. This implies you (generally) only
> write new code if you can get a test to fail, first, because it's not there.

It's not that obvious, because there is nothing that can stop you
implementing more, leading to features that were not asked for.

The problem is that even if you start with a failing test, you will
write a piece of implementation code that is not necessarily minimal
with respect to the given test (and even being minimal can introduce
features that were not asked for!). You will just implement something,
try to see whether the test still fails and if so, come back to coding,
and so on. There is absolutely no provision that this way you will
always implement *only* the features asked for in the test.

This is where derived requirements come from, very often forgotten.

> That forces all your code to be testable

The point is that it's not the case. Granted, with lots of practice you
can come to some optimial balance, but it is not something given. Many
people at the beginning will fall into the trap of small test and big
lump of implementation code, with little test coverage as a result.


-- 
Maciej Sobczak : http://www.msobczak.com/
Programming    : http://www.msobczak.com/prog/

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Phlip wrote:
> Rune Allnor wrote:

> > I'm a hobby C++ programmer who have reached the point where
> > strain of testing and debugging code are about to outweigh the
> > fun and usefulness of my projects.

> > As is said in most books (e.g. "Pragmatic programmer"), "code a
> > little, test a lot."

> > However, this might be very good advice, but how does one go
> > about actually achieving this?

> Take all the time you spend, after coding, in debugging.

> Transfer it to time spent, before coding, writing test cases.

Design would be more useful.

> You will have lots of time left over.

How does writing the tests before you write the code, rather
than later, change the amount of time spent in debugging.  If
you don't design your code, you're going to spend a lot of time
in debugging.  If you do, you're going to spend a lot less.
Regardless of when you write the tests.

> There are those who call this "test-driven development". Write failing
> tests, then write code to pass the tests. This implies you (generally) only
> write new code if you can get a test to fail, first, because it's not there.

Which neglects the entire problem of specification.
"test-driven development" is one of those catch words, invented
when there are no other good arguments.  The very first thing
you have to do when developing software is to decide what it has
to do---what the requirements specifications are.  Then, design.
Only once you've done these two steps can you start writing
code.

And of course, the requirements almost always contain aspects
which are *not* testable.  Where I work, for example, it is a
requirement for almost everything I write that it be thread safe
(impossible to test), and understandable by others (even more
impossible to test).

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                    Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

James Kanze wrote:

>> Take all the time you spend, after coding, in debugging.
>
>> Transfer it to time spent, before coding, writing test cases.
>
> Design would be more useful.

That's why test-first is a design technique. It is useful.

>> You will have lots of time left over.
>
> How does writing the tests before you write the code, rather
> than later, change the amount of time spent in debugging.

Because your designs will strongly resist bugs, you will have fewer of 
them,
and you will have much more time left to devote to the remaining ones.

>  If
> you don't design your code, you're going to spend a lot of time
> in debugging.  If you do, you're going to spend a lot less.
> Regardless of when you write the tests.

So, by that reasoning, if we _do_ design code before writing it, and if we
discover another design technique which happens to use test cases, then we
will spend even less time debugging.

There are those who practice Test-Driven Development - even in advanced
networky C++ - and never feel the need to invoke a debugger. That's not the
same thing as never debugging, but it's very close.

(The specific issue with a debugger is that, for each manual experiment you
perform with it, you could instead have written a new test case. So if your
code is already completely test-ready - because it's well designed - then
you have almost no reason to invoke a debugger.)

>> There are those who call this "test-driven development". Write failing
>> tests, then write code to pass the tests. This implies you (generally) 
>> only
>> write new code if you can get a test to fail, first, because it's not 
>> there.
>
> Which neglects the entire problem of specification.

I also did not mention you use a keyboard. Yes, you also use a keyboard
while doing this technique. And a steno chair, mouse, etc.

> "test-driven development" is one of those catch words, invented
> when there are no other good arguments.  The very first thing
> you have to do when developing software is to decide what it has
> to do---what the requirements specifications are.  Then, design.
> Only once you've done these two steps can you start writing
> code.

And that is exactly why another of these "catch words" is "story test" - a
test written for your business-liaison to review, to help convert verbal
requirements into mechanical specifications.

> And of course, the requirements almost always contain aspects
> which are *not* testable.  Where I work, for example, it is a
> requirement for almost everything I write that it be thread safe
> (impossible to test), and understandable by others (even more
> impossible to test).

Right. Things that are impossible to test are generally "computer science".
Example: Does a complex algorithm fit within an OlogN performance profile?
Putting the OlogN part into a test case would be worse than useless.

However, once you pick an algorithm, its specification will lead to 
numerous
small details, each perfectly ripe for testing.

-- 
   Phlip
   http://www.greencheese.us/ZeekLand <-- NOT a blog!!!


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Phlip" <phlipcpp@yahoo.com> wrote:
> > Design would be more useful.
>
> That's why test-first is a design technique. It is useful.

I don't see TDD as a design technique, I see it as an implementation
technique.

TDD says, "Before you write any code, write a test which fails, then write
code to make that test pass".  That helps ensure that the class's methods
perform as specified.  It doesn't help you figure out what the methods will
do, or what methods you should have.

For example, I sat through a code review of a double-linked list container
class the other day.  One of the reviewers objected to the fact that the
author had included an insertAt() method.  His opinion was that because it
was an O(N) operation, it should not have been provided.

Whether or not a double-linked list container class should implement
insertAt() is a design question.  TDD helps ensure that the method is
correctly implemented, but does nothing to tell you if it SHOULD be
implemented.

As a meta issue, one must wonder why we were wasting our time implementing
yet another double-linked list class when there are so many out there to
pick from already?  That's a good question.  A design question.  One which
TDD doesn't help you answer. :-)

--
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Phlip wrote:
> James Kanze wrote:

> >> Take all the time you spend, after coding, in debugging.

> >> Transfer it to time spent, before coding, writing test cases.

> > Design would be more useful.

> That's why test-first is a design technique. It is useful.

Except that you can't write the tests until you've done at least
the requirements specifications, and you can't capture many
important features of the design in the tests.  Tests do not
replace design.

> >> You will have lots of time left over.

> > How does writing the tests before you write the code, rather
> > than later, change the amount of time spent in debugging.

> Because your designs will strongly resist bugs, you will have
> fewer of them, and you will have much more time left to devote
> to the remaining ones.

Do you have any concrete measurements to back this up.  I don't
see any relationship.  Tests have nothing to do with design, per
se, and most of the criteria for good design aren't testable.
I've written a lot of robust code which has been delivered
without a single test (although that's not a good policy
either), but I've never seen any robust code which has been well
designed.

> > If you don't design your code, you're going to spend a lot
> > of time in debugging.  If you do, you're going to spend a
> > lot less.  Regardless of when you write the tests.

> So, by that reasoning, if we _do_ design code before writing
> it, and if we discover another design technique which happens
> to use test cases, then we will spend even less time
> debugging.

> There are those who practice Test-Driven Development - even in
> advanced networky C++ - and never feel the need to invoke a
> debugger. That's not the same thing as never debugging, but
> it's very close.

I don't use a debugger myself.  And unit tests certainly
contribute to this.  But they don't replace good design, which
is essential as well.

> (The specific issue with a debugger is that, for each manual
> experiment you perform with it, you could instead have written
> a new test case. So if your code is already completely
> test-ready - because it's well designed - then you have almost
> no reason to invoke a debugger.)

The specific issue with a debugger is that you can't use it
until you have a fully linked executable.  By which time, if you
have good design and good code review, there are very, very few
errors left anyway.  And of course, if you've done your design
work, you know what input produces what output, and which steps
the code goes through for each possible input.  Which means that
you can usually find the error just by studying the sources,
knowing the input and the actual output, in less time than it
takes to fire up a debugger.

> >> There are those who call this "test-driven development".
> >> Write failing tests, then write code to pass the tests.
> >> This implies you (generally) only write new code if you can
> >> get a test to fail, first, because it's not there.

> > Which neglects the entire problem of specification.

> I also did not mention you use a keyboard. Yes, you also use a keyboard
> while doing this technique. And a steno chair, mouse, etc.

Obviously.  And?  Even the worst hackers I know use a keyboard.

> > "test-driven development" is one of those catch words, invented
> > when there are no other good arguments.  The very first thing
> > you have to do when developing software is to decide what it has
> > to do---what the requirements specifications are.  Then, design.
> > Only once you've done these two steps can you start writing
> > code.

> And that is exactly why another of these "catch words" is "story test" - a
> test written for your business-liaison to review, to help convert verbal
> requirements into mechanical specifications.

Except that my business-liaison is usually incapable of
understanding what my programs should do.  At least at a level
that is "testable".  On the other hand, my business-liaison does
know that he doesn't what the server to core dump (even if he
doesn't know what a core dump is), and there's no way to be sure
that it won't by testing.

> > And of course, the requirements almost always contain aspects
> > which are *not* testable.  Where I work, for example, it is a
> > requirement for almost everything I write that it be thread safe
> > (impossible to test), and understandable by others (even more
> > impossible to test).

> Right. Things that are impossible to test are generally "computer science".
> Example: Does a complex algorithm fit within an OlogN performance profile?
> Putting the OlogN part into a test case would be worse than useless.

Most threading problems aren't testable either.

> However, once you pick an algorithm, its specification will
> lead to numerous small details, each perfectly ripe for
> testing.

In sum, once you've got a good design, it should be possible to
develop useful unit tests.  Totally agreed.  Once you've got a
good design, it's also possible to code the functions
immediately.  Whether you code the unit tests or the functions
first is totally irrelevant.  And if you're a careful coder, you
may just discover that the unit tests work first go.  I usually
feel that I've been unnecessarily careless if they don't.

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                    Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Roy Smith wrote:

> I don't see TDD as a design technique, I see it as an implementation
> technique.
>
> TDD says, "Before you write any code, write a test which fails, then write
> code to make that test pass".  That helps ensure that the class's methods
> perform as specified.  It doesn't help you figure out what the methods will
> do, or what methods you should have.

Then that is the distinction between requirements and implementation.
Both need a design process.

> For example, I sat through a code review of a double-linked list container
> class the other day.  One of the reviewers objected to the fact that the
> author had included an insertAt() method.  His opinion was that because it
> was an O(N) operation, it should not have been provided.
>
> Whether or not a double-linked list container class should implement
> insertAt() is a design question.  TDD helps ensure that the method is
> correctly implemented, but does nothing to tell you if it SHOULD be
> implemented.

Yes it does. If you can't link a line of code thru tests to storytest
requirements, you don't write it. The performance is irrelevant.

> As a meta issue, one must wonder why we were wasting our time implementing
> yet another double-linked list class when there are so many out there to
> pick from already?  That's a good question.  A design question.  One which
> TDD doesn't help you answer. :-)

One powerful design concept here is "architect the negative space".
TDD _requires_ you to omit the insertAt() until you find a client with
a true need.

If your homegrown linked list has test cases that help document its
existing design, then whoever writes the client that needs insertAt()
can add it without interrupting the linked list's author. And that
client will be best positioned to guess whether O(n) performance (for
a very small O!) is acceptable.

We are not all writing generic publication-ready libraries here, and
pretending we are slows us down. Test cases help us deploy minimal
application-specific classes.

--
   Phlip


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

James Kanze wrote:

> Except that you can't write the tests

You don't write _all_ the tests - this isn't Big Test Up Front.

> until you've done at least
> the requirements specifications, and you can't capture many
> important features of the design in the tests.  Tests do not
> replace design.

I didn't say they replace design.

If you know the next line of code to write (and if you already have a
test harness), then you must perforce be able to think of a simple
test that will fail if the code is not there. The test needn't be
perfect - bad code could pass it. You simply write the test to confirm
you need the code, then write the code to pass the test. Repeat in
tiny cycles.

> Do you have any concrete measurements to back this up.  I don't
> see any relationship.  Tests have nothing to do with design, per
> se, and most of the criteria for good design aren't testable.

Another topic slip: We are not testing that the design is good. We are
using tests to apply pressure to the design, to help us improve it.

> Except that my business-liaison is usually incapable of
> understanding what my programs should do.

That's why you write the acceptance test as literately as possible,
using a Domain Specific Language.

A completely different verbiage wrapped around the same concept is the
Ada SPARKS project. Its devotees also run all their tests - oops I
mean "proofs" - after every few edits...

-- 
  Phlip


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Mar 7, 2:13 am, "Phlip" <phlip2...@gmail.com> wrote:
> James Kanze wrote:

      [...]
> If you know the next line of code to write (and if you already have a
> test harness), then you must perforce be able to think of a simple
> test that will fail if the code is not there.

That's simply false.  I often write things like:

     boost::mutex::scoped_lock lock( someMutex ) ;

when I know that I cannot possibly write a test case which will
fail on my machine (a single processor Sparc), usually, cannot
possibly write a test case guaranteed to fail anywhere.  (But I
know that the code will fail, once in a blue moon, without the
lock.)

My code is also full of asserts of things I know can't happen.
They're there to protect the maintenance programmer: they
document expectations, and even if he misses the documentation,
the code won't get very far if he doesn't meet them.

I also write documentation in the form of comments.  How does
your test test that they are correct?

> The test needn't be
> perfect - bad code could pass it. You simply write the test to confirm
> you need the code, then write the code to pass the test. Repeat in
> tiny cycles.

This is simply bad engineering.  You write code because it is
necessary to solve a problem.  Whether you can test for the
problem or not.  Parts of that problem space are almost by
definition not testable: how do you test the maintainability,
for example.  Other parts are typically not ammendable to
reasonable tests: if you run a test long enough, you might even
be able to check some of the threading issues, but a test which
runs for several years just won't cut it (and I've seen
threading issues which will only cause a problem about once a
year, or less, under intensive use).

--
James Kanze (GABI Software)             email:james.kanze@gmail.com
Conseils en informatique orient�e objet/
                    Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34



-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

James Kanze wrote:

> > If you know the next line of code to write (and if you already have a
> > test harness), then you must perforce be able to think of a simple
> > test that will fail if the code is not there.
>
> That's simply false.  I often write things like:
>
>      boost::mutex::scoped_lock lock( someMutex ) ;

Then learn about mock objects.

--
  Phlip


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Roy Smith wrote:
> "Phlip" <phlipcpp@yahoo.com> wrote:
> 
>>>Design would be more useful.
>>
>>That's why test-first is a design technique. It is useful.
> 
> 
> I don't see TDD as a design technique, I see it as an implementation
> technique.
> 
> TDD says, "Before you write any code, write a test which fails, then write
> code to make that test pass".  That helps ensure that the class's methods
> perform as specified.  It doesn't help you figure out what the methods will
> do, or what methods you should have.
> 
> For example, I sat through a code review of a double-linked list container
> class the other day.  One of the reviewers objected to the fact that the
> author had included an insertAt() method.  His opinion was that because it
> was an O(N) operation, it should not have been provided.
> 
> Whether or not a double-linked list container class should implement
> insertAt() is a design question.  TDD helps ensure that the method is
> correctly implemented, but does nothing to tell you if it SHOULD be
> implemented.
> 
> As a meta issue, one must wonder why we were wasting our time implementing
> yet another double-linked list class when there are so many out there to
> pick from already?  That's a good question.  A design question.  One which
> TDD doesn't help you answer. :-)

It seems that no matter what we do, there will be overlap between 
requirements, design, and test.  Frankly, I consider the insertAt 
question more of a requirements issue than a design issue.  Does a 
particular client need it?

Nothing is perfect.  TDD does have the benefit, however, of encouraging 
a good deal of reflection.


----
Michael Feathers
http://www.objectmentor.com
http://michaelfeathers.typepad.com
author of: Working Effectively with Legacy Code (Prentice Hall 2005)

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

James Kanze wrote:
>>>How does writing the tests before you write the code, rather
>>>than later, change the amount of time spent in debugging.
>
>
>>Because your designs will strongly resist bugs, you will have
>>fewer of them, and you will have much more time left to devote
>>to the remaining ones.
>
>
> Do you have any concrete measurements to back this up.  I don't
> see any relationship.  Tests have nothing to do with design, per
> se, and most of the criteria for good design aren't testable.
> I've written a lot of robust code which has been delivered
> without a single test (although that's not a good policy
> either), but I've never seen any robust code which has been well
> designed.

I think there are a few more studies ongoing, but practice does tend to 
move faster than research at times.

Here's a study from IBM: http://portal.acm.org/citation.cfm?id=776892

> In sum, once you've got a good design, it should be possible to
> develop useful unit tests.  Totally agreed.  Once you've got a
> good design, it's also possible to code the functions
> immediately.  Whether you code the unit tests or the functions
> first is totally irrelevant.  And if you're a careful coder, you
> may just discover that the unit tests work first go.  I usually
> feel that I've been unnecessarily careless if they don't.

If you do TDD then, it will probably feel a bit familiar.  You end up 
using roughly the same thought process.  One thing that is a bit of a 
tripping point is that you may, depending upon your design style, find 
that you end up with smaller classes and member functions when you do 
TDD.  The reason why is because you are considering testability at a 
fine grain and that tends to push you toward more decoupling.

Again, it depends on what your current design style is like.  I'd expect 
that it's probably very good in your workplace, but I notice as I go 
from company to company that people don't often realize how coupled 
their code is until you ask them to take an individual class and test it 
  independently in a test harness.

----
Michael Feathers
http://www.objectmentor.com
http://michaelfeathers.typepad.com
author of: Working Effectively with Legacy Code (Prentice Hall 2005)

-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Michael Feathers a �crit :
> James Kanze wrote:
>[snip]
>> In sum, once you've got a good design, it should be possible to
>> develop useful unit tests.  Totally agreed.  Once you've got a
>> good design, it's also possible to code the functions
>> immediately.  Whether you code the unit tests or the functions
>> first is totally irrelevant.  And if you're a careful coder, you
>> may just discover that the unit tests work first go.  I usually
>> feel that I've been unnecessarily careless if they don't.
> 
> If you do TDD then, it will probably feel a bit familiar.  You end up 
> using roughly the same thought process.  One thing that is a bit of a 
> tripping point is that you may, depending upon your design style, find 
> that you end up with smaller classes and member functions when you do 
> TDD.  The reason why is because you are considering testability at a 
> fine grain and that tends to push you toward more decoupling.
> 
> Again, it depends on what your current design style is like.  I'd expect 
> that it's probably very good in your workplace, but I notice as I go 
> from company to company that people don't often realize how coupled 
> their code is until you ask them to take an individual class and test it 
>  independently in a test harness.

IMVHO that is the idea behind TDD that even a bad designer can produce
acceptable code: minimal interface if not complete, modular class (and
easily refactored), with test unit to understand how to use the class
.... and this make regression test part of the developement process where
people tend to be overconfident sometimes.

I see where it can be useful in a group of C++ developers which don't
have the same level of design skill. Now, it is also possible to make
bad test. The author of Anti-Pattern has begun an Anti-Test list on his
blog.

Again IMHO, a mechanical TDD doesn't yield reusable software because too
  specific problem oriented. In this regard, I don't consider it as a
design technic but a quality process. But then compagnies are more
focused on having a tested, maintainable, marketable product rather than
a collection of highly reusable class; this may be part of the success
of TDD.

Michael


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Michael DOUBEZ wrote:

> IMVHO that is the idea behind TDD that even a bad designer can produce
> acceptable code

I read a science fiction story once, in which a modestly-mentally-
endowed extraterrestrial species, at about the Age of Exploration
level of their progress, accidentally discovered a low-tech form of
space travel.

They located Earth, and attempted to conquer us. We kept outsmarting
them, essentially using the karate technique of pulling when your
opponent pushes.

The aliens' scientists developed a theory for why their seeming
victories kept turning out as failures. They decided that we were
primitive and inferior creatures who possessed a form of "faux-
intelligence"; an animal cunning that, to an intelligent alien, could
sometimes appear as true intelligence.

So maybe our theory here, about TDD, is that it forms a kind of "faux
design skills". It can produce designs which, to an intelligent
designer, can sometimes appear as true design skills?

--
    Phlip
    http://www.greencheese.us/ZeekLand <-- NOT a blog!!!


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Wed,  7 Mar 2007 16:41:00 CST, Michael Feathers 
<mfeathers@objectmentor.com> wrote:
[Test-driven development]
> using roughly the same thought process.  One thing that is a bit of a 
> tripping point is that you may, depending upon your design style, find 
> that you end up with smaller classes and member functions when you do 
> TDD.  The reason why is because you are considering testability at a 
> fine grain and that tends to push you toward more decoupling.

In my experience, it /forces/ you towards looser coupling.

For example, where you need mock objects, you need[1] to derive both the
mock class and your real, live class from some abstract base class
which serves no real purpose in the final program.

For me, that's one big problem with unit tests. I prefer a design that
explicitly tries to solve the problem at hand and nothing else. I
can reason better about such code, and I get better help from the type
system.

Like Phlip said elsewhere in the thread: "We are not all writing
generic publication-ready libraries here, and pretending we are slows
us down."

> [...] I notice as I go 
> from company to company that people don't often realize how coupled 
> their code is until you ask them to take an individual class and test it 
>   independently in a test harness.

That they don't realize it is a bad thing ...  But I disagree with the
implicit assumption that tight coupling is a bad thing, all or most of
the time.

BR,
/Jorgen

[1] Unless I misunderstand something about how people implement mock
     objects in C++. I have only used them in dynamically typed
     languages.

-- 
   // Jorgen Grahn <grahn@        Ph'nglui mglw'nafh Cthulhu
\X/     snipabacken.dyndns.org>  R'lyeh wgah'nagl fhtagn!

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Jorgen Grahn wrote:

> That they don't realize it is a bad thing ...  But I disagree with the
> implicit assumption that tight coupling is a bad thing, all or most of
> the time.

Define "coupling" as "A must change only because B changed". Define "cohere"
as "A shares legitimate reasons to change with B".

Tight coherency is a good thing.

--
  Phlip
  http://www.greencheese.us/ZeekLand <-- NOT a blog!!!


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Mar 15, 4:19 am, "Phlip" <phlip...@yahoo.com> wrote:
> Define "coupling" as "A must change only because B changed". Define "cohere"
> as "A shares legitimate reasons to change with B".

My two pence..

:-)

I've always understood coherency as single-mindedness of purpose.

I.e. it's an attribute/aspect of a single class or function.

Coupling is an attribute/aspect of a collection of classes or
functions.

The best description of this I read was in Steve McConnel's "Code
Complete" (thoroughly recommended reading).


Thanks, Fazl


-- 
      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]