How do clc folks test a C program ?

I have this another thread titled "Array based Binary Heap in C". My test 
was simple one, just add some elements and remove some and if it works 
fine, that's great. (That is my usual way of testing)

My seniors want me to do automatic test which can add/remove around a 
million elements without any manuals intervention, the input just have to 
be configurable. Someone advised to write a Client-Server program using 
threads to test it where client will send input but I did not like that 
very complex idea.

So I want to know, how do people here at clc test a C program ?






-- 
www.lispmachine.wordpress.com
my email is @ the above blog.

On Nov 2, 8:43=A0pm, arnuld <sunr...@invalid.address> wrote:
> How do clc folks test a C program ?
>
> I have this another thread titled "Array based Binary Heap in C". My test
> was simple one, just add some elements and remove some and if it works
> fine, that's great. (That is my usual way of testing)
>
> My seniors want me to do automatic test which can add/remove around a
> million elements without any manuals intervention, the input just have to
> be configurable. Someone advised to write a Client-Server program using
> threads to test it where client will send input but I did not like that
> very complex idea.
>
> So I want to know, how do people here at clc test a C program ?

Here are some C unit testing frameworks:
http://en.wikipedia.org/wiki/List_of_unit_testing_frameworks#C

If your management is asking you to do comprehensive testing, that is
a terrible error on their part.
Comprehensive testing should *always* be performed by a separate
testing body and *NOT* by the programmer.  To do otherwise is an
absolutely certain invitation for disaster.

Be that as it may, I have some simple rules for my own unit testing
that you might find handy:
1. If the input is small (IOW, 5 bytes or so maximum) then I will
exhaustively test the function (5 bytes of input will be the bit
pattern 0x0000000000 through 0xffffffffff spread across the inputs).
2. Test all edge cases --> E.G.: every pointer input should get a NULL
pointer, a pointer to an empty field, and a pointer to a maximally
populated field).
3. Invalid input must be tested (e.g. floating point functions should
get data outside of the acceptable domain, NaNs, +/-INF, etc.).
4. Valid input must be tested (seems a no brainer, but if you
concentrate on steps 1-3 up above, it might be left out).
5. If the input is large (IOW, strings are allowed as input, or many
bytes of parameters, etc.) then I will statistically test the input
[step 1 above is not possible].
6. Data should be provided in various distributions (e.g. all data
identical, random data, sorted data, etc.)  I have about 10
distributions that I typically use.

HTH

In <pan.2009.11.03.04.40.22@invalid.address>, arnuld wrote:

> How do clc folks test a C program ?
> 
> I have this another thread titled "Array based Binary Heap in C". My
> test was simple one, just add some elements and remove some and if
> it works fine, that's great. (That is my usual way of testing)
> 
> My seniors want me to do automatic test which can add/remove around
> a million elements without any manuals intervention, the input just
> have to be configurable. Someone advised to write a Client-Server
> program using threads to test it where client will send input but I
> did not like that very complex idea.
> 
> So I want to know, how do people here at clc test a C program ?

Volume testing is certainly one important step.

I'm not sure why you are being advised to involve threads in the test 
harness - that's more likely to introduce new bugs than to uncover 
old ones.

Test every possible code path. That isn't always easy, and it's 
probably fair to say that it isn't always /possible/, but it should 
be attempted, at least.

It is at testing time that you discover how useful program scaffolding 
can be. For example, one big problem with testing is "for 
error-handling tests, how do you make a resource fail in exactly the 
place you want?" It can be tricky to get a particular, exact "right 
the hell this one here" malloc call to fail, for example, and yet 
have all the prior and subsequent calls succeed. But a relatively 
trivial wrapper around the allocation routines can give you this 
flexibility (and can be shed for production code).

-- 
Richard Heathfield <http://www.cpax.org.uk>
Email: -http://www. +rjh@
"Usenet is a strange place" - dmr 29 July 1999
Sig line vacant - apply within

On 2009-11-03, arnuld <sunrise@invalid.address> wrote:
> How do clc folks test a C program ?

It depends a lot on the program and its scope or intended function.
I might well do nothing but eyeball results for a one-off that I just
wrote to calculate some results.  For a larger program, I'd try to
come up with test cases which exercise its common usage pattern.

> My seniors want me to do automatic test which can add/remove around a 
> million elements without any manuals intervention, the input just have to 
> be configurable. Someone advised to write a Client-Server program using 
> threads to test it where client will send input but I did not like that 
> very complex idea.

Unless there's something in a program that I expect to be affected
specifically by size, I usually favor merely testing every code path I can
conceive of.  So, once you've tested a handful of sorted and unsorted inputs,
etcetera, that's usually enough to confirm that it'll work.

> So I want to know, how do people here at clc test a C program ?

Depends hugely on its intended use, etcetera.  Something that's going to be
a fundamental piece of a shipping product gets a lot more attention than a
one-off "what happens if..."

-s
-- 
Copyright 2009, all wrongs reversed.  Peter Seebach / usenet-nospam@seebs.net
http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!

> On Tue, 03 Nov 2009 05:46:50 +0000, Richard Heathfield wrote:

> ..SNIP...

> It is at testing time that you discover how useful program scaffolding
> can be. For example, one big problem with testing is "for error-handling
> tests, how do you make a resource fail in exactly the place you want?"
> It can be tricky to get a particular, exact "right the hell this one
> here" malloc call to fail, for example, and yet have all the prior and
> subsequent calls succeed. But a relatively trivial wrapper around the
> allocation routines can give you this flexibility (and can be shed for
> production code).


Moved it to /comp.programming/, does not seem like a C question anymore.



-- 
www.lispmachine.wordpress.com
my email is @ the above blog.