Hello,

I recently had a colleague introduce me to the variable macro
DBL_EPSILON and the function _isnan(), because some of my functions
return 1.#QNANO values. However, I'm not sure of of how best to use
DBL_EPSILON and _isnan(). Which is the better way to test for a small
double:

Method 1:  _isnan( really_small_double );

Method 2: if( really_small_double < DBL_EPSILON ) // do stuff

Thanks,

 - Olumide

Olumide <50295@web.de> writes:
> I recently had a colleague introduce me to the variable macro
> DBL_EPSILON and the function _isnan(), because some of my functions
> return 1.#QNANO values. However, I'm not sure of of how best to use
> DBL_EPSILON and _isnan(). Which is the better way to test for a small
> double:
>
> Method 1:  _isnan( really_small_double );
>
> Method 2: if( really_small_double < DBL_EPSILON ) // do stuff

Neither.

C99 has a standard macro isnan(); if your implementation doesn't
support it, it might instead have something similar called "_isnan()".
You'll have to consult your implementation's documentation to find out
exactly what it does.  But it almost certainly tests whether its
argument is a NaN -- which stands for "Not a Number".  It's not a
really small number; it's not a number at all.

DBL_EPSILON is the difference between 1.0 and the next representable
number above 1.0.  Whether that qualifies as "really small" is
something you'll have to decide for yourself.  It depends on the
requirements of your application.  There is no single answer.

Oh, and (x < DBL_EPSILON) is true for very large negative numbers.

I *think* this applies to both C and C++ (I'm reading and posting in
comp.lang.c), but cross-posts between comp.lang.c and comp.lang.c++
are almost never a good idea.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

On Thu, 4 Jun 2009 17:21:30 -0700 (PDT), Olumide <50295@web.de> wrote:

>Hello,
>
>I recently had a colleague introduce me to the variable macro
>DBL_EPSILON and the function _isnan(), because some of my functions
>return 1.#QNANO values. However, I'm not sure of of how best to use
>DBL_EPSILON and _isnan(). Which is the better way to test for a small
>double:
>
>Method 1:  _isnan( really_small_double );
>
>Method 2: if( really_small_double < DBL_EPSILON ) // do stuff

You misunderstand DBL_EPSILON.  It is not a very small number.  It is
the smallest number that can be added to 1.0 to produce a result >
1.0.  If a double has d digits of precision, then DBL_EPSILON is on
the order of pow(10,-d).  Almost all floating point representations
can handle numbers much smaller.

-- 
Remove del for email

Barry Schwarz wrote:
> On Thu, 4 Jun 2009 17:21:30 -0700 (PDT), Olumide <50295@web.de> wrote:
> 
>> Hello,
>>
>> I recently had a colleague introduce me to the variable macro
>> DBL_EPSILON and the function _isnan(), because some of my functions
>> return 1.#QNANO values. However, I'm not sure of of how best to use
>> DBL_EPSILON and _isnan(). Which is the better way to test for a small
>> double:
>>
>> Method 1:  _isnan( really_small_double );
>>
>> Method 2: if( really_small_double < DBL_EPSILON ) // do stuff
> 
> You misunderstand DBL_EPSILON.  It is not a very small number.  It is
> the smallest number that can be added to 1.0 to produce a result >
> 1.0.  If a double has d digits of precision, then DBL_EPSILON is on
> the order of pow(10,-d).  Almost all floating point representations
> can handle numbers much smaller.
> 

DBL_MIN is a very small number.

-- 
pete

> You misunderstand DBL_EPSILON. =A0It is not a very small number. =A0It is
> the smallest number that can be added to 1.0 to produce a result >
> 1.0. =A0If a double has d digits of precision, then DBL_EPSILON is on
> the order of pow(10,-d). =A0Almost all floating point representations
> can handle numbers much smaller.

Thanks everyone. I'm sorry I didn't express myself clearly enough. By
"too small" I mean cannot be represented by a double, thus the value
1.#QNANO (BTW, I'm using Visual Studio .NET 2003). Therefore, I meant
to ask what is the proper technique for testing the result of a
computation is not representable as a double?

Thanks.

Olumide wrote:
> 
> Thanks everyone. I'm sorry I didn't express myself clearly enough. By
> "too small" I mean cannot be represented by a double, thus the value
> 1.#QNANO (BTW, I'm using Visual Studio .NET 2003). Therefore, I meant
> to ask what is the proper technique for testing the result of a
> computation is not representable as a double?
> 

There's no such thing as too small to represent as a double. Really 
really small values become zero. NaNs come from things like 0/0, 
inf-inf, etc.

Try this:

double f = 0.0;
while (f != 0)
	{
	std::cout << f << std::endl;
	/* printf("%f\n", f); for the C cross-post */
	f /= 2;
	}

-- 
   Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com) Author of
"The Standard C++ Library Extensions: a Tutorial and Reference"
(www.petebecker.com/tr1book)

Pete Becker wrote:
> Olumide wrote:
>>
>> Thanks everyone. I'm sorry I didn't express myself clearly enough. By
>> "too small" I mean cannot be represented by a double, thus the value
>> 1.#QNANO (BTW, I'm using Visual Studio .NET 2003). Therefore, I meant
>> to ask what is the proper technique for testing the result of a
>> computation is not representable as a double?
>>
> 
> There's no such thing as too small to represent as a double. Really 
> really small values become zero. NaNs come from things like 0/0, 
> inf-inf, etc.
> 
> Try this:
> 
> double f = 0.0;

Whoops:

	double f = 1.0;

> while (f != 0)
>     {
>     std::cout << f << std::endl;
>     /* printf("%f\n", f); for the C cross-post */
>     f /= 2;
>     }
> 


-- 
   Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com) Author of
"The Standard C++ Library Extensions: a Tutorial and Reference"
(www.petebecker.com/tr1book)

> > Thanks everyone. I'm sorry I didn't express myself clearly enough. By
> > "too small" I mean cannot be represented by a double, thus the value
> > 1.#QNANO (BTW, I'm using Visual Studio .NET 2003). Therefore, I meant
> > to ask what is the proper technique for testing the result of a
> > computation is not representable as a double?
>
> There's no such thing as too small to represent as a double. Really
> really small values become zero. NaNs come from things like 0/0,
> inf-inf, etc.

Perhaps then the problem is a NaN. My functions perform lots of
computation of the type: a^b, and occasionally, a is a very small
number, while b is negative, so that the result tends to 1/0 . Might
this explain why my function occasionally returns 1.#QNANO?

Thanks.

On Fri, 05 Jun 2009 00:41:47 -0400, pete <pfiland@mindspring.com>
wrote:

>Barry Schwarz wrote:
>> On Thu, 4 Jun 2009 17:21:30 -0700 (PDT), Olumide <50295@web.de> wrote:
>> 
>>> Hello,
>>>
>>> I recently had a colleague introduce me to the variable macro
>>> DBL_EPSILON and the function _isnan(), because some of my functions
>>> return 1.#QNANO values. However, I'm not sure of of how best to use
>>> DBL_EPSILON and _isnan(). Which is the better way to test for a small
>>> double:
>>>
>>> Method 1:  _isnan( really_small_double );
>>>
>>> Method 2: if( really_small_double < DBL_EPSILON ) // do stuff
>> 
>> You misunderstand DBL_EPSILON.  It is not a very small number.  It is
>> the smallest number that can be added to 1.0 to produce a result >
>> 1.0.  If a double has d digits of precision, then DBL_EPSILON is on
>> the order of pow(10,-d).  Almost all floating point representations
>> can handle numbers much smaller.
>> 
>
>DBL_MIN is a very small number.

Actually it is a very small positive number but it is still larger
than most (half+1) floating point values the system can represent.

-- 
Remove del for email

Olumide <50295@web.de> writes:
>> > Thanks everyone. I'm sorry I didn't express myself clearly enough. By
>> > "too small" I mean cannot be represented by a double, thus the value
>> > 1.#QNANO (BTW, I'm using Visual Studio .NET 2003). Therefore, I meant
>> > to ask what is the proper technique for testing the result of a
>> > computation is not representable as a double?
>>
>> There's no such thing as too small to represent as a double. Really
>> really small values become zero. NaNs come from things like 0/0,
>> inf-inf, etc.
>
> Perhaps then the problem is a NaN. My functions perform lots of
> computation of the type: a^b, and occasionally, a is a very small
> number, while b is negative, so that the result tends to 1/0 . Might
> this explain why my function occasionally returns 1.#QNANO?

It's hard to tell.

Is a^b supposed to be a raised to the power b?  If so, I'd expect
something like, say, 1e-20 ^ (-100) to yield Infinity, not a NaN.
You'll need to analyze your code, perhaps using a debugger, to
determine where it's going wrong.  If you can narrow your code down to
a small self-contained program that exhibits the problem, we can
probably help you -- or, very likely, you'll find the problem yourself
in the process of narrowing it down.

But please decide which language you're using, C or C++, and post just
to the appropriate newsgroup.

Another thing, you see the line "Olumide <50295@web.de> writes:" at
the top?  That's an attribution line.  Please leave those in place for
any quoted text.  Thanks.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

In article <3e6h25heser5l5mvip7isnp6jhk8ifmh5f@4ax.com> Barry Schwarz <schwarzb@dqel.com> writes:
....
 > You misunderstand DBL_EPSILON.  It is not a very small number.  It is
 > the smallest number that can be added to 1.0 to produce a result >
 > 1.0.

You misunderstand DBL_EPSILON.  It is the difference between 1.0 and the
next larger number.  With standard rounding:
    1.0 + 3 * DBL_EPSILON / 4 > 1.0
-- 
dik t. winter, cwi, science park 123, 1098 xg amsterdam, nederland, +31205924131
home: bovenover 215, 1025 jn  amsterdam, nederland; http://www.cwi.nl/~dik/

In article <a86cd80f-39b1-4bc7-b2ab-49f74a95565e@21g2000vbk.googlegroups.com> Olumide <50295@web.de> writes:
....
 > Thanks everyone. I'm sorry I didn't express myself clearly enough. By
 > "too small" I mean cannot be represented by a double, thus the value
 > 1.#QNANO (BTW, I'm using Visual Studio .NET 2003). Therefore, I meant
 > to ask what is the proper technique for testing the result of a
 > computation is not representable as a double?

You misunderstand.  A NAN does not mean what you think it means.  It means
either that the result is not a real number (but complex, like sqrt(-2.0))
or that the question asked makes no sense (like 0.0/0.0).

The proper way to check for it is to test with an isnan() function, if
available, otherwise the test  a != a  should yield true for a NAN, but
beware of compilers that are too eager in their optimisation.
-- 
dik t. winter, cwi, science park 123, 1098 xg amsterdam, nederland, +31205924131
home: bovenover 215, 1025 jn  amsterdam, nederland; http://www.cwi.nl/~dik/

On Sat, 6 Jun 2009 00:59:35 GMT, "Dik T. Winter" <Dik.Winter@cwi.nl>
wrote:

>In article <3e6h25heser5l5mvip7isnp6jhk8ifmh5f@4ax.com> Barry Schwarz <schwarzb@dqel.com> writes:
>...
> > You misunderstand DBL_EPSILON.  It is not a very small number.  It is
> > the smallest number that can be added to 1.0 to produce a result >
> > 1.0.
>
>You misunderstand DBL_EPSILON.  It is the difference between 1.0 and the
>next larger number.  With standard rounding:
>    1.0 + 3 * DBL_EPSILON / 4 > 1.0

I took the definition right off of K&R II, page 258.

Your definition is from (or consistent with) n1256 but that document
gives five possible rounding methods, at least two of which render
your assertion false.

-- 
Remove del for email

Barry Schwarz wrote:
> On Sat, 6 Jun 2009 00:59:35 GMT, "Dik T. Winter" <Dik.Winter@cwi.nl>
> wrote:
> 
>> In article <3e6h25heser5l5mvip7isnp6jhk8ifmh5f@4ax.com> Barry Schwarz <schwarzb@dqel.com> writes:
>> ...
>>> You misunderstand DBL_EPSILON.  It is not a very small number.  It is
>>> the smallest number that can be added to 1.0 to produce a result >
>>> 1.0.
>> You misunderstand DBL_EPSILON.  It is the difference between 1.0 and the
>> next larger number.  With standard rounding:
>>    1.0 + 3 * DBL_EPSILON / 4 > 1.0
> 
> I took the definition right off of K&R II, page 258.
> 
> Your definition is from (or consistent with) n1256 but that document
> gives five possible rounding methods, at least two of which render
> your assertion false.

Actually, it defines five  different values for FLT_ROUNDS, but I would 
not consider the value of -1 to describe a specific rounding method; and 
it allows for other possible rounding modes.
The standard uses the phrase "default rounding" many times, and that may 
be what he meant by "standard rounding", but as far as I can tell the 
standard never defines what default rounding is. Since most of those 
uses are found in Annex F, it's probably defined by IEC 60559, in which 
case it would not applicable unless __STDC_IEC_559__ is pre-defined by 
the implementation.

However, his point was that DBL_EPSILON is defined by subtraction, not 
addition, is a valid one, and significant, since the two definitions 
will not, in general, define the same number.

Barry Schwarz <schwarzb@dqel.com> writes:

> On Fri, 05 Jun 2009 00:41:47 -0400, pete <pfiland@mindspring.com>
> wrote:
> 
> >Barry Schwarz wrote:
> >> On Thu, 4 Jun 2009 17:21:30 -0700 (PDT), Olumide <50295@web.de> wrote:
> >> 
> >>> Hello,
> >>>
> >>> I recently had a colleague introduce me to the variable macro
> >>> DBL_EPSILON and the function _isnan(), because some of my functions
> >>> return 1.#QNANO values. However, I'm not sure of of how best to use
> >>> DBL_EPSILON and _isnan(). Which is the better way to test for a small
> >>> double:
> >>>
> >>> Method 1:  _isnan( really_small_double );
> >>>
> >>> Method 2: if( really_small_double < DBL_EPSILON ) // do stuff
> >> 
> >> You misunderstand DBL_EPSILON.  It is not a very small number.  It is
> >> the smallest number that can be added to 1.0 to produce a result >
> >> 1.0.  If a double has d digits of precision, then DBL_EPSILON is on
> >> the order of pow(10,-d).  Almost all floating point representations
> >> can handle numbers much smaller.
> >> 
> >
> >DBL_MIN is a very small number.
> 
> Actually it is a very small positive number but it is still larger
> than most (half+1) floating point values the system can represent.

Apparently someone is being deliberately obtuse.  Of course what
was meant (and I expect how it was read by most people) is that
DBL_MIN is a (positive) number with a very small magnitude.  The
earlier discussion makes this reading the only sensible one.

On 06 Jun 2009 07:31:42 -0700, Tim Rentsch <txr@alumnus.caltech.edu>
wrote:

>Barry Schwarz <schwarzb@dqel.com> writes:
>
>> On Fri, 05 Jun 2009 00:41:47 -0400, pete <pfiland@mindspring.com>
>> wrote:
>> 
>> >Barry Schwarz wrote:
>> >> On Thu, 4 Jun 2009 17:21:30 -0700 (PDT), Olumide <50295@web.de> wrote:
>> >> 
>> >>> Hello,
>> >>>
>> >>> I recently had a colleague introduce me to the variable macro
>> >>> DBL_EPSILON and the function _isnan(), because some of my functions
>> >>> return 1.#QNANO values. However, I'm not sure of of how best to use
>> >>> DBL_EPSILON and _isnan(). Which is the better way to test for a small
>> >>> double:
>> >>>
>> >>> Method 1:  _isnan( really_small_double );
>> >>>
>> >>> Method 2: if( really_small_double < DBL_EPSILON ) // do stuff
>> >> 
>> >> You misunderstand DBL_EPSILON.  It is not a very small number.  It is
>> >> the smallest number that can be added to 1.0 to produce a result >
>> >> 1.0.  If a double has d digits of precision, then DBL_EPSILON is on
>> >> the order of pow(10,-d).  Almost all floating point representations
>> >> can handle numbers much smaller.
>> >> 
>> >
>> >DBL_MIN is a very small number.
>> 
>> Actually it is a very small positive number but it is still larger
>> than most (half+1) floating point values the system can represent.
>
>Apparently someone is being deliberately obtuse.  Of course what
>was meant (and I expect how it was read by most people) is that
>DBL_MIN is a (positive) number with a very small magnitude.  The
>earlier discussion makes this reading the only sensible one.

DBL_MIN was introduced into a discussion in which it had no relevance,
or at least none that I could find.  Or maybe your ability to infer
intended meaning is just sharper than mine.

On the other hand, considering your penchant for clarity (such as your
post in the VLA thread 45 minutes earlier), I guess you are precise
while I am obtuse.  I can live with that.  I am also stubborn as well
as persistent.

-- 
Remove del for email

Barry Schwarz <schwarzb@dqel.com> writes:

> On 06 Jun 2009 07:31:42 -0700, Tim Rentsch <txr@alumnus.caltech.edu>
> wrote:
> 
> >Barry Schwarz <schwarzb@dqel.com> writes:
> >
> >> On Fri, 05 Jun 2009 00:41:47 -0400, pete <pfiland@mindspring.com>
> >> wrote:
> >> 
> >> >Barry Schwarz wrote:
> >> >> On Thu, 4 Jun 2009 17:21:30 -0700 (PDT), Olumide <50295@web.de> wrote:
> >> >> 
> >> >>> Hello,
> >> >>>
> >> >>> I recently had a colleague introduce me to the variable macro
> >> >>> DBL_EPSILON and the function _isnan(), because some of my functions
> >> >>> return 1.#QNANO values. However, I'm not sure of of how best to use
> >> >>> DBL_EPSILON and _isnan(). Which is the better way to test for a small
> >> >>> double:
> >> >>>
> >> >>> Method 1:  _isnan( really_small_double );
> >> >>>
> >> >>> Method 2: if( really_small_double < DBL_EPSILON ) // do stuff
> >> >> 
> >> >> You misunderstand DBL_EPSILON.  It is not a very small number.  It is
> >> >> the smallest number that can be added to 1.0 to produce a result >
> >> >> 1.0.  If a double has d digits of precision, then DBL_EPSILON is on
> >> >> the order of pow(10,-d).  Almost all floating point representations
> >> >> can handle numbers much smaller.
> >> >> 
> >> >
> >> >DBL_MIN is a very small number.
> >> 
> >> Actually it is a very small positive number but it is still larger
> >> than most (half+1) floating point values the system can represent.
> >
> >Apparently someone is being deliberately obtuse.  Of course what
> >was meant (and I expect how it was read by most people) is that
> >DBL_MIN is a (positive) number with a very small magnitude.  The
> >earlier discussion makes this reading the only sensible one.
> 
> DBL_MIN was introduced into a discussion in which it had no relevance,
> or at least none that I could find.  Or maybe your ability to infer
> intended meaning is just sharper than mine.

The earlier discussion shows pretty clearly that DBL_EPSILON is a
positive number with a relatively small (but not inordinately
small) magnitude.  But if DBL_EPSILON "is not a very small
number" and DBL_MIN is, it's more likely what's being talked
about is the magnitude than the sign.  I admit, I thought
that was obvious from the discussion.


> On the other hand, considering your penchant for clarity (such as your
> post in the VLA thread 45 minutes earlier), I guess you are precise
> while I am obtuse.  I can live with that.  I am also stubborn as well
> as persistent.

I try to write both clearly and precisely, perhaps moreso than
I should.  On the other hand, I know not everyone does.  If a
statement seems incorrect or not to make sense, I think it helps
to try to give the writer the benefit of the doubt -- assume
that they were trying to say something reasonable, but perhaps
just phrased it poorly.  Sometimes doing that leads to more
confusion rather than less, but usually I think it helps much
more than it hurts.  What's the most sensible meaning (reading)
a statement (especially a confusing or ambiguous one) can have?
I recommend taking that approach, and not just passively but
actively.

Also, my apologies if my comment about deliberate obtuseness
was misplaced.

pete wrote:
> Barry Schwarz wrote:
>> On Thu, 4 Jun 2009 17:21:30 -0700 (PDT), Olumide <50295@web.de> wrote:
>>
>>> Hello,
>>>
>>> I recently had a colleague introduce me to the variable macro
>>> DBL_EPSILON and the function _isnan(), because some of my functions
>>> return 1.#QNANO values. However, I'm not sure of of how best to use
>>> DBL_EPSILON and _isnan(). Which is the better way to test for a small
>>> double:
>>>
>>> Method 1:  _isnan( really_small_double );
>>>
>>> Method 2: if( really_small_double < DBL_EPSILON ) // do stuff
>>
>> You misunderstand DBL_EPSILON.  It is not a very small number.  It is
>> the smallest number that can be added to 1.0 to produce a result >
>> 1.0.  If a double has d digits of precision, then DBL_EPSILON is on
>> the order of pow(10,-d).  Almost all floating point representations
>> can handle numbers much smaller.
>>
> 
> DBL_MIN is a very small number.
> 
But not the smallest. There are a plethora of smaller values called 
subnormal or denormal.

-- 
Joe Wright
"Memory is the second thing to go. I forget what the first is."

Tim Rentsch <txr@alumnus.caltech.edu> writes:
> Barry Schwarz <schwarzb@dqel.com> writes:
>> On Fri, 05 Jun 2009 00:41:47 -0400, pete <pfiland@mindspring.com>
>> wrote:
>> >Barry Schwarz wrote:
>> >> On Thu, 4 Jun 2009 17:21:30 -0700 (PDT), Olumide <50295@web.de> wrote:
>> >>> I recently had a colleague introduce me to the variable macro
>> >>> DBL_EPSILON and the function _isnan(), because some of my functions
>> >>> return 1.#QNANO values. However, I'm not sure of of how best to use
>> >>> DBL_EPSILON and _isnan(). Which is the better way to test for a small
>> >>> double:
>> >>>
>> >>> Method 1:  _isnan( really_small_double );
>> >>>
>> >>> Method 2: if( really_small_double < DBL_EPSILON ) // do stuff
>> >> 
>> >> You misunderstand DBL_EPSILON.  It is not a very small number.  It is
>> >> the smallest number that can be added to 1.0 to produce a result >
>> >> 1.0.  If a double has d digits of precision, then DBL_EPSILON is on
>> >> the order of pow(10,-d).  Almost all floating point representations
>> >> can handle numbers much smaller.
>> >> 
>> >
>> >DBL_MIN is a very small number.
>> 
>> Actually it is a very small positive number but it is still larger
>> than most (half+1) floating point values the system can represent.
>
> Apparently someone is being deliberately obtuse.  Of course what
> was meant (and I expect how it was read by most people) is that
> DBL_MIN is a (positive) number with a very small magnitude.  The
> earlier discussion makes this reading the only sensible one.

But note that one of the tests the OP proposed was:

    if (really_small_double < SOME_TINY_CONSTANT)

The OP needs to think about how to handle negative numbers.

And yes, DBL_MIN is a very small positive number, but it's not useful
in determining whether some given number is very small.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

On Thu, 04 Jun 2009 19:30:04 -0700, Keith Thompson <kst-u@mib.org>
wrote:

>Olumide <50295@web.de> writes:
>> I recently had a colleague introduce me to the variable macro
>> DBL_EPSILON and the function _isnan(), because some of my functions
>> return 1.#QNANO values. However, I'm not sure of of how best to use
>> DBL_EPSILON and _isnan(). Which is the better way to test for a small
>> double:
>>
>> Method 1:  _isnan( really_small_double );
>>
>> Method 2: if( really_small_double < DBL_EPSILON ) // do stuff
>
>Neither.
>
>C99 has a standard macro isnan(); if your implementation doesn't
>support it, it might instead have something similar called "_isnan()".
>You'll have to consult your implementation's documentation to find out
>exactly what it does.  But it almost certainly tests whether its
>argument is a NaN -- which stands for "Not a Number".  It's not a
>really small number; it's not a number at all.
>
>DBL_EPSILON is the difference between 1.0 and the next representable
>number above 1.0. =20

Not really.  It specifies the guaranteed accuracy of mathematical
operations, not the resolution.  The difference can be pretty subtle,
but consider: if you are dealing with largish quantities, say
Avogadro's number, and you want to know the number of atoms delta
resolution that can be represented it would still be many trillions.
But the accuracy ratio scales directly with the size of the numbers
represented, and this is but one of the contributing components to a
property called numerical instability.

>Whether that qualifies as "really small" is
>something you'll have to decide for yourself.  It depends on the
>requirements of your application.  There is no single answer.
>
>Oh, and (x < DBL_EPSILON) is true for very large negative numbers.
>
>I *think* this applies to both C and C++ (I'm reading and posting in
>comp.lang.c), but cross-posts between comp.lang.c and comp.lang.c++
>are almost never a good idea.

JosephKK wrote:
> On Thu, 04 Jun 2009 19:30:04 -0700, Keith Thompson
> <kst-u@mib.org> wrote:

>>DBL_EPSILON is the difference between 1.0 and the next
>>representable number above 1.0.  
> 
> Not really.  It specifies the guaranteed accuracy of
> mathematical operations, not the resolution.

Yes it does. C++98 final draft, 18.2.1.2/20 says:

|   static T epsilon() throw();
|
| Machine epsilon:  the difference between 1 and the least value 
| greater than 1 that is representable. 8)
|
| 8) Equivalent to FLT_EPSILON, DBL_EPSILON, LDBL_EPSILON.

The C++ standard does not define accuracy, but IEEE754 mandates that 
the relative error of basic operations not exceed 1 ulp which roughly 
means the interval [-2*epsilon, epsilon].


Martin

-- 
Quidquid latine scriptum est, altum videtur.

On Sat, 6 Jun 2009 00:59:35 GMT, "Dik T. Winter" <Dik.Winter@cwi.nl>
wrote:

>In article <3e6h25heser5l5mvip7isnp6jhk8ifmh5f@4ax.com> Barry Schwarz =
<schwarzb@dqel.com> writes:
>...
> > You misunderstand DBL_EPSILON.  It is not a very small number.  It is
> > the smallest number that can be added to 1.0 to produce a result >
> > 1.0.
>
>You misunderstand DBL_EPSILON.  It is the difference between 1.0 and the
>next larger number.  With standard rounding:
>    1.0 + 3 * DBL_EPSILON / 4 > 1.0

No. See my previous explanation.  It represents the accuracy ratio.

Martin Eisenberg wrote:

> The C++ standard does not define accuracy, but IEEE754 mandates
> that the relative error of basic operations not exceed 1 ulp
> which roughly means the interval [-2*epsilon, epsilon].

Correction: [-1,1]*epsilon.


Martin

-- 
Quidquid latine scriptum est, altum videtur.

JosephKK wrote:
> On Thu, 04 Jun 2009 19:30:04 -0700, Keith Thompson <kst-u@mib.org>
> wrote:
....
>> DBL_EPSILON is the difference between 1.0 and the next representable
>> number above 1.0.  
> 
> Not really. ...

Yes, really. The C standard defines it as: "the difference between 1 and 
the least value greater than 1 that is representable in the given 
floating point type ..." (5.2.4.2.2p11). Since this is cross-posted to 
C++, I'll also mention that the C++ standard incorporates the C standard 
library by reference (1.2p2) and specifically mentions both <float.h> 
(D.5p1) and DBL_EPSILON (C.2p4) as features of that library which are so 
incorporated, and specifies that for <cfloat>　”The contents are the 
same as the Standard C library header <float.h>”． (18.2.2p4).

 > ... It specifies the guaranteed accuracy of mathematical
> operations, not the resolution   The difference can be pretty subtle,
> but consider: if you are dealing with largish quantities, say
> Avogadro's number, and you want to know the number of atoms delta
> resolution that can be represented it would still be many trillions.
> But the accuracy ratio scales directly with the size of the numbers
> represented, and this is but one of the contributing components to a
> property called numerical instability.

The definition of DBL_EPSILON is cited above. It's quite simple, and 
makes no mention of any of those issues. Those issues are certainly 
related to it, but not in any way that justifies saying "Not really.".

JosephKK wrote:
> On Thu, 04 Jun 2009 19:30:04 -0700, Keith Thompson <kst-u@mib.org>
> wrote:
> 
>> Olumide <50295@web.de> writes:
>>> I recently had a colleague introduce me to the variable macro
>>> DBL_EPSILON and the function _isnan(), because some of my functions
>>> return 1.#QNANO values. However, I'm not sure of of how best to use
>>> DBL_EPSILON and _isnan(). Which is the better way to test for a small
>>> double:
>>>
>>> Method 1:  _isnan( really_small_double );
>>>
>>> Method 2: if( really_small_double < DBL_EPSILON ) // do stuff
>> Neither.
>>
>> C99 has a standard macro isnan(); if your implementation doesn't
>> support it, it might instead have something similar called "_isnan()".
>> You'll have to consult your implementation's documentation to find out
>> exactly what it does.  But it almost certainly tests whether its
>> argument is a NaN -- which stands for "Not a Number".  It's not a
>> really small number; it's not a number at all.
>>
>> DBL_EPSILON is the difference between 1.0 and the next representable
>> number above 1.0.  
> 
> Not really.  It specifies the guaranteed accuracy of mathematical
> operations, not the resolution.  The difference can be pretty subtle,
> but consider: if you are dealing with largish quantities, say
> Avogadro's number, and you want to know the number of atoms delta
> resolution that can be represented it would still be many trillions.
> But the accuracy ratio scales directly with the size of the numbers
> represented, and this is but one of the contributing components to a
> property called numerical instability.
> 

Yes really. The definition of EPSILON is in simple English:

    the  difference  between  1 and the least value greater
    than 1 that is  representable  in  the  given  floating
    point type, b^1-p.

In the case of float that is 2^-23. Consider:

00110100 00000000 00000000 00000000
Exp = 104 (-22)
   11101010
Man =   .10000000 00000000 00000000
1.19209290e-07

This is the representation of FLT_EPSILON. Adding 1 to it:

00111111 10000000 00000000 00000001
Exp = 127 (1)
   00000001
Man =   .10000000 00000000 00000001
1.00000012e+00

Consider the mantissa (significand?). The bit at b23 is the 1 and the 
bit at b0 is EPSILON. It shows us graphically the limits of precision 
and accuracy of a float. The term (1.0 + FLT_EPSILON) is useful in 
determining the granularity of a given floating point value.

-- 
Joe Wright
"Memory is the second thing to go. I forget what the first is."

James Kuyper wrote:

> incorporated, and specifies that for <cfloat>　”The contents are the 
> same as the Standard C library header <float.h>”． (18.2.2p4).

Sorry about that - but I've finally figured out what's going wrong. My 
computer has software installed that lets me easily insert characters in 
encodings and fonts designed for showing both Chinese and English text.d 
It switches modes automatically when I hit Shift-Space, or Ctrl-Space, 
key combinations that I've been hitting by accident far too frequently 
lately.

That was supposed to say:

 > incorporated, and specifies that for <cfloat.h> "The contents are the
 > same as the Standard C library header <float.h>" (18.2.2p4).

JosephKK wrote:
> On Thu, 04 Jun 2009 19:30:04 -0700, Keith Thompson <kst-u@mib.org>
> wrote:
>> 
>> DBL_EPSILON is the difference between 1.0 and the next representable
>> number above 1.0.  
> 
> Not really.

     Yes, really.  In comp.lang.c at any rate; comp.lang.c++
is beyond my knowledge or speculation.  5.2.4.2.2p10 *defines*
DBL_EPSILON as

	"the difference between 1 and the least value greater
	than 1 that is representable in [double]"

.... and goes on to say that it shall not exceed 1E-9.

-- 
Eric Sosman
esosman@ieee-dot-org.invalid

Keith Thompson wrote:

> Oh, and (x < DBL_EPSILON) is true for very large negative numbers.

Isn't it true for *all* negative numbers?

-- 
pete

pete <pfiland@mindspring.com> writes:
> Keith Thompson wrote:
>
>> Oh, and (x < DBL_EPSILON) is true for very large negative numbers.
>
> Isn't it true for *all* negative numbers?

Of course (and for 0.0), but the intended point of the comparison was
to determine whether x is "really small".

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

On Sat, 6 Jun 2009 22:28:59 +0000 (UTC), Martin Eisenberg
<martin.eisenberg@udo.edu> wrote:

>JosephKK wrote:
>> On Thu, 04 Jun 2009 19:30:04 -0700, Keith Thompson
>> <kst-u@mib.org> wrote:
>
>>>DBL_EPSILON is the difference between 1.0 and the next
>>>representable number above 1.0. =20
>>=20
>> Not really.  It specifies the guaranteed accuracy of
>> mathematical operations, not the resolution.
>
>Yes it does. C++98 final draft, 18.2.1.2/20 says:
>
>|   static T epsilon() throw();
>|
>| Machine epsilon:  the difference between 1 and the least value=20
>| greater than 1 that is representable. 8)
>|
>| 8) Equivalent to FLT_EPSILON, DBL_EPSILON, LDBL_EPSILON.
>
>The C++ standard does not define accuracy, but IEEE754 mandates that=20
>the relative error of basic operations not exceed 1 ulp which roughly=20
>means the interval [-2*epsilon, epsilon].
>
>
>Martin

I was talking in clc, was not mindful of crossposting.  Besides being
more of a hardware type i care more about IEEE-754 terminology.=20
j

On Sat, 06 Jun 2009 23:50:58 GMT, James Kuyper
<jameskuyper@verizon.net> wrote:

>James Kuyper wrote:
>
>> incorporated, and specifies that for <cfloat>=E3=80=80=E2=80?The =
contents are the=20
>> same as the Standard C library header <float.h>=E2=80?=EF=BC=8E =
(18.2.2p4).
>
>Sorry about that - but I've finally figured out what's going wrong. My=20
>computer has software installed that lets me easily insert characters in=
=20
>encodings and fonts designed for showing both Chinese and English text.d=
=20
>It switches modes automatically when I hit Shift-Space, or Ctrl-Space,=20
>key combinations that I've been hitting by accident far too frequently=20
>lately.
>
>That was supposed to say:
>
> > incorporated, and specifies that for <cfloat.h> "The contents are the
> > same as the Standard C library header <float.h>" (18.2.2p4).

Well shit.  The standards say what the standards say.  I am corrected
on what they say.  It does not alter the real properties of the real
hardware, nor the mathematical properties thereof.  Overextend your
trust in the programming language standards at your own risk.

On Sat, 06 Jun 2009 15:39:34 -0700,
"JosephKK"<quiettechblue@yahoo.com> wrote:

>On Sat, 6 Jun 2009 00:59:35 GMT, "Dik T. Winter" <Dik.Winter@cwi.nl>
>wrote:
>
>>In article <3e6h25heser5l5mvip7isnp6jhk8ifmh5f@4ax.com> Barry Schwarz =
<schwarzb@dqel.com> writes:
>>...
>> > You misunderstand DBL_EPSILON.  It is not a very small number.  It =
is
>> > the smallest number that can be added to 1.0 to produce a result >
>> > 1.0.
>>
>>You misunderstand DBL_EPSILON.  It is the difference between 1.0 and =
the
>>next larger number.  With standard rounding:
>>    1.0 + 3 * DBL_EPSILON / 4 > 1.0
>
>No. See my previous explanation.  It represents the accuracy ratio.

Yeah, i see what the standards say.  I am talking about how the
hardware and the math works.

JosephKK wrote:
> On Sat, 06 Jun 2009 15:39:34 -0700,
> "JosephKK"<quiettechblue@yahoo.com> wrote:
> 
>> On Sat, 6 Jun 2009 00:59:35 GMT, "Dik T. Winter" <Dik.Winter@cwi.nl>
>> wrote:
>>
>>> In article <3e6h25heser5l5mvip7isnp6jhk8ifmh5f@4ax.com> Barry Schwarz <schwarzb@dqel.com> writes:
>>> ...
>>>> You misunderstand DBL_EPSILON.  It is not a very small number.  It is
>>>> the smallest number that can be added to 1.0 to produce a result >
>>>> 1.0.
>>> You misunderstand DBL_EPSILON.  It is the difference between 1.0 and the
>>> next larger number.  With standard rounding:
>>>    1.0 + 3 * DBL_EPSILON / 4 > 1.0
>> No. See my previous explanation.  It represents the accuracy ratio.
> 
> Yeah, i see what the standards say.  I am talking about how the
> hardware and the math works.

It works as one would expect from reading the standard.

If FLT_ROUNDS is 1

and if

x = y = 4 * DBL_EPSILON / 3;

and if

z = 2.0;

then

x + y + z <  x + (y + z)

-- 
pete

In article <dkol25ted7lh52pq05gignjrvarstj90va@4ax.com> "JosephKK"<quiettechblue@yahoo.com> writes:
 > On Thu, 04 Jun 2009 19:30:04 -0700, Keith Thompson <kst-u@mib.org>
 > wrote:
....
 > >DBL_EPSILON is the difference between 1.0 and the next representable
 > >number above 1.0. =20
 > 
 > Not really.  It specifies the guaranteed accuracy of mathematical
 > operations, not the resolution.

It would be better if you read the standard before pronouncing things like
this.
-- 
dik t. winter, cwi, science park 123, 1098 xg amsterdam, nederland, +31205924131
home: bovenover 215, 1025 jn  amsterdam, nederland; http://www.cwi.nl/~dik/

JosephKK wrote:
....
> Well shit.  The standards say what the standards say.  I am corrected
> on what they say.  It does not alter the real properties of the real
> hardware, nor the mathematical properties thereof. ...

DBL_EPSILON Is not a mathematical abstraction, it is a macro defined in 
the C standard library, with a value whose meaning is specified by the C 
standard.

When you wrote "Not really.", you were objecting to a true statement 
about DBL_EPSILON, not a statement about "the real properties of real 
hardware".

When you wrote the sentence which starts "It specifies ...", the subject 
of that sentence was "It", which in context had to be a reference to the 
use of DBL_EPSILON in the message you were responding to. Again, what 
you said about "it" may have been true with regard to some entity 
related to "the real properties of real hardware", but you didn't 
identify which entity that is; it certainly is not DBL_EPSILON.

> ... Overextend your
> trust in the programming language standards at your own risk.

I trust DBL_EPSILON to be the difference between 1.0 and the next number 
after 1.0 that is representable as a double. Combining that fact with 
the definition of C's floating point model given in 5.2.4.2.2p1-2, and 
the value of FLT_RADIX, I can calculate the spacing between x and the 
next representable number for any value of x that satisfies (DBL_MIN <= 
x && x < DBL_MAX) || (-DBL_MAX <= x && x < -DBL_MIN).

Thinking that DBL_EPSILON means anything other than that would be 
"overextending" the definition (as CBFalconer so spectacularly 
demonstrated in a recent thread), but that's not because of trusting the 
programming language standards too much, but because of misunderstanding 
what they say.

On Mon, 08 Jun 2009 11:00:51 GMT, James Kuyper
<jameskuyper@verizon.net> wrote:

>JosephKK wrote:
>...
>> Well shit.  The standards say what the standards say.  I am corrected
>> on what they say.  It does not alter the real properties of the real
>> hardware, nor the mathematical properties thereof. ...
>
>DBL_EPSILON Is not a mathematical abstraction, it is a macro defined in=20
>the C standard library, with a value whose meaning is specified by the C=
=20
>standard.
>
>When you wrote "Not really.", you were objecting to a true statement=20
>about DBL_EPSILON, not a statement about "the real properties of real=20
>hardware".
>
>When you wrote the sentence which starts "It specifies ...", the subject=
=20
>of that sentence was "It", which in context had to be a reference to the=
=20
>use of DBL_EPSILON in the message you were responding to. Again, what=20
>you said about "it" may have been true with regard to some entity=20
>related to "the real properties of real hardware", but you didn't=20
>identify which entity that is; it certainly is not DBL_EPSILON.
>
>> ... Overextend your
>> trust in the programming language standards at your own risk.
>
>I trust DBL_EPSILON to be the difference between 1.0 and the next number=
=20
>after 1.0 that is representable as a double. Combining that fact with=20
>the definition of C's floating point model given in 5.2.4.2.2p1-2, and=20
>the value of FLT_RADIX, I can calculate the spacing between x and the=20
>next representable number for any value of x that satisfies (DBL_MIN <=3D=
=20
>x && x < DBL_MAX) || (-DBL_MAX <=3D x && x < -DBL_MIN).
>
>Thinking that DBL_EPSILON means anything other than that would be=20
>"overextending" the definition (as CBFalconer so spectacularly=20
>demonstrated in a recent thread), but that's not because of trusting the=
=20
>programming language standards too much, but because of misunderstanding=
=20
>what they say.

The C standard says what it says.  It defines a few thing in certain
ways.  Those definitions are often somewhat at odds with how the
hardware operates and the way that IEEE-754 specifies that they
operate.  Thus when the hardware acts differently than what the C
standard specifies, it is not the hardware that is defective.
;
Never mind we are talking past each other.

On Mon, 8 Jun 2009 10:19:57 GMT, "Dik T. Winter" <Dik.Winter@cwi.nl>
wrote:

>In article <dkol25ted7lh52pq05gignjrvarstj90va@4ax.com> =
"JosephKK"<quiettechblue@yahoo.com> writes:
> > On Thu, 04 Jun 2009 19:30:04 -0700, Keith Thompson <kst-u@mib.org>
> > wrote:
>...
> > >DBL_EPSILON is the difference between 1.0 and the next representable
> > >number above 1.0. =3D20
> >=20
> > Not really.  It specifies the guaranteed accuracy of mathematical
> > operations, not the resolution.
>
>It would be better if you read the standard before pronouncing things =
like
>this.

Well fudge.  The standard defines it the way that it does.  That
definition is in conflict with the way the IEEE-745 compliant hardware
operates.  And i am a hardware type.

JosephKK wrote:
....
> The C standard says what it says.  It defines a few thing in certain
> ways.  Those definitions are often somewhat at odds with how the
> hardware operates and the way that IEEE-754 specifies that they
> operate.  Thus when the hardware acts differently than what the C
> standard specifies, it is not the hardware that is defective.
> ;
> Never mind we are talking past each other.

I'd prefer not to be talking past each other. I'm aware that there are 
inconsistencies of the kind you mention in some other parts of the 
standard; in particular, I've heard that some of the specifications in 
Annex F about how signed zeros and signed infinities are supposed to be 
handled don't jibe with the IEEE-754.

But is there anything that the standard says about DBL_EPSILON 
specifically  which is "somewhat at odds with the hardware operates and 
the way that IEEE-754 specifies that they operate"? If so, how? I'm at a 
loss to imagine how it could be. As far as I know, all floating point 
representations have a unique next-representable value after 1.0, and 
that would seem to be the only requirement for the definition of 
DBL_EPSILON to be meaningful.

"JosephKK"<quiettechblue@yahoo.com> writes:
> On Mon, 8 Jun 2009 10:19:57 GMT, "Dik T. Winter" <Dik.Winter@cwi.nl>
> wrote:
>>In article <dkol25ted7lh52pq05gignjrvarstj90va@4ax.com>
>>"JosephKK"<quiettechblue@yahoo.com> writes:
>> > On Thu, 04 Jun 2009 19:30:04 -0700, Keith Thompson <kst-u@mib.org>
>> > wrote:
>>...
>> > >DBL_EPSILON is the difference between 1.0 and the next representable
>> > >number above 1.0. =20
>> > 
>> > Not really.  It specifies the guaranteed accuracy of mathematical
>> > operations, not the resolution.
>>
>>It would be better if you read the standard before pronouncing things like
>>this.
>
> Well fudge.  The standard defines it the way that it does.  That
> definition is in conflict with the way the IEEE-745 compliant hardware
> operates.  And i am a hardware type.

How so?  The definition of DBL_EPSILON isn't about the way the
hardware operates; it's an unambiguous definition of a number.  Just
what conflict do you see?

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Nokia
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

In message <36hr25179ntpbso5mml24vt96a8q95384c@4ax.com>, JosephKK 
<quiettechblue@yahoo.com> writes
>On Mon, 8 Jun 2009 10:19:57 GMT, "Dik T. Winter" <Dik.Winter@cwi.nl>
>wrote:
>
>>In article <dkol25ted7lh52pq05gignjrvarstj90va@4ax.com> 
>>"JosephKK"<quiettechblue@yahoo.com> writes:
>> > On Thu, 04 Jun 2009 19:30:04 -0700, Keith Thompson <kst-u@mib.org>
>> > wrote:
>>...
>> > >DBL_EPSILON is the difference between 1.0 and the next representable
>> > >number above 1.0. =20
>> >
>> > Not really.  It specifies the guaranteed accuracy of mathematical
>> > operations, not the resolution.
>>
>>It would be better if you read the standard before pronouncing things like
>>this.
>
>Well fudge.

"When in hole, stop digging".

>The standard defines it the way that it does.  That
>definition is in conflict with the way the IEEE-745 compliant hardware
>operates.

Since the definition is entirely in terms of how the hardware operates 
("next representable number above 1.0"), how can that be?

>And i am a hardware type.

I see. And I always thought hardware was based on logic.

-- 
Richard Herring