Home » Developing U++ » U++ Developers corner » Value with type float
Re: Value with type float [message #58239 is a reply to message #58235] |
Sat, 02 April 2022 09:43   |
 |
mirek
Messages: 14256 Registered: November 2005
|
Ultimate Member |
|
|
Tom1 wrote on Fri, 01 April 2022 14:03Hi,
The following code portrays the float rounding issue:
#include <Core/Core.h>
using namespace Upp;
CONSOLE_APP_MAIN
{
Value v(0.002f);
Cout() << v << "\n";
Cout() << FormatDouble(v) << "\n";
Cout() << FormatG(v,7) << "\n";
}
The result of running the above follows:
0.0020000000949949
0.0020000000949949
0.002
<--- Finished, press [ENTER] to close the window --->
I do not know how to solve this cleanly. In any case a regular user seeing 0.002000000095 in a field where he expects to see 0.002, will not be happy about it. For years I have used roundr() all over the code to fix this up, but having a solution hidden in Core would be awesome! 
Best regards,
Tom
OK, so it is correct behaviour. You can simplify that to
double x = 0.002;
DDUMP(x);
float fx = x;
x = fx;
DDUMP(x);
which produces exactly the same result. Now for an explanation what is going on here:
0.002 cannot be exactly represented as double. Normal double formatting (as used in DDUMP) rounds for 15 decimal digits which is guaranteed precision and it all yields a "correct" result (in both directions, closes value is choosen and everything is "fine").
Anyway, when you convert it to float, you have to round at equivalent of about 7 valid digits (you have to cut 28 bits of mantissa). In this case, the last mantissa bit of float is rounded up to 1 as that is the closest value to original double. When converting back to double, this cannot be undone, hence you get those "949949" digits at the end.
That said, all of this made me think that if we knew that it is float in Value, we might be able to apply for something like FormatG(v,7) instead when displaying it (e.g. in EditDouble).
Mirek
|
|
|
 |
|
Value with type float
|
 |
|
Re: Value with type float
By: mirek on Sat, 19 March 2022 10:38
|
 |
|
Re: Value with type float
|
 |
|
Re: Value with type float
By: Tom1 on Thu, 31 March 2022 13:49
|
 |
|
Re: Value with type float
By: Tom1 on Thu, 31 March 2022 15:14
|
 |
|
Re: Value with type float
By: mirek on Fri, 01 April 2022 11:07
|
 |
|
Re: Value with type float
By: Tom1 on Fri, 01 April 2022 11:13
|
 |
|
Re: Value with type float
By: Tom1 on Fri, 01 April 2022 12:06
|
 |
|
Re: Value with type float
By: Tom1 on Fri, 01 April 2022 14:03
|
 |
|
Re: Value with type float
By: Tom1 on Fri, 01 April 2022 15:26
|
 |
|
Re: Value with type float
By: mirek on Sat, 02 April 2022 01:31
|
 |
|
Re: Value with type float
By: mirek on Sat, 02 April 2022 01:31
|
 |
|
Re: Value with type float
By: mirek on Sat, 02 April 2022 09:43
|
 |
|
Re: Value with type float
By: Tom1 on Sat, 02 April 2022 10:55
|
 |
|
Re: Value with type float
By: mirek on Sat, 02 April 2022 11:26
|
 |
|
Re: Value with type float
By: Tom1 on Sat, 02 April 2022 13:31
|
 |
|
Re: Value with type float
By: mirek on Mon, 11 April 2022 16:52
|
 |
|
Re: Value with type float
By: Tom1 on Tue, 12 April 2022 09:26
|
 |
|
Re: Value with type float
By: mirek on Tue, 12 April 2022 13:40
|
 |
|
Re: Value with type float
By: Tom1 on Tue, 12 April 2022 13:54
|
 |
|
Re: Value with type float
By: Tom1 on Tue, 10 May 2022 13:21
|
 |
|
Re: Value with type float
By: mirek on Tue, 10 May 2022 15:44
|
 |
|
Re: Value with type float
By: Tom1 on Tue, 10 May 2022 20:08
|
 |
|
Re: Value with type float
By: mirek on Tue, 10 May 2022 21:26
|
 |
|
Re: Value with type float
By: Tom1 on Wed, 11 May 2022 08:20
|
 |
|
Re: Value with type float
By: mirek on Fri, 03 June 2022 10:52
|
 |
|
Re: Value with type float
By: Tom1 on Fri, 03 June 2022 12:03
|
 |
|
Re: Value with type float
By: mirek on Wed, 04 October 2023 09:55
|
 |
|
Re: Value with type float
By: Tom1 on Wed, 04 October 2023 12:08
|
 |
|
Re: Value with type float
By: mirek on Fri, 06 October 2023 12:51
|
 |
|
Re: Value with type float
By: Tom1 on Fri, 06 October 2023 13:40
|
 |
|
Re: Value with type float
By: mirek on Fri, 06 October 2023 13:54
|
 |
|
Re: Value with type float
By: Tom1 on Fri, 06 October 2023 14:28
|
 |
|
Re: Value with type float
By: mirek on Mon, 30 October 2023 10:34
|
 |
|
Re: Value with type float
By: Tom1 on Mon, 30 October 2023 12:27
|
 |
|
Re: Value with type float
By: mirek on Mon, 30 October 2023 13:37
|
 |
|
Re: Value with type float
By: mirek on Fri, 01 April 2022 11:03
|
 |
|
Re: Value with type float
By: jimlef on Tue, 06 September 2022 01:41
|
 |
|
Re: Value with type float
By: Oblivion on Tue, 06 September 2022 22:42
|
 |
|
Re: Value with type float
By: jimlef on Thu, 08 September 2022 14:49
|
Goto Forum:
Current Time: Thu May 01 01:22:36 CEST 2025
Total time taken to generate the page: 0.01149 seconds
|