Quantcast
Channel: Renesas Synergy Platform
Viewing all articles
Browse latest Browse all 23645

Forum Post: float number rounding issues

$
0
0
One question regarding the representation of a float number, because I've got rounding issues, which I can't explain: I had the following code:     float f = 0.1; If I watch the var f in the debugger, I would expect:  f = 0.1 but the debugger shows  f = 0.100000001 when I multiply now f with 5, I would expect:  f = 0.500000005 but the debugger shows  f = 0.5 (so as if f would be 0.1) I set  again f = 0.1; now i compute f = 1 - f; I would expect:  f = 0.9 but the debugger shows  f = 0.899999976 // next op f=2+f; debugger shows  f =2.9000001 // next op f*=2; debugger shows  f =5.80000019 flaot k = f; k*=2; debugger shows  k =11.6000004; If I use double I've got the same problem (with a higher resolution):     double d = 0.1;  the debugger shows  d = 0.10000000000000001 Do I have to set any other / further compiler switch/option to avoid that behavoir or what do I make wrong here? Any help is very appreciated. Thank you and best regards Ralph

Viewing all articles
Browse latest Browse all 23645

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>