I'm using this to plot points to a frame buffer so there could be something wrong with the function that is plotting the points but these numbers not matching my calculation seemed like a place to start.An example: The value you got was -4.371139e-08. Scale it using 1e05 for a calculation where you need an integer and it becomes -0.004371139 and then it is rounded to 0. So for applications where you require 6 or less significant digits, you will still get accurate results. For regular stuff needing 4 or 5 significant digits, I wouldn't worry about anything.
Statistics: Posted by DarkElvenAngel — Mon Jul 22, 2024 7:04 pm