We have float.ToString("Fn"), where "n" specifies the number of digits you want after the point. So that will convert "123.456" to "123.46". The documentation calls the "F" formatting string the "precision" specifier and even says: "The precision specifier indicates the desired number of decimal places".

Maybe I'm missing something but "precision" and "decimal place" aren't exactly the same thing. To me, "precision" means number of signinficant digits.

So according to me, the following values all have the same precision:

0.0012345
0.012345
0.12345
1.2345
12.345
123.45
1234.5
12345
123450
1234500

...yet if you were to specify ToString("F5"), the output won't look anything like the above.

So let's say I want to display a floating point value that can have a large range with a certain precision, how do I do that? Up to now I have used an algorithm that uses log10 to calculate the location of the point and then calculates the correct value for n in ToString("Fn").

Is there a better way?

26 minutes ago, DCMonkey wrote

@BitFlipper: "Gn" where n is "Precision specifier: Number of significant digits."

The problem with "Gn" is that it automatically switches between fixed-point and scientific notation, whichever is more compact. In my case I always want it to be in fixed-point notation.

It doesn't look like you can tell it to always use fixed point.

15 hours ago, BitFlipper wrote

*snip*

The problem with "Gn" is that it automatically switches between fixed-point and scientific notation, whichever is more compact. In my case I always want it to be in fixed-point notation.

It doesn't look like you can tell it to always use fixed point.

well "floating point"  by definition is not fixed ....

what about "R"  format as that is to make sure you can get back the same value ?

seems like that is really what you want ?

4 hours ago, figuerres wrote

*snip* 

well "floating point"  by definition is not fixed ....

what about "R"  format as that is to make sure you can get back the same value ?

seems like that is really what you want ?

"R" won't work either because it always uses the max precision that the float or double supports. I specifically want it to be rounded to a specified precision. I need this for display purposes, not round-trip purposes.

As far as calling it "fixed point", I'm simply using the same terminology that the documentation is using. Is this another example where the documentation is using the wrong terminology? It specifically states "The general ("G") format specifier converts a number to the most compact of either fixed-point or scientific notation". The "fixed-point" is referring to something like "123.456". I'm not a math expert but I always thought "fixed point" implied that the point is fixed at a certain location, not that it is necessarily an integer.

12 minutes ago, Blue Ink wrote

@BitFlipper: I'm afraid I don't understand your specification. How would you represent 1E8 in fixed point notation when you specify seven significant digits? 

It would be represented as "0.00000001". The specified precision is the max number of significant digits that should be returned. If the specific number has more digits than what is specified by the precision argument, it should be rounded to only have that number of significant digits.

Also, looking at the documentation for float and double, it specifies the "precision" that these types have, which is 7 for float and 15 for double. However each of these have a very large range. For instance float goes from 1E-44 to 3.4E+38, however it always has at least 7 digits of precision. This means that it can represent 0.000001234567 as well as 1234567000000 with the same 7 digits of precision.

Here are some examples:

Value              Precision        Result
1                           3                1
10000                   3                10000
12345                   3                12345    <- 12300 doesn't make sense in this case
12.345                  4                12.35
0.0000000123      2                0.000000012
0.99999                3                1

Negative numbers have the exact same result other having "-" prepended as expected.

11 minutes ago, BitFlipper wrote

*snip*

This means that it can represent 0.000001234567 as well as 1234567000000 with the same 7 digits of precision.

No. That's a matter of accuracy. Accuracy and precision are very different beasts.

7 minutes ago, AndyC wrote

*snip*

No. That's a matter of accuracy. Accuracy and precision are very different beasts.

I'm well aware of the difference between accuracy and precision, but I think they are the other way around from what you just said. Wikipedia states "The precision of a value describes the number of digits that are used to express that value".

The MSDN documentation states that float has a precision of 7 digits and double has a precision of 15 digits.

So in both cases precision == number of digits, which is exactly what I mean when saying "precision". None of what I have said so far addresses "accuracy" which has more to do with the source of the values than how to represent them.

Actually, one of the reasons I need to specify the precision is exactly because this data isn't very accurate, so using 7 digits to represent it is meaningless. I want to specify 3 or 4 digits.

There is no guarantee that two numbers which require the same level of precision (such as 0.000001234567 and 1234567000000 ) will actually both be stored to the same level of accuracy (in fact, it's pretty much guaranteed they won't for floating point math, as there are numbers with far less significant digits that can't be accurately represented.)

Yes that is correct however the issue here is simply displaying the number with a consistent number of significant digits, no matter the accuracy. All I want is a way to display a series of numbers that each could have values that are multiple orders of magnitude different from each other. So I want to display 12345 or 0.0012345. The currently available formatting options don't allow you to do that.

BTW, 0.000001234567 and 1234567000000 have no problem maintaining the stated precision (at least 7 digits for float or at least 15 for double), no matter that their values are so hugely different. Yes there are numbers they can't represent (like 1/10), but the error is still below the specified precision. Where you do lose precision is when you add/subtract values with large differences in size (this is a real problem in some types of digital filters for instance because the coefficients could have extreme large/small value).

Although none of the accuracy related issues are really very relevant to the problem I'm trying to solve.

Maybe the code below will clarify what I need to those that don't seem to understand the results I'm after.

private static string ToString(this double value, int precision, bool showPlus = false)
{
    var text = "";

    if (value == 0.0)
    {
        text = "0";

        if (precision > 1)
            text += ".".PadRight(precision, '0');
    }
    else
    {
        var absValue = Math.Abs(value);
        var threshold = (long)Math.Pow(10.0, precision - 1);
        var shift = 0;
        var shiftValue = absValue;
        var longValue = (long)Math.Round(shiftValue);

        while (longValue < threshold)
        {
            shiftValue *= 10.0;
            longValue = (long)Math.Round(shiftValue);
            shift++;
        }

        text = absValue.ToString("F" + shift);
    }

    if (value < 0.0)
    {
        return "-" + text;
    }
    else if (value > 0.0 && showPlus)
    {
        return "+" + text;
    }
    else
    {
        return text;
    }
}