C#: Numbers and Integer Precision [6 of 19]

Play C#: Numbers and Integer Precision [6 of 19]

The Discussion

  • User profile image
    Winston Smith

    4min 19sec "Holy crap!"

  • User profile image
    psymun
    I also enjoyed the exclamation of "holy crap!".
  • User profile image
    siralomarah​med
    Basically "double" deals with a limited set of rational numbers. Since rational numbers includes integers, no wonder it is very big in size. Rational number is any number that can be written by dividing an integer by another.

    The overflow effect is pretty intersting, I played with it a bit, and found pretty weird answers like these:

    int.MaxValue + int.MaxValue = -2
    int.MaxValue + int.MinValue = -1

    double.MaxValue + double.MaxValue = infinite
    double.MaxValue + double.MinValue = zero
  • User profile image
    Confused person

    I'm confused. With input of 15 / 72, I get an answer of 0, but if I input 1.5 / 7.2, it gives 0.208333333333333. These two equations are equivalent, and I thought that double was not supposed to truncate the answer. It seems to do the same thing every time both numbers are integers that were not assigned variables. So in the second example, (a + b) / c gives 5.25, where a=19, b=23, and c=8, but if I type it in as numbers, (19 + 23) / 8, it gives 8.
    Under which circumstances does double revert to int?

  • User profile image
    Confused person

    Typo in previous comment, should read "(19 + 23) / 8, it gives 5."

  • User profile image
    Kay

    In a previous video in this series, Scott mentioned "String literals", strings which are not stored as variables but are strings.
    @Confused Person, likewise
    15/72 will be interpreted as int literal, which gives a 0.
    1.5 / 7.2 will be interpreted as a double literal.
    And likewise, (19 + 23)/8 is interpreted as int.
    To test this out, try assigning the operations to a variable declared with var, and then call var.gettype() like so:
    var d = (19 + 23)/8 ;
    Console.WriteLine($"Type of d is {d.gettype()}");
    A better reference:
    https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/floating-point-numeric-types

Add Your 2 Cents