MapInfo Pro

 View Only

  • 1.  Decimal character limit in variables?

    Posted 04-28-2021 06:02
    Hello,

    I am trying to understand why I can't seem to store a coordinate greater than 6 characters in a variable. Whether string or float, a note of the variable returns something like -30.456 or 150.553

    Three characters either side.

    please help me understand....

    thanks in advance

    ------------------------------
    Bradley Trounson]
    NBN Co Limited]
    ------------------------------


  • 2.  RE: Decimal character limit in variables?

    Employee
    Posted 04-28-2021 08:23
    Hi Bradley

    That's the typical way Pro tries to generalize the floating values. You can use for example the Format$() function to "push the limit":

    Dim fValue As Float
    fValue = 123.4567890987654321
    Print fValue
    Print Str$(fValue)
    Print FormatNumber$(fValue)
    Print Format$(fValue, "#.########")

    Typically you will be able to get to 7 or 8 decimals using the Format$() function

    ------------------------------
    Peter Horsbøll Møller
    Principal Presales Consultant | Distinguished Engineer
    Precisely | Trust in Data
    ------------------------------

    Original Message


  • 3.  RE: Decimal character limit in variables?

    Posted 04-28-2021 08:47
    Thanks Peter.

    ------------------------------
    Bradley Trounson]
    NBN Co Limited]
    ------------------------------

    Original Message