MapInfo Pro

Expand all | Collapse all

Decimal character limit in variables?

  • 1.  Decimal character limit in variables?

    Posted 14 days ago
    Hello,

    I am trying to understand why I can't seem to store a coordinate greater than 6 characters in a variable. Whether string or float, a note of the variable returns something like -30.456 or 150.553

    Three characters either side.

    please help me understand....

    thanks in advance

    ------------------------------
    Bradley Trounson]
    NBN Co Limited]
    ------------------------------


  • 2.  RE: Decimal character limit in variables?

    Posted 14 days ago
    Hi Bradley

    That's the typical way Pro tries to generalize the floating values. You can use for example the Format$() function to "push the limit":

    Dim fValue As Float
    fValue = 123.4567890987654321
    Print fValue
    Print Str$(fValue)
    Print FormatNumber$(fValue)
    Print Format$(fValue, "#.########")

    Typically you will be able to get to 7 or 8 decimals using the Format$() function

    ------------------------------
    Peter Horsbøll Møller
    Principal Presales Consultant | Distinguished Engineer
    Precisely | Trust in Data
    ------------------------------



  • 3.  RE: Decimal character limit in variables?

    Posted 14 days ago
    Thanks Peter.

    ------------------------------
    Bradley Trounson]
    NBN Co Limited]
    ------------------------------